Why do I feel like I let myself down?

STOP COUNTING. If you think that counting calories and making yourself fit will make you happy, then it obviously is not. We don't eat to feel guilty. Eat moderately and feel happy about yourself. I don't say that it doesn't matter what you look like, but a genial and exuberant nature will attract more people than an over-self concsious-calorie counting person. You are definitely not the kind of person who could end up overeating and getting obese. Just chill. Have a balanced diet and enjoy life instead of spending it counting all that you eat and feeling guilty about it.