i would call myself fat but most of everyone i know is skinny, it is a combo of metabolism genetics and how much a person actually eats and exercises, furthermore those factors all weight out different in (ie one will over power the other). on my mothers side of the family they are somewhat large and live to an average 0f 75 and my dads side lives to 100 and has the fastest metabolism ive ever seen. i could eat the same or less than most people i know and still gain weight while they stay the same, they don't exercise, neither do i except for gym class and in the summer when i play sports. there are people who eat themselves to death i know of 2 such people and its horrible but they enjoy it and dont care. so its not always peoples fault but sometimes it can be
Metabolism is a function of muscle mass mostly. Build muscle mass, and you'll burn more calories while sitting idle just maintaining it. Also, not all foods are created equal. Some is more calorie dense than others (for example, 100g of chicken breast contains about 180 calories while the same 100g of Porc contains 224. 100g of Tofu, extra-firm, contains 150).
You don't have a slow metabolism. I was fat and ate junk. Like Hellhammer, these days, I'm skinny and fit and can barely keep the weight on pigging out on the good stuff. I eat more than I use to and if I'm not careful, I shed weight. I'm currently trying to keep a 2750/3000 cal diet.
Pay attention to the calorie content of what you eat in a day and you'll understand. Those people with "fast" metabolism ? They eat better than we did. I have one guy at work who claims a "fast metabolism", yet I watched what he ate, and his lunch was often times 1 microwave "Stouffers" type lunch. Checking the box on it, it had around 250 cal. He then ate an Apple. And drank water. Fast metabolism my ass. Sure he hate greasy foods sometimes, but that's the catch, that was like once every 2 weeks. He'd sit there and eat that crap and say "look, I can eat this stuff and no gain a lbs". Sure smartass, you've been living with a calorie deficit for the last 2 weeks.
That's not a healthy way to eat and his muscle mass is probably very low.
(Anecdote, I use to eat 4 pop tarts, the double fudge kind, for breakfeast. I "only" ate 4 because I thought eating more would be just bad, but I felt hungry like 10 minutes later every time. Checking the box last night at the grocery store just for fun, I find out those things are 210 cals per pop tart.

So I was eating 840 cal and feeling hungry still. No wait, I also usually had a tall glass of milk, the 3.25% kind, 120 cal per cup, about 1 cup. 960 calories, just getting up in the morning)
These days my usual breakfeast is around 675 cal. It's composed of 2 portions of all-bran, a mix of fruit (banana, strawberries, blue-berries, raspberries), 1 cup of 1% milk, 2 toasts whole-wheat high fiber and for some sweetness, usually a tbsp of maple syrup or honey and some low sugar jam. That gets me through a couple of hours. It's more food than the 4 pop tarts yet it's about 2/3s the calories.)
Education my friend. Read the labels, and start searching online for stuff that doesn't have labels :
http://caloriecount.about.com/