There are some really bad aftermarket batteries, but if you don't go for the absolute cheapest stuff, choose a brand that's fairly well reviewed (Wasabi seems okay), I don't think you'd have any issues. OEM batteries tend to be priced much higher than their quality justifies, simply because many people gladly pay more for the perceived peace of mind.
There are a few manufacturers that offer OEM battery replacements at sensible prices, but they are very few.
I got an additional OEM battery for my Nikon D80 (back when it was fresh off the press), and paid something like $80 for it. Money badly wasted. It was my first DSLR and I wanted to play it safe, but the reality was that considering how long that battery lasts in the first place, I never actually had to use it as a back up. I used both batteries in rotation, just to keep it active, but in retrospect, that money would be better spend on other accessories.