Pretty sure it's a bug, but this definitely annoys me too. However, I did figure out how to get the badges to show up. If the iPhone is locked when a reminder notification comes through, if you slide to view the reminder notification directly, that reminder will be reflected with a badge. If you don't slide to view the reminder OR the reminder came through while unlocked but you DISMISSED IT, then the only way to get the badge to show is to go view the reminder VIA the notification center (i.e. not by tapping the reminder icon), OR, clear the overdue reminder(s) from the notification center by tapping clear. Interestingly, if you have overdue reminders whose badges have not yet "posted" so to speak, and one comes through and you tap to view it (and only it), all of the other overdue reminders badges become visible on the app icon.
Apple needs to fix it so that as soon as a reminder notification comes in, it's badge instantly appears, just like when a text message comes in, regardless of whether or not you cleared it from the notification center.
EDIT: And yes, all this only applies to reminders that have an alert time. In the above description by "overdue" I mean the alert for the reminder has gone off. Obviously, there is no way to have reminders with no alert associated to be reflected in an icon badge, unlike most every 3rd party reminders app.