I manage our phones at work and we just rolled out a few hundred iPhone 5. There were a few of them that this did happen to. We ended up asking for them to be replaced as we feel they were defective.
One other thing to check, have you hear of any hissing coming from your speaker at random? That is an issue I am dealing with as well and that can drain your battery too. That is why I had to ask..
Try resetting network settings under the settings app. Ive had instances were this fixes the battery issue. Also under location services make sure that you don't have timezone settings on or location based iAds or diagnostic information as it will use gps and kill the battery as well.