Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Did you recently update the Weather Channel app? There is a known problem that they are working on fixing where it keeps the arrow on your status bar.

If you go to Settings > General > Reset > Reset Location Warnings, it'll go away.
 
If you have the Weather Channel app it's probably the absolutely horrible update they just did.
 
I shut off local weather and the gps still stayed on. I use the city option now. I still could not pin point it. I dont know if it was on or if the arrow was just stuck.
 
I had the same problem there a couple days ago, i went through my apps location settings one by one and it ended up being "the weather channel max" that was constantly doing something.
 
If you are running iOS 5 beta, then that is probably related to the Reminders.app. For some reason (probably why it's called a beta), Reminders.app uses a crapload of battery and GPS if you have a Geo-fenced reminder on there. So your best bet could be to just go in Settings -> Location Services -> and turn off permission for Reminders.app. That's what had happened to me a while back.
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.