Yes, I have a case. And a screen protector. And an external battery for recharging the phone. And a cable. I would prefer to have an all-in-one ruggedized iPhone.
And in my case it appears to be the GPS that drains the battery, so in your setup is the GPS on your Garmin? On a typical hillwalking trip I am walking about 10-12 hours/day, so generally I have to top up the phone during lunch and then overnight. Of course I carry a printed map and compass as a backup, but I just prefer the convenience of using the phone. Also, adding satellite functionality for SOS texts to the iPhone, if it is possible, would be reassuring, as would an attachment point for a lanyard. I think if Apple sold a sport version of the iPhone it would find an untapped market.