So I figured this out today by just trying... I didn't see this posted either, when I searched, it was mostly about jailbreaking to hide Safari.
Go into Settings > General > Restrictions > And then type your passcode or enable one. Then turn Safari off.
No more Safari after that and you're free to get something off the App Store.
I tried Dolphin and it worked fine. If you're using Sparrow, and some of the other apps that have their own web views to open links, they work fine. Not so much with the apps that want to open up Safari separately though.
Go into Settings > General > Restrictions > And then type your passcode or enable one. Then turn Safari off.
No more Safari after that and you're free to get something off the App Store.