Never jailbroken any device, but if I had to guess, I'd say free apps, tethering, features not supported natively to iOS, free apps.
Main thing for me is that it's nice to be ready to install any software you want. I haven't jailbroken because I don't trust this Taig and Pangu crap, but I miss it. If you want to generate the most random list possible, ask an avid jailbreaker why he jailbroke...
Unlimited skips and no ads on iTunes Radio.
Terminal and iFile, which I used to delete the stupid camera shutter sound and change the BBC app's notification sound.
Community-provided patches for SSL vulnerabilities and other things like the "effective power" bug way before Apple pushes real updates.
Emulators.
Free in-app purchases (in some cases).
The ability to delete ****ing Newsstand.