It allows capabilities that Apple restrict in their iOS products.
For example, tired of the old interface? Theme you iPod to be completely different.
Want Wireless internet tethering? Cydia has an App for that. (More useful on the iPhone though. )
Want to display info like messages, email, and even tweets on the lockscreen? They have an app for that.
Want an NES/SNES/N64/etc. emulator?
Before multitasking and folders were officially made, jailbreaking already had that. (Backgrounding and CategoriesSB, respectively)
Many of the Official App store apps can't do this, because developers have specific APIs that they are given from Apple to follow. With jailbreaking, it allows your iOS product to become an open-source device.
I "jailbroke" my 1st gen iPod touch last night with JailBreakMe.com. Pretty cool stuff. I've only done the theme customization at this point. Haven't messed with anything else.