As others have noted, the major advantage of the mini is that it is, well, mini. Due to the Developer Transition kit trade-in, I got a 17" Intellimac "free" and then bought a mini duo (and just got a MacBook Pro for work), so I feel like I'm qualified to comment on the differences.
The iMac is the fastest of the bunch (although, had work sprung for a 7200 RPM HD on the MBP, maybe not) and is noticeable faster graphics-wise than the mini.
That said, with enough RAM (1 GB is a minimum, 2 GB would be better and I wish I'd gotten it), the mini seems just as fast as the iMac in non-graphics-intensive work. I use it to drive a TV at 1920x1080, and it works flawlessly. As others have said, it's ideal in home theater setups, if you already have a monitor/KB, or if the small size is otherwise better for you.
If you don't have a KB/monitor and have room for the iMac, I would strongly recommend one of those instead, if for no other reason than the better HD and graphics. The fact the RAM is easily accessible is another real plus.
But... I love my mini as well, so you really can't go wrong. I'd say to avoid the solo unless you either cannot afford the duo or will be using it for non-challenging tasks.