In addition to the price difference, I think it ultimately comes down to "legacy" issues.
Do you need/want a ODD? Yes you can add an external drive, but as many have mentioned that defeats the purpose of an AIO machine. I occasionally burn blu-ray discs on my iMac and I always hate finding a spare outlet for the external drive and finding a space for it on my crowded desk.
Do you need/want firewire? Yes, Apple offers a thunderbolt to firewire converter, but it doesn't always work with bus powered devices and some users have reported slower transfer speeds.
Do you need/want an audio in port? I just noticed this was missing from the new iMac. There are probably some USB alternatives, but I often transfer old cassettes to CD and would really miss this feature.
So, if you don't need any of these things, the new iMac should work great for you. If you only need one or maybe two, it might work depending upon your specific equipment. If you're like me and need all three, buy a refurbished 2011 iMac.