I'd say go for it!
Many people make it sound as if the 2011 iMac is now an ancient piece of history, but I'm still very happy with mine. In my opinion, not that much has changed since 2011, looking at the base models. Sure they've lost their superdrive and gotten a bit thinner, but otherwise it's not that different. I wish we had seen more innovation since the day I got my iMac in 2011. Although 2011 sounds long ago, there has only been two releases of the iMac since. The 2012 Ivy Bridge, and the 2013 Haswell.
Comparing the baseline 2011 to baseline 2013 (ignoring the 1.4GHz-model):
2.5GHz i5 Sandy Bridge vs 2.7GHz i5 Haswell.
Ivy Bridge added a bit more oomph from Sandy Bridge, but as I understand it, Haswell mainly improved efficiency which improved battery life for the MacBook-line of computers. For the iMac, I doubt you'll see that much difference in speed between the two processors. Both are Quad Core and should stay relevant for several years.
4GB RAM vs 8GB RAM
The biggest difference between the two, but also one that is very simple to improve on with the 2011 model. I'm adding another 4GB to my iMac this week.

Note that you cannot upgrade the RAM in the new 21.5" iMac.
500GB 7200rpm HDD vs 1TB 5400rpm HD
The 2011 iMac may actually be a bit quicker.. Although any of these old spinning disks are ancient history today, whichever model you may go for, an SSD/Fusion Drive would be much preferred.
AMD 512MB 6750M vs Intel Iris Pro
Although integrated, I'm assuming the Intel Iris Pro will be way faster than the old AMD GPU.
Of course the 2013 is a better computer, no doubt about that. However, I think the 2011 iMac is still a very good computer, and if you can get it for $600 it's a bit of a bargain. Throw an SSD in it and upgrade the RAM to 32GB and you have a very fast desktop. I think the main limitation of the 2011 iMac is that it's only got USB 2.0, rather than 3.0 and as I understand from the techies here, the older SATA-interface of the 2011 may limit SSD speeds, in case you'd install one. Also, you won't be playing any of the recent games with the old AMD GPU I presume.
I think the Retina-iMac on the other hand, has brought significant changes to the line, seeing as it comes with a 5K display as well as a Fusion Drive or SSD as standard. However I don't see it as a mature product as of yet. It will be interesting to see where the iMac-line is in another 3 years. I sort of miss the PowerPC days. Every new iteration, G3 > G4 > G5 was such a major step. As of now, the iMac has looked the same for nearly 6 years (2009 - 2015). Wake up Apple!
I hope the 27" Retina iMac is a proving ground for Apple and their design, and in two years we'll see an upgrade to the entire line of iMacs. Brand new design from the ground up, designed specifically around the Retina Display across all sizes. Less bezel, maybe a swivel arm and new materials.
Haha, we'll see I guess!