Benchmark scores for CPUs alone have virtually doubled when compared to my 2010 MBP. Add a better GPU (and a much nicer display!) and yes, you would definitely notice an improvement--especially for graphically-intensive tasks, rendering, etc. That said, in everyday use it probably wouldn't be as drastic as the improvement you saw when you upgraded to an SSD--which for me was like night and day.
Don't underestimate how nice those retina displays are, though. Much, much easier on the eyes.