Safari's "Swipe to go back" re-loading page

Hi all,

I just got a new Mac and coming from Windows, I'd have to say almost all of the features are GREAT!!

BUT!!

I have been "dealing" with this annoying feature of Safari since I've got this rMBP. But I have had enough. So I am wondering if it is just me that's suffering?

When ever I swipe the track pad to go back to the last web page, the web page "re-loads" (the page doesn't actually "re-load", it's just the progress bar moving). During that time, the page seems to "freeze". I cannot scroll down, or do anything. I just have to wait a few seconds (without any indication of when it had finished) before I can start scrolling again. Yes, it's "small" and seems trivial. However it's frustrating.

Is there a way to stop Safari from reloading the page?

It's really really is annoying!!

PS: This isn't an issue if I click the "back button" next to the address bar. It only is an issue when everything freezes when I swipe to go back.