No! It is not about pixels. For the one site I've seen clear slowdown on (EBay new design page in Chromium a month or so ago), connect TWO external monitors so total number of pixels is >9million and scroll the page on the external -- no stutter. The number of pixels is irrelevant. In Chrome (because the bug tracker report was posted), it was because images were not being properly cached in HiDPI mode, and Safari was/is likely a similar bug.
retina display -> hidpi mode -> safari lags. Conclusion: it's the resolution.