For my use, Safari is much worse than Chrome.
My wife had been using only Safari on her MBA and I use only Chrome on my rMBP. She's been having these exact same problems where page loads just stall for minutes (sometimes even the beach ball). Especially when a 2nd page was loading, files downloading, or when Time Machine backups kicked in.
I switched routers, tried 2.4 and 5ghz banding on both. No luck. Wiped the system and re-installed Mavericks from scratch (didn't restore her previous profile and TM backup)...still no luck. Ultimately took her MBA to the Genius Bar a few days ago to get it checked out.
They replaced her SSD drive and wireless module. Took it home and set up again from scratch. Maybe somewhat better, but same issues encountered.
I installed Chrome on hers and when running TM backups, at least pages load. Little slower, but not a full stall. I then also fired up Safari on my rMBP and tried pages when running TM backup...completely stalled for me as well!
Ultimately, from my experiences, Safari sucks and doesn't seem to multi-task well. At least not in Mavericks on the 2013 MBA and 2013 rMBP. No issues on the iPads and iPhones though using Safari or Chrome, but then again, those aren't quite the same.