I did some more tests. The issue it definitely with the iphone 5, and and not the router. I'm pretty certain.
How I tested this time. I booted up my old Iphone 3gs and was comparing it against my Iphone 5 on wifi. I found a website with high res pictures on it.
Now, for kicks, I got out my old Linksys WRT54g router, which is probably 8 years old. I would try to open the pics on the iphone 5, then on the 3gs. The 3gs loads the high res pics almost double the speed.
I then hooked up my netgear wndr3700, (much newer router with G and Dual N) with iphone connected to the N speed, and the iphone 3gs on the G speed, and the 3gs still loaded the high res pics faster. What I did notice was the 3gs even loaded them quicker since being on the new router, but the iphone 5 didn't get any quicker, even on dual N.
The strange thing is, when I do a speed test, iphone 5 gets 52mb down, and 3gs 20 mb down, since it's on G. One would think the iphone 5 should load those pics quicker, but it doesn't.
Lastly, I took the iphone 5 off wifi, so it's on the 4g network (Lte not avail here) and the pics load over double the speed! So this rules out my routers. What it does say is that the iphone 5 hardware isn't doing something right. So right now, on 4g, everything loads quicker. Sites like gizmodo, and engadget load quicker on 4g than on wifi.
For the guy at work and others with the slow wifi, try taking off wifi and see if things load faster on 4g, etc.
EDIT, if I try to return this at apple, wasn't there something to check if it had a different wifi chip? I thought i saw this mentioned somewhere.