Well....yes it is a fact that at&t offers better speeds. However, since you are contemplating a move to Verizon it must be because at&t service is sub par for you where you live. Either you live in an area with spotty service where those high speeds come only if you are in the right spot, or you live in a city with over-saturation in which case the speeds drop dramatically even if you are on 3G with 5 bars.
If those are you issues then Verizon's slower speed will actually be much faster than what you get now with at&t, or you will have consistent, reliable service, even if a bit slower.
If you're in a great at&t coverage area then I don't see why you'd switch except for some kind of company discount.