I had that problem when i got my first TV, i had a modem and a wifi router and i had to play a little bit to get work good but in the end i removed the linksys router.
Rarely i get that issue now but i know i just unplug and plug back and speed go back to normal; might aswell check if you have intruders in your network.
Hope it helps.