If this works as advertised, they shouldn't run rampant. The OS observes how often and when you use these apps and provides sporadic updates when it thinks that you will open the app soon. If you very rarely open these games, the background refresh shouldn't affect these apps much. In other words, the OS optimises itself. Of course, when you don't need it at all for specific apps, you can turn it off to save perhaps a tiny bit of battery.
----------
It depends really. I always found the slight delay very annoying when opening apps like WhatsApp. It is nice that WhatsApp can refresh itself when it receives a push notification.