on each unit/computer, do you have to set the DNS to then point to the router instead of something like 8.8.8.8?
Computers / devices default to using whatever DNS the router uses. The router defaults to whatever DNS the ISP (internet provider) uses. For example, if you have Comcast, your router will by default use Comcast's DNS, and then your computers will also use Comcast's DNS.
If you manually change the DNS on your router to something else, all your devices will then use that specific DNS. So for example, even if you have Comcast, but you configure your router to specifically use openDNS, then all your computers and devices will also use openDNS. In this scenario, nothing needs to be changed on the computer / devices themselves; only on the router.