I bought the ATV4 for my elderly mother. The two big negatives that jumped out immediately were 1) The touchpad and 2) Trying to select characters (worse than ATV2). Bluetooth keyboard integration solves number 2. I don't know how to solve the touchpad problem except to have an older "silver" remote around with the click keys for faster navigation. I suspect the touchpad is not a problem for younger people/gamers who are already used to this sort of thing, but for older people trying to make subtle gestures on a 1" x 1" area on a piece of plastic, can be challenging. BTW, she has the iPad1 and loves it. A touchscreen on that size device is fine. And the ATV remote App does not run on older iPads so that is out for her too.
Based on my experience with ATV4, I see no reason at this time to upgrade from ATV2. The only real advantage is Siri but even that is not compelling enough.