When books were originally introduced to the wider public, many parents at the time were very concerned that books would corrupt their children, remove them from the real world, and make them lazy.
The same concern occurred with rock n roll, TV, video games, and cell phones.
AR/VR glasses is just the next technology on that list.
We already have an endless amount of things that distract, influence, and cut us off from the real world, and we all do them voluntarily, because we want to.
Same thing here.