Goggles will presumably have much more usable space for processing power and battery than a skinny brick in our pockets. Other than business motivations, why make them dependent on a phone? Yes, perhaps interface with phone for select functions better done by that "coprocessor" but I question the idea that the bulk of the computer horsepower for goggles should be in iPhone.
Depending on how big we want to imagine goggles/glasses, it seems they should have M4 chips in them vs. leaning on the A18 or whatever that will be at the time. If the rumor price of $3K for goggles is real, I suspect we have a powerful Mac or two on our heads... not some (mostly) tiny monitors leaning heavily on the phone in our pocket for much of what we see in and do with them.
However, that offered, your guess is as good as mine. Apple does like to make many things have hard dependencies on an iPhone in a pocket.