The infrastructure path to autonomy is a dead end. I've watched initiative after initiative in this space: V2I, V2V, V2X... If we can get real time position to the centimeter, if we have hyper-accurate maps, if we have gigabit upload speeds to the cloud.
It's never going to be perfect enough. The world is huge and always in flux. If you defined the perfect infrastructure, first it would bankrupt a nation to create it, then the first phase would be in need of replacement before the last phase was done.
Humans are perfectly capable of navigating the infrastructure that we have with two bad eyes and half a brain while managing fighting children, texting, and thinking about how we'd tell off our boss if only we'd thought of it at the time.
One critical piece of infrastructure that's widely deployed? Sirens and flashing lights-- yet Teslas keep hitting firetrucks.
We don't need better infrastructure, we need better software. It's just not there and we don't yet know how to create it.