Driving implies moral decisions and a human conscious, I don't think machines are better at that.
There are upsides and downsides to self driving vehicles. If everyone is going from Point A to B, you don't need that deep of a level of attention. This'll become more true the more widespread they become. It'll be an automated system, moving thousands of little cars along a series of tracks.
But even the best designed automated system can't account for every fly in the ointment, which is where something like a self driving car would fail. For that reason, I wouldn't make the system either/or. Your car is self driving when you want it to be, but can be easily overridden if you need or want to drive yourself.