Computers have a lot of advantages over humans when it comes to driving: they can see everywhere at once, they can see in the dark, they never get distracted by kids arguing in the back seat or urges to show off, they can have much faster reaction times, they can calculate F=ma to much higher precision trying to thread the needle between over and under reacting, they don't get drunk or fall asleep, and they're not likely to confuse the accelerator for the brake in a panic (though Tesla apparently still has some issues with this...).
They can also coordinate much better with other computers than people are ever willing to cooperate with other people, and improve traffic flows.
What computers lack is the ability to reason and adapt. They'll get there, but almost certainly not as soon as the evangelists think. Being able to drive suburban roads in Phoenix or Cupertino is a long way from solving the general problem.
The real justification for self driving cars isn't to give the driver some additional leisure time, it's to improve safety, traffic and energy efficiency. Worthy goals, but it's going to get worse before it gets better. We're also much more forgiving of people than we are of machines-- if I ask a friend for directions and they get it wrong, I react much differently than I do when Siri gets it wrong-- so they're going to have to do much better than people before we fully accept them.
Okay, but, until this is Minority Report type stuff, self-driving cars will always be a road hazard potential. Such a car can brake, or swerve, thousands of times faster than a human can, so its like running into someone, you go left, they go right, you go right, they go left... Think TCAS, when one pilot ignores its commanded movement. Until ALL cars are that technology, it's a bigger risk.
Like, this example: The car hits a curb, and stops instantly because it hit something, and that takes precedence. Anyone following that car isn't likely to have the same quick stopping drive/ability, and will have to either run into them, or swerve to avoid, meaning the latter would potentially involve other vehicles, or pedestrians. Reality gets messy... Even driving down a highway could cause the same cascade. It can become ugly quickly... There will be a lot more deaths and injuries on the way to a 'perfecter' self-driving vehicle.
And 'self-flying planes'? People are talking about them, and I sure as hell wouldn't want to be a passenger on one. Nope. But rich people are 'going into space' *COUGH* on self-flying 'spaceships'.