There is the difference that another person in the car can see what is ahead and is much less likely to start a question when there is something happening right in front of the car that requires the drivers attention.
Exactly. In fact, the person next to you helps your driving since he will react if something is wrong.
The real problem comes from information processing from your brain. Human beings are social creatures of language, any conversation is very high priority for our brain. A normal conversation, with a human being next to us, provides a lot of informations : words, but also non-word utterance (ahah, uh?, mmm...), body language, facial expressions, eye contact... Understanding is easy.
But if the person is not next to us, for instance is on the phone, our brain has to do with poorer information : we have no access to body language, the audio quality is inferior. So, our brain has to work much harder to understand. Worse, we tend to imagine the expressions of the person, as if we were in front of him, which is even more taxing to our brain.
What has been shown by research is that having a conversation over the phone exhausts our ability to process information. For instance, our eyes move a lot less and the result is a kind of tunnel vision. Likewise, our ability to process unusual information quickly plummets - for instance, it will take very long to identify something on the road that doesn't look as what you expect on the road (an animal, someone in a wheelchair). As a result, the driver can be surprised, doesn't see things in the periphery and reacts slowly.
This is the real danger. An headset or anything doesn't help that at all.
