You do know it's the conversation that is the distracting factor, right? Holding your hand to your head doesn't make you a worse driver.
Conversation is A distracting factor. But, working on that logic, radios and passengers should be banned, and all drivers should ensure they've had 8 hours sleep immediately before driving.
There are many factors that will influence the quality of your driving, and every trip is a compromise between safety and practicality. I would agree that taking phone calls while trying to navigate Cambridge's one-way system and adjust your stereo is going to result in a crash. But, on the other hand, if I'm driving down to the south of France, am sat in the slow lane of the motorway, and get a call from the parents just to find out where I am, the distraction is going to be negligible in the given circumstances, and would be prefereable to waiting until the next service station, and trying to get enough reception to return the call.
The main thing is to exercise caution and common sense. Also, one of the main reasons for banning handsets while driving is that, if you only have one hand available, your capacity to react to emergencies is severely limited.
With respect to the original poster, I am looking for a decent car kit. The Nokia CK series has served me well previously, but I am looking for a wired solution which, I think, does not exist.