That happens to me sometimes too.
Not sure, it might have to do with Apple's server.
Sometime when I connect all I see is a black screen too and then I have to hang up and dial facetime again and it usually goes thru.
Im on 4.1 and my wifes iphone is on 4.2.1 and it was doing the same before for months now.