Other than Scotch and Irish accents, I'd say the vast vast majority of Americans would have no idea where a UK accent would be from.
Would we know it's from the UK? Likely, but we're just as likely to be thrown off by South African and New Zealand accents as well.
Yup. Every country has its own accents and subaccents that the vast majority of people simply won't know about unless they live there. Here in the US, we basically only know what we mostly see on TV: Southern UK (sometimes with some Received Pronunciation thrown in for good measure), Scottish, and Irish. It's rare for someone here to know about geordie accents, or any of the other roughly 50,000 accents you have surrounding the various parts of London.
On the flip side, people in the UK probably aren't aware that there are at least three distinct kinds of southern and midwestern accents, the New England accents, The various New York/New Jersey accents, and however many accents they speak out on the west coast.
It's not crass ignorance, it's just mostly what you hear on TV. Just about every time I see a UKer on Youtube trying to imitate an American accent, it's either them doing deep south, or broadcast American (which is kinda our version of Received Pronunciation, I guess).
And you're right, both of us would probably be thrown by all the various South African accents, of which there are a bunch. I personally knew some people from SA, so I can pick up on it pretty easily (theees khahhn, and theees khaan't, hrrright) . But if I never met them, I would've figured it was another UK accent.