I agree with you. There's a lot of people here where I work, who have android phones and DON'T EVEN REALIZE IT. They basically went into the carrier store and wanted to see nice big touchscreen phones and picked one out that looked cool. Most of these people don't have music on the phone, and have never purchased an app from the android marketplace (or maybe just one to try it).
When I ask them what OS is on their phone, they don't know what I mean. To them their phone is an LG Whatever or HTC Whatever or Samsung Whatever. Half the time they don't even know the Whatever part of the name of the phone, they just know it's an LG or something because that's printed on the phone itself.
So I think to the general public, android has become a behind the scenes name for the OS that goes into a lot of phones, and then of course these manufacturers are then skinning the OS with their own thing, so their phone will look different than the others or have a cool animation that will get people to buy the phone in the store. I don't think they're building up a huge android following, though, because a lot of these people wouldn't know if they moved from an android to a webOS or symbian, other then noticing that things worked differently. They just think that's how that particular phone model works.