How long has it been there for? IIRC the VM alert is triggered by the network not the OS and it sometimes takes a while (up to an hour ime for the network to clear the alert)
I have had this happen on two occasions. Once it clears by calling myself from another phone and leaving a message. When I checked the message both were gone.
I also found it so stubborn and had to restore my iPhone from an encrypted iTunes backup to clear it.