I've found there are often voice recognition problems if the names aren't standard - if there's something different about them, such as (maybe) two capital letters instead of a "real" name.
I know three people with the same initials and I tried to differentiate them by using their first initial and a number (ex: S1, S2, S3). NO WAY could I get the voice recognition to handle those alpha-numeric identities. I'd get phone calls all over the map - totally unrelated.
The use of initials with no spaces between them may be part of your problem. But, I'd have to agree that, if you want to keep the contact numbers, then keep them as a standard listing. Otherwise, it's pretty obvious you are hiding something - or trying to.