Like the doctor said to the man who told him that it hurts when he raises his arm over his head - Just don't raise your your arm over your head.
If it's only a little dot, don't touch it; it's not hurting anything is it?
Actually, I can't imagine what it is but if it acts as a shortcut to your contacts, you can move the Contacts icon to another page to get it out of sight and free up room on page one for other stuff.
It could be a glitch in the software. You could try a restore if you really need to get rid of it.
It's an application on a jailbroken iPod Touch. Hold down that yellow light and you will see a list of all your applications at your fingertips (pun intended). Contacts just happens to be the first one shown.
Uninstall the application if you wish for this to disappear.