If your USB drives have been formatted in the Mac OS Extended filesystem then windows won't read them without special software. Formatting your drive to FAT should allow it to be read by both operating systems.
None of them have been formatted. They are all left in er... Flash Drive format whatever that it. I assume it's just free space. Put it this way, they work on a friends PC, and they work on my Mac running OSX, however not while running Windows XP. Very odd.
Are you plugging them into the actual USB ports or the extra ports on the keyboard? Sometimes the keyboard ports have trouble mounting some USB flash drives because they don't carry enough power.