If you examine the microscopic writing on both bricks, you'll find that the iPod charger is rated at 5 watts, while the iPad charger is rated at 10 watts. While the iPod charger will eventually charge the iPad, it will take longer and doesn't provide enough charge to power the screen while charging the device. It just isn't particularly useful, though my understanding is that it shouldn't harm anything.
I labeled our iPad-specific chargers when I brought them home, both at the brick and at the docking end of the cable. We have four of the original chargers in the house already, so this way when I grab at a cable in any given spot, I know at a glance whether or not I have the right one!