It's only a 3GS thing. In order to use it, you press the location pinging icon on the bottom left once to locate your position. Press it again and it will reorient the map to make the map face in the direction you are holding the phone (does not work well in a car IMO). Press it again to turn it off.
Speaking of which, I wish Apple would add a third option that would deem it useful in a car. For example:
>Press location icon once: pings your GPS location
>Press location icon twice: reorients the map in the direction the phone is facing
>Press location icon three times: reorients the map in the direction the phone is moving
>Press location icon four times:turns off the feature