Don't feel too bad--as a physics major, I was going to correct the semantics on what makes a laser a bit different from a flashlight, too.
To add a bit of info, a bright enough IR laser, even though you can't see it with your eye, could of course be dangerous. I don't know what the actual output of a laser on a mouse is, but it could certainly be bright enough that were you to stick the thing against your eye at the "right" angle and stare into it for a while would cause some damage to the retina. There is some minor added danger, since you don't really know when you're looking right into it--you can't see the light, after all. This is, presumably, why my Logitech laser mouse came with a little warning not to leave it sitting upside down or stare into the detector hole when it's on.
As for practical use, of course, no, it's perfectly safe--you would have to really work at it to damage your eyes.