Yes, completely normal. When the CPU is working harder, it heats up, and the fan speeds up in response. The 4000 range is completely normal; most can run up to 6000+, and in any case no matter how loud it gets it won't spin faster than it's designed to.
Playing video in general will use plenty of CPU, but if by "online" you meant "in a flash player" (like YouTube), then you have Adobe to thank for that--most Flash based things on the Mac max out an entire processor core, which in turn spins the fans up. It got so annoying to me that I eventually turned on "flashblock" in my browser so my fans won't go berserk every time I have a window with a Flash ad in it open (which is ALWAYS, since they're everywhere).
Flash has sucked on the Mac basically since its inception, and while the recent Player 10 improved the situation significantly, significantly better than horrible is still pretty bad.
Bottom line, though, you're fine.
