If you actually do overclock.... (which is fun)... get a fan (maybe even a new heatsink). If you are doing serious overclocking, the temperature changes very quickly. If you are just changing it to be up a few mhz you should be okay... but its better to be safe, than sorry.
There are three options to test the temperature....
#1 (Best) - Get an external temperature reader... you can search for one on the net farely easily.
#2 (Okay) - Use software (http://eshop.macsales.com/Tech/FTP/newertech/GaugePRO102.sit) that gives the temperature. This method I don't recommend for G4s (cause it doesn't work). Also software programs that give temperature are not the most accurate. There are other types of software as well.... search for others if you want.
#3 (Worst) - Put your hand on the cpu and see how long you last (joke)
I recently overclocked my iMac 333 before I sold it.
I also used that website. If you look in the iMac clock gizmo, anything above 333 says warning. That means temp. I used a thermometer to check before and after.(it was one of those that you can use outside that communicates wirelessly to a display in the house) Before it was at 113F. and when I moded. it was 137F. So I immediately turned of the comp. and bought a fan which I pointed directly at the proccessor and it lowered it to 117F. Just be careful when overclocking.