You mean, it's like smudged dirt? You see it all the time no matter what as long the LCD turned on?
If so, then it's physical damage on the LCD, iMac classic (read: common) problem. You need to bring it in and Apple will replace the display for you, problem solved. I hope you got AppleCare with it.
Or risk it away and try clean it yourself first if you're OOW, there's a famous thread here discussing that dirty display.
If not, it could be the GPU. Well not necessarily much better or easier, but at least your display is fine.