We don't know what machine you have or what OS X version. I realize it's hard to type on an iPhone (I have the same problem), but the info might help.
If it's a newer machine, I'd try to take it to an Apple store if there's one nearby.
If it's an older machine, well... get your OS X Install Disk, put it in the optical drive, and boot from it while holding down the "D" key. That should start up the Apple Hardware Test which could well tell you something.
How does one run the Apple Hardware Test on a newer (no OS disc) machine?
Your problem sounds very similar to one I and a number of others have encountered, but my problem is specific to certain 2007 and 2008 iMacs with ATI video cards. Does yours happen to fall in that category?
It could well be the graphics card, but I'm guessing that it could instead be the logic board or a memory problem. I wouldn't just replace the graphics card without more evidence that it's the problem.