Peggle is sex. Play it, now. Zen Bound 2 is fantastic. Gemmed! is a fun Bejeweled-type game. Words with Friends is probably what I play the most. Basically scrabble against other people, no rush you just make your move and wait until they move again. Uno you can play against people online in real time, great fun.
If you like first person shooters, get N.O.V.A. It can even use the gyroscope as a method of control, for example, you aim the iPhone 4 up, and the view in-game looks up, turn right, and the screen moves to the right. It's very impressive.