Well, you have the only one if that's the case. According to all others and OWC anything above 6 gb of RAM will result in a sluggish system once 4 gb of active ram are used:
http://eshop.macsales.com/shop/memory/iMac/Intel_Core_2_Duo_PC2-6400
http://eshop.macsales.com/shop/memory/iMac/Intel_Core_2_Duo_PC2-6400