Well with a 27" you are wrong, so go ahead and add more memory. With a 21.5" model, which has to be cracked and nearly the entire guts removed, yes you do void that warranty.
With the iMac being a single glued screen, strongly suggest getting the additional AppleCare insurance, and if you are in the US or Japan, make that AppleCare+.