Thanks everyone for the input.
I have the Mac Mini Server (Mid 2011) i7 (Intell 3000HD GPU) and ended up getting a great deal on some 4GB (4x2) Kingston HyperX memory 1600Mhz, CL-9
Link to the same memory:
http://www.newegg.com/Product/Product.aspx?Item=N82E16820104257
and so far so good! I had the following running:
* VM Ware fusion up (Running Windows 7 With Excel 2010, some web browsers, an equation program, notepad++, and a couple other things running)
* Photoshop CS 5.1
* InDesign CS 5.x
* Chrome
* Safari
* Ms Word 2011
* Acrobat Pro
* Some Finder windows
* iTunes playing a 1080p movie
* Duel Monitors
*Activity monitor
* Cirtix session open running through some Java VPN stuff
and had pretty good performance, I didn't notice any lag, and about 800MB free memory, and only about 122KB paged. Boot time is a little faster too.
Anyhow, I would recommend it for other people too.
When 16GB becomes cheaper and has a lower CL then I think I'll upgrade again, but for now this is a little rockin' machine for school work, my job, and personal stuff.
For the price of memory, I can't believe I was hesitant to spend more money to get the extra RAM, my productivity has gone way up and my frustration has gone way down.