A lot of really good responses, figured I would chime in as I looked closely at the Mini as a possible server for a friend's small business.
The mini uses a mobile class NIC card. NIC cards come in many flavors and just because it is a 10/100/1000 NIC doesn't mean it can handle the work load put upon it by 10 network clients. This was the main reason I did not recommend the Mini as a server. If Apple wants to sell this as a "server" they should double it's hight, add a desktop class CPU, and server class NIC card, and 4 drive bays. My server has 2 Intel ET 2 4 port NIC cards and the traffic flies! I spend an entire weekend testing testing testing speeds before I went live... NIC cards make a big difference when loaded!
This again brings up other issues. R/W speeds across your network. I have servers (Virtual Machines) running on RAID 5 SSD arrays, and servers running on RAID 5 HDD arrays, and I have Windows 7 running off a single SSD in a 2009 Mini (it's serving Quickbooks to 3 people and my remote PC desktop for quickbooks). It's days are numbered as I have been waiting for my new iMac before I retire the Mini and roll a new Virtual machine on my RAID 5 SSD array. I have tested how much faster Quickbooks will run on the SSD and it's vavavooooom! Can't wait!
Why the RAID 5 SSD array? The read/write speeds across my network off my RAID 5 SSD array are off the charts in comparison to my RAID 5 HDD array (and those are SATA III Enterprise class drives).
I also have CAT 6 across my network, Intel NIC cards in all our PCs, and a Cisco switch.
What can you take from all of this?
1. Yes, the SSD will help for reasons previous mentioned
2. Your NIC card in the Mini is a bottle neck, nothing you can do about it
3. I would open up the Mini and add 2 SSDs, and then run them on RAID 0. Then setup an external drive for Time Machine as a backup
4. Make sure your cables are Cat 5e and up (if you are using WIFI, then you are wasting your time with any upgrades here)
5. Check your switch, you can get an awesome unmanaged gigabit switch for around 200$
6. Get off that 5400 HDD, omg