Thats a good idea, one that I have considered. When I was looking at building a box I was looking at building something around FreeNas. Because I don't have any spare parts laying around, for not much more I can buy a NAS box and get support from one vendor if something mucks up. How has your experience been?
I built it using old ~1.7Ghz Pentium Dual Core (Core Duo-equivalent) parts plus a Silicon Image 4-port SATA PCI controller. Might be cheaply available off eBay.
It's actually been really really good. Far better than I expected given the low specs of the hardware I gave it.
One time I accidentally kicked the box and a cable fell loose. It continued to work, but was slower since a drive was missing. Plugged it back in, rebooted, and it let me know how much data it resynced back to that drive to bring it up to date.
Periodically, running "zpool status" let me know that I have actually gotten some IO checksum errors during heavy transfers, but that error correction worked every time. All those went away when I bought a better power supply. Running "zpool scrub" causes it to read back all the data and verify that everything checks out. Also demonstrates where your IO bottleneck happens to be too.
I did later build one using even older spare parts (P4, all ancient 160/250GB drives) using FreeBSD 9.1. It rebooted a few times under load before I read over the tuning guides.
The 1.7Ghz box is enough to max out gigabit ethernet while using 6 drives. The old P4 didn't. Not sure if it's the CPU or the age of the drives.
The short summary is that if you do decide to do it and can get parts all supported by OpenSolaris (I'm running an old build, snv_130 I think), go with that. If not, get a 64-bit CPU if you're going to run it on FreeBSD to avoid requiring tuning and recompiling the kernel.
At least 2GB ram.
