The base 1.4 ghz model with 4gb RAM and 500gb hard drive is more than adequate for an itunes server. I've been using one for over two years and it's very fast. If you're clear that you don't want it for anything else, then you would just be wasting money to upgrade. A SSD would add nothing for this use - it's just going to sit there in iTunes, that will not require much access of the system drive.
Just get some fast USB 3.0 hard drives for your media - these are cheap today also. I have all my media on a 4tb external drive, but that's a couple years old and you can get an 8tb drive for the same cost today. You will then want one or more additional drives for backup.
Now there is nothing wrong with an external SSD for other things though, I have a 2012 quad mini that boots from a 1TB Samsung T3 (the predecessor of the T5) and gave a 2012 base mini with a 500gb T3 to my daughter's family. I just don't think this makes sense for an iTunes server. Gigabit ethernet maxxes out at 120MB/sec, so you really gain nothing from a drive that delivers 500MB/sec (unless the server is also being used for other stuff).
The external hard drive with my iTunes library clocks at 180MB/sec and this is what I get when I access it as a shared drive on my LAN, so it's basically saturating the gigabit ethernet.