Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Storage Solutions for our OUT-OF-CONTROL iTunes Libraries
- Thread starter fivepoint
- Start date
- Sort by reaction score
Drobo with a dedicated mac mini hardwired into a router and wireless N to the aTV.
I am actually in the middle of ripping about 500 DVD's (110 so far and 3 weeks into it). It is extremely time consuming and the thought of loosing everything to a HD crash was not worth it. I looked into a RAID system, but I guess I'm getting old and just didn't want to deal with the hassle. With the Drobo if a drive fails you simple pull it out and plug in a new one and your done. No idea how it works but it does. You can't do that with RAID, so it was worth the extra expense too me.
One thing I did find was ripping DVDs on the mini was way too slow so I do all conversions on my Macbook Pro.
I am actually in the middle of ripping about 500 DVD's (110 so far and 3 weeks into it). It is extremely time consuming and the thought of loosing everything to a HD crash was not worth it. I looked into a RAID system, but I guess I'm getting old and just didn't want to deal with the hassle. With the Drobo if a drive fails you simple pull it out and plug in a new one and your done. No idea how it works but it does. You can't do that with RAID, so it was worth the extra expense too me.
One thing I did find was ripping DVDs on the mini was way too slow so I do all conversions on my Macbook Pro.
For this PowerMac G4 Digital Audio, I first added an Ultra PCI Sata card, bought two 500 GB internal drives (no desktop clutter and both drives are faster than firewire 400 or USB 2.0 could have handled regardless). One drive is used to make bootable backups of the other using Carbon Copy Clone. I created 3 partitions on each, transferred MacOSX Tiger and OS9 over from the stock 40GB drive (using a small 5GB partition for OS9, a 45GB partition for Tiger and left the rest for Leopard and storage of iTunes libraries). I have around 375 CDs in the library stored as Apple Lossless, which takes up over 120 GB and I also have a 2nd library converted from the Lossless one stored as 256kbit AAC to use on my iPod Touch (about 34GB worth there).
I've got another 320GB and 500GB drive on my PC and both are connected via Gigabit Ethernet to a NetGear Gigabit Draft N router. I then transferred a copy of both libraries over to the PC for a secondary backup in case the Mac ever goes south on me. Two AppleTV units are in the house along with an Airport Express DraftN that I can stream iTunes to via AirTunes from either the Mac or PC.
I think as long as I don't buy tv shows or movies, I won't need additional storage any time soon. Frankly, I still use DVDs for buying movies. I do rent HD movies using AppleTV, though. Of course, those don't stay on any drive since they're rentals. If I did need more storage, I could always add another Sata card and a couple of 1TB drives (or even 2TB drives in the future).
Despite the official count of like 3 internal hard drives, there's actually plenty of room for about 6 hard drives internally in this PowerMac if I can get more drive carrier plates (two are stacked on the far right side now; there are mounting holes on the left side and room in the middle as well.) The PC has room for two more hard drives as well internally.
I've got another 320GB and 500GB drive on my PC and both are connected via Gigabit Ethernet to a NetGear Gigabit Draft N router. I then transferred a copy of both libraries over to the PC for a secondary backup in case the Mac ever goes south on me. Two AppleTV units are in the house along with an Airport Express DraftN that I can stream iTunes to via AirTunes from either the Mac or PC.
I think as long as I don't buy tv shows or movies, I won't need additional storage any time soon. Frankly, I still use DVDs for buying movies. I do rent HD movies using AppleTV, though. Of course, those don't stay on any drive since they're rentals. If I did need more storage, I could always add another Sata card and a couple of 1TB drives (or even 2TB drives in the future).
Despite the official count of like 3 internal hard drives, there's actually plenty of room for about 6 hard drives internally in this PowerMac if I can get more drive carrier plates (two are stacked on the far right side now; there are mounting holes on the left side and room in the middle as well.) The PC has room for two more hard drives as well internally.
I put the master iTunes repository on External HDD #1. I then import the movies and TV shows into iTunes from HDD #2 and #3. And you're correct, I disabled "keep your files organized" in iTunes.
Not as much as I would like. Right now, I use the original DVDs as the backup for the Handbrake'd files. And for the iTunes purchased content, I make periodic DVD-ROM backups.
Any suggestions?
Ahhh... Ok, I see. But, if you buy something off of the iTunes store, what happens to the file? Does it put it in the 'master iTunes repository' and then you just go in manually and re-file it on the appropriate Hard Drive? I can see that.
Well, the DVDs are good, but a software backup would be better so you wouldn't have to re-rip all of that data after a catastrophic failure. I'm no expert (I started this thread to learn more about this and analyze the possiblities) but from what I know... I would say that a good choice for you would be to use software RAID to connect all three of your drives, re-compile all of the data from those three drives into your new software RAID drive and have iTunes begin organizing it all for you (reduced 'hand-work'). After that is done, get a Time Capsule or a Large External drive connected to an AEBS for a Time Machine drive, and have it back up your computer and your iTunes RAID drive.
This would give you quick 'lag-less' access to your iTunes library, eliminate your 'hand-work' media filing that you're doing now, and keep your large Time Machine drive away from your desk and out of the way.
Just a thought, I'm sure there are different and possibly better ways to go about it.
I don't think this would reduce the "hand work" at all. I mean when I rip TV Shows, I set up the folder struction right then and just do all my tagging and stuff like that. Having iTunes organize the files for me would actually make things less logical since I wouldn't be able to as easily navigate the folder structure to grab a specific show or season.
I see what you're saying... but when you're ripping some movies, and buying others from the iTunes Store, wouldn't it be easier to just keep all of the files in the same spot?
If you manually file them, then your iTunes Store downloads get put into the 'master iTunes' folder... and not with all of your ripped movies, right? Letting iTunes organize them all makes sure that however you get your content into iTunes, once it's there... it's all in the same spot.
Maybe I am missing something. If so, I apologize.
Well, the stuff from iTunes is all DRMed, so I don't really want to mix that up with my un-DRMed stuff.
When you started running out of space on your Mac, what long-term external storage solution did you switch to (or plan to switch to)?
Well, for me I built (currently have all the parts and am putting it together) and unRaid server.
List the hardware you've purchased.
I purchased a rocketfish case which was on sale at bestbuy for like $40. It will hold a lot of hard drives without me having to add SATA backplanes.
I purchased the Abit AB9 Pro motherboard as it is supported by unRaid and has 9 built in SATA ports.
I purchased an Intel e1200 for use in the server
I purchased the CORSAIR 550W power supply.
I purchased 2GB of generic Ram
I am putting 2 750GB drives and 2 500GB drives into the server.
All in all the computer part of the server was not expensive to build. It's the hard drives that make the cost go up.
What advice can you give to others?
really think about what you want to do with moving your info to another place. I had thought about just going the external hard drive route but after considering and weighting all the options i figured it would be better (and cheaper) in the long run to just build some sort of server and add drives as i need to.
What would you do differently?
Nothing really. For my needs and the time i wanted to put in I think i made the right choice. Each person is going to have different needs and wants and requirements.
If you could design the ultimate storage setup for your ever-expanding iTunes library... what would it be?
At this point in time i think i have the correct setup for my needs. It will allow me to expand as the need arises.
I currently use a 1TB external drive.. however for backups and other reasons its getting pretty full... I am looking at purchasing another 1 TB drive or maybe a cheap G4 desktop and loading it up with hard drive space.. Kinda not sure what i want to do.
I only have 200 gbs of itunes.. its growing, but not that the rate some of the others are.
I only have 200 gbs of itunes.. its growing, but not that the rate some of the others are.
I currently have 4 drives attached to my AEBS via a USB hub.
1. 500 GB Media drive (iTunes Library + iPhoto Library + other random stuff)
2. 1TB backup drive - I use Apple's Backup program to backup the Media drive overnight
3. Time Machine drive for me
4. Time Machine drive for my wife
It all works pretty well, however, every month or two, I lose connection to all drives and I have to restart the Airport Extreme. I'm not sure why that happens, but once it restarts, I can reconnect to everything without problem.
I have just a mess of drives hooked up to my computer. I have a 750GB drive holding my TV shows (that one is getting pretty close to full), and several 500GB drives to store my movies and disk images for encoding.
Very cool. How nice that you don't have to keep all of those drives connected directly to your computer. Have you also had success with streaming speed of iTunes content to the computer and/or AppleTV? No LAG to speak of?
For those of you using Software RAID... a question.
When one of your RAID 0 drives fails, is the other drive any good? I mean... with striped content can you read the data off of the second drive if the first one fails... or do the two drives (once connected through RAID) need eachother to work properly?
Also, with Drives connected through Software RAID, does it matter whether or not you daisy-chain the drives or not?
When one of your RAID 0 drives fails, is the other drive any good? I mean... with striped content can you read the data off of the second drive if the first one fails... or do the two drives (once connected through RAID) need eachother to work properly?
Also, with Drives connected through Software RAID, does it matter whether or not you daisy-chain the drives or not?
Saw this article this morning on Macdailynews.com, and thought it was particularly relevant to our conversation.
Apparently Apple will be including a new file system in Snow Leopard (ZFS) which will completely remove the need for RAID systems. I don't really understand the whole concept... but it sounds very promising for large future storage solutions.
No RAID cards or controllers
ZFS implements very fast RAID that fixes the performance knock-off against software RAID. In ZFS all writes are the fastest kind: full stripe writes. And the RAID is running on the fastest processor in your system (your Mac), rather than some 3-5 year old microcontroller.
Just add drives to your system and you have a fast RAID system. With Serial Attach SCSI and SATA drives youll pay for the drives (cheap and getting cheaper), cables and enclosures.
No more volumes
Every time you add a disk to your Mac you see another disk icon on the desktop. If you want to RAID some disks you use Disk Utility (or something) to create the volume. Slow, error-prone, confusing.
ZFS eliminates the whole volume concept. Add a disk or five to your system and it joins your storage pool. More capacity. Not more management.
Apparently Apple will be including a new file system in Snow Leopard (ZFS) which will completely remove the need for RAID systems. I don't really understand the whole concept... but it sounds very promising for large future storage solutions.
No RAID cards or controllers
ZFS implements very fast RAID that fixes the performance knock-off against software RAID. In ZFS all writes are the fastest kind: full stripe writes. And the RAID is running on the fastest processor in your system (your Mac), rather than some 3-5 year old microcontroller.
Just add drives to your system and you have a fast RAID system. With Serial Attach SCSI and SATA drives youll pay for the drives (cheap and getting cheaper), cables and enclosures.
No more volumes
Every time you add a disk to your Mac you see another disk icon on the desktop. If you want to RAID some disks you use Disk Utility (or something) to create the volume. Slow, error-prone, confusing.
ZFS eliminates the whole volume concept. Add a disk or five to your system and it joins your storage pool. More capacity. Not more management.
I have a stutter every now and then when streaming to Apple TV, but that's pretty rare, and once it gets playing again, it's fine.
There are many different kinds of RAID. Raid 5 allows you to lose 1 drive and be ok (if you lose more then one ALL data is lost) as it can rebuild that lost drive from the parity drive. Raid 0 (also called scary RAID) just stripes data across the drives. It is very fast at read and write but if it fails you are SOL. There are many other types of RAID. The one I plan to use (unRaid) is a "subset" (sorta) of raid 5. It allows for a parity disk and if you lose one drive you can rebuild the lost one. The difference between it and raid five is that if you lose 2 data drives, while you are still SOL, you will not lose the data that is on the drives that is not. Raid 5 stripes data across drive and unRaid does not.
For a more in depth read on RAID i suggest wiki.
EDIT:
Just figured i would go ahead and post back in here and say that i have finally gotten around to setting up my unRaid box. All went really smooth and i could not be happier right now. I did have one hickup but that was because the video card i was trying to use was bad. setting up everything is fairly straight forward. I am using the free version (allows 3 drives; 1 parity and 2 data) and am transferring everything over tonight...we shall see how long it takes. Going to look into setting it up so that my music can be served out over the internet using firefly/mt-daapd.
A couple things i have noticed:
typing //tower into a mac browser does not take my to the unRaid server like it is supposed to. Instead i must enter the ip address (which i made static on my internal LAN).
Thank you. I am aware of RAID 5 and it's many benefits. What I am wondering about however is RAID 0. If you lose one drive, what is that status of the other?
Very cool. This type of setup interests me greatly... I definitely want to try it sometime. One more question... with this setup using a networked iTunes Library, are you able to access the same library from multiple computers at once, and make changes from all comptuers simutaneously... which are soon reflected on each computer?
My Sharing setup
I had commented earlier how few weeks, the AEBS seems to lose connection to the USB drives and require a restart... Funny that how the same day I wrote this, I went through one of those network crashes that required restarting the AEBS!
To recap: I have an AEBS with USB hub and 4 drives attached. Media Library, Backup drive, and Time Machine drive for my MacBook Pro, and Time Machine drive for my wife's MacBook.
Keeping everything in sync is not automatic. I handle it two different ways:
Does anyone else have a similar setup? I'd appreciate any suggestions or pointers for optimizing.
Doug
I had commented earlier how few weeks, the AEBS seems to lose connection to the USB drives and require a restart... Funny that how the same day I wrote this, I went through one of those network crashes that required restarting the AEBS!
To recap: I have an AEBS with USB hub and 4 drives attached. Media Library, Backup drive, and Time Machine drive for my MacBook Pro, and Time Machine drive for my wife's MacBook.
Keeping everything in sync is not automatic. I handle it two different ways:
- For iTunes media (music and movies), my wife and I share the same files, but we have two separate iTunes libraries. That way, we can have different ratings for our songs, and we can create different playlists. We have Sharing enabled, so that I can see her playlists from my computer, and vice versa. The iTunes library XML files are still kept on our respective computers. The Apple TV is primarily associated with one computer. When I add some files to one library, I have to manually add them to the other.
- For our iPhoto library, we want to have one master library, so the library metadata files are actually stored on the networked drive. PROS: Either one of us can have access to our photos on either computer. CONS: Only one of us can have iPhoto open at a time. It's also slower with the library files on the network. Opening the library on a local computer can take a few seconds, on the network library, it can take up to minute.
Does anyone else have a similar setup? I'd appreciate any suggestions or pointers for optimizing.
Doug
Oh, ok... you do it a bit differently than I had imagined. I figured you had a 'folder' for each user in iTunes, with the playlists for that person inside of that folder. Then, you would only have one set of data, one set of .XML files, and one set of library data. Each individual computer would point to that same iTunes folder, and would share all of the organizational data as well.
If this was the setup you had, I am curious if 2 users could be using iTunes at the same time, and making changes, etc. at the same time.
Thecus N5200BR Pro
Hi - I have had similar issues with storage (currently counting 480 movies and 980 TV episodes - not to mention music). The only solution that worked for me was to get that external NAS and use it to as my default iTunes location. It works great (currently has 5x1TB on a RAID 5 arrangement which gives me about 3.7TB of storage). Plugged into my AEBS and utilising AFP over gigabit it works miracles with my 2
TVs and MacMini. Also, as it has a 4 port hub attached to it, you can add more gigabit ports to your AEBS without additional hubs/switches and power bricks. The machine runs great and power consumption and noise are kept to a minimum.
P.S. so glad about the genre option inTV - it takes ages to find a movie otherwise... if only frontrow would get through one of the pending updates...
Hi - I have had similar issues with storage (currently counting 480 movies and 980 TV episodes - not to mention music). The only solution that worked for me was to get that external NAS and use it to as my default iTunes location. It works great (currently has 5x1TB on a RAID 5 arrangement which gives me about 3.7TB of storage). Plugged into my AEBS and utilising AFP over gigabit it works miracles with my 2
TVs and MacMini. Also, as it has a 4 port hub attached to it, you can add more gigabit ports to your AEBS without additional hubs/switches and power bricks. The machine runs great and power consumption and noise are kept to a minimum. P.S. so glad about the genre option in
TV - it takes ages to find a movie otherwise... if only frontrow would get through one of the pending updates...Way Out of Control
Mine is way out of control and I'm in the process of transitioning. Bought a Mac Pro and originally had two 500gb Samsungs in it, but have since gone to three 750gb Samsungs with one partitioned into a 500 for iTunes and a 200 for movies. I am currently going naked on backup but am also in the process of creating my backup drive, an Addonics case, which holds up to eight drives. It currently has the two 500gb Samsungs, and will get four more Samsung 750s. I will need a PCIe card for the MacPro toi handle an eSATA connection, so only one card to access up to four external drives. I also have three LaCies, two 500gbs and one 250, all firewire, which will get spread out to my family network, our tastes don't run the same.
All of this will allow me to continue my obsession with music (over 4000 cds, 1500 lps, 2000 cassettes (over 30 years of collecting)), and also put the family history in photos onto one of the drives, the next purchase being a film scanner.
Does this qualify for out-of-control?
Mine is way out of control and I'm in the process of transitioning. Bought a Mac Pro and originally had two 500gb Samsungs in it, but have since gone to three 750gb Samsungs with one partitioned into a 500 for iTunes and a 200 for movies. I am currently going naked on backup but am also in the process of creating my backup drive, an Addonics case, which holds up to eight drives. It currently has the two 500gb Samsungs, and will get four more Samsung 750s. I will need a PCIe card for the MacPro toi handle an eSATA connection, so only one card to access up to four external drives. I also have three LaCies, two 500gbs and one 250, all firewire, which will get spread out to my family network, our tastes don't run the same.
All of this will allow me to continue my obsession with music (over 4000 cds, 1500 lps, 2000 cassettes (over 30 years of collecting)), and also put the family history in photos onto one of the drives, the next purchase being a film scanner.
Does this qualify for out-of-control?
Interesting thread. I've been at a crossroads recently with my library as it grew to over 300 Gb on my iMac's internal HDD.
If it wasn't for Apple TV, I'd have gone down the NAS route, but I couldn't find a NAS which supported ATV natively. So instead I picked up a G5 tower, stuck a couple of 1TB drives in it, mirrored them and moved my library to it. I then pointed all the devices in the house to it and job's a good 'un.
Or so I thought.
Then I started to discover some little annoyances. Like how Front Row wouldn't connect to the library - this is a known bug apparently and there's some workarounds/fixes which work for some but not for others.
Not a big deal, but when you connect to a shared iTunes library, things like movies, podcasts etc are presented as a playlist rather than a nice categorised front end.
Synching iPods from a shared library requires you to do a workaround which I posted here last week. Bit convoluted and a bit annoying considering I'm not wanting to do anything illegal, so I'd have thought there would be an official way to get round this problem.
Streaming of audio and video works fine, but I'm still trying to work out some of the little problems with it.
In the mean time I've upgraded the HDD in my iMac to 1TB as well, so I have the option of just putting everything back on there if I need to and backing up to an external HDD.
If it wasn't for Apple TV, I'd have gone down the NAS route, but I couldn't find a NAS which supported ATV natively. So instead I picked up a G5 tower, stuck a couple of 1TB drives in it, mirrored them and moved my library to it. I then pointed all the devices in the house to it and job's a good 'un.
Or so I thought.
Then I started to discover some little annoyances. Like how Front Row wouldn't connect to the library - this is a known bug apparently and there's some workarounds/fixes which work for some but not for others.
Not a big deal, but when you connect to a shared iTunes library, things like movies, podcasts etc are presented as a playlist rather than a nice categorised front end.
Synching iPods from a shared library requires you to do a workaround which I posted here last week. Bit convoluted and a bit annoying considering I'm not wanting to do anything illegal, so I'd have thought there would be an official way to get round this problem.
Streaming of audio and video works fine, but I'm still trying to work out some of the little problems with it.
In the mean time I've upgraded the HDD in my iMac to 1TB as well, so I have the option of just putting everything back on there if I need to and backing up to an external HDD.