Help ripping ~30,000 CDs, Was “Help digitizing […]” ;) Options

[trail]

Hey! I'm a newbie around these forums, but hopefully I'll be around quite a bit. I really like the community here, and hope I can contribute in the future. But enough introduction, here's the interesting stuff:

I work for a college radio station, and we've decided to undertake the rather ambitious project of digitizing the CDs we've acquired over the years. This is a pretty monumental undertaking, so I'm looking to make this as painless and quick as possible. We have a very rough approximation of about 30,000 CDs that we're looking to convert to digital files, and it's my job to work out many of the more technical aspects of the project.

The problem with being a college radio station is that we're on a pretty limited budget. We can't afford any sort of robot or anything like that to help the process along, nor can we afford any sort of service, so we're stuck doing it ourselves. Thankfully, we have a bunch of people willing to put the time and effort in. We also aren't terribly picky about getting every rip totally 100% perfect. But I've done a fair bit of research, and here's the kind of plan I had in mind:

Ideally, we have one pretty decent quad-core desktop that we're planning to outfit with four CD drives. We have software that allows us to rip multiple discs at once to V0 MP3s which are stored on a small RAID 1 array inside the computer. I've done some informal ripping tests, and have narrowed down the two pieces of software that seem to work best to fre:ac and dBpoweramp. I have also tried EAC and simply ripping with MediaMonkey, but freac and dBpoweramp seemed the most efficient and easy to use. Now, if I decide to use one of these pieces of software (if anyone has any suggestions, I'm 100% open to them!) how can I configure them to make them as painless as possible? Would using multiple drives be an option? I found very little information about software that provided ripping from multiple drives simultaneously, so I'm assuming this is not a common feature. If not, would using different computers be our best bet? If anyone has any other suggestions about ripping multiple discs at the same time or other ways to improve efficiency then that would probably make my life much easier.

thanks for your time!

[Porcus]

Since this is turning into a discussion of storage (on a budget), here's my uneducated two cents, subject to change upon anyone's better arguments:

- RAID is not backup. RAID is a way to reduce the number of times you need to resort to your backup. RAID does not protect against a thief, a lightening strike, or a 'holy s**t, what did I just do?'. RAID5/6/Z/2Z gives you a limited time to replace a broken drive, that's all. That's a big deal if you care about uptime, but on a budget, you don't. You would rather take the array offline until you are sure it is OK again.

- Striping -- i.e., spreading one file over multiple drives -- (basically all RAIDs except RAID1 ... and some nonstandard solutions) is a bit dangerous: even if you have a fault-tolerance of 1 faulty drive of, say, 4 then you need all the other 3 in order to read a single file. You also need the RAID setup. That is, you cannot take a single drive out of the array and get anything out of it -- and if you will take the 3 working drives out, then you need to mount them in a RAID array that can read it.

- There is a proprietary solution called UnRAID which eliminates the issues of striping: it simply dedicates a drive as parity, monitors the other drives, and whenever you write to a drive, it also updates the parity drive. That means, you can take drive #2 out of the array, mount it on a different computer, and every file on drive #2 is readable. If drive #2 AND the parity drive is ruined -- then retrieve merely drive #2 from your backup and clone it. There is a performance loss (writing takes twice the time), but if media files are basically write-once-read-many, that is no issue.


If you still want to do striping (like, RAID5):

- Enclosure RAID with port multiplier? The www is full of complaints about data loss, so I dare not even try. Yes port multipliers slow things down (everything has to go through the same channel), and that might be one reason for issues -- the OS might give up because it sees the drive as unresponsive.
(I'm using a port multiplier myself, but with 5 individual drives, no striping, and it is still a bit stressful: I thought it would be no issue as I only read the file I'm playing, no writing -- so I thought: but Windows writes to the NTFS journal all the time, or something like that.)

- Stay away from 'hardware RAID'. Mainly because you won't actually get hardware RAID on a budget, even though some weasels market it as such -- it is done in the drivers, and kind of gives you all issues of hardware RAID and all issues of software RAID. And if you actually go for a hardware RAID card, then you need two identicals, in order to have a backup if the card breaks, further violating the 'on a budget' purpose.

- Linux software RAID? Less issues. FreeNAS with ZFS' RAID-Z? Tried it once on a too old box, ZFS does require a bit of resources.