So, my next project when I have some cash is to make a big ole file server. The plan is to get a bunch of large, cheap IDE hard drives, stick them into a machine and rip all my music to it, losslessly (unconcerned with format here). After the music is there then I can do anything with the data.

I'm concerned with what happens should one of the massive hard drives die.

I figure that my biggest investment is the time to rip all my cd's (probably around 400 or so) to a standard, organize them, tag them, etc.

Having a hard drive die would be terrible if it would cause me to re-rip all the music.

What would be the best way to avoid that? I've been thinking RAID5 would give me the fault tolerance that I require. I can't see two hard drives dying at once...The cost there is one extra hard drive. I could even use software raid as performance is not too critical at all, just the fault tolerance. No extra time required.

I've seen that someone people use external drives as backups, which would require a 1:1 cost basis in hard drives. I think that's too rich for me. Of course, the backup factor is pretty good.

Another option would be to buy a dvd drive (handy to have anyway) and burn copies of the files on to it as they're ripped. I could even use some par2 data. The cost there would be the drive (good to have anyway) plus media. Requires a whole lot of time.

I could always just go with using the original cd's as backups (although I'll need to be very good about burning copies of downloaded live albums). Of course, it's a huge pain in the ass to have to re-rip. No cost except if something goes wrong, which is somewhat inevitable.

Are there other good options that I'm not thinking of?

What do you use? What are you thinking of using?

Pro RAID5:

Quite easy to set up
No manual work required
Low cost
Somewhat safer

Con RAID5:

Software: requires support in the OS (binds you to it)
Hardware: requires drivers
If you use same HDDs then they may die at the same time
Vulnerable to power surges/outages (good UPS required)
Vulnerable to viruses, software failures, OS bugs, RAID driver bugs
Vulnerable to fires, floods
HDDs in use wear faster than ones offline

Pro external HDDs:

Even easier to set up than RAID
Safer than RAID5
Can be put into safe to protect from fires and floods
Doesn't bind you to one OS
Doesn't require UPS

Con external HDDs:

Synchronising backup can be PITA
Slower than RAID5
More expensive than RAID5

Are you really sure about IDE drives ?

From what I have heard, IDE drive specs do not incorporate long-time runs (as being present in typical server environments) whereas SCSI drive specs do not handle lots of spinup/spindown cycles very well ...

RAID5 on IDE is definitely a good compromise between speed, pricing and data safety (S-ATA adds hotplugging, too) ... additionally, I would take care of HDD cooling (IDE drives do not feel well over 40°C - use filtered air to avoid some of the dust load on the drives) and vibration-absorbing drive suspension (something like that) ...

You also might want to consider using drives out of the 5400RPM class which do not create that much heat during operation and should be more reliable than 7200 or even 10000RPM drives ... a good SMART surveillance should be a thought as well so you'll be able to detect drive failures (SMART e.g. allows read/write error count readout) before they actually occur ...

Just my 2 cents ...

So, is an external drive in good storage conditions safer and longer-lasting than DVDs in storage conditions?

It depends on the brand of drive.

If it's an IBM Deathstar 45GXP, then I'd rather store the data on infamous IOMEGA 100MB ZIP disks

Here's some advice (for HDD):
- As JeanLuc said, take 5400RPM drives.
- Personally I'd choose Seagate (or maybe WD).
- Toshiba and IBM/Hitachi are a bit prone to mechanical failures
- WD's SMART feature is a bit paranoid about "possible" problems - but the drives seem to be VERY reliable. Run them for a few days/weeks before using them for backups.
- I don't have recent info about Quantum & others.
- If possible take external drives
- Do RAID1 or RAID5 !! If your OS can't do RAID5 on external drives, switch to a decent OS
- Do checksums (ie: MD5 / SHA1 / PAR)
- Check your drives regularly, and do not bump them vertically while they spin.
- Do not put tape on the drives.
- If possible fill your drives with random data several times, this will reveal bad sectors to the drives' electronics, and catch most early failures. If you use Windows, download a Knoppix CD and do "cat /dev/urandom > /dev/hda" several times (BE CAREFUL TO DO IT ON THE CORRECT DRIVE !!!!!). You can put this command in a batch file, and set an interval of ~ 2 hours to let the drive rest between passes. Then go visit your grandma (or GF) for the weekend
- DO NOT USE THE FAT16/FAT32 FILESYSTEMS ON THE DRIVES !!

I hope this helps.

If you're afraid about HDD failures, try DVD-RAM discs. There's a lower probability of serious data loss (and when it happens it's unlikely to lose the whole disc), but cost per MB is higher. Again, use ext3/UDF on the discs, not FAT32.

Have fun !

JeanLuc > ... whereas SCSI drive specs do not handle lots of spinup/spindown cycles very well

I've never seen this mentioned anywhere, and have used SCSI drives for years. Maybe this was an issue when drives didn't auto-park, but modern drives have been doing this for quite a few years. In many cases, the core drive from a particular mfr is identical between IDE and SCSI in any given size. Only the controller differs. A server here has (3) SCSI HD's (Seagates) and after (7) years of daily power ups & downs, only one drive has failed.

xen-uno

Well, I could not find any relating info on that topic ... in fact, one of the sys admins in my company told me so ... maybe he was referring to "older" days where SCSI drives were optimised for long-run operational mode only - nowadays, SCSI is used in desktop environments as well ... I should have clarified that.

I just need some clarification here.........I got 2 hds,one i use like my primary hd with os installed and for daily varius use,and one big ide drive-7200rpm for my audio and video files......my questions are.....
is it correct that 5400 rpm will last longer then 7200(less heat as mentioned above??)??????
I plan to add one more hd.........purpose is to put some more audio-video files on it...scsi is out of question......so...which will last longest(reliability and durability)..
usb,fireware or ide....or its the same..just different interface...
also.....is it any software which will let me check the drives...so i can see that hd is getting damaged...so i can replace before it breaks down......or it comes bundled with drives..im on windows xp-home/sp1.....
Thanks....

I've had this topic over at storage review and I am contineously amazed at how many people consider a RAID solution as a backup device.

A RAID is in no way a Backup!

The RAID provides recovery for faults within the system but if something happens to your RAID (stolen, virus, accidental deletion) you will find out the difference between a RAID and a backup.

I realize that it is extra cost, but you will have to determine if it something that you can lose, personally I think the cost of HD space warrents backing up most data by whichever method you find best for you

My advice -

1. External backup drives are a better option than internal ones, because they are not susceptible to viruses / electrical surges / IDE controllers going bad.

2. DVD disks are better as well for the exact same reasons.

3. If using DVD disks, use PAR2 data recovery to ensure that your files don't get damaged.

4. If you decide to use Raid5, make sure that you use 5400rpm drives, have good cooling in your case, use a good filing system (such as NTFS or a Linux biased FS) and you use a SMART monitoring tool to make sure your drives are OK. Also, get a good surge protector / updated virus monitoring software.

Also, if you want to do something cheap but effective, buy 2 internal drives, use 1 to keep in the computer and just hook the second one up once a month or so and back up the data. Its not all that big of a pain, its fast to back up the data once the drive is hooked up, and if you keep the drive in a fire proof safe, I think you're pretty safe at not loosing the data. Good Luck!

Thanks for the tips.

I realize that RAID is not a backup solution, I don't think anyone suggested it in the thread.

I figure if I need 3-4 large drives to store my music, and knowing my luck with hard drives, I'm bound to get a bad apple...Having some security to survive the death of one drive would be great. That's a good point about mixing drive brands, though.

Also on the 5400 rpm drives.

I don't trust the enclosures that I can afford too much so I'll probably stick with internal drives. I don't have an excess of space or money (in an apartment), so I need to make this self-contained within a decent-sized case.

I'm not too worried about being tied to one os...If I ever want to switch os's, then by that time I'm sure storage will have decreased in price enough to make it simple.

If, for example, I use softraid5 on a storage array, but the small drive where the os resides dies (or I similarly destroy the os), it's possible for a new os installation to pick up the existing array, right?

<deleted because I found another recent thread discussing my question>

No need for RAID, no need to make dozens of DVD-Rs, just buy a hard drive, put it in, back up, take it out.

I've had my eye on an Infrant ReadyNAS NV. Although pricey, has up to 2Terabytes RAID 0,1,5 hot-swappable storage using SATA drives with steaming support, automatic volume expansion, small footprint, and multi-protocol support.

I don't know too much about hard drives and all this stuff, but on our old computer is a WD 8 gigabyte hard drive, that has been through countless on/off cycles and has been left on for long periods. It's about 6 years old, and hasn't had any failures. Even after several format/operating install cycles, it hasn't caused any problems. So I don't know. It's not like you're backing up nuclear secrets or something.

A bit off topic, but how well do those IDE-to-USB conversion kits work? I mean, for turning an internal IDE drive into an external. Internal ones are cheaper, by quite a bit.

I just saw a good review at http://www.tomsnetworking.com/2006/03/03/i...nt_readynas_nv/


I have seen similar things I have drives that are 5+ years old and working fine and other new drives that die in under 2 years. Though I have started to buy drives with 5 year warrenties and I haven't had any those ones die (touch wood) have had them for about a year and a half now.


I have an IDE->USB2/Firewire case I really like it and am going to buy some in the future. I got the type of case where you buy the case and drive seperatly. I already had teh drives so was cheaper that way.

The 2 things I don't like about is they are nosiy as often have a small and cheap fan running at a high RPM. Also the drive's SMART information isn't transmitted so don't get warning of drive failures.

I transferred all my CD music to hard disk in .AIFF uncompressed format over the last couple of years. I started off with a smallish (160Gb) external firewire drive, then a 250Gb, and now have a nearly full 500GB lacie FW800 drive for music storage. I haven't re-used the 250Gb drive, but kept it as a backup of half my music collection (in a cupboard, not connected.

Like yourself I was *very* concerned about backups, and initially I made sure that when 4Gb worth of CDs had been transferred (and checked that the files all play ok) I'd copy to DVD & verify. Backing up as you go isn't too bad or time consuming. I noted what tracks were on which DVD using the "group" field of iTunes for the archive number.

Partway through the process of transferring all the CDs, I using two drives, the 160Gb and another 120Gb, then had a major problem (caused by bad firewire firmware in one of the enclosures) which resulted in the 160Gb disk (full of music) suffering serious data corruption - however, I was able to recover all my data as the problem had only wiped the root of the directory tree, not the files & folders beneath - so disk recovery software could rebuild this from the "bottom up" - it took many hours to run, but was well worth it!

I have needed to recover from the DVDs a couple of times (but that was finger trouble on my part - accidently deleting the wrong things!)

I am now considering getting a second 500Gb firewire either using it with RAID to mirror my main music hard disk, or manually copying on disk to another whenever I add new music. This is mainly because of the time it would take to reload all 500GB back from DVDs if the 500Gb crashed.

I'm also considering converting all my .AIFFs on hard disk to Apple Lossless format, (which would save considerably on disk space) but even then I'd still keep DVD backups of everything in uncompressed .AIFF format - just in case, Apple Lossless is fairly new it's possible some problem may be discovered in future, no third party hardware currently supports it & it may not last in the long term as a format. I don't want to end up with 500GB of music in a format that turned out to be a dead-end, or discover it had a bug that irrecoverably damaged the audio files.

I would maually do it when add new cds. So if only turn on once a month hopefully extend its life. Or maybe alternate them one live one in a cupbaord or at work (off site backup)

@tekno.mage

I wouldn't worry too much about converting to Apple Lossless, yuo can always reconvert back to aiff or wav, and its not like support is going to end tommorrow. It may be phased out in a few years, but you''l have plenty of time to convert it.

FLAC is also getting some support on OSX. There is a FLAC spotlight plugin, a test was made avaliabl on this site, i never tried it myself however. there are also OSX FLAC command line applications, and some other programs support it as well. Rest assured FLAC is going nowhere.

Another added benefit, other than giving yourself roughly 1/2 yuo used space back as free space, is audio integrity checking (Md5 summing in FLAC) and tagging support.

Seems like a no brainer to me, it'll just take a bit of time to convert them all

Kristian