Two weeks back I went to make an MP3 (128Kbps Radium codec) backup of my Faith No More Live album.

Shock horror - the CD is deteriorating. If I remember correctly it's flaking at the edges.

I have maintained an extensive collection of MP3s (95% my own CDs) for a few years now, and had recently begun to think about a better backup approach. I was at that time simply considering LAME 192Kbps VBR MP3. I know, I know...

However, now I am a lossless codec convert. I first came across Monkeys Audio well over a year ago, but never really did much with it.

I have been twisting and turning about whether to go with Monkeys Audio, FLAC or LA. Until this morning, when I read this thread, I was going to go with LA (even though it gets little press here), as its compression is definately superior. However, I may now go with the mid-point of these three - Monkeys Audio (Extra High). Who knows, tomorrow I may decide to go with FLAC (-8 / --best).

I don't see much point into getting into yet another "Which codec is best" debate - horses for courses/personal preference/weird Linux users/etc. I have been visiting these forums now for around two weeks, and I think I have seen enough to realise that there is no definative answer to this.

My question takes another tack on the "I want to backup my CDs" route:

What is your medium of choice?

CDs (surely not)? DVDs? Internal Harddrive? External Harddrive?

I know this may be slightly off-topic - but I would suggest that most lossless users are backup freaks, and therefore I see this as relevant, or at least of interest.

I have always been concerned with the longevity of CDs (see sentence 2). Are DVDs any better? NB: I currently only have a CD burner. If I get them all onto a 120GB hardrive (most likely external - whose transfer errors also concerns me) and only ever use that drive once in a blue moon, i.e.: if I discover that a CD has spontaneously combusted, will it last until I die?

I would be most interested to hear from users who have backed up to DVD for a while, or who use external (I have Firewire) hardrives. Oh yeah, and hot chicks. A hot chick with an external hardrive!

what abouy storage on optical mediea like CD and DVD
and do some PAR parity on another disk.

Kinda like a raid 3 but on optical medium instead of HDD's

I just don't see harddrives as a good solution. Maybe for near-line "hot" backup, but not for long term storage. If nothing else, consider that in a couple of years IDE hard drives may have gone the way old MFM and RLL drives went. If your computer doesn't have a controller, what will you do? What if firewire goes away? What if you store the drive in the closet and your girlfriend reaches up on that top shelf to pull a sweater down and pulls your drive with it?

There's a thread around here someplace where we discussed archiving to CD/DVD with PAR information. Even on CDs that were pretty well fragged (one guy put electrical tape over a section of the disc) a combination of Isobuster and PAR managed to recover all the data. A search on isobuster and par would probably turn it up.

So I've decided on flaccing CD images (ripped with EAC's "Create Image with CUE file" option) and then copying them to DVD (10- to 15- CDs per DVD) with PAR redundancy of about 10%, and then storing all the tools used (Flac, QuickPAR, Isobuster) in a Util folder on each DVD. That way if Isobuster, PAR and Flac all go away, I'll have the tools available at some time in the future to recapture the original wavs.
----
Moderation: removed long off-topic signature.

I use HD-s They are dirt cheap fast and very reliable (if you keep the temperature down). I have good cooling. I also use RAID1. Plus I have 2 SATA cables are lead outside the case of my server. This is how I back up onto some spare drives. The server only runs Linux built from sources (Gentoo). No virus. This PC never goes onto internet. It provides a read only samba mount to the other PC-s.

As for CD-s. What happens if your drive refuse to read your CD. PAR is pretty useless, then.

I use a USB2 external hard drive (Seagate 160 GB). Works nicely.

ff123

It's a good idea, but make sure to use PAR2's instead of the original PAR format. PAR2's will work with partial files, original PAR only worked with whole files. Use a pretty small blocksize too, I'd say. In fact, if you make the blocksize equal to the sector size on the CD, that might work out really well.

I agree--my problem with hard drive backups is that you're putting all of your eggs in one basket. One hard drive problem on the backup and poof it's all either gone or will require expensive hard drive work to recover it.

With CD/DVDs, your data can be cheaply spread across multiple locations. Even at $1 per DVDR and 10% PAR, you can archive a 40 gig archive for $10 each time around. So, one disc goes bad, so what?

It is quite true that CD/DVDs are more probably more prone to fail in some unrecoverable way than hard drives. But a single hard drive situation still has the problem that it fails badly, whereas if you burn a new copy of your archive every few months it'll cost you only a couple bucks and you'll have yet another layer of redundancy. Also, if you occassionally test the discs you can see problems developing and take corrective action immediately. It's harder to spot a hard drive going bad before it actually loses data irretrievably, though someone with more knowledge here might correct me.

I'm a little reticent to post as I don't want to stop the thread. However, I felt it necessary to thank everyone for the help so far.

I had seen PAR mentioned previously - by someone who was archiving using LA with PAR, but have not heard of it before that.

Thanks to Sven's original post I decided to investigate. I checked it out last night briefly. I don't really understand what I'm doing, but I do think it's something that I would use (pros outweigh cons/filesize). Thanks for the suggestions for redundancy and blocksize. I tested both par2cmdline and QuickPar - and got different results (filesizes) from each. I guess that must be to do with QuickPar's default settings/blocksize/redundancy values.

Like rfarris, it would be my intention to save any utils used on each CD disc (I'll add ISOBuster now as well). This is what confuses me when some people show concern about applications becoming obsolete - if you have been sensible enough to store the EXE then no-one can stop you from using it in the future - even if it is no longer supported and you miraculously cannot find it anywhere on the Internet. I don't get it...

I love the idea of have SATA cables accessible externally as Triza - but my motherboard is quite old and I don't have SATA. Definately something I'll consider when I eventually upgrade my PC.

I think I may be being persuaded toward a DVD solution at the moment - especially as the initial outlay would be less, and my wife is not keen on me spending £100 on an external harddrive at the moment! (another baby due imminently)

Please keep the suggestions coming though, it's been most valuable so far.

Thanks, I assume it is this thread. Looks like I have another app to check out.

Actually - it's this one.

I use TDK Armor Plated DVD-R's to back-up my mp3/mpc's.

Yep, that would've been me, I suppose, in the same thread you mentioned earlier.

Still solely on HDDs here. An internal one I use all the time for my FLACs, an external one that's touched maybe a couple of times a week for an extra copy of the collection transcoded to La, plus extra PAR2 reinforcement. The odds of both disks dying at the same time are negligible. Simple, fast, convenient and relatively cheap (2 x 250 GB of storage, worth 2 x 800 losslessly encoded albums, consider what that costs nowadays!).

Hard disks have mechanical parts & are open to viruses unlike write once media such CD/DVD. Hard disks are really a bad idea for backing important data.

If you don't have decent anti-virus protection, viruses or other malware can just as easily be written to CD/DVD. No storage solution will ever relieve you of the need for efficient security (anti-virus, firewall, anti-spyware, ...). To me, having the full redundancy of a back-up on an external HDD provides sufficient peace of mind.

Just a few thoughts:

As for viruses. I have a dedicated linux file server that built from sources (gentoo linux) and never goes onto internet or anything. I do not plan to update it frequently either. My everyday machine does only have read only access via Samba. If I want to dump something onto that machine there is a small pub directory is awaylable as a writable dir. Then I log onto my file server and move those files where they fit.

HD-s are also good because it is very easy to replicate. It takes less than 3 hours to replicate a 160GByte drive. No manual intervention is require. Similarly I have MD5SUM checksums, which are also signed by GPG so I can notice if somebody alters them :-) and I can check 160GByte unassisted. Try to replicate or just check your CD/DVD this fast.

I do not use PAR because I can afford multiple HD-s, which are at multiple locations. Also I find it useless if HD breaks or CD drive refuses to read the CD. I do not know how frequently this can happen.

Also strangly I never had a dying CD either despite the fact that I used that so-called 'CMC' manufacturer's CD-s that appears to be the worst. Well I was not very well informed before. I think this is because I burned 2x speed, which is unbearable as a backup solution.

I plan to skip DVD revolution altogether.

Suppose you mean 160 GB?

this thread deals with the same:
http://www.hydrogenaudio.org/forums/index....wtopic=23014&hl

I'll bet you that spinning hard drives aren't within two orders of magnitude of the reliability of CD/DVD media.

I thought you guys were talking about archiving to a hard drive and then putting it in the closet until needed. Even then I'd rather have the fail-soft aspect of multiple DVDs.

What he said!

I don't believe it. If you've got 100 pieces of media laying around, then sure, over the years you're going to see errors, but how many hard drives do you know of that have lasted even five years? Suppose you had 100 hard drives. How many do you think would still be alive after five years?

-- Rick

----
Moderation: removed long off-topic signature.

The anti-HD responses are a little extreme! IDE obsolete in a couple of years; viruses eating up your stuff on HD while it's sitting on your shelf!
DVD's are very time-consuming, but reliable I think if you have a good drive and media. Hard drives apparently can't be left safely for long periods (years) unused - that's what I hear.
I don't need ultra-reliable backup as my main stuff is unlikely to go and in the very unlikely case that my main stuff and backup both go I have the original CDs. So I'll go HD I think.
Unless anyone uses MiniDV sucessfully? That would be great for me.

Has anyone had this happen? I've not had any trouble with hard drives, and my family's had a lot of computers. CDRs I've had trouble with, but probably because of going with cheap discs.

I've had two drives fail after about six years each, but also a bunch of others that have yet to fail (though they were all used about five years and then sold to other people when I upgraded). I don't mean to sound anti-HD--I think the odds of something happening to both your main and your backup in a close enough time frame to preclude a restoration of your backup are very, very small, but you'll need to test your backup every now and then to make sure it's still in working order.

Part of the reason I go with CD/DVD is that I had a particularly bad experience once with a hard drive which, immediately upon being inserted into a WinXP computer, sent XP into a fit in which it first "repaired lost clusters" and then "repaired orphaned files." Needless to say, XP had orphaned the files in the first place, and when I tried to access the MP3s or other files I would be pointed towards parts of other files, which was the most bizarre thing I've ever seen happen to any file system. Oddly, upon putting it into another XP computer, all the files were, umm, reunited with their parents, and the drive functioned perfectly.

The point is that, from then on, I started using backups which could not be overwritten, like DVDRs. I think most people will be fine with an extra HD (I just bought the same for my brother for his birthday), and it is more convienent for a lot of people (I don't think burning a bunch of DVDs is that bad but, then again, my whole collection is only about 70 gigs--I just backup the compressed versions, they're transparent to me and I'm not going to edit them or the like), but I like the cost and multiplicity of CD/DVD backups.

As for some real facts, this article at PC Mag has an unsourced assertion that "The average time-till-failure of the 7200 rpm drives seems to be about 30 months; the average time-till-failure of 5400 drives is a lot longer -- about 60 months. In all cases, the drives are run pretty much constantly, day in and day out."

Most hard drives have mean time before failure of around 300,000 hours or so, which equates to 34 years, but I'm curious what they're actually measuring there, as that really doesn't fit within most people's practical experience. Storage Review has a reliability survey, but I can't seem to get on there right now.

Yes you will see a lot bad stories about CD-R, DVD-R etc. because of the amount of cheap media, cheap burners & excessive recording speeds plus that it’s rather fragile design compared to a hard disk drives which have thick steal protecting them.

If using CD-R I would follow these steps for something I don’t want to lose,

-Use quality media
-Burn at low speeds (8x)
-Check for errors after burn
-Store in a safe place (avoid Sunlight/Humidity/Water/Scratches/Extreme temperatures) - This is a hard disks advantage!
-Check for errors every 6 - 12 months

Yup. Sorry. I corrected my original post.

Yup. Although I have a RAID1 in my server which hosts all my FLAC-ed albums, occasionally I sync up its content with HD-s which are normally stashed away in various locations. Incidentally I just brought a copy of my entire collection on one 160GByte HD (I have only 300 CDs :-( ) to my parents' house, which is in a different country and few thousands km away where I live.

I agree, but here is the advantage of HD solution. It is faster to "burn" it. In fact if I just edit a few files I can easily update the other off-line HDs (by using rsync). You need to reburn those disks. Pertty manual.

Checking for errors are easier too. In fact I only have 300CDs so one 160GB can hold the lot, so I do not even need to change disks.

Triza

Are you kidding? Your family hasn't had nearly as many computers as any small (let along anything larger) business IT manager.

I don't know of a single IT manager that considers hard drives a valid backup media. I was willing to work with you guys when I thought you were talking about backing up to a hard drive and then putting it in a closet, but the idea of something that is online and in the same physical location as the data it is supposed to backing up is laughable.

This is not rocket-science, guys. A single slip of the finger, or a single virus can wipe out all your online storage at one time. RAID 1 and other redundant online storage schemes are designed to increase system availability. You will nowhere find it claimed that they are backup solutions.

-- Rick

----
Moderation: removed long off-topic signature.

As I said I have off-line copies in (namely at 3 different locations) on off-line HDs. Also if you look at my solution, there is virtually no chance for virus. I am very disciplined about it. Plus I have RAID1. But I know RAID1 is not backup. It does not prevent from accidental deletion and virus. But as I said I have several off-line copies in different geographical locations. In fact I am uniquely paranoid. Believe me!

One location is 2500 km-s away from me :-)

But again my off-line copies are HD based.

Triza

I use an offline HD.
About once a month, I plug it in, backup all the data again. Sometimes I completely delete it first from the backup. Anyway, I always browse through it from some time just after the backup in order to ensure that it works.
I don't see any egg problem. If the drive gets broken, I buy another one, and backup again. I can see if it gets broken every month. I wouldn't check DVD or CDR as often as that.
Considering how my CDRs behaved, I would have lost at least 30 % of what I've got if I did backup it on them. The CDRs failed no more than one year from each other, depending on the locations. In many cases, I would have lost both the files and the recovery files in all locations, unless I get them all back at home every three monthes in order to check them all.

In my opinion, the point is not what media, the point is how you handle your backup. ANY media will fail you if you store it somewhere and forget about it (exept maybe vinyl records). If you care about your data, check the backups regularly. For example, in the supermarkets for which I work, a full backup of the server is done every day and stored off line. All that people has to do, every day, is putting the DDS tape of the day in the player, and store the one inside the drive (from the day before) in its case.

I built a seperate server for all my data and store everything on a RAID 5 array - that way if I have an HD failure I can just replace the dodgy drive and rebuild whilst losing nothing. But for belt and braces I back up everything vital every few months on a set of DAT DDS 2 tapes - takes about 20 but I just let it run in the background and swap tapes when I notice they have popped out. All this stuff can be bought cheap on eBay and is well worth doing for peace of mind

I have also spent some time worrying about this problem, most solutions ignore the problems created by atmospheric detonations of neutron bombs too. I have started building a clustering mirrored server solution in a hardened bunker 40 miles from my home. I had some challenges with running the replication over this distance because of latency however these have been solved now (ESCOM is no longer part of the soltution). If you need help plotting blast zones, or calculating EMP intensities then PM me.

Love,
Fairy

I'm a little confused. Having investigated QuickPAR I must say that it is one of the more confusing UIs I've ever had the pleasure of interacting with. When you say "blocksize," do you mean the block size that you set on the "Options" page next to the "Exact" radio button? (BTW, blocksize for all DVD media is 2,048 bytes)

If so, how does that relate to the "Block allocation" block size? I can tell you one thing, if you make the "Block allocation" block size as small as you can (no where near 2,048 bytes) the "Compute time" becomes staggering.

On two 5 GB album-wavs, I set one for a block size of 2MB and it took about 1.5 minutes to compute. The other I set for a block size of 150KB and it took about 10 minutes. (P4-3.2, 1GB) All with 10% redundancy.

Any pointers?

@rfarris: I'd suggest checking out the QuickPar forum for storing on DVDR/CDR.

I agree with westgrov. The mean time between failures is far too high for a hard drive to consider it a 'backup' media. Taiyo Yuden CD-R is great media. Search the internet for reviews. Sony is great media too, surprise, it is mostly relabeled Taiyo Yuden.

I use hard drives plus CD-RW and/or DVD+RW to backup my data, which includes both personal and business data.

I have two PCs on my network, one handheld device (PDA) and a removable 60GB drive in my car. The PCs each have 40GB internals, and the PDA uses Memory Sticks up to 512MB in capacity. I back up the PDA to this PC (my primary client machine), then that data and all other data on this PC's hard drive get backed up to a shared external 250GB hard drive that is connected via Firewire to my server. The server's internal HD is backed up the same way to the external drive.

Then all critical data (anything irreplaceable) on any network drive gets burned to either CD-RW or DVD+RW, depending on data volume. I keep two such off-network backup sets stored in two different locations. The optical discs themselves are replaced with new ones every six months.

My music is kept on two hard drives (the 60GB in my car and the external 250GB network drive). Therefore my music collection exists in three places at all times: the original CDs and both hard drives. Any rare or irreplaceable music and/or video files are backed up to optical disc as well, though, just to be safe.

In the case of data corruption or losing an internal HD, I can do quick recoveries from the external HD. In the case of losing the external, I can recover data from one of the optical disc backups.

So, in all, data is kept on a minimum of three backup sets, and more for some data (like what's on the PDA, which is backed up in four places: this PC, the external HD and two optical discs). Incremental backups are fully automated and done nightly. Burning the opticals is done weekly by hand. Standard backup activities require 5 to 10 minutes of my direct involvement each week. The rest takes care of itself (but I still do periodic integrity checks, though).


Edit: My obsession with data integrity comes from working for 13 years in IT operations management.

I also use external hdds. It's very easy to use and synchronize (just copying) and I truly wanted to forget those long filenames and various ISO problems.

I wasted many hours burning cdrs, and when you have 100 CDRs, then you need to make an index to find which one has the information you want.

You can have those cdrs in a single hdd and if you're worried, make a copy on another hdd, it's so damn easy.

Arf!!!

This morning I wanted to see something I recorded on a CD a while ago, just out of curiosity. I took for granted it was going to be readable. How wrong I was!! argh

This is a Verbatim DataLifePlus (MetalAzo dye, or so they call it) written at 4x (max speed of my old sony crx-100e) in february 2002. Stored all this time in my room, so no sun, no humidity, no heat, anything.



arfg!!! and I thought this was top-quality media!! It's disintegrating like any crappy cdr else, it just lasted longer....

I won't ever trust optical media anymore. HDs with external enclosures for me from now on.

I second this. Many of my MPC files and html pages have too long filenames to be burned on CDR.

Lucas, did you check the CD for errors after burning? can you use a tool such as Nero InfoTool to check who the CD is made by?


Use UDF file system ; ) .

Well, I didn't check for errors, but I watched the content from the disc after I burned it. I can't seem to be able to use any ATIP reader to see the information, I don't know why in my computer I've never been able to see it, no matter what program I use (I just tried nero infotool, nero cdspeed, cdr identifier and cdr atip reader and neither worked...)

I use IDE HD for storage. Because hard drives are cheap, reliable and monitor-able with SMART. Optical storage has quite a random life duration/reliability.
As soon as I will have a Linux-proof computer, I will use software RAID 5. RAID 5 uses more than 3 drives, if one drive fails, no data is lost.

Optical storage has a much longer shelf life provided you use good media. Taiyo Yuden is well known for its high quality. Some internet research on backup media and methods would be a good idea. Tape, DVD, and CD-R are generally accepted in the IT community. www.tomshardware.com is a good site.

I have worked around long filename issues in the past by using WinRAR or 7zip. I didn't get a much smaller file, but it did a great job of preserving the filenames.

Edit: Added link

Tom's Hardware storage article

I archive music on Verbatim DVD-Rs, compressed using Apple Lossless.

I have never had any problems with long file names, neither on DVDs nor CDs, but that's probably because I don't use Windows. I usually use either UDF or hybrids of ISO, UDF and HFS.

Atici, thank you. That was exactly the pointer I needed.

For search purposes: Anyone who would like to use QuickPar or some other PAR2 (par2cmdline, say) parity generator to add redundancy to CD/DVD archives of media files should check out the QuickPar forum which includes tutorials and other helpful information for beginners.

-- Rick

Some drives can't read the ATIP after the CD is burned. Read it on blank CDRs.

Got a 120GB HDD hooked up for my Audio Needs. FLAC is my preffered method. If I dig the whole CD, like Miles Davis or The Doors or something, I'll archive it in a lossless codec. But, if I am only after two or three songs (majority of my CDs), I usually just rip with OGG Vorbis. Very few of my CDs are worthy of a full tracklisting lossless backup.

Ok, heres my take on this. I would invest some money and purchase and external harddrive. Keep all the packaging that comes with it. Fill the harddrive up. Package it backup in the box. Put it in a closet. Come back every couple of weeks and use (spin the harddrive).

I also consider Raid.