Help - Search - Members - Calendar
Full Version: Hard disc crash
Hydrogenaudio Forums > Lossy Audio Compression > MP3 > MP3 - General
locutus
I really need your help. I had sometimes ago a HD crash. I could rescue nearly 80% of my songs, but now i have detected that my recovery action was not succesful on every file.
On some songs there is a cut somewhere in the song and another songstarts, in the same file. Some kind of bit chaos in the file.

Now the question: Is there any tool to find all those corrupt files? Would be too hard to check over 17k of files manually by listening. wink.gif
garym
QUOTE(locutus @ Aug 15 2007, 07:07) *

I really need your help. I had sometimes ago a HD crash. I could rescue nearly 80% of my songs, but now i have detected that my recovery action was not succesful on every file.
On some songs there is a cut somewhere in the song and another songstarts, in the same file. Some kind of bit chaos in the file.

Now the question: Is there any tool to find all those corrupt files? Would be too hard to check over 17k of files manually by listening. wink.gif


Not sure it will help in your situation (but can't hurt): try Foobar 9.4.3 program. Load all files into playlist window, select all, then right click, select "UTIL" then "verify integrity" This checks whether each song file can be decoded properly, etc.
Wook
I use Encspot to check the integrity of files as it will fairly rapidly scan a batch of files and tell you if there are any sync errors in the files. That would certainly be the case if a particular file jumps from one track to another, or has missing sections, so you should be able to tell which files have been corrupted during the recovery process.
locutus
QUOTE(Wook @ Aug 15 2007, 08:13) *

I use Encspot to check the integrity of files as it will fairly rapidly scan a batch of files and tell you if there are any sync errors in the files. That would certainly be the case if a particular file jumps from one track to another, or has missing sections, so you should be able to tell which files have been corrupted during the recovery process.


Thanks a lot, this is very helpfull. The only thing i am now worried about are the files with "last bad frame". I know very little about MP3 files. Can you tell me something about that?
pdq
That probably means that the file is truncated, so the last frame in the file is missing some of its contents.
ernstblaauw
QUOTE(pdq @ Aug 15 2007, 09:05) *

That probably means that the file is truncated, so the last frame in the file is missing some of its contents.

It is also possible an old (or buggy) encoder has been used to encode those files. I forgot which ones did not write a correct last frame, but they exist(ed).
pdq
Perhaps, but given the OP's situation I would expect there to be lots of truncated files and probably not very many with this particular encoder flaw.

Would it be insensitive of me to bring up the value of backing up one's files?
Whelkman
I had this issue years ago when I copied to CD and deleted the originals before thoroughly examining the data. Every song was shifted by about 30 seconds; thus I could listen to full songs only if I constructed a playlist with the files in their original burn order, which wasn't necessarily alphabetically or any other logical sorting. I ended up tossing the thing. Even though I lost stuff that was no longer available, it just wasn't worth the aggravation.
plnelson
QUOTE(locutus @ Aug 15 2007, 09:07) *

I really need your help. I had sometimes ago a HD crash. I could rescue nearly 80% of my songs, but now i have detected that my recovery action was not succesful on every file.
On some songs there is a cut somewhere in the song and another songstarts, in the same file. Some kind of bit chaos in the file.

Now the question: Is there any tool to find all those corrupt files? Would be too hard to check over 17k of files manually by listening. wink.gif


How did you do your rescue? Do you still have the crashed HD? There are commercial services that can do a remarkably thorough job of recovering files off of crashed/damaged HD's.

I'd like to ask another question but I just want to say upfront I'm sincerely not trying to be snarky - I just genuinely don't "get" it - Why wasn't this backed-up? I've had only one HD crash in 25 years of computing, but I keep both local and offsite backups on a rotating schedule plus DVD's in a safe-deposit box. But I see people all the time who don't do any backups. Why? Even if the risk of an HD crash is low, there's also fire and theft. And for me the biggest uses of backups is when I do something stupid and trash a file all by myself!

pdq
There are two kinds of people in the world; those that back up their files, and those that haven't had a HD crash yet.
Whelkman
Proper backups present a pretty high barrier, which is why, by no coincidence, almost no one does them:
  • Dedicated backup hardware Even if the device is as simple as a low end PC, it still represents a cost being that most don't have spare PCs about.
  • 2x storage cost For backup space 2x the storage cost is a generously low estimate.
  • Increased physical space usage and electricity costs Many live in small areas and/or are not willing to assume an additional ~80 kWh onto their electrical bill.
  • Backups become more difficult over time Network and media read/write speeds have not scaled with capacity. Every year backing up becomes, relatively speaking, slower and slower.
  • Isolation Most importantly, proper backups require environmental isolation from the working data but still have a data path available. Even the most primitive arrangement is out of range for most people as correct execution requires a broad range of knowledge, skills, and tools.
Setups that work much of the time but don't count as backup:
  • In-machine copying With the "backup" next to the original, your data is still susceptible to loss from theft, environmental damage, or destruction due to e.g. bad voltage from the drive controller or file system corruption when a failed drive hangs the shared bus. Also, with storage so close and accessible, many will not resist the temptation to use it for non backup purposes.
  • External copying This is probably the preferred method for most as it requires the least hardware and electricity while retaining the potential for physical security, but the data is an archival snapshot, not a backup. Updating and managing the external data requires manual intervention and discipline if the data needs to be kept up to date.
While, of course, backups are absolutely essential for important data, the idea probably won't gain sufficient traction until someone successfully markets a "backup in a box" that includes all the storage, connectivity, and wizardry such that the buyer doesn't have to make any decisions beyond ten or less setup dialogs. Unfortunately, online backups are exponentially out of reach in the United States since all major ISPs strangle upload channels to dialup-like speeds.
plnelson
QUOTE(pdq @ Aug 16 2007, 13:27) *

There are two kinds of people in the world; those that back up their files, and those that haven't had a HD crash yet.

As I said - it's not just crashes - there's LOTS of reasons why you might wish you had a backup.

plnelson
QUOTE(Whelkman @ Aug 16 2007, 13:39) *

[*]External copying This is probably the preferred method for most as it requires the least hardware and electricity while retaining the potential for physical security, but the data is an archival snapshot, not a backup.

I'm not sure what the distinction is that you're making between an "archival snapshot" and a "backup". I think most people would consider a copy of their data that they make periodically onto an external USB HD for purposes of recovering their data in the event that the original is lost or damaged as a "backup". Certainly in this case it would have sufficed.

Also, I dispute the idea that using such a scheme is expensive, technically challenging, or demanding of network resources. In my case I have 3 USB harddrives on a rotating schedule so at least one of them is always offsite. But two (onsite+offsite) would be adequate and even one onsite USB HD would provide protection from crashes and even accidental file damage, if caught soon enough. The problem with having only one onsite backup HD is that it doesn't protect from fire or theft.

When considering the cost and trouble of backing up their data people need to consider how much they would care if they lost it. In my case years of studio photography with gorgeous models and lots of work, other once-in-a-lifetime photography, financial records, and MP3's ripped and tag-edited at great effort from maybe $18,000 worth of CD's (which would also be up in smoke in a fire), my personal journal, and decades of software that I've written, would be gone.

But if people have little of value on their PC's so any losses wouldn't justify the (small) effort of backing things up then I agree they needn't bother. I don't back up installed, commercial software because it's relatively easy to replace it.
Whelkman
QUOTE(plnelson @ Aug 16 2007, 15:28) *
I'm not sure what the distinction is that you're making between an "archival snapshot" and a "backup".

I should have termed "backup" better. Selecting files and copying to another drive is a data copy, not a backup. A backup system contains periodic deltas against occasional full data dumps. Proper data backups are done to an electronically controlled schedule. All one can say about a pulled hard drive with a data copy is that it has a snapshot at the time and most likely lacks past deltas.

QUOTE(plnelson @ Aug 16 2007, 15:28) *
In my case I have 3 USB harddrives on a rotating schedule so at least one of them is always offsite.

This is a decent system. In the event of a disaster, you're giving up some timely data in exchange for reduced costs. However, your plan requires planning beyond what most are willing to invest and regular manual intervention (discipline). Investing in 2-3 USB drives would also be viewed as an unreasonable cost by many people.

QUOTE(plnelson @ Aug 16 2007, 15:28) *
When considering the cost and trouble of backing up their data people need to consider how much they would care if they lost it.

From personal experience, I can say "not that much". The average person is willing to spend $100 or so in data protection and recovery; otherwise he'll live with the loss of his daughter's vacation photos. Many people don't even care to transfer data between computers upon a new purchase! Those who keep valuable professional data on their personal PCs are likely outnumbered 10 to 1 by those who don't, which could explain that, despite decades of warnings about the follies of living without backups, still no one seems to care.
plnelson
QUOTE(Whelkman @ Aug 16 2007, 17:29) *

QUOTE(plnelson @ Aug 16 2007, 15:28) *
I'm not sure what the distinction is that you're making between an "archival snapshot" and a "backup".
I should have termed "backup" better. Selecting files and copying to another drive is a data copy, not a backup. A backup system contains periodic deltas against occasional full data dumps. Proper data backups are done to an electronically controlled schedule. All one can say about a pulled hard drive with a data copy is that it has a snapshot at the time and most likely lacks past deltas.
I still think you're trying apply your own definitions. It doesn't make it "not a backup" because it's a full backup and not deltas, nor does it make it any less of a backup if somone uses a wind-up alarm clock or a trained ferret to notify them when to run it instead of an electronic timer.

I don't run my backups automatically because I don't want a backup to start while I'm in the middle of editing a file, nor do i want one to run if I haven't changed anything since the last one.

QUOTE
QUOTE(plnelson @ Aug 16 2007, 15:28) *
When considering the cost and trouble of backing up their data people need to consider how much they would care if they lost it.


From personal experience, I can say "not that much". The average person is willing to spend $100 or so in data protection and recovery; otherwise he'll live with the loss of his daughter's vacation photos. Many people don't even care to transfer data between computers upon a new purchase! Those who keep valuable professional data on their personal PCs are likely outnumbered 10 to 1 by those who don't, which could explain that, despite decades of warnings about the follies of living without backups, still no one seems to care.
But in the context of audio, what most people have is lots of music that they're either painstakingly ripped and lovingly edited the tags, or purchased online for tons of money. I inhabit lots of audio forums and I see people posting all the time about losing huge un-backed-up music collections and I STILL don't understand why, given all the time and money it reprsents why they don't at least have a static copy from a month beforeon some DVD's or something. All I'm asking is WHY?
Whelkman
QUOTE(plnelson @ Aug 16 2007, 18:40) *
But in the context of audio, what most people have is lots of music that they're either painstakingly ripped and lovingly edited the tags, or purchased online for tons of money. I inhabit lots of audio forums and I see people posting all the time about losing huge un-backed-up music collections and I STILL don't understand why, given all the time and money it reprsents why they don't at least have a static copy from a month beforeon some DVD's or something. All I'm asking is WHY?

Even in audio, a significant percentage feels the cost/effort of backing up is outweighed by the perceived marginal risk. People who feel otherwise establish backups. There's really nothing more to it.
locutus
QUOTE(Whelkman @ Aug 16 2007, 18:41) *

QUOTE(plnelson @ Aug 16 2007, 18:40) *
But in the context of audio, what most people have is lots of music that they're either painstakingly ripped and lovingly edited the tags, or purchased online for tons of money. I inhabit lots of audio forums and I see people posting all the time about losing huge un-backed-up music collections and I STILL don't understand why, given all the time and money it reprsents why they don't at least have a static copy from a month beforeon some DVD's or something. All I'm asking is WHY?

Even in audio, a significant percentage feels the cost/effort of backing up is outweighed by the perceived marginal risk. People who feel otherwise establish backups. There's really nothing more to it.


I donīt want to launch a backup discussion. I know it was stupid not to do a backup, but it is happened.

Back to the topic. Does Encspot Pro have some weakness discovering sync errors or how great is the chance to have the right checksum in a corrupt file?
I thought i had identified all corrupt files, but when listening, i found one file mixed with two songs. Also after removing any tag and reload in Encspot some sync errors disappear. How big is the chance that in case of a crash only the tag becomes wrong data?
Any explaination?
Whelkman
Back to your question: if I would hazard a guess, it sounds like your drive crash offset (some of) your filesystem tables in a manner similar to the following crude representation:
CODE

[-----file1-----][-----file2-----][...]          MPEG audio data
     [-----file1-----][-----file2-----][...]     file markers

Files are functional by coincidence. Your preferred decoder is reading superfluous tags midstream, rejecting them, and continuing with the adjacent MPEG audio. A file that's not adjacent to more MPEG audio data might crash the decoder, throw an exception mid-stream, or quietly cease playing. You'll need a tool, if one exists, that cuts the file upon encountering new data plus rejoins your MPEG chunks. Else you can try cutting and rejoining at the binary level. My opinion: it's too late to repair the damage--you'll need to repair the original file system and recopy.
jjack229
QUOTE(pdq @ Aug 16 2007, 11:27) *

There are two kinds of people in the world; those that back up their files, and those that haven't had a HD crash yet.


Very nice. I was in the first group myself, and now i am in the second. smile.gif
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2008 Invision Power Services, Inc.