The hard drive on which many of my FLAC encoded audio files were located has become corrupted. I have been able to salvage most of the songs using a recovery utility but, of those songs, a large number though reported as salvaged with 0 errors by the recovery utility have significant decoding errors.

1.) I want to test all salvaged songs in all directories, preferably generating a log of corrupted songs.
2.) I want restore any songs that have inaudible errors.

I am using Windows XP and FLAC 1.10.

* * *

With regard to testing all songs, there obviously is a test function in FLAC but it appears to work on a per file basis. There is no apparent way to scan a group of files, much less scan recursively through all subdirectories.

I found a DOS batch file that should test a group of songs (and which appears derived from a similar Shorten routine), the heart of which is:
for %%f in (*.flac) do flac.exe -t %%f
Unfortunately, this results in nothing but errors because it splits songs at the spaces in those songs, ie:

willie: ERROR initializing decoder
state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE
and: ERROR initializing decoder
state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE
lobo: ERROR initializing decoder
state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE
corazon: ERROR initializing decoder
state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE
salvaje.flac: ERROR initializing decoder
state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE

If I can't figure out a solution to the above, one alternative might be to pipe a find (or gnu grep) to flac to a text file, but I haven't explored that in detail yet.

* * *

So, as stated above, I manually tested a number of files, one at a time, and found that a good percentage were corrupt. I would like to restore these if the corruptions are inaudible. I am not sure whether there is an automated way to determine the severity of the file corruption (i.e. % of lost data) but... at the least I should be able to decode the songs, ignoring existing errors, and listen to them.

Unfortunately, decoding always stops even though I am using: "FLAC -d -F" Moreover, the error codes aren't even consistent:

jimmy eat world - the middle.flac: 65% complete
jimmy eat world - the middle.flac: *** Got error code 0:FLAC__STREAM_DECODER_ERROR_STATUS_LOST_SYNC
jimmy eat world - the middle.flac: *** Got error code 0:FLAC__STREAM_DECODER_ERROR_STATUS_LOST_SYNC
jimmy eat world - the middle.flac: WARNING, MD5 signature mismatch

or

drifters - there goes my baby.flac: 15% complete
drifters - there goes my baby.flac: *** Got error code 0:FLAC__STREAM_DECODER_ERROR_STATUS_LOST_SYNC
drifters - there goes my baby.flac: *** Got error code 2:FLAC__STREAM_DECODER_ERROR_STATUS_FRAME_CRC_MISMATCH
drifters - there goes my baby.flac: *** Got error code 0:FLAC__STREAM_DECODER_ERROR_STATUS_LOST_SYNC
drifters - there goes my baby.flac: WARNING, MD5 signature mismatch

or

barry mcguire - eve of destruction.flac: 24% complete
barry mcguire - eve of destruction.flac: ERROR while decoding data
state = 6:FLAC__STREAM_DECODER_UNPARSEABLE_STREAM

or the ugliest (and seemingly least promising):

david grisman - trinidadian rag.flac: *** Got error code 0:FLAC__STREAM_DECODER_ERROR_STATUS_LOST_SYNC
david grisman - trinidadian rag.flac: *** Got error code 1:FLAC__STREAM_DECODER_ERROR_STATUS_BAD_HEADER
[REPEATED FOR QUITE SOME TIME]
david grisman - trinidadian rag.flac: *** Got error code 2:FLAC__STREAM_DECODER_ERROR_STATUS_FRAME_CRC_MISMATCH
david grisman - trinidadian rag.flac: ERROR, channels is 1 in frame but 2 in STREAMINFO
david grisman - trinidadian rag.flac: ERROR while decoding data
state = 3:FLAC__STREAM_DECODER_READ_FRAME

Any help with the problems above would be appreciated...

Heh - been there, done that. Assuming you have all your FLAC files in subdirectories under some top directory, you can use a command like this:

FOR /R "C:\FLAC" %i IN (*.flac) DO flac.exe -t -s "%i" 2>> log.txt

This will go through all subdirectories recursively, test each .flac file, and log errors to log.txt. From there, you can see how much damage has been done, and contemplate re-ripping many CDs...

If you want to go further, download 4NT and make a batch file which generate its own shorter log, by checking FLAC's ERRORLEVELs.

By the way, what happened ? Your partition got corrupted ?

Cheers

Some errors can make the decoder stop even with -F. That one about the unparseable stream, what happened is that some noise actually looked like a correct frame header that was created by some future encoder. In badly damaged files this can happen.

It would be possible to make a decoder that was specifically designed to decode such heavily damaged files but in these cases you would probably get unacceptable dropouts or noise anyway that would need manual editing.

Josh

edit: Look on the bright side, with another format you might not get any data back from corrupted files!

Thanks all!

Your command line script did just what I needed! But why can't it work in a batch file? For example:

@echo off
cls
FOR /R "D:\dump\" %i IN (*.flac) DO flac.exe -t -s "%i" 2>> flaclog.txt
pause

* * *

I experienced the loss on a FAT32X partition on a removeable Firewire/USB hard drive (larger, more flexible, cheaper than an iPod). The hard drive itself is physically fine. The plugging in and out probably fouled up the file system. XP and PartitionMagic say the hard drive is unformatted. Partition Info and EasyRecovery see the file system with some help. Some of the files are crosslinked.

The moral of the story is use a journaled file system. (OK, and also don't keep anything unique on a removeable hard drive). Now, if only there were a journaled file system that could be used with all OS's... Maybe one day, someone will backport Reiser4 to Windows?

* * *

Yes, I have been nothing but pleased with FLAC. I am not a Golden Ear who values lossless for its sonic quality. I just appreciate the future-proof nature of the format and data-integrity checks. As you imply, Josh, I could never have done these tests with MP3 files, for example.

Still, the blessing of the checksums can also be a curse. I am not certain how bad the loss really is since many of the songs play perfectly until they exit... Since even CD rips are not bit-for-bit perfect, FLAC's error-checking may be overkill if inaudible errors require you to throw away the file.

Long term, there should be a way to decode through these errors, perhaps distinguishing between critical and noncritical errors. True, for CD rips, one can always rerip the CD. But I have read about radio stations/libraries interested in FLAC as an archival format for unique recordings (live events for example). Assuming that some corruption is inevitable, restoration would be required if there is only one remaining copy of audio file. And, in my case, I might have decided that the file sounded just fine and not reripped.

* * *

OK, one final question... before I blow away the partition and reformat, I want to do a raw file recovery.

What is the file signature and offset for FLAC? The file signature should be in hexidecimal format and not exceed 8 bytes. The signature offset should be 0 relative and less than 512 bytes.

The only relevant info I find on the documentation page is:
* the four byte string "fLaC"

Here's the error when I run the script as a batch file...

i" 2>> flaclog.txt was unexpected at this time.

This is usually the catch-22 with any digital data. It's perfect, but very "fragile" compared to good-ol' analog storage.



I seem to remember having trouble with batch files as well, but if that's the only problem being reported, try leaving the "2>> flaclog.txt" off of the command, and instead appending it to the command with the batch file, like "C:\>flac_check.bat 2>> flaclog.txt".

You have to replace "%i" with "%%i" in batch files.

-Eugene

Yeah, the heavy-duty repairer is on my list of stuff to do. What % of damaged files stopped even with -F? Could you post a few so that I can use them for testing the new tool?


That's it, it's at offset 0. If all your files had at least a seektable, vorbis comment, or padding, you can make it more robust by adding 4 more bytes: 'f' 'L' 'a' 'C' 00h 00h 00h 22h

Josh

%% did the trick for the batch file

How would I write the file signature in hexidecimal? (i.e. 1A 00 00 03 00 00 14 00)

66 4C 61 43

or

66 4C 61 43 00 00 00 22

Josh

I've got .FLAC files from 150 of my albums which have become corrupted.
Appears to have happened during tagging operations. Have not yet isolated
problem software. May be easytag, or may be id3convert. Both were used
on the corrupted files.

The '-F' flag still fails to get flac to produce a '.wav' file from these failures,
which look like this:

cjshaker@corel-linux: flac -t "You Don't Know My Mind.flac"

flac 1.1.0, Copyright © 2000,2001,2002,2003 Josh Coalson
flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
welcome to redistribute it under certain conditions. Type `flac' for details.

You Don't Know My Mind.flac: testing, 96% complete

You Don't Know My Mind.flac: ERROR while decoding data
state = 4:FLAC__STREAM_DECODER_END_OF_STREAM
cjshaker@corel-linux:


Also, my 'shntool' doesn't seem to support '.flac' files. I'm wondering if it's
'fix' option would fix this problem if it did...

Thank you,
Chris Shaker
cjshaker@hotmail.com