Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Yet another FLAC recovery problem thread (Read 7627 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Yet another FLAC recovery problem thread

Just as the other thread in this sub forum, I've had accidentally deleted a FLAC collection of mine and I went to recover it. Unlike the previous user most of my FLAC files turned out undamaged when recovering them but some were overwritten. Although I decided to recover the damaged files too, it went successfully and most of the FLAC files came intact. I played a few of them and I noticed one track that had a problem. When playing it in foobar (tried VLC also), at some random points it produces a unpleasant very loud screech (white-noise like). Now, as I had quite a few FLAC files in that collection, I would like to check them all for those errors, and preferably not through playback as that could be quite painful to the ears XD
I tried to run the -t parameter in the CLI flac encoder on the file, but it reports no errors. Any suggestions on what to use to detect those mishaps? (if it can do it in batch mode that's a plus! )

Software used:

Recording/Encoding - Audacity/libFLAC 1.2.1 20070917
Playback - FooBar, VLC
Recovery - Recuva
File System - NTFS
OS - WinXP SP3 x86

If anyone needs a sample of the problematic file, I can upload it somewhere, it's not copyrighted.

Thanks in advance ^^


Yet another FLAC recovery problem thread

Reply #1
Would Foobar2000's File Integrity tool work for this?

Yet another FLAC recovery problem thread

Reply #2
I tried to run the -t parameter in the CLI flac encoder on the file, but it reports no errors.
On a file that produces a loud burst of static? That’s very odd. Assuming of course that there’s a reason for that in the data itself, flac.exe should notice it and report an error in the overall MD5 at the very least, if not also the individual block(s) housing the change. Of course, it’s possible you’ve defied the odds and found a hash collision, but y’know, probably not!  So, I’m very curious about this. Have you tried the test on more than one file?

Yet another FLAC recovery problem thread

Reply #3
in lame man terms (HA pun intended), the way I understand it, error free FLAC = error free output; the audio is playing back as is.  unless there is a bug in decoder.

you may be able to "correct" the audio with Audacity.

if the tracks at not copyrighted, just try to find a legal download (hopefully not violating TOS). no copyright does NOT always equal free.

Yet another FLAC recovery problem thread

Reply #4
When playing it in foobar (tried VLC also), at some random points it produces a unpleasant very loud screech (white-noise like).
[...]
I tried to run the -t parameter in the CLI flac encoder on the file, but it reports no errors.
[...]
If anyone needs a sample of the problematic file, I can upload it somewhere, it's not copyrighted.


Yes please, I would like to see the file. If this is true, it might be a *very bad* bug which needs fixing soon, as the devs are working towards a new release of FLAC
Music: sounds arranged such that they construct feelings.

Yet another FLAC recovery problem thread

Reply #5
Would Foobar2000's File Integrity tool work for this?


Where would that tool be? I can't find it in fb2k's menus.

On a file that produces a loud burst of static? That’s very odd. Assuming of course that there’s a reason for that in the data itself, flac.exe should notice it and report an error in the overall MD5 at the very least, if not also the individual block(s) housing the change. Of course, it’s possible you’ve defied the odds and found a hash collision, but y’know, probably not!  So, I’m very curious about this. Have you tried the test on more than one file?


It could be in the data itself, maybe Recuva found this file as being damaged for having a few bytes (or blocks as you say) missing and it attempted to plaster those missing bits and make it checksum intact. Yes, I tried multiple files and some do expose the same problem. I forgot to also mention that the original audio (before deletion) didn't have those loud screechy bits (BTW, they aren't static, as they change in waveform and in the spectrum).

you may be able to "correct" the audio with Audacity.

if the tracks at not copyrighted, just try to find a legal download (hopefully not violating TOS). no copyright does NOT always equal free.


I don't think I can fix the audio anymore, what I'm trying to do is to detect the faulty files and replace them (by recording again). There is no download for this file as it's recorded by myself.

Yes please, I would like to see the file. If this is true, it might be a *very bad* bug which needs fixing soon, as the devs are working towards a new release of FLAC


Here you go. It has ~7sec of silence in the beginning, beware not to set your speakers too loud if you attempt to listen to it =P

Also here's the spectrogram:



And a close-up:


Yet another FLAC recovery problem thread

Reply #6
[…] the devs are working towards a new release of FLAC
I didn’t realise there was much happening with FLAC again; have any plans, etc. been released? Who are the developers now, anyway?

Where would [the File Integrity Verifier] be? I can't find it in fb2k's menus.
Try shift-right-clicking on the relevant file.

It could be in the data itself, maybe Recuva found this file as being damaged for having a few bytes (or blocks as you say) missing and it attempted to plaster those missing bits and make it checksum intact.
Notwithstanding that I doubt this or any other undeleting program has tailored support for FLAC or any other format, I would hope that none of them would attempt to cobble together a repair and fudge a checksum without at least asking the user – and advising them of how silly that would be.  At least, that’s my expectation based upon the assumption of a rational human world, so you never know!


Yet another FLAC recovery problem thread

Reply #8
Here you go. It has ~7sec of silence in the beginning, beware not to set your speakers too loud if you attempt to listen to it =P

Thanks for the sample. I'm pretty sure that this corruption has happened *before* converting this to FLAC, because for example the first error spans 5 FLAC blocks with start and end somewhere in a block, so it's not aligned. Furthermore, decoding with ffmpeg (which, I believe, is developed independent of libFLAC) yields the same result. Last, it is impossible that recuva somehow managed to get this into a FLAC file to get the MD5 to match: there are CRC's at the end of each frame and at the end of each header, so Recuva would have had to get a stream that matched with 1 MD5 and 10 CRCs... that's impossible.

edit: it might be possible that recuva found corrupt files, decoded them ignoring the errors and re-encoded them, but I think that's highly unlikely. Besides, FLAC usually embed silence on decoding through errors, not stuff like this. I don't know how this happened, but it's unlikely FLAC it the culprit.

[…] the devs are working towards a new release of FLAC
I didn’t realise there was much happening with FLAC again; have any plans, etc. been released? Who are the developers now, anyway?

Just a bugfix release, getting FLAC to compile nicely with current compilers. The new maintainers (Erik de Castro Lopo) main concern is libFLAC. Check the mailinglists for more information: http://lists.xiph.org/pipermail/flac-dev/ They are merging support for 7 and 8 channel audio as well. edit: Josh had some unreleased fixes in cvs as well, of course these will get in the new release too.
Music: sounds arranged such that they construct feelings.

Yet another FLAC recovery problem thread

Reply #9
Just a bugfix release, getting FLAC to compile nicely with current compilers. The new maintainers (Erik de Castro Lopo) main concern is libFLAC. Check the mailinglists for more information: http://lists.xiph.org/pipermail/flac-dev/ They are merging support for 7 and 8 channel audio as well. edit: Josh had some unreleased fixes in cvs as well, of course these will get in the new release too.


Glad to hear that the devs are improving the libs even more, would be fun to see the new features/fixes that are added ^^


Notwithstanding that I doubt this or any other undeleting program has tailored support for FLAC or any other format, I would hope that none of them would attempt to cobble together a repair and fudge a checksum without at least asking the user – and advising them of how silly that would be.  At least, that’s my expectation based upon the assumption of a rational human world, so you never know!


I don't think I would go that far to say that, but there's definately something that Recuva/deleting those files caused those screeches to show up. My other guess is that some files (or fragments of them) were placed over the deleted FLAC files thus damaging them, and so, in a recovery attempt, Recuva tried "extracting" the damaged files with the fragments of the files that replaced the deleted ones. I hope I was descriptive enough with this...

You might have to download the component. I don't think it's installed with Foobar2000 "out of the box".

http://www.foobar2000.org/components/view/foo_verifier


Thanks for the link, I tried verifying it and it didn't report any errors.

I'm pretty sure that this corruption has happened *before* converting this to FLAC, because for example the first error spans 5 FLAC blocks with start and end somewhere in a block, so it's not aligned.


That's also highly impossible, because I've listened to the audio before encoding (while recording) and after and it didn't have those screeches.

[...] but it's unlikely FLAC it the culprit.


With that in mind I suppose there's no automated way to detect the files with the screeches, so I'd probably have to go through the audio files one by one and check their spectrograms X.X

Thanks for your assistance, guys!

Yet another FLAC recovery problem thread

Reply #10
there's definately something that Recuva/deleting those files caused those screeches to show up. My other guess is that some files (or fragments of them) were placed over the deleted FLAC files thus damaging them, and so, in a recovery attempt, Recuva tried "extracting" the damaged files with the fragments of the files that replaced the deleted ones. I hope I was descriptive enough with this...
Well, yeah, this is the most basic scenario for what would happen to a partially overwritten file for which recovery was attempted, so nothing specific to any one program; again, this makes the question of why no verifier has apparently detected any mismatch all the more important.

Yet another FLAC recovery problem thread

Reply #11
I don't think I would go that far to say that, but there's definately something that Recuva/deleting those files caused those screeches to show up. My other guess is that some files (or fragments of them) were placed over the deleted FLAC files thus damaging them, and so, in a recovery attempt, Recuva tried "extracting" the damaged files with the fragments of the files that replaced the deleted ones. I hope I was descriptive enough with this...

But that doesn't explain why the MD5sum is correct. It is highly unlikely that it 'coincidentally' turns out to be the same MD5.

Anyway, I took a look at the WAV with a HEX editor, and I saw this:

Code: [Select]
advertisement-top
##.advertisement300x250
##.advertisement468
##.advertisementBlock
##.advertisementBox
##.advertisementColumnGroup
##.advertisementContainer
##.advertisementHeader
##.advertisementLabel
##.advertisementPanel
##.advertisementText
##.advertisement_300x250
##.advertisement_block_468_60
##.advertisement_btm
[...]


That's right: plain text! It looks like some kind of webbrowser cache stuff. It looks like parts of the file got overwritten by other things.


Quote
That's also highly impossible, because I've listened to the audio before encoding (while recording) and after and it didn't have those screeches.

Yes, but what if it was corrupted between recording and encoding or during encoding? This plain text showed up in the decoded WAV not in the FLAC-file itself, so it has had to be overwritten while lingering as a WAV file, not while being a FLAC file. That would explain why the FLAC file itself is intact and the MD5sum is correct.

Quote
With that in mind I suppose there's no automated way to detect the files with the screeches, so I'd probably have to go through the audio files one by one and check their spectrograms X.X

Yes, because the corruption took place in the WAV and the WAV container has no checks for corruption or checksums, you'll have to check by hand.
Music: sounds arranged such that they construct feelings.

Yet another FLAC recovery problem thread

Reply #12
Yes, but what if it was corrupted between recording and encoding or during encoding? This plain text showed up in the decoded WAV not in the FLAC-file itself, so it has had to be overwritten while lingering as a WAV file, not while being a FLAC file. That would explain why the FLAC file itself is intact and the MD5sum is correct.


Ah, now I realize that the order of events was different than I previously thought. ARGH.
I didn't encode directly to FLAC in Audacity, I encoded to WAV instead (have no idea why). When I accidentally deleted the collection, there were still wav files remaining that weren't converted to FLAC, so when I did the recovery, they were still in WAV format, that explains the plain text section in the file. Sorry for the mix-up and for the eventual head-scratching XD
Also thank you for taking your time and looking into this weird issue!