Help - Search - Members - Calendar
Full Version: Free utility to check all Wavpack files in a Directory.
Hydrogenaudio Forums > Lossless Audio Compression > WavPack
Pages: 1, 2
gl.tter
Updated - latest version: 1.0 (now with shell extension)

I've released a little standalone Windows utility that will check single Wavpack files, or all .wv files contained in a directory tree or drive for errors:
http://wavpack.gl.tter.org

Easy two-click operation. It writes the full paths of any bad files found to a text file for further processing. Useful when eg. like me you ripped 1500+ files with an overheating CPU, and had to track down the few corrupted ones.

Bug reports welcome (here or email).
Remedial Sound
Handy little app! I just scanned a few album images and it worked great.

One question: it seems to beep/ding at arbitrary times... is there a reason for this?

BTW nice avatar!
ssjkakaroto
Thanks gl.tter.
Synthetic Soul did something similar using .bat files, but I'm not sure if it checks for files in subdirectories http://www.hydrogenaudio.org/forums/index....verify++wavpack
gl.tter
QUOTE(Remedial Sound @ Nov 21 2006, 20:22) *

Handy little app! I just scanned a few album images and it worked great.


That was quick smile.gif.

QUOTE

One question: it seems to beep/ding at arbitrary times... is there a reason for this?


It's only supposed to beep when it finds an error, and at the end. If it doesn't let me know (my PC is minus a speaker right now but I can try it later).

QUOTE

BTW nice avatar!


Thanks. Yours would be great animated smile.gif.


QUOTE(ssjkakaroto @ Nov 21 2006, 20:25) *

Synthetic Soul did something similar using .bat files, but I'm not sure if it checks for files in subdirectories http://www.hydrogenaudio.org/forums/index....verify++wavpack


Bat files are good, but I fancied throwing some eyecandy at it. It's also easy to use for non-commandline fiends, just copy it to the dir where you want to scan from and double-click. If any errors are found, the text file with details is auto-launched into eg. Notepad at the end.
guruboolez
That's great. I crossed my fingers and hoped to see such program appear soon... since I had to recover the content of a 300 GB HDD full of WavPack & FLAC encodings last summer (partition table was corrupted). I was able to check the flac collection (no errors) but the WavPack's one is still unverified for the moment.
Thank you for coding this (I'm a complete nut with .bat files tongue.gif )
DARcode
Nice handy little proggy, pretty useful, many thanks!
Using it now tongue.gif ...
gl.tter
QUOTE(DARcode @ Nov 21 2006, 21:47) *

Nice handy little proggy, pretty useful, many thanks!
Using it now tongue.gif ...


Thanks guys. Uploaded a new version, see the site for changes.
Synthetic Soul
QUOTE(ssjkakaroto @ Nov 21 2006, 19:25) *
Synthetic Soul did something similar using .bat files, but I'm not sure if it checks for files in subdirectories http://www.hydrogenaudio.org/forums/index....verify++wavpack
Yes, it will.

QUOTE(gl.tter @ Nov 21 2006, 19:31) *
Bat files are good, but I fancied throwing some eyecandy at it. It's also easy to use for non-commandline fiends, just copy it to the dir where you want to scan from and double-click. If any errors are found, the text file with details is auto-launched into eg. Notepad at the end.
The BAT file is set up so that you can just drag the folder to check onto the BAT file icon and off it goes. Pretty simple. It records any fails in a temporary text file and then uses that to report any failed files in the same window, post-processing.

These things aside, I'm sure that many (most) people would prefer a nice EXE rather than a scary-sounding BAT file, so thanks for the work.

Edit: I've just had a look at the app - very nice indeed. I have found it more easy to put the EXE in a folder in PATH and just call "checkwavpackfiles" on the commandline. Other users may also be interested to know that you can also just drag a folder onto the EXE, or a shortcut to the EXE. I find the idea of having to move the EXE to the folder in question too much hassle. Thanks again.
gl.tter
QUOTE(Synthetic Soul @ Nov 22 2006, 08:35) *

The BAT file is set up so that you can just drag the folder to check onto the BAT file icon and off it goes. Pretty simple. It records any fails in a temporary text file and then uses that to report any failed files in the same window, post-processing.


Cool. My utility is useful for people who don't have the Wavpack utils, eg. people working in pro audio with my Wavelab plugin, but either is good.

QUOTE

Edit: I've just had a look at the app - very nice indeed. I have found it more easy to put the EXE in a folder in PATH and just call "checkwavpackfiles" on the commandline. Other users may also be interested to know that you can also just drag a folder onto the EXE, or a shortcut to the EXE. I find the idea of having to move the EXE to the folder in question too much hassle. Thanks again.


Good ideas, thanks. I keep mine in the root of my Rockbox'ed portable player.

BTW, has anybody compared the SSE2 version with the normal one? I haven't timed it but it seems a little faster on my PentiumD. It's not hand-written SSE2, just letting the compiler spit it out.
DrazardX
Wow. This is really nice. I'm obviously not a very active member, but I had to log in to say thanks. My music directory has gotten very messy recently. This will be very useful when I decide to organize everything again. I usually use MD5s to verify my files, but this will be great before creating an MD5 to check if i'm not hashing a bad file.

Thanks again.
jaybeee
QUOTE(guruboolez @ Nov 21 2006, 19:51) *

That's great. I crossed my fingers and hoped to see such program appear soon... since I had to recover the content of a 300 GB HDD full of WavPack & FLAC encodings last summer (partition table was corrupted). I was able to check the flac collection (no errors) but the WavPack's one is still unverified for the moment.
Thank you for coding this (I'm a complete nut with .bat files tongue.gif )

Hi guruboolez, what app/tool did you use to check the FLAC files?
Maybe just loading all in foobar would do it? I have noticed a few FLAC files in my collection that have corrupted


Apologies for going a little off topic (well from WavPack to FLAC).
guruboolez
flactester:
http://www.vuplayer.com/other.php
smile.gif
jaybeee
QUOTE(guruboolez @ Nov 23 2006, 11:39) *
Thanks mate.

Also, been looking for a freeware player that can submit disc info to Gracenote too! So player says it does it biggrin.gif
gl.tter
If the program crashes as soon as you run it, you'll need the latest runtimes from MS (thanks Bryant).

gl.tter

New version uploaded. See the site for changes: http://wavpack.gl.tter.org

I've also updated my Wavelab Wavpack plugins (same page).
DARcode
Thank you.

Have you noticed than even when it reaches the "Press Space to Exit." stage it still uses as much CPU as it's available and doesn't go really idle until you actually press the space bar?
gl.tter
QUOTE(DARcode @ Nov 28 2006, 03:10) *

Thank you.

Have you noticed than even when it reaches the "Press Space to Exit." stage it still uses as much CPU as it's available and doesn't go really idle until you actually press the space bar?


Thanks, download updated. Easy to miss on a dual-core.
DARcode
QUOTE(gl.tter @ Nov 28 2006, 16:08) *

QUOTE(DARcode @ Nov 28 2006, 03:10) *

Thank you.

Have you noticed than even when it reaches the "Press Space to Exit." stage it still uses as much CPU as it's available and doesn't go really idle until you actually press the space bar?


Thanks, download updated. Easy to miss on a dual-core.
Wow, that's one fast fix, thank you!
For your info my CPU isn't dual core, it's a mere Sempron 3400+.
gl.tter
QUOTE(DARcode @ Nov 28 2006, 20:29) *

Wow, that's one fast fix, thank you!


You're welcome, easy fix.

QUOTE(DARcode @ Nov 28 2006, 20:29) *

For your info my CPU isn't dual core, it's a mere Sempron 3400+.


I meant I have a (fairly newish) dualie. The problem is that you don't notice programs hogging the CPU on 'em as easily, as most will only use 50% of the (combined) CPU, so everything's still nice and responsive.
askoff
Does anyone else have problems with this utility giving random errors when checkin multiple files?
DARcode
QUOTE(askoff @ Nov 29 2006, 15:33) *

Does anyone else have problems with this utility giving random errors when checkin multiple files?
Nope, my 80 plus GBs of WV files were all reported as OK.
gl.tter
QUOTE(askoff @ Nov 29 2006, 14:33) *

Does anyone else have problems with this utility giving random errors when checkin multiple files?


Any chance you're overclocking/overheating, or have a dud memory module? Do the Wavpack utils work?
askoff
QUOTE(gl.tter @ Nov 29 2006, 17:06) *
Any chance you're overclocking/overheating, or have a dud memory module? Do the Wavpack utils work?

I thought of it too because the CPU is overclocked a little, but there are no other symptons of it. All problems link to wavpack only for some reason because also Foobar2000 verfy integrity tool gives same random checksum errors with wavpack files. I have about 450 song in wavpack format and they are all encoded with -hx6m switches. They all play fine and with second check all the corrupted files are OK. Foobar2000 doesn't report any checksum errors with mp3s and I have never got any false alarms when checking integrity with .sfv files.

EDIT: Both the SSE2 and non SSE2 versions of CheckWavpackFiles gives me random errors.

EDIT2: I will try without overclockings in couple of days.
askoff
It took more than couple of days for me to test without overclockings, but anyway it solved the problem. All Wavpack integrity check tools work now as the should, and gives no errors anymore.

[off topic]
This overclocking hobby is so strange. I can only wonder, why this computer hasn't crashed even at once, while it has been overclocked all these few months. Now I know good test utility to test this CPUs overclockin limits.
[/off topic]
disintegrated
Hi gl.tter,


is Win98 SE supported by CheckWavpack?
Because it's not working on my side. crying.gif


I copied the .exe into a folder with wavpacks, and doubleclicked:
CODE
- CheckWavpackFiles v0.94b -  (c) gl.tter (gl@r-i-l.net) 2006
    updates & goodies: http://wavpack.gl.tter.org
    supports Wavpack files up to v4.x.
__

Invalid path '??????????????????????????????????????????????????????????????????
????????????????????????????????????????????????????????????????????????????????
????????????????????????????????????????????????????????????????????????????????
????????????????????????????????????????????????????????????'

Press any key to exit.



Then I tried the drop-a-folder method without success:
CODE
- CheckWavpackFiles v0.94b -  (c) gl.tter (gl@r-i-l.net) 2006
    updates & goodies: http://wavpack.gl.tter.org
    supports Wavpack files up to v4.x.
__

Invalid path 'E:\WAVPACK\THECLASH'

Press any key to exit.




Regards,
disintegrated
gl.tter
QUOTE(disintegrated @ Dec 13 2006, 12:08) *

is Win98 SE supported by CheckWavpack?
Because it's not working on my side. crying.gif


No, I didn't think anybody was still using it (NT-based apps use Unicode strings, hence the garbage paths you were getting).

Try this version and let me know if it works.
disintegrated
QUOTE(gl.tter @ Dec 13 2006, 18:02) *

QUOTE(disintegrated @ Dec 13 2006, 12:08) *

is Win98 SE supported by CheckWavpack?
Because it's not working on my side. crying.gif


No, I didn't think anybody was still using it (NT-based apps use Unicode strings, hence the garbage paths you were getting).

Try this version and let me know if it works.


Thank You very, very much! Seems to work fine biggrin.gif
Overnight wv-checking is planned wink.gif
Only thing I noticed was wrong colourcodes here and there:
IPB Image

Regards,
disintegrated
gl.tter
QUOTE(disintegrated @ Dec 13 2006, 19:04) *

Only thing I noticed was wrong colourcodes here and there:


Strange, I'm not seeing that here (on XP). Might be some bugs in 9x consoles.

Try this one, it tidies up a couple of 9x things.
gl.tter
New version uploaded (see 1st post).

The files now work on 9x too, and also auto-delete the 'BadFiles.txt' file when no bad files were found (it was supposed to do that before but didn't).
disintegrated
QUOTE(gl.tter @ Dec 13 2006, 21:23) *

QUOTE(disintegrated @ Dec 13 2006, 19:04) *
Only thing I noticed was wrong colourcodes here and there:

Strange, I'm not seeing that here (on XP). Might be some bugs in 9x consoles.
Try this one, it tidies up a couple of 9x things.

Tried 0.95b.. still wrong colours.. just the same as in 0.94b_9x..
New screeny, this time with a selfmade corrupt file:
IPB Image


Everything else seems to just works fine..


Regards,
Disintegrated

gl.tter
QUOTE(disintegrated @ Dec 15 2006, 21:44) *

Tried 0.95b.. still wrong colours.. just the same as in 0.94b_9x..


Thanks for the reports - is anybody else seeing this, on what platform?
gl.tter
0.97b RC1 released. Now includes the new Wavpack 4.40 code.

Only thing holding back a 1.0 release is the colour issue. Please report if you get this problem with details. If it's just down to Win98 bugs it won't be fixed.

Also Bryant pointed out that it likely doesn't work on Win95 - anybody want to try?
satorippoi
Hey, gl.tter, just wanted to let you know that you program is really awesome...thanks, mate!!!

btw, i can't find SSE2 edition at your site...could point where i can get it?..

yet, are you planning to make it support dual cores checking two simulteneously?..
gl.tter
QUOTE(satorippoi @ Dec 23 2006, 01:14) *

Hey, gl.tter, just wanted to let you know that you program is really awesome...thanks, mate!!!


Cheers.

QUOTE(satorippoi @ Dec 23 2006, 01:14) *

btw, i can't find SSE2 edition at your site...could point where i can get it?..


I've removed it because it wasn't actually faster. It was just compiler-generated SSE2, to really speed things up requires hands-on code.

QUOTE(satorippoi @ Dec 23 2006, 01:14) *

yet, are you planning to make it support dual cores checking two simulteneously?..


I've thought about it, but it makes the display complicated, and you would also get some disk trashing, which may or may not be a problem depending on your setup.

Another way to support dual-cores would be to change the Wavpack code to unpack the stereo channels on seperate processors.

BTW, I added a couple more features and it's now at 0.99b RC2. This version creates an Explorer shell extension so that you can also scan folders by right-clicking them. Everyone, give this one a quick try so I can bump it to 1.0.
ssjkakaroto
Hi gl.tter, can you add non-ascii support to your app?
Checkwavpack can't check files with japanese characters (eg. "03. サンキュー!!.wv"), the error I get is couldn't open! ("can't open file")
gl.tter
QUOTE(ssjkakaroto @ Dec 26 2006, 02:16) *

Hi gl.tter, can you add non-ascii support to your app?
Checkwavpack can't check files with japanese characters (eg. "03. サンキュー!!.wv"), the error I get is couldn't open! ("can't open file")


I just took a look at this, turns out there's no easy solution. Almost none of my apps (including Wavelab and even the Wavpack utils themselves) can open this filename on my English XP - so far only Notepad has managed it (good old Notepad).

The problem is that the filename contains characters from a different language than what the OS is set to, which is particularly problematic with the ANSI (Win9x compatible) version. A workaround might be to temporarily change your OS language to Japanese - let me know if that works.

It does work with an all-Unicode version (with a couple of Wavpack code changes), but the problem is that the Jap characters cannot be displayed correctly on the console, nor are they listed correctly in BadFiles.txt. I don't know how much use that is, but try it.

IPB Image
ssjkakaroto
Thanks a lot gl.tter, that'll work for me since the directories are using ascii characters, so I'll know which song is which by the track number.

EDIT: It's working fine smile.gif
gl.tter
Managed to squeeze 1.0 in the last few hours of '06.

The default one is now the Unicode version, Win98 compatible is a seperate download if you need it.

Enjoy and keep reporting problems.
collector
QUOTE(gl.tter @ Dec 13 2006, 09:02) *

QUOTE(disintegrated @ Dec 13 2006, 12:08) *

is Win98 SE supported by CheckWavpack?
Because it's not working on my side. crying.gif

No, I didn't think anybody was still using it (NT-based apps use Unicode strings, hence the garbage paths you were getting).

I do. Not even SE smile.gif. Win98 is sufficient enough to run my MMM (multi media machine). It's only for fooling around with music and downloads. I am a newbie at Wavpack. So, I can't try your checker yet.
xequence
Very very usefull program, thanks alot smile.gif
Maglor
Hi all! smile.gif I've been reading a lot of posts lately about bad blocks or crc errors on wavpack files, and I can't find the solution to my problem, which is: recently I've bought a new external drive where I've passed all my rips still on folders, not rar's, and many of them have now been corrupted when they weren't before that.
So now what I want is to recover those files, because many of those CD's that I've ripped aren't with me anymore. So is there a little something like this that could recover those wavpack files?
I appreciate very much all the help! wink.gif
photographer
Unfortunately I'm having a little difficulty. When I run CheckWavpackFiles_1.0 I get an error stating "# Couldn't read MD5 hash.". I use Easy CD DA to rip my audio to my hard drive. I've tried it with files that were ripped using both the latest and the previous WavPack .dll's.
gl.tter
QUOTE(photographer @ Apr 2 2007, 20:38) *

Unfortunately I'm having a little difficulty. When I run CheckWavpackFiles_1.0 I get an error stating "# Couldn't read MD5 hash.". I use Easy CD DA to rip my audio to my hard drive. I've tried it with files that were ripped using both the latest and the previous WavPack .dll's.


Never had that error on any of my files. It could be a bug in the way Easy CD encodes Wavpack files. I don't use EasyCD, anybody else able to test it?

Try ripping to .wav and encoding with the Wavpack utilities. If that works, send an email to the EasyCD guys.

EDIT: also try verifying a bad file with the Wavpack utils (switch -v IIRC).
photographer
QUOTE(gl.tter @ Apr 2 2007, 15:48) *

QUOTE(photographer @ Apr 2 2007, 20:38) *

Unfortunately I'm having a little difficulty. When I run CheckWavpackFiles_1.0 I get an error stating "# Couldn't read MD5 hash.". I use Easy CD DA to rip my audio to my hard drive. I've tried it with files that were ripped using both the latest and the previous WavPack .dll's.


Never had that error on any of my files. It could be a bug in the way Easy CD encodes Wavpack files. I don't use EasyCD, anybody else able to test it?

Try ripping to .wav and encoding with the Wavpack utilities. If that works, send an email to the EasyCD guys.

EDIT: also try verifying a bad file with the Wavpack utils (switch -v IIRC).


Thanks for the prompt reply. I just installed WavPack 4.40 and Foobar 2000. Had it rip a single CD to a new directory and your utility verifies that all files are OK. I'll submit a bug report to the author of Easy CD-DA.

Thanks again. Good tool to have.
gl.tter
QUOTE(photographer @ Apr 2 2007, 21:16) *

Thanks for the prompt reply. I just installed WavPack 4.40 and Foobar 2000. Had it rip a single CD to a new directory and your utility verifies that all files are OK. I'll submit a bug report to the author of Easy CD-DA.


Cool, he's probably just writing the MD5 hash in slightly the wrong place (or maybe he's forgetting to write it altogether?).

It should be:

<pack all samples>
call WavpackStoreMD5Sum(...)
call WavpackFlushSamples()


(why do my new replies keep getting attached to my last reply??)

QUOTE(Maglor @ Mar 23 2007, 13:57) *

Hi all! smile.gif I've been reading a lot of posts lately about bad blocks or crc errors on wavpack files, and I can't find the solution to my problem, which is: recently I've bought a new external drive where I've passed all my rips still on folders, not rar's, and many of them have now been corrupted when they weren't before that.
So now what I want is to recover those files, because many of those CD's that I've ripped aren't with me anymore. So is there a little something like this that could recover those wavpack files?
I appreciate very much all the help! wink.gif


Maglor, it depends why the files are coming back corrupt. If there's a problem with the external drive's interface (USB?), then the files might actually be OK on the disc. If not, then the data really is corrupt and there's no getting it back.

Try connecting the external HD another way, or on another PC.
bryant
QUOTE(gl.tter @ Apr 2 2007, 15:35) *

Cool, he's probably just writing the MD5 hash in slightly the wrong place (or maybe he's forgetting to write it altogether?).

It should be:

<pack all samples>
call WavpackStoreMD5Sum(...)
call WavpackFlushSamples()

That will work fine but my documentation says:

CODE
  1. get a context and set block output function with WavpackOpenFileOutput()
  2. set the data format and specify modes with WavpackSetConfiguration()
  3. optionally write a RIFF header with WavpackAddWrapper()
  4. allocate buffers and prepare for packing with WavpackPackInit()
  5. actually compress audio and write blocks with WavpackPackSamples()
  6. flush final samples into blocks with WavpackFlushSamples()
  7. optionally write MD5 sum with WavpackStoreMD5Sum()
  8. optionally write RIFF trailer with WavpackAddWrapper()
  9. optionally append metadata tag with functions in next section
  10. optionally update number of samples with WavpackUpdateNumSamples()
  11. close the context with WavpackCloseFile()

which is actually wrong because the MD5 sum will not get written! There should be an additional step at 8.5 to call WavpackFlushSamples() again. Doh! sad.gif

Maybe a registered user should show this post to the author of Easy CD Extractor...

David


QUOTE(gl.tter @ Apr 2 2007, 15:35) *

(why do my new replies keep getting attached to my last reply??)

It's a feature so you can easily do this. smile.gif
gl.tter
QUOTE(bryant @ Apr 3 2007, 04:02) *

That will work fine but my documentation says:
<snip>


The function name is confusing - people might easily assume that it's only required after packing the samples.

How about requiring the call only once (as you said it works anyway), and calling it 'FlushStream' or 'FlushData' instead?

QUOTE

It's a feature so you can easily do this. smile.gif


Is that sarcasm or true? smile.gif If it's a feature, how do I turn it off?
photographer
Gentlemen,

The author of the software has made the corrections that you noted and has published a link to the updated .dll's in the thread at : http://www.poikosoft.com/forum/viewtopic.php?p=2573#2573 . Thanks very much for your help and your great softwares.

JD
bryant
QUOTE(gl.tter @ Apr 3 2007, 04:37) *

QUOTE(bryant @ Apr 3 2007, 04:02) *

That will work fine but my documentation says:
<snip>


The function name is confusing - people might easily assume that it's only required after packing the samples.

How about requiring the call only once (as you said it works anyway), and calling it 'FlushStream' or 'FlushData' instead?

Well, there actually is an odd case where the call would be required twice. If the audio only consumes a single WavPack block then the additional flush is required to make sure that the RIFF header and trailer are not in the same block. Not very likely, but possible...

And, of course, you know I'm not going to rename the function, B! smile.gif But I will update the docs.


QUOTE(gl.tter @ Apr 3 2007, 04:37) *

[
QUOTE

It's a feature so you can easily do this. smile.gif


Is that sarcasm or true? smile.gif If it's a feature, how do I turn it off?

I am never sarcastic! smile.gif

I haven't tried to turn it off (because I like it), but I think that the button labelled "+ QUOTE" with the tooltip "Toggle multiquote addition" might do it.

QUOTE(photographer @ Apr 3 2007, 10:52) *

The author of the software has made the corrections that you noted and has published a link to the updated .dll's in the thread at : http://www.poikosoft.com/forum/viewtopic.php?p=2573#2573 . Thanks very much for your help and your great softwares.
Wow, that was fast! smile.gif

Thanks for the link, but unfortunately only registered users can access that page. Does it seem to fix the problem?

Thanks,
David
sn0wman
hello, i am curious about one thing. it is written in the 'readme' file:
QUOTE

Each Wavpack block is tested for CRC errors, stored sample totals
are compared to the actual number of samples in the file, and MD5 fingerprints
(if present) are compared to the audio data.
on my computer, verifing with your app a set of files having stored fingerpring takes about 56 secs. when i use speek's wavpack frontend to verify these files (no switch), it takes almost the same time. little longer but still almost a minute takes to verify files with -m switch, which implies decoding and then hashing the stream with md5 algorithm. so, doing both verifing the integrity (blocks' crcs) and fingerprint should take about 2 minutes.
how you do it in a one ?
- i missunderstand something, or
- it is not written correctly in the readme that you do both things, or
- you really do it in some magic speedy way smile.gif

//edit
well, another thing suprises my but its somehow OT here, but btw, as far as i understand, in context of wavpack, veryfing means checking all blocks CRCs, and veryfing with -m switch is about decoding and checking md5 (is it correct?). if it is, then how is it possible to do both things within almost the same time ? decoded file is more than twice bigger, and md5 is little slower than crc algorithm. i am not questioning anything here, just wonder if i can do some things better in my own code.

//edit2
these may be the answer:
(from format specs)
QUOTE
uint32_t crc; // crc for actual decoded data

it means, that it is not crc for a block itself, but for block decoded ?


best regards, sn0wman.
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2009 Invision Power Services, Inc.