First, I'd like to thank Guru and the Mods for the dedicated WavPack forum. Very cool!

Despite the fact that it seems pretty quiet lately on WavPack development, there has actually been a lot going on. Today I'm posting another 4.4 alpha that has some major changes. I am also finishing up a reorganization of the source code and the creation of an official library and API, including a Windows dll.

The new alpha has a completely new high mode. Its decoding complexity falls just about halfway between the default mode and the old high mode, but its compression is closer to the high side. The old high mode is still available and is called very high (-hh).

This new mode is particularly suited for use on the iPod using Rockbox. The old high mode files play, but use so much CPU to decode that there isn't enough left over to perform audio effects (like EQ) or even allow a peakmeter on the display. There are plans to eventually use both CPU cores on Rockbox, but I think this new mode is a much better compromise between decoding demand and compression (even on the PC).

Also, the extra mode is completely redone. The old mode was useful for creating the smallest possible WavPack file for a given quality, but was so slow that it was useless for most applications (and the -x levels that were fast enough to be usable usually didn't do anything). The new extra mode has only three levels (1 to 3) and they are much faster than before, although they can't quite match the old mode's ultimate compression ratio. But, just using -x by itself (which is equivalent to -x1) makes such an improvement with so little extra processing that I am thinking of having it always on (once it's a little more optimized).

This version uses different decorrelation filter tuning in all modes, so don't be surprised if you see differences between this and the previous. For my corpus this results in a small average improvement, but there are certainly files that do worse (although with -x now usable all the time, this shouldn't be an issue).

I have not incorporated any of the MMX enhancements, and in fact have not done any optimization on this yet, so I expect some more speed gains (and the new -x mode is a good candidate for threading).

This is an alpha (and not well tested either), so please compress responsibly.

Any feedback or results are very much appreciated.

http://www.wavpack.com/wavpack44a2.zip

Are you planing to add MMX optimizations in WavPack 4.4 final or some later version? These unofficial tests with 4.3 version and MMX optimizations where quite promising.

Thanks David, great news!

Is the old -x6 switch still somehow accessible?

Also, any progress towards a single file encoder/decoder yet?

Wow, David this sounds good. I am going to play with this alpha during the week to test lossy and lossless modes.

Here's my results... (for a 40 min. album) (Original Size = 412,937 KBs)

-m is in there because I always use it.

-x does have a nice effect on the normal setting, but it's not good with either high settings. I think it's better to have it off by default, unless using either high setting would automatically turn it off. The new high setting does seem nice, but my PC handles the old high setting perfectly, so i'm probably gonna stick with that. I can see how the new high setting would be a great option for some people though.

Instead of changing the old high setting to "very high", couldn't you name the new one something different such as "new high" or "high new" kinda like OptimFrog? Then again, I really like how "very high" sounds. Only thing is getting used to -hhm instead of -hm, but I wouldn't exactly call that a problem.

I'm looking forward to see how the newest version turns out in the end. Thanks for all your work on WavPack.

i did a quick and dirty test of decoding speed with foobar

-h == 99.999x
-hh == 77.003x

CPU = AMD 64 3000+ @ 2.3

nice work

From brief testing its good to see that -x2 is doing something as opposed to the old -x2, and its not that slow. -x or -x1 is doing only auto-joint stereo like v4.31 from what I can tell. This -x1 should be speed optimised as I find it very useful in lossy mode. Currently I use -hx1 in v4.31 MMX. The new high is also working better than normal mode -x so one can use it instead of -hh and still get comptitive compression and lossy quality and quick decoding for devices.

I wil do more tests, but this looks like a step in the right direction. David you are the man !

What is the function of -x1 in 4.4 ? Looks like its doing some extra compression as well as auto-M/S ?

I compressed a file with 4.31 and 4.4

4.31 -h = 971 k

4.31 -hx1 = 971 k

4.4a -hh = 974 k

4.4a -hhx1 = 969 k

Ok, I get it: now -x1 is also doing extra compression and its quicker than the old -x1. I also think that defaulting an optimized -x1 would be a good idea.

Are the source code available for download?
That would be helpful for us non-Windows users!

http://wavpack.com/downloads.html

I think he meant the sources for 4.4.

I could have thought a bit before posting.... but the sources do not seem to be online.

Always good to see that lossless codecs are still improving. Keep up the good work bryant, and I might give up ape in favor of wavpack

I just tried 4.4 hx3b288s0 and hx3b352s0 on my standard problem samples trumpet, herding_calls and harp40_1 in comparison to 4.31 hx6bxxxs0.

4.4 is great. With 4.31 I could abx trumpet with hx6b288s0 at 9/10 and with hx6b352s0 at 8/10.
With 4.4 hx3s0 and same nominal bitrates I could not reliably abx.

4.4 hx3 better than 4.31 hx6? Will try again tomorrow, I'm pretty tired now.

Great work, David. Thanks a lot .

I wonder how the existing decoders are working when fed with this new high mode. Decompression must be different to the old high mode cause otherwise decompression performance couldn't be better.

But I guess I don't have to worry about compatibility.

You can get a rough idea by the lossless compression ratios - the one with the better compression should have the best s/n ratio. A small difference of say +- 5 kbit won't make much difference and you can also use -n switch to get the average noise report.

You perform an ABX test on a lossless audio codec?

remember that WavPack has a hybrid 'recoverable-lossy' mode. I think halb27 meant he ABX-ed the lossy WavPack stream/fork/whatever

Thanks everybody for your feedback; looks like no serious blow-ups yet...

askoff:
I am definitely interested in the MMX enhancements! I really want to get 4.4 out quickly, so it won't be before then, but I want to get it into SVN right after that. I haven't looked into that code for a while, but the last time I looked there wasn't anything for decoding, which I'd also like.

DARcode:
The old -x mode is no longer available. However, I do plan on merging some of the features of the old -x mode into the new one for those weird (mostly artificial) samples that really benefit from generating filters from scratch (which is what the old version did). This will probably end up being -x4, or something. Sorry, the combined program is still rather low on the list...

DrazardX:
I thought about calling the new mode "new high", but I never liked names like that because they don't tell you where the mode fits with the other modes (e.g. is "new high" higher or lower than "high"?). And, fact is, I'd rather have people who don't any better simply use the new mode so I don't see them post that their WavPack files sometimes skip on the iPod! I haven't decided about the default, and even if I make some modes use "extra" by default you'll always be able to turn it off with -x0. BTW, welcome to HA!

halb27:
I would be surprised if 4.4 hx3 is better than 4.31 hx6 on lossy mode! However, in the process of doing this I did discover a bug that would cause unbalanced noise when lossy mode switches on and off joint stereo (which the -x mode does). Perhaps you are hearing that. Oh, and I guess I forgot to mention that all this stuff is all backward compatible. I haven't used up all the WavPack 4.0 format functionality yet...

shadowking:
This new "extra" works very differently than the old version. For each mode I have 256 decorrelation filters which are organized into a binary search tree. So, I simply search the tree testing the current best filter against a new from the table. For -x1 I always compare against one other filter for each block, for -x2 I compare against 3 others and -x3 always searches through the whole tree (which takes 8 compares). So, unlike the old mode where different levels actually did different things, in this mode different levels simply so more of the same thing. Since joint stereo is one characteristic of the filter, that simply gets indirectly factored in. BTW, the -j[n] switch currently will not do anything in the new -x mode; haven't quite figured out what to do for that.

I am getting close to actually working on that smart noise shaping that we talked about a long time ago. In the meantime, do you think it would be worthwhile to include that prototype mode in a new release even considering that it can't work with correction files? Do you still use it when creating pure lossy files?

krmathis:
My next project is to merge all these changes into the new common source in SVN. As soon as I do that I will post so everyone can use it (I am actually becoming a pretty avid Ubuntu user myself).

BTW, if anyone is interested in reading a little more about how WavPack works internally, I finished up an article on it for an upcoming book. Here's the excerpt:

http://www.wavpack.com/WavPack.pdf

Thanks again everyone!

As you can see from the parameters I use wavPack as a very high quality lossy encoder.


As from former tests I expected herding_calls to be the most critical among the samples I tested. And I was very surprised that I could abx trumpet at high bitrate with 4.31hx6. There's a little 'blip' (hard to describe) within the critical passages. It's very subtle, but it's there. With 4.4hx3 I couldn't hear it. Maybe it's caused by the bug you mentioned.

As I will not use 4.4 in alpha state for productive purposes: can you do a correction for 4.31 concerning this bug?

I used correction files for dvd archiving to this point, so I didn't use it much. Now I think I'll only keep them only for certain albums. I reckon it won't hurt to put in a switch for it if the real thing won't make it into 4.4 final .. that way people who don't use correction files can still use it. If you think that it will be ready by 4.4x or 4.5 then that's fine too.

Nice to see you becoming a GNU/Linux user.
This hopefully mean you will be able to share alpha source code as well, so non-Windows users can take part of the testing as well. Which mean even more testers and feedback!

Are the WavPack SVN server a public one? If so, where can I find it?
I don't see it mentioned on the website, and the SourceForge project site only list an old CVS repository (18 months since last checkin). https://sourceforge.net/cvs/?group_id=74831

Thanks in advance!

I understand.
"hx6b352s0" did (does) not look like a parameter to me, so I did not catch it.

I redid my test with -b352hx3s0 on trumpet, herding_calls, harp40_1, and added Atemlied (which I could easily abx at 288kbps) and badvilbel (which should be abxable up to the 4xx kbps range).

Except for badvilbel I could not abx these samples, and even badvilbel was acceptable.

I will use 4.4 b352hx3s0 in the future when it's final.

Thanks again, David.

Maybe it's a stupid question, but why the WavPack encoder is not able to add Replay Gain tag directly without using wvgain.exe (just like FLAC does) ?

I have foobar engrave the replay gain info. No problems.

The SVN server is at http://svn.slomosnail.de/wavpack

There, now it's public!

The code in there should build on Linux and there are project files for MS Visual C++ 2005. It has been reorganized from the released sources with separate folders for the library and the cli modules, however the alpha code is not in there yet, nor are any of the other plugins (and the docs are not up-to-date either). I hope to have some of that done this weekend...





This would be possible for track gain, but I never figured out a good way to handle album gain this way unless wavpack was used to process a whole album at once (which EAC won't do). Having it a separate program allows easy processing of track and album gain. I'm not sure how FLAC handles album gain...

David, will there be a fix for the 4.31 stereo mode switching bug?

Thanks for your answer.

Excellent!
I will keep an eye on it for further updates.

Lets hope so, cause I really want to give the alpha 2 a go...

Okay, I have updated SVN with the alpha sources and also fixed a bug that could cause a crash when combining -c and -i and -r on a wav file that has the incorrect number of samples (thanks Marius!). I have also switched to Visual C++ 2005 (from 6.0) and compiled a new Windows version here:

http://www.wavpack.com/wavpack44a3.zip

I had been told that the new Microsoft compilers had better optimization than before. Well, this executable is much bigger and much slower than the previous (at least on my P4) so I'm not impressed yet. Perhaps I need to tweak the settings some...



No, I won't be going back to fix old versions when I'm just a few weeks away from a new, much better release. I would actually trust the current alpha more than a patched 4.31 anyway, but if you want to stick with the old version for now I understand.

I am planning to try your command args on "trumpet" to see if it actually does trigger that bug (at this point I am not sure that the bug actually exists).

Very glad to see WavPack getting better.. =)

Bug: When executing (simple exec, without passing any parameters!) this command file (it's invaid, I know)

, WavPack 4.44a3 and 4.44a2 crashes.

WavPack 4.43 doesn't crash on it.

I didn't realize you are just a few weeks away from a new release. Sure I can wait with my music which I want to encode at extremely high quality. Not so much at the moment anyway as I did encode most of my music collection.

Thanks for your great work.

Also I have a strange sample: a 546 MB wav file (CD with Russian and Soviet marches; very LQ, restorated)




(!) - an old x mode gives such significant improvement in some cases. Could you return it again, and change it's switch, f.e., to -xx ?

(!!) - SUCH a big difference between best mode of WavPack and very high mode of MAC (it also has -c5000 (insane) mode) -- is it normal, or it is unexpected WavPack's behaviuor?

Thanks!

It might be me, but I can't get rev. 7 to build.
Version 4.32 compiled with a plain "./configure && make", and the README file in SVN have the same instructions:But there are no configure file in the source, so it fails.
Making autogen.sh executable and run it end with errors as well. Like this:This process creates the needed configure file, but running "./configure && make" errors out like this:
You provide two different source code archives for 4.32, so perhaps this is the MS Windows one only?
So do anyone know how to proceed from here?

I only use Replay Gain to avoid clipping so I'd really enjoy an option to add track gain tag during the encoding process (like --replay-gain with FLAC), and also a modification in the Winamp plugin to enable clipping prevention without enabling Replay Gain (like in the Nullsoft Vorbis plugin).

Anyway, thanks for your great work on WavPack, I think I will transcode all my Monkey's Audio files to WavPack when the version 4.4 will be released.

Quick test with my usual sample WAV.


V4.4a3 -hhx gives better compression and is faster'n -hx3?
Also, v4.31 -hx compresses better'n v4.4a3 -hhx3?
The new version is pretty fast and -hhx3m is very efficient, tho it'd be very cool to be able to achieve v4.31 -hx6 compression ratio w/ it.

Re:http://www.hydrogenaudio.org/forums/index....st&p=422091Is that so really?

Last but surely not least many many thanks for your brilliant work David, appreciated!

EDIT:Grammar + misc imprecisions.

It's somewhat disappointing to see 4.31 tending to outperform 4.4a in terms of compression - but the differences aren't too great and I'm sure that the resulting benefits are worth it!

NeXT:
I'll try to look into that crashing problem you have there.

Yes, the old -x mode does often do a little better than the new -x mode, but at a huge speed penalty. I am planning on re-introducing it integrated into the new scheme, probably as simply -x4.

As for your file there, this looks very much like the old "mono" problem that would be addressed with the --optimize-mono switch. Have you tried that?

halb27:
I can see why you might think that a new release was not imminent because the first alpha was so long ago. But, I didn't do a release for the first alpha because I didn't think the only added feature (--optimize-mono) was significant enough to justify a new release. There are certainly enough new now to warrant it, however, so it won't be long.

krmathis:
Yes, the documentation is for the actual distribution. To build from SVN you will need libtool (I have version 1.5.22) and maybe autoconf. You should be able to easily find that for your distro. Hope that helps; I'm kinda new at this part myself...

The source packages are definitely not going to be separate anymore.

dutch109:
What I try to do is get basic functionality in there and worry about extra convenience later. Someday I'd like to combine all three command-line programs into one that will do everything (like FLAC), but so far I've been concentrating on adding features and functions that are not available any other way.

It turns out that clipping prevention is not needed with WavPack. In lossless mode you can't get clipping without ReplayGain, and in lossy mode the clipping prevention is automatically done during decode.

DARcode:
Yes, on 4.4a3, -hhx will normally compress better and faster than -hx3, but keep in mind that -hx3 will decode faster than -hhx (and play better on Rockbox). You either pay once up front, or every time you decode.

I don't think these very fast -x modes will ever quite match the old -x mode in ultimate compression, but I will be putting in a modified version of the original -x mode that should work as well (or slightly better).

What Roberto said is basically correct. I don't think the WavPack format will be able to get any better compression than 4.31 with -x (well, maybe a tiny bit more, or maybe better with certain files). What I have been trying to achieve is getting close to that level of compression without taking forever to get it. And of course there can always be speed improvements through optimization (like MMX).

dv1989:
That is just temporary situation until the old -x mode is re-integrated. The idea is that the usable compression is better.

Thanks David, I appreciate very much all the time you put in the development of your superb audio compressor and answering questions here on HA.

I agree on the "usable" compression take, but please do re-integrate the "old" -x mode, I'm already salivating at the prospect of an MMX/SSEx optimized -hx4 (x4 = v4.31 -x mode) :] !

Finally, is Rockbox the only option for WV on DAP or have you been made aware of further projects?

EDIT: "Usable" comopression and Rockbox parts

I should mention that that commandline is INVALID -- WavPack should do nothing and exit. So my example is not a PROBLEM, preventing from doing needed job, but just a bug, discovered acidently.



Oh, I never noticed this command, because it appears only in WavPack 4.4 internal help, but not in any html file (neither site, nor bundled User Manual), or internal help of WavPack 4.31. Yes, it helped, that wav compressed with --optimize-mono -hhx3 has 84 539 008 bytes now, but still diference with Monkey's Audio is 8 667 630 bytes. Waiting for returning old -x mode to test it with --optimize-mono..

I seem to be having problems accessing the sourcecode from the svn server. Here is the error message I receive:

Error * PROPFIND request failed on '/wavpack' PROPFIND of '/wavpack': Could not read status line: An existing connection was forcibly closed by the remote host. (http://svn.slomosnail.de)

I am using tortoisesvn.

Thanks

Unfortunately, no, Rockbox is the only DAP option right now, at least for portables. There is the Roku PhotoBridge which has a third-party plugin available for WavPack:

http://www.rokulabs.com/products/photobridge/index.php




I don't normally use tortoisesvn (I use svn under cygwin) but I do happen to have it installed now and just tried it and it worked fine. Not sure what could be up; I assume you tried it more than once...?




The reason the command isn't in the manual is because it was only introduced with alpha 1. You might want to look at this thread:

http://www.hydrogenaudio.org/forums/index....showtopic=43866

I probably should have mentioned this in this thread also.

But wait! I know there was an effort to get a wavpack codec supported in TCPMP (in fact I am testing it right now on my Treo 650). Bryant are you aware of this development?

Support on my Treo is one of the main reasons I will be switching my entire collection over to wavpack: one set of files that can be used for both my portable/Treo (space conscious hence lossy) and my home (space and bandwidth o'plenty). This new 4.4 release sounds good enough to wait for.

In the mean time I just have to get slimserver to support wavpack (.wv + wvc) including tagging to stream to my SliMP3 and my Roku Soundbridge M1001. I figure since it can support FLAC then we should be able to get wavpack going.

You must have libtool installed before running autogen.sh

I have tried several times. Now it is asking for a username and password to access svn.slomosnail.de.

Edit: I have moved my "Compile WavPack from SVN" problems to a new thread:
http://www.hydrogenaudio.org/forums/index....showtopic=47682

I'm not sure what I did, but SVN seems to be working.

I was aware of the TCPMP development, but I seem to forget about it because it's not official and I don't have one of the devices. Thanks for reminding me and I'm glad to hear it's working good!

I also have a Squeezebox and tried to get Slimserver playing WavPack with it. Unfortunately, I didn't know Perl so I got a Perl book and went through that, but then ran out of time for that project. I'll get back around to that one of these days; if anyone wants to jump in before me I'd be happy to help...

Recompressed a couple of test ablums (Faith No More - The Real Thing and Primus - Frizzle Fry) with 4.4a3 -hx3m and compared to 4.31 -hxm they are only ~200/250 Kb bigger (including full tags w/ RG), so the time save and better DAP support is definitely worth it, bravo David!

EDIT: Grammar.

EDIT2: Mo' gramma.

24-7 Spyz - Gumbo Millennium is just ~220 kb bigger too .

Any new alphas to play with yet ?
Can't wait to test -hx4 (x4 = old v4.31 -x mode), maybe MMX/SSEx optimized too.