Unfortunately I have not got a chance to work on -x4 yet (and I'm not even thinking about MMX). I have added commands to wvunpack to automate the extraction of embedded cuesheets, but that's just in SVN for now. Thanks for your (and everyone's) testing so far (and thanks for your patience)...

I did finish up a first pass at a test suite that should be a useful resource for developers including WavPack support in their applications (and is important for acceptance of WavPack in the Linux community). It also has some samples at various lossy bitrates that provide a demo of WavPack's capabilies outside pure lossless. Anyway, if anyone would like to check it out, it can be found here:

http://www.rarewares.org/wavpack/

BTW, many thanks to Roberto for the hosting!

Yow beauwty!

Looks like I may have to get SVN setup to play...

Thanks for listening David.

Actually, it's not too fancy. It doesn't edit the cuesheet or anything like that, but it does get it out without having to jump through hoops.

I could easily post another alpha if you're not set up to compile it and would like to beat on it...

Great to see you're able to allocate time to WV these days and things are progessing quite a bit, thank you David, being patient is the least we can do, such a brilliant product and free to boot, I'd like to help more (tho ain't no coder) or being able to donate at least.

P.S.
I'm a Kubuntu user too, albeit a total noob.

EDIT: Grammar grammar grammar.

I'm not currently setup for it, but I wouldn't want you to waste your time; that was not my intention.

Thanks for the offer.

SVN works over HTTP, so any web-downloader will work if you're in a pinch; just do a recursive download and you'll get the current revision of the mainline trunk.

(The WavPack SVN server doesn't seem to use trunks or branches, though.)

Here is a build from the source this morning:

http://www.4shared.com/file/3518984/727cc0...k_20060906.html

It was built with MSVC++ 2006 Express Edition, with the default project settings. If there are any problems, please let me know.

Thanks Bryant! Great software!

Well, it certainly would not be a waste of time (and probably wouldn't have taken 5 minutes) but lantern did it while I was sleeping (thanks!) and I checked his version and it seems to be fine. I would be interested to know if this meets at least some of the desired utility, although I know you are busy...

I actually managed to beat lantern to it, but didn't think to post my builds. I had TortoiseSVN here at work already, and installed Visual C++ 2005 Express with little trouble. It's more fun with a little effort.

What can I say, it does exactly what it says on the tin! I like the options to both output to STDOUT and to file on extraction. It's great that WVUNPACK asks for confirmation before overwriting an existing cuesheet, as it does for the WAV file.

I tested with a self-extracting file which I believe I successfully added a cuesheet to on encoding, but on extraction the cuesheet was not extracted. I may need to test again to be sure; it may be safer to encode to WV, check the cuesheet is there (-c switch ), and then prepend the wvselfx.exe file, before I test.

IMHO, if a self-extracting file has a cuesheet it will be required, and it would be great if it could be automatically extracted when the EXE is double-clicked. Is that achievable? There's something nice about the idea of giving someone an EXE and saying "double click that and you have everything you need to burn to CD in Nero". (If someone knew about EAC, foobar, or Burrrn they could handle the WV themselves).

I can't really think what else to test... the cuesheet is simply a piece of text (in this instance) so WVUNPACK has no concerns about compliancy, etc. I can't think how I could trip it over... it either extracts or it doesn't. Is there anything specific you would like tested?

Thanks again David.

Edit: OK, tested self-extract again using method above, and no cuesheet.

I second that; thanks, Bryant!

The wvselfx header is due for an update and that's where the logic needs to go to self-extract a cuesheet. My plan was to simply have it extract the cuesheet if there's one there; I can't imagine a scenario where that would be a problem.


Actually, the fact that you ran it on a different machine with a different image and cuesheet is a good test. I was mostly concerned whether this was enough functionality and that the options made sense. It turns out that it is a little more complicated than it seems because the cuesheet is stored as UTF-8 in the tag and needs to be converted to ANSI, and I had a bug where newlines were not being handled correctly, but I think it's all sorted out now.

Thanks for the feedback!

Superb. That would be ideal.

Well, we still have the idea of editing the cuesheet upon extraction to ensure that it points to the WAVE file, but I realise that this is a lot more effort, and possibly should simply be bounced back to the embedder's responsibility to ensure that the cuesheet points to a WAVE of the same name before embedding.

Unfortunately mine don't, and I guess EAC doesn't do that if you rip compressed. However, again, I suppose you could say that the embedder should have the skills to edit the cuesheet. It's all down to making it as easy as possible for the decrypter, not the encrypter.

Both would be good though.

When I made my changes to Tag I had to mess about to get new lines coming out OK in STDOUT. Hmmm... I wonder whether it barfs with non-ASCII characters.

That's why you do what I do, and I just stick my oar in.

David, I've been using your nice little proggy copytags a lot lately, it's definitely pretty handy, any intention of integrating its functionality into wavpack.exe?
If not in the near future what about when all executables will be merged into one?

Oops, sorry, for some reason I missed this post...

Glad you can use copytags. Just don't use it on anything important!

Yes, once wavpack.exe can accept wv files as input (for transcoding) then it would also copy any tags found.

As part of my Yalac testing I have just completed some tests with 4.4a3. For the full results, please see this post on the Yalac thread for info.

Here are the WavPack-related results though, settings in compression order.

REMOVED, please see post #68 below.

Thanks Synthetic Soul for adding the new WavPack version, I really appreciate all your work on this!

The results fit my expectations pretty much, however I think there's one error. I find it unlikely that "4.4a3" and "4.4a3 -f" would get exactly the same compression ratio. I think they must be both "-f". If that's the case then it's also interesting that the two decode speeds differ by almost 3% when they should be identical! I suspect that this indicates that your test method has a higher level of measurement error than your 4 significant digits suggest and that perhaps rounding to the nearest integer would be more appropriate.

Finally, a request. If I could add a single option combination it would be "-fx". I think this is significant because it offers faster decoding and better compression than "-f" alone, and with a fairly small increase in encode time. For those users who might consider WavPack as a FLAC replacement it is certainly the most logical choice. I would even be willing to give up the other -x2 options for this because they really don't offer too much over -x alone. But, of course, this is all up to your descretion.

Thanks again for the results.

Argh! I have just checked the scripts and both runs were using -f. I do try to check my results when I publish, as I am very concerned about spreading misinformation, but this obviously slipped my net. Many apologies.

Personally, I'm not suprised that the speeds vary slightly. My reports currently list TIMER's global speed which includes CPU and IO time. I suspect that the IO process, which in my case slows the process down, introduces some variables. Your point about using integer values only is extremely valid, and not something I had previously considered. I had a brief discussion about significant bits a while back; not being a math wiz I don't really consider the fact that the decimal places shown indicate the precision. I will change it ASAP. FYI: My scripts are now recording both CPU+IO and CPU only times, but I need time to update my report system to be able to display both/either. I would be a lot happier displaying CPU-only results.

Of course I will run -fx and add that in to the report.

Thanks for the feedback David, and again, apologies for the false info.

BTW, although it was posted in another thread, I have to say:

... has got to be the best analogy I have ever seen!

OK, I've updated the default figures immediately, as they were wrong. I'll run the -fx (and probably -fx2 and -fx3) tests today and get them up as soon as I can.

Here's the correct data:

And now with -fx(2/3):


http://synthetic-soul.co.uk/comparison/lossless/?All=1

I made reference in the Yalac thread when I posted my initial results that I had never previously considered WavPack's- -f option, and was surprised at the speed and compression. It's impressive that WavPack can compete with FLAC -5 in speed, and still surpass Flake in compression.

I also find it interesting that the changes in 4.4 are geared toward faster, less processor-intensive decoding, and that has come at a small cost in compression. I must admit, I think I may be sticking with -h (-hh), but it's very possible that I may switch to the new -h for a little faster decoding at the cost of a few MiB, as it seems the difference to me would only be around 4MiB per GiB encoded, which really is negligable.

Edit: David, can you see the presets changing at all for the release version, or will I be able to rename to 4.4 and leave the results as is?

Ah yes, those results make more sense now! Thanks for updating them so quickly and adding the -f results also.

Yeah, I find that this speed measurement business can be a real nightmare trying to get results that are consistent and make sense. There are enough variations between CPU's and OS's and I/O devices and compilers to make your head spin around! Anyway, I think having those numbers rounded to ints makes sense and doesn't convey more accuracy than is actually there.

I don't know if you originally encoded your files with -h or -hx, but in my corpus the new -hx3 actually compresses better than the old -h (i.e. new -hh). I notice that for some reason the -x mode achieves a lot less extra compression on your corpus than on mine. Probably that's because my corpus was designed to have a wide and varying range of material, which is exactly what the -x mode is trying to take advantage of.

I'm not really sure what I'm going to finally do with the presets. I am going to try to add a -x4 setting to work more like the old -x mode, but this is really only for special cases and certainly wouldn't make sense for your corpus. I would also like to see if I can make some quick performance tweaks while still staying in pure C, but time will tell whether that happens. Basically, I reserve the right to improve everything dramatically!

Oh, BTW, glad you liked the analogy...

I don't really like posting my "IO-infected" rates, but it's how I started reporting and I would need some time to switch totally to CPU-only. I am also still undecided as to how irrelevant IO+CPU times are, as this is what I would see. I've never meant my tests to be the definitive figures for each encoder, but it concerns me that people may think that is what I am trying to portray. Thanks for the suggestion to report integer values; it makes a lot of sense.

I use -hm at the moment. I assumed that -x must perform better with other corpuses than mine. A lot of people seem to use -x. The fact that it is not nearly such a performance hit as the old -x is great anyway, and does mean I may consider using it, in case my music taste gets a little more varied.

Hey, fine by me. I will re-test when you release 4.4; it may be interesting to compare the two results anyway, to ensure they match where they should.

Thanks David.

Out of interest, here's the results for my "Yalac" corpus in a different format.

Firstly, the speeds are CPU-only, so they are not affected by my hard drive. Bear in mind that, if you decode to file, you may not attain these speeds, even if using the same equipment, as writing to disc may introduce some further delay (refer to my results listed above to see the difference my hard drive makes, or take a look at the graph on this unfinished page).

Secondly, I have grouped the switches, and I think the result quite neatly demonstrates the effect of using -x in addition to a core setting, so I thought I would share.



I am currently running the same scripts on my "FLAC" corpus. When I have the results (around a day's time) I'll post them in this same format. I'm hoping the greater (although probably still not great) variety in my "FLAC" corpus may demonstrate -x's benefits more.

Synthetic Soul the build you're using doesn't have any MMX/SSEx optimizations right?

No, it doesn't. It's the version from post #30 in this thread.

I'm not overly interested in optimised versions until David applies them himself (at which point I will become very interested).

I am very interested in optimised versions since my PC isn't the fastest.

I currently have to work abroad, but I'll probably investigate in those optimisations again, when I'm back home in November. Anyway, I think David should release a final version first, so that we have a stable base to do the testing on.

It's really nice to see the recent improvements in the field of lessless encoding. Thanks to all codec authors for sharing their work with us!


Kind regards,
Jo.

A little more benefit with this corpus.

I agree. Currently, the function decorr_stereo_pass() is unchanged, so the previous patch could be dropped into the current code, but I don't think it's worth it for now because I plan on trying to do some more optimization on the C code.

Once the release is done I would be very interested in getting these MMX optimizations in, and I really appreciate your and wisodev's work on this.

David

i did not read all pages of this thread so i don't know if this is the wrong position to post this.

i have a feature request for the new wavpack version:
it would be great if the wavpack encoder would accept wavpack files as input. then you could easily update your "old" wavepack files to the new version. of course all tags/etc have to be copied also.
thanks and keep up the great work.

You can easily achive the audio conversion via pipes:



This has one disadvantage tho, the tags will not be preserved.

not a solution for me as this doesn't copy my embedded cuesheets and logs but thanks anyway

Unfortunately, this feature will not be able to make it into the new version, but it is something that is planned (especially now that FLAC can do it).

However, in my testing I use a few short commands that basically achieves the same thing (and I'm sure a simple batch file would simplify it further). I describe it here:

http://www.hydrogenaudio.org/forums/index....mp;#entry423126

And, of course, Foobar2000 or dBpowerAMP would both work fine (once they are updated with the new library).

I'm curious, how much more can lossless audio compression be optimized? Wavpack seems to have achieved its maximum compression ratio (while being statistically significant) .

I guess we have to face the fact that we're close to what can be achieved. All the good codecs like wavPack, FLAC, Monkey and others yield a compression ratio which is more or less the same (not counting peas). Progress is still there, but it's not very essential any more. The developers have a done a great job already.

But other things count as well, like David's progress when using wavPack on mobile DAPs. Top quality lossy versions of lossless codecs are very interesting too. I'm very curious about 4.4 final and I'm looking forward to it.

bryant, one question. When i remember right you did some programming for Slimdevices to add wavpack support to Sllimserver.
The Transporter of them now has a Ubicom IP3K series Proceessor @325MHz. The Squeezeboxes only had 250MHz.
I can imagine a -h encoded file should now be possible to be decoded natively in the unit.
Do you have any infos if something like this is planned?

I did try to get WavPack working with SlimServer, but got sidetracked. I still plan to go back to it at some point.

There was never any plan to get WavPack decoding natively because that source is closed (and I guess that the tools are probably expensive, so it won't be hacked either). However, if it actually was done I think it would work pretty well even at 250 MHz. Even decoding the -h (now -hh) mode of WavPack lossy (which is worst for CPU usage) uses less than 75 MHz on an ARM core, and I suspect that their processor isn't that different. And I think WavPack lossy is a natural for the Transporter because high-res 24/96 material compresses very nicely to around 1024 kbps.

But it turns out I'm not "in the loop" with them right now...

Thanks for the clarification. So lets hope it sometimes happens

Warning: I am a newbie.

Some time back (Oct 2005) I encoded all my CDs using WavPack 4.2. I have now about 180GB of Wavpack files.

the arguments used were
hm -w "Artist=%a" -w "Album=%g" -w "Track=%n" -w "Title=%t" -w "Year=%y" -w "Genre=%m" %s %d

the file path used was
F:\wav\%a\%g\%n - %t

1. Do I need to reencode them using the latest version of WavPack?

2. Can these files be played on any hard disk based player (espcially any protable). The idea is to stop using my CDs and used Hard disks (both at home and on the road) for playback. I presently use MP3s on an ipod on the road but there is a significant difference between my MP3 (--preset extreme) and my wavpack files and i'd love to use wavpack if possible.

I dont want to have a PC or Mac involved for playback as the boot time is a deterent.

1. Reencoding is not necessary. You will get slightly better compression, but it's just not worth it.

2. Any? No. You can find a list of hardware support for wavpack here. If you have an ipod, you can rockbox it and use wavpack on it.

Thanks Kanak, now that there is a 160GB ipod out storing wavpack files I am more serious about using my wavpack library.

Q1: If I use rockbox do I still need iTunes? If not how do I sync my ipod and create playlists?
Q2: My car CD player (Alpine) is designed to control my ipod. This works nicely using MP3s and an ipod. Will I use this compatibility if I switch to Rockbox?

If the new Ipods use the same hardware and encryption schemes as the 2nd generation Nano, you might have to wait a long while before seeing Rockbox support. Rockbox does not need Itunes at all, you "sync" music by plain and simply copying files to the Ipod with your tool of choice, and create playlists the usual way, either on your computer (then copying them to the Ipod), or by creating them directly on the Ipod with Rockbox.


Rockbox does not support this, but anyone is free to figure out how this works and code it for us.

how do we know what scheme is used by the new 80GB and 160GB Ipods? Is there any way to determine this before we buy the new ipods. Presently I have old 80GB ipods (Ipod Video).

Is there any other software (other than Rockbox) that will allow one to use wavpack files on an ipod?

Only way to find out is wait until someone opens one up and has a good look. The hardware does indeed seem to be similar to the 2nd gen. Nano, which means we have no data sheets on it, and I sincerely doubt they have downgraded their efforts at locking us out with encryption.
And no, nothing (to my knowledge) plays Wavpack apart from Rockbox. On Ipods you pretty much only have Rockbox and Ipodlinux to choose from, and the latter does not seem to be developed much anymore, and does in any case not support Wavpack.
Your old Ipod _is_ supported by Rockbox, and nothing is keeping you from using your Wavpack files on that.

Found a solution. Use new 160GB ipod with wavpack files and rockbox for portable listening with Bithead and good headphone (AKG 701 or Beyer DT990) and old 80GB ipod with MP3 files (EAC/LAME) for the Alpine car deck.

+
=
Not currently possible