LossyWav: average filesize reduction Options

[chrizoo]

hi. what is the average filesize reduction when LossyWav is used prior to lossless encoding?

losslessenconding(WAV)=100%
losslessenconding(lossy.WAV)= aprox. ... ?


Given that there are plenty of lossless encoders, my question concerns the encoder which has the best performance (on average) in terms of final file size (not in terms of savings compared to the original size (withot lossywav)).

[halb27]

Best lossless encoder: TAK.
Second best lossless encoder and most universally usable: FLAC.

Saving of lossyWAV -P --altpreset | lossless is 50-55% that of lossless on average when used on pop music. (52% is saved on my test set of various old and new pop music for which plain FLAC -8 results in an average bitrate of 788 kbps).
Saving is better for 'wild' pop music when the lossless codecs' performance is worse.
Saving is worse for music with a lot of quiet and/or non-complex parts in it when the lossless codecs' performance is good to very good. This applies to many tracks of classical music or music originating from one or very few instruments.

This post has been edited by halb27: Mar 12 2010, 14:27

[Nick.C]

My collection averages 882kbit/s in lossless FLAC and 378kbit/s lossyFLAC (--portable --altpreset) (i.e. 42.8% that of FLAC or 26.8% that of WAV).

This post has been edited by Nick.C: Mar 12 2010, 15:13

[chrizoo]

thanks folks for your much-to-the-point answers. exactly what I hoped for.

@halb27: do the 50-55% that you mentioned refer to TAK or all lossless encoders on average?

[halb27]

My experience is with FLAC.
Improving with TAK doesn't change the result very much however.
As you can see from what Nick.C and I wrote percentage saved depends more heavily on source material, and this comes mainly from the performance of pure lossless encodings which varies a lot (lossyWAV | lossless in contrary results in a more stable bitrate dpending a lot less on source material).
50-55% saving is more of my personal experience with pop music containing a lot of ballads and a lot of music originating from the time before the loudness war. With today's popular music especially of the vivid kind saving is better, it's more like what Nick.C wrote, or even better.

This post has been edited by halb27: Mar 12 2010, 15:20

[chrizoo]

halb27 and Nick.C,
you both mentioned FLAC as your preferred encoder for lossyWav.

Do you know if better compression can be achieved if another lossless encoder is is used in conjunction with lossyWav? What would be the best one in terms of smalles file size ?

[robert]

If you read post #2 again, you'll get your answer.

[chrizoo]

I wouldn't see why

[robert]

I wouldn't see why

Best lossless encoder: TAK.
Second best lossless encoder and most universally usable: FLAC.

[halb27]

Maybe I should have explicitly said: best resp. second best lossless encoder for encoding the lossyWAV result.

As well as Nick.C as myself are using lossyWAV on a mobile DAP. That's why we use FLAC.
If I'd use lossyWAV for archiving on a pc system I'd use TAK. (Temporarily I really did, but not much later ended up re-encoding [being a bit too interested in the lossyWAV progress in order not to do so], so I had to collect my lossless tracks from backups and partially had to re-rip).

This post has been edited by halb27: Mar 12 2010, 22:28

[chrizoo]

Maybe I should have explicitly said: best resp. second best lossless encoder for encoding the lossyWAV result.

Well, yes at first I guessed that's what you wanted to say. But then you went on arguing:

"My experience is with FLAC.
Improving with TAK doesn't change the result very much however.",

... which finally left me with not much of a clue.

How can TAK be the "best lossless encoder for encoding the lossyWAV result" when - simultaneously - "TAK doesn't change the result very much however" ?

[lvqcl]

My test for LossyWAV 1.2.0 --standard:

FLAC -8 -b 512 : 450 kbps
TAKC -e -p4m -fsl512: 430 kbps.

The difference is 450/430 = 1.0465 => 4.65%.

[halb27]

TAK yields the best performance on the lossyWAV result, but the difference isn't big usually from what I remembered.
lvqcl's results show however that the difference is bigger than what I memorized (though I didn't use -p4m).

Anyway:
If you do know that lacking TAK support isn't an issue, and if you want to have the filesize advantage of TAK no matter if you call it small or not-so-small: I suggest to use TAK.
In any other case: I suggest to use FLAC.

This post has been edited by halb27: Mar 12 2010, 22:47

[[JAZ]]

TAK best in terms of compression ratio

FLAC not too far behind in that comparison, while being much more compatible.

Does that make more sense now?


(Looks like i was slow this time!)

This post has been edited by [JAZ]: Mar 12 2010, 22:41

[chrizoo]

yes, the last 3 posts made it clear for me now. thank you!

as a follow-up question: lossyWav uses a blocksize of 512. Can this be changed? I couldn't see an appropriate setting in http://wiki.hydrogenaudio.org/index.php?ti...cation_settings

And for a codec to benefit from lossyWav, is it enough that it can be set to a blocksize of 512 manually, or does it ALSO have a built-in function similar to FLAC's "wasted bits" feature (i.e. question is if the codec must meet both requirements) ?

[Nick.C]

The blocksize is automatic and equates to approximately 11.6msec for 44.1kHz material and 10.6msec for 48kHz material. This is due to the default 1.5msec and 20msec (approximate) FFT window lengths used for the analysis.

For lossless encoding, the blocksize can be set to anything that you want, however for optimum encoding it should be set to 512 for 44.1/48kHz material, 1024 for 88.2/96kHz material and 2048 for 176.4/192kHz material.

This post has been edited by Nick.C: Mar 12 2010, 23:14

[lvqcl]

And for a codec to benefit from lossyWav, is it enough that it can be set to a blocksize of 512 manually, or does it ALSO have a built-in function similar to FLAC's "wasted bits" feature (i.e. question is if the codec must meet both requirements) ?

WMA Lossless doesn't have 'blocksize' option yet it can benefit from lossyWAV.
On the other side, CUETools.ALACEnc have blocksize option, but cannot effectively compress files after lossyWAV.

So, only the latter feature is absolutely necessary.

[chrizoo]

WMA Lossless doesn't have 'blocksize' option yet it can benefit from lossyWAV.On the other side, CUETools.ALACEnc have blocksize option, but cannot effectively compress files after lossyWAV.

So, only the latter feature is absolutely necessary.

I see, the latter is a must but both would be desirable.

What about .ape ? The wiki says it's no supported.
Is there anything I can do about this (parameters, etc.) ?
Or is anything planned for a future version?


I fail to see why a series of zeros would NOT be duly compressed by ANY lossless codec. A simple run-length encoding would do this and isn't that part of any lossless codec? [confused]

[chrizoo]

The blocksize is automatic and equates to approximately 11.6msec for 44.1kHz material and 10.6msec for 48kHz material. This is due to the default 1.5msec and 20msec (approximate) FFT window lengths used for the analysis.

For lossless encoding, the blocksize can be set to anything that you want, however for optimum encoding it should be set to 512 for 44.1/48kHz material, 1024 for 88.2/96kHz material and 2048 for 176.4/192kHz material.

thank you!
is it correct to speak of a "virtual blocksize" in this case? because I didn't know that .wav files have something like a "blocksize" at all? I mean, it's not the same thing as "blocksize" for mp3 for example, right? I know, the question is very badly phrased, but can you understand it, or should I try to rephrase ?

[lvqcl]

Is there anything I can do about this (parameters, etc.) ?

No.

Or is anything planned for a future version?

According to Changelog, the last time encoder engine was improved was in ver. 3.99 (April 29, 2004): "Improved entropy coder for increased compression."

[chrizoo]

I see. thanks, lvqcl.

This post has been edited by chrizoo: Mar 12 2010, 23:51

[halb27]

As Nick wrote for deciding on how many least significant bits can be rounded to zero lossyWAV has to do FFT analyses. These require a certain block which is 512 samples long for the usual 44.1 or 48 kHz sampled source material. The calculated number of bits that can be removed are removed for all the samples of the block, so in order for lossyWAV to be efficient the blocksize must be that small.
So lossyWAV blocksize at first is just an internal detail of the lossyWAV mechanism. But in its consequence the number of bits removed in the WAV data share the same block structure. For best efficiency of the overall process the block size of the lossless codec should be 512 samples as well. If another blocksize is used the lossless codec can only take advantage of the smallest number of bits removed among all the lossyWAV blocks that contribute to the current lossless block: waste!

In practice your choice is TAK, FLAC, or wvPack, and wvPack isn't very attractive as it desn't work well with small blocks.

This post has been edited by halb27: Mar 12 2010, 23:59

[chrizoo]

that was an excellent explanation to me.

So lossyWAV blocksize at first is just an internal detail of the lossyWAV mechanism.

right. that's why I tried to coin the term "virtual blocksize" because it is a variable used internally during the processing (with all the subsequent implications you described), but once you see the final .wav output, there are no "boundaries" ... the blocks are "invisible" or "virtual" ... hence my use of words.

In practice your choice is TAK, FLAC, or wvPack, and wvPack isn't very attractive as it desn't work well with small blocks.

I like your "in practice" approach. thanks.

This post has been edited by chrizoo: Mar 13 2010, 00:14

[chrizoo]

For lossless encoding, the blocksize can be set to anything that you want,

how ? I used the --longhelp switch, but I couldn't find a parameter for setting the blocksize.

[halb27]

Nick was talking about the blocksize setting of the lossless codec, from lossyWAV's point of view (that is neglecting the possibilities for such a thing with the specific lossless encoder used).