Efficient LossyWAV correction file encoding?, Split from Topic ID: 65499 Options

[monoton]

Well, I did inspect the correction file with Audacity... I ran my test with the ACDC "Smash N Grab" file, and boosted the correction file for easier viewing only by 24db, you can see that in the attachment. When looking at it, I noticed the big jumps from sample to sample and thought it could be a reason for Flac not being able to predict as good as desired.

Anyway... then I thought I could prevent some of this by simply setting every other value to its negative counterpart. So -32768 is going to be +32767, -100 will become +99 and so on. Wrote a little app to just do that. The upper (stereo) wave in the picture is the original lwcdf and the lower (stereo) wave the result after processing every other sample to its negative value (multiplied by -1 and subtracted 1).

From the looks I expected at least some better compression. But the results were just on par, with my idea being even slightly worse.

Original wav-file: 41.4MB
Original flac-file: 34.1MB (Flac quality settings "6" for all)
lossy Flac: 14.0MB (quality -4 "awful" to get maximum effect)
lwcdf Flac: 22.4MB (upper wave in picture, boosted only for better viewing, not for processing)
negated 2nd Flac: 22.5MB (lower wave in picture, boosted only for better viewing, not for processing)




Actually I'm a bit puzzled about the result, that the predictors of Flac can take care about this that efficiently. Gotta go and deep digger then I guess

Any comments welcome, and I'd be happy to open another thread about this if you think it may hijack your original thread intention, Nick!

This post has been edited by monoton: Mar 20 2009, 03:11

[2Bdecided]

The second waveform looks like it should be easier to code than the first, but I think (note: lossless coding isn't my thing!) that just multiplying every second sample by -1 is a very primitive form of predictor - so this could easily be accounted for by FLAC already - hence you doing it manually made very little difference.

The correction files are modulated white virtually random noise (slightly less random where noise shaping is used). The only thing FLAC can do is efficiently throw away the leading zeros in each block - what's left isn't very compressible.

Cheers,
David.

[halb27]

'what's left isn't very compressible'
My thought as well. For such small random-like signals I guess FLAC can't make good use of the predictor at all. I suppose FLAC's work consists mainly of exact residual coding.

[Nick.C]

I'll try a test where the correction file is converted to mid/side and see what (if anything) happens to the resultant FLAC -5 bitrate. I would hazard a guess that it will simply average out to be the same overall with individual files being higher or lower in mid/side than they were in simple stereo.

[monoton]

Thanks a bunch to greynol for handling this topic split so fast!

Actually you'd think both materials are about the same... yet with everything inverted I simply ruled out what looked like a not-so-high-frequent AM modulation (that's erm... 22,050Hz to be exact) - I find it funny that FLAC could handle the other better.

But yeah, white noise is the hardest... actually noise as it is not really predictable...

This post has been edited by monoton: Mar 20 2009, 20:04

[Nick.C]

Yup - he's one of those kind mods I was referring to....

This has got me thinking. The difference signal is a signed integer which varies between -2^(bits_to_remove-1) and +2^(bits_to_remove-1)-1 and should happily exist in a bits_to_remove bit envelope. All of these could be joined together with pure silence at the end of the correction buffer. The downside is that this would produce an amount of white noise.

I'll get the release of v1.2.0 out of the way and then set to work looking at the correction file.

This post has been edited by Nick.C: Mar 20 2009, 20:14

[monoton]

I'll try a test where the correction file is converted to mid/side and see what (if anything) happens to the resultant FLAC -5 bitrate. I would hazard a guess that it will simply average out to be the same overall with individual files being higher or lower in mid/side than they were in simple stereo.

I doubt that will have lots of effect or be even quite a bit worse as I think the channel intercorrelation is quite low: major parts of the left or right channel are almost silent whereas the other side contains lots of information. Actually it may be this many on and offs that cause FLAC some problems. But I'm not really familiar with their predictors yet.