New FLAC encoder Options

[Nayru]

A new FLAC encoder has been developed, which is proposed for inclusion in ffmpeg. The author claims that in high-compression mode, it can compress certain files better than libFLAC.

http://mplayerhq.hu/pipermail/ffmpeg-devel...May/011385.html

[jcoalson]

it seems like a nice clean interface. some notes:

1. at the higher compression settings it is using large block sizes (16k for 44.1kHz) and high lpc order (32) which increase the decoding burden with little improvement (at least in my tests). (e.g. I'm not sure the squeezebox supports 16k blocks due to buffer requirements.) also, performance with larger blocksizes in more dependent on the audio characteristics so his results might be biased by his sample.

2. it has an interesting way of selecting the stereo decorrelation method which is faster than the reference encoder; whether it's better on not needs some testing. same for fixed/lpc subframe selection.

3. it's guessing at the max frame size without supporting verbatim encoding, which might explain it crashing

4. the lpc analysis stage looks just like the reference encoder.

so I think any compression gain is just due to the larger blocksize and high lpc order, which you can do with the reference encoder (though I wouldn't encourage that for general use).

Josh

[Justin Ruggles]

Hi,
I am the person writing the encoder mentioned above. It is in an extremely experimental phase at the moment. I am fairly new to audio encoding, so most of what I am doing is just learning by trying things out and reading as much as I can. I do enjoy it though. I want to say that I greatly admire what Josh has done with the flac format and reference software. Its openness and good documentation are a rarity and a pleasure. I am writing an encoder for FFmpeg because one of the goals of that project is to write its own decoders and encoders so as not to rely on too many external libraries, and because of licensing (which I don't think is really much of an issue in the case of FLAC). Also, I am doing it for the shear pleasure of learning something new. My claims of better encoding are more due to excitement at the results than anything else. Beating the reference encoder is not my goal. It's just a good motivator, both for the excitement when I succeed and for the challenge when I can't seem to get anywhere close.

it seems like a nice clean interface. some notes:

1. at the higher compression settings it is using large block sizes (16k for 44.1kHz) and high lpc order (32) which increase the decoding burden with little improvement (at least in my tests). (e.g. I'm not sure the squeezebox supports 16k blocks due to buffer requirements.) also, performance with larger blocksizes in more dependent on the audio characteristics so his results might be biased by his sample.

That is true. I have found some odd results in my tests though. Many times I have seen my encoder do much better on a particular clip with, say, max lpc order of 32, but do not see the same benefit on the same clip with the reference encoder. I couldn't say why or how though. I am mostly just trying to test anything I can and observing what settings work best in which situations. You do have a point about buffer requirements. I had to increase the SDL audio buffer size in ffplay to get the high blocksize clips to play without skipping.

The issue with block sizes and different audio characteristics is one I am still struggling with. It seems to be one of the biggest drawbacks to really good compression quality. Variable blocksize would be ideal, but I know the compatiblity issues. Given the limitations, one of my objectives is to have good compatible modes, but also to provide more options for better compression. The reference encoder seems to go for the middle ground in many cases. This has its benefits, but personally, I like more options.

2. it has an interesting way of selecting the stereo decorrelation method which is faster than the reference encoder; whether it's better on not needs some testing. same for fixed/lpc subframe selection.

Yeah, the stereo decorrelation selection was just a guess that turned out to work okay. I haven't looked closely enough to see how the reference encoder does it. As for fixed/lpc subframe selection...I can't really remember how I was doing that on May 26 . I have had too many abandoned approaches to keep up... Right now I am using fixed mode (and mostly just order 2) only in the fastest encoding setting and for small frames (usually just the last frame).

3. it's guessing at the max frame size without supporting verbatim encoding, which might explain it crashing

I have no idea about the crashing on windows, as I use Linux. As for verbatim encoding, I actually started the program as verbatim only just to get the bitstream right. I later removed it. I probably should add it back in as a backup mechanism for audio that really just won't compress...a rarity, but it would save the guessing you mentioned. For now, I have at least added a warning & skip frames that are too big. It's not ideal, nor lossless, but at least it gives a warning...

4. the lpc analysis stage looks just like the reference encoder.

Yeah, it pretty much is. I have modified the autocorrelation slightly and implemented my own quantizer. The levinson-durbin function is the same. I experimented some with other ways of estimating the best predictor order, but it was not as good as the reference encoder. When I coupled the 2 together and selected the best by counting the bits I got slightly better encoding...that might have just been chance though I could have probably selected an order at random and done the same thing. I guess I'll have to keep researching. It's not that I think the reference encoder has a bad solution...I just like a challenge. I recently added, and am currently experimenting with, window functions before autocorrelation. Yesterday I was pointed to a Hydrogenaudio thread on how this is being added to the reference encoder as well. Very exciting stuff!

so I think any compression gain is just due to the larger blocksize and high lpc order, which you can do with the reference encoder (though I wouldn't encourage that for general use).

Blocksize I understand, but what's wrong with a higher LPC order? True, it isn't ideal for every situation, but it does help in many cases. I haven't noticed much of an impact on decoding, and I have a quite slow computer.

Anyway, I just thought I would respond. I am new to Hydrogenaudio, but since I seem to have thrown myself into the fray I might as well join in on the discussion.

-Justin