I read an interesting post over at the doom9.org forum, concerning a program which "optimizes the huffman tables" in Vorbis files, losslessly adding about 3% compression. Very cool.

As far as I know, Huffman encoding is used in other file formats besides Vorbis, isn't it? Would it be possible to develop a program to optimize MP3 files as well? Or other formats?

I am planning to write the recomp3. www.sf.net/projects/recomp3
It will contain such feature.

Be sure this is only an armchair plan.

CBR: No
VBR: Yes. For most encoded files I expect bit savings between 0 and 0.1%.

Does the existance of that tool translate into "Vorbis' huffman coding could be better"?

This is great!

Just tried it on an album:
Before: 96 203 208 bytes
After: 94 342 426 bytes

Savings: 1 860 782 bytes (1.77 MB)
Average bitrate dropped 3.6 kbps

This means same quality for lesser diskspace!

Hope this will be implemented in oggenc/oggdrop, maybe with --twopass option as in BeSweet?

yeah, would be really fine. John, could you perhaps add it to OggdropXPd ? Oggenc.exe should be modified too!

Depends on your point of view. It could be better (and is, with rehuff) when you allow a second processing pass, but two-pass encoding throws the possibilty for on-the-fly encoding and streaming out the window, one thing that Vorbis was specifically designed to be able to do. In the grand scheme of things, I would consider streamability as more important than 2% space savings.

Just my 2 cents,

Carsten.

But for people who use Vorbis to compress CD's the rehuff option would be nice. The savings couldn't be there when you had to do a stream of the music (but, how often do people stream -q 6 over the Internet?), but they would be nice to have on a portable player.

Not so great:

When playing the rehuff'ed oggs with winamp, seeking was impossible. When jumping to a position, there was just silence for a while, then it stopped playing. Sometimes the volume was very very low.

However, by decoding and comparing .wavs with the un-huffed ogg it was bit-identical.

So I guess this rehuff messes up seek-points?

how about AAC ?

im recompressing all my .AAC files with winrar to save around 3-6%

then using XOR winamp plugin to automatic decompress and playback the .RAC file (Rar compressed AaC)

---Edit ---

Does anybody know if they have fixed rehuuf so that you can muxe the enhanced vorbis stream into an .ogm file ?

Could you tell us a little more on this project please. The summary on Sourceforge does not explain a lot:

REMP3 can compress/change the playback volume/cut some part of mp3 file. Of cource output is MP3 file too. If you have a portable MP3 player, this tool would be very useful to store more MP3s into limited space from huge/high qulity archive.

Do you mean some kind of 'bitrate peeling' for mp3, or such thing as lowpass filtering an mp3 without reecoding. This kind of utilities would be very useful if technically possible.

Segher Boessenskool did the rehuff optimizations for Vorbis and like Frank said it increases compression by about 0.1 %. The rehuff source was attached to one of the developers mailing list archived posts.




This is essentially along the lines of how bitstream peeling would work as well. Make m passes through the residue partitions in VQ/Lattice Foam Codebooks I believe and you can you shave off a decent percentage of the bitstream, nothing significant though. Works that way with the Residue types at least. Maybe not with the floor backends as they are scalar Huffman coded.

Yes, it will peel the bitrate. Input is CBR/VBR mp3 file, and output will be VBR.

recomp3 will peel the bitrate with two strategy. one is lossless and another is lossy.

lossless method inclueds: Optimize Huffman coding, scalefactor storing, bit-reservoir use.
lossy method includes: small quantized spectrum trancation, intensity stereo, lowpass filter.

Note: This is only plan. There's no lines have written for this project. and lossless method will be not able to compress the mp3 file which encoded by the latest LAME because it already optimized.