Why no Vorbis 2-pass? Options

[Phaedras]

I know this question has been brought up before, most specifically on the Vorbis mailing list, but I still don't quite understand why 2-pass encoding isn't supported by Oggenc. Sure, the savings are minimal, but isn't that what lossy coding is all about? To make every bit count?

[Benjamin Lebsanf...]

maybe in the future rehuff could be integrated, maybe john can create a special version

[john33]

On a single example, rehuff compresses a 6016Kb file to 5925Kb. The saving is about 3kbps on this file encoded at -q 6.25. It would not be difficult to integrate this as an option into oggenc2.1 (my hybrid with lossless decoding support), but is there really a demand for it?

[Benjamin Lebsanf...]

it would be a nice feature to play with and if its not too difficult go ahead, we would love it

Because of the sub-optimal one pass method, it is possible to gain some space by recompressing an Ogg Vorbis file, but space savings as dramatic as 10% could not be reproduced with the current version of the encoder. (For example, a quick test by myself resulted in a pitiful 0.2% gain on a randomly chosen file from my collection.)

Segher Boessenkool is currently developing a tool that optimizes the Huffman codes for a given Ogg Vorbis file in order to shave off a few percent of the file size. He reports that the resulting space savings are normally under 5%, and typically between 1% and 2%.

[robUx4]

In video, double pass is not used for saving space, but to maximize quality for a given size/bitrate. I think it should be the same for audio.

[Tripwire]

I don't know if this is doable, but a 2nd pass could scan the compressed file for heavy compression damages and reencode the block with a higher bitrate. And do the same on highbitrate blocks, recompressing them on a lower bitrate.

[Slo Mo Snail]

How do you want to detect artefacts? There's no way to teach a computer, how an artefact "looks" like

[amp]

I don't know if this is doable, but a 2nd pass could scan the compressed file for heavy compression damages and reencode the block with a higher bitrate. And do the same on highbitrate blocks, recompressing them on a lower bitrate.

No, I think the problem is more or less like this: The 2nd pass is really worth only with ABR-like bitrate managment. With video codecs like DivX or Xvid most of the times you're targeting for a certain bitrate, much like LAME ABR. So the problem is: if the codec, as it only 'knows' the complexity of the signal progressively, wastes too much bits with the first sections of the movie or the audio, there'll less space for the later part. And, if the codec is conservative at the first part 'waiting' for more complex signals, maybe at the end it'll never get that complex signals so, again, bits are wasted.

So, at the first pass the encoder only analyzes the complexity of the signal (therefore the first pass is much faster than the second) for a correct 'bit distribution' at the second pass.

I think the method discused here for Vorbis is other, that is in relation with Huffman compression (lossless).

Excuse me if this explanation is poor technically speaking, or incorrect. Maybe someone can explain it better than me .

This post has been edited by amp: Dec 31 2002, 16:43

[john33]

@amp: Your explanation is perfectly understandable to me.

The other comment about 2 pass and quality, since most people concerned with quality use the quality settings, and these use VBR, the encoder grabs whatever bits it needs to achieve the desired quality level in the first place.

[tangent]

In video, double pass is not used for saving space, but to maximize quality for a given size/bitrate. I think it should be the same for audio.

Actually in video, the main purpose of 2-pass is to obtain a target output filesize after encoding while keeping quality consistent through the entire video. The one-pass method of obtaining a target output filesize (CBR) gives too many bits to the easy scenes and too few bits to the hard scenes therefore quality noticably suffers in the hard scenes. 2-pass allows target filesize to be reached while maintaining a consistent quality.

In video, a target output filesize is often desired because people often rip DVDs to CDRs and target the maximum capacity of 1 or 2 CDRs. In audio, 2-pass is not really in demand because it would be very rare that an audio ripper would require a target filesize for the encoding.

Ogg Vorbis constant quality single pass will already give you the best quality the codec can give you for its filesize (not counting the use of rehuff), it is just that the output filesize is unpredictable. Video encoding can also be done with a constant quality by encoding with a constant quantizer (option available for xvid and divx4/5), as usual, filesize is unpredictable, but this is the recommended method where you do not need a target filesize.

The other factor why you don't see 2-pass audio encoding is that it's harder to implement than for video. In video, there is only 1 variable (quantizer) which is varied between frames to affect the size of the frame towards a target bitrate. For audio, there are a lot more variables which the encoder will have to consider.


Audio encoding 2-pass has been asked many times before, it really should go into a FAQ. Pio2001, you there?

[Mgz]

WMAv9 support 2-pass encoding,too

/me

[Mac]

Going back to john33, saving 3kbs is exactly what it's about You could do it like that, just shaving the file-size down.. you still have the quality defined by q6.25, but in a smaller file size.

If it was the size you were after, you can just encode slightly higher at q6.4, and get 3kbs more of quality added for the file you were expecting

[dev0]

Besides from the various problems reported about rehuff at the Doom9 forums, I'd definetly like tosee this function in oggenc2.1.
But don't call it 2 Pass! As explained above the meaning of 2pass is different from what rehuff does, so adding an option --rehuff might be just fine.
dev0

[john33]

For those who are interested, oggenc2.2 is now available from Mirror 1.

Additional option is '--rehuff'. It processes quite quickly with a fair amount of info displayed to the console. The '-Q' (quiet) option from the command line will also kill the console output from the rehuff pass.

Have fun!!

[kheops]

it's a big improvement, i hope we'll see this 2 pass mode in oggmachine soon (for now i only use ogg to convert from ac3 files)

thanks

[sven_Bent]

does the ogg vidoe container support the rehuff savings ?

if i use ogmmux to mux avi videi and some vorbis file with rehuff

is the audio still done with the rehuff savings ?

[john33]

does the ogg vidoe container support the rehuff savings ?

if i use ogmmux to mux avi videi and some vorbis file with rehuff

is the audio still done with the rehuff savings ?

So far as I'm aware, the content and appearance of the rehuffed file to 'ogg aware' software should be no different than for any other ogg file. All the headers are correct, it simply has more efficient compression. I know that this is not an unequivocal answer, but I don't believe it should behave in any different way to any other ogg file.

[schlauf]

OK, now I post it here.

I just tried the --rehuff option while ripping and encoding a whole audio CD - nothing special about that.
For ripping I use Exact Audio Copy in conjunction with OggEnc2, command line reads as follows:

-q 6 %s %d -t "%t" -a "%a" -l "%g" -d "%y" -N "%n" -G "%m" --rehuff

This has worked well so far except the --rehuff option.

Now, having tried to encode this CD, I have indeed received files which were smaller than they used to when using OggEnc(1), but unfortunately, seeking and jumping in files has been corrupted.

For playback I use Winamp 2.81 and in_vorbis.dll 1.2.9 latest build.

When trying to seek inside the file to any position after half of the whole file right after starting, playback instantly stops.

Did anybody else track this problem down? It would be interesting if it is an encoder or decoder problem ... and, strangely, it didn't occur on ALL files, just on about 20% ...

Bye
MS

EDIT: OK, now I've found out at the Doom9 Forum that this problem has been encountered in previous releases too ... unfortunately there seems to be no solution, right?

http://forum.doom9.org/showthread.php?thre...ighlight=rehuff

This post has been edited by species007: Jan 1 2003, 16:35

[S_O]

Now, having tried to encode this CD, I have indeed received files which were smaller than they used to when using OggEnc(1), but unfortunately, seeking and jumping in files has been corrupted.

Yes, I noticed the same problem with all players.

EDIT: OK, now I've found out at the Doom9 Forum that this problem has been encountered in previous releases too ... unfortunately there seems to be no solution, right?

I found a solution: remux the file (not reencode!) with Tobias DirectShow-filters, build a graph in GraghEdit: infile.ogg -> Ogg Splitter -> Ogg Multiplexer -> outfile.ogg
The file has nearly the same size (27 Byte bigger actually) but seeking works perfect again.

does the ogg vidoe container support the rehuff savings ? if i use ogmmux to mux avi videi and some vorbis file with rehuff is the audio still done with the rehuff savings ?

Yes. My test:
without rehuff: 1.458.758 Byte
with rehuff: 1.409.311 Byte (96,610% from the normal file) seeking is broken!
with rehuff and remuxed: 1.409.338 Byte (96,612% from the normal file) file perfectly seekable!

So this problem is not unfixable and not a general result of rehuffing.

[john33]

Hmmm, I can't comment at this time other than to confirm the same behaviour. The files play fine from start to finish, it's the seeking that screws up.

[schlauf]

Yeah, that GraphEdit-Thingo works indeed. Unfortunately, this procedure is way too complex to apply it to a whole bunch of .OGG files, so I do hope this issue can be handled very soon ....

Thanks John, we appreciate your effort!

[john33]

BTW, I was just trawling through the vorbis-dev mailing lists and came across a comment by Segher Boessenkool, the author of rehuff, referring to a bug in Libogg as being the cause of the problems. The upcoming patch release relates primarily to the ogg side of things, ogg file I believe, so maybe this will enable a proper release of rehuff in the not too distant future.

[rc55]

John,

It seriously makes me bang my head against the wall - you are so bloody helpful!
It's very much a case of "It'd be nice if vorbis did...." and before I've finished the
sentence you've gone and done it. Superb.

I'll have to come to Berkshire and buy lots of beer for you.

Ruairi.

[LCtheDJ]

Yesterday I downloaded oggenc2.2 and tested it using the --rehuff switch; it seems that skipping and searching within a track is working correctly now.

My filesize savings due to using --rehuff came to about 1 - 2 percent.

Are there plans to make available an ICL7 compile optimised for Pentium 4?

My thanks to john33!

[john33]

Yesterday I downloaded oggenc2.2 and tested it using the --rehuff switch; it seems that skipping and searching within a track is working correctly now.

My filesize savings due to using --rehuff came to about 1 - 2 percent.

Are there plans to make available an ICL7 compile optimised for Pentium 4?

My thanks to john33!

No sooner requested, than done! You'll find it at Mirror 1.