Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: lame3100k - bringing constraint VBR to Lame (Read 53493 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

lame3100k - bringing constraint VBR to Lame

You can download it from here. For some background see this and the subsequent posts.

What’s lame3100k?

It's an extension to Lame 3.100 alpha2 which offers an additional constraint vbr mode option --cvbr x, x from 1 to 10, fractional values allowed, to be used additionally to standard VBR option -Vn.

constrained vbr mode means bitrate is controlled not to go too low.
It comes at the cost of an increased average bitrate. This extra cost can be kept very low (down to 0%), while offering remarkable quality improvement for special issues.
For an easy personal listening experience compare the results of -V3 and -V3 --cvbr 2 (+2% avg. bitrate for a collection of various pop music) on the 3.0 sec. issue of problem sample eig. Any other -Vn --cvbr x combination shows the quality improvement as well.
For the first two seconds of sample lead-voice the improvement of -V3 --cvbr 2 over plain -V3 is also easily audible.
On sample harp40_1 -V3 --cvbr 3 is better than plain -V3. The difference is not so very obvious as with the other samples mentioned.

--cvbr 1 ... --cvbr 4
Situations with a potential temporal resolution (or related) issue are taken good care of.
Care is taken also of other issues, but to a minor extent because focus here is on keeping bitrate increase low.
--cvbr 1 is useful for -V5 (+3% avg. bitrate).
--cvbr 2 is useful for -V4 to -V2 (+3% / +2% / +0% avg. bitrate for -V4 / -V3 / -V2).
--cvbr 3 is useful for -V3 to -V1 (+5% / +2% / +0% avg. bitrate for -V3 / -V2 / -V1).
--cvbr 4 is useful for -V3 to -V2 (+11% / +4% avg. bitrate)

--cvbr 5 ... --cvbr 8
Any situation with a potential issue is taken care of.
--cvbr 5 is useful for -V3 to -V1 (+17% / +7% / +1% avg. bitrate for -V3 / -V2 / -V1).
--cvbr 6 is useful for -V2 to -V1 (+15% / +3% avg. bitrate for -V2 / -V1).
--cvbr 7 is useful for -V1 to -V0 (+10% / +0% avg. bitrate for -V1 / -V0).
--cvbr 8 is useful for -V0 (+3% avg. bitrate).

above --cvbr 8
This is kind of an 'insane mode' for the people who are extremely fearful of audible issues.
--V0 -cvbr 9 yields an average bitrate of 289 kbps.
--V0 -cvbr 10: average bitrate is 317 kbps.

--cvbr auto
The functional extension choses an appropriate --cvbr x value for the user's -Vn value. For -V5 and -V4 more or less only temporal resolution issues are taken care of, whereas from -V3 to -V0 more and more good care is taken of any situation with a potential issue.
Average bitrate for a collection of various pop music is for -V5: +3%, -V4: +3%, -V3: +5%, -V2: +6%, -V1: +3%, -V0: +1%.

--bCVBR x, 134 <= x  <= 317
x is the desired average bitrate for a collection of various pop music.
Instead of using -Vn --cvbr auto and trying different n's to arrive at the desired bitrate, this option can be used.


Installation

lame3100k.exe was compiled with Visual C++ 2010. For this reason it is necessary to install the Microsoft Visual C++ 2010 Redistributable Package vcredist_x86.exe. You can download it from http://www.microsoft.com/en-us/download/details.aspx?id=8328.

lame3100k.exe uses the fast and lossless mp3packer tool internally to squeeze the otherwise unused bits out of the mp3 file. You can download mp3packer from http://www.hydrogenaudio.org/forums/index....st&p=282289.
Put mp3packer.exe into the same folder where lame3100k.exe is located. Many thanks to Omion for this great tool.
In case there is no mp3packer.exe in lame3100k.exe’s folder lame3100k.exe will work, but the mp3 files will be somewhat larger than necessary.
lame3995o -Q1.7 --lowpass 17

lame3100k - bringing constraint VBR to Lame

Reply #1
Many thanks for your work!

I tested the lame3100k, and made a graph of a real bitrate of albums.(5min snippet of Pops and Jazz).


I'm currently listening to the output mp3s. Many settings were used but no obvious flaws so far.
I think --cvbr number should be redefined to specify the lowest bitrate in kbps, as final version of this encoder should be something easier to understand.
Ideally, rate–distortion optimization should be used rather than to set both the typical and lowest bitrate and complicate the situation.

lame3100k - bringing constraint VBR to Lame

Reply #2
Thanks a bunch halb27.  It took me a little bit to figure out what was going on with the various settings in 3100k as they are significantly more complicated (at first glance)  than those of your earlier variants.  As of now, I've figured out all 3 "modes", and will enjoy listening to various conversions over the coming weeks.  I won't be doing any abx testing unless something really jumps out at me (doubtful). 

Best regards, and thanks again.

lame3100k - bringing constraint VBR to Lame

Reply #3
... I think --cvbr number should be redefined to specify the lowest bitrate in kbps, as final version of this encoder should be something easier to understand.
Ideally, rate–distortion optimization should be used rather than to set both the typical and lowest bitrate and complicate the situation.

The difference between the earlier -Vx+ user interface (together with a bunch of additional options not very easy to understand) and the --cvbr user interface is that you can vary the constrained vbr (cvbr) intensity now in an easy way. With the earlier -Vx+ interface the cvbr intensity is fixed, an in order to vary it you have to go into the complexity of the --adbr_xxx paramters.
I've tried to give some hints for the usage of --cvbr depending on -Vn value.
--cvbr x corresponds to a combination of the minimum audio data bitrate of normal, short, start and stop blocks and more. This combination can't be put into one parameter while addressing each of them. I think that these particular parameters (which were available in previous versions) were hardly used by somebody, so I think it's better to make a predefined combination available to the user and give the user an easy accees to various combinations which correspond to --cvbr value.

What does 'rate–distortion optimization' mean?

Thank you for your testing. I've always considered only -V levels downto -V5, but your graph shows that --cvbr auto or --cvbr 1 maybe be useful downto -V6 or even -V8.
lame3995o -Q1.7 --lowpass 17

lame3100k - bringing constraint VBR to Lame

Reply #4
Thanks for your work, as always, halb27.  I have two questions, a couple of comments, and a request:

(Question 1) My testing indicates that -V0 and -V0+ are now identical.  Is that correct?
(Question 2) It also appears that -q0 is not needed with -V0.  Is that correct?

(Comment 1) I would suggest reversing the --cvbr scale, and putting it on a 0 (best) to 9 (worst) scale instead.  This would line it up with standard LAME functions such as -q and -V.
(Comment 2) I would suggest adding the setting --cvbr -1 to disable the new cvbr function, and making --cvbr auto standard (i.e. it is implemented if no --cvbr option is passed).  I'm not sure why anyone would use your extension and then disable cvbr, but it might be useful for testing.

(Request) I always use the --replaygain-accurate setting to assist with volume normalization.  However, I would rather calculate for the newer EBU R128 standard.  Based on your understanding of ReplayGain and EBU R128, would it be difficult to add this into your extension?

lame3100k - bringing constraint VBR to Lame

Reply #5
Thanks for your work, as always, halb27.  I have two questions, a couple of comments, and a request:

(Question 1) My testing indicates that -V0 and -V0+ are now identical.  Is that correct?
(Question 2) It also appears that -q0 is not needed with -V0.  Is that correct?

(Comment 1) I would suggest reversing the --cvbr scale, and putting it on a 0 (best) to 9 (worst) scale instead.  This would line it up with standard LAME functions such as -q and -V.
(Comment 2) I would suggest adding the setting --cvbr -1 to disable the new cvbr function, and making --cvbr auto standard (i.e. it is implemented if no --cvbr option is passed).  I'm not sure why anyone would use your extension and then disable cvbr, but it might be useful for testing.

(Request) I always use the --replaygain-accurate setting to assist with volume normalization.  However, I would rather calculate for the newer EBU R128 standard.  Based on your understanding of ReplayGain and EBU R128, would it be difficult to add this into your extension?

Question1: There isn't any -Vn+ any more. But lame3100j -V0+ is identical with lame3100k -V0 --cbvr 10.
Question2: I've never cared about -q x. It's used here like in standard Lame, and AFAIK doesn't make sense to be used other than with the default.
Comment1: Yes, your suggestion would be in line with -V and -q scale philosophy.
Comment2: Sure 3100k is only interesting if --cvbr (or --bCVBR) is used. So: yes, it makes sense to default to --cvbr auto.
Request: Sorry, replaygain stuff is beyond my scope. Can't you use foobar to do what you want to do?

As a result: question to any (potential) lame3100k user:

a) should I default -Vx usage to -Vx --cvbr auto?
b) should I use a scale --cvbr 0...10 (0 yielding highest quality) instead of current 1...10 (10 yielding highest quality)?
lame3995o -Q1.7 --lowpass 17

lame3100k - bringing constraint VBR to Lame

Reply #6
As a result: question to any (potential) lame3100k user:

a) should I default -Vx usage to -Vx --cvbr auto?
b) should I use a scale --cvbr 0...10 (0 yielding highest quality) instead of current 1...10 (10 yielding highest quality)?

Thanks for the quick reply!  One small comment: I am proposing 0...9 (0 yielding highest quality), not 0...10.  0 to 9 would result in the same size scale as the current 10 to 1.

lame3100k - bringing constraint VBR to Lame

Reply #7
... One small comment: I am proposing 0...9 (0 yielding highest quality), not 0...10.  0 to 9 would result in the same size scale as the current 10 to 1.

OK, so the question is: --cvbr 0...9 (0 yielding highest quality) instead of current scale?
lame3995o -Q1.7 --lowpass 17

lame3100k - bringing constraint VBR to Lame

Reply #8
I agree with the cvbr scale 0 to 9 scale.  (0 being highest quality).

question for halb27- Is the cvbr 10 setting yield the same result as --bCVBR x, 134 <= x <= 317?

Also, would changing the Vn setting to default to cvbr auto mess up the current cvbr 10 to 1 settings?

Thank you.

lame3100k - bringing constraint VBR to Lame

Reply #9
--cvbr 10 is useful only for -V0 or at least a -V level close to -V0. -V0 --cvbr 10 is identical with --bCVBR 317.
Defaulting -Vn to current -Vn --cvbr auto would mean exactly this and wouldn't mess up with personal -Vn --cvbr x settings.
lame3995o -Q1.7 --lowpass 17

lame3100k - bringing constraint VBR to Lame

Reply #10
Hi,

sorry to say this, but as an ordinary user, your new settings are a bit confusing. I might suggest, that you rethink them again.
I did a quick table about your suggestions, showing only the possible combination. The value in the matrix is the increase of size.:

Code: [Select]
-V / -cvbr   1    2    3    4    5    6    7    8
 5        +3%
 4              +3%
 3              +2%  +5% +11% +17%
 2              +0%  +2%  +4%  +7% +15%
 1                  +0%      +1%  +3% +10%
 0                                            +3%

I don't think that users will get into this table that easy.

Instead of trying to figure out, which -cvbr value is suitable/possible for a given -V value, I suggest that you simplify the settings just on the base of the potential increase of the bitrate.
Lets say, you have an option -boost=0, 1, 2, 3, 4, 5
with
-boost=0: +0%
-boost=1: +1% - +2%
-boost=2: +3% - +4%
-boost=3: +5% - +10%
-boost=4: +11% - +15%
-boost=5: >+16%

Then you could translate the -boost settings into your -cvbr settings this way
Code: [Select]
-V / -boost  0   1   2   3   4   5
 5          1  1  1  1  1  1           
 4          2  2  2  2  2  2
 3          2  2  2  3  4  5
 2          2  3  4  5  5  6
 1          3  5  6  7  7  7
 0          8  8  8  8  8  8
In that case, a user would not have to think about, if the option is possible or not, he just would choose, how much percentage he would accept for a better quality. How this is internally managed, would be nothing he would need to care about.

Only my suggestion.

lame3100k - bringing constraint VBR to Lame

Reply #11
I'm sorry that the new options are confusing.
I think this is due to a) the new user interface as such b) my documentation which should have been clearer.

A -boost option would focus on average bitrate increase.
The purpose of a user-selectable constraint vbr level however is to select minimum audio data bitrate. The higher the --cvbr value, the higher is the required minimum audio data bitrate. Average bitrate increase just comes as a side effect (which is not welcome and should not be in focus in the first place).

For an easy usage of --cvbr just use --cvbr auto.
For special needs like having average bitrate increase of -V0 --cvbr x below 1% you can use specific --cvbr values (-V0 --cvbr 7 in the example).
lame3995o -Q1.7 --lowpass 17

lame3100k - bringing constraint VBR to Lame

Reply #12
Question1: There isn't any -Vn+ any more. But lame3100j -V0+ is identical with lame3100k -V0 --cbvr 10.

3.100k accepts both -V0 and -V0+ (and -V1 / -V1+ , etc.) in the commandline.  If you intended this, then thank you as it makes compatibility easier.  But I thought I should mention it in case you did not


EDIT: As far as making --cvbr auto the default, as mentioned above I support this.  However, this would require the option to disable --cvbr if the user desires.

lame3100k - bringing constraint VBR to Lame

Reply #13
I tested the lame3100k again, and made a graph of a real bitrate of albums.(5min snippet of Pops and Jazz).



Ideally, rate–distortion optimization should be used rather than to set both the typical and lowest bitrate and complicate the situation.

What does 'rate–distortion optimization' mean?

In case quality increase is minimal per bitrate increase, do not increase the quality and save the bit. In case bitrate increase is minimal per quality increase, increase the quality, at the cost of very little more filesize. Wikipedia has the article of rate–distortion optimization.




lame3100k - bringing constraint VBR to Lame

Reply #14
3.100k accepts both -V0 and -V0+ (and -V1 / -V1+ , etc.) in the commandline. ...

Thanks for the info. As for the processing of the -V parameter 3100k behaves exactly like standard Lame. Any characters after the last digit are ignored. So -Vn+ = -Vn.
Sure due to the history of the functional extension this should be avoided and an error situation should be produced. I'll take care of this.
lame3995o -Q1.7 --lowpass 17

lame3100k - bringing constraint VBR to Lame

Reply #15
In case quality increase is minimal per bitrate increase, do not increase the quality and save the bit. In case bitrate increase is minimal per quality increase, increase the quality, at the cost of very little more filesize. Wikipedia has the article of rate–distortion optimization.

I see. Sure in an ideal world --cvbr auto should be according to rate-distortion optimization. Unfortunately we don't really know the rate-distortion optimum point. The situation is a bit similar to the choice of -Vn where it's pretty personal where to put the sweet spot. That's not exactly the situation though because I know a bit about --cvbr level dependent quality improvement and I have tried to share a bit of that in the documentation.
As for --cvbr auto: IMO -V5 and -V4 users are very aware of low bitrate, so I use a setting that yields only a small avg. bitrate increase. Quality improvement for samples like eig can be very audible though.
-V3 and more so -V2 users are much more quality aware and less sensitive to a somewhat higher avg. bitrate, that's why I allow for --cvbr 3 resp. nearly 5 here. This way probability for improvement of any issue is much higher, especially when using -V2 --cvbr auto. For -V1 and more so -V0 a pretty high --cvbr level 6 and higher can be used while keeping avg. bitrate increase small.
Individual needs may be different, that's why I gave an imagination for any --cvbr n which -Vx levels are most interesting to be used together with --cvbr n.
To me this makes sense, but I welcome any discussion about it.
lame3995o -Q1.7 --lowpass 17

lame3100k - bringing constraint VBR to Lame

Reply #16
I want to do the following:

a) -Vn will be constrained VBR by default according to what's now --cvbr auto
b) Usage of -Vn+ will lead to an error situation
c) --cvbr level will go from 0 to 9, 0 being highest cvbr level
d) --cvbr off will turn -Vn into standard VBR
e) --cvbr auto will not be available as an explicit option because it's default behavior
f) try to improve the documentation to make it easier to understand

I dislike having to create a version 100l so soon after having come out with 100k. For my own peace of mind I will try to do also some more internal optimizations, but they will be very, very minor (if they should happen at all) as I just did exactly this.

I want to start doing theses changes next Monday (June 10).
So please come up with any change request until then.
lame3995o -Q1.7 --lowpass 17

lame3100k - bringing constraint VBR to Lame

Reply #17
I want to do the following:
d) --cvbr off will turn -Vn into standard VBR


Sounds great!  Thanks, as always, halb27.
For what it's worth, I agree...there is no need to release a new version so soon after k for such minor changes.  It would make more sense to wait until you decide to make a release for other reasons.

One comment on point d) above.  The reason I suggest --cvbr -1 instead of --cvbr off is because -1 is the standard used by LAME to turn other options (such as lowpass and highpass) off.
Also, now that you have explained how native LAME works, I am not certain if point b) is needed after all.  -Vn+ will not cause an error condition in standard LAME, after all.

The only other request I can think of is one I plan to make for standard LAME: that a commandline option be added which treats frames that are near the minimum threshold for human hearing as silence.  Let me give an example.  Many CDs feature "hidden tracks" at the end of the album.  The primary track and the "hidden track" will be separated either by silence, or by near-silence white noise.  If the tracks are separated by silence, all silent frames will be encoded at the minimum size of 32kbps.  If they are at near-silence, however, they will be encoded at the user requested size--even though the user generally cannot hear them.  I would like an option to encode these at 32kbps as well.

lame3100k - bringing constraint VBR to Lame

Reply #18
OK, I'll use --cvbr -1 to switch constrained VBR behavior off.

As for your other request: What Lame treats as silence isn't exactly silence, but near-silence as you want it to be. Lame's silence is frame contents below the threshold of hearing. In such a situation my Lame variant behaves exactly like standard Lame.
Probably you want the near-silence threshold to be higher, but what should be the criterion for that, and how shall I ensure that such a 'rounding down to silence' wouldn't affect other spots in the music you want to be taken good care of?
Can't you do a manual track seperation using a wave editor before encoding, or mp3directcut after encoding? Or silencing the very low volume part down to silence using a wave editor or mp3directcut if you don't want seperate tracks?
lame3995o -Q1.7 --lowpass 17

lame3100k - bringing constraint VBR to Lame

Reply #19
Interesting!  I didn't realize that LAME already treated near-silence in this way.  You are correct then.  I simply want the threshold to be higher or to be customizable.

I do separate "hidden tracks" in WAV before encoding...so the "white noise" example that I gave actually would not be affected by the suggestion.  Instead, it would only affect a few frames in each track.  As such, I see this as a way to free up a few more bits for adjacent frames or to increase the compression slightly.  As for your question on how to avoid "rounding down to silence" in important sections...I do not have a good answer .

At any rate, the suggestion is probably better suited for the primary version of LAME, IF it is implemented anywhere.

lame3100k - bringing constraint VBR to Lame

Reply #20
As you do separate the tracks before encoding, I guess your problem affects only a second or so of your track. Shouldn't bring average bitrate really down. While separating: can't you just bring your very low-volume parts around the separation point down to real silence? Guess that's the most appropriate solution if you really care about this and shouldn't mean a lot of extra work.
I am afraid that any option in Lame to do that might have a negative effect on real contents.
lame3995o -Q1.7 --lowpass 17

lame3100k - bringing constraint VBR to Lame

Reply #21
I plotted some real-bitrate vs quality graphs of my latest listening test at 224kbps. Of course, it includes rate and quality distribution of halb27's lame3100j(Previous version).
I think the graphs helps developers to improve the behavior of the encoders.
http://www.hydrogenaudio.org/forums/index....mp;#entry836209

lame3100k - bringing constraint VBR to Lame

Reply #22
Thanks for your graphs. Unfortunately it doesn't help for further progress as it shows average bitrate. These results are more or less as expected though Helix's extreme averge bitrate variation compared to Lame's is very interesting.
You're very welcome to contribute if you like to. If it's not too much work for you could you compare your samples Trust (most important to me), Tom's Dinner, Experiencia, and Quizas @224 kbps for Lame3.99.5, Lame3.100.a2, lame3100i, lame3100k?
lame3995o -Q1.7 --lowpass 17

lame3100k - bringing constraint VBR to Lame

Reply #23
Thanks halb27. --cvbr auto seems very easy to use so I will test soon .

lame3100k - bringing constraint VBR to Lame

Reply #24
Thanks for your graphs. Unfortunately it doesn't help for further progress as it shows average bitrate.

You mean average bitrate of one sample, which is typically 20 seconds long? I calculated each sample's bitrate by filesize(Bytes)*8/audiolength(seconds).