lame 3.97 alpha 6 testing thread Options

[guruboolez]

I am wondering if there was a regression between 3.90.3 and 3.96.1 and no one spotted it, or if there is a regression between 3.96.1 and 3.97a6.

I could check it (quickly) but I fear that I can't post results before next monday.

[uart]

I could check it (quickly) but I fear that I can't post results before next monday.

Great . I'm very interested in the results as I'm currently using 3.96.1 exclusively.

[Gabriel]

The q1 and q0 problem has now been identified and corrected, thank you very much for the report.

[guruboolez]

APPENDIX TO TEST#4: --preset standard

I am wondering if there was a regression between 3.90.3 and 3.96.1 and no one spotted it, or if there is a regression between 3.96.1 and 3.97a6.

It appeared previously that lame 3.97a6 -V 2 quality was inferior to -V 2 --vbr new and also to lame 3.90.3 --preset standard. The purpose of the following test is to verify the encoder’s evolution, from 3.96.1 to 3.97a6, in order to see if the cause of the measured degradations occurred during this developing phase or if it happened before it.

I've used the same 20 samples again, but I didn't test the three ones which sounded identical during the last test (BachS1007.wav: too transparent, castanets2.wav: too smeared, thear1.wav: too deafening).


Results

CODE

ATrain          3,5      3,5
BachS1007        not tested      
BeautySlept     3,0      3,5
Blackwater      4,0      4,0
castanets2       not tested      
dogies          3,5      2,5
FloorEssence    4,0      3,0
fossiles        4,0      3,9
SinceAlways     3,0      4,0
Layla           3,0      2,5
LifeShatters    4,0      4,0
LisztBMinor     3,0      4,0
macabre         2,0      3,0
MidnightVoyage  3,0      3,5
Orion II (2.1)  3,5      3,5
rawhide         3,0      4,0
thear1           not tested      
TheSource       3,0      2,5
Waiting         2,0      2,0
wayitis         3,0      2,0

Click here for log files


Results are unclear. Changes between 3.96.1 and 3.97a6 are sometimes audible, but they are equally distributed between both encoders. 7 samples sounded identical; at least no difference appeared during ABX phase. 5 were better with 3.96.1 (confirmed by positive ABX results) and 5 were better with 3.97a6 (also confirmed).

It seems that differences heard previously between -V2 and -V2 --vbr-new aren't correlated to a bug introduced in 3.97 alphas. I would therefore conclude that 3.90.3 --preset standard is better than 3.96.1 3.97a6 at same preset and finally that 3.97a6 -V 2 --vbr-new beats them all.

Of course, this conclusion is based on a limited set of sample (most are not critical for lossy encoders) and on my current sensitivity. It would be really helpful I suppose to hear other results confirming or infirming my own results.

N.B. According to ANOVA & Tukey parametric analysis, both encoders are tied.


EDIT: replaced 3.96.1 (now crossed) by 3.97a6.

This post has been edited by guruboolez: Dec 29 2005, 23:07

[Jebus]

I know this is just one listening test, but would it be possible to remap --preset standard to use --vbr-new if it is determined to be of higher quality, sort of like --r3mix was changed in the last version?

[ezra2323]

Its been over a week with no news of any developments. Any progress being made?

Yes - I am impatient
Yes - the wonderful developers and testers do not "owe" me an answer
Yes - I'm asking anyway

I would love to help test but fear my ears will only harm the process by muddying the test results. I try and aid the HA community in other ways - like helping newbies get their questions answered. I leav ethe listening tests to the golden eared and the programming to the programmers.

Looking forward to a stable 3.97 release soon!

[dev0]

I know this is just one listening test, but would it be possible to remap --preset standard to use --vbr-new if it is determined to be of higher quality, sort of like --r3mix was changed in the last version?

I'd prefer making vbr-new the default VBR mode, but that would need some additional testing.

[[proxima]]

guruboolez results about the new VBR mode are really interesting and unexpected! Maybe these findings could be a starting point for a new HA recommanded version. Certainly further, intensive testing is needed.
Unfortunately i'm busy with my exams at this time

[Jechdery]

I am using 3.96.1 -V 2 rather than 3.90.3 aps for encoding speed reasons.

From guruboolez tests, 3.97a6 -V 2 --vbr-new > 3.90.3 aps > 3.96.1 and 3.97a6 -V 2.

Has anybody compared 3.96.1 -V 2 --vbr-new to 3.90.3 aps ? i.e. does the new VBR algorithm appear better than the old, both for 3.96.1 and 3.97a6 ?

This would allow me to know if I keep on using 3.96.1 or regress to 3.90.3.

[Wombat]

After some reading it seems i finally found the thread that has dethroned 3.90.3

Maybe this is the right place to add something.
There is the new recommendation using 397beta together with vbr-new.
As the listening tests show there seems nothing wrong with it. One thing that makes me wonder is the artifacts that have to be tolerated when using it.
Here is what you developers and others that still can change lame behaviour may check.
At least 2 samples that produce real ugly articacts with 397b1 -V2 --vbr-new don´t do this with 3961 -V2 --vbr-new.
I talk about Birds and s53_wind_saxophone_a from guruboolez.
Maybe comparing behavior with these easy to hear artifacts can help in some way?

[Wombat]

I don´t know where to post this and i am waiting for the 3.97 sample regression thread.
So i will add it here. Even Roels number 1 sample "velvet.wav" has some funny rhytmic plops added with 3.97b -V2 -vbr--new. Mostly at second 2-3 and 9-10.

[robert]

@wombat
could you check 3.98a2 with the velvet.wav? The "sandpaper" problem with "birds" is still there in 3.98a2.

[Wombat]

Sure i will! As soon as i am home tonight.