Possible LAME 3.97 -V 2 killer samples., Since those tracks are transparent on LAME 3.90 APS Options

[/mnt]

I found a couple of tracks that are easy to ABX on LAME 3.97 -V 2 --vbr-new while I found it hard and failed to do on LAME 3.90.3 --alt-preset standard.

Metallica - Until It Sleeps [Load]

CODE

foo_abx 1.3.1 report
foobar2000 v0.9.4.4
2007/10/08 21:21:38

File A: F:\Listen Tests\Untli It Sleeps.wav
File B: F:\Listen Tests\Untli It Sleeps LAME 3.97 V2.mp3

21:21:38 : Test started.
21:22:18 : 01/01  50.0%
21:22:33 : 02/02  25.0%
21:22:48 : 03/03  12.5%
21:23:12 : 04/04  6.3%
21:23:34 : 05/05  3.1%
21:23:37 : Test finished.

----------
Total: 5/5 (3.1%)

Makes a metallic warbling noise on the drums at around 2.14 - 2.20.

Fear Factory - New Breed (Spoetnik Mix) [Hatefiles]

CODE

foo_abx 1.3.1 report
foobar2000 v0.9.4.4
2007/10/08 20:38:49

File A: F:\Listen Tests\New Breed (Spoetnik Mix).wav
File B: F:\Listen Tests\New Breed (Spoetnik Mix) LAME3.97 V2.mp3

20:38:49 : Test started.
20:39:33 : 01/01  50.0%
20:40:07 : 02/02  25.0%
20:41:21 : 03/03  12.5%
20:41:35 : 04/04  6.3%
20:41:58 : 05/05  3.1%
20:42:59 : Test finished.

----------
Total: 5/5 (3.1%)

Metallic warbling and precho at the start, while sounds transparent on LAME 3.90.

This post has been edited by /mnt: Oct 11 2007, 14:30

[Bourne]

can you please provide the WAV samples in lossless compressed format ( 30 seconds max ).

[/mnt]

can you please provide the WAV samples in lossless compressed format ( 30 seconds max ).

Yeah i could, but it might effect the problem though.

[Gow]

How? Lossless compression is bit for bit identical to the WAV.

[/mnt]

How? Lossless compression is bit for bit identical to the WAV.

The encoding behavior would change dramatic when encoding from a 30 sec instead of the full track, thats what i ment the 30 sec sample might reduce used bits on the bit reservoir.

[SoleBastard]

21:21:38 : Test started.
21:22:18 : 01/01 50.0%
21:22:33 : 02/02 25.0%
21:22:48 : 03/03 12.5%
21:23:12 : 04/04 6.3%
21:23:34 : 05/05 3.1%
21:23:37 : Test finished.

----------
Total: 5/5 (3.1%)

Fischer would call this coincidence.

[/mnt]

Metallica - Until It Sleeps sample:

http://www.sendspace.com/file/hk6ijp

Fear Factory - New Breed (Spoetnik Mix) sample:

http://www.sendspace.com/file/nq60re

[shadowking]

We should be only testing 3.98 vs 3.97 and not test 3.90 any more.

[slks]

Why not? If it's possible 3.90 performs better in some situations, the problem should be analyzed and fixed so that the new versions are the best they can be.

[greynol]

If it's possible 3.90 performs better in some situations, the problem should be analyzed and fixed so that the new versions are the best they can be.

I agree completely, but maybe it would be more helpful if these samples were checked using 3.98 instead of 3.97 since that's the version currently in development.

[shadowking]

--alt-preset standard = --vbr-old
--alt-preset fast standard = --vbr-new

You were testing 3.90 vbr-rh (old) vs 3.97 mtrh (new).To get a better picture one should test:

3.97 -V2 --vbr-new vs 3.90 --alt-preset fast standard
3.97 -V2 --vbr-old vs 3.90 --alt-preset standard

Testing 3.90 is problematic to me as the stable version is 3.97 and current version is 3.98. If 3.90 is any reference then are 3.92, 3.93, 3.94, 3.95 , 3.96 any less valid and they should also be tested ?. the other issue is what the hell is the point ? Should we also start testing Vorbis 1.01 ? MPC 0.9, WMA 7, Psytel AAC ?

This post has been edited by shadowking: Oct 9 2007, 08:17

[Rio]

I agree with shadowking. 3.90 APS should be abx'ed by the OP vs. 3.97 -V2 old method (and most would say 3.98, but may be both old and new method as well).

It has been already noted in the wiki however that the new algorithm may suffer in quality vs. faster speed as compared to the old method.

[shadowking]

It has been already noted in the wiki however that the new algorithm may suffer in quality vs. faster speed as compared to the old method.

In some cases its true. I was nearly going to use 'old' but then discovered a range of samples where new is better. Can't make up my mind. I stuck with new - its quicker, not worse overall and is now default.

[robert]

We should be only testing 3.98 vs 3.97 and not test 3.90 any more.

Why? I don't see anything wrong with using 3.90 as a reference point for comparison. Version 3.98 is our current development branch, feedback for it is always welcome.

[/mnt]

I did the abx again on the full track since the sample i uploaded has not been affected.

LAME 3.97 -V 2 --vbr-new

CODE

foo_abx 1.3.1 report
foobar2000 v0.9.4.4
2007/10/09 11:53:22

File A: F:\Listen Tests\Untli It Sleeps.wav
File B: F:\Listen Tests\Untli It Sleeps LAME 3.97 V2.mp3

11:53:22 : Test started.
11:54:03 : 01/01  50.0%
11:54:23 : 02/02  25.0%
11:54:41 : 03/03  12.5%
11:54:54 : 04/04  6.3%
11:55:16 : 05/05  3.1%
11:55:38 : 06/06  1.6%
11:55:51 : 07/07  0.8%
11:56:09 : 08/08  0.4%
11:56:32 : 09/09  0.2%
11:56:50 : 10/10  0.1%
11:57:54 : 11/11  0.0%
11:58:06 : 12/12  0.0%
11:58:10 : Test finished.

----------
Total: 12/12 (0.0%)

The artifact goes away at -V0, but its not present on LAME 3.90.3 and LAME 3.98b5 at V2 or APS.

LAME 3.98 beta 5 -V 2

CODE

foo_abx 1.3.1 report
foobar2000 v0.9.4.4
2007/10/09 12:03:32

File A: F:\Listen Tests\Untli It Sleeps.wav
File B: F:\Listen Tests\Untli It Sleeps (LAME 3.98b5 V2).mp3

12:03:32 : Test started.
12:04:10 : 01/01  50.0%
12:04:31 : 01/02  75.0%
12:04:38 : 02/03  50.0%
12:04:42 : 02/04  68.8%
12:04:56 : 03/05  50.0%
12:04:59 : 03/06  65.6%
12:05:16 : 03/07  77.3%
12:05:21 : 03/08  85.5%
12:05:27 : 03/09  91.0%
12:05:31 : 04/10  82.8%
12:05:49 : 04/11  88.7%
12:05:53 : 05/12  80.6%
12:06:52 : Test finished.

----------
Total: 5/12 (80.6%)

This post has been edited by /mnt: Nov 27 2007, 01:02

[halb27]

Nice to see that 3.98b5 is fine, even at -V2.
IMO 3.98b5 has the best vbr quality among all the Lame versions I know (also better than that of 3.90).

As for 3.97 it is known that there are specific issues (most of concern: the 'sandpaper noise' problem found by Wombat) though overall quality is great. The problems existed already with 3.96 though to a minor degree. This is why I preferred 3.90 over 3.97, but with 3.98 things have changed.

Hopefully 3.98 will soon be final and will be the HA recommended version then.

@ the Lame devs: Though I understand well that standard behavior should be so that Lame encoded mp3s are playable using WMP, I'd love to see an option in the final version to allow for an unrestricted or nearly unrestricted use of bit reservoir with 320 kbps frames.

This post has been edited by halb27: Oct 9 2007, 22:06

[/mnt]

Nice to see that 3.98b5 is fine, even at -V2.
IMO 3.98b5 has the best vbr quality among all the Lame versions I know (also better than that of 3.90).

As for 3.97 it is known that there are specific issues (most of concern: the 'sandpaper noise' problem found by Wombat) though overall quality is great. The problems existed already with 3.96 though to a minor degree. This is why I preferred 3.90 over 3.97, but with 3.98 things have changed.

Hopefully 3.98 will soon be final and will be the HA recommended version then.

@ the lame devs: Though I understand well that standard behavior should be so that Lame encoded mp3s are playable using WMP, I'd love to see an option in the final version to allow for an unrestricted or nearly unrestricted use of bit reservoir with 320 kbps frames.

Yeah I am also hoping that LAME 3.98 will be the new recommended version since that imfamus sandpaper noise problem on 3.97 while 3.97 was recomended and has gone on LAME 3.98.

Also atleast LAME 3.98 still uses 320 frames for VBR on strict ISO mode unlike Fhg's new mp3 surround encoder (uses 32 - 256 for strict ISO).

This post has been edited by /mnt: Oct 9 2007, 13:57

[greynol]

--alt-preset standard = --vbr-old
--alt-preset fast standard = --vbr-new

You were testing 3.90 vbr-rh (old) vs 3.97 mtrh (new).To get a better picture one should test:

3.97 -V2 --vbr-new vs 3.90 --alt-preset fast standard
3.97 -V2 --vbr-old vs 3.90 --alt-preset standard

Your attempt to impose these arbitrary rules only hinders future development.

Glad to see that 3.98 --vbr-new is not affected like 3.97 --vbr-new on these samples.

This post has been edited by greynol: Oct 9 2007, 18:25

[shadowking]

Not rules, Common sense .. vbr engine affects quality. We try to match 2 encoder quality settings unless we are testing one engine vs the other.

[greynol]

We try to match 2 encoder quality settings unless we are testing one engine vs the other.

You used the word "we". Does that mean you're on the development team?

So, if there's a sample that's a problem for --vbr-new that doesn't exist for --vbr-old, then you should just give up?

This post has been edited by greynol: Oct 9 2007, 18:33

[pdq]

I think all that he is trying to say is that you don't want to change too many variables at the same time. Either compare --vbr-old vs. --vbr-new in the same lame version, or else compare the same vbr engine in two different versions of lame. Changing both of them doesn't tell you where any difference is coming from.

[greynol]

I actually got that, but this shouldn't be about comparing versions; it should be about improving the new one.

Suppose 3.98 -V2 --vbr-old didn't exhibit the problem but 3.98 -V2 did, do "we" just throw "our" hands up in the air?

I was under the impression that --vbr-new was supposed to supersede --vbr-old, so given the same quality level, -V2 (or standard), the new engine should be just as good as the old engine regardless of the version being tested. Robert seems to think that 3.90 --alt-preset standard is fair game, and I'm certain he knows which engine is being used in each version.

This post has been edited by greynol: Oct 9 2007, 20:23

[guruboolez]

I was under the impression that --vbr-new was supposed to supersede --vbr-old, so given the same quality level, -V2 (or standard), the new engine should be just as good as the old engine regardless of the version being tested.

--vbr-new superiority is much more an overall one than a constant, systematical one. The same apply to most comparisons (vbr vs cbr, 3.90 vs 3.98, 3.97 vs 3.98, etc...). Strong progress usually contains some regress (the opposite -few progress for global regression- also works very well).

I supposes that one part of the hard-job for developers is to correct some occasional issues without breaking everything.

This post has been edited by guruboolez: Oct 9 2007, 20:43

[Bourne]

how long until 3.98b is 3.98 final? it's about the same time last year the final lame was released... and I don't see any news updates on the changelog from lame.....

[greynol]

I absolutely agree with you, guruboolez, but this doesn't mean that people should be dissuaded from bringing attention to killer samples if they don't fit someone else's notion of common sense.

This post has been edited by greynol: Oct 9 2007, 20:58