Help - Search - Members - Calendar
Full Version: Lame 3.97 Alpha 7 Testing Thread
Hydrogenaudio Forums > Lossy Audio Compression > MP3 > MP3 - Tech
Pages: 1, 2
Gabriel
abr/cbr: enabled ath adjustement
vbr new: adjustement of scalefactors allocation
vbr old from 9 to 4: adjustements.


Regarding vbr, I'd be interested by preset medium results compared to previous versions. If results are positives, I'll perhaps apply the same settings to preset standard.


Results from 3.97a6:
http://www.hydrogenaudio.org/forums/index....showtopic=30631

edit: -q1 and -q0 should now work properly
AtaqueEG
The changes sound interesting...

Too bad I will we working the whole weekend and I am away from my computer and Sennheisers

Hope that Guruboolez has some time wink.gif
AtaqueEG
By the way, it is not at Rarewares...
ezra2323
Yes, is there a link to 3.97 a7? It is not at rarewares.
krmathis
^ Its available here: ftp://cedric.vabo.cz/pub/linux/apps/lame/
john33
QUOTE (AtaqueEG @ Feb 6 2005, 05:38 AM)
By the way, it is not at Rarewares...
*

It is now!!!! smile.gif Forgive me for having a life away from HA and Rarewares, I was away overnight!!! wink.gif
Madrigal
@john33:

Thank you, john33, for all your innovations, hard work, and overall contributions to the audio community. You are very much appreciated.

Regards,
Madrigal
ezra2323
QUOTE
Forgive me for having a life away from HA and Rarewares, I was away overnight!!!


You did not get the memo? You are not allowed a life. Only devotion to digital audio. wink.gif
john33
QUOTE (ezra2323 @ Feb 6 2005, 09:09 PM)
You did not get the memo? You are not allowed a life. Only devotion to digital audio.  wink.gif
*

Forgive me, Master. How am I to be punished???? tongue.gif
Canar
QUOTE (john33 @ Feb 6 2005, 02:05 PM)
Forgive me, Master. How am I to be punished???? tongue.gif
*


20 lashes with a wet noodle. :B
rjamorim
I'd better not comment. You guys amaze me...
tHiNkBoNe
Does 3.97 --decode recognize and process replaygain information if available ?
frodoontop
Since my brother has better ears than me, I let him do an ABX test of lame 3.96.1 versus lame 3.97a7 with: --preset medium --vbr-new. Personally I've tried and tried and never ever could distinguish both files from original.

The sample I used is from the cd : Judas Priest - Turbo Lover, track 6: Out in the cold. I chose this sample because the recording is not too loud (replaygain -3db) and quite a lot instruments occur at the same time. Although he initially chose the wav file as the mp3, he was able to ABX the difference (p = 10/13 = 0.046). That is for lame 3.97a7. He wasn't able to abx the 3.96.1 one though. Still he gave the latest alpha a 4.5 out of 5. In fact he was quite impressed that mp3 could sound so well smile.gif .

Now the strange thing is that the 3.96.1 encode was at much less bitrate than the 3.97a7 one. The 3.97a6 file was 128kbit/s (vbr) and the 3.97a7 file was 154 kbps. I find this difference to be significant. Of course this is only based on one track, but I found the bitrate 'inflation' to be quite constant for several tracks I tried. I guess everybody was hoping for improvement, sorry for reporting the bad news..
Gabriel
QUOTE
Since my brother has better ears than me, I let him do an ABX test of lame 3.96.1 versus lame 3.97a7 with: --preset medium --vbr-new.

I would advise you to use either "--preset fast medium" or "-V4 --vbr-new" but not "--preset medium --vbr-new".
--preset are applying their internal parameters immediately, so "--preset medium" is applying the parameters for "normal" medium preset. After, you add --vbr-new. On initialisation, the correct parameters for -V4 --vbr-new will not be used as they have already been changed from their default status.

On the other hand, if you use "--preset fast medium", the preset will know that it will be used with vbr-new, and will apply proper parameters.
If you use "--vbr-new -V4" or "-V4 --vbr-new" you will also be fine.
frodoontop
This will probably explain the strange results, I'll see if I can get another test.
mithrandir
Scalefactor fix for vbr-new is nice. I've seen some files shrink by almost half between 397a7 and earlier versions.
crazyman
QUOTE (Gabriel @ Feb 10 2005, 07:16 AM)
QUOTE
Since my brother has better ears than me, I let him do an ABX test of lame 3.96.1 versus lame 3.97a7 with: --preset medium --vbr-new.

I would advise you to use either "--preset fast medium" or "-V4 --vbr-new" but not "--preset medium --vbr-new".
--preset are applying their internal parameters immediately, so "--preset medium" is applying the parameters for "normal" medium preset. After, you add --vbr-new. On initialisation, the correct parameters for -V4 --vbr-new will not be used as they have already been changed from their default status.

On the other hand, if you use "--preset fast medium", the preset will know that it will be used with vbr-new, and will apply proper parameters.
If you use "--vbr-new -V4" or "-V4 --vbr-new" you will also be fine.
*



Gabriel,
proposed V4 gives rather too low bitrates, around 165. How about V3, which is very near to 192?

Regards,

Crzmn
mithrandir
The current V4 (in 3.97a7) is much more usable than the old one in terms of bitrate. Before there was a huge jump between V5 and V4; sometimes V4 used as many bits as the old 3.90.3 APS. Now the bitrate scale is more linear. Switching from -X 3,3 to -X 9,9 was a good move I think...of course, it's too early to comment on quality.
robert
I would like to add one point to the list to test with current alpha:

I am interested in testsamples where "-V2 --vbr-new" is ABX'able to "-V2 --vbr-new -b32"
uart
Wow the alpha7 is producing much smaller files sizes at the -V4 setting I usually encode at. For example I just encoded Nirvana (MTV Unplugged in NY) and the album average bit rate was only 148kbps compared with 170kbps with 3.96.1 using the same -V4 setting.
memomai
take a look at EncSpot for a file encoded with Lame 3.97 alpha 7 with --alt-preset standard:
huh.gif
The lowpass filter shows 18600, I thought usually was the filter on 19000

Is this a bug of EncSpot or a new experimental setting? blink.gif

Will you perhaps use the new lowpass filter frequency in the following stable versions? blink.gif

greetings memomai biggrin.gif
Seymour
Subj lame vs one_love.wav
foo_abx v1.2 report
foobar2000 v0.8.3
2005/02/13 18:06:33

File A: file://D:\Program Files\foobar2000\one_love.wav
File B: file://D:\Program Files\foobar2000\_ - _.MP3

18:06:33 : Test started.
18:06:53 : 01/01 50.0%
18:07:01 : 02/02 25.0%
18:07:09 : 03/03 12.5%
18:07:20 : 04/04 6.3%
18:07:31 : 05/05 3.1%
18:07:37 : 06/06 1.6%
18:07:44 : 07/07 0.8%
18:07:49 : 08/08 0.4%
18:07:58 : 09/09 0.2%
18:08:03 : 10/10 0.1%
18:08:20 : 11/11 0.0%
18:08:25 : Test finished.

----------
Total: 11/11 (0.0%)

EDIT: I've forgotten to specify lame settings: -V2 --vbr-new
beto
please, make the sample available somewhere.

What exactly did you test here? Is it a -V2 --vbr-new against -V2 --vbr-new -b32 test??
robert
Thanks, Seymour.

@ beto: I asked him here to abx. you find the sample in that thread too.
Seymour
@ robert: i found this sample ABX'able very easily with the settings marked above by me (as at low frequency (some wobble bass at first 5 secs) so at high frequency (tone practically masked by first "clap" sound)).
But for example I couldn't find difference (sincerely without hard listening work smile.gif) at LAME --alt-preset insane v3.90.3. I tried it just for interest.
guruboolez
one_love.wav vs one_love.mp3 (-V 2 --vbr-new):

foo_abx v1.2 report
foobar2000 v0.8.3
2005/02/14 18:41:59

File A: file://C:\AUDIO\2_ Samples\samples téléchargés\one_love.wav
File B: file://C:\AUDIO\2_ Samples\samples téléchargés\one_love -V 2 --vbr-new.mp3

18:42:02 : Test started.
18:42:09 : 01/01 50.0%
18:42:13 : 02/02 25.0%
18:42:16 : 03/03 12.5%
18:42:22 : 04/04 6.3%
18:42:26 : 05/05 3.1%
18:42:36 : 06/06 1.6%
18:42:40 : 07/07 0.8%
18:42:44 : 08/08 0.4%
18:42:48 : 09/09 0.2%
18:42:53 : 10/10 0.1%
18:42:57 : 11/11 0.0%
18:43:01 : 12/12 0.0%
18:43:03 : Test finished.

----------
Total: 12/12 (0.0%)

Very simple to ABX (basic pre-echo/unprecise sound to my ears).
guruboolez
QUOTE (Gabriel @ Feb 5 2005, 04:11 PM)
abr/cbr: enabled ath adjustement
*

Does it mean that --athaa-sensitivity 1 command is now possible with alpha 7 on ABR/CBR?
I'd like to test, but unfortunately, I don't have access to my main computer (and samples). On the few samples I currently have, the switch doesn't lead to audible difference (tested on ABR 128). Did I something wrong?
Gabriel
QUOTE
Does it mean that --athaa-sensitivity 1 command is now possible with alpha 7 on ABR/CBR?

Yes, but a value of 1 will not change a lot compared to the default value.
Before, there was no ATH adjustement on abr/cbr. Now, it is on by default
Acid Orange Juice
aps_Killer_sample.wav

ABX report

Settings: Lame 3.97a7; -V 2
aps_Killer_sample.wav versus aps_Killer_sample.mp3
( 10/10 ) Very easy to ABX: ugly flanging effect


Settings: Lame 3.97a7; -V 2 --vbr-new
aps_Killer_sample.wav versus aps_Killer_sample.mp3
( 10/10 ) Very easy to ABX: the same ugly flanging effect again
guruboolez
3.97a7 TEST #1: -V 4 & -V 4 --vbr-new full test



Some literature first…


1/ SAMPLES

During previous listening tests, I’ve used a selection of various music (ff123’s samples libraries) and in order to complete it I added two problem samples (castanet2, Orion II) and one classical music (macabre.wav). For the following test, I’ve decided to change this, and to break the previous unity into three separated categories:
• GROUP1 = various music
• GROUP2 = classical music only
• GROUP3 = problem samples

— To build GROUP1, I’ve used the same samples than before (ff123 12+4+1 samples + Dev0’s “Since Always”). I’ve discarded the five “classical music” samples included in ff123’s ensemble (LizstBMinor, BachS1007, Macabre, fossiles, and BeautySlept) and put them in GROUP2. Total = 13 samples. I replaced them with 5 additional samples used in previous collective listening tests; they must be still available on rarewares (thanks to Roberto for this). The 5 samples are: 41_30sec; experienca, NewYorkCity, Quizas, Scars. TOTAL = 18 samples.

— For GROUP2, I’ve began with my previous classical selection (already used for various listening tests I’ve published in the past), which contains 15 samples. I’ve add the five samples excluded from GROUP1 (LizstBMinor, BachS1007, Macabre, fossiles, and BeautySlept) : total attains 20 samples. Three samples were removed: Bruhns and Bayle (problem samples which will join GROUP3) and the harpsichord sample called BeautySlept (I’ve already two harpsichord samples: Couperin and BachBWV1034). I preferred to this last one a guitar sample, called Murcia - Fandango. TOTAL = 18 samples.

— GROUP3. Instead of using one sample representing pre-echo (castanet2) and another one which reveal problems with micro-attacks (Orion II), I’ve used this time 6 samples representing each problem. Hope it will help to have a better idea of real performance of each codec/setting with this kind of issues. There’s also a third issue I’d like to test. It’s a problem I’ve noticed some years ago with Miles Davis’ trumpet: the signal is very tonal, and despite of this, lame introduce severe distortions. With VBR encodings, the bitrate drops to unusual low bitrate. It occurs with all modern lame builds. With time, I’ve collected more samples which suffer from this issue. I’ve therefore put in this group 6 samples representing this “highly-tonal” problem. TOTAL = 6+6+6 = 18 SAMPLES.

In summary: 3 different groups for three different kind of samples; 18 samples for each group; 54 samples as total.


2/ SETTINGS

-V 4 (aka preset medium) is not as easy to test as 128 kbps encodings, especially with common music. It needs more attention, and tests are therefore quickly exhausting. With 54 samples, I was forced to drastically limit the number of challengers. Comparing -V 4 with alpha 6 and alpha 7 was the real priority. It could be enough. Nevertheless, I’ve noticed previously so strange disparities between old-school VBR mode and the --vbr-new routine (in favor of this one) that I’ve decided to add alpha 7 --vbr-new in this test, which doesn’t apparently changed since alpha 6 (see bitrate table below)

In summary: 3 tested settings: alpha 6 V4 // alpha 7 V4 // alpha 7 V4 --vbr-new.


3/ BITRATE

A complete bitrate table is available here.
Summary:
CODE
        alpha6    alpha7          alpha6    alpha7
        -V 4      -V 4              -V4 vbr-new

GR1       176       162  [-8%]      164       163
GR2       172       165  [-3%]      161       162
GR3       158       162  [+3%]      163       162


• Difference is really important between a6 and a7 with -V 4. The biggest difference happened in GROUP1: bitrate is much lower. With metal tracks, the bitrate diminution exceed 15% (<=> 30 kbps)! With classical music (GROUP2), same tendency, but with lower amplitude. It might be interesting to note that bitrate could be seriously higher (+5…+10%) on some tracks with alpha 7. Four tracks are in this situation. These four tracks share the same characteristic: very low energy in high-frequencies (which doesn’t necessary imply low replaygain value). The situation is entirely different for GROUP3: the new alpha7 produces now higher bitrate encodings (+3%) on problem samples than alpha 6.

• There are quite no difference between a6 and a7 with -V 4 --vbr-new.

• With alpha 7, there are no serious difference between old VBR mode and the new one - vbr-new - (at least with V4). Except maybe for classical music (GROUP2), for which --vbr-new produces slightly lighter encodings.

Of course, bitrate doesn’t tell anything about quality. Now, results :-)

In summary: biggest change occurs with -V 4 between old and new alpha. --vbr-new looks unchanged.


4/ RESULTS





4.1/ GROUP1: various music

CODE
a6V4     a7V4    a7V4_new
3,23     3,97      3,99


COMMENTS:
Much less problems with alpha7 than with alpha6. Encodings are more robust, with no serious ringing issues anymore. It’s a nice progress, especially if we keep in mind that bitrate is 15 kbps smaller on average! Difference between -V4 and -V4 --vbr-new is now marginal: some samples are better with default VBR mode, some others sound better with --vbr-new. Layla.wav is the only exception: I’ve ABXed with success the benefit of defaulted VBR compared to vbr-new.

4.2/ GROUP1: classical music

CODE
a6V4     a7V4    a7V4_new
3,14     3,97      4,19


COMMENTS:
Again, alpha6 V4 is worse than alpha7, and even worse when compared to --vbr-new. I think it could be safe to say that ATH issues occur more often with classical. That’s maybe why --vbr-new performed better on average than defaulted vbr mode. I’ve ABXed with success this positive difference on 6 samples. It’s not that much... But even small, it’s still a real good point for --vbr-new. Other good points are: smaller bitrate (-2%) and of course faster encoding (+100%). Maybe some additional tweaking on ATH could fill the gap between the two VBR mode?

4.3/ GROUP3: problem samples

CODE
a6V4     a7V4    a7V4_new
1,53     2,51      2,68


COMMENTS:
— micro-attacks samples: ANOVA analysis: alpha 6 [2.00] < alpha 7 [3.00] < alpha 7 --vbr-new [3.67]. It’s the only ANOVA analysis which conclude on --vbr-new superiority for the WHOLE test.
— highly tonal samples: alpha6 (1.17 / 5) is clearly worse than alpha7 encodings. Marginal difference between -V4 and -V4 --vbr-new with alpha 7 (2.20 & 2.30).
— transient samples: alpha6 is worse again. Results were so bad than I wasn’t able to find difference between alpha7 -V4 and -V4 --vbr-new, except on the marimba sample, which sounded sharper with default VBR mode [ABXed 16/16].


5/ GENERAL CONCLUSIONS

• ATH modifications made by Gabriel on -V4 are highly positive: bitrate is much smaller with ‘modern music’, slightly smaller with ‘classical music’ and slightly higher with problem samples; quality is now much better, whatever the samples’ category. Efficiency and usability is therefore much greater. Some problems are nevertheless still audible.
• As consequence of this double progress, the --vbr-new mode is less attractive than it appeared to be during previous tests. Bitrate is similar, quality is comparable, but with a tiny advantage for --vbr-new with classical music and with ‘micro attacks’ samples. Moreover --vbr-new has a big and uncontestable advantage on encoding speed. I wouldn’t say anymore than defaulted mode is “more secure” than --vbr-new or that this one could increase the bitrate. On contrary, for this test, average bitrate is (very slightly) smaller, quality is (very slightly) higher and speed twice higher :-)


ABX log files are here.
guruboolez
3.97a7 TEST #2: ABR 128 and “ringing samples”


After VBR, I’ve decided to give a try to ABR. The --athaa-sensitivity is now enabled by default, and should greatly improve the quality on samples which suffered with previous alpha from ringing, especially after MP3Gain/ReplayGain process.
The following test is very similar to a previous one, for which I have tweaked the basic preset with the --athlower switch. I’ve now compared the full evolution since 3.90.3:

• lame 3.90.3 --preset 128
• lame 3.96.1 --preset 128
• lame 3.97a6 --preset 128
• lame 3.97a7 --preset 128

CODE
                        3.90.3        3.96.1       3.97a6         3.97a7
                   
Die Lotosblume            2.0           1.0          2.5            4.0
Lamento della ninfa       4.0           1.5          2.5            4.0
LisztBMinor               2.0           1.0          3.5            4.0
Quis non posset           2.0           1.5          3.5            4.5
                 Means   2.50          1.25         3.00           4.13

ABX log files are here.


Quick ANOVA analysis of results:
CODE
FRIEDMAN version 1.24 (Jan 17, 2002) http://ff123.net/
Blocked ANOVA analysis

Number of listeners: 4
Critical significance:  0.05
Significance of data: 8.48E-004 (highly significant)
[...]

---------------------------- p-value Matrix ---------------------------

        3.97a6   3.90.3   3.96.1  
3.97a7   0.031*   0.005*   0.000*  
3.97a6            0.287    0.003*  
3.90.3                     0.020*  
-----------------------------------------------------------------------

3.97a7 is better than 3.97a6, 3.90.3, 3.96.1
3.97a6 is better than 3.96.1
3.90.3 is better than 3.96.1



3.96.1 is last; 3.97a7 is the ‘winner’ and reaches near transparency, which should be the expected result for so ‘easy’ samples. On the other side, the reference 3.90.3 suffers from severe ringing/background noise reduction. Difference between all version could be illustrate with this animated gif (220 kb). These positive results are another good point for 3.97a7 (which is also twice faster, if not more). This additional progress for lame leads me to perform another test, and to compare ABR to VBR at ~130 kbps.
guruboolez
3.97a7 TEST #3: ~130 kbps ABR / VBR encodings on classical music

Unfortunately, my time is not unrestricted, and my resistance to effort is even more limited. I couldn’t therefore assume a complete test. As consequence, I’ve limited my efforts to test the performance of newest alpha on classical music. Two reasons for that:
• alpha 6 already appeared to be superior to 3.90.3 on general music (see this test.
• alpha 6 was less convincing with classical music, and reaches similar quality than 3.90.3 whereas --athlower 10 lead to significant progress (but lowered the quality on ‘modern’ music). See this test.
• -V 5 has already proved its quality on general music (see Roberto’s multiformat tests including 3.95.1 -V 5). But with classical music, I really fear that -V 5 won’t perform as well, at lest with some samples.

The following test is therefore limited to two encoders and three settings:
• alpha 6 --preset 135
• alpha 7 --preset 135
• alpha 7 -V 5 --vbr-new

I’ve used 135 as value in order to target 128 kbps (it’s an old tip which works with classical only, but works great since lame 3.90). I’ve used -V 5 --vbr-new because this mode seems to have less troubles with ringing issues. Note that I didn’t use --athaa-sensitivity 1. Average bitrate for VBR encodings reaches 130 kbps (and 128 kbps for ABR).

It is important to note that I didn’t ABX anything. The test consists on quick ABC evaluations. I nevertheless take care to evaluate properly each distorted files, in order to not give a better note to a file which doesn’t really sound better than another one. But without ABX confirmation, wrong evaluations may occur.


Results :



ABX log files are here.


Comments :

• alpha 7 is clearly better than alpha 6. Ringing was lowered again (it was also lowered between alpha 5 and alpha 6). I’ve nevertheless noticed four samples for which quality was apparently lower compared to alpha 6. But I repeat, without a direct ABX comparison, this evaluation is not safe (same comment applies for progress though - but I’ve noticed eight samples for which alpha 7 sounded better).

• VBR is not safe at all at this bitrate with classical music. -V 5 performs very well on ‘modern music’ (synonym of compressed/overcompressed music), but with the wide dynamic range of classical music, it sucks (sorry). When VBR performs better than ABR, bitrate is also clearly higher (less efficiency): 140-145 kbps on average, not only on short samples but also on full album. VBR could also produce lower bitrate (it compensate higher one), but in this case quality is systematically worse than ABR. Piano samples (Liszt, Brahms) are both crap, with huge ringing and underwater-sound. Nevertheless, quality is clearly better than ABR with some samples (with higher bitrate though…). ReplayGain/MP3gain could even increase the gap between ABR and VBR.

I also recall that I didn’t use the --athaa-sensitivity 1 switch. This command line is well-known for reducing ringing issues; reverse side is a very slight increase of average bitrate. But with classical, this command line really boosts the bitrate. For these 18 samples as example, bitrate goes from 130 to 137 kbps, with still a big disparity between different albums (120…~160 kbps). In other word, user can’t be sure to store two hours of music on a 128 MB player with -V 5, and chances are near zero with --athaa-sensitivity 1 command line. And I don’t consider -V 6 as a working solution with classical: from my experience, quality could be acceptable with some albums but is most often really annoying - probably more annoying than ABR encodings at similar bitrate.


N.B. The compostelle.wav samples has very strange warbling artefact with ABR and VBR (but it’s more audible with ABR). It could be worth to take a look at this unusual artefact.
guruboolez
3.97a7 TEST #4: VARIOUS + links

Last, some other comments.

4.1. -q0 switch

I didn’t insist, but on few tracks I’ve tried, the -q 0 switch doesn’t produces terrible ringing artefacts anymore. No bugs, but I don’t know if this switch really improves quality.


4.2. -b32 switch and -V 2 --vbr-new

Honestly, I must say that I don’t really like this idea. In my experience, the -b 128 was a good security against optimistic bitrate allocation with lame VBR encodings. Some flaws noticed in the previous test [-V 5 against --abr 128] are probably a consequence of the usage of frames at excessive low bitrate. Therefore, I was very confident to find without effort problem with -V 2 --vbr-new -b32 due to the usage of unrestricted low bitrate frames.
I’ve tried, I’ve tried again, but I didn’t found anything…

• First, I’ve selected samples at very low volume (very high RG adjustment): already was OK, and few frames were encoded below 128 kbps.
• Then, I’ve selected the samples used for GROUP3_tonal, which are strangely encoded at unusual low bitrate. Some frames are now inferior to 96kbps, but again, I didn’t found any additional problems.
• Last, I’ve decided to cheat, and to use very, very low volume samples and to push the volume of my amp to the maximum. I did it for my MP3 DECODERS test performed last year, and decided to use exactly the same samples. Results: can’t detect any difference.

These results surprised me a lot. I’ve checked all files with encspot, which indicate the presence of frames encoded below 128 kbps. There must logically be some differences... Despaired, I’ve launched bit-to-bit comparisons between files encoded with regular preset and files encoded with -b 32 floor… and it gives me the answer: most files were bit-to-bit identical, despite of the usage of frames at lower bitrate!! In rare cases, they were some objective differences. They are *visible* in a frequency editor, but they appear to be very limited (one or two consecutive frames - never more), and I can’t ABX any difference.
I made 2 temporary html pages in order to illustrate this:
http://guruboolez.free.fr/b32impact/b32floor.html
http://guruboolez.free.fr/b32impact/b32floor2.html

Honestly, I don’t know what kind of samples could be a problem for -b 32 additional command? Does someone have an idea?


4.3. LINKS


I’ve uploaded all samples, excepted those ones used for -V 4 GROUP1. They are available on:
http://www.ff123.net
and on:
http://www.rjamorim.com/test/samples/

GROUP2: 18 classical samples (14.3 MB)
GROUP3: 6 micro-attacks samples (6.27 MB)
GROUP3: 6 highly tonal samples (4.46 MB)
GROUP3: 6 preecho samples (4.61 MB)

To make samples accessible to everybody, I’ve encoded them with WavPack lossless with self-extract module. That’s why they are all in *.exe format. To decode them very easily, just select all of them and press [ENTER]. The DOS windows will disappear by themselves. On Linux or Mac, or if you don’t like *.exe files, just remove the *.exe extension: files can then be played by any player supporting WavPack or could be decoded with official wavunpack.exe decoder.
crazyman
QUOTE (guruboolez @ Feb 21 2005, 12:27 AM)
3.97a7 TEST #4: VARIOUS + links

Last, some other comments.

4.1. -q0 switch

I didn’t insist, but on few tracks I’ve tried, the -q 0 switch doesn’t produces terrible ringing artefacts anymore. No bugs, but I don’t know if this switch really improves quality.


4.2. -b32 switch and -V 2 --vbr-new

Honestly, I must say that I don’t really like this idea. In my experience, the -b 128 was a good security against optimistic bitrate allocation with lame VBR encodings. Some flaws noticed in the previous test [-V 5 against --abr 128] are probably a consequence of the usage of frames at excessive low bitrate. Therefore, I was very confident to find without effort problem with -V 2 --vbr-new -b32 due to the usage of unrestricted low bitrate frames.
I’ve tried, I’ve tried again, but I didn’t found anything…

• First, I’ve selected samples at very low volume (very high RG adjustment): already was OK, and few frames were encoded below 128 kbps.
• Then, I’ve selected the samples used for GROUP3_tonal, which are strangely encoded at unusual low bitrate. Some frames are now inferior to 96kbps, but again, I didn’t found any additional problems.
• Last, I’ve decided to cheat, and to use very, very low volume samples and to push the volume of my amp to the maximum. I did it for my MP3 DECODERS test performed last year, and decided to use exactly the same samples. Results: can’t detect any difference.

These results surprised me a lot. I’ve checked all files with encspot, which indicate the presence of frames encoded below 128 kbps. There must logically be some differences... Despaired, I’ve launched bit-to-bit comparisons between files encoded with regular preset and files encoded with -b 32 floor… and it gives me the answer: most files were bit-to-bit identical, despite of the usage of frames at lower bitrate!! In rare cases, they were some objective differences. They are *visible* in a frequency editor, but they appear to be very limited (one or two consecutive frames - never more), and I can’t ABX any difference.
I made 2 temporary html pages in order to illustrate this:
http://guruboolez.free.fr/b32impact/b32floor.html
http://guruboolez.free.fr/b32impact/b32floor2.html

Honestly, I don’t know what kind of samples could be a problem for -b 32 additional command? Does someone have an idea?


4.3. LINKS


I’ve uploaded all samples, excepted those ones used for -V 4 GROUP1. They are available on:
http://www.ff123.net
and on:
http://www.rjamorim.com/test/samples/

GROUP2: 18 classical samples (14.3 MB)
GROUP3: 6 micro-attacks samples (6.27 MB)
GROUP3: 6 highly tonal samples (4.46 MB)
GROUP3: 6 preecho samples (4.61 MB)

To make samples accessible to everybody, I’ve encoded them with WavPack lossless with self-extract module. That’s why they are all in *.exe format. To decode them very easily, just select all of them and press [ENTER]. The DOS windows will disappear by themselves. On Linux or Mac, or if you don’t like *.exe files, just remove the *.exe extension: files can then be played by any player supporting WavPack or could be decoded with official wavunpack.exe decoder.
*


Guruboolez,
I admire your hard work on listening tests and their presentation. Do you think it will be worth doing comparison between previous alpha 7 with standard setting V2 vbr-new and the hottest alpha 7 with intended standard V4? I think decreasing of bitrate in V4 may cause also some decreasing the quality if compared with former 3.97a7 (1).

Cheers,

Crazyman
Gabriel
Thank you very much for those detailled results, they are very usefull.
AtaqueEG
QUOTE (crazyman @ Feb 21 2005, 02:42 AM)
Guruboolez,
I admire your hard work on listening tests and their presentation. Do you think it will be worth doing comparison between previous alpha 7 with standard setting V2 vbr-new and the hottest alpha 7 with intended standard V4? I think decreasing of bitrate in V4 may cause also some decreasing the quality if compared with former 3.97a7 (1).

Cheers,

Crazyman
*



What do you mean?
What "hottest" alpha 7 are you talking about.
There is only one 3.97 alpha at Rarewares and Mitiok does not even have it.

And what could be the point in comparing V2 and V4?

If LAME could achieve V2/aps quality at V4 bitrates, it would be HUGE news. But it just won't happen. By definition.
AtaqueEG
QUOTE (Gabriel @ Feb 21 2005, 03:08 AM)
Thank you very much for those detailled results, they are very usefull.
*


Congratulations to you, Gabriel.

This alphas are looking good, and sounding even better.
crazyman
QUOTE (AtaqueEG @ Feb 21 2005, 01:16 AM)
QUOTE (crazyman @ Feb 21 2005, 02:42 AM)
Guruboolez,
I admire your hard work on listening tests and their presentation. Do you think it will be worth doing comparison between previous alpha 7 with standard setting V2 vbr-new and the hottest alpha 7 with intended standard V4? I think decreasing of bitrate in V4 may cause also some decreasing the quality if compared with former 3.97a7 (1).

Cheers,

Crazyman
*



What do you mean?
What "hottest" alpha 7 are you talking about.
There is only one 3.97 alpha at Rarewares and Mitiok does not even have it.

And what could be the point in comparing V2 and V4?

If LAME could achieve V2/aps quality at V4 bitrates, it would be HUGE news. But it just won't happen. By definition.
*



Alpha 7 was several times upgraded at Rarewares, the last version issued Feb 20, previous Feb 12, and there were even some more before, I don´t exactly recollect when. From this point of view, I speak about more Alphas.

Gabriel´s recommendation to test V4 setting was raised already after first "versions" of alpha 7 were out on Rarewares. Just for those not much in technique involved music fans could be good to know which alternative to former APS V2 where target bitrate was around 190-210, is in latest alpha 7 where V2 yields even significantly quite higher bitrate.
Gabriel
QUOTE
Honestly, I don’t know what kind of samples could be a problem for -b 32 additional command? Does someone have an idea?

Those which trigger a failure of the psychoacoustic model.
The problem beeing that we do not really know when the model will fail.
indybrett
Would this be the version of LAME where vbr-new starts to become the standard? I ask because I see it being used more and more in the listening tests.
jaybeee
Guruboolez - truly great work. I applaud you.

The testing of -V 4 (aka preset medium) has been something I've been most interested in. I'm someone who has enough hard disk space to store lossless files, so my lossy encoder of choice can vary. However, owning an iRiver HP120 portable player has made me lean more towards LAME again. This is simply because Ogg Vorbis uses too much battery life.

However, using preset standard, to me with my ears, is a bit of an overkill and those that own a portable will know that the lower the bitrate the longer the battery life.

So, if LAME can achieve near transparency with -V 4 (preset medium), then this would play into the hands of those who own portables and want better battery life.

Who knows, with more testing maybe HA will recommend -V 4 as the switch of choice for portables??

BTW I do think the use of mp3s on portables should be an important element in LAMEs development. Sure, -V 4 might not be quite as good as preset standard (makes sense, it's not throwing as much bits at it etc), but mp3 is here to stay, regardless of the current HA community's encoder of choice. And we all know that portable digital audio players are here to stay, so all in all a very exciting time for LAME.

If such develoment in the coding and quality v's bitrate can be successfully ported to LAME 4, then we could be in for some even more exciting encoding times.

Well done LAME devs and ABX'ers.

jb
PoisonDan
And I just created MP3s of my entire CD collection this weekend, using -V4 with 3.96.1. Maybe I should have waited a little longer. wink.gif

Oh well, recreating them is just a matter of running a batch convert run in foobar2000 for me. Not much effort.
robert
@guruboolez
From me too, a big THANK-YOU. I'm overwhelmed of your detailed feedback.
Madrigal
QUOTE (robert @ Feb 21 2005, 07:40 AM)
@guruboolez
From me too, a big THANK-YOU. I'm overwhelmed of your detailed feedback.
*

@guruboolez
Ditto. Amen. Huge thanks for your hard work. I am very excited about the progress you are almost single-handedly enabling.

Regards,
Madrigal
guruboolez
My work is few thing compared to all development efforts. Thanks to you smile.gif
Enig123
Thanks for your hard work, guruboolez.

I'm most interested in ~130 test, your listenling test told me to choose abr @ that bitrate.

BTW, I have the same prefered gere as yours, classical. smile.gif
Madrigal
QUOTE (Gabriel @ Feb 21 2005, 04:08 AM)
Thank you very much for those detailled results, they are very usefull.
*

Huge thanks to you too, Gabriel ... I really appreciate all your hard work and communication, as well as the combined efforts of all the developers. The progress being made with 3.97 is really exciting, and hopefully we are not all that far away from another stable release, with --vbr-new as the default for all vbr levels.

Keep up the good work, and once again, thanks a million!

Regards,
Madrigal
uart
Brilliant work guruboolez. Very good information and just what I was interested in as I just started using the alpha7 with "-V4 --vbr-new" for my portable encodes about one week ago. Those results are looking really positive. smile.gif
alex2210
Hi

I found one very interesting test sample - performanse of trumpet*
Sample is 10sec. long (trumpet.flac - 964KB)

I make simple 128kbps test with Lame 3.96.1, 3.97alpha7 and old 3.90.3

3.90.3 with "-h --abr 128"
3.96.1 with "-h --abr 128" and "-V 5 --athaa-sensitivity 1"
3.97a7 with "-h --abr 128" and "-V 5 --vbr-new"

You must listen this! Result is acceptable with old 3.90.3 ,
bad with 3.96.1 and very bad with new 3.97alpha7.

I upload test file
here
- in .zip format

Sorry for my english ;-)

* DON ELLIS "Just One Of Those Things"
New York, Nola Penthouse Studios, 21 aprile 1961
Dal CD "Out Of Nowhere" (Candid CCD 79032)
----------------------------------------------------------
Jechdery
[quote=alex2210,Feb 21 2005, 02:37 PM]
Hi

I found one very interesting test sample - performanse of trumpet*
Sample is 10sec. long (trumpet.flac - 964KB)


This trumpet sample is easy to ABX between the original and 3.97a7 -V4 :

WinABX v0.42 test report
02/21/2005 22:18:59

A file: trumpet.wav
B file: trumpet V4.wav

Start position 00:00.0, end position 00:10.0
22:19:41 1/1 p=50.0%
22:19:57 2/2 p=25.0%
22:20:02 3/3 p=12.5%
22:20:07 4/4 p=6.2%
22:20:13 4/5 p=18.8%
22:20:18 5/6 p=10.9%
22:20:23 6/7 p=6.2%
22:20:28 7/8 p=3.5%
22:20:33 8/9 p=2.0%
22:21:28 9/10 p=1.1%
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2009 Invision Power Services, Inc.