Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Why does LAME 3.93 have higher SNR than 3.98? (Read 15252 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Why does LAME 3.93 have higher SNR than 3.98?

Greetings.

I am analysing SNR (signal to noise ratio) of files and i noticed one exact problem: On CBR or ABR, same bitrate on newer 3.98.4 would produce (sometimes much) worse results than old 3.93.1
For example, on 192 kbits on my test sample (Vivaldi 4 seasons, CD rip 16bit 44100hz) endoding would produce ~27 on newer lame versus ~34 on older lame!

Is this a well known bug in 3.98.4? Or why does a latter version produce worser signal-to-noise ration than earlier version?

Why does LAME 3.93 have higher SNR than 3.98?

Reply #1
First tell us how you are measuring SNR, and then maybe we can shed some light on this.

Why does LAME 3.93 have higher SNR than 3.98?

Reply #2
Or why does a latter version produce worser signal-to-noise ration than earlier version?


Probably due to differences in the low pass filter settings.  Assuming you have the same low pass and bitrate settings, the SNR doesn't really depend much on the encoder. 

Is this a well known bug in 3.98.4?


No.  SNR is not meaningful for mp3 files, so theres no expectation that higher numbers are better, or that it should have any particular value.  Its basically just a random number determined by how the mp3 format works, the lowpass/sample rate settings, and the bitrate.

Why does LAME 3.93 have higher SNR than 3.98?

Reply #3
2pdq

I am measuring SNR.... erm... like it is supposed to be measured.

First i offset the resulting file to match the source PCM file, measure to measure.
Then i calculate Asignal**2 / Anoise**2, as said in wikipedia http://en.wikipedia.org/wiki/Signal-to-noise_ratio

This means that, for example, for each second (44100 measures) (assuming mono signal for simplicity) i am calculating (f1[0]**2 + f1[1]**2 + ... + f1[44099]**2) / ((f1[0] - f2[0])**2 + (f1[1] - f2[1])**2 + ... + (f1[44099] - f2[44099])**2), where f1 is source signal and f2 is encoded-decoded signal, and that is the SNR for this second of the signal.

So, medium value over the file is ~27 when comparing source file to 3.98.4 encoded file, and ~34 when comparing source file to 3.93.1 encoded file.
I can upload the files in question so you could check in your own SNR measuring program if you wish.

2 saratoga

I see, how do i get to know what ow pass settings i am using? Bitrate is the same, --cbr -b 192
I know SNR isnt totally meaningful to understand mp3 file quality, but still, it does corellate with quality. SNR for bigger bitrate would always be bigger than for lower bitrate.
Such a difference in signal-to-noise ration feels like the quality of 3.98.4 went down compared to 3.93.1.
Here is an example of topic where person says that listening tests confirm that http://www.claessonedwards.com/forum/viewtopic.php?p=9839

Why does LAME 3.93 have higher SNR than 3.98?

Reply #4
Not "isn't totally meaningful," rather such measurements are useless (except for extreme/trivial/uninteresting cases), and are forbidden as a means to support claims about the sound quality of lossy compression on this forum.

These are the terms to which you agreed in order to participate here (specifically #8 in this case):
http://www.hydrogenaudio.org/forums/index.php?showtopic=3974

If you aren't already familiar, you should familiarize yourself with what are the proper means to support sound quality claims:
http://www.hydrogenaudio.org/forums/index....showtopic=16295

Why does LAME 3.93 have higher SNR than 3.98?

Reply #5
I once did some tests evaluating the PSNR metric for different encoders. As expected, the PSNR didn't have a high correlation with perceived audio quality. I compared AoTuV vorbis, LAME and Nero AAC. Nero AAC produced much higher PSNR at most bitrates than LAME, and AoTuV was mostly below those two. But listening tests so far have shown that AoTuV is at least tied to LAME and Nero AAC in terms of perceived quality if not better. Also, following the video encoding world where the metric PSNR is often used, one can see that it also has only some correlation with perceived quality. Encoders with good psychoacoustic/psychovisual models do not necessarily produce higher PSNR numbers.

In the video world is has become a bit common now to use the SSIM metric to evaluate the visual quality, from what I read it correlates better with the perceived quality than (P)SNR. You can find some papers about possible audio quality metrics, including one about SSIM, on Google Scholar, maybe it includes some useful ones. I would be quite interested to see tests of correlation of results of public listening tests with said metrics.

That said, the agreed upon method on Hydrogenaudio to evaluate lossy encoders is the blind listening test. It basically says that the only meaningful metric to assess a lossy encoder is the perceived difference from the source material, and the listener is not allowed to know which sample is which (blind test) during evaluation. If you can not reliably tell the lossy encode from the source, the lossy encode is considered transparent, i.e. perceptually indistinguishable from the source. That means that the encoder works properly and well at that chosen settings. So a more meaningful test than comparing SNR values would be ABXing LAME 3.93.1 and LAME 3.98.4 encodes. I expect them to be quite close contestors.
It's only audiophile if it's inconvenient.

Why does LAME 3.93 have higher SNR than 3.98?

Reply #6
2 greynol
I dont make any claims. Really. I am not making a statement. If it did sound like that please forgive me, english isnt my native tongue.
All i wonder is why. Why does it happen. Why does latter lame provide more noisy signal (less SNR). I'm not saying "why does latter lame have lower quality" however even that wouldnt be a statement, more of a wrong formed question (a question that implies something that isnt a fact, like a question "Does your mother know you are a gay" impiles you are a gay, and whichever you answer, yes or no, makes you agree with it).

Once again, if i sound like i make a statement please forgive me. I do not. I am just showing SNR calculations i made and ask why can it happen like that.

2 Kohlrabi
Yeah, i do understand listening tests are the way to determine quality. However, i am not able to do that.
So, i am right now stuck with SNR. And i have now tested 3.90.3 as well - both 3.90.31 and 3.93.1 provide a bigger SNR than 3.98.3

Now my question is why can this happen. Did defaults change? Did pshycoacoustic model change? Did something else change?
In other words, my queston is "Can i do something to my 3.98.3 to make it produce a file of similar SNR to 3.93.1 or 3.90.3?"

Why does LAME 3.93 have higher SNR than 3.98?

Reply #7
Here is some output data:
(--resample 44100 is effectively not used since input file is 44100)

lame_3_93_1.exe --cbr -b 192 ..\CNT\8.wav --resample 44100 -m s tmp
LAME version 3.93 MMX  (http://www.mp3dev.org)
(Win32 binaries from:  http://www.hot.ee/smpman/mp3)
CPU features: i387, MMX (ASM used), SIMD, SIMD2
Using polyphase lowpass  filter, transition band: 18671 Hz - 19205 Hz
Encoding ..\CNT\8.wav to tmp
Encoding as 44.1 kHz 192 kbps stereo MPEG-1 Layer III (7.3x) qval=2
    Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
  7722/7724  (100%)|    0:13/    0:13|    0:13/    0:13|  15.499x|    0:00
average: 192.0 kbps  LR: 7725 (100.0%)

lame_3_90_3.exe --cbr -b 192 ..\CNT\8.wav --resample 44100 -m s tmp
..\lame_3_90_3.exe: unrec option --cbr
LAME version 3.90.3 MMX  (http://www.mp3dev.org/)
CPU features: i387, MMX (ASM used), SIMD, SIMD2
Using polyphase lowpass  filter, transition band: 18671 Hz - 19205 Hz
Encoding ..\CNT\8.wav to tmp
Encoding as 44.1 kHz 192 kbps stereo MPEG-1 Layer III (7.3x) qval=5
    Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
  7722/7725  (100%)|    0:16/    0:16|    0:16/    0:16|  12.534x|    0:00
average: 192.0 kbps  LR: 7725 (100.0%)

lame_3_98_4.exe --cbr -b 192 ..\CNT\8.wav --resample 44100 -m s tmp
LAME 3.98.4 32bits (http://lame.sf.net/)
CPU features: MMX (ASM used), SSE (ASM used), SSE2
Using polyphase lowpass filter, transition band: 18671 Hz - 19205 Hz
Encoding ..\CNT\8.wav to tmp
Encoding as 44.1 kHz stereo MPEG-1 Layer III (7.3x) 192 kbps qval=3
    Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
  7724/7724  (100%)|    0:22/    0:22|    0:22/    0:22|  8.9612x|    0:00
-------------------------------------------------------------------------------
  kbps        LR  %    long switch short %
  192.0      100.0        99.8  0.1  0.1
Writing LAME Tag...done
ReplayGain: -2.3dB

Why does LAME 3.93 have higher SNR than 3.98?

Reply #8
Since LAME version 3.94, the ABR/CBR/VBR modes use some presets, so no more plain old GPsycho psy model. (look out for NSPsycho, Naoki Shibata's psy tunings)

http://lame.cvs.sourceforge.net/viewvc/lam...l?revision=HEAD

Quote
Gabriel Bouvigne:
    work on presets[/li][li]use presets by default for cbr/abr[/li]
  • use presets by default for vbr

Why does LAME 3.93 have higher SNR than 3.98?

Reply #9
So, robert, can i using some input parameters "emulate" 3.93 behavior on 3.98? I mean, can i tweak 3.98 by changing some settings so it would encode like 3.93 does by default?

Why does LAME 3.93 have higher SNR than 3.98?

Reply #10
Before v3.94 you would need to use --alt-preset cbr 192

That is CBR using presets like 3.98.4.  --cbr -b is the old code until 3.94.  Everything after that default to presets - -b192, --cbr 192, --alt-preset 192 are all the same since 3.94


Why does LAME 3.93 have higher SNR than 3.98?

Reply #11
So, robert, can i using some input parameters "emulate" 3.93 behavior on 3.98? I mean, can i tweak 3.98 by changing some settings so it would encode like 3.93 does by default?


No.

Why does LAME 3.93 have higher SNR than 3.98?

Reply #12
Now my question is why can this happen. Did defaults change? Did pshycoacoustic model change? Did something else change?
In other words, my queston is "Can i do something to my 3.98.3 to make it produce a file of similar SNR to 3.93.1 or 3.90.3?"


That's not "in other words"; those are two different questions.

Question 1. What changed?
Answer: maybe the LAME changelog has more details, or a LAME dev might respond to email in a timely manner. The answer may or may not be useful to you.

Question 2. Can I set some settings to get back the old SNR?
Answer: why do you want to do this? Purely for academic purposes? To learn a little more about how LAME works?

Why does LAME 3.93 have higher SNR than 3.98?

Reply #13
Okay, i understand my questions are maybe too uncertain, then let me rephrase it:

Are you sure that i am not doing something wrong, which makes me get less SNR on 3.98 than on 3.93?
And - are you sure that the reduction in SNR i am getting when encoding with 3.98 when compared to 3.93 does not at all indicate a drop in quality?
And - what would be your most probable explanation of why did such SNR drop happen when going from 3.93 to 3.98?

PS: Yes, you can call this interest of mine "purely academic".

Why does LAME 3.93 have higher SNR than 3.98?

Reply #14
Please stop referring to this as SNR. What you are looking at is the remainder after subtracting the decoded file from the original, which, as has already been pointed out, typically bears little or no relationship with quality.

Several times per month someone arrives here with the revelation that all you have to do is subtract the original from the decoded file and miraculously you now have a numeric representation of quality. It doesn't work. So please stop pestering us with this nonsense and come back when you have some serious questions that you want to ask.

Why does LAME 3.93 have higher SNR than 3.98?

Reply #15
a wrong formed question

This is exactly the underlying problem with this topic of yours.

I am just showing SNR calculations i made and ask why can it happen like that.

Welcome to the wonderful world of psychoacoustics!

both 3.90.31 and 3.93.1 provide a bigger SNR than 3.98.3

...and completely irrelevant if you're trying to determine whether there is a difference in perceived sound quality.

Can i do something to my 3.98.3 to make it produce a file of similar SNR to 3.93.1 or 3.90.3?

Which begs the question: even if you could, why would you want to?

Why does LAME 3.93 have higher SNR than 3.98?

Reply #16
And - are you sure that the reduction in SNR i am getting when encoding with 3.98 when compared to 3.93 does not at all indicate a drop in quality?


Perhaps I was unclear when I said that your SNR measurement was a meaningless number with no connection to quality.  Let me be more clear:  its completely worthless.  No use at all.  You might as well flip coins. 

And - what would be your most probable explanation of why did such SNR drop happen when going from 3.93 to 3.98?


Have you verified that the low pass filter settings are the same?

Why does LAME 3.93 have higher SNR than 3.98?

Reply #17
On topic that what i am calculating is not SNR:
Is wikipedia wrong then? I am calculating exactly what SNR it is said to be on wikipedia, Psignal/Pnoise.
http://en.wikipedia.org/wiki/Signal-to-noise_ratio

On topic that SNR is worthless and does not relate to quality:
How do you explain then that same file encoded with lesser bitrate AWAYS has smaller SNR than same file encoded with bigger bitrate.
And (i hope) there is no argument that SNR is telling EXACTLY how much the signal was distorted. Since it is by definition SIGNAL to NOISE ratio. The bigger it is, the less noise was added. The lower it is, the more noise was added.
However, there is also no argument that signal distortion != percieved distortion, since our ear has its own hearing quirks. Thats what psychoacoustics are about. And I never stated that SNR corelates to HEARING QUALITY and i know SNR does not measure that.
But I am talking about signal distortion here. And it is clear that 3.98.3 provides more signal distortion than 3.93.1. Now the argument that this distortion does not mean by itself that it sounds worse is not relevant to me. I never argued about that! What i need to know is wether i am doing something wrong with 3.98.3 that i receive a more distorted file, or wether i am doing nothing wrong, and drop in SNR is intended, and indeed, less SNR brings more HEARING QUALITY in this case.
Just look here http://www.claessonedwards.com/forum/viewtopic.php?p=9839
Believe your eyes. More waveform distortion with 3.98.3, less with 3.93.1

About lowpass:
I typed the console output. Since all three say "Using polyphase lowpass filter, transition band: 18671 Hz - 19205 Hz" can i assume that lowpass settings are the same? What parameters shall i input to make sure that ALL filter settings are the same?

About why do i want it:
If you know how to do it, tell me. If you dont, tell me you dont know how to do it. If i cannot explain to you importance of something it doesnt mean it is not important to me. A kid cannot explain importance of healthy food, or how does it affect his body health, but it is still important to him, right?

Why does LAME 3.93 have higher SNR than 3.98?

Reply #18
On topic that SNR is worthless and does not relate to quality:
How do you explain then that same file encoded with lesser bitrate AWAYS has smaller SNR than same file encoded with bigger bitrate.


Obviously, lower bitrate means higher level of distortion (apart from, maybe, specific artificial cases). But lower bitrate does not universally imply lower quality, depending on particular definition of the latter. So please, either stop using the word "quality", or provide us with your definition of it before talking about lower/higher quality.

Quote
What i need to know is wether i am doing something wrong with 3.98.3 that i receive a more distorted file, or wether i am doing nothing wrong, and drop in SNR is intended, and indeed, less SNR brings more HEARING QUALITY in this case.


How can you say you do something wrong if you use exactly same command line parameters? Obviously, something within the encoder itself has changed, probably the masking thresholds used for perceptual analysis. You need to compare the source code for different versions to find out the exact reasons.

As for whether the decreased SNR means degradation in codec quality - no, there is no direct connection, since preserving the waveform (i.e. minimizing the absolute amount of distortion) is not the codec's purpose, nor a design goal. If you want some numeric characteristic of the codec quality then first, you need to compute SNR in frequency domain, and second, the computation must be weighted according to masking thresholds calculated for each analysis window. And even then you can't say that the codec with higher metric is "better", since both may appear transparent in a listening test.

If you are trying to enforce codec to serve the different purpose, then you need to develop the different analysis model. If your goal is to maximize average SNR, then you need to replace the psychoacoustic model with the model that would calculate scalefactors according to the maximum possible average SNR criterion (this probably would mean uniform quantization noise ditribution across all bands, although I'm not sure). Of course you should not hope for perceptual transparency at any bitrate in this case.


Why does LAME 3.93 have higher SNR than 3.98?

Reply #19
2 alexeysp
Okay, i will stop using the world quality in the sense "reverse of the amount of noise added" and call it distortion, ok?

I am thinking that i may be using same parameters, but defaults have changed somehow. I understand that preserving signal is not the goal, i just want to make sure that i'm not doing something wrong with the codec. That the data i gathered (and not only me, see link in prev. post) does not in any way mean the files i get with 3.98.3 are lower in quality (hearing quality) than those i get with 3.93.1

 

Why does LAME 3.93 have higher SNR than 3.98?

Reply #20
The site you linked doesn't contain any data, just an image of two waveforms put onto each other. Please explain what these images are supposed to convey, it's not self-evident to me. Also:

Quote
But the visual impression of the waveform comparisons matches actually the results of my (and Jesse's) blind ABX (listening) testings.
What does that even mean?

To say it again, LAME is most certainly not tuned to produce high SNR, but rather to achieve transparency. So, yeah, in a way you can say the drop in SNR going from one version to the other is "intended" because the SNR metric is not important for lossy codec development.

If you really care so much about high SNR, use the old codec.
If you really want to understand the reasons why the SNR dropped in the newer version, study lossy codec development and LAME specifically.

Otherwise this discussion will just go in circles like it does now.
It's only audiophile if it's inconvenient.

Why does LAME 3.93 have higher SNR than 3.98?

Reply #21
Those images are supposed to convey the same thing that I try to state: SNR dropped in 3.98 compared to 3.93. What you quoted means that blind listening tests confirmed that files made with 3.93 sound better than those with 3.98, i presume.

The discussion goes circles first of all because people dont understand that if something is useless to them, it isnt imperative to be useless to everybody, and try to impose "If i see no point in this, he has no point too, so he must see that i am right and he also has no point in this".


Why does LAME 3.93 have higher SNR than 3.98?

Reply #22
If you call LAME with --verbose command line parameter, you will notice, 3.98 does some scaling of the input data. FWIW, add --scale 1.

Why does LAME 3.93 have higher SNR than 3.98?

Reply #23
Those images are supposed to convey the same thing that I try to state: SNR dropped in 3.98 compared to 3.93.
No. The mean amplitude of the signal generated by mixing the uncompressed and encoded signals, one inverted, increased. pdq pointed out how useless this is as an indication of quality a good number of posts back. Surely you understand the point of psychoacoustics? If the listener can’t hear it and it reduces bitrate, the codec can do whatever it wants to the input audio signal.

Quote
What you quoted means that blind listening tests confirmed that files made with 3.93 sound better than those with 3.98, i presume.
I see no ABX logs in the linked thread, or any other information that might enable other listeners to evaluate those posters’ claims. Boring.

Quote
The discussion goes circles first of all because people dont understand that if something is useless to them, it isnt imperative to be useless to everybody, and try to impose "If i see no point in this, he has no point too, so he must see that i am right and he also has no point in this".
Again, you are misunderstanding the purpose and mechanics of psychoacoustic modelling, despite ample opportunity to learn how it actually works, despite numerous past threads on the folly of using visual or mathematic indices as any indication of perceived quality, despite ToS8, etc.… Who’s going in circles here, again?

Why does LAME 3.93 have higher SNR than 3.98?

Reply #24
The discussion goes circles first of all because people dont understand that if something is useless to them, it isnt imperative to be useless to everybody, and try to impose "If i see no point in this, he has no point too, so he must see that i am right and he also has no point in this".

You can only expect a meaningful discussion if the participants understand the topic and reasoning. You always state that perceived quality is no concern to you, yet you link to a page where people draw haphazard conclusions from comparing green dots with ABX tests.

It's only audiophile if it's inconvenient.