graph shows 320 kbps LAME 'never achieves transparency'? Options

[krabapple]

Someone is waving this in my face at another forum,after I asserted that people who can tell modern 320kbs LAME encodes from source in ABX constitute the a tiny minority of listeners, and even they may require 'killer samples' , rather than being able to do it all the time, e.g., with a random mix of music.

You talk about LAME. Let's look at that: http://wiki.hydrogenaudio.org/index.php?title=Lame

Here is a graph from that page:


It shows that it *never* achieves transparency regardless of data rate.

I called bullsh*t on that rhetoric (e.g., on the very same wiki page, we see "-V0 (~245 kbps), -V1 (~225 kbps), -V2 (~190 kbps) or -V3 (~175 kbps) are recommended. These settings will normally produce transparent encoding (transparent = most people can't distinguish the MP3 from the original in an ABX blind test). Audible differences between these presets exist, but are rare. "

IMO, this graph is not evidence that 320 'never achieves transparency'. I know some people can ABX 320kbps. That is not the issue -- I have never claimed 320 is transparent to EVERYONE, all the time. Just that it's a rare ability, typically requiring training to hear specific artifacts, and possibly requiring carefully-chosen samples to reveal them.

The graph also lacks error/variation bars -- where's from, btw?

Dude (who is a former colleague of JJ) refuses to come over here and argue his case. I told him I'd bring it here for him. What say you, LAME developers and users?


full post:
http://www.avsforum.com/avs-vb/showpost.ph...p;postcount=348

This post has been edited by krabapple: Aug 26 2011, 19:33

[[JAZ]]

Either you, or the guys that told you about that graph missed the point.

The graph has one single use: To describe that when increasing the bitrate, the quality increases in smaller steps.
Or said it in other words: Doubling the bitrate does not mean doubling the quality.

Also, nowhere in the graph talks about transparency. It talks about quality.
Except hybrid codecs, or codecs designed specifically for such thing (which are rare), lossy codecs are not lossless at their highest setting.
As such, quality can't be 100% with any setting.

If you want to add there the transparency point, it would be somewhere around 8, but killer samples tend to disagree.


Addenum: Note that the graph is not based on math. The graph could be considered as "the average/usual case". That's another reason not to make more conclusions than those mentioned.

This post has been edited by [JAZ]: Aug 26 2011, 19:34

[krabapple]

So, what is the definition of 'quality' being used in the context of that graph? Numerical identity with source? Likelihood of being ABX-able from source?

(and where's the graph from originally?)

This post has been edited by krabapple: Aug 26 2011, 19:42

[saratoga]

So, what is the definition of 'quality' being used in the context of that graph? Numerical identity with source? Likelihood of being ABX-able from source?

How many arbitrary units of quality the file has.

[C.R.Helmrich]

IMO, this graph is not evidence that 320 'never achieves transparency'.

Completely agree. The graph is missing at least 4 important things:

• Confidence intervals around all data points
• Mean and confidence interval of the hidden reference, since most of the time that reference does not end up getting quality 10!
• a description of what exactly is meant by "perceived listening quality", i.e. the y-axis label
• a description of the test material used to obtain those quality results.

As such, quality can't be 100% with any setting.

Why not? Full perceptual transparency for every listener in the universe implies 100% quality.

If you want to add there the transparency point, it would be somewhere around 8...

Why? Or rather, were did you read that? In MUSHRA tests, 80% and above means "excellent/broadcast quality". Transparency, though, is when the confidence intervals of a codec and the hidden reference overlap. That's the reason why the hidden reference is needed in that plot.

Chris

This post has been edited by C.R.Helmrich: Aug 26 2011, 19:55

[greynol]

Also, nowhere in the graph talks about transparency. It talks about quality.

More specifically, it says "perceived listening quality". As such it is not unreasonable to assume from the graph that mp3 never reaches transparency.

This post has been edited by greynol: Aug 26 2011, 20:09

[krabapple]

Also, nowhere in the graph talks about transparency. It talks about quality.

More specifically, it says "perceived listening quality". As such it is not unreasonable to assume from the graph that mp3 never reaches transparency.

In which case I wonder if the labeling of the graph isn't just misleading. What is meant by 'quality' on that graph, not to mention 'transparency'? What does '10' represent on the graph? Are these not aggregate results of ABC/hr tests? Surely those points have confidence intervals associated with them?

Where does this graph come from??

This post has been edited by krabapple: Aug 26 2011, 22:19

[C.R.Helmrich]

Exactly what I said.

Someone called Jan: http://wiki.hydrogenaudio.org/index.php?ti...ame-chart-2.png

Chris

[lvqcl]

Where does this graph come from??

http://www.hydrogenaudio.org/forums/index....2288&st=100

[krabapple]

apparently this, to be precise

http://www.hydrogenaudio.org/forums/index....st&p=329974


I suppose I'll have to read that whole thread to find out why in hell this was done:

Arbitrarily, I'd give the following quality levels:

--abr 56: 3
--abr 90: 5
-V5: 7
-V4: 8
-V3:8.5
-V2: 8.7
-V0: 9.1
-b 320: 9.2

and what 'XLS' is.


And why the HA wiki for LAME is using a cryptic graph about an old version of the codec, to inform the public. (Don't worry, I know the answer -- ITS A WIKI, IF YOU DON'T LIKE IT, FIX IT)

This post has been edited by krabapple: Aug 26 2011, 22:42

[db1989]

Yeah, it seems at best rather useless and at worst misleading. If it’s trying to convey that there are diminishing returns of perceived quality with increasing bitrate, that seems much better—and easier—expressed with words. Hey, I just did it!

And why the HA wiki for LAME is using a cryptic graph about an old version of the codec, to inform the public. (Don't worry, I know the answer -- ITS A WIKI, IF YOU DON'T LIKE IT, FIX IT)

I’ll happily remove it if that’s the general desire.

[lvqcl]

and what 'XLS' is

MS Excel spreadsheet

[krabapple]

and what 'XLS' is

MS Excel spreadsheet

LOL. I should've seen that coming.


OK, whose? (hovers over link) ..ok, it's Synthetic Soul's, but I can't locate the original context in which he posted the graph. The plot thickens...

This post has been edited by krabapple: Aug 26 2011, 23:17

[krabapple]

Dude still won't post here, but does have this to say:

The graph only has one meaning. That increasing bit rate never achieves transparency.

[db1989]

Is the objection limited solely to a somewhat misleading graph and a blatant misinterpretation thereof, or is amirm railing against MP3 using a line of reasoning that has not been subjected to a double-blind test? Confidence of pronouncement often varies inversely with willingness to proffer evidence, something that should be abundant and close to hand if the claimant is correct. Ironic!

[MichaelW]

The graph only has one meaning. That increasing bit rate never achieves transparency.

That is the sort of thing a trial lawyer or a fundamentalist would say. No point arguing, he's hooked up on a reading of an old and imprecise graph, and won't change what he says.

Or, as they say, can't tell if troll or really stupid.

[krabapple]

Is the objection limited solely to a somewhat misleading graph and a blatant misinterpretation thereof, or is amirm railing against MP3 using a line of reasoning that has not been subjected to a double-blind test? Confidence of pronouncement often varies inversely with willingness to proffer evidence, something that should be abundant and close to hand if the claimant is correct. Ironic!

Well, he also now says I'm not conveying his beliefs correctly, but here goes (feel free to wade through that thread if you want to):

Apparently (to me) he believes I claim LAME 320kbps CBR or 192kbps VBR mp3s are transparent universally (to everyone, all the time). Apparently he thinks the graph shows mp3s can never be transparent. You see the artificial gulf there?


First belief: simply wrong. I never claimed that anywhere, and never would. Saying 100 random listeners using random tracks would probably fail an ABX of 320kbps vs source isn't saying that everyone always would. It's just saying 'that's how damn good LAME/320kbps is'

Second belief: hinges on the strictest definition of 'transparent', and puts a burden on that graph that it wasn't meant to bear. Misses the forest for the trees.


IMHO. He thinks you guys are beating me up here, so what do I know? ;>

This post has been edited by krabapple: Aug 29 2011, 00:50

[saratoga]

IMHO. He thinks you guys are beating me up here, so what do I know? ;>

amirm failed spectacularly at comprehending that example chart. That makes him at least clueless about codec testing, and probably stupid as well since its not really that hard to understand if you can use words and understand ideas. (my apologies if he simply does not speak good english and so could not read the chart properly) Hes arguing about codec testing. You know hes ignorant about codec testing. Why are you even having an argument with him?

Point out hes too ignorant to have useful ideas about this and move on.

[Notat]

What is the definition of "transparent"?

[greensdrive]

I am probably incorrect, so please inform me if so.

I was thinking about the definition of transparency as well. I assumed "transparency" meant something like a lossy encode being audibly equivalent to a lossless encode, according to human listeners. on the other hand, I also assumed "perceived listening quality" is how good it sounds according to human listeners.

in either case of what I thought, they were basically the same thing: an opinion of the listener.

[Notat]

Which listener(s)? What source material? How are they compared?

[LithosZA]

I think by full 'Quality' on the graph they mean that it will never be bit exact quality from the source file. You never will be guaranteed that it will be lossless even if you pump up the bitrate to 320Kbit/s.
Transparent might be the wrong word. Transparent means that is 'sounds' transparent to the user.
Even though 192Kbit/s or even 320Kbit/s Vorbis sounds 'transparent' to me doesn't mean that is is full 'Quality'.

The graph probably wants to show that Lossy != Lossless.

[dhromed]

The graph probably wants to show that Lossy != Lossless.

The graph wants to show that quality and bitrate (or "quality setting") do not increase in perfect tandem with eachother, as stated earlier in this very thread. This is hardly news for anyone who's ever compressed any sort of media file.

[greynol]

I think by full 'Quality' on the graph they mean that it will never be bit exact quality from the source file.

...not if one is to interpret the legend literally!

As I said already:

it says "perceived listening quality"

The legend is blatantly incorrect, of course.

Regarding what the other guy is saying from the other forum, who cares. He can either come here and argue or you guys can go there. I do not think it is appropriate for us host an argument by proxy.

This post has been edited by greynol: Aug 29 2011, 17:22

[krabapple]

Hey, I agree argument by proxy is lame. Amir (who claims to be an EXPERT on lossy codecs, via his Microsoft/JJ connection) offered up the HA wiki graph as proof of his claim. I told him he really should check his interpretation with the folks who actually sponsor the page, because I thought he was misinterpreting it and I thought the graph itself had some problems, like the lack of 'error' bars (not expecting him to just take my word for it). Anyway, he's got a link to this thread, I hope he's reading along and learning from it.

As for transparency, meaning of, HA's own knowledgebase has this to say:

http://wiki.hydrogenaudio.org/index.php?title=Transparency

This post has been edited by krabapple: Aug 29 2011, 19:54