IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
Expert Listeners?
tima
post Jan 5 2011, 00:19
Post #1





Group: Members
Posts: 14
Joined: 15-December 10
Member No.: 86520



I've seen discussions about various studies that refer to "expert listeners" who can distinguish 320 Kbps audio from lossless. What characterizes these listeners? For example, are they able to pick out a 30 second clip at random from a collection of classical music and ABX it 10/10 times without breaking a sweat? How do they describe the differences they perceive? Do they use flowery audiophile-speak or what?
Go to the top of the page
+Quote Post
dyneq
post Jan 5 2011, 00:35
Post #2





Group: Members
Posts: 143
Joined: 29-December 05
Member No.: 26719



Could you please provide a reference to the studies you read so we can understand their context?

I can say that when it comes to evaluating lossy audio encoders the terminology here at HA often includes terms like 'pre echo' and 'smearing'. But it is not like wine, chocolate, coffee tasting where they use a standardized flavor wheel with acceptable words, etc. If you look through some of the older listening test threads you'll see some recurring terms though.

Did you read the wiki yet? It has some links to further reading on ABX, etc.
Go to the top of the page
+Quote Post
Porcus
post Jan 5 2011, 01:05
Post #3





Group: Members
Posts: 1779
Joined: 30-November 06
Member No.: 38207



QUOTE (tima @ Jan 5 2011, 00:19) *
For example, are they able to pick out a 30 second clip at random from a collection of classical music and ABX it 10/10 times without breaking a sweat?


Not 10/10, that's for sure. I would suppose it is closer to "rejecting the null hypothesis of uniform drawing".


--------------------
One day in the Year of the Fox came a time remembered well
Go to the top of the page
+Quote Post
DVDdoug
post Jan 5 2011, 03:11
Post #4





Group: Members
Posts: 2441
Joined: 24-August 07
From: Silicon Valley
Member No.: 46454



Since artifacts & compression limitations depend on program material, I'm not sure if anyone can ABX random samples 100% of the time. Most people who "bash" MP3 have never done any actual blind ABX tests... There is a LOT of nonsense in the audiophile community. Many in the audiophile community don't believe in blind testing!!!!!

Take a look at this Artifact Training Page. But, be careful.... It might be better if you can enjoy your MP3s without knowing what to listen for! biggrin.gif

I'm sure there's a bit of natural ability as well as a bit of training & experience. Some people can hear details that others can't... Some people have better hearing than others... A very few musicians have perfect pitch, and that's something that can't be learned if you don't possess the natural ability.
Go to the top of the page
+Quote Post
greynol
post Jan 5 2011, 03:34
Post #5





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



I think the chances are more likely with random sample of heavy metal for an individual with acute hearing than a random sample of classical music as it typically demands higher bitrates when using VBR and because the music is more homogeneous throughout.

This post has been edited by greynol: Jan 5 2011, 03:40


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
tima
post Jan 5 2011, 03:40
Post #6





Group: Members
Posts: 14
Joined: 15-December 10
Member No.: 86520



QUOTE (DVDdoug @ Jan 4 2011, 20:11) *
Since artifacts & compression limitations depend on program material, I'm not sure if anyone can ABX random samples 100% of the time. Most people who "bash" MP3 have never done any actual blind ABX tests... There is a LOT of nonsense in the audiophile community. Many in the audiophile community don't believe in blind testing!!!!!


Well, the motivation for my post was someone answering the "ABX challenge" in a more or less audiophile forum by presenting perfect foobar2000 results for a (claimed) randomly chosen orchestral snippet encoded at 320 Kbps (LAME 3.97, wouldn't recode with 3.98.4), which to him was easy and perfectly ordinary. I've also encountered people who claim to be able to easily distinguish 128, 160, 192, 256, and 320, from lossless and each other. Then there are those who love to say that lossy isn't even good enough for their car systems. If they provide foobar2000 results, I take them at face value, as I can't set up and monitor the tests myself, but I still can't help but wonder if "expert listeners" in controlled, monitored tests are able to routinely pull off these feats with arbitrary, ordinary music, as opposed to "killer samples". I know I can't, and I'm just curious if these claimed capabilities are real, and how common they are.
Go to the top of the page
+Quote Post
benski
post Jan 5 2011, 03:59
Post #7


Winamp Developer


Group: Developer
Posts: 669
Joined: 17-July 05
From: Ashburn, VA
Member No.: 23375



It is about knowing what to listen for and experience. In a typical 3 minute song there's probably only 2 or 3 small artifacts at 320 kbps. So, no, random 30 second samples wouldn't work, even for "expert listeners".

For the tos #8 hounders, search my post history for a successful 320kbps ABX on classical music. smile.gif
Go to the top of the page
+Quote Post
greynol
post Jan 5 2011, 04:12
Post #8





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



I searched and saw your one successful ABX attempt. In one post about it you say it's the exception. You're telling us now that it is perfectly usual for you with any track?


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
benski
post Jan 5 2011, 04:26
Post #9


Winamp Developer


Group: Developer
Posts: 669
Joined: 17-July 05
From: Ashburn, VA
Member No.: 23375



No. I'm saying that you have to try really hard and there's only a few isolated artifacts and they require a lot of attention to locate. It is impossible to be given a clip and say "oh that's 256" because its not a gradual degradation, its a change in probability of artifacts (at least at higher bitrates). I know you know this, greynol, just clarifying for the OP.

However, if/when an artifact is identified, its often easy to ABX the track. in my experience, ABX is an all or nothing deal.
Go to the top of the page
+Quote Post
Arnold B. Kruege...
post Jan 5 2011, 14:47
Post #10





Group: Members
Posts: 3536
Joined: 29-October 08
From: USA, 48236
Member No.: 61311



QUOTE (benski @ Jan 4 2011, 22:26) *
No. I'm saying that you have to try really hard and there's only a few isolated artifacts and they require a lot of attention to locate. It is impossible to be given a clip and say "oh that's 256" because its not a gradual degradation, its a change in probability of artifacts (at least at higher bitrates). I know you know this, greynol, just clarifying for the OP.


FWIW, also my experience with ABXing MP3s.

QUOTE
However, if/when an artifact is identified, its often easy to ABX the track.


Also my experience.

QUOTE
in my experience, ABX is an all or nothing deal.


That's what we designed ABX to be.

However, we often end up with mixed bags and arguable results when ABXing audio hardware. ABX's sharp edge has been mostly advantageous because reality has generally been muddy.
Go to the top of the page
+Quote Post
lvqcl
post Jan 5 2011, 14:58
Post #11





Group: Developer
Posts: 3212
Joined: 2-December 07
Member No.: 49183



QUOTE
ABX is an all or nothing deal.

So, ABX result 16/20 is less probable than 19/20 or 20/20?

added: by "less probable" I mean "less likely to obtain".

This post has been edited by lvqcl: Jan 5 2011, 15:17
Go to the top of the page
+Quote Post
Josh358
post Feb 11 2011, 02:23
Post #12





Group: Members
Posts: 74
Joined: 4-November 10
Member No.: 85212



QUOTE (DVDdoug @ Jan 4 2011, 21:11) *
A very few musicians have perfect pitch, and that's something that can't be learned if you don't possess the natural ability.


Just as a matter of interest, while it was once believed that perfect pitch was inherited, it's now known that it isn't. An adult can acquire it with about a year's systematic practice. Kids acquire it if they learn to associate the names of notes with the pitches they represent before IIRC the age of five.

A good example of how people can become more expert listeners with practice, and it raises some interesting questions about whether some people can hear distinctions that are considered inaudible on the basis of psychoacoustic testing. I'm not aware of any research that shows they can, but there are studies that show that some people are more adept than others at consistently evaluating speakers in blind tests, and that training can improve this skill.
Go to the top of the page
+Quote Post
audioskeptic
post Mar 23 2011, 11:01
Post #13





Group: Members
Posts: 1
Joined: 23-March 11
Member No.: 89233



First post here: Please explain how ABX testing relates to expert listeners as opposed to average listeners.

It seems to me that testing for the ability to hear differences in types or equipment or formats, or even testing to hear if something is "audible" is not the same as testing, let's say, in a drug trial.
In simple terms, in a drug trial you are testing for efficacy significantly above a certain threshold (something well above a placebo effect); otherwise the use of the drug can't be seen as useful.

But in listening tests, unless you are trying to determine what the "average" human is capable of, isn't there a situation where a small subset of listeners can "hear" something that typical users can't? I have a musician friend who clearly perceives aspects of classical recordings that I'm not sensitive to. So, in other words, isn't it possible for example, for ABX testing to show that there is no statistically significant audible difference between 2 pieces of equipment or 2 differently formatted recordings, yet there still exist a small number of listeners who can differentiate between them?

Thanks. Sorry for the newbie question.
Go to the top of the page
+Quote Post
db1989
post Mar 23 2011, 11:17
Post #14





Group: Super Moderator
Posts: 5157
Joined: 23-June 06
Member No.: 32180



QUOTE
isn't there a situation where a small subset of listeners can "hear" something that typical users can't?
QUOTE
isn't it possible for example, for ABX testing to show that there is no statistically significant audible difference between 2 pieces of equipment or 2 differently formatted recordings, yet there still exist a small number of listeners who can differentiate between them?
Of course!

Thatís why, in any experimental design, one must try to use a large and representative/unbiased sample (listeners) when aiming to draw general conclusions.

In most such cases, such an ABX test would be approximating the Ďaverageí listener; and for general purposes, isnít that good? Audio technology for general consumption is targeting exactly them, after all.

If the sample is not sufficiently large or is biased, the test cannot be taken as indicative of any sampling unit (listener) beyond the one/few it was applied to. Thatís not necessarily bad: If you conducted an ABX test and found that you had slightly better or worse hearing than average and could/had to adjust your encoding settings accordingly, that makes senseófor you. Equally, in designing a codec for Ďexpertí listeners, the creator would have to preferentially test that subset of people.

TL;DR: In any experiment, use sampling units representative of the population you want your results to apply to. emot-science.gif
Go to the top of the page
+Quote Post
googlebot
post Mar 23 2011, 11:23
Post #15





Group: Members
Posts: 698
Joined: 6-March 10
Member No.: 78779



Results from large "average listener" sets cannot be extrapolated with high certainty to the capabilities of possible expert listeners. The nice thing is, any person claiming to belong to the latter group can be challenged to conduct an ABX test himself. If the outcome is positive there is a high certainty that he does, if not otherwise.
Go to the top of the page
+Quote Post
Arnold B. Kruege...
post Mar 23 2011, 13:39
Post #16





Group: Members
Posts: 3536
Joined: 29-October 08
From: USA, 48236
Member No.: 61311



QUOTE (audioskeptic @ Mar 23 2011, 06:01) *
First post here: Please explain how ABX testing relates to expert listeners as opposed to average listeners.


Is that really a question? You seem to be offering a declaration of your answer to it below:

QUOTE
It seems to me that testing for the ability to hear differences in types or equipment or formats, or even testing to hear if something is "audible" is not the same as testing, let's say, in a drug trial.


Drug testing is usually thought to be related to a passive activity, while participation in a listening test is an activity that presumes active engagement. So, yes they are different.

QUOTE
In simple terms, in a drug trial you are testing for efficacy significantly above a certain threshold (something well above a placebo effect); otherwise the use of the drug can't be seen as useful.


In a listening test we are in a likewise manner testing for audibility that is significantly above a certain threshold.

QUOTE
But in listening tests, unless you are trying to determine what the "average" human is capable of, isn't there a situation where a small subset of listeners can "hear" something that typical users can't?


IME listening tests by definition involve people who at least think that they are capable of hearing things that casual listeners don't necessarily notice. Inevitably, listening tests involve people who think that they can hear differences that most people don't bother with. This is a point where the comparisons with drug testing break down quite badly. People aren't interested in medicines that require special perceptual skills to obtain a benefit from. They want to receive the treatment and have the healing commence forthwith.

QUOTE
I have a musician friend who clearly perceives aspects of classical recordings that I'm not sensitive to.


You are mixing apples and oranges. Musical differences and sound quality differences are *not* the same. The parts of the brain that are involved with each are different. Musicians even have a fairly bad reputation for hearing audible differences due to technical changes because they tend to get distracted from technical differences by the musical issues.

QUOTE
So, in other words, isn't it possible for example, for ABX testing to show that there is no statistically significant audible difference between 2 pieces of equipment or 2 differently formatted recordings, yet there still exist a small number of listeners who can differentiate between them?


All things are possible, but not all observations are helpful.

Sure, its possible for an ABX test involving one group of listeners to produce a null result, when another similar ABX test involving a different group of listeners produces a positive result.

Given that there is a well-known and highly recommended activity called "listener training", which tends to improve the sensitivity of listeners, we are pretty sure that we can contrive things so that things like this are observed.

One of the goals of setting up a good listening test is to get the most sensitive possible results.

This is just Science. All findings of Science are provisional. They are only relevant until some better and more reliable finding is obtained.

But let's put this into perspective. At this time 100's and even 1,000's or 10,000's of listening tests have been done. We know how to relate the measured performance of audio products with the reliable findings of the most sensitive listeners we can train up. Many of the technical differences that some people claim to hear are bullocks in the sense that they are far, far smaller than the differences that even the most highly trained listeners are able to reliably hear. OTOH, there are differences that are relatively easy for any reasonably well-trained listener to hear. The trick is knowing which is which. ;-)
Go to the top of the page
+Quote Post
krabapple
post Mar 23 2011, 23:55
Post #17





Group: Members
Posts: 2159
Joined: 18-December 03
Member No.: 10538



QUOTE (tima @ Jan 4 2011, 21:40) *
QUOTE (DVDdoug @ Jan 4 2011, 20:11) *
Since artifacts & compression limitations depend on program material, I'm not sure if anyone can ABX random samples 100% of the time. Most people who "bash" MP3 have never done any actual blind ABX tests... There is a LOT of nonsense in the audiophile community. Many in the audiophile community don't believe in blind testing!!!!!


Well, the motivation for my post was someone answering the "ABX challenge" in a more or less audiophile forum by presenting perfect foobar2000 results for a (claimed) randomly chosen orchestral snippet encoded at 320 Kbps (LAME 3.97, wouldn't recode with 3.98.4), which to him was easy and perfectly ordinary.


I'd say that person is mistaken, fibbing, or using some weird software/hardware combo. Even the few here with hearing (and training) exceptional enough to be able to ABX 320 from lossless, don't claim they'd be able to do it with a randomly-chosen sample of music (and classical is not necessarily the ''hardest' music to encode).

Go to the top of the page
+Quote Post
Canar
post Mar 24 2011, 00:14
Post #18





Group: Super Moderator
Posts: 3327
Joined: 26-July 02
From: princegeorge.ca
Member No.: 2796



Hi! I've ABXed 320 from lossless repeatedly. The eig, castanets, and fatboy samples have all fallen before my "ABX skills". Universally, all fail due to pre-echo issues. Sharp attacks just aren't sharp enough. It's not obvious at all; if I didn't have lossless to compare against, I probably wouldn't even notice. I might be able to do this with other samples, but... I'd really rather not ruin lossy encoding for myself entirely.

I'd be quite surprised to hear that someone can ABX an arbitrary orchestral sample at 320 vs FLAC on any kind of reliable basis. But... weirder things have happened. /mnt, here on these forums, ABXed a statistically-significant chunk of certain albums, for example.

This post has been edited by Canar: Mar 24 2011, 00:19


--------------------
∑:<
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 20th April 2014 - 10:10