IPB

Welcome Guest ( Log In | Register )

DBT Is Flawed Because Bob Stuart Says So, Split from Topic ID #11442
2Bdecided
post Aug 6 2012, 11:22
Post #1


ReplayGain developer


Group: Developer
Posts: 4945
Joined: 5-November 01
From: Yorkshire, UK
Member No.: 409



I don't think it's helpful criticising someone for not running proper DBTs of loudspeakers. As far as I know, only one or two members here have tried.

I know the loudspeaker comments were part of a longer post that had many other issues, but I don't think ABX-ing loudspeakers is a realistic thing to jump on.

Cheers,
David.
Go to the top of the page
+Quote Post
 
Start new topic
Replies
audioclaudio
post Aug 8 2012, 20:17
Post #2





Group: Members
Posts: 33
Joined: 3-December 11
Member No.: 95560



The trouble with ABX testing is always the fact if you want it to be truly objective you have to collect a large enough number of subjective test results to start with, and then apply statistics in order to filter out the unwanted subjectiveness part. This is because every individual test result always has a significant chance of being highly inaccurate, so you need lots of people to participate if you want the final conclusions to be reliable ones.
Often, if not practically always, it is too time consuming and/or too uneconomical to conduct an ABX test properly, i.e. in such way that expectation bias doesn't creep in through the back door etcetera. Experts in auditory neuroscience and psychoacoustics have gathered experimental evidence which appears to indicate the following. Humans who remember different things are perfectly capable of hearing the same sounds differently as a result of remembering different things. For example, Bob Stuart of Meridian, who has a Ph.D in neuroscience, believes that it is perfectly possible for a person to not hear a specific detail in a piece of music when it is played back on one particular system "A", then to discover this specific detail by listening to the same music again on a better, more resolving system "B" next, and then, finally, to turn back to the previous system "A" and always hear this detail on system "A" even though the detail could previously not be heard on system "A". Moreover, Bob Stuart believes rapid switching between sounds can inevitably cause humans to perceive sound objects differently.
Go to the top of the page
+Quote Post
Arnold B. Kruege...
post Aug 11 2012, 10:27
Post #3





Group: Members
Posts: 3536
Joined: 29-October 08
From: USA, 48236
Member No.: 61311



QUOTE (audioclaudio @ Aug 8 2012, 15:17) *
The trouble with ABX testing is always the fact if you want it to be truly objective you have to collect a large enough number of subjective test results to start with, and then apply statistics in order to filter out the unwanted subjectiveness part.


Wow! I've never seen so many critical mistakes in one sentence!

First off there is no intent that ABX tests be objective. The goals are that they be relevant, reliable and accurate. Tests involving listening by people are inherently subjective and that is a good thing. In fact the goal is to involve subjects (people) and in some sense be as subjective as possible.

The reason for collecting modest numbers of test results is to manage the inherent variations in subjective tests near the threshold of reliable detection. If a test necessarily involves a large number of trials, the effect being studied is probably so small that it is actually not significant to normal listening. When something is clearly audible statistical significance is easily obtained with a fairly small number of trials.

QUOTE
This is because every individual test result always has a significant chance of being highly inaccurate, so you need lots of people to participate if you want the final conclusions to be reliable ones.


Wrong. The above is only true for very small differences. Trained listeners can be very accurate and reliable if they are listening to a difference that is even just a little bit above the threshold of audibility.

One of the more obvious things about listening tests that one obtains from practical experience is that when they involve things that can be actually heard, they are generally fast and easy.

Most of the anecdotes about zillions of trials and frustrated listeners come from people who in retrospect were trying to hear things that aren't audible.
Go to the top of the page
+Quote Post
audioclaudio
post Aug 11 2012, 14:08
Post #4





Group: Members
Posts: 33
Joined: 3-December 11
Member No.: 95560



QUOTE (Arnold B. Krueger @ Aug 11 2012, 10:27) *
QUOTE (audioclaudio @ Aug 8 2012, 15:17) *
The trouble with ABX testing is always the fact if you want it to be truly objective you have to collect a large enough number of subjective test results to start with, and then apply statistics in order to filter out the unwanted subjectiveness part.


Wow! I've never seen so many critical mistakes in one sentence!

First off there is no intent that ABX tests be objective. The goals are that they be relevant, reliable and accurate. Tests involving listening by people are inherently subjective and that is a good thing. In fact the goal is to involve subjects (people) and in some sense be as subjective as possible.

The reason for collecting modest numbers of test results is to manage the inherent variations in subjective tests near the threshold of reliable detection. If a test necessarily involves a large number of trials, the effect being studied is probably so small that it is actually not significant to normal listening. When something is clearly audible statistical significance is easily obtained with a fairly small number of trials.

You basically just confirmed the whole thing I said. Because the problem with ABX testing is the fact it cannot always be used to produce truly objective results, it cannot always be considered relevant, accurate and reliable.
http://en.wikipedia.org/wiki/Objectivity_(science)
QUOTE (Arnold B. Krueger @ Aug 11 2012, 10:27) *
QUOTE
This is because every individual test result always has a significant chance of being highly inaccurate, so you need lots of people to participate if you want the final conclusions to be reliable ones.


Wrong. The above is only true for very small differences. Trained listeners can be very accurate and reliable if they are listening to a difference that is even just a little bit above the threshold of audibility.

That is only your opinion. What you call "small differences" might very well be less small to me.
QUOTE (Arnold B. Krueger @ Aug 11 2012, 10:27) *
One of the more obvious things about listening tests that one obtains from practical experience is that when they involve things that can be actually heard, they are generally fast and easy.

Most of the anecdotes about zillions of trials and frustrated listeners come from people who in retrospect were trying to hear things that aren't audible.

...Or the bias that ABX fails to eliminate is exactly what's causing the fact things aren't audible.

Go to the top of the page
+Quote Post

Posts in this topic
2 Pages V   1 2 >


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 19th April 2014 - 05:23