IPB

Welcome Guest ( Log In | Register )

Biophysics, Limitations of Shannon and Issues with ABX Testing, Split from Topic ID: 57406 (TOS #5)
theorist1
post Jun 14 2013, 04:32
Post #1





Group: Members
Posts: 4
Joined: 14-June 13
Member No.: 108646



As a scientist (biophysics) and an audiophile, I have my feet in both worlds, and have thus enjoyed this discussion. Here are some thoughts:

I. Open Access.

Several have written it’s unfortunate that this paper, which is of general public interest, is not available to those without an AES subscription (that includes myself). Agreed; and note that the landscape for scientific publication has been changing: researchers are increasingly realizing the value of “open access” papers – ones available freely, without a subscription. Some scientific journals have gone entirely open access, while others (for an additional fee) offer the option of open access publication of your paper. Let me suggest that, if AES doesn’t offer this option, it should, and that those publishing papers of broad interest should endeavor (if financially possible) to avail themselves of this.


II. ABX testing.

I assume from the discussion that this paper makes use of sequential ABX testing (by “sequential” I mean: present A, then B, then X, in sequence). I’d like to see basic research done into sequential ABX testing itself. In particular, sequential ABX testing (double-blinding will be assumed throughout this discussion) is highly regarded for assessing perceptual discrimination, because (with proper statistical analysis) it eliminates false positives – subjects can’t “pass” (i.e., achieve successful discrimination in) a sequential ABX test unless they truly can reliably distinguish A from B. But what about the converse? Achieving accurate discrimination in sequential ABX testing is hard, because it requires accurate memory of both A and B when judging X. Therefore I’d like to offer the hypothesis that subjects can “fail” sequential ABX testing even when they can successfully discriminate between A and B, and suggest a way of testing this. Essentially, one would use simultaneous ABX testing to eliminate the memory requirement, and then compare the results to those for sequential ABX testing. I can’t think of how you’d do simultaneous ABX testing with hearing, but it could certainly be done with vision (using color tiles). The experiment would be divided into three phases:

1) Find the smallest color difference that can be reliably distinguished during simultaneous ABX testing (present all three tiles – A, B, and X – simultaneously, so that the subjects can line them up next to each other for comparison).
2) Repeat the experiment, using that same color difference, with sequential ABX testing.
3) If the subjects fail phase 2 (thus confirming my hypothesis), find the smallest color difference that can be reliably distinguished using sequential ABX testing, and compare to the results of phase 1.

If it is found that subjects indeed fail to reliably distinguish colors when presented sequentially that they can reliably distinguish when presented simultaneously, this will suggest that sequential ABX testing may not be a good method for assessing our perceptual limits (i.e., for determining transparency) -- and not just for vision, but possibly for hearing as well.


III. High-resolution audio and the Shannon sampling theorem.

Here I’ll apologize in advance if my questions are na´ve – I’m no expert at signal processing. Anyways:

Those who are dismissive of high-res audio based on theory alone typically cite the Shannon sampling theorem (typically misattributed to Nyquist), noting correctly that we can’t hear over 20 kHz (if that), and that we only need >40 kHz sampling to accurately reproduce this. But 2 x max. frequency is not the theorem’s only requirement. It’s my understanding that it also assumes an infinite signal, perfect sampling, and perfect interpolation. I’ve never seen any of these assumptions discussed in this context, so I’d like to ask how much practical effect these requirements would have on Redbook (16 bit/44.1 kHz) vs. high-res conversion:

1) Infinite signal. I assume we can effectively satisfy this with signal length >> 1/frequency, and that this is thus a non-issue.

2) Perfect sampling. Clearly, sampling need not be perfect, but simply close enough to perfect to be transparent in amplitude and time. Timing errors lead to jitter (right?). So (and this is an engineering question): what’s the relationship between sampling rate and how easy it is to eliminate audible jitter errors?

3) Perfect interpolation. Another engineering question: Naively, unless implementing near-perfect (i.e., transparent) interpolation is trivial, I would think it would be easier to achieve transparent interpolation with a higher sampling rate, because the points are more closely spaced. Is transparent interpolation so easily achievable that the sampling rate has no practical effect?


IV. The effect of mastering.

Many have mentioned that the reason high-res disks do indeed sound better than CDs of the same performance is that they are mastered differently – as labors of love, and without the usual commercial pressures to alter the sound. Given this, I think it would be a pubic service if someone could produce a Blu-Ray disk corresponding to the songs tested in this study, containing both the high-res and Redbook versions of each, and make it available for sale. That way people could easily experiment for themselves.


V. General thoughts.

I think the reason for the continued controversy about digital audio performance is that we don’t completely understand the biophysics of human hearing (which is why it continues to be an active area of research). If we did, we would know, a priori, what constitutes a complete specification set sufficient to determine transparency, and thus could engineer transparent electronic gear (I say electronic because I am excluding transducers) without listening to it. To the best of my understanding, this is not yet the case, since the errors that our auditory system is capable of detecting can be extraordinarily subtle, and what would constitute a complete set of scalar specifications sufficient to ensure transparency thus remains an open question.

This post has been edited by greynol: Jun 14 2013, 14:02
Reason for edit: Added link to original discussion from which this one was split for being off-topic.
Go to the top of the page
+Quote Post
 
Start new topic
Replies
greynol
post Aug 5 2013, 04:33
Post #2





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



How can something that serves to make distinguishing two samples more difficult also serve as a tell?

It doesn't.

ABX testing, as I know it, hands the power of switching to the testee. A click will only be a tell if it occurs for one sample and not the other for both the unknown X and the corresponding known (A or B, but not both). That a click occurs as the result of switching is meaningless in alerting the testee to the switch since the testee initiated it.

This post has been edited by greynol: Aug 5 2013, 04:55


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
mzil
post Aug 6 2013, 17:15
Post #3





Group: Members
Posts: 422
Joined: 5-August 07
Member No.: 45913



QUOTE (greynol @ Aug 4 2013, 23:33) *
How can something that serves to make distinguishing two samples more difficult also serve as a tell?

When the click noise has not been verified, pre-test, to be uniform in its sound quality (and/or timing) when transitioning to either A or B; then it can act as a tell.

Arnold B. Krueger mentions this problem can exist in hardware ABX testing and personally being able to reliably distinguish the sound of the QSC ABX switcher transitions, at least in the absence of music playing, here. He also mentions it occasionally can be an issue with PCABX (at least when using certain computers) in that post.

A slightly different but equally problematic issue is when the click sound doesn't immediately identify the signal itself, however is does indicate to the listener if the transition was to the same signal vs. the alternate signal [For example, the artifact click sound switching from A to X, where X = A, vs. the sound of A to X where instead X = B].

Kees de Visser, who started that thread, mentions a difference in the click sounds (and gives some examples for us to listen to) in software ABX testing, in this post. His belief at the time, if I understood correctly, was that it was random in nature [a good thing] however his response to a question here seemed to suggest otherwise to me.

The unrelated reason why we ideally don't want clicks at all, is because the presence of any other obtrusive sound, during the transition period, be it a blast from a bull horn, an annoying loud beep, or even just a loud click/thump can muck up our short term memory for things like loudness (or I would assume, also tonality). This I would assume is rather self evident. Sorry if I wasn't more clear about the "tell" part.
Go to the top of the page
+Quote Post
Kees de Visser
post Aug 22 2013, 08:59
Post #4





Group: Members
Posts: 613
Joined: 22-May 05
From: France
Member No.: 22220



QUOTE (mzil @ Aug 6 2013, 18:15) *
Kees de Visser, who started that thread, mentions a difference in the click sounds (and gives some examples for us to listen to) in software ABX testing, in this post. His belief at the time, if I understood correctly, was that it was random in nature [a good thing] however his response to a question here seemed to suggest otherwise to me.
Sorry for being late. The artifacts I found are indeed random. They also appear when switching between identical audio, which is strange since this is a trivial task and should be 100% lossless, e.g. with a simple linear fade. The artifacts I was/am worried about will only appear when switching between different audio streams, which is not a lossless process, not a trivial task and might be audible, depending on many variables.
Hope this helps.
Go to the top of the page
+Quote Post
Woodinville
post Aug 24 2013, 13:11
Post #5





Group: Members
Posts: 1401
Joined: 9-January 05
From: JJ's office.
Member No.: 18957



QUOTE (Kees de Visser @ Aug 22 2013, 00:59) *
QUOTE (mzil @ Aug 6 2013, 18:15) *
Kees de Visser, who started that thread, mentions a difference in the click sounds (and gives some examples for us to listen to) in software ABX testing, in this post. His belief at the time, if I understood correctly, was that it was random in nature [a good thing] however his response to a question here seemed to suggest otherwise to me.
Sorry for being late. The artifacts I found are indeed random. They also appear when switching between identical audio, which is strange since this is a trivial task and should be 100% lossless, e.g. with a simple linear fade. The artifacts I was/am worried about will only appear when switching between different audio streams, which is not a lossless process, not a trivial task and might be audible, depending on many variables.
Hope this helps.


Two things come to mind:

1) bad time alignment, although with 2 identical files that would seem unlikely
2) bad crossfade window design

If the click when switching between A/X is more noticible than B/X, you can be sure subjects will latch on to that.


--------------------
-----
J. D. (jj) Johnston
Go to the top of the page
+Quote Post
Arnold B. Kruege...
post Aug 25 2013, 01:21
Post #6





Group: Members
Posts: 3535
Joined: 29-October 08
From: USA, 48236
Member No.: 61311



QUOTE (Woodinville @ Aug 24 2013, 08:11) *
QUOTE (Kees de Visser @ Aug 22 2013, 00:59) *
QUOTE (mzil @ Aug 6 2013, 18:15) *
Kees de Visser, who started that thread, mentions a difference in the click sounds (and gives some examples for us to listen to) in software ABX testing, in this post. His belief at the time, if I understood correctly, was that it was random in nature [a good thing] however his response to a question here seemed to suggest otherwise to me.
Sorry for being late. The artifacts I found are indeed random. They also appear when switching between identical audio, which is strange since this is a trivial task and should be 100% lossless, e.g. with a simple linear fade. The artifacts I was/am worried about will only appear when switching between different audio streams, which is not a lossless process, not a trivial task and might be audible, depending on many variables.
Hope this helps.


Two things come to mind:

1) bad time alignment, although with 2 identical files that would seem unlikely
2) bad crossfade window design

If the click when switching between A/X is more noticeable than B/X, you can be sure subjects will latch on to that.


That's for sure! One good test is to run an ABX test with no music playing. If there are switching artifacts or background noises that relate to one alternative but not the other. a careful listener can detect them and do pretty well!

While none of the switchboxes that were sold by the ABX company had this problem, I did encounter a sample of a competitive product that I could score 16/16 with, with nothing attached to it at all.

This sort of problem can be inherent in the switchbox, or it can be due to an error in the setup of the test.

It is also problem to have noises that are truly random. They are a less severe problem but if noticeable enough they can distract the listener and reduce the probability of reliable detection when it is possible.
Go to the top of the page
+Quote Post

Posts in this topic
- theorist1   Biophysics, Limitations of Shannon and Issues with ABX Testing   Jun 14 2013, 04:32
- - pdq   I don't know what you mean by "sequential...   Jun 14 2013, 05:17
|- - theorist1   QUOTE (pdq @ Jun 13 2013, 20:17) I don...   Jun 14 2013, 05:34
|- - greynol   QUOTE (theorist1 @ Jun 13 2013, 21:34) Th...   Jun 14 2013, 06:15
|- - 2Bdecided   QUOTE (greynol @ Jun 14 2013, 06:15) QUOT...   Jun 14 2013, 09:38
|- - Martel   QUOTE (2Bdecided @ Jun 14 2013, 10:38) Ex...   Aug 6 2013, 17:20
- - Kees de Visser   QUOTE (theorist1 @ Jun 14 2013, 05:32) (b...   Jun 14 2013, 10:33
- - greynol   I hate to do this, since drawing allusions to sigh...   Jun 14 2013, 13:55
- - drewfx   QUOTE (theorist1 @ Jun 13 2013, 23:32) Ac...   Jun 14 2013, 17:54
|- - theorist1   Thanks for all your comments! QUOTE (greynol ...   Jun 15 2013, 03:11
|- - theorist1   QUOTE ([JAZ] @ Jun 14 2013, 09:32...   Jun 15 2013, 03:12
||- - [JAZ]   QUOTE (theorist1 @ Jun 15 2013, 04:12) So...   Jun 15 2013, 16:43
||- - greynol   QUOTE (theorist1 @ Jun 15 2013, 04:12) So...   Jun 17 2013, 07:00
|- - krabapple   QUOTE (theorist1 @ Jun 14 2013, 22:11) ly...   Jun 20 2013, 11:17
- - [JAZ]   I will try to address the theory and reasoning of ...   Jun 14 2013, 18:32
|- - pdq   QUOTE ([JAZ] @ Jun 14 2013, 13:32...   Jun 14 2013, 18:44
||- - splice   QUOTE (pdq @ Jun 14 2013, 09:44) ... The ...   Jun 15 2013, 00:10
|- - phofman   QUOTE ([JAZ] @ Jun 14 2013, 19:32...   Jun 16 2013, 19:49
- - Woodinville   QUOTE (theorist1 @ Jun 13 2013, 20:32) If...   Jun 15 2013, 07:16
- - Porcus   There is a sampling/interpolation/hearing issue th...   Jun 15 2013, 18:38
|- - Woodinville   QUOTE (Porcus @ Jun 15 2013, 10:38) Anywa...   Jun 16 2013, 07:34
|- - Porcus   QUOTE (Woodinville @ Jun 16 2013, 08:34) ...   Jun 17 2013, 10:53
|- - Woodinville   QUOTE (Porcus @ Jun 17 2013, 02:53) A 19 ...   Jun 18 2013, 03:10
|- - Porcus   QUOTE (Woodinville @ Jun 18 2013, 04:10) ...   Jun 18 2013, 10:41
|- - knutinh   QUOTE (Porcus @ Jun 18 2013, 10:41) Now i...   Jun 18 2013, 13:01
||- - Porcus   QUOTE (knutinh @ Jun 18 2013, 14:01) It a...   Jun 18 2013, 23:26
||- - knutinh   QUOTE (Porcus @ Jun 18 2013, 23:26) QUOTE...   Jun 19 2013, 06:24
|- - Woodinville   QUOTE (Porcus @ Jun 18 2013, 02:41) Now i...   Jun 20 2013, 06:26
- - knutinh   Scientific curiosity aside: If it was found that s...   Jun 17 2013, 07:44
|- - Porcus   QUOTE (knutinh @ Jun 17 2013, 08:44) If [...   Jun 17 2013, 09:56
|- - knutinh   QUOTE (Porcus @ Jun 17 2013, 09:56) Long-...   Jun 18 2013, 10:19
- - antz   QUOTE (theorist1 @ Jun 14 2013, 04:32) If...   Aug 4 2013, 13:46
|- - greynol   QUOTE (theorist1 @ Jun 14 2013, 04:32) If...   Aug 4 2013, 18:31
|- - Woodinville   QUOTE (antz @ Aug 4 2013, 05:46) The fact...   Aug 4 2013, 21:42
|- - antz   QUOTE (Woodinville @ Aug 4 2013, 21:42) Q...   Aug 4 2013, 22:14
- - Light-Fire   QUOTE (theorist1 @ Jun 13 2013, 23:32) .....   Aug 4 2013, 17:43
- - mzil   QUOTE And what level-roving experiments show is th...   Aug 4 2013, 22:54
|- - Woodinville   QUOTE (mzil @ Aug 4 2013, 14:54) however ...   Aug 5 2013, 03:22
- - greynol   How can something that serves to make distinguishi...   Aug 5 2013, 04:33
|- - mzil   QUOTE (greynol @ Aug 4 2013, 23:33) How c...   Aug 6 2013, 17:15
|- - Kees de Visser   QUOTE (mzil @ Aug 6 2013, 18:15) Kees de ...   Aug 22 2013, 08:59
|- - Woodinville   QUOTE (Kees de Visser @ Aug 22 2013, 00:5...   Aug 24 2013, 13:11
|- - Arnold B. Krueger   QUOTE (Woodinville @ Aug 24 2013, 08:11) ...   Aug 25 2013, 01:21
- - 2Bdecided   Two mono samples, A and B. sequential: Listen to ...   Aug 5 2013, 10:46
- - Wayne Highwood   In light of the limits of serial audio testing rai...   Aug 5 2013, 19:14
|- - Arnold B. Krueger   QUOTE (Wayne Highwood @ Aug 5 2013, 14:14...   Aug 13 2013, 01:38
- - greynol   I see a distinction without a difference. Subject...   Aug 5 2013, 19:23
|- - Wayne Highwood   QUOTE (greynol @ Aug 5 2013, 12:23) I see...   Aug 5 2013, 19:59
- - 2Bdecided   So you are exploring the possibility that the reas...   Aug 5 2013, 20:59
|- - Wayne Highwood   QUOTE (2Bdecided @ Aug 5 2013, 13:59) So ...   Aug 5 2013, 22:05
|- - saratoga   QUOTE (Wayne Highwood @ Aug 5 2013, 17:05...   Aug 5 2013, 22:27
- - 2Bdecided   People who look for reasons to doubt DBTs generall...   Aug 6 2013, 10:15
|- - Wayne Highwood   QUOTE (2Bdecided @ Aug 6 2013, 03:15) Peo...   Aug 6 2013, 16:47
|- - [JAZ]   QUOTE (Wayne Highwood @ Aug 6 2013, 17:47...   Aug 6 2013, 17:45
|- - 2Bdecided   QUOTE (Wayne Highwood @ Aug 6 2013, 16:47...   Aug 6 2013, 18:16
|- - Wayne Highwood   I just hope you guys in the industry don't end...   Aug 6 2013, 21:13
|- - Wayne Highwood   QUOTE (2Bdecided @ Aug 6 2013, 11:16) * -...   Aug 15 2013, 00:31
|- - greynol   QUOTE (Wayne Highwood @ Aug 14 2013, 16:3...   Aug 15 2013, 00:45
|- - Woodinville   QUOTE (Wayne Highwood @ Aug 14 2013, 16:3...   Aug 15 2013, 01:01
||- - Wayne Highwood   QUOTE (Woodinville @ Aug 14 2013, 18:01) ...   Aug 15 2013, 14:39
||- - Woodinville   QUOTE (Wayne Highwood @ Aug 15 2013, 06:3...   Aug 17 2013, 04:57
|- - Arnold B. Krueger   QUOTE (Wayne Highwood @ Aug 14 2013, 19:3...   Aug 22 2013, 13:14
- - greynol   The point is that ABX is useful for more than the ...   Aug 15 2013, 15:08
|- - Wayne Highwood   QUOTE (greynol @ Aug 15 2013, 08:08) The ...   Aug 15 2013, 16:21
|- - saratoga   QUOTE (Wayne Highwood @ Aug 15 2013, 11:2...   Aug 15 2013, 19:26
- - greynol   I live on the West Coast so there's still time...   Aug 17 2013, 01:10


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 17th April 2014 - 19:53