IPB

Welcome Guest ( Log In | Register )

2 Pages V   1 2 >  
Reply to this topicStart new topic
How do perform a controlled listening test at home
Rich B
post Nov 16 2013, 11:36
Post #1





Group: Members
Posts: 34
Joined: 16-November 13
Member No.: 112247



Hi all,

I want to learn how to perform a bias-controlled test at home. Are there step-by-step tutorials that can guide amateurs? I would like to train myself how to do this so that I can apply this knowledge in other scenarios. I have friends who have bought expensive gear who have challenged me to prove that they can't hear differences between equipment. They would entertain a listening test in their own homes.

I would like to set the record straight. But first I need to learn how to set up a test for myself. If I can't test this for myself then it's not worth the hassle. I have equipment at home (separate dacs and amplifiers) and I can hear differences quite easily between them, but I also know the listening test is not reliable. I want the wool to be pulled over my eyes and to have the knowledge to put these claims to rest.

Any help or assistance in learning would be most appreciated. Thanks!

This post has been edited by Rich B: Nov 16 2013, 11:36
Go to the top of the page
+Quote Post
Destroid
post Nov 16 2013, 11:53
Post #2





Group: Members
Posts: 544
Joined: 4-June 02
Member No.: 2220



In an scientifically objective audio listening test, the listener is unaware of the circumstances (i.e. DAC 1 or 2) in random multiple succession to eliminate the guessing-factor. If you have a a second party to facilitate the changes and record (and then those results if beyond the the guess-factor of >85% [:unsure]) may establish the listener detects an audible difference.

However, a "bias-controlled test?" I will defer to the experts... wink.gif

This post has been edited by Destroid: Nov 16 2013, 12:00


--------------------
"Something bothering you, Mister Spock?"
Go to the top of the page
+Quote Post
Rich B
post Nov 16 2013, 12:07
Post #3





Group: Members
Posts: 34
Joined: 16-November 13
Member No.: 112247



I need to learn how to conduct a listening test so that it can reveal whether differences are heard or not. I assumed the test needed to be bias-controlled in order for the results to be meaningful.
Go to the top of the page
+Quote Post
greynol
post Nov 16 2013, 12:14
Post #4





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



Check this topic which is pinned to the top of the general audio forum:
http://www.hydrogenaudio.org/forums/index....showtopic=16295

This post has been edited by greynol: Nov 16 2013, 12:14


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
Rich B
post Nov 16 2013, 12:15
Post #5





Group: Members
Posts: 34
Joined: 16-November 13
Member No.: 112247



QUOTE ("Destroid")
However, a "bias-controlled test?" I will defer to the experts..


I've been reading up on bias, so I assumed the test must be controlled for that. I'm not asking for lab conditions, but how would I go about testing the claims in my own home? If I was alone, and I wanted to devise a test, how would I go about it?

I've been reading into ABX and double blind, and I've read articles that claim that amplifiers sound the same with certain provisions, so I would like to put those claims to the test in my own home. Nothing better than testing these things out for yourself first hand.

If I am successful in learning the basics then I can test my friends too. At the end of the day, I want to know the truth. I've been told various things by various people over the years, but I am only interested in the results.
Go to the top of the page
+Quote Post
Rich B
post Nov 16 2013, 12:17
Post #6





Group: Members
Posts: 34
Joined: 16-November 13
Member No.: 112247



QUOTE (greynol @ Nov 16 2013, 13:14) *
Check this topic which is pinned to the top of the general audio forum:
http://www.hydrogenaudio.org/forums/index....showtopic=16295


I appreciate the link. But it does not detail how the tests are done. I would like to do these tests at home, but I don't even know where to start.

Have you done these tests at home? If so, how did you do them?
Go to the top of the page
+Quote Post
greynol
post Nov 16 2013, 12:24
Post #7





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



Amplifiers do not necessarily sound the same, especially if they are of the tube variety. Amps that are designed to be transparent will, if done successfully.

ABXing hardware in the home requires either an ABX box or an additional person or two. The person doing the switching must be concealed both audibly and visually from the listener so that there are no tells.

You may also attempt to record clips which must be time-aligned and use ABD software, though this has obvious pitfalls.

Level-matching is critical.

This post has been edited by greynol: Nov 16 2013, 12:28


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
Rich B
post Nov 16 2013, 12:30
Post #8





Group: Members
Posts: 34
Joined: 16-November 13
Member No.: 112247



QUOTE (greynol @ Nov 16 2013, 13:24) *
Amplifiers do not necessarily sound the same, especially if they are of the tube variety. Amps that are designed to be transparent will, if done successfully.

ABXing hardware in the home requires either an ABX box or an additional person or two. The person doing the switching must be concealed both audibly and visually from the listener so that there are no tells.


So without an ABX box or helpers, it can't be done? I assume everyone is in the same boat as far as this is concerned. People have asked me to do my own testing which I very much would like to do. But I don't have access to an ABX box and I live alone. Does this mean that I cannot perform such testing? Is it entirely necessary that the result is double blind in order for the test to have meaning?
Go to the top of the page
+Quote Post
Rich B
post Nov 16 2013, 12:33
Post #9





Group: Members
Posts: 34
Joined: 16-November 13
Member No.: 112247



QUOTE (greynol @ Nov 16 2013, 13:24) *
Amplifiers do not necessarily sound the same, especially if they are of the tube variety. Amps that are designed to be transparent will, if done successfully.

You may also attempt to record clips which must be time-aligned and use ABD software, though this has obvious pitfalls.

Level-matching is critical.


Just noticed the edit. It seems like doing the test is not something that everyone can do. How would I go about level-matching? I do have an SPL meter. Would that help?
Go to the top of the page
+Quote Post
greynol
post Nov 16 2013, 12:33
Post #10





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



Also, ABX tests can only show whether A and B can be distinguished with statistical significance and are primarily relevant to the person taking the test, the time the test was taken, and the specific conditions of the test. One must be careful when generalizing beyond these circumstances.


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
Brand
post Nov 16 2013, 12:39
Post #11





Group: Members
Posts: 312
Joined: 27-November 09
Member No.: 75355



If you want to compare line level outputs of a DAC (soundcard, CD player etc.) you can:
1. record the output from various devices using the same audio material and then compare the recordings in a blind listening test (once you have the digital recordings this is easy to do with software)
2. test the outputs with something like RMAA (or some dedicated testing hardware)

I believe you can also do this for a headphone-amplified output, by using a Y splitter: headphone plugged in one output, test signal on the other. In this case the test signal would be influenced by the headphone load and it will vary with different headphones.

Obviously you'd have to trust whatever input ADC you're using for recording/testing to be good enough to not mask the differences between the tested outputs. Same for the output if you're doing a listening test of the recordings.
There are also small amplitude/voltage variations you need to take into account and compensate at both DAC and ADC stages.
Go to the top of the page
+Quote Post
greynol
post Nov 16 2013, 12:39
Post #12





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (Rich B @ Nov 16 2013, 03:33) *
I do have an SPL meter. Would that help?

An appropriate meter is essential for hardware testing. Consider what would happen if there were level differences. I've read that a volume change as small as 0.1 dB can be detected.


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
Rich B
post Nov 16 2013, 12:40
Post #13





Group: Members
Posts: 34
Joined: 16-November 13
Member No.: 112247



Okay, but if I want to know whether I can hear a difference between two components, like two amplifiers, then what type of testing do I need to do?
Go to the top of the page
+Quote Post
greynol
post Nov 16 2013, 12:41
Post #14





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



We just told you.


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
Rich B
post Nov 16 2013, 12:43
Post #15





Group: Members
Posts: 34
Joined: 16-November 13
Member No.: 112247



QUOTE (greynol @ Nov 16 2013, 13:39) *
QUOTE (Rich B @ Nov 16 2013, 03:33) *
I do have an SPL meter. Would that help?

An appropriate meter is essential for hardware testing. Consider what would happen if there were level differences. I've read that a volume change as small as 0.1 dB can be detected.


I thought 0.1 dB could not be heard. I know I can hear 1 dB changes with an SPL meter, but it's very slight. I don't think I could discern 0.1 dB. Aren't solid state amplifiers have a +- 0.1 dB across the frequency range?
Go to the top of the page
+Quote Post
Rich B
post Nov 16 2013, 12:45
Post #16





Group: Members
Posts: 34
Joined: 16-November 13
Member No.: 112247



QUOTE (greynol @ Nov 16 2013, 13:41) *
We just told you.


I was busy replying to another post and I had not yet seen the post from Brand.
Go to the top of the page
+Quote Post
greynol
post Nov 16 2013, 12:50
Post #17





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (greynol @ Nov 16 2013, 03:24) *
The person doing the switching must be concealed both audibly and visually from the listener so that there are no tells.

Also, the person interfacing with the listener must not know which is playing.


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
Rich B
post Nov 16 2013, 12:50
Post #18





Group: Members
Posts: 34
Joined: 16-November 13
Member No.: 112247



QUOTE (Brand @ Nov 16 2013, 13:39) *
If you want to compare line level outputs of a DAC (soundcard, CD player etc.) you can:
1. record the output from various devices using the same audio material and then compare the recordings in a blind listening test (once you have the digital recordings this is easy to do with software)
2. test the outputs with something like RMAA (or some dedicated testing hardware)

I believe you can also do this for a headphone-amplified output, by using a Y splitter: headphone plugged in one output, test signal on the other. In this case the test signal would be influenced by the headphone load and it will vary with different headphones.

Obviously you'd have to trust whatever input ADC you're using for recording/testing to be good enough to not mask the differences between the tested outputs. Same for the output if you're doing a listening test of the recordings.
There are also small amplitude/voltage variations you need to take into account and compensate at both DAC and ADC stages.


That sounds like a lot of work and a lot of uncertainty involved. How would I know whether my ADC was good enough not to mask differences? Then I need to adjust for voltage variations to .. less than 0.1 dB? This is all very confusing.
Go to the top of the page
+Quote Post
greynol
post Nov 16 2013, 13:00
Post #19





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (Rich B @ Nov 16 2013, 03:43) *
Aren't solid state amplifiers have a +- 0.1 dB across the frequency range?

This may or may not matter. If it does matter and two amps have a didferent response then they will sound different.

Personally, I think understanding how expectation bias can influence how we perceive what we hear and that true auditory memory is extremely short are far more important than attempting to prove you can hear differences that will likely only be nuanced at best (assuming they even exist at all outside your brain).

This post has been edited by greynol: Nov 16 2013, 13:02


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
Brand
post Nov 16 2013, 13:01
Post #20





Group: Members
Posts: 312
Joined: 27-November 09
Member No.: 75355



QUOTE (Rich B @ Nov 16 2013, 12:50) *
That sounds like a lot of work and a lot of uncertainty involved. How would I know whether my ADC was good enough not to mask differences? Then I need to adjust for voltage variations to .. less than 0.1 dB? This is all very confusing.

It's not that much work. My advice is to familiarize yourself with RMAA testing (assuming you're using Windows). Once you get the hang of it, it's easy.
It should give you an idea of the noise levels, frequency responses etc. of whatever soundcard(s) you're using. Then you can choose one that is better suited for testing and doing test recordings. There are also RMAA tests for various soundcards floating around the internet.

And yeah, I'd definitely recommend to level match to less than 0.1dB. But this is easy if you have digital audio files, any audio editing program (even free ones like Wavosaur, Audacity...) can do it. You'd also use that to time-match the recordings down to the sample.
Go to the top of the page
+Quote Post
Rich B
post Nov 16 2013, 13:24
Post #21





Group: Members
Posts: 34
Joined: 16-November 13
Member No.: 112247



QUOTE (Brand @ Nov 16 2013, 14:01) *
QUOTE (Rich B @ Nov 16 2013, 12:50) *
That sounds like a lot of work and a lot of uncertainty involved. How would I know whether my ADC was good enough not to mask differences? Then I need to adjust for voltage variations to .. less than 0.1 dB? This is all very confusing.

It's not that much work. My advice is to familiarize yourself with RMAA testing (assuming you're using Windows). Once you get the hang of it, it's easy.
It should give you an idea of the noise levels, frequency responses etc. of whatever soundcard(s) you're using. Then you can choose one that is better suited for testing and doing test recordings. There are also RMAA tests for various soundcards floating around the internet.

And yeah, I'd definitely recommend to level match to less than 0.1dB. But this is easy if you have digital audio files, any audio editing program (even free ones like Wavosaur, Audacity...) can do it. You'd also use that to time-match the recordings down to the sample.


Would I do this to test the difference between amplifiers as well? So most of this I can do on my computer. I assumed I might need a volt meter to level match.

I am using Windows 8. Right now I have no experience using RMAA. But I could certainly look into it and try to familarise myself with it.
Go to the top of the page
+Quote Post
Rich B
post Nov 16 2013, 13:28
Post #22





Group: Members
Posts: 34
Joined: 16-November 13
Member No.: 112247



QUOTE (greynol @ Nov 16 2013, 14:00) *
QUOTE (Rich B @ Nov 16 2013, 03:43) *
Aren't solid state amplifiers have a +- 0.1 dB across the frequency range?

This may or may not matter. If it does matter and two amps have a didferent response then they will sound different.

Personally, I think understanding how expectation bias can influence how we perceive what we hear and that true auditory memory is extremely short are far more important than attempting to prove you can hear differences that will likely only be nuanced at best (assuming they even exist at all outside your brain).


I have a basic understanding. Attempting to prove whether I can hear differences or whether other people can hear differences is important - for me, because it means I can probably spend less money than I used to. For other people, it means I can show them the truth, and if it means they will save money in future rather than getting ripped off from buying fancy cable or other things, then I certainly would like to help. Telling people that you shouldn't expect to hear differences because of XYZ isn't good enough. If they see the results first hand then it's a different story. I think that is fair to say.

This post has been edited by Rich B: Nov 16 2013, 13:30
Go to the top of the page
+Quote Post
Brand
post Nov 16 2013, 15:12
Post #23





Group: Members
Posts: 312
Joined: 27-November 09
Member No.: 75355



QUOTE (Rich B @ Nov 16 2013, 13:24) *
Would I do this to test the difference between amplifiers as well?

You couldn't test a loudspeaker output with it. This is suitable for analog signals which can be captured with a soundcard, basically line-level stuff, but also amplified signals with headphone loads.
The amp signal that powers passive speakers is a different kind and would require different equipment.

All RMAA (or any computer program) can do is output a test signal, which goes through your analog gear and comes back, eventually converted back to digital. Then it compares the two, and by analyzing the differences it makes an estimate of how the analog path influences/distorts/colors the original signal.
It can't test every aspect of an audio device, but I still think it's useful and convenient.
Go to the top of the page
+Quote Post
greynol
post Nov 16 2013, 17:35
Post #24





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



http://www.hydrogenaudio.org/forums/index....showtopic=84680


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
saratoga
post Nov 16 2013, 21:31
Post #25





Group: Members
Posts: 4718
Joined: 2-September 02
Member No.: 3264



Amps should be level matched with a voltmeter or even a line in rather than an spl meter. This is more accurate and much cheaper.

SPL is only needed for speaker testing .
Go to the top of the page
+Quote Post

2 Pages V   1 2 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 20th April 2014 - 07:28