Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: How do perform a controlled listening test at home (Read 12969 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

How do perform a controlled listening test at home

Hi all,

I want to learn how to perform a bias-controlled test at home. Are there step-by-step tutorials that can guide amateurs? I would like to train myself how to do this so that I can apply this knowledge in other scenarios. I have friends who have bought expensive gear who have challenged me to prove that they can't hear differences between equipment. They would entertain a listening test in their own homes.

I would like to set the record straight. But first I need to learn how to set up a test for myself. If I can't test this for myself then it's not worth the hassle. I have equipment at home (separate dacs and amplifiers) and I can hear differences quite easily between them, but I also know the listening test is not reliable. I want the wool to be pulled over my eyes and to have the knowledge to put these claims to rest.

Any help or assistance in learning would be most appreciated. Thanks!

How do perform a controlled listening test at home

Reply #1 –
In an scientifically objective audio listening test, the listener is unaware of the circumstances (i.e. DAC 1 or 2) in random multiple succession to eliminate the guessing-factor. If you have a a second party to facilitate the changes and record (and then those results if beyond the the guess-factor of >85% [:unsure]) may establish the listener detects an audible difference.

However, a "bias-controlled test?" I will defer to the experts...
"Something bothering you, Mister Spock?"

How do perform a controlled listening test at home

Reply #2 –
I need to learn how to conduct a listening test so that it can reveal whether differences are heard or not. I assumed the test needed to be bias-controlled in order for the results to be meaningful.

How do perform a controlled listening test at home

Reply #3 –
Check this topic which is pinned to the top of the general audio forum:
http://www.hydrogenaudio.org/forums/index....showtopic=16295

How do perform a controlled listening test at home

Reply #4 –
Quote
However, a "bias-controlled test?" I will defer to the experts..


I've been reading up on bias, so I assumed the test must be controlled for that. I'm not asking for lab conditions, but how would I go about testing the claims in my own home? If I was alone, and I wanted to devise a test, how would I go about it?

I've been reading into ABX and double blind, and I've read articles that claim that amplifiers sound the same with certain provisions, so I would like to put those claims to the test in my own home. Nothing better than testing these things out for yourself first hand.

If I am successful in learning the basics then I can test my friends too. At the end of the day, I want to know the truth. I've been told various things by various people over the years, but I am only interested in the results.

How do perform a controlled listening test at home

Reply #5 –
Quote from: greynol on
Check this topic which is pinned to the top of the general audio forum:
http://www.hydrogenaudio.org/forums/index....showtopic=16295


I appreciate the link. But it does not detail how the tests are done. I would like to do these tests at home, but I don't even know where to start.

Have you done these tests at home? If so, how did you do them?

How do perform a controlled listening test at home

Reply #6 –
Amplifiers do not necessarily sound the same, especially if they are of the tube variety. Amps that are designed to be transparent will, if done successfully.

ABXing hardware in the home requires either an ABX box or an additional person or two.  The person doing the switching must be concealed both audibly and visually from the listener so that there are no tells.

You may also attempt to record clips which must be time-aligned and use ABD software, though this has obvious pitfalls.

Level-matching is critical.

How do perform a controlled listening test at home

Reply #7 –
Quote from: greynol on
Amplifiers do not necessarily sound the same, especially if they are of the tube variety. Amps that are designed to be transparent will, if done successfully.

ABXing hardware in the home requires either an ABX box or an additional person or two.  The person doing the switching must be concealed both audibly and visually from the listener so that there are no tells.


So without an ABX box or helpers, it can't be done? I assume everyone is in the same boat as far as this is concerned. People have asked me to do my own testing which I very much would like to do. But I don't have access to an ABX box and I live alone. Does this mean that I cannot perform such testing? Is it entirely necessary that the result is double blind in order for the test to have meaning?

How do perform a controlled listening test at home

Reply #8 –
Quote from: greynol on
Amplifiers do not necessarily sound the same, especially if they are of the tube variety. Amps that are designed to be transparent will, if done successfully.

You may also attempt to record clips which must be time-aligned and use ABD software, though this has obvious pitfalls.

Level-matching is critical.


Just noticed the edit. It seems like doing the test is not something that everyone can do. How would I go about level-matching? I do have an SPL meter. Would that help?

How do perform a controlled listening test at home

Reply #9 –
Also, ABX tests can only show whether A and B can be distinguished with statistical significance and are primarily relevant to the person taking the test, the time the test was taken, and the specific conditions of the test. One must be careful when generalizing beyond these circumstances.

How do perform a controlled listening test at home

Reply #10 –
If you want to compare line level outputs of a DAC (soundcard, CD player etc.) you can:
1. record the output from various devices using the same audio material and then compare the recordings in a blind listening test (once you have the digital recordings this is easy to do with software)
2. test the outputs with something like RMAA (or some dedicated testing hardware)

I believe you can also do this for a headphone-amplified output, by using a Y splitter: headphone plugged in one output, test signal on the other. In this case the test signal would be influenced by the headphone load and it will vary with different headphones.

Obviously you'd have to trust whatever input ADC you're using for recording/testing to be good enough to not mask the differences between the tested outputs. Same for the output if you're doing a listening test of the recordings.
There are also small amplitude/voltage variations you need to take into account and compensate at both DAC and ADC stages.

How do perform a controlled listening test at home

Reply #11 –
Quote from: Rich B on
I do have an SPL meter. Would that help?

An appropriate meter is essential for hardware testing.  Consider what would happen if there were level differences. I've read that a volume change as small as 0.1 dB can be detected.

How do perform a controlled listening test at home

Reply #12 –
Okay, but if I want to know whether I can hear a difference between two components, like two amplifiers, then what type of testing do I need to do?

How do perform a controlled listening test at home

Reply #14 –
Quote from: greynol on
Quote from: Rich B on
I do have an SPL meter. Would that help?

An appropriate meter is essential for hardware testing.  Consider what would happen if there were level differences. I've read that a volume change as small as 0.1 dB can be detected.


I thought 0.1 dB could not be heard. I know I can hear 1 dB changes with an SPL meter, but it's very slight. I don't think I could discern 0.1 dB. Aren't solid state amplifiers have a +- 0.1 dB across the frequency range?

How do perform a controlled listening test at home

Reply #16 –
Quote from: greynol on
The person doing the switching must be concealed both audibly and visually from the listener so that there are no tells.

Also, the person interfacing with the listener must not know which is playing.

How do perform a controlled listening test at home

Reply #17 –
Quote from: Brand on
If you want to compare line level outputs of a DAC (soundcard, CD player etc.) you can:
1. record the output from various devices using the same audio material and then compare the recordings in a blind listening test (once you have the digital recordings this is easy to do with software)
2. test the outputs with something like RMAA (or some dedicated testing hardware)

I believe you can also do this for a headphone-amplified output, by using a Y splitter: headphone plugged in one output, test signal on the other. In this case the test signal would be influenced by the headphone load and it will vary with different headphones.

Obviously you'd have to trust whatever input ADC you're using for recording/testing to be good enough to not mask the differences between the tested outputs. Same for the output if you're doing a listening test of the recordings.
There are also small amplitude/voltage variations you need to take into account and compensate at both DAC and ADC stages.


That sounds like a lot of work and a lot of uncertainty involved. How would I know whether my ADC was good enough not to mask differences? Then I need to adjust for voltage variations to .. less than 0.1 dB? This is all very confusing.

How do perform a controlled listening test at home

Reply #18 –
Quote from: Rich B on
Aren't solid state amplifiers have a +- 0.1 dB across the frequency range?

This may or may not matter. If it does matter and two amps have a didferent response then they will sound different.

Personally, I think understanding how expectation bias can influence how we perceive what we hear and that true auditory memory is extremely short are far more important than attempting to prove you can hear differences that will likely only be nuanced at best (assuming they even exist at all outside your brain).

How do perform a controlled listening test at home

Reply #19 –
Quote from: Rich B on
That sounds like a lot of work and a lot of uncertainty involved. How would I know whether my ADC was good enough not to mask differences? Then I need to adjust for voltage variations to .. less than 0.1 dB? This is all very confusing.

It's not that much work. My advice is to familiarize yourself with RMAA testing (assuming you're using Windows). Once you get the hang of it, it's easy.
It should give you an idea of the noise levels, frequency responses etc. of whatever soundcard(s) you're using. Then you can choose one that is better suited for testing and doing test recordings. There are also RMAA tests for various soundcards floating around the internet.

And yeah, I'd definitely recommend to level match to less than 0.1dB. But this is easy if you have digital audio files, any audio editing program (even free ones like Wavosaur, Audacity...) can do it. You'd also use that to time-match the recordings down to the sample.

How do perform a controlled listening test at home

Reply #20 –
Quote from: Brand on
Quote from: Rich B on
That sounds like a lot of work and a lot of uncertainty involved. How would I know whether my ADC was good enough not to mask differences? Then I need to adjust for voltage variations to .. less than 0.1 dB? This is all very confusing.

It's not that much work. My advice is to familiarize yourself with RMAA testing (assuming you're using Windows). Once you get the hang of it, it's easy.
It should give you an idea of the noise levels, frequency responses etc. of whatever soundcard(s) you're using. Then you can choose one that is better suited for testing and doing test recordings. There are also RMAA tests for various soundcards floating around the internet.

And yeah, I'd definitely recommend to level match to less than 0.1dB. But this is easy if you have digital audio files, any audio editing program (even free ones like Wavosaur, Audacity...) can do it. You'd also use that to time-match the recordings down to the sample.


Would I do this to test the difference between amplifiers as well? So most of this I can do on my computer. I assumed I might need a volt meter to level match.

I am using Windows 8. Right now I have no experience using RMAA. But I could certainly look into it and try to familarise myself with it.

How do perform a controlled listening test at home

Reply #21 –
Quote from: greynol on
Quote from: Rich B on
Aren't solid state amplifiers have a +- 0.1 dB across the frequency range?

This may or may not matter. If it does matter and two amps have a didferent response then they will sound different.

Personally, I think understanding how expectation bias can influence how we perceive what we hear and that true auditory memory is extremely short are far more important than attempting to prove you can hear differences that will likely only be nuanced at best (assuming they even exist at all outside your brain).


I have a basic understanding. Attempting to prove whether I can hear differences or whether other people can hear differences is important - for me, because it means I can probably spend less money than I used to. For other people, it means I can show them the truth, and if it means they will save money in future rather than getting ripped off from buying fancy cable or other things, then I certainly would like to help. Telling people that you shouldn't expect to hear differences because of XYZ isn't good enough. If they see the results first hand then it's a different story. I think that is fair to say.

How do perform a controlled listening test at home

Reply #22 –
Quote from: Rich B on
Would I do this to test the difference between amplifiers as well?

You couldn't test a loudspeaker output with it. This is suitable for analog signals which can be captured with a soundcard, basically line-level stuff, but also amplified signals with headphone loads.
The amp signal that powers passive speakers is a different kind and would require different equipment.

All RMAA (or any computer program) can do is output a test signal, which goes through your analog gear and comes back, eventually converted back to digital. Then it compares the two, and by analyzing the differences it makes an estimate of how the analog path influences/distorts/colors the original signal.
It can't test every aspect of an audio device, but I still think it's useful and convenient.

How do perform a controlled listening test at home

Reply #24 –
Amps should be level matched with a voltmeter or even a line in rather than an spl meter.  This is more accurate and much cheaper.

SPL is only needed for speaker testing .