How to compare dacs Options

[Yahzi]

People generally claim that dacs sound different and I'm sure in some cases they can and do sound different, especially if their levels are not matched, unless the
frequency response of both were so grotesquely different, but I assume that many measure close to flat.

My question is, how does one go about level matching dacs? Is there a right and wrong way, or just a standard method?

[skamp]

You could record both DACs with appropriate output and recording levels, and Replaygain the resulting files.

[Yahzi]

I'm asking because I had a discussion with an EE over this. He is a firm believer in using dacs so I brought up the issue of level matching, in which case he replied with :

"All digital source components are level matched, using a 1kHz sine wave, to 1V RMS analogue. The reference WAV test tone is created to be at a level of -1dB from absolute, sampled at 96kHz with 24 bit depth. Not trying to be funny but I've been doing this for a while".

[krabapple]

Tell him to Google 'CD player output voltage' and get back to you after he's done reading.

[Yahzi]

Can you explain to me, a laymen, what the glaring issues are in his testing process? ; ) Of course, I haven't asked him whether the test was sighted or blind, but the level matching process doesn't seem right. I'm not a EE, but he is, and he clearly thinks he knows his stuff, so if I can undermine his testing methodology that will put a smile on my face for sure. ; )

This post has been edited by Yahzi: Nov 21 2012, 15:00

[Brand]

You can try RMAA. It won't measure every aspect of the DAC, but it's good for what it is.

[Yahzi]

Tell him to Google 'CD player output voltage' and get back to you after he's done reading.

Please explain.

[Kees de Visser]

"All digital source components are level matched, using a 1kHz sine wave, to 1V RMS analogue. The reference WAV test tone is created to be at a level of -1dB from absolute, sampled at 96kHz with 24 bit depth. Not trying to be funny but I've been doing this for a while".

Looks fine to me. If there's clipping somewhere in the path, it's probably clearly audible and should be avoided by lowering the analog gain after the DAC.

[Yahzi]

Looks fine to me. If there's clipping somewhere in the path, it's probably clearly audible and should be avoided by lowering the analog gain after the DAC.

So level matching to a single frequency is the preferred method of level matching for listening and/or testing purposes?

[pdq]

In an age when DACs exhibit near-perfectly flat frequency response, level matching at a single frequency should be just fine.

[Arnold B. Kruege...]

People generally claim that dacs sound different and I'm sure in some cases they can and do sound different, especially if their levels are not matched, unless the
frequency response of both were so grotesquely different, but I assume that many measure close to flat.

My question is, how does one go about level matching dacs? Is there a right and wrong way, or just a standard method?

Back in the day we put a high resolution attenuator on the analog outputs of both DACs, played test CDs with a good selection of test tones on them, and matched the outputs at the speaker terminals of the monitoring system.

Here is the rest of the equpment we used then.

http://home.provide.net/~djcarlst/abx_ldm.htm

http://home.provide.net/~djcarlst/abx_hcm.htm

http://home.provide.net/~djcarlst/abx_rm2.htm

Later on QSC sold these:

http://home.provide.net/~djcarlst/abx_qsc.htm

[greynol]

As to whether or not one should level-match outpus, one should at least verify they are matched, rather than take an EE's word for it. FWIW, this is coming from an EE.

[Arnold B. Kruege...]

As to whether or not one should level-match outpus, one should at least verify they are matched, rather than take an EE's word for it. FWIW, this is coming from an EE.

We both know the following, but for other readers:

You shouldn't need to be an EE to understand the need for level matching! It seems pretty logical - if something is louder or softer it sounds different but this is a trivial difference. The purpose of equipment tests is to find non trivial differences. All trivial but relevant differences should be removed from the experiment.

This post has been edited by Arnold B. Krueger: Nov 22 2012, 20:44

[Yahzi]

Arny, please correct me if I'm wrong here. Level matching is certainly important for comparisons, but the fact the test was sighted - wouldn't that invalidate the test process regardless whether the equipment was level matched or not? So while level matching closes the gap in difference between components, that isn't enough.

[greynol]

I, for one, had assumed the removal of expectation bias had already been taken into account. It also seemed clear to me from the original post that we were discussing level-matching rather than double-blind methodology. Anyway...

Level matching, double-blinding and time-synchronizing (if quick switching) must all be taken into account when testing for audible differences between two things.

And of course Arny is right, it doesn't take an EE to understand what should be common sense. My point was that appealing to authority is not a valid argument against good scientific practice.

This post has been edited by greynol: Nov 23 2012, 16:59

[Yahzi]

I, for one, had assumed the removal of expectation bias had already been taken into account. It also seemed clear to me from the original post that we were discussing level-matching rather than double-blind methodology. Anyway...

No, expectation bias was, to my knowledge, still in full flight in this case. He level matched the dacs using the methodology I referenced earlier, but he is clearly biased and has made no bones about that. I was hoping to find some glaring issue in his testing methodology, but assuming both dacs had identical frequency response then matching to a single frequency would be appropriate, unless there is a more accurate method for level matching.

This post has been edited by Yahzi: Nov 23 2012, 17:32

[greynol]

Lack of DBT is legitimate grounds to dismiss the results outright. There is no need even to entertain level matching when this criteria has not been satisfied.

This post has been edited by greynol: Nov 23 2012, 17:36

[Yahzi]

I'm curious, how does the McGurk effect enter into the equation? If someone tells you what you should expect to hear in a piece of audio equipment then, if I understand things correctly, you may be predisposed to hear those qualities whether they exist or not, or at least are more susceptible due to the suggestion. Is that right, or have I gone completely off the rails here? ; )

The reason I ask is because I'm sure this has a large degree on the outcome, skewing the end result. So not only do you have expectation bias - you want to believe that there will be an improvement - and then, in addition, people tell you that "x" sounds "bright" or "warm" or whatever and these suggestions skew the result further. Perhaps you read a review online or in a magazine or a friend gave you friendly advice and these suggestions kind of "imprinted" themselves into your subconscious. It just sounds like such a mountain to climb if you want to establish any form of truth outside of participating in a DBT.

This post has been edited by Yahzi: Nov 23 2012, 21:49

[Arnold B. Kruege...]

Arny, please correct me if I'm wrong here. Level matching is certainly important for comparisons, but the fact the test was sighted - wouldn't that invalidate the test process regardless whether the equipment was level matched or not?

Yes, an evaluation that lacks bias controls and carefully synchronized source material is invalid, level-matched or not.

So while level matching closes the gap in difference between components, that isn't enough.

Agreed.

[Yahzi]

Yes, an evaluation that lacks bias controls and carefully synchronized source material is invalid, level-matched or not.

Thanks Arny!

[Yahzi]

Do I have a handle on the McGurk effect, or is my understanding completely unrelated to the effect?