It seems to me that procedures outlined in ABX-related discussions are somewhat inadequate for real-world situations. For instance, I see the following issues with blind testing of cables (let's say 2 power cords, to be specific):

1. Without even touching the subject of transparency of ABX box - there's no ABX box in possession of average audiophile. That leaves us with the test protocol that excludes "double" from double-blind, since the helper will know what cable is currently plugged in - which seemingly runs contrary to what is considered acceptable practice here.

2. Ability to reproduce results of the test - that seems impractical, since it assumes having exactly the the same system in exactly the same room. The only possible way to recreate it is to invite the person who wishes to reproduce results to your home - which is also impractical for obvious reasons.

3. Finally, idea of reproduceability seems even more bizarre, considering that ear/brain combination is vastly different from one person to another. That would mean that results of particular test pertain only to single person that performed the test, and single system the test was performed on (including room), thus rendering the test pretty much worthless to anybody else.

Anybody care to try explaining what I'm missing here?

1. You see, most, if not all, tests conducted by or reported to these forums are about audio codecs, which you can compare in a double-blind fashion using well-established, dedicated software. There might indeed be a lack of proper equipment for comparisons of hardware. But that does not imply that we need to accept results below the standard we have set for ourselves. Someone who wanted to prove an audible difference between, for example, cables, would simply need to try harder.


2.+3. I would disagree with that requirement for reproducibility. If you successfully ABX two cables, why would you come to the conclusion that the difference can only be heard in your room? On the contrary, proving the difference independent of the listening environment would actually add to the credibility of the claims. If we again look at the years of experience with ABX tests of audio samples, these can usually be reproduced with completely different equipment by another user on the other side of the world. Why would it be any different with hardware?

Of course, it's impractical for the average audiophile to conduct ABX tests on hardware. But, if you're going to publish a claim that one power cable sounds better than another you need to back it up! A lot of nonsense gets published in the "audiophile community", and we try to separate the truth from the nonsense!

It just gets silly when an "audiophile" says he can hear differences in non-blind tests, and then when he fails a blind test, he starts telling you about all of the the "inadequacies" of blind testing. If you can only hear the differences in a sighted test, I'm going to question your sighted test!!!!

Blind testing isn't required for everything. Professional* audio work (recording & mixing, etc.) is all done without blind testing, and nobody does blind testing when tweaking their equalizer... Nobody is going to question me if I say that my home theater system sounds better than my boombox, or if I say that I've got an old vinyl record with dull-sounding highs. But, if I tell you that "a veil was lifted" when I changed speaker cables, the people here are going to say "prove it!".

Of course, people do vary in their hearing ability. (And, dogs & rats can hear higher frequencies than humans! ) You've probably heard of "golden ear" audiophiles**.

If you do some tests at home and you can hear a difference (in a blind test) that's useful information. But, proper (scientific) studies require more than one listener. If only one person can hear a difference 100% of the time, that is notable and should be reported. If half of the people can hear a difference 100% of the time, that should be reported too.

Statistics can be tricky and there is always some possibility of false result. This is why scientific results are always reported with a confidence level such as, "We are 95% certain that the drug is effective in more than 50% of cases." (The FDA requires blind testing too! ...They are even more strict about it than HydrogenAudio!)

If the room makes a difference, I need to know that. Usually these are advertising claims, and I need to know if this high-end cable is going to work in my room.





*There is a fair bit of mythology & nonsense in the pro audio world too, but as far as I know the pros don't worry too much about "little things" like cables. As long as the cable is rugged and reliable they don't expect it to affect the sound.

** I'd like to see the golden ear audiophile tested before I give him any extra credibility.

As long as the helper is not visible or otherwise potentially communicating back to the person taking the test, the test is double blind. This is pretty simple to ensure. You put the helper in another room or behind an opaque curtain. You call out commands to the helper such as "A", "B", and "X". BTW this is not a hypothetical solution - I've actually done ABX tests this way.



As long as you aren't testing speakers, the room should have negligable effects on the UUT. Coming up with a playback system with sufficient resolution should not be mission impossible. If you've got say a DAC that can only be heard to have an advantage in one room and one audio system in the Known universe, it hardly has any general application, now does it?




Same argument. If you're the only person in the known universe who can hear a certain something, why would anybody else in the known universe be interested in it?

You're running counter to typical audiophile behavior which is to breathlessly post to some forum that you've found a seemingly-magical component that lifts the 7 veils that have been obscuring everybody's enjoyment of music, and that everybody who reads your post should run right out and listen to it and buy it.

If you can establish that YOU hear difference (and that this is not merely a coincidence), then you can conclude that the difference is audible at least to SOMEBODY. If repeated testing shows that YOU are the only one who can, then ... well, then you are unique :-D

But testing in the high end is about establishing "audible to SOMEBODY". Not to everybody, because high-end audio is not for everybody (just like the musical properties of a Stradivarius between my hands is really ... not anything worth paying for). The big issue is "audible difference or not". Once "audible difference" is established, we can start to discuss "how important?".

Of course you have the logical issue that "audible difference to somebody" can never be disproved in a listening test, but again: until you can find any musician who can utilize the Stradivarius, then why pay $$s for hiring the instrument from the museum?

http://www.hydrogenaudio.org/forums/index....showtopic=45432

In keeping with your question at the top - here was a long-term, low-stress, power cable ABX test. Multiple participants. You may find the results interesting.

One thing this thread is illustrating, is how much a good wiki page is needed addressing the OP's issues with blind testing, all of which have been offered contra DBT many, many times before on many, many audio forums (going back long before the internets got popular) , by many, many, skeptics both open-minded and not. I have no hope that the 'issues' will be finally laid to rest -- no matter how many times the 'problems' have been addressed before, they deathlessly rise again, with the high-end press as a dependable provocateur -- but at least there would be a one-stop URL to send people to so the standard rebuttals don't have to be typed over and over. Something perhaps along the lines of talkorigins.org's 'Five Major Misconceptions About Evolution"

I think that a page also listing links to test results would be good too. Telling people on other sites to go to hydrogenaudio.org for test results is less than ideal...

The real reason for my post is a test I conducted over the weekend, different aspects of which raised the qiestions in the OP.

First, I was comparing 2 power cords sighted, and thought the difference is there, but pretty insignificant, so these two weren't good candidates for blind test (which I would've most likely failed). My next thought was to compare one of the two (the one I prefer) to the stock cord, but finally, since I own several cords, I decided to pick for blind testing the one I thought sounded significantly different (worse) in sighted auditions.

Here's how test was organized:

1. Power cords were utilized on amplifier, going straight to the wall.

2. NO LEVEL MATCHING was performed - since both cords are of sufficient (and rather heavy) gauge, I don't see any need for that, and would dismiss without further discussion any notion that only loudness was affected by replacing the cords.

3. I enlisted my wife, who doesn't care for sound quality, but acknowledges visual appeal of certain components (including speakers), as helper.

4. I hanged old shower curtain, sufficiently opaque, from the ceiling using several pins so that it hid all components (but not speakers) from the sight at listening armchair. Wife was located behind the audio rack so she wasn't visible, either, and was instructed not to manifest herself in any way.

5. All of the above was performed in my listening room, which is 17'x11'x7.5' finished part of the basement, and is separated by a door from unfinished part. Some acoustic treatments present (first reflection points and couple of bass traps).

6. The procedure was as follows:

- I listened to short fragment (~15 sec.) of the same track sighted, with wife replacing cords A and B. Twice to each was enough for me to establish the differences. I determined that A is "better" and B is "worse".
- I left the room, closing the door. Wife was instructed to replace the cords in arbitrary order, making note of which one is currently plugged under sequential #. On wife's command, I returned to the room, and listened to exactly the same ~15 sec. fragment used during sighted audition. I made a note of which one, in my opinion - A or B - was used, under sequential #.
- Previous step was repeated 10 times, after which we compared notes. Intention was to continue, if results are unsatisfactory - but that turned out not to be the case, so test was stopped. It wasn't my intention to post it here, or anywhere else for that matter, otherwise it would continue.

In short, all 10 tries were successful, with me reliable identifying correct cord.

7. Relevant details of the system:

- Simaudio Supernova CD player
- Reference Line Preeminence 1B passive "preamp"
- Plinius SB-300 amplifier
- KEF Reference 3.2 speakers

8. Track used:

- Saturnus - beginning of the track "Embraced by Darkness", from "Veronica Decides to Die" album.


Now, back to my questions:

- what useful information can anyone but myself take away from that experiment?
- would anyone be willing to spend money on the same exact power cords, considering they are moderately expensive ($200-500 range used), to repeat the experiment?
- without using the same exact system - how would we know that another person's system is of equal resolution to mine, or at least of enough resolution to detect these differences?
- how would we know that another person's hearing is in good enough order?
- how would we account for possible bias NOT to hear any differences (seems pretty actual here)?


I'm all ears.

Before we can extrapolate from your results we need to understand what you are hearing and why.

Your results seem to contradict those of other carefully performed experiments in this area. BTW, there may be a million dollar prize waiting for you.

Am I the only one who thinks, while this raises a quite good question over how one deals with fake results, it doesn't pass the smell test?
A day after a post expressing uncertainty over how to do an ABX test w/o an expensive box we get a post claiming almost every technique Arnold laid out in response had been previously done.


EDIT: Not that I want to fall into the trap of attacking.

elementary question: was wife in the room during the tests? (see: Clever Hans effect)


Define 'useful'. A validated 'positive' DBT can be good evidence for a real difference existing. The best validation would be repetition under the most stringent controls, with trusted observers.



'Anyone' is a broad category; so the answer is probably yes, someone, somewhere would probably be willing. Or, JREF might be willing to pay you $1 million if you can repeat your performance.


This starts from the assumption that every possible change stands a good chance of making an audible difference. That's not the case.


we don't conclude from one person's negative that NO ONE can hear it. However, we also don't ignore a century of audiological data, nor physics.


DBTs can be very highly 'resolving'. Multiple blind trials can actually reveal evidence of real difference even if the listener claims not to have heard anything different.

ISTR within the past few days there was a post asking how we know DBT reports here aren't lies. (Another impetus for my 'full moon' remark...but I've been informed it's always a full moon for me.)

Jessenov1, seriously, if a kid asked you where electricity was made, would you tell him "in the outlet"?

I always ask myself how you people get the idea, that after many, many kilometer of cabling, several stages of transformation, above streets, below streets, at least a dozen meter cheap standard copper cabling under plaster and walls through the house, probably passing several screwed joints - how in hell do you get the idea that the last two meters could matter in any way???

There are at least 5 million audiohpiles on this planet. It would be highly improbable, that not one of them would get 10/10 once in a while. So with this in mind and the fact that electricity isn't created in outlets and the fact how (conceptually) easy it is to get a clean power rail for home audio purposes from a suboptimal 110V/230V source, I think it is not too ignorant to not get too excited about your findings.



It would fit your suspected pattern, if Jessenov1 would react to my post with something like "oh, I forgot to mention that the cables were plugged into a power conditioner".

Answering in order:

1. Nothing at all. The test was too poorly designed to result in any usable predictions being generated. The only potential effect that may be considered positive is to get someone else to perform the test correctly.

2. Perhaps. But since it is another flawed test and generally uninteresting, chances are pretty low that any one person would want to cough up the money.

3. Your question is irrelevant. The point of testing a hypothesis is to come up with a prediction model for an observed phenomenon. If the effect you're experiencing is unrepeatable anywhere except in your lab (or on your rig), chances are that the variables being tested aren't isolated enough. See the Fleishmann-Pons announcement for a classic example of this in action - http://en.wikipedia.org/wiki/Cold_fusion#F...ns_announcement

4. One test does not significantly strengthen a hypothetical model. Documented, repeatable results that can be peer-observed and verified strengthen the predictive effect of the model. In effect: if you observe something that only you can observe, to the exclusion of all the rest of existence, it's not very interesting or useful.

5. Similar to #4 above. A sufficiently large, random sample of listeners is what you're looking for. Not a couple of buddies that came over for the weekend.

You seem to be arguing against conducting badly designed & performed tests.

Well, responses so far range from "Results are fake" to "Results are wrong" to "Test wasn't performed properly" (without telling what exactly was so wrong that could potentially affect the results), with general idea being "you and your results are not welcome here".

I sincerely hope that nobody expects me to respond to any of the above.

Well, I would actually like to hear, how you explain why 2 meters of $200-cable should matter after many 1000 meters standard cable to the power plant(s). Do you have one of those magic outlets that produce electricity by themselves?

Why not carefully record the signal coming from the output terminals of your amp and provide samples for us to evaluate?

Isn't that a natural-response audiophile excuse for your objections anyway? If he's a real audiophile I bet he did have them plugged to a power conditioner, probably a very expensive one.

I'll start a list:
1. Non-blind participant involved & remained in the room during the tests.
2. No level matching.
3. No set test protocol before the test & observation of the results as "acceptable", making a decision to stop testing based on that.
4. Marginal number of trials conducted
5. Lack of an ABX switch box (calling into question whether or not connectivity issues played a part in the result)

A real audiophile would also never forget to mention a beloved piece of gear, if it actually was part of his setup.



I'll think you're a little too strict. Level matching for a mains cable test???

If the test was conducted as it was, it was already much better than nothing (or what we else often get from this clientele) and not completely worthless. As said no 10/10 once in a while would be highly improbable. So this doesn't have to be fought off for formal reasons in any case.

Greynol's proposal is actually very good. Of course it leaves the backdoor, that a specific ADC wasn't able to capture the high-endness of Jessenov1's speakers.

Easy enough to bump the dial a touch while you're fiddling about with the cables. Nothing intentional necessary. But without knowing that the volume didn't change during the test while other manual changes were made to the system, the results are weakened.

If you cannot even trust somebody to be capable to pull out a plug and insert another without messing up a volume knob, I think all internet-reported tests, that haven't happened under your personal control, must be irrelevant (edit: weakened) for you.

But I agree that a wife calling to come in, while having knowledge about the current choice, is a source of (possibly subconscious) error, that is purposely excluded in scientific studies for a reason.

Let's say I did, you listened, and found the samples sound different. In that case, we're back to square one - what would you do with these results? Look at the post by Tahnru above - nothing of what I could've recorded and you could've listened matters, according to him (and nobody rushed to object) - statistical sample won't be large enough, especially considering that probably only precious few could detect the difference.

Besides, why would I go for considerable waste of time to do that - to convince quite obviously hostile crowd here, at HA? That's not something I strive for, and is impossible anyway - you know that better than me.

Unless you are implying that the volume knob got knocked one way for cable A and and the opposite way for cable B and repeated in a consistent manner then even this far-reaching hypothesis can not explain the results as presented.

*EDIT: Grammar

And yet, it would be a simple matter of record-keeping to have done during the test.

"Trial 1 -
Position of volume knob before start:
Cord being tested:"

It's not like it would have taken more than a half-seconds work, but it was deemed unnecessary. Why?

Side note - yes, I'm pretty strict (probably overly so). For example, I hate that button in Foobar that allows the tester to see their progressive results during an ABX test. There's no good reason for it. Pretty sad, yeah? Hating a button ...

It's quite understandable that, since most (all?) of involved here do not consider themselves "audiophiles", there would be some remarks about what audiophiles "think" or "do" that sound funny to an audiophile, but some things are just over the top:

- amplifier plugged into power conditioner, expensive or otherwise;
- "Easy enough to bump the dial a touch" - has anybody tried to "bump" 24-position manual switch?
- "High-endness" of my speakers - if we're recording at speaker terminals, what does that have to do with anything?

It's simple test protocol - things that can be variable in the system are checked to ensure they remain the same throughout the test, except the variable in question. Why ignore such an obvious one?

But yes, I'll cop to a negative thought there. I was trying to be nice and not imply that the test was being manipulated, but that was definitely what was running through my head.

And I suppose similarly, someone could just lie on the testing log sheet. Cord A - volume @ 4. Cord B - volume @ 4. When in fact, it's being moved to 5 every time B comes up.

In all fairness:
If different power cables should result in audibly different levels, isnt that a remarkable finding in itself?

-k

I would say that properly executed, reliably reported ABX-tests with significant results are what one could hope for from one person. Demanding presentation of physical models would exclude far too many people for a forum like this.

-k

@ Knutinh - On a broader note, an explanation of the difference that allowed positive identification has been noticeably absent in the discussion.

Level matching would have been interesting for exactly the same reason as you're pointing out.
- Method 1 - logging the hardware conditions. Difference in volume caused by the power cable, with the amplifier unchanged & logged. Provides strong support for additional testing performed on the cables to determine why.
- Method 2 - Measurement of volume levels between cables. During hardware setup, it would become apparent that hardware adjustment was needed to match the levels. Again, the cables are quickly isolated for additional electrical testing.

I think realizing that

1. Somehow the electricity has to get to ones outlet from the power plant.
2. Very probably that connection does not consist of $200/m wire (Manhattan bay liquid cooled super conductors excluded).

is not really such a complex concept to comprehend. And if one is willing to spend $200-$300 on 2 of many 1000 meters of wire, I expect that there is at least some concept on his mind, and that's what I'm interested to hear.

6. Cables should be tested to show no measurable differences significant enough to cause predictable differences with current theory. (ie - proof one isn't defective)

For that, IMHO, is the real point of this - and why the backlash came so hard and so fast. What is being proposed is nothing so insignificant as audible differences in high-end gear, but rather claimed evidence that there exist significant holes in modern electrical theory.

Bold claims require bold evidence.

Thus Randi's bold offer (on a different, yet quite similar issue).

Missed this response, and it's an important one.

With two files - one made with Cord A, one made with Cord B, the number of trials a reviewer could conduct is unlimited.

Please excuse, some people here are just pulling your leg. You believe in audible differences between main cables. To many here that's the same category as people, who put expensive plastic stickers on their mobiles to increase reception.



Well, "otherwise" you are really just extending miles of cheapest standard copper with 2 meters expensive. Just think about this analogy: there is a 10 km long pipe, 2 inches thick, to transport water. Do you think it would improve throughput considerably, if you replace the last two meters with a 20 inch thick pipe?



That was actually pulling your leg again. Sorry!

I'm not sure what, after I described that I was actually able to reliably distinguish 10 times out of 10 between 2 cords under blind conditions, my beliefs or lack thereof have to do with anything.

It's certainly your prerogative to continue insisting that results were rigged.

If the 2 inch pipe is used to fill a large tank then yes, the 20 inch pipe can make a large difference.

I've noticed something over the years... Just about the only time the HA regulars are perceived as "hostile" is when someone is strongly suspected of bullshitting them.


Only after the throughput has been the smaller value for the first 9998 meters. But honestly... are you really assuming the "quality of power" would have a material difference?

Edit: Oh, never mind. That appears to be exactly what the OP is trying to prove.

Don't say I went from pulling a leg to shooting into my own!

Why would there be a difference? As I understand it, at the transition from 2 inch to 20 inch you get less speed, more pressure, at the transition from 20 inch to tank size you get even less speed, even more pressure. What does the intermediate step change?

I'm sure most here wouldn't mind calling themselves audiophiles if "audiophiles" hadn't hijacked the term. I'm sure everybody here loves audio and knowing about audio, and from what I see, many know much more than the average "audiophile". That's probably why "audiophool" was coined, but that term might be a bit too derogatory if you wanna have a reasonable conversation with an "audiophile".

Of course you're making things easy for yourself by considering only the one possibility of two, which just ahppens to favor your own opinions and biases. You are only seeing the possibility of evidence that favors you. Guess what! Life in the real world doesn't always work out that way. ;-)

Let's say you recorded the outputs of your amp, we listened, and found the that the recordings are indistingushable.

Let's say you recorded the outputs of your amp, we measured, and found that the differences between the samples were vanishingly small.

Several of us were recently involved in a situation where someone was "demagnetizing" vinyl LPs, and as usual claiming mind-blowing differences. They were able to provide recordings of the results of 2 experimental trials. Doing this sort of thing isn't exaclty rocket science. I've personally done it dozens of times.



Not to mention the fact that one of the major purposes of the power supplies in audio gear is to isolate the circutry from any funny stuff that might be coming in via the power line.

YOu can feed well made equipment with power that is square, rather than sine waves, and nothing about how it works changes, except that maybe a power amp powered with square waves might deliver a little more power because square waves have more energy in them than sine waves with the same amplitude.




Good point. The use of a steady sequence of X's is sufficient to invalidate the test.




Good point. I was an audiophile from about 1959 to 1973, before the serious craziness started. By the late 1980s, I didn't want to be known as an audiophile because it was synonymous with being an audiophool.



Not if the 2 meter 20 inch pipe is fed through a 10 km 2 inch pipe.



This opening comment is an example of the extreme hysteria that afflicts so many modern audiophiles. Why would it be difficult to build an ABX box that switches between 2 power cords? Of course if you answer that question in a reasonable way, you also explain why power cord ABXs have to be among the most futile wastes of time there ever was.

For that matter, why would it be difficult to build an ABX box that switches between 2 power amplifiers, 2 DACs or even 2 preamplifiers?

I still remember taking gas from hyterical audiophiles, high end audio dealers, and high end audio reviewers in the late 1970s because our ABX box had the temerity to route line-level audio signals through relays that were purpose-designed and built for the very purpose.

People weren't thinking - why were there such things as relays that were purpose-designed for routing line-level signals? The answer is that relays were commonly used in those days to route signals through high end audio mixing consoles. We simply found out who was making the best relays for the purpose, and bought the same for our purpose which was electronically the same. Our consulant for that part of the project was a PhD who studied relays for Bell labs who worked for the pleasure of assisting us with DBTs.

BTW, I don't know if it was mentioned, but the swapping of the cables, shouldn't it have been random? As in, the wife should have been flipping a coin instead of "randomly" doing it herself.

Sorry if this is not well articulated, but what I'm not understanding is this: back in post #9, jessenov1 wrote:

"My next thought was to compare one of the two (the one I prefer) to the stock cord, but finally, since I own several cords, I decided to pick for blind testing the one I thought sounded significantly different (worse) in sighted auditions."

Why not have a disinterested person put all of his power cords into a bucket, and pull out 2 of them. If the test subject could not distinguish between those 2 at a level better than chance (say more than 15 out 20), would that not provide something more meaningful than the test subject himself choosing the 2 cords for the blind test?

In my opinion, discussion assumes ability of all parties to freely express themselves. Since it doesn't seem to be the case with me (several of my replies were blocked, without explanation of course), it seems logical to stop participating alltogether.

There won't be any more replies from me in this thread.

For those who do not know the history, jessenov1 was placed on moderator preview a few months back for this post stemming from either ignorance or a lack of respect for this forum's rules:

http://www.hydrogenaudio.org/forums/index....showtopic=72493

It was stated that the preview be removed when he demonstrated that our rules were worth his adherence. As far as I'm concerned, the jury is still out.

EDIT: Abuse of PM privileges certainly doesn't help in getting back unmoderated posting privileges.

Actually, I was the first to mention a potential flaw, I guess you failed to read it:

Was your wife in the room during your trials?




Actually I don't expect you to respond to it, Mr. Bond, I expect you to....depart in a huff.

Look, I am a skeptic myself, but there are a few issues the howling wolfpack should have thought of first, before throwing out comments based on "the result is impossible so therefore the test must have been wrong or the results fake".

- is the tested object merely a "cable", or is it a "filter"? There are mains "filter cables", i.e. "power conditioners in cable shape". They might do something your in-wall copper does not, namely a bit of cleansing. That is, electrically, not due to "transparent" transmission properties, rather the contrary.

- have you never encountered a broken mains connection? I have -- it was not a mains-to-apparatus link, it was part of my installation. Noisy damn thing until I realised it was actually getting hot. It need not necessarily be "the expensive thing improves", it could be "cheap thing was not durable". In between "perfectly OK" and "so bad that you could tell the difference by connecting a lamp to it"[*] there's an interval where you could detect.

Of course, these effects could be eliminated -- the second very inexpensively by replacing with just another cheap mains cable. But that does not mean the test results are impossible.





[*] I kid you not ... Then I opened the box, stressed it a bit mechanically and saw the sparks -- and then *finally* my landlord called out the electrician, and voilà! Better sound too.

My point was that if the 2 inch pipe provides sufficient average flow for the application but inadequate peak flow then the tank plus 20 inch pipe is a solution. I don't claim that this is a particularly good analogy to a line conditioner. I was just responding to someone else's analogy.

I apparently need to remind you that an occasional apparent exception does not disprove a good general rule. You give us this anecdote about an obviously broken electrical mains connection. My goodness, sparks were flying out! How does that relate to the zillions of properly-working connections? It doesn't.