What's the problem with double-blind testing? |
What's the problem with double-blind testing? |
Oct 19 2005, 03:03
Post
#1
|
|
|
Group: Members Posts: 11 Joined: 15-October 05 Member No.: 25138 |
I happened to pick up an issue of stereophile at a record store I visited and I was pretty shocked to see a seemingly intelligent person in the correspondence section bashing double blind testing as being unreliable. I'm afraid I don't understand his angle of attack. I don't see how anything could be a more reliable test of sound quality differences than a properly conducted double blind listening test.
I'm almost afraid to read the rest of the magazine if this is the kind of letter they think is worth publishing. Is there an audio magazine that isn't filled with this kind of thinking? |
|
|
|
![]() |
Oct 19 2005, 11:54
Post
#2
|
|
|
WinABX developer Group: Developer Posts: 1578 Joined: 1-October 01 Member No.: 137 |
The "goosebump", emotional factor can be caused by placebo effect, you can't be sure that is not the cause. People has used same principle (emotional response and the like) to say cables sound different, for example.
You can do long-term blind tests too, not just the quick-switch, short snippet, typical abx test. This post has been edited by KikeG: Oct 19 2005, 11:55 |
|
|
|
Oct 19 2005, 12:06
Post
#3
|
|
|
Group: Members Posts: 224 Joined: 8-July 05 Member No.: 23210 |
QUOTE (KikeG @ Oct 19 2005, 06:54 PM) The "goosebump", emotional factor can be caused by placebo effect, you can't be sure that is not the cause. People has used same principle (emotional response and the like) to say cables sound different, for example. You can do long-term blind tests too, not just the quick-switch, short snippet, typical abx test. What do you mean by long term abx? how do I do this? Play entire songs? actually can you give me a link that explains placebo. thx |
|
|
|
Oct 19 2005, 18:31
Post
#4
|
|
![]() Group: Members Posts: 2525 Joined: 25-July 02 From: South Korea Member No.: 2782 |
QUOTE (Donunus @ Oct 19 2005, 08:06 PM) The word placebo, when used in audio circles, is a little different from the medical term, where people can actually benefit a little from dummy treatments. According to ff123, it's closer to expectation bias, where a person's expectation can influence his perception, if I understand correctly. -------------------- http://blacksun.ivyro.net/vorbis/vorbisfaq.htm
|
|
|
|
mirrorsawlljk What's the problem with double-blind testing? Oct 19 2005, 03:03
bubka some people can actually detect specific codecs by... Oct 19 2005, 03:07
TheQat QUOTE (mirrorsawlljk @ Oct 18 2005, 06:03 PM)... Oct 19 2005, 03:27
singaiya QUOTE (TheQat @ Oct 18 2005, 06:27 PM)QUOTE (... Oct 19 2005, 04:31
Yaztromo QUOTE (TheQat @ Oct 19 2005, 03:27 AM)Edit: H... Oct 19 2005, 23:05
Tahnru The closest thing I have seen to a legitimate crit... Oct 19 2005, 04:29
Axon The Audio Critic is notably pro-DBT. Oct 19 2005, 07:06
PoisonDan QUOTE (Axon @ Oct 19 2005, 08:06 AM)The Audio... Oct 19 2005, 12:18
onthejazz Interesting publication. I like it, too bad its no... Oct 19 2005, 10:03
Donunus The writer of the letter in stereophile is incorre... Oct 19 2005, 11:00
Danimal QUOTE (Donunus @ Oct 19 2005, 05:00 AM)The wr... Oct 19 2005, 20:20
Donunus QUOTE (Danimal @ Oct 20 2005, 03:20 AM)QUOTE ... Oct 20 2005, 01:51
stephanV QUOTE (Donunus @ Oct 20 2005, 02:51 AM)Well, ... Oct 20 2005, 08:48

bryant QUOTE (stephanV @ Oct 19 2005, 11:48 PM)QUOTE... Oct 20 2005, 18:11


Pio2001 QUOTE (bryant @ Oct 20 2005, 07:11 PM)To tell... Oct 20 2005, 20:48



singaiya Thanks for the papers, David. The first one is a g... Oct 20 2005, 22:13



duff QUOTE (Pio2001 @ Oct 20 2005, 07:48 PM)QUOTE ... Oct 20 2005, 22:52



stephanV QUOTE (duff @ Oct 20 2005, 11:52 PM)It's ... Oct 20 2005, 23:11




duff QUOTE Might be... or might be not. You can't c... Oct 20 2005, 23:36




KikeG QUOTE (duff @ Oct 20 2005, 11:36 PM)I think i... Oct 21 2005, 00:00





rjamorim QUOTE (KikeG @ Oct 20 2005, 09:00 PM)Also, ps... Oct 21 2005, 00:15





duff QUOTE (KikeG @ Oct 20 2005, 11:00 PM)QUOTE (d... Oct 21 2005, 00:29





rjamorim QUOTE (duff @ Oct 20 2005, 09:29 PM)I'm n... Oct 21 2005, 01:03





Mike Giacomelli QUOTE (duff @ Oct 20 2005, 04:29 PM)The audit... Oct 21 2005, 05:56





KikeG QUOTE (duff @ Oct 21 2005, 12:29 AM)From the ... Oct 21 2005, 08:10




stephanV QUOTE (duff @ Oct 21 2005, 12:36 AM)I think i... Oct 21 2005, 08:34




Woodinville QUOTE (stephanV @ Oct 20 2005, 11:34 PM)Sorry... Oct 21 2005, 21:19



krabapple QUOTE (duff @ Oct 20 2005, 04:52 PM)QUOTE (Pi... Oct 21 2005, 05:25



antz QUOTE (duff @ Oct 20 2005, 10:52 PM)QUOTE (Pi... Oct 21 2005, 13:39



Woodinville QUOTE (antz @ Oct 21 2005, 04:39 AM)Seems to ... Oct 21 2005, 21:20


stephanV QUOTE (bryant @ Oct 20 2005, 07:11 PM)It turn... Oct 20 2005, 21:43

Donunus QUOTE (stephanV @ Oct 20 2005, 03:48 PM)QUOTE... Oct 21 2005, 04:10

stephanV QUOTE (Donunus @ Oct 21 2005, 05:10 AM)I do n... Oct 21 2005, 08:43

user I recall, we have had successful abx tests even he... Oct 21 2005, 09:20

Pio2001 QUOTE (user @ Oct 21 2005, 10:20 AM)I recall,... Oct 21 2005, 12:18


ff123 QUOTE (Pio2001 @ Oct 21 2005, 03:18 AM)QUOTE ... Oct 21 2005, 17:43

krabapple QUOTE (user @ Oct 21 2005, 03:20 AM)I recall,... Oct 21 2005, 21:54
Cartoon QUOTE (Donunus @ Oct 20 2005, 02:51 AM)I do h... Nov 12 2005, 13:40

Woodinville QUOTE (kjoonlee @ Oct 19 2005, 09:31 AM)QUOTE... Oct 21 2005, 01:27

duff QUOTE If you consider that we are wired to detect ... Oct 21 2005, 01:38

Woodinville QUOTE (duff @ Oct 20 2005, 04:38 PM)So we are... Oct 21 2005, 21:15

duff QUOTE (Woodinville @ Oct 21 2005, 08:15 PM)QU... Oct 21 2005, 23:43

Woodinville QUOTE (duff @ Oct 21 2005, 02:43 PM)When I sa... Oct 22 2005, 08:00
Lyx QUOTE (Donunus @ Oct 19 2005, 01:06 PM)QUOTE ... Oct 20 2005, 06:07
Donunus QUOTE (Lyx @ Oct 20 2005, 01:07 PM)QUOTE (Don... Oct 20 2005, 16:51
Lyx QUOTE (Donunus @ Oct 20 2005, 05:51 PM)QUOTE ... Oct 20 2005, 21:10
KikeG Well, I think expectation effects in listening tes... Oct 19 2005, 19:33
krabapple QUOTE (KikeG @ Oct 19 2005, 10:33 AM)Well, I ... Oct 19 2005, 20:12
krabapple QUOTE (mirrorsawlljk @ Oct 18 2005, 06:03 PM)... Oct 19 2005, 20:06
Halcyon There is nothing inherently bad about double blind... Oct 19 2005, 20:29
ff123 Tests that try to distinguish very small effects *... Oct 19 2005, 20:53
krabapple QUOTE (ff123 @ Oct 19 2005, 11:53 AM)Tests th... Oct 19 2005, 23:40
ff123 QUOTE (krabapple @ Oct 19 2005, 02:40 PM)Inte... Oct 20 2005, 05:38
fcmts There is another problem similar to wine blind tes... Oct 19 2005, 22:15
stephanV I don't see how that is a problem. The goal of... Oct 19 2005, 22:22
duff One distinction relevant to this issue is the diff... Oct 20 2005, 23:00
duff QUOTE The opposite of "blind", I suppose... Oct 21 2005, 01:27
Jun-Dai I think it's pretty clear that double-blind te... Oct 21 2005, 07:22
KikeG For helping doing long-term, casual listening-like... Oct 21 2005, 08:30
duff QUOTE Seems to be a lack of understanding of somet... Oct 21 2005, 16:48
stephanV QUOTE (duff @ Oct 21 2005, 05:48 PM)The reaso... Oct 21 2005, 17:29

duff QUOTE (stephanV @ Oct 21 2005, 04:29 PM)QUOTE... Oct 21 2005, 20:06

stephanV QUOTE (duff @ Oct 21 2005, 09:06 PM)QUOTE
Wh... Oct 21 2005, 20:21

bryant QUOTE (stephanV @ Oct 21 2005, 11:21 AM)QUOTE... Oct 21 2005, 22:53

rjamorim QUOTE (bryant @ Oct 21 2005, 07:53 PM)But I d... Oct 21 2005, 23:12

stephanV QUOTE (bryant @ Oct 21 2005, 11:53 PM)So, if ... Oct 22 2005, 00:00

ChiGung QUOTE (bryant @ Oct 21 2005, 10:53 PM)What... Oct 22 2005, 00:25


bryant QUOTE (ChiGung @ Oct 21 2005, 03:25 PM)I have... Oct 22 2005, 23:39


ChiGung just thinking -i do get all sorts of weird sensati... Oct 23 2005, 00:34

Woodinville QUOTE (bryant @ Oct 21 2005, 01:53 PM)So, if ... Oct 22 2005, 07:52

bryant QUOTE (Woodinville @ Oct 21 2005, 10:52 PM)QU... Oct 22 2005, 23:32

Woodinville QUOTE (bryant @ Oct 22 2005, 02:32 PM)The onl... Oct 24 2005, 21:51
Lyx QUOTE (duff @ Oct 21 2005, 05:48 PM)QUOTE See... Oct 21 2005, 17:35
krabapple QUOTE (duff @ Oct 21 2005, 10:48 AM)QUOTE See... Oct 21 2005, 22:01
duff QUOTE It sounds counterintuitive that my brain wou... Oct 22 2005, 00:30
ChiGung QUOTE (duff @ Oct 22 2005, 12:30 AM)Frankly, ... Oct 22 2005, 00:47

duff QUOTE (ChiGung @ Oct 21 2005, 11:47 PM)QUOTE ... Oct 22 2005, 01:14

ChiGung QUOTE (duff @ Oct 22 2005, 01:14 AM)The reduc... Oct 22 2005, 01:26
Woodinville QUOTE (duff @ Oct 21 2005, 03:30 PM)Filtering... Oct 22 2005, 08:07
ChiGung A simplistic example - a sine wave of exactly 3122... Oct 22 2005, 00:59
Woodinville QUOTE (ChiGung @ Oct 21 2005, 03:59 PM)A simp... Oct 22 2005, 08:08
ChiGung QUOTE (Woodinville @ Oct 22 2005, 08:08 AM)QU... Oct 22 2005, 14:09
ChiGung -Sorry for flooding a bit there. Tasty subject Oct 22 2005, 01:01
duff So sorry for the long delay. I didn't forget a... Nov 10 2005, 20:10
Woodinville QUOTE (duff @ Nov 10 2005, 11:10 AM)Woodinsid... Nov 10 2005, 20:23
duff I don't understand why you would want to maint... Nov 11 2005, 00:41
Woodinville QUOTE (duff @ Nov 10 2005, 03:41 PM)Somehow, ... Nov 11 2005, 21:52
Pio2001 QUOTE (duff @ Nov 11 2005, 01:41 AM)The ABX p... Nov 12 2005, 03:00
Woodinville QUOTE (Pio2001 @ Nov 11 2005, 06:00 PM)I can ... Nov 12 2005, 09:33
user There aren't theoretical flaws with abx or DB ... Nov 12 2005, 10:52
KikeG Sorry, you still have provided no evidence that wh... Nov 11 2005, 14:34
Woodinville QUOTE (KikeG @ Nov 11 2005, 05:34 AM)Sorry, y... Nov 11 2005, 21:56
Pio2001 QUOTE (Woodinville @ Nov 11 2005, 10:56 PM)Su... Nov 12 2005, 02:32
duff Let's back up a moment...
There are at least ... Nov 11 2005, 18:37
KikeG About the first point, your degraded examples are ... Nov 11 2005, 19:57
duff QUOTE Well, if it's inaudible, there's a g... Nov 11 2005, 21:19
Woodinville QUOTE (duff @ Nov 11 2005, 12:19 PM)In other ... Nov 11 2005, 21:59![]() ![]() |
|
Lo-Fi Version | Time is now: 22nd May 2013 - 06:28 |