IPB

Welcome Guest ( Log In | Register )

Minimum number of required ABX trials, Split from from topic ID: 92851
sauvage78
post Jan 12 2012, 00:41
Post #1





Group: Members
Posts: 677
Joined: 4-May 08
Member No.: 53282



The minimal number of trials depends on how successfull you are.
How quickly you are sucessfull shows how confident in yourself you are.

The time & number of sucessfull trials are tied, you should never separate them when judging an ABX log.

With F2K ABX component, 8 sucessfull trials in a row (& if all successfull in a row, it usually means quick) trials is the minimum for me.

As soon as you begin to fail you can easyly increase to 10 or 12 to try to "erease" your failures.
In this case if you fail once or twice you can usually still get a signifiant result although it usually means the ABXing was hard, & by consequence longer as you begin to hesitate.

Usually if you begin to fail more than 3 times on 12 trials, it begins to be so hard & you have so much hesitation that it begins to take forever to ABX. At this stage I usually give up by myself & declare that I cannot ABX as in general it means I am not sure that the audio part I am focusing on actually contains any real artefact.

This post has been edited by sauvage78: Jan 12 2012, 00:42


--------------------
CDImage+CUE
Secure [Low/C2/AR(2)]
Flac -4
Go to the top of the page
+Quote Post
 
Start new topic
Replies
nesf
post Jan 12 2012, 11:08
Post #2





Group: Members
Posts: 39
Joined: 2-January 12
Member No.: 96196



From a complete newbie perspective: It could have been handled in a more friendly fashion but someone has to point out the flaws in the methodology of the test early for the person to go on and do more useful tests for the community. You catch more flies with honey sure, but if you child is holding their pencil wrong when writing their letters you correct them early before the habit forms. You then make sure they get a real good patting on the head any time you see them doing it correctly though. Perhaps the user should have been told how to do it correctly for two samples and then given a lot of praise for doing it right when they came back with their results?

Anyway, thanks for the thread, very interesting.
Go to the top of the page
+Quote Post
saratoga
post Jan 12 2012, 23:15
Post #3





Group: Members
Posts: 4715
Joined: 2-September 02
Member No.: 3264



QUOTE (nesf @ Jan 12 2012, 05:08) *
From a complete newbie perspective: It could have been handled in a more friendly fashion but someone has to point out the flaws in the methodology of the test early for the person to go on and do more useful tests for the community. You catch more flies with honey sure, but if you child is holding their pencil wrong when writing their letters you correct them early before the habit forms. You then make sure they get a real good patting on the head any time you see them doing it correctly though. Perhaps the user should have been told how to do it correctly for two samples and then given a lot of praise for doing it right when they came back with their results?


I think we were pretty friendly in general, its just that by the time the OP told anyone what he was doing he had already made up his mind and it was too late to help him come up with a better test. IMO it wasn't until I pointed out that he had gone 5 out of 5 trials the wrong way on one test that he started to realize what ABX testing was supposed to do, and by then he was frustrated at the wasted effort and just gave up.

Of course, since it wasn't just the OP, but also some experienced posters in this thread who misunderstood how ABX testing should work, perhaps our documentation needs to be made more accessible. I'm not exactly sure how that should be done though. I'm hesitant to put up guides to doing tests that imply that there is an absolute right way to do testing. Maybe some examples of sound test setups for comparing different codecs on a problem sample would help people understand though. I get the feeling that the wall of text on most of our links discourages people.
Go to the top of the page
+Quote Post

Posts in this topic
- sauvage78   Minimum number of required ABX trials   Jan 12 2012, 00:41
- - greynol   It is expected that you choose the number of trial...   Jan 12 2012, 00:44
- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 18:41) Th...   Jan 12 2012, 00:45
- - sauvage78   Well there is the theory & there is real life ...   Jan 12 2012, 00:56
|- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 18:56) To...   Jan 12 2012, 01:05
- - greynol   He should be conducting sets of 16 trials at first...   Jan 12 2012, 01:10
- - sauvage78   I never said I judged this test valid, I only gave...   Jan 12 2012, 01:13
|- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 19:13) I ...   Jan 12 2012, 01:17
|- - greynol   QUOTE (sauvage78 @ Jan 11 2012, 16:13) I ...   Jan 12 2012, 01:20
- - sauvage78   saratoga: Yes, I had the feeling that you were thi...   Jan 12 2012, 01:27
|- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 19:27) Ye...   Jan 12 2012, 01:32
|- - greynol   QUOTE (saratoga @ Jan 11 2012, 16:32) del...   Jan 12 2012, 01:42
- - sauvage78   Well I know this topic isn't about me but my...   Jan 12 2012, 02:31
- - greynol   The difference between you and the OP is that you ...   Jan 12 2012, 02:54
- - sauvage78   I don't even need a log anymore to trust /mnt ...   Jan 12 2012, 03:30
- - IgorC   A lot of discussion here but it won't change t...   Jan 12 2012, 03:55
|- - saratoga   QUOTE (IgorC @ Jan 11 2012, 21:55) A lot ...   Jan 12 2012, 04:06
- - sauvage78   Even if I think 5 trials is too low to convince ot...   Jan 12 2012, 04:01
|- - greynol   QUOTE (sauvage78 @ Jan 11 2012, 19:01) Ig...   Jan 12 2012, 04:13
|- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 22:01) Ev...   Jan 12 2012, 04:13
- - IgorC   I think I understand what sauvage78 wants to say. ...   Jan 12 2012, 04:46
|- - saratoga   QUOTE (IgorC @ Jan 11 2012, 22:46) It...   Jan 12 2012, 04:50
|- - IgorC   QUOTE (saratoga @ Jan 12 2012, 00:50) QUO...   Jan 12 2012, 04:57
- - sauvage78   QUOTE Are you claiming that I (or greynol) have no...   Jan 12 2012, 04:54
|- - saratoga   QUOTE (sauvage78 @ Jan 11 2012, 22:54) QU...   Jan 12 2012, 05:02
|- - IgorC   QUOTE (sauvage78 @ Jan 12 2012, 00:54) It...   Jan 12 2012, 05:03
|- - greynol   QUOTE (IgorC @ Jan 11 2012, 20:03) So it...   Jan 12 2012, 05:19
- - sauvage78   IgorC: I was more trying to say that if TOS8 is ve...   Jan 12 2012, 05:02
|- - IgorC   QUOTE (sauvage78 @ Jan 12 2012, 01:02) Ig...   Jan 12 2012, 05:06
- - nesf   From a complete newbie perspective: It could have ...   Jan 12 2012, 11:08
|- - saratoga   QUOTE (nesf @ Jan 12 2012, 05:08) From a ...   Jan 12 2012, 23:15
- - apodtele   Please read Fallacy of p-value. I just want to po...   Jan 12 2012, 17:09
|- - Porcus   QUOTE (apodtele @ Jan 12 2012, 17:09) Ple...   Jan 12 2012, 22:07
- - krabapple   It looks like we're groping towards a discussi...   Jan 12 2012, 18:00
- - nesf   A Dummies guide for doing some basic two sample an...   Jan 13 2012, 01:59


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 18th April 2014 - 09:40