IPB

Welcome Guest ( Log In | Register )

Probability of passing a sequencial ABX test, Split from "Mad Challenge - My Results"
KikeG
post Nov 10 2003, 14:39
Post #1


WinABX developer


Group: Developer
Posts: 1578
Joined: 1-October 01
Member No.: 137



Edit : this discussion was started there.

I like viewing my ABX results during the test, and stopping when I have reached low enough p. That's what some people call a "sequential" ABX test. As Pio2001 says, for such a kind of test, getting a p = 5% is not enough pass the test. It seems (someone correct me if I'm wrong, please) that, for such a kind of sequential test, getting a p<1% is is enough for saying the the test has been passed.

This post has been edited by tigre: Nov 12 2003, 16:48
Go to the top of the page
+Quote Post
 
Start new topic
Replies
Continuum
post Nov 10 2003, 19:17
Post #2





Group: Members
Posts: 473
Joined: 7-June 02
Member No.: 2244



QUOTE (Gabriel @ Nov 10 2003, 07:04 PM)
....
119 of 200, p = 0.004

...

(am I god?)

What version of ABCHR are you using? There was some doubt whether the random number generator used in older versions is reliable or not.

The result does indeed mean: The probability to score 119 or better out of 200 by guessing is 0.0043.
Go to the top of the page
+Quote Post

Posts in this topic
- KikeG   Probability of passing a sequencial ABX test   Nov 10 2003, 14:39
- - guruboolez   QUOTE (KikeG @ Nov 10 2003, 02:39 PM)I like v...   Nov 10 2003, 14:55
- - KikeG   5% is valid only if either you have fixed in advan...   Nov 10 2003, 15:18
- - tigre   Is there a rule of thumb for tests without fixed n...   Nov 10 2003, 15:22
- - guruboolez   QUOTE (KikeG @ Nov 10 2003, 03:18 PM)5% is va...   Nov 10 2003, 15:25
- - KikeG   QUOTE (guruboolez @ Nov 10 2003, 03:25 PM)Sho...   Nov 10 2003, 15:51
- - KikeG   QUOTE (tigre @ Nov 10 2003, 03:22 PM)Is there...   Nov 10 2003, 15:53
- - tigre   QUOTE (KikeG @ Nov 10 2003, 04:53 PM)I don...   Nov 10 2003, 16:10
- - ScorLibran   QUOTE (tigre @ Nov 10 2003, 10:10 AM)I want t...   Nov 10 2003, 16:18
- - Lev   QUOTE Usually, a p <= 0.05 is considered a sign...   Nov 10 2003, 16:19
- - tigre   QUOTE (ScorLibran @ Nov 10 2003, 05:18 PM)QUO...   Nov 10 2003, 16:43
- - tigre   QUOTE (Lev @ Nov 10 2003, 05:19 PM)How is the...   Nov 10 2003, 16:44
- - guruboolez   QUOTE (KikeG @ Nov 10 2003, 03:51 PM)QUOTE (g...   Nov 10 2003, 17:33
- - AstralStorm   I'd very much appreciate an option (in ABC/HR ...   Nov 10 2003, 17:45
- - Gabriel   Based on the current discussion, I tryed an experi...   Nov 10 2003, 17:54
- - Gabriel   Still continuing the experiment: 54 of 85, p = 0...   Nov 10 2003, 17:58
- - guruboolez   QUOTE (Gabriel @ Nov 10 2003, 05:54 PM)Based ...   Nov 10 2003, 18:01
- - Gabriel   .... 119 of 200, p = 0.004 ... (am I god?)   Nov 10 2003, 18:04
- - ErikS   QUOTE (Gabriel @ Nov 10 2003, 06:04 PM).... 1...   Nov 10 2003, 18:15
- - guruboolez   QUOTE (Gabriel @ Nov 10 2003, 06:04 PM).... 1...   Nov 10 2003, 18:25
- - guruboolez   On ABX tests, if you did one error each three tria...   Nov 10 2003, 18:49
- - Moneo   QUOTE 119 of 200, p = 0.004 I wonder what does Win...   Nov 10 2003, 19:00
- - Continuum   QUOTE (Gabriel @ Nov 10 2003, 07:04 PM).... 1...   Nov 10 2003, 19:17
- - Continuum   QUOTE (guruboolez @ Nov 10 2003, 03:55 PM)Doe...   Nov 10 2003, 19:28
- - ff123   QUOTE (Gabriel @ Nov 10 2003, 08:58 AM)Still ...   Nov 10 2003, 19:31
- - schnofler   Just a few thoughts on this whole discussion: In g...   Nov 10 2003, 19:34
- - Moneo   The randomizer in WinABX does seem to be deficient...   Nov 10 2003, 19:36
- - guruboolez   QUOTE (schnofler @ Nov 10 2003, 07:34 PM)This...   Nov 10 2003, 20:14
- - tigre   QUOTE (Continuum @ Nov 10 2003, 08:28 PM)QUOT...   Nov 10 2003, 20:56
- - Continuum   QUOTE (tigre @ Nov 10 2003, 09:56 PM)Do you k...   Nov 10 2003, 21:06
- - Mac   QUOTE (Moneo @ Nov 10 2003, 06:36 PM)The rand...   Nov 10 2003, 21:18
- - AstralStorm   Heh, 116/200 is nearly 35% chance of missing accor...   Nov 10 2003, 21:28
- - Pio2001   The problem is obvious : in his random test, Gabri...   Nov 10 2003, 21:37
- - Continuum   QUOTE (Mac @ Nov 10 2003, 10:18 PM)QUOTE (Mon...   Nov 10 2003, 21:39
- - Continuum   QUOTE (AstralStorm @ Nov 10 2003, 10:28 PM)He...   Nov 10 2003, 21:42
- - Pio2001   QUOTE (Continuum @ Nov 10 2003, 09:42 PM)QUOT...   Nov 10 2003, 21:47
- - Mac   I was going on the standard deviation of a 202 tri...   Nov 10 2003, 21:57
- - schnofler   I think one little suggestion is necessary here: P...   Nov 10 2003, 22:00
- - Pio2001   You're right. I'd like to see a graph of ...   Nov 10 2003, 22:12
- - Moneo   QUOTE (Mac @ Nov 10 2003, 09:18 PM)Unless my ...   Nov 10 2003, 22:13
- - Continuum   QUOTE (Mac @ Nov 10 2003, 10:18 PM)both of th...   Nov 10 2003, 22:14
- - AstralStorm   Check with what probability you can get this resul...   Nov 10 2003, 22:30
- - schnofler   QUOTE (Moneo)Well, you could help debunking them b...   Nov 10 2003, 22:54
- - Mac   Erg, I mixed myself up a little When saying +/- ...   Nov 10 2003, 23:18
- - schnofler   QUOTE (Mac)Choosing all A: 98/200, p=63.8 Choosing...   Nov 10 2003, 23:20
- - Pio2001   Here's the graph for the Pval of my 200 answer...   Nov 10 2003, 23:20
- - Pio2001   Schnofler, could you post your program, of the log...   Nov 10 2003, 23:23
- - Gabriel   another test: 15 of 24, p = 0.154 16 of 25, p ...   Nov 10 2003, 23:34
- - schnofler   QUOTE (pio2001)Schnofler, could you post your prog...   Nov 10 2003, 23:42
- - Moneo   QUOTE (schnofler @ Nov 10 2003, 10:54 PM)OK, ...   Nov 10 2003, 23:46
- - Moneo   Anyway, I'd like to know what did KikeG use to...   Nov 11 2003, 00:00
- - Pio2001   Thanks, Schnofler, this is exactly what I wanted. ...   Nov 11 2003, 00:24
- - Pio2001   Note in the first graph, that if we only had the f...   Nov 11 2003, 00:28
- - schnofler   QUOTE (Pio2001)The second graph behaves exactly ho...   Nov 11 2003, 00:36
- - phong   There are many "weak" pseudorandom numbe...   Nov 11 2003, 02:27
- - ff123   QUOTE (phong @ Nov 10 2003, 05:27 PM)There ar...   Nov 11 2003, 02:58
- - Pio2001   Here's how the p value behaves, according to t...   Nov 11 2003, 05:05
- - KikeG   QUOTE (Continuum @ Nov 10 2003, 07:28 PM)But ...   Nov 11 2003, 09:33
- - KikeG   About WinABX PRNG, from v0.4 it also uses the Mers...   Nov 11 2003, 09:36
- - Moneo   QUOTE (KikeG @ Nov 11 2003, 09:36 AM)About Wi...   Nov 11 2003, 11:11
- - KikeG   QUOTE (Moneo @ Nov 11 2003, 11:11 AM)Could yo...   Nov 11 2003, 11:58
- - Moneo   QUOTE (KikeG @ Nov 11 2003, 11:58 AM)I tried ...   Nov 11 2003, 12:49
- - KikeG   QUOTE (Moneo @ Nov 11 2003, 12:49 PM)Maybe yo...   Nov 11 2003, 15:37
- - schnofler   Ok, I tested the new version, here you go: 1. 100...   Nov 11 2003, 16:10
- - tigre   the 3 following posts have been moved from Upsampl...   Nov 12 2003, 15:18
- - KikeG   QUOTE (tigre @ Nov 12 2003, 03:18 PM)I'm ...   Nov 12 2003, 15:35
- - tigre   Continuum's corrected p-values are only correc...   Nov 12 2003, 16:42
- - KikeG   QUOTE (tigre @ Nov 12 2003, 04:42 PM)Continuu...   Nov 12 2003, 17:15
- - Continuum   I'm not a statistics guru, but here is what I ...   Nov 12 2003, 18:51


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 20th April 2014 - 06:49