Probability of passing a sequencial ABX test, Split from "Mad Challenge - My Results" |
Probability of passing a sequencial ABX test, Split from "Mad Challenge - My Results" |
 Nov 10 2003, 14:39
Post
#1
|
|
|
WinABX developer Group: Developer Posts: 1578 Joined: 1-October 01 Member No.: 137  |
Edit : this discussion was started there.
I like viewing my ABX results during the test, and stopping when I have reached low enough p. That's what some people call a "sequential" ABX test. As Pio2001 says, for such a kind of test, getting a p = 5% is not enough pass the test. It seems (someone correct me if I'm wrong, please) that, for such a kind of sequential test, getting a p<1% is is enough for saying the the test has been passed. This post has been edited by tigre: Nov 12 2003, 16:48 |
 |
 
|
 |
|
Nov 10 2003, 19:17
Post
#2
|
|
 Group: Members Posts: 473 Joined: 7-June 02 Member No.: 2244 |
QUOTE (Gabriel @ Nov 10 2003, 07:04 PM) .... 119 of 200, p = 0.004 ... (am I god?) What version of ABCHR are you using? There was some doubt whether the random number generator used in older versions is reliable or not. The result does indeed mean: The probability to score 119 or better out of 200 by guessing is 0.0043. |
|
|
|
KikeG Probability of passing a sequencial ABX test Nov 10 2003, 14:39
guruboolez QUOTE (KikeG @ Nov 10 2003, 02:39 PM)I like v... Nov 10 2003, 14:55 KikeG 5% is valid only if either you have fixed in advan... Nov 10 2003, 15:18 tigre Is there a rule of thumb for tests without fixed n... Nov 10 2003, 15:22 guruboolez QUOTE (KikeG @ Nov 10 2003, 03:18 PM)5% is va... Nov 10 2003, 15:25 KikeG QUOTE (guruboolez @ Nov 10 2003, 03:25 PM)Sho... Nov 10 2003, 15:51 KikeG QUOTE (tigre @ Nov 10 2003, 03:22 PM)Is there... Nov 10 2003, 15:53 tigre QUOTE (KikeG @ Nov 10 2003, 04:53 PM)I don... Nov 10 2003, 16:10 ScorLibran QUOTE (tigre @ Nov 10 2003, 10:10 AM)I want t... Nov 10 2003, 16:18 Lev QUOTE Usually, a p <= 0.05 is considered a sign... Nov 10 2003, 16:19 tigre QUOTE (ScorLibran @ Nov 10 2003, 05:18 PM)QUO... Nov 10 2003, 16:43 tigre QUOTE (Lev @ Nov 10 2003, 05:19 PM)How is the... Nov 10 2003, 16:44 guruboolez QUOTE (KikeG @ Nov 10 2003, 03:51 PM)QUOTE (g... Nov 10 2003, 17:33 AstralStorm I'd very much appreciate an option (in ABC/HR ... Nov 10 2003, 17:45 Gabriel Based on the current discussion, I tryed an experi... Nov 10 2003, 17:54 Gabriel Still continuing the experiment:
54 of 85, p = 0... Nov 10 2003, 17:58 guruboolez QUOTE (Gabriel @ Nov 10 2003, 05:54 PM)Based ... Nov 10 2003, 18:01 Gabriel ....
119 of 200, p = 0.004
...
(am I god?) Nov 10 2003, 18:04 ErikS QUOTE (Gabriel @ Nov 10 2003, 06:04 PM)....
1... Nov 10 2003, 18:15 guruboolez QUOTE (Gabriel @ Nov 10 2003, 06:04 PM)....
1... Nov 10 2003, 18:25 guruboolez On ABX tests, if you did one error each three tria... Nov 10 2003, 18:49 Moneo QUOTE 119 of 200, p = 0.004
I wonder what does Win... Nov 10 2003, 19:00 Continuum QUOTE (guruboolez @ Nov 10 2003, 03:55 PM)Doe... Nov 10 2003, 19:28 ff123 QUOTE (Gabriel @ Nov 10 2003, 08:58 AM)Still ... Nov 10 2003, 19:31 schnofler Just a few thoughts on this whole discussion: In g... Nov 10 2003, 19:34 Moneo The randomizer in WinABX does seem to be deficient... Nov 10 2003, 19:36 guruboolez QUOTE (schnofler @ Nov 10 2003, 07:34 PM)This... Nov 10 2003, 20:14 tigre QUOTE (Continuum @ Nov 10 2003, 08:28 PM)QUOT... Nov 10 2003, 20:56 Continuum QUOTE (tigre @ Nov 10 2003, 09:56 PM)Do you k... Nov 10 2003, 21:06 Mac QUOTE (Moneo @ Nov 10 2003, 06:36 PM)The rand... Nov 10 2003, 21:18 AstralStorm Heh, 116/200 is nearly 35% chance of missing accor... Nov 10 2003, 21:28 Pio2001 The problem is obvious : in his random test, Gabri... Nov 10 2003, 21:37 Continuum QUOTE (Mac @ Nov 10 2003, 10:18 PM)QUOTE (Mon... Nov 10 2003, 21:39 Continuum QUOTE (AstralStorm @ Nov 10 2003, 10:28 PM)He... Nov 10 2003, 21:42 Pio2001 QUOTE (Continuum @ Nov 10 2003, 09:42 PM)QUOT... Nov 10 2003, 21:47 Mac I was going on the standard deviation of a 202 tri... Nov 10 2003, 21:57 schnofler I think one little suggestion is necessary here: P... Nov 10 2003, 22:00 Pio2001 You're right.
I'd like to see a graph of ... Nov 10 2003, 22:12 Moneo QUOTE (Mac @ Nov 10 2003, 09:18 PM)Unless my ... Nov 10 2003, 22:13 Continuum QUOTE (Mac @ Nov 10 2003, 10:18 PM)both of th... Nov 10 2003, 22:14 AstralStorm Check with what probability you can get this resul... Nov 10 2003, 22:30 schnofler QUOTE (Moneo)Well, you could help debunking them b... Nov 10 2003, 22:54 Mac Erg, I mixed myself up a little When saying +/- ... Nov 10 2003, 23:18 schnofler QUOTE (Mac)Choosing all A: 98/200, p=63.8
Choosing... Nov 10 2003, 23:20 Pio2001 Here's the graph for the Pval of my 200 answer... Nov 10 2003, 23:20 Pio2001 Schnofler, could you post your program, of the log... Nov 10 2003, 23:23 Gabriel another test:
15 of 24, p = 0.154
16 of 25, p ... Nov 10 2003, 23:34 schnofler QUOTE (pio2001)Schnofler, could you post your prog... Nov 10 2003, 23:42 Moneo QUOTE (schnofler @ Nov 10 2003, 10:54 PM)OK, ... Nov 10 2003, 23:46 Moneo Anyway, I'd like to know what did KikeG use to... Nov 11 2003, 00:00 Pio2001 Thanks, Schnofler, this is exactly what I wanted. ... Nov 11 2003, 00:24 Pio2001 Note in the first graph, that if we only had the f... Nov 11 2003, 00:28 schnofler QUOTE (Pio2001)The second graph behaves exactly ho... Nov 11 2003, 00:36 phong There are many "weak" pseudorandom numbe... Nov 11 2003, 02:27 ff123 QUOTE (phong @ Nov 10 2003, 05:27 PM)There ar... Nov 11 2003, 02:58 Pio2001 Here's how the p value behaves, according to t... Nov 11 2003, 05:05 KikeG QUOTE (Continuum @ Nov 10 2003, 07:28 PM)But ... Nov 11 2003, 09:33 KikeG About WinABX PRNG, from v0.4 it also uses the Mers... Nov 11 2003, 09:36 Moneo QUOTE (KikeG @ Nov 11 2003, 09:36 AM)About Wi... Nov 11 2003, 11:11 KikeG QUOTE (Moneo @ Nov 11 2003, 11:11 AM)Could yo... Nov 11 2003, 11:58 Moneo QUOTE (KikeG @ Nov 11 2003, 11:58 AM)I tried ... Nov 11 2003, 12:49 KikeG QUOTE (Moneo @ Nov 11 2003, 12:49 PM)Maybe yo... Nov 11 2003, 15:37 schnofler Ok, I tested the new version, here you go:
1. 100... Nov 11 2003, 16:10 tigre the 3 following posts have been moved from Upsampl... Nov 12 2003, 15:18 KikeG QUOTE (tigre @ Nov 12 2003, 03:18 PM)I'm ... Nov 12 2003, 15:35 tigre Continuum's corrected p-values are only correc... Nov 12 2003, 16:42 KikeG QUOTE (tigre @ Nov 12 2003, 04:42 PM)Continuu... Nov 12 2003, 17:15
Continuum I'm not a statistics guru, but here is what I ... Nov 12 2003, 18:51 |
|
Lo-Fi Version | Time is now: 26th May 2013 - 07:07 |
