Here's a calculator which can calculate p-values for an ABX session:
http://ff123.net/abx/abx.html
ff123
Full Version: ABX calculator for your amusement
Just out of curiosity, are these p-values for a standard binomial distribution, or is some other crazy fancy statistical method used?
Thanks ff123. 
QUOTE
Originally posted by Delirium
Just out of curiosity, are these p-values for a standard binomial distribution, or is some other crazy fancy statistical method used?
Just out of curiosity, are these p-values for a standard binomial distribution, or is some other crazy fancy statistical method used?
ff123's site seems to use Monte Carlo simulation, but indeed the values will correspond with those from a standard binomial distribution.
--
GCP
QUOTE
Just out of curiosity, are these p-values for a standard binomial distribution, or is some other crazy fancy statistical method used?
Simulation is about as non-fancy as one can get. The basic idea and code is extremely simple:
CODE
/* abxpval.c
* Determines the p-value for an ABX test via simulation.
* Runs a session (trials in a session = NUMTRIALS) and
* repeats NUMSIMS times. The percentage of times that
* a session equals or exceeds the NUMCORRECT is the p-value.
*/
#include <stdio.h>
#include <math.h>
#define NUMTRIALS 16
#define NUMCORRECT 12
#define NUMSIMS 10000000L
int main(void) {
float pval; /* p-value of simulation */
long i; /* sims index */
long j; /* trials index */
long c; /* num of sessions exceeding NUMCORRECT */
long d; /* number correct */
int val; /* value of trial, 1 = correct */
for (c=0,i=0; i<NUMSIMS; i++) {
for (d=0,j=0; j<NUMTRIALS; j++) {
val = rand() % 2;
if (val == 1)
d++;
}
if (d >= NUMCORRECT)
c++;
}
pval = (float)c / NUMSIMS;
printf("pval = %7.5fn", pval);
return 0;
}
interesting, but since i dont know cr*p about C (or math), can u explain:
whats that L?
and
CODE
#define NUMSIMS 10000000L
whats that L?
and
CODE
pval = (float)c / NUMSIMS;
printf("pval = %7.5fn", pval);
please :confused: printf("pval = %7.5fn", pval);
QUOTE
interesting, but since i dont know cr*p about C (or math), can u explain:
Perhaps you should read up a little on either C, or basic maths, first. Here is the code in Python, which you may find easier to read. Moan again, and I'll give it to you in FORTRAN.
CODE
from random import randrange
def one_ABX_run(num_trials, num_correct):
"""
Performs one simulated ABX run,
returns 0 if the trial was unsuccessful
1 if the trial was successful
success happens if we get more than
num_correct trials correct, out of num_trials
"""
c = 0
for i in range(num_trials):
if (randrange(2) == 1):
c += 1
return c >= num_correct
def calculate_pval(num_trials, num_correct, num_sims):
"""
Estimates the probability that you will be successful
by chance in an ABX test where you are required to get
num_correct correct guesses out of num_trials.
Estimation is by performing an ABX with the appropriate
parameters num_sims times.
"""
c = 0
for i in range(num_sims):
c += one_ABX_run(num_trials, num_correct)
return float(c) / num_sims
num_trials = 16
num_correct = 12
num_sims = 10000
print "pval =",calculate_pval(num_trials, num_correct, num_sims)
Thinking about it, perhaps my Python code was too unreadable for you. Here's a better version:
from random import randrange
from operator import add
num_trials = 16
num_correct = 12
num_sims = 10000
print "pval =",float(len(filter(lambda i: reduce(add, map(lambda i: randrange(2), range(num_trials)))>=num_correct, range(num_sims))))/num_sims
I think we can all agree that this is much superior.
CODE
from random import randrange
from operator import add
num_trials = 16
num_correct = 12
num_sims = 10000
print "pval =",float(len(filter(lambda i: reduce(add, map(lambda i: randrange(2), range(num_trials)))>=num_correct, range(num_sims))))/num_sims
I think we can all agree that this is much superior.
tnx Jon, i think i did something
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.