Statistics For Abx |
Statistics For Abx |
 Aug 27 2002, 16:58
Post
#1
|
|
 ABC/HR developer, ff123.net admin Group: Developer (Donating) Posts: 1396 Joined: 24-September 01 Member No.: 12  |
Hopefully in the near future, I can implement an indicator of whether or not a listener should continue to perform ABX testing based on certain specified parameters:
alpha: probability of stating that a difference occurs when it does not (this is the parameter we are typically concerned with, which is usually set to 0.05) beta: probability of stating that no difference occurs when it does p0: the expected proportion of correct decisions when the samples are identical (0.5 for ABX) p1: the expected proportion of correct decisions when the odd sample is detected (other than by guess). We have historically not concerned ourselves with beta and p1, but I think it would be advantageous to do so for tests of very subtle differences. ff123 |
 |
 
|
ff123 Statistics For Abx Aug 27 2002, 16:58
Continuum Is this related to my question concerning guessing... Aug 27 2002, 18:51 ff123 Ideally, I would pop up a graph with the y axis sh... Aug 27 2002, 20:57 Guest ff13,
You may want to consider expressing the run... Aug 27 2002, 21:52 shday ... hmm, I can post when not logged in???
ff123,
... Aug 27 2002, 21:54 ff123 Well, it turns out I don't understand the stat... Aug 28 2002, 07:28 Delirium I don't have my statistics books with me and i... Aug 28 2002, 10:24 ff123 I guess what I'm having trouble understanding ... Aug 28 2002, 14:50 ff123 One more thing, Table 1 on the page I listed above... Aug 28 2002, 16:49 shday QUOTE (ff123 @ Aug 28 2002, 09:50 AM)I guess ... Aug 28 2002, 16:49 ff123 I didn't understand the previous thread then, ... Aug 28 2002, 17:15 Continuum QUOTE (shday @ Aug 27 2002, 10:54 PM)... hmm,... Aug 28 2002, 20:16 Continuum Situation 1: the number of trials is determined. D... Aug 28 2002, 20:23 ff123 Here is a web page of the clearest explanation I h... Aug 28 2002, 23:45 shday QUOTE (Continuum @ Aug 28 2002, 03:16 PM)shda... Aug 29 2002, 00:45 shday QUOTE (ff123 @ Aug 28 2002, 06:45 PM)The best... Aug 29 2002, 02:01 Continuum I just did a quick calculation using a modified pa... Aug 29 2002, 08:22 ff123 I hacked a quick and dirty sequential ABX simulato... Aug 29 2002, 08:59 Continuum I've wrote a program for evaluating the correc... Aug 29 2002, 12:14 ff123 I think I'll probably create several different... Aug 29 2002, 15:43 ff123 One more thing: I'm not sure if it hurts to l... Aug 29 2002, 16:35 shday QUOTE (ff123 @ Aug 29 2002, 11:35 AM)One more... Aug 29 2002, 17:10 ff123 QUOTE If you want the tool to be statistically sou... Aug 29 2002, 18:27 Continuum QUOTE The best way to minimize this problem would ... Aug 29 2002, 20:11 ff123 Here is the lookup table I would use for the 28-tr... Aug 29 2002, 20:25 shday QUOTE (shday @ Aug 29 2002, 12:10 PM)QUOTE (f... Aug 29 2002, 22:21 shday QUOTE (ff123 @ Aug 29 2002, 01:27 PM)Are ther... Aug 29 2002, 23:31 ff123 QUOTE (shday @ Aug 29 2002, 02:31 PM)QUOTE
P... Aug 30 2002, 00:20 shday QUOTE (ff123 @ Aug 29 2002, 07:20 PM)To answe... Aug 30 2002, 00:51 ff123 QUOTE (shday @ Aug 29 2002, 03:51 PM)Sorry if... Aug 30 2002, 01:46 shday I guess most of the time a tester wouldn't be ... Aug 30 2002, 02:35 ff123 QUOTE (shday @ Aug 29 2002, 05:35 PM)I'd ... Aug 30 2002, 03:20 Continuum Here's a new version of the Excel-sheet. It al... Aug 30 2002, 09:40 Continuum QUOTE (shday @ Aug 29 2002, 11:21 PM)It is tr... Aug 30 2002, 12:52 Continuum This should be a mode that allows 5/5 with total s... Aug 30 2002, 12:54 ff123 QUOTE (Continuum @ Aug 30 2002, 03:54 AM)This... Aug 30 2002, 13:27 shday QUOTE (Continuum @ Aug 30 2002, 07:52 AM)On d... Aug 30 2002, 16:04 shday QUOTE (Continuum @ Aug 30 2002, 07:52 AM)QUOT... Aug 30 2002, 16:12 ff123 QUOTE (Continuum @ Aug 30 2002, 12:40 AM)The ... Aug 30 2002, 17:19 Continuum QUOTE How about something like the following, wher... Aug 30 2002, 17:36 Continuum QUOTE (shday @ Aug 30 2002, 05:12 PM)QUOTE (C... Aug 30 2002, 17:40 ff123 I have uploaded an updated binary to:
http://ff123... Aug 30 2002, 17:52 shday QUOTE (Continuum @ Aug 30 2002, 12:40 PM)It d... Aug 30 2002, 17:57 ff123 I've been comparing the simulation vs. the bin... Aug 30 2002, 19:54 ff123 QUOTE (Continuum @ Aug 30 2002, 12:40 AM)QUOT... Aug 30 2002, 21:42 Continuum QUOTE (ff123 @ Aug 30 2002, 06:52 PM)But in a... Aug 30 2002, 21:44 ff123 QUOTE (Continuum @ Aug 30 2002, 12:44 PM)QUOT... Aug 30 2002, 22:10 ff123 I've been thinking some more about the in-betw... Aug 30 2002, 22:17 ff123 Ok, here is my corrected 28-trials profile
CODE10... Aug 30 2002, 22:51 ff123 I updated seqsimsource.zip and seqsim.zip on my we... Aug 30 2002, 23:37 shday QUOTE (ff123 @ Aug 30 2002, 05:10 PM)As I sai... Aug 31 2002, 00:01 ff123 QUOTE (shday @ Aug 30 2002, 03:01 PM)QUOTE (f... Aug 31 2002, 00:31 shday QUOTE (ff123 @ Aug 30 2002, 12:52 PM)Any thou... Aug 31 2002, 02:24 ff123 QUOTE (shday @ Aug 30 2002, 05:24 PM)QUOTE (f... Aug 31 2002, 02:40 ff123 Investigating the rand() error a little further, t... Aug 31 2002, 03:16 Continuum QUOTE (ff123 @ Aug 31 2002, 12:37 AM)But goin... Aug 31 2002, 09:40 Continuum First Step: Interprete Pascal triangle as abx resu... Aug 31 2002, 09:44 ff123 I still don't know why the simulation doesn... Aug 31 2002, 18:56 Continuum QUOTE (ff123 @ Aug 31 2002, 07:56 PM)I still ... Aug 31 2002, 20:57 ff123 QUOTE (Continuum @ Aug 31 2002, 11:57 AM)at l... Aug 31 2002, 23:48 ff123 Yay!
I made up my own spreadsheet using the P... Sep 1 2002, 03:54 Continuum So there is a mistake in the code. If you can find... Sep 1 2002, 09:21 ff123 Great! The code looks like it should be very ... Sep 1 2002, 16:53 Continuum Another thing to consider is, that this test allow... Sep 1 2002, 18:36 shday I think this is a major improvement to ABX testing... Sep 1 2002, 20:33 Continuum QUOTE (ff123 @ Aug 30 2002 - 11:17 PM)I'v... Sep 2 2002, 07:43 ff123 I'm having difficulty understanding your point... Sep 2 2002, 16:36 Continuum QUOTE (ff123 @ Sep 2 2002 - 05:36 PM)I'm ... Sep 2 2002, 17:29 ff123 Here's how I was calculating the in-between po... Sep 2 2002, 17:58 Continuum QUOTE (ff123 @ Sep 2 2002 - 06:58 PM)Ok, I... Sep 2 2002, 18:19 ff123 Regarding trials 1-4, I am thinking about how I ha... Sep 2 2002, 18:19 ff123 QUOTE (Continuum @ Sep 2 2002 - 09:19 AM)Yes,... Sep 2 2002, 18:25 Continuum QUOTE (ff123 @ Sep 2 2002 - 07:25 PM)I'm ... Sep 2 2002, 18:59 ff123 How should one modify the simulation?
Right now t... Sep 2 2002, 19:40 Continuum QUOTE (ff123 @ Sep 2 2002 - 08:40 PM)How shou... Sep 2 2002, 20:07 ff123 I still don't see the problem.
The listener c... Sep 2 2002, 20:21 Continuum QUOTE The listener can decide to stop at any in-be... Sep 2 2002, 20:42 ff123 Overall type 1 risk = overall alpha. The probabil... Sep 2 2002, 20:54 Continuum QUOTE The current optimum strategy for a listener ... Sep 2 2002, 21:29 ff123 Looks like
If one has 12/18, it is optimum to sto... Sep 2 2002, 22:22 Continuum QUOTE If one has 12/18, it is optimum to stop at t... Sep 3 2002, 06:45 ff123 QUOTE (Continuum @ Sep 2 2002 - 09:45 PM)What... Sep 3 2002, 15:10 ff123 Probably the best solution is to disallow stop poi... Sep 3 2002, 15:28 Continuum I think the easiest and savest method would be to ... Sep 3 2002, 15:44 ff123 QUOTE (Continuum @ Sep 3 2002 - 06:44 AM)I th... Sep 3 2002, 15:49 Continuum Hybrid version - sounds interesting!
Especial... Sep 3 2002, 15:58 shday Allowing in-between stops increases the chances of... Sep 3 2002, 22:57 ff123 Good point about the type 2 error, although the 28... Sep 4 2002, 02:03 ff123 The table brings up an interesting point. Should ... Sep 4 2002, 04:26 Continuum QUOTE (ff123 @ Posted on Sep 4 2002 - 03:03 A... Sep 4 2002, 06:42 ff123 duplicate post Sep 4 2002, 06:59 ff123 QUOTE Hmm, couldn't that lead to incorrect con... Sep 4 2002, 07:00 Continuum QUOTE However, the converse (program forces the li... Sep 4 2002, 07:13 ff123 I think it is enough to display what the overall a... Sep 4 2002, 07:45 Continuum QUOTE I think it is enough to display what the ove... Sep 4 2002, 08:02 shday I think the test should be strictly pass or fail. ... Sep 4 2002, 14:05 Continuum QUOTE (shday @ Posted on Sep 4 2002 - 03:05 P... Sep 4 2002, 14:40 ff123 I still don't see the problem with calculating... Sep 4 2002, 15:26 ff123 QUOTE (Continuum @ Sep 3 2002 - 11:02 PM)Woul... Sep 4 2002, 15:29
ff123 I think shday's idea is a good one. I think w... Sep 4 2002, 17:01  |
|
Lo-Fi Version | Time is now: 22nd May 2013 - 20:53 |
