QUOTE (worker1 @ Jun 9 2009, 22:19)

New test with 2 tracks.
1st track: 24/96
2nd track: 16/44
track 2 was converted from track 1 using sox with dither
I copied them many times randomly and mixed them around- then dragged them in a bunch to foobar with no signs (bitrate, etc) visible. I listened to a total of 37 tracks and marked down + or - for 24 and 16 respectively
I identified 22 out of 37 correctly. That's 59.46% success rate.
Not so good. If you were flipping coins, you'd do only 9.46% worse.
To put this into perspective you need to compare your results to a like number of coin flips.
Here's a tool for doing that:
http://stattrek.com/Tables/Binomial.aspxThis page also explains many of the reasons why.
I plugged your results in there and got a "cumulative probability" of about 0.90
Here's their definition of cumulative probability:
"A cumulative binomial probability refers to the probability of getting AT MOST a specific number of successes in a specific number of trials. For instance, we might ask: What is the probability of getting AT MOST 2 Heads in 3 coin tosses. That probability (0.875) would be an example of a cumulative binomial probability"
IOW the probability of getting at most 22 right out of 37 tries is about 0.90. There's about 1 chance in 10 of doing what you did by dumb, pure luck. The usual guideline for testing is that you want your cumulative probability at the 0.95 or 0.99 level. You are way short of that.
I've done a lot of testing and it is not that unusual to get scores in the 0.90 range.
If you think there is a chance of better results, run a test with a ton more trials and see what happens. The more trials you run, the lower percentage of right answers will be required for acceptable results.
Usually what happens is that after running more trials your results converge asymptotically to 50% right, and you have compelling evidence that you were guessing all along.
OTOH you may be onto something. Don't fret, don't fume, either buy into the data you have obtained so far, or make more! ;-)
I would also like to hear about your test enviroment - audio interface, transducers, etc.