1. Why does it never swap files under the Play A & Play B buttons? For example, I always get mp3 file under Play A button and the wav file under the Play B button, no matter how many tests I perform in a row, it only swaps files under the Play X & Play Y buttons.

2. Why sometimes after about 20 tests it displays N/A instead of percentage score?

It's the way ABX works - what's wrong with that?

I thought that double blind test is when files under both A & B and X & Y buttons get swapped, and if only X & Y is swapped then it's a signle blind test... Am I wrong?
And what's with the N/A issue?

Yes. Double blind means that the person performing the test (here the listener) as well as the one who provides test objects (here the software ) doesn't know what's tested. The term 'double-blind' makes more sense for e.g. medical tests: both, the patient as well as the doctor who hands him the medicine, don't know if he gets the placebo or the real medicine. Otherwise (if the doctor knew what the patient got,) the way the doctor behaves could influence the result.


If the number of trials is not fixed before the test starts (e.g. "I'll do 20 trials, not more, not less") The p-value (Probability you're guessing) is higher than the one calculated by fb2k (or other ABX tools). The more trials you are willing to take before you give up (if you don't reach a certain p-value), the higher the 'real' p-value gets. I assume that the limit with foobar2000 has been introduced to avoid the danger of false-positive results related to this.