44.1 vs 88.2 ABX report at AES Options

[2Bdecided]

Here:

http://www.aes.org/events/128/papers/?ID=2252

P18-6 Sampling Rate Discrimination: 44.1 kHz vs. 88.2 kHz—Amandine Pras, Catherine Guastavino, McGill University - Montreal, Quebec, Canada
It is currently common practice for sound engineers to record digital music using high-resolution formats, and then down sample the files to 44.1 kHz for commercial release. This study aims at investigating whether listeners can perceive differences between musical files recorded at 44.1 kHz and 88.2 kHz with the same analog chain and type of AD-converter. Sixteen expert listeners were asked to compare 3 versions (44.1 kHz, 88.2 kHz, and the 88.2 kHz version down-sampled to 44.1 kHz) of 5 musical excerpts in a blind ABX task. Overall, participants were able to discriminate between files recorded at 88.2 kHz and their 44.1 kHz down-sampled version. Furthermore, for the orchestral excerpt, they were able to discriminate between files recorded at 88.2 kHz and files recorded at 44.1 kHz.
Convention Paper 8101

Was anyone at the presentation? Has anyone bought the paper?

Cheers,
David.

[Pio2001]

Thanks for joining the discussion, Amandine.

Your work is very interesting. But I think that it is necessary to run all the raw data into a global analysis instead of picking seemingly significant results among all possible comparisons.

I see that you calculated the p values for many different cases. Basically, for the 16 listeners as individuals, then for all of them as a group, then for 3 of them, then for 13 of them, in each case for 3 formats times 5 samples, and that you also included two-tailed results in addition to one-tailed results.

It gives a total of [(16 x 5) + (16 x 3) + (1 +1 + 1) x 3 x 5] x 2 = 346 possible p-values, out of which you got 12 significant ones. However, out of 346 p-values, we should expect in average 346 / 20 = 17.3 of them to be significant by chance, that is false positives !

In reality, it is more complicated, because the 346 p values are not independant at all. The values for the 13 listeners group are close to the ones for the 16-listeners group. The values by format for the 3-listeners group are also conditionned by the fact that this group was selected according to their individual p-values for all formats etc.

But it is right that results by format, genre, and listeners should be evaluated independantly. The trick is to use a suited analysis that takes into account all these variables at once, but is not prone to false positive picking.

I don't know what method should be used in this case, but i'm sure that some forum members, more knoledgeable than me in statistics, can help.


Edit : correction of calculus.

This post has been edited by Pio2001: Jul 19 2010, 12:39