Need listening test grades statistics, … or raw data sets Options

[Serge Smirnoff]

Could anybody recommend some articles/papers on listening tests with statistical analysis of collected grades (mean, variance, distribution, normality …) or may be with listings of raw data sets. Going to work out some procedure for automated exclusion of outliers from grades are being received by SoundExpert. As there is no possibility to post-screen “insensitive” listeners in SE tests design the data themselves are the only objects for analysis.

[C.R.Helmrich]

Maybe a look at the procedures the EBU used in their 2007 multichannel listening test can help you. On page 19 they list three conditions of post-screening.

Chris

[Serge Smirnoff]

On page 19 they list three conditions of post-screening.

Thank you Chris. Although those conditions are not applicable in SE case directly, the distributions of scores in Appendix 5 are really helpful.

[Woodinville]

You might do better to consider partitioning the data, to see if you have sets of listeners who are reacting very differently to the auditory cues.

This way, you can find out if there are multiple listening modalities, which we all know do exist, but nobody really has attempted to sort out the "outliers' who use (for specific example) more emphasis on spatial issues.

This way you can also separate out the random outliers from groups of "outliers" who are in fact using different hearing modalities.

[Serge Smirnoff]

You might do better to consider partitioning the data, to see if you have sets of listeners who are reacting very differently to the auditory cues.

If you mean clusterization of listeners according to their listening (auditory) specificities, then it is interesting. AFAIK in standard listening tests the listeners whose grades are not correlated enough with majority of listeners are just removed. May be in case of large sample of grades there is possibility to pick out different groups of listeners ... In any case this can't be applied to SE grades because they are all anonymous. Only recent grades are stored with IP addresses which can't be used for reliable identification of their authors as well. The problem is to remove only obvious outlying grades. It was done manually but starts to consume more and more time. So some simple and reliable procedure is necessary.

[Serge Smirnoff]

I've done some research of grades submitted to SE. The purpose was to find simple and adjustable mechanism to reject obvious outlying grades. Finally the mechanism looks as follows:

• Median ± 1.5 for inlying grades
• min. number of grades = 15

I would like to ask people who conducted any listening tests (especially here @HA) and have raw sets of collected grades either to test the above rejection rule or to send those grades to SE. Grades could be depersonalized, just grade sets. Thanks in advance!