High Bitrate ~320kbps MP3 vs. Lossless audio, Spoiler: mp3s were TRANSCODED from Lame free-format. Options

[Archimago]

Happy New Year and greetings everyone! I know the folks here at Hydrogen audio have been around and back with regards to these tests... Nonetheless, I've been collecting data from around the audio forums since early Dec 2012.

Since early December 2012, I've opened up a survey to see if music lovers & "audiophiles" around the world can tell the difference between high bitrate ~320kbps MP3 against the original un-lossy-compressed CD audio. I'm only planning to collect data until the end of January 2013.

So far, I have >70 responses from around the world with 5 continents represented. Folks have been using everything from inexpensive (but good) headphones to megabuck separates >$50K for this. There's 2 weeks left so if you've ever been curious about participating in a blind test or just want to add to the dataset, here's your chance :-).

The "test" is relatively simple and consists of 3 musical passages encoded as "Set A" vs. the same songs "Set B"; one of which was MP3 encoded.

Come to my blog to download the music and fill out the survey:
Archimago's Musings

Thanks and have a wonderful 2013!

OOps - the title should have read 'ends Jan 31'... [fixed]

This post has been edited by db1989: Jan 15 2013, 21:16

[Kohlrabi]

Reading the test description one flaw seems to be that you encoded one set with a lossless encoder (let's call it "A"), and the other with a lossy MP3 encoder ("B"). Since you don't enforce the use of a software which ensures the test is being conducted in a double-blind fashion, so that the experimenter himself doesn't know which file is which until the test is finished, the following chain of events might occur:

Someone might analyse the spectra of the files and look for MP3 signatures beforehand, or conduct a successful test, and thus be able to determine correctly that file "B" is the lossy one. He then might publish it, which will affect any listeners who don't use a DBT software.

To me, the only way to ensure a proper test is to ensure that no cheating is possible in the first place, i.e. the test is done by enforcing the use of a DBT software, and ideally the result cannot be tampered with. Similar to how the ABC/HR tests have been done here in the past.

The questionnaire reduces the test to a single question about whether "A" or "B" is the inferior source. If the listener did not perform a full ABX test with a high enough number of trials, but rather a single listening or guess, this is the same as conducting only a single ABX test, where a person who guessed has a 50% chance to guess correctly. How will you be able to exclude guesses from your results, if you only have this single question at the beginning, and not a full test result with a number of trials that reduce your likelihood of the result occurring by chance to below 0.05?

This post has been edited by Kohlrabi: Jan 16 2013, 14:20