High Bitrate ~320kbps MP3 vs. Lossless audio, Spoiler: mp3s were TRANSCODED from Lame free-format. Options

[Archimago]

Happy New Year and greetings everyone! I know the folks here at Hydrogen audio have been around and back with regards to these tests... Nonetheless, I've been collecting data from around the audio forums since early Dec 2012.

Since early December 2012, I've opened up a survey to see if music lovers & "audiophiles" around the world can tell the difference between high bitrate ~320kbps MP3 against the original un-lossy-compressed CD audio. I'm only planning to collect data until the end of January 2013.

So far, I have >70 responses from around the world with 5 continents represented. Folks have been using everything from inexpensive (but good) headphones to megabuck separates >$50K for this. There's 2 weeks left so if you've ever been curious about participating in a blind test or just want to add to the dataset, here's your chance :-).

The "test" is relatively simple and consists of 3 musical passages encoded as "Set A" vs. the same songs "Set B"; one of which was MP3 encoded.

Come to my blog to download the music and fill out the survey:
Archimago's Musings

Thanks and have a wonderful 2013!

OOps - the title should have read 'ends Jan 31'... [fixed]

This post has been edited by db1989: Jan 15 2013, 21:16

[C.R.Helmrich]

From your blog:

This utilizes LAME's "free format" to create initially a 400kbps MP3 without the usual lowpass filter in place, then runs the resulting file through the MP3 encoder again but at a lower 350kbps bitrate (again with low-pass turned off) which closer approximates the 320kbps target bitrate for the test. By doing this, even though the resulting MP3 size is slightly larger by 30kbps, the degradation in sound quality by objective measures is in fact approximately the same or slightly worse than if the audio were processed directly through 320kbps but without the tell-tale sign of the strong low-pass filter.

Wait... so:

• You're not comparing lossless to the claimed 320-kbps MP3 (free-format LAME is not MP3, and 400-kbps/350-kbps tandem coding certainly isn't!)
• You don't do an ABX test (like pdq/DonP said; A and B might not be ranked significantly different even though a significant number of listeners heard a difference between A and B).
• People can look at the spectra of A and B before listening, and it's trivial for them to do so. How about HA's method of providing test software which randomly assigns A and B and encrypts the results to prevent tampering with them? You'd still be able to watch the spectra while the samples are being played back, but I claim that far fewer people would know how to do that. Edit: And providing such software, like ABC-HR (latest version here?), would not increase the download size by 30%, as you claimed.
• You tried to make it harder for people to identify MP3, and by doing so you crippled LAME's psychoacoustic model. An RMS error measure averaged over a test item does not tell you if two items (here true 320-MP3 vs. your 350-tandem-coding) sound the same. You need to do that measure in the perceptual (masking) domain and also look at instantaneous worst-cases. And since how to exactly replicate that perceptual domain in software is partly unknown, you shouldn't do this at all, I would say.

Sorry, you idea was great, but I can't see how you'll be able to draw any conclusion from your test which cannot be torn into pieces in a subsequent discussion.

Chris

This post has been edited by C.R.Helmrich: Feb 2 2013, 18:28