The Absolute Sound FLAC article Options

[SoAnIs]

My father subscribes to "The Absolute Sound," and in the latest issue they have an article about FLAC. Of course they have no (or very minimal) double-blind testing (something both my father and I believe is necessary for any review to be valid) and a history of endorsing bullshit, so we take their results with a truckload of salt.
They score their comparisons on a scale from 0-200 points, with CD audio as 100 points. The scores are generated by A/B comparison (not A/B/X) to a variety of reference qualities. The first issue crops up with this. 0 = Kazoo. 50 = 320kbps MP3. 100 = Red Book CD, 44.1khz/16 bit. 110 = 48khz/24 bit upconverted CD. 120 = 88/24 upconverted. 130 = 96/24 upconverted. 150 = 176/24 upconverted. 160 = 192/24 upconverted. 170 = 192/32 upconverted. 180 = SACD/HRx/High-res downloads. 200 = "Master Tape?". There are no references between 50 and 100. Furthermore, a "very large" difference of 50 points would "require only one A/B comparison and is so large that a 24-hour hiatus between listening sessions would still elicit the typical audiophile expression of 'Holy Cow! This is a night and day difference!'"
Both their reference systems used on-board sound cards capable of 192khz S/PDIF output into a PS Audio PWD DAC. Motherboard brand and model is not disclosed. Actual sound-card model is not disclosed.
This seems quite suspicious to me. I have been unable to successfully detect differences between 320kbps CBR MP3, wave ripped from CD using Exact Audio Copy, and CD Audio using Foobar2000's ABX comparator plugin. I used Epica's "Design Your Universe" tracks "White Waters" and "Burn to a Cinder" as my references this time. I used the onboard audio of my motherboard (an ASUS P6X58D Premium, Realtek ALC889 audio, with 24-bit/192khz default output set in Windows 7) and a set of Grado SR-60 stereo headphones. My CD drive is a Lite-On iHAS424. I don't have a fancy $4000 DAC. My ABX test results were no better than guessing, nowhere near a "night and day" difference audible with a 24-hour gap. I sincerely doubt that the lower quality of my headphones and DAC would be enough to mask such a difference. Thus, I doubt their measurement scale's accuracy.
They did their playback comparisons using JRMC (proprietary) with 192/32 realtime upsampling. They never once mention using the reference FLAC encoder/decoder, or even FLAC frontend.
The top 3 programs in sound quality were:
JRMC: 150 score for HRx 176.1khz/24-bit to FLAC
dBPowerAmp: 145
Foobar2000: 140
Original HRx wave file: 180
So they're claiming a large drop in sound quality.
Later they claim that converting from FLAC to WAV with Foobar can increase sound quality (where would the extra data be coming from?) and that other programs decrease sound quality. In all cases they claim a decrease in sound quality when compared to the original CD.
They also (via listening tests, no md5 hashing or such) claim that Wav->FLAC->Wav conversions create cumulative degradation. This is, of course, a very big claim.
So I took a wav file (a rip of "White Waters", mentioned above) and used the FLAC reference encoder to compress it. Then to decompress. I compared the MD5 hashes of the two .wav files, and they were the same. Just in case some strange miracle had occurred and I'd found an MD5 collision I compared the SHA-512 sums, and those were also the same. The two files were bit-for-bit identical. Just for fun, I repeated the wav->FLAC->wav process 10 times, and the files still had identical hashes. I then ABX tested the original and final files, and was unable to do better than guessing.

TL;DR: Bullshit detected.

[SoAnIs]

I'm not asking anything. I'm describing the article, and why I think it's wrong. I advise everyone to avoid this magazine due to the ridiculous claims they are making with insufficient evidence.

FLAC is lossless, with a properly implemented decoder. "The Absolute Sound" claims it isn't, and don't test the reference implementation. They don't test the file I/O, and use very subjective listening tests.

[Charlie Freak]

I couldn't find that article on their website, admittedly I didn't look very hard either. I'm guessing it's only in the physical magazine?

I did find this discussion in their forums, which contains many comments from one of the article's authors:
http://www.avguide.com/forums/computer-music-audio-quality

This post has been edited by Charlie Freak: Jan 7 2012, 07:14

[mjb2006]

http://www.avguide.com/forums/computer-music-audio-quality

Thanks for that link. I enjoyed this quote the most:
For example, compare identical music files played back from an optical drive, a defragmented hard drive, and a memory stick: they will all sound different.