I've been listening to more music than usual lately, and my collection's begun to span several formats: Musepack, lossless, MP3, and even some AAC and Vorbis as of late.

I'm becoming more convinced that I can hear a difference between the lossless files and the lossy files, even the Musepack-encoded stuff. Yet, as hard as I try to ABX, as many different techniques I try, I can't get any meaningful data out of it all. It's immensely frustrating, because I feel that if I could somehow learn how to detect these sensations better, I could really help out. Then again, I'm struggling with myself. My intuition tells me for certain that there's a difference. Yet logically, I cannot verify that intuition whatsoever.

Has anyone else run into this? Is it possible to overcome? I know I'm probably going to get blasted with posts that say that if it isn't ABXable or otherwise scientifically verifiable, it's not there. I try to repeat that mantra to myself until I believe it, but the intuitive side of me refuses to accept that. And, as both my logic errors and my intuitive errors run about even, I'm not sure which way the truth lies in.

I suppose this is the HA equivalent of believing in ghosts, little green men, or occult phenomena. Yet the anecdotal evidence supporting these isn't trivial, either, and although there's a lack of scientific proof, for obvious reasons there's also a lack of scientific disproof.

So... any comments? I'm almost frightened to post this lest I get flamed into oblivion, but ah well... there goes the Post button.

Minutes from a recent Placebo Sufferers' Anonymous meeting. (PSA)

Den: Hi, my name is Den.

All: Hi Den.

Den: I am a HA member, and I have a collection of many different Lossy formats and some lossless also. No matter how hard I try, I believe I can sometimes sense a difference in the various files. * breaks down into sobs * I tell myself over and over * sniff * that I am imagining things. I put myself through * sniff * endless blasted ABX tests, and I fail everytime, but....I can't help it, I still think I can hear something... * sniff *

DAMN YOU PLACEBO!

All: * applause *

Councillor: Don't worry Den. We're here for you mate.

Den: * sniff * geez, thanks guys...

Welcome to the club Canar, you're in good company...


Seriously though, I get similar feelings from time to time, but I can never hit it with ABX testing so I'm now over it.

There are for sure other/alternative ways to prove (in a scientific way) that there are differences between the various formats ... but none of them incorporates your own listening as good as blind testing ...

As an example, a spectral difference analysis of original and ecoded file will tell you what codec and what type of setting affects your original sound file's spectrum most but our "flawed" average ears are not able to pinpoint these differences exactly and reliably, although a scientific method has been used.

Blind Testing is, after all, a means which incorporates your whole listening chain from file -> soundcard -> amplification -> playback -> your ears. That's the good point which makes ABX'ing a very useful means of testing, although it is limited in one major way ... the point is that you might be able to hear (and thus reliably ABX) more differences when you are:

• using better (in terms of more accurate) listening equipment (headphones, soundcard etc.) or you are
• concentrating harder on the source material (because close listening is a form of education/concentration and depends on your daily mood/sensitivity as well - that's why it is called psycho-acoustics) or you are
• beginning to learn what kind of sample/music/effect causes easily-abxable problems with lossy encoders.

After all, blind testing aims at one purpose only ... to find out what sounds transparent to your ears, not others' ears.

If you can tell a difference and you definitely do not know what is playing at the moment, it isn't always placebo ...

BTW ... I have not been able to ABX differences between various lossy encoders at high quality settings yet ... and I sometimes "feel" like hearing a difference as well ...

Does the feeling disappear when you run the ABX test, or is it still there, but on the wrong file ?

Perhaps somebody (who is skilled) should write a foobar2k- and/or winamp plugin for "long term ABXing"... All tracks in a playlist should be available in at least two formats (lossy + lossless) and that plugin picks a random format each time - but the user only sees his usual playlist. Of course everything will be logged and there should be a button somewhere saying "I have this feeling now"... Just an idea...

Regards, fileman.

Interesting idea Fileman... sounds like it would work, and id be interested in trying it if someone wrote it...

Problem is that ABX'ing is more likely to fail on "normal" track lengths (e.g. 04:30) than on specific test samples like the ones widely used by the HA community ... a plugin (very good idea fileman) that would handle a certain directory structure where original file samples and encoded file samples (like fatboy.wav etc.) are being stored would be more convenient IMO ... then you could set your player to "Blind Test Mode" whenever you feel the need of doing so ... and you could create some kind of statistics with your test runs to judge your personal listening progress ...

I do all my critical listening in my couch with my eyes closed. The sensation between open and closed eyes is very different. With open eyes my mind seems to wander and sometimes might even focus on an imperfection on the wall or the dust on the speakers. With eyes closed I can concentrate on the music alone and hear so much more.

If you are concentrating on the test itself and possible fumbling with a computer/remote/pen to change tracks and note results, you are not giving the music your full concentration.


<Canar: Has anyone else run into this?>

More or less. I have my entire collection in a lossless format. But some months ago I decided I wanted to expand my collection. So I downloaded a lot of MPCs and MP3s from the net. I seldom listen to more than 2 new albums a day. More than that gives too many sensations for me to cannot process. So I just put the lossy files in the collection and would listen to them later. On several occations I had to stop playback during casual listening of the lossy files. It was not because I disliked the music, I just wanted to sit in silence.


Questions: Do feel that you enjoy lossless music more? Do you think listening to lossy music makes you listen less? If yes to both questions, you can do a long term blind test like this:

Get some extra disk space and make three copies of your music collection.
1. Lossless files only
2. Collection 1 encoded in lossy format 1
3. Collection 1 encoded in lossy format 2
(you will need to hide the file extention or something like that)

Now listen to music, but only one collection per day. Every evening note how much music you played that day and from which collection. After 180 days you should have 60 results for each collection. Simply sum the results for each collection and see which collection you played most.

really good idea fileman. If a person's computer is fast enough, he could set the plugin parameters to certain encoding settings that he wants to test, and then have the program temporarily encode from lossless the next couple songs in his list to random formats within those parameters. When he clicked "I have a feeling", that temporary encode would be backed up and not deleted, so as to allow further analysis later. If he doesn't click "I have a feeling" then the temporary encode would be deleted after playback. This solution still requires a lot of hard disk space for ones lossless collection, and a fast computer for on the fly que encoding, but it is do-able.

@Canar: I have no input, except to say that your reasoned approach to this ticklish subject is the best I can remember having seen, and IMHO there is no way under the sun that you should get flamed for it. Your post could well serve as a textbook case for others, as to the mature way to present a proposition for discussion.

Regards,
Madrigal

There's one weak link in the ABX scheme as usaully done here. Any chance that there is a (possibly hardware dependant) difference in sound between playing back a compressed file on the fly vs decoding to wav and playing back the wav file?

I have certainly heard gross problems with hardware players that couldn't keep up with a high bitrate, or streaming audio whose buffer runs dry.

I've had similar experiences as Canar, but mine have proven "shallow" by comparison, as I've always tended to disprove what I "thought" I was hearing (so far).

The times when I think I can hear a difference between my Vorbis files and FLACs, for instance, I'll ~50% of the time choose the Vorbis file as sounding "better", so I take that as evidence that I'm suffering the effect of placebo, and then trying in vain to rationalize which track sounds closer to source. These are the same files, mind you, that I cannot ABX apart from each other in a direct, controlled test session.

Just the other day, I (thought I) had overwritten a few albums encoded in Vorbis 1.0 -q4.25 on the HDD in my car with the same albums encoded in Vorbis 1.0.1 -q5.00. When I listened to the results, I was totally convinced that I heard significantly better sound quality.....only to find out that I had not actually copied over the old -q4.25's after all. Boy was I embarrased, and no one even knew about it but me! (The app that manages the music on my car's HDD works on a concept of file-views rather than real-time updating, and I failed to physically overwrite anything on the first try.) That damn placebo gremlin completely fooled me!!!

So I guess you could say I'm agnostic to this issue. I'll believe that long-term listening results show differences that direct ABXing can't only when I can see confirmable results. However, I'll participate in such a test if an effective one is designed (already begun here by sshd and fileman?) and if I have the time to commit to it.


Edit: tpyos

Yes, and specifically car audio systems (in my experience) run into two instances of this issue...

-1- As you pointed out, different decoders have different overhead requirements for playing back particular file types. On the Kenwood Music Keg, for example, WAV and FLAC playback have always been flawless (the best I could tell), as have all MP3s up to 320kbps (and all VBR modes I tried). But Vorbis at anything higher than -q 5 for some tracks "breaks" during decoding as the buffer empties faster than the decoder can process the input, thus not keeping up with real-time playback. The effect is skipping and/or pitch distortion...sometimes mild, sometimes more severe.

-2- Different sources in a multi-source system. Playing back a FLAC file from the Music Keg can truly sound different than playing the same song from the CD in the head unit because of many factors (source volume leveling, source EQ, different output stages/pre-amps/DACs in the different components, different cables with different levels of shielding (more sensitive in a noisy EM environment such as in a vehicle). As a matter of fact, I'm trying to devise a good way of doing a blind test between different playback sources in a car audio system to compare sound quality between CD player and HDD components.

Only in a truly controlled environment can truly controlled testing be performed effectively, where the "playing field is level" and you can eliminate the problems of varying requirements for different decoders, different physical source components with their own settings, etc.

Therein lies the essential problem... The less controlled the test, the less you can trust it's results.

It disappears... The intuitive sense is very faint at best. I'm not sure I could qualitatively describe it, even. It's such a conglomeration of sensations... I hesitate to describe it further, lest the lesser experienced users say, "Well, Canar heard this kind of artifacting before". Even if I wanted to, though, it'd be really vague.

@Madrigal & den: Thanks for the support, guys.

Be carefull there...... "sounding better" and "sounding more original" are two completely different things.

You know? I'm one of the big critics of vorbis, because at first sight, it "sounds better"....... well "better than the original" to be honest.

Well, i'm not writing this to bash vorbis, but instead because lossy compression isn't about making music sound "as good as possible", but instead about "sounding as close to the original as possible". This is an important difference which i do feel is a problem in non-ABX-audio-comparision - it allows lossy formats to "cheat"(be it through unwanted bugs, or intentionall "cheating")

Well, another reason to use ABX-results i guess.

- Lyx

You're right. What I should have said (and what I was thinking) was 50% of the time I think the Vorbis file sounds closer to source than FLAC.....an obvious case of either simple placebo effect, or as you say, if there is an audible difference it's impossible to relate which file really sounds closer to source without a controlled test.

Thanks for calling me on that!

off-topic
ScorLibran: If you want to continue to like vorbis, don't read the following(and there are perfectly valid reasons for that - if it's okay right now, why change the situation?)

Do an ABX-test, comparing the wave-source and vorbis(try q3 to make it more easy)
- pick a song with cymbals, snares and drums
- try to concentrate on which one sounds more "clear" and "sharp"(especially with the cymbals, snares, etc.).
then repeat the comparision, but with LAME vs. WAV ;-)
/off-topic

- Lyx

Try visiting the PCABX Training Room if you haven't done so already. Also, try focusing on a very small segment - a fraction of a second. Often times that's the only way I can notice a difference between two samples.

@Lyx: I did a similar Vorbis listening test to the one yu suggested recently. Normally, I have a higher success rate ABXing with my headphones (for obvious reasons) than my speakers. However, when I was testing vorbis -q4 I had a higher success rate on almost all the samples with my speakers. Why, I don't know - it's very strange.

One approach to codec sensations, this will sound quite unscientific, is to not "try too hard". When you are ABXing, don't sit and listen for artifacts or differences, just let the music play and let your mind wander, play tetris (or that Nokia snake game) and listen in the background. With this approach, I seem to develop a sort of "awareness" of which is the lossy sample. Obviously you need long samples for this, but it really seems to work.

This is very different from sshd's advice and his is probably more in the spirit of a listening test, but for phenomena that are hard to track down it's worth trying multiple approaches.

[off-topic]
I did that very thing a few months ago between PCM WAV, LAME and Vorbis at -q 3.00 and up. I found that -q 4.25 was a "borderline", with all the tracks I initially tested being transparent to my ears. All levels I tested above that one are equally transparent to me. All the ABX tests I've done with music from my collection recently between LAME --aps, Vorbis -q 5.00 and WAV have given negative results.

I did experience what you are alluding to regarding the lower bitrates. And interestingly, in the 64kbps listening test I even rated Vorbis the lowest of all tested codecs. Yet still I like Vorbis as my primary format, at -q 5.00 that is. I have many other requirements beyond transparency, though, including gapless playback on all my players and tag format compability. So I'm not leaving Vorbis any time soon.
[/off-topic]

And through it all...I'd still be interested in doing some sort of controlled long-term test on my Vorbis-encoded files, with the risk that discovering a marginal sound quality issue might make me want to encode my collection over again. It's a risk this agnostic is willing to take!

I remember Differenciam had a similar thread a while back, I wonder if he's been able to do any more long-term ABX'ing with his method. It would be really interesting if someone could devise a methodolgy for this type of test (one that didn't require a week for each sample!). I guess Cabbagerat's suggestions won't help in Steve's transcoding test, but I'll give it a try anyway.


Cheers, Joey.