I'm wondering if it wouldn't be a good idea to set up some sort of a page which features examples of encoder artifacts at various levels of difficulty. It could have categories such as the following (which may overlap):

transient smearing (pre- and post-echo)
chirping
warble
noise pumping
stereo collapse
fluttering
metallic sound
flanging
underwater sound
fuzzy sound on transients
increase/decrease in background hiss
ringing
frequency bandwidth limiting
increase/decrease in sibilance
unclear/blurred sound
distortion which sounds like speaker breakup

I would want to start with very obvious artifacting, since I would intend this to be a training guide for people who don't have much experience in listening for bad things in codecs.

Any comments or suggestions? Favorite samples?

ff123

Excellent idea, ff.
That's pretty much everything I can say.

The stuff that I hear the most difference is with music that already has distortion in it. For example, Radiohead's "Airbag" which has alot of distorted stuff in the source. The trick is to get that distortion to sound the same in the alternate codec...

I also notice that some Trumpet blasts tend to break up when I try to compress them. Miles Davis' stuff gives the codecs a hard time; "Right Off" is a perfect example...

I look forward to learning the appropriate terms and types of artifacts, so I can can specifically, and clearly, communicate stuff like the two cited examples...

Great idea ff123 I'll provide few cases which could be good examples soon.

I too have noticed this. There is one song on "The Who By Numbers" that is distorted at the beginning of the song on the original CD. I can't recall which song it is at the moment.

Anyway, the original is distorted, and the MP3 seems to make it even more distorted. I noticed this first on the MP3, and then pulled out the CD to see what the deal was. It too was distorted, but not as much as the MP3. I would describe the distortion as a buzzing sound.

If anyone is curious, I can see exactly which song this when I get home tonight.

Brett

I've got some really obvious ones at hand:

castanets: encoded with Xing at 128 (pre-echo)
applaud: encoded with Radium at 128 (warble)
daughter: encoded with mp3enc31 at 128 (chirping)

I was thinking of making two mp3's available (as well as the original file). One encoded with the obvious artifact, and the other encoded with --alt-preset insane. This should save the newbie from having to download lpac or flac, etc., or from having to download megabytes of data, while still providing a pretty good comparison.

Hopefully, I can also limit the artifacting to mp3 codecs, but possibly some types of artifacting, like noise-pumping, might be hard to generate (WMA8 seems to be a good codec for generating those types of files).

ff123

Great idea ff123!

This chirping in Daughter encoded with mp3enc, is that the sound that I hear strongest on sec. 4 and 5? It sounds almost like knocking on some hollow wooden thing to me. Only it begins with a "tsj". I guess that's why you call it chirping.

Actually, I was reconsidering how I should describe that artifact. I agree that "knocking on wood" is a better description.

I also was thinking about how to capture the sound of artifacting in other codecs while still keeping it simple for newbies: transcode to mp3 --alt-preset insane. So now I can capture noise-pumping. Any favorite examples of that?

ff123

Please don't. Different types of artifacts-assuming that all compression codecs have some sort of artifacts, etc.-and their sounds, and relevance/impact to source material, are the things that factor into a codec choice...

I don't see this as an all or nothing thing. From a strict htpc perspective, why can't you use ogg for some stuff, mpc for others and aac for others, and mp3 for others? Maybe use 2 or 3 of these? Is iot not possible that some codecs work better for different types of music? Is it possible that identifiable artifacts/sound quality is associated uniquely with the codec? It would seem to me that this would be true. Some artifacts might be acceptable to my hearing, but give your aesthetics a chill; or vice versa..?

Could the determining factors for codec choice would be which format handles that type of music best? That's what I am interested in figuring out. Maybe one format that is good for loud guitars, and maintaining the integrity of the source distortion, but is not so good for classical music, or music with alot of silence. My point is, if you limit it to one codec, the unique traits of all the other codecs won't be explored, even though they are in play, making an informed codec chopice that much more difficult...

Again, great idea. Godspeed.

In the categories of "very obvious" or "obvious" artifacting, I think I can get away with transcoding to mp3 --alt-preset insane to capture the artifacting. More precisely, I would give listeners the option to download the transcoded mp3. The file in its original format would still be available. The "very obvious" and "obvious" categories would be for inexperienced listeners, and it would be helpful to make these available in a format that almost everybody should have.

For the other, more subtle categories, I would put up the files in their original formats only.

ff123

I think this would be a really good idea also. I think multiple formats should be used to emphasize particular artifacts, but transcoding to mp3 afterwards (with 320kbps) should work I think.

Some samples I'd use (big surprises here ):

death2 - Shows post-echo in addition to pre-echo. Post-echo generates a different sound to me so I think it'd be interesting from that approach.

fatboy, short, ravebase - distorted impulses, sounds much different than pre-echo on normal attacks.

bloodline, piano, serioustrouble - with some settings can show the effect of ringing/watery/metallic/underwater (all very similar) sounds due to not enough background noise being encoded (for various reasons)

2nd_vent_clip - different type of noise pumping.. problem with masking I assume

drone - shows noise on impulses and pre-echo/post-echo on transients. Sometimes emphasizes inaccurate triggering of block switching (second part of loop with some encoders sounds much worse than the first even though the sounds are basically identical)

headache (from MrDrew) - shows the effect of noise pumping or decrease and fadeout of background noise when not appropriate

I have quite a few other samples which would probably be good for showing.

If you're interested I should be able to provide the samples and encoder information and settings where these problems can become the most audible.

I've started a page:

http://ff123.net/training/training.html

ff123

Thanks for this; it's very useful. Now perhaps I can actually try to hear for myself the artifacting rather than relying on other people to tell me which codecs are higher quality than others. =]

Very good idea to put out samples with example artifacts. This is a great idea for developers and enthusiasts alike.

I'm sure I could learn tons from this and I'm willing to participate and help out as a test if needed.

cheers,
Halcyon

Added left-dist.wav encoded with RC3 -q0 (fluttering). Am I getting the ratings about right? These artifacts seem very obvious or obvious to me with my headphones at home, but when I listened on my computer speakers at work, where it's kind of noisy, I wasn't so sure.

BTW, one of my co-workers had the gall to prefer 41_30sec.wav encoded with scmpx (dist10) at 128 to lame --alt-preset cbr 128 (the cymbals were harsher for him using lame). I think I'll add that one next. It was so awful last night with Blade that I didn't think anybody could find something like that even close to listenable, much less preferable to the lame setting, but I was proven wrong

ff123

It would not be difficult to design a web-based test using these samples. The idea would be to distinguish the bad mp3s from the pseudo-reference mp3s. One of the instructions on a future listening test could be: "make sure you can pass this test." I would probably just test for the obvious/very obvious samples.

comments?

ff123

Yeah, I support that kind of screening test.
Btw. Darryl, I sent you a PM.

Wouldn't this bias the test? I mean: the samples you put up there are those that _you_ feel are easy to pick out. But other people with different hearing may have a different opinion.... So then the pre screening would sort out those, and only leave those that hear the same kind of artifacts that you do.

Btw, do you think you could include a sample of "collapsed stereo" on your training page? I've seen it mentioned here a couple of times, but I think I never actually heard it.

/Erik, tin-eared

Actually, I do want to bias future tests. I want to bias them such that the target population are those who are bothered by artifacting with mp3 at 128 kbit/s. Yes, there is the possibility that some people might be insensitive to the artifacts I chose while being sensitive to others. But I've chosen files with very gross defects on purpose -- if somebody doesn't notice that these files are bad, well, I'm not sure I'd want them on a listening test.

I will try to find a nice stereo collapsing file.

ff123

Yeah, the defects are SO bad that there's just no excuse if somebody doesn't hear those.
Well, should be pretty easy if you use the buggy FastEnc or VQF..
The first sample I though would be good was very good to demonstrate FastEnc bug: "Sichia" (de Lucia) track 4, 2:00-2:35 CD: Passion Grace and Fire (McLaughlin, Di Meola, de Lucia) from http://www.minidisc.org/mc_sich.wav
Of course almost any file that has high stereo separation is good to demonstrate the FastEnc bug, but this sample has very loud guitars very much separated in stereo field.

Brilliant move! I might even learn what "chirping" is according to you folks...

Regards, stoff

Thanks ff !

I always thought i wasn't that sensitive to artifacts but tonight you have proven me wrong.
At first i just listened to the samples and read the description you made. After that i abx'ed all of the samples and got all of them right in a breeze.

The hardest sample for me was "wait", the most simple one was "blackbird" that sounded really horrible. I'm looking forward for more ..

I must say that it gets easier after some trials as i know what to listen for then.

So this is how the fastenc stereo collapse bug sounds like. Nice sample (mc_sich_short). I like this training a lot!

ff
You are a genius.............. I particularly appreciate the commentary on each artifact.
Thanks
crk

Hey thanks. I particularly want to know if people disagree with a comment I've made, or hear something else which they find more annoying, or if they find it very difficult to hear something which I've classified as "very obvious" or "obvious."

ff123

This is a great page!!!

Thanks ff123.

One note: on my system, blackbird_insane plays silently. Is the file damaged? I'm using WMP6 (where did Winamp go when I re-installed?! ;-))

Cheers,
David.

Very educational.

I can see this turning into a thesis for somebody -

"The classification and epistemology of 'artifacts' in audio data reduction codecs"

So far ff123 has a high distinction!

I downloaded and checked the file with MP3Trim. Everything seems ok. I can play it both with Winamp and with WMP 7.

ff123

stoff wrote:


I put up 41_30sec.wav as encoded by SCMPX. This is what i would call "chirping." Basically, its a really bad case of warbling. I also had another reason for putting up this sample. Some yahoo in alt.music.mp3 was claiming that SCMPX at 128 sounds just as good as FastEnc 128 or Lame --alt-preset cbr 128. He couldn't hear how nasty SCMPX made applaud.wav sound, so I put this sample up. This is pretty much as nasty as artifacting can get.

ff123