First of all I am not a professional in audio technologies I am just an audiophile who is very strict when it comes to audio quality , so this test might be wrong, the purpose of this test is just to make readers aware about some encoders behavior about noise responsiveness.

The other day I did a test on a wav track which contains some noise effects, the purpose of this test is to determine Noise responsiveness and quality on mainstream audio formats
Test Date : 2009/10/10
Source Format : Wav 44.1kHz Stereo 1411Kbps PCM.
Target Format : Ogg, Mp3, Mpc, AAC.
Settings : default settings of each encoder.
Testing hardware : Sony TwinTurbo Earphones on Realtek HD Audio card (output format : 24bit).
Testing Software : Foobar2k 0.9.6.4 (Encoding and listening) Enabled plugins : Equalizer + vLevel.



From the resulted bit rates it shows the weird responsiveness of Nero AAC and Musepack to noise with Nero AAC being the worst unlike Ogg and Mp3 which responded well and gave a very nice noise quality.
to make sound distinguishable I used Foobar's built-in Equalizer to clarify the sound on each format. equalizer setting file is attached with this post.
it is recommended to use vLevel to be able to hear the noise clearly in all files
Archive contents : Original Wav File + Equalizer setting + 4 encoded Audio files.
best regards.

Archive

Critics or comments are welcome

Perhaps look into ABC/HR and run the test again.

http://wiki.hydrogenaudio.org/index.php?title=ABC/HR

My scoring was according to the scoring noted in the link you provided, NeroAAC and MPC are realy annoying with MPC being better than NAAC
if you mean ABX testing I can easily identify the MPC and NeroAAC encoded track in the first Seconde of the track

Thank you.

I run again the test for MPC and NAAC at Q6 and Q7
MPC showed better responsiveness at Q6 and Q7 124kbps 171kbps respectively but NAAC still have quality degradation even at higher quality

Thank you.

I am just curious. At what bit rate do you stop perceiving the difference between lossy and lossless?

the used track is just 2s length of noise, so the bitrate is almost the same (very low) for nero and MPC
MPC :
NAAC :
as stated above, ~120kbps seem to give good noise quality.

No, you've missed the point I think. Let's review:

ABX testing, in an audio context, provides evidence as to whether or not an audible difference exists.
ABC/HR testing also provides evidence as to the audibility of a difference, and ALSO (important point here) provides a blind way of measuring perceived quality.

An ABC/HR test is the appropriate way to measure quality difference between several lossy samples. ABX and sighted tests are not. I recommended you re-conduct the test using this tool because, properly done, it will strengthen the usefulness of your results. And it's good practice to use the appropriate test and tools for a given experiment.

You've got one hell of a smiley-face going on with your eq. -6db to +11db? What makes you think this is valid eq setting for real listening situations? Such a configuration just seems like it is defeating the equal-loudness curve, which all encoders rely upon. So even if your results are confirmed by ABX testing, they may not apply to anybody who does not have as vastly a poor of a frequency response as what this test utilizes.

Why is vlevel necessary?

Besides that... I tend to support this general kind of listening, insofar as codecs ought to be configured so that they allow for transparency under any kind of linear distortion operation (such as eq and to a lesser extend dynamic range compression). But it is highly unlikely that can be achievable at these sorts of bitrates. After all, it's dangerously close to perceptual transparency in the first place.

can you please tell or point me to a link on how to conduct this test because the link you gave me above does not discribe the way of testing.

Thank you.

at least it's my favourite setting .
the audio file was encoded to different format directly without using any DSP plugin, but while listening I used Equaliser and vLevel to amplify the noise and make it clearer.
Thank you.

If you listen with a lot of equalization then the proper way to do lossy compression is to apply the equalization first, then encode it. That is the only way that the encoder can rationally determine what will be audible and what won't.

what surprises me in this test, is why NeroAAC and MPC behave differently to Lame and Ogg in same circumstances, regardless of filters.
does NAAC and MPC ignore Noise purposely ?

Isn't it supposed to be a good thing that noise isn't given much bits?

I think sometimes noise should be kept as is, no ? I think it's quality problem here not bit rate problem
thank you

Why should noise be kept, or not kept, as is? What's the rationale?

what if noise is put on purpose on an audio track, should it be neglected because noise is mostly neglected ?
IMO when encoding to a given quality level everything should be encoded at this level whether it's noise or anything else <-- Just an opinion

This noise is inaudible at normat listening conditions. So there's no need to keep it.
Lossy encoders don't assume heavy postprocessing of encoded files.

it's seems that NAAC ignores signals at a low level
I tried to encode the same wav file but these time by applying vLevel filter at encode time
result is : better responsiveness with 201kbps rate

You mean you originally encoded noise at a very low level and then wondered why the encoders weren't keeping them?
After you increased the volume of the wav and encoded it, you found that the encoder exhibited normal behaviour? Gosh. This seems to show that Nero AAC and Musepack are better than the Vorbis and LAME encoders you used.

1 - Musepack showed responsiveness to noise at Q7 but NeroAAC didn't at Q7
2 - NAAC showed responsiveness but only after normalizing the sound (vLevel Filter)

In my opinion signals shouldn't be ignored or given low bitrate whether they are low level or noise, everything should be encoded according to the given quality setting. But I might be wrong

Thank you.

Your points 1 and 2 indicate to me that Nero AAC is actually more advanced than Musepack, LAME and Vorbis... part of the job description of an efficient lossy encoder. If it can't be normally heard, it doesn't need to be encoded. Then again, we've been so used to Replaygain values that are negative that I wonder if encoders are already able to deal with samples that eventually require highly positive RG values. Without further information from the various codec developers, we won't know for sure.

And if we don't know for sure, we'll be foolish to present our opinions as facts.

These results are differences of the designs (or/and tunings) of the encoders basically.

The libvorbis(include aoTuV) assumes that sound of the small volume is played at big volume .
Since an encoder cannot prescribe play volume beforehand, I think this supposition to be be a necessary thing.
However, the behavior may be superabundant in 16bit source.

Thank you very much for your comments, I was able to understand many things from this topic
Thank you.