Nero AAC, listening test :-) Options

[Ivan Dimkovic]

Hi, - before releasing final version I would like to hear audiophiles opinion about two noise allocation strategies we can use.

These two versions will be called "build_a" and "build_b"

I've packed some of the essential "codec killers" samples in build_a.zip and build_b.zip - originals could be easily found on hydrogenaduio and mp3dev

http://www.psytel-research.co.yu/test/

BUILD A:

http://www.psytel-research.co.yu/test/build_a.zip


BUILD B:

http://www.psytel-research.co.yu/test/build_b.zip


Which one is better, and for which samples?

Files are CBR - 80, 96 and 128 kbps - 32 and 44.1 kHz sampling rate - ADIF AAC format

This post has been edited by Ivan Dimkovic: Nov 27 2002, 13:21

[Dibrom]

I moved the thread to the news section.. it'll probably get more views that way. Hope you don't mind

I downloaded the files btw.. I'll try to submit some results a little later.

[Dibrom]

Result So Far:

Castanets - 44.1khz@128kbps - Build_b is better. Build_a has a terrible whoosing/tearing sound (most audible in the left channel) during the fast attacks that makes it much more annoying. Neither are transparent, though I haven't abx'd yet.

Fatboy - 44.1khz@128kbps - Build_b is better. Build_a has a sort of stuttering and skipping sound at one point. Neither are transparent.

Btw, these results are from only about 10-15 seconds listening to each sample and without having compared directly to the original (it's just off memory) or having done abx testing..

More later.. (including more thorough testing)

[Guest_SK1_*]

OH YEAH!
OK, starting to test. This thread will be edited and will contain detailed, ahm, details ..

41_30sec_32khz_80.aac - build a sounds better, was hard to choose.
41_30sec_32khz_96.aac - build a sounds better, hard to choose.
41_30sec_44khz_80.aac - build b sounds better.
41_30sec_44khz_96.aac - build a sounds better.
41_30sec_44khz_128.aac - build a sounds better, hard to choose.
applaud_32khz_80.aac - build b sounds better
applaud_32khz_96.aac - build a, hard to choose
applaud_44khz_80.aac - build b, hard to choose
applaud_44khz_96.aac - build a
applaud_44khz_128.aac - build b
bullet_32khz_80.aac - build a, hard...
bullet_32khz_96.aac - build a, hard
bullet_44khz_80.aac - build a, both have terrible noise..
bullet_44khz_96.aac - build a better, again both have terrible noise ..
bullet_44khz_128.aac - build a, noice again
castanets_32khz_80.aac - build b sounds better
castanets_32khz_96.aac - build b
castanets_44khz_80.aac - build b
castanets_44khz_96.aac - build b
castanets_44khz_128.aac - build b
fatboy_32khz_80.aac - build b
fatboy_32khz_96.aac - build b, although it has a noticable bad clicking noise, hard to notice quality difference between a and b other than the click though.
fatboy_44khz_80.aac - build b better
fatboy_44khz_96.aac - build b
fatboy_44khz_128.aac - build b, BUT, terrible clicking noise at one point like in 32khz_96, but other than that quality is much better than a, unlike 32khz_96
NIN_becoming_32khz_80.aac - build b
NIN_becoming_32khz_96.aac - build a
NIN_becoming_44khz_80.aac - build b
NIN_becoming_44khz_96.aac - build b
NIN_becoming_44khz_128.aac - build a
si02_32khz_80.aac - build b, easy
si02_32khz_96.aac - build b, easy
si02_44khz_80.aac - build b, easy
si02_44khz_96.aac - build b, easy
si02_44khz_128.aac - build b, all easy
velvet_32khz_80.aac - build b, hard
velvet_32khz_96.aac - build b, hard
velvet_44khz_80.aac - build b
velvet_44khz_96.aac - build a
velvet_44khz_128.aac - build a


Weeeeew...... FINISHED.
As you can see, took much time, i like my results accurate ...

Glad to help.

This post has been edited by SK1: Nov 27 2002, 19:04

[guruboolez]

··· gURuBoOleZZ results (work in progress) ···

I suggest a unique message for each participant There are many files that need tests : not easy to publish detailed results for 40 ABX comments. Better complete in the next day the same place in order to avoid chaos.

Soundcard : Terratec DMX6Fire
Headphone : BeyerDynamic DT-531
Software : ABC/HR
Language : french

41_30.aac --- 128kb/s

passage : 0.00 - 3.00
Sample A : Note = 2.0/5 ABX = 12/12
Sample B : Note = 4.0/5 ABX = 12/13
General comments :
sampleA : heavy distorsion at the beginning
sampleB : tremulous signal

passage : 3.00 - 6.00
Sample A : Note = 3.5/5 ABX = 12/12
Sample B : Note = 3.5/5 ABX = 12/12
General comments :
sampleA : muffled sound. Loss of color and treble. Cymbal slightly metallic
sampleB : same remarks

passage : 15.00 - 18.00
Sample A : Note = 2.0/5 ABX = 12/12
Sample B : Note = 2.2/5 ABX = 11/12
General comments :
sampleA : muffled sound. Distorsion. Strange phenomenon : there is a 'gap' in the midle, where I found the signal to be blurred, without life and details, like a strong video post-preocessing.
sampleB : same remarks. Maybe better treble
I tried to ABX sampleA and sampleB : failure.

passage : 27.00 - 30.00
Sample A : Note = 3.0/5 ABX = 12/12
Sample B : Note = 4.0/5 ABX = 12/12
General comments :
sampleA : embarrassing distorsions. Saxo is altered.
sampleB : Distorsions, but better sound, esp. on saxo.
I tried to ABX sampleA and sampleB : failure, but good beginning (maybe lake of concentration).
==> build B is better

41_30.aac --- 96kb/s --- 44.1Khz

Hard to choose. But build B seems to be little sharper on cymbals, but maybe distorted too. I gave a better note on build B on cymbals. Difficult to isolate a passage that distinguish A and B.

==> build A & build B are nearly the same

41_30.aac --- 96kb/s --- 32Khz

passage : 0.00 - 3.00
Main difference :
sampleA : Awful distorsion at the beginning
I tried to ABX sampleA and sampleB : Success (11/12)

passage : 14.5-16.6
Main difference :
sampleA : Same phenomenon : like a dry cleaning of noise and détails [I called it "post-processing" a moment ago] : unatural sound.
I tried to ABX sampleA and sampleB : Success (10/10)

==> build B is far better on some point.

41_30.aac --- 80kb/s --- 44.1Khz

Not easy to choose. I focused on cymbal again [4.7-5.5], and found build B to sound worse. I ABXed with hesitation the difference betwwen two files : 12/16 and 14/20, I found A > B.

==> I prefer build A

This post has been edited by guruboolez: Nov 28 2002, 00:08

[Guest_SK1_*]

Forget ABXing, what's the use?? It's not like you need to try to distinguish from the original or something , it's just about what sounds better to you. not determining the original..
(edited previous post)

[guruboolez]

Forget ABXing, what's the use??

ABC/HR :

Removing placebo effect.
Giving more precision for some passage for the same sample.

But I agree : ABX scores (12/12, etc...) are useless here.

[Guest_SK1_*]

Fair enough ...

[Mac]

I wish I could help more, I don't have the ears or equipment for it

I've tried testing, just can't come up with anything conclusive or repeatable.

[JohnV]

What bothers me is the NIN-becoming sample and its very fluttering background noise. This is imo very annoying at lower bitrates and it's still quite noticeable in 128kbps. Old Psytel 2.15 was much better in this regard from 80 to 128kbps.

Anothing thing is the (pumpkins) bullet sample. In Psytel 2.15 with -normal vbr there's very clear panning effect especially at about 6.8 seconds when the singer sings "of my *rage*". (See here for the thread)
This is not at all so bad at 128cbr even in psytel 2.15. So I was wondering if the panning effect has been fixed in the new plugin's vbr mode also?

[Mac]

I noticed the start of the NIN sample was horrible as well. It sounds as though there should be background hiss, but it stutters and sounds like bad mp3 artifacting. Didn't have the original song, so couldn't compare with 2.15.

[wildboar]

The beginning of the NIN clip should have a constant hiss going on with a really airy sounding background with stuff that sounds like shoes moving and squeeking and things being moved around on a hard surface floor, like marble or something. The sound on these clips wavers back and forth like JohnV said... the original problem I found was with Psytel 2.15 giving a squished artifact sound when the sharp impulses came in later in the song. This has been cleaned up really well

[JohnV]

I noticed the start of the NIN sample was horrible as well.  It sounds as though there should be background hiss, but it stutters and sounds like bad mp3 artifacting.  Didn't have the original song, so couldn't compare with 2.15.

Original can be found here:
http://static.hydrogenaudio.org/extra/Psytel/

[smok3]

44 khz/128 kbit:
-----------
velvet_44khz_128_aac

orig vs a
abx 13/16
not preecho, but some burps added to the last part of the drum (post echo?).

orig vs b
abx 13/16 (not easy)
hiss in the background is a bit to loud or artifical, especialy in the right channel.

a vs b
abx 12/16 (p=0.038)
hard to say which one is better, probably a.

----------------------------------------------
applaud_44khz_128_aac

orig vs a
abx 5/5 (didnt bother)
smearing in the background at the end of the sample and the 'thank you' part has some high frequencies that arent there in original.

orig vs b
didnt bother
preecho, a bit less smearing in the background

a vs b
abx 12/16
hard to say which one is better, probably a.
----------------------------------------------
bullet_44khz_128_aac

orig vs a
abx 5/5
didnt bother, the same stereo error as with fatboy at the beggining, other than that i cant hear any problems

orig vs b
didnt bother, the same stereo error as with fatboy at the beggining, other than that i cant hear any problems

cant abx a vs b
----------------------------------------------
fatboy_44khz_128_aac

orig vs a
1st try
cant abx orig vs a
2nd try
abx 10/11 (p=0.006) - 1st time i overlooked the stereo error, this time i just listened from 1st second on to ignore the error. (less defined than sample b, but without dropouts)

orig vs b
abx 16/16 stereo error at the beginning
2nd try - this time ignoring the stereo error at the beginning
abx 8/8 something weird on 3s-5s - short dropouts
i cant say which one is better, probably a.

conclusion: i will always rate sample that is less defined better than sample with hiss or high-frequency drop-outs, for me most of this samples (128kbit/44khz) are close to transparency and it wasnt easy to abx them against original.

This post has been edited by smok3: Dec 1 2002, 18:40

[JohnV]

As Ivan said, all the originals can be found in these places:
http://lame.sourceforge.net/download/samples/
http://static.hydrogenaudio.org/extra/Psytel/
http://static.hydrogenaudio.org/extra/samp...s/test_samples/
(si02 is the castanets2 sample on HA)

[smok3]

tnx johnV i found most, which one is 41_30s ? (this isnt easy via dialup )

[JohnV]

tnx johnV i found most, which one is 41_30s ? (this isnt easy via dialup )

Hmm, you are right, it's not there. I thought it was because it's one of the most used clips nowadays. Anyway, I'll upload it to the:
http://static.hydrogenaudio.org/extra/samp...s/test_samples/
but you can also get it from ff123's sample archive:
http://www.ff123.net/samples/

[Ivan Dimkovic]

Ok, attack issues with NIN_becoming.wav should be fixed now - this is the output from (very soon to be released ;-) version:

http://www.psytel-research.co.yu/temp/v2.4/

[Phobos]

OHH BOY, THOSE NEW SAMPLES ARE MAD!!! I bet 192kbps audio can be finaly transparent

[guruboolez]

It's hard to find a difference between 2.30 sample and 2.40 one.
I tried to ABX velvet.mp4
The first time, I find something different that I can't defined between the two versions.
I ABXed velvet 230 and velvet 240, and obtain a 11/16, with 2.30 beeter than 2.40
I ABXed this a second time, the first 0.4 second, and I clearly (but hard to obtain) find 2.40 worse than 2.30. I can't defined it : a real small distorsion is the more precise description I'm able to make. I's not obvious, but I ABX this two time.

I haven't compared other samples yet.


EDIT : forget what I wrote. I did a big mistake... I decoded the sample with CoolEdit2, and the sound is really bad compared to in_mp4.dll (faad2 from august). Must try again.

This post has been edited by guruboolez: Dec 10 2002, 21:11

[Ivan Dimkovic]

There is a big difference, at least with NIN_becoming.mp4 - check out the first several drum "attacks" - old version made "shquish" sound on third hihat hit, also three others were clearly distorted sounding like bicycle ring bell.

2.40 encodes them without problems

[guruboolez]

I will try.
I tried again velved, properly decoded, and it was impossible to perform the same result.

The mp4 engine from CoolEdit2 seems to be the FAAC.flt, dated from 08.26.02
Can someone confirm the muffled sound I heard compared to in_mp4 ?

[guruboolez]

Confirmed. (I'm not dreaming : there is no NIN sample in 2.30 folder ?!)

But I find a small problem by comparing NIN sampleB (128 kbps) to NIN 2.40.
At 27.0-27.7 exactly, there is a typical sound, mixing electronic sample to cymbal. I find (and ABX easily : 38/50 with a worse beginning) a small pre-echo, especially on right channel with one sample. The other was too hard to ABX. The problem file was NIN_becoming encoded with 2.40.

Can someone confirm this ?
I find this flaw more annoying, because of the nature and the supposed redundancy (I'm only gessing) of this problematic sound.

[menno]

The mp4 engine from CoolEdit2 seems to be the FAAC.flt, dated from 08.26.02
Can someone confirm the muffled sound I heard compared to in_mp4 ?

Yes that version is broken. I'm not sure if a newer compile is available at Roberto's page. Should be.

Menno

[guruboolez]

I downloaded this file :
http://www.inf.ufpr.br/~rja00/files/faacplug.zip
input and output plug-ins for cooledit.

Same date, same problem. Are all encoding affected (high bitrate esp.) ? I sometime used CoolEdit for decoding file in order to perform blind test.