Hi, - before releasing final version I would like to hear audiophiles opinion about two noise allocation strategies we can use.

These two versions will be called "build_a" and "build_b"

I've packed some of the essential "codec killers" samples in build_a.zip and build_b.zip - originals could be easily found on hydrogenaduio and mp3dev

http://www.psytel-research.co.yu/test/

BUILD A:

http://www.psytel-research.co.yu/test/build_a.zip


BUILD B:

http://www.psytel-research.co.yu/test/build_b.zip


Which one is better, and for which samples?

Files are CBR - 80, 96 and 128 kbps - 32 and 44.1 kHz sampling rate - ADIF AAC format

I moved the thread to the news section.. it'll probably get more views that way. Hope you don't mind

I downloaded the files btw.. I'll try to submit some results a little later.

Result So Far:

Castanets - 44.1khz@128kbps - Build_b is better. Build_a has a terrible whoosing/tearing sound (most audible in the left channel) during the fast attacks that makes it much more annoying. Neither are transparent, though I haven't abx'd yet.

Fatboy - 44.1khz@128kbps - Build_b is better. Build_a has a sort of stuttering and skipping sound at one point. Neither are transparent.

Btw, these results are from only about 10-15 seconds listening to each sample and without having compared directly to the original (it's just off memory) or having done abx testing..

More later.. (including more thorough testing)

OH YEAH!
OK, starting to test. This thread will be edited and will contain detailed, ahm, details ..

41_30sec_32khz_80.aac - build a sounds better, was hard to choose.
41_30sec_32khz_96.aac - build a sounds better, hard to choose.
41_30sec_44khz_80.aac - build b sounds better.
41_30sec_44khz_96.aac - build a sounds better.
41_30sec_44khz_128.aac - build a sounds better, hard to choose.
applaud_32khz_80.aac - build b sounds better
applaud_32khz_96.aac - build a, hard to choose
applaud_44khz_80.aac - build b, hard to choose
applaud_44khz_96.aac - build a
applaud_44khz_128.aac - build b
bullet_32khz_80.aac - build a, hard...
bullet_32khz_96.aac - build a, hard
bullet_44khz_80.aac - build a, both have terrible noise..
bullet_44khz_96.aac - build a better, again both have terrible noise ..
bullet_44khz_128.aac - build a, noice again
castanets_32khz_80.aac - build b sounds better
castanets_32khz_96.aac - build b
castanets_44khz_80.aac - build b
castanets_44khz_96.aac - build b
castanets_44khz_128.aac - build b
fatboy_32khz_80.aac - build b
fatboy_32khz_96.aac - build b, although it has a noticable bad clicking noise, hard to notice quality difference between a and b other than the click though.
fatboy_44khz_80.aac - build b better
fatboy_44khz_96.aac - build b
fatboy_44khz_128.aac - build b, BUT, terrible clicking noise at one point like in 32khz_96, but other than that quality is much better than a, unlike 32khz_96
NIN_becoming_32khz_80.aac - build b
NIN_becoming_32khz_96.aac - build a
NIN_becoming_44khz_80.aac - build b
NIN_becoming_44khz_96.aac - build b
NIN_becoming_44khz_128.aac - build a
si02_32khz_80.aac - build b, easy
si02_32khz_96.aac - build b, easy
si02_44khz_80.aac - build b, easy
si02_44khz_96.aac - build b, easy
si02_44khz_128.aac - build b, all easy
velvet_32khz_80.aac - build b, hard
velvet_32khz_96.aac - build b, hard
velvet_44khz_80.aac - build b
velvet_44khz_96.aac - build a
velvet_44khz_128.aac - build a


Weeeeew...... FINISHED.
As you can see, took much time, i like my results accurate ...

Glad to help.

··· gURuBoOleZZ results (work in progress) ···

I suggest a unique message for each participant There are many files that need tests : not easy to publish detailed results for 40 ABX comments. Better complete in the next day the same place in order to avoid chaos.

Soundcard : Terratec DMX6Fire
Headphone : BeyerDynamic DT-531
Software : ABC/HR
Language : french



passage : 0.00 - 3.00
Sample A : Note = 2.0/5 ABX = 12/12
Sample B : Note = 4.0/5 ABX = 12/13
General comments :
sampleA : heavy distorsion at the beginning
sampleB : tremulous signal

passage : 3.00 - 6.00
Sample A : Note = 3.5/5 ABX = 12/12
Sample B : Note = 3.5/5 ABX = 12/12
General comments :
sampleA : muffled sound. Loss of color and treble. Cymbal slightly metallic
sampleB : same remarks

passage : 15.00 - 18.00
Sample A : Note = 2.0/5 ABX = 12/12
Sample B : Note = 2.2/5 ABX = 11/12
General comments :
sampleA : muffled sound. Distorsion. Strange phenomenon : there is a 'gap' in the midle, where I found the signal to be blurred, without life and details, like a strong video post-preocessing.
sampleB : same remarks. Maybe better treble
I tried to ABX sampleA and sampleB : failure.

passage : 27.00 - 30.00
Sample A : Note = 3.0/5 ABX = 12/12
Sample B : Note = 4.0/5 ABX = 12/12
General comments :
sampleA : embarrassing distorsions. Saxo is altered.
sampleB : Distorsions, but better sound, esp. on saxo.
I tried to ABX sampleA and sampleB : failure, but good beginning (maybe lake of concentration).
==> build B is better



Hard to choose. But build B seems to be little sharper on cymbals, but maybe distorted too. I gave a better note on build B on cymbals. Difficult to isolate a passage that distinguish A and B.

==> build A & build B are nearly the same



passage : 0.00 - 3.00
Main difference :
sampleA : Awful distorsion at the beginning
I tried to ABX sampleA and sampleB : Success (11/12)

passage : 14.5-16.6
Main difference :
sampleA : Same phenomenon : like a dry cleaning of noise and détails [I called it "post-processing" a moment ago] : unatural sound.
I tried to ABX sampleA and sampleB : Success (10/10)

==> build B is far better on some point.



Not easy to choose. I focused on cymbal again [4.7-5.5], and found build B to sound worse. I ABXed with hesitation the difference betwwen two files : 12/16 and 14/20, I found A > B.

==> I prefer build A

Forget ABXing, what's the use?? It's not like you need to try to distinguish from the original or something , it's just about what sounds better to you. not determining the original..
(edited previous post)

ABC/HR :

Removing placebo effect.
Giving more precision for some passage for the same sample.

But I agree : ABX scores (12/12, etc...) are useless here.

Fair enough ...

I wish I could help more, I don't have the ears or equipment for it

I've tried testing, just can't come up with anything conclusive or repeatable.

What bothers me is the NIN-becoming sample and its very fluttering background noise. This is imo very annoying at lower bitrates and it's still quite noticeable in 128kbps. Old Psytel 2.15 was much better in this regard from 80 to 128kbps.

Anothing thing is the (pumpkins) bullet sample. In Psytel 2.15 with -normal vbr there's very clear panning effect especially at about 6.8 seconds when the singer sings "of my *rage*". (See here for the thread)
This is not at all so bad at 128cbr even in psytel 2.15. So I was wondering if the panning effect has been fixed in the new plugin's vbr mode also?

I noticed the start of the NIN sample was horrible as well. It sounds as though there should be background hiss, but it stutters and sounds like bad mp3 artifacting. Didn't have the original song, so couldn't compare with 2.15.

The beginning of the NIN clip should have a constant hiss going on with a really airy sounding background with stuff that sounds like shoes moving and squeeking and things being moved around on a hard surface floor, like marble or something. The sound on these clips wavers back and forth like JohnV said... the original problem I found was with Psytel 2.15 giving a squished artifact sound when the sharp impulses came in later in the song. This has been cleaned up really well

Original can be found here:
http://static.hydrogenaudio.org/extra/Psytel/

44 khz/128 kbit:
-----------
velvet_44khz_128_aac

orig vs a
abx 13/16
not preecho, but some burps added to the last part of the drum (post echo?).

orig vs b
abx 13/16 (not easy)
hiss in the background is a bit to loud or artifical, especialy in the right channel.

a vs b
abx 12/16 (p=0.038)
hard to say which one is better, probably a.

----------------------------------------------
applaud_44khz_128_aac

orig vs a
abx 5/5 (didnt bother)
smearing in the background at the end of the sample and the 'thank you' part has some high frequencies that arent there in original.

orig vs b
didnt bother
preecho, a bit less smearing in the background

a vs b
abx 12/16
hard to say which one is better, probably a.
----------------------------------------------
bullet_44khz_128_aac

orig vs a
abx 5/5
didnt bother, the same stereo error as with fatboy at the beggining, other than that i cant hear any problems

orig vs b
didnt bother, the same stereo error as with fatboy at the beggining, other than that i cant hear any problems

cant abx a vs b
----------------------------------------------
fatboy_44khz_128_aac

orig vs a
1st try
cant abx orig vs a
2nd try
abx 10/11 (p=0.006) - 1st time i overlooked the stereo error, this time i just listened from 1st second on to ignore the error. (less defined than sample b, but without dropouts)

orig vs b
abx 16/16 stereo error at the beginning
2nd try - this time ignoring the stereo error at the beginning
abx 8/8 something weird on 3s-5s - short dropouts
i cant say which one is better, probably a.

conclusion: i will always rate sample that is less defined better than sample with hiss or high-frequency drop-outs, for me most of this samples (128kbit/44khz) are close to transparency and it wasnt easy to abx them against original.

As Ivan said, all the originals can be found in these places:
http://lame.sourceforge.net/download/samples/
http://static.hydrogenaudio.org/extra/Psytel/
http://static.hydrogenaudio.org/extra/samp...s/test_samples/
(si02 is the castanets2 sample on HA)

tnx johnV i found most, which one is 41_30s ? (this isnt easy via dialup )

Hmm, you are right, it's not there. I thought it was because it's one of the most used clips nowadays. Anyway, I'll upload it to the:
http://static.hydrogenaudio.org/extra/samp...s/test_samples/
but you can also get it from ff123's sample archive:
http://www.ff123.net/samples/

Ok, attack issues with NIN_becoming.wav should be fixed now - this is the output from (very soon to be released ;-) version:

http://www.psytel-research.co.yu/temp/v2.4/

OHH BOY, THOSE NEW SAMPLES ARE MAD!!! I bet 192kbps audio can be finaly transparent

It's hard to find a difference between 2.30 sample and 2.40 one.
I tried to ABX velvet.mp4
The first time, I find something different that I can't defined between the two versions.
I ABXed velvet 230 and velvet 240, and obtain a 11/16, with 2.30 beeter than 2.40
I ABXed this a second time, the first 0.4 second, and I clearly (but hard to obtain) find 2.40 worse than 2.30. I can't defined it : a real small distorsion is the more precise description I'm able to make. I's not obvious, but I ABX this two time.

I haven't compared other samples yet.


EDIT : forget what I wrote. I did a big mistake... I decoded the sample with CoolEdit2, and the sound is really bad compared to in_mp4.dll (faad2 from august). Must try again.

There is a big difference, at least with NIN_becoming.mp4 - check out the first several drum "attacks" - old version made "shquish" sound on third hihat hit, also three others were clearly distorted sounding like bicycle ring bell.

2.40 encodes them without problems

I will try.
I tried again velved, properly decoded, and it was impossible to perform the same result.

The mp4 engine from CoolEdit2 seems to be the FAAC.flt, dated from 08.26.02
Can someone confirm the muffled sound I heard compared to in_mp4 ?

Confirmed. (I'm not dreaming : there is no NIN sample in 2.30 folder ?!)

But I find a small problem by comparing NIN sampleB (128 kbps) to NIN 2.40.
At 27.0-27.7 exactly, there is a typical sound, mixing electronic sample to cymbal. I find (and ABX easily : 38/50 with a worse beginning) a small pre-echo, especially on right channel with one sample. The other was too hard to ABX. The problem file was NIN_becoming encoded with 2.40.

Can someone confirm this ?
I find this flaw more annoying, because of the nature and the supposed redundancy (I'm only gessing) of this problematic sound.

Yes that version is broken. I'm not sure if a newer compile is available at Roberto's page. Should be.

Menno

I downloaded this file :
http://www.inf.ufpr.br/~rja00/files/faacplug.zip
input and output plug-ins for cooledit.

Same date, same problem. Are all encoding affected (high bitrate esp.) ? I sometime used CoolEdit for decoding file in order to perform blind test.

Yes, everything is affected. I'll make sure Roberto compiles some new compiles.

Menno

Thanks.
I used CoolEdit for one practical reason. With faad, I always have problems in ABC/HR : by clicking on the .wav file obtained with faad, there is no sound. And after that, others wave file have problems [i must change the selection, or clickin many time on other A, B or Reference boxes in order to restore sound - and if I click again on the .wav faad file, same problem].

I just try again in order repeat the problem, it immediatly occurs when I decode the mp4 file provided by Ivan Dimkovic. The faad decoder I tried are :
· Faad-based AAC drag'n'drop frontend (>here<
· a faad version bundled with aacenc.zip 2.15 version.

My soundcard : Terratec DMX6fire.

Is that a knwon bug ? Should I enter a parameter in order to decode properly the file ? Where can I find a recent version ?

Ok, new CoolEdit filter is now available from RareWares.

I also noticed the problem with playing the files under ABC/HR, I'll have to check what the exact problem is.

Menno

There was thread regarding this issue somewhere (WMP doesn't play faad-decoded files).
Something with wav header, IIRC.
I did pair of attempts: decoding mp4 with faad, then opening it with wave editor and saving as ACM waveform:

This is fc /b faad-decoded against saved by editor (two files).

Thanks, it works fine now.
Glad to see that I'm not alone to have this problem with ABC/HR...

ABC/HR has a problem with some WAV files which apparently have incorrect information about where the end of the WAV is. The WAV files can be corrected by opening them in a sound editor, slicing a small bit of information from the beginning or end, and saving back to WAV. A future version of ABC/HR should perform a check for such files.

ff123

I'm going through a similar problem here. Only the first second of the reference file is played back.

Detail is, if I load the test config and try to play back the reference right away, it plays from start to end.

But if I play a sample first, and then try playing the reference, only a fraction of the first second of it is played.

Here's how my config file looks like:



It doesn't seem like a WAV error to me, since it's played normally at first, and later borks.

Am I doing something wrong?

Thanks for any info.

Regards;

Roberto.

Edit: The reference file has been encoded and later decoded with flac.exe. Is that an issue?
All the samples are decoded from mp4, with FAAD. They play fine.

rjamorim,

Regarding your recent inquiry (2 June 03),

In the situation you describe above, where the sample file seems to break the reference file playback, the issue is that one of the files (I think I remember it being the first one, your sample) does not provide abc/hr with the correct ending time. abc/hr uses an MCI function to look this up. I don't know exactly where this function gets its information, but my guess was that it was in the WAV header.

I need to know how long the WAV files are so that I can do things like offsets properly.

I haven't really tracked the problem down further than that.

ff123

I did another test.

This time, I used another sample: ATrain.wav

The same bug occurs. After I listen to one of the samples, the reference file doesn't plays anymore.

Interestingly, I used the reference file (exactly the same file) as sample05. When I try to play back it it plays well - but if I try the reference just after, it borks playback (of the reference only. Sample 5 keeps being playable).

My config file:


Therefore, it's surely an issue in ABC/HR. Seems a wav file only borks when it's being used as reference, since this wav file plays perfectly if being used as sample.

Regards;

Roberto.

I assume you either got ATrain.wav from the zipfile during the 64 kbit/s test or from my samples page, so it hasn't been modified in any way.

I have the zipfile for ATrain saved on my hard drive, so I can reload those files, and then include ATrain.wav as a sample file to try to verify the particular behavior you're describing. But when I was looking at what was going on before, the problem was definitely traced back to the WAV files themselves, with abc/hr being involved because it assumed that the MCI function was returning the correct information from the WAV file.

BTW, the way I remember it was: the first time you play one of those problem files, it will play ok (but it gives abc/hr the wrong end-time information). If you play another file after that, things will be borked. The second file may be ok as a WAV file, even though it doesn't play after things are borked.

ff123

P.S., if you flac and zip your files, and upload them to my site, I can directly verify what you see.

Upload to ff123.net
userid: anonymous@ff123.net
(note: userid must be exactly as shown above, with the @ sign and everything)
use anything as the password, it doesn't matter what
go to the incoming directory and upload to that
you will not get a directory listing when you have finished uploading (a security measure)

I am uploading the file now. It's the exact same file I would distribute as part of the test, except that it lacks the readme ATM.

The files should be uncompressed in ABC/HR's dir, and then you should run
Bin/Sample02.bat

Good luck.

Regards;

Roberto.

roberto,

I got your file.

I verified that when I loaded up the five samples, and then added the original as the 6th sample, it worked (or rather it didn't work) just like you said.

Then I made all of the samples the original, and that worked properly. I think I remember that I check the WAV properties of each file prior to each test. That is, I need to know which file will end last, so that I can play this file all the way to the end instead of cutting it off short.

The bottom line is that this behavior is still caused by an WAV file that is different somehow from the typical WAV file.

I opened atrain_1.wav in Cool Edit, and then saved it as a new WAV file, atrain_1a.wav. The old file causes problems, but the new one doesn't. A binary file comparison (using the DOS command "fc") yields the following differences:

Comparing files ATrain_1.wav and ATRAIN_1A.WAV
0000001C: 90 10
0000001D: C0 B1
0000001E: CD 02

I'll look this up to see what the difference is.

ff123

Great news, ff123. Thanks a lot for taking a look at it.

So, I believe FLAC is outputting (kinda)bork files? Maybe I should use another lossless format for my test?

No, flac is not at fault here.

The problem appears to be with the FAAD decoded WAV files. The field that is different is the "average bytes per second" field.

I copied the WAV format description to my conv2wav program here:

http://ff123.net/convert/conv2wav.c.txt

If you have the time and energy, maybe you can figure out what FAAD is doing wrong.

ff123

BTW, it looks like that field is redundant, and that it can be calculated assuming that all the other information in the format header is correct. The correct approach is to fix FAAD, but another solution would be to create a small "fixit" program to fix the header. You could run such a program after running FAAD to fix the decoded files. That way you don't have to distribute flac'd wav files -- you could still distribute the much smaller mp4 files.

ff123

Thanks for the info.

I guess the problem is in this file, somewhere inside the write_wav_header function
http://pessoal.onda.com.br/rjamorim/audio.c.txt

I'll take a look at it, and also ask for Menno's help.

Regards;

Roberto.

Hello ff123,

Yes FAAD2 used to have a little bug in the wav file writing, your tool made me aware of that a few months ago. Most other programs seem to play the files just fine (apparantly they don't look at the average bytes per sec).

But of course the problem has already been fixed for a few months

Greetings,

Menno

Indeed, my compile was dated December

Everything works perfect now.

If there are no more issues, test starts tonight.

What's this test all about? Obviously I know that it's an AAC test. Is it about Ivan's original request for listening responses or is it something else? Just curious.

ff123

Since AAC has been getting quite some news coverage lately (Apple, AOL, iRiver...), some people and I thought it would be interesting to do a test to see what codec performs better at 128kbps (whch seems to be a good compromise for portable usage and DSL streaming).

Besides, I plan to take the winner and throw it against Vorbis, Lame, MPC and FhG MP3 in another test. (Maybe including WMA and/or Atrac3 also)

Cool. How many samples are you going to try? Sounds like fun.

I've took your 64kbps test samples.

Then, I replaced BachS1007 and LisztBMinor with death2 and 41_30sec, since some killer samples would be interesting at these bitrates.

So, 12 samples in all. All the packages together will sum to 47Mb worth of downloads.

The sample packages are being uploaded to the server now.

If you take WMA9, be sure to use both Std and Pro.

As I understand, there's no difference between them.

WMA pro is just WMA for high bit depths, high sampling rates and multichannel. It isn't even possible to encode a 44100/stereo/16bit sample with it.

How do Codecs with VBR fit in with 128Kbps? to be fair to WMA it should be VBR 2 pass, not that it will win