IPB

Welcome Guest ( Log In | Register )

64kbps public listening test, FINISHED
rjamorim
post Sep 22 2003, 09:10
Post #1


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



Hello.

There have been some delays, but here are the so awaited results of the latest 64kbps group blind listening test.

http://audio.ciara.us/test/64test/results.html

And the final plot:
Each vertical line segment represents the 95% confidence interval (using ANOVA analysis) for each codec.


Note: Lame MP3 is 128kbps "high anchor" in this test, FhG MP3 is the "low anchor"

Zoomed version without the anchors:


Codec specifications:
  • Ahead/Nero 6.0.0.15 HE AAC VBR profile Streaming :: Medium, high quality
  • Ogg Vorbis post-1.0 CVS -q 0
  • MP3pro (from Adobe Audition 1.0) VBR quality 40, Current Codec, allow M/S and IS, allow narrowing, no CRC
  • Real Audio Gecko (from Real Producer 9.0.1 64kbps
  • Windows Media Audio v9 VBR quality 50
  • QuickTime 6.3 AAC LC 64kbps, Best Quality
  • Lame MP3 encoder 3.90.3 --alt-preset 128 --scale 1. (high anchor)
  • FhG MP3 encoder (from Adobe Audition 1.0) 64kbps CBR, Current codec, allow M/S, no I/S, allow narrowing, no CRC. (low anchor)
Thanks to everyone that participated and helped!

Best regards;

Roberto.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
 
Start new topic
Replies
[proxima]
post Sep 22 2003, 14:34
Post #2





Group: Members
Posts: 197
Joined: 12-October 02
From: Italy
Member No.: 3537



This is the table of my results:



According to the averege the best my preferred encoder seems to be HE AAC even if i should say that iím quite surprised for the result of mp3PRO (or maybe iíve overestimated HE AAC).

Iíve done a sort of ranking for the best, real antagonists at 64 Kbps. The first position is colored in green, the second in yellow and the third in red. With this direct comparison Mp3PRO is (according to my preferences) often better than HE AAC.

Mp3Pro has shown a detectable lowpass (16Khz) but this is not the real problem for a 64 Kbps, artifacts are more annoying. There is an interesting thing iíve perceived: with HE AAC the high frequencies seem unnatural, attenuated, as if it was lowpassed. While doing the blind test i imputed this to the lowpass, but later iíve discovered that the HE AAC files are lowpassed at about 20 kHz (surely inaudible for me). Does FAAD use dithering when decoding ? If not, i think that the fact could be explained with the SBR ďproblemĒ of which guruboolez give us an excellent description.

While the two codecs above scored a very close quality level, i canít say the same for Vorbis that is often behind the others two. I surely agree with guruboolez: Vorbis in this bitrate range is easy detectable because of noise and exaggerated highs (with sharp attacks the result is quite annoying).
We all are waiting for the 1.1 version that should give better result with this type of artifacts.

At the end, i sincerely want to thank Roberto for his effort organizing this useful test. The number of participants is increased and this is a clear indication of good organization.


--------------------
WavPack 4.3 -mfx5
LAME 3.97 -V5 --vbr-new --athaa-sensitivity 1
Go to the top of the page
+Quote Post

Posts in this topic
- rjamorim   64kbps public listening test   Sep 22 2003, 09:10
- - bond   yummie hm, first time that i voted most of the c...   Sep 22 2003, 09:15
- - neoufo51   Interesting... I expect Vorbis to be a little hig...   Sep 22 2003, 09:20
- - rjamorim   Bloody 5:20 AM here. I'll read criticisms, co...   Sep 22 2003, 09:21
- - rjamorim   BTW: If the server starts with that redirection fa...   Sep 22 2003, 09:22
- - rjamorim   One last thing: People are invited to announce it ...   Sep 22 2003, 09:24
- - Ivan Dimkovic   Roberto, can you please move the anchors (low-hig...   Sep 22 2003, 09:26
- - Jon Ingram   The group results seem to be: Lame > He AAC, MP...   Sep 22 2003, 09:27
- - Gabriel   QUOTE the anchors (low-high) to the right - and/or...   Sep 22 2003, 09:28
- - rjamorim   QUOTE (Ivan Dimkovic @ Sep 22 2003, 05:26 AM)...   Sep 22 2003, 09:32
- - ScorLibran   Here are the averaged results of my own tests on j...   Sep 22 2003, 09:35
- - guruboolez   I obtained different results, with bad notation fo...   Sep 22 2003, 09:41
- - Jon Ingram   QUOTE (ScorLibran @ Sep 22 2003, 08:35 AM)The...   Sep 22 2003, 09:49
- - bond   edit: my zoomed in pic is already on the official ...   Sep 22 2003, 09:56
- - mcbevin   Hey, I think it might be worth emphasizing (or at ...   Sep 22 2003, 10:29
- - 2Bdecided   QUOTE (guruboolez @ Sep 22 2003, 08:41 AM)I o...   Sep 22 2003, 10:43
- - 2Bdecided   btw(!) Thanks for conducting such an excellent...   Sep 22 2003, 10:44
- - guruboolez   Note than Vorbis sufffered by an artifact (reverbe...   Sep 22 2003, 10:52
- - Continuum   QUOTE (Jon Ingram @ Sep 22 2003, 10:27 AM)The...   Sep 22 2003, 10:58
- - ScorLibran   QUOTE (Jon Ingram @ Sep 22 2003, 04:49 AM)Wit...   Sep 22 2003, 11:27
- - eloj   Slashdot is going to be hell if this is posted and...   Sep 22 2003, 11:31
- - teetee   QUOTE (2Bdecided @ Sep 22 2003, 10:44 AM)EDIT...   Sep 22 2003, 11:55
- - schnofler   My results looked like this: Lame, HE AAC > MP3...   Sep 22 2003, 12:04
- - phong   WOW, very interesting! The results are so far...   Sep 22 2003, 12:08
- - Garf   QUOTE (eloj @ Sep 22 2003, 12:31 PM)Also, how...   Sep 22 2003, 12:12
- - Vietwoojagig   Will the next test be a 96kbps test, with the same...   Sep 22 2003, 12:29
- - bond   QUOTE (Vietwoojagig @ Sep 22 2003, 01:29 PM)W...   Sep 22 2003, 12:31
- - phong   QUOTE (guruboolez)I'm not at ease with vorbis ...   Sep 22 2003, 12:36
- - Garf   QUOTE (bond @ Sep 22 2003, 01:31 PM)he-aac is...   Sep 22 2003, 12:46
- - Garf   QUOTE (phong @ Sep 22 2003, 01:36 PM)I hope 1...   Sep 22 2003, 12:47
- - bond   QUOTE (Garf @ Sep 22 2003, 01:46 PM)QUOTE (bo...   Sep 22 2003, 13:20
- - JohnV   QUOTE (bond @ Sep 22 2003, 03:20 PM)QUOTE (Ga...   Sep 22 2003, 13:42
- - bond   QUOTE (JohnV @ Sep 22 2003, 02:42 PM)Umm, jus...   Sep 22 2003, 13:48
- - JohnV   QUOTE (bond @ Sep 22 2003, 03:48 PM)QUOTE (Jo...   Sep 22 2003, 13:52
- - listen   Gosh! I only listened to one sample (New York...   Sep 22 2003, 14:00
- - sthayashi   Anyone else think it interesting that Ogg Vorbis w...   Sep 22 2003, 14:03
- - JohnV   QUOTE (sthayashi @ Sep 22 2003, 04:03 PM)Anyo...   Sep 22 2003, 14:06
- - Jojo   Great test! However, I wonder who is doing 64k...   Sep 22 2003, 14:09
- - JohnV   QUOTE (Jojo @ Sep 22 2003, 04:09 PM)By the wa...   Sep 22 2003, 14:18
- - Jon Ingram   QUOTE (Jojo @ Sep 22 2003, 01:09 PM)I hope th...   Sep 22 2003, 14:23
- - [proxima]   This is the table of my results: According to t...   Sep 22 2003, 14:34
- - amano   hmm, surprisingly wma was very close to the low an...   Sep 22 2003, 14:34
- - Atlantis   QUOTE (amano @ Sep 22 2003, 03:34 PM)hmm, sur...   Sep 22 2003, 14:36
- - rjamorim   QUOTE (Vietwoojagig @ Sep 22 2003, 08:29 AM)W...   Sep 22 2003, 14:41
- - bond   QUOTE (rjamorim @ Sep 22 2003, 03:41 PM)@bond...   Sep 22 2003, 14:44
- - Gecko   At first, let me thank Roberto for his efforts: th...   Sep 22 2003, 14:52
- - Garf   QUOTE (Jojo @ Sep 22 2003, 03:09 PM)Great tes...   Sep 22 2003, 15:01
- - Garf   QUOTE Many people found it hard to detect artifact...   Sep 22 2003, 15:04
- - rjamorim   Hello. Menno came with an idea: Replacing "L...   Sep 22 2003, 15:10
- - guruboolez   QUOTE (rjamorim @ Sep 22 2003, 03:10 PM)Hello...   Sep 22 2003, 15:21
- - rjamorim   QUOTE (guruboolez @ Sep 22 2003, 11:21 AM)Is ...   Sep 22 2003, 15:43
- - smok3   @rjamorim: tnx for organizing and conducting the ...   Sep 22 2003, 15:43
- - AstralStorm   Thank you rjamorim for making my results 'anon...   Sep 22 2003, 15:46
- - rjamorim   QUOTE (smok3 @ Sep 22 2003, 11:43 AM)flame ve...   Sep 22 2003, 15:47
- - rjamorim   QUOTE (AstralStorm @ Sep 22 2003, 11:46 AM)Th...   Sep 22 2003, 15:48
- - bond   good idea calling it low/high anchor (perhaps you ...   Sep 22 2003, 15:50
- - guruboolez   Glad to know anon1's name. It's one of the...   Sep 22 2003, 15:51
- - 2Bdecided   I was sad to be anon05 - I'll have to demand c...   Sep 22 2003, 16:18
- - ff123   QUOTE (2Bdecided @ Sep 22 2003, 07:18 AM)I wa...   Sep 22 2003, 16:31
- - c_haese   QUOTE (rjamorim @ Sep 22 2003, 09:47 AM)OK, I...   Sep 22 2003, 17:44
- - Niknak   Can you give us the actual bitrates or filesizes w...   Sep 22 2003, 17:49
- - de Mon   I can't understand why didn't the best ver...   Sep 22 2003, 18:42
- - Continuum   GT3 is only real tuned for high bitrates (>160 ...   Sep 22 2003, 18:45
- - sony666   the mp3PRO result is interesting.. too bad the for...   Sep 22 2003, 18:56
- - JohnV   QUOTE (de Mon @ Sep 22 2003, 08:42 PM)I can...   Sep 22 2003, 19:10
- - JohnV   QUOTE (sony666 @ Sep 22 2003, 08:56 PM)the mp...   Sep 22 2003, 19:17
- - AstralStorm   I can't read the comments from the site. If I...   Sep 22 2003, 19:22
- - de Mon   Thanks you for your replies, Continuum and JohnV. ...   Sep 22 2003, 21:54
- - Dologan   Heh, I find it weird that I usually scored MP3Pro ...   Sep 22 2003, 22:00
- - ff123   Results have been slashdotted: http://slashdot.or...   Sep 22 2003, 22:42
- - AstralStorm   My results: CODELAME     MP3Pro   H...   Sep 22 2003, 23:03
- - pseudoacoustic   Great test Roberto. However, I also cannot access ...   Sep 22 2003, 23:28
- - rjamorim   QUOTE (ff123 @ Sep 22 2003, 06:42 PM)Results ...   Sep 23 2003, 03:11
- - ff123   I always find stuff to reply to in the /. threads ...   Sep 23 2003, 04:17
- - Dologan   QUOTE (rjamorim @ Sep 22 2003, 08:11 PM)That ...   Sep 23 2003, 04:20
- - rjamorim   QUOTE (ff123 @ Sep 23 2003, 12:17 AM)I always...   Sep 23 2003, 04:47
- - phong   I added a big rant to the /. thread, but I don...   Sep 23 2003, 05:07
- - ScorLibran   QUOTE (From Slashdot) The poster offers an intere...   Sep 23 2003, 05:24
- - JohnV   ok ok.. everybody knows what kind of place slashdo...   Sep 23 2003, 05:31
- - rjamorim   LOL!   Sep 23 2003, 05:41
- - rjamorim   QUOTE (ScorLibran @ Sep 23 2003, 01:24 AM)Int...   Sep 23 2003, 05:50
- - phong   QUOTE (ScorLibran)Some people are like defective p...   Sep 23 2003, 06:10
- - ScorLibran   QUOTE (phong @ Sep 23 2003, 01:10 AM)QUOTE (S...   Sep 23 2003, 06:34
- - ff123   QUOTE (phong @ Sep 22 2003, 09:10 PM)Can anyo...   Sep 23 2003, 06:35
- - [proxima]   QUOTE (phong @ Sep 23 2003, 06:10 AM)Can anyo...   Sep 23 2003, 09:48
- - joey_m   Ditto with WMA... I rated it even better than LAME...   Sep 23 2003, 15:58
- - vinnie97   If you can easily detect upper-range frequencies, ...   Sep 23 2003, 17:01
- - rjamorim   QUOTE (vinnie97 @ Sep 23 2003, 01:01 PM)Rober...   Sep 23 2003, 17:51
- - [proxima]   QUOTE (rjamorim @ Sep 23 2003, 05:51 PM)You s...   Sep 23 2003, 18:07
- - Volcano   Just wanted to say... great work as usual, Roberto...   Sep 23 2003, 18:12
- - rjamorim   QUOTE ([proxima),Sep 23 2003, 02:07 PM] I...   Sep 23 2003, 18:29
- - [proxima]   Roberto, these sort of mistakes clearly indicate ...   Sep 23 2003, 19:22
- - Latexxx   QUOTE (joey_m @ Sep 23 2003, 06:58 AM)Ditto w...   Sep 23 2003, 19:54
- - rjamorim   Hello. I just uploaded the bitrate tables. They a...   Sep 23 2003, 20:12
- - phong   Thanks everyone for the comments on WMA, that...   Sep 23 2003, 20:28
- - spoon   I am thinking that certain manufacturer claims of ...   Sep 23 2003, 22:36
- - XXX   It's always a mistake to scale a chart that na...   Sep 23 2003, 23:00
- - Dibrom   QUOTE (rjamorim @ Sep 23 2003, 12:12 PM)Hello...   Sep 23 2003, 23:06
- - Dibrom   QUOTE (spoon @ Sep 23 2003, 02:36 PM)I am thi...   Sep 23 2003, 23:09
- - Jon Ingram   You mention in the blurb that it's interesting...   Sep 23 2003, 23:13
2 Pages V   1 2 >


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 17th April 2014 - 20:41