IPB

Welcome Guest ( Log In | Register )

Personal Listening Test of MP3 encoders at 224kbps, ABC/HR blind test, 1 Listener
Kamedo2
post May 16 2013, 17:59
Post #1





Group: Members
Posts: 168
Joined: 16-November 12
From: Kyoto, Japan
Member No.: 104567



Abstract:
Blind Comparison between Lame 3.100i V2+, Lame 3.99 V1, LAME 3.98 CBR 224kbps -q 0 , Helix -V146, BladeEnc CBR 224kbps(low anchor).

Encoders:
LAME 3.100i
http://www.hydrogenaudio.org/forums/index....showtopic=99483
LAME 3.99.5 VBR V1
http://www.rarewares.org/mp3-lame-bundle.php
LAME 3.98.4 CBR 224kbps -q 0(slowest)
Helix mp3enc v5.1 Open Source encoder 2005-12-20 -V146
http://www.rarewares.org/mp3-others.php
BladeEnc 0.94.2 CBR 224kbps (low anchor)

Settings:
lame3100i -S -V2+ input.wav output.mp3
lame3.99.5 -S -V1 input.wav output.mp3
lame3.98.4 -S -q 0 -b 224 input.wav output.mp3
hmp3 input.wav output.mp3 -X2 -U2 -V146
bladeenc -quit -nocfg input.wav output.mp3 -224

Samples:
25 Sounds of various genres.

Hardwares:
Sony PSP-3000 + RP-HT560.

Results:



Conclusions & Observations:
I could not a significant difference except the low anchor. There are no big differences in the average quality of these four encoders.

Anova analysis:
CODE
FRIEDMAN version 1.24 (Jan 17, 2002) http://ff123.net/
Blocked ANOVA analysis

Number of listeners: 25
Critical significance: 0.05
Significance of data: 0.00E+000 (highly significant)
---------------------------------------------------------------
ANOVA Table for Randomized Block Designs Using Ratings

Source of Degrees Sum of Mean
variation of Freedom squares Square F p

Total 124 23.56
Testers (blocks) 24 7.75
Codecs eval'd 4 9.24 2.31 33.80 0.00E+000
Error 96 6.56 0.07
---------------------------------------------------------------
Fisher's protected LSD for ANOVA: 0.147

Means:

Helix-V1 3.100iV2 3.98CBR 3.99V1 BladeEnc
4.60 4.58 4.57 4.54 3.90

---------------------------- p-value Matrix ---------------------------

3.100iV2 3.98CBR 3.99V1 BladeEnc
Helix-V1 0.829 0.706 0.419 0.000*
3.100iV2 0.871 0.553 0.000*
3.98CBR 0.666 0.000*
3.99V1 0.000*
-----------------------------------------------------------------------

Helix-V146 is better than BladeEncCBR
3.100iV2+ is better than BladeEncCBR
3.98CBR is better than BladeEncCBR
3.99V1 is better than BladeEncCBR

Raw data:
CODE
% MP3 224kbps ABC/HR Score
% This format is compatible with my graphmaker, as well as ff123's FRIEDMAN.
3.100iV2+ 3.99V1 3.98CBR Helix-V146 BladeEncCBR
%feature 7 LAME LAME LAME Other Other
4.700 4.600 4.000 4.300 3.100
4.300 4.200 4.600 4.800 3.800
4.500 4.500 4.400 5.000 4.700
4.800 5.000 4.600 5.000 4.300
4.700 4.500 4.200 4.400 3.500
4.700 4.300 5.000 4.600 4.500
4.400 5.000 3.800 4.700 3.900
4.200 4.500 4.400 4.500 3.800
4.300 4.200 4.000 4.500 3.200
4.400 4.300 5.000 4.600 3.400
4.000 4.300 4.500 4.600 3.500
4.500 4.200 4.400 4.600 3.600
4.200 4.500 5.000 4.700 4.000
4.300 4.100 4.300 4.100 3.500
4.200 4.200 4.400 4.600 3.900
5.000 4.500 5.000 5.000 4.100
5.000 4.300 4.700 4.400 4.000
4.500 4.400 4.200 4.000 3.200
5.000 5.000 5.000 4.500 4.400
5.000 5.000 4.700 5.000 4.400
5.000 4.800 4.800 4.600 4.200
4.700 5.000 5.000 4.500 3.900
4.800 5.000 5.000 4.600 4.200
5.000 5.000 4.700 5.000 4.200
4.400 4.100 4.600 4.400 4.100
%samples 41_30sec hihats
%samples finalfantasy cemb
%samples ATrain Jazz
%samples BigYellow Pops
%samples FloorEssence Techno
%samples macabre orch
%samples mybloodrusts guitar
%samples Quizas Latin
%samples VelvetRealm Techno
%samples Amefuribana Pops
%samples Trust Gospel
%samples Waiting Rock
%samples Experiencia Latin
%samples Heart to Heart Pops
%samples Tom's Diner Vocal
%samples Reunion Blues Jazz
%samples French Speech
%samples undelete Pops
%samples Dimmu Borgir Metal
%samples Run up Pops
%samples German Speech
%samples ItCouldBeSweet Pops
%samples OnTheRoofWith Pops
%samples easy game Pops
%samples Tears Infection Pops

Bitrates:
CODE
259222 250962 224500 270110 224109
212206 190894 224404 200702 224012
210626 223963 224651 248472 224040
256744 246869 224559 243848 224081
272211 268745 224645 225813 224060
212126 222561 224771 234717 224101
227229 274100 224802 226008 224228
252353 237478 224475 243034 224060
264467 270433 225449 245293 224317
230309 214030 224517 219325 224051
243315 240742 224427 240749 224024
232944 226612 224709 251829 224129
256994 236299 224619 229343 224034
245990 237097 224533 243966 224121
220848 204723 224825 225298 224235
226596 224930 224500 248784 224110
274433 235266 224408 176326 224012
240666 234456 224458 230376 224032
218750 222924 224796 232772 224208
234844 237687 224774 229799 224189
210745 167946 224583 104856 224026
219320 180893 224796 161194 224211
211124 209905 224500 200196 224110
214539 204183 224648 182829 224167
226791 224161 224673 215760 224121
average:
235016 227514 224641 221256 224112
Go to the top of the page
+Quote Post
 
Start new topic
Replies
Kamedo2
post Jun 4 2013, 19:27
Post #2





Group: Members
Posts: 168
Joined: 16-November 12
From: Kyoto, Japan
Member No.: 104567



I plotted bitrate vs quality distribution of those individual samples(25 samples per encoder).
LAME3.100 V2+ really is a cvbr behavior. No samples go below 210kbps.
LAME 3.98.4 CBR is a total CBR; bitrate barely changes and only quality differs.
Helix has more distribution to the lower bitrates but surprisingly keeps quality at these low bitrates.
And Helix has the most VBR-ish distribution; narrow quality distribution and wide bitrate fluctuation.

Go to the top of the page
+Quote Post

Posts in this topic
- Kamedo2   Personal Listening Test of MP3 encoders at 224kbps   May 16 2013, 17:59
- - Gainless   Thanks for the work, nice to have such a detailed ...   May 16 2013, 19:46
|- - Kamedo2   QUOTE (Gainless @ May 17 2013, 03:46) Tha...   May 17 2013, 19:04
|- - Destroid   Astounding listening test, and quite interesting. ...   May 18 2013, 01:06
- - halb27   It must have been a hard test, thank you very much...   May 16 2013, 20:26
- - halb27   I welcome very much if you could test Lame3.100 al...   May 17 2013, 19:24
- - greynol   It seems the basis here is ~224 kbits. If there i...   May 17 2013, 19:39
- - halb27   ??? I think it's interesting to see how 3.100a...   May 17 2013, 19:45
- - greynol   Neither of those are on-topic. See TOS #5 and #7 i...   May 17 2013, 19:54
- - IgorC   Kamedo2, Thank You Very Much for sharing this te...   May 19 2013, 04:10
|- - Kamedo2   QUOTE (IgorC @ May 19 2013, 12:10) It...   May 19 2013, 07:07
- - Dynamic   Thanks for the effort and ability you put into thi...   May 19 2013, 11:09
|- - Kamedo2   QUOTE (Dynamic @ May 19 2013, 19:09) I wo...   May 21 2013, 15:57
|- - Kohlrabi   QUOTE (Kamedo2 @ May 21 2013, 16:57) QUOT...   May 21 2013, 16:30
|- - Kamedo2   QUOTE (Kohlrabi @ May 22 2013, 00:30) QUO...   May 21 2013, 18:09
- - greynol   BladeEnc != modern encoder   May 21 2013, 18:16
|- - Kamedo2   QUOTE (greynol @ May 22 2013, 02:16) Blad...   May 21 2013, 19:15
- - greynol   I just like stating the obvious.   May 21 2013, 21:51
- - shadowking   Very good and interesting test. It shows that for ...   May 24 2013, 09:09
- - Gecko   Thank you very much for the test! I realize y...   May 24 2013, 18:23
- - greynol   It seems that if you're particularly sensitive...   May 24 2013, 19:01
|- - Gecko   QUOTE (greynol @ May 24 2013, 20:01) It s...   May 24 2013, 22:07
|- - Kamedo2   QUOTE (Gecko @ May 25 2013, 06:07) I...   May 25 2013, 18:03
- - Kamedo2   The ABX criteria was 15+/20(p=0.02). All samples a...   May 24 2013, 20:11
- - greynol   It would help since Guruboolez's data is class...   May 24 2013, 20:45
|- - Dynamic   QUOTE (greynol @ May 24 2013, 19:45) It w...   May 24 2013, 23:49
- - greynol   It was a general comment. Stop consonants in spee...   May 24 2013, 22:52
- - halb27   But as you can see they can be encoded transparent...   May 24 2013, 23:16
- - Kamedo2   I plotted bitrate vs quality distribution of those...   Jun 4 2013, 19:27


Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 18th April 2014 - 12:23