IPB

Welcome Guest ( Log In | Register )

9 Pages V  « < 5 6 7 8 9 >  
Reply to this topicStart new topic
Public MP3 Listening Test @ 128 kbps - FINISHED
level
post Nov 27 2008, 03:42
Post #151





Group: Members
Posts: 42
Joined: 18-May 05
Member No.: 22125



@/mnt

Please; Could you perform the test again with the same samples?

but now with this command line:

-V122 -X2 -HF2 -SBT500 -TX0 -C0

Copy and paste EXACTLY as this is written.

Thanks.

This post has been edited by level: Nov 27 2008, 03:45
Go to the top of the page
+Quote Post
Steve999
post Nov 27 2008, 04:00
Post #152





Group: Members
Posts: 88
Joined: 16-December 03
Member No.: 10473



QUOTE (Sebastian Mares @ Nov 26 2008, 13:50) *
Full results are online now. Hopefully I didn't miss anything.


Thanks so much everyone! Sebastian, I sympathize with you, after putting forth the effort and conducting a very nice test, there are so many arguments! I had agonized over whether the switch from Lame 3.97 to Lame 3.98 was worth it, and the answer appears to be yes, quite likely, very much so, for some types of samples. So I will keep using lame 3.98. Thanks to all of the lame developers for your good work.

The helix results look very interesting. I am not much of an expert. I downloaded the hellix encoder from rarewares, but I don't see a guide on how to set the switches. I like to try new things just for fun. I'm odd that way.

SO, if someone could help me, if I want to use helix in the low-to-mid 200 kbps range (which is what I use nowadays since memory and hard disk space are so much cheaper), what would be the standard-issue switch settings? I assume I could use it with EAC (which is what I use)? Is there a link to a guide for setting the switches? I can't find the documentation. For me a decent encoder at this range is fine. I'm interested in the speed and trying something new.

And most of all, thank you SO MUCH to everyone who participated in setting up and conducting the testing.

This post has been edited by Steve999: Nov 27 2008, 04:01
Go to the top of the page
+Quote Post
greynol
post Nov 27 2008, 04:13
Post #153





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (Steve999 @ Nov 26 2008, 19:00) *
I had agonized over whether the switch from Lame 3.97 to Lame 3.98 was worth it, and the answer appears to be yes, quite likely, very much so, for some types of samples.

There really isn't much legitimate to argue about. The problem is that people seem to want to extrapolate the results for a single sample to the entire genre of that sample; and it extends beyond this particular thread. Without conducting tests on additional samples and showing correlation such claims simply aren't credible.


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
Alexxander
post Nov 27 2008, 09:45
Post #154





Group: Members
Posts: 457
Joined: 15-November 04
Member No.: 18143



QUOTE (Neasden @ Nov 27 2008, 01:53) *
Is this basically the end of mp3 by reaching its maximum level of quality being that all codecs are tied?

Define "end of mp3". Judging from this listening test clearly there's room for quality improvement as some encoders perform clearly better on some samples and worse on others.
Go to the top of the page
+Quote Post
Alex B
post Nov 27 2008, 10:29
Post #155





Group: Members
Posts: 1303
Joined: 14-September 05
From: Helsinki, Finland
Member No.: 24472



QUOTE (Alexxander @ Nov 27 2008, 10:45) *
Judging from this listening test clearly there's room for quality improvement as some encoders perform clearly better on some samples and worse on others.

That is one of the reasons I started the sample specific threads. I hope the threads would help the developers (mainly the LAME developers who are active here) to better understand what kind of problems the testers noticed.

I have added new threads for the samples #3 and #4. More is coming.
http://www.hydrogenaudio.org/forums/index.php?showforum=40


--------------------
http://listening-tests.freetzi.com
Go to the top of the page
+Quote Post
Synthetic Soul
post Nov 27 2008, 10:39
Post #156





Group: Super Moderator
Posts: 4887
Joined: 12-August 04
From: Exeter, UK
Member No.: 16217



QUOTE (Neasden @ Nov 27 2008, 00:53) *
Is this basically the end of mp3 by reaching its maximum level of quality being that all codecs are tied?
biggrin.gif You seem to enjoy throwing inflammatory one-liners into this thread. Nice avatar BTW.

QUOTE (Steve999 @ Nov 27 2008, 03:00) *
Thanks so much everyone! Sebastian, I sympathize with you, after putting forth the effort and conducting a very nice test, there are so many arguments!
I suspect that Sebastian is very pleased about the debate that his test has created.


--------------------
I'm on a horse.
Go to the top of the page
+Quote Post
halb27
post Nov 27 2008, 11:02
Post #157





Group: Members
Posts: 2414
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



QUOTE (guruboolez @ Nov 27 2008, 00:50) *
QUOTE (halb27 @ Nov 26 2008, 22:47) *

There is a chance that users can get significant encoder differentiation for his individual needs.

A very small chance. ...

You're right, as you can read in my last post.
BTW I guess I can see what our differences are about: I care about other things than you do. For instance it's not a question to me whether encoder X is better than encoder Y, in the first place because of the difficulty in the meaning of 'better'. I think most members here including you are happy when there would have been a good ordering in overall results with well-separated confidence intervals. While in a formal sense this is meaningful (also to me) this is not the most important thing to me.
If for instance the best encoder in this sense would have had modest scores on samples 3, 4, 7, 10 (those samples with very special importance to me - see my last post) this encoder would not be interesting to me.
In the end it comes to that I like only results on those samples I care about (1-8, 10, 13, with special emphasis on 3, 4, 7, 10), and I'd like to see results which are pretty close to 5 (cause only in this cases there is a strong agreement among all the participants that the outcome of the particular encoder on this sample is good). Sure 'pretty close to 5' is a weak statement, but at 128 kbps I can't expect to give a strong formulation in case I want to get some answer.
And I do want an answer cause the question about saving storage space has arrived at me as I plan to use a Meizu M6 SL DAP (guess I'll get it for Xmas) which unfortunatley has only 8 GB. This test came in quite handy, and I was surprised I could only ABX pretty few encoders on pretty few samples (not mentioning the very low anchor). So my very high quality demands I had so far (and which I could easily have with my 40 GB iRiver H140) are inappropriate. So I take these test results as a starting point for my encoder choice, I have to reconsider those problem samples I care about (I still do, but I will drop some of minor practical significance to me and will be content when an encoder gets at the non-obvious issue level status), and once I have found a good candidate based on this (shouldn't be too hard), I will do intensive listening tests with 'normal' music of my favorite kind.

This post has been edited by halb27: Nov 27 2008, 11:16


--------------------
lame3100m --bCVBR 300
Go to the top of the page
+Quote Post
Sebastian Mares
post Nov 27 2008, 11:04
Post #158





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



QUOTE (Synthetic Soul @ Nov 27 2008, 10:39) *
QUOTE (Steve999 @ Nov 27 2008, 03:00) *
Thanks so much everyone! Sebastian, I sympathize with you, after putting forth the effort and conducting a very nice test, there are so many arguments!
I suspect that Sebastian is very pleased about the debate that his test has created.


Of course. Otherwise, the test would be pretty much pointless and would give me the feeling that nobody cares. smile.gif


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
/mnt
post Nov 27 2008, 11:48
Post #159





Group: Members
Posts: 697
Joined: 22-April 06
Member No.: 29877



QUOTE (level @ Nov 27 2008, 03:42) *
@/mnt

Please; Could you perform the test again with the same samples?

but now with this command line:

-V122 -X2 -HF2 -SBT500 -TX0 -C0

Copy and paste EXACTLY as this is written.

Thanks.


Since am busy today, i can only do a few tests.

Anyway, did a few tracks.

CODE
foo_abx 1.3.3 report
foobar2000 v0.9.6 beta 3
2008/11/27 10:29:05

File A: C:\Temp\Helix Mp3\V122\Metallica - Ride The Lightning\04. Fade To Black.mp3
File B: C:\Rips\Metallica - Ride The Lightning\04. Fade To Black.flac

10:29:05 : Test started.
10:30:15 : 00/01 100.0%
10:30:29 : 01/02 75.0%
10:30:33 : 02/03 50.0%
10:30:39 : 03/04 31.3%
10:30:45 : 04/05 18.8%
10:30:50 : 05/06 10.9%
10:30:55 : 06/07 6.3%
10:31:00 : 07/08 3.5%
10:31:05 : 08/09 2.0%
10:31:10 : 09/10 1.1%
10:31:14 : 10/11 0.6%
10:31:22 : 11/12 0.3%
10:31:49 : 12/13 0.2%
10:31:54 : 13/14 0.1%
10:32:06 : 14/15 0.0%
10:32:08 : Test finished.

----------
Total: 14/15 (0.0%)


Warbling is almost gone at 3:55, but i think i spotted a preecho though after the drum snare. But alot better.

CODE
foo_abx 1.3.3 report
foobar2000 v0.9.6 beta 3
2008/11/27 10:32:57

File A: C:\Rips\Metallica - Ride The Lightning\05. Trapped Under Ice.flac
File B: C:\Temp\Helix Mp3\V122\Metallica - Ride The Lightning\05. Trapped Under Ice.mp3

10:32:57 : Test started.
10:33:17 : 01/01 50.0%
10:33:24 : 02/02 25.0%
10:33:31 : 03/03 12.5%
10:33:50 : 04/04 6.3%
10:34:01 : 05/05 3.1%
10:34:07 : 06/06 1.6%
10:34:22 : 07/07 0.8%
10:34:31 : 08/08 0.4%
10:34:39 : 09/09 0.2%
10:34:47 : 10/10 0.1%
10:34:55 : 11/11 0.0%
10:34:56 : Test finished.

----------
Total: 11/11 (0.0%)


Gutiar warbling from the start is still there, but alot better.

CODE
foo_abx 1.3.3 report
foobar2000 v0.9.6 beta 3
2008/11/27 10:36:14

File A: C:\Rips\Fear Factory - Digimortal\05. Linchpin.flac
File B: C:\Temp\Helix Mp3\V122\Fear Factory - Digimortal\05. Linchpin.mp3

10:36:14 : Test started.
10:36:30 : 01/01 50.0%
10:36:37 : 02/02 25.0%
10:36:48 : 03/03 12.5%
10:36:58 : 04/04 6.3%
10:37:05 : 05/05 3.1%
10:37:12 : 06/06 1.6%
10:37:20 : 07/07 0.8%
10:37:27 : 08/08 0.4%
10:37:32 : 09/09 0.2%
10:37:40 : 10/10 0.1%
10:37:48 : 11/11 0.0%
10:37:56 : 12/12 0.0%
10:37:57 : Test finished.

----------
Total: 12/12 (0.0%)


Hardly any improvement with sample 11, warbling and smearing.


--------------------
"I never thought I'd see this much candy in one mission!"
Go to the top of the page
+Quote Post
sld
post Nov 27 2008, 12:32
Post #160





Group: Members
Posts: 1015
Joined: 4-March 03
From: Singapore
Member No.: 5312



Maybe LAME 3.98.2 is far into the realm of diminishing returns at this stage (for 128 kbps), when taking into account the tradeoff in encoding speed.

Does LAME have to trade its quality to reach Helix's speed? Does Helix have to trade its speed for LAME's quality (consistency and less metal artifacting)?
Go to the top of the page
+Quote Post
DigitalDictator
post Nov 27 2008, 13:44
Post #161





Group: Members
Posts: 313
Joined: 9-August 02
From: SoFo
Member No.: 3002



QUOTE
-V122 -X2 -HF2 -SBT500 -TX0 -C0

where did you get that command line from? I've been asking about this before, we should explore the existing switches to come up with the best possible. It's the best we can do, since tuning Helix doesn't seem like an option.

This post has been edited by DigitalDictator: Nov 27 2008, 13:44
Go to the top of the page
+Quote Post
kwanbis
post Nov 27 2008, 14:13
Post #162





Group: Developer (Donating)
Posts: 2355
Joined: 28-June 02
From: Argentina
Member No.: 2425



Two things to consider is that this where all problem samples and that people where paying attention as much as they can to hear errors.

I think in normal conditions, people would not be so picky when listening.

Like, when i'm listening to music and working, i don't pay not half the attention to the music.

Same if i'm in a meeting with friends, and we have background music.

I think in "reality" , encoders sound better than on tests like this one.


--------------------
MAREO: http://www.webearce.com.ar
Go to the top of the page
+Quote Post
Polar
post Nov 27 2008, 16:59
Post #163





Group: Members
Posts: 266
Joined: 12-February 04
Member No.: 11970



QUOTE (Sebastian Mares @ Nov 26 2008) *
They'll also still be available at
http://listeningtest.vanquickel.be/Sample01.zip
http://listeningtest.vanquickel.be/Sample02.zip
http://listeningtest.vanquickel.be/Sample03.zip
http://listeningtest.vanquickel.be/Sample04.zip
http://listeningtest.vanquickel.be/Sample05.zip
http://listeningtest.vanquickel.be/Sample06.zip
http://listeningtest.vanquickel.be/Sample07.zip
http://listeningtest.vanquickel.be/Sample08.zip
http://listeningtest.vanquickel.be/Sample09.zip
http://listeningtest.vanquickel.be/Sample10.zip
http://listeningtest.vanquickel.be/Sample11.zip
http://listeningtest.vanquickel.be/Sample12.zip
http://listeningtest.vanquickel.be/Sample13.zip
http://listeningtest.vanquickel.be/Sample14.zip

To throw in my own 2 perhaps simplistic cents, all contenders yielding a 4.5ish tie, I personally see little interest in endless debates about which encoder comes out best from whichever point of view, as all have once again proven to have bumped into the boundaries of the practically testable. One of the few, maybe the only, conclusion to draw imho, is that except for killer samples, 128k on any modern codec is no longer interesting to test at this scale.
Go to the top of the page
+Quote Post
Neasden
post Nov 27 2008, 17:53
Post #164





Group: Banned
Posts: 185
Joined: 1-July 08
Member No.: 55148



QUOTE (Synthetic Soul @ Nov 27 2008, 06:39) *
biggrin.gif You seem to enjoy throwing inflammatory one-liners into this thread. Nice avatar BTW.


Did you find my one-liner inflammatory? I don't think it is. I think perhaps it was a naive question that was interpreted as sarcasm (which is quite used by other members, often unremarked) but I really meant in this way: if there were more room for MP3 improvements, because of the known limitations bounded to the format.
Go to the top of the page
+Quote Post
TechVsLife
post Nov 27 2008, 18:47
Post #165





Group: Members
Posts: 195
Joined: 29-May 07
Member No.: 43837



Does this latest test suggest that the recommended settings for LAME be changed (or any other part of the ha wiki)? And one specific point: is -v2 still the recommended minimum setting for transparency for most listeners in good listening environments etc., or should that be lower now?



Thanks for any comments (and thanks to Sebastian Mares for conducting the test).
Go to the top of the page
+Quote Post
gerwen
post Nov 27 2008, 19:04
Post #166





Group: Members
Posts: 64
Joined: 18-September 08
From: Sparta, Ontario
Member No.: 58419



QUOTE (kwanbis @ Nov 27 2008, 08:13) *
Two things to consider is that this where all problem samples and that people where paying attention as much as they can to hear errors.

I think in normal conditions, people would not be so picky when listening.

Like, when i'm listening to music and working, i don't pay not half the attention to the music.

Same if i'm in a meeting with friends, and we have background music.

I think in "reality" , encoders sound better than on tests like this one.


True enough, but do you really want your encoder to let you down when you do decide to pay specific attention?

Imo, pick your encoder and bitrate on very critical listening, and tough samples. At least then you can be reasonably confident that you won't hear artifacts when you do crank it up.
Go to the top of the page
+Quote Post
Synthetic Soul
post Nov 27 2008, 19:38
Post #167





Group: Super Moderator
Posts: 4887
Joined: 12-August 04
From: Exeter, UK
Member No.: 16217



QUOTE (Neasden @ Nov 27 2008, 16:53) *
QUOTE (Synthetic Soul @ Nov 27 2008, 06:39) *
biggrin.gif You seem to enjoy throwing inflammatory one-liners into this thread. Nice avatar BTW.
Did you find my one-liner inflammatory? I don't think it is. I think perhaps it was a naive question that was interpreted as sarcasm (which is quite used by other members, often unremarked) but I really meant in this way: if there were more room for MP3 improvements, because of the known limitations bounded to the format.
Yes, I do see the suggestion that this is the "end of mp3" as inflammatory. If you say that is not your intention then so be it. FYI, I have compiled a few more of your statements from this thread as background for my remark, as way of explanation:

QUOTE (Neasden @ Nov 24 2008, 22:00) *
Does that make Helix the new recommended MP3 encoder, or has it to be LAME because it's open source?
Edit: Both are open source.
QUOTE (Neasden @ Nov 24 2008, 22:25) *
They are all techincally tied, but Helix outperformed all of them. Also, the encoding speed compared to LAME is absurd faster. Could these two arguments qualify Helix for the new recommended MP3 encoder? (LAME being the second recommended)
QUOTE (Neasden @ Nov 26 2008, 17:50) *
* Facts
...
Helix performed a bit better than LAME in this test.
QUOTE (Neasden @ Nov 26 2008, 18:51) *
I have nothing to add, like I am not saying Helix is BETTER than LAME, I didn't say that... but the numbers are there, and I am gonna stick with the numbers. You can't tell against the numbers.
QUOTE (Neasden @ Nov 27 2008, 00:53) *
Is this basically the end of mp3 by reaching its maximum level of quality being that all codecs are tied?
I'll say no more on the subject now.


--------------------
I'm on a horse.
Go to the top of the page
+Quote Post
Destroid
post Nov 27 2008, 19:41
Post #168





Group: Members
Posts: 544
Joined: 4-June 02
Member No.: 2220



Interesting results. It appears 128kbps really is enough for general use MP3 encoding after all. I actually stick to lossless, but I was satisfied with CBR 128kbps Helix on the old P3-733 system because it's advantageous speed and acceptable quality.

If anything, these results should help rid those ridiculous threads about customized encoding setting (you know, the ones with "-V 0 -m s -b320 -B320" et cetera).


--------------------
"Something bothering you, Mister Spock?"
Go to the top of the page
+Quote Post
greynol
post Nov 27 2008, 19:50
Post #169





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



QUOTE (TechVsLife @ Nov 27 2008, 09:47) *
Does this latest test suggest that the recommended settings for LAME be changed (or any other part of the ha wiki)?
No.

QUOTE (TechVsLife @ Nov 27 2008, 09:47) *
is -v2 still the recommended minimum setting for transparency for most listeners in good listening environments etc., or should that be lower now?
-V2 is not recommended as the minimum setting for transparency. If you're talking about the wiki which is intended only to be a general guideline the minimum setting is -V3. This is not to say that -V4 might also deliver transparency to a large number of people over a large number of tracks, let alone -V5. Consider that -V5.7 was used in this test for Lame 3.98


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
TechVsLife
post Nov 27 2008, 20:10
Post #170





Group: Members
Posts: 195
Joined: 29-May 07
Member No.: 43837



Thanks. I must have been mistakenly thinking of -v2 as the minimum standard for transparency from some even older recommendation. I'll consider -v3 as a conservative and safe min. standard level for transparency for most (the vast majority?) of listeners under even very good conditions (i.e. quiet, but not abx testing). I understand this is only a very general guideline, and listeners vary, etc.

The reason I mention this test as possibly changing the wiki, is that if the perceived quality of the mp3 encoders at 128kps is now higher than it used to be over a couple of years ago (e.g. getting closer to "5"), then that might shift the recommendations a bit. The graph on the wiki of LAME -v settings against both resulting quality and filesize is very useful; I'm assuming that's still consistent with the latest results.

Scroll down a little from here for the graph:
http://wiki.hydrogenaudio.org/index.php?ti...ate.29_settings


QUOTE (greynol @ Nov 27 2008, 13:50) *
-V2 is not recommended as the minimum setting for transparency. If you're talking about the wiki which is intended only to be a general guideline the minimum setting is -V3. This is not to say that -V4 might also deliver transparency to a large number of people over a large number of tracks, let alone -V5. Consider that -V5.7 was used in this test for Lame 3.98


This post has been edited by TechVsLife: Nov 27 2008, 20:17
Go to the top of the page
+Quote Post
Neasden
post Nov 27 2008, 23:00
Post #171





Group: Banned
Posts: 185
Joined: 1-July 08
Member No.: 55148



QUOTE
Yes, I do see the suggestion that this is the "end of mp3" as inflammatory.


Sorry if I offended anyone with those one-liners.
Exiting the thread.
Go to the top of the page
+Quote Post
jimmy69
post Nov 28 2008, 02:48
Post #172





Group: Members
Posts: 82
Joined: 25-May 06
Member No.: 31129



Has anyone else noticed that the itunes encoder has actually performed very well from looking at the results. I mean all this time members f this forum have strongly recommended against using the mp3 encoder in itunes, only to find that it performed just as well as LAME 3.97, which was considered to be a very good encoder for a long time. On top of that Apple have now updated itunes with the fix version of there mp3 encoder. From these results I think it should be ok to say that people who use itunes and would prefer the ease of use in using the built in mp3 encoder, go for it.
Go to the top of the page
+Quote Post
halb27
post Nov 28 2008, 09:26
Post #173





Group: Members
Posts: 2414
Joined: 9-October 05
From: Dormagen, Germany
Member No.: 25015



QUOTE (jimmy69 @ Nov 28 2008, 03:48) *
... I mean all this time members f this forum have strongly recommended against using the mp3 encoder in itunes ...

Yes, I've never liked tendencies like these on HA (I've never seen strong reason for this), and I like the outcome of this test so everybody can pick his favorite also for non-quality related reasons.


--------------------
lame3100m --bCVBR 300
Go to the top of the page
+Quote Post
kwanbis
post Nov 28 2008, 12:42
Post #174





Group: Developer (Donating)
Posts: 2355
Joined: 28-June 02
From: Argentina
Member No.: 2425



QUOTE (jimmy69 @ Nov 28 2008, 01:48) *
I mean all this time members f this forum have strongly recommended against using the mp3 encoder in itunes,

Probably based on the last mp3 listening test where it SUCKED.

It is really good that now it does not.


--------------------
MAREO: http://www.webearce.com.ar
Go to the top of the page
+Quote Post
lvqcl
post Nov 28 2008, 17:14
Post #175





Group: Developer
Posts: 3221
Joined: 2-December 07
Member No.: 49183



QUOTE (kwanbis @ Nov 28 2008, 14:42) *
QUOTE (jimmy69 @ Nov 28 2008, 01:48) *
I mean all this time members f this forum have strongly recommended against using the mp3 encoder in itunes,

Probably based on the last mp3 listening test where it SUCKED.

It is really good that now it does not.

It is really good that Sebastian have dualcore computer, and Alex B doesn't wink.gif
Go to the top of the page
+Quote Post

9 Pages V  « < 5 6 7 8 9 >
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 24th April 2014 - 19:15