IPB

Welcome Guest ( Log In | Register )

8 Pages V   1 2 3 > »   
Reply to this topicStart new topic
MP3 at 128kbps public listening test, Discussion thread
rjamorim
post Dec 10 2003, 02:17
Post #1


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



Hello.

As I already mentioned here some times, I plan to conduce an MP3 @ 128kbps listening test in January.

The reason for the early discussion thread is that I'll travel for the holidays to a place with difficult internet connection. So, I'd rather have everything sorted out in advance because I can then start the test soon after I return.

So, here is what I am planning:

As usual, the test would run for 11 days, starting on a wednesday.

There would be 5 codecs and an anchor:

-Lame - --alt-preset 128
-FhG Audition Legacy Slow - VBR 60~70
-FhG Audition Current - VBR 60~70
-FhG Audioactive - 128kbps high quality
-iTunes - 128kbps
-Xing 128kbps as anchor

EDIT: Audition Legacy Slow -> Legacy Fast
Radium -> iTunes


Of course, these are open to debate and critics.

The sample suite would be the same as the 64kbps test. Again, if someone has some constructive criticism on it, speak out loud.

If things go as I plan, the test starts Jan 14th 2004 and ends Jan 25th.


Also, it's worth mentioning my future plans: in february, I plan to conduce another AAC@128kbps test, that would compare:

-Nero AAC
-Compaact
-Faac
-QuickTime
-NCTU-Faac (if they sort out their situation until then. Otherwise, I'll find something else)
-Anchor

And, in march, another multiformat test:

-AAC winner
-MP3 winner
-Vorbis GT3b2 or 1.0.1
-Musepack
-WMA standard
-Anchor

Please comment and give your ideas.

Best regards;

Roberto.

This post has been edited by rjamorim: Dec 10 2003, 06:01


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
sthayashi
post Dec 10 2003, 03:29
Post #2





Group: Members
Posts: 494
Joined: 16-April 03
From: Pittsburgh, PA
Member No.: 5997



Roberto, I have a request for your 2nd test. Include Ogg Vorbis in your AAC test. Ogg Vorbis's direct competition in terms of format is AAC. In your last test, Vorbis did not perform quite as well as the best AAC at the time. Since Vorbis has not improved significantly since then (Garf's tunings notwithstanding, since they're not applicable at 128kbps), the results will not be as interesting.

However, seeing Ogg Vorbis compared to other AAC implementations will be more interesting (at least to me). For example, to the end user, OggEnc is more like FAAC than Quicktime or Nero, and a comparison between the two should be most interesting to observe.

This is just my opinion though, and you're welcome to conduce your tests however you like.
Go to the top of the page
+Quote Post
saratoga
post Dec 10 2003, 03:29
Post #3





Group: Members
Posts: 4715
Joined: 2-September 02
Member No.: 3264



QUOTE
-Lame - --alt-preset 128
-FhG Audition Legacy Fast - VBR 60~70
-FhG Audition Current - VBR 60~70
-FhG Audioactive - 128kbps high quality
-Radium - 128kbps
-Xing 128kbps as anchor


Which of this is closest to iTunes? I'd sort of like to know how good its MP3 encoder is since a lot of people still use it.
Go to the top of the page
+Quote Post
elmar3rd
post Dec 10 2003, 04:01
Post #4





Group: Members
Posts: 144
Joined: 5-May 02
Member No.: 1974



Well, I see the theoretical sense of a 128kbps-mp3-test regarding the upcoming tests.
But 1. can we gather enough participants and 2. is 128kbps still a valid "border"?

What will we proove with these tests? Is the 128kbps-range still that important to make three tests?

Let's talk about the targets first.

This post has been edited by elmar3rd: Dec 10 2003, 04:03
Go to the top of the page
+Quote Post
rjamorim
post Dec 10 2003, 04:04
Post #5


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (sthayashi @ Dec 10 2003, 12:29 AM)
Roberto, I have a request for your 2nd test.  Include Ogg Vorbis in your AAC test.  Ogg Vorbis's direct competition in terms of format is AAC.  In your last test, Vorbis did not perform quite as well as the best AAC at the time.  Since Vorbis has not improved significantly since then (Garf's tunings notwithstanding, since they're not applicable at 128kbps), the results will not be as interesting.

However, seeing Ogg Vorbis compared to other AAC implementations will be more interesting (at least to me).  For example, to the end user, OggEnc is more like FAAC than Quicktime or Nero, and a comparison between the two should be most interesting to observe.

This is just my opinion though, and you're welcome to conduce your tests however you like.

Well, I would gladly include vorbis. The problem is, I already have 6 codecs there (IMO, the maximum for a 128kbps test), and even if NCTU dropped, there is already another one in line.

Besides, calling it an AAC test would no longer make sense. (Although I'll probably have to consider another format for anchor anyway)

So, I probably won't include Vorbis in this test. But I would gladly help people interested in conducing such AAC vs. Vorbis test.

QUOTE
Which of this is closest to iTunes? I'd sort of like to know how good its MP3 encoder is since a lot of people still use it.


Veeery good point. I think it's worthing dropping one of the FhG codecs in favour of SoundJam/iTunes.

Any suggestion? Personally, I would drop Radium.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
rjamorim
post Dec 10 2003, 04:18
Post #6


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (elmar3rd @ Dec 10 2003, 01:01 AM)
Well, I see the theoretical sense of a 128kbps-mp3-test regarding the upcoming tests.
But 1. can we gather enough participants

I hope so smile.gif

Keep in mind that MP3@128 is still what the majority of people consider CD quality. So, it is of interest, even if only to see what encoder is best at this mythical bitrate.

QUOTE
and 2.  is 128kbps still a valid "border"?


Anything above this is almost untestable due to near-transparency.
And below this, you start getting what people consider low bitrates (96kbps-)

QUOTE
What will we proove with these tests?


The best encoders at 128kbps.

You can't really extrapolate to higher bitrates, but since anything above that will flood my mailbox with ranked references, 128 will have to do.

Also, regarding AAC, it'll be a good measure to find out how codecs improved since the latest AAC test.

QUOTE
Is the 128kbps-range still that important to make three tests?


Do you suggest another range? smile.gif

QUOTE
Let's talk about the targets first.


OK. My targets are the non-audiophile users (I.E, most people that don't visit to HA, and even some HA visitors) that are happy with MP3 at 128kbps but are interested in broadening their horizons, trying out the new batch of codecs that is being developed.

And my targets are the HA users that use medium bitrates for portable usage or simply because they admittedly don't have golden ears.

Also, one could say my targets are the format developers, that will probably be interested in finding out the flaws people detected, in order to improve their codecs. (Some won't give a damn, like Microsoft. But some probably will - Ahead, Apple, Xiph...)

So there you have my reasoning smile.gif

Regards;

Roberto.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
CosmoKramer
post Dec 10 2003, 04:47
Post #7





Group: Members
Posts: 133
Joined: 3-January 03
Member No.: 4399



What about --alt-preset cbr 128? I for one would like to see that one.
Go to the top of the page
+Quote Post
rjamorim
post Dec 10 2003, 05:09
Post #8


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (CosmoKramer @ Dec 10 2003, 01:47 AM)
What about --alt-preset cbr 128? I for one would like to see that one.

No problem, but that would replace Lame ABR.

Opinions? Maybe make the test all CBR?


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
ff123
post Dec 10 2003, 05:44
Post #9


ABC/HR developer, ff123.net admin


Group: Developer (Donating)
Posts: 1396
Joined: 24-September 01
Member No.: 12



QUOTE (rjamorim @ Dec 9 2003, 05:17 PM)
-Lame - --alt-preset 128
-FhG Audition Legacy Fast - VBR 60~70
-FhG Audition Current - VBR 60~70
-FhG Audioactive - 128kbps high quality
-Radium - 128kbps
-Xing 128kbps as anchor

I believe that Audioactive and Radium are pretty much the same thing.

What's the difference between FhG Audition Legacy Fast and Audition Current? It's my (wild-assed) guess that both are based on FastEnc (which is what FhG VBR has been based on in the past), and that the faster one should be dropped, if it's anything like the difference between the codecs in Cool Edit Pro 2.

I agree that the iTunes mp3 encoder should be tested if at all possible (I don't think it's FhG based).

There's also the FhG slow codec (high quality codec within fastencc.exe)

ff123
Go to the top of the page
+Quote Post
Cey
post Dec 10 2003, 05:46
Post #10





Group: Banned
Posts: 105
Joined: 18-September 03
Member No.: 8889



QUOTE (rjamorim @ Dec 9 2003, 07:18 PM)
OK. My targets are the non-audiophile users (I.E, most people that don't visit to HA, and even some HA visitors) that are happy with MP3 at 128kbps but are interested in broadening their horizons, trying out the new batch of codecs that is being developed.

If you are targeting the 'average' non-HA user, then perhaps you should use some of the codecs *they* typically use. That way you can show them how much better other codecs are. If you use codecs they don't use, then they would have nothing to relate your results to their experience.

Such as MusicMatch (with several settings for the processing level), RealOne, iTunes, and so on.

Those three (or 5, if you count the three levels in MusicMatch) among the more common codecs that average people use.

(I'm assuming most people don't buy any of the mp3 codecs for WMP from Microsoft.... Although it would be interesting to find out the quality of those!)


And for the record, I do fall into the category of HA visitors who don't have "golden ears" except for the occasional song. (Better sound card & better headphones would probably help me a lot...[grin])
Go to the top of the page
+Quote Post
Cey
post Dec 10 2003, 05:50
Post #11





Group: Banned
Posts: 105
Joined: 18-September 03
Member No.: 8889



QUOTE (ff123 @ Dec 9 2003, 08:44 PM)
I agree that the iTunes mp3 encoder should be tested if at all possible (I don't think it's FhG based).

I think I remembering reading in the LAME mailing list archives that a developer for that mp3 encoder said that it was originally based on the original ISO, but that it was very very VERY heavily modified. So much so that it is pretty much a new codec.
Go to the top of the page
+Quote Post
rjamorim
post Dec 10 2003, 05:57
Post #12


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (ff123 @ Dec 10 2003, 02:44 AM)
I believe that Audioactive and Radium are pretty much the same thing.

Good to know. I'll drop Radium then, since latest Audioactive version is newer.

QUOTE
What's the difference between FhG Audition Legacy Fast and Audition Current?  It's my (wild-assed) guess that both are based on FastEnc (which is what FhG VBR has been based on in the past), and that the faster one should be dropped, if it's anything like the difference between the codecs in Cool Edit Pro 2.


I was under the impression that FhG Current was the slow codec. In this case, I'll do current and legacy slow. Thanks for the tip. I'll update my first post now.

QUOTE
I agree that the iTunes mp3 encoder should be tested if at all possible (I don't think it's FhG based).


Yes, I'm quite sure it is SoundJam.

QUOTE
There's also the FhG slow codec (high quality codec within fastencc.exe)


Huh? So fastencc isn't the fast codec?

Anyway, fastencc is buggy and AFAIK this has been fixed on later releases of that library. Where can I find it? Audition? Musicmatch?

QUOTE
I think I remembering reading in the LAME mailing list archives that a developer for that mp3 encoder said that it was originally based on the original ISO, but that it was very very VERY heavily modified. So much so that it is pretty much a new codec.


Yeah, I'm pretty confident it's SoundJam. I'll replace Radium with iTunes then smile.gif\


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
Todesengel
post Dec 10 2003, 06:02
Post #13





Group: Members
Posts: 26
Joined: 3-September 03
Member No.: 8658



Someone tell me what is Anchor??? Is it new AAC encoder or, maybe, new format???
Go to the top of the page
+Quote Post
harashin
post Dec 10 2003, 06:03
Post #14





Group: Members
Posts: 339
Joined: 20-February 02
From: Kyoto, Japan
Member No.: 1362



Ordinary people in Japan still prefers Gogo (2, 3)...


--------------------
Folding@Home Hydrogenaudio.org Team ID# 32639
http://folding.stanford.edu/
Go to the top of the page
+Quote Post
rjamorim
post Dec 10 2003, 06:15
Post #15


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (Cey @ Dec 10 2003, 02:46 AM)
Such as MusicMatch (with several settings for the processing level), RealOne, iTunes, and so on.

Musicmatch = Audition, I would guess (both use latest FhG libs)
Real = Xing :-P
iTunes is already in smile.gif

And I'll mention these associations at the results page.

QUOTE
(I'm assuming most people don't buy any of the mp3 codecs for WMP from Microsoft....  Although it would be interesting to find out the quality of those!)


Well, it's not like there's a huge amount of codecs in the market. You have Lame, FhG and dist10. iTunes uses a proprietary one, what else do we have?

I would guess these WMP codecs are all based on FhG.

QUOTE
And for the record, I do fall into the category of HA visitors who don't have "golden ears" except for the occasional song.  (Better sound card & better headphones would probably help me a lot...[grin])


From the results of my 128kbps multiformat test, transparency in the terms of the ITU (creators of teh ABC/HR test) - higher than 4 points - can easily be reached with modern codecs.

The reason of using 4 as the transparency level is that the ITU takes in consideration that their tests are done in a very controlled environment, equipment is top notch and the listeners are trained and extremely attentive - very far from the everyday situation.

Even considering my tests aren't done with the same level of profissionalism as the ITU ones and we cut some slack - say, .5 point more to reach transparency - one can say MPC reaches it and QuickTime gets pretty close.

(More about the ITU transparency border here and at ITU R.BS 1116-1 document)

Regards;

Roberto.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
rjamorim
post Dec 10 2003, 06:18
Post #16


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (Todesengel @ Dec 10 2003, 03:02 AM)
Someone tell me what is Anchor??? Is it new AAC encoder or, maybe, new format???

Nono, anchor is a compressed stream encoded at low quality, or maybe with a lowpass. It is there to put the results into perspective and to avoid bad codecs getting too low rankings because there was nothing worse.

QUOTE
Ordinary people in Japan still prefers Gogo (2, 3)...


Well, ordinary people in the west still use Xing (AudioCatalyst, RealJukebox, RealOne...)

So, it's hard to please all markets sad.gif


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
Cey
post Dec 10 2003, 06:27
Post #17





Group: Banned
Posts: 105
Joined: 18-September 03
Member No.: 8889



QUOTE (rjamorim @ Dec 9 2003, 09:15 PM)
QUOTE
(I'm assuming most people don't buy any of the mp3 codecs for WMP from Microsoft....  Although it would be interesting to find out the quality of those!)


Well, it's not like there's a huge amount of codecs in the market. You have Lame, FhG and dist10. iTunes uses a proprietary one, what else do we have?

I would guess these WMP codecs are all based on FhG.

I don't know. I've never heard anybody say what they are based on or how good they are.

I even got curious once and checked some warez sites just to see if I could find one and look through it with a hex editor (I never install warez... Way too much chance of virus or trojan) and I couldn't even find a copy of any of the three official mp3 codecs on the warez sites!

So I've kind of gotten the feeling that they are pretty rare.

I did read a comment here in HA from one user who had bought one and was very disappointed with it. But that's about all I've heard.
Go to the top of the page
+Quote Post
westgroveg
post Dec 10 2003, 06:36
Post #18





Group: Members
Posts: 1235
Joined: 5-October 01
Member No.: 220



I think LAME will perform better with ABR @ 128 so you should use --alt-preset 128 to answer, does highest quality FhG out perform highest quality LAME @ 128?


I would like to see Xing with full weapons, Newest encoder, VBR, JS, etc.
Go to the top of the page
+Quote Post
rjamorim
post Dec 10 2003, 06:42
Post #19


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



QUOTE (westgroveg @ Dec 10 2003, 03:36 AM)
I would like to see Xing with full weapons, Newest encoder, VBR, JS, etc.

No problem.

I'll have to play around with Xing settings some first, though. For some strange reason, I never played with them much tongue.gif


(And yes, "newest weapon" would mean installing RealOne)
The things I do for you guys... biggrin.gif


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
Cey
post Dec 10 2003, 06:51
Post #20





Group: Banned
Posts: 105
Joined: 18-September 03
Member No.: 8889



QUOTE (rjamorim @ Dec 9 2003, 08:09 PM)
QUOTE (CosmoKramer @ Dec 10 2003, 01:47 AM)
What about --alt-preset cbr 128? I for one would like to see that one.

No problem, but that would replace Lame ABR.

Opinions? Maybe make the test all CBR?

[grimace]

I realize that many newbies do use CBR (I used to) but I think you need to draw the line somewhere, and that's a good place.

Maybe do one common encoder at CBR and VBR, just to give newbies an example of how much better the VBR is.

For a newbie friendly test like this, I don't see any point to ABR. It should be VBR and one CBR for comparison.
Go to the top of the page
+Quote Post
rjamorim
post Dec 10 2003, 06:52
Post #21


Rarewares admin


Group: Members
Posts: 7515
Joined: 30-September 01
From: Brazil
Member No.: 81



Lame VBR doesn't work well at bitrates smaller than ~160 :/

That's why I chose ABR.


--------------------
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org
Go to the top of the page
+Quote Post
harashin
post Dec 10 2003, 07:17
Post #22





Group: Members
Posts: 339
Joined: 20-February 02
From: Kyoto, Japan
Member No.: 1362



QUOTE (rjamorim @ Dec 10 2003, 02:18 PM)
Well, ordinary people in the west still use Xing (AudioCatalyst, RealJukebox, RealOne...)

So, it's hard to please all markets sad.gif

Right.

And Xing and Blade, we already know its quality in previous testing, are still popular here, too.


--------------------
Folding@Home Hydrogenaudio.org Team ID# 32639
http://folding.stanford.edu/
Go to the top of the page
+Quote Post
Cey
post Dec 10 2003, 07:43
Post #23





Group: Banned
Posts: 105
Joined: 18-September 03
Member No.: 8889



QUOTE (rjamorim @ Dec 9 2003, 09:42 PM)
(And yes, "newest weapon" would mean installing RealOne)
The things I do for you guys... biggrin.gif

The full 320kbps mp3 encoder is still linked on the RealOne page:

http://realone.real.com/

it goes to:

http://realone.real.com/r/url/?10442991173750

Which redirects to:

http://forms.real.com/real/jukebox/320k_re...l?src=r1central

I assume the one that comes with the free RealOne is probably limited below 128k, which I why I mention this...

I assume, of course, that the paid version of RealOne (which probably includes a full mp3 encoder) and the free downloadable version are the same encoder, rather than the possibility of the paid version including a decent encoder.

I mention that because it's still the same old RealJukebox link, which makes me wonder if they've updated it since the days of RJB.
Go to the top of the page
+Quote Post
magic75
post Dec 10 2003, 09:27
Post #24





Group: Members
Posts: 511
Joined: 2-December 02
Member No.: 3959



Some of my opinions:

I don't think all should be set to CBR. I do however vote for a mixture of typical defaults, like Xing & FhG @ 128 CBR, which would be what the typical non-HA user would get when using MusicMatch, Real or WMP. BTW, I am almost 100% sure that WMP uses FhG. Anyway, these "standard" encodings of a typical non-HA person would be interesting to compare with encodings done with more advanced codecs and settings, like lame -preset 128 and the FhG high quality settings, so that we get a comparision of how much better quality you would get by using these more advanced codecs/settings.

Personally I would really like to se Gogo in the test as well as a fast and free alternative (I have very old computer...) There are already three FhG codecs in the test, and personally I could drop one of them in favour of Gogo. But I guess the majority may very well be more interested in FhG so... And I guess fastenc is pretty fast as well...

To sum up: What I would like to see is a test that shows the newbie or non-HA person why dropping software like Musicmatch, Real or WMP can be a good idea.
Go to the top of the page
+Quote Post
magic75
post Dec 10 2003, 09:32
Post #25





Group: Members
Posts: 511
Joined: 2-December 02
Member No.: 3959



QUOTE (Cey @ Dec 9 2003, 09:27 PM)
I even got curious once and checked some warez sites just to see if I could find one and look through it with a hex editor (I never install warez... Way too much chance of virus or trojan) and I couldn't even find a copy of any of the three official mp3 codecs on the warez sites!

So I've kind of gotten the feeling that they are pretty rare.

I am not quite sure I understand you correctly, but if you are talking about the fact that you can't find any Lame, FhG or dist10 on any warez sites, the explanation is quite straightforward. Warez sites contain cracked and illegal software, but both Lame and FhG can be obtained legally for free, so why would anyone publish them on a warez site?
Go to the top of the page
+Quote Post

8 Pages V   1 2 3 > » 
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 17th April 2014 - 00:33