Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Autumn 2006 Listening Test (Read 142067 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Autumn 2006 Listening Test

Reply #51
As a quick rehearsal I tried this sample

http://www.hydrogenaudio.org/forums/index....showtopic=47370

using the following encoders & settings:

LAME 3.97b2, -V5 --vbr-new
WMP10 / FhG, 128 kbps CBR default settings (I used the acmenc command line interface)
Helix (hmp3enc.exe 23.7.2005), -X2 -U2 -V60 -HF2 -F17000
iTunes 6.0.5.20, 128 kbps, VBR, Quality: Highest, Joint Stereo, Smart enabled, Filter below 10 Hz enabled
iTunes 6.0.5.20, 128 kbps, CBR, Joint Stereo, Smart enabled, Filter below 10 Hz enabled
Gogo 3.13, -a -b 133 -m j -q 2

The first few distorded rock guitar chords sent all other encoders except LAME straight to the Hell.

I ABXed them all 8/8, but LAME was difficult and the others were quick and easy.

In general, LAME was quite good, not annoying at all, FhG was barely usable, the others were bad.

I don't know if the encoder versions and settings I used were optimal, but if the actual test samples behave even partially like this I think we may have a good chance to find a clear winner this time.

Personally, after this rehearsal, I would use only LAME at this bitrate. Even the other samples would be transparent with all encoders a standard rock quitar sound is too important for me.

Edit: fixed the link

Autumn 2006 Listening Test

Reply #52
As a quick rehearsal I tried this sample

http://www.hydrogenaudio.org/forums/index....showtopic=47313

using the following encoders & settings:

LAME 3.97b2, -V5 --vbr-new
WMP10 / FhG, 128 kbps CBR default settings (I used the acmenc command line interface)
Helix (hmp3enc.exe 23.7.2005), -X2 -U2 -V60 -HF2 -F17000
iTunes 6.0.5.20, 128 kbps, VBR, Quality: Highest, Joint Stereo, Smart enabled, Filter below 10 Hz enabled
iTunes 6.0.5.20, 128 kbps, CBR, Joint Stereo, Smart enabled, Filter below 10 Hz enabled
Gogo 3.13, -a -b 133 -m j -q 2

i always had mismatch in WMP version vs. encoder used version. Can you please enlighten me on which encoder is the WMP10 / FhG ? Is it l3enc based (v1.2.x, slow) or fastenc based (v3.x, very fast)?

thanks, J.M.

Autumn 2006 Listening Test

Reply #53
Fraunhofer IIS mpeg Layer-3 Codec (professional)

It is very fast.


Edit:

V. 3.3.2 (Build 44)


Edit 2:

http://www.hydrogenaudio.org/forums/index....showtopic=47313
The correct sample link is: http://www.hydrogenaudio.org/forums/index....showtopic=47370

Autumn 2006 Listening Test

Reply #54
Fraunhofer IIS mpeg Layer-3 Codec (professional)

It is very fast.


Edit:

V. 3.3.2 (Build 44)

thanks (this is my favourite encoder at 128kbps cbr  )
J.M.

Autumn 2006 Listening Test

Reply #55
Because of the problem samples which are included in the test I suggest for the high anchor to use very high bitrate ABR or CBR. Guess most members like to see Lame 3.97b2 as the high anchor encoder, and as for this I suggest to use ABR 256 or similar.

This should still show up typical mp3 problems while not depending on specific weaknesses. The high anchor should really be a high anchor.
lame3995o -Q1.7 --lowpass 17

Autumn 2006 Listening Test

Reply #56
It makes no sense using an ABR setting as a high anchor... I don't get it.  It's not recommended anywhere else. I know you have a weak spot for it but I think the rest are quite happy with, let's say -V2.
//From the barren lands of the Northsmen

Autumn 2006 Listening Test

Reply #57
ABR is just using brute force, so why not using Lame CBR 320 kbps instead then? Anyway, if a high anchor is used, it should probably be reviewed first.

I would like to remention my interest in speed measurements. I wouldn't mind assisting in doing some benchmarks. (If something is called fast I want to know HOW fast  )
"We cannot win against obsession. They care, we don't. They win."


Autumn 2006 Listening Test

Reply #59
ABR is just using brute force, so why not using Lame CBR 320 kbps instead then? ...

Yes, CBR 320 is a good thing for using as an mp3 high anchor IMO.
lame3995o -Q1.7 --lowpass 17

Autumn 2006 Listening Test

Reply #60
Quote
I am still wondering if we really need a high anchor since we didn't have one in the multiformat listening test at 128 kbps.

High anchor is not for fun, but should be used for a purpose 
If all competitors quality is low, high anchor is needed to make marks to reflect the real situation more closely (w/o it marks could be too high).
IMHO, 128 Kbit for now is rather near to transparency, so high anchor isn't needed on this bitrate...

Autumn 2006 Listening Test

Reply #61
IIRC, Sebastian Mares wanted the high anchor because of the killer samples. It's quite useless for normal samples since LAME is pretty much transparent @ ~128 kbps (geee... it's the second time I say that). The question is then, will the listening test be split in half or what?
//From the barren lands of the Northsmen

Autumn 2006 Listening Test

Reply #62
Yep, that's my problem. Without the killer samples, LAME would already be like a high anchor (it was already transparent and only a little bit worse than the contenders during the multiformat listening test).

Autumn 2006 Listening Test

Reply #63
Ok, point taken.
But, anyway:

1. Are you sure that those samples are really "killer" ? 
(It can be verified)

2. When the competitors are near transparent, may be original should be used to compare, not another codec ?

Autumn 2006 Listening Test

Reply #64
So why not using, as an high anchor, an encoder that has different types of artifacts, like aoTuV beta 4.51 at a 160 kb/s bitrate?

Autumn 2006 Listening Test

Reply #65
The need of a high anchor can be decided later.

The contenders, fair encoding settings and samples are more important now.

Autumn 2006 Listening Test

Reply #66
As for the killer samples I suggest to use
  • harp40_1
  • trumpet
  • herding_calls
  • castanets
(among others).
It's all natural music with the special effect that without knowing in advance that these are problem samples one would not easily beleive how big the problems are for encoders and settings that are usually transparent on most music (with the exception of castanets).
lame3995o -Q1.7 --lowpass 17

Autumn 2006 Listening Test

Reply #67
OK, I decided not to use a high anchor at all. As for the low anchor, I will have a look at l3enc.

Now to the contenders (which shouldn't be more than 5 IMO).

LAME is for sure LAME 3.97b2 -V5 --vbr-new.
FhG - WMP's CBR encoder or MMJB's VBR encoder?
iTunes - CBR or VBR?
Gogo and Helix - will look into it ASAP

Autumn 2006 Listening Test

Reply #68
Hmm... I'd really like to see lame 3.98. It seems like it has made som progress since 3.97b2. But I know, it's still alpha... but perhaps there's a beta coming up pretty soon?
//From the barren lands of the Northsmen

Autumn 2006 Listening Test

Reply #69
...
FhG - WMP's CBR encoder or MMJB's VBR encoder?

I'm sure FHG performs better in cbr (I have yet to see a FHG encoder that doesnt force plain Stereo in VBR). so I would choose the WMP's CBR ACM encoder (also because I think it is very good quality and would like to see how it compares to Lame).

J.M.

Autumn 2006 Listening Test

Reply #70
I just talked / am talking to Roberto who claims that iTunes also performs better in CBR mode. It also seems sensible that Apple focused on AAC and didn't tune the VBR mode of their MP3 encoder.

So, with l3enc, FhG and iTunes, we have 3 encoders in CBR mode. LAME, Helix and Gogo will most likely use VBR. I am now waiting for a CBR vs. VBR discussion to start and make this thread explode.

Autumn 2006 Listening Test

Reply #71
So, with l3enc, FhG and iTunes, we have 3 encoders in CBR mode. LAME, Helix and Gogo will most likely use VBR. I am now waiting for a CBR vs. VBR discussion to start and make this thread explode.
You know what I've been thinking?  Maybe we should test LAME -b 128 with a high-anchor;  I'd be interested in seeing how LAME performs, "on equal terrain" with the other CBR encoders.  And we could get guaranteed 128kbps bitrate, as opposed to the approximate 135 or so we'll get with vbr...


Autumn 2006 Listening Test

Reply #73
I think you should always use VBR if it is possible and produces higher quality files then CBR. Why limit one encoder just because others are worse (can't produce VBR). If you decide to use CBR for some encoders, you should encode first all VBR tracks and then set the CBR bitrate as average of the achieved bitrates. I'm affraid all VBR encoders will go above 128 (I'd guess the average will be around 135 kbps). You might also contact developers to suggest you best settings as Doom9 does before his video codecs comparisons.

As for speed testing - everyone can try it by himself. It is an objective test while listening test is subjective.

Autumn 2006 Listening Test

Reply #74
Quote
As for speed testing - everyone can try it by himself. It is an objective test while listening test is subjective.

You can't do subjective testing for yourself?

I just thought it would be interesting to gather results from different machines, that's all. Maybe not.
"We cannot win against obsession. They care, we don't. They win."