Alternative Multiformat Listening Test @ 128 kbps, at SoundExpert with the same contenders Options

[Ivan Dimkovic]

So, using layers at higher than sweet spot bitrates is inefficient and could be reasonable for compatibility only.

I don't think that is really completely true - "Layers" were introduced basically because of pure performance reasons - to provide "scaling" in quality vs. CPU performance (there was probably also another reason, but it is out of scope of this forum

Since MPEG-1 codecs were standardized in 1989/1990, therefore it was very important also to maintain low performance requirement for at least some coding modes because during that time DSPs being able to run audio coding were very expensive, and MPEG probably needed to keep everybody happy - therefore 3 layers were defined.

Layer III flaws, that prevent it to always outperform others were introduced by design requirement to be somehow "backwards compatible" with Layers I and II (which it is of course not) - without these limitations, MP3 would be very similar in its nature to LC-AAC excluding TNS and window shape switching, and with slightly inferior huffman coding

(AAC is going to be an exclusion from this rule – time and tests will show)

It has been already shown, some 9 years ago - AAC @320 kbps was better than MP2 @640 kbps for 5.1 channel material

This post has been edited by Ivan Dimkovic: Jan 16 2006, 23:46

[halb27]

So, using layers at higher than sweet spot bitrates is inefficient and could be reasonable for compatibility only.

What about vorbis? Vorbis seems to be great from 80 kbps to 256 kbps and more.
What is vorbis' sweet spot?

[Serge Smirnoff]

[What about vorbis? Vorbis seems to be great from 80 kbps to 256 kbps and more.
What is vorbis' sweet spot?

Sweet spot concept is true for MPEG family codecs. It may not for others. Vorbis seems to perform efficiently everywhere.

[Ivan Dimkovic]

"Sweet spot" comes from the relatively straightforward spectral codecs from late 90's.

For example, MP2, MP3, LC-AAC - have "sweet spots" that could be described as "point below which quality starts to drop significantly"

If we use 44.1 khz material,

For MP3 - it was 120-130 kbps, for LC=AAC 90-110 kbps, etc...

BUT

- It is not true that above "sweet spot" codecs should not be used - e.g. LC-AAC outperforms MP2 at any bit rate, etc...

- Furthermore - by adding of various tools (lossy stereo, PNS, SBR, spatial audio coding, etc...) codecs become very sophisticated, and able to operate optimally at much wider range of bits/sample than simple perceptual codecs.

For example, Vorbis might use "lossy stereo" to increase its coding efficiency at lower bit rates - AAC could also use lossy stereo (IS) but it can also use PNS and SBR, etc... So, the "sweet spot" does not have to much sense there.

[Sagittaire]

Just my 2 cents ... ;-)

1) Well ... not really 128 Kbps test but 140 Kbps listening test

CODE

   Sample (Duration in Seconds)   iTunes     LAME     Nero     AoTuV     WMA Professional
   --------------------------------------------------------------------------------------
   BigYellow (24)                 139        141      139      147       138
   bodyheat (25)                  136        146      138      139       143
   Carbonelli (17)                128        121      142      143        92
   Coladito (20)                  145        152      140      152       162
   DontLetMeBeMisunderstood (30)  143        162      137      163       165
   yello (9)                      142        160      152      175       115
   Elizabeth (29)                 128        109      137      112       117
   eric_clapton (25)              141        153      138      146       153
   ReunionBlues (30)              144        155      137      143       163
   LesJoursHeureux (20)           136        146      141      180       119
   macabre (17)                   133        147      142      149       125
   MysteriousTimes (28)           143        148      137      146       153
   ravel (28)                     140        149      137      131       157
   School (19)                    144        153      141      163       150
   Senor (17)                     135        137      142      132       131
   SongForGuy (15)                133        144      144      161       126
   TheDraperyFalls (30)           138        146      137      156       140
   WhiteAmerica (30)              128        113      137      125       113
   --------------------------------------------------------------------------------------
   Average: 22.94                 137.56     143.44   139.89   147.94    136.78

2) Target bitrate is always a big problem for me

1- Vorbis with rating 4.79 and 147 Kbps for bitrate
2- I-tunes with rating 4.74 and 137 Kbps for bitrate

It's 7% difference for bitrate. In comparison RDO advantage for MPEG4 ASP is only 6% in metric test (VHQ0 vs VHQ4 for example). Make visual test with XviD at 930 Kbps and DivX at 1000 Kbps is certainely strange test too ...

I'am really curious to see test with that:

1- Vorbis with rating X and 127 Kbps for bitrate
2- I-tunes with rating 4.74 and 137 Kbps for bitrate

or test with that:

1- Vorbis with rating X and 116 Kbps for bitrate
2- Vorbis with rating Y and 128 Kbps for bitrate
3- Vorbis with rating Z and 140 Kbps for bitrate


3) Ogg Vorbis has a very good and fine quality preset. Why use q4.25 ... and make undersize target. Why not q4.00 or other setting for better target bitrate ... ???


4) The good point is the test (finally) for (old) wma9 pro.

This post has been edited by Sagittaire: Jan 18 2006, 13:34

[Gabriel]

1) Well ... not really 128 Kbps test but 140 Kbps listening test

Did you read the test preparation thread?
The resulting overall on overall music is about 128kbps. Local bitrate of short test samples is not important.

It is the same as if you were doing a video comparison: encode a whole movie using an overall target bitrate, and then compare the difficult parts against different contenders. Would you care if the specific part you are comparing has an higher bitrate with encoder A than with encoder B, as long as the overall size is the same?

[Alex B]

The bitrates were measured with a big amount of complete tracks.

Bitrates estimates and pretest discussion:

http://www.hydrogenaudio.org/forums/index....showtopic=38955

http://www.hydrogenaudio.org/forums/index....showtopic=38723

Actually, the correct name would be something like guruboolez used in his personal test: 130...135 kbps test. I guess 128 kbps is close enough and generally used.

It was not possible to get all test contenders to produce 128 kbps average bitrates because only Vorbis has almost steplessly adjustable quality settings. The Vorbis setting was tweaked to be fair with the others.

edit: typo

This post has been edited by Alex B: Jan 18 2006, 13:49

[Alex B]

BTW, this thread is about the alternative Sound Expert test (http://www.soundexpert.info/coders128.jsp), not the Public Multiformat Listening Test @ 128 kbps (http://www.hydrogenaudio.org/forums/index....showtopic=40607).

[Sagittaire]

The resulting overall on overall music is about 128kbps.

well I can understand that (your experience is really better) ...

Local bitrate of short test samples is not important.

but certainely not that ...

For example easy for me to advantage codec with higher variability bitrate -> more bitrate and more quality for "difficult part" but less bitrate and less quality for "easy part" too. IMO if you compare only these "difficult" part you must use (if possible) same bitrate for these difficult part ...


But If I understand well this test is not exactly average 128 Kbps listening test but simply comparison between +/- comparable 128 Kbps VBR bitrate profil for each codec (q4.25 for Ogg Vorbis, q50 for wma9 pro ... etc etc)


always my 2 cents ...

This post has been edited by Sagittaire: Jan 18 2006, 14:26

[Garf]

The ability of a codec to distribute more bits to hard parts and less bits to easy parts is a significant part of it's performance.

One assumption you make is that 'less bits to easy parts means less quality'. That's just wrong. Why is it an easy part? Because it doesn't need much bits to be good!

A codec that can't make the distinction well will waste bits on easy areas (where they can't improve anything) and not assign enough to hard parts (which will cause a lot of artifacting).

Just making the variability of a codec bigger would just make things worse - because the above still applies *inside* the short samples, too.

This post has been edited by Garf: Jan 18 2006, 14:33

[pepoluan]

Just noticed that on SoundExpert site, the lower bitrates all use oggenc 1.0.

Considering Aoyumi's extensive tuning on lower bitrates, shouldn't the low-bitrate pages be re-tested with aoTuV beta 4 or 4.51?

EDIT: Added link to the SoundExpert site on HA Wiki Listening Tests page. <-- Is it better per test or just one link?

This post has been edited by pepoluan: Jan 19 2006, 18:56

[Serge Smirnoff]

Considering Aoyumi's extensive tuning on lower bitrates, shouldn't the low-bitrate pages be re-tested with aoTuV beta 4 or 4.51?

For sure. Just preparing.

EDIT: Added link to the SoundExpert site on HA Wiki Listening Tests page. <-- Is it better per test or just one link?

Thank you. One link is enough.