Oh I forgot to say that I want to thank you nevertheless (see my last post), Roberto!

In the next listening we should use more songs with lower than average bitrates.

Big_Berny

Well, they're the ones with the hardware.

Some one could easily record all the clips to their MD recorder via a digital link (from a non-resampling sound card), and then copy them all back into a PC. Then the three versions of each clip (original, software encoded, hardware encoded) could be the subject of a mini listening test. EDIT: like the lame 3.90.3 vs 3.96 test, not like the present one.

I'm not suggesting Roberto should carry out such a test - I'm just saying it would be easy to prove this one way or the other.

As you said, it probably won't happen. Let's face it, MD users aiming for decent results aren't using this setting anyway, because it adds audible artefacts so often.

Cheers,
David.

If have Net MD I will definitely do that test.

People who care about quality probably won't use MD.

The implementation in QT 6.5.1 / iTunes 4.5 is VBR see here

ABR and recognizing silence is not the same as VBR.

If the codec knows its encoding difficult music, it still can't flex the bitrate higher.

How do you know its unable to do that?

Because you can only set bitrates and the produced file has exactly that average bitrate?

It can vary in the song, but it can't vary among songs.

small clarification.
As far as I know there is no CBR aac encoders.
Two systems are used: VBR and ABR.
VBR - is quality based mode - bitrate is adjusted to maintain constant quality (measured by S/N ratio for example).
ABR - bitrate based mode. Average bitrate should be as defined.
CBR can (unsure) be used with aac, but by ITU specs it is not defined.
AAC always use ABR mode. If bitrate fluctuations are no more than defined by standart it is considered as CBR.
So, technically, you can consider iTunes encoding as CBR, if requrements mentioned above are met.
BTW: I remeber Ivan Dimkovic explained this somewhere on this forum, but I can't find it, period...

It seems (from testing, try it) that average bitrate remains the same...

iTunes 4.2 is CBR. Frames are at a constant bitrate. Search for Ivan's explination. I believe he refers to it directly as CBR actually.

Problem is, unfortunately, codec quality is one of the least concerns of hardware manufacturers. They have much more to worry about first: hardware requirements for decoding, ease to implement on hardware, licensing fees, user demand...

Uhh. Found it at the end.

See this thread: http://www.hydrogenaudio.org/forums/index....showtopic=8835&

You make it sound like MPC is tweaked for winning listening tests. I believe it is tweaked to sound consistent trough the entire track. I'm no expert, but it sounds to me like you kind of confuse VBR with ABR.

I'll try to explaing my view on this matter. Then some of the experts can correct it

The average bitrate you see with a certain VBR quality setting, is kind of a coincidence. When alot of encoded material has a certain consistent quality during the whole track, it just happens to average to e.g. 128 kbps. If you take the settings used in this test and encode metal or another demanding genre, you won't see this 128kbps average anymore. The way you explain it, it sounds like it has to sacrifice large parts of the song to be able to boost the bitrate on the hard parts. That's not the idea of VBR (more like ABR but not really that either). When a VBR setting uses only 80 kbps on certain parts, it's because it doesn't need more bits to reach the desired quality. It could have used more if it was needed.

Have you actually heard that the quality in between problem samples is lower? I'm not sure how people ended up providing the samples that they did, for this test. If they listened for problems, then from your reasoning, they wouldn't have provided the problem samples themselves, but rather the low quality parts between problem samples, as that low quality probably would stand out from the rest of the track.

CBR: Variable quality. High quality on easy parts, low quality on hard parts.
ABR: As constant quality as possible within the bitrate limitation. Use bits where they are most useful.
VBR: Constant quality.

So, from my point of view your idea is totaly wrong. I think it would be possible, and a cynical marketing department might be tempted, but I would say that it isn't very likely the case here. These codecs are used every day, right?

IIRC Nvidia or ATI or both, tried to optimize their graphics drivers to win 3Dmark tests and make their cards look better, but I'm not sure if they sacrificed anything by doing it though.

Now, am I totally wrong?

I wonder why the test didn't compare files that were the same size, though I know it would be a royal PITA to repeatedly encode to get that. And then somebody would complain that time to encode should be the equalizing measure ...

No! Sorry if it sounded like this! I think that this is one of the objectivest audio tests!

Sorry you misunderstood me. It's very difficult to explain it for me cause I speak german.
I'll try it again:
I only try to say that MPC perhaps only is so good in this tests because we test only samples with high bitrates! (BTW I know what ABR and VBR is!)
In this test we used quality-settings to reach an average bitrate of 128kbits, right? I know that not the average bitrate of the samples should be 128kbits, but the average bitrate of hole musiccollections with different genres. Right?
If you now look at the bitratetable you'll see that most of the samples encoded with MPC have a bitrate over 128kbits! I don't say that MPC is optimized for the listening test, but its bitrate spreading is very high! And if we only test samples with high bitrates MPC will probably give good results. But there must also be songs with a bitrate under 128kbits because of the average bitrate of approx. 128kbits! Right?
And perhaps this "easy" samples could sound bad because they have a very lower bitrate than the other codecs at this qualitysetting!

Does someone understand this theory? It's only a theory without any prove! But if you look at the testresults you can see that the sample "debussy" with a very low bitrate sounded bad with MPC!

Big_Berny

Big_Berny

you are just saying that musepack has an efficient vbr model... but saying this as if it is wrong or confusing or unfair...

this makes no sense


later

@Big_Berny:
I see what you mean after looking at the data for the Debussy sample. At first I thought your reasoning behind this low bitrate was for the wrong reason, but now I see that it probably wasn't.

I would also like to see a test like the one you suggest. If musepack or any other codec, uses too few bits in places it should have used more, then I suppose it would be useful to investigate it.

Btw: Explaining what I mean in english, isn't my strength either...

Edit: Added the part about the Debussy sample

No, that's not what I want to say. (Very happy that upNorth seems to have understood)!

I want to say that MPC has a very strong VBR-mode with very different bitrates at the same qualitysettings. In this test there were bitrates from 91kbits to 155kbits. You can call it efficient if you want.
But the problem is that: Only two of the samples we tested had a bitrate under 128kbits! And one of them was rated very bad! I just want to say that perhaps the variation of the bitrate is too high but nevertheless MPC will be rated very good in this test overall because we (almost) only tested samples with bitrates over the average (for this qualitysetting).
In the next test we should perhaps also test some samples with very low bitrates because that could be a serious problem of MPC.
If you only test difficult samples on an codec with a high bitrate variation you'll get good ratings if the codec recognizes that the sample is difficult because it will give it a high bitrate. But on the other hand it gives very little bitrate to non-problem-samples so that they perhaps have a too small bitrate. And then MPC will sound worse than the other codecs which have a smaller bitrate variation and will give a higher bitrate on this sample.


Example: Sample Kraftwerk (high bitrate)
Codecs: iTunes MPC Vorbis Lame WMA Atrac3
Bitrates: 128 152 135 141 127 132
Ratings: 4.30 4.78 4.30 3.32 3.11 2.29

Example: Sample Debussy (low bitrate)
Codecs: iTunes MPC Vorbis Lame WMA Atrac3
Bitrates: 128 98 120 108 129 132
Ratings: 4.67 3.53 4.91 3.75 3.95 4.54

You see now what I mean? Perhaps MPC is like "too efficient"!

Big_Berny

To: Big_Berny, upNorth.
Hey, guys, problem with mpc bitrate exists and was discussed on the page 4 of this thread.
See ff123 comments on this issue and how to avoid this in the future, if possible.
EDIT: grammar.

@SirGrey: I read it now, thanx.
But I think you only mentioned that it is a problem of the test and not that it could be a problem of the MPC encoder!

Big_Berny

I understand the point of Big_Berny, never thought about that.

He means that perhaps mpc could be failing on easy tracks, and as the test is featuring mostly hard tracks, it is performing good.

It might also be a matter of track loudness. As an example, the current Lame version is likely to fail in vbr when using a low volume track, as it is not estimating loudness before encoding.

Perhaps this isn't the right time for me to comment on it but I think it would have been superior to test LAME 3.96 ABR instead of VBR, i.e. "--preset 128" or something similar. At lower bitrates ABR is generally considered more effective than VBR. Theoretically VBR should be best but that's not going to be the case in the real world always.

Of course I have not been keeping track of much that is going on so perhaps I missed something.

This was checked before the test. Based on pre-tests, vbr was choosed.


For bitrates around 128kbps, it is now time to change this consideration.

Me too
ff123 mentioned that at page I point to... Never think that way before.
Interesting question to think of.

He-he.
That depends on your point of view and methodology.
Someone in that discussion porposed to use mpc with different setting for every song - to be sure avg bitrate is 128Kbit. So, in this case mpc will have no problems !
But from another perspective, using just one setting is much more consistant...
BTW: I never used mpc for encoding, except for testing.
As I understand, it is tweaked layer 2 encoder, so for bitrates less than ~130Kbit it should produce low quality output... (?)
May be somebody familiar with mpc could explain it's behaviour ?

Oh. Thing to do for version 4 ?

Thank you! That's exactly what I mean! You explained most of my thoughts in one simple sentence...

Big_Berny

I had read the whole thread already, but forgot about that discussion, sorry.

I still have the feeling that the bitrate is a topic because of the fact that this is a 128kbps test. If the motivation for adding more samples with low bitrate is only, or partly, to make the average bitrate look better (closer to 128kbps), then I would say the results of such a test would be less interesting. As ff123 has said already, some problems arise because of a too low bitrate (as seen with Debussy sample), and as I see it, that is a valid reason for using more such low bitrate samples. Add it because it is another type of problem sample, not because it makes the average closer to 128kbps.

Shouldn't the samples also be picked so that all codecs has something to struggle with? Like two samples wma struggles with, two mp3 struggles with and so on? Or maybe that would be too much to ask if as many genres as possible should be covered?

My point of view is that when it comes to bitrate, the only thing that counts is the long time average. Doing some artificial tweaking to make all codec have the same average bitrate on all these short samples, would ruin the value of the test for me. For short I agree with the way things are done already.

Then a question, or more like hearing if my understanding is right: If a codec where perfect, wouldn't it then, at a specific quality setting, recieve the same rating for all samples?

I'm sorry if all of this is old "news" and covered in another thread, then I would be greateful if someone could point me too it.
I'm just trying to see if my understanding and way of thinking is right.

Btw: As it takes me a while to write, alot has happend in the meantime. I see now that Gabriel has picked up on the point Big_Berny made.

Yep. Of course. And here the problem with mpc lies.
So, to summarize, my IMHO, formed by last test and this thread:
1. MPC is a VBR encoder (or at least vbr is the mode it performs much better).
2. Test on many different albums showed (see 128Kbit test discussion thread) that 4.15 setting produces and average bitrate of ~130Kbit and that is ok.
3. The idea the samples for the test are selected is to make encoders job harder.
4. So, average mpc bitrate rises for test samples from 130 to 142Kbit.
5. To maintain avg mpc bitrate about projected (as a result - 136Kbit) additional easy sample(s) (one was selected ocassionally) was chosen.
6. MPC failed on this samples.
So, the question - do mpc have a high score because of it's quality or because of samples selected ?
Correct me, if I wrong somewhere...
BTW: ff123 idea to use equal number of overbitrated and underbitrated samples can correct a situation. May be. That's why I wrote, that error could be in test setup, not in mpc...

The Debussy sample is a great sample for Frank Klemm when he does future MPC tuning. If indeed the bitrate was too low for MPC to give good quality, then that is an issue that needs to be fixed.

From the Vorbis side, I think guruboolez tested the 1.0 encoder on one sample (may have been creaking sample or brahms) at q 0 that produced a low bitrate of 40 kbps or something. It sounded pretty bad. Monty took note of this and made some tweaks to produce 1.0.1 which now gives a more realistic bitrate (somewhere close to nominal 64 kbps) and sounds much better now.

Quickly contribution some more humor to this thread. Someone posted the results on some Windows tech forum, someone else found it funny to reply with this:


http://www.neowin.net/forum/index.php?showtopic=171304

QuantumKnot> I've sent last years two samples: Stockhausen - Stimmung (vocal music) and Liszt - Harmonies Poétiques et Religieuses (piano). Both at very low volume (original, I didn't changed it).

With 1.01 encoder, the problem corrected (or close to be so).


I've played yesterday with mpc 1.14 -q4.15. I've encoded 10 CD of piano music. Average bitrate was < 100 kbps (for a complete work of Erik Satie (5h30, digital recording), bitrate was ~90 kbps. In other words, the average bitrate of the Debussy isn't an accident... or is a very usual one!

Low volume is just a part of the quality problem. I've encoded a piece of contempory music, very quiet too, and without background noise (lossless encoding reached 20%): bitrate was inferior to 80 kbps, and quality was much better, far from the disaster of Debussy.wav. This mean that MPC could achieve good reproduction even with very low bitrate...
Problem for mpc is maybe low volume + background noise? To be verified...

Heh... just as the Slashdot hammering on my site starts to subside, I get Slashdotted again.

In Japan!!!

I wonder if the SNR there is as big as in slashdot.org...



Edit: BTW, I recommend you don't use Babelfish



Your head will explode.

Roberto,

Can you clarify the sample sizes again? What your N means, is that the total people that listened to all songs or the number of people per song? And what is the sample size for the final ANOVA?

Btw, can you make the actual dataset available for other people to analyze?

As a final comment, I think this is too low a sample size.

Jaester

What do you mean by sample sizes?

N is the amount of results I received for that sample minus discarded results.



It's already there.
http://www.rjamorim.com/test/multiformat12...s/comments.html



?

Hi !

I just switched on my iRiver H120 in shuffle mode. I'm currently hearing a Madonna Song, I encoded many years ago with a Fraunhofer encoder in 128 kbps. It sounds very cool, probably due to the usage of intensity stereo. I guess LAME could benefit of intensity stereo usage in the 128-ish bitrate area. I really wonder what ranking it would have got...

I'm currently too lazy to search all postings but in case anyone knows: why has LAME been chosen for mp3 encoding in this test ? It lacks intensity stereo support.

I guess the aoTuV beta 2 encoder would sound lousy in the 128-ish area if it would not make use of intensity stereo (or point-stereo, whatever you want to call it)

bye,
SebastianG

Typically, a sample size of 30 is recommended to be representative of a population.

However, we already know going into these types of open tests that the group of listeners who'll respond are not going to be representative of the general population. So if you accept the proposition that this group of listeners represents the smaller population of "listeners who care (LWC)," then all you need are enough samples to produce a significant result.

You can get significant results this way from just one person, as long as he represents the LWC (e.g. Dibrom tuning lame). However, more listeners are desirable, of course, to average out individual biases (for example, my limited high-frequency ability predisposes me to the sound of *gasp* wma9 standard). As was shown in the Vorbis and mp3 listening pre-tests, even a handful of listeners listening to multiple samples can produce reliable and accurate result.

ff123

FhG does not use intensity stereo at 128 kbit/s. IS is a low bitrate technique, in the same vein as spectral band replication, and isn't meant to produce near transparent encodings.

However, that isn't to say that old FhG encodings can't sound competitive. Roberto's last mp3 test (designed to find the best mp3 encoder at 128 kbit/s, and which lame won) did not include the super slow FhG encoder, which many people with very good high frequency hearing might like best as their mp3 encoder at 128 kbit/s.

ff123

IMHO because:
1. It is most widely used mp3 encoder.
2. It, I think, is on pair (or even better, may be) than old mp3enc31.
I 'm sure Gabriel could answer this question more correctly.
BTW, do you know that mp3enc31 cost 199$ ? And you can not purchase it now.
I think, most people used it illegally
Oh, and I think lame USES joint-stereo for 128Kbit bitrate.
And IS (intensity stereo) corrupts stereo image, thus it is not recommended for such a *high* bitrate as 128Kbit.
EDIT: ff123 was faster
EDIT2:

Forgot to mention it as 3. Sorry...

Lame is the most widely used MP3 encoder? Perhaps around here. I would have to say most people are using variations on royalty paying MP3 encoders, probably Fraunhafer, that are included in various all-in-one music solutions. Because of its so so legal status none of these programs can incorporate Lame, even if many popular applications work with it. What about Music Match, it comes on nearly every PC?

Frankly, I am amazed at all the tiny details that seem so fascinating around here.

IMO, Roberto's test is a blockbuster. Look at the politics. An unofficial build of the open source ogg vorbis encoder blows everything away. Two proprietary solutions, one from the hated MS and the other from Sony, a mega copyright holder, make a weak showing. The hightly compatible and easy to find Lame MP3 shows it has a bunch of life left in it. That is headline news in digital audio compression if there ever was any.

Heh. You are probably right.
Personally, I (nor my friends) 've never use any *box* solutions(except nero, which comes with all my writers), so I simply did not count them. My fault
Musicmatch, ITunes and so on have a huge auditory, really...

I've used an old l3enc which DOES make use of IS, even at 192 kbps.

As for "near transparency": Current Vorbis encoders make use of IS at up to -q5.99. They just don't call it Intensity stereo. Monty seems to have a very different (official) point of view regarding this. He talks about diffuse and point images in the specification. Well, it's basically the same as intensity stereo.

(Maybe seeing/interpreting things from a different angle helps avoiding patent issues, I don't know...)

Anyway, I'm surprised that LAME peforms so well WITHOUT Intensity Stereo in the 128-ish bitrate area - Same for FAAC. (no IS AFAIK)
I guess Vorbis will have strong competitorw when LAME and FAAC start making use of IS for that kind of bitrates (perhaps PNS for FAAC, too).

time will tell.

bye,
Sebi

Believe it or not. I registered l3enc back in 1997 together with the WinPlay3 software for Win3.11



Yeah, I wasn't talking about joint stereo coding. LAME does M/S coding as one possible Joint-Stereo coding technology but not IS.

How you define stereo image ?
It is a widely believed fact that we are unable to perceive phase differences of high frequencies, so IS is an appropriate tool, even for near transparency encodings.

bye,
Sebi

It did. Audioactive (I.E, slowenc with some tunings done in AudioActive)

Thank-you, but that's the reason conducing listening tests is nearly impossible now

Buoy...

From l3enc documentation (taken from good ol' ReallyRareWares):

*****For l3enc 2.0*****




*****For l3enc 2.72*****



What means that, if you got IS at 192kbps, you were messing where you shouldn't



I always understood Vorbis' implementation as a variation on M/S stereo, not IS.

After all, it's very well known that IS completely ruins the stereo image. There were some pre tests for my listening tests that came to that conclusion (look for a post by tigre, IIRC)



Same for MPC and iTunes.

Actually, IS was once available in MPC, and IIRC Andree removed it because it had no place in a codec targeted at high quality



I keep my point that using IS at bitrates above 96kbps is a very bad idea, except on very specific cases.

Regards;

Roberto.

afaik, you are wrong, you are required t pay a license for the right to implement/use an MP3 encoder, so after that, you can use LAME if you want LEGALLY.

Audioactive is a different beast from the very slow codec, which is best represented by mp3enc31, or by using fastencc.exe in -hq mode (this version of the very slow codec has a higher lowpass than mp3enc31).

Audioactive/Opticom/"radium" can be grouped together, but not in the same family as mp3enc31/fastencc.exe -hq.

mp3enc31 is recognizable by low frequency glitches. Ironically, bAdDuDeX (an mp3 connoiseur from long ago), who could hear a 16 kHz lowpass in applaud.wav, loved mp3enc31 despite the glitching and despite its relatively low 14.5 kHz lowpass because it was free from high-frequency ringing.

ff123

I agree completely with both of these statements. They both, in their own way, provide interesting reading. Roberto's tests "always" provide informative, useful and constructive information, while the former simply serves to amuse and utter the occasional "WTF is that about?"

Later.

It seems to be even worse than the one in .org. I found some guys say that Sony rules and this test sucks, or they should adopt Japanese songs for samples. A guy who has his doubts about the test didn't even know that has been a double blind test

The problem with MP3 IS is that it´s not possible to restrict IS usage to certain frequencies - you can only switch stereo modes on a block level, not on a frequency one.

Ahhh!

I don't know what was going wrong....
I just trimmed a frame out of the middle and checked for myself:

Header: FF FB 90 64 = 1111 1111 | 1111 1011 | 1001 0000 | 0110 0100
=>
MPEG 1, Layer 3, 128 kbps, 44100 Hz, Joint-Stereo
mode_extension = 10 => M/S coding: yes and IS coding: no.

I apologize for that. Might have been fuzzy memory or something.
Also, I wasn't that experienced back in 1997.

But I remember that i fed l3enc with out-of-phase high frequency sines that got cancelled ...

edit:

As for Vorbis: Trust me. It's intensity stereo for q<6 (called "point-stereo")
I stick to "IS is very powerful if done right."
Vorbis Stereo Stuff
At q<6 and at a certain frequency Vorbis encoders switch from lossless coupling to point-stereo. In point-stereo the angle value will always br zero. Therefore, the (unscaled) MDCT samples will be the same for both channels after inverse square polar mapping. Intensity is controlled by the floor curves.

In Vorbis, decorrelation and intensity stereo is achieved by square-polar-mapping and channel-interleaved vector quantization.

bye,
Sebastian

well, in mp3 you can signal the use of IS stereo without using IS at all**.
if IS is used, you will have to use it for the whole frequency range beginning from the last scalefactor band down to some arbitrary but fixed scale factor band. you can use L/R or M/S coding for the lower bands.

---
**) I'm actually not sure if the sfb21--where no scalefactor band exists--wouldn't have to be IS coded with 0 degree direction in this case