That's how I usually compare new LAME versions with reference one (3.90.3).

Create 3 .bat files: "cycle.bat", "proc.bat" and "params.bat" inside LAME folder.

"cycle.bat":
for /L %%i in (1,1,5) do call proc.bat

"proc.bat":
call params.bat master.wav
lame.exe --decode "master.mp3"
del "master.mp3"
del master.wav
ren "master.mp3.wav" master.wav

"params.bat":
lame.exe -b 128 -q 0 --nspsytune --athtype 2 --lowpass 17.5 --ns-bass -6 --nogap -X 3 %1

Of course params.bat may be different.

First I put a short sample into LAME folder and name it "master.wav", then I run "cycle.bat". After all done "master.wav" contains the result. The same operation is performed with another LAME version (.bat files are the same!) and then I compare those two "master.wav"s via simple listening test.
I'm not sure this method is suitable for low-bitrate tests, but with 256kbps it gives very accurate results (especially when increasing number of cycles from 5 to about 100).

Actually its not suitable for any listening test here. Read the faq on abx and lame recommended settings.

Terms Of Service:
http://www.hydrogenaudio.org/forums/index.php?showtopic=3974

Rule 8 of the above:
http://www.hydrogenaudio.org/forums/index....974#entry149481

HA.org FAQ:
http://www.hydrogenaudio.org/forums/index....t=ST&f=5&t=7516

Re-Encoding part of the FAQ:
http://www.hydrogenaudio.org/forums/index....=ST&f=15&t=6938

Recommended LAME Settings:
http://www.hydrogenaudio.org/forums/index....t=ST&f=15&t=203

It seems that you missed the above and didn't made yourself familiar with these forums before posting - should i google for a link to the netiquette?

- Lyx

Multi-generation encoding isn't usefull for making assertions about the quality of the first generation encode. All you are testing is the loss between generations and indeed a fifth generation encode sounds worse than a first generation encode. MP3 was designed to make the first generation encode sound good. The quality of >1 generation encodes is irrelevant.

Hey, maybe you understood me wrong - I meant I'm not comparing original sample with its fifth generation, I'm comparing fifth generation from reference LAME and fifth generation from "tested" LAME version. There's no need to argue that fifth generation sounds worse than first - actually the first generation waveform is deleted during batch processing as you mentioned (I bet you didn't even try).

Gecko's conclusion still partially applies. If i understand you right, then you're intentionally re-encoding to make it easier to notice artifacts. The problem with that approach is that you are not just comparing artifacts but also generation loss - so, if an encoder would create exceptionally good 1st gen encodes, but introduces a high generation-loss when re-encoding, then this encoder would look pretty bad in your "test", although it would be the best one in this fictional example.

Or more simple said:
In your testing method you are not just comparing encoder-quality, but also generation loss...... however, for real-world listening generation-loss is irrelevant and therefore distorts your test-results.

You are trying to make it easier to detect encoder-flaws, but with your method to achieve that you introduce another error-source which distorts the results.

- Lyx

Ok, then please explain, how do you define "generation-loss" term? And why do you differ "artifacts" and "generation-loss"?
I thought that "perfect" encoder (particularly lossy) should encode any source fine, not only "pure" wav but even "mp3-decoded wav" (even white noise, etc.). If artifacts are accumulating, I have only one explanation - encoder is "not perfect". But the same encoder cannot produce different artifacts (assuming all its internal parameters remain unchanged). So, if one encoder gives more artifacts, and another one gives less, but I cannot hear their difference (maybe other people can) I just make the artifacts accumulate to be AUDIBLE. That's all.

I think maybe you should learn a bit more about what lossy audio encoding is my friend. If im right, you are saying that even if you 10th generation encoded an audio file, it should still sound the same, and if not, the encoder has imperfections rite? Well, thats just nonsense. That may be true of a lossless encoder, but not lossy.

No, it can't do that. There's a very good thread in the FAQ that explains why, but basically think of it this way: The audio codec "approximates" the audio signal. In some senses, it blurs or smears it in both the time and frequency domains - but not so much that you can hear the smearing. If it smeared it any more, you'd hear problems. If it didn't smear it quite so much, it would still sound fine but it would take more bits to encode it. So, there's an optimal amount of smearing.

If you take an optimally "smeared" file, and then encode it again - then you're smearing it even more. So, it's obvious that at some point you'll start to hear the problems - it'll smear the signal beyond the region where you can't hear the smearing, into the region where you can. Your 10th generation file won't sound the same as the first, exactly because the codec is doing its job properly.

The other issue is that if A sounds the same as B (but B is a little different - just not enough to notice), and the same applies to B and C, C and D, D and E, etc etc then you'll find that Z sounds very different from A, even though each change was inaudible. So, again, the codec can do its job, but fail your test.



There's some logic in that. It may be false logic though: Maybe both codecs are transparent on the sample you've chosen at the bitrate you've chosen - so there's no point running them 10x over to make them fail. For sensible use (encode once, listen) they're equivalent. However, there may be another audio sample which you haven't tested which causes serious problems for one of the codecs - and there's no reason why the codec which performed worse in your test should be the one that has a problem with a different audio sample. Different codecs trip up on different signals.

Taking a signal which an encoder can encode transparently, and encoding it to several generations will tell you something interesting about what the codec is doing internally. But it's quite useless when you want to judge what it will sound like in normal use, because there will be other signals which the encoder can't encode transparently - it's much more useful to try to find those, count how many of them there are, and how bad each one is when encoded once.

Hope this makes some sense.

Cheers,
David.

And again you didn't understand me. Imagine: during first generation encoder builds the encoded stream truncating (smearing) inaudible parts of the signal to fit it into desired bitrate (or "quality"). That's ok for me. Then you decode the stream into wav. This wav is already missing some information comparing to original. So why the encoder truncates some more INFORMATION again during second pass, if THE SAME stream can be (theoretically) built again without any additional loss?
I was talking about "perfect encoder", theoretically possible - but practically...

Losssy encoders do not remove parts of the signal.
They are adding quantization noise, that is a big difference.

On each pass, the encoder is not "removing signal" but instead adding noise.
This "noisy" signal will be the input of your next encoding stage, where the encoder will try to reproduce as acurately as possible the input signal. That also means reproducing the noise of the previous generation.

Either i misunderstood you, or you do not understand the workings and intention of lossy codecs. As gecko already pointed out in one of the first replies in this thread, lossy codecs are tuned to get the best possible FIRST GENERATION encode.

>1-generation encodes are irrelevant.

So, if i understand you right, then you're searching for the best codec to re-encode lossy files. Simple answer: lossy codecs aren't meant for re-encoding and therefore perform bad in that regard - thats why re-encoding is strongly frowned upon on these forums, unless its inevitable.

- Lyx

And lossy codecs assume non-lossy source!

Triza

We do, but you don't understand us, or how lossy codecs work - but that's OK!



What you're describing isn't an encoder - it's an un-decoder.

If you assume that what's in the .wav file is the output of an mp3 encoder, then it's theoretically possible to figure out what the original mp3 file was, and re-create it. (Though, in practice, even FhG, the creators of mp3, haven't managed to do this properly - and they've tried!). The reason it's virtually impossible is because the frequency and time blocks in an mp3 overlap. While these blocks are uniquely expressed within the mp3 file itself, they're drawn from overlapping data in the original wav file, and used to generate new overlapping data in the output wav file - that's how the smearing occurs, and that's why you can't un-decode an mp3, even if you wanted to.

The reason any useful encoder doesn't work like an "un-decoder" is because, in most cases, the input audio hasn't been encoded. So the encoder has got to look at the source audio, and distort it (smear it, add noise) in order to fit it into the desired bitrate. The perfect un-decoder (if you could make it) would be useless as a normal encoder.

What's more, because of the overlapping time and frequency blocks, the encoder doesn't see the same data second time around. It sees the original signal plus the encoding noise, but it doesn’t only see the encoding noise for the current time/frequency block; the overlapping means that it also sees some noise from adjacent time/frequency blocks - that's the reason why it can't generate the same encoded version second, third, fourth etc time around - because the unique information of that mp3 file has been lost during decoding.

Does that make sense?


Cheers,
David.

Ok, now I see I was wrong. Thanks a lot. Sorry for wasting your time.