Hi All,

I am playing around with the LAME source, and am looking for a good way to test if I have broken anything along the way: a sort of basic regression. So far I have been encoding a set of wav files with default options and diffing the resulting mp3 with an mp3 produced from a clean version of LAME. If they differ, I try running mp3guessenc on each file and looking for differences in that output. I found an extension to foobar2000 that diffs mp3 files, but it is not very robust, and as far as I can tell does the same thing as diffing mp3guessenc outputs. Additionally, I could manually look for differences using statistics from EncSpot.

My question is: what do the pros do to test their changes?

I would prefer that my changes result in bit-exact results, but I've noticed that even small changes such as moving a chunk of code into a separate function sometimes results in some bit differences. Given such volatility, I wouldn't mind having "good-enough" results, but I'm not sure what kind of tolerance to allow for, and how to determine the results are still correct. In my wildest dreams this would all be automated via a simple shell/perl script (and cross-platform if I should be so greedy...). In essence I want an mp3diff command line utility.

I think the mp3 frame analyzer included with LAME is probably promising for narrowing down differences, but I have had a heck of a (unsuccessful) time trying to compile it on Windows.

Thanks for your input!

So, apparently, there is a python script included in the source that exercises a whole bunch of different encoding options, given a wav input. Presumably this is what one would use to test their changes.

I say presumably, because there is a file labeled HACKING in the lame root that speaks of such a script, but for the life of me I can't find this test script anywhere in the source. According to this file it should be in the test directory.... but I don't have a test directory in the lame-3.97 source that I have.

Does anyone know where I can find this script?

Isn't this what you mean?
It appears in the sourceforge tarball / cvs.