fpMP3Enc: a multi-core MP3 encoder based upon LAME 3.98.2 Options

[GeorgeFP]

Hello,

I'd like to introduce my project "fpMP3Enc" - a multicore MP3 encoder based on LAME 3.98.2. It is a sample application/case study for my multicore library "Fiber Pool".

The source code can be downloaded here: http://www.fiberpool.de/en/downloads.html

The current version is still mostly single-threaded for single-file encoding but scales for multiple-files encoding.

The multicore tweaks used in this version are:
- parallel/asynchronous conversion/scaling of PCM data to float samples
- parallel/asynchronous computation of replay gain
- I/O ordering (WAV files are read into memory first before MP3 files are written to disk)

With the following test system
- Intel Q9450@2.66GHz (Quad Core)
- 8 GiB RAM
- Windows Vista x64 (Superfetch disabled)
- 61 WAV files (2.99 GiB, about 5 hours play time)

I've got the following results:
LAME 3.98.2 (x32; rarewares): 24.6x
LAME64 3.98 (x64; mp3tech): 20.8x
fpMP3Enc (x64; single): 23.3x
fpMP3Enc (x64; multi): 80.2x

Single-file encoding is slower than the original LAME because I haven't SSE-optimized the code.
The scale factor in multiple-files encoding is 3.3 compared to LAME and 3.4 compared to "fpMP3Enc single", which is quite good for a first version.

The next steps will be to perform the psycho acoustics computation for each frame in parallel, then the MDCT, and so on.

Compile and usage:
- You need Visual Studio 2008 (and obviously Windows) to compile the project.
- SSE4 is enabled by default. To disable: #undef USE_SSE4
- Memory control is disabled by default in this version. This can lead to pagefile swapping if too many files are to be encoded. You can enable memory control by using a different 'MEMORY_MODE' macro (experimental).
- Win32 version: The total WAV file size sum must not exceed 400MiB.
- Win64 version: The total WAV file size sum must not exceed 400GiB.
- Input: WAV 16-bit stereo, 44.1kHz
- Output: MP3 Stereo/Joint-Stereo, 44.1kHz

I'd like to know what you think about this project so contact me if you have any comments or feedback.
George

[GeorgeFP]

Hi again,

meanwhile, I've added ID3 tagging support to fpMP3Enc to make it usable for frontends. If there's something missing, please let me know.

I was also able to make the "vbr-new" algorithm even faster: Running with 3 threads on my quad-core system the speedup was 2.59x compared to LAME and 2.75x to single-threaded fpMP3Enc which gives a 3-core efficiency of 92%.

If you download the file, you will find another sample application inside, called "fpStream", which in my eyes is the next level of multicore stream processing:

While fpMP3Enc is "single/multi file -> single encoding", fpStream is "single/multi file -> single/multi encoding". This means you can take one or more input files and perform one or more encodings on each of them.

For example, you can take one WAV file and encode one CBR MP3 and one VBR MP3 file.

I've also added Vorbis support which is in initial state because you cannot set any parameters. You will need x64 versions of libogg.dll and libvorbis.dll to use it.

fpStream is the framework that is responsible for CPU, memory and disk usage. The actual stream processing is done in the plugins, provided by me or by others. Unfortunately, the code is not well documented.

Next, I will add some parameters for Vorbis encoding and implement a plugin for FLAC encoding and decoding.