caudec: a multiprocess audio converter for Linux and OS X, Leverages multi-core CPUs with lots of RAM Options

[skamp]

I'd like to introduce a little program I wrote.

Caudec is a BASH script that transcodes audio files from one format (codec) to another. It leverages multi-core CPUs with lots of RAM by copying input files to a tmpfs mount, and running multiple processes concurrently (one per file and per codec).

• Supported input codecs: WAV, FLAC, WavPack, Monkey's Audio, TAK, Apple Lossless.
• Supported output codecs: WAV, FLAC, Flake, WavPack, Monkey's Audio, TAK, lossyWAV, LAME, Ogg Vorbis, Nero AAC.
• Optimized I/O: input files are copied onto a tmpfs mount sequentially, so as to get the best performance out of the underlying medium (e.g. a hard drive). Transcoding however is done concurrently. Example: file 1 gets copied. When that's done, transcoding of file 1 starts. Meanwhile, file 2 gets copied, etc… Very little time is lost reading the files.
• Transcoding to several different codecs at once is possible. In that case, decoding of input files is done only once.
• Metadata is preserved (as much as possible) from one codec to another.
• Uses existing, popular command line encoders/decoders.

Tested under Arch Linux. Download here. I tried my best to make sure that it'll run properly outside of my own environment, but I might have missed a couple of things. Please use the issues tracker to report any bugs. Feedback is most welcome!

[skamp]

I dug up an old version (before 1.0) that didn't copy input files to a tmpfs. Here are the results when transcoding FLACs from my hard drive to Ogg Vorbis, with 8 processes, on a 2 CD album with 37 files (same external encoders):

• old version: 71.41 seconds (15.0 MB/s) (124.3x)
• latest caudec: 58.71 seconds (18.2 MB/s) (151.1x)

That's roughly a 21% speed increase. Maybe not quite as dramatic as one could hope, but substantial nonetheless.

Obviously I dropped filesystem caches before each run.

This post has been edited by skamp: Jun 2 2012, 11:29

[skamp]

That's roughly a 21% speed increase. Maybe not quite as dramatic as one could hope, but substantial nonetheless.

The benefit gets more obvious as CPU time decreases (the HDD becomes more of a bottleneck). Here's a case where the difference becomes "dramatic": encoding WAVs to FLAC (-q 5, FLAC's default compression level).

• old version: 70.63 seconds (22.2 MB/s) (125.7x)
• latest caudec: 38.33 seconds (40.9 MB/s) (231.4x)

That's a 84% speed increase YMMV of course.