FLACCL: CUDA-enabled FLAC encoder by Gregory S. Chudov (prev. FlaCuda), Formerly "lossless codecs and CUDA" Options

[gib]

With my recent purchase of a 9000 series nVidia graphics card, I started thinking, has anyone investigated if nVidia's CUDA could be useful for lossless compression? I'm not even remotely close to being a programmer, so I haven't a clue how the code works, but it seems like CUDA is valuable for coding/decoding. I know nVidia is already holding a contest to speed up LAME (which ends in about 2 weeks), so perhaps it could be used to speed up lossless compressors? The fastest modes of several codecs are already blazing fast, approaching the limits of hard drives, but perhaps the high-compression modes could be sped-up through CUDA. Maybe, if the speed-up is enough, developers could even implement more ways to gain compression while still maintaining good encoding rates. It would be pretty cool if compression levels like La's best could be done at 50x or something.

Anyway, my curiosity is large, so just thought I'd ask. :)

[sauvage78]

IMHO audio or video encoding will not help nvidia survive long because if the only purpose of buying GPU become to accelerating encoding then you'd better buy a higher-end CPU, being written in a lower process than GPU, CPU will always have the advantage in brute encoding force vs. power consumption & heat.

As for a multithreaded flac encoder, AFAIK there is none, I think I recall I read about some very experimental proof-of-concept code on some mailing list, but nothing serious.

Maybe we should start a donation to buy a quad core for Josh, it cannot be more useless than buying a PC for Klemm afterall

[GeSomeone]

As for a multithreaded flac encoder, AFAIK there is none, ..

The simpelest way to use multi threading for any encoder is to run multiple encoders simultaneously (foobar2000 can do that). The number of usable threads depends on when the hard disk becomes the bottleneck.

[odyssey]

As for a multithreaded flac encoder, AFAIK there is none, ..

The simpelest way to use multi threading for any encoder is to run multiple encoders simultaneously (foobar2000 can do that). The number of usable threads depends on when the hard disk becomes the bottleneck.

Now we just need a way to simultanously run CUDA and CPU encoders

[alvaro84]

Now we just need a way to simultanously run CUDA and CPU encoders

Sounds fun, though I'm afraid we'd bump into a strong bottleneck because of disk head positioning Even converting with 2 threads one HDD seeks like crazy - but it's still a lot faster than 1 thread.
NCQ in AHCI mode should help a lot with more threads, but it didn't when I tested it a while ago. Physically different source/target drives can alleviate this bottleneck quite a bit.
Fast SSDs are worth a try too
This CUDA encoder can be a different solution, in case of one instance it's faster than the reference encoder running on one core of my CPU (converting one file at a time is the least disk-bottlenecked way to do it).
A natively multithreaded CPU-based encoder (working on segments of one single track) is another option.

[skamp]

Sounds fun, though I'm afraid we'd bump into a strong bottleneck because of disk head positioning Even converting with 2 threads one HDD seeks like crazy - but it's still a lot faster than 1 thread. […] A natively multithreaded CPU-based encoder (working on segments of one single track) is another option.

Ideally you would run multiple instances of a single-threaded encoder (one track per CPU core) and one instance of the CUDA encoder per GPU at the same time - it's just a matter of making sure that all instances are kept busy.

When the number of remaining tracks gets lower than the number of available cores, you prioritize the GPU instance (since it's faster than a single-threaded encoder on a single CPU core), but also run (if available) a multi-threaded encoder; one MT encoder over two cores is likely to be slower than two instances of a ST encoder over the same number of cores (see the Lancer builds of the Ogg Vorbis encoder). In other words, an MT encoder is particularly useful for keeping CPU cores busy when the workload dries up.

In short, the priorities go like this (if you have a multi-core CPU, that is):
ST * n CPU cores > GPU > MT

As for the I/O bottlenecks, that's when a large enough RAMdisk comes in very handy. Even just 1GiB is often enough for encoding a whole album (WAV + FLAC or FLAC + Ogg Vorbis or whatever on the RAMdisk).

I already use all available CPU cores when I encode my rips to FLAC or any other codec (one track per core); what I could really use, even before a MT FLAC encoder comes up, is a simple, command-line, multi-threaded Replay Gain utility. As I've said in the past, computing RG values on an album now takes longer than encoding it in the first place (because the former uses only one core while the latter uses all 4 cores on my quadcore CPU).

[alvaro84]

As for the I/O bottlenecks, that's when a large enough RAMdisk comes in very handy. Even just 1GiB is often enough for encoding a whole album (WAV + FLAC or FLAC + Ogg Vorbis or whatever on the RAMdisk).

You're absolutely right, I don't know how I could forget about RAMdisks. I used them all the time when 8MiB felt plenty of RAM, but somehow I never thought about them since we have multiple GiBs at our disposal... talk about contradictions...