Hello,

Does anybody know if there's version of FLAC that will use both cores of Intel cpu's (Core Duo),for 1.2.1? I searched and found one but for FLAC version 1.1.4, I would prefer 1.2.1.

TIA

Edit: I never found an optimized version to run on multi-core cpu's.

I dunno, but depending on the application you're using it might be simpler to just set up two encoding threads. If you're batch converting stuff in Foobar2K it'll run as many encoding processes simultaneously as you have cores (at least it does two on my own Core2Duo). In EAC you can set in the options how many encoding threads you want it to launch when ripping. I assume other similar software has similar functionality.

I think dBpoweramp Reference will use both cores... Looks like FLAC v1.2.0 though.

I'm not positive on this, but I think I recall reading about it awhile ago.

I'm using EAC with REACT. I just looked through the EAC options and can not find that option. Can you please explain to me where the option is located.


Thank You for all the responses

Found that option in EAC it's in EAC options---->Tools Trying out now.


EDIT: That did not work, FLAC still used one core. Any other suggestions?

Quick answer? No, there's no common compressor that I know of that will use more than one core/cpu for a single encoding job. It's a limitation of how these compressors were written, I suppose.

The best you're going to find is ways to launch more than one instance of the encoder at once. dBpoweramp will use multiple compressor instances; EAC can do it, too; so can foobar2000 (when converting; I assume it can when ripping, as well). But one instance per core is the best you're going to do right now.

Couldn't codec developers do a trick: split the audio in half (in case of two CPU threads), encode both halves and just do some voodoo in the middle of the whole audio file, so the compressed audio won't be different from a file that was processed by a single thread encode?

Ok, how do I setup EAC so it use two instances of X encoder? I thought that's what I tried earlier with EAC and didn't work.

Did it not do it? Or did it take so longer rip one track that it never got a chance to launch more than one encoder at a time? I do secure T&C, so even though I allow up to three encoder instances, the encoding always happens much faster than the ripping (i.e. I/O), so it functionally doesn't matter.

After I made the change in EAC only the one FLAC encoder ran not two. Maybe I'm doing something wrong? Or It might not work since I'm using EAC with REACT?

Can you explain what exactly you do?

Thank You

May be stupid to ask, but did you restart EAC after making the change? It does say right below the option that "it is necessary to restart EAC to change the number of compression threads."

LOL, I overlooked that part That's what happens when you don't sleep. I'm going to try it again in a bit I will post if it works or not.


Thank You for the help.

It's working THANK YOU!!!!

But the question remains if a FLAC encoder could be written in such a manner that it uses multiple cores. It could speed up the encoding process in cases where one has more cores than tracks or use programs which don’t support multiple FLAC instances (for example, simple batch files).

Doesn’t FLAC encode a file after splitting it into pieces first? Could those pieces be encoded distributed on the CPU cores?

yes, but it sounds simpler than it actually is.

Indeed, it isn't so simple, especially coming from single-threaded C code. Writing thread-safe code isn't trivial... when two or more threads are operating on the same stream, you have to deal with all sorts of things. Even flac just spawning itself twice (to encode the stream in two pieces) isn't easy - the threads still need to pass information back and forth.

In something like Java or C#, it is somewhat easier. C has no native threading libraries, and the standard libraries typically aren't thread-safe to begin with.

I'm sure it will get done eventually, but it isn't a weekend coding project by any stretch.

I think the headers for voodoo are in iomanip.h!

j/k

with my C experience & what little of that has anything to do with DSP I would wager to say that would be an incorrect way of going about things. While certainly not impossible, adding multi-threading to any app can exponentially increase the overall complexity of the code making the developer having to choose between trade-offs such as, are we better off running two separate instances at once or are are we better off using multi-threading to complete one instance faster.

In this case you'll find that most developer will probably want to run two separate instances at once as you can run into some pretty weird race conditions, especially when you're using difficult transforms and such for compression.

I'm not a developer, but I play one in real life.

There's going to be more and more threads like this, as multi-core CPUs become more and more common… There are several ways to leverage the power of multi-threading though. Maybe one of the simpler ones could involve only the parts of the encoder that run several algorithms sequentially on the same audio chunk and choose the most efficient one?



Btw, I hate Intel for trademarking "Core Duo": people already seem to think it's a generic term when it's really a specific brand. The correct terms are multi-core, multiprocessing, parallel processing, symmetric multiprocessing, etc…

i'm just a little surprised how most responses in this thread have been of the "it's much too complex" variety, yet as the OP says:



i would think if 1.1.4 was accomplished then to do the same for 1.2.1 couldn't be HARDER could it?

I doubt the OP actually found a modified version of FLAC 1.1.4 with the ability to use multiple cores for a single track. He was probably confused about what the app in question was actually doing.

You are correct. I some how read this tread and for some reason got confused with sse and multi core. oops

It will be nice to have a multi core version

Joining of FLAC or other compressed formats can easily be done losslessly, correct? If that's the case, it would be easy to write a program to split an input WAV into sections for n cores, invoke the encoder for each section in a separate thread, then join the resulting files. The overhead of joining the files may not be worth it, though. Well, I bet it would, it would just be a little slower than having the encoder itself multithreaded.

The way I do it on Linux is that, if I have 10 songs, I open two terminals and start one flac with the first 1-5 songs, and in the other terminal start flac with the songs 6-10.

The trouble with that approach is that if the song sizes aren't distributed evenly, one of the jobs might finish long before the other. That'll leave a CPU idle unnecessarily.

My little set of audio tools uses a simple distributor which starts one flac encoding process per CPU. Then, as waitpid(2) tells it a process finishes, the distributor starts up another. That keeps all CPUs busy until the number of remaining tracks gets smaller than the CPU count.

It's certainly an easier short term method than trying to make the flac encoder itself into a multithreaded application, so long as the "one file per track" method of CD ripping remains popular.

That's currently the best solution indeed. Another way would be to encode different chunks of the same source and then join them together losslessly, but that's not currently possible with any codec, AFAIK, unless Matroska was used as a container, maybe. OptimFROG has planned support for joining .ofr files though.

Personally I think multithreading should get a higher priority than any other gadget-like improvement, given how complete FLAC (and others) already is. Also, multithreading would yield much higher speed improvements than any assembly code would.

Perhaps something like that could be implemented in flac. E.g. when more than one file is specified it uses multi core?

That's implemented easily enough in shell scripts and third-party encoding apps. There's not much point in adding bloat to the reference encoder. What would make sense in the core libraries is a low-level multi-threaded encoding API, but Josh doesn't seem too keen on doing that.

Easiest way is using two instances of flac as foobar2000 does already.
Having a frontend like Flac frontend that will make use of it like the Monkeys Audio frontend will already be a big step.
I bet modifying the existant Flac frontend will be easier than to reingeneer flac itself.

an alternative to instances is to compile flac with ICC10/ICC9 with multi-threading and profiling, that's what I whant to do, it will not be as effective as instances - but then instances have their own drawbacks (you need 2 physical hard drives, and make sure the data is split between them, so that the hard drive activity would not hold back the other thread, worse is that hard drives (and raids too) are not very good at working on more than one file/operation - you could move some of the data to other physical drives, but that would not make it any faster as you will still have to wait until it is transferred, and defeats the purpose)

Reflacer 1.04 allows FLAC 1.2.1 to use both cores of a dual core processor.

http://www.hydrogenaudio.org/forums/index....st&p=551735

I assume you mean Reflacer allows for encoding multiple files concurrently (your sentence gives the impression that Reflacer makes FLAC itself multi-threaded).

You are correct. Sorry for the confusion.

As mentioned above, dBpoweramp Reference will encode flac files on multiple cores. Keeps my quad core very busy... Also flac v1.2.1 is available as of 10/02/2007 in their beta forum.

If FLAC (or any other codec, for that matter) offered an option to losslessly join files together (i.e. blocks of compressed data), third-party tools would be able to encode chunks in parallel and join them into a single file at the end. FLAC already provides the ability to encode parts of PCM audio with the --skip and --until parameters.
Ghido had that functionality in his TODO list for OptimFROG (joining files together) but never got around to implementing it, apparently.

Edit: hmm, I'm repeating myself.

The newly released GCC 4.3.0 has an auto-parallelisation switch, based on OpenMP (you need to build GCC OpenMP support). For those daring enough, give it a go.

-ftree-parallelize-loops=n

Where n is the number of threads. 2*cores would be my suggestion.

Adding -ftree-parallelize-loops=8 to my C[XX]FLAGS didn't improve encoding or decoding speed with FLAC, WavPack nor Monkey's Audio in my tests.

Heh, it won't even compile for me.

Oh well.

Yeah I fixed that by compiling the replay gain code without that flag. Same thing happens with wvgain (WavPack's implementation).

Just a thought, do you have a processor with shared L2 cache? OpenMP apparently is only beneficial with such processors.... I see you used 8, so I guess combined with my suggestion you have four cores, is it a Core rather than a Phenom or a dual X2?

Phenom 9600, shared L3 cache but dedicated L2 cache for each core.

That may be the problem then, I believe that the L3 is just used as a really quick prefetch store. But I may be wrong.