Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: A multicore FLAC encoder (Read 134446 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

A multicore FLAC encoder

Reply #25
You may use it together with metaflac with linking to cmd.exe as encoder in the way it is explained here in the Wiki for AAC. I am using this logic with Flacuda with no problem. You even can add the tags with Metaflac.
One downside of cause of these direct cmd-lines is that no album-gain can be set corretly.

http://wiki.hydrogenaudio.org/index.php?title=EAC_and_AAC

maybe this helps:
/c ""C:\path\CUETools.FlaCuda.exe" -8 --verify %s -o %d && "C:\path\metaflac.exe" --set-tag="Album=%g" --set-tag="Artist=%a" --set-tag="Title=%t" --set-tag="Tracknumber=%n" --set-tag="Date=%y" --set-tag="Genre=%m" --set-tag="Comment=FlaCuda 0.6 -8" %d"

good luck
Is troll-adiposity coming from feederism?
With 24bit music you can listen to silence much louder!

A multicore FLAC encoder

Reply #26
interesting project you have going here. i know that as mentioned before... gregory chudov was working on a similar project called "flaCUDA" using the Nvidia CUDA api's... i always find this stuff interesting to watch though. i have a couple of questions for you:

1) the encoding algorithm is standardized, correct? that is, obviously it is going to provide lossless encoding/decoding, and compress to (at least) the same filesize as one encoded with the original FLAC.exe? i know there were some projects going on trying to increase performance/compression on this codec.... which i don't like, standards are made for reasons 

2) you say the current algorithm for encoding is i/o bound, i'm assuming this means the encoder can encode faster than the hard drive can read/supply the data, is this what you mean? unfortunately, this probably means that on faster systems, the speed improvement (vs. the original encoder) will have a tendency to decline the faster the system gets. fortunately, this also will improve with current and future tech like SSD drives... 

3) i would also like the verification function. as far as i know, isn't this a simple matter of computing the MD5 checksum of the raw audio data, encoding, then decompressing and comparing the MD5 values? (the MD5 checksum itself is stored within the FLAC file, obviously you know this) splitting up the threading and stuff could make this complicated if it was calculated in real-time, but if the MD5 was made based on the source WAV, then turn right back around and decode to a temp file, and compare. this would obviously add time to the overall process, plus require 'decoder' code implemented. the original FLAC program seems to do the "Verify OK" instantaneously (or maybe my computer is just really fast)... can you extrapolate on what would be necessary to include this function?  you already answered this in the above post 

4) i would be willing to write or help out with a GUI portion of this, or at least a frontend would be pretty easy to whip up to help simplify for some...


but the batch file posted, wouldn't this simple trick work -> copy the batch file to your desktop or Send To menu, (taking in %1 or argument as a variable) and then encoding that --- that is, basically drag n' drop the WAV file onto the shortcut icon (or send to menu) to encode the file?

Quote
One downside of cause of these direct cmd-lines is that no album-gain can be set correctly.
unless it was an image (single wav) rip, right?


i'm definitely going to test it out though. good work. i will report back my results.


noticed one comsmetic bug  in your command line help, when citing the examples of input/output, you still have reference to *MP3* files... i'm assuming just a copy and paste where you forgot to change? just letting you know 

Quote
Examples:

Encode one WAV file to one MP3:
FPSTREAM readfile in -f audio.wav
      + wavin wav -s in
      + fpmp3enc mp3enc -s wav --vbr-new
      + writefile out -s mp3enc -f audio.mp3

Encode one WAV file to one MP3 and one OGG with tags:
FPSTREAM readfile in -f audio.wav
      + wavin wav -s in
      + tag tag "TITLE=Audio Title" "ARTIST=Audio Artist"
      + fpmp3enc mp3enc -s wav --vbr-new -tag tag
      + vorbisenc oggenc -s wav -tag tag
      + writefile mp3file -s mp3enc -f audio.mp3
      + writefile oggfile -s oggenc -f audio.ogg

A multicore FLAC encoder

Reply #27
Alright! I'm impressed. I have to say that I didn't think the  improvement would be that great on my system, but here are my results (based on about a 53 minute CD):

original FLAC encoder (-8) = 1 minute, 15 seconds.   
and George FP's encoder (-8) = 24 seconds!   

i didn't believe it, so I did a bit-compare on the two files:

Code: [Select]
All tracks decoded fine, no differences found.
    
    Comparing:
    "X:\encode1.flac"
    "X:\encode2.flac"
    No differences in decoded data found.


and then I did a verify on the FLAC files, to make sure they were identical (audio data-wise):

Code: [Select]
Item: "X:\encode1.flac"
    MD5: 66B84449D7A9899065A11FA190C994AC
    CRC32: D199E9A2
    No problems found.
    
    All items decoded successfully.
    
    
    Item: "X:\encode2.flac"
    MD5: 66B84449D7A9899065A11FA190C994AC
    CRC32: D199E9A2
    No problems found.
    
    All items decoded successfully.



finally,  to prove that it wasn't EXACTLY THE SAME THING, i have noted the file  sizes and hashes of both FLAC files... different! both files are unique.

Code: [Select]
File: encode1.flac
    Size: 357MB (374,398,783 bytes)
    CRC-32: fa984f9e
       MD5: 6eb400219c22f1aa1525d13f5f7f1135
     SHA-1: 70aa7e4a65ed635eb78b52ef3d2f1b1b8b435692
    
    File: encode2.flac
    Size: 356MB (374,328,806 bytes)
    CRC-32: eac29087
       MD5: a8d830f2defbe50af368ce60a5a69460
     SHA-1: c356080e0e508d26b8d7b625229af0d2ca2dde7b


so George... you have made a believer of me! empirical evidence, right there for me and everyone else to see.... congrats!
I am about to do a second test on a second system, then I will post those results below.

Results from test #2:

original FLAC encoder (-8 compression) = 1 minute, 07 seconds
and George FP's encoder (-8 compression) = 21 seconds

just for kicks I tried flaCUDA...

flaCUDA encoder (-11 compression) = 58 seconds
flaCUDA encoder (-8 compression) = 16 seconds

Code: [Select]
E:\Temp>CUETools.FlaCuda.exe -11 encode.wav -o encode3.flac
      CUETools.FlaCuda, Copyright (C) 2009 Gregory S. Chudov.
    Filename  : encode.wav
      File Info : 44100kHz; 2 channel; 16 bit; 00:52:53.4930000
      Results   : 53.89x; 373021162 bytes in 00:00:58.8901034 seconds;
      
      E:\Temp>CUETools.FlaCuda.exe -8 encode.wav -o encode4.flac
      CUETools.FlaCuda, Copyright (C) 2009 Gregory S. Chudov.
      Filename  : encode.wav
      File Info : 44100kHz; 2 channel; 16 bit; 00:52:53.4930000
      Results   : 198.27x; 373969399 bytes in 00:00:16.0056281 seconds;


but there is a discrepancy.... flaCUDA's compression scheme goes up to 11, and that is what the 58 second result was based on... when I use compression -8, the encoding time ends up only taking 16 seconds. jeez. I'm not sure how the compression numbers crossover though. all 4 output files are identical to the original and each other (same audio MD5's), the two files produced by FLAC and fpFLAC both have a bitrate of 944, whereas the flaCUDA -11 encode shows a 940 bitrate, and the flaCUDA -8 encode shows a 943 bitrate (in foobar2000) -- all end-product FLAC files are sized between 355MB and 357MB, so negligible difference, size wise. *shrug* i have no idea.... already put waaaay too much thought into this as it is haha

anyways, with differing compression schemes, it's hard to make a solid conclusion.... other than that both of these two are VERY FAST encoders!

oh yeah, and of course, I forgot to include what specs I was using to test:

Intel i7-860 (2.8Ghz stock, 3.47Ghz in turbo mode -- it's a Quad-core with Hyper-Threading, so it sees it as 8 CPU's)
4GB DDR3-1600 RAM
Windows7 x64... I used of course the 64 bit version and the SSE4 instructions as George said in the readme
(and video card is Nvidia GeForce 9600 GSO)


 

A multicore FLAC encoder

Reply #28
Here is a screenshot how to configure EAC to use fpFLAC:



Of course, you can set a compression level other than "-8".

On "EAC -> EAC options -> Tools", set the number of "simultaneous external compressor thread(s)" to 1.


Could you provide a screenshot to configure foobar to use fpFLAC?


A multicore FLAC encoder

Reply #30
Could you provide a screenshot to configure foobar to use fpFLAC?

Today, I've finished the implementation of dedicated tools for specific tasks. The names of the tools are self-explanatory: fpMP3, fpFLAC, fpFLAC2FLAC and fpFLAC2MP3. I will release them tomorrow.

Tomorrow, I will also provide new screenshots for EAC and foobar.

A multicore FLAC encoder

Reply #31
Wow, I'm impressed.
Thanks for all the work effort you put into this project!

A multicore FLAC encoder

Reply #32
Such a nice upgrade to EAC,  thankyou



A multicore FLAC encoder

Reply #35
Hi!

The new version can be downloaded here:

64-Bit version
32-Bit version (not supported)

Just extract the archive and execute "setup.exe".

To use "fpMP3" or "fpFLAC2MP3" you have to download the source code and build the "fpMP3_DLL" project (Blend, SSE2, or SSE4 version). The built DLLs have to be registered with "regsvr32" (admin rights required for that).

Here is the screenshot for EAC:



In this example, I've used compression level 8.

If you like to add some tags, use the "-T" option, e.g.
Code: [Select]
-T "ARTIST=%a" -T "TITLE=%t" ...

EAC FAQ: Which flags can I use in the external compression scheme “User Defined MP3 Encoder”?

I will upload the foobar screenshot in a few minutes.

A multicore FLAC encoder

Reply #36
And here is the screenshot for foobar2000:


You should also set the number of converter threads to 1:

File -> Preferences -> Advanced -> Tools -> Converter -> Thread count : 1

Note: Because of a programming bug the encoder may hang in some (rare) cases. It's under investigation.

A multicore FLAC encoder

Reply #37
To use "fpMP3" or "fpFLAC2MP3" you have to download the source code and build the "fpMP3_DLL" project (Blend, SSE2, or SSE4 version). The built DLLs have to be registered with "regsvr32" (admin rights required for that).

I would like to use fpFLAC2MP3. But I'm not able to build the project. I'm just not into this subject.
Maybe somebody smart who owns all the needed software likes to build the project and upload it here in the forum? 

That would be very appreciated.

A multicore FLAC encoder

Reply #38
64-Bit version
32-Bit version (not supported)
Just extract the archive and execute "setup.exe".


Hi GeorgeFP,

Thanks for your effort and kindly sharing your great work with us, can you change the German installer into English?

Sorry but I'm not even a native English speaker, the installer seems fail to install but I can't read the error message.

Thanks again!
Hong Kong - International Joke Center (after 1997-06-30)

A multicore FLAC encoder

Reply #39
64-Bit version
32-Bit version (not supported)
Just extract the archive and execute "setup.exe".


Hi GeorgeFP,

Thanks for your effort and kindly sharing your great work with us, can you change the German installer into English?

Sorry but I'm not even a native English speaker, the installer seems fail to install but I can't read the error message.

Thanks again!

I will try to change the setup language to English. Can you send me a screenshot of the failed installation? Then, I'll be able to do two things at once.

A multicore FLAC encoder

Reply #40
Considering that EAC only passes one file at a time and has its own internal ability to make use of multiple processes, can someone please explain how you derive a benefit from this program with EAC?

A multicore FLAC encoder

Reply #41
So are you saying that it is pointless to use this version for ripping? That it is really only useful for batch converting?

A multicore FLAC encoder

Reply #42
I know I'm usually a sarcastic cynic, but this time I'm honestly asking for information.  After considering this some more, I'd like to see rationale for using it with foobar2000 as well.

To answer your question though, with my limited understanding of all this, yes it does seem quite pointless.

A multicore FLAC encoder

Reply #43
This encoder is useful, because it can use all the available processor cores when compressing a single file.
This is especially useful when ripping not as separate tracks, but as a single image.
CUETools 2.1.6

A multicore FLAC encoder

Reply #44
So it's a departure from this?
http://www.hydrogenaudio.org/forums/index....showtopic=73790
Quote
The current version is still mostly single-threaded for single-file encoding but scales for multiple-files encoding.

Ok, seeing the initial post here I do see
Quote
The files were encoded one after the other, not in parallel.
...for the benchmark test.  Presumably demonstrating that the program does indeed process each file using more than one core.

So yeah, I would expect only a slight speedup for individual tracks unless there's a really big track that was ripped towards the end of the disc.

A multicore FLAC encoder

Reply #45
Wouldn't the time to compress a single image file be a lot shorter than the time it took to rip it anyway, even using just a single core?


A multicore FLAC encoder

Reply #47
I'm just saying that the time to encode after you finish ripping is fairly short relative to the time spent ripping. It probably takes you longer to replace that CD with the next one.


A multicore FLAC encoder

Reply #49
How hard will it be to add replay gain support? (Please?)  With the addition of Verify support (mentioned above), I'll be all set 

Oops, I missed this post.

Replay gain should not be a problem. I will add it in one of the next versions.
And verify support.