A multicore FLAC encoder Options

[Wombat]

You may use it together with metaflac with linking to cmd.exe as encoder in the way it is explained here in the Wiki for AAC. I am using this logic with Flacuda with no problem. You even can add the tags with Metaflac.
One downside of cause of these direct cmd-lines is that no album-gain can be set corretly.

http://wiki.hydrogenaudio.org/index.php?title=EAC_and_AAC

maybe this helps:
/c ""C:\path\CUETools.FlaCuda.exe" -8 --verify %s -o %d && "C:\path\metaflac.exe" --set-tag="Album=%g" --set-tag="Artist=%a" --set-tag="Title=%t" --set-tag="Tracknumber=%n" --set-tag="Date=%y" --set-tag="Genre=%m" --set-tag="Comment=FlaCuda 0.6 -8" %d"

good luck

[Chinch]

interesting project you have going here. i know that as mentioned before... gregory chudov was working on a similar project called "flaCUDA" using the Nvidia CUDA api's... i always find this stuff interesting to watch though. i have a couple of questions for you:

1) the encoding algorithm is standardized, correct? that is, obviously it is going to provide lossless encoding/decoding, and compress to (at least) the same filesize as one encoded with the original FLAC.exe? i know there were some projects going on trying to increase performance/compression on this codec.... which i don't like, standards are made for reasons

2) you say the current algorithm for encoding is i/o bound, i'm assuming this means the encoder can encode faster than the hard drive can read/supply the data, is this what you mean? unfortunately, this probably means that on faster systems, the speed improvement (vs. the original encoder) will have a tendency to decline the faster the system gets. fortunately, this also will improve with current and future tech like SSD drives...

3) i would also like the verification function. as far as i know, isn't this a simple matter of computing the MD5 checksum of the raw audio data, encoding, then decompressing and comparing the MD5 values? (the MD5 checksum itself is stored within the FLAC file, obviously you know this) splitting up the threading and stuff could make this complicated if it was calculated in real-time, but if the MD5 was made based on the source WAV, then turn right back around and decode to a temp file, and compare. this would obviously add time to the overall process, plus require 'decoder' code implemented. the original FLAC program seems to do the "Verify OK" instantaneously (or maybe my computer is just really fast)... can you extrapolate on what would be necessary to include this function? you already answered this in the above post

4) i would be willing to write or help out with a GUI portion of this, or at least a frontend would be pretty easy to whip up to help simplify for some...


but the batch file posted, wouldn't this simple trick work -> copy the batch file to your desktop or Send To menu, (taking in %1 or argument as a variable) and then encoding that --- that is, basically drag n' drop the WAV file onto the shortcut icon (or send to menu) to encode the file?

One downside of cause of these direct cmd-lines is that no album-gain can be set correctly.

unless it was an image (single wav) rip, right?


i'm definitely going to test it out though. good work. i will report back my results.


noticed one comsmetic bug in your command line help, when citing the examples of input/output, you still have reference to *MP3* files... i'm assuming just a copy and paste where you forgot to change? just letting you know

Examples:

Encode one WAV file to one MP3:
FPSTREAM readfile in -f audio.wav
+ wavin wav -s in
+ fpmp3enc mp3enc -s wav --vbr-new
+ writefile out -s mp3enc -f audio.mp3

Encode one WAV file to one MP3 and one OGG with tags:
FPSTREAM readfile in -f audio.wav
+ wavin wav -s in
+ tag tag "TITLE=Audio Title" "ARTIST=Audio Artist"
+ fpmp3enc mp3enc -s wav --vbr-new -tag tag
+ vorbisenc oggenc -s wav -tag tag
+ writefile mp3file -s mp3enc -f audio.mp3
+ writefile oggfile -s oggenc -f audio.ogg

This post has been edited by Chinch: Dec 2 2009, 06:35

[Chinch]

Alright! I'm impressed. I have to say that I didn't think the improvement would be that great on my system, but here are my results (based on about a 53 minute CD):

original FLAC encoder (-8) = 1 minute, 15 seconds.
and George FP's encoder (-8) = 24 seconds!

i didn't believe it, so I did a bit-compare on the two files:

CODE

All tracks decoded fine, no differences found.
    
    Comparing:
    "X:\encode1.flac"
    "X:\encode2.flac"
    No differences in decoded data found.

and then I did a verify on the FLAC files, to make sure they were identical (audio data-wise):

CODE

Item: "X:\encode1.flac"
    MD5: 66B84449D7A9899065A11FA190C994AC
    CRC32: D199E9A2
    No problems found.
    
    All items decoded successfully.
    
    
    Item: "X:\encode2.flac"
    MD5: 66B84449D7A9899065A11FA190C994AC
    CRC32: D199E9A2
    No problems found.
    
    All items decoded successfully.

finally, to prove that it wasn't EXACTLY THE SAME THING, i have noted the file sizes and hashes of both FLAC files... different! both files are unique.

CODE

File: encode1.flac
    Size: 357MB (374,398,783 bytes)
    CRC-32: fa984f9e
       MD5: 6eb400219c22f1aa1525d13f5f7f1135
     SHA-1: 70aa7e4a65ed635eb78b52ef3d2f1b1b8b435692
    
    File: encode2.flac
    Size: 356MB (374,328,806 bytes)
    CRC-32: eac29087
       MD5: a8d830f2defbe50af368ce60a5a69460
     SHA-1: c356080e0e508d26b8d7b625229af0d2ca2dde7b

so George... you have made a believer of me! empirical evidence, right there for me and everyone else to see.... congrats!
I am about to do a second test on a second system, then I will post those results below.

Results from test #2:

original FLAC encoder (-8 compression) = 1 minute, 07 seconds
and George FP's encoder (-8 compression) = 21 seconds

just for kicks I tried flaCUDA...

flaCUDA encoder (-11 compression) = 58 seconds
flaCUDA encoder (-8 compression) = 16 seconds

CODE

E:\Temp>CUETools.FlaCuda.exe -11 encode.wav -o encode3.flac
      CUETools.FlaCuda, Copyright (C) 2009 Gregory S. Chudov.
    Filename  : encode.wav
      File Info : 44100kHz; 2 channel; 16 bit; 00:52:53.4930000
      Results   : 53.89x; 373021162 bytes in 00:00:58.8901034 seconds;
      
      E:\Temp>CUETools.FlaCuda.exe -8 encode.wav -o encode4.flac
      CUETools.FlaCuda, Copyright (C) 2009 Gregory S. Chudov.
      Filename  : encode.wav
      File Info : 44100kHz; 2 channel; 16 bit; 00:52:53.4930000
      Results   : 198.27x; 373969399 bytes in 00:00:16.0056281 seconds;

but there is a discrepancy.... flaCUDA's compression scheme goes up to 11, and that is what the 58 second result was based on... when I use compression -8, the encoding time ends up only taking 16 seconds. jeez. I'm not sure how the compression numbers crossover though. all 4 output files are identical to the original and each other (same audio MD5's), the two files produced by FLAC and fpFLAC both have a bitrate of 944, whereas the flaCUDA -11 encode shows a 940 bitrate, and the flaCUDA -8 encode shows a 943 bitrate (in foobar2000) -- all end-product FLAC files are sized between 355MB and 357MB, so negligible difference, size wise. *shrug* i have no idea.... already put waaaay too much thought into this as it is haha

anyways, with differing compression schemes, it's hard to make a solid conclusion.... other than that both of these two are VERY FAST encoders!

oh yeah, and of course, I forgot to include what specs I was using to test:

Intel i7-860 (2.8Ghz stock, 3.47Ghz in turbo mode -- it's a Quad-core with Hyper-Threading, so it sees it as 8 CPU's)
4GB DDR3-1600 RAM
Windows7 x64... I used of course the 64 bit version and the SSE4 instructions as George said in the readme
(and video card is Nvidia GeForce 9600 GSO)


This post has been edited by Chinch: Dec 2 2009, 08:18

[jamesbaud]

Here is a screenshot how to configure EAC to use fpFLAC:

Of course, you can set a compression level other than "-8".

On "EAC -> EAC options -> Tools", set the number of "simultaneous external compressor thread(s)" to 1.

Could you provide a screenshot to configure foobar to use fpFLAC?

[JacksonGrey]

Could you provide a screenshot to configure foobar to use fpFLAC?

I'd like to second this request.

[GeorgeFP]

Could you provide a screenshot to configure foobar to use fpFLAC?

Today, I've finished the implementation of dedicated tools for specific tasks. The names of the tools are self-explanatory: fpMP3, fpFLAC, fpFLAC2FLAC and fpFLAC2MP3. I will release them tomorrow.

Tomorrow, I will also provide new screenshots for EAC and foobar.

[gottkaiser]

Wow, I'm impressed.
Thanks for all the work effort you put into this project!

[Glenda]

Such a nice upgrade to EAC, thankyou

[Glenda]

Such a nice upgrade to EAC, thankyou

Except I'm getting an error with some tracks

[Glenda]

Such a nice upgrade to EAC, thankyou

Except I'm getting an error with some tracks

No errors when running EAC under Win7x64 as admin.

[GeorgeFP]

Hi!

The new version can be downloaded here:

64-Bit version
32-Bit version (not supported)

Just extract the archive and execute "setup.exe".

To use "fpMP3" or "fpFLAC2MP3" you have to download the source code and build the "fpMP3_DLL" project (Blend, SSE2, or SSE4 version). The built DLLs have to be registered with "regsvr32" (admin rights required for that).

Here is the screenshot for EAC:

In this example, I've used compression level 8.

If you like to add some tags, use the "-T" option, e.g.

CODE

-T "ARTIST=%a" -T "TITLE=%t" ...

EAC FAQ: Which flags can I use in the external compression scheme “User Defined MP3 Encoder”?

I will upload the foobar screenshot in a few minutes.

[GeorgeFP]

And here is the screenshot for foobar2000:

You should also set the number of converter threads to 1:

File -> Preferences -> Advanced -> Tools -> Converter -> Thread count : 1

Note: Because of a programming bug the encoder may hang in some (rare) cases. It's under investigation.

[gottkaiser]

To use "fpMP3" or "fpFLAC2MP3" you have to download the source code and build the "fpMP3_DLL" project (Blend, SSE2, or SSE4 version). The built DLLs have to be registered with "regsvr32" (admin rights required for that).

I would like to use fpFLAC2MP3. But I'm not able to build the project. I'm just not into this subject.
Maybe somebody smart who owns all the needed software likes to build the project and upload it here in the forum?

That would be very appreciated.

[johnsonlam]

64-Bit version
32-Bit version (not supported)
Just extract the archive and execute "setup.exe".

Hi GeorgeFP,

Thanks for your effort and kindly sharing your great work with us, can you change the German installer into English?

Sorry but I'm not even a native English speaker, the installer seems fail to install but I can't read the error message.

Thanks again!

This post has been edited by johnsonlam: Dec 4 2009, 13:26

[GeorgeFP]

64-Bit version
32-Bit version (not supported)
Just extract the archive and execute "setup.exe".

Hi GeorgeFP,

Thanks for your effort and kindly sharing your great work with us, can you change the German installer into English?

Sorry but I'm not even a native English speaker, the installer seems fail to install but I can't read the error message.

Thanks again!

I will try to change the setup language to English. Can you send me a screenshot of the failed installation? Then, I'll be able to do two things at once.

[greynol]

Considering that EAC only passes one file at a time and has its own internal ability to make use of multiple processes, can someone please explain how you derive a benefit from this program with EAC?

[pdq]

So are you saying that it is pointless to use this version for ripping? That it is really only useful for batch converting?

[greynol]

I know I'm usually a sarcastic cynic, but this time I'm honestly asking for information. After considering this some more, I'd like to see rationale for using it with foobar2000 as well.

To answer your question though, with my limited understanding of all this, yes it does seem quite pointless.

[Gregory S. Chudo...]

This encoder is useful, because it can use all the available processor cores when compressing a single file.
This is especially useful when ripping not as separate tracks, but as a single image.

This post has been edited by Gregory S. Chudov: Dec 4 2009, 20:05

[greynol]

So it's a departure from this?
http://www.hydrogenaudio.org/forums/index....showtopic=73790

The current version is still mostly single-threaded for single-file encoding but scales for multiple-files encoding.

Ok, seeing the initial post here I do see

The files were encoded one after the other, not in parallel.

...for the benchmark test. Presumably demonstrating that the program does indeed process each file using more than one core.

So yeah, I would expect only a slight speedup for individual tracks unless there's a really big track that was ripped towards the end of the disc.

This post has been edited by greynol: Dec 4 2009, 20:34

[pdq]

Wouldn't the time to compress a single image file be a lot shorter than the time it took to rip it anyway, even using just a single core?

[greynol]

EAC doesn't start encoding with an external compressor until after the file is ripped.

[pdq]

I'm just saying that the time to encode after you finish ripping is fairly short relative to the time spent ripping. It probably takes you longer to replace that CD with the next one.

[greynol]

On my system it still takes longer to encode @ -8 with flac 1.2.1 than it takes to rip unless the disc is giving the drive trouble.

[GeorgeFP]

How hard will it be to add replay gain support? (Please?) With the addition of Verify support (mentioned above), I'll be all set

Oops, I missed this post.

Replay gain should not be a problem. I will add it in one of the next versions.
And verify support.