IPB

Welcome Guest ( Log In | Register )

5 Pages V  < 1 2 3 4 > »   
Reply to this topicStart new topic
A multicore FLAC encoder
Wombat
post Dec 2 2009, 03:36
Post #26





Group: Members
Posts: 950
Joined: 7-October 01
Member No.: 235



You may use it together with metaflac with linking to cmd.exe as encoder in the way it is explained here in the Wiki for AAC. I am using this logic with Flacuda with no problem. You even can add the tags with Metaflac.
One downside of cause of these direct cmd-lines is that no album-gain can be set corretly.

http://wiki.hydrogenaudio.org/index.php?title=EAC_and_AAC

maybe this helps:
/c ""C:\path\CUETools.FlaCuda.exe" -8 --verify %s -o %d && "C:\path\metaflac.exe" --set-tag="Album=%g" --set-tag="Artist=%a" --set-tag="Title=%t" --set-tag="Tracknumber=%n" --set-tag="Date=%y" --set-tag="Genre=%m" --set-tag="Comment=FlaCuda 0.6 -8" %d"

good luck
Go to the top of the page
+Quote Post
Chinch
post Dec 2 2009, 06:25
Post #27





Group: Members
Posts: 90
Joined: 22-July 09
Member No.: 71664



interesting project you have going here. i know that as mentioned before... gregory chudov was working on a similar project called "flaCUDA" using the Nvidia CUDA api's... i always find this stuff interesting to watch though. i have a couple of questions for you:

1) the encoding algorithm is standardized, correct? that is, obviously it is going to provide lossless encoding/decoding, and compress to (at least) the same filesize as one encoded with the original FLAC.exe? i know there were some projects going on trying to increase performance/compression on this codec.... which i don't like, standards are made for reasons wink.gif

2) you say the current algorithm for encoding is i/o bound, i'm assuming this means the encoder can encode faster than the hard drive can read/supply the data, is this what you mean? unfortunately, this probably means that on faster systems, the speed improvement (vs. the original encoder) will have a tendency to decline the faster the system gets. fortunately, this also will improve with current and future tech like SSD drives... wink.gif

3) i would also like the verification function. as far as i know, isn't this a simple matter of computing the MD5 checksum of the raw audio data, encoding, then decompressing and comparing the MD5 values? (the MD5 checksum itself is stored within the FLAC file, obviously you know this) splitting up the threading and stuff could make this complicated if it was calculated in real-time, but if the MD5 was made based on the source WAV, then turn right back around and decode to a temp file, and compare. this would obviously add time to the overall process, plus require 'decoder' code implemented. the original FLAC program seems to do the "Verify OK" instantaneously (or maybe my computer is just really fast)... can you extrapolate on what would be necessary to include this function? you already answered this in the above post wink.gif

4) i would be willing to write or help out with a GUI portion of this, or at least a frontend would be pretty easy to whip up to help simplify for some...


but the batch file posted, wouldn't this simple trick work -> copy the batch file to your desktop or Send To menu, (taking in %1 or argument as a variable) and then encoding that --- that is, basically drag n' drop the WAV file onto the shortcut icon (or send to menu) to encode the file?

QUOTE
One downside of cause of these direct cmd-lines is that no album-gain can be set correctly.

unless it was an image (single wav) rip, right?


i'm definitely going to test it out though. good work. i will report back my results.


noticed one comsmetic bug smile.gif in your command line help, when citing the examples of input/output, you still have reference to *MP3* files... i'm assuming just a copy and paste where you forgot to change? just letting you know wink.gif

QUOTE
Examples:

Encode one WAV file to one MP3:
FPSTREAM readfile in -f audio.wav
+ wavin wav -s in
+ fpmp3enc mp3enc -s wav --vbr-new
+ writefile out -s mp3enc -f audio.mp3

Encode one WAV file to one MP3 and one OGG with tags:
FPSTREAM readfile in -f audio.wav
+ wavin wav -s in
+ tag tag "TITLE=Audio Title" "ARTIST=Audio Artist"
+ fpmp3enc mp3enc -s wav --vbr-new -tag tag
+ vorbisenc oggenc -s wav -tag tag
+ writefile mp3file -s mp3enc -f audio.mp3
+ writefile oggfile -s oggenc -f audio.ogg



This post has been edited by Chinch: Dec 2 2009, 06:35
Go to the top of the page
+Quote Post
Chinch
post Dec 2 2009, 07:23
Post #28





Group: Members
Posts: 90
Joined: 22-July 09
Member No.: 71664



Alright! I'm impressed. I have to say that I didn't think the improvement would be that great on my system, but here are my results (based on about a 53 minute CD):

original FLAC encoder (-8) = 1 minute, 15 seconds. mellow.gif
and George FP's encoder (-8) = 24 seconds! shock1.gif

i didn't believe it, so I did a bit-compare on the two files:

CODE
All tracks decoded fine, no differences found.
    
    Comparing:
    "X:\encode1.flac"
    "X:\encode2.flac"
    No differences in decoded data found.


and then I did a verify on the FLAC files, to make sure they were identical (audio data-wise):

CODE
Item: "X:\encode1.flac"
    MD5: 66B84449D7A9899065A11FA190C994AC
    CRC32: D199E9A2
    No problems found.
    
    All items decoded successfully.
    
    
    Item: "X:\encode2.flac"
    MD5: 66B84449D7A9899065A11FA190C994AC
    CRC32: D199E9A2
    No problems found.
    
    All items decoded successfully.



finally, to prove that it wasn't EXACTLY THE SAME THING, i have noted the file sizes and hashes of both FLAC files... different! both files are unique.

CODE
File: encode1.flac
    Size: 357MB (374,398,783 bytes)
    CRC-32: fa984f9e
       MD5: 6eb400219c22f1aa1525d13f5f7f1135
     SHA-1: 70aa7e4a65ed635eb78b52ef3d2f1b1b8b435692
    
    File: encode2.flac
    Size: 356MB (374,328,806 bytes)
    CRC-32: eac29087
       MD5: a8d830f2defbe50af368ce60a5a69460
     SHA-1: c356080e0e508d26b8d7b625229af0d2ca2dde7b


so George... you have made a believer of me! empirical evidence, right there for me and everyone else to see.... congrats!
I am about to do a second test on a second system, then I will post those results below.

Results from test #2:

original FLAC encoder (-8 compression) = 1 minute, 07 seconds
and George FP's encoder (-8 compression) = 21 seconds

just for kicks I tried flaCUDA...

flaCUDA encoder (-11 compression) = 58 seconds
flaCUDA encoder (-8 compression) = 16 seconds

CODE
E:\Temp>CUETools.FlaCuda.exe -11 encode.wav -o encode3.flac
      CUETools.FlaCuda, Copyright (C) 2009 Gregory S. Chudov.
    Filename  : encode.wav
      File Info : 44100kHz; 2 channel; 16 bit; 00:52:53.4930000
      Results   : 53.89x; 373021162 bytes in 00:00:58.8901034 seconds;
      
      E:\Temp>CUETools.FlaCuda.exe -8 encode.wav -o encode4.flac
      CUETools.FlaCuda, Copyright (C) 2009 Gregory S. Chudov.
      Filename  : encode.wav
      File Info : 44100kHz; 2 channel; 16 bit; 00:52:53.4930000
      Results   : 198.27x; 373969399 bytes in 00:00:16.0056281 seconds;


but there is a discrepancy.... flaCUDA's compression scheme goes up to 11, and that is what the 58 second result was based on... when I use compression -8, the encoding time ends up only taking 16 seconds. jeez. I'm not sure how the compression numbers crossover though. all 4 output files are identical to the original and each other (same audio MD5's), the two files produced by FLAC and fpFLAC both have a bitrate of 944, whereas the flaCUDA -11 encode shows a 940 bitrate, and the flaCUDA -8 encode shows a 943 bitrate (in foobar2000) -- all end-product FLAC files are sized between 355MB and 357MB, so negligible difference, size wise. *shrug* i have no idea.... already put waaaay too much thought into this as it is haha wink.gif

anyways, with differing compression schemes, it's hard to make a solid conclusion.... other than that both of these two are VERY FAST encoders!

oh yeah, and of course, I forgot to include what specs I was using to test:

Intel i7-860 (2.8Ghz stock, 3.47Ghz in turbo mode -- it's a Quad-core with Hyper-Threading, so it sees it as 8 CPU's)
4GB DDR3-1600 RAM
Windows7 x64... I used of course the 64 bit version and the SSE4 instructions as George said in the readme
(and video card is Nvidia GeForce 9600 GSO)



This post has been edited by Chinch: Dec 2 2009, 08:18
Go to the top of the page
+Quote Post
jamesbaud
post Dec 2 2009, 11:00
Post #29





Group: Members
Posts: 107
Joined: 15-December 03
Member No.: 10468



QUOTE (GeorgeFP @ Nov 23 2009, 01:34) *
Here is a screenshot how to configure EAC to use fpFLAC:



Of course, you can set a compression level other than "-8".

On "EAC -> EAC options -> Tools", set the number of "simultaneous external compressor thread(s)" to 1.


Could you provide a screenshot to configure foobar to use fpFLAC?
Go to the top of the page
+Quote Post
JacksonGrey
post Dec 2 2009, 13:40
Post #30





Group: Members
Posts: 122
Joined: 5-February 09
Member No.: 66629



QUOTE (jamesbaud @ Dec 2 2009, 10:00) *
Could you provide a screenshot to configure foobar to use fpFLAC?


I'd like to second this request. smile.gif
Go to the top of the page
+Quote Post
GeorgeFP
post Dec 3 2009, 09:50
Post #31





Group: Members
Posts: 81
Joined: 2-August 09
Member No.: 71959



QUOTE (jamesbaud @ Dec 2 2009, 11:00) *
Could you provide a screenshot to configure foobar to use fpFLAC?

Today, I've finished the implementation of dedicated tools for specific tasks. The names of the tools are self-explanatory: fpMP3, fpFLAC, fpFLAC2FLAC and fpFLAC2MP3. I will release them tomorrow.

Tomorrow, I will also provide new screenshots for EAC and foobar.
Go to the top of the page
+Quote Post
gottkaiser
post Dec 3 2009, 12:26
Post #32





Group: Members
Posts: 171
Joined: 7-January 05
From: Germany
Member No.: 18891



Wow, I'm impressed.
Thanks for all the work effort you put into this project!
Go to the top of the page
+Quote Post
Glenda
post Dec 3 2009, 14:14
Post #33





Group: Members
Posts: 66
Joined: 27-November 07
Member No.: 49067



Such a nice upgrade to EAC, thankyou
Go to the top of the page
+Quote Post
Glenda
post Dec 3 2009, 14:22
Post #34





Group: Members
Posts: 66
Joined: 27-November 07
Member No.: 49067



QUOTE (Glenda @ Dec 3 2009, 09:14) *
Such a nice upgrade to EAC, thankyou



Except I'm getting an error with some tracks
Go to the top of the page
+Quote Post
Glenda
post Dec 3 2009, 14:36
Post #35





Group: Members
Posts: 66
Joined: 27-November 07
Member No.: 49067



QUOTE (Glenda @ Dec 3 2009, 09:22) *
QUOTE (Glenda @ Dec 3 2009, 09:14) *
Such a nice upgrade to EAC, thankyou



Except I'm getting an error with some tracks




No errors when running EAC under Win7x64 as admin.
Go to the top of the page
+Quote Post
GeorgeFP
post Dec 4 2009, 12:27
Post #36





Group: Members
Posts: 81
Joined: 2-August 09
Member No.: 71959



Hi!

The new version can be downloaded here:

64-Bit version
32-Bit version (not supported)

Just extract the archive and execute "setup.exe".

To use "fpMP3" or "fpFLAC2MP3" you have to download the source code and build the "fpMP3_DLL" project (Blend, SSE2, or SSE4 version). The built DLLs have to be registered with "regsvr32" (admin rights required for that).

Here is the screenshot for EAC:



In this example, I've used compression level 8.

If you like to add some tags, use the "-T" option, e.g.
CODE
-T "ARTIST=%a" -T "TITLE=%t" ...

EAC FAQ: Which flags can I use in the external compression scheme “User Defined MP3 Encoder”?

I will upload the foobar screenshot in a few minutes.
Go to the top of the page
+Quote Post
GeorgeFP
post Dec 4 2009, 12:47
Post #37





Group: Members
Posts: 81
Joined: 2-August 09
Member No.: 71959



And here is the screenshot for foobar2000:


You should also set the number of converter threads to 1:

File -> Preferences -> Advanced -> Tools -> Converter -> Thread count : 1

Note: Because of a programming bug the encoder may hang in some (rare) cases. It's under investigation.
Go to the top of the page
+Quote Post
gottkaiser
post Dec 4 2009, 13:08
Post #38





Group: Members
Posts: 171
Joined: 7-January 05
From: Germany
Member No.: 18891



QUOTE (GeorgeFP @ Dec 4 2009, 12:27) *
To use "fpMP3" or "fpFLAC2MP3" you have to download the source code and build the "fpMP3_DLL" project (Blend, SSE2, or SSE4 version). The built DLLs have to be registered with "regsvr32" (admin rights required for that).

I would like to use fpFLAC2MP3. But I'm not able to build the project. I'm just not into this subject.
Maybe somebody smart who owns all the needed software likes to build the project and upload it here in the forum? rolleyes.gif

That would be very appreciated.
Go to the top of the page
+Quote Post
johnsonlam
post Dec 4 2009, 13:25
Post #39





Group: Members
Posts: 226
Joined: 12-January 03
From: Kowloon, Hong Kong
Member No.: 4533



QUOTE (GeorgeFP @ Dec 4 2009, 19:27) *
64-Bit version
32-Bit version (not supported)
Just extract the archive and execute "setup.exe".


Hi GeorgeFP,

Thanks for your effort and kindly sharing your great work with us, can you change the German installer into English?

Sorry but I'm not even a native English speaker, the installer seems fail to install but I can't read the error message.

Thanks again!

This post has been edited by johnsonlam: Dec 4 2009, 13:26


--------------------
Hong Kong - International Joke Center (after 1997-06-30)
Go to the top of the page
+Quote Post
GeorgeFP
post Dec 4 2009, 14:03
Post #40





Group: Members
Posts: 81
Joined: 2-August 09
Member No.: 71959



QUOTE (johnsonlam @ Dec 4 2009, 13:25) *
QUOTE (GeorgeFP @ Dec 4 2009, 19:27) *
64-Bit version
32-Bit version (not supported)
Just extract the archive and execute "setup.exe".


Hi GeorgeFP,

Thanks for your effort and kindly sharing your great work with us, can you change the German installer into English?

Sorry but I'm not even a native English speaker, the installer seems fail to install but I can't read the error message.

Thanks again!

I will try to change the setup language to English. Can you send me a screenshot of the failed installation? Then, I'll be able to do two things at once.
Go to the top of the page
+Quote Post
greynol
post Dec 4 2009, 18:12
Post #41





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



Considering that EAC only passes one file at a time and has its own internal ability to make use of multiple processes, can someone please explain how you derive a benefit from this program with EAC?


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
pdq
post Dec 4 2009, 19:51
Post #42





Group: Members
Posts: 3304
Joined: 1-September 05
From: SE Pennsylvania
Member No.: 24233



So are you saying that it is pointless to use this version for ripping? That it is really only useful for batch converting?
Go to the top of the page
+Quote Post
greynol
post Dec 4 2009, 19:57
Post #43





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



I know I'm usually a sarcastic cynic, but this time I'm honestly asking for information. After considering this some more, I'd like to see rationale for using it with foobar2000 as well.

To answer your question though, with my limited understanding of all this, yes it does seem quite pointless.


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
Gregory S. Chudo...
post Dec 4 2009, 20:04
Post #44





Group: Developer
Posts: 683
Joined: 2-October 08
From: Ottawa
Member No.: 59035



This encoder is useful, because it can use all the available processor cores when compressing a single file.
This is especially useful when ripping not as separate tracks, but as a single image.

This post has been edited by Gregory S. Chudov: Dec 4 2009, 20:05


--------------------
CUETools 2.1.4
Go to the top of the page
+Quote Post
greynol
post Dec 4 2009, 20:10
Post #45





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



So it's a departure from this?
http://www.hydrogenaudio.org/forums/index....showtopic=73790
QUOTE
The current version is still mostly single-threaded for single-file encoding but scales for multiple-files encoding.

Ok, seeing the initial post here I do see
QUOTE
The files were encoded one after the other, not in parallel.
...for the benchmark test. Presumably demonstrating that the program does indeed process each file using more than one core.

So yeah, I would expect only a slight speedup for individual tracks unless there's a really big track that was ripped towards the end of the disc.

This post has been edited by greynol: Dec 4 2009, 20:34


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
pdq
post Dec 4 2009, 20:20
Post #46





Group: Members
Posts: 3304
Joined: 1-September 05
From: SE Pennsylvania
Member No.: 24233



Wouldn't the time to compress a single image file be a lot shorter than the time it took to rip it anyway, even using just a single core?
Go to the top of the page
+Quote Post
greynol
post Dec 4 2009, 20:36
Post #47





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



EAC doesn't start encoding with an external compressor until after the file is ripped.


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
pdq
post Dec 4 2009, 21:25
Post #48





Group: Members
Posts: 3304
Joined: 1-September 05
From: SE Pennsylvania
Member No.: 24233



I'm just saying that the time to encode after you finish ripping is fairly short relative to the time spent ripping. It probably takes you longer to replace that CD with the next one.
Go to the top of the page
+Quote Post
greynol
post Dec 4 2009, 21:29
Post #49





Group: Super Moderator
Posts: 10000
Joined: 1-April 04
From: San Francisco
Member No.: 13167



On my system it still takes longer to encode @ -8 with flac 1.2.1 than it takes to rip unless the disc is giving the drive trouble.


--------------------
Your eyes cannot hear.
Go to the top of the page
+Quote Post
GeorgeFP
post Dec 4 2009, 21:37
Post #50





Group: Members
Posts: 81
Joined: 2-August 09
Member No.: 71959



QUOTE (howesz @ Dec 2 2009, 03:14) *
How hard will it be to add replay gain support? (Please?) With the addition of Verify support (mentioned above), I'll be all set cool.gif

Oops, I missed this post.

Replay gain should not be a problem. I will add it in one of the next versions.
And verify support.
Go to the top of the page
+Quote Post

5 Pages V  < 1 2 3 4 > » 
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 16th April 2014 - 15:07