New FLAC encoder Options

[Nayru]

A new FLAC encoder has been developed, which is proposed for inclusion in ffmpeg. The author claims that in high-compression mode, it can compress certain files better than libFLAC.

http://mplayerhq.hu/pipermail/ffmpeg-devel...May/011385.html

[kurtnoise]

For those who want to test it, I've made a compile for win32 plateforms.


booo...it crashes.

This post has been edited by kurtnoise: May 26 2006, 14:45

[jcoalson]

it seems like a nice clean interface. some notes:

1. at the higher compression settings it is using large block sizes (16k for 44.1kHz) and high lpc order (32) which increase the decoding burden with little improvement (at least in my tests). (e.g. I'm not sure the squeezebox supports 16k blocks due to buffer requirements.) also, performance with larger blocksizes in more dependent on the audio characteristics so his results might be biased by his sample.

2. it has an interesting way of selecting the stereo decorrelation method which is faster than the reference encoder; whether it's better on not needs some testing. same for fixed/lpc subframe selection.

3. it's guessing at the max frame size without supporting verbatim encoding, which might explain it crashing

4. the lpc analysis stage looks just like the reference encoder.

so I think any compression gain is just due to the larger blocksize and high lpc order, which you can do with the reference encoder (though I wouldn't encourage that for general use).

Josh

[Justin Ruggles]

Hi,
I am the person writing the encoder mentioned above. It is in an extremely experimental phase at the moment. I am fairly new to audio encoding, so most of what I am doing is just learning by trying things out and reading as much as I can. I do enjoy it though. I want to say that I greatly admire what Josh has done with the flac format and reference software. Its openness and good documentation are a rarity and a pleasure. I am writing an encoder for FFmpeg because one of the goals of that project is to write its own decoders and encoders so as not to rely on too many external libraries, and because of licensing (which I don't think is really much of an issue in the case of FLAC). Also, I am doing it for the shear pleasure of learning something new. My claims of better encoding are more due to excitement at the results than anything else. Beating the reference encoder is not my goal. It's just a good motivator, both for the excitement when I succeed and for the challenge when I can't seem to get anywhere close.

it seems like a nice clean interface. some notes:

1. at the higher compression settings it is using large block sizes (16k for 44.1kHz) and high lpc order (32) which increase the decoding burden with little improvement (at least in my tests). (e.g. I'm not sure the squeezebox supports 16k blocks due to buffer requirements.) also, performance with larger blocksizes in more dependent on the audio characteristics so his results might be biased by his sample.

That is true. I have found some odd results in my tests though. Many times I have seen my encoder do much better on a particular clip with, say, max lpc order of 32, but do not see the same benefit on the same clip with the reference encoder. I couldn't say why or how though. I am mostly just trying to test anything I can and observing what settings work best in which situations. You do have a point about buffer requirements. I had to increase the SDL audio buffer size in ffplay to get the high blocksize clips to play without skipping.

The issue with block sizes and different audio characteristics is one I am still struggling with. It seems to be one of the biggest drawbacks to really good compression quality. Variable blocksize would be ideal, but I know the compatiblity issues. Given the limitations, one of my objectives is to have good compatible modes, but also to provide more options for better compression. The reference encoder seems to go for the middle ground in many cases. This has its benefits, but personally, I like more options.

2. it has an interesting way of selecting the stereo decorrelation method which is faster than the reference encoder; whether it's better on not needs some testing. same for fixed/lpc subframe selection.

Yeah, the stereo decorrelation selection was just a guess that turned out to work okay. I haven't looked closely enough to see how the reference encoder does it. As for fixed/lpc subframe selection...I can't really remember how I was doing that on May 26 . I have had too many abandoned approaches to keep up... Right now I am using fixed mode (and mostly just order 2) only in the fastest encoding setting and for small frames (usually just the last frame).

3. it's guessing at the max frame size without supporting verbatim encoding, which might explain it crashing

I have no idea about the crashing on windows, as I use Linux. As for verbatim encoding, I actually started the program as verbatim only just to get the bitstream right. I later removed it. I probably should add it back in as a backup mechanism for audio that really just won't compress...a rarity, but it would save the guessing you mentioned. For now, I have at least added a warning & skip frames that are too big. It's not ideal, nor lossless, but at least it gives a warning...

4. the lpc analysis stage looks just like the reference encoder.

Yeah, it pretty much is. I have modified the autocorrelation slightly and implemented my own quantizer. The levinson-durbin function is the same. I experimented some with other ways of estimating the best predictor order, but it was not as good as the reference encoder. When I coupled the 2 together and selected the best by counting the bits I got slightly better encoding...that might have just been chance though I could have probably selected an order at random and done the same thing. I guess I'll have to keep researching. It's not that I think the reference encoder has a bad solution...I just like a challenge. I recently added, and am currently experimenting with, window functions before autocorrelation. Yesterday I was pointed to a Hydrogenaudio thread on how this is being added to the reference encoder as well. Very exciting stuff!

so I think any compression gain is just due to the larger blocksize and high lpc order, which you can do with the reference encoder (though I wouldn't encourage that for general use).

Blocksize I understand, but what's wrong with a higher LPC order? True, it isn't ideal for every situation, but it does help in many cases. I haven't noticed much of an impact on decoding, and I have a quite slow computer.

Anyway, I just thought I would respond. I am new to Hydrogenaudio, but since I seem to have thrown myself into the fray I might as well join in on the discussion.

-Justin

[kurtnoise]

Hi Justin,

Thank you very much for this work (fork ?)...

I've made a benchmarck on my AMD64 3200+. I've tested the last flake 0.2 and flac from the last cvs entry with a wav sample of 1h11m55s (mostly electro and pop rock).

Here are my results:

1°) Flake 0.2 compiled with gcc 4.1.0 & CFLAGS = -g -O3 -Wall -pedantic -std=c99 -march=k8

CODE

|  level  |   Real Time   |       Bytes        |  Ratio   |  kbps |
|    0     |  2m11.511s  |  530 951 235  |  0,697  | 984   |
|    1     |  2m34.031s  |  513 566 686  |  0.675  | 952   |
|    2     |  2m34.206s  |  499 679 481  |  0,656  | 926   |
|    3     |  2m36.720s  |  491 472 721  |  0,645  | 911   |
|    4     |  2m35.962s  |  488 370 918  |  0,641  | 905   |
|    5     |  2m50.350s  |  487 493 404  |  0,640  | 904   |
|    6     |  2m51.587s  |  487 565 406  |  0,640  | 904   |
|    7     |  2m50.688s  |  490 601 577  |  0,644  | 910   |
|    8     |  4m13.558s  |  490 820 897  |  0,644  | 910   |

2°) FLAC CVS 20060603 compiled with gcc 4.1.0

CODE

|  level  |   Real Time   |       Bytes        |  Ratio   |  kbps |
|    0     |  1m16.089s  |  531 245 719  |  0,698  | 984   |
|    1     |  1m20.170s  |  510 802 749  |  0.671  | 947   |
|    2     |  1m25.361s  |  505 186 083  |  0,663  | 940   |
|    3     |  1m32.409s  |  501 361 052  |  0,652  | 933   |
|    4     |  1m35.588s  |  489 589 293  |  0,643  | 908   |
|    5     |  1m50.881s  |  488 073 198  |  0,641  | 905   |
|    6     |  2m04.660s  |  487 691 205  |  0,641  | 904   |
|    7     |  6m59.237s  |  486 851 452  |  0,640  | 903   |
|    8     |  9m00.266s  |  485 704 730  |  0,638  | 901   |

Now, I've a question : why flake crashes during encoding on win32 plateform ? In fact, compilation is fine but running fails. Any idea ?

This post has been edited by kurtnoise: Jun 3 2006, 16:53

[Justin Ruggles]

Thank you very much for this work (fork ?)...

You're welcome. Fork? Well, I suppose that if I get the LPC order selection code replaced with my own then I can claim it to be nearly non-derived . The Levinson-Durbin LPC generation code is nearly identical to that in several other programs I've seen. I copied the UTF-8 code basically because I'm not 100% clear on how that encoding works. I suppose I should separate these 3 or 4 functions into a single file with proper copyright notice and attribution...or at least put a small notice above each and include both a COPYING.libFLAC file and COPYING.lgpl file. It wasn't really a fork though. I basically started from scratch using the specs and some reused FFmpeg code. I even tried my own UTF-8 function working backwards from the decoding in FFmpeg...I could never get it to work though, so I copied in that function. When adding non-fixed LPC I tried my hand at that as well, but fell short. So it's more of an independent implementation with crutches. The structure is my own, but the puzzle had to be filled-in in a few places.

I've made a benchmarck on my AMD64 3200+. I've tested the last flake 0.2 and flac from the last cvs entry with a wav sample of 1h11m55s (mostly electro and pop rock).

Here are my results:

1°) Flake 0.2 compiled with gcc 4.1.0 & CFLAGS = -g -O3 -Wall -pedantic -std=c99 -march=k8

| level | Real Time | Bytes | Ratio | kbps |
[...]
| 5 | 2m50.350s | 487 493 404 | 0,640 | 904 |

2°) FLAC CVS 20060603 compiled with gcc 4.1.0

| level | Real Time | Bytes | Ratio | kbps |
[...]
| 8 | 9m00.266s | 485 704 730 | 0,638 | 901 |

I also noticed similar results (not quite as good as libFLAC) on samples where flake did the best with smaller block sizes (i.e. level 5, which is blocksize 2048), but that the reference encoder did best at level 8 (blocksize 4608). I am still trying to wrap my head around this issue. It is possibly due to whatever libFLAC does in "exhaustive" mode? Or maybe my version of stereo decorrelation works well only on audio with certain characteristics? Testing, testing, and more testing. I'm working on my own horribly slow "brute-force" mode to help me out in my tests.

Now, I've a question : why flake crashes during encoding on win32 plateform ? In fact, compilation is fine but running fails. Any idea ?

I really have no clue here. What does the crash look like? Is it a segfault? Any gdb info? If I had Windows I would test it myself. If MinGW is being used, maybe it needs "--enable-memalign-hack"...? It has been several..5..6.. years since I have done any gcc compilation in Windows.

-Justin

[kurtnoise]

I really have no clue here. What does the crash look like? Is it a segfault? Any gdb info? If I had Windows I would test it myself. If MinGW is being used, maybe it needs "--enable-memalign-hack"...? It has been several..5..6.. years since I have done any gcc compilation in Windows.

Through wine, here is the debug :

lionel@kubuntu:~/flake$ wine ./flake.exe -2 /home/lionel/wav/input.wav /home/lionel/wav/output.flac
Flake: FLAC audio encoder
Signed 16-bit 44100 Hz stereo
samples: 190288560
wine: Unhandled page fault on write access to 0x7f9a0dd4 at address 0x405703 (thread 0009), starting debugger...
WineDbg starting on pid 0x8
Unhandled exception: page fault on write access to 0x7f9a0dd4 in 32-bit code (0x00405703).
Register dump:
CS:0073 SS:007b DS:007b ES:007b FS:1007 GS:0033
EIP:00405703 ESP:7fbafdcc EBP:7fbafdd8 EFLAGS:00210206( - 00 - RIP1)
EAX:001f4514 EBX:ffffffff ECX:7f9a0dd4 EDX:7fd085e0
ESI:00401210 EDI:7ffdf760
Stack dump:
0x7fbafdcc: 7fcf0020 004018f6 ffffffff 7fbafeb8
0x7fbafddc: 00403306 7fd085e0 7fbafe60 0b5792b0
0x7fbafdec: 00402fd4 7fcf0020 00000000 7fcf0020
0x7fbafdfc: 00402fcf 7fd08c38 7fd0945c 7fbafe28
0x7fbafe0c: 00000010 7fcf0000 00000000 7fd084b0
0x7fbafe1c: 00000000 7fcf0000 7fc946dc 7fbafe48
0200: sel=1007 base=7fe6c000 limit=00001fff 32-bit rw-
Backtrace:
=>1 0x00405703 in flake (+0x5703) (0x00405703)
2 0x00403306 in flake (+0x3306) (0x00403306)
3 0x004011d9 __mingw_CRTStartup+0xc9 [/build/buildd/mingw32-runtime-3.8/build_dir/src/mingw-runtime-3.8/crt1.c:226] in flake (0x004011d9)
4 0x00401223 in flake (+0x1223) (0x00401223)
5 0x7fc5b311 in kernel32 (+0x4b311) (0x7fc5b311)
6 0xb7ed8ddb wine_switch_to_stack+0x17 in libwine.so.1 (0xb7ed8ddb)
0x00405703: orl $0,0x0(%ecx)
Modules:
Module Address Debug info Name (19 modules)
PE 0x00400000-00438000 DIA flake
ELF 0x7bf00000-7bf03000 Deferred <wine-loader>
ELF 0x7f93f000-7f9a0000 Deferred msvcrt<elf>
\-PE 0x7f950000-7f9a0000 \ msvcrt
ELF 0x7fbee000-7fcf0000 Export kernel32<elf>
\-PE 0x7fc10000-7fcf0000 \ kernel32
ELF 0x7fe07000-7fe11000 Deferred libnss_files.so.2
ELF 0x7fe11000-7fe26000 Deferred libnsl.so.1
ELF 0x7fe3a000-7fe5c000 Deferred libm.so.6
ELF 0x7fe63000-7fe6c000 Deferred libnss_nis.so.2
ELF 0x7fe70000-7ff66000 Deferred libwine_unicode.so.1
ELF 0x7ff66000-7ffe0000 Deferred ntdll<elf>
\-PE 0x7ff80000-7ffe0000 \ ntdll
ELF 0xb7d71000-b7d7a000 Deferred libnss_compat.so.2
ELF 0xb7d7b000-b7d7e000 Deferred libdl.so.2
ELF 0xb7d7e000-b7ead000 Deferred libc.so.6
ELF 0xb7ead000-b7ebf000 Deferred libpthread.so.0
ELF 0xb7ed4000-b7eee000 Export libwine.so.1
ELF 0xb7ef1000-b7f07000 Deferred ld-linux.so.2
Threads:
process tid prio (all id:s are in hex)
00000008 (D) Z:\home\lionel\flake\flake.exe
00000009 0 <==

Hope this helps....


About fork. It was just a joking. Keep up the good work.

[Justin Ruggles]

Through wine, here is the debug :

lionel@kubuntu:~/flake$ wine ./flake.exe -2 /home/lionel/wav/input.wav /home/lionel/wav/output.flac
Flake: FLAC audio encoder
Signed 16-bit 44100 Hz stereo
samples: 190288560
wine: Unhandled page fault on write access to 0x7f9a0dd4 at address 0x405703 (thread 0009), starting debugger...
WineDbg starting on pid 0x8
Unhandled exception: page fault on write access to 0x7f9a0dd4 in 32-bit code (0x00405703).

It's kinda hard to tell without more symbol names, but my thoughts still go to memory alignment in the structs. I tried rearranging the structs so that non-DWORD values are always at the end. Maybe that will help? If that's even the issue. It seems to be happening in the small section in flake.c around flake_encode_init. This is when the flac encoder structures are first accessed. Some of those values are pointers, so if the alignment is off, the pointers could end up pointing to unaccessible memory, which could cause a page fault.

-Justin

This post has been edited by Justin Ruggles: Jun 4 2006, 08:28

[Justin Ruggles]

Hi,
I have made a small webpage to post updated versions and various test results from my FLAC encoder. The page will be updated frequently. I have included a Linux i386 binary along with the source code, and I hope I can get some help providing a Windows binary. Hopefully, the more people I can get to test it out, the more I can improve it.

http://home.earthlink.net/~jruggle/flake/

Please note that this pre-release is not intended for general use, only testing. It does not currently calculate MD5 sums, do any tagging, or add any padding. The wav parser is not very robust either.

Thanks,
-Justin

[kurtnoise]

Hi Justin,

Great stuff...Now, flake 0.4 works fine too on win32 os. I uploaded a windows build here.

[krmathis]

flake dont seem to work correctly on Mac OS X (PowerPC).

The source compiles fine, and the flake executable does it work to my source .wav file.
But it dont compress the file at all (actually it is larger than the original), and the output is just noice.

flake 0.4:

CODE

Flake: FLAC audio encoder
© 2006 Justin Ruggles

Signed 16-bit 44100 Hz stereo
samples: 28563276

blocksize: 4608
max order: 8
order method: estimate
stereo method: estimate

progress: 100% | ratio: 1.025 | bitrate: 1447.0 kbps | bytes: 117148084

flac 1.1.2:

CODE

flac 1.1.2, Copyright © 2000,2001,2002,2003,2004,2005 Josh Coalson
flac comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
welcome to redistribute it under certain conditions. Type `flac' for details.

options: -P 4096 -b 4608 -m -l 8 -q 0 -r 3,3
wrote 78408059 bytes, ratio=0.686

I have tried multiple input files, from different sources, but the same happends every time.
Is this a known problem?

[Justin Ruggles]

flake dont seem to work correctly on Mac OS X (PowerPC).

The source compiles fine, and the flake executable does it work to my source .wav file.
But it dont compress the file at all (actually it is larger than the original), and the output is just noice.

...

I have tried multiple input files, from different sources, but the same happends every time.
Is this a known problem?

Dang. I thought I had fixed the endian issue. I'll keep trying though. It's hard to test when I don't have a big-endian machine.
-Justin

[krmathis]

Hope you nail this, cause this looks like an interresting application.

If you need a test dummy, just give me a hint.

[jcoalson]

Yesterday I was pointed to a Hydrogenaudio thread on how this is being added to the reference encoder as well. Very exciting stuff!

btw in the FLAC code in CVS there is a src/libFLAC/window.c that may save you some time.

Blocksize I understand, but what's wrong with a higher LPC order? True, it isn't ideal for every situation, but it does help in many cases. I haven't noticed much of an impact on decoding, and I have a quite slow computer.

the number of multiply-adds in the FIR filter equals the order. flac -8 uses max 12, so order 32 requires 20 more multiplies+adds per sample than what is widely used.

Josh

[Justin Ruggles]

Yesterday I was pointed to a Hydrogenaudio thread on how this is being added to the reference encoder as well. Very exciting stuff!

btw in the FLAC code in CVS there is a src/libFLAC/window.c that may save you some time.

Thanks. I did see that. I chose to go with the Welch window. It is simple to calculate and gives above-average results most of the time.

Blocksize I understand, but what's wrong with a higher LPC order? True, it isn't ideal for every situation, but it does help in many cases. I haven't noticed much of an impact on decoding, and I have a quite slow computer.

the number of multiply-adds in the FIR filter equals the order. flac -8 uses max 12, so order 32 requires 20 more multiplies+adds per sample than what is widely used.

Once I started benchmarking the decoding, I saw what you were talking about. Using max order 32 seems to increase decoding time about 30%. I didn't notice until I actually timed it because I was using short clips. So, I moved the higher block sizes and orders to extended levels 9-15 and modified 0-8 to use similar output parameters to the reference encoder. The higher block sizes don't seem to do much now that I've changed the encoder quite a bit. I'm strongly considering removing the preset levels for 16384 and maybe even 8192.

-Justin

[ggf31416]

40.964.828 bytes 16-bit Stereo 44100Hz WAV 3:52 "La cosecha de mujeres"

CODE

FLAKE 0.4
Level | Ratio | Bitrate | Time
0 | 0.788 | 1112.6 kbps | 6.484
1 | 0.747 | 1054.7 kbps | 7.328
2 | 0.741 | 1045.1 kbps | 7.250
3 | 0.733 | 1034.5 kbps | 7.468
4 | 0.729 | 1029.2 kbps | 7.500
5 | 0.726 | 1024.1 kbps | 7.906
6 | 0.721 | 1017.5 kbps | 11.250
7 | 0.721 | 1017.5 kbps | 12.718
8 | 0.719 | 1015.0 kbps | 14.343
9 | 0.719 | 1014.8 kbps | 29.046
10 | 0.719 | 1015.2 kbps | 20.671
11 | 0.719 | 1014.3 kbps | 120.562
12 | 0.720 | 1016.7 kbps | 21.093
13 | 0.720 | 1016.1 kbps | 119.875
14 | 0.722 | 1019.5 kbps | 21.703
15 | 0.722 | 1019.1 kbps | 120.828

FLAC 1.1.2 CVS
Level | Ratio | Time
0 | 0,782 | 2.687
1 | 0,778 | 2.890
2 | 0,776 | 3.609
3 | 0,731 | 3.750
4 | 0,723 | 4.812
5 | 0,722 | 6.031
6 | 0,722 | 6.625
7 | 0,721 | 21.531
8 | 0,720 | 28.140

In some samples a large block size can reduce the compression ratio.
WAV 111.266.108 bytes 16-bit Stereo 44100Hz 10:30 "La navidad de los pobres"

CODE

Parameters | Ratio | Bitrate | Bytes | Time
FLAKE -8 -b 1152 | 0.645 | 909.7 kbps | 71722600 | 39.640
FLAKE -8 -b 2048 | 0.642 | 906.0 kbps | 71436942 | 40.031
FLAKE -8 -b 2304 | 0.642 | 906.2 kbps | 71446745 | 39.656
FLAKE -8 -b 4096 | 0.644 | 909.3 kbps | 71697454 | 39.609
FLAKE -8 -b 4608 | 0.645 | 910.6 kbps | 71796426 | 38.843
FLAKE -8 -b 8192 | 0.652 | 920.1 kbps | 72549094 | 37.812
FLAC CVS -8 | 0.647 | ~913 kbps | 71971927 | 78.421

I suggest testing several block sizes in the high levels, using a lower compression level. Then use the best value to compress the sample:
Example:

CODE

-b x,[n]

-b x Block size [16 - 65535] (default: 4608)
-b 1 Try -b 2048 and -b 4608
-b 2 Try -b 2048, -b 2304, -b 4096, -b 4608 and -b 8192.
-b 3 Try -b 192, -b 576, -b 1152, -b 2304, -b 4608, -b 256, -b 512, -b 1024, -b 2048, -b 4096, -b 8192 and -b 16384.

n Use compression level n for the tries. Default: 5 for -b 1, -b 2 or -b 3.

Compression Levels:
[1-9] Without changes
10 = -l 12 -o 4 -s 2 -b 1,5  Estimated Time: ~  45 (2* level 5 + level 9)
11 = -l 12 -o 4 -s 2 -b 2,5  Estimated Time: ~  69 (5*5+9)
12 = -l 32 -o 4 -s 2 -b 2,5  Estimated Time: ~ 160 (5*5+11)
13 = -l 32 -o 4 -s 2 -b 3,5  Estimated Time: ~ 215 (12*5+11)
14 = -l 32 -o 4 -s 2 -b 3,8  Estimated Time: ~ 293 (12*8+11)

All Times are Process Time (Timer 3.01). Celeron D325 2.53 Ghz under Windows XP SP2.
Estimates times are based on the first sample ("La cosecha de mujeres").

This post has been edited by ggf31416: Jun 15 2006, 04:20

[Justin Ruggles]

I suggest testing several block sizes in the high levels, using a lower compression level. Then use the best value to compress the sample:
Example:

-b x Block size [16 - 65535] (default: 4608)
-b 1 Try -b 2048 and -b 4608
-b 2 Try -b 2048, -b 2304, -b 4096, -b 4608 and -b 8192.
-b 3 Try -b 192, -b 576, -b 1152, -b 2304, -b 4608, -b 256, -b 512, -b 1024, -b 2048, -b 4096, -b 8192 and -b 16384.

Along those same lines, I am working on implementing block switching. The resulting files would not be Subset compliant, but they would still decode with at least the reference decoder and with FFmpeg/MPlayer, which are the only decoders I have tested with at this point. I have found that 8192 only rarely increases compression and 16384 almost never does. I have now removed them as presets (pending pre-release 5) in favor of an 8-level order search.

The advantage of block switching would be with audio which compresses better at 2048 or 1024 blocksize. What you're suggesting seems like it could be accomplished with a 2-pass encoding, which I guess I could try to implement as another option.

Thanks,
Justin

[Justin Ruggles]

Hi,
Pre-release 5 is now out.

* I hope the PPC endian issue is fixed. If not, please let me know.

* Added support for streamed input and output. For example,
cat test.wav | flake - - | flac -d -o test2.wav -

* Added 8-bit input support. 24-bit input is also supported, but only converts to 16-bit for now, so it's not lossless.

* Changed up the extended level options. 8192 and 16384 block sizes have now been removed as preset options. Also, I added an 8-level prediction order search (levels 9 & 11).

-Justin

[krmathis]

Pre-release 5 is now out.
* I hope the PPC endian issue is fixed. If not, please let me know.

It seems to work fine now. The track encodes, and the resulting FLAC files play fine.
Thanks!

* now on to testing this application *

[Justin Ruggles]

Hi all,

Sorry to keep posting updates so often. The online version of my FLAC encoder is now pre-release 6.

Changes are mostly header-related.
* added MD5 support! yay!
* added padding in the header to make room for metadata
* added a placeholder for a future vendor tag

I am at a good stopping point for now. My wishlist is 90% done. I will finally start work on an FFmpeg patch to integrate the current version.

Also, it would be really great to see some other peoples' test results. My computer is pretty darn slow, so I have only been able to test small files when doing everyday sort of benchmarking. I would love to see how it performs, say, on a whole CD.

Thanks,
Justin

[krmathis]

Album: Toto - Legend (in a single file)
Playing time: 72m 35s
Original File Size: 768284896 bytes
Computer specs: Apple PowerBook G4, 1.5GHz, 1GB ram, 7200rpm HDD.
Encoders: flac 1.1.2 and flake 06

CODE

flac:

level   enc time     bytes       ratio     kbps      dec time
-----  ---------   --------      -----    ------    ---------
  0     1m 50s     518214304     0.675               1m 15s
  5     3m 34s     496252171     0.646               1m 14s
  8    13m 46s     494499692     0.644               1m 26s

flake:

level   enc time     bytes       ratio     kbps      dec time
-----  ---------   --------      -----    ------    ---------
  0     3m 21s     526551866     0.685     967.2     1m 21s
  5     6m 53s     495548928     0.645     910.3     1m 30s
  8    13m 37s     492004491     0.640     903.8     1m 14s

The compression rate is comparable to flac, but it encodes noticeable slower at -0 and -5.
Anyway, you have done great work so far!

Download the Mac OS X univeral binary here: direct link

[Justin Ruggles]

Album: Toto - Legend (in a single file)
Playing time: 72m 35s
Original File Size: 768284896 bytes
Computer specs: Apple PowerBook G4, 1.5GHz, 1GB ram, 7200rpm HDD.
Encoders: flac 1.1.2 and flake 06

CODE

flac:

level   enc time     bytes       ratio     kbps      dec time
-----  ---------   --------      -----    ------    ---------
  0     1m 50s     518214304     0.675               1m 15s
  5     3m 34s     496252171     0.646               1m 14s
  8    13m 46s     494499692     0.644               1m 26s

flake:

level   enc time     bytes       ratio     kbps      dec time
-----  ---------   --------      -----    ------    ---------
  0     3m 21s     526551866     0.685     967.2     1m 21s
  5     6m 53s     495548928     0.645     910.3     1m 30s
  8    13m 37s     492004491     0.640     903.8     1m 14s

The compression rate is comparable to flac, but it encodes noticeable slower at -0 and -5.
Anyway, you have done great work so far!

Thank you for the feedback.
It looks like I have a long way to go to make level 5 comparable to the reference encoder on PPC. At least level 8 is on par.

I finally let my system run for a while to test a whole album. Here is what I got at level 8.

Smashing Pumpkins - Siamese Dream
playing time: 1h2m15.8s

flac: encode time=1h1m35.42s ratio=0.583
flake: encode time=35m43.75s ratio=0.582

I know...it really is about time for a major upgrade...I'm still holding out for 10 good years out of my '97 IBM.

Thanks,
Justin

[krmathis]

Thank you for the feedback.
It looks like I have a long way to go to make level 5 comparable to the reference encoder on PPC. At least level 8 is on par.

You are welcome!
Looks like FLAC have optimized their code for PowerPC, while you of course have consentrated on i386 only.
I guess it would be hard for you to improve the performance on PowerPC as well, since you dont have access to one. But that might not be that important since Apple are moving their line to Intel, and at the current stage they only have two PowerPC computers left.

I finally let my system run for a while to test a whole album. Here is what I got at level 8.

Smashing Pumpkins - Siamese Dream
playing time: 1h2m15.8s

flac: encode time=1h1m35.42s ratio=0.583
flake: encode time=35m43.75s ratio=0.582

I know...it really is about time for a major upgrade...I'm still holding out for 10 good years out of my '97 IBM.

Wow, you have cut the -8 encoding time on your i386 machine to half.

That you still use a 300MHz computer from '97 are just insane. My computer are only two (2) years old, and I am seriously looking for a replacement (probably next generation MacBook Pro)!

[miros]

Here are my results for four albums lying around. CPU AMD K8 1.8GHz, 64-bit OS, everything cached in memory, so the harddisk wouldn't be a bottleneck. Included is also decoding speed of another decoder that is very sensitive to higher LPC order. Versions used: flake-06 and flac CVS.

CODE

garbage / beautiful garbage

lvl|  ratio       speed (x realtime)    |  ratio       speed (x realtime)  
   |           flake   flac -d  flacdec |            flac   flac -d  flacdec
0  |  0.713     52.1    202.9    418.1  |  0.727    135.5    210.0    424.3
1  |  0.678     50.7    184.7    391.6  |  0.703    137.6    205.9    408.5
2  |  0.678     50.6    184.8    391.2  |  0.701     90.4    206.4    416.7
3  |  0.670     50.0    179.7    382.4  |  0.694     90.9    179.5    385.5
4  |  0.668     51.1    176.0    373.5  |  0.668     77.6    170.7    351.5
5  |  0.666     46.5    170.4    352.9  |  0.667     58.5    172.3    354.6
6  |  0.664     31.2    169.0    344.0  |  0.666     43.9    172.0    355.2
7  |  0.664     27.9    170.8    348.9  |  0.665      9.7    172.8    359.0
8  |  0.661     24.1    163.6    309.1  |  0.663      7.2    168.4    329.1
9  |  0.661     18.4    164.2    311.9  |  
10 |  0.661     14.6    165.3    314.6  |  
11 |  0.657     11.8    110.0    118.7  |  
12 |  0.657      4.5    114.4    122.4  |  

u2 / the joshua tree

lvl|  ratio       speed (x realtime)    |  ratio       speed (x realtime)  
   |           flake   flac -d  flacdec |            flac   flac -d  flacdec
0  |  0.589     57.0    217.9    450.5  |  0.585    141.0    223.1    448.4
1  |  0.553     55.8    194.2    422.7  |  0.573    142.7    218.2    430.5
2  |  0.553     55.3    195.3    419.7  |  0.570     92.4    219.2    438.7
3  |  0.545     55.0    189.2    411.7  |  0.554     93.3    192.1    411.7
4  |  0.542     56.4    185.7    404.0  |  0.540     79.1    181.1    374.4
5  |  0.540     50.9    179.3    376.7  |  0.539     59.3    180.7    377.6
6  |  0.535     33.1    178.2    365.3  |  0.538     44.5    181.9    378.1
7  |  0.535     29.4    180.2    372.0  |  0.536      9.7    182.6    384.9
8  |  0.533     25.3    173.5    332.3  |  0.535      7.3    180.2    360.5
9  |  0.533     19.1    175.2    336.3  |  
10 |  0.533     15.0    175.2    337.8  |  
11 |  0.530     12.2    127.7    144.5  |  
12 |  0.530      4.5    131.3    147.8  |  

smashing pumpkins / gish

lvl|  ratio       speed (x realtime)    |  ratio       speed (x realtime)  
   |           flake   flac -d  flacdec |            flac   flac -d  flacdec
0  |  0.654     54.6    211.1    434.1  |  0.635    138.9    216.9    436.2
1  |  0.599     54.1    193.0    411.7  |  0.634    140.2    213.4    425.5
2  |  0.599     54.0    192.5    411.1  |  0.633     91.4    213.4    428.1
3  |  0.582     53.6    188.4    400.4  |  0.581     93.1    190.2    404.5
4  |  0.577     54.9    183.2    392.5  |  0.575     79.0    179.6    370.6
5  |  0.571     49.8    178.0    367.7  |  0.574     59.3    181.3    372.1
6  |  0.566     32.7    178.8    365.3  |  0.573     44.6    180.7    372.1
7  |  0.565     29.0    181.1    369.1  |  0.570      9.9    181.9    378.2
8  |  0.561     25.2    175.4    332.2  |  0.568      7.4    179.6    359.2
9  |  0.560     19.0    175.4    333.8  |  
10 |  0.560     15.0    176.2    336.2  |  
11 |  0.559     12.4    149.8    183.7  |  
12 |  0.559      4.6    155.9    190.2  |  

walk the line OST

lvl|  ratio       speed (x realtime)    |  ratio       speed (x realtime)  
   |           flake   flac -d  flacdec |            flac   flac -d  flacdec
0  |  0.628     54.7    217.1    445.7  |  0.654    137.9    220.3    441.0
1  |  0.608     52.9    193.6    410.6  |  0.621    141.0    216.5    425.3
2  |  0.607     51.9    194.8    409.3  |  0.618     91.8    219.3    440.2
3  |  0.602     52.2    190.5    402.7  |  0.637     91.9    191.5    406.6
4  |  0.602     53.7    186.4    397.6  |  0.603     78.5    183.9    380.1
5  |  0.601     48.7    180.3    377.2  |  0.602     59.0    185.4    387.1
6  |  0.600     31.8    176.9    357.3  |  0.601     44.2    185.7    386.5
7  |  0.599     28.5    183.4    377.8  |  0.599      9.7    187.9    395.1
8  |  0.599     25.1    180.6    354.8  |  0.599      7.2    186.6    383.0
9  |  0.599     18.9    180.5    361.4  |  
10 |  0.599     14.9    180.1    365.1  |  
11 |  0.598     12.4    163.3    255.1  |  
12 |  0.598      4.5    167.8    263.1  |

Justin, are you planning to implement a variable blocksize support?

[Justin Ruggles]

Justin, are you planning to implement a variable blocksize support?

Thanks for the decoding benchmarks. To answer your question, yes, eventually. My current order of business is integrating the current version with FFmpeg. There has already been a very basic version committed to current FFmpeg SVN which is along the lines of using "flac -0" but with even less compression. It was more of a test-run to get things properly integrated & generating compliant FLAC files. I will reply here again when FFmpeg has competitive encoding, not just compliant encoding.

The variable blocksize support will mostly likely be after my newest, forthcoming, FFmpeg patch has been applied & optimized by more experienced FFmpeg developers.

-Justin