New FLAC encoder Options

[Nayru]

A new FLAC encoder has been developed, which is proposed for inclusion in ffmpeg. The author claims that in high-compression mode, it can compress certain files better than libFLAC.

http://mplayerhq.hu/pipermail/ffmpeg-devel...May/011385.html

[kurtnoise]

Hi Justin,

Thank you very much for this work (fork ?)...

I've made a benchmarck on my AMD64 3200+. I've tested the last flake 0.2 and flac from the last cvs entry with a wav sample of 1h11m55s (mostly electro and pop rock).

Here are my results:

1°) Flake 0.2 compiled with gcc 4.1.0 & CFLAGS = -g -O3 -Wall -pedantic -std=c99 -march=k8

CODE

|  level  |   Real Time   |       Bytes        |  Ratio   |  kbps |
|    0     |  2m11.511s  |  530 951 235  |  0,697  | 984   |
|    1     |  2m34.031s  |  513 566 686  |  0.675  | 952   |
|    2     |  2m34.206s  |  499 679 481  |  0,656  | 926   |
|    3     |  2m36.720s  |  491 472 721  |  0,645  | 911   |
|    4     |  2m35.962s  |  488 370 918  |  0,641  | 905   |
|    5     |  2m50.350s  |  487 493 404  |  0,640  | 904   |
|    6     |  2m51.587s  |  487 565 406  |  0,640  | 904   |
|    7     |  2m50.688s  |  490 601 577  |  0,644  | 910   |
|    8     |  4m13.558s  |  490 820 897  |  0,644  | 910   |

2°) FLAC CVS 20060603 compiled with gcc 4.1.0

CODE

|  level  |   Real Time   |       Bytes        |  Ratio   |  kbps |
|    0     |  1m16.089s  |  531 245 719  |  0,698  | 984   |
|    1     |  1m20.170s  |  510 802 749  |  0.671  | 947   |
|    2     |  1m25.361s  |  505 186 083  |  0,663  | 940   |
|    3     |  1m32.409s  |  501 361 052  |  0,652  | 933   |
|    4     |  1m35.588s  |  489 589 293  |  0,643  | 908   |
|    5     |  1m50.881s  |  488 073 198  |  0,641  | 905   |
|    6     |  2m04.660s  |  487 691 205  |  0,641  | 904   |
|    7     |  6m59.237s  |  486 851 452  |  0,640  | 903   |
|    8     |  9m00.266s  |  485 704 730  |  0,638  | 901   |

Now, I've a question : why flake crashes during encoding on win32 plateform ? In fact, compilation is fine but running fails. Any idea ?

This post has been edited by kurtnoise: Jun 3 2006, 16:53

[Justin Ruggles]

Thank you very much for this work (fork ?)...

You're welcome. Fork? Well, I suppose that if I get the LPC order selection code replaced with my own then I can claim it to be nearly non-derived . The Levinson-Durbin LPC generation code is nearly identical to that in several other programs I've seen. I copied the UTF-8 code basically because I'm not 100% clear on how that encoding works. I suppose I should separate these 3 or 4 functions into a single file with proper copyright notice and attribution...or at least put a small notice above each and include both a COPYING.libFLAC file and COPYING.lgpl file. It wasn't really a fork though. I basically started from scratch using the specs and some reused FFmpeg code. I even tried my own UTF-8 function working backwards from the decoding in FFmpeg...I could never get it to work though, so I copied in that function. When adding non-fixed LPC I tried my hand at that as well, but fell short. So it's more of an independent implementation with crutches. The structure is my own, but the puzzle had to be filled-in in a few places.

I've made a benchmarck on my AMD64 3200+. I've tested the last flake 0.2 and flac from the last cvs entry with a wav sample of 1h11m55s (mostly electro and pop rock).

Here are my results:

1°) Flake 0.2 compiled with gcc 4.1.0 & CFLAGS = -g -O3 -Wall -pedantic -std=c99 -march=k8

| level | Real Time | Bytes | Ratio | kbps |
[...]
| 5 | 2m50.350s | 487 493 404 | 0,640 | 904 |

2°) FLAC CVS 20060603 compiled with gcc 4.1.0

| level | Real Time | Bytes | Ratio | kbps |
[...]
| 8 | 9m00.266s | 485 704 730 | 0,638 | 901 |

I also noticed similar results (not quite as good as libFLAC) on samples where flake did the best with smaller block sizes (i.e. level 5, which is blocksize 2048), but that the reference encoder did best at level 8 (blocksize 4608). I am still trying to wrap my head around this issue. It is possibly due to whatever libFLAC does in "exhaustive" mode? Or maybe my version of stereo decorrelation works well only on audio with certain characteristics? Testing, testing, and more testing. I'm working on my own horribly slow "brute-force" mode to help me out in my tests.

Now, I've a question : why flake crashes during encoding on win32 plateform ? In fact, compilation is fine but running fails. Any idea ?

I really have no clue here. What does the crash look like? Is it a segfault? Any gdb info? If I had Windows I would test it myself. If MinGW is being used, maybe it needs "--enable-memalign-hack"...? It has been several..5..6.. years since I have done any gcc compilation in Windows.

-Justin

[kurtnoise]

I really have no clue here. What does the crash look like? Is it a segfault? Any gdb info? If I had Windows I would test it myself. If MinGW is being used, maybe it needs "--enable-memalign-hack"...? It has been several..5..6.. years since I have done any gcc compilation in Windows.

Through wine, here is the debug :

lionel@kubuntu:~/flake$ wine ./flake.exe -2 /home/lionel/wav/input.wav /home/lionel/wav/output.flac
Flake: FLAC audio encoder
Signed 16-bit 44100 Hz stereo
samples: 190288560
wine: Unhandled page fault on write access to 0x7f9a0dd4 at address 0x405703 (thread 0009), starting debugger...
WineDbg starting on pid 0x8
Unhandled exception: page fault on write access to 0x7f9a0dd4 in 32-bit code (0x00405703).
Register dump:
CS:0073 SS:007b DS:007b ES:007b FS:1007 GS:0033
EIP:00405703 ESP:7fbafdcc EBP:7fbafdd8 EFLAGS:00210206( - 00 - RIP1)
EAX:001f4514 EBX:ffffffff ECX:7f9a0dd4 EDX:7fd085e0
ESI:00401210 EDI:7ffdf760
Stack dump:
0x7fbafdcc: 7fcf0020 004018f6 ffffffff 7fbafeb8
0x7fbafddc: 00403306 7fd085e0 7fbafe60 0b5792b0
0x7fbafdec: 00402fd4 7fcf0020 00000000 7fcf0020
0x7fbafdfc: 00402fcf 7fd08c38 7fd0945c 7fbafe28
0x7fbafe0c: 00000010 7fcf0000 00000000 7fd084b0
0x7fbafe1c: 00000000 7fcf0000 7fc946dc 7fbafe48
0200: sel=1007 base=7fe6c000 limit=00001fff 32-bit rw-
Backtrace:
=>1 0x00405703 in flake (+0x5703) (0x00405703)
2 0x00403306 in flake (+0x3306) (0x00403306)
3 0x004011d9 __mingw_CRTStartup+0xc9 [/build/buildd/mingw32-runtime-3.8/build_dir/src/mingw-runtime-3.8/crt1.c:226] in flake (0x004011d9)
4 0x00401223 in flake (+0x1223) (0x00401223)
5 0x7fc5b311 in kernel32 (+0x4b311) (0x7fc5b311)
6 0xb7ed8ddb wine_switch_to_stack+0x17 in libwine.so.1 (0xb7ed8ddb)
0x00405703: orl $0,0x0(%ecx)
Modules:
Module Address Debug info Name (19 modules)
PE 0x00400000-00438000 DIA flake
ELF 0x7bf00000-7bf03000 Deferred <wine-loader>
ELF 0x7f93f000-7f9a0000 Deferred msvcrt<elf>
\-PE 0x7f950000-7f9a0000 \ msvcrt
ELF 0x7fbee000-7fcf0000 Export kernel32<elf>
\-PE 0x7fc10000-7fcf0000 \ kernel32
ELF 0x7fe07000-7fe11000 Deferred libnss_files.so.2
ELF 0x7fe11000-7fe26000 Deferred libnsl.so.1
ELF 0x7fe3a000-7fe5c000 Deferred libm.so.6
ELF 0x7fe63000-7fe6c000 Deferred libnss_nis.so.2
ELF 0x7fe70000-7ff66000 Deferred libwine_unicode.so.1
ELF 0x7ff66000-7ffe0000 Deferred ntdll<elf>
\-PE 0x7ff80000-7ffe0000 \ ntdll
ELF 0xb7d71000-b7d7a000 Deferred libnss_compat.so.2
ELF 0xb7d7b000-b7d7e000 Deferred libdl.so.2
ELF 0xb7d7e000-b7ead000 Deferred libc.so.6
ELF 0xb7ead000-b7ebf000 Deferred libpthread.so.0
ELF 0xb7ed4000-b7eee000 Export libwine.so.1
ELF 0xb7ef1000-b7f07000 Deferred ld-linux.so.2
Threads:
process tid prio (all id:s are in hex)
00000008 (D) Z:\home\lionel\flake\flake.exe
00000009 0 <==

Hope this helps....


About fork. It was just a joking. Keep up the good work.

[Justin Ruggles]

Through wine, here is the debug :

lionel@kubuntu:~/flake$ wine ./flake.exe -2 /home/lionel/wav/input.wav /home/lionel/wav/output.flac
Flake: FLAC audio encoder
Signed 16-bit 44100 Hz stereo
samples: 190288560
wine: Unhandled page fault on write access to 0x7f9a0dd4 at address 0x405703 (thread 0009), starting debugger...
WineDbg starting on pid 0x8
Unhandled exception: page fault on write access to 0x7f9a0dd4 in 32-bit code (0x00405703).

It's kinda hard to tell without more symbol names, but my thoughts still go to memory alignment in the structs. I tried rearranging the structs so that non-DWORD values are always at the end. Maybe that will help? If that's even the issue. It seems to be happening in the small section in flake.c around flake_encode_init. This is when the flac encoder structures are first accessed. Some of those values are pointers, so if the alignment is off, the pointers could end up pointing to unaccessible memory, which could cause a page fault.

-Justin

This post has been edited by Justin Ruggles: Jun 4 2006, 08:28