Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: flac: alpha support for riff chunks (Read 12181 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

flac: alpha support for riff chunks

I recently added support to flac.exe (in CVS) for storing "foreign metadata" in FLAC files when encoding and restoring when decoding.  foreign metadata is any non-audio data in the source file's container.  currently only RIFF WAVE and AIFF/AIFF-C are supported.

[attachment=3631:attachment]
each foreign chunk gets stored in a FLAC APPLICATION metadata block when the new --keep-foreign-metadata option is used for encoding.  when that option is used for decoding and the FLAC file has foreign metadata that matches the output type, it is restored.  note that it does not do container transcoding (and probably never will), e.g. if the FLAC file has RIFF chunks and the output type is AIFF, no foreign metadata is restored.

for almost all cases, using this option on both encode and decode will yield the exact same WAVE file as the original, metadata and all.  but there are a few corner cases now where the restoration process does not try to match the original metadata exactly (but it could with some improvement).  the cases are:

1. original wav was >2 channels and needed remapping to FLAC order
2. original wav is not spec compliant, e.g. 20 bps in WAVEFORMATEX; restored file will still be a compliant WAVEFORMATEXTENSIBLE
3. other wierd corner cases where the "fmt" chunk is not exactly identical due to there being multiple ways to represent the same thing
4. original aiff is in AIFF-C form with compression type "sowt" or "NONE"; restored file will always be AIFF (uncompressed) form

I need help with running the new option through the ringer before it goes into a release.  if you have any wav/aiff files with metadata, can you please try:

1. encoding with flac.exe --keep-foreign-metadata
2. decoding back to original with flac.exe -d --keep-foreign-metadata
3. comparing the decoded wav/aiff with the original, bytewise or else audio-wise
4. reporting any successes or bugs along with the output of "iffscan.exe file.wav" (iffscan.exe included in the zip file; it just prints info about the chunks in a wav/aiff file)

thanks in advance!

Josh

P.S. forgot to mention this is extremely alpha so don't use for real archiving!

(edit: uploaded new zip with bug fixes)

flac: alpha support for riff chunks

Reply #1 –
Evidently a Visual Studio debug DLL is needed (msvcrtd.dll) - I found a 6.0.xxx version online, and while iffscan.exe runs fine with it, flac.exe crashes every time straight to the standard Dr. Watson dialog box, even with regular 16/44 WAV files with no RIFF data...

I'm running XP SP2.
"Not sure what the question is, but the answer is probably no."

flac: alpha support for riff chunks

Reply #2 –
Quote from: mixminus1 on
Evidently a Visual Studio debug DLL is needed (msvcrtd.dll) - I found a 6.0.xxx version online, and while iffscan.exe runs fine with it, flac.exe crashes every time straight to the standard Dr. Watson dialog box, even with regular 16/44 WAV files with no RIFF data...

I'm running XP SP2.


I don't believe that msvcrtd.dll is legally redistributable.  I think we may need a modified re-up.

flac: alpha support for riff chunks

Reply #3 –
ack, sorry about that, will u/l a non-debug build shortly...

ok, updated download link above...

flac: alpha support for riff chunks

Reply #4 –
I created quickly two small 16/44.1 wave files. One with Audition 2 and another one with Wavelab 5.

I added something to all possible metadata fields in both programs. AFAIK, these programs use standard compliant RIFF chunks for metadata.

When I use the "--keep-foreign-metadata" switch the test encoder crashes immediately (on XP SP2).

Without the switch I get this:

Code: [Select]
U:\test\Riff_FlacTest>FLAC 02_1a.wav

flac 1.2.0, Copyright (C) 2000,2001,2002,2003,2004,2005,2006,2007  Josh Coalson
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

02_1a.wav: ERROR reading foreign metadata: invalid WAVE file: multiple "fmt " ch
unks (005)

U:\test\Riff_FlacTest>

The files are available in this rar package: Riff_FlacTest.rar

Edit: link removed

flac: alpha support for riff chunks

Reply #5 –
hmm, maybe something's bad with the windows build, both work fine for me on linux.  will check it out tonight.  thanks for the examples, that's exactly what I was looking for.

flac: alpha support for riff chunks

Reply #6 –
With alpha2, I get the exact same results as Alex B with files created with Cool Edit 2000 1.1.
"Not sure what the question is, but the answer is probably no."

flac: alpha support for riff chunks

Reply #7 –
ok, found the problem, I uploaded another zip with the fix.  sorry about that!

flac: alpha support for riff chunks

Reply #8 –
Quote from: jcoalson on
ok, found the problem, I uploaded another zip with the fix.  sorry about that!

No problemo - that's why they're called "alphas." 

Everything's good with alpha3 - two files created with Cool Edit 2000, with all metadata fields filled, compress with no problems, and when decompressed are bit-for-bit identical to the originals.

Here's the output of iffscan:

Code: [Select]
C:\>iffscan rifftest1.decode.wav
block:[RIFF] size=001aebee=(   1764334) type:[WAVE]
block:[fmt ] size=00000010=(        16)
block:[data] size=001aeaa0=(   1764000)
block:[LIST] size=00000106=(       262)
block:[DISP] size=00000014=(        20)

C:\>iffscan rifftest1.wav
block:[RIFF] size=001aebee=(   1764334) type:[WAVE]
block:[fmt ] size=00000010=(        16)
block:[data] size=001aeaa0=(   1764000)
block:[LIST] size=00000106=(       262)
block:[DISP] size=00000014=(        20)

C:\>iffscan rifftest2.decode.wav
block:[RIFF] size=001aec24=(   1764388) type:[WAVE]
block:[fmt ] size=00000010=(        16)
block:[data] size=001aeaa0=(   1764000)
block:[LIST] size=0000013e=(       318)
block:[DISP] size=00000012=(        18)

C:\>iffscan rifftest2.wav
block:[RIFF] size=001aec24=(   1764388) type:[WAVE]
block:[fmt ] size=00000010=(        16)
block:[data] size=001aeaa0=(   1764000)
block:[LIST] size=0000013e=(       318)
block:[DISP] size=00000012=(        18)
"Not sure what the question is, but the answer is probably no."

flac: alpha support for riff chunks

Reply #9 –
Tried a CD image (length 45:29, uncompressed size 459 MB) just for kicks, and this time added a small (24 KB) .bmp image in Cool Edit's Info box - no problems at all, decompressed file opened perfectly in Cool Edit with all data intact and was bit-for-bit identical to the original.

Lookin' good! 
"Not sure what the question is, but the answer is probably no."

flac: alpha support for riff chunks

Reply #10 –
This version works for me too. Here's what I tried:

Code: [Select]
U:\test\Riff_FlacTest>flac --keep-foreign-metadata -o wavelab_test.flac wavelab.
wav
NOTE: --keep-foreign-metadata is a new feature; make sure to test the output fil
e before deleting the original.

flac 1.2.0, Copyright © 2000,2001,2002,2003,2004,2005,2006,2007  Josh Coalson
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

wavelab.wav: wrote 608385 bytes, ratio=0,502

U:\test\Riff_FlacTest>flac --keep-foreign-metadata -o audition_test.flac auditio
n.wav
NOTE: --keep-foreign-metadata is a new feature; make sure to test the output fil
e before deleting the original.

flac 1.2.0, Copyright © 2000,2001,2002,2003,2004,2005,2006,2007  Josh Coalson
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

audition.wav: wrote 615715 bytes, ratio=0,508

U:\test\Riff_FlacTest>flac -d --keep-foreign-metadata wavelab_test.flac
NOTE: --keep-foreign-metadata is a new feature; make sure to test the output fil
e before deleting the original.

flac 1.2.0, Copyright © 2000,2001,2002,2003,2004,2005,2006,2007  Josh Coalson
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

wavelab_test.flac: done

U:\test\Riff_FlacTest>flac -d --keep-foreign-metadata audition_test.flac
NOTE: --keep-foreign-metadata is a new feature; make sure to test the output fil
e before deleting the original.

flac 1.2.0, Copyright © 2000,2001,2002,2003,2004,2005,2006,2007  Josh Coalson
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

audition_test.flac: done

U:\test\Riff_FlacTest>comp wavelab_test.wav wavelab.wav
Comparing wavelab_test.wav and wavelab.wav...
Files compare OK

Compare more files (Y/N) ? y
Name of first file to compare: audition_test.wav
Name of second file to compare: audition.wav
Option:
Comparing audition_test.wav and audition.wav...
Files compare OK

Compare more files (Y/N) ? n

U:\test\Riff_FlacTest>iffscan wavelab_test.wav
block:[RIFF] size=00128d68=(   1215848) type:[WAVE]
block:[fmt ] size=00000010=(        16)
block:[LIST] size=000001de=(       478)
block:[bext] size=00000520=(      1312)
block:[smpl] size=00000024=(        36)
block:[inst] size=00000008=(         8)
block:[PAD ] size=00000882=(      2178)
block:[data] size=00127d70=(   1211760)

U:\test\Riff_FlacTest>iffscan wavelab.wav
block:[RIFF] size=00128d68=(   1215848) type:[WAVE]
block:[fmt ] size=00000010=(        16)
block:[LIST] size=000001de=(       478)
block:[bext] size=00000520=(      1312)
block:[smpl] size=00000024=(        36)
block:[inst] size=00000008=(         8)
block:[PAD ] size=00000882=(      2178)
block:[data] size=00127d70=(   1211760)

U:\test\Riff_FlacTest>iffscan audition_test.wav
block:[RIFF] size=0012a9da=(   1223130) type:[WAVE]
block:[fmt ] size=00000010=(        16)
block:[data] size=00127d70=(   1211760)
block:[LIST] size=00000216=(       534)
block:[cue ] size=0000001c=(        28)
block:[LIST] size=00000052=(        82)
block:[PAD ] size=00000dea=(      3562)
block:[SyLp] size=000000b8=(       184)
block:[smpl] size=00000024=(        36)
block:[bext] size=00000520=(      1312)
block:[cart] size=00000802=(      2050)
block:[DISP] size=00000006=(         6)
block:[_PMX] size=00000d6c=(      3436)
block:[DISP] size=00000010=(        16)

U:\test\Riff_FlacTest>iffscan audition.wav
block:[RIFF] size=0012a9da=(   1223130) type:[WAVE]
block:[fmt ] size=00000010=(        16)
block:[data] size=00127d70=(   1211760)
block:[LIST] size=00000216=(       534)
block:[cue ] size=0000001c=(        28)
block:[LIST] size=00000052=(        82)
block:[PAD ] size=00000dea=(      3562)
block:[SyLp] size=000000b8=(       184)
block:[smpl] size=00000024=(        36)
block:[bext] size=00000520=(      1312)
block:[cart] size=00000802=(      2050)
block:[DISP] size=00000006=(         6)
block:[_PMX] size=00000d6c=(      3436)
block:[DISP] size=00000010=(        16)

U:\test\Riff_FlacTest>
Unfortunately my audio editors seem to not save metadata to AIFF files so I can't help with that format.

flac: alpha support for riff chunks

Reply #11 –
I just remembered that iTunes can tag AIFF files. I think it uses Apple's own variant of id3v2.

I tagged an AIFF file with iTunes 7.2. I wrote something to all tag fields and added a small jpg file too. As you explained, looks like preserving this type of metadata is not supported in your test version, but perhaps you could add support for it.

In addition, without the "foreign metadata" switch the encoding/decoding process seems to produce erratic results when an iTunes tagged AIFF file is used as a source. The test version of flac.exe crashes on decoding.

Here's the test file: itunesaiff.rar

[!--sizeo:1--][span style=\"font-size:8pt;line-height:100%\"][!--/sizeo--]Edit: link removed to make room on my server...[/size]

flac: alpha support for riff chunks

Reply #12 –
I'm not sure if this applies to this, but I tried a file that I was getting this error on:

ERROR: WAVEFORMATEXTENSIBLE chunk with unsupported channel
mask=0x003B

when I tried it with this alpha (without --keep-foreign-metadata) I got this error:

ERROR reading foreign metadata: invalid WAVE file: multiple
"fmt " chunks (005)

and with the --keep-foreign-metadata switch I got the same WAVEFORMATEXTENSIBLE chunk error.

The file is a 24bit 96kHz 5 channel file.

Iffscan says this:

Code: [Select]
block:[RIFF] size=0593e98c=(  93579660) type:[WAVE]
block:[fmt ] size=00000028=(        40)
block:[data] size=0593e950=(  93579600)

flac: alpha support for riff chunks

Reply #13 –
My wave files are edited in Sound Forge 8.0 where I use markers to mark regions. They are successfully preserved.

Code: [Select]
C:\flac-experimental>iffscan.exe test.wav
block:[RIFF] size=05cfc618=(  97502744) type:[WAVE]
block:[fmt ] size=00000010=(        16)
block:[data] size=05cfbc7c=(  97500284)
block:[cue ] size=000005a4=(      1444)
block:[LIST] size=000003c4=(       964)

C:\flac-experimental>iffscan.exe test-dec.wav
block:[RIFF] size=05cfc618=(  97502744) type:[WAVE]
block:[fmt ] size=00000010=(        16)
block:[data] size=05cfbc7c=(  97500284)
block:[cue ] size=000005a4=(      1444)
block:[LIST] size=000003c4=(       964)

flac: alpha support for riff chunks

Reply #14 –
my quick test, appears fine;

iffscan orig.wav
block:[RIFF] size=009ef824=(  10418212) type:[WAVE]
block:[fmt ] size=00000010=(        16)
block:[data] size=009ef800=(  10418176)

iffscan decoded.wav
block:[RIFF] size=009ef824=(  10418212) type:[WAVE]
block:[fmt ] size=00000010=(        16)
block:[data] size=009ef800=(  10418176)

fc orig.wav decoded.wav
Comparing files orig.wav and DECODED.WAV
FC: no differences encountered
PANIC: CPU 1: Cache Error (unrecoverable - dcache data) Eframe = 0x90000000208cf3b8
NOTICE - cpu 0 didn't dump TLB, may be hung

flac: alpha support for riff chunks

Reply #15 –
thanks for the feedback so far everyone.

one more thing to mention: I found out that sounddevices is already using the "RIFF" app ID so I will be changing the case of the app IDs I use officially in the release version to avoid a conflict.