Hi All

I've been having a few seeking problems with flac files and think some of the problem may be to do with flac.exe

Context:
I rip CDs using EAC in image+cue mode, converting to flac using foobar2k

I had seeking problems in Winamp2 and Musicbox (linux jukebox software). The problem was solved by adding 1 second seekpoints using metaflac.

On new rips, I use foobar2k's cliencoder with flac.exe - for adding seekpoints during the conversion I use the switch:
--seekpoint=1s

The resulting flac files also have quite major seeking problems. On comparing the seek table generated by flac.exe and metaflac.exe, they are very different. If I remove the seektable using metaflac, and then add it back (again using metaflac), the files play with no seeking problems.

The seektable on a typical album is much bigger when using flac.exe. I can't remember how many points it has as I don't have the files in front of me, but it's roughly about 3-4 times as big as the metaflac generated seektable.

Has anyone else come across this or can anyone else confirm this? flac files really need the seekpoints to make them useable, but i don't want to go through a 2 stage process in creating them (mainly because I don't know how to configure foobar2k to do a 2 stage process).
I used to use .ape, but have switched to flac recently as the linux jukebox software mentioned above can't use .ape

Maybe Josh knows if there is a difference in the code between flac and metaflac.

Thanks
-Audio Spyder

you're probably on to something. can you do 'metaflac --list' on both files so I can see the difference? theoretically the only difference should be placeholder points in the first file. also, show the exact flac and metaflac command-lines you used to make the two files. thanks.

Josh

sure, I've got the --list output saved to a text file at home. I can try and upload them here and will also let you know the exact commands used in each case.

one more thing to note... according to the seeking bug on sourceforge the problem only manifests on FLACs created with the windows flac.exe, not on unix, which may explain why I have never had any seeking problems with FLAC myself.

that gives me another idea... AudioSpyder, do you have a linux box on which you could run 'metaflac --list' on the same two files? if they give different answers then that could point to a bug on the windows side while parsing the metadata.

another thing to note is that the worst thing that could be wrong is a bad seektable and/or bad seek algorithm, both correctable. the underlying audio is fine.

Josh

I've now run a fairly extensive set of tests on windows and linux and it looks like there is a difference between the parsing of metadata.
I'll post a file containg the results here later today.

-Audio Spyder

ok - here are my findings.

-----------------------------------------------------
Background:

Flac files were produced on a WinXP box or Slackware Linux box, with files being stored on a WinXP shared folder

I also used foobar2k v0.8beta8 as a comparison to using the command line tools (flac.exe)

-----------------------------------------------------
File naming convention:

Text files in the (hopefully) attached zip file are named as follows:
[does anyone know how to attach a zip file to a post? I've cut and paste some of the data, but Josh may need to see more]

X_aaa_bbb.txt

X: "W" for files created using metaflac.exe on Windows and "L" using metaflac on Linux
aaa: "win" for Windows versions, "lin" for Linux versions, "foobar" for using foobar2k
bbb: "flac" if the seektable was created using flac, "meta" if it was created using metaflac - I'll come back to the foobar ones later

When encoding using flac.exe or flac, the command line was:
flac test.wav

------------------------------------------------------
Output from the flac when it finished encoding:

Linux:
options: -P 4096 -b 4608 -m -l 8 -q 0 -r 3,3
test.wav: wrote 523020382 bytes, ratio=0.632

Windows:
options: -P 4096 -b 4608 -m -l 8 -q 0 -r 3,3
test.wav: wrote 523060608 bytes, ratio=0.632

The interesting thing here is the different sizes although the input file was the same.

------------------------------------------------------
Command lines used:

when using metaflac to add seekpoints, I used a flac file with no seektable:
metaflac --add-seekpoint=1s filename.flac

Using metaflac to list the metadata block:
metaflac --list filname.flac > X_filename.txt

------------------------------------------------------
Flac files which give a seek error in Winamp:

foobar_flac.flac
foobar_cli_seek.flac
win_flac.flac

i.e. only files created in Windows (but not all of them).

------------------------------------------------------
Results: - since I can't attach the files, I've cut a few lines of the metaflac output.

It looks like windows metadata parsing is doing something different with stream_offset and frame_samples.

Also, there is a difference in the windows flac generated seektable to the linux flac generated seektable

-------------
flac generated seektable

W_win_flac.exe
METADATA block #1
type: 3 (SEEKTABLE)
is last: false
length: 8424
seek points: 468
point 0: sample_number=0, stream_offset=0, frame_samples=0
point 1: sample_number=437760, stream_offset=0, frame_samples=829014

L_win_flac.exe
METADATA block #1
type: 3 (SEEKTABLE)
is last: false
length: 8424
seek points: 468
point 0: sample_number=0, stream_offset=0, frame_samples=4608
point 1: sample_number=437760, stream_offset=829014, frame_samples=4608

W_lin_flac.flac
METADATA block #1
type: 3 (SEEKTABLE)
is last: false
length: 8424
seek points: 468
point 0: sample_number=0, stream_offset=0, frame_samples=0
point 1: sample_number=437760, stream_offset=0, frame_samples=829097

L_lin_flac.falc
METADATA block #1
type: 3 (SEEKTABLE)
is last: false
length: 8424
seek points: 468
point 0: sample_number=0, stream_offset=0, frame_samples=4608
point 1: sample_number=437760, stream_offset=829097, frame_samples=4608


----------------
metaflac generated seektable

W_win_meta.flac
METADATA block #1
type: 3 (SEEKTABLE)
is last: false
length: 84384
seek points: 4688
point 0: sample_number=0, stream_offset=0, frame_samples=0
point 1: sample_number=41472, stream_offset=0, frame_samples=10086

L_win_meta.flac
METADATA block #1
type: 3 (SEEKTABLE)
is last: false
length: 84384
seek points: 4688
point 0: sample_number=0, stream_offset=0, frame_samples=4608
point 1: sample_number=41472, stream_offset=10086, frame_samples=4608

W_lin_meta.flac
METADATA block #1
type: 3 (SEEKTABLE)
is last: false
length: 84384
seek points: 4688
point 0: sample_number=0, stream_offset=0, frame_samples=0
point 1: sample_number=41472, stream_offset=0, frame_samples=10086

L_lin_meta.flac
METADATA block #1
type: 3 (SEEKTABLE)
is last: false
length: 84384
seek points: 4688
point 0: sample_number=0, stream_offset=0, frame_samples=4608
point 1: sample_number=41472, stream_offset=10086, frame_samples=4608

------------------------------------------------------
Foobar results:

Foobar seems to do something very strange when encoding flac files.
I tried in 3 ways:
1) use the cliencoder and flac.exe, with default options
- -0 %d%
2) use the cliencoder and flac.exe, with a seektable command line switch
- --seekpoint=1s -o %d%
3) use the FLAC encoder, default options

I hope I've given the correct switches here as I'm writing them down from memory, but the general structure is correct I think

results:
1) creates a file with a seektable
2) creates a file with a very big seektable (much bigger than the equivalent seektable generated by metaflac)
3) creates a file with no seektable

------------------------------------------------------
I hope I've explained things clearly here and it helps nail the seeking problem.

-Audio Spyder

Edited for some formatting problems

You can upload a zip-file to the Upload forum, and then link to it from here.

hmm, from this it looks like it is either:

1. a bug parsing the seekpoints in libFLAC on windows
2. a bug printing the seekpoints in metaflac on windows

I'm guessing it's #2 since seeking would be totally broken if #1 (because all stream_offsets seem to be 0). if it's #2, it still wouldn't explain the other bug of not being able to seek to samples at n*44100.

I'll step through this tonight.

Josh

ancl - thanks for the link, exactly what I was looking for.
I've now uploaded the results, follow this link:
zip file of results

Okey

OK, I found and fixed two bugs.

the first is only on windows. it explains why the seektable looks wrong with 'metaflac --list'. but the second bug is the real cause of I believe all the seeking problems so far reported (see also bug #851155).

the first bug is that the metaflac code was using printf() w/ %llu for printing out unsigned long long ints (ANSI C), which MSVC6 does not support. it was treating them like unsigned int and screwing up. MSVC6 takes a non-standard %I64u, so I fixed it in CVS.

so the seektables in the FLAC file are fine, and are parsed fine, they just aren't printed properly by metaflac 1.1.0 on windows.

the second bug is much more subtle. what is basically happening is that in some sporadic cases, the byte offset where libFLAC calculated the seek target to be happened to land in front of audio data that looked exactly like a frame header from a future version of FLAC. this is rare since the frame header also has a CRC (I've never run into it myself) but it can happen. the workaround of 1-second seekpoints is consistent with this, since the farther apart seek points are spaced, the more likely for this to happen.

so to the seek routine it looked like it landed in a valid spot in the middle of a FLAC stream it didn't know how to parse, and bailed out. I have fixed it to not do this and that has fixed all the reported cases that I have test data for.

I'm contemplating a bug fix release since I'm still trying to finish up the Ogg FLAC work and it probably shouldn't wait until then.

Josh

Excellent news. Your description of the second bug explains why only some flac files exhibited a seeking problem.
Thanks for looking into this, your efforts are really appreciated.

- Audio Sypder

Thanks for fixing this! I just rebuilt flac from CVS and I can now seek in several files that were previously problematic.

Josh, I am curious about something. I've always encoded using --best (or -8), and the default 10s seektable interval has always seemed sufficient, since seeking still works - or appears to work - as expected, and I can seek to any point in the file. In his initial post, Kjetil Limkjær said the problem disappears when he uses -8... so is there still a chance I could encounter this bug?

So far, the only time I've ever received an error of any sort from FLAC is when I've tried piping it to SoX. For example, using Speek's Batchenc the following line should downmix a stereo source to mono:

But instead, it produces the following message:

The test FLAC in question was encoded from CD-quality (16 bit, 44.1 kHz stereo) audio, using the FLAC 1.1.0 reference encoder.

- M.

there's still a chance. it will happen any time something that looks like a valid but unparseable frame header (i.e. uses reserved bits) occurs somewhere in the middle of the audio data and a seek lands on it.


does it happen for any FLAC file or just one particular file? I would also expect it to give the same error if you did

Josh

This problem seems to be universal, although it only manifests itself in conjunction with SoX. For example, the following line works perfectly well in Batchenc:
I've now tested the FLAC/SoX line with some forty-odd files, all of which came from different sources and were encoded at different times. Same result, every time.

- M.

Josh or other FLAC enthusiasts,

Sorry for resurrecting this post but this is important for me at this point in time.

I read this post over and over and it seems that the encoding of FLAC 1.1.0 was OK. Is this true?

On one hand we have 2 bugs: one which is is just a printout probem of metaflac, another which has something to do with seeking. This latter one implies decoding. So it seems that encoding was fine.

On the other hand Audio_Spyder pointed out that the linux filesize differs from the Windows makes me worry that I misunderstood the thread. It just makes me feel that in the end this was a encoder bug.

I encoded in Windows by EAC with this command line



and I must admit I never experienced the bug, but I rarely seek. I just want to know if I need to reencode all my music collection now to correct this or indeed encoding was fine. In the latter case could somebody explain why the filesize difference.

Many thanks,

Triza

Hello again,

I do not want to look pedantic here, but please bear with me because the question is to reencode my 250 CD flac correction.

I looked up the related defect in Sourceforge and it seems that files encoded on Linux does not exhibit this problem. Josh said this was a fluke, but mnetown says



So things do not add up to me at least. Also I think the flac files should be identical regardless whether we encode in Linux or Windows unless there is a reference in the flac file that tells the OS type or something

Many thanks for possible answers.

Triza

I encoded with flac 1.1.0 on Solaris 9 (UltraSparc), NetBSD 1.6 (i386), and Windows 2000 (i386) and got the same result each time - by checking with metaflac --list on each platform.

The MD5 signature and seektable was the same everywhere.

But you should still run flac -t against your collection if you're still worried.

--jth

Hi jth,

Thanks. I am in the process to set up a Linux machine as we speak and I am gonna do some tests.

BTW I was worried about the seektable only. I know that the actual music is fine because I encoded with --verify option and also decoded it and compared it against EAC CRC too. I just want to make sure that my collection is in perfect shape.

Triza

Several people around here have experienced bugy seeking with flac when playing cue sheet single file rips. MOst changed to Monkey's audio, which is also free. The other alternative is to rip to separate files. I keep asking why anyone would want to compress an entire album to a single file, as opposed to separate files plus a cue sheet, but have not been able to make sense of the answers.

well, if your player supports cue input, then it doesn't make much of a difference, does it?

...especially if your burner software supports cue/wav --> = perfect replication of CD if done right


later

ah, maybe sox is closing the pipe after thinking it's got all the wave data. that would be an unrelated problem.



there is no encoding problem. if you read the thread carefully, the difference in filesizes is because he used different ways of adding the seektables in some of the tests.

furthermore, it is not required for a file to encode to the exact same bit pattern on two different platforms. there can be differences because of, for example, different floating point implementations. that doesn't mean there is anything wrong with the encoder or encoding.

Josh

I don't understand that one at all. I'm not doubting that it may be true (I have no idea), but switching to Monkey's Audio from FLAC because of this "maybe" seeking problem (that seems, from what I've read, not to be encoder related) seems crazy.

I'm not trying to be defensive of some kind of apostle here, but the analysis that led me to choose FLAC over MA in the first place would make switching to MA unacceptable.

MA's advantages:
- slightly better compression and file sizes (the better compression rarely exceeds 10mb or so, and on a 380mb file 10mb hardly seems like a substantial gain)
- sometimes encodes faster than FLAC (like 12 vs. 13 minutes or something, so not much of an advantage if you plan on ripping an album once)

MA's disadvantages:
- substantially longer decode than FLAC (FLAC is like 100% faster or something)
- more difficult to decode than FLAC (meaning portables that use it, if and when there are any, will have shorter battery life when playing MA files)
- MA doesn't stream as far as I know
- MA has no hardware support, limited player support
- MA isn't open -source (before you counter "but its available to see" it isn't the same thing as open source)

Someone correct me if I'm wrong or missed one of MA's notable strengths somewhere. I just don't understand why MA is still even viable as an option with FLAC on the table, but I'm no expert so maybe I'm missing something.

-rt

@rt

I base my statement about Monkey's on the posts (some here and some elsewhere) of several other HA members, at least two of which were developers. Your comparison of flac and MA is correct, as far as I can remember. However, the seeking problem is real when using large full album flac files with cue sheets. With flac files of 17 minutes duration and no cue sheet, I could not find a problem with seeking.

Personally, I don't think that lossless compression makes sense on a portable, even one with a 40 gig drive. Open source is nice, but only if you need to hack the code & know how to do it.

However, after reading some threads on using a single file and a cue sheet for an album, I can't see why some like it, be it Monkey's or flac.

I sometimes use flac myself to do a quick and dirty (burst mode) rip of an album to my hard drive so I can listen to it while leaving my CD/DVD burner free for other things. The P4 compile of flac runs faster on this machine than Monkey's does. Separate files for each track, so seeking is not an issue. These don't stay around too long.

As far as wanting another 10mb of compression on a 380mb file is concerned, making a big deal about small differences is definitely a HA trait.

MA supports embedded cuesheets with PERFORMER and TITLE information.

I wasn't questioning your point, just kind of amazed at the fact. Maybe MA is attractive to all the PETA and animal lovers on HA.

As for using FLAC on a portable you are entirely right; with current HD densities and FLAC file sizes it isn't really practicaly to use a lossless codec on a DAP. I actually was originally attracted to the Rio Karma because I thought I would use AFLAC on it. However, after finding I could only fit 40 or so albums on a 20gb drive I switched to OGG for use on the portable (at 192kbps I can fit just about my whole collection on the Karma).

That said, I CAN easily see a day in the future when DAP drive densities will increase to the point where carrying lossless music around will be completely practical. At the current rate I would guess we'll see the first 100gb mini-drive inside of 2-3 years or so, and then you have the MASSIVE storage options we'll see when all the holographic media research begins to pay off (figure 5-10 years).

So down the road having a lossless archive will leave you well situated to take advantage of evolving tech. That said, I do still have a current use for FLAC as the "core" of my music system. I keep a full FLAC archive stored on a network drive for use on all my home devices, including my computer, my stereo, etc.

To the future, HAZAH!

-rt

Is that as opposed to external CUE sheets which I'm assuming (have no idea) you need to use with FLAC? Don't all CUE sheets have the ability to include the normal ID data for the album or files?

Seems like a gross oversight if not, and even Performer/Title seems kinda limited (year? genre?). I mean if you can't include normal ID data for the albums I can't imagine I would ever elect to go with a single file over separates. Doesn't that really suck when playing back the albums in something like Winamp?

-rt

Yes, that's as opposed to external cuesheets. I'm just going to refer you to the FLAC FAQ for more information.

I like to have a single FLAC file for archival purposes. It plays back perfectly in foobar2000 (my preferred player). I don't know about Winamp and others.

Hi Eagleray,

Your comment is interesting. I thought I did not experience the bug probably because I rarely seek, but perhaps because I store every track in a different file.

I also would like to thank for Josh for the clarification. I still surprised that Audio_Spyder did not have problems when he encoded several problematic samples in Linux, but in a way you explained that the FLAC file format have some flexibility in the format (floating point representation) so I take it that the variant used in Linux just happens not to exhibit this problem.

A bit of digression

I am in Eagleray camp full-heartedly. This one file per disk seems to be totally unnecessary and considering the difficulties with it I do not get what people wish to achieve. I have every file as a separate track without removing the gaps (gaps appended at the end of the track) and I also save the CUE sheet although I know that apart from the totally useless INDEX00 info I could resurrect the CD correctly even without CUE sheet.

Finally I did very careful consideration when I chose FLAC and the little bit of bigger filesize does not bother me. Robustness, free open SW, and the principles such as it remains pure lossless and free from DRM and thorough testbed for regression testing and fast decoding is what is important. Good work, Josh. Just please keep it pure as it is.

I don't think MA supports that, it's a feature of the fb2k plugin isn't it? the same thing could be done with FLAC.

Josh

True...it is a fb2k feature. But FLAC won't let me store a cuesheet with that information so what tag would fb2k read?

FLAC doesn't prevent it. tags are only one type of metadata in FLAC. cuesheet is another, and that doesn't store title/artist/etc. but you could put the cuesheet in the tags the same way they are put in MA tags.

Josh

It may be difficult with the native FLAC framing, but it certainly istn when Ogg or MKA are used as containers. Personally i dont have any use for single songs, as i will always listen to a complete album ( maybe because of my preferred music styles, Jazz and Classic ) and for me the one file = one album solution is clearly the way to go.
Please leave this decision to each user, and avoid to generalize based on your personal preferences ....

ChristianHJW,

I always listen to complete albums and almost never individual tracks. I have no compilations as of today not even in a playlist form. Yet I use separate file for each track. Personally I do not want to bother with Ogg or MKA container. Rather I would like to keep it simple stupid (KISS) in FLAC. With my comment I try to help. Ogg and MKA might be nice, but it has more limited support than plain FLAC and I am convinced that a large number of newbies who constantly bombard HA with the same question would benefit if somebody would tell them that one file per album has no advantages, but it has disadvantages. Even you failed to name one (real) advantage.

Triza