A post over in the lossless section made me remember a mysterious issue I ran into a year and a half ago when encoding with Ogg Vorbis that I am just getting back to. I was having random but reproduceable occurances of bit and/or filesize differences when encoding the same set of FLACs into multiple redundant sets of Ogg Vorbis files. I ruled out bad memory and CPU overclocking of my computer as I tested and obtained similar results on other computers. I was using a Pentium 4 optimized version of OggDropXPd based on vorbis Post Release 1.0.1 (I know it is an old one, but I don't think the version really matters). John33 couldn't reproduce the differences on his own, but I was never able to get my test files to him. He even compiled a version of the program for me that would forcibly restrict the serial number of every outputted file to a static value in batch mode so that this header information would not change and throw off bit differences. I have gone back and can still reproduce this, and would like other people's input.

To be specific, I am identically encoding two or more times a set of FLACs into Ogg Vorbis files with NO metatags passed and am forcing them to all have the same serial number for when comparing Vorbis to Vorbis. Then I am decoding each set of Vorbis files into WAVs and I compare each set of WAVs bit for bit to the others with a comparison program. I also compare the Vorbis files to each other. I will sometimes have all the files be off and then other times a random portion of them are off and the rest identical. The random files that are different as Vorbis files are the same ones that are different as WAV files, which means that I am getting random differently encoded audio, not just header differences.

So, as far as just the Vorbis files, is there other header information besides the serial number that will change per repeated encode of the same FLAC file? And, with regards to the WAV files, is there any minute part of the Q profiles or some sort of random value rounding that occurs at encode time that would create random results like this?

No takers? I'll try to shorten the original message soon if I still get no responses.

Thanks

hmm... I have never noticed any filesize differences when encoding 1 test file multiple times with the same settings... haven't tried doing a bitwise comparision though, I'll give it a go though and see what happens.

Does this happen with any file you use for input?

Yep. You should be able to produce it with any file...but, it could be that I have found some that are MORE likely...I actually have a complete test case set up on disc for others to try out, but over the internet, I don't currently have to bandwidth or server space to allow people to download it...it is 124 MB.

I did some further testing with the help of John33. He compiled three new test binaries of OggDropXPd with various codec versions that would force a static serial number. I received the same results on two of the three. (aoTuVb4.51 and standard 1.1.2)

The third compile, however, is 1.1.2MOD, by QuantumKnot, and, while I didn't do a completely thorough test, it was the only one that produced the same bit identical files every time. QuantumKnot, do you have any ideas as to why your 1.1.2MOD compile is different from the others and looks as to always produce bit identical results?

Well, IIRC, the only modifications of mine that were included into a special version of OggDropXPd by John33 were the impulse trigger profile (ITP) options, which simply allow you to change Ogg Vorbis' sensitivity in switching long to short blocks. Other than that, the encoders should be identical (since ITP is not used by default).

Yeah, that looks like it is the only difference in the options. And, I'm not using the options anyway, so that is odd that it would change the output, or make it somehow more discrete.

Edit: I tested it again, and this time, even the Modified (QuantumKnot) version produced differences. So, back to square one.
Edit 2: Scratch that, I forgot to convert to WAVs first before comparing bits. So, actually, 1.1.2MOD DOES produce bit identical wavforms, while the others don't.

I tested it again, and this time, even the Modified (QuantumKnot) version produced differences. So, back to square one.

Edit: Scratch that, I forgot to convert to WAVs first before comparing bits. So, actually, 1.1.2MOD DOES produce bit identical wavforms, while the others don't.

lol.. I forgot about this thread... oops..

but I do remember doing a bitwise comparison with foobar on 2 ogg vorbis files (same source file and encoding settings) and they were 100% identical

and you can force the serial number to whatever you want with the -s switch (i think its -s anyways )

use foobar's bit-compare and post the results on those weird encodings of yours

By the way, see above, because I actually forgot to convert the vorbis files back to wavs before I did the comparision on the 1.1.2MOD this more recent time...so my previous assumption about it producing bit-identical wavforms and the others not was actually correct the entire time!

Also, I'm actually using a program called FileCompare to do the comparisions...is the foobar2000's bit-compare only in the full version of fb2k, or is it just in the new 0.9?

By the way, just to let everyone know, these are not weird clips that I am testing, they are actually Soundgarden, Skunk Anansie, and Mission Impossible tracks!

I don't know this FileCompare utility, but make sure ti lets you see the differences between the files. They may all be header changes or something else. I use HexWorkshop's "Tools-->Compare" option. It generates a list of all differences and it highlights them too.

1. What about other codecs?
2. Did you already exclude possible hardware problems? If not - get an MD5 checker. Create a file (zip, rar) about 2 - 4 Gb (it should be several times larger than your RAM). Calculate its MD5 checksumm. Then verify MD5 checksum about 20-100 times. Or if you have a RAR archiver you can just verify your archive 20-100 times. I had similar troubles with with incompatible motherboard+CD-ROM+RAID controller.

i have v0.8.3 full version, i just highlight both tracks in the playlist, right click, and bit-compare
I just used the vorbis tracks directly in fb2k

Best test for floating point instabilities and possible memory errors i've found is Prime95, the best pure memory test around is Memtest86+. Prime95 seems to abort fairly quickly for me on errors, memtest will take some time. Both can be left to run unattended.

If the issue occured on multiple machines, did they have the same CPU make/model? Which compilers and switches were used for the versions that do and do not exhibit it? There has to be a pattern somewhere.

Also, a quick way of bit comparing files with near absolute certainty is md5sum, see http://www.etree.org/md5com.html

Yeah, I have tested my computer with Prime95 whenever I have done this test, and I have gotten no errors. Also, when I initially tested this, it was a P4 build of oggdropXPd, but I have since then reproduced it with the above mentioned non-P4 builds.

I think I have tested it on just Pentium 4s, so I will have to test it on an AMD proc, I guess.

Or a Intel P3, P-M, Core, Core-2, etc..

Hi all

I did the same tests as mmortal3 and these are the results:

Scenario 1:

• Tracks ripped to FLAC
• FLAC converted to ogg with oggdropXPd V.1.8.9 (non ITP version)
• Bit compare of the two ogg files (without converting to wav)

Scenario 2:

• Tracks ripped to FLAC
• FLAC converted to ogg with oggdropXPd V.1.8.9 (non ITP version)
• ogg files converted to fixed-point wav files (44kHz, 16 bit) with Foobar2000
• Bit compare of the two resultant wav files

Scenario 3:

• Tracks ripped to FLAC
• FLAC files converted to ogg with oggdropXPd V.1.8.9 (non ITP version)
• ogg files converted to 32bit floating-point wav files with Foobar2000
• Bit compare of the two resultant wav files

As you can see, there were no problems on bit-comparison of the ogg files themselves. However, when the ogg files were converted to fixed-point wave files (44kHz, 16 bit) with Fb2k, bit-comparison of the two wave files revealed differences. However, there were no differences if the ogg files were converted to 32 bit floating-point wave files in Fb2k.

However, the strangest thing is what follows:
I converted the ogg files to 44kHz, 16 bit wave files using dBpowerAMP Music Converter. I assume these are fixed-point wave files. Bit-comparison of these wave files revealed no differences.

dBpowerAMP does not appear to have any floating-point wave mode but it can convert the ogg files to 44kHz, 24 bit wave files. I do not know if these are floating-point files. Can anybody clarify on this? Anyway, bit comparison of the 44kHz, 24 bit wave files showed no differences either.

So, only the 44kHz, 16 bit wave files converted with fb2k showed differences. The 44kHz, 16 bit wave files converted with dMC showed no differences. This is really puzzling, to say the least

I know I have not been too successful in helping here, rather the result seems to be more confusion!!
audiomars

edit: spelling

Maybe you used dither?