Hydrogenaudio Forums > Can't get float files to decode properly

sbooth

May 3 2007, 17:12

I've been pulling my hair out trying to get decoding of float wavpack files to work correctly.

I'm working off of a custom framework build of the library (4.40) on OS X (I'm not using the tinydecoder). I am able to decode all bit depths of integer files successfully (8, 12, 16, 24, and 32), so I don't think it is an issue with my build.

The code I'm using is fairly straightforward; but no matter what I try float files are decoded to nothing but white noise. I figured I would paste the relevant code excerpt here and see if anyone notices anything that looks funky in the float processing.

I tried swab'ing the buffer, but still got white noise.

It is worth mentioning that the command-line client (wvunpack) produces a silent WAV from the same 32-bit float input file (32bit_float.wv from the test suite).

CODE

- (void) fillPCMBuffer
{
    void                *writePointer;
    int32_t                inputBuffer [WP_INPUT_BUFFER_LEN];
    uint32_t            sample;

    for(;;) {
        UInt32    bytesToWrite                = RING_BUFFER_WRITE_CHUNK_SIZE;
        UInt32    bytesAvailableToWrite        = [[self pcmBuffer] lengthAvailableToWriteReturningPointer:&writePointer];
        UInt32    spaceRequired                = WP_INPUT_BUFFER_LEN /* * [self pcmFormat].mChannelsPerFrame */ * (32 / 8);

        if(bytesAvailableToWrite < bytesToWrite || spaceRequired > bytesAvailableToWrite) {
            break;
        }

        // Wavpack uses "complete" samples (one sample across all channels), i.e. a Core Audio frame
        uint32_t samplesRead = WavpackUnpackSamples(_wpc, inputBuffer, WP_INPUT_BUFFER_LEN / [self pcmFormat].mChannelsPerFrame);

        // Handle floating point files
        if(MODE_FLOAT & WavpackGetMode(_wpc)) {

            if(127 != WavpackGetFloatNormExp(_wpc)) {
                NSLog(@"Floating point data not scaled to +/- 1.0");
                return;
            }

            float    *inputFloatBuffer    = (float *)inputBuffer;
            float    *floatBuffer        = (float *)writePointer;
            float    audioSample            = 0;

            for(sample = 0; sample < samplesRead * [self pcmFormat].mChannelsPerFrame; ++sample) {
                audioSample        = inputFloatBuffer[sample];
                *floatBuffer++    = (audioSample < -1.0 ? -1.0 : (audioSample > 1.0 ? 1.0 : audioSample));
            }

            [[self pcmBuffer] didWriteLength:samplesRead * [self pcmFormat].mChannelsPerFrame * (32 / 8)];
        }
        else {
            float    *floatBuffer    = (float *)writePointer;
            double    scaleFactor        = (1LL << WavpackGetBitsPerSample(_wpc));
            double    audioSample        = 0;

            for(sample = 0; sample < samplesRead * [self pcmFormat].mChannelsPerFrame; ++sample) {

                if(0 <= inputBuffer[sample]) {
                    audioSample = (double)(inputBuffer[sample] / (scaleFactor - 1));
                }
                else {
                    audioSample = (double)(inputBuffer[sample] / scaleFactor);
                }

                *floatBuffer++ = (float)(audioSample < -1.0 ? -1.0 : (audioSample > 1.0 ? 1.0 : audioSample));
            }

            [[self pcmBuffer] didWriteLength:samplesRead * [self pcmFormat].mChannelsPerFrame * (32 / 8)];
        }

        // EOS?
        if(0 == samplesRead) {
            [self setAtEndOfStream:YES];
            break;
        }
    }
}

bryant

May 3 2007, 17:51

QUOTE(sbooth @ May 3 2007, 16:12)

I've been pulling my hair out trying to get decoding of float wavpack files to work correctly.

Hmm, I don't see anything obvious in your code that could cause that. I assume that this is not an Intel-based MAC, right?

You shouldn't have that check in there for the WavpackGetNormFloatExp() because that might prevent you from working with some files. Just set the OPEN_NORMALIZE flag when you open the file instead. But obviously that's not causing this problem.

What happens when you use the -m option with wvunpack when you unpack the file? If that reports an incorrect value, then there's an obvious problem in my code (that is almost certainly an endian issue). If that's correct, then there must be another issue.

Please stop pulling out your hair! I'm sure we can easily figure out what's going on and I appreciate you helping me with it...

David

bryant

May 3 2007, 18:20

Also not releated to your white noise issue, I found a problem in the integer code. Instead of

CODE

double scaleFactor = (1LL << WavpackGetBitsPerSample(_wpc));

I think that

CODE

double scaleFactor = (1LL << (WavpackGetBytesPerSample(_wpc) * 8 - 1));

would be correct.

This should have caused most integer files to be 6 dB soft, and the 12 and 20 bits files should have been loud and badly clipping.

sbooth

May 3 2007, 18:53

QUOTE(bryant @ May 3 2007, 15:51)

Hmm, I don't see anything obvious in your code that could cause that. I assume that this is not an Intel-based MAC, right?

Correct, this is a PowerBook G4.

QUOTE

You shouldn't have that check in there for the WavpackGetNormFloatExp() because that might prevent you from working with some files. Just set the OPEN_NORMALIZE flag when you open the file instead. But obviously that's not causing this problem.

OK, thanks. I thought of it more as a sanity check, since I'm passing 0 as the offset_exp when the file is opened.

QUOTE

What happens when you use the -m option with wvunpack when you unpack the file? If that reports an incorrect value, then there's an obvious problem in my code (that is almost certainly an endian issue). If that's correct, then there must be another issue.

Here is the output:

CODE

$ ./wvunpack -m ~/Music/Samples/test_suite/bit_depths/32bit_float.wv

WVUNPACK  Hybrid Lossless Audio Decompressor  Darwin Version 4.40.0
Copyright (c) 1998 - 2006 Conifer Software.  All Rights Reserved.

original md5:  8036f57e0fd91bef33604b2de3fb4430
unpacked md5:  65e15559bde044c947bebaeb1dc9711b
MD5 signatures should match, but do not!

So perhaps there is a problem in the library itself?

QUOTE(bryant @ May 3 2007, 16:20)

Also not releated to your white noise issue, I found a problem in the integer code. Instead of

CODE

double scaleFactor = (1LL << WavpackGetBitsPerSample(_wpc));

I think that

CODE

double scaleFactor = (1LL << (WavpackGetBytesPerSample(_wpc) * 8 - 1));

would be correct.

This should have caused most integer files to be 6 dB soft, and the 12 and 20 bits files should have been loud and badly clipping.

I used this method because I found some sample code on Apple's web site (http://developer.apple.com/documentation/D..._section_5.html) that did a similar thing. After some googling and reading on cycling74's web site, it seemed to me the consensus is that there is no consensus on the "best" way to perform the conversion.

However, what you say is true; the 12 and 20 bit files did sound a bit like crap

Edit: OK, I missed what your fix did at first. You are correct- I left my error here for posterity.

bryant

May 3 2007, 20:01

QUOTE(sbooth @ May 3 2007, 17:53)

QUOTE

What happens when you use the -m option with wvunpack when you unpack the file? If that reports an incorrect value, then there's an obvious problem in my code (that is almost certainly an endian issue). If that's correct, then there must be another issue.

Here is the output:

CODE

$ ./wvunpack -m ~/Music/Samples/test_suite/bit_depths/32bit_float.wv

WVUNPACK  Hybrid Lossless Audio Decompressor  Darwin Version 4.40.0
Copyright (c) 1998 - 2006 Conifer Software.  All Rights Reserved.

original md5:  8036f57e0fd91bef33604b2de3fb4430
unpacked md5:  65e15559bde044c947bebaeb1dc9711b
MD5 signatures should match, but do not!

So perhaps there is a problem in the library itself?

Indeed!

In the meantime I did a little googling myself and found that GCC allocates bit-fields the same on big-endian and little-endian machines. Unfortunately, I expected it to mirror the changes in integers and floating-point numbers and am using a bitfield to "define" a 32-bit float.

When I reversed the bitfield definition in my code, I got the same incorrect MD5 as you get on that file!

If you have the time (and interest) you can also reverse the definition in src/wavpack_local.h and see if this fixes all your problems (both the wvunpack and your player). Just change this:

CODE

typedef struct {
    unsigned mantissa : 23;
    unsigned exponent : 8;
    unsigned sign : 1;
} f32;

to this:

CODE

typedef struct {
    unsigned sign : 1;
    unsigned exponent : 8;
    unsigned mantissa : 23;
} f32;

Obviously this will break Intel builds, so I need some generic solution that will probably involve some ugly macros. I see GCC 4 has a pragma for this called reverse_bitfields, but it's a little late.

Anyway, thanks for finding this problem, and this should let you limp along until I get a real fix.

David

sbooth

May 3 2007, 21:43

QUOTE(bryant @ May 3 2007, 18:01)

If you have the time (and interest) you can also reverse the definition in src/wavpack_local.h and see if this fixes all your problems (both the wvunpack and your player). Just change this:

CODE

typedef struct {
    unsigned mantissa : 23;
    unsigned exponent : 8;
    unsigned sign : 1;
} f32;

to this:

CODE

typedef struct {
    unsigned sign : 1;
    unsigned exponent : 8;
    unsigned mantissa : 23;
} f32;

Obviously this will break Intel builds, so I need some generic solution that will probably involve some ugly macros. I see GCC 4 has a pragma for this called reverse_bitfields, but it's a little late.

Anyway, thanks for finding this problem, and this should let you limp along until I get a real fix.

Thanks for the quick response! I made the changes you suggested and was able to decode to WAV successfully with matching md5 sums.

FWIW, I added a conditional guard for __BIG_ENDIAN__ around the reversed bitfield and I believe the code should work for both OS X architectures (assuming it worked on Intel previously, it works for me now on PPC).

bryant

May 4 2007, 13:27

QUOTE(sbooth @ May 3 2007, 20:43)

Thanks for the quick response! I made the changes you suggested and was able to decode to WAV successfully with matching md5 sums.

FWIW, I added a conditional guard for __BIG_ENDIAN__ around the reversed bitfield and I believe the code should work for both OS X architectures (assuming it worked on Intel previously, it works for me now on PPC).

Thanks for trying that and letting me know; I'm glad that it worked!

I think using a conditional based on the endian-ness will work fine for you for now, and I thought about just doing that myself. However, the C spec makes it pretty clear that developers should not depend on the allocation of bitfields to be any particular way, so I think the prudent thing for me to do is switch to macros. Fortunately, it's uglier than it is difficult...