Help - Search - Members - Calendar
Full Version: main_data_begin and audio data length
Hydrogenaudio Forums > Lossy Audio Compression > MP3 > MP3 - Tech
viktor09
Hi!

I wrote a small parser for mp3 streams. Now when I look at the output for some mp3s I see something like this: (mdb = main_data_begin, adl = ancillary data length)

frame 1: mdb = 0, adl = 64
frame 2: mdb = 64, adl = 112
frame 3: mdb = 176, adl = 114
frame 4: mdb = 290, adl = 114
frame 5: mdb = 290, adl = 0
frame 6: mdb = 100, adl = 0

So this means frame 2 - 4 have a main_data_begin != 0 but don't use the bitreservoir, as all pointers up to frame 5 point to the same byte in the reservoir.

2 questions here: First is this possible or is my parser not working correct? If it is correct, why not set the mdb to 0 for frames 2 - 4?

Another thing I observed was that the length of the audio data as caluclated by summing up the part2_3_length values can exceed the size of the frame. Again 2 questions: Is this correct or is my parse not working? If it is correct, does it mean that the calculated length includes the bits from the reservoir?

Thanks for your answers.

Martin
SebastianG
QUOTE (viktor09 @ Aug 20 2005, 05:33 PM)
frame 1: mdb = 0, adl = 64
frame 2: mdb = 64, adl = 112
frame 3: mdb = 176, adl = 114
frame 4: mdb = 290, adl = 114
frame 5: mdb = 290, adl = 0
frame 6: mdb = 100, adl = 0

So this means frame 2 - 4 have a main_data_begin != 0 but don't use the bitreservoir, as all pointers up to frame 5 point to the same byte in the reservoir.
*

how do you compute "adl" ?
what's the size of each side info block ?
what are the frames' sizes ?
what do your numbers tell us ? (bits or bytes?)
maybe you could upload/post a link to a file containing the first couple frames of your file.

QUOTE (viktor09 @ Aug 20 2005, 05:33 PM)
2 questions here: First is this possible or is my parser not working correct? If it is correct, why not set the mdb to 0 for frames 2 - 4?
*

I can't answer that one. You didn't give us all necessary informations.

QUOTE (viktor09 @ Aug 20 2005, 05:33 PM)
Another thing I observed was that the length of the audio data as caluclated by summing up the part2_3_length values can exceed the size of the frame. Again 2 questions: Is this correct or is my parse not working? If it is correct, does it mean that the calculated length includes the bits from the reservoir?
*

1.) Yes, it's possible that the sum exeeds the frame size
2.) Bingo

I suspect you don't fully understand what the reservoir thing is about and/or what main_data_begin actually says.


Sebi
viktor09
QUOTE (SebastianG @ Aug 20 2005, 08:27 PM)
QUOTE (viktor09 @ Aug 20 2005, 05:33 PM)
frame 1: mdb = 0, adl = 64
frame 2: mdb = 64, adl = 112
frame 3: mdb = 176, adl = 114
frame 4: mdb = 290, adl = 114
frame 5: mdb = 290, adl = 0
frame 6: mdb = 100, adl = 0

So this means frame 2 - 4 have a main_data_begin != 0 but don't use the bitreservoir, as all pointers up to frame 5 point to the same byte in the reservoir.
*

how do you compute "adl" ?
what's the size of each side info block ?
what are the frames' sizes ?
what do your numbers tell us ? (bits or bytes?)
maybe you could upload/post a link to a file containing the first couple frames of your file.


I compute adl size with the following formula (all values in bits):
adl = size_of_frame - size_of_header - size_of_crc - size_of_side_info - size_of_audio_data
size_of_header = 32
size_of_crc = 16 or 0
size_of_side_info = 136 for 1 channel or 256
size_of_audio_data = sum of part2_3_length values in side_info

The value I gave in my posting is in bytes (adl / 8) because main_data_begin is in bytes (right?).

QUOTE (SebastianG @ Aug 20 2005, 08:27 PM)
QUOTE (viktor09 @ Aug 20 2005, 05:33 PM)
2 questions here: First is this possible or is my parser not working correct? If it is correct, why not set the mdb to 0 for frames 2 - 4?
*

I can't answer that one. You didn't give us all necessary informations.

Is it possible to upload files here? Meanwhile here is a more elaborate info about the first frames. All size values are in bytes. Frame 0 contains no audio data but an info tag.

CODE
0   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 0, fr size = 626, audio length = 0, anc length = 590
1   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 0, fr size = 626, audio length = 525, anc length = 64
2   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 64, fr size = 627, audio length = 478, anc length = 112
3   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 176, fr size = 627, audio length = 476, anc length = 114
4   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 290, fr size = 627, audio length = 477, anc length = 114
5   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 404, fr size = 627, audio length = 496, anc length = 94
6   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 498, fr size = 627, audio length = 588, anc length = 2
7   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 500, fr size = 627, audio length = 589, anc length = 1
8   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 501, fr size = 627, audio length = 593, anc length = 0
9   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 498, fr size = 627, audio length = 588, anc length = 2
10   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 500, fr size = 627, audio length = 592, anc length = 0

So taking your last answer into accout, the audio data length in frame 8 contains 3 ( = 501 - 498) bytes from the reservoir which leaves 590 bytes in the frame itself. Given that header and sideinfo cosumes 36 bytes it would just fill the whole frame.

But I still don't understand why frames 1 - 7 have mdb != 0. Why not set it to 0?

QUOTE (SebastianG @ Aug 20 2005, 08:27 PM)
QUOTE (viktor09 @ Aug 20 2005, 05:33 PM)
Another thing I observed was that the length of the audio data as caluclated by summing up the part2_3_length values can exceed the size of the frame. Again 2 questions: Is this correct or is my parse not working? If it is correct, does it mean that the calculated length includes the bits from the reservoir?
*

1.) Yes, it's possible that the sum exeeds the frame size
2.) Bingo

I suspect you don't fully understand what the reservoir thing is about and/or what main_data_begin actually says.


That's possible. But I'm trying.

Martin
SebastianG
QUOTE (viktor09 @ Aug 21 2005, 10:19 AM)
The value I gave in my posting is in bytes (adl / 8) because main_data_begin is in bytes (right?).
*

Yes.

QUOTE (viktor09 @ Aug 21 2005, 10:19 AM)
Is it possible to upload files here? Meanwhile here is a more elaborate info about the first frames. All size values are in bytes. Frame 0 contains no audio data but an info tag.
*

Yes. In the upload forum.

QUOTE (viktor09 @ Aug 21 2005, 10:19 AM)
CODE
0   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 0, fr size = 626, audio length = 0, anc length = 590
1   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 0, fr size = 626, audio length = 525, anc length = 64
2   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 64, fr size = 627, audio length = 478, anc length = 112
3   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 176, fr size = 627, audio length = 476, anc length = 114
4   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 290, fr size = 627, audio length = 477, anc length = 114
5   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 404, fr size = 627, audio length = 496, anc length = 94
6   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 498, fr size = 627, audio length = 588, anc length = 2
7   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 500, fr size = 627, audio length = 589, anc length = 1
8   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 501, fr size = 627, audio length = 593, anc length = 0
9   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 498, fr size = 627, audio length = 588, anc length = 2
10   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 500, fr size = 627, audio length = 592, anc length = 0

So taking your last answer into accout, the audio data length in frame 8 contains 3 ( = 501 - 498) bytes from the reservoir which leaves 590 bytes in the frame itself. Given that header and sideinfo cosumes 36 bytes it would just fill the whole frame.
*

No. The length of the frame's main data is 593 bytes. Due to mdb being 501 you get the first 501 bytes of the main data from the reservoir and the remaining 92 bytes from the actual frame's main data section.

BTW: you need to map the version/layer bits first to the correct value.
MPEG version: 11 = MPEG 1, 10 = MPEG 2, 01 = MPEG 2.5
Layer: 11 = Layer 1, 10 = Layer 2, 01 = Layer 3

BTW2: Your "adl" is NOT the length of the ancillary data.

QUOTE (viktor09 @ Aug 21 2005, 10:19 AM)
But I still don't understand why frames 1 - 7 have mdb != 0. Why not set it to 0?
*

According to your log, frame 1 has mdb=0.

QUOTE (viktor09 @ Aug 21 2005, 10:19 AM)
QUOTE (SebastianG @ Aug 20 2005, 08:27 PM)

I suspect you don't fully understand what the reservoir thing is about and/or what main_data_begin actually says.

That's possible. But I'm trying.
*



This image might help you understand the reservoir thing:

mdb is actually a pointer which tells you how many main data bytes you have to step BACK prior decoding the main data. 0 means "no bytes from the reservoir are used". The reservoir is actually the last 511 bytes from the main data sections of previous frames.

HTH,
Sebi
viktor09
QUOTE (SebastianG @ Aug 21 2005, 12:00 PM)
QUOTE (viktor09 @ Aug 21 2005, 10:19 AM)
The value I gave in my posting is in bytes (adl / 8) because main_data_begin is in bytes (right?).

Yes.

QUOTE (viktor09 @ Aug 21 2005, 10:19 AM)
CODE
0   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 0, fr size = 626, audio length = 0, anc length = 590
1   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 0, fr size = 626, audio length = 525, anc length = 64
2   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 64, fr size = 627, audio length = 478, anc length = 112
3   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 176, fr size = 627, audio length = 476, anc length = 114
4   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 290, fr size = 627, audio length = 477, anc length = 114
5   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 404, fr size = 627, audio length = 496, anc length = 94
6   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 498, fr size = 627, audio length = 588, anc length = 2
7   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 500, fr size = 627, audio length = 589, anc length = 1
8   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 501, fr size = 627, audio length = 593, anc length = 0
9   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 498, fr size = 627, audio length = 588, anc length = 2
10   a MPAHeader (vers 3, layer 1, frq=44100, chn=stereo, t=0.02612244897959184)
     mdb = 500, fr size = 627, audio length = 592, anc length = 0


BTW: you need to map the version/layer bits first to the correct value.
MPEG version: 11 = MPEG 1, 10 = MPEG 2, 01 = MPEG 2.5
Layer: 11 = Layer 1, 10 = Layer 2, 01 = Layer 3

BTW2: Your "adl" is NOT the length of the ancillary data.

QUOTE (viktor09 @ Aug 21 2005, 10:19 AM)
QUOTE (SebastianG @ Aug 20 2005, 08:27 PM)

I suspect you don't fully understand what the reservoir thing is about and/or what main_data_begin actually says.

That's possible. But I'm trying.
*



This image might help you understand the reservoir thing:

mdb is actually a pointer which tells you how many main data bytes you have to step BACK prior decoding the main data. 0 means "no bytes from the reservoir are used". The reservoir is actually the last 511 bytes from the main data sections of previous frames.

HTH,
Sebi
*



Thanks for the answer. I think I got it now. But the issue with mdb != 0 in almost all frames still bugs me.

Say a frame number n does not need the bitreservoir. It sets mdb = 0. Now frame (n + 1) may still use spare data of frame (n - 1) and frame n as bitreservoir if it wishes to do so, right? So the choice of frame n to have mdb = 0 or shift its data partly into the reservoir (and by doing so increasing its own ancillary data size by that amount) is a matter of taste, yes? The "virtually" continuous bitreservoir is not affected by that.

Martin
SebastianG
QUOTE (viktor09 @ Aug 22 2005, 09:35 PM)
Thanks for the answer. I think I got it now. But the issue with mdb != 0 in almost all frames still bugs me.

Say a frame number n does not need the bitreservoir. It sets mdb = 0. Now frame (n + 1) may still use spare data of frame (n - 1) and frame n as bitreservoir if it wishes to do so, right? So the choice of frame n to have mdb = 0 or shift its data partly into the reservoir (and by doing so increasing its own ancillary data size by that amount) is a matter of taste, yes? The "virtually" continuous bitreservoir is not affected by that.

Martin
*


Why shouldn't frame n use unused space in frame (n-1) even if the main data completelyx fits into frame n ? If it does use the unused space frame n will have more room for the following frame's main data. If you set mdb to zero in frame n you'd waste unused space in frame (n-1). This space cannot be used by frame (n+1) because of several reasons (i.e. mdb is only a 9 bit pointer).

Another thing: Your "ancillary data size" is simply the amount of bytes in a frame that are not used by itself at the end. Only a small portion of this is actually "ancillary data" 'cause the follwing frame also uses some of those bytes to store its main data in there. So, the real ancillary data size is actually your computed adl minus the mdb of the following frame. wink.gif

Sebi
robert
iirc main data may point up to 9 frames back, not just 1 frame.
SebastianG
QUOTE (robert @ Aug 23 2005, 12:46 AM)
iirc main data may point up to 9 frames back, not just 1 frame.
*


Right. (I made some assumptions about the frame sizes and stuff when I mentioned the 9 bit pointer thing.)

It depends on the frames' sizes. If you got CBR 192 kbps at 44 kHz the frame size is
around 627 bytes. Each frame size has a main data section of around 593 bytes. Since the highest 9 bit number is 511 you can only point back 511 bytes. In this case you are in the previous frame.

The worst case for MPEG1 is 32 kbps 48000 Hz stereo with a frame size of 96 bytes and a main data section size of 96-4-30=62 bytes. 511/62 = 8.242. There you have your worst case dependency of 9 previous frames.

What viktor09 suggested was the following: if the main data of a frame completely fits into this frame don't make use of the reseroir, This does not make sense since it leaves holes of unused space in previous frames that in most situations can't be used by a following frames because two things must be satisfied: A following frame needs to be able to address this hole which is not possible if the amount of main data bits of the current frame + number of unused bits is higher than 511*8. Secondly the size of this "hole" needs to be large enough so all main data bits of the following frame can be stored in there because the main data bits can't be be split into multiple non-adjacent blocks (*). So, his suggestion is actually a bad idea.

(* counting only the concatenated main data sections)

Sebi
viktor09
QUOTE (SebastianG @ Aug 23 2005, 12:40 PM)
QUOTE (robert @ Aug 23 2005, 12:46 AM)
iirc main data may point up to 9 frames back, not just 1 frame.
*


Right. (I made some assumptions about the frame sizes and stuff when I mentioned the 9 bit pointer thing.)

It depends on the frames' sizes. If you got CBR 192 kbps at 44 kHz the frame size is
around 627 bytes. Each frame size has a main data section of around 593 bytes. Since the highest 9 bit number is 511 you can only point back 511 bytes. In this case you are in the previous frame.

The worst case for MPEG1 is 32 kbps 48000 Hz stereo with a frame size of 96 bytes and a main data section size of 96-4-30=62 bytes. 511/62 = 8.242. There you have your worst case dependency of 9 previous frames.

What viktor09 suggested was the following: if the main data of a frame completely fits into this frame don't make use of the reseroir, This does not make sense since it leaves holes of unused space in previous frames that in most situations can't be used by a following frames because two things must be satisfied: A following frame needs to be able to address this hole which is not possible if the amount of main data bits of the current frame + number of unused bits is higher than 511*8. Secondly the size of this "hole" needs to be large enough so all main data bits of the following frame can be stored in there because the main data bits can't be be split into multiple non-adjacent blocks (*). So, his suggestion is actually a bad idea.

(* counting only the concatenated main data sections)

Sebi
*



I think I'm almost there now. But I did not understand your last remark about "can't be split into multiple non-adjacent blocks". Do you mind explaining this in more detail?

Martin
SebastianG
QUOTE (viktor09 @ Aug 23 2005, 08:01 PM)
I think I'm almost there now. But I did not understand your last remark about "can't be split into multiple non-adjacent blocks". Do you mind explaining this in more detail?

Martin
*


Look at the picture above: If you ignore the frame headers and side infos and only look at the frame's main data sections: Each frame's main infos are always consecutive bits. You just can't tell a decoder: get the first bytes of my main infos from the reservoir at offset x, skip y bytes and then continue to decode main infos. Each frame's main infos are a single block of bits which is ONLY split by headers+side-infos and by nothing else (i.e. another frame's main data). This is because the only thing you store in MP3 frames about the reservoir usage is one single 9 bit pointer -- NOT a long descrption of where to find all the main data bits and how they are distributed.

What does a decoder do ?
- it memorizes the last 511 bytes of the frames' main data sections (reservoir)
- it decodes the header / side infos
- if (MDB>0) -> the last MDB bytes of the reservoir are (virtually) inserted in the current frame right after the side infos / at the beginning of the main data section
- the main data section is then decoded


I think I provided sufficient informations about the reservoir.


Sebi
viktor09
QUOTE (SebastianG @ Aug 23 2005, 11:27 PM)
QUOTE (viktor09 @ Aug 23 2005, 08:01 PM)
I think I'm almost there now. But I did not understand your last remark about "can't be split into multiple non-adjacent blocks". Do you mind explaining this in more detail?

Martin
*


Look at the picture above: If you ignore the frame headers and side infos and only look at the frame's main data sections: Each frame's main infos are always consecutive bits. You just can't tell a decoder: get the first bytes of my main infos from the reservoir at offset x, skip y bytes and then continue to decode main infos. Each frame's main infos are a single block of bits which is ONLY split by headers+side-infos and by nothing else (i.e. another frame's main data). This is because the only thing you store in MP3 frames about the reservoir usage is one single 9 bit pointer -- NOT a long descrption of where to find all the main data bits and how they are distributed.

What does a decoder do ?
- it memorizes the last 511 bytes of the frames' main data sections (reservoir)
- it decodes the header / side infos
- if (MDB>0) -> the last MDB bytes of the reservoir are (virtually) inserted in the current frame right after the side infos / at the beginning of the main data section
- the main data section is then decoded


I think I provided sufficient informations about the reservoir.
*


Yes you did. Thanks again.

Martin
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2009 Invision Power Services, Inc.