Hydrogenaudio Forums > MP3 repacker

Hydrogenaudio Forums > Lossy Audio Compression > MP3 > MP3 - Tech

Pages: 1, 2, 3, 4, 5, 6, 7, 8, 9

j7n

Jan 5 2007, 02:47

I think the repacker is mostly useless for files you encode today with LAME. Its useful for files other people have encoded before. To estimate the size decrease if repacked (-z), check your MPEG Layer-3 file with a hex editor. Padding data is usually written as 0x00 or 0xFF and visible as empty space inside the file. For such files I have decreased the avg bitrate from 320 to 250 kBit/s.

halb27

Jan 5 2007, 02:54

QUOTE(j7n @ Jan 5 2007, 10:47)

I see. Bitrate decrease is great. I didn't think of other peoples' encodings but do it myself - streamripped internet mp3 at high bitrate (usually 192 to 256 kbps). So the repacker may be useful to me too.

bukem

Jan 5 2007, 09:01

@halb27:
As j7n already mentioned you can't expect a big bitrate decrease when compressing LAME VBR (from my experience for LAME VBR (in fact any VBR) you get 1~3% decrease). The main reason I'm using mp3packer is to convert high bitrate CBR (~256 and more) to VBR which allows me to save around 10~15% of size which is precious for my portable mp3 player, but that of course is not a rule. In some rare cases I was able to save even 30~40% especially when I was mp3packing 320 CBR mp3 with contemplation/meditation music which tends to have long silence periods. Other reason why I'm using mp3packer is to rebuild broken/bad mp3 - I know that it's not main purpose of that tool but it came to be very efficient for repairing synchronization errors and etc.

halb27

Jan 5 2007, 09:56

Thanks a lot, bukem and j7n. I will use mp3repacker on those internet radio streamripped files which I want to keep.

Martin F.

Jan 20 2007, 14:10

On this file, mp3packer reports that there were 8 buffer errors, but it displays only one.

QUOTE

>mp3packer -z 8_buffer_errors.mp3

*** '8_buffer_errors.mp3' -> '8_buffer_errors-vbr.mp3'
WARNING: buffer underflow; frame 0 will be entirely replaced with silence
100% done with 334 frames

WARNING: There were 8 buffer errors

Light-Fire

Jan 20 2007, 15:30

It is a good software to convert VBR files to equivalent CBR to use with players that don't work well with VBR.

Omion

Jan 20 2007, 20:34

@Martin F.:

It looks like there is a bit of a discrepancy in what mp3packer thinks is a buffer error.

The errors displayed are only the frames which have data outside the buffer.
The number of buffer errors at the end is the number of frames which have data offsets before the beginning of the buffer

It would seem that 1 would imply 2; however, if the frame's data starts outside the buffer but the data has 0 length, the number of errors will be incremented but there will be no data loss.

I think I'll change it so it only counts an error if there was data loss. I'll fix that ASAP.

QUOTE(Light-Fire @ Jan 20 2007, 14:30)

It is a good software to convert VBR files to equivalent CBR to use with players that don't work well with VBR.

Thanks! A lot of people seem to use mp3packer for that.

Omion

Jan 20 2007, 22:29

1.14 is out.

The output issues that Martin F. raised have been fixed. The repacker itself was not changed.

Get it at the beginning of this thread.

bukem

Feb 18 2007, 11:32

@Omion:

Found another mp3 that crashes mp3packer (v1.14-159):

QUOTE

*** 'volume 10/07. j. dilla - walkinonit.mp3' -> 'volume 10.new/07. j. dilla - walkinonit.mp3'
WARNING: sync error; expected frame 1 at 1586
WARNING: recompression skipped on frame 1: Invalid_argument(index out of bounds)
100% done with 1 frames

WARNING: There was 1 sync error
Fatal error: exception Mp3types.Too_many_bytes

You should get that file in your mailbox right now. Greets.

Omion

Feb 18 2007, 17:10

@bukem:

Thanks for the file. It looks like the first frame has the "original" bit set, but no other frames have it. Then it randomly finds another frame which overflows. I took off the original-bit checking for valid frames, and it seems to work for that file. I'll put out a release soon.

(In the meantime, you can set byte 544 to 0x00 in that file and it will work, but that's not really a generalizable solution

)

Omion

Feb 18 2007, 20:34

OK. Released 1.15. I just made it so that mp3packer does not check for the same "original" bit throughout the entire file. I've never heard of this being a problem until bukem's sample, though.

Nothing else wan changed. In fact, the entire change to the code was 6 bytes, whereas I had to add ~130 bytes to the changelog

bukem

Feb 19 2007, 11:21

QUOTE(Omion @ Feb 19 2007, 03:34)

I'm bad a.s - nothing but problems

. Thanks for quick response. Be back soon

EDIT:
I'm getting a lot of following errors on one album - is it any bad?

CODE

WARNING: error decompressing frame 1880; copying frame data instead
WARNING: error decompressing frame 1881; copying frame data instead
WARNING: error decompressing frame 1882; copying frame data instead
WARNING: error decompressing frame 1891; copying frame data instead
WARNING: error decompressing frame 1902; copying frame data instead
WARNING: error decompressing frame 1909; copying frame data instead
WARNING: error decompressing frame 1915; copying frame data instead
WARNING: error decompressing frame 1921; copying frame data instead
WARNING: error decompressing frame 1935; copying frame data instead
WARNING: error decompressing frame 1940; copying frame data instead
WARNING: error decompressing frame 1956; copying frame data instead
WARNING: error decompressing frame 1957; copying frame data instead
WARNING: error decompressing frame 1962; copying frame data instead

Omion

Feb 19 2007, 20:34

QUOTE(bukem @ Feb 19 2007, 10:21)

EDIT:
I'm getting a lot of following errors on one album - is it any bad?

That means that the MP3 data was bad. Repacking it will not make it any worse though, since the data was just passed unmodified.

j7n

Feb 20 2007, 09:46

I have a similar question...

On some Blade-encoded files (the ones Mp3Packer is useful on) your program sometimes gives me these error msgs:

QUOTE

WARNING: bitstream error on frame %d.

QUOTE

WARNING: error recompressing frame %d: Invalid_argument(%s)

The output files play ok though. What do these warnings mean?

stigc

Feb 20 2007, 12:10

I don't get the -z flag. Is it loss-less?

Omion

Feb 20 2007, 13:28

@j7n:
Those warnings are the pre-1.14 versions of the ones bukem was getting. It means that there was a problem in decompressing the actual frame data, so the -z switch was turned off for that frame. The data should still be exactly the same (even though it may not have been valid in the first place)

QUOTE(stigc @ Feb 20 2007, 11:10)

I don't get the -z flag. Is it loss-less?

Yes, it's lossless. The -z switch is equivalent to unpacking a .zip file, and re-packing it with a higher compression level.

maadjordan

Mar 2 2007, 02:47

thanks OMION for this masterpeiece.. but what i request is somthing else

i would request to add the first phase of -z command which is to decode the huffs of the frame into expanded mp3 form and a feature to restore or repack it like -z command to that format type.. in order to test the possiblity to achieve even more compression ratio via lossless programs like winrar and 7-zip so if worked it would be great.. and if the resulted file would be mp3 readable to any decoder then it could be used with any player that support packed music files aka foobar..

So please add this necessary feature to test this capability..even in a special build.. Thanks in advance for the Mp3packer and speicaly for the -z command..

is there going to be OGGpacker ?? on cbr files which is rare not like mp3 and has a huffman coding.. to retest

j7n

Mar 9 2007, 00:15

QUOTE

is there going to be OGGpacker ??

I believe there is not even a good cutter for OggS. It's a big advantage of classic MPEG formats, that there is no file header required and any sequence of frames can be decoded.

I've found a problem file where Mp3Packer fails to skip garbage at the beginning.

CODE

*** 'D:/_temp/dcfinal/1996 - Around the World Hit Singles - the Journey So Far/0
2_Deep.mp3' -> 'D:/_temp/dcfinal/1996 - Around the World Hit Singles - the Journ
ey So Far/02_Deep-vbr.mp3'
WARNING: buffer under AND overflow; frame 0 is garbage and will be replaced by s
ilence
WARNING: sync error; expected frame 1 at 721
WARNING: buffer overflow; frame 1 will be truncated
WARNING: sync error; expected frame 2 at 2065
WARNING: buffer overflow; frame 2 will be truncated
WARNING: sync error; expected frame 3 at 3409
WARNING: buffer overflow; frame 3 will be truncated
WARNING: sync error; expected frame 4 at 1521790
WARNING: buffer overflow; frame 4 will be truncated
WARNING: sync error; expected frame 5 at 2708486
WARNING: buffer overflow; frame 5 will be truncated
WARNING: sync error; expected frame 6 at 6864529
WARNING: buffer overflow; frame 6 will be truncated
WARNING: sync error; expected frame 7 at 6865873
WARNING: buffer overflow; frame 7 will be truncated
WARNING: sync error; expected frame 8 at 6867217
WARNING: buffer overflow; frame 8 will be truncated
100% done with 8 frames

WARNING: There were 9 buffer errors and 8 sync errors

The output 02_Deep-vbr.mp3 is 2040 bytes long. It happens that the first 2 bytes of the input file look like a valid MPEG Layer 3 frame with CRC error detection (i.e. FF FA), but in fact this turned out to be a random occurence in the first (incomplete) frame. I believe the first frame got cut because of removal of a bogus ID3v2 header, but that's not really important.

Omion

Mar 9 2007, 09:57

QUOTE(maadjordan @ Mar 2 2007, 01:47)

I'm afraid I don't quite follow you here. You want to be able to store the decompressed frequency values in the hope that you could apply your own compressor to the values?

If that is the case, it's quite impossible that it would be able to be decoded by normal mp3 players, even with archive support. When you unpacked the archive, you would end up with... decompressed frequency values, which does not make for a valid MP3 file.

The only way it could work is if I made mp3packer artificially inflate the Huffman values, in the hope that the archive format would pick up the compression better. It would be a valid mp3, but not compressed as much. This is possible, but I highly doubt that it will result in any improvement.

As for OggPacker, I'm not going to write it. Learning everything there is to know about one audio format is enough for me. (Learning about mp3 actually was my primary goal of this program; I hardly have any mp3s in my collection...) There was a program by Garf (I think) which optimized the Huffman tables in a Vorbis stream, but I seem to remember it being somewhat buggy.

@j7n:

Yeah. That's because mp3packer only checks one frame to see if it's valid, rather than making sure there's another valid frame which follows directly after it. That little shortcut has been nagging at me for quite a while, but there's no easy fix for it other than to completely re-make the mp3 reading portion of the program.
Rewriting large parts of mp3packer is not out of the question -- I've done it twice before -- but now that I've got a full-time job and another program out to support, my time is stretching thin(*) A fix should happen... eventually. No idea when, though.

In the mean time, use a hex editor to delete the fake sync pattern. You seem to know what you're doing

Other than that, I don't really know what can be done.

(*) especially now that I've been re-playing Oblivion. Man, that game takes a lot of time.

j7n

Mar 9 2007, 13:20

It's no problem.

RogerG

Mar 9 2007, 20:17

Hi!

nice application!
- sometimes out is larger than in. Could you make an option to not overwrite in if out is larger?
- What do you think about an option to remove space padding from id3v2 tags? This would decrease file size even more.
- What will happen to my file if there are buffer errors?
- Why does it corrupt my files ("buffer underflow; frame 0 will be truncated", "WARNING: buffer underflow; frame 0 will be entirely replaced with silence") ? Not sure if it does but the warnung messages don't sound well. How can your program be lossless if it inserts silence into my files?

Maybe you can make a small FAQ

Regrads

Omion

Mar 9 2007, 23:47

QUOTE(RogerG @ Mar 9 2007, 19:17)

sometimes out is larger than in. Could you make an option to not overwrite in if out is larger?

The only time the output is larger than the input should be when the only thing that's added is the LAME header. In this case the file is only ~500 bytes larger. I suppose that any increase is bad, but I figured that 500 bytes is not enough to worry about. I can look into not using the output file if it's larger, but if it's hard I don't think I'll bother.

Have you seen any files that get larger than about 500 bytes? If so, then that's probably a bug.

QUOTE

What do you think about an option to remove space padding from id3v2 tags? This would decrease file size even more.

I try to avoid id3 tags altogether, especially id3v2. It's quite a nasty tagging spec, and judging by the files that I get, it tends to get broken quite a bit. (And I'd have to implement yet another CRC generator... not fun)

QUOTE

What will happen to my file if there are buffer errors?

That depends on what options you have set. By default both the output and input files will be kept, and a warning will be displayed. If you change the --keep-bad option, various things happen:
--keep-bad in: Keeps only the input file. Good for only repacking the files without errors
--keep-bad out: Keeps only the output file. Good if you want to repack all the files no matter what
--keep-bad both: Default. Won't delete either file.
Similarly, the --keep-ok switch determines which files to keep when no errors occur.
Note also that if you use the -u switch, the output file will be the same name as the original input, whereas the input file will be renamed something else.

QUOTE

Why does it corrupt my files ("buffer underflow; frame 0 will be truncated", "WARNING: buffer underflow; frame 0 will be entirely replaced with silence") ? Not sure if it does but the warnung messages don't sound well. How can your program be lossless if it inserts silence into my files?

That warning, when you see it only on the first few frames, generally means that the file was split improperly. The data for that frame doesn't exist in the file, so the only thing that can be done to make it normal again is replace it with a frame which is silent.

The warning message is telling you that the input file is bad, not that the processing did something horrible to the file.

When a bitstream error occurs, you are right that mp3packer is no longer lossless on that frame, but only because the data is already lost and mp3packer has to "recreate" it.
For example, if half of a frame is cut off, you can do a few things:

Pad the frame with zeros and decode like normal (tends to be completely wrong)
Start decoding from the beginning of the valid data (also generally completely wrong, as there is no way to tell how far into the frame the valid data starts)
Only keep the parts of the frame which have no missing data, and set the rest of the frame to silence (what mp3packer does by default; usually the best-sounding option)
Replace the whole frame with silence (what mp3packer does if the -w option set; saves a bit more space sometimes)

There is basically no "lossless" way to even read the mp3 data, as some of it is already lost!

However, all frames which do not produce a warning message should be lossless. If not, there's something wrong.

Martin F.

Mar 19 2007, 09:21

mp3packer -z 8bp026-05-sabastian_boaz-cold_hard_fusion.mp3

QUOTE

WARNING: error decompressing frame 57; copying frame data instead

How can the new file be without decompressing errors if the data is simply copied?

Omion

Mar 19 2007, 11:44

QUOTE(Martin F. @ Mar 19 2007, 08:21)

mp3packer -z 8bp026-05-sabastian_boaz-cold_hard_fusion.mp3

QUOTE

WARNING: error decompressing frame 57; copying frame data instead

How can the new file be without decompressing errors if the data is simply copied?

The new file will have the exact same decompression errors as the input file. mp3packer was not designed to fix errors, but rather to not create new ones. The reasoning behind this is simple: the best way to make the process lossless is to make sure that both files have the same errors. (mp3packer does fix sync errors, but that's only because it would be impossible to exactly recreate them)

I checked the file, and it seems to be a simple data overflow in the frame. The frame says it uses 42 bits, but it actually uses either 41 or 47. Most decoders just toss out the data from the end if this happens, but mp3packer keeps the bad data and lets the player deal with it (just like the original file)

Martin F.

Mar 19 2007, 12:04

QUOTE(Omion @ Mar 19 2007, 18:44)

The new file will have the exact same decompression errors as the input file.

But why doesn’t MP3packer complain about them?

CODE

mp3packer -z 8bp026-05-sabastian_boaz-cold_hard_fusion.mp3

*** '8bp026-05-sabastian_boaz-cold_hard_fusion.mp3' -> '8bp026-05-sabastian_boaz-cold_hard_fusion-vbr.mp3'
WARNING: error decompressing frame 57; copying frame data instead
WARNING: error decompressing frame 69; copying frame data instead
WARNING: error decompressing frame 133; copying frame data instead
WARNING: recompressing frame 145 failed; copying frame data instead
WARNING: error decompressing frame 147; copying frame data instead
WARNING: error decompressing frame 210; copying frame data instead
WARNING: error decompressing frame 222; copying frame data instead
WARNING: recompressing frame 232 failed; copying frame data instead
WARNING: error decompressing frame 261; copying frame data instead
WARNING: error decompressing frame 279; copying frame data instead
WARNING: recompressing frame 290 failed; copying frame data instead
WARNING: error decompressing frame 299; copying frame data instead
WARNING: error decompressing frame 346; copying frame data instead
WARNING: error decompressing frame 357; copying frame data instead
WARNING: error decompressing frame 365; copying frame data instead
WARNING: error decompressing frame 366; copying frame data instead
WARNING: error decompressing frame 376; copying frame data instead
WARNING: error decompressing frame 395; copying frame data instead
WARNING: recompressing frame 470 failed; copying frame data instead
WARNING: error decompressing frame 623; copying frame data instead
WARNING: error decompressing frame 624; copying frame data instead
WARNING: error decompressing frame 652; copying frame data instead
WARNING: error decompressing frame 749; copying frame data instead
WARNING: error decompressing frame 750; copying frame data instead
WARNING: error decompressing frame 758; copying frame data instead
WARNING: error decompressing frame 815; copying frame data instead
WARNING: error decompressing frame 816; copying frame data instead
WARNING: error decompressing frame 824; copying frame data instead
WARNING: error decompressing frame 826; copying frame data instead
WARNING: error decompressing frame 845; copying frame data instead
WARNING: error decompressing frame 874; copying frame data instead
WARNING: error decompressing frame 891; copying frame data instead
WARNING: error decompressing frame 903; copying frame data instead
WARNING: recompressing frame 917 failed; copying frame data instead
WARNING: error decompressing frame 1083; copying frame data instead
WARNING: error decompressing frame 1125; copying frame data instead
WARNING: recompressing frame 1180 failed; copying frame data instead
WARNING: error decompressing frame 1222; copying frame data instead
WARNING: error decompressing frame 1236; copying frame data instead
WARNING: error decompressing frame 1259; copying frame data instead
WARNING: error decompressing frame 1264; copying frame data instead
WARNING: recompressing frame 1298 failed; copying frame data instead
WARNING: error decompressing frame 1303; copying frame data instead
WARNING: error decompressing frame 1437; copying frame data instead
WARNING: recompressing frame 1438 failed; copying frame data instead
WARNING: recompressing frame 1457 failed; copying frame data instead
WARNING: error decompressing frame 1486; copying frame data instead
WARNING: error decompressing frame 1504; copying frame data instead
WARNING: recompressing frame 1515 failed; copying frame data instead
WARNING: recompressing frame 1547 failed; copying frame data instead
WARNING: error decompressing frame 1571; copying frame data instead
WARNING: error decompressing frame 1590; copying frame data instead
WARNING: error decompressing frame 1591; copying frame data instead
WARNING: error decompressing frame 1601; copying frame data instead
WARNING: recompressing frame 1620 failed; copying frame data instead
WARNING: recompressing frame 1623 failed; copying frame data instead
WARNING: error decompressing frame 1624; copying frame data instead
WARNING: error decompressing frame 1690; copying frame data instead
WARNING: error decompressing frame 1695; copying frame data instead
WARNING: recompressing frame 1796 failed; copying frame data instead
WARNING: recompressing frame 1848 failed; copying frame data instead
WARNING: error decompressing frame 1853; copying frame data instead
WARNING: recompressing frame 1872 failed; copying frame data instead
WARNING: error decompressing frame 1877; copying frame data instead
WARNING: error decompressing frame 1896; copying frame data instead
WARNING: error decompressing frame 1974; copying frame data instead
WARNING: error decompressing frame 1983; copying frame data instead
WARNING: error decompressing frame 2040; copying frame data instead
WARNING: error decompressing frame 2041; copying frame data instead
WARNING: error decompressing frame 2049; copying frame data instead
WARNING: error decompressing frame 2051; copying frame data instead
WARNING: error decompressing frame 2070; copying frame data instead
WARNING: error decompressing frame 2099; copying frame data instead
WARNING: error decompressing frame 2104; copying frame data instead
WARNING: error decompressing frame 2116; copying frame data instead
WARNING: error decompressing frame 2128; copying frame data instead
WARNING: recompressing frame 2183 failed; copying frame data instead
WARNING: error decompressing frame 2202; copying frame data instead
WARNING: error decompressing frame 2204; copying frame data instead
WARNING: error decompressing frame 2213; copying frame data instead
WARNING: error decompressing frame 2233; copying frame data instead
WARNING: error decompressing frame 2308; copying frame data instead
WARNING: recompressing frame 2389 failed; copying frame data instead
WARNING: error decompressing frame 2405; copying frame data instead
WARNING: error decompressing frame 2407; copying frame data instead
WARNING: error decompressing frame 2409; copying frame data instead
WARNING: error decompressing frame 2430; copying frame data instead
WARNING: error decompressing frame 2444; copying frame data instead
WARNING: error decompressing frame 2461; copying frame data instead
WARNING: error decompressing frame 2489; copying frame data instead
WARNING: error decompressing frame 2525; copying frame data instead
WARNING: recompressing frame 2528 failed; copying frame data instead
WARNING: error decompressing frame 2654; copying frame data instead
WARNING: error decompressing frame 2662; copying frame data instead
WARNING: recompressing frame 2663 failed; copying frame data instead
WARNING: error decompressing frame 2695; copying frame data instead
WARNING: error decompressing frame 2714; copying frame data instead
WARNING: error decompressing frame 2715; copying frame data instead
WARNING: error decompressing frame 2729; copying frame data instead
WARNING: recompressing frame 2740 failed; copying frame data instead
WARNING: error decompressing frame 2796; copying frame data instead
WARNING: error decompressing frame 2807; copying frame data instead
WARNING: error decompressing frame 2815; copying frame data instead
WARNING: error decompressing frame 2826; copying frame data instead
WARNING: recompressing frame 2845 failed; copying frame data instead
WARNING: error decompressing frame 2863; copying frame data instead
WARNING: recompressing frame 2920 failed; copying frame data instead
WARNING: error decompressing frame 2997; copying frame data instead
WARNING: error decompressing frame 3073; copying frame data instead
WARNING: error decompressing frame 3074; copying frame data instead
WARNING: recompressing frame 3094 failed; copying frame data instead
WARNING: error decompressing frame 3102; copying frame data instead
WARNING: error decompressing frame 3132; copying frame data instead
WARNING: error decompressing frame 3141; copying frame data instead
WARNING: error decompressing frame 3199; copying frame data instead
WARNING: error decompressing frame 3200; copying frame data instead
WARNING: error decompressing frame 3208; copying frame data instead
WARNING: error decompressing frame 3266; copying frame data instead
WARNING: recompressing frame 3274 failed; copying frame data instead
WARNING: error decompressing frame 3276; copying frame data instead
WARNING: error decompressing frame 3295; copying frame data instead
WARNING: recompressing frame 3341 failed; copying frame data instead
WARNING: error decompressing frame 3353; copying frame data instead
WARNING: error decompressing frame 3361; copying frame data instead
WARNING: error decompressing frame 3400; copying frame data instead
WARNING: error decompressing frame 3408; copying frame data instead
WARNING: error decompressing frame 3427; copying frame data instead
WARNING: error decompressing frame 3429; copying frame data instead
WARNING: recompressing frame 3438 failed; copying frame data instead
WARNING: error decompressing frame 3458; copying frame data instead
WARNING: error decompressing frame 3533; copying frame data instead
WARNING: recompressing frame 3630 failed; copying frame data instead
WARNING: error decompressing frame 3658; copying frame data instead
WARNING: error decompressing frame 3686; copying frame data instead
WARNING: recompressing frame 3714 failed; copying frame data instead
WARNING: error decompressing frame 3724; copying frame data instead
WARNING: error decompressing frame 3753; copying frame data instead
WARNING: error decompressing frame 3811; copying frame data instead
WARNING: error decompressing frame 3820; copying frame data instead
WARNING: error decompressing frame 3879; copying frame data instead
WARNING: error decompressing frame 3887; copying frame data instead
WARNING: recompressing frame 3888 failed; copying frame data instead
WARNING: recompressing frame 3907 failed; copying frame data instead
WARNING: error decompressing frame 3936; copying frame data instead
WARNING: error decompressing frame 3941; copying frame data instead
WARNING: error decompressing frame 3954; copying frame data instead
WARNING: recompressing frame 3965 failed; copying frame data instead
WARNING: error decompressing frame 4021; copying frame data instead
WARNING: error decompressing frame 4040; copying frame data instead
WARNING: error decompressing frame 4041; copying frame data instead
WARNING: error decompressing frame 4051; copying frame data instead
WARNING: recompressing frame 4070 failed; copying frame data instead
WARNING: recompressing frame 4145 failed; copying frame data instead
WARNING: error decompressing frame 4298; copying frame data instead
WARNING: error decompressing frame 4299; copying frame data instead
WARNING: error decompressing frame 4327; copying frame data instead
WARNING: error decompressing frame 4425; copying frame data instead
WARNING: error decompressing frame 4491; copying frame data instead
WARNING: recompressing frame 4499 failed; copying frame data instead
WARNING: error decompressing frame 4501; copying frame data instead
WARNING: error decompressing frame 4520; copying frame data instead
WARNING: error decompressing frame 4549; copying frame data instead
WARNING: error decompressing frame 4566; copying frame data instead
WARNING: error decompressing frame 4578; copying frame data instead
WARNING: error decompressing frame 4586; copying frame data instead
WARNING: recompressing frame 4633 failed; copying frame data instead
WARNING: recompressing frame 4652 failed; copying frame data instead
WARNING: error decompressing frame 4654; copying frame data instead
WARNING: recompressing frame 4663 failed; copying frame data instead
WARNING: error decompressing frame 4683; copying frame data instead
WARNING: error decompressing frame 4758; copying frame data instead
WARNING: recompressing frame 4855 failed; copying frame data instead
WARNING: error decompressing frame 4883; copying frame data instead
WARNING: error decompressing frame 4957; copying frame data instead
WARNING: error decompressing frame 4969; copying frame data instead
WARNING: error decompressing frame 5033; copying frame data instead
WARNING: recompressing frame 5045 failed; copying frame data instead
WARNING: error decompressing frame 5047; copying frame data instead
WARNING: error decompressing frame 5064; copying frame data instead
WARNING: error decompressing frame 5072; copying frame data instead
WARNING: recompressing frame 5084 failed; copying frame data instead
WARNING: error decompressing frame 5122; copying frame data instead
100% done with 5208 frames

On this new file MP3packer only reports the recompression errors:

CODE

mp3packer -z 8bp026-05-sabastian_boaz-cold_hard_fusion-vbr.mp3

*** '8bp026-05-sabastian_boaz-cold_hard_fusion-vbr.mp3' -> '8bp026-05-sabastian_boaz-cold_hard_fusion-vbr-vbr.mp3'
WARNING: recompressing frame 145 failed; copying frame data instead
WARNING: recompressing frame 232 failed; copying frame data instead
WARNING: recompressing frame 290 failed; copying frame data instead
WARNING: recompressing frame 470 failed; copying frame data instead
WARNING: recompressing frame 917 failed; copying frame data instead
WARNING: recompressing frame 1180 failed; copying frame data instead
WARNING: recompressing frame 1298 failed; copying frame data instead
WARNING: recompressing frame 1438 failed; copying frame data instead
WARNING: recompressing frame 1457 failed; copying frame data instead
WARNING: recompressing frame 1515 failed; copying frame data instead
WARNING: recompressing frame 1547 failed; copying frame data instead
WARNING: recompressing frame 1620 failed; copying frame data instead
WARNING: recompressing frame 1623 failed; copying frame data instead
WARNING: recompressing frame 1796 failed; copying frame data instead
WARNING: recompressing frame 1848 failed; copying frame data instead
WARNING: recompressing frame 1872 failed; copying frame data instead
WARNING: recompressing frame 2183 failed; copying frame data instead
WARNING: recompressing frame 2389 failed; copying frame data instead
WARNING: recompressing frame 2528 failed; copying frame data instead
WARNING: recompressing frame 2663 failed; copying frame data instead
WARNING: recompressing frame 2740 failed; copying frame data instead
WARNING: recompressing frame 2845 failed; copying frame data instead
WARNING: recompressing frame 2920 failed; copying frame data instead
WARNING: recompressing frame 3094 failed; copying frame data instead
WARNING: recompressing frame 3274 failed; copying frame data instead
WARNING: recompressing frame 3341 failed; copying frame data instead
WARNING: recompressing frame 3438 failed; copying frame data instead
WARNING: recompressing frame 3630 failed; copying frame data instead
WARNING: recompressing frame 3714 failed; copying frame data instead
WARNING: recompressing frame 3888 failed; copying frame data instead
WARNING: recompressing frame 3907 failed; copying frame data instead
WARNING: recompressing frame 3965 failed; copying frame data instead
WARNING: recompressing frame 4070 failed; copying frame data instead
WARNING: recompressing frame 4145 failed; copying frame data instead
WARNING: recompressing frame 4499 failed; copying frame data instead
WARNING: recompressing frame 4633 failed; copying frame data instead
WARNING: recompressing frame 4652 failed; copying frame data instead
WARNING: recompressing frame 4663 failed; copying frame data instead
WARNING: recompressing frame 4855 failed; copying frame data instead
WARNING: recompressing frame 5045 failed; copying frame data instead
WARNING: recompressing frame 5084 failed; copying frame data instead
100% done with 5208 frames

j7n

Mar 22 2007, 00:43

Mp3Packer does not preserve gapless information from an "Info" frame (Xing frame in constant bitrate files).

Omion

Mar 22 2007, 01:03

QUOTE(j7n @ Mar 21 2007, 23:43)

Mp3Packer does not preserve gapless information from an "Info" frame (Xing frame in constant bitrate files).

I've had no problems with it. I just did a quick CBR128 encode, which registers in Foobar as having the same sample offsets. Which LAME version are you using? Somebody may have screwed up the spec when I wasn't looking. (and an example file would be helpful too)

@Martin F.
Sorry for the delay. I got sick last week and haven't gotten up the energy to do much debugging. However, what you are describing does sound like a bug (as I said, the same errors should be in both files) I'll look into it as soon as the flu gives me the chance to. In the meantime, I'm going to bed

j7n

Mar 22 2007, 01:16

I manully added the gappless information using Foobar 0.9.4.1. I just checked that it didn't create the complete frame, only number of frames, filelength and gapless data. This was the problem, complete Info frames added by Foobar 0.8.3 were preserved.

However, VBR frames containing only the most important information were also created by earlier versions of Mp3DirectCut, and shouldn't be ignored. There is no way Foobar 9 will reindex an mp3 file it detects it as CBR.

Omion

Mar 23 2007, 00:12

QUOTE(j7n @ Mar 22 2007, 00:16)

I see the problem now. It's Foobar's fault.

It looks like Foobar still makes LAME tags without bothering to set the CRC. I already worked around this problem once in 1.07, and I don't want to bend the spec any more than I already have.
The CRC info at the end of a LAME tag is by far the best way of determining whether or not it's a LAME tag or just an XING tag with random garbage at the end. Bad CRC = no LAME tag.

The only exception I made to this was if all the extra LAME info was 0 except for the padding, it would still register as a LAME tag. This was specifically to work around Foobar not updating the frame correctly. However, it looks like now Foobar is smart enough to fill in some other info, but still can't get the CRC right.

In a nutshell, I won't fix this problem, simply because it is not mine to fix. The best way to get it fixed would be to post the issue in Foobar's support forum. The only other reference to the issue is in this post, so it's possible that the Foobar guys don't know about it yet.

Omion

Mar 23 2007, 00:42

@Martin F.:

I see the problem now. mp3packer says it keeps the input data on those frames, but in fact it still uses the recompressed data

It actually took me a bit of time trying to find the problem, since I had already fixed it in my working copy of mp3packer

. There is a certain type of bitstream error that is fairly common and all MP3 players seem to deal with it the same. In all the release versions this type of error was fixed nicely, but a warning came up saying the error was not fixed... The next version will simply fix the error and go on repacking.

(I know I'm kind of contradicting myself here: I said before that mp3packer will faithfully reproduce the input errors, but now I'm saying that it will fix those errors. However, the errors that I am fixing here are a small subclass of errors; they are extremely minor, relatively common, and all players deal with them the same. In these cases I think it is wise to simply fix the error, as playback will still be exactly the same.)

j7n

Mar 24 2007, 19:56

QUOTE

The best way to get it fixed would be to post the issue in Foobar's support forum. The only other reference to the issue is in this post, so it's possible that the Foobar guys don't know about it yet.

Done. But I'm afraid that post will gonna get ignored, since it says between the lines that Fb 0.8 did something better than 0.9 does.

Martin F.

Mar 30 2007, 09:05

Thanks for your reply, Omion. Interesting insights in some details of the MP3 format.

So here’s my next question

I noticed mp3packer -z is very fast on some files, for example on this (cough) 34 MB sized file (faster than on a usual file one tenth in size). The files on which I noticed this exceptional speed have a sampling rate of 22050 Hz in common. I don’t know why the sampling rate could have an effect on the compression speed, and the sampling rate and the speed may have no correlation at all

but it’s the only unusual attribute I could find on these files. Do you have an explanation for this?

// Edit: I’m having trouble using mp3packer on Windows 98 with a large file (2.12 GB):

CODE

D:\Martin>dir
[…]
FFN-CH~1 MP3 2.278.070.131  01.07.06  23:46 ffn-charts.mp3
MP3PAC~1 EXE       647.168  18.02.07  19:13 mp3packer.exe
[…]
         2 Datei(en)         2.278.717.299 Bytes
         4 Verzeichnis(se)       11.211,95 MB frei

D:\Martin>mp3packer -z ffn-charts.mp3
Fatal error: exception Failure("'ffn-charts.mp3' does not exist")

D:\Martin>mp3packer -z ffn-ch~1.mp3
Fatal error: exception Failure("'ffn-ch~1.mp3' does not exist")

j7n

Mar 30 2007, 11:58

I have a question about the bit reservoir

I was under impression that if I repack a medium bitrate stream with -b 320 parameter, the bit reservoir won't be used for most frames in the stream. I needed to edit a live recording @ 192 kBit/s and followed this path. The output stream contained lots of nulls, but still I was unable to cut the stream without glitches.

How is bit reservoir treated when repackaging mp3 data?

Omion

Mar 30 2007, 18:11

QUOTE(Martin F. @ Mar 30 2007, 08:05)

Thanks for your reply, Omion. Interesting insights in some details of the MP3 format.

So here’s my next question

but it’s the only unusual attribute I could find on these files. Do you have an explanation for this?

Actually, I seem to remember something about not enabling the -z switch with MPEG2 files (which are what files < 32KHz are) I don't remember why, but I'll see what happens when I re-enable it. I'm at work right now

so I can't do much here, but I'll test it out when I get home.

QUOTE

// Edit: I’m having trouble using mp3packer on Windows 98 with a large file (2.12 GB):

CODE

Accessing files larger than 2GB in Win 9x is not straightforward. I don't think OCaml can do it...

QUOTE(j7n @ Mar 30 2007, 10:58)

Actually, the way it works is that almost every frame will use the entire bit reservoir. Since almost no frames use all that space, the next frame will use up as much of the reservoir as possible. What that means is that using "-b 320" will actually lose the most data at the split point.

I have been thinking about trying to minimize the bit reservoir if the program determines that it won't take up any space, but when not using the "-b" switch this will need to cache very large chunks of the file. I'll see if there's a reasonable middle-ground to this.

What I would do to properly cut mp3 files is use pcutmp3, which is designed to avoid any errors. If whatever you use to play them back doesn't support LAME tags, then it won't quite be gapless, but it's as close as mp3s can get.

Jojo

Apr 29 2007, 21:39

I'm assuming that the program's output is lossless? I'm partially wondering about the -z switch. Is it lossless as well?

thanks

gib

Apr 29 2007, 22:19

It is lossless, with or without the -z switch. Obviously, if you have any concerns, you could always do a foobar bit-compare to verify the packed mp3 is lossless before tossing the original. I do that, but then I tend to be a bit overly cautious. heh

nemoW

May 1 2007, 11:32

Why MP3Packer (I use WinMP3Packer 1.13) rewrite 'encoder' field in mp3 files?

Omion

May 1 2007, 12:14

QUOTE(nemoW @ May 1 2007, 10:32)

Why MP3Packer (I use WinMP3Packer 1.13) rewrite 'encoder' field in mp3 files?

I did a little test with Tag&Rename, and it looks like it will guess the encoder if it is not available in a field anywhere.

LAME will put its version data in the padding info (you have to put something in the padding, so it might as well be useful) If Tag&Rename doesn't see any actual encoder field, it will use this padding data as the encoder.
That's great, except that the padding is exactly what mp3packer gets rid of! So when you put the files through mp3packer, that prevents Tag&Rename from guessing correctly.

So really, it's not rewriting the encoder field, but it is rewriting a hint that Tag&Rename uses to guess at the encoder. If the file stores the encoder data in the tags or in a LAME/XING frame, mp3packer will save it to the output file.

It is odd, though. When mp3packer can't find an encoder field, it will write its own name as the encoder (the header that mp3packer uses requires it) but it looks like Tag&Rename doesn't recognize this; it just assumes Xing for everything...

nemoW

May 1 2007, 15:05

QUOTE(Omion @ May 1 2007, 20:14)

I did a little test with Tag&Rename, and it looks like it will guess the encoder if it is not available in a field anywhere.

I am not familiar with mp3 file structure, but it seems like original files (from my 1st post) doesn't have LAME/XING header (here first bytes of file).

QUOTE

If the file stores the encoder data in the tags or in a LAME/XING frame, mp3packer will save it to the output file.

It is odd, though. When mp3packer can't find an encoder field, it will write its own name as the encoder (the header that mp3packer uses requires it) but it looks like Tag&Rename doesn't recognize this; it just assumes Xing for everything...

Hmm...

I've made aditional tests. Algorthm:
1. Encode small wav file with Lame 3.96.1 to cbr & vbr;
2. Convert with mp3packer cbr>vbr & vbr>cbr;
3. Convert again vbr>cbr & cbr>vbr;
Here interesting results.

Omion

May 1 2007, 15:42

QUOTE(nemoW @ May 1 2007, 14:05)

I am not familiar with mp3 file structure, but it seems like original files (from my 1st post) doesn't have LAME/XING header (here first bytes of file).

Yeah. That file just goes right into the MP3 data. No headers whatsoever.

QUOTE

mp3packer uses a slightly different header for CBR and VBR files, although they are functionally the same. It looks like that program is picking up the encoder from the VBR header, but not from the CBR header. But this doesn't explain why the VBR header isn't recognized in the files you mentioned on your first post, though...

buktore

May 3 2007, 08:18

I found something interrest. if i convert VBR > CBR > VBR

you got converted VBR file that (a bit)smaller than the original VBR.

maybe you can add an option to "squeeze VBR file" option or something.

Omion

May 3 2007, 08:58

QUOTE(buktore @ May 3 2007, 07:18)

I found something interrest. if i convert VBR > CBR > VBR

you got converted VBR file that (a bit)smaller than the original VBR.

maybe you can add an option to "squeeze VBR file" option or something.

You may notice that converting VBR > VBR gives the same improvement. The command line will take any MP3 file and repack it to a VBR by default. It looks like with the GUI you have to set "Input Types" to "All" to do the same thing.

How much smaller is the output file, though? It's rare to be able to compress it more than a couple hundred bytes without using the "-z" switch.

buktore

May 3 2007, 10:19

Ah.. you are right. VBR > VBR give the same file as VBR > CBR > VBR. Silly me.

and yeah. it get only tiny bit smaller. but i remember i once see that it's smaller enough to not show 0.0% (but how much.. i can't remember)

and thanks for this great tool

i love it.

PS.sorry if my ENG is bad

Omion

May 9 2007, 23:50

1.16 is out! Huzzah!

I basically completely re-wrote the entire thing, which is why it took almost three months. I did fairly extensive regression testing, but there may still have been bugs that slipped through. You are strongly encouraged to check the files before deleting the originals!

Here's what changed:

The -z switch now does something with MPEG2 / MPEG2.5 files (should fix j7n's issue)
It will now only resync if 3 consecutive frames can be found. Should fix j7n's other issue as well as various other people who have e-mailed me.
If the -b switch is used, the bit reservoir is minimized. This behavior may be changed by the -r and -R switches. This addresses j7n's impression. If, for example, a CBR320 file is outputted then the vast majority of its frames will not use the bit reservoir.
Major code cleanup to implement all of this. The source looks much better than before...

Also, along the way, I noticed and fixed the following issues:

If the first part of a frame is missing, the function which replaced it with nothing was not working correctly.
Occasionally the -z switch will output frames which are larger than the input, especially for low-bitrate files. mp3packer now recognizes this condition and will only use the repacked frame if it is smaller. (I should really fix the -z switch to never give a larger frame, but that will take a while)

The -r and -R switches are interesting. mp3packer used to try to minimize frame size, then maximize bit reservoir usage in the frame. This is exactly what you need to make the file as small as possible, but it had a tendency to peg the reservoir when the -b switch was used. I address this issue at the bottom of this post. mp3packer will now figure out the minimum bit reservoir for a frame to use without screwing up the bitrate of any frames after it. However, if the -b switch is not used the analysis may take a lot of memory, and probably won't do anything at all. Therefore, the bit reservoir is only minimized if the -b switch is given. Otherwise it is maximized like before.

The -r switch makes it so that the bitrate reservoir is always minimized, whereas the -R switch will maximize the reservoir (the normal operation for mp3packer before 1.16)

Whew! Done at last!

Martin F.

May 10 2007, 12:34

foo_bitcompare says

Comparing:
csr064_full-load-of-king_elevator-to-the-gallows_5_envelope-infrared_part-1.mp3
csr064_full-load-of-king_elevator-to-the-gallows_5_envelope-infrared_part-1-vbr.mp3
Comparing failed (length mismatch).

Comparing:
csr064_full-load-of-king_elevator-to-the-gallows_7_envelope-infrared_part-3.mp3
csr064_full-load-of-king_elevator-to-the-gallows_7_envelope-infrared_part-3-vbr.mp3
differences found: 3132 sample(s), starting at 71.0415193 second(s), peak: 0.0008336 at 71.0554422 second(s), 1ch

Also happened with the previous version (that reported lots of re- and/or decompression errors). I’ve used the -z switch.

Taken from csr064, author: Full Load of King, CC licensed

Omion

May 11 2007, 20:40

@Martin:

Well, part 1 looks OK with 1.16. I've noticed that in Foobar sometimes you have to force a reload in order for the length to be updated.

Part 3 is indeed doing something odd. I'm trying to figure it out now...

Martin F.

May 12 2007, 04:11

foobar2000 says

QUOTE

csr064_full-load-of-king_elevator-to-the-gallows_5_envelope-infrared_part-1.mp3
Duration : 4:02.770 (10706159 samples)

csr064_full-load-of-king_elevator-to-the-gallows_5_envelope-infrared_part-1-vbr.mp3
Duration : 4:02.796 (10707311 samples)

A reload doesn’t change it. I opened it in Cool Edit 2000 and it shows an additional (short) silent part at the beginning of the VBR file.

Omion

May 23 2007, 23:43

1.17 is out.

I completely remade the -z switch with the hopes of it getting twice as fast and slightly higher compression. That didn't happen. Instead, I got a -z switch which is 40% slower and slightly higher compression. Most of the time I spent went toward optimizing it (before optimization it went 3x slower

) Oh well. The -z switch was never meant to be fast.

The compression improvement (and speed decrease) is mainly evident in low-bitrate encodes. For high bitrates I noticed a whopping 0.03% improvement in the file size versus 1.16.

I also fixed a few bugs related to the -i switch. I hadn't tested it out properly when I released 1.16, so it printed a bunch of garbage and got one of the results wrong. This didn't affect the repacking at all, though.

Jojo

May 24 2007, 16:10

I assume neither of the changes affect WinMP3Packer, so that I just have to replace the .exe?

gottkaiser

May 24 2007, 18:54

@Omion

Thanks for developing this great tool!

Could you add a option to keep the time stamps of the old mp3 files?

This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.