MP3 repacker Options

[Omion]

@ominion

i dont have fobar her
but i did two diskwrites of the same mp3 as before (the original and not the repackwd)
and they came out bit identical

MD5
b526274595bd1b40fa744673c8b25f08 *first.wav
b526274595bd1b40fa744673c8b25f08 *Second.wav

CRC32
first.wav 75FDC377
Second.wav 75FDC377

So it definatly seems like the repack stuff is NOT lossless to the decoded stream

I figured it out. It's actually probably Winamp's fault. My program outputs a VBR stream, and so adds a VBR header at the front. This header is designed such that programs which don't support it will simply see an empty frame.

Winamp suppports the header, but it still plays the frame as an empty frame. Therefore, Winamp will add an extra 1152 null samples to a file with a VBR header. If the input file to mp3packer doesn't have a header, then Winamp will add an extra 1152 samples to the output file (which always has a header), thus making the MD5 different for the decoded files.

At least, I think that's how Winamp works. Foobar thinks my two test files are exactly the same, whereas Winamp outputs files with different MD5s.

In order to make sure, I took the two Winamp output files, deleted the WAV headers of the 2 (because the files have a different number of samples, which is encoded in the header) then deleted the first 1152 samples (4608 bytes) of the file from my program. The resulting files had the same MD5. That indicates that Winamp's output is the exact same, except for 1152 samples added to the front. If that makes any sense...

This post has been edited by Omion: Mar 20 2005, 03:39

[moozooh]

Okay, version 0.02 works. But still with some issues…

CODE

C:\Files\progs\Lame stuff\mp3packer>perl -v

This is perl, v5.6.1 built for MSWin32-x86-multi-thread
(with 1 registered patch, see perl -V for more detail)

Copyright 1987-2001, Larry Wall

<…>

CODE

C:\Files\progs\Lame stuff\mp3packer>perl mp3packer.pl ae01.mp3 ae01repacked.mp3
WARNING: Bit reservoir overflow on frame 8
WARNING: Bit reservoir overflow on frame 9
WARNING: Bit reservoir overflow on frame 10
WARNING: Bit reservoir overflow on frame 11
WARNING: Bit reservoir overflow on frame 12
WARNING: Bit reservoir overflow on frame 13
WARNING: Bit reservoir overflow on frame 14
WARNING: Bit reservoir overflow on frame 15
WARNING: Bit reservoir overflow on frame 16
On frame 18

ERROR! Found more data than could be encoded in one frame!
Frame number 17 uses 1214 bytes, of which 1170 bytes need to be stored in
the current frame. (Note that the max data stored in a frame is 1009)
Try increasing the additional reservoir (-a switch), currently 0.
(The additional reservoir must be from 0 to 511)

C:\Files\progs\Lame stuff\mp3packer>perl mp3packer.pl -a 511 ae01.mp3 ae01repacked.mp3

Here it does massive BR overflow on almost every frame then halts and writes this:

CODE

ERROR! Found more data than could be encoded in one frame!
Frame number 3064 uses 1257 bytes, of which 1062 bytes need to be stored in the current frame. (Note that the max data stored in a frame is 1009)
Try increasing the additional reservoir (-a switch), currently 511.
(The additional reservoir must be from 0 to 511)

The file was 9:23 track encoded (not by me though) in 256 kbps cbr (lame 3.91). Encspot doesn't report any problems.
What do I do now?

[Omion]

Okay, version 0.02 works. But still with some issues…

CODE

C:\Files\progs\Lame stuff\mp3packer>perl mp3packer.pl -a 511 ae01.mp3 ae01repacked.mp3

Here it does massive BR overflow on almost every frame then halts and writes this:

CODE

ERROR! Found more data than could be encoded in one frame!
Frame number 3064 uses 1257 bytes, of which 1062 bytes need to be stored in the current frame. (Note that the max data stored in a frame is 1009)
Try increasing the additional reservoir (-a switch), currently 511.
(The additional reservoir must be from 0 to 511)

The file was 9:23 track encoded (not by me though) in 256 kbps cbr (lame 3.91). Encspot doesn't report any problems.
What do I do now?

Try increasing the -p switch. It's 100 by default. Use "mp3packer -a 256 -p 200". If THAT doesn't work, then I'll need to do something about my frame-too-big handling. I'll start working on a better solution now... The problem is that, with many LAME encodes (especially CBR 256 -- you picked the worst one) the file might require at least n bytes in the reservoir, but my program tries to minimize the amount of bytes in the bit reservoir, so many times there's fewer than n bytes, so the program panics.

Encspot wouldn't notice any problems in the input file, simply because there really aren't any problems. It's just the way my program handles (or fails to handle) some files.

I'm pretty sure I can do some sort of "pre-pass" on the file, log all the frames which are larger than 320kbps, and try to only pad the reservoir right before them. I'll get to work on it, but I have no idea when it'll be done.

This post has been edited by Omion: Mar 20 2005, 23:36

[moozooh]

It works now, but only with -p 250 or more, the result is just stripping a few kbps.
Edit: “A few” = 2-8 kbps (tested on 224-256-320 kbps files).

This post has been edited by Mo0zOoH: Mar 22 2005, 20:56

[Jojo]

I haven't had the chance yet to give it a try...but is it true that CBR files will end up as VBR files?

[Omion]

It works now, but only with -p 250 or more, the result is just stripping a few kbps.
Edit: “A few” = 2-8 kbps (tested on 224-256-320 kbps files).

Hmmm... Do you know what program made the files? In my experience with both LAME and the iTunes encoder, CBR 320 files go down to around 290. I can believe that 224 won't give much improvement, though. The lower the bitrate, the more the files actually need all of it.

I haven't had the chance yet to give it a try...but is it true that CBR files will end up as VBR files?

Yup. That's how it works. It uses the minimal size for each frame, which almost always results in a VBR file. (Just like I said in post #10 ) A valid LAME VBR header is written, so programs will handle it the same way as they do every VBR file, if that's what you're wondering.

By the way, I made a new version which intelligently figures out where padding needs to be placed, so you don't need to screw around with the -a and -p switches. The downside to this is it either has to read the file twice (goes almost half as fast) or store the entire file into memory (uses lots of memory) I'll post it once I finish testing it.

[Omion]

Just released version 0.03. Now you no longer need to fiddle with the -a and -p switches (actually they aren't even supported any more). It shouldn't complain about not having enough bits in the bit reservoir again.

I do recommend that you use the -m switch, as it tends to go about twice as fast. It crams the entire file into memory, so it's not good for large files. For standard 1-song-per-file MP3s it's fairly safe to use -m.

[kevinsham]

I get this error for one file

On frame 7435 of 7536 (98substr outside of string at mp3packer/mp3.pm line 996.

and the resulting mp3 has no header


Also, i find some 320kbps files that are not packed at all.

[Lyx]

@Omion

Would it be possible to add a parameter to the m-switch where you can specify a value in megabytes. If the file is larger than the specified value, then it will not be loaded into memory, if it is smaller or equal then it will be loaded into memory. This could be useful for batch-repacking in the future when your tool becomes stable.

- Lyx

[Antonski]

I get this error for one file

On frame 7435 of 7536 (98substr outside of string at mp3packer/mp3.pm line 996.

and the resulting mp3 has no header


Also, i find some 320kbps files that are not packed at all.

I also got the same error when I tried to re-pack a new-Xing vbr file (according to EncSpot).
However other 9 cbr files (also Xing, old and new) were repacked without errors.

BTW, I don't think that Perl is not a "real" programming language

[djdance]

Boys, please let me simply download .EXE without any compiling, well?
Would anybody likes to upload exe and put the link here?

[kjoonlee]

Um, you don't need a repacker binary. Use perl to run it.

[Omion]

I get this error for one file

On frame 7435 of 7536 (98substr outside of string at mp3packer/mp3.pm line 996.

and the resulting mp3 has no header


Also, i find some 320kbps files that are not packed at all.

I also got the same error when I tried to re-pack a new-Xing vbr file (according to EncSpot).
However other 9 cbr files (also Xing, old and new) were repacked without errors.

BTW, I don't think that Perl is not a "real" programming language

THAT's not good. Looks like I've made more bugs than I've squashed with the lastest version. I may have missed something whilst re-reading the XING tag spec. Can either of you send, say, the first 1KB of the problem file to me? I've PMed you my email address.

Unfortunately, I don't know how most people cut up a file, so I can't tell you how to send the first 1KB, but I'm sure someone here does. I use an old hex editor called "Hedit" which doesn't even exist any more, so it's not really of any use.

By the way, does this only happen with version 0.03, or does 0.02 do it too?
[edit:] Also, does the -m switch affect the problem? [/edit]

@Lyx:
Adding a cutoff value for the -m switch would not only be possible, it would be extremely easy and useful. I don't know why I didn't think of it before. I'll add it to the next version, along with the fix to this bug people are getting.

This post has been edited by Omion: Mar 26 2005, 08:22

[kjoonlee]

For chopping, I use dd from GNU coreutils.

dd if=input.mp3 of=chopped_file bs=1000 count=1

edit: If you're running Windows, MinGW, Cygwin, or GnuWin32 should let you run coreutils.

This post has been edited by kjoonlee: Mar 26 2005, 10:00

[moozooh]

0.3 works good for me, thanks.

[Antonski]

Can either of you send, say, the first 1KB of the problem file to me? I've PMed you my email address.

Sure, I'll send it in Monday (that file is on my office desktop).
Meanwhile I'll try to reproduce the problem on my home PC (if my wife allow me to go closer to the comp )

BTW, the problem could be with that file and not with your program. I even don't remember where I have downloaded it from.

Tony

[Omion]

Released version 0.04.

Hopefully the previous "substr outside of string" error has been fixed. Thanks, Antonski and kevinsham, for sending me the files that broke.

Also added a "-M" option, which has one argument. If the file size is smaller than the argument, in MiB, then the "-m" option is turned on. Otherwise, it's turned off.

There was another problem which may or may not have existed, but it definitely shouldn't exist any more. The calculation for the XING TOC was off by around 100 bytes, which may have caused the XING tag to overflow and corrupt the first frame on very small files.

[Antonski]

Hi Reed,

I'm afraid I've encounered a new problem with repacker, this time workng on my old home sweet Win98SE. I have Active Perl 5.6.1.
After repacking of every file the program generates a message "Illegal division by zero at mp3.pm line 785." and there is no Xing header in the output file (as far as I can see).
There is no problem with Win2000 though.
If you are interested, I would send you the the headers of the original file and both outputs - correct and incorrect.

BTW, I really like that program a lot. And Perl

~Tony

[Omion]

@Antonski -
Hmm.... That's where the program divides by the size of the output file. Maybe win98 doesn't update the size of open filehandles or something. Try replacing your mp3.pm with this one. If that doesn't work, then I'll have to think of something else.

This post has been edited by Omion: Apr 16 2005, 22:07

[Antonski]

Try replacing your mp3.pm with this one.

Works perfectly!
Thank you

[Deep_Elem]

I've been tinkering with v.0.04 using the latest version of Perl on Win XP. I'm getting pretty good results, even with 192 kbps CBR LAME files. But I have noticed two issues:

1. My original LAME files have proper LAME tags, but the new files have no LAME tags (so I am adding them manually via fb2k).

2. Encspot now reports the encoder used as Xing or Gogo but the original files are definitely LAME.

[Omion]

1. That's weird. If the input files have LAME tags, the output files should too. Otherwise, a simple XING tag is added. What version of LAME do you have, and how did you check if the files have the tag? It's possible that my program thinks the tag is incorrectly formed, and so just stores a standard XING tag. (If it doesn't write an XING tag either, that's very bad)

2. Well, the audio data is the same, so it shouldn't matter. Encspot just sees that the files are arranged differently from what LAME usually does, and guesses something else. The quality is the same as the original. (Except if a LAME tag is written, in which case EncSpot parrots the encoder string)

This post has been edited by Omion: Apr 18 2005, 00:33

[asonicboom]

Sorry to resurrect this thread on my first post but I really do like MP3 Packer and had a few questions...

Omion I use your program on all my old --preset insane files to remove tags and in the process shave a few kbits off the filesize. Am I right in saying that it also strips off any "bad last frames" that it encounters (I think you confirmed this in an earlier post) ?

When I recently tried to repack a 192kbps CBR mp3 Encspot showed the outputted file as having 224, 256 or 320 kpbs bit rates for some frames. Is this possible ? Is the mechanism similar to:

If LAME decides a frame needs more then 320kbps, it will take the needed bits from the bit reservoir.... As of version 0.03, very large frames are identified early enough to pack the reservoir dynamically.

I sincerely hope you are still developing this great little proggie, batch processing and perhaps an executable and a GUI would be very welcome indeed although I am perfecty content with the app in its current form (if anyone reading could compile MP3 Repacker into a stable binary or show me how to go about it I would be eternally grateful ). As it is I currently use Florian's amazing MP3Tag program with mp3repacker inserted as a tool like so:

CODE

Name: MP3 Packer
Path: C:\WINDOWS\system32\cmd.exe
Parameter: /c "cd /d "E:\Backup\Programs\MP3 Packer\MP3 Packer v0.04" && perl mp3packer.pl -mst "%_path%" "%_folderpath%\%_filename%vbr.mp3""

with "for all selected files" ticked. Provided the user adjusts the absolute path to the two MP3 Packer files accordingly then it allows any number of mp3s in a directory to be packed to output files in the same folder but with vbr appended to the end of the filename by simply highlighting the mp3s, right clicking and selecting the MP3 Packer tool. This pseudo-batch processing comes in quite handy so I hope those instructions help anyone else in the same situation as me. Omion I was wondering if you could tell me if it's "safe" to run so many MP3 Packers concurrently ?

Lastly I was hoping you could tell me if you are aware of any bugs that would make running Mp3 Packer on all of my old unreplaceable mp3s a bad decision...

Thanks for all your hard work and effort

--asonicboom

[Omion]

Sorry to resurrect this thread on my first post but I really do like MP3 Packer and had a few questions...

Omion I use your program on all my old --preset insane files to remove tags and in the process shave a few kbits off the filesize. Am I right in saying that it also strips off any "bad last frames" that it encounters (I think you confirmed this in an earlier post) ?

If a bad last frame is encountered, it is simply added to the "non-MP3 junk at the end" category. That means that if you use the -s option the bad frame will be thrown out, otherwise it will be included but not optimized.

When I recently tried to repack a 192kbps CBR mp3 Encspot showed the outputted file as having 224, 256 or 320 kpbs bit rates for some frames. Is this possible ? Is the mechanism similar to:

If LAME decides a frame needs more then 320kbps, it will take the needed bits from the bit reservoir.... As of version 0.03, very large frames are identified early enough to pack the reservoir dynamically.

It is possible, expected, and happens quite a lot . Due to the bit reservoir, the amount of data stored in an MP3 frame has very little relation to the bitrate of that frame. For example, here's a graph of a 192-CBR file. The red is the size of each frame (constant, of course) but the black dots are the actual amount of data in each frame. The blue is how many bits are in the reservoir per frame.

You can see that the bit reservoir is relatively large compared to the frame size; this means that data for one frame is far away from the actual frame header. The output of my program, though of similar size, has a much different structure:

You can see that there is less data in the reservoir, and the frames have sizes from 32-320kbps, depending on how much actual data there is. The top row of red dots is 320, the next one down is 256, etc.

I sincerely hope you are still developing this great little proggie, batch processing and perhaps an executable and a GUI would be very welcome indeed although I am perfecty content with the app in its current form (if anyone reading could compile MP3 Repacker into a stable binary or show me how to go about it I would be eternally grateful ).

I've been trying to figure out how to make binaries from Perl, but my Perl doesn't seem to want to.

I don't have any experience or desire to make a GUI for the program, but if you want to, feel free .

As it is I currently use Florian's amazing MP3Tag program with mp3repacker inserted as a tool like so:

CODE

Name: MP3 Packer
Path: C:\WINDOWS\system32\cmd.exe
Parameter: /c "cd /d "E:\Backup\Programs\MP3 Packer\MP3 Packer v0.04" && perl mp3packer.pl -mst "%_path%" "%_folderpath%\%_filename%vbr.mp3""

with "for all selected files" ticked. Provided the user adjusts the absolute path to the two MP3 Packer files accordingly then it allows any number of mp3s in a directory to be packed to output files in the same folder but with vbr appended to the end of the filename by simply highlighting the mp3s, right clicking and selecting the MP3 Packer tool. This pseudo-batch processing comes in quite handy so I hope those instructions help anyone else in the same situation as me. Omion I was wondering if you could tell me if it's "safe" to run so many MP3 Packers concurrently ?

As long as you don't have them writing to the same file, running multiple repackers in parallel isn't a problem. (If you do have them writing to the same file, they won't corrupt it; they'll just exit with a "permission denied" error)

Lastly I was hoping you could tell me if you are aware of any bugs that would make running Mp3 Packer on all of my old unreplaceable mp3s a bad decision...

I don't know of any bugs in the program; it seems fairly stable. However, if you want to extra assurance, you can run the input and output files through Foobar's "bit-compare files". If that says they're OK, then you can delete the old one. If it says they're different, then post to this thread, as it would indicate a bug.

Thanks for all your hard work and effort

--asonicboom

Thank you for the feedback! It's good to know that people are using my program.

[edit from 7 years in the future: fixed the images to point to my current server]

This post has been edited by Omion: Aug 2 2012, 06:05

[asonicboom]

Thanks for replying so soon and so comprehensively Omion, your answers were very helpful Again good work on making such a great little program.

If that says they're OK, then you can delete the old one. If it says they're different, then post to this thread, as it would indicate a bug.

Will do Till then...

--asonicboom