IPB

Welcome Guest ( Log In | Register )

 
Reply to this topicStart new topic
special compressor for mpc and mp3, best lossless compression software for mpc and mp3 files
tripex
post Jun 23 2007, 23:35
Post #1





Group: Members
Posts: 15
Joined: 19-January 07
Member No.: 39799



Hello

I have a music library of about 40GB of mpc files (all -standard) and mp3 files (most --r3mix). The mpc part is about 80%.

I want to transfer this library completely via Internet from my home computer to my portable hard disk. I have no physical access to my computer but remote via Highspeed (192kbit upload) connection. This is not very fast so I want to precompress the library to maybe 60-80% with a lossless! compression software that is specialized in audio files (preferred mpc or mp3). At my Laptop I would decompress all files and enjoy bit-identical files.

What software/plugin/parameters are the best for this job?

Thanks for any tips.

CU
Andy
Go to the top of the page
+Quote Post
bryant
post Jun 24 2007, 00:12
Post #2


WavPack Developer


Group: Developer (Donating)
Posts: 1287
Joined: 3-January 02
From: San Francisco CA
Member No.: 900



Generally, MPC and MP3 files are already compressed about as well as they can be. You might be able to squeeze a few percent out of some of them, but probably not enough to make it worthwhile. If they were more compressable, then it would already have been used to reduce the bitrate!

A while back there was a discussion about a program written (in Russia, IIRC) that did a little better compressing encoded audio files, but I haven't heard anything about it lately.
Go to the top of the page
+Quote Post
Heliologue
post Jun 24 2007, 00:13
Post #3





Group: Members
Posts: 100
Joined: 17-September 06
Member No.: 35303



I know of no compression that will pack a lossy music file with anything approaching efficacy. You'll just have to put up with a long transfer.
Go to the top of the page
+Quote Post
TREX6662k6
post Jun 24 2007, 01:20
Post #4





Group: Members
Posts: 311
Joined: 20-August 06
From: United Kingdom
Member No.: 34237



Well if you can remote desktop you can use WMP to stream the sounds over the connection.

If you wana get technical you can set-up a VPN so you can access your SMB shares and stream them.

...Or you can put up with a slow download.


--------------------
http://www.last.fm/user/TREX6662k5/
Go to the top of the page
+Quote Post
DOS386
post Jun 24 2007, 02:21
Post #5





Group: Members
Posts: 64
Joined: 16-June 07
Member No.: 44412



QUOTE
This is not very fast so I want to precompress the library to maybe 60-80% with a lossless! compression software that is specialized in audio files (preferred mpc or mp3). At my Laptop I would decompress all files and enjoy bit-identical files.


Compress them by factor 3...5 ? Lostlessly ? shock1.gif

Impossible. Nonsense. Forget it. sad.gif

You won't "compress" them more than 1.05x at best ... they are already compressed.

QUOTE
discussion about a program written (in Russia, IIRC) that did a little better compressing encoded audio files, but I haven't heard anything about it lately.


YES, there are / used to be some around. All based on one the same idea: FRAUD.

They can make the file much smaller, and "restore" it later to original size and file type - but it's NOT lostless.


--------------------
/\/\/\/\/\/\
Go to the top of the page
+Quote Post
bryant
post Jun 24 2007, 06:14
Post #6


WavPack Developer


Group: Developer (Donating)
Posts: 1287
Joined: 3-January 02
From: San Francisco CA
Member No.: 900



QUOTE (DOS386 @ Jun 23 2007, 18:21) *
QUOTE

discussion about a program written (in Russia, IIRC) that did a little better compressing encoded audio files, but I haven't heard anything about it lately.


YES, there are / used to be some around. All based on one the same idea: FRAUD.

They can make the file much smaller, and "restore" it later to original size and file type - but it's NOT lostless.

No, this one was for real and was tested by several people here. The compression wasn't great (about 15% on music MP3 files) but it was lossless. Here's the thread:

http://www.hydrogenaudio.org/forums/index....showtopic=44102
Go to the top of the page
+Quote Post
DOS386
post Jun 24 2007, 07:09
Post #7





Group: Members
Posts: 64
Joined: 16-June 07
Member No.: 44412



QUOTE
No, this one was for real and was tested by several people here. The compression wasn't great (about 15% on music MP3 files) but it was lossless.


5% 10% 20% 25% lostless MP3 or JPG re-compression - is still far below request of topic starter ... and IMHO absolutely useless even if it works. It puts a bad light on JPG and MP3 formats and the guys should rather help improving useful lostless or lossy formats (FLAC, OGG Vorbis) or replacing them (JPG is obsolete ... replacement needed).


--------------------
/\/\/\/\/\/\
Go to the top of the page
+Quote Post
Seiitsu
post Jun 24 2007, 12:05
Post #8





Group: Members
Posts: 82
Joined: 26-January 07
Member No.: 40026



QUOTE (bryant @ Jun 24 2007, 07:14) *
QUOTE (DOS386 @ Jun 23 2007, 18:21) *

QUOTE

discussion about a program written (in Russia, IIRC) that did a little better compressing encoded audio files, but I haven't heard anything about it lately.


YES, there are / used to be some around. All based on one the same idea: FRAUD.

They can make the file much smaller, and "restore" it later to original size and file type - but it's NOT lostless.

No, this one was for real and was tested by several people here. The compression wasn't great (about 15% on music MP3 files) but it was lossless. Here's the thread:

http://www.hydrogenaudio.org/forums/index....showtopic=44102

15% is still quite a bit seeing as that mp3 files are rather heavily compressed already.
Go to the top of the page
+Quote Post
Fandango
post Jun 24 2007, 15:36
Post #9





Group: Members
Posts: 1546
Joined: 13-August 03
Member No.: 8353



In case you have many CBR MP3 files you can try at least to squeeze a few extra bytes out of them by repacking them into VBR files... losslessly.

http://www.hydrogenaudio.org/forums/index....showtopic=32379

I have no idea how much you'll gain.
Go to the top of the page
+Quote Post
kornchild2002
post Jun 24 2007, 16:19
Post #10





Group: Members
Posts: 2043
Joined: 8-April 05
From: Cincinnati, OH
Member No.: 21277



You could always use WinRAR to compress albums (or your entire library) without any quality loss. WinRAR works in the same way that WinZIP does. However, I agree with the other posts stating that you should just go on with the slow transfer speeds. Even with 40GB of music, it shouldn't take your computer more than a week and a few days to download all of your music.
Go to the top of the page
+Quote Post
Garf
post Jun 24 2007, 16:54
Post #11


Server Admin


Group: Admin
Posts: 4853
Joined: 24-September 01
Member No.: 13



QUOTE (Seiitsu @ Jun 24 2007, 13:05) *
QUOTE (bryant @ Jun 24 2007, 07:14) *

QUOTE (DOS386 @ Jun 23 2007, 18:21) *

QUOTE

discussion about a program written (in Russia, IIRC) that did a little better compressing encoded audio files, but I haven't heard anything about it lately.


YES, there are / used to be some around. All based on one the same idea: FRAUD.

They can make the file much smaller, and "restore" it later to original size and file type - but it's NOT lostless.

No, this one was for real and was tested by several people here. The compression wasn't great (about 15% on music MP3 files) but it was lossless. Here's the thread:

http://www.hydrogenaudio.org/forums/index....showtopic=44102

15% is still quite a bit seeing as that mp3 files are rather heavily compressed already.


It's trivial if you replace the huffman coder by a range or arithmetic coder. The technique is well known, it's used by H264 for example, to get an advantage over ASP.

There is a serious speed penalty.
Go to the top of the page
+Quote Post
prankstare
post Jun 24 2007, 18:30
Post #12





Group: Members
Posts: 94
Joined: 13-July 03
From: Brazil
Member No.: 7733



Could somebody explain to me how can an audio file compressed with a lossy encoder could ever reduce its filesize using another lossless codec? Is not the topic about transcoding lossy -> lossy?

QUOTE
... so I want to precompress the library to maybe 60-80% with a lossless! compression software that is specialized in audio files (preferred mpc or mp3).

Are you referring to compressing programs like WinRAR and 7-Zip? If so, there's no software in that category that is able to work out the filesize if the file has already been encoded with a lossy coder (pretty much on the contrary, it might increase instead).
Go to the top of the page
+Quote Post
Sebastian Mares
post Jun 24 2007, 19:06
Post #13





Group: Members
Posts: 3629
Joined: 14-May 03
From: Bad Herrenalb
Member No.: 6613



Even lossy codecs make use of lossless coding (Huffman). In order to save space, one can replace the Huffman coder with an arithmetic coder for example as suggested by Garf. This, however, will break compatibility to the existing MP3 standard.

The reason why WinZIP will not compress better is that it takes the file as-is and tries to compress it. The program mentioned here however decodes the Huffman codes first and then recompresses the decoded data using a more efficient algorithm. Basically, you can imagine it in this way: you have lots of stuff which you put in a box. However, you work inefficiently and don't think very well where to put each thing so you end up having "holes" between the elements. With WinZIP, you then take the box with the stuff and put it inside another box without gaining anything. The program described here however will take the box, unpack everything and rearrange the things in a more efficient way so that you end up having less or no holes between the things inside the box.

This post has been edited by Sebastian Mares: Jun 24 2007, 19:11


--------------------
http://listening-tests.hydrogenaudio.org/sebastian/
Go to the top of the page
+Quote Post
Fandango
post Jun 24 2007, 20:31
Post #14





Group: Members
Posts: 1546
Joined: 13-August 03
Member No.: 8353



Or in other words, this special russian compressor "re-writes" the MP3 files, while a normal compressor only compresses the MP3s files without needing to know anything about the format.

Actually I doubt that this MP3 packer can work lossless on every MP3 you throw at it, there's always a chance that it misses something when disassembling and then reassembling the MP3, or am I wrong about the MP3 standard here?
Go to the top of the page
+Quote Post
SebastianG
post Jun 25 2007, 12:29
Post #15





Group: Developer
Posts: 1317
Joined: 20-March 04
From: Göttingen (DE)
Member No.: 12875



QUOTE (Garf @ Jun 24 2007, 17:54) *
QUOTE (Seiitsu @ Jun 24 2007, 13:05) *

15% is still quite a bit seeing as that mp3 files are rather heavily compressed already.

It's trivial if you replace the huffman coder by a range or arithmetic coder. The technique is well known, it's used by H264 for example, to get an advantage over ASP.

It's really difficult to compare because there are a lot of things done differently in H264. Though, I remember an article of the German computer magazine "c't" where they said that by using CABAC instead of CAVLC you can save 15% of space. I always wanted to verify this using x264 in "constant quality" mode...

What about AAC-BSAC? Is this any good (let's ignore its complexity)?

btw: Nice analogy, Mr. Mares. wink.gif

Cheers!
SG

This post has been edited by SebastianG: Jun 25 2007, 12:36
Go to the top of the page
+Quote Post
tripex
post Sep 25 2007, 04:36
Post #16





Group: Members
Posts: 15
Joined: 19-January 07
Member No.: 39799



Thank you very much for all the replies. For reasons unknown I've got all the auto reply notifiers (about 14?) in compressed form not until today, so I thought nobody is caring about my issue.

In fact something like SoundSlimmer is what I was looking for. I don't care about the incompatibility as long as I can reverse the process to absolutely checksum identical files on my computer. This provides correctness via Internet transfer and I don't need to "trust" any lossless converter/optimizer. Unfortunately it works only for mp3 files which is the minority in my library.

The mp3repacker is also an idea but I guess it's almost the same than first decoding it to wav and second reencoding it via vbr. But even if it is better than the manual method, I won't use it b/c only a few of my anyway few mp3s are in cbr and I don't want to risk loosing any quality on them.

After playing around with paql compressor one night and gaining only 1 or 2% in exchange for a ridicules speed I decided to discarded the whole project. It was more an idea than necessary. I'll have physical access to my computer soon.

But again thanks for the ideas!

CU
Andy
Go to the top of the page
+Quote Post
pdq
post Sep 25 2007, 13:32
Post #17





Group: Members
Posts: 3305
Joined: 1-September 05
From: SE Pennsylvania
Member No.: 24233



QUOTE (tripex @ Sep 24 2007, 23:36) *
The mp3repacker is also an idea but I guess it's almost the same than first decoding it to wav and second reencoding it via vbr. But even if it is better than the manual method, I won't use it b/c only a few of my anyway few mp3s are in cbr and I don't want to risk loosing any quality on them.

Actually it is not the same at all. mp3repacker is truly lossless in that the results are bit identical when decoded. This is how it works: When you encode to CBR you are often telling the encoder to use more bits than it really needs, so there is unused space in the file. mp3repacker then closes up the unused spaces by making some of the frames smaller, thus converting the CBR file to VBR. The data in the file has not changed, only the unused space has been removed.

Of course, this doesn't help with your MPC files.
Go to the top of the page
+Quote Post
tripex
post Sep 25 2007, 15:40
Post #18





Group: Members
Posts: 15
Joined: 19-January 07
Member No.: 39799



QUOTE (pdq @ Sep 25 2007, 04:32) *
Actually it is not the same at all. mp3repacker is truly lossless in that the results are bit identical when decoded.


Sounds interesting though. Does that really mean I can convert any? cbr mp3 into vbr mp3 and reverse as often as I want and the checksum is always exactly the same? Does this also mean that the cbr mp3 is NEVER bigger than the vbr counterpart?

And finally I assume there is no "second" file with all the information necessary to rebuild the bit identical cbr, isn't it?

CU
Andy
Go to the top of the page
+Quote Post
pdq
post Sep 25 2007, 15:51
Post #19





Group: Members
Posts: 3305
Joined: 1-September 05
From: SE Pennsylvania
Member No.: 24233



QUOTE (tripex @ Sep 25 2007, 10:40) *
QUOTE (pdq @ Sep 25 2007, 04:32) *
Actually it is not the same at all. mp3repacker is truly lossless in that the results are bit identical when decoded.


Sounds interesting though. Does that really mean I can convert any? cbr mp3 into vbr mp3 and reverse as often as I want and the checksum is always exactly the same? Does this also mean that the cbr mp3 is NEVER bigger than the vbr counterpart?

And finally I assume there is no "second" file with all the information necessary to rebuild the bit identical cbr, isn't it?

CU
Andy

Yes, you can convert back and forth as many times as you want and the wav file decoded from the mp3 will always be identical.

The vbr should never be bigger than the cbr (at worst it will leave it unchanged) but the same is not true going from vbr to cbr. Sometimes though going from vbr to vbr, the file could get smaller, depending on how well the encoder packed the frames.

ALL of the information for decoding to wav is still in a single mp3 file, and the mp3 file is still compatible with all compliant decoders.

Edit: As I recall, the original incentive for writing mp3repacker had to do with 320 kbit cbr lame encodes. In some versions of lame there was quite a lot of unused space in these files sometimes, and mp3repacker would reduce their size significantly without any loss in sound quality. Other versions of lame gave much smaller savings though.

This post has been edited by pdq: Sep 25 2007, 15:57
Go to the top of the page
+Quote Post
tripex
post Sep 26 2007, 03:17
Post #20





Group: Members
Posts: 15
Joined: 19-January 07
Member No.: 39799



Well, I see that we are still in the mpc forum but I will give this lossless cbr<->vbr mp3 converter called mp3repacker a try whenever I have some time later. Even I don't have too much cbr mp3s I'll convert them to vbr and again to cbr just to check the crc32. If it matches I can surely delete the hopefully much bigger cbr ones.

CU
Andy
Go to the top of the page
+Quote Post
saratoga
post Sep 26 2007, 04:58
Post #21





Group: Members
Posts: 4715
Joined: 2-September 02
Member No.: 3264



QUOTE (tripex @ Sep 25 2007, 19:17) *
Well, I see that we are still in the mpc forum but I will give this lossless cbr<->vbr mp3 converter called mp3repacker a try whenever I have some time later. Even I don't have too much cbr mp3s I'll convert them to vbr and again to cbr just to check the crc32. If it matches I can surely delete the hopefully much bigger cbr ones.


You need to compare the decoded wav files, not the mp3s themselves (may not match), so just decode one to wav, repack it, and then decode that to wav.
Go to the top of the page
+Quote Post
slks
post Sep 29 2007, 19:38
Post #22





Group: Members
Posts: 374
Joined: 31-March 06
From: Houston, Texas
Member No.: 29046



Or better yet, just use foobar's Bit-Compare tool.
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 17th April 2014 - 20:20