Help - Search - Members - Calendar
Full Version: To rar or not to rar, music collection ;)...
Hydrogenaudio Forums > Hydrogenaudio Forum > General Audio
MusicLover
I think that CD-R recording is handy... But should I rar or zip mpcs? There is an opinion on the forum, that this is more reliable... But it is not so handy to listen rared mpcs, isn't it? (If it is just possible)...
LoKi128
There is no real reason to RAR or ZIP any type of already compressed music files. It is not for the reliability, since I have never been able to fix a ZIP archive. I have no experience in that respect with RARs tho. It's not for the compression either, since MPCs will not compress well at all.

I would recomend just burn them straight into the CDR. Oh and make sure they are all tagged with your favorite tagger so you can recover any really long filenames that might be truncated by Joliett.
atici
Don't compress with rar. The resulting file will possibly be larger otherwise. Rule of information theory. If rar could compress better than mpc when it comes to music, then we'd all be using rar anyway:) After a single mpc pass your data should ideally be totally random. Any means of compression in the universe make use of the redundancies in the information. That means any general data compressor could not compress mpc file any further.
dgover2
QUOTE(atici @ Feb 20 2003 - 10:38 AM)
Don't compress with rar. The resulting file will possibly be larger otherwise. Rule of information theory. If rar could compress better than mpc when it comes to music, then we'd all be using rar anyway:) After a single mpc pass your data should ideally be totally random. Any means of compression in the universe make use of the redundancies in the information. That means any general data compressor could not compress mpc file any further.

Isn't it more accurate to say that an MPC file is encoded, it's not really compression is it? I mean, yes the file won't compress any further because MPC also uses lossless compression - but basically, MP3, MPC, etc are encoders that also use compression to limit the filesize further, right?

Cheers,
-dave
dgover2
QUOTE
I would recomend just burn them straight into the CDR. Oh and make sure they are all tagged with your favorite tagger so you can recover any really long filenames that might be truncated by Joliett.


Or maybe check out the Romeo filesystem. I've not used it at all yet but AFAIK it lets you use up to 128 character filenames. I don't know if that breaks compatibility with mp3 players etc though. If you are just writing your MP3's to CD for backup purposes it should be fine though, I guess.

Cheers,
-dave
dreamliner77
I still like the recovery record option with WinRAR
Neo Neko
QUOTE(dgover2 @ Feb 20 2003 - 12:48 PM)
QUOTE(atici @ Feb 20 2003 - 10:38 AM)
Don't compress with rar. The resulting file will possibly be larger otherwise. Rule of information theory. If rar could compress better than mpc when it comes to music, then we'd all be using rar anyway:) After a single mpc pass your data should ideally be totally random. Any means of compression in the universe make use of the redundancies in the information. That means any general data compressor could not compress mpc file any further.

Isn't it more accurate to say that an MPC file is encoded, it's not really compression is it? I mean, yes the file won't compress any further because MPC also uses lossless compression - but basically, MP3, MPC, etc are encoders that also use compression to limit the filesize further, right?

Cheers,
-dave

The files are encoded with an auditory compression scheme. It is compression. And MPC is not lossless.
LoKi128
QUOTE(dreamliner77 @ Feb 20 2003 - 02:47 PM)
I still like the recovery record option with WinRAR

Hmm... a neat experiment would be:

1. make a RAR with a recovery record
2. open it in notepad
3. add a few random characters here and there... try to pretend you are a "dirty CD"
4. chop off the last few bytes... cause your TCP connection just got dropped and the server is gone
5. save and try to reopen with WinRAR

to be quite honest, i doubt you will be able to get anything back. Thats the whole deal with compression... EVERY bit counts. WinRAR only "protects" up to 10% of the filesize... so basically only 10% of the file is protected. With perfect compression, recovery of the data with even just one bit wrong would be impossible.

If your data is that valuable, just make two copies. And keep one at a different location.
Jospoortvliet
QUOTE(LoKi128 @ Feb 20 2003 - 09:40 PM)
QUOTE(dreamliner77 @ Feb 20 2003 - 02:47 PM)
I still like the recovery record option with WinRAR

Hmm... a neat experiment would be:

--

to be quite honest, i doubt you will be able to get anything back. Thats the whole deal with compression... EVERY bit counts. WinRAR only "protects" up to 10% of the filesize... so basically only 10% of the file is protected. With perfect compression, recovery of the data with even just one bit wrong would be impossible.

actually, this IS what winRAR claims: the recovery record should be able to fix this destroyed data.

its error-corection, like on an normal audio cd: if there are scratches on an audio cd, which are not longer than some .5-1 cm, the error-correction can recover the data, and you wont hear clicks... re-reads and advanced error correction (EAC, Plextools) can even recover more.

but the files will be larger, and there must be other ways for adding an recovery record - im sure special progs are there to protect data much better than winRAR can do.

about the compression: for creating mpc files, first all data that can be trown away is deleted by mppenc, than A NORMAL (HUFFMAN) COMPRESSION ALGORITM is used to compress the data left. as far as I know. thats why winRAR cant compress more.

but maybe it whould be possible to use, instead of the hoffman coding, another algoritm which compresses better. a kind a combo of mpc and ape might further improve compression, i dont know enough about these matters.

grtz
ViPER1313
QUOTE
Hmm... a neat experiment would be:

1. make a RAR with a recovery record
2. open it in notepad
3. add a few random characters here and there... try to pretend you are a "dirty CD"
4. chop off the last few bytes... cause your TCP connection just got dropped and the server is gone
5. save and try to reopen with WinRAR

to be quite honest, i doubt you will be able to get anything back. Thats the whole deal with compression... EVERY bit counts. WinRAR only "protects" up to 10% of the filesize... so basically only 10% of the file is protected. With perfect compression, recovery of the data with even just one bit wrong would be impossible.

If your data is that valuable, just make two copies. And keep one at a different location.


I have much of my music collection backed up using .rar with no compression / 5% recovery rate. When I was burning the files to CD-R's, some of the CD's had burning errors / NERO failed to at data verification. The .rar files would not copy as a whole to the HD either. Because I was interested, I tried to recover the files from the CD's, and it WORKED - not a single problem getting all the files back out of the .rar B) . I feel pretty safe using .rar to backup my music collection after that - at least safer than with no error correction.
Messer
QUOTE(dreamliner77 @ Feb 20 2003 - 09:47 PM)
I still like the recovery record option with WinRAR

For this task I'd rather use Parchive to create as much recovery files as one think is needed and keep them with the originals.
buzzy
I agree, if you're worried about the files, then create some par files, those could be used to recover any corrupted mpcs, but you can still keep the file set in mpc format.
LoKi128
Actually I have to retract my above statement about WinRAR. I had a little break so I made a small RAR and played around with UltraEdit in binary mode.

I changed some bits around. First just one bit, then one byte, then more bytes etc. In most cases WinRAR could recover the file. I was using 10% recovery record. It took quite a bit of changing before the file was unrecoverable.

I was plesantly surprised. It handled a good amount of errors in the file before messing up. Things sure have improved since the days of ZIP repair. Still, I would just save two copies of any important files just because I'm old fashioned like that.
mithrandir
I would store them uncompressed so you can easily play them right from the CD-R.
Negative Zero
Exactly. There's no point in trying to ZIP or RAR compressed audio files to save space...
Neo Neko
You miss the point it is not to save space per say. More for the extra file recovery features winrar offers. So all in all they are expending more space.
Lev
"To rar or not to rar"?

Not to rar
krmathis
QUOTE(mithrandir @ Feb 21 2003 - 06:18 AM)
I would store them uncompressed so you can easily play them right from the CD-R.

If you have missed it I will inform you about this:
foobar2000 plays audio files directly from *.rar and *.zip via external Archive reader plugin...

Working great here.. biggrin.gif
dgover2
QUOTE
The files are encoded with an auditory compression scheme. It is compression. And MPC is not lossless.


What I actually meant is, that an MP3, Vorbis, MPC or whatever file is not actually a compressed WAV (for example), it's actually a seperate file format with data that roughly represents the original waveform, with a more accurate representation the higher the bitrate you use. (Somebody correct me if I'm wrong here).

I read somewhere else that MPC also uses lossless compression. Not in the same sense that FLAC of SHN uses lossless compression but to actually further compress the data to reduce the filesize that bit more. I suppose that the MP3, MPC, whatever representation of the waveform may still contain some redundancy.

Anyway this is just what I've managed to "guess" from what little information I have found about "compressed" (I prefer to say encoded) audio formats that is written in plain english.

I'd actually be interested to hear a plain english description of the process used in creating an MP3 or MPC file from somebody with good knowledge of the subject.

Is there anybody out there?

Cheers,
-dave
ChristianHJW
It makes sense to ask for extra safety when burning MPCs on disc, as current MPC container doesnt offer any ECC/EDC components, so in case you have heavy scratches on your CD you might loose the file. Highly unlikely, but possible sad.gif ...

Note that Frank Klemm is working with the matroska team on making MPC SV8 available in matroska container files, and matroska can handle any amount of ECC/EDC elements ( thanks to the flexibility of EBML ) to protect the vital data, such as headers, huffman tables, etc. . We are already working on a tool that will allow the user to add a certain amount of safety elelements, depending on his perferences.

This tool was originally planned for mode2 form2 burning of video files, allowing us to burn 800 MB movies on a normal 80 mins CD, but with reduced safety as there is no 3rd layer ECC being used in mode2 form2. The XCD project does that aleady today for OGM, MP4 and even AVI files, but it will simply make a copy of the first few sectors of the file ( where the header is in ) in a separate mode2 form1 track, so there is no special safety added for important parts within the file, which is important for music backups as you may have dozens of files on there, and all need their headers to play correctly, not just the first one.

The new tool we are working on ( with the support of DeXT and avih, the XCD authors ) can do a bit more than this. Lets suppose you have a movie with 765 MB and your buner can burn 810 MB nicely on a normal 80 mins CD in mode2 form2 ( slight overburning, very normal for S-VCDs ). Our tool will now allow the user to add 810 - 765 = 45 MB of ECC/EDC safety elements throughout the file, thus making very best use of the available space on the CD by giving the most intelligent safety.

Its well possible that we will create a special MPC music backup profile for that tool , now that we see there might be interest in some extra protection for important music collections. Of course, the tool can also be used for normal mode2 form1 burning, it doesnt force you to use form2. if you have any questions about this tool pls. drop an email to the responsible project manager for it to spyder at matroska dot org .

Christian
Althalus
i would go with par's, but remember that the size of the pars will be the same as the biggest of the files. so if you have songs that are 1mb to 20mb each par will be 20mb smile.gif

but if you rar (store) them to say 5mb parts the pars will only be 5mb

"to par or not to par"

to par (... if it is important data)
Neo Neko
QUOTE(dgover2 @ Feb 21 2003 - 04:36 AM)
QUOTE

The files are encoded with an auditory compression scheme. It is compression. And MPC is not lossless.


What I actually meant is, that an MP3, Vorbis, MPC or whatever file is not actually a compressed WAV (for example), it's actually a seperate file format with data that roughly represents the original waveform, with a more accurate representation the higher the bitrate you use. (Somebody correct me if I'm wrong here).


There are 2 types of compression Lossless(ZIP,RAR,ACE,TAR,MAC,FLAC,SHN) Lossy(MP3,WMA,Vorbis,MPC,VQF,ADPCM). Just because a compressed file is not bit identical to the original that does not mean it was not compressed. Encoded is a propper term here. But so is compressed. One does not preclude the other.

QUOTE
Originally posted by kvs
[B]I read somewhere else that MPC also uses lossless compression. Not in the same sense that FLAC of SHN uses lossless compression but to actually further compress the data to reduce the filesize that bit more. I suppose that the MP3, MPC, whatever representation of the waveform may still contain some redundancy.


MPC is generally regarded as the next best thing to lossless. But it is lossy.

QUOTE
Originally posted by kvs
[B]I'd actually be interested to hear a plain english description of the process used in creating an MP3 or MPC file from somebody with good knowledge of the subject.

Is there anybody out there?

Cheers,
-dave


Well unless you are ok with casually throwing around terms like Huffman Tables, Fast Fourrier Transform, Sub Band Processing, etc I doubt you will get anything suitably plain here. Some of the guys here know their stuff so well that they can break out the techno jargon and acronyms so fast your eyes will glaze over instantly.
Jojo
Hehe, I think I'm the person that is responsible for all that...since I started a similar thread biggrin.gif
Well, I came to a conclusion...I'll just burn everything 2 times. On one CD I'll store the plain mp3 files, so that I can easily listen to it and take it with me...the second copy will be a backup, so I put a 10% recovery record to each *.rar file. I will never use the second CD unless it is necessary so I keep the CD in a closet with no light and stuff...
BadMath
For whats it's worth I use .zip's to store all my music on CD-R's. I don't care about error correction or compression (as was mentioned you will get maybe a few kilobytes of saved space zipping mp3/ogg/mpc). The reason I zip them is to avoid the stupid Joliet filename length limit. I could use a program to rename the files from tags but it's faster to right click on the folder and select "add to archive <fildername>" before I burn.

Also my school likes to delete mp3's from our network drives and our webspace so zipping the albums allows me to use the schools network drives without fear of them wiping it all away. Plus I take pleasure in the fact that they are backing up files they don't even want on their drives!

It also makes uploading to my friends ftp's easier. My schools connection is terrible and everytime I want to upload a file I have to kind of "re-establish" a connection. I can't explain it exactly but it takes a long time between one upload ending and the next one to begin. I literally save hours by uploading a full album at once instead of song by song.

Your Pal,

Jerry P.
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2008 Invision Power Services, Inc.