I have compressed my CD collection using Monkey's Audio, and now i want to backup the files on CD-Rs.

I will store two albums on each CD-R. Then i will use QuickPar to create par2 filesets for the audio files (one fileset for each album). The par2 filesets i will also store on CD-Rs (6-7 filesets on each CD-R).

This is the way i prefer to do it, but i am afraid it might cause problems. If a CD-R containing par2 filesets gets so many errors that windows can't read it, and that i have to create an ISO image of the disc, will i be able to use all of the filesets on the disc? Or just one of them, randomly chosen by QuickPar? What if i mount the ISO image in a virtual drive (using Daemon Tools, Alcohol 120% or some other software), would i then be able to copy the different par2 filesets (with errors) to my harddrive?

(and by the way, is it ok to store my Monkey's Audio files in two different folders on a CD-R, one for each album?)


Just one more thing i am wondering about, and that is how to create the necessary ISO image when a disc has gone bad. Yesterday i burned a CD-R, and scratched it a lot. I could still access and read it from the windows explorer, so i scratched it a bit more. Now windows couldn't read the disc, the drive appeared to be empty in windows explorer. Now i wanted to use IsoBuster to create an ISO image of the disc, but the drive appears to be empty in IsoBuster too! ("No media present"). How am i supposed to make the image of the damaged CD-R?

If you copy a bunch of corrupted PAR2s to your HD and open them, quickpar will use all recoverable blocks in those PAR2s. So yeah, even broken PAR2s are useful.

I hope I understood correctly what you want...
There's software capable of copying files containing corrupt sectors to HDD, e.g. IsoBuster. Some of these programs even have error correction algorithms similar to EAC for audio CDs (i.e. repeated re-reads until several re-reads match), e.g. CD Data Rescue or CDCheck.
Even if such a problematic file extracted from CD-R to HDD contains errors, PAR2 will notice them and just use the error-free blocks for recovery.

Argh! I know that it is possible to use damaged par2 files!

I'll explain again:

1. I store several par2 filesets on a single CD-R (6-7 filesets)

2. If the disc goes bad, i'll have to use IsoBuster (for example) to create an ISO image of the disc. When i open the ISO image in QuickPar, will i be able to choose which fileset to use, or will QuickPar simply use the first it finds?

Anyway, PeterBClements has confirmed that if i am forced to use an ISO image, QuickPar will use only one set of PAR2 files and ignore the rest. Then i won't be able to use all filesets on a disc if it goes bad. Because of that, i think i will have to reconsider whether it really is a good idea to store several par2 filesets on a single CD-R.

A question that still hasn't been answered: Will i be able to mount the ISO of the damaged CD-R in a virtual drive, and then copy the individual par2 filesets? ...If that is the case, it is safe to store several par2 filesets on the same CD-R after all.

Why do you have to create an iso image and mount it?

With the other programs I mentioned you can simply copy damaged files (i.e. files containing bad CD-R sectors) to HDD. Unrecoverable sectors are either replaced by "0"'s or by what the software assumes to be (closest to) correct. PAR2 will detect using internal checksums if the blocks in these files are usable or not and automatically decide if it uses them or not based on this.

I suspect there's still a misunderstanding

I have been explained that if the TOC of the CD-R containing my par2 filesets becomes unreadable, i will have to create an ISO image.

Ah. Now I understand. I have no experience with CD-Rs with damaged TOC. I see a chance that specialised recovery software as I mentioned above can extract something from such CDs without iso image. You would have to ask the manufacturer / read the documentation. About PAR2 and iso images: I can't answer your questions.

I'm currently working on a utility program to help out someone with a trashed HD. The've recovered most of the lost data by using CHKDSK, but of course the files are now all called FILE*.CHK. They had lots of PAR1 files so the utility that I am creating will analyse them all, work out which are PAR files and which are data files, and then rename and repair them all in one operation.

Thinking about this, I could create a utility to extract that PAR2 files from an ISO file. The utility would simply scan the entire image file looking for PAR2 packets. The PAR2 packets would be classified according to which set they are for, and all packets for the same set would be written out to a single PAR2 file. You would then be able to open each PAR2 file in turn using QuickPar and tell QuickPar to "Add" the ISO file to find the data.

Sounds very nice, something like that i am hoping for.

OK, I've written a PAR2 packet extractor.

I just need to test it on an ISO file for a disk with multiple PAR2 sets.

That was fast, excellent! Will it be available for downloading soon?

It needs some work, but if you want I'll email you a copy of the initial version for you to try out.

I'm not in a hurry, i have no damaged CDs to test it on yet

If you email me at Garathor@syndig.com i'll take a look at least.

Its on its way.

I've just added a "Save config" function that creates a small .INI with the settings, I added .FLAC to the default File mask... :-)
Now its a little more usefull :-)

Now the proggy is at:
http://www.inutils.tk


----------------------------------------------------------------

Hi all.
I coded this week par2 commandline frontend myself.
Its still in beta state :-) and lack of config file and a couple of details...

Resuming:
It creates a set of Par2 files for every folder recursively using as "block-size" multiplus of CDR and DVDR clusters (2048 and 2352)

I coded it to ease the task of create par2 files in my MP3 backups in CDRom...

Here you have a snapshot:


And you can download it here if you want to give it a try...
http://www.inutils.tk
Works fine for me :-)

Best regards to all.

b:.

PD.: Congratulations Peter, Quickpar/par2 are great tools.

Thankyou Peter. I created an ISO file containing several par2 filesets, and the program extracted each of them perfectly

I will take a look at your frontend bodhy, but QuickPar is working very well for me.

Great! That's what I like to hear.

The functionality it provides does not really fit into QuickPar as it stands, so I'll probably release it as a standalone utility.

At present the PAR2 files get given names that are simply the 16 byte set-id. I'll put some code in that attempts to figure out a better name based on the files that the set is for. It'll use the same algorithm that QuickPar uses when you create PAR2 files.

Dang. This is the first I'm reading about different cluster sizes for DVD. Everyone's been recommended a block size in multiples of 2048 for CDR/DVDR. Where can I find more about this 2352 cluster size for DVDR and how come no one until has recommended this multiple?

EDIT: bodhy, I just read the help file that came with your program and it helped explain (somewhat) the 2352 cluster size for DVDR (for burning in Mode 2 - whatever that is - guess I'll search that next).

Nice program you got here. Exactly what us newbies need for creating par2 files for archiving on optical media. Can you add *.FLAC though as one of the default extensions in the file mask? When I enter it and close it doesn't save. Thanks again for this cool little program

EDIT #2: I notice when using quickpar that when I lower the block size to the lowest possible multiple of 2048, the efficiency rating drops significantly. I also read that the efficiency rating should be as high as possible. I also read one should use the lowest possible multiple of 2048 for archiving on optical media. So which is it? Low multiple of 2048 and low efficiency, or a higher multiple and better efficiency? I'm using this mostly on lossless files in the 25-45MB size range if that makes a difference.

Mildly off-topic: what will you do if one CD is completely trashed, e.g. broken?

Probably cry in my soup....

Still, I'd rather lose one disc worth of data than an entire hard drive. My plan is to backup on DVDR with par2 and a separate backup on an external hard drive. Nothing is perfect, so I'm not going to worry about a disc cracking in half. Shouldn't happen as long as I take care of it.

Hi.



For CDR is sure there is a Burning Mode that uses "block sizes" of 2352, I think its called usually Mode2 (I think Mode2 / XA, or Mode 2 Form2 in other burning programs), for the DVDR I'm not sure of the existance of this "block size"...

I'll try to confirm.

http://www.disctronics.co.uk/technology/cd...D-ROM%20Sectors



I'll try to add a .INI file this evening (or at least add .FLAC to the file mask :-)



Using smaller block sizes par2 takes much more time to create the recovery files, but with smaller block sizes, error recovery becomes more flexible I think you'll can correct a higher number of errors... (I think)

The "optimal" value for par2 blocksize is to be the same than the CDR blocksize.

The "efficiency" value is related to time comsumption of par2 creation only?
Using smaller block sizes, you'll get a better protection against errors, but par2 will take much more time working.

Yesterday I left my office's computer at night with the frontend and It taked about 10 hours to process 6Gbs (block size 2048 - redundancy 10%) in a AMDXp 1800.

I can add in the blocksize combobox higher "blocksizes", for instance 4096, or 8192
so the process will speed up significally. Or something like a "speed-up multiplier" X2 (4096) X4 (8192) X8 (16384)...

If someone has any correction to make... please do it, It will be great appretiated :-)
Best regards to all.

b:.

[QUOTE=Daffy,Feb 18 2004, 12:26 AM][QUOTE=bodhy,Feb 17 2004, 05:12 PM]EDIT #2: I notice when using quickpar that when I lower the block size to the lowest possible multiple of 2048, the efficiency rating drops significantly.[/QUOTE]

The efficiency rating shows the average amount of repair that a given amount of recovery data can achieve.

You will get low efficiency ratings for both very large and very small block sizes.

For large block sizes the efficiency will drop if the source files have sizes that are not exact multiples of the block size. The last block in a file is not a full block, but it will still require a full block of recovery data to repair.

For very small block sizes a greater fraction of the PAR2 file consists of packet headers and verification data.

An efficiency ration of 50% means that if damage has effected 1MB of data, the PAR2 files you would need for repair would be about 2MB in size.

When you are trying to fill up free space on a CD/DVD, the higher you get the efficiency, the greater the total amount of damage you will be able to repair.

Of course, using too large a block size means that if the damage is small and highly scattered then repair may not be possible, but if damage is localised, repair will be possible.

Peter, thanks for that explanation. If I understand this correctly, then using the lowest multiple of 2048 does not necessarily result in the most efficient repair set. Instead, we should aim for highest block size (that's still a multiple of 2048) that results in the the highest efficiency rating. For example, on a set of FLAC files I get these settings:

Block Size <-> Efficiency
8192 <-> 36.3%
51200 <-> 85.0%
102400 <-> 93.0%
374784 <-> 98.0%
3080192 <-> 94.5%

In this example, my efficiency rating starts to peak at 374,784 block size, which is 2048 * 183. Would this be the ideal setting for this particular set of files? At 374,784 block size, only 65 recovery blocks are created, and the estimated time is only 1m9s. Something still doesn't seem right if that's the optimal setting.

Daffy, there is one additional thing you need to do when creating PAR2 files for CD/DVD, and that is to set the number of PAR2 files to 1.

I've just done a test with a set of 30 files with sizes varying from 9MB to 30MB (and a total size of 606MB) and get the following results (the first percentage is for variable sized PAR2 files and the second is the a single PAR2 file):

20,480 bytes = 52.5% (12 files) or 88.2% (1 file)
22,528 bytes = 61.2% (12 files) or 89.2% (1 file)
32,768 bytes = 71.7% (11 files) or 92.7% (1 file)
51,200 bytes = 80.6% (11 files) or 95.0% (1 file)
96,256 bytes = 90.0% (10 files) or 97.2% (1 file)
188,416 bytes = 95.0% (9 files) or 98.3% (1 file)
253,952 bytes = 96.3% (8 files) or 98.6% (1 file)
514,048 bytes = 97.5% (7 files) or 98.4% (1 file)
1,024,000 bytes = 96.9% (7 files) or 97.3% (1 file)

Peter - thanks for that test. I've seen similar results.

So you agree that the sweet spot should be:

(1) the highest multiple of 2048 that
(2) gives the highest efficiency rating for
(3) one file set

In your example, that would be a block size in the range of 188,416 - 514,048. Does that look right to you? The process time would be really quick and the number of recovery blocks would be low. Would that still yield the optimal results?

There are several conflicting things you are trying to achieve when selecting a block size:

1) Getting as many recovery blocks as possible.

2) Getting as high an efficiency as possible.

3) Mimimizing the time to create.

I've just done another test with a similar set of files (but this time I've adjusted the redundancy to get as near as possible to 100MB of PAR2 files), and gathered a little more information:

CODE

block   redundancy  recovery  efficiency  compute
size                blocks                time

20,480  15.04%     4,675     91.2%       4h 11m
40,960  15.79%     2,455     95.8%       1h 57m
81,920  16.13%     1,255     97.8%          56m
122,880  16.21%       842     98.3%          37m
163,840  16.26%       634     98.6%          27m
204,800  16.28%       508     98.8%          22m
262,144  16.27%       397     98.8%          17m
393,216  16.23%       265     98.7%          11m
524,288  16.20%       199     98.4%           8m
786,432  16.18%       133     98.0%           6m

As you can see, the increased efficiency lets you have a slightly higher redundancy setting, but recovery block count does drop off very rapidly when you increase the block size to the value necessary for the peak efficiency.

I would suggest that as long as you are within a few % of the peak efficiency, you are probably OK, and it may be more important to have a reasonably high recovery block count.

At some point in the future I will be working on features specific to VD/DVD usage and when I do that I will set things up so that instead of specifying the redundancy setting to use, you will specify how much free space you want to fill up.

Thanks again Peter for your hard work on this and attention to detail. It appears then that a compromise of settings that gives relatively good efficiency and high recovery block count is the way to go. I think I now have a much better understanding of the relationships and how I should be preparing my file sets for archive on DVDR. Thanks.

Daffy

Garathor:

I just have to say I'm very confused by what you're trying to do here. Why do you prefer to do it that way? Is there some reason you actually want to go to so much extra work?

You're storing your APE files on CD-R's. You're worried about the CD-R's becoming damaged, in which case you would have to create an ISO to attempt to repair it. You are going through all your files album by album to make par2 recovery files.

So, why don't you simply make a par2 file for each ISO to begin with? I was under the impression that's how everybody does it. You seem to have figured out the major drawbacks to doing file-by-file par2 creation when you're trying to protect removable media, yet you say that's the way you prefer to do it?

Just put your albums on CD-R's and then make a par2 recovery file for each ISO. If the disc becomes damaged, you extract, repair, and re-burn. This will potentially save you countless headaches.

So what settings would you recommend from backing up to DVD? Too many different tables without any definite guidelines for me to following