EAC secure mode test proposal, Testing its real accuracy Options

[Pio2001]

Wow ! This gives (1-20/10000)*100 = 99.8 % accuracy. This is below the C2 accuracy of some drives ...and above the C2 accuracy of others...

Thank you very much for your work, Tigre. Very useful.

[Canar]

Wow ! This gives (1-20/10000)*100 = 99.8 % accuracy. This is below the C2 accuracy of some drives ...and above the C2 accuracy of others...

Thank you very much for your work, Tigre. Very useful.

So this means everything we know is wrong and on hard CDs, EAC doesn't give perfect rips?

Or am I reading this wrong?

[tigre]

Wow ! This gives (1-20/10000)*100 = 99.8 % accuracy. This is below the C2 accuracy of some drives ...and above the C2 accuracy of others...

Thank you very much for your work, Tigre. Very useful.

So this means everything we know is wrong and on hard CDs, EAC doesn't give perfect rips?

Or am I reading this wrong?

I'd rather say: We knew before that EAC doesn't give 100% security (-> "different Test & Copy CRCs on error-free reported Secure Mode rip (Cache disabled, C2 off)" reports). This is why some ppl use Test & Copy + Secure mode.

IMO it's dangerous do generalize ("EAC doesn't give perfect rips."), because ...


I only tested 1 (pressed) CD with ~ 20 minutes of music ... - and "artificial" scratches (I drew a line with a marker and noticed that it caused too much trouble, so I removed it with "brute force". If I try to read the CD in secure mode, it fails (= unrecoverable errors).

My conclusions:
- On this CD EAC secure mode would have reported (many) unrecoverable errors anyway, so a few unnoticed error positions (all of them near to the unrecoverable ones) won't do any harm.

- If the pattern of blocks re-read on an error found would be changed (but same ammount of re-reads) more errors (in my test all) would be covered by secure mode's multiple re-reads.

- The test can show how to do more similar tests, maybe with
-- CDs damaged due to RL usage,
-- CDs that are less damaged, so no unrecoverable errors are shown in secure mode's report and/or
-- dying CD-Rs,
- but the result shouldn't be taken too serious; I'd say it plays in the same league as "different Test & Copy CRCs on error-free reported Secure Mode rip (Cache disabled, C2 off)" reports ...

So I invite everyone interested to repeat this test.

[tigre]

I tested the same scratched CD with another drive: LG DVD-ROM DRD 8120B

Results:

Total time: 20:32

Total Errors: 1369
Undetected without extended re-read[/b]: 86
P = 93.7 %

Undetected with extended re-read: 36 at 1 position.

This time the undetected faulty position was surrounded by > 200 correct blocks before and after, so no chance to be detected correctly.

BTW: The difference is inaudible at this position, its peak amplitude is -59dB.

[DrDoogie]

I'd rather say: We knew before that EAC doesn't give 100% security (-> "different Test & Copy CRCs on error-free reported Secure Mode rip (Cache disabled, C2 off)" reports). This is why some ppl use Test & Copy + Secure mode.

IMO it's dangerous do generalize ("EAC doesn't give perfect rips."), because ...

...because you have never heard the story of the white swans?

Claim: There are only white swans in the world.
Proof: Find all swans, prove that you have found all swans, and show that they are all white and have always been white.
(In other words, exterminate the swans).
Counter-proof: Find one swan that is not white.

...Can you see the "not-white" swan?

[tigre]

You're right about swans ...

I just havn't hacked EAC so I can see every bit it gets from the drive and what it does with it - I just tried to simulate/approximate what it does, so be careful ...

Example: Some minutes EAC + my LG drive finished extracting the scratched CD in secure mode ... no errors reported - and the rip was bit identical to the reference. At this very moment I let my LG drive extract the position not detected correctly in my "simulation" over and over again in secure mode - until now (~15 rips) either perfect result or unrecoverable errors reported ...

A guess: The position I'm talking about lies ~ 250 blocks after a position with many errors where error correction kicks in every time. So probably when reading on after finishing error correction, the drive's speed is still reduced when getting to the position undetected in the test, so the error will be corrected immediately by the drive - or noticed.

EDIT:
BTW: Noone ever claimed EAC gives perfect security. My test is to get an idea of how many black swans you have to expect - and I just say the results don't tell much about this yet.

This post has been edited by tigre: May 28 2003, 16:33

[Pio2001]

Saying EAC is not perfect, and not using it because of this is like saying "I prefer calculate by hand, with a paper and a pencil, because there is a slight chance that an electronic pocket calculator returns a false result in 1000 years. Therefore a calculator won't always give me an exact result".

[Pio2001]

Tigre, it seems what you call extended reread is assuming that all errors in the rereading range are detected.
One should rip 16 different versions of the CD, and keep the samples that are 8 or 9 times the same as good, then see if they actually are.
In this case, your 99.8 and 97.4 % results are just upper bounds for the actual accuracy, because among the 1930 and 86 other errors, many will remain undetected after rereading, since they are consistent. The 80 % accuracy is then not unlikely.

These consistent errors would be just samples damaged beyond recovery. Each time they are read, an error occurs, and they are always interpolated, thus returning a consistent result.

[tigre]

Tigre, it seems what you call extended reread is assuming that all errors in the rereading range are detected.

You're right. I just tried to find the errors that will remain undetected for sure ...

One should rip 16 different versions of the CD, and keep the samples that are 8 or 9 times the same as good, then see if they actually are.

... because I'm too lazy to rip 16 more times - and do 15+ more comparisons. You're right: One should ... - but this "one" won't be me.

In this case, your 99.8 and 97.4 % results are just upper bounds for the actual accuracy, because among the 1930 and 86 other errors, many will remain undetected after rereading, since they are consistent. The 80 % accuracy is then not unlikely.

Hard to tell without testing. I'd rather think that most of the errors are detected during 1st 16 re-reads (because of reduced speed and the finding I mentioned in the post before: in secure mode (C2 off, Drive caches checked) the same drive + same CD either give perfect results or report errors at bad positions - 10 trials, no undetected error).

These consistent errors would be just samples damaged beyond recovery. Each time they are read, an error occurs, and they are always interpolated, thus returning a consistent result.

The way I understand how audio CDs work, there's quite some damage necessary to get an unrecoverable error (C1 + C2 failing). So as several damaged = "unsecure" bits read in raw data are needed to cause an error (that makes the drive interpolate), it's rather unlikely that all of these bits are returned identically on several re-reads resulting in identical interpolation.

Whatever ... My conlusions so far:
- More tests needed
- Real life secure mode performs much better on error detection than my "simulation" for so far unknown reasons.
- It seems like Secure mode (C2 off, correct cache settings) is more secure in error detecting than Burst mode Test & Copy with CRC check.

This post has been edited by tigre: Jun 5 2003, 00:14

[Pio2001]

I think I'll perform some little tests myself.

About the 16 files, it would be easier to write a program that loads the 16 wavs and performs an automated detection.

[tigre]

About the 16 files, it would be easier to write a program that loads the 16 wavs and performs an automated detection.

This should work with SOX + a batch file. For my test I prefered to use CEP because there weren't so many operations that could be automated - and I wanted visual control to be sure that e.g. after multiplying a wave with itself all values are positive. (If dither is enabled in CEP accidentially you'll get a few negative values too.)

I think I'll perform some little tests myself.

Good luck. Feel free to ask e.g. if you need details about something I did.

[DrDoogie]

Saying EAC is not perfect, and not using it because of this is like saying ...

Oh no, that's not what I meant at all.
I'm just sayin' it's healthy to keep a realistic perspective on the limitations inherent in the reading process.

EAC isn't perfect => Don't trust EAC alone

One should keep in mind that the reason Andre has given for writing EAC, was the lack of accuracy of "some" drives in reporting C2 errors.
Meaning they went by unnoticed, in which case re-reading was (and still is) a good idea.

However, now that the C2 error-reading capability of "some" drives are rather good, the next aspect we can grasp, is the inability of any possible drive in correcting an uncorrectable error.

I certainly do not in any way, shape or form "blame" this on EAC - that would have been a logical fallacy along the line of "killing the messenger bringing bad news".

[Pio2001]

I agree with you. But the reason for which EAC was written was not the lack of C2 acuracy, but the fact that all rippers just ignored the errors reported.

[kdo]

- It seems like Secure mode (C2 off, correct cache settings) is more secure in error detecting than Burst mode Test & Copy with CRC check.

I'd rather re-phrase that conclusion, to make it more specific:
it is more secure in detecting individual errors.

Because the plain "test&copy CRC OK" will leave undetected errors only if ALL those errors are consistent ("repeatable", "bogus-corrected") AND there are NO inconsistent errors. The probability of this event can be "measured" (estimated).
My understanding is that so far it was reported to be very low.
Exception is the recent report by DrDoogie, but it has to be verified whether the result can really be trusted.

In this Tigre's test there were always inconsistent errors, so there would always be CRC mis-match - indication of the fact "there are some errors" (but their number and location is unknown).

(Well, except those rare cases when CRC-algorithm itslef would fail, but then a bit-to-bit comparison would reveal the difference.)

Hope my understanding is right.

[Pio2001]

(Well, except those rare cases when CRC-algorithm itslef would fail, but then a bit-to-bit comparison would reveal the difference.)

What do you mean ?

[kdo]

(Well, except those rare cases when CRC-algorithm itslef would fail, but then a bit-to-bit comparison would reveal the difference.)

What do you mean ?

Nothing ground-breaking. Negligible theoretical chanse that two different files have the same CRC.
consistent error is a lot more serious trouble to worry about, no doubt.

[Pio2001]

No doubt, no doubt !