Generating Accuraterip checksums, Looking for a commandline Linux tool or sourcecode/specification Options

[Xenion]

Hello,
Long story short: I have Image+CUE+Log EAC-ripped audio disks i'd like to split to individual tracks. Since the EAC-log contains the Accuraterip checksums i'd like to compare the splitted files against the accuraterip checksum that is printed in the EAC-log in order to ensure that the splitting progress (i use shn-tool under linux) is error-free.

Now what i need is a tool that is able to generate accuraterip checksums. Since everything is taking place under linux with commandline, a tool available through the linux package management would be first choice.

If nothing is available could also write a tool on my own but i cannot find an official specification or sourcecode of the accuraterip checksum algorithm

Any ideas?

[spoon]

Example code in c++:

http://forum.dbpoweramp.com/showthread.php...CRC-Calculation

[leo-bogert]

Example code in c++:

http://forum.dbpoweramp.com/showthread.php...CRC-Calculation

Thanks for your reply. I'm a friend of the original poster and the developer who requested him to ask for this.
So I've implemented your code in a C99 commandline program which allows computation of the checksums for WAV singletrack files.
I've implemented it in a way which allowed me to copy-paste most of your code and only fill in the blanks.

- I wanted to copy-paste most of it instead of re-implementing it to make sure that I don't break anything.
Can you please change your thread to apply the GPLv3 license to both the V1 and V2 code so I can publish the sourcecode of my tool, which will be GPLv3 itself?

This post has been edited by leo-bogert: Oct 27 2012, 19:50

[spoon]

We do not claim copyright on those few lines of code, you are free to do as you want with it.

[leo-bogert]

We do not claim copyright on those few lines of code, you are free to do as you want with it.

Nice, thank you!

Here you go: https://github.com/leo-bogert/accuraterip-checksum
I would be glad if you posted the link in the thread with your sample code.

I have tested the code with this album and all V2 checksums match.
It also contains code for generating the V1 checksums, however I have not tested it yet as I have no EAC LOG with V1 checksums at hand.
Therefore, there is no user interface for generating the V1 checksums yet.

If you can give me the V1 checksums for the said album, I will implement a user interface for generating V1 checksums as well.

[Dario]

If you can give me the V1 checksums for the said album, I will implement a user interface for generating V1 checksums as well.

If we're talking about the Island Records (828 513-2) re-issue, then here are the checksums:

CODE

Track 1: Accurately ripped (confidence 41) [575D11C1]
Track 2: Accurately ripped (confidence 42) [F71CCA86]
Track 3: Accurately ripped (confidence 44) [BD5F8095]
Track 4: Accurately ripped (confidence 42) [7B29D6E2]
Track 5: Accurately ripped (confidence 42) [0FD65A2C]
Track 6: Accurately ripped (confidence 42) [FC9FAEE9]
Track 7: Accurately ripped (confidence 42) [D882CA85]
Track 8: Accurately ripped (confidence 41) [2D4068B0]
Track 9: Accurately ripped (confidence 43) [96F3E348]
Track 10: Accurately ripped (confidence 42) [5B7EF1C9]
Track 11: Accurately ripped (confidence 41) [512C6514]
Track 12: Accurately ripped (confidence 40) [06DA6436]
Track 13: Accurately ripped (confidence 41) [512C6514]
Track 14: Accurately ripped (confidence 40) [9D58954A]
Track 15: Accurately ripped (confidence 41) [4165F749]
Track 16: Accurately ripped (confidence 39) [358F2D1B]

Do you have any plans to turn that thingy into a fully-fledged AR verification tool?

[korth]

Actually Dario, I think these are at the correct offset. Yours are offset by -763.

CODE

[AccurateRip ID: 001f5c76-01776ea7-c50cac10] found.
Track [ V1 | V2 ] Status
01 [06147ac6|12a17108] (034+006/222) Accurately ripped
02 [0567e4bf|b8b12553] (035+006/223) Accurately ripped
03 [5a73b8c8|ee647f8c] (035+006/223) Accurately ripped
04 [750a306a|198d1b89] (036+006/224) Accurately ripped
05 [2dd8dd28|443f683f] (035+006/221) Accurately ripped
06 [304de972|d4ea2156] (035+006/221) Accurately ripped
07 [345a3cc2|23a1013a] (035+006/222) Accurately ripped
08 [ce0330f2|59846adb] (035+006/214) Accurately ripped
09 [728d9f1a|562695d4] (035+006/221) Accurately ripped
10 [8b44748a|f8f465ac] (036+006/219) Accurately ripped
11 [7fc8a851|779aa114] (036+006/221) Accurately ripped
12 [ab1a3355|826fdc3d] (035+006/218) Accurately ripped
13 [17af540f|08e26e72] (035+006/218) Accurately ripped
14 [76b1e2a9|bb90680d] (035+006/214) Accurately ripped
15 [31fe666d|37aac77a] (035+006/216) Accurately ripped
16 [eba39f2a|1a3cd493] (035+006/216) Accurately ripped

This post has been edited by korth: Oct 28 2012, 03:36

[leo-bogert]

Actually Dario, I think these are at the correct offset. Yours are offset by -763.

CODE

[AccurateRip ID: 001f5c76-01776ea7-c50cac10] found.
Track [ V1 | V2 ] Status
01 [06147ac6|12a17108] (034+006/222) Accurately ripped
02 [0567e4bf|b8b12553] (035+006/223) Accurately ripped
03 [5a73b8c8|ee647f8c] (035+006/223) Accurately ripped
04 [750a306a|198d1b89] (036+006/224) Accurately ripped
05 [2dd8dd28|443f683f] (035+006/221) Accurately ripped
06 [304de972|d4ea2156] (035+006/221) Accurately ripped
07 [345a3cc2|23a1013a] (035+006/222) Accurately ripped
08 [ce0330f2|59846adb] (035+006/214) Accurately ripped
09 [728d9f1a|562695d4] (035+006/221) Accurately ripped
10 [8b44748a|f8f465ac] (036+006/219) Accurately ripped
11 [7fc8a851|779aa114] (036+006/221) Accurately ripped
12 [ab1a3355|826fdc3d] (035+006/218) Accurately ripped
13 [17af540f|08e26e72] (035+006/218) Accurately ripped
14 [76b1e2a9|bb90680d] (035+006/214) Accurately ripped
15 [31fe666d|37aac77a] (035+006/216) Accurately ripped
16 [eba39f2a|1a3cd493] (035+006/216) Accurately ripped

Thank you. I've added code to validate version 1 checksums and your checksums matched:

CODE

AccurateRip checksum of track 01: 06147AC6, expected 06147AC6. OK.
AccurateRip checksum of track 02: 0567E4BF, expected 0567E4BF. OK.
AccurateRip checksum of track 03: 5A73B8C8, expected 5A73B8C8. OK.
AccurateRip checksum of track 04: 750A306A, expected 750A306A. OK.
AccurateRip checksum of track 05: 2DD8DD28, expected 2DD8DD28. OK.
AccurateRip checksum of track 06: 304DE972, expected 304DE972. OK.
AccurateRip checksum of track 07: 345A3CC2, expected 345A3CC2. OK.
AccurateRip checksum of track 08: CE0330F2, expected CE0330F2. OK.
AccurateRip checksum of track 09: 728D9F1A, expected 728D9F1A. OK.
AccurateRip checksum of track 10: 8B44748A, expected 8B44748A. OK.
AccurateRip checksum of track 11: 7FC8A851, expected 7FC8A851. OK.
AccurateRip checksum of track 12: AB1A3355, expected AB1A3355. OK.
AccurateRip checksum of track 13: 17AF540F, expected 17AF540F. OK.
AccurateRip checksum of track 14: 76B1E2A9, expected 76B1E2A9. OK.
AccurateRip checksum of track 15: 31FE666D, expected 31FE666D. OK.
AccurateRip checksum of track 16: EBA39F2A, expected EBA39F2A. OK.

I've updated the code in the repository, you can now request V1 checksums with "--version1".
Notice that I had to re-create the repository because I made a mistake when creating it the first time. The URL is still okay but git pull probably won't work, please do a fresh checkout of the repository.

[leo-bogert]

A binary compiled for Ubuntu12.04 amd64 has been added to the downloads of the git project.

[leo-bogert]

Do you have any plans to turn that thingy into a fully-fledged AR verification tool?

I have already written code which reads the checksums from an EAC LOG and validates WAV singletracks according to them.
However, the code is no standalone tool but part of my perfect-flac-encode script.
I could theoretically write a standalone tool which does this, but right now I am not very motivated to do so because I have no purpose for it. A small donation might change this of course

[leo-bogert]

I have a problem with the following CUE:

CODE

REM GENRE Sixties
REM DATE 1991
REM DISCID BB0AD80D
REM COMMENT "ExactAudioCopy v1.0b3"
CATALOG 0000000000000
PERFORMER "Spencer Davis"
TITLE "Keep On Running"
FILE "Spencer Davis - Keep On Running.wav" WAVE
  TRACK 01 AUDIO
    TITLE "Keep On Running"
    PERFORMER "Spencer Davis"
    INDEX 00 00:00:00
    INDEX 01 00:00:33
  TRACK 02 AUDIO
    TITLE "Somebody Help Me"
    PERFORMER "Spencer Davis"
    INDEX 00 03:06:05
    INDEX 01 03:07:68
  TRACK 03 AUDIO
    TITLE "I'm A Man"
    PERFORMER "Spencer Davis"
    INDEX 00 05:47:58
    INDEX 01 05:50:35
  TRACK 04 AUDIO
    TITLE "Give Me Some Lovin'"
    PERFORMER "Spencer Davis"
    INDEX 00 09:24:70
    INDEX 01 09:26:23
  TRACK 05 AUDIO
    TITLE "Blood Runs Hot"
    PERFORMER "Spencer Davis"
    INDEX 00 12:18:70
    INDEX 01 12:20:50
  TRACK 06 AUDIO
    TITLE "Don't Want You No More"
    PERFORMER "Spencer Davis"
    INDEX 00 16:18:55
    INDEX 01 16:20:38
  TRACK 07 AUDIO
    TITLE "Love Is On A Roll"
    PERFORMER "Spencer Davis"
    INDEX 00 20:24:18
    INDEX 01 20:25:55
  TRACK 08 AUDIO
    TITLE "Crossfire"
    PERFORMER "Spencer Davis"
    INDEX 00 24:13:20
    INDEX 01 24:14:35
  TRACK 09 AUDIO
    TITLE "Private Number"
    PERFORMER "Spencer Davis"
    INDEX 00 28:16:60
    INDEX 01 28:19:33
  TRACK 10 AUDIO
    TITLE "No Other Baby"
    PERFORMER "Spencer Davis"
    INDEX 00 32:35:15
    INDEX 01 32:37:10
  TRACK 11 AUDIO
    TITLE "Mistakes"
    PERFORMER "Spencer Davis"
    INDEX 00 35:58:43
    INDEX 01 35:59:48
  TRACK 12 AUDIO
    TITLE "It Must Be Love"
    PERFORMER "Spencer Davis"
    INDEX 00 39:17:70
    INDEX 01 39:19:23
  TRACK 13 AUDIO
    TITLE "Such A Good Woman"
    PERFORMER "Spencer Davis"
    INDEX 01 43:06:20

I split the WAV image to WAV singletracks with

CODE

shntool split -P dot -d "$outputdir_relative" -f "Spencer Davis - Keep On Running.cue" -n %02d -t "%n - %t" -- "Spencer Davis - Keep On Running.wav"; then

which resulted in a track 0 being generated by shntool as hidden track one audio.

I wrote a bash script which uses my accuraterip-checksum tool to validate the checksums of the split files. It reads the expected checksums from the EAC LOG.
I get the following result:

CODE

Skipping tracknumber 0 as this is a hidden track, EAC won't list AccurateRip checksums of hidden track one audio
AccurateRip checksum mismatch for track 01: expected='7A8E95CB'; actual='84ED3380'
AccurateRip checksum of track 02: C91686C5, expected C91686C5. OK.
AccurateRip checksum of track 03: 9CFD1941, expected 9CFD1941. OK.
AccurateRip checksum of track 04: 18A20AEC, expected 18A20AEC. OK.
AccurateRip checksum of track 05: 6A6CA0E6, expected 6A6CA0E6. OK.
AccurateRip checksum of track 06: 5E1759B1, expected 5E1759B1. OK.
AccurateRip checksum of track 07: 32DC9E69, expected 32DC9E69. OK.
AccurateRip checksum of track 08: 404B6006, expected 404B6006. OK.
AccurateRip checksum of track 09: 30CC9C04, expected 30CC9C04. OK.
AccurateRip checksum of track 10: D84F3ADA, expected D84F3ADA. OK.
AccurateRip checksum of track 11: 22814312, expected 22814312. OK.
AccurateRip checksum of track 12: BBD7AC32, expected BBD7AC32. OK.
AccurateRip checksum mismatch for track 13: expected='BA3B1A5A'; actual='7F147ED5'

The EAC LOG reports:

CODE

AccurateRip summary

Track  1  accurately ripped (confidence 5)  [7A8E95CB]  (AR v1)
Track  2  accurately ripped (confidence 5)  [C91686C5]  (AR v1)
Track  3  accurately ripped (confidence 5)  [9CFD1941]  (AR v1)
Track  4  accurately ripped (confidence 5)  [18A20AEC]  (AR v1)
Track  5  accurately ripped (confidence 5)  [6A6CA0E6]  (AR v1)
Track  6  accurately ripped (confidence 5)  [5E1759B1]  (AR v1)
Track  7  accurately ripped (confidence 4)  [32DC9E69]  (AR v1)
Track  8  accurately ripped (confidence 5)  [404B6006]  (AR v1)
Track  9  accurately ripped (confidence 5)  [30CC9C04]  (AR v1)
Track 10  accurately ripped (confidence 5)  [D84F3ADA]  (AR v1)
Track 11  accurately ripped (confidence 5)  [22814312]  (AR v1)
Track 12  accurately ripped (confidence 5)  [BBD7AC32]  (AR v1)
Track 13  accurately ripped (confidence 5)  [BA3B1A5A]  (AR v1)

All tracks accurately ripped

This post has been edited by leo-bogert: Nov 6 2012, 15:10

[greynol]

It would be more correct to say that AR doesn't store checksums for HTOA. Don't blame EAC.

Did you remember to skip the first five frames of the first track when calculating its checksum?

This post has been edited by greynol: Nov 6 2012, 16:01

[leo-bogert]

It would be more correct to say that AR doesn't store checksums for HTOA. Don't blame EAC.

Since both are not open source we have no idea what to blame

Did you remember to skip the first five frames of the first track when calculating its checksum?

See above: The source code was copy-pasted from the example code so I don't even have to remember anything.
And yes, the example code contains frame skipping. See https://github.com/leo-bogert/accuraterip-c...erip-checksum.c
The code DOES work for non-HTOA albums with all tracks.

Thanks for your remark though.

This post has been edited by leo-bogert: Nov 6 2012, 21:26

[leo-bogert]

Joining track 0 and 1 does not fix the issue.
The checksum of track 0 is also not the one we are looking for (which is 7A8E95CB)

CODE

$ shntool join 00\ -\ pregap.wav 01\ -\ Keep\ On\ Running.wav
Joining [00 - pregap.wav] (0:00.33) --> [joined.wav] (3:07.68) : 100% OK
Joining [01 - Keep On Running.wav] (3:07.35) --> [joined.wav] (3:07.68) : 100% OK
No padding needed.
$ ~/accuraterip-checksum --version1 joined.wav 1 13
C342F5FE
$ ~/accuraterip-checksum --version1 00\ -\ pregap.wav 1 13
A7051BF2
$ ~/accuraterip-checksum --version1 01\ -\ Keep\ On\ Running.wav 1 13
84ED3380

[pablogm123]

A suggestion regarding the rip you have shown: You shouldn't enable overread if you drive cannot overread into lead-out. I have tested many LG drives based on Renesas chipset (+667), and none can read into lead-out. If you enable overread and drive doesn't support it, EAC may not read properly the last sectors of CD, yet Accuraterip won't report anything because ignores the last five sectors of image.

Regarding the HT0A tracks, EAC will report its CRC in the log only if you rip that "track" as range selecting the proper range.

[greynol]

Since both are not open source we have no idea what to blame

You mean to say you don't know who to blame. Now you do, though this really isn't about blame, it is about history which is very well documented on this forum. Perhaps you can take a bit more initiative and search.

Joining track 0 and 1 does not fix the issue.

Of course it doesn't.

The checksum of track 0 is also not the one we are looking for (which is 7A8E95CB)

This is also hardly surprising.

The code DOES work for non-HTOA albums with all tracks.

That's great. Did you make sure that you're generating the correct disc ID? While HTOA is not included in the AR database it does affect the disc ID. Again, the code for that is available here on the forum.

This post has been edited by greynol: Nov 6 2012, 23:58

[korth]

AccurateRipID should be 013-0015cba5-00dcc9ab-bb0ad80d based on the CUE file (and assuming the last track length at 3:10:48 or 14298 sectors).

[leo-bogert]

A suggestion regarding the rip you have shown: You shouldn't enable overread if you drive cannot overread into lead-out. I have tested many LG drives based on Renesas chipset (+667), and none can read into lead-out. If you enable overread and drive doesn't support it, EAC may not read properly the last sectors of CD, yet Accuraterip won't report anything because ignores the last five sectors of image.

Thanks. Do you have any indication that this affects my drive?
EAC says: "Used drive : HL-DT-STDVDRAM GSA-U10N Adapter: 1 ID: 0"

Regarding the HT0A tracks, EAC will report its CRC in the log only if you rip that "track" as range selecting the proper range.

AccurateRip CRC or EAC CRC?

[leo-bogert]

Since both are not open source we have no idea what to blame

You mean to say you don't know who to blame. Now you do, though this really isn't about blame, it is about history which is very well documented on this forum. Perhaps you can take a bit more initiative and search.

Yes, I agree, this is not about blaming, it's about software development. We created this thread for research on how to compute AccurateRip checksums. Of course there are other threads about it but we didn't know whether they are still up to date so we created a fresh one.

Joining track 0 and 1 does not fix the issue.

Of course it doesn't.

The checksum of track 0 is also not the one we are looking for (which is 7A8E95CB)

This is also hardly surprising.

I don't those are very "of course". If there is an issue with a tracks' checksum it might have been because too much was left out/taken in.
But please read further, we have misunderstood each others because I didn't make it clear enough what I am trying to do:

The code DOES work for non-HTOA albums with all tracks.
That's great. Did you make sure that you're generating the correct disc ID? While HTOA is not included in the AR database it does affect the disc ID. Again, the code for that is available here on the forum.

I am NOT trying to obtain AccurateRip checksums from the server!
I wrote a tool which takes a correctly ripped WAV image as reported by the EAC LOG and splits it to singletrack WAVs using shntool according to the CUE which EAC produced when ripping the image.
Then, for each track, it passes the tracknumber and the total track count to my accuraterip-checksum tool which is advertised in this thread.
The accuraterip-checksum tool computes the AR checksum from the singletrack-WAVs, the tracknumbers and total track count.
Then it compares the sums to the ones in the EAC LOG which were reported as correct for the input WAV image. This comparison fails for track 1 and 13 for the said disc. No disc ID or AccurateRip server queries involved!
Notice: The code has been tested with different ARv1 and ARv2 discs for which all tracks are validated as correct.

This post has been edited by leo-bogert: Nov 7 2012, 11:23

[pablogm123]

EAC features a tool to detect overreading capabilities: The detect read sample offset correction button, located at Offset / Speed tab. If this test reports that the drive doesn't overread at all, or just overread into lead-in (pregap of first track, actually) with a drive with positive offset correction, don't enable overread for that drive.


EAC CRC, or CRC32 of raw headerless audio stream.

[greynol]

That's correct. Positive read offset corrections require overreading into the lead-out.

Regarding drives that may attempt to overread into the lead-in (those configured with a negative read offset correction), do be careful not to confuse this with a drive's ability to extract HTOA, as there are drives that are cabable of doing one and not the other.

Anyway, configuring a drive to overhead when it cannot can cause problems with data that would otherwise be read correctly when using EAC. This could very easily affect the data past the five frames that AR ignores.

Again, this is not new territory on the forum.

I don't have an issue with a redundant topic on the subject, but please don't use the idea that information is stale from conducting research. Once this issue has been identified it will be because of a problem with the implementation and not because something changed between then and now.

This post has been edited by greynol: Nov 7 2012, 18:10

[leo-bogert]

I figured out what was wrong with my tool accuraterip-checksum and the said Spencer Davis release and unfortunately it was induced by the imprecise specification of the AccurateRip V2 algorithm:

The original snippet says:

CODE

    DWORD AC_CRCNEW = 0;
    DWORD MulBy = 1;
    for (;; )
    {
        DWORD Value = {complete 32 bit sample comprising left and right channels};

        unsigned __int64 CalcCRCNEW = (unsigned __int64)Value * (unsigned __int64)MulBy;
        DWORD LOCalcCRCNEW = (DWORD)(CalcCRCNEW & (unsigned __int64)0xFFFFFFFF);
        DWORD HICalcCRCNEW = (DWORD)(CalcCRCNEW / (unsigned __int64)0x100000000);
        AC_CRCNEW+=HICalcCRCNEW;
        AC_CRCNEW+=LOCalcCRCNEW;

        MulBy++;
    }

Obviously, you have to guess the boundaries of the for-loop. What I guessed is:

CODE

    int DataDWORDSize = DataSize / sizeof(DWORD);

    DWORD AC_CRCNEW = 0;
    DWORD MulBy = 1;
    for (int i = 0; i < DataDWORDSize; i++)
    {
        DWORD Value = pAudioData[i];

        unsigned __int64 CalcCRCNEW = (unsigned __int64)Value * (unsigned __int64)MulBy;
        DWORD LOCalcCRCNEW = (DWORD)(CalcCRCNEW & (unsigned __int64)0xFFFFFFFF);
        DWORD HICalcCRCNEW = (DWORD)(CalcCRCNEW / (unsigned __int64)0x100000000);
        AC_CRCNEW+=HICalcCRCNEW;
        AC_CRCNEW+=LOCalcCRCNEW;

        MulBy++;
    }

But what is actually right is to add the parts of the V1-code which ignore the first and last sectors:

CODE

    //---------AccurateRip CRC checks------------
    DWORD AR_CRCPosCheckFrom = 0;
    DWORD AR_CRCPosCheckTo = DataSize / sizeof(DWORD);
#define SectorBytes 2352        // each sector
    if (TrackNumber == 1)            // first?
        AR_CRCPosCheckFrom+= ((SectorBytes * 5) / sizeof(DWORD));
    if (TrackNumber == AudioTrackCount)        // last?
        AR_CRCPosCheckTo-=((SectorBytes * 5) / sizeof(DWORD));

    int DataDWORDSize = DataSize / sizeof(DWORD);

    DWORD AC_CRCNEW = 0;
    DWORD MulBy = 1;
    for (int i = 0; i < DataDWORDSize; i++)
    {
        if (MulBy >= AR_CRCPosCheckFrom && MulBy <= AR_CRCPosCheckTo)
        {
        DWORD Value = pAudioData[i];

        unsigned __int64 CalcCRCNEW = (unsigned __int64)Value * (unsigned __int64)MulBy;
        DWORD LOCalcCRCNEW = (DWORD)(CalcCRCNEW & (unsigned __int64)0xFFFFFFFF);
        DWORD HICalcCRCNEW = (DWORD)(CalcCRCNEW / (unsigned __int64)0x100000000);
        AC_CRCNEW+=HICalcCRCNEW;
        AC_CRCNEW+=LOCalcCRCNEW;
        }
        MulBy++;
    }

This is fixed in version 1.4. I must admit that it is my fault that I had my brain disabled while writing this, it SHOULD have been possible to notice. Nevertheless, I would urge the original developer to please update the reference code snippets because algorithm specifications should never involve making people have to guess parts of the code.

In addition, there was a separate bug in my code which was purely my fault and which was fixed in 1.3. I waited until the end of the post to admit this so the issue with the specification would get noticed

[spoon]

I thought I would be obvious to anyone who has implemented v1 CRCs that v2 are calculated over the same data range, no where in the loop shown for (;;) actually shows which data points are used, it is there just to represent a loop, it is partial code.

I have added a disclaimer to this effect...