ARCue.pl - Check whole CD rips with AccurateRip Options

[Christopher Key]

Hello All,

I've throw together a pure perl script that calculates the track checksums for a cue/wav file pair and then checks them against the AccurateRip database. It's available from:

http://www.srcf.ucam.org/~cjk32/ARCue/ARCue.pl

Simply save it somewhere sensible, and from a command prompt, run:

ARCue.pl file1.cue file2.cue ...


A few points to bear in mind:

1) This is written in pure perl, so is quite slow, maybe 30s per track. If any perl gurus have any tips of speeding it up, they'd be much appreciated.

2) The range of cue files supported is probably very limited. It works with those generated by EAC. If it doesn't work with yours, please let me know.

3) The script relies on 32bit integer arithmetic, so I'm not sure how it'll behave on 64bit systems, any feedback will be gratefully recieved.

4) For reasons I can't fathom, approx 1 in 30 tracks checksums are incorrectly calculated. If a track is reported as inaccurately ripped, try running again before reripping. If anyone can shed any light on what might be going on, I'd be very grateful. So far, I can discern no noticable pattern as to when it happens.


Finally, as noted in the script, please read,

http://www.accuraterip.com/3rdparty-access.htm

before use.


Chris Key

[gib]

I have not tried it, but it sounds very cool. I suspect a lot of folks can use this tool, especially those that rip to an image+cue, which accuraterip does not support.

[bhoar]

I agree, very cool. Something I am likely to take a look at using in the future.

-brendan

[ssjkakaroto]

Sounds very good Christopher. Now AccurateRip can be used with single-wav albums

[brainsoft]

This is exactly what I've been looking for, and at 30sec/track it won't be too much slower than the alternative I was attempting with ripping as tracks and re-combining to lossless image after. Hopefully I can get this rigged up in React for track verification with image ripping!

[brainsoft]

Do you know if there are any particular settings to use or avoid to create a wav/cue image that can be verified with this script? I'm trying to rip single track wave and cue sheet but I'm not having much luck verifying the image afterwords. The tracks verify with AR when ripping as tracks, but switching to images with the same settings is not yielding the same results. GAP reading related perhaps?

Hello All,

I've throw together a pure perl script that calculates the track checksums for a cue/wav file pair and then checks them against the AccurateRip database. It's available from:

http://www.srcf.ucam.org/~cjk32/ARCue/ARCue.pl

Simply save it somewhere sensible, and from a command prompt, run:

ARCue.pl file1.cue file2.cue ...


A few points to bear in mind:

1) This is written in pure perl, so is quite slow, maybe 30s per track. If any perl gurus have any tips of speeding it up, they'd be much appreciated.

2) The range of cue files supported is probably very limited. It works with those generated by EAC. If it doesn't work with yours, please let me know.

3) The script relies on 32bit integer arithmetic, so I'm not sure how it'll behave on 64bit systems, any feedback will be gratefully recieved.

4) For reasons I can't fathom, approx 1 in 30 tracks checksums are incorrectly calculated. If a track is reported as inaccurately ripped, try running again before reripping. If anyone can shed any light on what might be going on, I'd be very grateful. So far, I can discern no noticable pattern as to when it happens.


Finally, as noted in the script, please read,

http://www.accuraterip.com/3rdparty-access.htm

before use.


Chris Key

[gib]

I finally got around to installing Perl and gave this script a try. It worked! It did seem perhaps a bit slow, but it steadily chugged along and verified that all the tracks were ripped successfully, displaying both the checksum and confidence level. At this point I've only tried one CD, but with AccurateRip previously being useless to me since I rip to image+cue, this script looks great. Many thanks, Christopher.

[wraithdu]

For the programming uninitiated, how do you go about installing Perl to use this script?

Oh, and if someone has a good way to integrate this into the REACT2 config for FLAC images, that would be great too.

This post has been edited by wraithdu: Mar 27 2007, 04:29

[gib]

For the programming uninitiated, how do you go about installing Perl to use this script?

Download and install ActivePerl and you're ready to go.

[caligae]

First of all big thanks. I've started using AccurateRip only recently (and already have ripped lots of CDs). I wanted to write something like this myself but didn't want to go through the hassle and apply for 3rd party access.

I'm on *nix. To parse the file correctly, I need to convert the .cue files to UNIX format and use '/' as path seperator. (No big problem)

edit: Hmm, seems like it only works with a single wave file, not with multiple wavs.

edit 2: I finally got it to work. There were strange errors that didn't happen using ActivePerl. As far as I can tell the default data types seem to be different and so I got integer overflows here (perl v5.8.8 on FreeBSD).

Disclaimer: I've never done anything in perl, so this is probably not the right fix, but at least it works for me and should give a hint what's wrong.

CODE

--- ARCue.pl.orig       Tue Mar 27 17:18:37 2007
+++ ARCue.pl    Tue Mar 27 17:12:24 2007
@@ -140,7 +140,7 @@
                        my $chunkTrackCount = unpack("c",substr($arCrcData,$ptr,1));
                        my $chunkDiscId1 = unpack("l",substr($arCrcData,$ptr+1,4));
                        my $chunkDiscId2 = unpack("l",substr($arCrcData,$ptr+5,4));
-                       my $chunkCddbDiscId = unpack("l",substr($arCrcData,$ptr+9,4));
+                       my $chunkCddbDiscId = unpack("I",substr($arCrcData,$ptr+9,4));
                        $ptr +=13;

                        if ( $chunkTrackCount != $trackCount
@@ -154,7 +154,7 @@
                        # How if it flagged that a track is not in the database?
                        for (my $track = 0; $track < $trackCount; $track++) {
                                my $trackConfidence = unpack("c",substr($arCrcData,$ptr,1));
-                               my $trackCrc = unpack("l",substr($arCrcData,$ptr+1,4));
+                               my $trackCrc = unpack("I",substr($arCrcData,$ptr+1,4));
                                $ptr += 9;

                                if ($arCrcCount == 0){

This post has been edited by caligae: Mar 27 2007, 16:20

[Christopher Key]

Do you know if there are any particular settings to use or avoid to create a wav/cue image that can be verified with this script? I'm trying to rip single track wave and cue sheet but I'm not having much luck verifying the image afterwords. The tracks verify with AR when ripping as tracks, but switching to images with the same settings is not yielding the same results. GAP reading related perhaps?

The disc id is generated based upon the length of every track in the album, so this info is required in the cue sheet in order for ARCue to function. I'm not familiar with how single track cue sheets / wav files work. If the info is there, then there's no reason why it can't work. The code was however written with whole album wav files in mind, so will need some updating. Could you send me a copy of the cue sheet you're trying to check, along with a description of exactly what the wav file contains, and I'll see what I can do.

Regards,

Chris

[greynol]

Besides the lengths of all the tracks, the start point of the first track is also necessary.

[caligae]

I was able to verify a multiple files rip. First converting the cuesheet using cuetools, then using foobar2000's "convert to single file" functionality.

This failed with other albums. Haven't checked yet, what's the problem. The calculated Disc ID is wrong for some reason.

[greynol]

Check my previous post.

A data track will influence the Disc ID as well.

This post has been edited by greynol: Mar 27 2007, 19:26

[caligae]

1) This is written in pure perl, so is quite slow, maybe 30s per track. If any perl gurus have any tips of speeding it up, they'd be much appreciated.

Using the String-CRC32 module might speed it up (As stated above, I never used perl.)

Check my previous post.

A data track will influence the Disc ID as well.

I tested some more discs and most of them worked. The one with the data track failed - this one also fails if I use "copy image". While the calculated CRCs seem to match, the Disc ID is wrong.

[Christopher Key]

[quote name='caligae' post='481280' date='Mar 27 2007, 11:12']
edit: Hmm, seems like it only works with a single wave file, not with multiple wavs.
[/quote]

Yes, currently only designed to work with single wav files. I'll update it to work with multiple wav file cue sheets.

[quote name='caligae' post='481280' date='Mar 27 2007, 11:12']
edit 2: I finally got it to work. There were strange errors that didn't happen using ActivePerl. As far as I can tell the default data types seem to be different and so I got integer overflows here (perl v5.8.8 on FreeBSD).
[/quote]

The pack documenation is available from
http://perldoc.perl.org/functions/pack.html

I used 'l' as that's exactly 32bits. Did you try it with 'L', I suspect that the problem may be because you're using a 64bit processor, and differences in the way unsigned integers are used.

Now, having looked more closely at the documenation, I've updated the code to use 'V' so that it's no longer architecture dependent, although the result is less than pretty. I've also improved the efficiency a bit. The new script is available from:

http://www.srcf.ucam.org/~cjk32/ARCue/ARCue-new.pl

Could you give this a go and see how it behaves. Once I've got a few reports that this works, I'll replace the older version.

[quote name='caligae' post='481280' date='Mar 27 2007, 11:12']
Disclaimer: I've never done anything in perl, so this is probably not the right fix, but at least it works for me and should give a hint what's wrong.
[/quote]
I've been using perl on and off for some time, but still feel something of a novice. I think it's that sort of language!

Regards,

Chris


[quote name='greynol' post='481367' date='Mar 27 2007, 18:25']
Check my previous post.

A data track will influence the Disc ID as well.
[/quote]

If anyone has an album with a data track, I'd be grateful if they could send me the cue sheet, along with the correct discids so I can add support.

[quote name='caligae' date='Mar 27 2007, 19:44' post='481388']
Using the String-CRC32 module might speed it up (As stated above, I never used perl.)
[/quote]
The algorithm isn't a CRC unfortunately.

I tested some more discs and most of them worked. The one with the data track failed - this one also fails if I use "copy image". While the calculated CRCs seem to match, the Disc ID is wrong.
[/quote]

Could you send me the cue sheet, along with the correct disc ids, and I'll have a go at adding support.

Regards,

Chris

[brainsoft]

Do you know if there are any particular settings to use or avoid to create a wav/cue image that can be verified with this script? I'm trying to rip single track wave and cue sheet but I'm not having much luck verifying the image afterwords. The tracks verify with AR when ripping as tracks, but switching to images with the same settings is not yielding the same results. GAP reading related perhaps?

The disc id is generated based upon the length of every track in the album, so this info is required in the cue sheet in order for ARCue to function. I'm not familiar with how single track cue sheets / wav files work. If the info is there, then there's no reason why it can't work. The code was however written with whole album wav files in mind, so will need some updating. Could you send me a copy of the cue sheet you're trying to check, along with a description of exactly what the wav file contains, and I'll see what I can do.

Regards,

Chris

I think it may be gap related but I've tried every setting in as many combinations as I can think of all with the same effect. The track CRCs don't match what is reported by EAC when I do TEST AND COPY (verified by AR). Maybe I've just got something screwed up here. Hell, if you "Rip image and create cuesheet" the cue sheet is different from the "create cuesheet > single file" cue. The DiscID is being generated correctly, I recognize the results confidence pattern for this album really well now.

I tested this with a few other albums without much luck, but this is the album I've been using for most of my testing. I'm currently doing fresh application installs to make sure I've not forgotten some setting I changed in the process. I'm going rip in Fb2k as well and see if I atleast get consistant results. And if the new file can check multi-wave files I'll verify that those are coming out right as well. I'll get back with cues when i know a bit more.

[caligae]

The pack documenation is available from
http://perldoc.perl.org/functions/pack.html

I used 'l' as that's exactly 32bits. Did you try it with 'L', I suspect that the problem may be because you're using a 64bit processor, and differences in the way unsigned integers are used.

Now, having looked more closely at the documenation, I've updated the code to use 'V' so that it's no longer architecture dependent, although the result is less than pretty. I've also improved the efficiency a bit. The new script is available from:

http://www.srcf.ucam.org/~cjk32/ARCue/ARCue-new.pl

Could you give this a go and see how it behaves. Once I've got a few reports that this works, I'll replace the older version.

Thanks, this seems to work (and also is noticeably faster). BTW, I'm using a 32bit processor, but perl was built with -Duse64bitint.

I tested some more discs and most of them worked. The one with the data track failed - this one also fails if I use "copy image". While the calculated CRCs seem to match, the Disc ID is wrong.

Could you send me the cue sheet, along with the correct disc ids, and I'll have a go at adding support.

Regards,

Chris

Here's the cue sheet:

CODE

REM GENRE Rock
REM DATE 1970
REM DISCID 7A0AD30A
REM COMMENT "ExactAudioCopy v0.95b4"
CATALOG 0724352190102
PERFORMER "David Bowie"
TITLE "The Man Who Sold The World"
FILE "Range.wav" WAVE
TRACK 01 AUDIO
TITLE "The Width Of A Circle"
PERFORMER "David Bowie"
ISRC USJT19900014
INDEX 01 00:00:00
TRACK 02 AUDIO
TITLE "All The Madmen"
PERFORMER "David Bowie"
ISRC USJT19900015
INDEX 00 08:06:22
INDEX 01 08:08:57
TRACK 03 AUDIO
TITLE "Black Country Rock"
PERFORMER "David Bowie"
ISRC USJT19900016
INDEX 00 13:47:52
INDEX 01 13:49:45
TRACK 04 AUDIO
TITLE "After All"
PERFORMER "David Bowie"
ISRC USJT19900017
INDEX 00 17:24:25
INDEX 01 17:25:50
TRACK 05 AUDIO
TITLE "Running Gun Blues"
PERFORMER "David Bowie"
ISRC USJT19900018
INDEX 00 21:19:00
INDEX 01 21:21:10
TRACK 06 AUDIO
TITLE "Saviour Machine"
PERFORMER "David Bowie"
ISRC USJT19900019
INDEX 00 24:34:05
INDEX 01 24:36:20
TRACK 07 AUDIO
TITLE "She Shook Me Cold"
PERFORMER "David Bowie"
ISRC USJT19900020
INDEX 00 29:02:37
INDEX 01 29:05:00
TRACK 08 AUDIO
TITLE "The Man Who Sold The World"
PERFORMER "David Bowie"
ISRC USJT19900021
INDEX 00 33:18:65
INDEX 01 33:21:45
TRACK 09 AUDIO
TITLE "The Supermen"
PERFORMER "David Bowie"
ISRC USJT19900022
INDEX 00 37:19:27
INDEX 01 37:21:40

It obviously can't work since the data track isn't included in the cue sheet. (There is an entry for the data track in the multiple wavs/non-compliant cue sheet, but I think its length is not included)

I don't know if there's a good solution for this one, maybe letting the user enter the data track manually?

EAC reports: [Disc ID: 000fe2a0-7a0ad30a]

I think this EAC screenshot should contain enough information to calculate the correct Disc ID.


Moderation: Codebox.

This post has been edited by greynol: Apr 9 2008, 06:21

[Christopher Key]

Could you send me the cue sheet, along with the correct disc ids, and I'll have a go at adding support.

Here's the cue sheet:

CODE

REM GENRE Rock
REM DATE 1970
REM DISCID 7A0AD30A
REM COMMENT "ExactAudioCopy v0.95b4"
CATALOG 0724352190102
PERFORMER "David Bowie"
TITLE "The Man Who Sold The World"
FILE "Range.wav" WAVE
  TRACK 01 AUDIO
    TITLE "The Width Of A Circle"
    PERFORMER "David Bowie"
    ISRC USJT19900014
    INDEX 01 00:00:00

...

  TRACK 09 AUDIO
    TITLE "The Supermen"
    PERFORMER "David Bowie"
    ISRC USJT19900022
    INDEX 00 37:19:27
    INDEX 01 37:21:40

It obviously can't work since the data track isn't included in the cue sheet. (There is an entry for the data track in the multiple wavs/non-compliant cue sheet, but I think its length is not included)

I don't know if there's a good solution for this one, maybe letting the user enter the data track manually?

EAC reports: [Disc ID: 000fe2a0-7a0ad30a]

Yes, that sounds like the best solution, prompting for a data length for any data tracks. I'm moving over the weekend, but next week I'll have a go at tidying things up to support non compliant cue sheets and so on.

Regards,

Chris

[caligae]

It obviously can't work since the data track isn't included in the cue sheet. (There is an entry for the data track in the multiple wavs/non-compliant cue sheet, but I think its length is not included)

I don't know if there's a good solution for this one, maybe letting the user enter the data track manually?

EAC reports: [Disc ID: 000fe2a0-7a0ad30a]

Yes, that sounds like the best solution, prompting for a data length for any data tracks. I'm moving over the weekend, but next week I'll have a go at tidying things up to support non compliant cue sheets and so on.

Have fun while moving. I'll probably do some more testing in the meantime.

Once the program works stable, we really should look into optimization. I fear this can't be done in pure perl very well. I don't know what options you have in perl. At least I know you can use C code in a module - but that will complicate distribution.

I've quickly hacked the checksumming in C. The perl script needed about 100 seconds for verifying an image I used for testing. The C version was limited by disk speed. With data already in cache, it took about 1.3 seconds.

[madxcream]

Great program. I found a cd that doesn't have a data track, and the disc ID is reported wrong by arcue. Here is the cuesheet:

CODE

REM GENRE "Christian Rock"
REM DATE 2006
REM DISCID 29107115
REM COMMENT "ExactAudioCopy v0.95b4"
PERFORMER "Further Seems Forever"
TITLE "Hope This Finds You Well"
FILE "Further Seems Forever - [2006] Hope This Finds You Well.flac" WAVE
TRACK 01 AUDIO
TITLE "The Moon Is Down"
PERFORMER "Further Seems Forever"
INDEX 01 00:00:00
TRACK 02 AUDIO
TITLE "Pride War"
PERFORMER "Further Seems Forever"
INDEX 00 03:12:57
INDEX 01 03:13:57
TRACK 03 AUDIO
TITLE "Hide Nothing"
PERFORMER "Further Seems Forever"
INDEX 00 06:20:40
INDEX 01 06:21:40
TRACK 04 AUDIO
TITLE "Snowbirds And Townies"
PERFORMER "Further Seems Forever"
INDEX 00 09:18:19
INDEX 01 09:19:19
TRACK 05 AUDIO
TITLE "Light Up Ahead"
PERFORMER "Further Seems Forever"
INDEX 00 13:46:24
INDEX 01 13:47:24
TRACK 06 AUDIO
TITLE "Against My Better Judgement"
PERFORMER "Further Seems Forever"
INDEX 00 16:56:04
INDEX 01 16:57:04
TRACK 07 AUDIO
TITLE "The Bradley"
PERFORMER "Further Seems Forever"
INDEX 00 20:38:58
INDEX 01 20:39:58
TRACK 08 AUDIO
TITLE "New Year's Project"
PERFORMER "Further Seems Forever"
INDEX 00 23:38:35
INDEX 01 23:39:34
TRACK 09 AUDIO
TITLE "How To Start A Fire"
PERFORMER "Further Seems Forever"
INDEX 00 27:43:51
INDEX 01 27:43:69
TRACK 10 AUDIO
TITLE "Someone You Know"
PERFORMER "Further Seems Forever"
INDEX 00 30:35:54
INDEX 01 30:36:04
TRACK 11 AUDIO
TITLE "The Sound"
PERFORMER "Further Seems Forever"
INDEX 00 33:52:47
INDEX 01 33:53:01
TRACK 12 AUDIO
TITLE "Wearing Thin"
PERFORMER "Further Seems Forever"
INDEX 00 37:31:47
INDEX 01 37:32:15
TRACK 13 AUDIO
TITLE "Bleed"
PERFORMER "Further Seems Forever"
INDEX 00 40:18:49
INDEX 01 40:19:48
TRACK 14 AUDIO
TITLE "Pictures Of Shorelines"
PERFORMER "Further Seems Forever"
INDEX 00 43:15:66
INDEX 01 43:16:65
TRACK 15 AUDIO
TITLE "For All We Know"
PERFORMER "Further Seems Forever"
INDEX 00 46:17:62
INDEX 01 46:18:61
TRACK 16 AUDIO
TITLE "Vengeance Factor (feat. Chris Carrabba)"
PERFORMER "Further Seems Forever"
INDEX 00 49:05:25
INDEX 01 49:06:24
TRACK 17 AUDIO
TITLE "There, Now I've Said It (feat. Jason Gleason)"
PERFORMER "Further Seems Forever"
INDEX 00 51:52:61
INDEX 01 51:53:28
TRACK 18 AUDIO
TITLE "Say It Ain't So (feat. Chris Carrabba)"
PERFORMER "Further Seems Forever"
INDEX 00 54:59:00
INDEX 01 54:59:74
TRACK 19 AUDIO
TITLE "Light Up Ahead (Acoustic) (feat. Jon Bunch)"
PERFORMER "Further Seems Forever"
INDEX 00 59:03:31
INDEX 01 59:03:67
TRACK 20 AUDIO
TITLE "Bye Bye Bye (feat. Jason Gleason)"
PERFORMER "Further Seems Forever"
INDEX 00 62:07:22
INDEX 01 62:07:47
TRACK 21 AUDIO
TITLE "Justice Prevails (feat. Chris Carrabba)"
PERFORMER "Further Seems Forever"
INDEX 00 65:30:56
INDEX 01 65:31:55

disc ID reported by EAC when ripping tracks : 0034a1e1-29107115
disc ID reported by ARCue: 0034a1e1-28107115

Thanks for any help.

Moderation: Codebox.

This post has been edited by greynol: Apr 9 2008, 06:23

[madxcream]

I have another problem with a cd I just tried. With this disc ID: 00115f65-890a960a reported correctly with EAC and ARCue. Here is the problem.

EAC reports for track 1: Accurately Ripped (confidence 3) [eccc8d81]
ARCue reports for track 1: Track not present in database. [eccc8d81]

Here is copy and past of the disc ID also to clarify:

EAC : [Disc ID: 00115f65-890a960a]
ARCue: [Disc ID: 00115f65-890a960a]

This happens with all the tracks!

Anyone else getting problems with cd's that don't have data tracks also like me?

This post has been edited by madxcream: Mar 31 2007, 14:36

[Christopher Key]

I have another problem with a cd I just tried. With this disc ID: 00115f65-890a960a reported correctly with EAC and ARCue. Here is the problem.

EAC reports for track 1: Accurately Ripped (confidence 3) [eccc8d81]
ARCue reports for track 1: Track not present in database. [eccc8d81]

Here is copy and past of the disc ID also to clarify:

EAC : [Disc ID: 00115f65-890a960a]
ARCue: [Disc ID: 00115f65-890a960a]

This happens with all the tracks!

Anyone else getting problems with cd's that don't have data tracks also like me?

First part of the move completed...

Is this using the newer version of the script? Does it work correcty with the older version? I did have problems like this myself, it essentially boiled down to perl comparing two identical 32bit values as if one were signed and the other weren't. That was what the {use integer; $CRC = ... + 0} was to fix. I'll have another dig.

Regards,

Chris

[Christopher Key]

disc ID reported by EAC when ripping tracks : 0034a1e1-29107115
disc ID reported by ARCue: 0034a1e1-28107115

Should be fixed now.

For future reference, the latest version will now always be ARCue.pl, and i'll keep an archive of older versions online, named as ARCue-rXX.pl.

Chris

[Christopher Key]

Have fun while moving. I'll probably do some more testing in the meantime.

First stage done, I've got two weeks at home now before heading off again.

Once the program works stable, we really should look into optimization. I fear this can't be done in pure perl very well. I don't know what options you have in perl. At least I know you can use C code in a module - but that will complicate distribution.

I've quickly hacked the checksumming in C. The perl script needed about 100 seconds for verifying an image I used for testing. The C version was limited by disk speed. With data already in cache, it took about 1.3 seconds.

Indeed, perl really isn't written for doing this sort of thing.

I'm keen to keep a pure perl version for folks who just want something simple, but I agree a faster alternative would be better.

The best option I came up with was to try to create a generic perl module to do the majority of the work. This would either call compiled c code, or do that calculation in native perl if that option was unavailable. On win32 platforms, it would also be set up to call functions from AccurateRip.dll to query the database etc where appropriate. I did start having a go at writing an XS module that would call AccurateRip.dll; I got as far as writing skeleton code in C that would call most of the functions successfully.

Chris