Welcome Guest ( Log In | Register )

Reply to this topicStart new topic
Linux command-line tool for converting WAV-image to FLAC singletracks, Input: EAC LOG + WAV + CUE, no Internet / user interaction required
post Oct 28 2012, 18:03
Post #1

Group: Members
Posts: 44
Joined: 27-October 12
Member No.: 104130

This is an encoder script for EAC "Test & copy image" mode.
It splits the WAV image to WAV singletracks according to the EAC CUE and encodes them to FLACs.
Its goal is to produce perfect FLAC files by verifying checksums in all steps and being totally paranoid.
For guaranteeing this, it does the following checks and aborts if any of them fails:
  • It checks the EAC LOG to make sure that AccurateRip reported a flawless rip.
  • It computes the AccurateRip checksums of the splitfiles which it has created and compares them with the ones from the EAC LOG.
  • It re-joins the singletrack files to an image and compares the checksum with the checksum of the original image from EAC to make sure that it would be possible to burn an identical CD again.
  • It encodes the singletracks to FLAC with very carefully chosen settings. The full manpage of FLAC was read by me when chosing the settings.
  • It runs flac --test on each singletrack which makes FLAC test the integrity of the file.
  • It decodes each singletrack to WAV again and compares the checksums with the checksums of the original WAV splitfiles.
Additional features are:
  • The script has 4 stages of output: Split singletracks, rejoined image from singletracks, encoded FLAC from singletracks, decoded FLAC to singletracks again. There are 4 config variables which can be used to make the script deliberately cause damage to any of the 4 stages. This can be used for making very sure that the checksum validation actually detects corruption.
Website: https://github.com/leo-bogert/perfect-flac-encode

Sample output:

perfect-flac-encode.sh Version BETA2 running ...
BETA VERSION - NOT for productive use!

Album: Paul Weller - Wild Wood
Checking EAC LOG for whether AccurateRip reports a perfect rip...
AccurateRip reports a perfect rip.
Splitting WAV image to singletrack WAVs...
shntool [split]: warning: discarding initial zero-valued split point
Splitting [Paul Weller - Wild Wood.wav] (54:04.55) --> [Stage1_WAV_Singletracks_From_WAV_Image/01 - Sunflower.wav] (4:06.62) : .......... OK
Splitting [Paul Weller - Wild Wood.wav] (54:04.55) --> [Stage1_WAV_Singletracks_From_WAV_Image/02 - Can you heal us (holy man).wav] (3:41.68) : .......... OK
Splitting [Paul Weller - Wild Wood.wav] (54:04.55) --> [Stage1_WAV_Singletracks_From_WAV_Image/03 - Wild wood.wav] (3:22.30) : .......... OK
Splitting [Paul Weller - Wild Wood.wav] (54:04.55) --> [Stage1_WAV_Singletracks_From_WAV_Image/04 - Instrumental one (Part 1).wav] (1:37.05) : .......... OK
Splitting [Paul Weller - Wild Wood.wav] (54:04.55) --> [Stage1_WAV_Singletracks_From_WAV_Image/05 - All the pictures on the wall.wav] (3:56.47) : .......... OK
Splitting [Paul Weller - Wild Wood.wav] (54:04.55) --> [Stage1_WAV_Singletracks_From_WAV_Image/06 - Has my fire really gone out.wav] (3:51.38) : .......... OK
Splitting [Paul Weller - Wild Wood.wav] (54:04.55) --> [Stage1_WAV_Singletracks_From_WAV_Image/07 - Country.wav] (3:38.60) : .......... OK
Splitting [Paul Weller - Wild Wood.wav] (54:04.55) --> [Stage1_WAV_Singletracks_From_WAV_Image/08 - Instrumental two.wav] (0:49.52) : .......... OK
Splitting [Paul Weller - Wild Wood.wav] (54:04.55) --> [Stage1_WAV_Singletracks_From_WAV_Image/09 - 5th season.wav] (4:54.00) : .......... OK
Splitting [Paul Weller - Wild Wood.wav] (54:04.55) --> [Stage1_WAV_Singletracks_From_WAV_Image/10 - The weaver.wav] (3:43.08) : .......... OK
Splitting [Paul Weller - Wild Wood.wav] (54:04.55) --> [Stage1_WAV_Singletracks_From_WAV_Image/11 - Instrumental one (Part 2).wav] (0:33.42) : .......... OK
Splitting [Paul Weller - Wild Wood.wav] (54:04.55) --> [Stage1_WAV_Singletracks_From_WAV_Image/12 - Foot of the mountain.wav] (3:37.60) : .......... OK
Splitting [Paul Weller - Wild Wood.wav] (54:04.55) --> [Stage1_WAV_Singletracks_From_WAV_Image/13 - Shadow of the sun.wav] (7:36.43) : .......... OK
Splitting [Paul Weller - Wild Wood.wav] (54:04.55) --> [Stage1_WAV_Singletracks_From_WAV_Image/14 - Holy man (Reprise).wav] (1:50.72) : .......... OK
Splitting [Paul Weller - Wild Wood.wav] (54:04.55) --> [Stage1_WAV_Singletracks_From_WAV_Image/15 - Moon on your pyjamas.wav] (4:02.00) : .......... OK
Splitting [Paul Weller - Wild Wood.wav] (54:04.55) --> [Stage1_WAV_Singletracks_From_WAV_Image/16 - Hung up.wav] (2:41.68) : .......... OK
Comparing AccurateRip checksums of split WAV singletracks to AccurateRip checksums from EAC LOG...
AccurateRip checksum of track 01: 12A17108, expected 12A17108. OK.
AccurateRip checksum of track 02: B8B12553, expected B8B12553. OK.
AccurateRip checksum of track 03: EE647F8C, expected EE647F8C. OK.
AccurateRip checksum of track 04: 198D1B89, expected 198D1B89. OK.
AccurateRip checksum of track 05: 443F683F, expected 443F683F. OK.
AccurateRip checksum of track 06: D4EA2156, expected D4EA2156. OK.
AccurateRip checksum of track 07: 23A1013A, expected 23A1013A. OK.
AccurateRip checksum of track 08: 59846ADB, expected 59846ADB. OK.
AccurateRip checksum of track 09: 562695D4, expected 562695D4. OK.
AccurateRip checksum of track 10: F8F465AC, expected F8F465AC. OK.
AccurateRip checksum of track 11: 779AA114, expected 779AA114. OK.
AccurateRip checksum of track 12: 826FDC3D, expected 826FDC3D. OK.
AccurateRip checksum of track 13: 08E26E72, expected 08E26E72. OK.
AccurateRip checksum of track 14: BB90680D, expected BB90680D. OK.
AccurateRip checksum of track 15: 37AAC77A, expected 37AAC77A. OK.
AccurateRip checksum of track 16: 1A3CD493, expected 1A3CD493. OK.
Joining singletrack WAV temporarily for comparing their checksum with the original image's checksum...
Joining [Stage1_WAV_Singletracks_From_WAV_Image/01 - Sunflower.wav] (4:06.62) --> [Stage2_WAV_Image_Joined_From_WAV_Singletracks/joined.wav] (54:04.55) : .......... OK
Joining [Stage1_WAV_Singletracks_From_WAV_Image/02 - Can you heal us (holy man).wav] (3:41.68) --> [Stage2_WAV_Image_Joined_From_WAV_Singletracks/joined.wav] (54:04.55) : .......... OK
Joining [Stage1_WAV_Singletracks_From_WAV_Image/03 - Wild wood.wav] (3:22.30) --> [Stage2_WAV_Image_Joined_From_WAV_Singletracks/joined.wav] (54:04.55) : .......... OK
Joining [Stage1_WAV_Singletracks_From_WAV_Image/04 - Instrumental one (Part 1).wav] (1:37.05) --> [Stage2_WAV_Image_Joined_From_WAV_Singletracks/joined.wav] (54:04.55) : .......... OK
Joining [Stage1_WAV_Singletracks_From_WAV_Image/05 - All the pictures on the wall.wav] (3:56.47) --> [Stage2_WAV_Image_Joined_From_WAV_Singletracks/joined.wav] (54:04.55) : .......... OK
Joining [Stage1_WAV_Singletracks_From_WAV_Image/06 - Has my fire really gone out.wav] (3:51.38) --> [Stage2_WAV_Image_Joined_From_WAV_Singletracks/joined.wav] (54:04.55) : .......... OK
Joining [Stage1_WAV_Singletracks_From_WAV_Image/07 - Country.wav] (3:38.60) --> [Stage2_WAV_Image_Joined_From_WAV_Singletracks/joined.wav] (54:04.55) : .......... OK
Joining [Stage1_WAV_Singletracks_From_WAV_Image/08 - Instrumental two.wav] (0:49.52) --> [Stage2_WAV_Image_Joined_From_WAV_Singletracks/joined.wav] (54:04.55) : .......... OK
Joining [Stage1_WAV_Singletracks_From_WAV_Image/09 - 5th season.wav] (4:54.00) --> [Stage2_WAV_Image_Joined_From_WAV_Singletracks/joined.wav] (54:04.55) : .......... OK
Joining [Stage1_WAV_Singletracks_From_WAV_Image/10 - The weaver.wav] (3:43.08) --> [Stage2_WAV_Image_Joined_From_WAV_Singletracks/joined.wav] (54:04.55) : .......... OK
Joining [Stage1_WAV_Singletracks_From_WAV_Image/11 - Instrumental one (Part 2).wav] (0:33.42) --> [Stage2_WAV_Image_Joined_From_WAV_Singletracks/joined.wav] (54:04.55) : .......... OK
Joining [Stage1_WAV_Singletracks_From_WAV_Image/12 - Foot of the mountain.wav] (3:37.60) --> [Stage2_WAV_Image_Joined_From_WAV_Singletracks/joined.wav] (54:04.55) : .......... OK
Joining [Stage1_WAV_Singletracks_From_WAV_Image/13 - Shadow of the sun.wav] (7:36.43) --> [Stage2_WAV_Image_Joined_From_WAV_Singletracks/joined.wav] (54:04.55) : .......... OK
Joining [Stage1_WAV_Singletracks_From_WAV_Image/14 - Holy man (Reprise).wav] (1:50.72) --> [Stage2_WAV_Image_Joined_From_WAV_Singletracks/joined.wav] (54:04.55) : .......... OK
Joining [Stage1_WAV_Singletracks_From_WAV_Image/15 - Moon on your pyjamas.wav] (4:02.00) --> [Stage2_WAV_Image_Joined_From_WAV_Singletracks/joined.wav] (54:04.55) : .......... OK
Joining [Stage1_WAV_Singletracks_From_WAV_Image/16 - Hung up.wav] (2:41.68) --> [Stage2_WAV_Image_Joined_From_WAV_Singletracks/joined.wav] (54:04.55) : .......... OK
No padding needed.
Computing checksums...
Original checksum: e47804481da6959636de7fbad147e3f2b2131c682aa54f07a2405d2b27aa7bdf
Checksum of joined image: e47804481da6959636de7fbad147e3f2b2131c682aa54f07a2405d2b27aa7bdf
Checksum of joined image OK.
Encoding singletrack WAVs to FLAC ...
NOTE: --keep-foreign-metadata is a new feature; make sure to test the output file before deleting the original.
Running flac --test on singletrack FLACs...
Decoding singletrack FLACs to WAVs to validate checksums ...
NOTE: --keep-foreign-metadata is a new feature; make sure to test the output file before deleting the original.
Generating checksums of original WAV files...
Validating checksums of decoded WAV singletracks ...
01 - Sunflower.wav: OK
02 - Can you heal us (holy man).wav: OK
03 - Wild wood.wav: OK
04 - Instrumental one (Part 1).wav: OK
05 - All the pictures on the wall.wav: OK
06 - Has my fire really gone out.wav: OK
07 - Country.wav: OK
08 - Instrumental two.wav: OK
09 - 5th season.wav: OK
10 - The weaver.wav: OK
11 - Instrumental one (Part 2).wav: OK
12 - Foot of the mountain.wav: OK
13 - Shadow of the sun.wav: OK
14 - Holy man (Reprise).wav: OK
15 - Moon on your pyjamas.wav: OK
16 - Hung up.wav: OK
All checksums OK.
Go to the top of the page
+Quote Post
post Oct 28 2012, 19:32
Post #2

Group: Members
Posts: 704
Joined: 12-May 06
From: Colorado, USA
Member No.: 30694

How is HTOA handled?

Does "Go fuck yourself" really need to be one of the messages?
Go to the top of the page
+Quote Post
post Oct 28 2012, 19:41
Post #3

Group: Members
Posts: 44
Joined: 27-October 12
Member No.: 104130

QUOTE (mjb2006 @ Oct 28 2012, 19:32) *
How is HTOA handled?

perfect-flac-encode itself does not parse CUE-sheets. They are interpreted by shntool.
Therefore, the HTOA handling is up to shntool.
The current commandline of shntool which is used is the following:
shntool split -P dot -d "$outputdir_relative" -f "$1.cue" -n %02d -t "%n - %t" -- "$1.wav"

($1 = filename of cue/wav)
The commandline of shntool is still up for discussion. If you feel like different settings should be chosen for better HTOA handling, feel free to suggest those.

QUOTE (mjb2006 @ Oct 28 2012, 19:32) *
Does "Go fuck yourself" really need to be one of the messages?

Consider it as an easter-egg laugh.gif
- The tool is aimed for being used in a setup to produce absolutely perfect rips, incorrect rips are not welcome wink.gif
I've changed the wording though, thanks for reporting this, I had forgotten about it.

This post has been edited by leo-bogert: Oct 28 2012, 20:33
Go to the top of the page
+Quote Post
post Oct 31 2012, 19:10
Post #4

Group: Members
Posts: 44
Joined: 27-October 12
Member No.: 104130

BETA3 available now.

- Fix possible bug in startup temp dir deletion when using directory names with spaces
- Make syntax match the syntax which is described in README.md: BETA2 used to assume that the CUE/WAV/LOG reside in a "<Artist> - <Title>" subdirectory of the directory which is the first parameter. We now assume that the first parameter is the directory where they reside in.
- Move output FLACs to a newly created "<Artist> - <Title>" subdirectory upon success (and include this directory in the "delete existing output?" check upon startup).
- Copy CUE/LOG to output directory ("<Artist> - <Title>") upon success
- Delete temporary output directories upon success
- Print an explicity SUCCESS message with the name of the output dir
- Add TODO which shall be resolved in BETA4: What about the checksum in the EAC-LOG? Is it a plain CRC32 or a magic checksum with some samples excluded? If it is no plain checksum then we should keep the sha256sum as a file in the output dir for the user so he can check the sum in case he ever re-joins the FLACs to a WAV image.
- Internal cleanup: Rename variables / functions to make their purpose more obvious
Go to the top of the page
+Quote Post
post Nov 1 2012, 03:58
Post #5

Group: Members
Posts: 44
Joined: 27-October 12
Member No.: 104130

BETA4 available now.
  • New output: Generate a .sha256 file which contains the SHA256-checksum of the original WAV image. If the user ever needs to burn a CD which is bit-identical to the original disc, he will have to decode the FLACs and re-join the WAVs to an image. For making sure that this produced an image which is identical to the original one, he can use the SHA256-checksum. Notice that the CRC from the EAC LOG is not suitable for this because it is a "custom" CRC which does not include the full file - therefore we generate the said SHA256.
  • Add TODOs for automatically generating a README in the output directory with information about the rip, instructions how to restore the original image from the FLACs, etc
  • Add TODO for validating the checksum of the original image against the EAC checksum from the EAC LOG
  • Documentation: Update REAMDE.md to include the sha256 feature
  • Documentation: Update README.md to include the secondary goal of perfect-flac-encode: Perfect backup of CDs.
  • Internal Documentation: Explain that and why we use a generated silent WAV for damaging the decoded WAV singletracks
  • Internal cleanup: Rename functions to expicitely state that they will quit upon any errors
Go to the top of the page
+Quote Post
post Nov 1 2012, 17:02
Post #6

Group: Members
Posts: 25
Joined: 14-February 12
Member No.: 97152

The CRC outputted by EAC excludes the RIFF header, just hashes the raw PCM audio data. If you rip to wav and delete the first 44 bytes of wav file, CRC of that file will match the reported by EAC. So, you could change the routine so that ignores the first 44 bytes of outputted wav fie, and hashes just the audio data. It is a plain CRC32, nothing magic, like AccurateRip's checksum.
Go to the top of the page
+Quote Post
post Nov 1 2012, 17:46
Post #7

Group: Developer
Posts: 1343
Joined: 4-May 04
From: France
Member No.: 13875

To compute an EAC-compatible CRC hash:

mkfifo 'fifo'
shncat -q -e 'file.wav' > 'fifo' &
crc="$( cksfv -b 'fifo' | fgrep -v ';' )"

There must be a more straightforward way, but I haven't investigated any further.

Go to the top of the page
+Quote Post
post Nov 4 2012, 17:48
Post #8

Group: Members
Posts: 44
Joined: 27-October 12
Member No.: 104130

QUOTE (skamp @ Nov 1 2012, 17:46) *
To compute an EAC-compatible CRC hash:

mkfifo 'fifo'
shncat -q -e 'file.wav' > 'fifo' &
crc="$( cksfv -b 'fifo' | fgrep -v ';' )"

There must be a more straightforward way, but I haven't investigated any further.
Thank you. Yes, I found a more straightforward way which doesn't need a FIFO and uses a pipe instead.
Using that, I've published a tool to compute the EAC CRC on github:

I hope you are OK with me adding a donation address to it. I decided for this because my script works quite different to yours.

This post has been edited by leo-bogert: Nov 4 2012, 18:01
Go to the top of the page
+Quote Post
post Nov 4 2012, 18:01
Post #9

Group: Members
Posts: 44
Joined: 27-October 12
Member No.: 104130

BETA6 available now.
The changelog of BETA5 should be considered, BETA6 was needed for a simple quickfix and contains no actual changes.

BETA5/6 Changelog:
  • Add code which checks whether the Test and Copy CRCs from the EAC LOG match
  • Add code which validates the EAC Copy CRC from the EAC LOG against the CRC of the actual EAC image. This requires the availability of a new tool called eac-crc which I will create a new repository on github for soon. Thanks to pablogm123 and skamp on hydrogenaudio for explaining the EAC CRC algorithm!
  • Also add a unit test for checking whether the EAC CRC validation actually works. For this the input file will be renamed and replaced with a silent WAV.
Go to the top of the page
+Quote Post
post Nov 7 2012, 00:47
Post #10

Group: Members
Posts: 44
Joined: 27-October 12
Member No.: 104130

BETA7 available. Changelog follows. Notice that there are quite a few questions in in which I would request feedback for. Thank you!
  • Remove "--keep-foreign-metadata" FLAC parameter. This parameter was originally added because we believed that it would be needed for being able to restore the original WAV files including all WAV headers. We believed that the checksum of the singletrack WAVs or at least the re-joined full-disc-image WAV would be wrong. However testing has shown that the checksums are okay. Question / TODO: What IS the "foreign-metadata" actually?
  • Explicitely tell the user whether hidden track one audio (HTOA) exists.
  • Print detected track count
  • Upon AccurateRip checksum validation failure, do not abort immediately anymore. Instead, print all AccurateRip checksums and then abort. This helps debugging.
  • Bugfix AccurateRip checksum validation: Also allow AccurateRip v1 checksums. I've learned that EAC will still produce v1 checksums for some discs. Maybe because there are not enough v2 checksums for those in the database?
  • Bugfix HTOA AccurateRip handling: Exclude track 0 in computations.
  • Bugfix removal of leading 0 in tracknumber for HTOA: shntool will number the hidden track as 0, the old code would remove the whole tracknumber. We only remove the first 0 now.
  • Bugfix code for getting total track count for AccurateRip+HTOA: Use trackcount from CUE instead of number of WAV files (requires cuetools). The hidden track is a separate WAV file but should not be included in total track count because it is labled as track 0. In other words: The track count of the old code would be one too much, resulting in the accuraterip-checksum tool not being able to detect the last track which should have caused a wrong checksum of the last track. What is weird: The v1 checksum of the last track did not change with the new code. Maybe the AccurateRip developers can answer why.
  • Add TODO: shntool generates a "00 - pregap.wav" for HTOA. Decide what to do about this and check MusicBrainz for what they prefer. Options are: Keep as track 0? Merge with track 1? Shift all tracks by 1?
  • Add TODO: I saw that "shntool len" is able to detect common problems of WAV files. Maybe we should do "shntool len" and check for problems? Or will those problems never happen with EAC?
  • Add TODO: Decide about a good filename for the CUE/LOG which are copied to the output. Ideally we should use a filename which makes it easy to understand for non-experts what those files are for. Something which explains that the CUE is for being able to burn the original disc again and the LOG is a proof of quality. On the other hand, we already have writing out a README.txt on the TODO list already, we will explain the purpose of CUE/LOG there anyway.
  • Add TODO: "Pre-"Tag FLACs from cuesheet with a separate tool I wrote. This tool will be available on github soon. "Pre-" because it will only copy the UPC/EAN/ISRC because those are actually stored on the disc. The real tagging will happen with MusicBrainz because it is the perfect tagging solution.
  • Add TODO: We should really write all output to a perfect-flac-encode.log and copy it to the output directory.
  • Add TODO: Replace Windows/Mac reserved filename characters in filenames generated from CUE by shntool
  • Internal stuff: Cleanup regexp usage: Use grep+sed instead of bash regexp because the bash ones seem to be buggy and grep+sed is less lines of code than the bash workarounds. Also, get rid of copypasted regexp, use variables instead. Further, there was some pure grep regexp parsing. Use grep+sed to make it more readable (we use sed for direct access to capturing groups by number) and more failsafe.
  • Internal stuff: Make regexp match full lines instead of parts of lines. This makes them more failsafe
  • Internal stuff: Abort when wrong parameters are passed to functions
  • Internal stuff: Make variables local which were global accidentially

This post has been edited by leo-bogert: Nov 7 2012, 00:55
Go to the top of the page
+Quote Post
post Nov 8 2012, 02:56
Post #11

Group: Members
Posts: 44
Joined: 27-October 12
Member No.: 104130

BETA8 available now. Changelog:
  • New feature: Pre-tag FLAC files with tags from the CUE. Only the tags which are physically stored on the disc (and our version info) are used. This is because CUEs are not sufficient as tag storage and EAC is not sufficient as a tagging tool. I recommend MusicBrainz for the rest as it is really a perfect tagging tool. The tags which are added are: TRACKTOTAL CATALOGNUMBER ISRC ENCODEDBY (perfect-flac-encode version + flac version) TRACKNUMBER COMMENT (this is the "track message". Question: Is this actually stored on the CD?) TOTALTRACKS.
  • Add TODO: We should check MusicBrainz for whether they want HTOA to be stored as track 0 or track 1 or merged with the actual track 1.
  • Internal stuff: Rename get_total_wav_tracks_without_hiddentracks to remove the "wav", the function gets the trackcount from the CUE so WAV is irrelevant
Go to the top of the page
+Quote Post
post Nov 9 2012, 05:46
Post #12

Group: Members
Posts: 44
Joined: 27-October 12
Member No.: 104130

BETA9/10 available.
10 is only a bugfix for 9, so here is the changelog of 9:
  • New feature: A README.txt is written to the output directory. It tries to explain in layman's terms how the rip was created, which software was used, that the rip should be high quality, which files are in the directory, and how to restore an identical CD.
  • Add TODO: We should append the output of perfect-flac-encode to the EAC LOG maybe. First, we should investigate whether there are any tools to validate the checksums of EAC LOGs. If there are, we have to create a new checksum and append it. We should also check whether this breaks those tools. If it does, we should write our own log file. Notice that this feature is referenced in the README.txt even though it is not implemented yet.
  • Internal stuff: Capitalize the name of the global variable "WORKING_DIR_ABSOLUTE". Document "globals should be uppercase" as naming convention. More to follow.
  • Internal stuff: I did the time consuming job of searching the whole script for "=" to find all implicit variable declarations. Variables which were global by accident are now local.
  • Internal stuff: Remove unused commented out code. We don't generate FLAC images because the support of players for them is poor.

Sample REAMDE.txt:
About the quality of this CD copy:
These audio files were produced with perfect-flac-encode version BETA10, using
flac 1.2.1.
They are stored in "FLAC" format, which is lossless. This means that they can
have the same quality as an audio CD.
This is much better than MP3 for example: MP3 leaves out some tones because
some people cannot hear them.

The used prefect-flac-encode is a program which converts good Exact Audio Copy
CD copies to FLAC audio files.
The goal of perfect-flac-encode is to produce CD copies with the best quality
This is NOT only about the quality of the audio: It also means that the files
can be used as a perfect backup of your CDs. The author of the software even
trusts them for digital preservation for ever.
For allowing this, the set of files which you received were designed to make it
possible to burn a disc which is absolutely identical to the original one in
case your original disc is destroyed.
Therefore, please do not delete any of the contained files!
For instructions on how to restore the original disc, please visit the website
of perfect-flac-decode. You can find the address below.

Explanation of the purpose of the files you got:
"Paul Weller - 1994 - Wild Wood.cue"
This file contains the layout of the original disc. It will be the file which
you load with your CD burning software if you want to burn a backup of the
disc. Please remember: Before burning it you have to decompress the audio with
perfect-flac-decode. If you don't do this, burning the 'cue' will not work.

"Paul Weller - 1994 - Wild Wood.sha56"
This contains a so-called checksum of the original, uncompressed disc image. If
you want to burn the disc to a CD, you will have to decompress the FLAC files
to a WAV image with perfect-flac-decode. The checksum file allows
perfect-flac-decode to validate that the decompression did not produce any

"Paul Weller - 1994 - Wild Wood.log"
This file is a 'proof of quality'. It contains the output of Exact Audio Copy
and perfect-flac-encode as well as their versions. It allows you to see the
conditions of the copying. You can check it to see that there were no errors.
Further, if someone finds bugs in the future in certain versions of the
involved software you will be able to check whether your audio files are

perfect-flac-encode: http://leo.bogert.de/perfect-flac-encode
perfect-flac-decode: http://leo.bogert.de/perfect-flac-decode
FLAC: http://flac.sourceforge.net/
Exact Audio Copy: http://www.exactaudiocopy.de/
Go to the top of the page
+Quote Post
post Nov 10 2012, 05:39
Post #13

Group: Members
Posts: 44
Joined: 27-October 12
Member No.: 104130

BETA11 available. Changelog:
  • New feature: Use shntool len to check for any of the following problems with the input WAV:
    * Data is not CD quality
    * Data is not cut on a sector boundary
    * Data is too short to be burned
    * Header is not canonical
    * Contains extra RIFF chunks
    * Contains an ID3v2 header
    * Audio data is not block‐aligned
    * Header is inconsistent about data size and/or file size
    * File is truncated
    * File has junk appended to it
  • New feature: Tell shntool to replace Windows reserved filename characters with "_". We recommend MusicBrainz Picard for tagging and it does it the same way.
  • Remove TODOs which suggestt he FLAC parameter "--keep-foreign-metadata". The script's own checksum testing shows that it is not needed for being able to restore an identical WAV image. Further, Hydrogenaudio folks say that it is not needed as well: http://www.hydrogenaudio.org/forums/index....showtopic=97781
  • Not all internal changes are listed in the changelog anymore. See the commit log of github if you are interested.
Go to the top of the page
+Quote Post
post May 30 2013, 18:43
Post #14

Group: Members
Posts: 44
Joined: 27-October 12
Member No.: 104130

The beta phase is over and the first and second stable version have been released!

Version 1:
This is the changelog of the first stable version of perfect-flac-encode.
It has been tested with over 100 audio CD images and was able to encode them all without any errors.
The unit tests have been run for 10 discs and all behaved as expected.
Since BETA11, there have been 209 commits which is quite a lot.
There has been a HUGE improvement in code quality and a lot of bugfixes. This was possible because I made a lot of effort to learn about common Bash pitfalls (special thanks goes to Greg's wiki for the pitfall list: http://mywiki.wooledge.org/BashPitfalls). In other words: BETA11 should not be used anymore, it has a lot of dangerous misuses of Bash.
What follows is the list of externally visible changes. Internal changes which are not interesting to non-developers have been excluded. Bugfixes also have been excluded, just try if it works now.

  • New feature: Copy a specification of the settings which were used with EAC to the output directory as "Exact Audio Copy Settings.txt". This requires a new input file "(release name).txt". You should obtain it from my settings-exact-audio-copy repository. Further instructions are in README.md.
  • Update README.md - this is the README which github displays on the main page of the repository. It gives a detailled overview of the input and output of the script and you should read it if you want to get an idea of what perfect-flac-encode produces in addition to FLAC files.
  • Make output directory selectable by the user instead of writing the output to the input directory. It now is a mandatory parameter. You don't need any write access to the input directory anymore. So if you don't trust PFE, make it write-protected.
  • Write all non-audio files except README.txt into a subdirectory "Proof of Quality" of the output directory.
  • Implement writing of a log file "Perfect-FLAC-Encode.log". I decided against appending to the EAC LOG. We shouldn't mess with well-known file formats.
  • Improve version output on the terminal and in ENCODEDBY tag: Add version numbers of all used tools which touch the audio data (currently accuraterip-checksum, cuetools, eac-crc, flac and shntool). If bugs are found in the tools, you can check whether your audio files are affected by examining the version numbers. Notice that this requires you to have the latest version of accuraterip-checksum/eac-crc because the previous versions did not support printing the version number. Also document the advantage of storing the version numbers in README.md and in the README.txt which perfect-flac-encode generates in the output directory. Remove the version numbers from README.txt. This file shall be easy to understand by laymen and we already store versions in ENCODEDBY and in the future perfect-flac-encode.log
  • Clean up usage of stdout, stderr and the log file: All critical error messages should go to stderr and the log file now. You can safely ignore stdout now and only watch stderr if you want to. The purpose of the log is also documented in the generated README.txt and in the github README.md
  • Do not put the track title into the filename of the singletracks anymore. EAC will use the local charset for track titles in the CUE instead of UTF-8 and it is difficult detect which charset it is. This would break bash processing of the filenames under certain conditions. Also, the metadata which EAC puts into the CUE is imprecise: It does not provide a proper way of selecting releases from MusicBrainz when using the FreeDB gateway. So the filenames would have to be changed when tagging with Picard anyway.
  • Use "Exact Audio Copy.cue", "Exact Audio Copy.log" and "Exact Audio Copy.sha256" as filenames instead of "(release name).cue/log/sha256". The name of the release is usually imprecise when the image is ripped because EAC does not allow proper selection of releases from MusicBrainz. Further, using "Exact Audio Copy" as basename will make the user less likely to be confused about what the files are for.
  • Do not abort if there is no CATALOG in the cuesheet: EAC sometimes produces cuesheets without a catalog. Probably there are discs which do not contain a CATALOG.
  • Update required accuraterip-checksum and eac-crc version
  • Resolve TODO of proof reading the FLAC encoding parameters. My soundman is ok with them as well.
  • Resolve TODO of asking my soundman to review the "shntool split" sytnax. He is ok with it.
  • Change expected location of accuraterip-checksum/eac-crc to be $PATH instead of the home directory. This clearly is the standard Unix way of doing it and I don't know why I didn't do this right from the beginning on.
  • Move all TODOs from the shell script to a file "TODO.txt". I don't want to use the Github bugtracker because it provides no official way of exporting issues and I don't want Github to be a single point of failure.
  • Check whether an unit test was enabled at the very end of the script and abort with "Unit test failure". The program should abort if any errors happen so reaching the end of it with unit tests enabled shows that they failed. Also document this.
  • I've run the script through ShellCheck and it didn't find any serious issues.
  • Resolve all TODOs with regards to Cuesheets: I mailed the EAC developer and obtained a specification of the EAC CUE format from him.
  • Print full output directory when asking the user to delete it.

Version 2:
  • Fix and improve hidden track one audio (HTOA) handling:
    * Version 0001 accidentally wrote "Hidden track one audio not found" into the log file even when there was HTOA. This did not have any negative side effects, it is merely cosmetic.
    * Unit test fix: If testing on data with HTOA, do not use track 0 as test track for checking whether the AccurateRip checksum validation works. Track 0 does not have an AccurateRip checksum.
    * New feature: Only keep hidden track one audio as a separate track if it is longer than a second. I have tested perfect-flac-encode with over 200 disc images. When tagging the resulting FLAC files, I noticed that VERY MANY discs (over 50) contain HTOA. It is usually shorter than a second. Therefore, it is more likely to be a mastering error than an actual hidden track. Typically, those sub-second tracks do not appear as a separate track in MusicBrainz and would make tagging very difficult. So as a workaround we merge them with track 1 now.
    * New feature: Alert the user of HTOA on stdout, not only in the log file.
    * New feature: Include the length of the HTOA in the message on stdout/in the log file.
  • Paranoia improvement: Generate checksums of original WAV image and split singletracks as early as possible. In the previous version, they were generated AFTER feeding the WAV files as input to other programs (shntool / flac), now we generate them before. This guards against the very unlikely case of shntool/flac damaging their input files.
Go to the top of the page
+Quote Post

Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:


RSS Lo-Fi Version Time is now: 16th April 2014 - 09:23