I have been using Monkey's audio for almost a year. I am obsessed with sound quality and use the highest settings in EAC with AccurateRip to rip all my CDs. Recently, I was furious to find out that the f****** Monkey's Audio does not provide any error correction/detection. This means that all my hard work could have been going down the drain. Could anyone please tell me what APE encoder/decoder can I use instead of that piece of shit Monkey's Audio? I read about WavPak, but am not sure how it works. I assume it encodes APE, but I have been unable to install Seek's frontend. Please recommend me something ASAP.

There is no 'quality' settings in lossless and error correction has nothing to do with this either. Any monkey encoder is as lossless as any wavpack or flac encoder.

Anything that encodes/decodes APE will no doubt be using the Monkey's Audio SDK, although it sounds like foobar uses adapted code, so it is possible it is a better decoder than MAC.EXE

WavPack creates WV file - completely different format.

You should do some reading in the wiki. Follow the link through to the comparison page and make your decision.

It is always possible that you could stick with APE and use alternative/additional error recovery, like PAR2.

APE writes an MD5 of the source audio data, it is upto the program you are encoding to, to decode the audio file and check it was written without error.

Excelsius... you're wrong in several things :

First, MAC does have some form of error detection, it was discussed recently here. It just seems that it is dependant on the decoder, and of course, detection doesn't mean correction.

Next, MAC is open source (since only around 2~3 years IIRC), but afaik, there are no other variants that encode to .APE, so it's the only software to create them (not counting the commandline and library interfaces).

Wavpack is a good lossless codec. It is not as fast as MAC but it offers several other features that make it good, as well as having an active development.

About frontends, i can't suggest any, since i don't use them.



@shadowking : that answer of yours was a bit precipitated, wasn't it?

As far as I know, there are currently no lossless format providing error correction. When an error occurs, decoders can't correct it. But some of them could skip the error; such skipping usually imply the loss of a small musical part.
The amount of loss is bigger with Monkey's Audio (compared to flac or wavpack) and it increases with the strength of the compression (you can loose several seconds with -c5000/insane profile). In some cases, the decoder can't even resync the stream; in other words you can loose the whole part of the file located after the error.

[JAZ]> you can't say that WavPack is not as fast than Monkey's. It doesn't mean anything if you don't explicit the encoding profiles. By defaut WavPack is clearly faster than Monkey's on both encoding and decoding side - but the ratio isn't as good then.

Data integrity on hardware or file system level is not the responsibility of a lossless audio codec IMO.

File organization isn't the responsability of the encoder either, but all of them are providing tagging support. It's called a feature and features are playing an important part in the choice of lossless format

Ahh. That is what I am trying to say. Nothing will save you from crummy hardware. Shouldn't blame the codec as the data is no longer lossless. No codec or any software can correct catastrophic hardware failure - they can report faults, maybe try to correct it but software is only as good as the hardware.

You're right. I was talking for a similar compression ratio.

First of all I want to state that "Excelsius XS" should choose his words with more care.
Monkey's Audio sure is everything but crap! If you don't like what it does, write your own lossless compressor that can reconstruct files from your scratched and stained CDs.
What I really hate are people who cannot even write a "hello world" sample in any programming language but have the audacity to talk of excellent software as a piece of shit. Shame on you!

Wow, I did not know that higher compression equaled more errors, Guru. I have been using MAC's "High" compression, but from now on I will only use the "Fast" mode. So are you saying that flac has better error prevention than APE? If so, I can switch to flac because I have been using it on and off. The problem is that APE has a much better tagging system that works great. I tried the tagging system with flac about a year ago and it sucked.

For all you pro's out there, please give me some information about preserving my files. I have amassed a large classical library of about 8000 files and growing (about 100GB). I am going to keep this library forever and want to know what can I do to preserve the integrity of the files. As I said, I use EAC to rip to .wav and then use MAC to convert the .wav to .ape. The thought that I could copy a perfect file from EAC and then lose the integrity just because I encoded it by MAC is horrible. If there are no "lossless" programs that have complete error correction, then maybe we should stop using that damn word "lossless." In my vocabulary, it can either be completely lossless or lossy. How can you dare calling something lossless even if you are losing a fraction of the data from that given file? Amazing.
And what would you recommend about the hardware? I read that some people suggest making a back up on a dvd, but I don't agree. While the disks of hard drive are almost hermetically sealed in a metal case, a dvd is in the open and a single scratch on the label side will cause permanent data loss. Even if the hard drive dies, you can pay someone to take out your data. It might be expensive (up to $1000), but at least you know that you could get your data back, especially if the data is worth over 100K.

Following your definition even copying of a text file from for instance a harddisk to an usb stick is lossy, because someone could break the latter into pieces...

Amongst so many others (), how on earth did you arrive at this conclusion?

I don't have to choose my words with care and it is not up to you to say what people can and cannot say in this forum. Apparently you're the only one "offended" by something I said that is not even offensive. Maybe I can't write a program, but when a program is called "lossless" I expect no data loss. You probably wouldn't care anyway, but I have spent almost four years painstakingly building a large library and now I feel betrayed and some of my work is trashed because someone decided to call their program "lossless" without making all the specifications clear.

And by the way, if I thought the same way you did, then I would have to say that over 90% of the worlds population, including you, are idiots because they have no knowledge of basic physics, but I don't think that way because I know that everyone is different and has a right to their own opinion, whether right or wrong.

"lossless" means that de decoded audio is bit-identical to the original wave file, not more and not less. If the carrier of encoded data gets some (some!!) corrupt sectors that have not been automatically moved to the hotspare area of the disk (if it was stored on a hard disk), maybe some formats will be able to correct more loss than APE. But my experience is: one corrupt sector does never come alone!
Keeping your archive forever means:
# read all data carriers every few years, check for errors, copy those with errors to new ones.
# move your archive from time to time to an up to date storage medium if you don't want to fumble around with obsolete hardware

"lossless" never ever meant the ability to correct errors introduced either by hardware failures nor by media failures. "Complete error correction" belongs to the realm of fairy tales, not to reality because it is simply impossible!

Even if complete error correction is impossible, at least it seems that flac is better than APE in terms of errors. I really like APE, but given that it has more potential for errors than flac I have no choice but to switch. Maybe we need new software that uses ape or flac encoders but also incorporates an elaborate error correction, much like EAC.

If your files are being corrupted then you have a serious problem in your hardware. Check the RAM and always test the files after compressing.

Again, how is it that you arrived at this erroneous conclusion?

My hardware is excellent. I have a custom brand new computer. I am not saying that my files are getting corrupted, I'm say that I MIGTH be getting little errors every time I compress and decompress my ape files since MAC does not detect errors.
Regarding the errors in APE I read about it in wikipedia where is compares all the lossless software and basically says that MAC has no error detection. Guruboolez also confirmed that you get less errors with flac and wavpak just a few posts above.

What EAC does is completely different to what occurs on corrupt data media.
Audio data is stored with a very weak error correction. in case of more serious damage the CD reader has to guess some samples!
The error correction implemented in filesystems can correct some errors - or it discards further reading with an error message. Normally the erroneous data won't even get to your lossless decoder of choice, unless you use something like "unstoppable copier" to read a CD. This will fill up unreadable sectors with zeros.
What you (not "we", at least not "I") need is some level of RAID system that distributes all data plus correction information over several disks so that 1 disk may get lost.



Again, error detection is done by the filesystem.

No, he didn't.

I hate to break it to you, but you're terribly mistaken about most everything you've said so far.

What guruboolez is trying to tell you is that *IF* your ape files happen to get corrupted, you'll not be able to get as much data back from them as you would *IF* your flac files happen to encounter the same corruption.

This does not mean that Monkey's Audio introduces errors while encoding.

Error correction in EAC and error correction as it relates to data are two totally different things. It is pointless to even compare them. BTW, despite what is shown in the EAC GUI, EAC performs no error correction.

I see no such mention about lack of error detection in wikipedia. It does talk about error robustness which is not the same thing (and what it says isn't exactly correct, BTW); otherwise please provide a link to support your claim.

I also suggest you acquaint yourself with TOS #2. You've violated this twice already.

From Monkey's Audio FAQ:



I agree that 32 bit CRC is not the best error detection money can buy, but the chance that only one frame is randomly changed and produces the same CRC32 is negligible to me.

Yeah, that's up to the TOS:
http://www.hydrogenaudio.org/forums/index.php?showtopic=3974

So please go read it.

I read it. I haven't violated anything, including #2, which is a very subjective rule. If a member claims that my post does not conform to his/her aesthetic standards, I can claim the same thing about that member. If you don't mind, I'd like to get back to the discussion...

greynol, what you say about APE makes it seem better, but it's still a problem that you can't get as much out of a corrupted APE than from FLAC. The word "Corruption" seems to be an ambiguous. It seems that if you define it scrupulously enough, it would mean that files in a computer get "corrupted" all the time and APE file, or any file for that matter, will eventually become completely useless because every little "unnoticeable" problem will add up and as time goes to infinity the corruption function will approach infinity as well. Is there anyone in here who knows what "corruption" really means? Is there a mathematical function to describe it? Considering that computers are based on 0's and 1's (linear algebra) I would think that there COULD be a corruption formula. That's intriguing and could elucidate a lot of aspects of this ambiguous discussion.

I smell a troll rather than an idiot here.

Excelsius XS,

Check it out, I'll lead you to the water, but it's up to you if you want to take a drink.

If you don't want to use Monkey's Audio because it can't recover as much data as flac or Wav Pack and because you are paranoid, that's your business. But don't think you're going to get away with trashing a perfectly good piece of software just because you are ignorant.

You're right, I'm the ignorant one. MAC can't recover the data, but it's "PERFECTLY GOOD." That about says it. By the way, you just violated TOS#2.

...nor can flac or Wav Pack recover corrupted data. I guess by your reasoning they aren't perfectly good either?

I suggest you read over this thread a little bit harder.

Perhaps instead of saying that you are ignorant I should have said that you don't know what you're talking about; it is painfully obvious.

Again, there are no lossless format which (currently) offers any possibility to restore the corrupted datas.
I'm not an expert but I guess that such feature would necessary take some space and therefore decrease the compressing ratio.

If you're so anxious about corruption (which should only appear on rare situation) I suggest you to go with dedicated solution such as PAR2.


edit> greynol was faster.

Maybe we should make a thread or a poll to ask some of the people who are really good at programming to develop or perfect the lossless front. Are there any programmers in here? I am sure that many people would sacrifice compression to have a more error-free file quality. Since storage becomes cheaper and cheaper, maybe it would be best to just store the files in wav format. I know that currently the industry is trying hard to popularize SACD and DVD-audio, which will have a much better quality that current CDs. Just the fact that a vinyl has a much higher quality than any CD or DVD-audio says a lot about how far back we're in terms of audio technology.

How would you insure that your waves don't get corrupted?

I still don't think you're grasping the concept here.
EDIT: Now I *know* you still aren't grasping the concept here.

Uh-oh! Now he's just violated TOS #8.

seanyseansean, I'm beginning to think you're right.

Well, i wasn't sure but...


"MAC can't handle errors well" has been a good topic for trolling and here is another one.

If he is a troll, he isn't very good or entertaining.

greynol, enough is enough. If you make one more post that is similar to what you've been saying I will see that the moderators put you back where you belong, since it seems they have been quick to respond to me.

I'll answer you this time: you cannot be 100% that any file remains pristine, but at least if you save the file in wav and lose a fraction, you won't lose as much as you would if you had an APE. APE is compressed and you would loss more data for the equivalent sized loss.
By quoting TOS#8 you just slandered me because you have no idea what i am talking about. It is perhaps beyond your comprehension, but vinyl indeed has a better quality. In simple terms, the reason is that digital audio approximates the waves in a quantized manner, whereas vinyl is engraved with the exact waveform of the music (it is not approximated).

Wrong, wrong, wrong. That waveform claim has been refuted so many times it's not even funny anymore.

Store your online files on a RAID-5 array, and your offline files on optical media enhanced with PAR2 files or dvddisaster correction, and then use whichever lossless codec you want. And stop worrying about the codecs.

-brendan

The easiest way to avoid errors is to stop using files.

From your posts it seems that there is no simple solution. Oh well. I'll try to come back and address this issue in a forum next year. By that time perhaps we'll all be using SACDs or DVD-audios and audio file storage will be perfected. I'll see if using PAR2 will not be time consuming because this is the first time I am hearing about it.

And Firon, I'm not going to repeat this again. I don't care what you have heard or what you have thought you have heard (or read what some kid has posted). CDs throw out a lot of the data that vinyls have. I have seen the data and the graphs myself. I have a friend who has been making home stereo and amplifiers for 37 years and he knows that too. There was also an editorial in New York Times about this. I don't what your opinion is, but do not misinform the people who come to get some information from this forum.

Many, many audio engineers (such as the creator of MP3 and AAC) who know far more than most of us have proven that your previous claim is essentially junk. If you don't want to believe that, suit yourself.
YOU are the one misinforming people here.

Oh, and if you want to test accuracy on a vinyl, try recording a sine wave on it and look at the output with an oscilloscope. Then do the same with a CD.
Let's not forget the harmonic distortions and the fact that there is an analog noise floor (which is not very low) with limits the "resolution" on vinyl.

Back to keeping your files safe, creation of PAR2 files is slow, but it is very reliable and very useful. With good quality media and taking care of your discs (ie keep them in dark places and in cases), the PAR2s should be more than sufficient for making sure your data is fine in the long term (sans fire or destruction)

You are still seriously missing the point. Lossless codecs do compress WAVE files and create no errors. However, post-encoding errors can occur (Edit: I refer to disc scratchs and bad sectors on hard drives here - corruption that can happen to any file type and are unrelated to the encoding process). Some formats, such as WavPack and FLAC, may be better at decoding through those errors (people's views seem to vary). No codec can recover those errors.

As I, and someone else, have already suggested, I would look at PAR2 parity data if you want some help recovering corrupt data completely.

You need to start listening, and doing some reading. This thread is close to being closed for trolling.

I know I've pretty much given up on it, but not before supplying a link:
http://www.quickpar.org.uk/

Synthetic, when I said that I indeed did not understand at what point the errors were introduced. I already know what you said. And storing my files on 35 DVDs is not my idea of a safe back up. I have been trying hard to keep my CDs scratch free, but I haven't been able to do that. If I accidentally drop the DVDs or if there is an earthquake,the label side will get damaged and I'll have a pile of plastic garbage. Raid-5 is a temporary and expensive solution for now that I might try.
I don't know what do you mean by trolling, but this thread is over anyway. Thanks to everyone who gave me some useful information or leads.

Firon, if you don't mind please provide the links to the proof that you have found. I am open minded as long as there is scientific evidence to back it up. MP3 was invented by about 5 guys at the Fraunhofer Institute in Germany. If you have a link that shows that the institute says that CDs have better quality than vinyls I will be surprised.

Lets recap:

Monkeys has 2 levels of error detection, a CRC for the smaller 'frames' and an MD5 which can be compared against the whole decoded audio data (very few programs will do this, probabbly because the crc32 will catch 99.9999999% of errors).

When compared to FLAC and wv Monkeys will sometimes (depending on compression mode) recover less data than the others, if the file has become corrupted.

No current audio codec, lossless or wav will recover and stay lossless if there are errors in the datastream.

Invest in a firesafe, removable hard disk and DVD-Rs (ie two types of backup) and you will be safe from earthquakes, fire and user missuse. Better still upload your audio to one of hte online datastorage backup places.

For someone without serious knowledge you sound very arrogant.


what data did you see there? was ist the difference between CD and vinyl?
Of course they are not equal because vinyl players introduce some wow&flutter, harmonic distortions, non harmonic distortions, groove noise, tracking angle errors, crackles, pops and so on.
CD throws them all away, that's right

Argh, I can't believe I'm even posting in this rediculous thread of misinformations!!!

As said before, it's not up to a *file format* to provide *error correction*. Although most filesystems DO have error correction/detection, and it would be very rare that files gets corrupted without an internal CRC error shows up.

Have you never heard of backup? There are many ways you can securely backup the data. Say, CD/DVD's doesn't last forever, but they last long enough if you keep them safe! But keeping them on such a media, I would definately use some kind of error correction data - I've seen projects before, that stores x% recovery data on media, or you could just duplicate every backed up DVD.

A combination of both an offline harddrive and backed up DVD's could be the best solution though.

AND FOR CHRIST SAKE STOP YELLING ABOUT INTEGRITY ON SPECIFIC FORMATS! IT'S POINT/USELESS IN THIS CASE!!!

...and vinyl is crap

http://www.skepticforum.com/viewtopic.php?t=85
Reading into that thread, there's references to books on basic sampling theory and so on that will explain everything (if you don't believe what Mr. Johnston says). And he is one of the co-inventors of MP3 and MPEG-2 AAC.
There's more I could probably dig up, but I'm not really going to bother at 6am.

I'm afraid no amount of links can help in this case...

Sorry but you don't have the faintest idea of what you are talking about. Your waveformat claim is a very common misconception of people who are not familiar with digital sampling theory and the nyquist-shannon sampling theorem.

You can reconstruct the original wave if sampled at least twice the highest frequency. The typical hearing range for humans goes from 20Hz to 20kHz. CDs are sampled at 44kHz which makes possible to reconstruct waves up to 22kHz above the hearing threshold so you do not lose anything from the waveformat.
Please don't argue against this because otherwise everyone will be sure you are a troll.

Are you really sure?
Double checking with wikipedia, GPFS and ZFS only provide Error Correction.

Never seen someone so paranoid about data integrity that he would question the format of the data, and I thought A removeable hard drive in a cupboard would be enough for the average joe.

Few filesystems actually provide error correction, because we are again looking at the wrong place. Error correction should be media dependent to be able to correct typical errors for that media, so in reality (on all drives you could remotely call modern) error correction for hard-disks is handled by the disk/controller internally, independent of the file system.

If a sector is read with wrong checksum, the drive uses ECC data to try to correct it and the sector is "reallocated" so that all further requests to it are redirected to a spare "healthy" sector. All this generally happens without any intervention or knowlege from the operating system, but you can use a SMART monitor to find out how many sectors your drive has already reallocated.

Unfortunately there is not much you can do to get better protection on that layer, so adding an ECC feature to filesystems does make sense.