Help - Search - Members - Calendar
Full Version: Hard discs: no error correction required?
Hydrogenaudio Forums > Hydrogenaudio Forum > Scientific Discussion
hollowman
On p.31 of the book Mastering Audio: The Art and the Science (by Bob Katz; 2002 Focal Press; ISBN: 0240805453), the author notes: "Hard discs [...] generally do not require error-correction, since their error rates are extremely small." The book is geared toward the recording industry. How true is this comment for playback as well as recording?

What about error rates for flash-based memory devices? There is some "consensus" that, e.g., hard-drive-based PDAPs (portable digital-audio players) sound better than flash-based PDAPs. But I'm not sure the correlation/causation for Katz's argument, in its context, is relative here.

On a somewhat-related topic regarding PC-based audio, a friend and fellow audiophile/DIYers noted:

"What makes the huge difference is that by using the PC for volume control, I was able to remove the preamp completely, and that is where the CD player [i.e., external/stand-alone unit, as in a typical home-audio system, and not a PC-based ROM drive] loses the race. It takes a few tricks to do volume control on the PC right, but once you do, you have no bit loss at the lower volume settings." See his system as well as more technical remarks here:

http://didnt.doit.wisc.edu/audio/DDDAC16/d...16usb_ASIO.html
http://didnt.doit.wisc.edu/audio/DDDAC16/dddac16usb.html

So, in conclusion:

Perhaps if media companies managed the DRM "issue(s)" better, and allowed consumers direct purchase of the digital master's "source code", we would all have some really high-quality stuff.
Firon
Hard drives do still require error correction, and all of them have it.
legg
Woah, Hard disks, error correction, volume control on the PC and DRM in one thread. tongue.gif


Don't know if HDDs have error correction per se, but they do use coding schemes to store the data.
spoon
Where to begin?...

>What about error rates for flash-based memory devices? There is some "consensus" that, e.g., hard-drive
>-based PDAPs (portable digital-audio players) sound better than flash-based PDAPs. But I'm not sure the
> correlation/causation for Katz's argument, in its context, is relative here.

Many top end PCs / Laptops use just flash memory for speed, no hard disk, by your analogy they must sound worse than hard disk based pcs, makes no difference. If a system is working correctly - be that an iPod with Hard disc or iPod nano with flash, the audio data from the hard disk or flash is identical, errors only happen when something is not working correctly, for flash and hard disks that should be a rare occurance. Audio components close to a hard disc might suffer, as there is a metalic object spinning around.

>Hard discs [...] generally do not require error-correction, since their error rates are extremely small

When you format a hard disk, you are setting up a frame work to detect errors. It could be that some pro audio equipment formats a disk with little error correction, apart from a small % extra space, nothing is gained.
Mike Giacomelli
QUOTE(hollowman @ Mar 29 2007, 23:32) *


What about error rates for flash-based memory devices? There is some "consensus" that, e.g., hard-drive-based PDAPs (portable digital-audio players) sound better than flash-based PDAPs. But I'm not sure the correlation/causation for Katz's argument, in its context, is relative here.


There is no difference between the two. Anyone who tells you that flash or disk choice has any impact on quality has no idea what they're talking about.

QUOTE(hollowman @ Mar 29 2007, 23:32) *

"What makes the huge difference is that by using the PC for volume control, I was able to remove the preamp completely, and that is where the CD player [i.e., external/stand-alone unit, as in a typical home-audio system, and not a PC-based ROM drive] loses the race. It takes a few tricks to do volume control on the PC right, but once you do, you have no bit loss at the lower volume settings."


Hes just using the amp in the sound card instead of an external amplifier. I don't see how this is an advantage or disadvantage for the PC. Actually, that whole point doesn't make much sense at all.

And looking at his page, he seems to believe all that kmixer nonsense. Was this the same guy who told you about flash and HD's?
cabbagerat
QUOTE(hollowman @ Mar 29 2007, 22:32) *

: "Hard discs [...] generally do not require error-correction, since their error rates are extremely small." The book is geared toward the recording industry. How true is this comment for playback as well as recording?

Hard disks have very robust built in error correction and coding mechanisms. The chances of a hard drive reporting an incorrect value for a read bit (instead of not reading it at all) are extremely, extremely small. It happens, and upsets scientific computing people, but it doesn't happen often enough for it to have any effect what so ever on audio quality.

QUOTE(hollowman @ Mar 29 2007, 22:32) *

What about error rates for flash-based memory devices? There is some "consensus" that, e.g., hard-drive-based PDAPs (portable digital-audio players) sound better than flash-based PDAPs. But I'm not sure the correlation/causation for Katz's argument, in its context, is relative here.
Hmm, no. If anything, flash based players should have cleaner power, and better sound quality - all other things being equal. If you mean "expensive" by "hard drive" and "cheap and cheerful" by "flash-based" then, yeah, flash based players are likely to suck, comparitively.
hollowman
QUOTE(cabbagerat @ Mar 30 2007, 13:43) *

QUOTE(hollowman @ Mar 29 2007, 22:32) *

What about error rates for flash-based memory devices? There is some "consensus" that, e.g., hard-drive-based PDAPs (portable digital-audio players) sound better than flash-based PDAPs. But I'm not sure the correlation/causation for Katz's argument, in its context, is relative here.
Hmm, no. If anything, flash based players should have cleaner power, and better sound quality - all other things being equal. If you mean "expensive" by "hard drive" and "cheap and cheerful" by "flash-based" then, yeah, flash based players are likely to suck, comparitively.


Yes: the latest flash-based memory should (theoretically) sound better. The reasons I think HD-based units may beat out flash players are as follows:

- HD players are (and have "traditionally" been) incorporated into any particular manuf's top-end model (perhaps because of extra capacity). The top-end models may get extra "topological attention" WRT audio circuit design. For example: because HD-based PDAPs are usually larger, perhaps the designer can use (fit in) discrete electronic components -- which typically sound better -- rather than integrated circuits. I think this is along the line with your comment re "expensive" vs. "cheap".

- HD designs are still superior to flash (but the latter may soon catch up and supersede)
hollowman
QUOTE(spoon @ Mar 30 2007, 13:21) *

>Hard discs [...] generally do not require error-correction, since their error rates are extremely small

When you format a hard disk, you are setting up a frame work to detect errors. It could be that some pro audio equipment formats a disk with little error correction, apart from a small % extra space, nothing is gained.


I assume you mean generic data EC (error-correction) -- via the OS and effective for all data I/O'ing thru the HD -- and not "audio-specific" EC, such as Reed–Solomon.

While I'm sure OS-based EC is quite robust, effective and efficient for most computer apps and processes, I'm wondering whether an ad hoc (audio- or music-specific) EC approach wouldn't work better.
Light-Fire
QUOTE(hollowman @ Mar 30 2007, 01:32) *

...There is some "consensus" that, e.g., hard-drive-based PDAPs (portable digital-audio players) sound better than flash-based PDAPs...


That makes no sense at all!!!

You should do some research on Boolean Algebra and, then, you will understand.
hollowman
QUOTE(Light-Fire @ Mar 30 2007, 18:55) *


You should do some research on Boolean Algebra and, then, you will understand.


Boolean Algebra is anitiquated terminology. Not sure how that term came to be applied in the first place but it is more-correctly know as Boolean Arithmetic:

http://en.wikibooks.org/wiki/Practical_Ele...lean_Arithmetic
Woodinville
QUOTE(Firon @ Mar 30 2007, 01:44) *

Hard drives do still require error correction, and all of them have it.



Well, yes, it's built into the hard drive, though. It's not generally necessary to include any kind of error protection in the stored file.

Just back everything up.
hollowman
QUOTE(Woodinville @ Mar 30 2007, 21:52) *

QUOTE(Firon @ Mar 30 2007, 01:44) *

Hard drives do still require error correction, and all of them have it.



Well, yes, it's built into the hard drive, though. It's not generally necessary to include any kind of error protection in the stored file.


Do you mean in the firmware of the HD drive (in some native chip)? This makes sense on a meta level. I think the OS also does some error checking as the data I/O's the HD.
Synthetic Soul
There are certain "Consumer Electronics" hard drives, used in PVR machines, that have little or no error correction to ensure smoother recording and playback.

http://www.xtendedplay.co.uk/faq.php#8

Edit: Segate CE drives have the suffix "ACE" or "SCE" in the model number.

http://www.seagate.com/www/en-us/products/...ics/db35_series


hollowman
QUOTE(Synthetic Soul @ Mar 30 2007, 23:29) *

There are certain "Consumer Electronics" hard drives, used in PVR machines, that have little or no error correction to ensure smoother recording and playback.

http://www.xtendedplay.co.uk/faq.php#8

Edit: Segate CE drives have the suffix "ACE" or "SCE" in the model number.

http://www.seagate.com/www/en-us/products/...ics/db35_series


Thanks for replying with references! I am partial to Seagate drives, due to personal experience with them, so I may experiment with the DB35 Series. A PDF (White paper) link on the page you ref'd above goes into some more details:

http://www.seagate.com/content/pdf/whitepa...45_DynaPlay.pdf

Specifically, the WP notes:

QUOTE
Improve stream integrity through time-limited commands

Uninterrupted, smooth video delivery is a fundamental customer expectation for any application recording television in the home. Hard drives designed for PC applications are optimized for data integrity through enhanced error detection and correction routines. Consumer video recording devices, however, need to be optimized for stream integrity to avoid problems caused by over-execution of error checking, which can cause problems in consumer video recording devices. The Seagate DB35 series drives offer time-limited commands, which allow manufacturers to adjust error detection and correction in favor of a consistent, smooth video stream. In the rare instance that an error in the video data is detected, the consumer would likely experience a virtually imperceptible drop of a video frame, but would continue to enjoy an uninterrupted stream of entertainment content.
Manufacturers can adjust time-limited commands to preferred specifications, using standard commands to define the drives’ timeout performance in error checking. Alternatively, Seagate can configure the drives’ time-limited commands settings before the drives ship. Seagate DB35 Series hard drives with time-limited commands capability support a wide range of OEM middleware, operating systems, motherboards and design standards.
mixminus1
Found this site that gives at least some basic descriptions of the error correction used in hard drives.

It seems to be pretty similar to CD's: Reed-Solomon error correction code is written with each sector by the hard drive firmware, and then read and computed on a read operation, and used for correction if necessary, again at the hard drive level. Multiple reads (a la EAC wink.gif ) are done if needed, and if all goes well, the OS is never aware that any correction was necessary.
Light-Fire
Error detection and correction is done by OS. Your OS will detect a corrupt file that it can't correct and let you know about it. Then you replace it from the back-up files you (wisely) kept stored somewhere else.
Mike Giacomelli
QUOTE(hollowman @ Mar 30 2007, 16:07) *

QUOTE(cabbagerat @ Mar 30 2007, 13:43) *

QUOTE(hollowman @ Mar 29 2007, 22:32) *

What about error rates for flash-based memory devices? There is some "consensus" that, e.g., hard-drive-based PDAPs (portable digital-audio players) sound better than flash-based PDAPs. But I'm not sure the correlation/causation for Katz's argument, in its context, is relative here.
Hmm, no. If anything, flash based players should have cleaner power, and better sound quality - all other things being equal. If you mean "expensive" by "hard drive" and "cheap and cheerful" by "flash-based" then, yeah, flash based players are likely to suck, comparitively.


Yes: the latest flash-based memory should (theoretically) sound better.


What the heck? No they shouldn't. Both will sound the same until one of them fails, at which point it will most likely not sound like much at all.

QUOTE(hollowman @ Mar 30 2007, 16:07) *

Boolean Algebra is anitiquated terminology. Not sure how that term came to be applied in the first place but it is more-correctly know as Boolean Arithmetic:

http://en.wikibooks.org/wiki/Practical_Ele...lean_Arithmetic


That link calls it boolean algebra. Regardless, the point of bringing it up was to subtly remind you that we're talking about digital media.
hollowman
QUOTE(Mike Giacomelli @ Mar 31 2007, 22:11) *

QUOTE(hollowman @ Mar 30 2007, 16:07) *

QUOTE(cabbagerat @ Mar 30 2007, 13:43) *

QUOTE(hollowman @ Mar 29 2007, 22:32) *

What about error rates for flash-based memory devices? There is some "consensus" that, e.g., hard-drive-based PDAPs (portable digital-audio players) sound better than flash-based PDAPs. But I'm not sure the correlation/causation for Katz's argument, in its context, is relative here.
Hmm, no. If anything, flash based players should have cleaner power, and better sound quality - all other things being equal. If you mean "expensive" by "hard drive" and "cheap and cheerful" by "flash-based" then, yeah, flash based players are likely to suck, comparitively.


Yes: the latest flash-based memory should (theoretically) sound better.


What the heck? No they shouldn't. Both will sound the same until one of them fails, at which point it will most likely not sound like much at all.


Perhaps we should not call them flash but, more prosaically, "solid-state memory devices". In any event, some content from this source is what I had in mind -- as in this excerpt:

"[...] what's the benefit of a high-speed SSD when the entire system cannot support it? The evolution of CPUs in terms of performance has far surpassed the development of the data storage system. The HDD is actually limiting the potential of a computer system when they are not able to keep up with the performance of the other components, particularly the CPU. If an HDD is replaced by a high-speed SSD, the performance would significantly improve. The resulting system would be a high-powered set-up that can go beyond the demands of high-speed business or military environments."

BTW, the sonic difference(s) I am speculating exist between an optimial solid-sate device and an optimal HDD are not as great as, say, the diff. between (all things being equal) a Red Book CD and a DVD-Audio disc.
Ken S
QUOTE(hollowman @ Apr 1 2007, 11:01) *

BTW, the sonic difference(s) I am speculating exist between an optimial solid-sate device and an optimal are not as great as, say, the diff. between (all things being equal) a Red Book CD and a DVD-Audio disc.


For me that would be <0 ...

... and all things would be equal. laugh.gif

Can't speak for others though.

Ken

BTW by "optimal" I assume you mean one that works (i.e. returns the stored values, reliably)
Kees de Visser
QUOTE(hollowman @ Apr 1 2007, 11:01) *
BTW, the sonic difference(s) I am speculating exist between an optimal solid-sate device and an optimal HDD are not as great as, say, the diff. between (all things being equal) a Red Book CD and a DVD-Audio disc.
As long as a storage medium is capable of reliably sending all the (audio) bits in time to the DA converter(s), it shouldn't have any influence on the sound.

Red Book CD and DVD-Audio don't contain necessarily the same audio format (samplerate and wordlength) so you can't compare them without being more specific.

Hard drives can be noisy. If solid-state devices don't need noisy cooling, they might be much quieter. That could be an interesting sonic difference. smile.gif
MedO
QUOTE(Light-Fire @ Apr 1 2007, 06:03) *

Error detection and correction is done by OS. Your OS will detect a corrupt file that it can't correct and let you know about it. Then you replace it from the back-up files you (wisely) kept stored somewhere else.


Nope, AFAIK, this is done transparently by the HD/Controller. The OS will only get a "read error" from the controller if the HD fails to read or recover the sector. Defective sectors are silently remapped to "spare" positions on the disk, again without any intervention by the OS, though you can usually get some statistics by reading the HD's SMART data. This is all done below even the filesystem level, which is why you almost never see defective sectors in scandisk on modern drives. Reading the SMART data (Value "Reallocated Sector Count") will give you the real count.

I'm talking about standard consumer drives here. Also, it doesn't apply to old drives AFAIK.
bhoar
QUOTE(Synthetic Soul @ Mar 31 2007, 02:29) *
There are certain "Consumer Electronics" hard drives, used in PVR machines, that have little or no error correction to ensure smoother recording and playback.
http://www.xtendedplay.co.uk/faq.php#8
Edit: Segate CE drives have the suffix "ACE" or "SCE" in the model number.
http://www.seagate.com/www/en-us/products/...ics/db35_series


More specifically, they have just as much error detection and correction *data* stored on the platters as the non-PVR targeted drives.

However, the firmware has been tweaked to prefer PVR-friendly realtime I/O strategy (A) over the more PC-friendly aggressive data-recovery strategy (B):

(A) PVR-friendly - When a read or write error is detected, abort the operation *quickly* and report an error to the caller, as the caller (presumably a streaming data recorder, such as a PVR) is expecting real time I/O capabilities and prefers to handle errors itself as a trade-off for fast response to disk I/O calls.

(B) PC-friendly - When a read or write error is detected, retry the read/write several times and more aggressively try to save marginally corrupted data by moving it to a spare sector. While this can lead to the drive becoming unresponsive for a significantly longer period of time than in (A), the fact that your spreadsheet was read into memory AND the data was recovered from a marginally corrupted sector to a good sector was never noticed by you, which is a good thing. Also, from time to time, run additional I/O-sapping calibration and correction routines so as to ensure the best error-correction capabilities are always available.

Summary: it's better to experience a 2-3 second glitch in playback of streaming data (presumably a stream type that allows quick re-lock onto the stream) than to have the screen freeze and the hard drive stutter for 30 seconds of clacking due to the delay from several re-calibrations trying to read (and perhaps re-map) several bad sectors found in a row...

-brendan
hlloyge
But when dealing with audio recording, IF this happens during recording, there is possibility that when using these "optimised" hard drives we couldn't be sure the reproduction will be flawless, because HDD would, in theory, write the data on possibly corrupted part of a drive, and just keep recording on, and later on there will be problems with reading the same exact data...?
I understand how it works with mpeg video streams, mpeg video has autocorecting algorythms inside the stream which will try to correct the badly read bits... but wavs don't have these, as I remember...
digital
.
I feel that I am qualified to answer this one: I'm a Microsoft certified Tech with a decade and a half of experience in the industry.

1) Hard Disk Drive certainly do have error detection / correction

2) You can delay (any) hard disk drive from implementing it's write-back caching routine, thus data drop / lag, with OS settings. We used to set up a lot of secondary SCSI drives this way to avoid frame-drops in video capture 'back in the day'.

3) If you [really] want to hear whether Hard Disk Drives 'sound different' than solid-state drives, simply copy a .FLAC or .WAV to a USB 2.0 thumb-drive and keep a copy on your hard disk drive. Listen to them both, back and forth (drag and drop into WinAmp), as many times as you want... hear any difference? No, neithor do I...

What an odd discussion thread...

Andrew D.
www.cdnav.com

.
spockep
QUOTE(hlloyge @ Apr 2 2007, 14:53) *

But when dealing with audio recording, IF this happens during recording, there is possibility that when using these "optimised" hard drives we couldn't be sure the reproduction will be flawless, because HDD would, in theory, write the data on possibly corrupted part of a drive, and just keep recording on, and later on there will be problems with reading the same exact data...?
I understand how it works with mpeg video streams, mpeg video has autocorecting algorythms inside the stream which will try to correct the badly read bits... but wavs don't have these, as I remember...



That's why a lot of professionals still use Digital Reels.
Pio2001
Don't professional recorders report the amount of errors after the recording ?
hlloyge
QUOTE(spockep @ May 28 2007, 04:37) *

That's why a lot of professionals still use Digital Reels.


They could as well use RAIDed hard drives, if speed is of importance; but I am 100% certain that same music file from normal hard drive and optimized one sounds the same provided it is decoded and reproduced through the same equipment. Modern HDDs are more than capable enough for reproducing ANY audio format, even uncompressed. Well, maybe not these 7.1 ch 32bit 192kbit streams, but if you need high bandwidth, you use RAIDs. And I seriously doubt that much speed is gained with disabled caching.
Roobar
QUOTE(digital @ May 28 2007, 11:46) *

.

What an odd discussion thread...

.


I have to agree: odd indeed!

As a a IT guy also, I have to say that, for the purposes of this discussion, there really is no practical difference between flash disks and hard disks in terms of quality and reliability of the digital data that they store. IT folks get quite upset if the storage media drops even one bit (as in a binary 1 or 0) of data. That sort of thing can corrupt a bank's database. So any digital media is very good at error correction and recovery.

If you are storing streaming media especially video, then the key issue is the speed at which data can be read from the disk. But in this regard, disks running at 7200 rpm on a high speed (personal computer) data bus can easily deliver data fast enough for most home use purposes. USB 2.0 and Firewire 400 is generally good enough for most streaming purposes at home. Hollywood studios running massive graphics rendering systems have very different requirements of course.

Audio data, including lossless formats like flac, don't have any difficulty with data streaming problems though. If you watch the activity light on any HDD, including USB external drives, you will see that they only just flicker from time to time. Basically, player software reads a chunk of data, stores it in memory, decodes it and passes the data to the DAC chip for us to hear. It then reads another chunk of data, and decodes it. The larger the memory available, the bigger the chunk of data that can be cached, the less often that the HDD needs to be read, and the less the need for high-speed disk storage. In fact, most mp3 tracks can be read from disk in one read operation and the whole track cached for processing. Ipods, in effect, send the HDD to sleep between read operations. This is very different from the way that DVDs and CDs work of course.

If you are recording a/v data, the write operation will be slower depending on speed of data coming from the a/v device (eg a digital video camera or DVD). So, when you plug your camera into your home computer to store the a/v data onto your HDD, you generally want to make sure that you have not other apps running and competing for the HDD or using CPU or memory resources. Same if you are ripping a CD or (especially) a DVD to disk. DVDs store a lot more data than CDs that needs to be processed quickly: so faster CPUs, lots more memory, 7200 rpm HDDs, and USB2.0 and Firewire 400 & 800 have quickly entered the home computer market to cater for this.


Anyway, I hope this this helps. rolleyes.gif
pepoluan
QUOTE(hlloyge @ Apr 3 2007, 01:53) *
But when dealing with audio recording, IF this happens during recording, there is possibility that when using these "optimised" hard drives we couldn't be sure the reproduction will be flawless, because HDD would, in theory, write the data on possibly corrupted part of a drive, and just keep recording on, and later on there will be problems with reading the same exact data...?
Not really. A failing sector on a HD is not a 'catastrophic' (i.e. sudden) event. It degrades over time. During reading, the drive's circuitry will detect the degradation, perform error correction, and move the physical sector to another physical sector on the same track. The OS keeps accessing the same logical sector, unaware that the logical sector has been mapped by the firmware to a different physical sector.

Problem happens if there is no more unused sector on the same track. But before this happens, SMART should indicate that the drive is in dire need of replacement.
odyssey
QUOTE(pepoluan @ May 29 2007, 08:43) *

QUOTE(hlloyge @ Apr 3 2007, 01:53) *
But when dealing with audio recording, IF this happens during recording, there is possibility that when using these "optimised" hard drives we couldn't be sure the reproduction will be flawless, because HDD would, in theory, write the data on possibly corrupted part of a drive, and just keep recording on, and later on there will be problems with reading the same exact data...?
Not really. A failing sector on a HD is not a 'catastrophic' (i.e. sudden) event. It degrades over time. During reading, the drive's circuitry will detect the degradation, perform error correction, and move the physical sector to another physical sector on the same track. The OS keeps accessing the same logical sector, unaware that the logical sector has been mapped by the firmware to a different physical sector.

Problem happens if there is no more unused sector on the same track. But before this happens, SMART should indicate that the drive is in dire need of replacement.

Amen!

This thread should go in Recycle Bin...
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2008 Invision Power Services, Inc.