Nika Aldrich published a very good explanation of dither at http://www.cadenzarecording.com/Dither.html .

The paper was originally posted in the prosoundweb forums, where it can be discussed : http://www.prosoundweb.com/recpit/viewtopic.php?t=1284

Other papers are coming, about Sample Rate Conversion, jitter...
Post your whishes at http://www.prosoundweb.com/recpit/viewtopic.php?t=1299

Images are huge and have bad quality.
Use a lossless encoder for the images like
PNG, it's shorter.

Example (9 KB instead of 150 KB):

[img]b176dc6100[/img]

The page contains several very basic mistakes.
The "noise shaping" explained on this page has nothing to do with noise shaping. You can noise shape without any dither. You don't need a
random number generator or a noise generator for
noise shaping. Noise shaping forms the quantization noise, it do not add additional noise from an external noise source.

Simple Quantization:

X = [x]

Simple Dithering:

X = [x + n]

Colored Dithering:

X = [x + (n,F)]

Noise Shaping:

X = [x + ((X-x),F)]

Dithering + Noise Shaping:

X = [x + ((X-x),F) + (n,F)]

I told him about pictures, he now uses Gif.

GIF is covered by the Unisys patent, and several
web hoster have got trouble from Unisys.

PNG is
- more flexible (paletted, grayscale, color, 1...16 bit + opacity)
- often smaller (50%...95% are often reached)
- patent free
! does not support animated images


edit: shorter -> smaller

Frank,

Thanks for that clarification of noise-shaping.

However, forgive me if I'm at a loss in regards to the "very basic mistakes" in the paper. I must have missed them.

Can you elaborate? Thanks in advance.

When I read the "paper", I thought "here's someone who has just grapsed the basics of dither, and is trying to explain them to to someone who hasn't".

Was it Albert E. who said "Things should be explained as simply and possible, but no simpler" (or something like that)? In this case, I think we've moved into "simpler" terratory!

I wanted to point out some mistakes, but didn't want to look like a smart ass. I'm glad Frank has beaten me to it!



Wrong. TPDF is optimum - Gaussian is sub-optimum. It's close, but TPDF is better AND easier to implement digitally. Gaussian can occurr naturally, so it has been used in 16-bit A>D conversion in the past, and is unnavoidable at 24-bit A>D conversions (i.e. they'll probably be some Gaussian-like noise on the signal).




Distortion is distortion. Noise is noise. Correct dither prevents quantisation distortion, but adds noise. This is the simple trade-off. I think the author realises this and mentions it, but this mangled sentence nearly hides the fact!




Yes, and it will, because any real world signal contains more than enough noise to do this. It's not an extra step you have to (or even can) worry about. A 6 ohm resistor in circuit will give enough noise to dither the last bit of a 24-bit convertor (I'm quoting someone here - I haven't verified it, but it's about right).


Is the Apogee UV22 really dither? Not quite, according to their advertising. It's a high frequency signal that acts like dither.



It looks like a high frequency waveform - it's possible (though not easy) to see that. I bet it's a long way from TPDF




I wish he'd explained this - it's quite interesting. Why adding this wonderful "dither" gives artefacts - is it really a bug in the graph program? I'm very interested (seriously) in what he's found here.


What's not said is that a correctly dithered system gives a theoretically infinite dynamic range (though some of it lies below the noise - this is important because our ears are very good at hearing sounds within noise), zero distortion, and zero noise modulation. Using the wrong dither (e.g. rectangular, or even TPDF where the amplitude isn't exactly right) can cause noise modulation (the noise goes up and down with the signal) and distortion.


However, it's good that someone has tried to explain dither to people who don't understand it in an approachable way that us techies would probably never have thought of (I like the dice bit!).


After reading his article, you could do worse than read:

Any articles on dither by Lipshitz and Vanderkooy - there seem to be none online, but loads in the AES. They have written the definitive papers on this subject.

http://www.mtsu.edu/~dsmitche/rim420/readi...420_Dither.html

http://www.digido.com/ditheressay.html

http://privatewww.essex.ac.uk/~djmrob/mp3decoders/24bit.html
http://privatewww.essex.ac.uk/~djmrob/mp3d...ers/24bit2.html

Cheers,
David.

P.S. for all this talk, to most people the audible effect is tiny.

This is good:

http://www.tonmeister.ca/main/textbook/digital/02.html


This is great (though only touches on dither):

http://www.digital-recordings.com/publ/pubrec.html


I like this:

http://www.abyssmall.com/washingt.htm



you get the idea...

D.

Thanks for the comments, but it would be interesting to discuss it with the author :

http://www.prosoundweb.com/recpit/viewtopic.php?t=1284

It would be OK to discuss it with the author, but I was reticent to post anything to the prosoundweb board. I don't know anything about that specific board, but from the name I feared it would be like other "recording people's" boards I've read: the people know more about music making than technology.

This is fine and good (especially if you're making music!) but it also means that, if you state a scientific or mathematical fact, you tend to spend the next 20 pages explaining it and defending it, and at the end they dismiss it, just saying "well, I'll just trust my ears".

Trusting your ears is the ultimate gold standard, but if your ears are wrecked, your monitors came from radioshack, and someone can prove that what you're doing is damaging the sound, you should take notice - not just say "I trust my ears" and carry on as you always have.

(sorry - end of rant against bad audio!)


So, from what I've surmised of the "pro" recording world, Nika has written them a nice summary of dither that is understandable. It does have mistakes, but I don't think most people would care about them. I'm not saying these folks are stupid: I'm saying they're not really bothered about this - they just want to feel like they understand, and get on with making music. So if me or Frank posts there and describes the errors, it won't matter - in fact we'll be the ones at fault! If you previously didn't understand dither, and now you do, the fact that the explanation wasn't 100% correct doesn't matter to you.


I see Frank's posting was cross posted, and was described as a "nit pick". Point proven. I already warned that I was going to be in dangerous "smart ass" territory, so please don't cross post mine!

Actually, go on - I'll let you - I have a quiet week ahead, and 20 pages of discussion is just what I need ;-)

Cheers,
David.
http://www.David.Robinson.org/

more (most of this is nit-picking, though some mistakes are fundamental errors)



You don't reduce noise. You reduce distortion. You increase noise.




To my knowledge, it isn't, but I could be wrong. To dither is to be indecisive - to not be able to make your mind up. To be "in a dither" is to be confused about the appropriate course of action. The nearest it gets to "vibration" is the idea of people moving between the two possibilities (e.g. should I go left or right? I'll go left <starts to go left> - no, I'll go right <starts go to right>).




they're steps, not square waves. (told you this was nit picking). The amplitude (height) of each step is equal to the step-size of the quantiser, and the duration (length) of each step is dependent on the slope of the original waveform, which arises due to its amplitude and frequency.




I prefer "one that reaches digital full scale" to "fully maximized". The distortion peaks will be lower for a higher amplitude signal. The reason is because there's the same amount of quantisation error whatever the signal, but it's signal correlated (i.e. harmonic distortion) for a low level signal, but semi-decorrelated for a high level signal (i.e. it appears as broadband noise).




No it's not! Where did you get this from? The idea persists in the whole discussion that what we really want is nice analogue gaussian noise, but TPDF dither will do. This is rubbish!

What we want to do is linearise the quantiser: in English, the quantiser has a step function - anything below a certain level falls into one numerical bin, anything above it falls into the next bin. If you plot an input output function, you get a staircase. We don't want this - we want to remove the steps, and have a nice straight line. We want the graph y=x - it means what you put in is what you get out. Perfect!

We can't get a straight line, but if we add the right about of noise, what happens is that a signal 20% above the limit between two quantisation steps has a 80% chance of falling on the lower step, and a 20% chance of falling on the upper step. A signal half way between the two steps has a 50% chance of falling on either.

Because of the noise, any given signal between two steps could now fall on either step (as opposed to always falling on the lower one - the case without dither, if you round down) - but the chance of it falling on either step is proportional to the actual level of the signal. So, if you average the signal, the steps disappear! Luckily, our ears do perform a kind of averaging - if you look in the frequency domain (which our ears, and all these pretty spectrograms do) you have to average in the time domain - you average over many many samples, and the stair case evens out.

So far, so good. This is using RECTANGULAR pdf noise. The problem is that the staircase is linear (there's no harmonic distortion) BUT signals half way between the two steps are never moved by more than 50% of a step, whereas signals at other locations can be moved further - up to 99.9% in fact. So the amount a signal gets shifted is dependent on its amplitude. This gives rise to noise modulation - it's more subtle than harmonic distortion, but it can be audible. This is solved by adding another RECTANGULAR pdf noise signal. The two together give rise to a TRIANGULAR noise signal. This sometimes pushes the signal over TWO steps, and (though I really can't explain it without a diagram!) means that signals of any amplitude are, on average, shifted by the same amount.

And this is ideal. no distortion, because the steps have been linearised, and no noise modulation, because the same amount of noise is added to any amplitude signal. So that's the first and second order problems of the quantiser fixed. There must be third, fourth, fifth etc problems too, but whereas you could quantify them using a mathematical analysis, your ear can't hear them, so we stop here!

So triangular dither is ideal. NOT gaussian - gaussian DOES linearise the quantiser, but it doesn't quite remove the noise modulation. The amount of noise modulation you have left is very small, but it's not zero. So analogue Gaussian dither (at the ADC) will do nicely, but in the digital domain there's no advantage to Gaussian dither at all.




It's called a rectangular PDF - I've not heard anyone call it square before, but I've probably not read the correct textbooks.




Hang on - if you use those dice (or a rect. or tri. PDF random number generator) to drive a digital to analogue converter, you WILL get white noise, up to fs/2. "TRUE white noise" is noise with a long term flat spectrum. Those dice are giving you TRUE white noise. The idea that Gaussian is TRUE white noise and triangular isn't TRUE white noise is complete nonsense.



Noise shaping - really bad explanation. And whether you call the UV22 system "dither" or not (I wouldn't, but then I like to call ideal dither "dither", and anything else "non-ideal noise" ;-) ), the UV22 system does not use noise shaping (according to their publication, though they could be hiding the truth for commercial reasons).

UV22 adds high frequency noise. Noise shaped systems don't have to add any noise, but what they do have to do (to be called noise shaping) is take the output, and feed part of it back to the input - this negative feedback system, with appropriate filtering in the negative feedback path, will push any quantisation noise and/or dither to higher frequencies. The UV22 system doesn't do any "pushing" - it just puts the noise there to start with and hopes. (repeat: according to their publication, though they could be hiding the truth for commercial reasons).




It's true that lower frequencies give rise to quantisation distortion that is highly correlated with the signal, and hence the resulting harmonic distortion is louder - or the quantisation error is more tone-like and less noise-like. But (inaudible) pure high frequency signals can give rise to large amounts of in-band quantisation distortion. Try a -30dB 20kHz sine wave in cool edit pro, and convert it to 8-bit without dither - you can hear the distortion tone! Use dither, and it goes away.

The thing is, the quantising error doesn't have an anti-alias filter applied to it - so any harmonic components that should have been above the nyquist limit of fs/2 will be folded back down into the audible range.




I can't find any more errors. If I do contact the author, I'll send him a copy of the best papers by Lipshitz and Vanderkooy. The maths can be hard to follow, but the explanations are great.

Cheers,
David.

http://www.David.Robinson.org/

Another German nitpicker entering ...I dont have a clue about dither, but above statement should be a bit inprecise if i remember correctly, because this will be highly depending on the signal amplitude you are referring to ? Certainly not true for a 100 V pp signal, but maybe for a 0 dB = 775 mV ?

The signal isn't usually 100 V pp entering an analogue to digital convertor ;-).

Seriously, the figure depends on the bandwidth of the device (e.g. a convertor running at 44.1kHz will give 22kHz bandwidth, while a convertor running at 96kHz will give 48kHz bandwidth) and the temperature of the resistor. The signal voltages are defined by standards (well, convention) in the home and (at a slightly higher level) in the studio.

The point is it's rather difficult (i.e. impossible!) to design a useable circuit where the noise isn't enough to dither a 24-bit convertor.

Maybe we shyould change to audio signal voltages that are large enough to kill you if you were to touch the wires. But I think that would bring its own design problems.

There's lots of noise analysis here.

http://www.aurastar.com/Noise%20Analysis.htm

But I'm not confident enough in this area to quote an exact figure. Less than 100 ohms seems about right given conventional signal amplitudes.

Cheers,
David.

If I understand well, dither increase the definition of low frequencies, because if we convert a 16 bits dithered signal to 24 bits, then lowpass it, the original shape will be back.

Correct me if I'm wrong, but I'm under the feeling that if we use a high-passed white noise as dither on a low-passed 24 bits wave as original, it will be possible to keep perfectly the 24 bits resolution (statistically) with no noise as long as the gap between the lowpass and the highpass is large enough so that the statistical fluctuations of the white noise (limited with the high pass)over the highest period of the wav (limited by the lowpass) are below the 24 bits resolution, therefore perfectly averaging the wave shape.

Does it make sense ?

No.

What is the ampliture of the Triangular PDF for the correct amount of dithering?

Is this amplitute expressed in LSB and is stated as P-P or RMS?

Peak-to-Peak is 2 LSB. RMS is sqrt(1/6) LSB.

Well, I'll try to explain better :

Let's consider a 24 bits 6 Khz wav file, and let's be the elementary level of the 16 bits resolution our unit of measurment.

Let's resample the wav to, say 6 MHz (ideally infinite sample rate)Now, let's add some 6 kHz highpassed square spread dither, at a level of 1 bit out of 16 (1 unit) peak to peak.

The dither being highpassed with no frequency below 6 kHz, the average of all added values within a 1/6000 s time interval is zero (no "DC offset").
Let's assume for convenience, that the 24 to 16 bits conversion is made by nearest neighbour instead of truncation. Last, let's assume that we'll convert the 6Mhz 16 bits back to 6 kHz 24 bits averaging the 1000 (ideally infinite) samples that are inside the 1/6000 s time window. (Rough antialias).

If the original wave level was zero for this sample, all dithered values will be between 0.5 and -0.5, therefore rounded to 0, and the average will be 0.

If the original wave level was 0.5, dithered values will be equally (square dither) distrubuted in the 0-1 interval, therefore 50% will be rounded to 0, and 50 % to 1. Therefore the average will be 0.5

Original=0.6
-> dithered equally distributed within 0.1 and 1.1
-> 40% rounded to 0, 60 % rounded to 1
-> average = 0.6

Original=x :
(where frac(x) is x modulo 1)

(1-frac(x)) *100 % are rounded to int(x)
frac(x) *100 % are rounded to int(x)+1

-> average = (1-frac(x))*int(x)+frac(x)*(int(x)+1)
=1*int(x)-frac(x)*int(x)+ frac(x)*(int(x)+1)
=int(x)+frac(x)*(int(x)+1-int(x))
=int(x)+frac(x)
=x

Therefore the original 6kHz 24 bits wave is perfectly kept at 16 bits if the dither is a square spread with infinite sample rate highpassed at at least 6 kHz.

Does it make more sense ?

No.

You need some math knowledge in statistics, system theory
and coding theory to be able to understand the problems. I only know the German vocabulary for the stuff you need to understand
the background of the different quantization processes.

And please don't look at the wave forms of a PCM signal.
The ear can't see the waveform. That's why ADPCM is not the
best lossy compressor.

Frank,

I hope no offence was taken by my "nit-picking" reference. I was just trying to steer the topic towards the issue at hand: dithering. Nika's references to noise-shaping were asides to the dithering issue, as I understood.

Most of the issues taken with Nika's paper, as far as I can see, are with interpretations of phrasology, terminology and verbage. I have not seen any refutation of Nika's explaination of dither in a basic scientific level as has been claimed.

Example: What is the difference between "square probability" and "rectangular probability"? Verbage. Or, ego, perhaps...either way, parsing words can be fun, but not always entirely useful or time-efficient.



Amen, good brother.

I, too, have been frustrated by this very issue on the "audio" boards. But guess what: That's why I am so encouraging of peeps like Nika and his efforts to explain these scientific issues in ways that the average Joe can understand.

As a composer of original music, I depend upon engineers to translate my works in the highest quality possible, with cost-efficient needs in mind.

As such, I have taken engineers to task over issues such as the supposed audible benefits of recording at 96Kz, and the supposed benefits (or lack thereof) of mastering to DSD.

The lack of understanding in the underlying basics of these media on the part of the general audio engineering community amazes me. There is an information gap. So, if I seem extraordinarily exhuberent in my defence of peeps like Nika, who are taking steps to fill that gap, please excuse me.

I do not trust my own ears entirely, so I will not trust anyone else's ears entirely, either. I personally would like a mathematical reference from which to base my ear's impressions. The two should go hand-in-hand. We listen, yes, but at the same time, we are "looking under the hood" for a way to put those listening experiences into context.

Dr. Stanley Lipshitz has written some comprehensive mathematical treatises on dithering. A few of us can understand them. Nika Aldrich has a way of explaining those concepts so that more can understand them. I believe it behooves us all to encourage that sort of effort, in the spirit of "Open Source."

Your mileage may vary...

Forget it, then, it doesn't matter...

I'll read the more technical papers. (I gave up math when they tried to demonstrate us that the Christoffel Symbol was not a tensor, but I should be able to follow on anyway )

So what dither settings would be recommended for use with the mppdec24? The description in the mpp manual.txt is a little over my head:

mppdec16, mppdec24 and mppdec32 should be
compiled from the current source files.
Otherwise the may be possible enhancements
of these versions are smaller than the
effect of non-fixed bugs. The binaries are really outdated.

The current implementation uses --dither 1
for a perfect amount of dither which is absolutely
distortion free.

If the source contains some noise or if some
distortions are allowed, dither can be reduced.
experiment with

--scale 256 --bits 8 --dither 0.00
...
--scale 256 --bits 8 --dither 1.00

Also
--shape 0
...
--shape 3

can be useful. With these settings you degrade
your sound card to a 8 bit sound card,
16 bit soundcards have 48, 24 bit 96 dB more
headroom, so don't listenen to the quanization
noise at too high sound presure levels.
Noise should have at most 25 dB(A) for 16 bit
and -25 dB(A) for 24 bit sound cards. This is
enough for 140 dB max. SPL.

These should be recent enough: http://www.saunalahti.fi/~cse/mppdec_1.01j.zip

Edit: new version

Hope this compiled from 1.01j20, not from 1.01j11...13. Call mppdec16 and look
for the range proposal of --dither:

old: 0.0-1.7
new: 0.00-1.00

Old versions use different dither, new
version uses more sophisticated dither,
noise shaping remains the same.

Linux binaries also available on my
webpage.

A good listening example is
on http://www.uni-jena.de/~pfk/mpp/testcase.html

the sample:
http://www.uni-jena.de/~pfk/mpp/audio2/Submarine.pac

(420 KByte). Encode it as braindead, decode it
using the settings I mentioned above.

Also a short list of all quantization methods I
know:

http://www.uni-jena.de/~pfk/mpp/Dieter.txt

It's new, the range is 0.00-1.00.

Pio2001, Frank,


what Pio2001 is suggesting is like DSD, only with more bits. OK, so DSD noise-shapes the error signal, rather than "just" adding high frequency dither, but I'm sure at the level Pio2001 was proposing the idea (i.e. a very basic explanation), he wasn't worry about such issues - and what he was saying, in a very basic sense, was right.

His system exists too - it's called DSD-wide, uses 8-bits and runs at 2.something MHz. It's used in the production of SACDs, and (unlike the 1-bit signal which goes onto the final disc) is theoretically perfectable. The low-pass starts at 100kHz and is quite gentle.

Cheers,
David.
http://www.David.Robinson.org/

Do you mean that DSD uses it's own "quantization" (if such a word exists in DSD) noise as a dither :eek: ? I had never thought about it !

For a perfect quantization process you need at least 2 LSBs dither. With a 1 bit system you can't add 2 LSBs dither.
for a 2 bit system it reduces FS level by a factor of 3 (9.54 dB).

So you can't dither a 1 bit system properly. You need the help
of the noise shaping process (noise shaping has nothing to do
with dither, see http://www.uni-jena.de/~pfk/mpp/Dieter.html).
But these only reduces distortions, and some amount remains.

You can construct signals which can't be reproduced by DSD,
they sound very different from the original. With PCM this is
not possible.

With 2.88 MBit/s you can have a system with 96 kHz/16 bit
with a flat response until 38 kHz, a gentle slope down to
-70 dB until 58 kHz using some noise shaping to reach
134 dB of SNR.

This is half the data rate of DSD, distortion free, easier to handle,
double the frequency response I need, a impulse response of
80 µs.

If jitter is a problem of digital audio, also note that DSD is
by about a factor of 1:1000 more sensitive to jitter than PCM.

I'd like to read about how DSD - SACD works.
I've already read many times that each bit meant that the signal should go up or down. But with an around 2 Mhz rate, it would mean about 50 bits each 40000th of a second. Therefore, in order to be able to code a full scale 20 kHz wave, the size of the step must be at least 1/50th of full scale, which would mean no more than 6 bits of resolution (2^6=64)

Therefore this description of SACD must be wrong somewhere...unless there is some strong ultrasonic dither at work...Is that it ?

DSD uses 64 times the sample frequency like the CD and
and exactly 1 bit per sample. Normally this would produce
a SNR of 25.8 dB relative to 44.1 kHz. The 120 dB are achieved
via really heavy noise shaping. To my mind so heavy noise shaping makes a lot of problems, also so called 1-bit-Converter
often use 7, 9 or 11 steps (instead of 2) to have less problems with noise shaping.

Note, that even when DSD have some advantages, this will be
killed by technical high end gadgets like digital controlled
loudspeakers, sound field synthesis, because they must be
converted to normal PCM before you can do such nice things.