Digital Audio: Conversion: Dither

Introduction to dither

===

FGR. 23 Dither can be applied to a quantizer in one of two ways. In (a) the dither is linearly added to the analog input signal, whereas in (b) it’s added to the reference voltages of the quantizer.

===

At high signal levels, quantizing error is effectively noise. As the audio level falls, the quantizing error of an ideal quantizer becomes more strongly correlated with the signal and the result is distortion. If the quantizing error can be decorrelated from the input in some way, the system can remain linear but noisy. Dither performs the job of decorrelation by making the action of the quantizer unpredictable and gives the system a noise floor like an analog system.

The first documented use of dither was by Roberts in picture coding.

In this system, pseudo-random noise (see Section 3) with rectangular probability and a peak-to-peak amplitude of Q was added to the input signal prior to quantizing, but was subtracted after reconversion to analog. This is known as subtractive dither and was investigated by Schuchman and much later by Sherwood. Subtractive dither has the advantages that the dither amplitude is non-critical, the noise has full statistical independence from the signal and has the same level as the quantizing error in the large signal undithered case.

Unfortunately, it suffers from practical drawbacks, since the original noise waveform must accompany the samples or must be synchronously recreated at the DAC.

This is virtually impossible in a system where the audio may have been edited or where its level has been changed by processing, as the noise needs to remain synchronous and be processed in the same way. All practical digital audio systems use non-subtractive dither where the dither signal is added prior to quantization and no attempt is made to remove it at the DAC.

The introduction of dither prior to a conventional quantizer inevitably causes a slight reduction in the signal-to-noise ratio attainable, but this reduction is a small price to pay for the elimination of non-linearities. The technique of noise shaping in conjunction with dither will be seen to overcome this restriction and produce performance in excess of the subtractive dither example above.

The ideal (noiseless) quantizer of FGR. 19 has fixed quantizing intervals and must always produce the same quantizing error from the same signal. In FGR. 23 it can be seen that an ideal quantizer can be dithered by linearly adding a controlled level of noise either to the input signal or to the reference voltage which is used to derive the quantizing intervals. There are several ways of considering how dither works, all of which are equally valid.

The addition of dither means that successive samples effectively find the quantizing intervals in different places on the voltage scale. The quantizing error becomes a function of the dither, rather than a predictable function of the input signal. The quantizing error is not eliminated, but the subjectively unacceptable distortion is converted into a broadband noise which is more benign to the ear.

Some alternative ways of looking at dither are shown in FGR. 24.

Consider the situation where a low-level input signal is changing slowly within a quantizing interval. Without dither, the same numerical code is output for a number of sample periods, and the variations within the interval are lost. Dither has the effect of forcing the quantizer to switch between two or more states. The higher the voltage of the input signal within a given interval, the more probable it becomes that the output code will take on the next higher value. The lower the input voltage within the interval, the more probable it’s that the output code will take the next lower value. The dither has resulted in a form of duty cycle modulation, and the resolution of the system has been extended indefinitely instead of being limited by the size of the steps.

Dither can also be understood by considering what it does to the transfer function of the quantizer. This is normally a perfect staircase, but in the presence of dither it’s smeared horizontally until with a certain amplitude the average transfer function becomes straight.

In an extension of the application of dither, Blesser has suggested digitally generated dither which is converted to the analog domain and added to the input signal prior to quantizing. That same digital dither is then subtracted from the digital quantizer output. The effect is that the transfer function of the quantizer is smeared diagonally (FGR. 25). The significance of this diagonal smearing is that the amplitude of the dither is not critical. However much dither is employed, the noise amplitude will remain the same. If dither of several quantizing intervals is used, it has the effect of making all the quantizing intervals in an imperfect convertor appear to have the same size.

===

FGR. 24 Wideband dither of the appropriate level linearizes the transfer function to produce noise instead of distortion. This can be confirmed by spectral analysis. In the voltage domain, dither causes frequent switching between codes and preserves resolution in the duty cycle of the switching.

===

FGR. 25 In this dither system, the dither added in the analog domain shifts the transfer function horizontally, but the same dither is subtracted in the digital domain, which shifts the transfer function vertically. The result is that the quantizer staircase is smeared diagonally as shown top left. There is thus no limit to dither amplitude, and excess dither can be used to improve differential linearity of the convertor.

===

Requantizing and digital dither

The advanced ADC technology which is detailed later in this section allows as much as 24-bit resolution to be obtained, with perhaps more in the future. The situation then arises that an existing sixteen-bit device such as a digital recorder needs to be connected to the output of an ADC with greater wordlength. The words need to be shortened in some way.

Section 3 showed that when a sample value is attenuated, the extra low-order bits which come into existence below the radix point preserve the resolution of the signal and the dither in the least significant bit(s) which linearizes the system. The same word extension will occur in any process involving multiplication, such as digital filtering. It will subsequently be necessary to shorten the wordlength. Clearly the high-order bits cannot be discarded in two's complement as this would cause clipping of positive half-cycles and a level shift on negative half-cycles due to the loss of the sign bit. Low-order bits must be removed instead.

Even if the original conversion was correctly dithered, the random element in the low-order bits will now be some way below the end of the intended word. If the word is simply truncated by discarding the unwanted low-order bits or rounded to the nearest integer the linearizing effect of the original dither will be lost.

===

FGR. 26 Shortening the wordlength of a sample reduces the number of codes which can describe the voltage of the waveform. This makes the quantizing steps bigger, hence the term requantizing. It can be seen that simple truncation or omission of the bits does not give analogous behavior. Rounding is necessary to give the same result as if the larger steps had been used in the original conversion.

===

FGR. 27 In a simple digital dithering system, two's complement values from a random number generator are added to low-order bits of the input. The dithered values are then rounded up or down according to the value of the bits to be removed. The dither linearizes the requantizing.

===

Shortening the wordlength of a sample reduces the number of quantizing intervals available without changing the signal amplitude. As FGR. 26 shows, the quantizing intervals become larger and the original signal is requantized with the new interval structure. This will introduce requantizing distortion having the same characteristics as quantizing distortion in an ADC. It then is obvious that when shortening the wordlength of a twenty-bit convertor to sixteen bits, the four low order bits must be removed in a way that displays the same overall quantizing structure as if the original convertor had been only of sixteen bit wordlength. It will be seen from FGR. 26 that truncation cannot be used because it does not meet the above requirement but results in signal dependent offsets because it always rounds in the same direction. Proper numerical rounding is essential in audio applications. Rounding in two's complement is a little more complex than in pure binary as can be seen in Section 3.

Requantizing by numerical rounding accurately simulates analog quantizing to the new interval size. Unfortunately the twenty-bit convertor will have a dither amplitude appropriate to quantizing intervals one sixteenth the size of a sixteen-bit unit and the result will be highly non-linear.

In practice, the wordlength of samples must be shortened in such a way that the requantizing error is converted to noise rather than distortion.

One technique which meets this requirement is to use digital dithering23 prior to rounding. This is directly equivalent to the analog dithering in an ADC. It will be shown later in this section that in more complex systems noise shaping can be used in requantizing just as well as it can in quantizing.

Digital dither is a pseudo-random sequence of numbers. If it’s required to simulate the analog dither signal of FGR. 23 and 24, then it’s obvious that the noise must be bipolar so that it can have an average voltage of zero. Two's complement coding must be used for the dither values as it’s for the audio samples.

FGR. 27 shows a simple digital dithering system (i.e. one without noise shaping) for shortening sample wordlength. The output of a two's complement pseudo-random sequence generator (see Section 3) of appropriate wordlength is added to input samples prior to rounding. The most significant of the bits to be discarded is examined in order to determine whether the bits to be removed sum to more or less than half a quantizing interval. The dithered sample is either rounded down, i.e. the unwanted bits are simply discarded, or rounded up, i.e. the unwanted bits are discarded but one is added to the value of the new short word.

The rounding process is no longer deterministic because of the added dither which provides a linearizing random component.

If this process is compared with that of FGR. 23 it will be seen that the principles of analog and digital dither are identical; the processes simply take place in different domains using two's complement numbers which are rounded or voltages which are quantized as appropriate. In fact quantization of an analog dithered waveform is identical to the hypothetical case of rounding after bipolar digital dither where the number of bits to be removed is infinite, and remains identical for practical purposes when as few as eight bits are to be removed. Analog dither may actually be generated from bipolar digital dither (which is no more than random numbers with certain properties) using a DAC.

Dither techniques

The intention here is to treat the processes of analog and digital dither as identical except where differences need to be noted. The characteristics of the noise used are rather important for optimal performance, although many sub-optimal but nevertheless effective systems are in use. The main parameters of interest are the peak-to-peak amplitude, the amplitude probability distribution function (pdf) and the spectral content.

The most comprehensive ongoing study of non-subtractive dither has been that of Vanderkooy and Lipshitz. and the treatment here is based largely upon their work.

FGR. 28 (a) Use of rectangular probability dither can linearize, but noise modulation (b) results. Triangular pdf dither (c) linearizes, but noise modulation is eliminated as at (d). Gaussian dither (e) can also be used, almost eliminating noise modulation at (f).

Rectangular pdf dither

Section 3 showed that the simplest form of dither (and therefore the easiest to generate) is a single sequence of random numbers which have uniform or rectangular probability. The amplitude of the dither is critical.

FGR. 28(a) shows the time-averaged transfer function of one quantizing interval in the presence of various amplitudes of rectangular dither.

The linearity is perfect at an amplitude of 1Q peak-to-peak and then deteriorates for larger or smaller amplitudes. The same will be true of all levels which are an integer multiple of Q. Thus there is no freedom in the choice of amplitude.

With the use of such dither, the quantizing noise is not constant. FGR. 28(b) shows that when the analog input is exactly centered in a quantizing interval (such that there is no quantizing error) the dither has no effect and the output code is steady. There is no switching between codes and thus no noise. On the other hand, when the analog input is exactly at a riser or boundary between intervals, there is the greatest switching between codes and the greatest noise is produced. Mathematically speaking, the first moment, or mean error is zero but the second moment, which in this case is equal to the variance, is not constant. From an engineering standpoint, the system is linear but suffers noise modulation: the noise floor rises and falls with the signal content and this is audible in the presence of low-frequency signals.

The dither adds an average noise amplitude of Q/___ 12 rms to the quantizing noise of the same level. In order to find the resultant noise level it’s necessary to add the powers as the signals are uncorrelated. The total power is given by:

[...] and the rms voltage is Q/__ 6. Another way of looking at the situation is to consider that the noise power doubles and so the rms noise voltage has increased by 3 dB in comparison with the undithered case. Thus for an n-bit wordlength, using the same derivation as expression (1) above, the signal to noise ratio for Q pk-pk rectangular dither will be given by: [...]

Unlike the undithered case, this is a true signal-to-noise ratio and linearity is maintained at all signal levels. By way of example, for a sixteen-bit system 95.1 dB SNR is achieved. The 3 dB loss compared to the undithered case is a small price to pay for linearity.

Triangular pdf dither

The noise modulation due to the use of rectangular-probability dither is undesirable. It comes about because the process is too simple. The undithered quantizing error is signal dependent and the dither represents a single uniform-probability random process. This is only capable of decorrelating the quantizing error to the extent that its mean value is zero, rendering the system linear. The signal dependence is not eliminated, but is displaced to the next statistical moment. This is the variance and the result is noise modulation. If a further uniform probability random process is introduced into the system, the signal dependence is displaced to the next moment and the second moment or variance becomes constant.

Adding together two statistically independent rectangular probability functions produces a triangular probability function. A signal having this characteristic can be used as the dither source.

FGR. 28(c) shows the averaged transfer function for a number of dither amplitudes. Linearity is reached with a peak-to-peak amplitude of 2Q and at this level there is no noise modulation. The lack of noise modulation is another way of stating that the noise is constant. The triangular pdf of the dither matches the triangular shape of the quantizing error function.

The dither adds two noise signals with an amplitude of Q/___ 12 rms to the quantizing noise of the same level. In order to find the resultant noise level it’s necessary to add the powers as the signals are uncorrelated. The total power is given by:

... and the rms voltage is Q/__ 4. Another way of looking at the situation is to consider that the noise power is increased by 50 percent in comparison to the rectangular dithered case and so the rms noise voltage has increased by 1.76 dB. Thus for an n-bit wordlength, using the same derivation as expressions (1) and (2) above, the signal to noise ratio for Q peak-to-peak rectangular dither will be given by:

Continuing the use of a sixteen-bit example, a SNR of 93.3 dB is available which is 4.8 dB worse than the SNR of an undithered quantizer in the large-signal case. It’s a small price to pay for perfect linearity and an unchanging noise floor.

Gaussian pdf dither

Adding more uniform probability sources to the dither makes the overall probability function progressively more like the Gaussian distribution of analog noise. FGR. 28(d) shows the averaged transfer function of a quantizer with various levels of Gaussian dither applied. Linearity is reached with 1/2Q rms and at this level noise modulation is negligible. The total noise power is given by:

...and so the noise level will be Q__ 3 rms. The noise level of an undithered quantizer in the large signal case is Q___ 12 and so the noise is higher by a factor of:

Thus the SNR is given by 6.02(n - 1) + 1.76 dB. A sixteen-bit system with correct Gaussian dither has a SNR of 92.1 dB.

This is inferior to the figure in expression (3) by 1.1 dB. In digital dither applications, triangular probability dither of 2Q peak-to-peak is optimum because it gives the best possible combination of nil distortion, freedom from noise modulation and SNR. Using dither with more than two rectangular processes added is detrimental. Whilst this result is also true for analog dither, it’s not practicable to apply it to a real ADC as all real analog signals contain thermal noise which is Gaussian. If triangular dither is used on a signal containing Gaussian noise, the results derived above are not obtained. ADCs should therefore use Gaussian dither of Q/2 rms and the performance will be given by expression (4).

It should be stressed that all the results in this section are for conventional quantizing and requantizing. The use of techniques such as oversampling and/or noise shaping require an elaboration of the theory in order to give meaningful SNR figures.

===