Quantisation or quantization has a number of related meanings in data compression. These refer to encoding techniques where a number of different values are mapped to a single smaller value, which is then encoded. When the data is decoded, the value is scaled up again to give a value close to the input.
Although this produces a loss in the information transmitted, quantisation is useful in situations where it is not essential to transmit data absolutely accurately. Since quantisation can result in a large degree of compression, a small loss of quality is often acceptable.
The simplest technique is to divide the input value by a constant, as in JPEG image coding. The quotient is then rounded and encoded using Huffman coding. The Huffman codebook is designed so that small values can be encoded with fewer bits than large values. In JPEG encoding, quantisation by a large factor commonly produces long runs of zeros, which can be encoded very efficiently. JPEG achieves further efficiency gains by using a larger divisor in coefficients that are less noticeable.
Quantisation also refers to a related process where values are reduced in magnitude by a non-linear formula. The mapping used in AAC (Advanced Audio Coding) is:
y = sign(x) * ( abs(x) ^ (3/4) )
where sign(x) = -1, 0 or 1 depending whether x is negative, zero or positive, and abs(x) gives the absolute value of x.
If you measure accuracy by an absolute difference (rather than a percentage) such an encoding will mean that small values are encoded more accurately than large values.
The following table shows the values from 0 to 29 quantised by 2 functions. The first is a division by a constant, in this case 5, similar to that used in JPEG. The second uses the non-linear formula from AAC.
Input / 5 ^ 3/4
---------------------
0 0 0
1 0 1
2 0 2
3 1 2
4 1 3
5 1 3
6 1 4
7 1 4
8 2 5
9 2 5
10 2 6
11 2 6
12 2 6
13 3 7
14 3 7
15 3 8
16 3 8
17 3 8
18 4 9
19 4 9
20 4 9
21 4 10
22 4 10
23 5 11
24 5 11
25 5 11
26 5 12
27 5 12
28 6 12
29 6 12
Quantisation is the British spelling, quantization is the American.