2. Sampling, Quantization and Source Representation

Introduction and Context

A digital communication system does not usually transmit a continuously varying physical waveform in its original form. Speech, music, images, and sensor outputs are analog sources, whereas modern communication, storage, and processing systems are predominantly digital. The interface between these two domains is therefore fundamental. Sampling converts a continuous-time signal into a discrete-time sequence, quantization converts continuous amplitude values into a finite set of levels, and source representation expresses the resulting information efficiently as binary symbols.

These operations determine the quality, bandwidth, complexity, and reliability of the complete communication system. An inadequate sampling rate causes irreversible aliasing; coarse quantization introduces distortion; inefficient source representation wastes channel capacity. Conversely, well-designed source conversion permits error control, encryption, multiplexing, digital switching, and regeneration over long distances.

This chapter develops the theory and practice of sampling, reconstruction, pulse-code modulation (PCM), differential PCM (DPCM), delta modulation (DM), companding, quantization noise, and source coding. The discussion assumes familiarity with signals and systems, probability, and elementary digital communication theory.

Learning Objectives

After studying this chapter, the reader should be able to:

1. State and apply the sampling theorem and explain aliasing.

2. Describe practical signal reconstruction and the role of anti-aliasing filters.

3. Derive the main steps of PCM and evaluate its bit rate and quantization noise.

4. Compare uniform and nonuniform quantization, including companding.

5. Explain DPCM and DM, including slope-overload and granular noise.

6. Distinguish lossless and lossy source representation.

7. Apply entropy, Huffman coding, and run-length methods to simple sources.

8. Select an appropriate representation method by considering quality, bandwidth, complexity, and application requirements.

From Continuous Waveforms to Digital Sequences

Let an analog signal be represented by $x(t)$. Ideal uniform sampling takes measurements at intervals $T_s$, producing

$$x_s[n]=x(nT_s), \qquad f_s=1/T_s,$$

where $f_s$ is the sampling frequency. Sampling does not itself quantize amplitude: the samples may still take infinitely many values. Digital representation requires a second operation, quantization, in which each sample is assigned to one of a finite number of amplitude levels.

The distinction is important. A signal can be sampled and remain amplitude-continuous, as in a discrete-time analog system. It becomes digital only after amplitude quantization and binary encoding. In practice, an analog-to-digital converter (ADC) performs sampling, quantization, and coding, often with additional filtering and timing functions.

The sampling theorem

If $x(t)$ is band-limited to frequencies below $B$ hertz, it can be reconstructed exactly from uniformly spaced samples if

$$f_s>2B.$$

The quantity $2B$ is the Nyquist rate. The term Nyquist frequency refers to half the sampling frequency, $f_s/2$, and should not be confused with the Nyquist rate. At the theoretical boundary, ideal conditions are required; practical systems normally use a sampling frequency greater than twice the highest significant frequency.

In the frequency domain, sampling produces periodic replicas of the original spectrum separated by $f_s$. If these replicas do not overlap, an ideal low-pass reconstruction filter can recover the original signal. If $f_s<2B$, spectral replicas overlap. This phenomenon, called aliasing, causes different analog frequencies to produce the same sampled sequence. Once aliasing has occurred, it cannot be removed by digital processing because information has already been lost.

For example, a sinusoid at frequency $f$ sampled at $f_s$ is indistinguishable from sinusoids whose frequencies differ from it by integer multiples of $f_s$. A 9 kHz tone sampled at 10 kHz appears at 1 kHz. This is not a mere amplitude error; it is a false frequency component.

Practical sampling and reconstruction

Real systems use an anti-aliasing low-pass filter before the sampler. The filter attenuates frequencies above the intended signal band and provides a transition band between the highest desired frequency and the Nyquist frequency. The sampling rate must be chosen with the filter's achievable roll-off in mind. After digital processing, a reconstruction or anti-imaging filter smooths the output and suppresses spectral images generated by the digital-to-analog converter.

Sampling-clock jitter produces uncertainty in the sampling instant. For a sinusoid, timing error becomes more serious as signal frequency and amplitude increase. Thus high-resolution systems require not only adequate amplitude resolution but also a low-jitter clock. Oversampling relaxes analog filter requirements and can improve practical performance, although it increases the raw sample rate.

Quantization: Converting Amplitude into Finite Precision

Quantization maps each sample to one of $L$ permitted levels. For an $N$-bit quantizer,

$$L=2^N.$$

In a uniform quantizer covering a range from $-V_{ ext{max}}$ to $+V_{ ext{max}}$, the step size is approximately

$$\Delta=\frac{2V_{ ext{max}}}{L}.$$

The quantization error is the difference between the input sample and its reconstructed level. For a properly operated uniform quantizer, the error is often modeled as a uniformly distributed random variable in $[-\Delta/2,\Delta/2]$. Its mean-square value is

$$\sigma_q^2=\frac{\Delta^2}{12}.$$

This model is useful when the signal exercises many quantization levels and is not strongly correlated with the quantizer error. It is an approximation, not a universal law. At low amplitudes or for periodic inputs, quantization error may be deterministic and correlated with the signal, producing tones rather than white noise.

For a full-scale sinusoid and an ideal uniform quantizer, the signal-to-quantization-noise ratio is approximately

$$SQNR \approx 6.02N+1.76\text{ dB}.$$

Each additional bit improves the ideal SQNR by about 6 dB. The expression assumes a sinusoid occupying the available range. If the signal uses only a small fraction of the range, the effective SQNR is lower. Overload occurs when the input exceeds the quantizer range and causes much larger distortion.

Uniform and nonuniform quantization

Uniform quantization uses equal intervals and is appropriate when the amplitude distribution is relatively flat or when simplicity is important. Speech has a different characteristic: small amplitudes occur much more frequently than large amplitudes. Uniform quantization therefore gives unnecessarily fine absolute resolution to large signals and inadequate relative resolution to weak signals.

Nonuniform quantization uses smaller steps near zero and larger steps at high amplitudes. It can be implemented directly or through companding: compressing the input dynamic range before uniform quantization and expanding it after decoding. Companding improves the effective signal-to-quantization-noise ratio for weak signals without requiring a large number of uniform levels.

Two widely used speech companding laws are the $$-law and A-law. Their detailed formulas and parameter conventions depend on the telecommunications standard, but both approximate a logarithmic amplitude characteristic. The compressor increases the representation resolution of low-level signals; the expander reverses the operation. Companding is not lossless, but it allocates quantization precision in accordance with the source statistics.

Pulse-Code Modulation

PCM is the basic method for representing an analog waveform by a sequence of binary codewords. Its functional chain is:

analog input → anti-aliasing filter → sampler → quantizer → binary encoder → digital channel.

At the receiver, the inverse process uses a decoder, a hold or digital-to-analog converter, and a reconstruction filter.

If the sample rate is $f_s$ samples per second and each sample is represented by $N$ bits, the PCM bit rate is

$$R_b=Nf_s,$$

before framing, synchronization, error-control, and other overheads. For a source band-limited to $B$, the theoretical minimum sampling rate is just above $2B$, but engineering margins often require a higher rate.

A PCM code may be designed as a natural binary code, offset binary code, sign-magnitude code, or another mapping. The numerical code assignment does not alter quantization distortion by itself, but it affects implementation and error behavior. Gray coding, for example, can make adjacent levels differ by one bit, reducing the amplitude effect of a single bit error in some systems.

PCM's principal advantages are predictable quality, compatibility with digital switching and error control, and resistance to cumulative noise after regeneration. Its costs are sampling and quantization distortion, increased bandwidth, and conversion complexity.

Differential Representations: DPCM and Delta Modulation

PCM encodes the absolute value of every sample. In many sources, adjacent samples are strongly correlated, so the difference between successive samples has a smaller dynamic range than the samples themselves. DPCM exploits this property. A predictor estimates the current sample from past reconstructed samples, and only the prediction error is quantized and encoded.

If $\hat{x}[n]$ is the predicted value, the prediction error is

$$e[n]=x[n]-\hat{x}[n].$$

Because $e[n]$ generally has lower variance than $x[n]$, DPCM can use fewer bits for a specified quality. The decoder uses the same predictor and adds the decoded error to reconstruct the sample. Encoder and decoder must use compatible predictors; otherwise, errors can propagate through the prediction loop.

Adaptive DPCM changes the predictor or quantizer according to signal behavior. Adaptive prediction is useful when correlation varies with time, while adaptive quantization changes step size to follow the error amplitude.

Delta modulation is a one-bit differential system. At each sampling instant, the encoder sends whether the signal should be increased or decreased by a fixed step size $\delta$. The reconstructed signal is a staircase waveform. DM is simple and requires only one bit per sample, but its sampling rate is usually much higher than that required by PCM.

Two characteristic DM errors are important. Slope-overload distortion occurs when the input changes faster than the staircase can follow. A rough condition for avoiding it is

$$\frac{\delta}{T_s} \geq \max\left|\frac{dx(t)}{dt}\right|.$$

Granular noise occurs when the step size is too large for a slowly varying or nearly constant input; the staircase alternates around the signal. Adaptive delta modulation varies $\delta$: it increases the step size during rapid changes and decreases it during quiet intervals. This reduces the conflict between slope tracking and granular accuracy.

Source Representation and Coding Efficiency

Source representation concerns how information is mapped into symbols or bits. It is distinct from channel coding. Source coding removes statistical redundancy, whereas channel coding deliberately adds structured redundancy to protect against noise and errors.

For a discrete source with symbol probabilities $p_i$, the entropy is

$$H(X)=-\sum_i p_i\log_2 p_i,$$

measured in bits per source symbol. Entropy is the theoretical lower bound on the average length of an ideal lossless code under standard assumptions. A source with highly unequal probabilities has redundancy and may be compressed efficiently; a uniformly distributed source has maximum entropy for a fixed alphabet.

Lossless source coding

Lossless coding permits exact recovery. Huffman coding repeatedly combines the two least probable symbols and assigns binary branches to produce a prefix code. No codeword is the prefix of another, so decoding is unambiguous. Huffman coding is optimal among symbol-by-symbol binary prefix codes when the probabilities are known and codeword lengths are integral.

Arithmetic coding represents an entire sequence by a progressively narrowed numerical interval. It can approach entropy more closely than ordinary Huffman coding, especially when symbol probabilities are skewed or fractional-bit average lengths are useful. Dictionary methods such as Lempel–Ziv coding replace recurring strings with references and are effective when repetition is present. Run-length coding is especially suitable for long runs of identical symbols, such as flat regions in an image or binary control data.

Lossy representation

Lossy coding deliberately discards information judged less important to perception or application performance. Quantization in PCM is already a lossy operation. Transform coding, predictive coding, subband coding, and vector quantization are common approaches. A transform concentrates signal energy into a smaller number of coefficients; insignificant coefficients can then be coarsely quantized or omitted.

Lossy design is fundamentally a rate–distortion problem. A lower bit rate generally requires greater distortion, but the relationship depends on source statistics, perceptual criteria, and coding structure. Mean-square error may be appropriate for instrumentation but may correlate poorly with human perception in audio or images. Therefore, professional codec design uses application-specific objective and subjective measures.

An Applied Case: Digitizing a Speech Link

Background

Consider an illustrative engineering project in which a university laboratory must digitize a speech channel whose useful frequency range extends to 3.4 kHz. The link must support multiplexing and digital error control while maintaining intelligible and natural speech.

Situation and analysis

A sampling rate below 6.8 kHz would violate the ideal sampling condition. The design team selects 8 kHz, leaving a practical transition band for the anti-aliasing filter. If ordinary 8-bit PCM is used, the source bit rate is

$$R_b=8\times 8000=64\text{ kbit/s}.$$

However, speech amplitudes are concentrated near low levels. Uniform quantization gives poor relative resolution during quiet speech, so a companding law is selected. The compressor expands the representation of weak signals before quantization, and the receiver applies the inverse characteristic.

Approach and outcome

The system includes an analog anti-aliasing filter, an 8 kHz sample clock, an 8-bit companding quantizer, framing, and channel coding. At the receiver, decoded samples pass through a digital-to-analog converter and reconstruction filter. The design provides a standard digital speech rate and allows repeaters to regenerate clean binary data rather than amplify accumulated analog noise.

If bandwidth becomes the dominant constraint, DPCM or adaptive predictive coding could reduce the average rate by exploiting sample correlation. That alternative would require careful predictor synchronization and would be more sensitive to error propagation. The final choice therefore depends not only on compression ratio but also on network architecture, acceptable delay, implementation cost, and maintenance capability.

Learning points

The case illustrates that a sampling decision is linked to filter design, that bit rate depends jointly on sample rate and quantizer resolution, and that companding is a statistical allocation of precision rather than a replacement for sampling. It also shows why a technically efficient representation may not be the best operational choice if it increases system fragility.

Design Risks, Limitations, and Professional Practice

Common errors include confusing Nyquist rate with Nyquist frequency, sampling before adequate anti-aliasing, ignoring overload, assuming quantization noise is always white, and comparing PCM and DPCM solely by nominal bit rate. A predictor that performs well for speech may perform poorly for a rapidly changing sensor signal. Similarly, a perceptual codec may be unsuitable for scientific measurement because information considered inaudible or invisible may be diagnostically important.

Engineering practice should include range calibration, clock-jitter analysis, filter verification, overload tests, and measurements using representative—not only idealized—signals. Quantizer performance should be reported with the input distribution, bandwidth, and measurement criterion. Reproducible specifications are important in regulated or safety-related applications.

Digital conversion also raises professional and governance issues. Sampling choices can affect evidentiary integrity in medical, legal, and industrial records. Lossy compression must not be applied without authorization where exact recovery is required. Metadata should identify sampling rate, bit depth, coding method, and any processing that may affect interpretation. In networked systems, source data may contain personal or sensitive information; secure storage, access control, and responsible retention are therefore part of system design.

Current Directions

Modern converters increasingly use oversampling, noise shaping, and sigma–delta architectures. These techniques push quantization noise toward frequencies that can later be filtered, achieving high resolution with comparatively simple analog filtering. Digital filter banks and multirate processing permit different sampling rates in different parts of a system.

Adaptive and learned source representations are also developing. Statistical models can predict source behavior and allocate bits dynamically, while machine-learning codecs can optimize representations for speech enhancement, immersive audio, images, or sensor streams. Such systems must still be evaluated for latency, computational cost, robustness to distribution changes, interpretability, and failure behavior. Machine learning does not remove the sampling theorem or the need to preserve information relevant to the application.

Best-Practice Design Sequence

A defensible conversion design normally proceeds as follows:

1. Specify the source bandwidth, amplitude range, dynamic range, and quality criterion.

2. Select the sampling rate with allowance for filter transition bands and clock imperfections.

3. Design and verify the anti-aliasing filter.

4. Select quantizer resolution and range to balance noise against overload risk.

5. Consider companding or prediction when source statistics justify it.

6. Calculate the resulting bit rate, storage, latency, and channel bandwidth.

7. Choose lossless or lossy source coding according to recovery and application requirements.

8. Test with realistic signals, disturbances, and worst-case operating conditions.

9. Document format, timing, coding, metadata, and reconstruction assumptions.

Chapter Summary

Sampling converts a continuous-time waveform into discrete-time samples, while quantization and binary encoding produce a digital representation. The sampling theorem requires a rate greater than twice the highest signal frequency, and anti-aliasing filters are essential in practical systems. Quantization introduces an error whose ideal mean-square value is related to the square of the step size; increasing resolution improves ideal SQNR by approximately 6 dB per bit.

PCM directly encodes quantized samples and offers predictable, regenerable digital transmission. DPCM reduces the representation rate by encoding prediction errors, while delta modulation uses one-bit upward or downward steps and faces slope-overload and granular-noise trade-offs. Companding allocates more effective resolution to weak speech signals. Source coding removes statistical redundancy: lossless methods permit exact reconstruction, whereas lossy methods trade distortion for lower rate. System design must integrate source characteristics, filters, clocks, bandwidth, quality, complexity, security, and professional responsibility.

Key Terms

Sampling: Measurement of a continuous-time signal at discrete time instants.

Sampling frequency: Number of samples taken per second.

Nyquist rate: Twice the highest frequency present in a band-limited signal.

Nyquist frequency: Half the sampling frequency of a sampled system.

Aliasing: Irreversible spectral overlap caused by insufficient sampling.

Anti-aliasing filter: Analog low-pass filter placed before sampling.

Quantization: Mapping continuous amplitude values to discrete levels.

Quantization error: Difference between an input sample and its quantized value.

PCM: Representation in which each quantized sample is encoded as a binary word.

SQNR: Ratio of signal power to quantization-noise power.

Companding: Compressing amplitude range before quantization and expanding it after decoding.

DPCM: Differential PCM that encodes prediction errors rather than absolute samples.

Delta modulation: One-bit differential method using upward or downward signal steps.

Slope-overload distortion: DM error caused when the staircase cannot follow signal slope.

Granular noise: DM error caused by an excessively large step size.

Entropy: Theoretical average information per source symbol.

Lossless coding: Coding that permits exact source reconstruction.

Lossy coding: Coding that accepts controlled information loss to reduce rate.

Prefix code: Code in which no valid codeword is the prefix of another.

Rate–distortion trade-off: Relationship between representation rate and reconstruction quality.

Review Questions

Short Answer Questions

1. Distinguish the Nyquist rate from the Nyquist frequency.

2. Why is an anti-aliasing filter placed before an ADC?

3. State the ideal mean-square quantization-noise formula for a uniform quantizer.

4. What is the purpose of companding in speech PCM?

5. Name the two principal distortion mechanisms in delta modulation.

Long Answer Questions

1. Explain the sampling theorem in time and frequency domains, including aliasing and reconstruction.

2. Describe the complete PCM transmitter and receiver and derive its bit-rate expression.

3. Compare uniform quantization, nonuniform quantization, and companding.

4. Explain DPCM operation, prediction, error quantization, and error-propagation concerns.

5. Discuss lossless source coding methods and explain why entropy is a lower bound on average code length.

Analytical / Application Questions

1. A signal is band-limited to 5 kHz and is sampled at 8 kHz. Determine whether ideal alias-free sampling is possible and explain the practical filtering requirement.

2. A 10-bit uniform quantizer has a full-scale range from –2 V to +2 V. Find the approximate step size and ideal quantization-noise variance.

3. A speech system samples at 8 kHz using 8 bits per sample. Compare its raw PCM rate with a hypothetical 4-bit DPCM system, and discuss why the lower nominal rate may involve additional design risks.

22. Multiple Choice Questions with Answers

Question 1

A 4 kHz band-limited signal is sampled at 6 kHz. What is the principal consequence?

A. Perfect reconstruction is guaranteed

B. Aliasing can occur because the sampling rate is below 8 kHz

C. Only quantization noise occurs

D. The signal becomes amplitude-continuous

Correct Answer: B. Aliasing can occur because the sampling rate is below 8 kHz

Question 2

For an ideal uniform quantizer, increasing resolution by one bit approximately changes SQNR by

A. 1 dB

B. 3 dB

C. 6 dB

D. 12 dB

Correct Answer: C. 6 dB

Question 3

The main purpose of companding speech before uniform quantization is to

A. Increase the sampling frequency

B. Give relatively finer resolution to low-amplitude signals

C. Remove all channel noise

D. Eliminate the need for binary encoding

Correct Answer: B. Give relatively finer resolution to low-amplitude signals

Question 4

A PCM system samples at 10 kHz and uses 8 bits per sample. Its raw bit rate is

A. 1.25 kbit/s

B. 8 kbit/s

C. 18 kbit/s

D. 80 kbit/s

Correct Answer: D. 80 kbit/s

Question 5

DPCM can reduce the required representation rate primarily because

A. It eliminates sampling

B. Adjacent samples are often correlated

C. It uses no quantizer

D. It increases prediction error variance

Correct Answer: B. Adjacent samples are often correlated

Question 6

In delta modulation, slope-overload distortion occurs when

A. The step size is too large for a slowly varying signal

B. The staircase cannot follow the input slope

C. The sampling frequency is infinite

D. The quantizer has no output levels

Correct Answer: B. The staircase cannot follow the input slope

Question 7

Which method permits exact recovery of the original coded data?

A. Lossy transform coding

B. Coarse PCM quantization

C. Lossless Huffman coding

D. Perceptual coefficient suppression

Correct Answer: C. Lossless Huffman coding

Question 8

If a quantizer has step size Δ and the uniform-error model applies, its quantization-noise variance is

A. Δ/12

B. Δ²/12

C. 12Δ²

D. Δ²/2

Correct Answer: B. Δ²/12

Question 9

A reconstruction filter after a DAC is primarily used to

A. Create aliasing

B. Suppress spectral images and smooth the output

C. Increase quantization error

D. Predict future source samples

Correct Answer: B. Suppress spectral images and smooth the output

Question 10

Which statement best distinguishes source coding from channel coding?

A. Source coding removes redundancy; channel coding adds structured redundancy for protection

B. Both always remove redundancy

C. Source coding only applies to analog signals

D. Channel coding guarantees zero quantization noise

Correct Answer: A. Source coding removes redundancy; channel coding adds structured redundancy for protection