6. Multiresolution Analysis and Image Compression

Introduction and Context

Digital images contain information at many spatial scales. A fine texture, a sharp edge, a broad region of uniform intensity, and the overall arrangement of objects are all meaningful, but they do not occupy the same frequency range or require the same representation. Multiresolution analysis provides a principled way to examine and represent images at several levels of detail. Image compression, in turn, seeks to represent those images using fewer bits while preserving the information required for a particular application.

These topics are closely related. Pyramids and scale-space methods describe how an image changes as its resolution or observation scale changes. Wavelets provide localized representations of changes across scale and position. Compression algorithms exploit these representations by concentrating important information into relatively few coefficients and coding the remaining information efficiently. The choice of compression method is therefore not merely a question of reducing file size; it involves a rate–distortion decision involving quality, computational cost, latency, robustness, and application requirements.

This chapter develops the mathematical and practical foundations of pyramids, scale-space, wavelets, transform coding, and modern image-compression systems. It also examines JPEG, JPEG 2000, and learned compression methods, with attention to professional design choices and limitations.

Learning Objectives

After completing this chapter, the reader should be able to:

1. Explain the purpose of multiresolution representations and scale-space analysis.

2. Construct and interpret Gaussian and Laplacian pyramids.

3. Describe wavelet decomposition using filter banks and subbands.

4. Distinguish lossless, lossy, near-lossless, and perceptual compression.

5. Explain transform, quantization, prediction, and entropy coding stages.

6. Analyze rate–distortion trade-offs using appropriate quality measures.

7. Compare JPEG, JPEG 2000, and learned image-compression systems.

8. Select a compression strategy for scientific, medical, industrial, or consumer applications.

9. Identify common artifacts, implementation errors, and governance concerns.

Images at More Than One Scale

A discrete image is a sampled function, commonly written as \(f[m,n]\). Its apparent structure depends on the scale at which it is examined. At a coarse scale, small details disappear and large regions become dominant. At a fine scale, edges, textures, and noise become visible. A multiresolution representation makes these different views explicit rather than forcing one image grid to carry every type of information equally.

Scale-space representation

A classical scale-space representation is generated by convolving an image with Gaussian kernels of increasing standard deviation:

\[

L(x,y,\sigma)=G(x,y,\sigma)*f(x,y),

\]

where

\[

G(x,y,\sigma)=\frac{1}{2\pi\sigma^2}\exp\left[-\frac{x^2+y^2}{2\sigma^2}\right].

\]

The parameter \(\sigma\) controls the observation scale. Small values preserve fine structures, whereas large values suppress increasingly small structures. Gaussian scale-space is important because it satisfies useful consistency properties: increasing scale should not create spurious fine structures, and the family should vary smoothly with scale. In continuous theory, it is closely connected with the diffusion equation

\[

\frac{\partial L}{\partial \sigma}=\sigma\nabla^2L,

\]

or, with an alternative scale parameter, the heat equation. This connection explains why Gaussian smoothing can be interpreted as controlled diffusion.

Scale-space is widely used for blob detection, feature extraction, image registration, object recognition, and denoising. However, smoothing also weakens edges and can merge nearby structures. A scale must therefore be selected according to the spatial dimensions of the phenomenon being studied.

Gaussian and Laplacian pyramids

A Gaussian pyramid is a sequence of progressively reduced and smoothed images. A typical construction applies a low-pass filter and then downsamples by a factor of two:

\[

G_{l+1}=\downarrow 2\,(h*G_l),

\]

where \(h\) is a low-pass kernel and \(G_0\) is the original image. Before downsampling, low-pass filtering is essential; otherwise, high-frequency content above the new Nyquist frequency aliases into lower frequencies.

The Gaussian pyramid provides smaller images at coarser resolutions. It is useful for efficient search, coarse-to-fine optimization, image blending, and object detection. A coarse level contains global structure, while a fine level contains additional detail.

A Laplacian pyramid stores the difference between a Gaussian level and an expanded version of the next coarser level:

\[

L_l=G_l-\operatorname{Expand}(G_{l+1}).

\]

The expansion operation upsamples and filters the coarse image. Each Laplacian level is therefore a band-pass-like residual. With the final Gaussian level, the original image can be reconstructed by repeated expansion and addition. The Laplacian pyramid is especially valuable for compression and progressive transmission because it separates coarse structure from detail residuals.

Pyramids are conceptually simple and computationally efficient, but their subbands are not usually as sharply localized in frequency as ideal wavelet subbands. Redundancy can also arise because filtering and interpolation are not always arranged as a critically sampled, perfectly reconstructing filter bank.

Wavelets and Filter-Bank Representations

Wavelet analysis represents a signal using functions localized in both position and scale. Unlike the Fourier transform, which describes global frequency content, a wavelet transform can indicate where a particular frequency-scale event occurs. This is valuable for images containing edges, singularities, and localized texture.

One-dimensional two-channel decomposition

A basic discrete wavelet transform uses a low-pass analysis filter \(h[n]\) and a high-pass analysis filter \(g[n]\). Filtering followed by decimation produces approximation and detail coefficients:

\[

a[k]=\sum_n h[n-2k]x[n], \qquad d[k]=\sum_n g[n-2k]x[n].

\]

The approximation coefficients describe slowly varying content, while the detail coefficients describe changes. The factor-of-two downsampling prevents the representation from doubling in size. A corresponding synthesis filter bank reconstructs the signal. Perfect reconstruction requires appropriate relationships among the analysis and synthesis filters and careful treatment of boundaries.

Two-dimensional decomposition

For images, separable filtering applies low-pass and high-pass filters along rows and columns. One level produces four subbands:

• LL: low-pass horizontally and vertically; it contains the coarse approximation.
• LH: low-pass in one direction and high-pass in the other; it emphasizes directional detail.
• HL: the complementary directional detail band.
• HH: high-pass in both directions; it often contains diagonal detail and fine texture.

Further decomposition is usually applied to the LL subband, producing a multilevel tree. The result is a compact description of coarse structure and progressively finer details. Wavelet coefficients near strong edges often have large magnitudes, while coefficients in smooth regions are small. This concentration of energy supports effective quantization and entropy coding.

The Haar wavelet is simple and computationally inexpensive but produces block-like behavior. Daubechies, biorthogonal, and Cohen–Daubechies–Feauveau wavelets offer different compromises among compact support, smoothness, symmetry, and perfect reconstruction. JPEG 2000 commonly uses the reversible 5/3 wavelet for lossless or near-lossless operation and the irreversible 9/7 wavelet for lossy coding.

Wavelet packets and directional extensions

Standard wavelet trees recursively decompose only the approximation band. Wavelet packets allow additional decomposition of detail bands and can provide a better basis for particular signals, although the computational and selection complexity increases. More advanced directional systems, including contourlet- and shearlet-like methods, are designed to represent elongated edges more efficiently than separable wavelets. These methods are useful in research and specialized applications but are not automatically superior for every compression task.

From Representation to Compression

Compression removes redundancy and, in lossy systems, information judged less important. Three broad forms of redundancy are commonly considered:

1. Spatial redundancy, caused by correlation among neighboring pixels.

2. Spectral or intercomponent redundancy, caused by correlation among color channels or related measurements.

3. Coding redundancy, caused by using inefficient codewords for symbols with known probabilities.

Lossless compression reconstructs the exact original image. It is required when every pixel value matters, as in some scientific, legal, archival, and medical contexts. Lossy compression permits controlled changes in exchange for substantially lower rates. Near-lossless methods constrain the maximum reconstruction error, whereas perceptual methods optimize an approximation of visual or task-relevant quality.

The transform-coding pipeline

A conventional lossy codec commonly follows this sequence:

1. Convert or predict the input to reduce correlation.

2. Transform the residual or image block into coefficients.

3. Quantize the coefficients.

4. Reorder and represent likely zeros efficiently.

5. Apply entropy coding.

6. Store side information, headers, and metadata.

The transform itself is usually reversible in principle. Quantization is the principal irreversible operation. For a scalar quantizer,

\[

\hat{c}=Q(c)=\operatorname{round}(c/\Delta),

\]

where \(c\) is a transform coefficient and \(\Delta\) is a step size. A larger step size lowers the bitrate but introduces greater error. Entropy coding then represents quantized symbols according to their probabilities. Huffman coding uses prefix codes; arithmetic and range coding approach the entropy more closely when probability models are accurate.

Lossless predictive codecs may encode a residual such as

\[

r[m,n]=f[m,n]-\hat{f}[m,n],

\]

where \(\hat{f}\) is predicted from neighboring samples. If residual values cluster near zero, entropy coding becomes efficient. Run-length coding is particularly useful when quantization produces long runs of zero coefficients.

Rate–distortion reasoning

Compression quality should be evaluated jointly with rate. Let \(R\) denote bits per pixel and \(D\) denote distortion. A rate–distortion curve shows the attainable relationship between them. A general optimization objective is

\[

J=D+\lambda R,

\]

where \(\lambda\) expresses the relative cost of distortion and bitrate. Increasing \(\lambda\) favors lower rate; decreasing it favors quality.

Mean squared error (MSE) and peak signal-to-noise ratio (PSNR) are convenient mathematical measures:

\[

\operatorname{MSE}=\frac{1}{MN}\sum_{m,n}(f[m,n]-\hat f[m,n])^2,

\]

\[

\operatorname{PSNR}=10\log_{10}\left(\frac{MAX^2}{\operatorname{MSE}}\right).

\]

They are not complete measures of visual quality. Structural similarity, multiscale structural similarity, perceptual metrics, task accuracy, and expert assessment may be more appropriate. A codec that achieves a high PSNR may still introduce objectionable ringing around edges, while a codec with lower PSNR may preserve perceptually important structure more effectively.

JPEG: Block Transform Compression

The baseline JPEG process normally converts RGB to a luminance–chrominance space such as YCbCr. Since human vision is generally less sensitive to high-frequency chrominance errors than to luminance errors, chroma components may be subsampled. Each component is divided into 8 × 8 blocks, and a discrete cosine transform is applied:

\[

C(u,v)=\alpha(u)\alpha(v)\sum_{x=0}^{7}\sum_{y=0}^{7}f(x,y)

\cos\left[\frac{(2x+1)u\pi}{16}\right]

\cos\left[\frac{(2y+1)v\pi}{16}\right].

\]

Low-frequency coefficients are concentrated near the upper-left corner of the transformed block. Quantization tables assign larger steps to less important high-frequency coefficients. The quantized coefficients are scanned in zigzag order, which tends to place zeros together. Differential coding is used for DC coefficients, while run-length and entropy coding represent AC coefficients.

JPEG is computationally mature, broadly supported, and effective for photographs at moderate rates. Its principal weakness is block structure. At low rates, independent quantization of 8 × 8 blocks can cause blocking, ringing, loss of texture, and color bleeding. The JPEG quality setting is not a universal bitrate or quality standard because implementations use different quantization tables and sampling choices.

JPEG 2000: Wavelet-Based Coding

JPEG 2000 replaces block DCT coding with a wavelet transform over tiles or the complete image. It supports both irreversible 9/7 and reversible 5/3 transforms. Its coding architecture uses quantized wavelet coefficients, embedded bit-plane organization, and arithmetic coding. Important capabilities include progressive quality and resolution transmission, region-of-interest coding, error resilience, large-image support, and lossless-to-lossy scalability.

Because the transform is not based on independent 8 × 8 blocks, JPEG 2000 usually avoids conventional blocking artifacts. At aggressive compression, it may exhibit ringing or smooth, wavelet-related distortions. Its implementation and computational demands can be greater than those of baseline JPEG, and software or hardware ecosystem support has historically been less universal.

For digital cinema, geospatial imagery, archival systems, and some medical or scientific workflows, JPEG 2000's scalability and reversible mode can be more valuable than simple consumer compatibility.

Learned Image Compression

Contemporary learned codecs use neural networks to learn an analysis transform, a quantized latent representation, and a synthesis transform. A simplified architecture is

\[

y=g_a(x), \qquad \hat y=Q(y), \qquad \hat x=g_s(\hat y).

\]

A probability model estimates the distribution of latent symbols, enabling entropy coding. Training commonly minimizes a rate–distortion objective:

\[

\mathcal{L}=R(\hat y)+\lambda D(x,\hat x).

\]

Hyperprior models provide side information that improves estimates of local latent distributions. Autoregressive or context models can further improve probability prediction, although they may increase decoding latency. Some systems optimize pixel fidelity, while others use perceptual or adversarial objectives. A perceptually attractive reconstruction may not be faithful to the original, particularly in textured or medical imagery; this distinction must be explicit.

Learned compression can adapt its representation to data distributions and may achieve competitive rate–distortion performance. Nevertheless, it introduces model-storage requirements, training-data dependence, hardware demands, reproducibility concerns, and possible distribution shift. A model trained on ordinary photographs may perform poorly on satellite images, microscopy, or industrial radiographs. Neural decoders can also hallucinate plausible details, which is unacceptable when the reconstruction is treated as evidence rather than visualization.

Selecting a Compression Strategy

The correct codec depends on the information and operational requirements, not solely on nominal compression ratio. Relevant questions include:

• Must the reconstruction be pixel-exact?
• Is the image used for human viewing, measurement, classification, or archival evidence?
• What latency, memory, and energy constraints apply?
• Is progressive transmission or region-of-interest access required?
• Will the image be decoded by standard software over many years?
• How sensitive is the application to ringing, blocking, color errors, or hallucinated detail?

Lossless PNG-like methods, reversible wavelets, or predictive coding are appropriate when exact recovery is mandatory. JPEG remains practical for broad compatibility and photographic content. JPEG 2000 is attractive when scalability, large images, region-of-interest coding, or reversible operation matters. Learned codecs are promising where a controlled deployment can support model management and where the quality objective is carefully defined.

Applied Case: Archiving and Remote Review of Microscopy Images

Background

A research hospital needs to store high-resolution microscopy images and provide remote reviewers with rapid access. The original images contain diagnostically important fine structures, and a subset will be retained as archival evidence. Reviewers also need low-resolution previews and the ability to inspect selected regions without downloading the entire slide.

Situation and problem

Uncompressed storage is expensive and remote transmission is slow. Ordinary JPEG produces visible artifacts around thin structures at aggressive rates. A single compression policy is unsuitable because preview, routine review, and archival preservation have different requirements.

Analysis

The archive requires exact or demonstrably bounded reconstruction for selected master images. Review images can tolerate controlled lossy distortion if diagnostic performance is validated. Progressive resolution and region-of-interest access are operational advantages. A metric based only on PSNR would be inadequate; expert review, segmentation accuracy, and preservation of small structures must also be assessed.

Approach and solution

The organization adopts a tiered policy. Master images are retained using a reversible wavelet-based mode or a validated lossless codec. Remote previews use a lossy, multiresolution representation with several resolution levels. Regions likely to contain diagnostically important structures are assigned higher quality, while background areas use a lower rate. The system records codec version, parameters, checksums, color information, and whether a displayed image is lossy.

Before deployment, compression levels are tested using representative samples and blinded expert evaluation. Automated checks compare measurements and downstream segmentation results between original and reconstructed images. Learned compression is considered only for previews, with a fixed model version and a clear prohibition against treating synthesized details as original evidence.

Outcome

The tiered design reduces transmission and storage demands while preserving an exact archival path. Progressive viewing improves usability, and region-of-interest coding reduces waiting time for remote reviewers. Governance metadata makes it possible to distinguish original, lossless, and lossy derivatives.

Learning points

The case demonstrates that compression is a systems decision rather than a file-format decision. Multiple representations may be appropriate for different stages of a workflow. Validation must involve the actual task, and metadata, provenance, and reversibility are part of technical quality.

Limitations, Risks, and Common Mistakes

Aliasing is a frequent pyramid error: downsampling without adequate low-pass filtering creates irreversible false patterns. Boundary handling can produce artifacts in wavelet and transform coding, particularly when padding is inconsistent between encoder and decoder. Excessive quantization can erase weak but important features. Comparing codecs at different chroma-subsampling settings or different image bit depths can lead to misleading conclusions.

PSNR should not be interpreted as a complete quality judgment. Conversely, perceptual optimization should not be used uncritically for scientific or medical data. A visually plausible reconstruction may contain invented structures. Repeated lossy recompression can accumulate artifacts, and converting between color spaces or bit depths may introduce additional error.

Operational risks include undocumented encoder settings, proprietary formats, obsolete decoders, corrupted metadata, and inadequate error detection. For long-term preservation, organizations should maintain format specifications, checksums, migration plans, and representative validation images.

Ethical, Professional, and Governance Considerations

Compression can influence decisions when images support diagnosis, surveillance, scientific measurement, or legal interpretation. Users must know whether an image is original, losslessly reconstructed, or altered by a perceptual or generative decoder. Compression parameters should be documented, and quality thresholds should be validated for the intended task rather than assumed from vendor labels.

Privacy is also relevant. Image files may contain embedded location, patient, device, or acquisition metadata. A compression pipeline should define which metadata are retained, removed, encrypted, or audited. Neural codecs require additional governance concerning training data, model provenance, version changes, security vulnerabilities, and distribution shift. In regulated environments, validation records and change control may be as important as the compression ratio.

Current Directions and Best Practices

Research is progressing toward content-adaptive transforms, neural entropy models, variable-rate and region-adaptive coding, semantic compression, and joint compression for machine vision. In machine-oriented applications, the best representation may preserve classification or detection accuracy rather than pixel fidelity. Edge devices increasingly require codecs that balance energy, memory, latency, and rate.

Best practice is to:

1. Define the task and acceptable distortion before selecting a codec.

2. Separate archival masters from access derivatives.

3. Prevent aliasing before subsampling.

4. Report bitrate, bit depth, color sampling, codec parameters, and quality metrics.

5. Evaluate both objective metrics and task or expert performance.

6. Test on data representative of deployment conditions.

7. Preserve provenance, checksums, metadata, and decoder information.

8. Treat learned reconstruction as a model-dependent estimate, not automatically as evidence.

9. Examine failure cases, not only average rate–distortion results.

10. Reassess the codec when sensors, data distributions, or downstream tasks change.

Chapter Summary

Multiresolution analysis represents images at several scales, allowing coarse structure and fine detail to be studied separately. Gaussian scale-space uses progressive smoothing; Gaussian pyramids add subsampling for efficient coarse-to-fine processing; Laplacian pyramids store detail residuals. Wavelet transforms use localized filter-bank subbands and provide a powerful basis for multiscale representation and compression.

Image compression combines prediction or transformation, quantization, and entropy coding. Lossless methods preserve every sample, while lossy methods trade distortion for rate. Rate–distortion analysis formalizes this trade-off, but PSNR and MSE should be supplemented by structural, perceptual, and task-specific evaluation. JPEG uses block DCT coding and remains widely compatible; JPEG 2000 uses wavelets and supports scalable and reversible operation; learned codecs optimize analysis transforms and probability models using neural networks but introduce model, governance, and hallucination concerns.

A responsible compression design begins with application requirements, validates quality using representative tasks, and preserves provenance and reversibility where necessary.

Key Terms

Scale-space: A family of progressively smoothed representations of an image indexed by observation scale.

Gaussian pyramid: A multilevel sequence formed by low-pass filtering and downsampling.

Laplacian pyramid: A multilevel representation of detail residuals between adjacent Gaussian-pyramid levels.

Wavelet: A localized basis function used to represent signal variation across position and scale.

Filter bank: A collection of analysis and synthesis filters used to decompose and reconstruct signals.

Subband: A component containing information from a particular frequency or directional range.

Approximation coefficients: Low-frequency wavelet coefficients representing coarse structure.

Detail coefficients: Wavelet coefficients representing changes, edges, and fine-scale information.

Aliasing: False low-frequency structure produced when sampling cannot represent high-frequency content.

Transform coding: Compression based on representing data in a decorrelating transform domain.

Quantization: Mapping continuous or high-precision coefficients to a finite set of values.

Entropy coding: Variable-length or arithmetic representation based on symbol probabilities.

Rate–distortion trade-off: The relationship between compressed bitrate and reconstruction error.

PSNR: A logarithmic quality measure derived from mean squared error and signal peak value.

Lossless compression: Compression that permits exact reconstruction of the original data.

Lossy compression: Compression that permits controlled information loss to reduce bitrate.

Progressive coding: Transmission or decoding that improves quality or resolution incrementally.

Region-of-interest coding: Allocating better representation to selected image regions.

Learned compression: Neural-network-based analysis, synthesis, quantization, and entropy modeling for coding images.

Perceptual distortion: Error assessed according to human or task-specific importance rather than only pixel differences.

Review Questions

Short Answer Questions

1. Why is low-pass filtering required before downsampling in a Gaussian pyramid?

2. What information is represented by the LL, LH, HL, and HH wavelet subbands?

3. Why is quantization generally the irreversible stage of a lossy codec?

4. Distinguish lossless, lossy, and near-lossless compression.

5. State two reasons why PSNR alone is insufficient for evaluating image quality.

Long Answer Questions

1. Explain Gaussian scale-space and relate it to Gaussian and Laplacian pyramids.

2. Describe two-dimensional wavelet decomposition using a filter-bank model.

3. Explain the complete JPEG encoding pipeline and discuss its principal artifacts.

4. Compare JPEG and JPEG 2000 in terms of transform, scalability, artifacts, and applications.

5. Explain the rate–distortion objective used in learned image compression and discuss its practical limitations.

Analytical / Application Questions

1. A satellite-imaging system needs lossless archival storage, progressive browsing, and high-quality access to selected regions. Design a compression strategy and justify the codec features required.

2. Two codecs produce the same PSNR, but one preserves edge localization while the other produces ringing. Which additional tests would you conduct before selecting a codec for industrial inspection?

3. A neural codec gives attractive reconstructions of textured medical images but occasionally generates plausible structures absent from the source. Analyze the governance risks and propose deployment restrictions and validation procedures.

22. Multiple Choice Questions with Answers

Question 1

What is the primary reason for applying a low-pass filter before downsampling an image pyramid?

A. To increase the number of image samples

B. To prevent aliasing caused by discarded high-frequency content

C. To convert a lossy representation into a lossless one

D. To make all image regions have equal intensity

Correct Answer: B. To prevent aliasing caused by discarded high-frequency content

Question 2

In a Laplacian pyramid, a level is principally formed by:

A. Adding two independently sampled images

B. Quantizing only the highest-frequency coefficients

C. Subtracting an expanded coarse image from the corresponding finer image

D. Applying a color transform to the original image

Correct Answer: C. Subtracting an expanded coarse image from the corresponding finer image

Question 3

Which wavelet subband represents low-pass filtering in both spatial directions?

A. LL

B. LH

C. HL

D. HH

Correct Answer: A. LL

Question 4

In a conventional lossy transform codec, which operation normally introduces irreversible error?

A. Color-component reordering

B. Quantization

C. Zigzag scanning

D. Huffman code lookup

Correct Answer: B. Quantization

Question 5

A compression system uses the objective J = D + λR. Increasing λ generally causes the optimization to:

A. Favor lower bitrate more strongly

B. Ignore bitrate entirely

C. Guarantee zero distortion

D. Increase image dimensions

Correct Answer: A. Favor lower bitrate more strongly

Question 6

Which artifact is most characteristically associated with aggressive baseline JPEG compression?

A. Perfect reconstruction error in every pixel

B. Blocking caused by independently coded transform blocks

C. Aliasing caused only by wavelet downsampling

D. Loss of all chrominance information

Correct Answer: B. Blocking caused by independently coded transform blocks

Question 7

Which feature most directly distinguishes JPEG 2000 from baseline JPEG?

A. It uses no transform coefficients

B. It is restricted to binary images

C. It uses wavelet coding and can support scalable and reversible operation

D. It cannot perform lossy compression

Correct Answer: C. It uses wavelet coding and can support scalable and reversible operation

Question 8

Why might a learned codec be unsuitable for archival medical evidence even when its images look highly realistic?

A. Neural networks cannot represent any low-frequency information

B. It may synthesize plausible details that were not present in the source

C. Entropy coding is impossible after neural transformation

D. Learned codecs always require uncompressed transmission

Correct Answer: B. It may synthesize plausible details that were not present in the source

Question 9

For a scientific workflow requiring exact recovery of every pixel, which choice is most appropriate?

A. A validated lossless or reversible codec

B. A perceptual codec optimized only for visual appeal

C. Maximum JPEG quantization

D. A generative decoder without provenance records

Correct Answer: A. A validated lossless or reversible codec

Question 10

Why should a codec evaluation include task performance or expert assessment in addition to PSNR?

A. PSNR does not measure bitrate

B. Pixel error may not reflect preservation of diagnostically or operationally important structure

C. Expert assessment eliminates the need for any objective metric

D. Task performance is independent of reconstructed image content

Correct Answer: B. Pixel error may not reflect preservation of diagnostically or operationally important structure