Linear Time-Invariant Systems and Convolution

Introduction and Context

Signals and systems theory provides the mathematical foundation for analysing communication channels, control systems, audio and image processing, radar, biomedical instrumentation, and embedded electronics. A system transforms an input signal into an output signal. The central question is not merely what output results from one input, but how the system behaves for arbitrary inputs and how its properties can be predicted, measured, and designed.

Among all systems, linear time-invariant (LTI) systems occupy a special position. Linearity permits decomposition of complicated signals into simpler components, while time invariance permits predictable translation of system behaviour across time. Together, these properties lead to the convolution representation, which states that an LTI system is completely characterised by its response to an impulse.

Although real systems are often only approximately LTI, the model is remarkably powerful. It supports filter design, channel equalisation, noise analysis, spectral analysis, simulation, and system identification.

Learning Objectives

After studying this chapter, the reader should be able to:

1. Define systems, linearity, time invariance, causality, memory, stability, and invertibility.
2. Test whether a continuous-time or discrete-time system is LTI.
3. Derive and interpret the convolution integral and convolution sum.
4. Compute convolution analytically, graphically, and algorithmically.
5. Determine important system properties from the impulse response.
6. Relate convolution in time to multiplication in frequency.
7. Apply LTI theory to communication, filtering, and practical engineering systems.
8. Recognise modelling assumptions, numerical limitations, and implementation trade-offs.

1. Systems and Their Mathematical Description

A system maps an input signal to an output signal. For a continuous-time system, this may be written as

where x(t) is the input, y(t) is the output, and T{·} denotes the system operation. For a discrete-time system,

where n is an integer-valued time index.

A system can be physical, such as an amplifier or transmission channel, or mathematical, such as a moving-average operation. The input and output may be voltage, current, sound pressure, sampled data, image intensity, or another measurable quantity. Unless explicitly stated, signals are assumed to be real-valued; the theory also applies to complex-valued signals.

1.1 Linearity

A system is linear if it satisfies both additivity and homogeneity. Equivalently, for arbitrary signals x₁ and x₂ and arbitrary constants a and b,

The output corresponding to a weighted sum of inputs is the same weighted sum of the individual outputs. This is the superposition principle.

For example, y(t) = 3x(t) is linear. In contrast, y(t) = x²(t) is not generally linear because squaring changes both addition and scaling relationships. A system with a nonzero constant offset, such as y(t)=2x(t)+5, is affine rather than linear.

1.2 Time Invariance

A system is time invariant if delaying the input delays the output by the same amount, without otherwise changing it. If

then time invariance requires

for every permitted shift t₀. In discrete time, the condition is

for every integer shift n₀.

The system y(t)=x(t−2) is time invariant. The system y(t)=t x(t) is not time invariant because its gain explicitly depends on the absolute time coordinate.

1.3 Testing Linearity and Time Invariance

A reliable test is to apply the definition rather than relying on visual intuition. For a proposed system:

1. Compute the output for a general input.
2. Replace the input by a linear combination to test linearity.
3. Replace the input by a shifted version to test time invariance.
4. Compare the resulting expressions with the required conditions.

For example, consider y(t)=x(2t). It is linear because

T{a x₁(t)+b x₂(t)}=a x₁(2t)+b x₂(2t).

However, it is not time invariant. The output for a shifted input is x(2t−t₀), whereas a shifted original output is x(2t−2t₀). These are generally unequal.

2. Additional System Properties

Linearity and time invariance are central, but other properties determine physical behaviour and implementation requirements.

2.1 Memoryless and Dynamic Systems

A system is memoryless if its output at a time depends only on the input at that same time. The system y(t)=4x(t) is memoryless. The delay system y(t)=x(t−1) has memory because its output depends on a past input.

An LTI system is memoryless if and only if its impulse response has the form

for continuous time, or

for discrete time, where K is a constant and δ is the impulse. This statement assumes ideal mathematical impulses and the usual convolution model.

2.2 Causality

A system is causal if the output at a given time does not depend on future input values. For an LTI continuous-time system,

is the necessary and sufficient condition for causality. For discrete time,

is the corresponding condition.

A noncausal system may be mathematically valid but cannot operate in real time without prediction or access to stored future data. Offline signal processing can implement noncausal filters because the complete record is available.

2.3 Bounded-Input, Bounded-Output Stability

An LTI system is BIBO stable if every bounded input produces a bounded output. For continuous time, the condition is

For discrete time, it is

These are absolute-integrability and absolute-summability conditions, respectively. They assume the standard convolution operation and finite-valued bounded inputs.

For example, h(t)=e^−2tu(t) is stable because

where u(t) is the unit-step function. The impulse response h(t)=u(t) is not stable because its absolute integral over the positive time axis is infinite.

2.4 Invertibility

A system is invertible if distinct inputs produce distinct outputs and an inverse system can recover the input. In frequency-domain terms, an LTI system with response H(ω) is ideally invertible over a frequency range if H(ω) does not vanish there and the inverse response is physically implementable and sufficiently stable.

A theoretical inverse may exist but be unsuitable in practice. Very small values of H(ω) cause noise amplification when division by H(ω) is attempted. This issue is fundamental in channel equalisation and deconvolution.

3. The Impulse and the Impulse Response

The continuous-time unit impulse δ(t) is defined by

It also satisfies the sifting property

The discrete-time unit impulse is

and therefore

The impulse response is the output produced by an LTI system when the input is an impulse:

Every signal can be represented as a weighted collection of shifted impulses. Linearity determines the response to each weighted impulse, and time invariance determines that a shifted impulse produces a shifted impulse response. This reasoning leads directly to convolution.

4. Convolution Representation

4.1 Continuous-Time Convolution Integral

Represent the input as

The impulse at location τ has weight x(τ). Its system response is h(t−τ). By superposition,

This operation is denoted by y(t)=x(t)*h(t), where * means convolution. An equivalent form is

The integral exists under appropriate conditions, such as absolute integrability of one factor together with boundedness or suitable integrability of the other.

4.2 Discrete-Time Convolution Sum

Using the impulse expansion of a discrete-time signal,

Equivalently,

For finite-length sequences, only a finite number of terms contribute. If x[n] has length N and h[n] has length M, with both sequences indexed from zero, their linear convolution has length N+M−1.

4.3 Graphical Interpretation

To evaluate y(t)=∫x(τ)h(t−τ)dτ, one may:

1. Choose a value of t.
2. Reverse h(τ) to obtain h(−τ).
3. Shift it by t to obtain h(t−τ).
4. Multiply the overlapping functions.
5. Integrate over τ.
6. Repeat for other values of t.

Figure: Conceptual graphical procedure for evaluating continuous-time convolution.

5. Properties of Convolution

For signals for which the expressions exist, convolution has the following properties:

Commutativity

Associativity

Distributivity

Identity

Shift property

If y(t)=x(t)*h(t), then

These properties are useful both analytically and computationally. Associativity, for example, allows cascaded LTI systems to be combined into a single equivalent impulse response.

6. Worked Examples

Example 1: Continuous-Time Convolution of Rectangular Pulses

Let

and

Both signals are rectangular pulses of unit amplitude, with durations 1 and 2 seconds. Their convolution equals the length of overlap between the intervals 0≤τ≤1 and t−2≤τ≤t. Therefore,

The output has duration 3 seconds, equal to the sum of the input durations. Its area is also the product of the input areas: 1×2=2. Direct integration confirms the result. The maximum value is 1 because the shorter pulse is fully covered during the interval 1≤t<2.

Example 2: Discrete-Time Convolution

Let x[0]=1, x[1]=2, and x[2]=1, with all other samples zero. Let h[0]=1 and h[1]=1, with all other samples zero. Then

Thus, the output sequence is {1,3,3,1}. Its length is 3+2−1=4, as expected.

7. LTI Systems in the Frequency Domain

For suitable signals, the Fourier transform converts convolution into multiplication:

where X(ω), H(ω), and Y(ω) are the Fourier transforms of the input, impulse response, and output, respectively. The function H(ω) is the frequency response or transfer function in the frequency-domain description.

This relation is important because each frequency component of an LTI system is scaled and phase-shifted independently. If

then |H(ω)| is the amplitude response and ∠H(ω) is the phase response. A sinusoidal input at angular frequency ω₀ produces a sinusoid at the same frequency, modified in amplitude and phase, provided the system is LTI and the relevant steady-state conditions apply.

For discrete-time systems, the discrete-time Fourier transform similarly gives

with periodicity in ω of 2π. In sampled-data systems, frequency-domain interpretation must also account for aliasing and the sampling frequency.

8. Cascades, Parallel Systems, and Block Diagrams

If two LTI systems with impulse responses h₁ and h₂ are connected in cascade, the equivalent response is

In the frequency domain,

If systems are connected in parallel, their equivalent impulse response is

and their frequency responses add. These results support modular design: amplifiers, filters, sensors, and communication-channel models can be analysed as components before being combined.

9. Finite-Dimensional and State-Space Perspectives

Convolution describes an input-output relationship directly. State-space models instead describe internal variables. A continuous-time linear time-invariant state-space model is

where x(t) is the state vector, u(t) the input, y(t) the output, and A, B, C, and D constant matrices of compatible dimensions. With zero initial conditions, the impulse response is related to the state transition matrix by

for a causal system. The term e^At is the matrix exponential. This representation connects convolution with modern control, estimation, and digital signal-processing methods. Nonzero initial conditions produce an additional natural response, which must be distinguished from the zero-state response represented by convolution.

10. Efficient Computation and Implementation

Direct discrete convolution of sequences of lengths N and M generally requires approximately NM multiplications, giving computational complexity O(NM). This is appropriate for short filters and real-time embedded systems.

For long sequences, the convolution theorem permits block processing using the fast Fourier transform (FFT). A linear convolution can be computed by zero-padding the sequences so that circular convolution does not wrap the result into the wrong indices. If the transform length is at least N+M−1, multiplication in the transform domain followed by an inverse transform produces the linear-convolution result, subject to numerical precision.

Common methods include:

• Overlap-add: partitions the input into blocks, convolves each block, and adds overlapping outputs.
• Overlap-save: uses overlapping input blocks and discards corrupted circular-convolution samples.
• Polyphase implementation: decomposes filters for efficient interpolation and decimation.
• Finite impulse response filtering: uses a finite set of coefficients and is naturally implemented by a convolution sum.

Quantisation, finite word length, overflow, coefficient sensitivity, and latency must be considered in hardware and software implementations. FFT methods may reduce arithmetic cost but introduce block latency and numerical overhead.

11. Applied Case: Equalisation of a Multipath Communication Channel

Background

Consider an illustrative digital communication link in which a transmitter sends symbols through a wireless channel. Reflections from buildings produce delayed signal replicas. A simplified discrete-time channel may be represented by

The received sequence is

where s[n] is the transmitted symbol sequence and w[n] is additive noise.

Problem

Each received sample contains contributions from several transmitted symbols. This intersymbol interference can cause detection errors, especially when symbol decisions are made independently.

Analysis

The channel frequency response is

Its magnitude varies with frequency, so some spectral components are attenuated more than others. A formal zero-forcing equaliser would use

where the denominator is nonzero. However, if H(e^jω) is very small at any frequency, the equaliser greatly amplifies noise.

Approach and Solution

A practical receiver estimates the channel impulse response using known training symbols. It then chooses an equaliser according to the noise level and performance objective. A zero-forcing equaliser prioritises removal of channel distortion, whereas a minimum-mean-square-error equaliser balances distortion and noise enhancement. In a simplified frequency-domain form, the MMSE equaliser is

where H*(ω) is the complex conjugate, and S_w(ω) and S_s(ω) are noise and signal power spectral densities. This expression assumes a linear model, known or estimated spectra, and second-order statistical descriptions of signal and noise.

Outcome

After equalisation, the received symbols exhibit reduced intersymbol interference. The MMSE design may leave some residual distortion, but it generally avoids the severe noise amplification that can occur with zero forcing. Performance is evaluated using measures such as bit-error rate, error-vector magnitude, latency, computational complexity, and robustness to channel-estimation errors.

Learning Points

1. Convolution models multipath propagation naturally.
2. The impulse response reveals delay spread and individual path amplitudes.
3. Frequency-domain multiplication simplifies equaliser design.
4. Exact inversion is not always optimal in the presence of noise.
5. Engineering design requires a trade-off among accuracy, stability, complexity, and latency.

12. Limitations, Assumptions, and Contemporary Practice

The LTI model assumes that system behaviour does not change with time and that superposition holds. Real communication channels may vary rapidly; amplifiers may saturate; sensors may exhibit hysteresis; and biological or economic signals may involve nonlinear dynamics. In such situations, local linearisation, linear time-varying models, Volterra models, or data-driven nonlinear models may be more appropriate.

Impulse responses are also estimated from finite, noisy data. Windowing and truncation can create spectral leakage and model error. A long impulse response may reflect genuine physical memory, poor synchronisation, or an inadequate model. Regularisation is often necessary in inverse problems to avoid unstable solutions.

Machine-learning methods increasingly assist system identification, channel prediction, adaptive filtering, and denoising. Nevertheless, an LTI model remains valuable because it is interpretable, mathematically testable, and computationally efficient. A responsible engineering workflow should validate assumptions, report uncertainty, protect communication data, and avoid treating a fitted model as universally valid outside its calibration conditions.

Best practices include:

• state whether the model is continuous or discrete;
• specify initial-condition assumptions;
• distinguish linear convolution from circular convolution;
• check causality and BIBO stability;
• verify sampling-rate and unit consistency;
• use independent validation data for estimated systems;
• inspect both time-domain and frequency-domain behaviour;
• quantify latency, noise sensitivity, and computational cost.

Chapter Summary

An LTI system obeys superposition and behaves consistently under time shifts. Its response to any input is completely determined by its impulse response. Continuous-time outputs are obtained through the convolution integral, while discrete-time outputs are obtained through the convolution sum. Convolution is commutative, associative, and distributive, making it especially useful for cascaded and parallel systems.

For LTI systems, causality is determined by whether the impulse response vanishes before the origin, and BIBO stability is determined by absolute integrability or summability of the impulse response. In the frequency domain, convolution becomes multiplication, allowing amplitude and phase effects to be studied separately across frequency. Practical implementations use direct convolution for short filters and FFT-based methods for long signals. Communication-channel equalisation illustrates how convolution provides both a physical model and a basis for engineering design, while also showing why inversion must account for noise and model uncertainty.

Key Terms

Linearity: Property in which a system satisfies superposition.

Time invariance: Property in which an input shift produces an identical output shift.

LTI system: A system that is both linear and time invariant.

Impulse: Ideal unit-area continuous-time signal or unit sample in discrete time.

Impulse response: Output of a system produced by an impulse input.

Convolution: Operation combining an input and impulse response to produce an LTI output.

Causality: Dependence of the present output only on present and past inputs.

Memoryless system: System whose output depends only on the input at the same time.

BIBO stability: Property that every bounded input produces a bounded output.

Frequency response: Complex response of an LTI system as a function of frequency.

Transfer function: Frequency- or transform-domain representation of system behaviour.

Superposition: Combined principle of additivity and homogeneity.

Linear convolution: Convolution without periodic wraparound.

Circular convolution: Periodic convolution associated with finite-length transforms.

Intersymbol interference: Overlap of symbols caused by channel memory or filtering.

Equaliser: System designed to compensate for channel distortion.

Zero-forcing equaliser: Equaliser that attempts to invert the channel response.

MMSE equaliser: Equaliser that balances distortion and noise according to mean-square error.

State-space model: Matrix-based description using internal states, inputs, and outputs.

FFT: Efficient algorithm for computing the discrete Fourier transform and its inverse.

Review Questions

Short Answer Questions

1. State the superposition condition for a linear system.
2. What is the impulse response of an LTI system?
3. Give the causality condition for a discrete-time LTI system.
4. Write the discrete-time convolution sum.
5. Why does frequency-domain multiplication simplify LTI analysis?

Long Answer Questions

1. Explain, with mathematical tests, how linearity and time invariance are verified for a system.
2. Derive the continuous-time convolution integral from the impulse decomposition of an arbitrary signal.
3. Discuss causality, memory, BIBO stability, and invertibility in terms of the impulse response.
4. Explain the properties of convolution and their significance in cascaded and parallel systems.
5. Compare direct convolution and FFT-based convolution, including computational complexity and implementation limitations.

Analytical / Application Questions

1. Compute the convolution of x[n]={1,2,1} and h[n]={1,1}, assuming both sequences begin at index zero.
2. For h(t)=e^−3tu(t), determine whether the system is causal and BIBO stable, and justify both conclusions mathematically.
3. A communication channel has impulse response h[n]=δ[n]+0.5δ[n−1]. Explain how it creates intersymbol interference and compare the conceptual advantages and risks of zero-forcing and MMSE equalisation.

Multiple Choice Questions with Answers

Question 1

Which condition defines linearity for a system T{·}?

A. T{x(t−t₀)} = T{x(t)}−t₀

B. T{a x₁+b x₂}=aT{x₁}+bT{x₂}

C. T{x(t)}=t x(t)

D. T{x(t)}=x(2t)

Correct Answer: B. T{a x₁+b x₂}=aT{x₁}+bT{x₂}

Question 2

For a continuous-time LTI system to be causal, which condition must hold?

A. h(t)=0 for t>0

B. h(t)=1 for all t

C. h(t)=0 for t<0

D. ∫h(t)dt=0

Correct Answer: C. h(t)=0 for t<0

Question 3

The output of an LTI system is determined by which operation?

A. Differentiation only

B. Convolution of input and impulse response

C. Time reversal only

D. Division of input by time

Correct Answer: B. Convolution of input and impulse response

Question 4

If x[n] has length 3 and h[n] has length 2, the length of their linear convolution is:

A. 2

B. 3

C. 4

D. 6

Correct Answer: C. 4

Question 5

Which condition guarantees BIBO stability for a discrete-time LTI system?

A. ∑|h[k]| is finite

B. h[k] is nonzero for every k

C. h[k] is periodic

D. ∑h[k] is zero

Correct Answer: A. ∑|h[k]| is finite

Question 6

For an LTI system, convolution in time corresponds to what operation in the frequency domain?

A. Addition

B. Differentiation

C. Multiplication

D. Time reversal

Correct Answer: C. Multiplication

Question 7

The convolution of x(t)=u(t)−u(t−1) and h(t)=u(t)−u(t−2) has what maximum value?

A. 0.5

B. 1

C. 2

D. 3

Correct Answer: B. 1

Question 8

A system y(t)=x(2t) is:

A. Linear and time invariant

B. Nonlinear and time invariant

C. Linear but not time invariant

D. Nonlinear and not causal by definition

Correct Answer: C. Linear but not time invariant

Question 9

Why can a zero-forcing equaliser be problematic?

A. It always removes all noise

B. It can amplify noise where the channel response is small

C. It cannot use an impulse response

D. It necessarily makes the channel nonlinear

Correct Answer: B. It can amplify noise where the channel response is small

Question 10

For a cascade of two LTI systems, the equivalent impulse response is:

A. h₁+h₂

B. h₁−h₂

C. h₁/h₂

D. h₁*h₂

Correct Answer: D. h₁*h₂