2. Linear Time-Invariant Systems and Convolution

Introduction and Context

A system transforms an input signal into an output signal. In electrical engineering, the system may represent an electrical network, a sensor, a communication channel, a control element, a mechanical plant, or a numerical algorithm. The central problem is to predict the output produced by a specified input and to determine the properties of the system from its mathematical description.

Linear time-invariant (LTI) systems occupy a uniquely important position because their behavior can be characterized completely by their response to an impulse. Once this response is known, the response to almost any admissible input follows through convolution. This result connects continuous-time and discrete-time analysis, differential and difference equations, frequency-domain methods, filtering, communications, and control theory.

The chapter develops the impulse-response representation of LTI systems and uses it to examine convolution, eigenfunctions, causality, memory, invertibility, and bounded-input bounded-output (BIBO) stability. These properties are not merely formal classifications: they determine whether a system can be implemented in real time, whether it requires stored information, whether its output remains physically acceptable, and whether an inverse system is practical.

Learning Objectives

After studying this chapter, the reader should be able to:

1. State and apply the principles of linearity and time invariance.

2. Derive continuous-time and discrete-time convolution representations.

3. Determine impulse responses from differential and difference equations.

4. Analyze LTI systems using eigenfunctions and frequency response.

5. Test systems for causality, memory, invertibility, and BIBO stability.

6. Interpret convolution physically and compute it accurately.

7. Recognize practical limitations associated with ideal models and inverse systems.

Linearity and Time Invariance

A system is linear if it satisfies superposition. If an input x produces y, then for arbitrary constants a and b, the input

x(t)=a x₁(t)+b x₂(t)

must produce

y(t)=a y₁(t)+b y₂(t).

This property combines additivity and homogeneity. Additivity requires that the response to x₁+x₂ equal the sum of individual responses; homogeneity requires that scaling the input by a constant scale the output by the same constant. A system involving saturation, squaring, multiplication of signals, or a nonzero constant offset is generally nonlinear.

A system is time invariant if delaying the input delays the output by the same amount. If

x(t) → y(t),

then time invariance requires

x(t−t₀) → y(t−t₀)

for every allowable t₀. In discrete time, the corresponding condition is x[n−n₀] → y[n−n₀]. A system whose parameters vary with time, such as y(t)=t x(t), is not time invariant because the factor multiplying the signal changes under a shift.

Linearity and time invariance are independent properties. A system may be linear but time varying, time invariant but nonlinear, both, or neither. The LTI combination is valuable because it permits decomposition of complicated signals into shifted impulses and treats every shifted impulse in a predictable way.

The Impulse Response as a Complete System Description

The unit impulse δ(t) is defined in continuous time by

∫δ(t)dt=1,

with the integral taken over any interval containing t=0, and δ(t)=0 elsewhere in the generalized-function sense. The discrete-time unit impulse δ[n] equals 1 at n=0 and 0 otherwise.

Let h(t) denote the response of an LTI system to δ(t):

δ(t) → h(t).

For a shifted impulse δ(t−τ), time invariance gives

δ(t−τ) → h(t−τ).

Any continuous-time input can be represented formally as

x(t)=∫_{−∞}^{∞}x(τ)δ(t−τ)dτ.

By linearity, the corresponding output is

y(t)=∫_{−∞}^{∞}x(τ)h(t−τ)dτ.

This is the convolution integral, written

y(t)=x(t)*h(t)=∫_{−∞}^{∞}x(τ)h(t−τ)dτ.

For a discrete-time signal,

x[n]=Σ_{k=−∞}^{∞}x[k]δ[n−k],

and therefore

y[n]=Σ_{k=−∞}^{∞}x[k]h[n−k]

or, equivalently,

y[n]=x[n]*h[n].

Thus, the impulse response completely characterizes an LTI system, subject to the usual conditions ensuring that the convolution exists. The expressions also show that convolution is commutative, associative, and distributive:

xh=hx,

(xh)g=x(hg),

x(h+g)=xh+x*g.

These identities have direct engineering interpretations. Commutativity means that mathematically the input and system response may be interchanged; associativity permits cascaded LTI systems to be replaced by one equivalent system; distributivity describes parallel combinations.

Understanding Convolution Operationally

To evaluate y(t)=x(t)*h(t), one may follow four conceptual operations: reverse one function, shift it, multiply the overlapping functions, and integrate over the variable. Using the form

y(t)=∫x(τ)h(t−τ)dτ,

h(t−τ) is h reversed and shifted. The output at a particular t is the area of the product over the interval where the two functions overlap.

For finite-duration signals, convolution is often most easily found by identifying the overlap limits before integrating. For example, let x(t)=1 for 0≤t≤T and h(t)=1 for 0≤t≤T, and zero otherwise. The output is the length of overlap between the two intervals. Hence

y(t)=0, t<0;

y(t)=t, 0≤t≤T;

y(t)=2T−t, T≤t≤2T;

y(t)=0, t>2T.

The result is triangular. This simple example illustrates why convolution generally increases duration: the output exists while the shifted input and impulse response overlap.

In discrete time, convolution is a weighted sum. If h[n] is finite, each output sample is formed by multiplying neighboring input samples by the corresponding impulse-response coefficients and adding them. A moving-average filter, for example, has h[n]=(1/M) for 0≤n≤M−1. Its output is the average of the most recent M input samples, making the relationship between convolution and digital filtering explicit.

Differential and Difference-Equation Models

Many physical systems are described by linear constant-coefficient differential equations. A general model is

a_N dᴺy/dtᴺ+⋯+a₁dy/dt+a₀y(t)

=b_M dᴹx/dtᴹ+⋯+b₁dx/dt+b₀x(t).

When initial conditions are zero, the impulse response is obtained by setting x(t)=δ(t) and solving the equation. Alternatively, the Laplace transform gives the transfer function

H(s)=Y(s)/X(s),

and h(t) is the inverse transform of H(s), with attention to the region of convergence and system conditions.

For discrete-time systems, a linear constant-coefficient difference equation may be written as

y[n]+a₁y[n−1]+⋯+a_Ny[n−N]

=b₀x[n]+b₁x[n−1]+⋯+b_Mx[n−M].

Under initial rest, the z-transform produces

H(z)=Y(z)/X(z).

The impulse response can then be found by inverse z-transform or by recursively applying the difference equation to x[n]=δ[n]. A recursive system may possess an impulse response of infinite duration even when the input is finite in duration. Such systems are called infinite impulse response (IIR) systems; systems whose impulse response has finite duration are finite impulse response (FIR) systems.

The zero-input or natural response must be distinguished from the zero-state response represented by convolution. Convolution describes the output due to the input when initial energy is zero. In a physical circuit, stored capacitor or inductor energy can produce an additional natural response.

Eigenfunctions and Frequency Response

Complex exponentials are eigenfunctions of LTI systems. For continuous time, let x(t)=e^{st}. Then

y(t)=∫h(τ)e^{s(t−τ)}dτ

=e^{st}∫h(τ)e^{−sτ}dτ

=H(s)e^{st}.

The input waveform is preserved; only its amplitude and phase are changed by the eigenvalue H(s). For sinusoidal steady-state analysis, s=jω, yielding

H(jω)=∫h(τ)e^{−jωτ}dτ,

which is the frequency response. A sinusoid at frequency ω emerges at the same frequency, multiplied by the complex gain H(jω). Its magnitude determines amplitude scaling and its phase determines phase shift.

In discrete time, e^{jωn} is an eigenfunction and

y[n]=H(e^{jω})e^{jωn},

where

H(e^{jω})=Σh[k]e^{−jωk}.

This property explains the importance of Fourier analysis in filtering and communications. An arbitrary signal can be interpreted as a combination of complex exponentials; an LTI system acts independently on each frequency component.

Structural Properties of LTI Systems

Causality

A system is causal if the output at time t depends only on input values at times no later than t. For an LTI system, causality is equivalent to

h(t)=0 for t<0

in continuous time, or h[n]=0 for n<0 in discrete time. A noncausal impulse response requires future input values and cannot be implemented in real time without prediction or offline processing.

A two-sided impulse response may still be mathematically valid and useful in smoothing, image processing, and offline signal reconstruction, but it is not causal.

Memory

A memoryless system produces y(t) from x(t) at the same instant only. For an LTI system, this requires h(t)=Kδ(t), or h[n]=Kδ[n]. Any delayed or distributed impulse response represents memory because past or future samples contribute to the output.

Invertibility

A system is invertible if distinct inputs produce distinct outputs and an inverse system can recover the input. If y=x*h, an inverse impulse response h⁻¹ must satisfy

h*h⁻¹=δ.

In the transform domain, the formal condition is H(s)H⁻¹(s)=1 or H(z)H⁻¹(z)=1. Invertibility does not automatically imply that the inverse is causal or stable. For example, a system may attenuate some frequency almost to zero. Its inverse must greatly amplify that frequency, making recovery highly sensitive to noise and numerical error.

BIBO Stability

BIBO stability means that every bounded input produces a bounded output. For a continuous-time LTI system, a sufficient and necessary condition under standard assumptions is

∫_{−∞}^{∞}|h(t)|dt<∞.

For discrete time, the condition is

Σ_{n=−∞}^{∞}|h[n]|<∞.

To see the continuous-time result, if |x(t)|≤B, then

|y(t)|≤∫|x(τ)||h(t−τ)|dτ≤B∫|h(τ)|dτ.

Thus the output is bounded by a fixed constant independent of t. An impulse response that does not have finite absolute area may produce an unbounded output from a bounded input. Pole locations provide an equivalent test for many rational systems: continuous-time poles must lie in the open left half-plane for stability, while discrete-time poles must lie strictly inside the unit circle, provided the relevant system representation is used.

Causality and stability are separate properties. A system may be causal but unstable, stable but noncausal, both, or neither. For instance, an ideal two-sided smoothing response can be stable but noncausal; an accumulator is causal but not BIBO stable.

A Unified Analysis Procedure

A reliable analysis of an LTI model can proceed as follows:

1. Identify the signal domain and determine whether the system is continuous or discrete time.

2. Test linearity and time invariance rather than assuming them from notation.

3. Obtain h from an impulse input, a governing equation, or a transform description.

4. Express the output through convolution and determine whether the sum or integral exists.

5. Use support and overlap to establish causality and output duration.

6. Examine h for memory and absolute summability or integrability for BIBO stability.

7. Determine H(jω) or H(e^{jω}) when frequency-selective behavior matters.

8. If recovery is required, examine the inverse for causality, stability, noise sensitivity, and implementation feasibility.

This procedure prevents a common error: treating a transfer function as a complete practical description without considering initial conditions, regions of convergence, or system realization.

Applied Case Study: Digital Sensor Smoothing

Background

Consider an illustrative industrial sensor that samples vibration at regular intervals. Individual measurements contain high-frequency electrical noise, while the engineering decision depends mainly on the slowly varying vibration level.

Situation and Problem

A three-sample moving-average filter is proposed:

y[n]=(x[n]+x[n−1]+x[n−2])/3.

The design team must determine its impulse response, causality, memory, stability, and limitations.

Analysis

The equation can be written as a convolution with

h[n]=1/3, n=0,1,2;

h[n]=0 otherwise.

The system is LTI because each input sample is multiplied by a fixed coefficient and the sum is linear; shifting the input shifts the output. It is causal because h[n]=0 for n<0. It has memory because y[n] depends on earlier samples. It is BIBO stable because Σ|h[n]|=1. Its frequency response is

H(e^{jω})=(1/3)(1+e^{−jω}+e^{−j2ω}).

The response has unit gain at zero frequency and attenuates selected higher-frequency components. Consequently, it smooths noise but also reduces rapid genuine changes.

Approach and Outcome

The filter is implemented as a finite impulse response structure. In practice, the previous two samples are stored, and each new output is computed from three additions and a scaling operation. The design is computationally simple and has predictable bounded behavior. However, it introduces delay in the interpretation of changing signals and cannot remove noise whose frequency overlaps the useful vibration band.

Learning Points

The case demonstrates that convolution is not an abstract calculation: it is the operation performed by a digital filter. The impulse response reveals implementation requirements, causality, memory, stability, and frequency selectivity simultaneously. Filter selection must therefore consider the signal spectrum and decision timing, not merely the fact that averaging reduces visible noise.

Limitations, Risks, and Common Errors

Several mistakes recur in LTI analysis. First, linearity is sometimes inferred merely because an equation looks algebraic; terms such as x²(t), |x(t)|, or x(t)y(t) invalidate linearity. Second, time invariance is confused with linearity. A linear system with time-dependent coefficients is not necessarily time invariant.

Third, convolution limits are often chosen incorrectly. The correct limits arise from the intersection of the supports of the shifted functions. Fourth, stability is incorrectly inferred from a finite value of H(0); stability requires the impulse-response integrability or summability condition, not merely a finite DC gain. Fifth, a stable system is assumed to have a stable inverse. Inverse filtering can amplify noise and may be noncausal.

Ideal impulses, infinite-duration signals, and exact poles are mathematical models. Physical systems have finite bandwidth, finite dynamic range, quantization, delays, and parameter uncertainty. Near instability, small coefficient changes or finite-precision arithmetic can produce substantial output growth. These engineering effects must be included when translating theoretical results into hardware or software.

Professional and Technological Relevance

LTI models remain central in circuit simulation, digital signal processing, radar, audio, biomedical instrumentation, image reconstruction, telecommunications, and control. Software tools calculate convolutions, frequency responses, pole-zero diagrams, and numerical stability margins. Fast convolution algorithms based on the discrete Fourier transform are essential when long signals or long impulse responses make direct computation expensive.

Modern systems may also contain adaptive, nonlinear, or time-varying components. An LTI approximation is then used locally around an operating point or over a restricted frequency and amplitude range. Engineers must state this scope explicitly. A model that is accurate for small perturbations may fail under saturation, switching, changing loads, or sensor clipping.

Best Practices

Use the impulse response as the primary bridge between a system model and its input-output behavior. State initial conditions when using differential or difference equations. Check support before evaluating convolution. Separate causality, memory, invertibility, and stability because none implies all the others. When designing an inverse or filter, inspect frequency response, noise amplification, delay, computational cost, and finite-precision effects. Finally, validate analytical conclusions with representative bounded inputs and limiting cases such as an impulse, a constant signal, and a sinusoid.

Chapter Summary

An LTI system obeys superposition and is invariant under time shifts. Its impulse response completely determines its zero-state response through convolution: an integral in continuous time and a sum in discrete time. Convolution can be interpreted as the accumulated weighted overlap of an input with a shifted impulse response and provides the mathematical basis of filtering and cascaded-system analysis.

Differential and difference equations describe many LTI systems, while transform methods provide transfer functions and frequency responses. Complex exponentials are eigenfunctions, so each frequency component is scaled and phase-shifted independently. Causality is determined by one-sided support of the impulse response; memorylessness requires an impulse concentrated at the origin; invertibility requires a convolutional inverse; and BIBO stability requires absolute integrability or summability. These properties must be assessed separately, especially when implementing filters or designing inverse systems.

Key Terms

Linearity: The property that a system obeys additivity and homogeneity.

Time invariance: The property that an input shift produces an identical output shift.

LTI system: A system that is both linear and time invariant.

Unit impulse: A generalized signal concentrated at one instant with unit area, or a discrete sequence equal to one at the origin.

Impulse response: The output of a system when the input is a unit impulse.

Convolution: The integral or sum that combines an input with an impulse response to produce the LTI output.

Zero-state response: The response caused by the input when initial stored energy is zero.

Transfer function: The transform-domain ratio of output to input under zero initial conditions.

Eigenfunction: An input whose system output is a scalar multiple of the same input.

Frequency response: The complex gain of an LTI system as a function of frequency.

Causal system: A system whose output does not depend on future input values.

Memoryless system: A system whose output at an instant depends only on the input at that instant.

Invertible system: A system for which the input can be uniquely recovered from the output.

BIBO stability: The property that every bounded input produces a bounded output.

FIR system: A discrete-time system with a finite-duration impulse response.

IIR system: A discrete-time system with an impulse response of infinite duration.

Natural response: The response caused by stored initial energy rather than the applied input.

Region of convergence: The transform-domain region in which a transform representation converges.

Pole: A value of the transform variable at which a transfer function becomes unbounded.

Support: The set of time or index values where a signal is nonzero.

Review Questions

Short Answer Questions

1. State the superposition condition for a linear system.

2. Define the impulse response of an LTI system.

3. Write the continuous-time and discrete-time convolution formulas.

4. What condition on h(t) indicates causality?

5. State the BIBO stability condition for a discrete-time LTI system.

Long Answer Questions

1. Derive the convolution integral from the representation of a signal as a continuous distribution of shifted impulses.

2. Explain how differential equations and transfer functions are used to obtain an LTI system’s impulse response.

3. Demonstrate why complex exponentials are eigenfunctions of LTI systems.

4. Compare causality, memory, invertibility, and BIBO stability, emphasizing why these properties are logically distinct.

5. Explain the operation and frequency response of a three-sample moving-average filter.

Analytical / Application Questions

1. Determine the convolution of two rectangular pulses of duration T and interpret the resulting waveform.

2. For h[n]=(1/2)^n u[n], determine whether the system is causal and BIBO stable, and justify both conclusions.

3. A system has H(e^{jω})=1+e^{−jω}. Determine its impulse response and explain its effect on a sinusoidal input.

22. Multiple Choice Questions with Answers

Question 1

Which condition is required for a system to be time invariant?

A. A scaled input must produce a scaled output

B. A delayed input must produce an equally delayed output

C. The output must be bounded for every input

D. The impulse response must have finite duration

Correct Answer: B. A delayed input must produce an equally delayed output

Question 2

For an LTI system, the response to x(t) is represented by

A. x(t)+h(t)

B. x(t)h(t)

C. ∫x(τ)h(t−τ)dτ

D. x(t)/h(t)

Correct Answer: C. ∫x(τ)h(t−τ)dτ

Question 3

An LTI system is causal when its impulse response satisfies

A. h(t)=0 for t>0

B. h(t)=1 for all t

C. h(t)=0 for t<0

D. ∫h(t)dt=0

Correct Answer: C. h(t)=0 for t<0

Question 4

Which impulse response describes a memoryless continuous-time LTI system?

A. u(t)

B. e^(−t)u(t)

C. δ(t−2)

D. Kδ(t)

Correct Answer: D. Kδ(t)

Question 5

The discrete-time LTI system with h[n]=(1/2)^n u[n] is

A. Noncausal and unstable

B. Causal and BIBO stable

C. Causal but not BIBO stable

D. Memoryless and stable

Correct Answer: B. Causal and BIBO stable

Question 6

Why are complex exponentials eigenfunctions of LTI systems?

A. They vanish outside a finite interval

B. Convolution changes their frequency into a different frequency

C. Convolution produces the same exponential multiplied by a scalar

D. They are always bounded for every complex exponent

Correct Answer: C. Convolution produces the same exponential multiplied by a scalar

Question 7

For a three-sample moving-average filter, the impulse response is

A. h[n]=1 for all n

B. h[n]=1/3 for n=0,1,2 and zero otherwise

C. h[n]=3δ[n]

D. h[n]=(1/3)^n u[n]

Correct Answer: B. h[n]=1/3 for n=0,1,2 and zero otherwise

Question 8

Which statement about invertibility is correct?

A. Every stable system has a stable inverse

B. An inverse must always be causal

C. An inverse must satisfy h*h⁻¹=δ

D. A noncausal system cannot be invertible

Correct Answer: C. An inverse must satisfy h*h⁻¹=δ

Question 9

A continuous-time LTI system is BIBO stable when

A. h(t) is periodic

B. ∫|h(t)|dt is finite

C. h(t) has nonzero value at t=0

D. H(0)=0

Correct Answer: B. ∫|h(t)|dt is finite

Question 10

The output of a causal LTI system at time n can depend on

A. Future input samples only

B. Past and present input samples

C. No input samples

D. Only the input sample at n if the system is stable

Correct Answer: B. Past and present input samples