Deep Learning for Signals, Images and Sequences

Introduction and Context

Electrical engineering systems generate heterogeneous data: voltage and current waveforms, vibration signals, thermal images, aerial inspection photographs, phasor streams, smart-meter records, and event logs. Traditional signal-processing methods remain valuable, but many modern applications require models that can learn useful representations directly from large, complex datasets. Deep learning provides such models through multilayer networks that transform raw or lightly processed measurements into predictions, classifications, reconstructions, or decisions.

This chapter develops the main deep-learning architectures used for electrical-engineering data: feedforward neural networks, convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory (LSTM) networks, gated recurrent units (GRUs), autoencoders, and transformers. The emphasis is not merely on architectural descriptions, but on matching an architecture to the physical structure of a problem while controlling overfitting, latency, memory, and computational cost.

Typical applications include:

• waveform classification for power-quality disturbances;
• image-based inspection of insulators, conductors, solar modules, and printed circuit boards;
• load and renewable-generation forecasting;
• fault and event recognition in industrial equipment;
• anomaly detection when labelled failures are scarce.

A successful engineering solution requires more than high test accuracy. Data leakage, class imbalance, temporal dependence, calibration, safety, explainability, and deployment constraints must also be addressed.

Learning Objectives

After studying this chapter, a student should be able to:

1. Explain how deep networks learn representations from electrical-engineering data.
2. Select CNN, RNN, autoencoder, or transformer architectures for signals, images, and sequences.
3. Formulate convolution, pooling, recurrent updates, attention, and common training losses.
4. Prepare datasets without temporal or equipment-level leakage.
5. Control overfitting through regularization, validation design, augmentation, and transfer learning.
6. Evaluate models using engineering-relevant metrics rather than accuracy alone.
7. Design a practical deep-learning workflow subject to computational and operational constraints.

1. Neural-Network Foundations

A neuron computes an affine transformation followed by a nonlinear activation:

Here, x is an input vector, w is a weight vector, b is a bias, z is the pre-activation value, φ is an activation function, and a is the output. The inputs may be normalized sensor values, image pixels, spectral coefficients, or engineered features.

Common activations include the rectified linear unit (ReLU), sigmoid, and hyperbolic tangent:

ReLU is widely used in hidden layers because it is computationally simple and reduces saturation for positive inputs. Sigmoid is suitable for a binary output probability, while softmax is commonly used for mutually exclusive multiclass classification:

where z_k is the logit for class k and the sum includes all classes. The outputs satisfy 0 ≤ p_k ≤ 1 and ∑ₖ p_k = 1.

For binary classification, binary cross-entropy is

where y ∈ {0,1} is the target and p is the predicted probability. For K-class classification, categorical cross-entropy is

where y_k is a one-hot target. For regression, mean squared error (MSE) is

where y_i and ŷ_i are the measured and predicted values, respectively. MSE has squared units; mean absolute error, MAE = (1/N)∑ᵢ|yᵢ − ŷᵢ|, has the same units as the target and is less sensitive to large outliers.

Training adjusts parameters θ to minimize a loss over training examples. In mini-batch gradient descent:

where η is the learning rate and L_B is the average loss for a mini-batch B. Backpropagation applies the chain rule to calculate gradients. In practice, optimizers such as stochastic gradient descent with momentum or Adam modify this basic update.

Data normalization

For a feature x, standardization is commonly defined as

where μ_train and σ_train are computed from the training set only. The same values must be applied to validation, test, and deployment data. Computing them using the full dataset leaks information from the test set.

2. Convolutional Networks for Signals and Images

2.1 The convolutional principle

A convolutional layer uses a small learnable kernel that scans across an input. For a one-dimensional discrete signal x[n], a cross-correlation operation, which is the operation usually implemented in neural-network libraries, can be written as

where K is the kernel length, w[k] are kernel weights, and b is a bias. A true mathematical convolution reverses the kernel; the distinction does not affect learning because the weights are learned.

For a two-dimensional image or feature map:

Channels are included in the practical operation. A layer learns multiple filters, each producing one output feature map. Early filters may respond to edges or oscillations; deeper filters combine these into patterns such as cracks, harmonic signatures, or partial-discharge structures.

If an input of length N is processed with kernel size K, stride S, padding P, and dilation D, the output length is

This formula applies to the standard discrete convolutional-layer geometry. For a two-dimensional layer, the formula is applied independently to height and width.

2.2 Pooling, stride, and receptive field

Max pooling retains the largest activation in a local region; average pooling computes its mean. Pooling can reduce computational cost and provide limited translation tolerance, but it may discard precise timing or location information. Strided convolution performs downsampling while learning the filtering operation.

The receptive field is the portion of the original input that can influence a unit. Stacking layers enlarges the receptive field. This matters in waveform analysis: a short receptive field may detect a local transient, whereas a larger one may identify a complete fault signature or several cycles of a power waveform.

For periodic electrical signals, the sampling frequency, window length, and kernel size must be interpreted physically. A 1,000-sample window at 10 kHz represents 0.1 s. A kernel of 25 samples represents 2.5 ms, so it may capture a transient but not a complete 50-Hz cycle, whose period is 20 ms.

2.3 CNNs for electrical waveforms

A one-dimensional CNN can process current, voltage, vibration, acoustic emission, or temperature sequences. Inputs may have shape (time, channels), where channels represent phases, sensors, or axes. Important design choices include:

• sampling rate and anti-aliasing;
• window duration and overlap;
• number of channels;
• kernel sizes expressed in physical time;
• causal versus noncausal processing;
• preservation of phase and amplitude information.

A CNN can classify a window as normal, sag, swell, interruption, harmonic distortion, or transient. It can also perform sequence-to-sequence denoising or regression.

2.4 CNNs for images

Image models generally use three-channel colour images or one-channel thermal, radiographic, or grayscale images. A final global average pooling layer reduces each feature map to one value and can replace a large fully connected layer, reducing parameter count. Transfer learning from a model trained on a large image dataset may be useful, but the source and target domains should be compared. Thermal images and visible-light images have different physical meanings; blindly transferring all assumptions can be harmful.

3. Recurrent Networks for Sequential Data

A recurrent network maintains a hidden state that summarizes previous observations:

where x_t is the input at time t, h_t is the hidden state, W_x and W_h are weight matrices, b_h is a bias, and φ is commonly tanh or another activation. An output may be produced as

The same parameters are reused at each time step, which allows variable-length sequences and reduces parameter growth relative to a separate network for every time index.

3.1 Vanishing and exploding gradients

During backpropagation through time, gradients involve repeated multiplication by recurrent Jacobian matrices. If their effective magnitudes are smaller than one, gradients can vanish; if larger than one, they can explode. Vanishing gradients make it difficult to learn long-term dependencies, while exploding gradients can destabilize training. Gradient clipping limits an update when its norm is too large. For a threshold c, a simple norm-clipping rule is

where g is the gradient vector and ||g|| is its norm. This changes the gradient only when ||g|| > c.

3.2 LSTM and GRU networks

An LSTM introduces gates and a cell state. One common formulation is

The symbols i_t, f_t, and o_t are input, forget, and output gates; c_t is the cell state; h_t is the hidden state; ⊙ denotes elementwise multiplication; and σ is the sigmoid function. The gates regulate information flow and improve long-range learning.

A GRU uses update and reset gates and has fewer state variables than an LSTM. It often trains faster, although neither architecture is universally superior.

RNNs are useful for load forecasting, sequential fault diagnosis, and event recognition. They can process data online, but recurrent computation is sequential across time, limiting parallelism and sometimes increasing latency.

4. Attention and Transformers

Attention allows a model to relate one position to other positions directly. Given query Q, key K, and value V matrices, scaled dot-product attention is

where d_k is the key-vector dimension. The scaling factor reduces excessively large dot products, which can otherwise make the softmax poorly conditioned.

A transformer typically contains multi-head attention, position information, residual connections, normalization, and a feedforward subnetwork. Since attention alone does not identify order, positional encodings or learned position embeddings are needed for sequences. Transformers can model long-range relationships and parallelize training across time, but standard self-attention has approximately quadratic memory and computation growth with sequence length. This is significant for high-rate waveforms.

Practical strategies include downsampling after appropriate anti-alias filtering, patching a waveform into segments, local or windowed attention, linear-attention approximations, and hierarchical models. A transformer is not automatically better than a CNN or LSTM; the choice depends on sequence length, dataset size, latency, and the required dependency range.

5. Autoencoders and Representation Learning

An autoencoder contains an encoder and decoder:

where x is the input, z is a lower-dimensional latent representation, and x̂ is the reconstruction. The reconstruction objective may be

for continuous signals under a squared-error assumption. A bottleneck forces the model to represent dominant structure, although a sufficiently powerful model may simply learn an identity mapping unless capacity or regularization is controlled.

A denoising autoencoder receives a corrupted input x̃ but is trained to reconstruct the clean input x. It can remove sensor noise or reconstruct missing measurements. An anomaly detector may train mainly on normal data and use reconstruction error

as an anomaly score. A threshold τ is selected using validation data, operational cost, and an acceptable false-alarm rate. Reconstruction error is not automatically a probability of failure; it may also increase because of sensor drift, a new operating regime, or a distribution shift.

Variational autoencoders impose a probabilistic latent structure and are useful for generation and uncertainty modelling, but their interpretation and calibration require care.

6. Designing a Deep-Learning Workflow

6.1 Define the engineering task

First specify the decision, prediction horizon, available measurements, and cost of errors. Event recognition may require a label at every time step, a label for a window, or a single label for an entire record. Load forecasting must define the horizon, such as 15 minutes ahead or day-ahead, and whether forecasts are point estimates or prediction intervals.

6.2 Acquire and partition data correctly

Randomly splitting overlapping windows from the same recording can produce severe leakage: nearly identical samples may appear in both training and test sets. Better strategies include splitting by time period, feeder, machine, site, customer, or inspection campaign. The split should reflect deployment. For a future forecasting task, training data must precede validation and test data chronologically.

Labels require engineering review. A fault label based on a protection relay may differ in timing from a sensor waveform. Clock synchronization, missing values, sensor saturation, calibration, and changes in instrumentation must be documented.

6.3 Establish baselines

A baseline may be a persistence forecast, seasonal average, logistic regression on engineered features, random forest, matched filter, or classical CNN. Baselines reveal whether deep learning provides meaningful value. For load forecasting, a persistence model predicts the latest measured load; for periodic demand, a same-hour-previous-day model may be stronger.

6.4 Handle imbalance

If failures are rare, accuracy can be misleading. A classifier that predicts “normal” for every sample may have high accuracy and zero usefulness. Precision, recall, F1 score, balanced accuracy, and precision–recall area are informative. For positive events:

where TP, FP, and FN denote true positives, false positives, and false negatives. The F1 score is

The operating threshold should be selected according to engineering consequences. A protective-monitoring application may prioritize recall, while an inspection system with expensive manual review may constrain false alarms.

Class-weighted loss, focal loss, oversampling, and targeted augmentation can help, but oversampling must be performed within the training set only. Synthetic fault data should preserve physically plausible amplitude, frequency, phase, and temporal properties.

6.5 Control overfitting

Overfitting occurs when a model fits training-specific patterns rather than generalizable physical relationships. Useful controls include:

• reducing model depth or width;
• weight decay, which penalizes large parameters;
• dropout, which randomly removes units during training;
• early stopping based on validation performance;
• data augmentation consistent with physics;
• transfer learning;
• cross-validation at the equipment or site level;
• careful feature and label auditing.

Dropout or augmentation must not alter a class-defining property. For example, arbitrary time reversal may be invalid for a causal transient, and amplitude scaling may destroy a fault category based on magnitude.

6.6 Evaluate beyond a single score

Classification should include a confusion matrix, class-specific recall, calibration, and threshold analysis. Regression should include MAE, RMSE, bias, and performance by operating regime. Root mean squared error is

with the same units as y. A model can have a good average score but fail during peak demand, extreme weather, switching events, or lightly represented equipment states.

Calibration examines whether predicted probabilities correspond to observed frequencies. For safety-related decisions, uncertainty estimates, abstention rules, and human review may be preferable to forced predictions.

7. Computational Cost and Deployment

The parameter count of a dense layer with n_in inputs and n_out outputs is

including one bias per output. A convolutional layer with kernel dimensions K_h × K_w, C_in input channels, and C_out filters has

These formulas exclude optional normalization parameters. Convolution usually uses fewer parameters than a dense layer connected to every pixel or time sample.

Deployment constraints include memory, inference latency, energy, connectivity, and cybersecurity. Quantization represents parameters or activations with fewer bits; pruning removes small or redundant connections; knowledge distillation trains a smaller student model to imitate a larger teacher. These techniques can reduce cost but may reduce accuracy or alter calibration. They should be validated on the actual target hardware, not only on a workstation.

For edge monitoring, a small 1-D CNN may outperform a transformer because it has predictable latency and low memory use. For a central server processing long multivariate records, a transformer or hybrid CNN-transformer may be justified. Causal architectures are necessary when future samples cannot be used at inference time.

8. Worked Design Example: Waveform Event Recognition

Consider a three-phase monitoring device sampling voltage and current at 12.8 kHz. The task is to identify normal operation, voltage sag, switching transient, and harmonic distortion.

A 10-cycle window at 50 Hz lasts

The number of samples per channel is

If six channels are used—three voltages and three currents—the input tensor has dimensions 2,560 × 6 for each window. A 1-D CNN may use kernels of fifteen samples. In physical time, this receptive element initially covers

This is suitable for local transient structure, while deeper layers combine information over longer intervals. Windows should be divided by recording session or feeder, not randomly after overlap. A chronological test set should contain later operating conditions.

Suppose the test confusion matrix for the “switching transient” class gives TP = 180, FP = 20, and FN = 30. Then

The model is therefore strong but misses approximately 14.3% of switching transients in this test subset. Before deployment, engineers should inspect whether missed events occur at particular voltage levels, loads, sensor locations, or signal-to-noise ratios.

A useful comparison would include a frequency-domain baseline using harmonic magnitudes and a smaller CNN. If the larger model improves F1 only marginally but doubles latency, the smaller model may be the better engineering choice.

9. Detailed Applied Case: Illustrative Solar-Module Inspection

Background

A utility operates a large photovoltaic plant. Periodic drone images are collected to identify hot spots, cracked modules, soiling, and normal panels. Thermal images vary with irradiance, wind, viewing angle, and module loading.

Problem

Manual inspection is slow, and a classifier trained on images from one inspection campaign may fail during another campaign. The engineering requirement is to flag suspicious modules for human review, not to authorize automatic replacement.

Analysis

The dataset is organized by inspection date and physical site. If adjacent modules from the same string appear in both training and test sets, spatial similarity may inflate performance. Therefore, entire strings or inspection campaigns are held out. Class counts are examined, and uncertain labels are reviewed by an electrical engineer.

Approach and solution

A transfer-learned CNN is used as a feature extractor, followed by a small classification head. Images are normalized using training-set statistics. Physically reasonable augmentation includes modest rotation, cropping, and brightness variation; augmentation that erases a small hot spot is restricted. A confidence threshold sends uncertain cases to manual inspection.

Because faults are rare, recall and precision–recall analysis are emphasized. The model is also tested separately under low irradiance, high wind, and different cameras. Grad-CAM-like saliency visualization is used as an inspection aid, but it is not treated as proof of causality. A thermal anomaly may result from a connector, shadow, sensor artifact, or environmental condition.

Outcome

In this illustrative case, the model reduces the number of panels requiring routine manual review while retaining a human decision for flagged modules. Performance is accepted only after site-level testing and comparison with the existing rule-based thermal threshold.

Learning points

1. The split must reflect the deployment environment.
2. Image classification is affected by measurement conditions, not only object appearance.
3. A high score does not establish the physical cause of a hot spot.
4. Human review, uncertainty thresholds, and maintenance records are part of the system design.

10. Limitations, Governance, and Contemporary Directions

Deep models may fail under distribution shift: a new sensor, firmware revision, climate, operating regime, or equipment population can change the data distribution. Monitoring should track input statistics, missingness, confidence, error rates, and false alarms after deployment. Retraining requires versioned data and reproducible evaluation.

Electrical-engineering applications may affect safety, reliability, privacy, and market operation. Smart-meter data can reveal occupancy patterns. Inspection images may contain sensitive infrastructure information. Access controls, encryption, retention limits, audit trails, and secure model-update procedures are essential. Models used in protection or control should be subject to formal verification, conservative fallback logic, and human or independent-system oversight where appropriate.

Explainability should be matched to the decision. Feature attribution can identify influential regions or time intervals, but it does not guarantee causal reasoning. Counterfactual explanations must obey physical constraints. A model should not be considered trustworthy merely because it produces a visually plausible explanation.

Current developments include self-supervised pretraining on unlabelled waveforms, multimodal models combining signals and maintenance text, physics-informed learning, neural operators, efficient attention, federated learning, and foundation models for time series. These methods are promising but do not remove the need for high-quality labels, physical validation, leakage prevention, and lifecycle monitoring.

Best-Practice Checklist

Before deployment, an engineering team should be able to answer the following:

• What exact decision does the model support?
• Are the train, validation, and test splits independent in the deployment sense?
• Were normalization and feature-selection operations fitted only on training data?
• Are labels time-aligned and physically meaningful?
• Is the model compared with a simple baseline?
• Are false positives and false negatives assigned realistic costs?
• Has performance been tested across sites, equipment, seasons, and operating regimes?
• Can the model meet latency, memory, energy, and cybersecurity requirements?
• Is uncertainty monitored, and is there a safe fallback or abstention mode?
• Are model, data, and software versions reproducible and auditable?

Chapter Summary

Deep learning converts raw electrical-engineering measurements into learned representations for classification, forecasting, reconstruction, and anomaly detection. CNNs exploit local structure in images and waveforms; RNNs, LSTMs, and GRUs represent sequential dependence; transformers use attention to model long-range relationships; and autoencoders learn compact or denoising representations. Architecture selection must follow the sampling structure, physical meaning, sequence length, data volume, and deployment constraints.

Reliable practice begins with a clearly defined engineering task, physically valid preprocessing, leakage-resistant dataset partitioning, and meaningful baselines. Overfitting is controlled through suitable capacity, regularization, augmentation, transfer learning, and validation at the equipment or site level. Evaluation should include class-specific metrics, calibration, operating-regime analysis, uncertainty, and computational cost. In safety- and infrastructure-related applications, governance, privacy, cybersecurity, human oversight, and post-deployment monitoring are integral parts of the design.

Key Terms

Neuron: A computational unit that applies a weighted sum, bias, and activation function.

Activation function: A nonlinear function applied to a neuron’s pre-activation.

Backpropagation: Gradient calculation using the chain rule through a network.

Convolution: A sliding local weighted operation used to detect patterns.

Receptive field: The input region that can influence a particular network unit.

Pooling: A downsampling operation such as maximum or average pooling.

CNN: A convolutional neural network designed to exploit local spatial or temporal structure.

RNN: A recurrent neural network that maintains a hidden state across time.

LSTM: A gated recurrent architecture with a memory cell for long-term dependencies.

GRU: A gated recurrent unit with a comparatively compact recurrent design.

Attention: A mechanism that weights relationships between positions in a sequence.

Transformer: An architecture primarily based on attention and position information.

Autoencoder: An encoder–decoder network trained to reconstruct its input or a target version.

Latent representation: A learned internal representation, often lower-dimensional than the input.

Overfitting: Learning training-specific patterns that do not generalize.

Data leakage: Unintended use of information from validation, test, or future data during training.

Calibration: Agreement between predicted probabilities and observed frequencies.

Quantization: Representing model values with reduced numerical precision.

Distribution shift: A change between training data and deployment data.

Gradient clipping: Limiting gradient magnitude to improve recurrent or unstable training.

Review Questions

Short Answer Questions

1. Why is standardization fitted on training data only?
2. State one advantage and one limitation of a CNN for waveform analysis.
3. What problem do LSTM gates help address?
4. Why does a transformer require positional information for ordinary time sequences?
5. Distinguish precision from recall.

Long Answer Questions

1. Explain the operation of a one-dimensional CNN for electrical waveform classification, including the roles of kernel size, stride, padding, and receptive field.
2. Compare CNN, LSTM, GRU, and transformer architectures for load forecasting and event recognition.
3. Describe a leakage-resistant workflow for developing an equipment-fault classifier.
4. Explain how autoencoders can be used for denoising and anomaly detection, including the limitations of reconstruction error.
5. Discuss methods for controlling overfitting and computational cost in an edge-deployed inspection system.

Analytical / Application Questions

1. A 25.6-kHz recorder captures a 50-Hz waveform. Design a window containing five cycles, calculate its sample count, and discuss whether a kernel of 32 samples can capture a complete cycle.
2. A rare-event classifier has TP = 72, FP = 8, and FN = 18. Calculate precision, recall, and F1, and recommend whether the operating threshold should be changed if missed events are considerably more costly than false alarms.
3. An LSTM performs well on historical data but fails after a sensor firmware update. Propose an investigation and remediation plan covering data drift, validation, retraining, monitoring, and safe deployment.

Multiple Choice Questions with Answers

Question 1

Which architecture most directly exploits local spatial or temporal patterns through shared filters?

A. Convolutional neural network

B. Standard fully connected network

C. Decision tree

D. Naive Bayes classifier

Correct Answer: A. Convolutional neural network

Question 2

In the formula N_out = floor((N + 2P − D(K−1) − 1)/S + 1), what does S represent?

A. Sampling variance

B. Stride

C. Number of output classes

D. Signal-to-noise ratio

Correct Answer: B. Stride

Question 3

What is the principal purpose of gates in an LSTM?

A. To convert images into RGB format

B. To regulate information entering, leaving, and remaining in the cell state

C. To eliminate the need for training data

D. To guarantee zero classification error

Correct Answer: B. To regulate information entering, leaving, and remaining in the cell state

Question 4

Why can randomly splitting overlapping waveform windows cause data leakage?

A. It removes all temporal information

B. Nearly identical portions of one recording may occur in both training and test sets

C. It always creates class balance

D. It prevents normalization

Correct Answer: B. Nearly identical portions of one recording may occur in both training and test sets

Question 5

For TP = 90, FP = 10, and FN = 20, what is the precision?

A. 0.75

B. 0.80

C. 0.90

D. 0.95

Correct Answer: C. 0.90

Question 6

Which loss is normally appropriate for a continuous regression target when squared-error penalties are desired?

A. Mean squared error

B. Categorical cross-entropy

C. Hinge loss only

D. Accuracy

Correct Answer: A. Mean squared error

Question 7

What is a major computational limitation of standard self-attention for very long sequences?

A. It cannot process numerical data

B. Its memory and computation generally grow quadratically with sequence length

C. It has no trainable parameters

D. It requires images rather than sequences

Correct Answer: B. Its memory and computation generally grow quadratically with sequence length

Question 8

An autoencoder trained mainly on normal equipment data is commonly used for anomaly detection by monitoring what quantity?

A. Reconstruction error

B. Kernel wavelength

C. Number of output classes only

D. Training-set name

Correct Answer: A. Reconstruction error

Question 9

Which practice best reduces the risk of site-specific overestimation of model performance?

A. Putting records from every site into every split

B. Splitting data by equipment or site when deployment is on unseen equipment or sites

C. Computing normalization statistics from the test set

D. Increasing overlap between train and test windows

Correct Answer: B. Splitting data by equipment or site when deployment is on unseen equipment or sites

Question 10

If a 12.8-kHz recorder observes a 50-Hz signal for 0.2 seconds, how many samples are collected?

A. 256

B. 1,280

C. 2,560

D. 12,800

Correct Answer: C. 2,560