Data-Driven Mathematical Modeling and Emerging Practice

Introduction and Context

Engineering mathematics has traditionally represented physical systems through first-principles laws: Kirchhoff’s laws for circuits, Maxwell’s equations for electromagnetic fields, Newtonian mechanics, and differential equations for dynamic processes. These models remain indispensable. However, modern electronic and communication systems generate enormous quantities of data from sensors, network logs, embedded processors, satellites, mobile devices, and laboratory instruments. This has created a complementary modeling paradigm: data-driven mathematical modeling.

A data-driven model infers relationships, structures, or governing behavior from observations. The observations may be used alone, or together with physical laws. In electronics and communication engineering, applications include channel estimation, spectrum sensing, fault diagnosis, antenna optimization, adaptive filtering, traffic prediction, semiconductor process monitoring, and digital-twin development.

The central engineering challenge is not merely to fit a curve or train an algorithm. It is to construct a model that is mathematically defensible, physically plausible, computationally efficient, interpretable enough for its intended use, and reliable under conditions not represented in the training data.

Learning Objectives

After studying this chapter, the learner should be able to:

1. distinguish first-principles, empirical, data-driven, and hybrid models;
2. formulate a supervised learning problem using vectors, matrices, and objective functions;
3. explain regression, classification, regularization, validation, and uncertainty;
4. analyze time-series and dynamical-system models relevant to communication systems;
5. evaluate model quality using appropriate statistical and engineering metrics;
6. design a reproducible data-driven modeling workflow;
7. identify issues of identifiability, distribution shift, explainability, privacy, and safety;
8. critically assess emerging practices such as physics-informed learning, digital twins, and edge intelligence.

1. The Modeling Perspective

A mathematical model maps inputs to outputs or describes the internal evolution of a system. A static model may be written as

where x is the input vector, y is the output, θ is a vector of model parameters, and ε represents measurement noise, unmodeled effects, or random variation. The notation assumes that the relationship is sufficiently stable over the period of interest.

For a time-dependent system, the state-space form is more appropriate:

Here, xₜ is the hidden system state at time t, uₜ is a control or excitation input, yₜ is the measured output, wₜ is process disturbance, and vₜ is measurement noise. The functions g and h describe state evolution and measurement generation respectively.

1.1 Main classes of models

A model is not intrinsically “good” or “bad.” Its suitability depends on purpose, data quality, operating conditions, computational constraints, and the cost of incorrect predictions.

1.2 Forward and inverse problems

In a forward problem, known inputs and parameters are used to predict system outputs. In an inverse problem, measured outputs are used to estimate unknown parameters, sources, or states. Inverse problems are frequently ill-posed: a solution may not exist, may not be unique, or may be highly sensitive to measurement noise.

For example, estimating an unknown channel impulse response from transmitted and received sequences is an inverse problem. Regularization, prior physical knowledge, and careful experiment design are required to obtain a stable solution.

2. Data Representation and Preprocessing

Let a dataset contain n observations and p measured features. It is commonly represented by a matrix X ∈ ℝⁿˣᵖ and an output vector y ∈ ℝⁿ. Row i contains the feature vector xᵢᵀ and corresponding response yᵢ.

Before modeling, the engineer must determine:

• what each variable measures and in which units;
• whether samples are independent or temporally correlated;
• how missing values, outliers, saturation, and sensor faults arise;
• whether labels are reliable;
• whether training data represent the future operating environment;
• whether the sampling rate is adequate for the phenomena of interest.

2.1 Scaling

A common standardization transformation is

where μ is the training-set mean and σ is the training-set standard deviation. The transformation is applied feature by feature. It is appropriate when variables have different numerical scales and the learning algorithm is sensitive to scale. The values of μ and σ must be computed using the training data only; using test data causes information leakage.

For a variable measured in volts, amperes, or hertz, standardization is dimensionless, but the original units must remain documented for interpretation and deployment.

2.2 Training, validation, and test data

A dataset should generally be divided into:

• training data, used to estimate model parameters;
• validation data, used to select hyperparameters or compare candidate models;
• test data, used once, or very sparingly, for final performance estimation.

For time-series data, random shuffling can be invalid because future observations may influence the training set. Chronological splitting or rolling-origin evaluation is usually more defensible.

3. Regression as a Mathematical Learning Problem

In regression, the response is numerical. The linear model is

where X includes the measured features, β is the coefficient vector, and ε is a vector of random errors. The ordinary least-squares estimate minimizes the residual sum of squares:

If XᵀX is nonsingular, the solution is

This closed-form expression assumes a correctly specified linear mean relationship, finite observations, and an invertible design matrix. It does not require normally distributed errors for the coefficient estimate itself, although normality is useful for some classical confidence intervals and hypothesis tests.

3.1 Regularization

When features are correlated or p is large relative to n, least squares can be unstable. Ridge regression adds an L₂ penalty:

where λ ≥ 0 controls the penalty. With an unpenalized intercept and standardized predictors, its solution is

The penalty reduces variance but introduces bias. λ must be selected using validation or cross-validation, not by inspecting the test set.

Lasso regression uses an L₁ penalty:

where ||β||₁ is the sum of absolute coefficient values. Lasso can set some coefficients exactly to zero, which may aid feature selection, although correlated predictors can make the selected set unstable.

3.2 Evaluation metrics

For observed values yᵢ and predictions ŷᵢ, the mean absolute error is

and the root mean-square error is

Both have the same units as the response. RMSE weights large errors more strongly than MAE. A normalized metric may be useful when comparing signals with different scales, but the normalization convention must be reported.

The coefficient of determination is

where ȳ is the sample mean of the observed responses. R² measures improvement over the mean-prediction baseline on the evaluated data; it is not, by itself, proof of causality or physical validity.

4. Classification and Probabilistic Prediction

Classification assigns an observation to a category such as modulation type, fault class, or occupancy state. For binary logistic regression, the probability of class 1 is modeled by

where σ(a) = 1/(1 + e⁻ᵃ) is the logistic function. A threshold, often 0.5 but not necessarily so, converts probability into a class decision.

The binary cross-entropy loss for n observations is

where yᵢ ∈ {0,1} and pᵢ is the predicted probability. The logarithms require 0 < pᵢ < 1; implementations use numerical safeguards near the boundaries.

A confusion matrix gives true positives, false positives, true negatives, and false negatives. Important metrics include

Precision is important when false alarms are costly; recall is important when missed events are dangerous. Accuracy can be misleading in imbalanced datasets.

5. Time-Series and Dynamical Modeling

Electronic systems are rarely memoryless. A received communication signal, temperature trace, oscillator phase, or network load depends on earlier states. A discrete autoregressive model of order p is

where c is an intercept, φⱼ are autoregressive coefficients, and εₜ is an innovation term. Stationarity requires restrictions on the roots of the associated characteristic polynomial; simply fitting coefficients does not guarantee stationarity.

For a first-order scalar system,

a common weak-stationarity condition is |φ| < 1, assuming the innovations have finite variance and are appropriately uncorrelated over time.

5.1 State estimation

A state-space model separates hidden state evolution from measurement. In the linear Gaussian case, the Kalman filter recursively combines a prediction with a measurement update. The prediction equations are

where A and B are system matrices, P is state-estimation covariance, and Q is process-noise covariance. With measurement model yₜ = Cxₜ + vₜ and measurement-noise covariance R,

The filter assumes a linear model and Gaussian noise for its classical optimality interpretation. Extended and unscented variants address certain nonlinear systems, but their assumptions and failure modes must still be examined.

6. Neural and Kernel Models

A feed-forward neural network composes affine transformations and nonlinear activation functions:

where W and b are learned weights and biases, ϕ is an activation function, and ℓ indexes a layer. Training minimizes a loss function using an optimization method such as stochastic gradient descent.

Neural networks can approximate complex nonlinear mappings, but good interpolation does not ensure reliable extrapolation. The number of parameters, training data, architecture, regularization, and deployment environment all affect performance.

Kernel methods represent similarity through a kernel k(xᵢ,xⱼ). A radial basis function kernel is commonly written as

where σₖ > 0 controls the characteristic distance. Kernel methods can model nonlinear relationships without explicitly constructing high-dimensional features, but memory and computational costs may grow with dataset size.

7. A Rigorous Modeling Workflow

A defensible workflow is iterative rather than purely sequential.

1. Define the decision or engineering objective. Specify the output, prediction horizon, allowable error, latency, and cost of failure.
2. Formulate the data-generating process. Identify sensors, sampling mechanisms, operating regimes, and possible confounders.
3. Inspect and document data. Record units, calibration, missingness, timestamps, labels, and provenance.
4. Create a baseline. Examples include mean prediction, persistence prediction, linear regression, or a physics-only model.
5. Partition data appropriately. Prevent temporal, spatial, device-level, or user-level leakage.
6. Construct features or representations. Use domain knowledge, spectral transforms, lag variables, or learned representations.
7. Train and tune models. Compare models under the same evaluation protocol.
8. Quantify uncertainty and failure modes. Report confidence or prediction intervals where justified and test unusual conditions.
9. Validate physically and operationally. Check conservation laws, ranges, stability, latency, energy consumption, and safety constraints.
10. Deploy with monitoring. Track data drift, performance decay, calibration, and retraining triggers.

Reproducibility requires versioned data, code, preprocessing parameters, random seeds where relevant, model configurations, and a clear record of evaluation splits.

8. Worked Example: Predicting Received Signal Strength

Suppose an engineer wants to predict received signal strength, Pᵣ, in dBm from distance d in metres and obstruction count o. A small illustrative dataset is used to fit a linear model:

Assume the fitted coefficients, obtained from a training set, are β₀ = −42 dBm, β₁ = −0.08 dBm/m, and β₂ = −3.5 dBm/obstruction. For d = 50 m and o = 2,

The units are consistent: β₁d has units dBm, β₂o has units dBm because obstruction count is dimensionless, and the intercept has units dBm.

This prediction should not be interpreted as a universal propagation law. It is valid only to the extent that the training environment represents the deployment environment. Antenna height, frequency, polarization, multipath, weather, and shadowing may produce substantial errors. A physically informed alternative might include a path-loss term involving log₁₀(d), while a hybrid model could learn the residual between a propagation model and measured strength.

9. Detailed Applied Case: Predictive Maintenance of RF Power Amplifiers

Background

An illustrative telecommunications manufacturer operates RF power amplifiers in base-station equipment. Failure can cause service interruption, but replacing components too early increases maintenance cost.

Problem

The engineering team records drain current, heat-sink temperature, supply voltage, output power, reflected power, alarm history, and operating hours. The objective is to predict whether a critical fault will occur within the next seven days.

Analysis

The data are time-stamped and grouped by amplifier. A random row-wise split would allow measurements from the same physical unit to appear in both training and test sets, producing overly optimistic results. The team therefore uses unit-level separation and chronological validation. Faults are rare, so accuracy is not an adequate primary metric. Recall, precision, precision–recall curves, calibration, and expected maintenance cost are considered.

Features include rolling means and variances, temperature rise relative to ambient, reflected-power excursions, and recent alarm counts. All rolling features use only past observations. Missing sensor values are classified as either random communication loss or a possible symptom of failure; they are not indiscriminately replaced with a global mean.

Approach and solution

A persistence and rule-based alarm system provide baselines. A regularized logistic model is then compared with a gradient-boosted model. The final candidate is selected not solely by predictive score but by the cost of false negatives, maintenance capacity, inference latency, and interpretability. Probability calibration is assessed because technicians need trustworthy risk estimates.

A deployment policy may be expressed as:

where τ is chosen using operational cost, available technicians, and acceptable risk. The threshold need not be 0.5.

Outcome

The model is initially operated in a “silent” monitoring phase. Predictions are logged without triggering maintenance, allowing performance and data quality to be evaluated. After approval, it supports—not replaces—engineering judgment. Drift monitoring detects changes caused by a new amplifier revision and triggers model review.

Learning points

The case demonstrates that data partitioning, label timing, cost-sensitive decisions, calibration, human oversight, and post-deployment monitoring are as important as algorithm selection. A model that performs well in a laboratory may fail when sensors are recalibrated, firmware changes, or operating conditions shift.

10. Hybrid and Physics-Informed Modeling

Purely data-driven models may violate known physical constraints. Hybrid modeling incorporates a mechanistic model and learns an unknown component. If a known model predicts y_phys and the measured output is y, a residual model may be defined as

and a learner estimates r from operating conditions. The final prediction is ŷ = y_phys + r̂. This approach can reduce data requirements when the physical model captures dominant behavior.

Physics-informed neural networks and related methods include differential-equation residuals in an objective function. For a differential equation F[u](x,t) = 0, a simplified objective may be

where L_data measures mismatch to observations, L_phys measures violation of the governing equation, and λ_phys controls their relative weighting. The exact form must respect equation scaling, boundary conditions, initial conditions, and measurement uncertainty. Penalty methods do not automatically guarantee exact constraint satisfaction.

11. Identifiability, Uncertainty, and Generalization

A parameter is identifiable if distinct parameter values can be distinguished from available observations under the assumed model. Correlated inputs, insufficient excitation, and noisy measurements can make parameters practically non-identifiable even when a formal solution exists.

Uncertainty has at least two important sources:

• aleatoric uncertainty, arising from irreducible randomness or measurement noise;
• epistemic uncertainty, arising from limited data, model inadequacy, or unfamiliar operating conditions.

A point prediction alone conceals these distinctions. Prediction intervals, ensembles, Bayesian methods, conformal prediction, or calibrated probabilistic models may provide useful uncertainty information, but their coverage claims depend on assumptions such as exchangeability or adequate calibration data.

Generalization should be tested across the dimensions that matter operationally: time, device, geography, temperature, frequency band, workload, and user population. A single random test split may measure interpolation within a narrow regime rather than true deployment performance.

12. Ethics, Governance, and Professional Responsibility

Data-driven engineering has governance implications even when the application appears technical. Sensor data may reveal location, behavior, communication patterns, or personal information. Engineers should apply data minimization, access control, retention limits, encryption, and appropriate anonymization. Consent and regulatory obligations depend on jurisdiction and application.

Important professional practices include:

• documenting data provenance and permitted use;
• assessing bias caused by unequal coverage of environments or users;
• preserving audit trails for model versions and decisions;
• providing human override for safety-critical actions;
• testing adversarial or corrupted inputs;
• distinguishing correlation from causal evidence;
• reporting uncertainty rather than overstating precision.

In safety-critical communication infrastructure, a model should fail safely. Its fallback behavior, alarm policy, update procedure, and responsibility allocation must be specified before deployment.

13. Emerging Practice

Digital twins

A digital twin is a computational representation linked to a physical asset through data and models. In electronics, it may combine a circuit simulator, sensor streams, degradation models, and maintenance history. A useful twin must define synchronization, fidelity, uncertainty, and the decisions it supports; a three-dimensional visualization alone is not a digital twin.

Edge intelligence

Deploying models near sensors reduces latency and bandwidth use. However, edge devices impose restrictions on memory, energy, numerical precision, and update mechanisms. Quantization, pruning, knowledge distillation, and hardware-aware design can reduce resource requirements, but accuracy and robustness must be re-evaluated after compression.

Federated and privacy-preserving learning

Federated learning trains a shared model using multiple devices without directly centralizing raw data. It does not automatically guarantee privacy: updates may leak information, and participating devices may be heterogeneous or unreliable. Secure aggregation, differential privacy, and threat modeling may be necessary.

Automated machine learning and foundation models

Automated model selection can accelerate experimentation, but it does not replace problem formulation, leakage control, domain validation, or responsible deployment. Large pretrained models may assist signal interpretation, code generation, or scientific discovery, yet their outputs require verification and their computational and data requirements may be unsuitable for embedded systems.

14. Best Practices for Postgraduate Engineering Work

A strong modeling study should report:

1. the engineering question and operational decision;
2. data sources, units, sampling, labels, and exclusions;
3. preprocessing performed without leakage;
4. baseline models and reasons for selecting candidate methods;
5. validation design and uncertainty estimates;
6. performance on relevant subgroups and operating regimes;
7. computational cost, latency, memory, and energy requirements;
8. physical checks and failure analysis;
9. limitations and conditions under which the model should not be trusted;
10. deployment monitoring, governance, and retirement criteria.

The most important principle is proportionality: model complexity should be justified by a measurable improvement in the engineering objective, not by novelty alone.

Chapter Summary

Data-driven mathematical modeling uses observations to estimate relationships, states, parameters, or decisions. It complements rather than eliminates first-principles modeling. Regression, classification, regularization, time-series analysis, state-space estimation, neural networks, and kernel methods provide a range of tools, but each depends on assumptions and suitable validation.

Reliable practice begins with a clearly defined engineering objective and continues through data provenance, leakage-free partitioning, baseline comparison, uncertainty analysis, physical validation, and deployment monitoring. Hybrid and physics-informed models can combine scientific structure with data flexibility. Postgraduate engineers must also address identifiability, distribution shift, privacy, explainability, safety, computational constraints, and professional accountability. Emerging practices—including digital twins, edge intelligence, federated learning, and automated modeling—are valuable only when their limitations are explicitly managed.

Key Terms

Data-driven model: A model whose structure or parameters are inferred substantially from observed data.

First-principles model: A model derived from physical laws or conservation principles.

Hybrid model: A model combining mechanistic and learned components.

Feature: A measured or constructed input variable used by a model.

Target: The response or output that a supervised model predicts.

Regularization: A method that controls model complexity by adding a penalty to the objective.

Ridge regression: Regression using an L₂ coefficient penalty.

Lasso regression: Regression using an L₁ coefficient penalty.

Overfitting: Fitting noise or sampling-specific patterns rather than general relationships.

Data leakage: Unintended use of information unavailable at prediction time.

Distribution shift: A change between training and deployment data distributions.

State-space model: A model describing hidden state evolution and measurement generation.

Identifiability: The ability to distinguish unknown parameters from available observations.

Aleatoric uncertainty: Uncertainty due to inherent randomness or measurement noise.

Epistemic uncertainty: Uncertainty due to limited knowledge or data.

Calibration: Agreement between predicted probabilities and observed frequencies.

Digital twin: A data-linked computational representation of a physical asset or process.

Edge intelligence: Model inference or learning performed near the data source.

Physics-informed learning: Learning constrained or guided by physical equations or principles.

Concept drift: Change over time in the relationship between inputs and outputs.

Review Questions

Short Answer Questions

1. Distinguish a first-principles model from a data-driven model.
2. Why must scaling parameters be computed using training data only?
3. State one difference between MAE and RMSE.
4. What is data leakage in a time-series modeling problem?
5. Define epistemic uncertainty.

Long Answer Questions

1. Explain the complete workflow for developing and deploying a data-driven engineering model.
2. Derive the ordinary least-squares estimator and state the assumptions required for its use.
3. Compare ridge regression, lasso regression, and an unregularized linear model.
4. Explain the role of state-space models and Kalman filtering in electronic systems.
5. Discuss how physics-informed and hybrid models address limitations of purely data-driven methods.

Analytical / Application Questions

1. A fault-detection dataset contains 1% faulty observations. Explain why accuracy may be misleading and propose an evaluation and threshold-selection strategy.
2. Design a leakage-free validation scheme for predicting future channel quality across several communication devices. Justify the split and identify possible sources of distribution shift.
3. A learned temperature predictor violates the known maximum operating temperature of a device. Propose at least two mathematical or system-level interventions and discuss their trade-offs.

Multiple Choice Questions with Answers

Question 1

Which expression represents ridge regression for a linear model?

A. minimize ||y − Xβ||²₂ + λ||β||²₂

B. minimize ||y − Xβ||²₂ − λ||β||²₂

C. minimize ||y − Xβ||₁ + λ||β||₁

D. minimize ||y − Xβ||²₂ with λ fixed at infinity

Correct Answer: A. minimize ||y − Xβ||²₂ + λ||β||²₂

Question 2

For a binary classifier, recall is defined as:

A. TP/(TP + FP)

B. TP/(TP + FN)

C. TN/(TN + FP)

D. (TP + TN)/(TP + TN + FP + FN)

Correct Answer: B. TP/(TP + FN)

Question 3

Which data split is generally most appropriate for evaluating a model that predicts future time-series observations?

A. Randomly shuffle every observation before splitting

B. Use only the observations with the largest values

C. Use chronological or rolling-origin validation

D. Use the test set to select hyperparameters

Correct Answer: C. Use chronological or rolling-origin validation

Question 4

In the model xₜ = c + φxₜ₋₁ + εₜ, a common weak-stationarity condition is:

A. φ = 0 only

B. |φ| < 1

C. |φ| > 1

D. φ must be an integer

Correct Answer: B. |φ| < 1

Question 5

If the observed response is −55 dBm and the predicted response is −53 dBm, what is the absolute prediction error?

A. −2 dBm

B. 0 dBm

C. 2 dBm

D. 108 dBm

Correct Answer: C. 2 dBm

Question 6

Which metric is usually more strongly influenced by a few very large errors?

A. RMSE

B. MAE

C. Recall

D. Precision

Correct Answer: A. RMSE

Question 7

What is the principal purpose of a validation set?

A. To estimate parameters while hiding all training data

B. To select models or hyperparameters before final testing

C. To replace the need for a test set in every situation

D. To guarantee causal interpretation

Correct Answer: B. To select models or hyperparameters before final testing

Question 8

A hybrid residual model commonly predicts:

A. The difference between measured and physically modeled output

B. Only the sensor serial number

C. A random value independent of the physical model

D. The training-set size

Correct Answer: A. The difference between measured and physically modeled output

Question 9

Which statement best describes identifiability?

A. The speed of a processor during inference

B. The ability to distinguish parameter values from available observations

C. The number of classes in a confusion matrix

D. The amount of memory used by a neural network

Correct Answer: B. The ability to distinguish parameter values from available observations

Question 10

Why is a 0.5 probability threshold not always appropriate for fault detection?

A. Probabilities cannot be used for classification

B. The threshold should reflect error costs, class imbalance, and operational capacity

C. A threshold must always equal the training-set mean

D. Changing the threshold changes the measured sensor units

Correct Answer: B. The threshold should reflect error costs, class imbalance, and operational capacity