Data-Driven Modelling, Governance and Emerging Practice

Introduction and context

Network Analysis and Synthesis traditionally begins with a physical description of a network: resistors, inductors, capacitors, controlled sources, ports, and interconnections. The governing equations are then obtained from Kirchhoff’s laws, constitutive relations, state-space models, or frequency-domain methods. Modern engineering practice adds a second source of knowledge: measured data. Voltage and current waveforms, impedance spectra, phasor measurements, switching records, environmental variables, and operational logs can be used to identify models, detect degradation, predict behaviour, and support design decisions.

Data-driven modelling does not replace physical network theory. Its most reliable form combines data with structural knowledge. Such hybrid modelling is particularly important when a network contains parasitic effects, nonlinear devices, ageing components, uncertain loads, or operating conditions that are difficult to represent exactly. At the same time, data introduce new responsibilities: measurement quality, privacy, cybersecurity, traceability, bias control, model validation, and governance.

In this chapter, a network means an interconnected electrical system represented by variables, parameters, topology, and input–output relations. The discussion applies to filters, power-electronic converters, power grids, battery networks, communication circuits, sensor networks, and electromechanical systems.

Learning objectives

After studying this chapter, the reader should be able to:

1. distinguish physics-based, data-driven, and hybrid network models;
2. formulate identification problems from measured electrical data;
3. select and validate models using appropriate error and generalisation criteria;
4. explain how uncertainty, data quality, and experiment design affect model reliability;
5. design a governance framework for network data and deployed models;
6. evaluate emerging uses of machine learning, digital twins, and automated synthesis critically;
7. apply these ideas to a realistic network-monitoring and synthesis problem.

1. From circuit equations to data-driven models

A physics-based model is derived from known laws. For a linear time-invariant network, a state-space representation may be written as

where x(t) is the state vector, u(t) is the input vector, y(t) is the measured output, and A, B, C, and D are matrices of compatible dimensions. The model assumes linearity and time invariance over the operating region, and the state variables must be chosen so that the system is adequately represented.

In the frequency domain, under zero initial conditions, the transfer matrix is

where s is the complex frequency and I is the identity matrix. This expression is applicable when the inverse exists and the state-space model is well defined. For a single-input, single-output system, G(s) is a scalar transfer function.

A data-driven model instead estimates a relationship from observations. Given sampled inputs u[k] and outputs y[k], a general predictor is

where fθ is a model class and θ is its parameter vector. The hat denotes a predicted or estimated value. The sampling interval must be sufficiently small to capture the relevant network dynamics, and the data must be time-aligned.

Three categories are useful:

A hybrid model may retain known dynamics while learning an unknown term:

Here p contains physical parameters and f_learned represents effects such as hysteresis, temperature dependence, parasitic coupling, or ageing. The learned term should not be allowed to violate essential constraints such as passivity, causality, dimensional consistency, or known safety limits.

2. What makes a network model acceptable?

A useful model is not merely one with a small training error. It should satisfy several properties.

Accuracy means that predicted voltages, currents, impedances, or power quantities agree with observations within an application-defined tolerance. Accuracy must be assessed on data not used to estimate the model.

Causality means that the output does not depend on future input. A physically implementable real-time model must obey this condition. Stability means that bounded inputs produce bounded outputs in the relevant operating region. For a continuous-time linear state model, asymptotic stability requires all eigenvalues of A to have negative real parts. For a discrete-time model x[k+1] = A_d x[k] + B_d u[k], the corresponding condition is that every eigenvalue of A_d lies inside the unit circle.

Passivity is important in impedance and admittance modelling. Informally, a passive network cannot generate net energy. For a one-port impedance Z(jω), passivity-related conditions include positive realness, subject to the adopted network convention and regularity assumptions. A fitted rational model that matches data but has nonphysical right-half-plane poles can destabilise a simulation or controller.

Interpretability concerns whether parameters and predictions can be explained in electrical terms. A resistance, pole, time constant, or coupling coefficient usually has more diagnostic value than an opaque feature vector.

Robustness concerns performance under noise, drift, missing sensors, changed loads, and operating conditions outside the training set. Robustness should be tested deliberately rather than assumed from a high coefficient of determination.

3. Data acquisition and experiment design

The quality of an identified model is constrained by the information contained in the experiment. Important design decisions include sensor placement, excitation amplitude, bandwidth, sampling frequency, duration, synchronisation, and operating-point coverage.

For a sampled signal with sampling frequency f_s, the Nyquist frequency is f_s/2. Frequencies above this limit can alias into lower frequencies unless suitable anti-alias filtering is applied. The sampling theorem requires a band-limited signal and adequate sampling; real engineering signals therefore require margin rather than operation exactly at the theoretical limit.

A practical experiment should:

• define the input and output variables and their units;
• calibrate sensors and record uncertainty specifications;
• synchronise channels and timestamp all records;
• excite the network over the frequency and amplitude range of interest;
• include normal, transient, and boundary operating conditions where safe;
• record temperature, load, control mode, and other contextual variables;
• preserve raw data before filtering or resampling.

Excitation may be a step, impulse approximation, pseudorandom binary sequence, multisine, swept sine, or naturally occurring operating variation. A multisine can efficiently probe several frequencies, while a transient experiment may reveal saturation and nonlinear recovery. The choice must reflect the intended model use: a small-signal filter model should not be identified solely from large-signal switching events.

A useful division is:

• training data, used to estimate parameters;
• validation data, used to select model structure or hyperparameters;
• test data, retained for final performance assessment.

Random splitting is unsafe for strongly autocorrelated time series because adjacent samples can place nearly identical information in training and test sets. Chronological or operating-condition-based splits are generally more credible.

4. Identification methods

4.1 Parametric identification

In parametric identification, the engineer assumes a model form and estimates a finite set of parameters. An autoregressive model with exogenous input, for example, can be written as

where c is an offset, aᵢ and bⱼ are coefficients, p and q are model orders, and e[k] is an unexplained error term. The units of coefficients depend on the scaling of variables. Parameter estimation commonly minimises a loss such as the mean squared error:

where N is the number of observations. J has the square of the output unit; it is not dimensionless unless the output has been normalised.

For a physically motivated equivalent circuit, parameters may be resistance R in ohms, inductance L in henries, and capacitance C in farads. A first-order RC time constant is

with units of seconds. Such parameters are easier to inspect and constrain than arbitrary regression coefficients.

4.2 Nonparametric and machine-learning models

Gaussian processes, kernel regression, tree ensembles, recurrent neural networks, and transformers can approximate complex relationships. They are useful when the state is only partially measured or the network has strong nonlinearities. However, high flexibility increases the risk of overfitting and poor extrapolation.

A neural network predictor may be expressed abstractly as

where z is a vector of measured features or time-history information. Standardisation of z can improve numerical conditioning, but the transformation must be recorded so that deployment uses the same procedure. A model should not be judged solely by training loss; regularisation, cross-validation designed for time series, and out-of-distribution tests are necessary.

4.3 Frequency-response and rational modelling

Measured frequency-response data are often fitted by a rational approximation:

where d is a direct term, rₖ are residues, pₖ are poles, and N is the model order. The variables have units determined by the input–output definition. For example, an impedance has units of ohms, whereas an admittance has units of siemens. The approximation is useful for simulation and control only after checking pole locations, stability, passivity, frequency range, and fit quality.

5. Validation, uncertainty, and model risk

For observations y[k] and predictions ŷ[k], common measures include the root mean square error

and the mean absolute error

RMSE emphasises large errors, while MAE is less sensitive to outliers. Neither measure alone establishes physical validity. Errors should be reported with units, operating conditions, confidence intervals where justified, and relevant baselines.

Residual analysis is essential. If residuals are correlated, the model has probably failed to capture dynamics. If residual variance changes with load or temperature, the model may be heteroscedastic. A residual plot against time, frequency, current, and temperature can reveal systematic deficiencies.

Uncertainty has several sources:

1. measurement uncertainty, caused by sensor resolution, calibration, noise, and timing;
2. parameter uncertainty, caused by finite and imperfect data;
3. model-form uncertainty, caused by an inadequate model class;
4. scenario uncertainty, caused by unknown future operating conditions.

A prediction interval is not the same as a confidence interval. A confidence interval concerns uncertainty in an estimated population quantity under a statistical model; a prediction interval concerns the likely range of a future observation and normally includes observation noise. Both require assumptions that should be stated.

Model risk is the possibility that a model produces an unacceptable decision or action because of error, misuse, drift, or invalid assumptions. In safety-critical networks, independent review, conservative operating limits, fallback controls, and documented approval gates are appropriate.

6. Governance of network data and models

Governance is the system of policies, roles, controls, and evidence through which data and models are managed. It is not limited to cybersecurity or legal compliance. It also concerns technical validity and accountability.

A governance framework should establish:

Data lineage is especially important. If a fitted impedance model is later used to approve a converter design, engineers should be able to identify the measurement campaign, sensor calibration, preprocessing code, estimator version, and validation results that produced it.

Cybersecurity and safety overlap. False measurements can cause a data-driven controller to change switching patterns or protection settings. Controls should therefore include authenticated acquisition, secure update mechanisms, anomaly detection, network segmentation, fail-safe modes, and manual override procedures. Security controls must be tested against realistic failure modes rather than documented only as intentions.

Privacy may arise when electrical data reveal household occupancy, industrial production schedules, or personal behaviour. Data minimisation, purpose limitation, access control, retention schedules, and aggregation should be applied where relevant. Compliance with applicable data-protection and critical-infrastructure requirements must be assessed for the jurisdiction and sector; governance is not satisfied by citing a generic standard without implementation evidence.

Fairness is relevant when models allocate reliability, maintenance, or service priority across customers or sites. A model that performs well on well-instrumented sites but poorly on rural or low-resource sites can produce systematic inequity. Performance should therefore be disaggregated by meaningful operating and deployment groups.

7. Data-driven synthesis and digital twins

In network synthesis, the objective is to construct a network whose response satisfies specifications. Classical synthesis uses positive-real functions, pole–zero constraints, Foster or Cauer forms, and physically realisable elements. Data-driven synthesis can assist by learning a mapping from specifications to candidate topologies or component values.

A safe workflow is constrained rather than purely generative:

1. define electrical specifications and admissible component libraries;
2. generate candidate topologies or parameters;
3. evaluate them using circuit simulation and physical constraints;
4. rank candidates using performance, cost, tolerance, thermal, and reliability criteria;
5. fabricate or prototype selected designs;
6. measure the prototype and update the model.

The learned proposal is therefore a search aid, not proof of realisability. Every candidate must be checked for causality, stability, passivity, component availability, tolerances, manufacturability, and compliance with applicable safety rules.

A digital twin is a continuously updated digital representation of a physical asset or network. It typically combines a structural model, live data, state estimation, and a lifecycle record. A twin is useful for what-if analysis, predictive maintenance, fault diagnosis, and commissioning. It is not automatically a faithful replica: sensor bias, missing states, communication delays, and model drift can make the digital representation misleading.

8. Illustrative case: hybrid ageing model for a battery-connected converter

Background

This fictional case concerns a 50 kW bidirectional converter connected to a lithium-ion battery pack. The engineering team has a small-signal equivalent-circuit model for control design, but the model becomes inaccurate as temperature and state of health change.

Problem

The original model predicts terminal impedance adequately near 25 °C and moderate state of charge, but it underestimates low-frequency impedance during cold operation. A controller tuned with this model produces slower-than-expected current regulation and may approach protection limits.

Analysis

The team collects synchronised current, voltage, temperature, and state-of-charge data from laboratory experiments and controlled field operation. Data are divided chronologically: early experiments for estimation, later experiments at unseen temperatures for validation, and a final field period for testing. Sensor calibration records and excitation amplitudes are retained.

A physics-based model contains an ohmic resistance and two polarisation branches. Temperature-dependent parameters are estimated using bounded least squares. A small neural network then learns the remaining residual as a function of temperature, state of charge, current history, and estimated state of health. The residual is constrained to zero at a reference operating point and is clipped to a validated domain.

Approach and solution

The team compares three models: the original fixed-parameter circuit, a fully data-driven recurrent model, and the hybrid model. The hybrid model has slightly higher training error than the recurrent model but lower validation error under unseen temperatures. Its parameters remain interpretable, and the residual network is disabled when inputs leave the validated domain. The controller uses conservative limits and falls back to the original model if sensor quality checks fail.

Outcome

The hybrid model reduces validation RMSE in terminal-voltage prediction from 42 mV to 18 mV over the tested operating envelope. The result is not claimed to apply outside that envelope. Field monitoring tracks residual error, temperature coverage, sensor health, and model version. A review is triggered if error exceeds the acceptance threshold for a defined period or if the battery chemistry changes.

Learning points

The case demonstrates that model quality depends on experiment design, operating-domain definition, and governance as much as on algorithm selection. Hybrid modelling preserves physical structure, while explicit fallback logic prevents an uncertain predictor from becoming an uncontrolled safety component.

9. Emerging practice and research directions

Several developments are shaping advanced network engineering.

Physics-informed learning incorporates differential equations, conservation laws, boundary conditions, or passivity penalties into training. This can reduce data requirements, but a poorly specified physical penalty can bias the model.

Operator learning seeks mappings between functions, such as excitation waveforms and responses, rather than only finite-dimensional vectors. It is promising for parameter sweeps and distributed networks but requires careful treatment of discretisation and generalisation.

Online and continual learning update models as new measurements arrive. Such systems need drift detection, bounded updates, rollback, and protection against contaminated or adversarial data.

Uncertainty-aware learning produces distributions or calibrated intervals rather than single predictions. Calibration must be evaluated on representative data; an interval that is wide but uninformative is not automatically useful.

Automated topology synthesis combines graph search, optimisation, simulation, and learned surrogates. Its central research challenge is ensuring that generated networks are physically realisable and robust to component tolerances.

Explainable and auditable AI is increasingly important in protection, grid operation, and industrial control. Explanations should be linked to engineering evidence, not treated as a substitute for validation. Feature-attribution plots, for example, do not prove causality.

10. Best-practice checklist

Before approving a data-driven network model, the engineer should ask:

• What physical quantity is predicted, and in what units?
• What operating domain and sampling conditions are covered?
• Are training, validation, and test records independent in an engineering sense?
• Have causality, stability, passivity, and conservation constraints been checked where applicable?
• What happens under missing, delayed, corrupted, or out-of-range data?
• What uncertainty is reported, and is it calibrated?
• Can the data, code, parameters, and model version be reproduced?
• Who is authorised to approve, modify, deploy, monitor, and retire the model?
• What is the fallback response when the model is unreliable?
• Has performance been assessed across relevant sites, temperatures, loads, and user groups?

A mature engineering organisation treats the model as a lifecycle artefact. Design, identification, validation, deployment, monitoring, updating, and retirement are connected activities rather than isolated software tasks.

Chapter Summary

Data-driven modelling extends network analysis by learning relationships from measured electrical behaviour. Physics-based models offer interpretability and physical extrapolation, while data-driven models can represent nonlinearities and effects that are difficult to derive analytically. Hybrid models often provide the most useful balance.

Reliable modelling requires carefully designed experiments, calibrated and synchronised measurements, appropriate time-series data separation, physical constraints, and validation under unseen operating conditions. RMSE and MAE are useful but incomplete indicators; residuals, uncertainty, stability, passivity, causality, and domain coverage must also be considered.

Governance provides ownership, provenance, quality control, access management, versioning, validation, monitoring, and retirement procedures. Cybersecurity, privacy, fairness, and safety are integral to engineering practice. Digital twins, physics-informed learning, continual learning, and automated synthesis offer significant opportunities, but none removes the need for physical verification and accountable professional judgement.

Key Terms

Data-driven model: A model estimated primarily from observed data.

Physics-based model: A model derived from physical laws and known device relationships.

Hybrid model: A model combining physical structure with learned components.

System identification: Estimation of a dynamic system model from input–output data.

Training data: Data used to estimate model parameters.

Validation data: Data used to select model structure or tuning settings.

Test data: Data reserved for final performance assessment.

Causality: Dependence of an output only on present and past inputs.

Passivity: A property associated with a network that does not generate net energy.

Model drift: Change in model performance caused by changes in data or the physical system.

Residual: Difference between an observed value and a model prediction.

RMSE: Square root of the mean squared prediction error.

Data provenance: Evidence describing data origin, processing, and history.

Data lineage: Traceability from source data through transformations to decisions.

Digital twin: A data-connected digital representation of a physical asset or system.

Model risk: Risk of harm caused by model error, misuse, or invalid assumptions.

Out-of-distribution condition: An operating condition not adequately represented in development data.

Physics-informed learning: Learning constrained or guided by physical laws.

Continual learning: Updating a model as new data become available.

Governance: Policies, roles, controls, and evidence governing data and models.

Review Questions

Short Answer Questions

1. Distinguish a physics-based model from a data-driven model.
2. Why is chronological splitting often preferable to random splitting for network time-series data?
3. Define causality and explain its importance in a real-time network model.
4. State two purposes of model governance.
5. What is the difference between validation data and test data?

Long Answer Questions

1. Explain the benefits and limitations of hybrid modelling for nonlinear electrical networks.
2. Discuss how excitation design, sampling, synchronisation, and calibration affect system identification.
3. Explain why accuracy metrics alone cannot establish the validity of a network model.
4. Develop a governance framework for a data-driven model used in converter control.
5. Evaluate the opportunities and risks of digital twins in network analysis and synthesis.

Analytical / Application Questions

1. A fitted impedance model has a low frequency-domain error but contains a right-half-plane pole. Analyse why this is unacceptable for many simulation and control applications and propose corrective checks.
2. Design an identification experiment for a temperature-dependent battery equivalent circuit. Specify measured variables, excitation, sampling considerations, data partitions, and validation tests.
3. A machine-learning model performs well at 25 °C but poorly at −10 °C. Diagnose possible causes and propose a governed hybrid-model solution, including fallback and monitoring measures.

Multiple Choice Questions with Answers

Question 1

Which model type explicitly combines physical equations with a learned correction term?

A. Purely empirical model

B. Hybrid model

C. Open-loop measurement

D. Static lookup table

Correct Answer: B. Hybrid model

Question 2

For a discrete-time linear state model, asymptotic stability requires the eigenvalues of the state matrix to be:

A. Outside the unit circle

B. On the imaginary axis

C. Inside the unit circle

D. Equal to one

Correct Answer: C. Inside the unit circle

Question 3

Which metric places greater emphasis on large prediction errors?

A. MAE

B. RMSE

C. Median absolute deviation only

D. Sample count

Correct Answer: B. RMSE

Question 4

Why can random splitting be misleading for strongly autocorrelated time-series data?

A. It changes sensor units

B. It can place nearly identical information in training and test sets

C. It removes all noise

D. It guarantees underfitting

Correct Answer: B. It can place nearly identical information in training and test sets

Question 5

What is the principal purpose of data provenance?

A. To increase sampling frequency automatically

B. To trace the origin and processing history of data

C. To replace model validation

D. To guarantee passivity

Correct Answer: B. To trace the origin and processing history of data

Question 6

For a first-order RC network, the time constant is:

A. R ÷ C

B. C ÷ R

C. RC

D. R + C

Correct Answer: C. RC

Question 7

Which property requires that a real-time model not depend on future inputs?

A. Causality

B. Linearity

C. Identifiability

D. Quantisation

Correct Answer: A. Causality

Question 8

A prediction interval primarily describes:

A. Only the uncertainty in a fixed parameter estimate

B. The likely range of a future observation under stated assumptions

C. The network’s nominal resistance

D. The sampling frequency

Correct Answer: B. The likely range of a future observation under stated assumptions

Question 9

Which action is most appropriate when deployed inputs leave the validated operating domain?

A. Ignore the condition

B. Automatically claim higher accuracy

C. Use defined fallback or conservative behaviour

D. Delete the original model

Correct Answer: C. Use defined fallback or conservative behaviour

Question 10

In data-driven network synthesis, a generated candidate should be accepted only after checking:

A. Its colour and file size

B. Physical realisability and relevant performance constraints

C. Whether it has the most parameters

D. Whether training error is exactly zero

Correct Answer: B. Physical realisability and relevant performance constraints