Data Analytics, AI and Cybersecurity

Introduction and Context

Modern electrical power systems generate data from supervisory control and data acquisition (SCADA), phasor measurement units (PMUs), smart meters, intelligent electronic devices (IEDs), weather services, market platforms and distributed-energy-resource controllers. This data supports state estimation, forecasting, asset management, protection supervision, demand response and cyber-incident detection.

The transition from conventional grids to cyber-physical smart grids creates both opportunity and risk. Analytics can improve reliability and economic operation, but inaccurate models, biased data, opaque algorithms and compromised communications can cause unsafe control actions. Consequently, data analytics, artificial intelligence (AI) and cybersecurity must be treated as an integrated engineering discipline rather than independent technologies.

Learning Objectives

After studying this chapter, the reader should be able to:

1. Distinguish descriptive, diagnostic, predictive and prescriptive analytics in power systems.
2. Formulate common forecasting, classification and anomaly-detection problems.
3. Evaluate models using technically appropriate metrics.
4. Explain how uncertainty enters AI-supported operational decisions.
5. Formulate a simple uncertainty-aware optimization problem.
6. Describe cybersecurity threats, controls and governance requirements for power utilities.
7. Design an integrated analytics and security workflow for a power-system application.

1. Analytics Architecture for Power Systems

A useful architecture has five layers:

Power-system data is heterogeneous. SCADA commonly has relatively slow and irregular measurements, whereas PMUs provide synchronized phasor measurements at much higher rates. Data must therefore be time-synchronized, quality-labelled and interpreted with awareness of topology, operating regime and measurement uncertainty.

A measurement vector can be represented as:

where z is the measurement vector, x is the system state, h(·) is the nonlinear measurement function determined by network physics, and e is measurement error. The equation assumes that the network model and measurement placement are known. In weighted least-squares state estimation, an estimate x̂ minimizes:

where R is the measurement-error covariance matrix. R must be positive definite (or appropriately regularized), and measurements must be sufficiently redundant for observability. This physical model is important because purely statistical models can produce plausible but electrically impossible results.

2. Forms of Analytics

2.1 Descriptive and diagnostic analytics

Descriptive analytics answers “what happened?” Examples include feeder peak demand, voltage-violation duration, transformer loading distribution and outage frequency. Diagnostic analytics asks “why did it happen?” It may correlate an outage with relay records, weather, vegetation, maintenance history or communication failures. Correlation does not by itself establish causation; engineering validation is required.

2.2 Predictive analytics

Predictive analytics estimates future or unobserved outcomes. Applications include:

• short-term load and renewable-power forecasting;
• transformer remaining-useful-life estimation;
• fault and disturbance classification;
• electricity-price prediction;
• cyber-anomaly detection.

For observations yᵢ and predictions ŷᵢ, common regression metrics are:

MAE is in the units of the target and treats all errors linearly. RMSE is also in target units but gives greater weight to large errors. ME indicates bias under the stated actual-minus-predicted convention: negative ME indicates average overprediction. Percentage metrics should be used cautiously when actual values are zero or close to zero.

For a binary classifier, let TP, TN, FP and FN denote true positives, true negatives, false positives and false negatives. Then:

Precision is important when unnecessary interventions are costly; recall is important when missed faults or attacks are dangerous. The F1 score is the harmonic mean:

Worked Example 1: Forecast-error interpretation

A feeder has actual and predicted demands, in kW, as follows:

The mean error is:

Thus, the forecast overpredicts by 1 kW on average. The MAE is:

The negative bias and nonzero MAE convey different information: the model is slightly biased high, but its typical absolute error is 6.5 kW.

2.3 Prescriptive analytics

Prescriptive analytics selects an action, not merely a prediction. In power systems, the action may be generator dispatch, battery charging, capacitor switching, demand-response activation or maintenance scheduling.

A deterministic economic-dispatch formulation is:

subject to:

where Cᵢ is generator cost, Pᵢ is generator output, D is demand and P_loss is network loss. Additional constraints may include ramp limits, reserve requirements, voltage bounds, line thermal limits and unit-commitment logic. A mathematically optimal result is not operationally acceptable unless the network model, constraints and data are credible.

3. AI Models and Their Engineering Use

3.1 Supervised learning

Supervised learning maps labelled inputs to outputs. Linear regression is interpretable and useful as a baseline:

The coefficients β are estimated from historical data. Its assumptions include an appropriate functional form, sufficiently independent errors for the intended inference, and stable relationships. Regularization can reduce overfitting. For ridge regression, the training objective is:

where λ ≥ 0 controls the penalty. A larger λ generally reduces coefficient magnitude but may increase bias.

Decision trees, random forests, gradient-boosting models and neural networks can represent nonlinear relationships. However, model complexity does not remove the need for feature validation, temporal cross-validation and physical plausibility checks.

3.2 Unsupervised learning

Clustering can group similar load profiles or operating conditions. Principal component analysis can reduce dimensionality, although its components may not correspond to physical variables. Autoencoders and density-based methods can identify unusual behaviour. An anomaly is not automatically a fault or cyberattack; it may result from a topology change, sensor replacement or unusual but valid weather.

3.3 Physics-informed and hybrid models

Hybrid models combine physical equations with machine learning. For example, a neural network may estimate a correction to a power-flow model rather than directly predicting all bus voltages. Such models can improve extrapolation and constrain outputs, but they require careful treatment of model mismatch and uncertainty. Physics-informed learning is not a substitute for validating the underlying network model.

4. Uncertainty-Aware Prediction and Decision Optimization

Point forecasts conceal uncertainty. A probabilistic forecast represents a conditional distribution p(yₜ₊ₕ | information available at time t), or provides quantiles. An interval [L, U] at nominal coverage 1 − α should satisfy approximately:

Coverage alone is insufficient: an interval can achieve coverage by being excessively wide. A useful interval score for central prediction intervals is:

where α is the total tail probability, I(·) is 1 when its condition is true and 0 otherwise. The score rewards narrow intervals but penalizes misses.

Chance-constrained operation

Suppose renewable generation W is uncertain and line flow F depends on W. A chance constraint requires:

where F_max is the thermal limit and ε is the permitted violation probability. This is meaningful only when the probability model is calibrated. If F is normally distributed with mean μ_F and standard deviation σ_F, a one-sided approximation is:

where z₁₋ε is the corresponding standard-normal quantile. The normal approximation is inappropriate for strongly skewed, heavy-tailed or multimodal flows; scenario-based or distributionally robust methods may then be preferable.

Worked Example 2: Risk-aware dispatch decision

A battery can discharge during a 1-hour interval. Each additional discharged kWh reduces expected energy cost by £0.12, but the operator assigns a risk cost of £0.20 per kWh for energy shortfall. Two actions are available:

• Conservative action: discharge 0 kWh; expected shortfall 0 kWh.
• Aggressive action: discharge 10 kWh; expected shortfall 3 kWh.

The net expected benefit of the aggressive action is:

If the risk cost rises to £0.50/kWh, then:

The conservative action is then preferred. This illustrates that a forecast does not determine an action independently of risk preference. In a full stochastic program, decisions made before uncertainty is revealed are separated from recourse decisions made afterward:

where x is the first-stage decision, ξ is the uncertain outcome, cᵀx is its cost and Q is the optimal recourse cost. Scenario probabilities, dependence between weather and demand, feasibility and computational burden must be explicitly documented.

5. Cybersecurity of Electrical Power Systems

Power-system cybersecurity protects confidentiality, integrity and availability, with integrity and availability often having immediate safety significance. Additional objectives include authenticity, accountability, resilience and safety.

5.1 Threats and attack surfaces

Important threats include:

• phishing and credential theft;
• malware and ransomware in enterprise or operational networks;
• denial-of-service and communication flooding;
• false-data injection into measurements;
• manipulation of time synchronization;
• insecure remote access;
• compromised firmware or supply chains;
• insider misuse;
• model poisoning and adversarial examples against AI systems.

A false-data attack may attempt to alter measurements so that an estimator produces a misleading state. If an attacker changes measurements by a vector a, then z′ = z + a. In simplified linear estimation, attacks aligned with the measurement-model subspace can be difficult to detect using only residual tests. Consequently, redundancy, secure phasor timing, topology validation, behavioural monitoring and independent physical checks are needed.

5.2 Defence-in-depth

A defensible architecture includes network segmentation, least privilege, multifactor authentication where operationally safe, secure configuration, patch and vulnerability management, application allow-listing, encrypted and authenticated communications, immutable logging, backup and recovery, security monitoring and tested incident response.

Controls must respect operational constraints. A patch that is safe for an office workstation may be unsafe for a protection relay without compatibility testing. Safety instrumented functions and emergency controls require independent analysis and fail-safe design.

5.3 AI-specific security and governance

AI systems introduce additional risks:

• training-data poisoning;
• evasion through crafted measurements;
• data leakage from models or logs;
• concept drift as operating conditions change;
• automation bias and overreliance on recommendations;
• non-reproducible model updates.

A model registry should record data lineage, feature definitions, training period, hyperparameters, validation results, uncertainty calibration, approval status and rollback version. Human operators need interpretable evidence, confidence information and clear authority to reject an automated recommendation.

6. Integrated Operational Workflow

A practical workflow is:

1. Define the operational decision and consequence of error.
2. Identify measurements, labels, latency and data-quality requirements.
3. Establish a physics-based baseline.
4. Clean and align data without leaking future information into training.
5. Split data chronologically where time dependence exists.
6. Train, calibrate and stress-test candidate models.
7. Evaluate technical metrics and operational consequences.
8. Integrate uncertainty into optimization or operator displays.
9. Conduct cybersecurity, safety and failover assessments.
10. Deploy in shadow mode before closed-loop use.
11. Monitor drift, bias, calibration, latency and security events.
12. Revalidate after topology, market, sensor or software changes.

Random train-test splitting is often misleading for time-series data because adjacent observations are correlated and future information may leak into the training set. Evaluation should include seasonal, extreme-weather and topology-change cases.

7. Illustrative Case Study: Distribution Feeder with Solar PV

Background

This illustrative case concerns a 33-kV distribution feeder with high rooftop photovoltaic penetration, a community battery and an automated voltage-control scheme.

Problem

Midday reverse power flow causes voltage excursions, while cloud transients create rapid ramps. A forecasting model also receives data through a remote connection that is vulnerable to credential misuse.

Analysis

The utility combines smart-meter demand, inverter telemetry, irradiance forecasts and feeder state estimation. Historical records show that a low average forecast MAE hides much larger errors during cloud transients. Security logs show repeated failed remote logins and anomalous configuration requests.

Approach

A hybrid model produces probabilistic PV forecasts. The dispatch optimizer minimizes energy cost, battery degradation and a penalty for predicted voltage violations. It imposes inverter apparent-power limits, battery state-of-charge bounds and a chance constraint on feeder voltage. The security design places remote access behind a controlled gateway, uses role-based privileges, monitors command sequences and requires independent approval for protection-setting changes. The AI model first operates in shadow mode, and a conventional rule-based controller remains available as a fallback.

Outcome

In the illustrative deployment, operators report fewer nuisance voltage-control actions because the system distinguishes forecast uncertainty from genuine violations. The principal benefit is not simply a lower forecast error; it is safer decisions under uncertain forecasts. Cybersecurity monitoring also identifies suspicious access before a control command is executed.

Learning Points

1. Prediction accuracy must be evaluated during critical operating regimes.
2. Probabilistic forecasts are useful only when calibrated.
3. Prescriptive optimization must include electrical, economic and security constraints.
4. AI should initially advise or supervise rather than directly control safety-critical equipment.
5. Fallback operation and human accountability are essential.

8. Limitations, Ethics and Regulation

Analytics can reproduce historical inequities, for example by allocating demand-response burdens disproportionately across customer groups. Data minimization, purpose limitation, access control and retention policies are necessary when customer-level data is used. Privacy-preserving aggregation may reduce detail but can protect consumers.

Utilities must maintain evidence of model validation, change control, access decisions and incident response. Relevant requirements depend on jurisdiction and may include critical-infrastructure cybersecurity standards, privacy law, sector-specific reliability rules and organizational risk-management frameworks. Compliance is a minimum; a compliant system may still be operationally fragile.

Important limitations include nonstationarity, rare-event scarcity, label uncertainty, sensor faults, adversarial adaptation, computational latency and the gap between benchmark performance and field performance. Research should therefore report confidence intervals, data provenance, baseline comparisons, external validation and failure cases rather than only a single accuracy value.

9. Contemporary Developments and Best Practices

Current research includes graph neural networks for network-structured data, federated learning across utilities, digital twins, foundation models for engineering documents, secure multiparty analytics and distributionally robust optimization. These approaches remain subject to data-quality, explainability, computational and governance constraints.

Best practice is to:

• combine data-driven and physics-based validation;
• report MAE, RMSE, bias and regime-specific performance;
• calibrate and monitor predictive uncertainty;
• optimize expected cost together with reliability and security risk;
• maintain strict separation between advisory and autonomous control functions;
• use adversarial testing and red-team exercises;
• document model lineage and rollback procedures;
• test rare but consequential scenarios;
• involve protection, operations, data, safety and cybersecurity specialists.

Chapter Summary

Data analytics converts power-system measurements into descriptive, diagnostic, predictive and prescriptive information. State estimation demonstrates why physical models and measurement uncertainty matter. AI methods can support forecasting, classification, anomaly detection and asset management, but require careful validation against temporal leakage, drift and physically impossible outputs. Metrics such as MAE, RMSE, bias, precision and recall must be selected according to operational consequences.

Prescriptive analytics links forecasts to actions through constrained optimization. Uncertainty-aware methods, including probabilistic forecasts, chance constraints and stochastic programming, allow operators to balance economic benefit against reliability risk. Cybersecurity must protect data, communications, models and control functions through defence-in-depth, secure governance and tested recovery. Safe deployment requires human accountability, fallback control, monitoring and continuous revalidation.

Key Terms

Data analytics: Systematic examination of data to obtain useful information.

Descriptive analytics: Analysis of what has occurred.

Diagnostic analytics: Analysis of causes or contributing factors.

Predictive analytics: Estimation of future or unobserved outcomes.

Prescriptive analytics: Selection of actions under objectives and constraints.

SCADA: Supervisory control and data acquisition system.

PMU: Device providing time-synchronized voltage and current phasors.

State estimation: Estimation of network operating state from measurements and a system model.

Feature: Input variable supplied to an analytical model.

Concept drift: Change in the statistical relationship learned by a model.

Calibration: Agreement between predicted probabilities or intervals and observed frequencies.

Chance constraint: Constraint required to hold with a specified probability.

False-data injection: Deliberate alteration of measurements to mislead monitoring or estimation.

Defence-in-depth: Multiple complementary security protections.

Least privilege: Granting only the access required for a task.

Model governance: Controls for approval, monitoring, documentation and change management.

Adversarial example: Input deliberately modified to mislead an AI model.

Digital twin: Dynamic computational representation of a physical system.

Review Questions

Short Answer Questions

1. Distinguish descriptive, predictive and prescriptive analytics.
2. Why can a negative mean forecast error indicate overprediction?
3. State two reasons why random train-test splitting can be unsuitable for power-system time series.
4. What is the purpose of a chance constraint?
5. Name four defence-in-depth cybersecurity controls.

Long Answer Questions

1. Explain the role of physical models in AI-supported state estimation and anomaly detection.
2. Compare MAE, RMSE, precision and recall in terms of engineering interpretation.
3. Discuss how probabilistic forecasting can improve renewable-energy dispatch.
4. Explain the principal cybersecurity risks introduced by AI models.
5. Design a governance process for deploying an AI recommendation system in a control centre.

Analytical / Application Questions

1. A forecast has errors of −8, 4, 6 and −2 MW under the actual-minus-predicted convention. Calculate ME and MAE and interpret both results.
2. A battery action provides an expected saving of £2.40 but creates an expected shortfall of 4 kWh. Determine whether it is preferred when the risk penalty is £0.50/kWh, and state the assumptions of the calculation.
3. Design an evaluation plan for a feeder cyber-anomaly detector that must distinguish sensor failure, topology change and malicious false-data injection.

Multiple Choice Questions with Answers

Question 1

For the worked feeder data in this chapter, what is the mean error using actual minus forecast?

A. −1 kW

B. 1 kW

C. 6.5 kW

D. −6.5 kW

Correct Answer: A. −1 kW

Question 2

For actual demands 100 kW and 120 kW with forecasts 110 kW and 114 kW, what is the MAE?

A. 2 kW

B. 6 kW

C. 8 kW

D. 10 kW

Correct Answer: C. 8 kW

Question 3

Which metric gives greater relative weight to large errors because it squares residuals before averaging?

A. MAE

B. ME

C. RMSE

D. Precision

Correct Answer: C. RMSE

Question 4

If TP = 90 and FP = 10, the precision is:

A. 0.10

B. 0.50

C. 0.90

D. 0.99

Correct Answer: C. 0.90

Question 5

A negative mean error under the convention y − ŷ most directly indicates:

A. Average underprediction

B. Average overprediction

C. Zero absolute error

D. Perfect calibration

Correct Answer: B. Average overprediction

Question 6

Which statement best describes prescriptive analytics?

A. It stores raw sensor readings.

B. It summarizes historical events only.

C. It selects actions subject to objectives and constraints.

D. It removes all uncertainty from forecasts.

Correct Answer: C. It selects actions subject to objectives and constraints.

Question 7

In a chance constraint P(F ≤ Fmax) ≥ 1 − ε, ε represents:

A. The maximum permitted violation probability

B. The forecast mean

C. The line resistance

D. The measurement sampling interval

Correct Answer: A. The maximum permitted violation probability

Question 8

Which control most directly limits the damage caused by compromised user credentials?

A. Increasing display brightness

B. Least privilege and multifactor authentication

C. Increasing forecast horizon

D. Removing event logs

Correct Answer: B. Least privilege and multifactor authentication

Question 9

Why is shadow-mode deployment useful for a new AI control recommendation?

A. It allows evaluation without immediately applying automated commands.

B. It guarantees zero model error.

C. It eliminates the need for cybersecurity testing.

D. It replaces physical network models.

Correct Answer: A. It allows evaluation without immediately applying automated commands.

Question 10

In a state-estimation measurement model z = h(x) + e, e denotes:

A. The estimated state

B. The measurement-error term

C. The network topology

D. The generator cost

Correct Answer: B. The measurement-error term