Parameter Estimation, System Identification and AI

Introduction and Context

A mechatronic system combines mechanical, electrical, electronic, sensing, control, and computational subsystems. Its simulation model is useful only when its parameters and structure represent the physical device with sufficient accuracy. A motor model with an incorrect resistance, inertia, friction coefficient, or torque constant may predict apparently plausible behavior while producing unsafe controller designs. Parameter estimation and system identification provide systematic methods for learning these quantities from experimental data.

Parameter estimation assumes that the model structure is known, at least approximately, and determines unknown numerical values. System identification is broader: it includes selecting a model structure, designing experiments, estimating parameters, validating the model, and deciding whether the model is adequate for its intended use. Artificial intelligence extends this toolbox through machine-learning models, physics-informed neural networks, Gaussian processes, and data-driven surrogate models. These methods can represent nonlinearities and unmodeled dynamics, but they introduce concerns about interpretability, bias, extrapolation, data quality, and governance.

The central engineering principle is that a model should be judged not by its predictive accuracy alone, but by whether it is accurate for a defined purpose, operating range, time scale, and decision. A model for controller design may require reliable phase and delay characteristics, whereas a model for fault diagnosis may prioritize sensitivity to abnormal behavior.

Learning Objectives

By the end of this chapter, the reader should be able to:

1. Distinguish parameter estimation, system identification, machine learning, and surrogate modeling.
2. Formulate regression-based estimators and interpret their assumptions.
3. Explain optimization-based estimation, constraints, regularization, and uncertainty.
4. Design an identification experiment and distinguish training, validation, and test data.
5. Identify common linear and nonlinear dynamic models for mechatronic systems.
6. Evaluate physics-informed, machine-learning, and hybrid models.
7. Diagnose overfitting, bias, poor excitation, leakage, and weak generalization.
8. Apply these methods to an engineering case involving a motor-driven mechanism.
9. Discuss interpretability, safety, ethics, and responsible deployment of AI in engineering.

1. Model Parameters, States, and Measurements

A continuous-time state-space model can be written as

Here, x(t) is the state vector, u(t) is the input vector, y(t) is the measured output, θ is the parameter vector, and f and h describe the process and measurement relationships. The terms w(t) and v(t) represent process and measurement disturbances. In a deterministic model these terms may be omitted; in a statistical model they are treated as random variables or stochastic processes.

A discrete-time representation is

where k indexes samples taken at interval T_s. The sampling interval must be sufficiently short relative to the fastest dynamics that matter to the application. Sampling too slowly can cause aliasing and loss of information; sampling much faster than necessary can increase data volume and computation without improving identification.

Parameters differ from states. A state, such as shaft speed, changes during operation and is often estimated online. A parameter, such as inertia J or resistance R, is usually constant or slowly varying for a particular operating condition. In practice, some parameters are time-varying because of temperature, wear, lubrication, battery state, or payload changes.

Example: Electromechanical actuator model

For a permanent-magnet DC motor coupled to a rigid load, a simplified model is

where L is armature inductance in henries, R is resistance in ohms, i is current in amperes, v is voltage in volts, k_e is the back-emf constant, ω is angular speed in radians per second, J is equivalent rotational inertia in kg·m², b is viscous friction in N·m·s/rad, τ_c is Coulomb friction torque in N·m, k_t is torque constant in N·m/A, and τ_L is external load torque in N·m. The sign function is discontinuous at zero speed, so it is often replaced in numerical optimization by a smooth approximation such as tanh(ω/ε), where ε controls the smoothing region.

The equations assume a rigid coupling, approximately constant parameters, negligible magnetic saturation, and a motor convention in which k_e and k_t are numerically consistent in SI units. These assumptions must be checked before interpreting estimated values physically.

2. Regression-Based Parameter Estimation

Regression is especially effective when the model can be rearranged to be linear in unknown parameters. Suppose measurements provide an equation of the form

where φ_k is a known regression vector, θ is the unknown parameter vector, and ε_k is measurement or modeling error. Stacking N observations gives

where y is an N × 1 output vector and Φ is the N × p regression matrix.

Ordinary least squares

The ordinary least-squares estimate minimizes the sum of squared residuals:

If Φ has full column rank, the solution is

The inverse exists only when the columns of Φ are linearly independent. If measurements have unequal variances, weighted least squares is more appropriate:

where W is a positive-definite weighting matrix. For independent measurements with variance σ_k², a common choice is W = diag(1/σ_1², …, 1/σ_N²).

The least-squares estimator is unbiased under conditions including a correctly specified linear-in-parameters model, zero-mean errors, and errors uncorrelated with the regressors. In dynamic systems, these conditions can fail because regressors may contain measured outputs affected by noise. This creates errors-in-variables and biased estimates.

Example: Estimating viscous friction

Suppose a motor runs at constant speeds and the measured torque required to maintain speed is modeled as τ = bω. Measurements are:

For a model through the origin, the least-squares estimate is

The numerator is 10×0.42 + 20×0.81 + 30×1.19 = 54.9. The denominator is 10² + 20² + 30² = 1400. Therefore,

The estimate is meaningful only if external load torque is known or negligible, acceleration is approximately zero, and Coulomb friction and other losses do not dominate. If a Coulomb term is important, the model should include an intercept or separate positive- and negative-speed terms rather than forcing all torque to be viscous.

Recursive least squares

For online estimation, recursive least squares updates the estimate as each new sample arrives. A standard form is

Here P is the parameter covariance or uncertainty matrix and λ is the forgetting factor, normally 0 < λ ≤ 1. λ = 1 gives no intentional forgetting; a smaller value allows adaptation to slowly changing systems but increases sensitivity to noise. Initial P should express prior uncertainty and must use units consistent with the parameter scaling.

3. Optimization-Based Estimation

Many useful models are nonlinear in parameters or states. Let a simulation produce predicted outputs ŷ_k(θ), and define residuals

A nonlinear least-squares estimate solves

where Q_k weights output components and sampling instants. If position is measured in radians and current in amperes, Q must account for their different scales; otherwise the numerically larger quantity may dominate the objective without being more important physically.

Engineering constraints can be included:

For example, inertia and resistance must be positive. Bounds can prevent physically impossible estimates, but inappropriate bounds may conceal model errors by forcing the optimizer to a boundary.

Regularization

When data are noisy, parameters are weakly identifiable, or a flexible model is used, regularization can improve stability:

Here λ ≥ 0 controls the penalty, L defines which parameter combinations are penalized, and θ₀ is a prior value. With L equal to the identity, the method is ridge-type regularization. Regularization introduces bias deliberately to reduce variance and ill-conditioning. Its value should be selected using validation data, physical reasoning, or a probabilistic prior—not simply chosen to obtain a visually attractive fit.

Optimization methods include gradient-based algorithms, Gauss–Newton, Levenberg–Marquardt, sequential quadratic programming, and global or derivative-free methods. Local methods are efficient when the model is smooth and the initial guess is reasonable. Global methods may help with multimodal objectives but can be computationally expensive. A robust workflow uses physically informed initial values, multiple initializations, scaled variables, and inspection of convergence diagnostics.

4. Identifiability and Experiment Design

A parameter is identifiable if the available input-output data contain enough information to distinguish its effect from the effects of other parameters. Structural identifiability concerns the ideal model and noiseless observations. Practical identifiability also considers finite data, noise, sampling, actuator limits, and operating conditions.

For linear regression, poor identifiability appears as an ill-conditioned Φ^TΦ. A common diagnostic is the condition number, defined as the ratio of the largest to smallest singular value of Φ. A large ratio indicates that some parameter combinations are weakly distinguished. Singular-value decomposition is generally more numerically reliable than directly forming a matrix inverse.

The input must excite the dynamics of interest. A constant input cannot identify time constants that are never activated. Useful excitation may include steps, ramps, chirps, pseudorandom binary sequences, multisine signals, or carefully designed trajectories. However, excitation must respect safety, saturation, thermal, vibration, and human-interaction constraints.

Experiment design balances information and risk. A high-frequency input may identify electrical dynamics but be attenuated by an actuator or filtered by a sensor. A large-amplitude input may reveal nonlinear friction or saturation but damage the mechanism. The experiment should therefore specify:

• the parameters and dynamics to identify;
• input amplitude, bandwidth, and duration;
• sensor locations, resolution, and synchronization;
• operating points such as temperature, load, and speed;
• safety limits and emergency shutdown conditions;
• calibration, filtering, and data-quality checks.

5. Dynamic System Identification

System identification commonly follows four decisions: model purpose, model structure, experiment, and validation.

Parametric models

A parametric model has a finite set of interpretable parameters. Examples include state-space models, transfer functions, ARX models, and physical energy-based models. For a single-input single-output discrete-time ARX model,

The coefficients can be estimated by linear regression, provided the noise assumptions are acceptable. ARX models are convenient, but because past measured outputs appear as regressors, output noise can bias estimates. Output-error and Box–Jenkins structures model noise separately and may be preferable when measurement noise is significant.

Nonparametric and frequency-domain methods

Frequency-response estimation applies sinusoidal or broadband excitation and estimates gain and phase as functions of frequency. It is valuable for controller design and resonance analysis. Nonparametric models can reveal dynamics without committing early to a physical structure, but they may require many experiments and do not automatically provide a reliable nonlinear or time-domain model.

Validation

A model must be tested on data not used for fitting. Important checks include:

1. Prediction error: compare measured and predicted outputs using RMSE, MAE, or application-specific loss.
2. Residual whiteness: residuals should contain no systematic dynamics left unexplained.
3. Residual independence: residuals should be uncorrelated with past inputs when the model is intended to capture input-output dynamics.
4. Physical plausibility: estimated mass, inertia, damping, delay, and energy behavior should be credible.
5. Operating-range performance: test different loads, temperatures, speeds, and trajectories.
6. Closed-loop relevance: a model used for control must be assessed in the frequency and amplitude ranges affecting stability and performance.

A useful distinction is one-step prediction versus free-run simulation. A model may predict the next sample well because it repeatedly corrects itself with measured outputs, yet drift badly when simulated independently. Free-run performance is often more demanding and more relevant to digital twins and trajectory prediction.

6. Physics-Informed, Machine-Learning, and Surrogate Models

Physics-based models

A physics-based model uses conservation laws, constitutive relations, geometry, and known mechanisms. Its advantages include interpretability, extrapolation within valid physical assumptions, and compatibility with safety constraints. Its limitations include uncertain parameters, unmodeled friction, hysteresis, backlash, manufacturing variation, and computational complexity.

Pure machine-learning models

A machine-learning model approximates a mapping from data. Examples include linear models, decision trees, random forests, support-vector regression, Gaussian processes, and neural networks. A feed-forward neural network may represent

where z is the input feature vector, W_i and b_i are learned weights and biases, g is an activation function, and L is the number of layers. Neural networks can represent highly nonlinear mappings, but their predictions may be unreliable outside the training distribution.

Surrogate models

A surrogate is a computationally cheaper approximation to a high-fidelity simulation or experiment. It may be used for optimization, uncertainty propagation, real-time control, or design-space exploration. A surrogate is not automatically a physical model: its validity is limited by the data and domain used to construct it.

Physics-informed and hybrid models

Physics-informed learning incorporates physical equations, constraints, or known parameters into training. A generic loss may be written as

where L_data measures measurement error, L_physics measures violation of governing equations, L_boundary measures violation of initial or boundary conditions, L_reg is a regularization term, and α, β, γ are weights. The terms must be scaled carefully; otherwise one component can dominate because of units or numerical magnitude.

Hybrid modeling combines a mechanistic model with a learned residual:

where r_ψ is a learned correction parameterized by ψ. This approach can preserve known structure while learning friction, hysteresis, thermal losses, or other difficult effects. It is most useful when the physical model is approximately correct but incomplete.

Comparison of model classes

7. Bias, Variance, Uncertainty, and Generalization

Bias is systematic error caused by incorrect assumptions, unrepresentative data, sensor calibration, omitted variables, or an objective that does not reflect the application. Variance is sensitivity to the particular sample. Increasing model complexity often reduces training bias but increases variance. Cross-validation, regularization, early stopping, and independent test data help manage this trade-off.

Data splitting must respect time and operating conditions. Randomly mixing adjacent time samples can leak nearly identical information into training and test sets and produce an overoptimistic result. For time-dependent systems, chronological splits or blocked validation are usually safer. If the model must operate across temperatures or payloads, entire operating conditions should be held out for evaluation.

Uncertainty has several sources:

• Measurement uncertainty: sensor noise, quantization, calibration, and synchronization.
• Parameter uncertainty: finite data and weak identifiability.
• Structural uncertainty: missing physics or an unsuitable model class.
• Distributional uncertainty: future conditions differ from training conditions.
• Algorithmic uncertainty: optimization or random initialization changes results.

Confidence intervals based on least-squares assumptions describe only certain statistical uncertainties; they do not guarantee safety under structural error or extrapolation. Bootstrap methods, Bayesian posteriors, ensembles, Gaussian processes, and interval analysis can provide additional information, but their assumptions must be examined.

Generalization should be reported as a function of operating domain, not as a single universal accuracy number. An error of 0.1 rad/s may be acceptable for a low-speed monitoring application but unacceptable for precision positioning. Metrics should therefore be linked to engineering requirements, such as settling time, maximum position error, energy prediction error, fault-detection rate, or constraint-violation probability.

8. Interpretability, Governance, and Responsible Engineering AI

Interpretability is essential when a model influences motion, energy, safety, maintenance, or compliance. A physically interpretable parameter estimate can reveal a worn bearing or increased friction. A black-box predictor may achieve lower average error but fail silently when a fault or novel condition occurs.

Good practice includes:

• documenting data origin, calibration, preprocessing, and exclusions;
• recording model version, parameter values, training conditions, and software dependencies;
• preserving units and coordinate conventions;
• testing for sensor bias and performance differences across operating conditions;
• using uncertainty estimates and out-of-distribution detection;
• establishing human oversight and safe fallback behavior;
• validating cybersecurity of data pipelines and update mechanisms;
• retaining traceable evidence for safety-critical decisions.

AI should not be used to replace a required safety function with an unvalidated prediction. A learned controller or diagnostic system should be bounded by hard constraints, watchdogs, plausibility checks, and fail-safe modes. Engineers remain responsible for the specification, verification, and deployment context even when optimization is automated.

9. Worked Application: Identification of a Servo Axis

The following is an illustrative case involving a laboratory servo axis. It is not a report of a particular commercial product.

Background and problem

A servo axis exhibits larger-than-expected settling error after a payload change. The team has voltage, current, encoder position, and commanded torque data. They need a model accurate enough to retune the controller and determine whether the main problem is inertia, friction, sensor delay, or an unmodeled flexible mode.

Analysis and approach

First, the team calibrates voltage and current sensors, verifies encoder direction, synchronizes timestamps, and removes samples recorded during saturation. They conduct three experiments: a low-amplitude frequency sweep, step commands at several payloads, and constant-speed runs in both directions. The tests are repeated at two temperatures.

A rigid-body model is initially selected:

Electrical resistance is estimated separately from steady current-voltage measurements. The mechanical parameters J, b, and τ_c are estimated by bounded nonlinear least squares using measured current as the torque-producing input. The objective weights speed and position according to their measurement uncertainty and application importance. Positive bounds are imposed on J, b, and τ_c.

The team then examines residuals. The rigid model fits constant-speed data well but shows oscillatory residuals after fast command changes. Frequency-response data reveal a resonance near the expected structural mode. A two-inertia model with shaft stiffness and damping is therefore tested. The additional model reduces validation error and reproduces the resonance, but it is rejected for low-speed embedded control because its computational and calibration requirements are unnecessary for that use. Instead, the two-inertia model is retained for offline controller analysis, while a reduced model is used in the controller with a validated notch filter.

A small neural network is also trained to predict the residual torque caused by friction and temperature. It receives speed, temperature, direction, and recent speed history. Training data are separated by temperature and payload so that validation includes conditions not seen during fitting. The network is used only as a bounded residual correction; the rigid-body equations remain active, and a fallback model is used when inputs lie outside the training range.

Outcome

The identified inertia increases with payload as expected, while friction increases at low temperature. The residual resonance explains the settling error more convincingly than a change in nominal viscous damping. The hybrid model improves free-run prediction over the tested domain, but the team does not claim universal validity. The model documentation records the operating ranges, uncertainty, excluded data, validation trajectories, and fallback conditions.

Learning points

1. Experiment design must separate competing explanations.
2. A good fit on one trajectory does not prove structural adequacy.
3. Residual analysis can reveal omitted dynamics.
4. Model complexity should match the decision and computational budget.
5. Hybrid AI is safer when it corrects a bounded physical model rather than replacing it without safeguards.

10. Best-Practice Workflow

A defensible identification workflow is:

1. Define the intended prediction, control, monitoring, or design task.
2. Establish coordinates, units, sign conventions, and measurement uncertainty.
3. Select a model structure based on physics, prior evidence, and required fidelity.
4. Design safe experiments with sufficient excitation.
5. Inspect, synchronize, calibrate, and document data before fitting.
6. Estimate parameters using numerically stable algorithms and physical bounds.
7. Scale variables and objective terms consistently.
8. Examine identifiability, parameter correlation, and convergence.
9. Validate with independent trajectories and operating conditions.
10. Compare against simpler baselines and relevant physical models.
11. Quantify uncertainty and detect out-of-distribution conditions.
12. Deploy with monitoring, version control, fallback behavior, and periodic reassessment.

Contemporary Developments

Current research increasingly combines differentiable simulators, automated experiment design, operator learning, Gaussian-process residuals, digital twins, and online adaptation. Differentiable simulation can provide gradients through a physical model, making parameter fitting efficient. Active learning chooses experiments that are expected to reduce uncertainty. Neural operators attempt to learn mappings between fields or trajectories rather than fixed-dimensional vectors. These developments are promising, but engineering adoption still depends on reproducibility, computational cost, explainability, safety evidence, and performance under distribution shift.

The most robust direction is not a contest between physics and AI. It is the deliberate integration of physical structure, statistical estimation, carefully designed data, uncertainty quantification, and domain-specific validation.

Chapter Summary

Parameter estimation determines unknown numerical quantities in a chosen model, whereas system identification additionally addresses model structure, experiment design, validation, and intended use. Least squares provides a transparent foundation for linear-in-parameter models; weighted and recursive forms handle unequal uncertainty and online adaptation. Nonlinear optimization supports realistic dynamic models but requires scaling, good initialization, constraints, and convergence checks.

Identifiability depends on excitation, measurement quality, sampling, and separation of parameter effects. Validation must use independent and representative data, examine residuals, distinguish one-step prediction from free-run simulation, and test the operating domain relevant to deployment. Physics-based models offer interpretability and constraints; machine-learning models offer flexibility; physics-informed and hybrid models combine these advantages but require careful loss scaling and uncertainty analysis.

Bias, variance, data leakage, structural uncertainty, and extrapolation are central threats to generalization. Responsible engineering AI requires traceability, calibrated data, out-of-distribution awareness, safety limits, fallback modes, human oversight, and clear documentation. The strongest practical workflow treats AI as part of a validated modeling system rather than as a substitute for engineering judgment.

Key Terms

Parameter estimation: Determining unknown numerical values in a specified model.

System identification: Selecting, estimating, and validating a dynamic model from experimental data.

Regression: Estimating relationships between measured outputs and explanatory variables.

Ordinary least squares: Minimization of the unweighted sum of squared residuals.

Weighted least squares: Least squares with weights representing measurement importance or variance.

Recursive least squares: An online estimator that updates parameters as new data arrive.

Identifiability: The ability to distinguish parameter or model effects from available observations.

Persistent excitation: Input variation sufficient to reveal the dynamics or parameters of interest.

Residual: Difference between measured and model-predicted output.

Regularization: A penalty or prior that controls ill-conditioning and model complexity.

Parametric model: A model described by a finite set of parameters.

Nonparametric model: A model whose complexity is not fixed in advance by a small parameter set.

Surrogate model: A computationally efficient approximation to a high-fidelity model or experiment.

Physics-informed model: A data-driven model trained with governing physical relationships or constraints.

Hybrid model: A model combining mechanistic equations with learned components.

Overfitting: Excessive adaptation to training data that harms performance on new data.

Data leakage: Unintended use of information from validation or test data during training.

Generalization: Performance on data or operating conditions not used for fitting.

Structural uncertainty: Uncertainty caused by an incomplete or unsuitable model form.

Out-of-distribution condition: A situation outside the data domain represented during model development.

Review Questions

Short Answer Questions

1. Distinguish a state from a parameter in a dynamic mechatronic model.
2. State the assumptions required for ordinary least squares to provide an unbiased estimate in a linear regression setting.
3. Why is persistent excitation important in system identification?
4. What is the purpose of a forgetting factor in recursive least squares?
5. Distinguish one-step prediction from free-run simulation.

Long Answer Questions

1. Explain a complete workflow for identifying a motor-drive model, including experiment design, estimation, validation, and documentation.
2. Compare physics-based, machine-learning, physics-informed, and hybrid models for a servo-control application.
3. Explain how scaling, constraints, and regularization affect nonlinear parameter estimation.
4. Discuss the sources of uncertainty in an identified model and methods for quantifying them.
5. Explain why random sample splitting can produce misleading validation results for time-dependent mechatronic data.

Analytical / Application Questions

1. A motor model fits a constant-speed experiment accurately but fails during acceleration. Propose at least three possible causes and design experiments to distinguish among them.
2. You must estimate inertia, viscous friction, and Coulomb friction from torque and speed data. Formulate a suitable regression or optimization problem and state the assumptions needed for credible estimates.
3. A neural-network residual model improves test error but occasionally commands physically impossible torque corrections. Propose a deployment architecture that retains its benefits while enforcing safety and physical plausibility.

Multiple Choice Questions with Answers

Question 1

Which condition is required for the ordinary least-squares closed-form estimator θ̂ = (Φ^TΦ)^-1Φ^T y to exist?

A. The regression matrix has full column rank

B. All measurements are noiseless

C. The system is nonlinear in parameters

D. The input is constant

Correct Answer: A. The regression matrix has full column rank

Question 2

In recursive least squares, a forgetting factor less than one primarily allows the estimator to do what?

A. Ignore all measurement noise

B. Adapt to slowly changing parameters

C. Guarantee unbiased estimates

D. Remove the need for excitation

Correct Answer: B. Adapt to slowly changing parameters

Question 3

What is the main purpose of persistent excitation in an identification experiment?

A. To reduce the sampling frequency

B. To ensure that relevant dynamics can be distinguished

C. To eliminate actuator saturation automatically

D. To guarantee a linear physical system

Correct Answer: B. To ensure that relevant dynamics can be distinguished

Question 4

Which method is most directly intended to reduce ill-conditioning by penalizing large or implausible parameter values?

A. Regularization

B. Aliasing

C. Quantization

D. Differentiation

Correct Answer: A. Regularization

Question 5

Why can random splitting of adjacent time-series samples cause overly optimistic validation?

A. It always increases sensor noise

B. Nearly identical information can appear in both training and test sets

C. It prevents any model from learning dynamics

D. It removes all temporal correlation

Correct Answer: B. Nearly identical information can appear in both training and test sets

Question 6

A hybrid model is best described as one that does what?

A. Uses only a neural network with no physical assumptions

B. Combines mechanistic equations with a learned component

C. Uses only manually selected lookup tables

D. Eliminates the need for validation

Correct Answer: B. Combines mechanistic equations with a learned component

Question 7

For the illustrative friction data, the through-origin estimate using b̂ = (∑ω_kτ_k)/(∑ω_k²) is approximately what?

A. 0.00392 N·m·s/rad

B. 0.0392 N·m·s/rad

C. 0.392 N·m·s/rad

D. 3.92 N·m·s/rad

Correct Answer: B. 0.0392 N·m·s/rad

Question 8

Which validation result most strongly suggests that a model has omitted dynamic behavior?

A. Residuals show systematic oscillations after command changes

B. The input units are recorded in SI

C. The parameter vector has a stated prior

D. The training loss decreases

Correct Answer: A. Residuals show systematic oscillations after command changes

Question 9

What is a major risk of using a machine-learning model outside its training operating range?

A. It necessarily becomes a physical law

B. Its prediction may be unreliable because of extrapolation

C. Its parameters become directly measurable

D. Its sampling interval becomes zero

Correct Answer: B. Its prediction may be unreliable because of extrapolation

Question 10

Which practice is most appropriate for a safety-relevant learned model?

A. Deploy it without monitoring if average error is low

B. Use hard constraints, monitoring, and a validated fallback mode

C. Train only on randomly shuffled samples

D. Remove all physical checks to increase flexibility

Correct Answer: B. Use hard constraints, monitoring, and a validated fallback mode