Calculus of Variations and Engineering Optimization

Introduction and Context

Mechatronic systems are designed to achieve objectives while satisfying physical, geometric, safety, and manufacturing constraints. A robot trajectory may need to minimize energy and avoid obstacles; a controller may require parameters that minimize tracking error; and a mechanical component may need to balance mass, strength, cost, and reliability. These are optimization problems.

Ordinary calculus optimizes a finite set of variables. Calculus of variations extends this idea to optimize an entire function, such as a trajectory x(t). Engineering optimization combines both viewpoints with numerical algorithms for problems that cannot be solved analytically.

This chapter develops the Euler–Lagrange equation, constrained optimization using Lagrange multipliers, gradient-based methods, and numerical search methods. The emphasis is on formulation, interpretation, assumptions, and practical implementation.

Learning Objectives

By the end of this chapter, a student should be able to:

1. Distinguish parameter optimization from functional optimization.
2. Derive and apply the Euler–Lagrange equation.
3. Formulate unconstrained and constrained engineering optimization problems.
4. Use gradients, Hessians, Lagrange multipliers, and numerical search methods.
5. Interpret optimization results in trajectory planning, parameter estimation, energy minimization, and design trade-offs.
6. Recognize numerical, modelling, and ethical limitations of optimization.

1. Optimization in Mechatronics

An optimization problem generally has the form

Here, z is the vector of decision variables, J is the objective function, g_i are inequality constraints, and h_j are equality constraints. A feasible point satisfies all constraints. A global optimum is at least as good as every feasible point; a local optimum is best only within a neighbourhood.

Examples include:

A minimization problem can represent maximization by minimizing the negative of the desired quantity. Units must be considered: adding dimensional quantities directly is invalid unless they have been normalized or assigned appropriate weights.

2. Finite-Dimensional Optimization

For a differentiable scalar objective J(x), where x = [x₁, …, x_n]ᵀ, the gradient is

The gradient points in the direction of greatest local increase. Therefore, −∇J is the direction of steepest local decrease. A necessary condition for an unconstrained interior stationary point is

The Hessian matrix is

At a stationary point, a positive-definite Hessian indicates a strict local minimum, a negative-definite Hessian indicates a strict local maximum, and an indefinite Hessian indicates a saddle point. These tests assume twice-continuous differentiability near the point.

Gradient Descent

The basic gradient-descent iteration is

where α_k > 0 is the step length. A small step may converge slowly, whereas a large step may cause oscillation or divergence. Line searches and adaptive step rules are often used.

For the quadratic objective J(x) = 1/2 xᵀQx + cᵀx, with symmetric positive-definite Q, the unique global minimizer satisfies Qx* = −c. Positive definiteness means yᵀQy > 0 for every nonzero vector y.

Worked Example 1: Gradient Descent

Let

The gradient is ∇J = [2(x−2), 4(y+1)]ᵀ. Starting at (0, 0), take α = 0.1:

∇J(0,0) = [−4,4]ᵀ.

Thus,

[x₁,y₁]ᵀ = [0,0]ᵀ − 0.1[−4,4]ᵀ = [0.4,−0.4]ᵀ.

The exact minimum is (2, −1), because both squared terms are minimized there. The first iteration moves toward that point, but not directly to it because the objective has different curvature in the two directions.

3. Calculus of Variations

In calculus of variations, the unknown is a function y(x), not merely a number. Consider the functional

where y′ = dy/dx, F is continuously differentiable, and the endpoint values y(a) and y(b) are fixed. The goal is to find the function y that makes I stationary.

Derivation of the Euler–Lagrange Equation

Perturb a candidate function y by

where ε is a small scalar and η(a) = η(b) = 0. Stationarity requires

Differentiating under the integral gives

Integration by parts gives

The boundary term vanishes because η(a) = η(b) = 0. Hence

Since η is arbitrary, the fundamental lemma of the calculus of variations yields the Euler–Lagrange equation:

This is a necessary condition for a stationary function. It is not by itself sufficient to prove a minimum; the stationary function may be a maximum or saddle, and additional convexity or second-variation analysis may be required.

For several functions y_i(x), the equations become

Example 2: Shortest Path

The length of a curve y(x) joining fixed endpoints is

Here, F = √(1 + y′²), which does not explicitly depend on y. Therefore, the Euler–Lagrange equation gives

d/dx(y′/√(1+y′²)) = 0.

Thus y′ is constant, and y is a straight line. The result agrees with geometric intuition: among smooth curves joining two fixed points, the straight line has minimum length.

4. Engineering Lagrangians and Optimal Trajectories

In mechanics, the Lagrangian is commonly

where T is kinetic energy, V is potential energy, q is a generalized coordinate, and q̇ is its time derivative. The equations of motion follow from

In trajectory planning, an engineering cost may instead be

where u is an actuator command, q̇ is velocity, q̈ is acceleration, and w_e, w_v, w_a are nonnegative weights. The weights must be chosen with attention to units, often after normalization. Boundary conditions such as q(0), q(T), q̇(0), and q̇(T) may be prescribed.

If the integrand depends on q̈, the higher-order Euler–Lagrange equation is

This equation assumes sufficiently smooth admissible trajectories and appropriate fixed boundary conditions.

5. Constrained Optimization and Lagrange Multipliers

Suppose an objective f(x) is minimized subject to an equality constraint h(x) = 0. Define the Lagrangian

A constrained stationary point satisfies

Geometrically, at the optimum the objective gradient is parallel to the constraint gradient. The multiplier λ measures the local sensitivity of the optimal objective to a change in the constraint, with sign depending on the chosen Lagrangian convention.

For several equality constraints,

For inequalities, the Karush–Kuhn–Tucker conditions supplement stationarity with feasibility and complementary slackness. For a minimization problem with g_i(x) ≤ 0, using ℒ = f + ∑λ_i g_i + ∑μ_j h_j, the conditions include λ_i ≥ 0 and λ_i g_i(x) = 0. Under suitable constraint qualifications and convexity, these conditions are especially powerful; otherwise they are generally necessary conditions rather than sufficient ones.

Example 3: Minimum Norm with a Constraint

Minimize f(x,y)=x²+y² subject to x + y = 1. The Lagrangian is

ℒ=x²+y²+λ(x+y−1).

Stationarity gives 2x + λ = 0 and 2y + λ = 0, so x = y. Using x + y = 1 gives x = y = 0.5. The minimum objective is 0.25 + 0.25 = 0.5. Since f is strictly convex and the constraint is affine, this is the unique global minimum.

6. Parameter Estimation as Optimization

Suppose a model predicts measurements ŷ_k(θ), where θ is an unknown parameter vector. A common least-squares objective is

The factor 1/2 simplifies derivatives and does not change the minimizer. If measurement errors have unequal variances, weighted least squares is more appropriate:

where w_k is commonly chosen proportional to the inverse error variance. The model, data quality, parameter identifiability, and noise assumptions determine whether the fitted parameters are meaningful.

For nonlinear models, Gauss–Newton approximates the Hessian using the Jacobian of residuals, while Levenberg–Marquardt adds damping to improve robustness. Training a small neural estimator is also an optimization problem, but low training error does not guarantee safe or physically valid mechatronic behaviour.

7. Numerical Search Methods

Analytical solutions are rare for realistic systems. Common methods include:

A practical workflow is: define variables and units; scale variables and objectives; identify constraints; choose an initial guess; solve; check feasibility; test sensitivity to initial conditions; and validate the result in a higher-fidelity model or experiment.

Global methods such as genetic algorithms, particle swarm optimization, and simulated annealing can explore nonconvex landscapes, but they may require many function evaluations and do not automatically guarantee a global optimum. Hybrid methods often use a global search followed by local refinement.

8. Illustrative Case: Energy-Efficient Robot Joint Motion

Background

Consider a single robot joint required to move from q(0) = 0 rad to q(T) = 1 rad in T = 2 s, with zero initial and final velocity. This is an illustrative case, not a specification for a particular robot.

Problem

A smooth trajectory should reduce actuator effort and avoid abrupt changes. Choose q(t) to minimize

subject to q(0) = 0, q(2) = 1, q̇(0) = q̇(2) = 0.

Analysis

Because F = q̈², the higher-order Euler–Lagrange equation is

d²/dt²(∂F/∂q̈)=0,

so q⁽⁴⁾(t) = 0. Therefore q(t) is a cubic polynomial:

The boundary conditions give a₀ = 0, a₁ = 0, q(2) = 4a₂ + 8a₃ = 1, and q̇(2) = 4a₂ + 12a₃ = 0. Subtracting the equations gives 4a₃ = −1, so a₃ = −0.25 and a₂ = 0.375. Hence

or, equivalently,

for 0 ≤ t ≤ 2. Its velocity is q̇(t) = 0.75t − 0.75t², which is zero at both endpoints. Its acceleration is q̈(t) = 0.75 − 1.5t, which is finite but changes sign.

Outcome and Interpretation

The cubic trajectory is stationary for the stated acceleration-squared objective and boundary conditions. In implementation, actuator torque limits, velocity limits, collision constraints, discretization, motor dynamics, and controller tracking error must also be checked. If jerk smoothness is important, an objective involving q⁽³⁾ or a penalized combination of acceleration and jerk may be used.

Learning Points

1. The decision variable can be a complete function.
2. Boundary conditions are part of the optimization formulation.
3. A mathematically optimal trajectory may be infeasible for real hardware.
4. Weight selection expresses an engineering trade-off and must be documented.

9. Design Trade-offs, Scaling, and Validation

Engineering objectives often conflict. Reducing mass may reduce stiffness; increasing speed may increase energy and vibration; improving accuracy may increase cost. A weighted objective such as

is meaningful only when component objectives are scaled comparably. One common dimensionless normalization is

where the reference limits must be defined carefully. Changing weights changes the preferred design. A Pareto-optimal design cannot improve one objective without worsening at least one other objective, within the feasible set.

Optimization results should be validated by independent simulation, uncertainty analysis, sensitivity analysis, and physical testing. A local optimum can be mistaken for a global optimum, and a numerically precise answer can still be physically wrong if the model is poor.

10. Limitations, Safety, and Responsible Use

Optimization is not a substitute for engineering judgement. Important limitations include nonconvexity, noisy measurements, inaccurate models, hidden constraints, numerical ill-conditioning, and dependence on initial guesses. A constraint omitted from the model is not enforced in the real system.

For safety-critical robots, medical devices, vehicles, and industrial machinery, optimization must operate within documented requirements, verification procedures, and applicable standards. Automatically generated designs should be reviewed by qualified engineers. AI-based optimizers should preserve traceability: data sources, objective definitions, constraints, random seeds, software versions, and validation results should be recorded. A system must not trade away safety merely because a weighted numerical objective improves.

Best-Practice Checklist

• Define every decision variable, unit, bound, and physical meaning.
• Use dimensionless scaling or justified weights for mixed objectives.
• Separate hard safety constraints from preferences.
• Check constraint qualifications and solver termination messages.
• Test several initial guesses for nonconvex problems.
• Compare analytical results with numerical solutions where possible.
• Validate optimized designs under uncertainty and off-nominal conditions.
• Report sensitivity, feasibility margins, and reproducibility information.

Chapter Summary

Optimization provides a systematic way to select parameters, trajectories, and designs. Finite-dimensional unconstrained optima satisfy a zero gradient under differentiability assumptions, while the Hessian helps classify stationary points. Calculus of variations optimizes functions and leads, through admissible perturbations and integration by parts, to the Euler–Lagrange equation. Higher-order functionals produce higher-order Euler–Lagrange equations.

Lagrange multipliers convert equality constraints into stationarity conditions, and KKT conditions extend the framework to inequalities. Gradient, Newton-type, least-squares, derivative-free, and global numerical methods each have appropriate uses and limitations. In mechatronics, optimization supports trajectory planning, parameter estimation, control tuning, energy reduction, and design trade-offs. Sound results require correct units, complete constraints, numerical checks, physical validation, and responsible safety-oriented engineering judgement.

Key Terms

Functional: A mapping that assigns a scalar to an entire function.

Variation: An admissible perturbation of a candidate function.

Euler–Lagrange equation: A necessary stationarity equation for a functional.

Lagrangian: An objective combined with constraint terms and multipliers.

Gradient: Vector of first partial derivatives of a scalar function.

Hessian: Matrix of second partial derivatives.

Stationary point: Point where the relevant first-order derivative vanishes.

Local minimum: Minimum within a neighbourhood.

Global minimum: Minimum over the complete feasible set.

Feasible point: Point satisfying all constraints.

Equality constraint: Condition written h(x) = 0.

Inequality constraint: Condition written g(x) ≤ 0 or g(x) ≥ 0.

Lagrange multiplier: Variable associated with a constraint sensitivity.

KKT conditions: First-order conditions for constrained optimization.

Gradient descent: Iterative method that moves opposite the gradient.

Least squares: Minimization of squared prediction residuals.

Pareto optimum: Design for which no objective improves without another worsening.

Convex function: Function whose line segments lie above its graph.

Sensitivity: Change in an outcome caused by a change in a parameter.

Admissible trajectory: Trajectory satisfying specified regularity and boundary requirements.

Review Questions

Short Answer Questions

1. What is the difference between a local and a global minimum?
2. State the Euler–Lagrange equation for a functional depending on y and y′.
3. What is the geometric meaning of a Lagrange multiplier condition?
4. Why should mixed engineering objectives be scaled or normalized?
5. State one limitation of gradient descent.

Long Answer Questions

1. Derive the Euler–Lagrange equation using an admissible perturbation and integration by parts.
2. Explain how the Hessian classifies an unconstrained stationary point.
3. Compare gradient descent, Newton, quasi-Newton, and derivative-free methods.
4. Formulate a constrained robot trajectory optimization problem and identify its variables, objective, and constraints.
5. Explain why numerical optimality does not necessarily imply physical feasibility.

Analytical / Application Questions

1. Minimize f(x,y) = x² + 4y² subject to x + y = 1 using Lagrange multipliers.
2. For a least-squares model ŷ(t;θ) = θ₁t + θ₂, explain how measurements are used to estimate θ₁ and θ₂ and identify one source of estimation error.
3. Design an optimization formulation for a two-joint robot that balances travel time, energy, and smoothness while enforcing joint, velocity, and collision constraints.

Multiple Choice Questions with Answers

Question 1

Which condition is necessary for an unconstrained differentiable interior minimum?

A. ∇J(x*) = 0

B. J(x*) = 0

C. H(x*) = 0

D. All constraints are active

Correct Answer: A. ∇J(x*) = 0

Question 2

The Euler–Lagrange equation for I[y] = ∫F(x,y,y′)dx is:

A. ∂F/∂y + d/dx(∂F/∂y′) = 0

B. ∂F/∂y − d/dx(∂F/∂y′) = 0

C. F = 0

D. dF/dx = 1

Correct Answer: B. ∂F/∂y − d/dx(∂F/∂y′) = 0

Question 3

In gradient descent, the update direction is generally:

A. The positive gradient

B. The Hessian only

C. The negative gradient

D. The constraint gradient only

Correct Answer: C. The negative gradient

Question 4

For f(x,y)=x²+y² subject to x+y=1, the minimizer is:

A. (0,1)

B. (1,0)

C. (−1,2)

D. (0.5,0.5)

Correct Answer: D. (0.5,0.5)

Question 5

What does a feasible point satisfy?

A. Only the objective gradient condition

B. All specified constraints

C. Only equality constraints

D. A zero objective value

Correct Answer: B. All specified constraints

Question 6

For a twice-differentiable stationary point, a positive-definite Hessian indicates a:

A. Strict local minimum

B. Strict local maximum

C. Saddle point

D. Constraint violation

Correct Answer: A. Strict local minimum

Question 7

In least-squares estimation, the residual is generally:

A. The measured value minus the predicted value

B. The Hessian determinant

C. The constraint multiplier

D. The step length

Correct Answer: A. The measured value minus the predicted value

Question 8

For an objective involving q̈, the higher-order Euler–Lagrange equation contains:

A. Only q

B. A second derivative of ∂F/∂q̈

C. No derivatives

D. Only the first derivative of q

Correct Answer: B. A second derivative of ∂F/∂q̈

Question 9

Complementary slackness in KKT conditions requires, for each inequality, that:

A. The multiplier and constraint function product is zero

B. The multiplier is always negative

C. Every constraint is active

D. The objective is zero

Correct Answer: A. The multiplier and constraint function product is zero

Question 10

A Pareto-optimal design is one for which:

A. Every objective has its individual minimum simultaneously

B. No objective can improve without worsening another objective

C. All design variables are zero

D. The Hessian is always indefinite

Correct Answer: B. No objective can improve without worsening another objective