Robotics, Planning and Human–Robot Interaction

Introduction

Artificial intelligence has transformed robots from pre-programmed machines into systems that can perceive uncertain environments, construct internal models, plan actions, and cooperate with people. In mechatronics engineering, these capabilities emerge from the integration of mechanics, sensors, actuators, control, embedded computing, artificial intelligence, and human factors.

A robot operating in a warehouse, factory, hospital, or service environment must answer four related questions:

1. Where am I? — localization and state estimation.
2. What is around me? — perception and mapping.
3. What should I do, and how should I do it? — task, path, and motion planning.
4. How should I coordinate with people? — human–robot interaction and shared autonomy.

These questions are coupled. A planning algorithm is only as reliable as the robot’s state estimate and environmental model. Similarly, a mathematically efficient trajectory may be unacceptable if it is unsafe, ergonomically poor, or difficult for a human worker to understand.

Learning Objectives

By the end of this chapter, the reader should be able to:

• distinguish localization, mapping, planning, and control;
• explain probabilistic localization and simultaneous localization and mapping (SLAM);
• formulate configuration-space and graph-based planning problems;
• compare sampling-based and optimization-based motion planning;
• describe AI methods for manipulation and mobile robots;
• design interaction principles for collaborative robots;
• evaluate human intent, ergonomics, safety, and shared autonomy;
• select meaningful performance and safety metrics for a robotic application.

1. The Robotic Decision-Making Architecture

A useful abstraction is the perception–planning–action loop. Sensors provide observations, an estimator infers the robot’s state, a world model represents relevant objects and obstacles, a planner selects actions, and a controller converts those actions into actuator commands.

Let the robot state at time t be [[M:x_t]], the control input be [[M:u_t]], and the sensor observation be [[M:z_t]]. A general stochastic robot model is:

Here, [[M:f]] is the motion model, [[M:h]] is the observation model, and [[M:w_t]] and [[M:v_t]] represent process and measurement uncertainty. The equations are applicable when the system can be approximated by discrete-time state and observation functions. Their dimensions must be consistent: for example, if [[M:x_t]] is a pose vector, [[M:f]] must return a pose vector and [[M:u_t]] must have units compatible with the robot’s actuators.

The estimator seeks the posterior probability:

This probability expresses uncertainty about the current state given all observations up to time t and previous controls. Kalman filters are effective for approximately linear, Gaussian systems; extended Kalman filters linearize nonlinear models; particle filters represent non-Gaussian distributions with samples. In robotics, uncertainty is not a secondary detail: it determines whether a planned clearance is genuinely safe.

A complete architecture normally contains:

2. Localization and Mapping

2.1 Localization

Localization estimates a robot pose relative to a reference frame. For a planar mobile robot, a pose may be written as:

where x and y are positions in metres and [[M:θ]] is orientation in radians. A differential-drive robot with wheel radius r, wheel separation L, and wheel angular velocities [[M:ω_R]] and [[M:ω_L]] has ideal body velocities:

For a short interval [[M:∆t]], a first-order pose update is:

These equations assume planar motion, no wheel slip, known wheel parameters, and sufficiently small sampling intervals. In practice, encoder drift, unequal wheels, floor irregularity, and slip cause errors that grow with distance.

Absolute localization can use landmarks, fiducial markers, GPS, or map matching. Relative localization uses wheel odometry, inertial measurement units, visual odometry, or lidar scan matching. Robust systems fuse multiple sources because each sensor has different failure modes.

2.2 Mapping and SLAM

A map may be geometric, semantic, or both. An occupancy-grid map divides space into cells and assigns each cell a probability of being occupied. A log-odds representation is often used:

where [[M:m_i]] is a map cell. Log odds make repeated evidence additive. The representation assumes that the sensor inverse model is defined and that the map-cell update approximation is acceptable; correlations between cells may be ignored.

In simultaneous localization and mapping, the robot estimates both its pose and an initially unknown map. The central difficulty is circular: an accurate map is needed for localization, while accurate localization is needed to construct the map. SLAM systems address this using scan matching, visual features, odometry, loop-closure detection, and graph optimization.

A pose-graph formulation represents robot poses as nodes and relative measurements as edges. Optimization seeks poses that best satisfy the measurements. Loop closure is especially important: when the robot revisits a known place, the new constraint can correct accumulated drift. However, false loop closures can produce severe map distortion, so perceptual similarity must be verified geometrically and statistically.

2.3 Practical localization design

A mechatronics engineer should specify:

• required position and orientation accuracy;
• update rate and latency;
• operating illumination and texture;
• maximum acceptable drift;
• environmental changes, such as moved shelving;
• failure detection and recovery behavior;
• computational and power limits.

A warehouse robot may tolerate centimetre-level localization in open aisles but require tighter accuracy at a docking station. Localization quality should therefore be evaluated against task tolerances, not only against an abstract benchmark.

3. Path and Motion Planning

3.1 Configuration space

Planning is performed in configuration space, whose coordinates describe the robot’s degrees of freedom. A mobile robot may have three configuration variables, whereas a six-axis manipulator has six joint coordinates. Obstacles are expanded by the robot’s geometry; the resulting forbidden region is called configuration-space obstacle space. This conversion allows collision checking to be expressed as a point-search problem in configuration space.

A path is a geometric sequence of configurations. A trajectory additionally specifies time, velocity, and acceleration. This distinction is essential: a collision-free path may be dynamically infeasible if it requires infinite acceleration or exceeds actuator limits.

3.2 Graph search

If the environment is discretized into nodes and edges, planning can be formulated as a shortest-path problem. Dijkstra’s algorithm is complete on a finite graph with nonnegative edge costs but may explore many unnecessary nodes. A* uses a heuristic:

where [[M:g(n)]] is the cost from the start to node n, and [[M:h(n)]] estimates the remaining cost. If the heuristic never overestimates the true remaining cost, A* is admissible and returns an optimal path under the selected cost function.

For a grid with unit-cost horizontal and vertical motion, Manhattan distance is admissible when diagonal motion is disallowed. Euclidean distance is admissible when straight-line distance is a lower bound on the actual cost. In real robots, the cost may include distance, turning, energy, risk, discomfort, or proximity to people.

3.3 Sampling-based planning

High-dimensional manipulators make exhaustive grids impractical. Sampling-based methods such as the rapidly exploring random tree (RRT) generate collision-free configurations and connect nearby samples. RRT is probabilistically complete under suitable conditions: if a feasible path exists and sampling continues indefinitely, the probability of finding a path approaches one. This does not mean that a solution is guaranteed within a fixed computation time, nor that the first solution is optimal.

RRT* improves asymptotic solution quality through rewiring, but it normally requires more computation. Sampling-based planners are useful for complex geometry, while their paths often require smoothing and time parameterization before execution.

3.4 Optimization-based planning

Trajectory optimization minimizes a cost such as:

The [[M:w_i]] are nonnegative weights, and each cost term must be scaled so that the weighted sum has meaningful numerical behavior. Typical constraints include joint limits, velocity limits, acceleration limits, dynamic equations, and collision-clearance requirements. Optimization can produce smooth, efficient motion but may converge to a local minimum and may fail if initialization is poor.

A robust planner often combines methods: a graph or sampling planner finds a feasible route, then optimization smooths and improves it. Replanning is required when obstacles or people move.

3.5 Risk-aware planning

For human-populated environments, distance alone is insufficient. A robot should consider relative velocity, approach direction, visibility, uncertainty, and the consequences of contact. A simple risk-sensitive objective can be written as:

where [[M:λ]] expresses the importance assigned to risk. This is a design parameter, not a universal constant. Safety constraints should not be replaced by a soft penalty when regulations or hazard analysis require hard limits.

4. Manipulation and Mobile Robots

4.1 Manipulator planning

A manipulator must coordinate joint motion, end-effector pose, grasp stability, and collision avoidance. Forward kinematics maps joint coordinates [[M:q]] to end-effector pose:

The Jacobian relates joint velocity to end-effector velocity:

where [[M:J(q)]] is the manipulator Jacobian. Near singular configurations, small Cartesian motions may require very large joint velocities, and force transmission may become poor. Planners should therefore penalize or avoid singularities where appropriate.

AI supports manipulation through object detection, pose estimation, grasp prediction, tactile classification, and learning from demonstration. Nevertheless, learned perception must be checked for uncertainty and domain shift. A grasp model trained on clean laboratory images may fail under reflective packaging, occlusion, or altered lighting.

4.2 Mobile robot navigation

A navigation stack commonly includes localization, a global planner, a local planner, and low-level control. The global planner uses a long-range map; the local planner reacts to dynamic obstacles and vehicle constraints. Cost maps may encode occupancy, inflation around obstacles, terrain difficulty, and social preferences such as avoiding doorways or maintaining personal space.

The planner must account for the robot’s kinematics. A holonomic platform can move independently in two planar directions, whereas a car-like robot cannot move sideways instantaneously. Ignoring these constraints produces paths that are geometrically valid but physically impossible.

4.3 Learning-based robotics

Reinforcement learning can learn policies through interaction, where an agent chooses actions to maximize cumulative reward. It is attractive for complex behaviors, but direct deployment on physical robots is difficult because exploration can damage equipment or injure people. Simulation, domain randomization, demonstrations, safety shields, and constrained policy execution reduce risk.

A practical principle is to use learning where perception or adaptation is difficult to model, while retaining verified model-based constraints for collision avoidance, actuator limits, and emergency stopping.

5. Human–Robot Interaction and Collaborative Robots

5.1 Interaction as a control problem

Human–robot interaction is not merely a user-interface issue. The human and robot form a coupled system in which each participant predicts the other’s behavior. Effective interaction requires the robot to communicate intent, interpret human actions, and adapt its motion without becoming unpredictable.

Interaction channels include:

• physical contact and force sensing;
• speech and natural language;
• gaze, gestures, and body posture;
• buttons, tablets, and wearable devices;
• visual projections, lights, sounds, and motion cues.

Redundant communication is beneficial in noisy environments. For example, a robot may indicate a planned direction through both an oriented body motion and a light signal.

5.2 Collaborative robot safety

Collaborative operation requires a risk assessment rather than reliance on the label “collaborative robot.” Relevant hazards include crushing, trapping, impact, sharp edges, unexpected restart, payload release, and software failure. Protective measures may include physical separation, safety-rated monitored stop, hand guiding, speed-and-separation monitoring, power-and-force limiting, protective sensing, and validated emergency stops.

The design process should identify reasonably foreseeable misuse, not only intended use. Safety functions require appropriate validation, documentation, and periodic inspection according to applicable machinery and workplace regulations. AI perception should not be treated as the sole safety layer unless its performance, failure modes, and certification status are demonstrably suitable.

5.3 Human intent recognition

Human intent may be inferred from gaze, hand trajectory, posture, speech, task context, or interaction history. A probabilistic formulation is:

where [[M:I]] is a candidate intent and [[M:O]] is observed behavior. [[M:P(I)]] is the prior probability of the intent and [[M:P(O|I)]] is the likelihood of the observations. The proportionality becomes an equality after normalization across all candidate intents.

Intent inference is uncertain and can be wrong. A robot should therefore communicate assumptions, request clarification when ambiguity is high, and select conservative actions when the cost of misunderstanding is large.

5.4 Ergonomics

A collaborative robot should improve the overall work system, not merely transfer difficult movements from a person to a machine. Ergonomic analysis considers reach distance, posture, repetition, force, vibration, visual demand, cognitive workload, and work-rest cycles. The robot may present parts at a comfortable height, carry heavy loads, or synchronize with a worker’s preferred pace.

A technically successful system can still fail if it increases cognitive interruptions, forces awkward waiting, or reduces the worker’s sense of control. Human factors evaluation should therefore include task time, physical workload, error rate, subjective workload, trust calibration, and acceptance.

5.5 Shared autonomy

In shared autonomy, the human supplies goals, preferences, or corrections while the robot handles detailed execution. Suppose the human provides a command [[M:u_h]] and the robot has an autonomous policy [[M:u_r]]. A blended command may be written:

where [[M:0 ≤ α ≤ 1]]. This simple model is useful conceptually, but a production system must account for command conflicts, coordinate frames, safety constraints, latency, and authority transfer. The blending parameter may depend on confidence, task phase, or detected human workload.

Shared autonomy works best when authority is legible. Users should know what the robot will do automatically, how to override it, and when control has shifted. Excessive automation can produce out-of-the-loop problems, while insufficient automation creates unnecessary workload.

6. Applied Case: Illustrative Collaborative Assembly Cell

Background

Consider an illustrative automotive subassembly cell in which a worker installs clips and connectors while a six-axis robot presents parts, holds a component, and performs repetitive fastening. The cell contains a depth camera, force-torque sensor, joint torque monitoring, an industrial controller, and a safety-rated scanner.

Problem

The original robot program used fixed timing and fixed waypoints. It caused delays when the worker changed pace, occasionally approached the worker too closely, and failed when parts were placed with small positional variation.

Analysis

The engineering team separates the problem into four layers:

1. Perception: detect the worker’s hand, part pose, and occupied workspace.
2. Estimation: fuse camera, robot encoder, and force information.
3. Planning: select a collision-free approach and adapt speed to the worker.
4. Interaction: communicate which part is ready and allow the worker to pause or redirect the robot.

A risk assessment identifies fastening and part presentation as separate hazard states. The robot is permitted to move quickly in a verified empty region but must reduce speed near the worker. A safety-rated scanner independently commands a protective stop if separation becomes unsafe.

Approach and solution

A global planner generates feasible robot configurations for reaching presentation poses. A local trajectory optimizer then minimizes path length and jerk subject to joint, collision, and speed constraints. The vision system estimates part pose but rejects detections with low confidence. If uncertainty exceeds a threshold, the robot pauses and requests confirmation rather than guessing.

The worker uses a physical enable device and a simple gesture to request the next part. The robot signals its intended motion using an indicator light and a short, visible pre-motion movement. Force sensing detects contact during insertion and switches the robot to a compliant mode. The safety system remains independent of the learned perception system.

Outcome

In the illustrative evaluation, cycle time improves because the robot adapts to worker pace, while unplanned pauses decrease because part pose variation is handled explicitly. More importantly, the system becomes easier to understand: the worker can predict robot behavior, stop it, and recover from uncertain perception.

Learning points

• Planning quality depends on state estimation and calibrated sensing.
• Human-aware motion requires more than obstacle avoidance.
• AI should be combined with deterministic safety functions.
• Confidence thresholds and safe fallback states are essential.
• Acceptance depends on workload, predictability, and control—not only productivity.

7. Evaluation, Limitations, and Best Practices

Important evaluation metrics include localization error, map consistency, planning time, success rate, path length, energy, minimum clearance, jerk, collision rate, task completion time, human workload, intervention frequency, and subjective trust. Metrics must be reported with operating conditions; an average success rate can conceal catastrophic failures in rare situations.

Common limitations include sensor occlusion, reflective or transparent objects, changing lighting, wheel slip, map ageing, computational delay, local minima, sim-to-real transfer errors, biased training data, and distribution shift. Human intent models may perform differently across cultures, body types, disabilities, or communication styles.

Best practices are to:

1. define the operational design domain before selecting an AI method;
2. model uncertainty explicitly and calibrate confidence;
3. combine global planning with local replanning;
4. enforce hard safety constraints outside unconstrained learning modules;
5. validate with representative users and realistic disturbances;
6. design clear override, stop, and recovery procedures;
7. log decisions, sensor states, interventions, and near misses;
8. protect personal and biometric data;
9. test failure modes, not only nominal performance;
10. document assumptions, limits, maintenance, and software versions.

Contemporary systems increasingly combine foundation models, semantic maps, tactile sensing, digital twins, and adaptive control. These technologies can improve flexibility, but language or vision models may hallucinate, misinterpret context, or provide unverifiable reasoning. Their outputs should therefore be bounded by task-specific planners, validated interfaces, and safety monitors.

Chapter Summary

Robotic intelligence integrates perception, state estimation, mapping, planning, control, and interaction. Localization estimates pose under uncertainty, while mapping represents geometry and meaning. SLAM jointly estimates both and relies on robust data association and loop closure. Path planning searches configuration space; graph search is effective in discretized environments, sampling methods address high-dimensional spaces, and optimization produces smooth, constrained trajectories. Mobile robots require planners that respect vehicle kinematics, while manipulators require attention to joint limits, singularities, grasp stability, and collision avoidance.

Human–robot interaction extends robotics beyond physical capability. Collaborative robots must communicate intent, interpret uncertain human behavior, support ergonomic work, and maintain safe authority boundaries. Shared autonomy combines human goals with robot execution, but must provide predictable control and reliable overrides. AI is most responsibly deployed when adaptive perception and planning are combined with independently validated safety functions, explicit uncertainty handling, representative testing, and careful governance.

Key Terms

Localization: Estimating a robot’s pose relative to a reference frame.

Mapping: Constructing a representation of the robot’s environment.

SLAM: Simultaneous localization and mapping.

Occupancy grid: A grid whose cells encode the probability of being occupied.

Configuration space: Space whose coordinates represent robot configurations.

Configuration-space obstacle: A region of configurations that causes collision.

Path: A geometric sequence of configurations.

Trajectory: A path together with timing and motion profiles.

A*: A heuristic graph-search algorithm.

RRT: A sampling-based planner for continuous configuration spaces.

Jacobian: A matrix relating joint velocities to task-space velocities.

Singularity: A configuration in which the Jacobian loses rank and motion or force capability is impaired.

Shared autonomy: Cooperation in which human and robot contribute to control or decision-making.

Intent recognition: Inferring a person’s goal from observations and context.

Ergonomics: Designing work systems to fit human physical and cognitive capabilities.

Collaborative robot: A robot designed and assessed for operation in proximity to people under specified safeguards.

Safety-rated monitored stop: A protective state that stops robot motion while monitoring conditions for restart.

Domain shift: A change between training conditions and deployment conditions.

Human-in-the-loop: A system in which a person contributes to supervision, decisions, or control.

Review Questions

Short Answer Questions

1. Distinguish localization from mapping.
2. What is the difference between a path and a trajectory?
3. Why is loop closure important in SLAM?
4. State two limitations of learning-based robotic perception.
5. What is meant by shared autonomy?

Long Answer Questions

1. Explain the perception–planning–action architecture of an autonomous mobile robot.
2. Compare graph-based, sampling-based, and optimization-based motion planning.
3. Discuss the role of uncertainty in localization, mapping, and human intent recognition.
4. Explain why collaborative robot safety requires more than obstacle detection.
5. Describe how ergonomics and interaction design influence acceptance of a robotic system.

Analytical / Application Questions

1. A warehouse robot experiences increasing pose error after travelling over a slippery floor. Propose a sensor-fusion and recovery strategy, explaining the assumptions and failure modes.
2. Design a planning architecture for a six-axis robot that must hand parts to a worker. Identify the global planner, local planner, safety functions, interaction signals, and evaluation metrics.
3. A vision system estimates two possible human intentions with similar probability. Explain how the robot should select an action using uncertainty, clarification, risk, and shared autonomy principles.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes a trajectory from a path?

A. A trajectory includes timing and motion variables, whereas a path is geometric.

B. A path always includes actuator torques, whereas a trajectory does not.

C. A trajectory is used only for mobile robots.

D. A path must be represented as a probability distribution.

Correct Answer: A. A trajectory includes timing and motion variables, whereas a path is geometric.

Question 2

For a differential-drive robot, which condition is assumed by the ideal wheel-odometry equations?

A. The robot moves only vertically.

B. There is no wheel slip and wheel parameters are known.

C. The robot has four independently steered wheels.

D. The sensor measurements are always noise-free.

Correct Answer: B. There is no wheel slip and wheel parameters are known.

Question 3

What is the principal purpose of loop closure in SLAM?

A. To increase motor voltage.

B. To detect actuator overheating.

C. To correct accumulated pose and map drift when a known place is revisited.

D. To remove all dynamic objects from camera images.

Correct Answer: C. To correct accumulated pose and map drift when a known place is revisited.

Question 4

Under what condition does A* retain optimality on a graph?

A. Its heuristic always overestimates the remaining cost.

B. Its heuristic is admissible and edge costs are nonnegative.

C. It uses random rather than systematic expansion.

D. It ignores the cost from the start node.

Correct Answer: B. Its heuristic is admissible and edge costs are nonnegative.

Question 5

What is a major limitation of the first solution produced by a basic RRT?

A. It can never operate in continuous spaces.

B. It is guaranteed to be globally shortest.

C. It may be feasible but inefficient or unsmooth.

D. It requires a complete occupancy grid at every joint angle.

Correct Answer: C. It may be feasible but inefficient or unsmooth.

Question 6

The manipulator Jacobian primarily relates:

A. Map cells to image pixels.

B. Joint velocities to end-effector velocities.

C. Battery voltage to wheel radius.

D. Human workload to cycle time.

Correct Answer: B. Joint velocities to end-effector velocities.

Question 7

Why should AI perception generally not be the only safety layer for a collaborative robot?

A. AI perception cannot process any sensor data.

B. Safety requires independent handling of foreseeable failures and hazardous states.

C. Collaborative robots do not use sensors.

D. Planning algorithms never interact with safety systems.

Correct Answer: B. Safety requires independent handling of foreseeable failures and hazardous states.

Question 8

In the intent model [[M:P(I | O) ∝ P(O | I)P(I)]], what does [[M:P(I)]] represent?

A. The prior probability of an intent.

B. The robot’s motor current.

C. The map resolution.

D. The collision penalty.

Correct Answer: A. The prior probability of an intent.

Question 9

Which design feature best supports predictable shared autonomy?

A. Hiding all autonomous decisions from the operator.

B. Removing the physical stop device.

C. Making authority, overrides, and control transitions clear.

D. Allowing the robot to change goals without notification.

Correct Answer: C. Making authority, overrides, and control transitions clear.

Question 10

Which practice is most appropriate when a robot’s perception confidence is low near a person?

A. Increase speed to finish before uncertainty grows.

B. Ignore the confidence estimate.

C. Pause or enter a safe fallback state and request clarification if needed.

D. Disable all safety monitoring.

Correct Answer: C. Pause or enter a safe fallback state and request clarification if needed.