AI-Assisted Engineering and Emerging Futures

Introduction

Generative artificial intelligence (AI) is changing how software is specified, designed, implemented, tested, documented, deployed, and maintained. Tools based on large language models can generate source code, explain unfamiliar systems, propose test cases, transform natural-language requirements into structured artefacts, and assist engineers during incident response. However, generated output is not automatically correct, secure, maintainable, or legally usable. AI-assisted engineering therefore requires more than tool adoption: it requires disciplined validation, traceability, security controls, human accountability, and an understanding of where automation is appropriate.

This chapter examines generative AI as an engineering capability rather than as a replacement for software engineering. It also considers related developments: low-code and no-code platforms, autonomous software agents, platform engineering, and emerging software delivery models. The central principle is that responsibility remains with the engineering organization and its accountable professionals, even when an AI system produces the immediate artefact.

Learning Objectives

By the end of this chapter, students should be able to:

1. Explain how generative AI can support requirements, coding, testing, documentation, and maintenance.
2. Distinguish assistance from delegation of engineering responsibility.
3. Design validation, security, provenance, and review controls for AI-generated artefacts.
4. Evaluate low-code platforms, autonomous agents, and platform engineering in context.
5. Analyse workforce, ethical, legal, and organizational implications.
6. Recommend an appropriate future software-delivery model for a given organization.

1. From Automation to AI-Assisted Engineering

Traditional automation follows explicitly programmed rules. Generative AI differs because it infers likely outputs from patterns learned from large datasets. A model may receive a prompt containing a requirement and produce prose, code, diagrams, test cases, or structured data. Its output is probabilistic: the same request can produce different results, and fluent language does not guarantee factual accuracy.

AI-assisted engineering is best understood as a socio-technical system involving:

• a human or team with a defined engineering objective;
• an AI model and its surrounding toolchain;
• project context such as repositories, specifications, policies, and telemetry;
• review and validation activities; and
• organizational accountability.

The model is useful for reducing mechanical effort and expanding the range of alternatives considered. It is less reliable when context is incomplete, requirements conflict, domain consequences are severe, or correctness depends on information absent from the prompt.

A useful distinction is between augmentation and autonomy. Augmentation keeps humans responsible for decisions while AI accelerates analysis or production. Autonomy allows an agent to select actions, invoke tools, and pursue a goal with limited intervention. Autonomy can increase throughput, but it also increases the potential impact of errors. The appropriate degree of autonomy depends on risk, reversibility, observability, and the quality of available controls.

2. Generative AI Across the Software Life Cycle

2.1 Requirements engineering

AI can support requirements work by:

• converting interview transcripts into candidate user stories;
• identifying ambiguous, contradictory, or missing requirements;
• generating acceptance criteria and edge cases;
• clustering stakeholder feedback;
• translating requirements into alternative formulations; and
• checking whether requirements are testable and traceable.

For example, an AI system may transform “users should receive fast notifications” into a candidate requirement such as “For 95% of notification requests under normal operating load, delivery shall begin within two seconds.” This is only a draft. Stakeholders must define “normal load,” the measurement location, the notification channels, and the consequences of failure.

AI should not silently decide business priorities, safety boundaries, privacy expectations, or legal obligations. Requirements are agreements among stakeholders, not merely text-generation tasks. A strong workflow stores the source evidence, the generated proposal, reviewer decisions, and the final approved requirement.

2.2 Design and architecture

Generative AI can propose architectural alternatives, explain patterns, generate interface contracts, and identify potential failure modes. It can also compare options such as a monolith and a set of services. Its suggestions should be evaluated against quality attributes including security, availability, performance, operability, accessibility, cost, and maintainability.

A common failure is accepting a familiar architecture because the model produces a persuasive explanation. Architecture decisions require explicit constraints and trade-offs. An AI-generated design record should therefore state assumptions, rejected alternatives, risks, and evidence supporting the decision.

2.3 Coding

Coding assistants can generate boilerplate, translate code between languages, complete functions, explain compiler errors, and suggest refactorings. Their value is highest when the developer supplies precise context and works within a tested repository. Their risks include:

• incorrect APIs or fabricated libraries;
• insecure defaults;
• overlooked error handling;
• hidden performance costs;
• licensing or provenance uncertainty; and
• code that passes superficial review but violates domain rules.

Generated code should be treated as an untrusted contribution. Compilation proves only that code satisfies certain syntactic and type constraints; it does not prove business correctness or security.

2.4 Testing

AI can generate unit tests, property-based test ideas, test data, mocks, regression tests, and exploratory test charters. It can identify equivalence classes and boundary conditions from a specification. It can also summarize failures and suggest likely causes.

A major danger is test imitation: an AI may generate tests that repeat the implementation’s assumptions rather than challenge them. For example, if a defective function returns an incorrect tax rate, generated tests may reproduce that rate and report success. Independent oracles, domain examples, mutation testing, contract tests, and human-designed boundary cases help reduce this risk.

Testing remains an evidence-gathering activity. A passing AI-generated test suite is not evidence of correctness unless the tests have adequate coverage of relevant behaviours and their expected results are independently justified.

2.5 Documentation and knowledge management

AI can produce API descriptions, release notes, onboarding guides, code comments, runbooks, and summaries of technical discussions. This is particularly valuable for legacy systems, where documentation is incomplete.

Documentation must be checked against the running system. Generated explanations may describe intended behaviour rather than actual behaviour, and summaries may omit exceptions. Documentation pipelines should identify the source version, generation date, model or tool, reviewer, and any unresolved uncertainty.

2.6 Maintenance and operations

In maintenance, AI can assist with dependency analysis, issue triage, log summarization, root-cause hypotheses, migration planning, and code modernization. An operational agent might detect an alert, inspect logs, propose a rollback, and open a change request.

High-impact production actions require graduated authorization. Read-only analysis may be automated, while changes to authentication, financial calculations, data deletion, or infrastructure require human approval and independent checks. Every action should be recorded so that engineers can reconstruct what happened.

3. Validation: From Plausibility to Evidence

Validation asks whether the software solves the right problem; verification asks whether an artefact conforms to its specified properties. AI-assisted development needs both. Fluent output creates a special risk because people may confuse plausibility with evidence.

A practical validation loop is:

1. Define the task, constraints, and acceptance conditions.
2. Generate one or more candidate outputs.
3. Inspect the output for factual, logical, security, and domain errors.
4. Execute automated checks in a controlled environment.
5. Compare results with independent expected behaviour.
6. Obtain review appropriate to the risk.
7. Record the evidence, decision, and provenance.

Useful controls include static analysis, type checking, dependency scanning, unit and integration testing, contract testing, security testing, performance testing, sandbox execution, and staged deployment. For critical systems, formal specifications or proofs may be appropriate for selected components, although they do not eliminate the need to validate the surrounding system.

One way to express test effectiveness is the defect detection rate:

Here, d_f is the number of relevant defects found by the test activity and d_e is the number found later by another trusted activity or in operation. The measure is meaningful only when the later discovery process is sufficiently observed and the defect populations are comparable. It is not a universal measure of software quality; a low value may indicate weak tests, poor observability, or incomplete defect recording.

Coverage metrics should be interpreted carefully. Statement coverage indicates which executable statements ran; branch coverage indicates which decision outcomes ran. Neither demonstrates that requirements, security properties, or combinations of conditions are correct. AI can increase coverage while decreasing test validity if it generates repetitive tests.

4. Security and Privacy

AI-assisted engineering introduces security risks at both the model boundary and the generated-artefact boundary.

Prompt and context risks

Prompts may contain source code, credentials, customer information, security findings, or proprietary designs. Organizations need data-classification rules, approved tools, retention controls, access management, and clear policies on whether submitted data is used for model training. Secrets must never be placed in prompts or repositories merely because an assistant requests them.

Output risks

Generated code may contain injection vulnerabilities, weak cryptography, insecure deserialization, excessive permissions, or unsafe file handling. Security review must therefore be based on the actual code and its execution context, not on the model’s explanation of that code.

Adversarial risks

An attacker may place malicious instructions in a document, issue, web page, or repository file that an agent reads. This is commonly described as indirect prompt injection. Agents should treat retrieved content as data, separate instructions from evidence, restrict tool permissions, and require approval for consequential actions.

Secure operating model

A defensible operating model includes:

• least-privilege identities for models and agents;
• isolated execution environments;
• allow-listed tools and repositories;
• secret scanning and dependency verification;
• input and output filtering;
• audit logs;
• rollback mechanisms; and
• human approval for high-impact actions.

Security is not achieved by adding a disclaimer to a prompt. It is achieved through architecture, access control, testing, monitoring, and organizational practice.

5. Provenance, Intellectual Property, and Accountability

Provenance is information about the origin, transformation, and custody of an artefact. In AI-assisted development, provenance may include the human author, model or tool, prompt or task description, retrieved sources, repository revision, generated output, edits, tests, and approvals.

Provenance supports reproducibility, incident investigation, licensing review, and accountability. It does not require retaining every private chain of thought from a model. Engineering records should preserve useful evidence—inputs, outputs, decisions, and validation results—without collecting unnecessary personal or sensitive information.

Organizations should establish policies for:

• acceptable use of external and internal models;
• open-source licence compatibility;
• attribution and copyright review;
• protection of confidential information;
• retention and deletion of AI interaction records;
• responsibility for generated artefacts; and
• disclosure where AI use is material to a customer or regulator.

Legal rules differ across jurisdictions and change over time. Engineers should not treat a model’s statement about ownership or licensing as legal advice. A generated component must be reviewed under the organization’s applicable legal and procurement processes.

6. Autonomous Agents and Control Boundaries

An autonomous software agent typically combines a model with memory, planning, tool invocation, and feedback. A simple agent loop is:

1. Observe a goal and available context.
2. Plan a next action.
3. Invoke a tool such as a compiler, browser, test runner, or deployment system.
4. Observe the result.
5. Revise the plan or stop.

This loop is powerful because the agent can perform multi-step work. It is dangerous because an incorrect plan can be repeated at speed. The engineering question is not whether an agent is “intelligent,” but whether its action space is bounded and its failure modes are controlled.

A useful control classification is:

Agents should have bounded goals, time limits, budget limits, tool allow-lists, transaction isolation, and an emergency stop. Evaluation should include normal tasks, ambiguous instructions, malicious inputs, tool failures, and recovery from partial completion.

7. Low-Code and No-Code Platforms

Low-code platforms provide visual development, declarative configuration, reusable components, and limited conventional coding. No-code platforms target users who build applications primarily through configuration. These approaches can shorten delivery time for forms, workflows, dashboards, and departmental applications.

Their benefits include rapid prototyping, participation by domain experts, standardized deployment, and reduced boilerplate. Their limitations include vendor lock-in, constrained architectures, opaque generated code, integration difficulty, performance limits, licensing costs, and governance problems when unsanctioned applications proliferate.

Low-code does not remove engineering. It relocates engineering decisions to data models, permissions, integration contracts, deployment settings, and platform governance. Professional teams should maintain inventories, security reviews, backup and export plans, environment separation, testing standards, accessibility checks, and ownership records.

Low-code is most appropriate when requirements fit the platform’s capabilities and the consequences of failure are manageable. It is less suitable when the system requires unusual algorithms, stringent latency, deep infrastructure control, or highly specialized security properties.

8. Platform Engineering and Internal Developer Platforms

Platform engineering creates internal products that provide developers with reliable “golden paths” for building and operating software. A platform may offer templates, identity integration, observability, deployment pipelines, databases, secrets management, and policy enforcement through self-service interfaces.

The platform team is not merely an infrastructure help desk. It designs an enabling product, studies developer needs, publishes service-level expectations, and measures adoption and outcomes. A good platform hides unnecessary complexity without hiding important consequences.

AI assistants are increasingly embedded in these platforms. For example, a developer may request a service template, receive generated infrastructure configuration, and obtain a pull request with tests and monitoring. The platform should constrain the generated result through policy-as-code, approved templates, security scanning, and cost controls.

Platform engineering can reduce cognitive load, but excessive standardization can obstruct legitimate innovation. Teams need an exception process and an architecture governance model that distinguishes risk-based controls from arbitrary preference.

9. Emerging Software Delivery Models

Several delivery models are developing simultaneously:

• AI-augmented teams: humans own goals and decisions while assistants accelerate routine work.
• Agentic delivery: agents perform multi-step development tasks under explicit permissions and review gates.
• Product-centred platforms: internal platforms provide reusable capabilities as products rather than isolated tools.
• Composable delivery: applications are assembled from managed services, APIs, reusable components, and low-code modules.
• Continuous verification: deployment decisions use automated evidence about tests, security, policy, performance, and operational health.
• Outcome-oriented engineering: teams measure customer and business outcomes alongside delivery speed.

These models are not mutually exclusive. A mature organization may use low-code for internal workflows, a platform for common deployment paths, coding assistants for implementation, and agents for low-risk maintenance.

A balanced evaluation should consider delivery lead time, reliability, change failure rate, recovery time, security incidents, operational cost, developer experience, accessibility, and customer outcomes. Speed alone is an inadequate measure: accelerating the production of defects is not engineering improvement.

10. Workforce and Professional Implications

AI changes tasks before it eliminates occupations. Routine code production may become less valuable, while skills in problem framing, architecture, verification, security, data governance, communication, and domain analysis become more important. Entry-level developers may face a particular challenge if assistants remove simple tasks that traditionally provided practice. Organizations should therefore design learning pathways rather than assuming that tool access creates competence.

Professional judgement remains essential in:

• negotiating ambiguous requirements;
• identifying unacceptable risks;
• understanding social and organizational context;
• deciding what evidence is sufficient; and
• taking responsibility for consequences.

Teams also need psychological safety. If productivity monitoring treats every generated line as an individual performance measure, developers may conceal AI use or optimize for volume rather than quality. Policies should focus on outcomes, review quality, security, and learning.

Bias and accessibility must be considered. Training data and generated outputs may reproduce stereotypes, exclude users with disabilities, or assume dominant cultural practices. Requirements and acceptance tests should explicitly include accessibility and inclusion, and affected users should participate in evaluation.

11. Illustrative Case: An AI-Assisted Benefits Portal

Background

A fictional public-sector organization operates a benefits portal used by citizens, case workers, and payment administrators. The existing system has incomplete documentation, a large test backlog, and frequent support requests. The organization proposes an AI assistant for code explanation, test generation, and incident triage.

Problem

The portal handles personal and financial information. Incorrect eligibility calculations or unauthorized disclosure could harm citizens and create legal consequences. The organization must improve maintenance speed without allowing generated code or agent actions to bypass controls.

Analysis

The highest-value, lower-risk uses are read-only code summarization, documentation discovery, duplicate issue detection, and suggestions for tests that are reviewed by domain experts. The highest-risk uses are changing eligibility rules, modifying access control, and deploying directly to production.

The team classifies data, prohibits personal records in external prompts, and deploys an approved model in a controlled environment. Repository access is read-only by default. Generated code can be proposed in a branch but cannot merge without peer review, automated checks, security scanning, and an approval from a benefits-domain specialist.

Approach and solution

The team establishes a traceability record connecting each change to a requirement, source issue, generated contribution, reviewer, test evidence, and deployment. It creates independent golden test cases from legislation and historical decisions. Mutation testing is used on eligibility calculations to determine whether tests detect deliberate defects. An agent may summarize production logs and draft an incident report, but rollback or data changes require explicit authorization.

The platform team supplies standard pipelines, dependency controls, secrets management, observability, and a deployment template. A quarterly review examines false suggestions, security findings, demographic impacts, and whether the assistant is causing over-reliance.

Outcome

The fictional organization reduces documentation time and improves issue triage while retaining human control over legally significant decisions. Some generated tests are rejected because their expected results reflect existing implementation rather than policy. This finding demonstrates why independent domain oracles are necessary.

Learning points

1. Risk classification should determine the level of autonomy.
2. Generated output must be tested against independent evidence.
3. Provenance makes review and incident investigation possible.
4. Platform controls can make safe practices easier to follow.
5. AI adoption is an organizational change programme, not merely a software purchase.

12. Best-Practice Framework

A practical governance framework can be organized into five questions:

1. Purpose: What problem is AI solving, and why is it preferable to a conventional method?
2. Boundaries: What data, tools, repositories, environments, and actions may the system access?
3. Evidence: Which tests, reviews, simulations, and monitoring demonstrate acceptable behaviour?
4. Accountability: Who approves, owns, audits, and responds to failures?
5. Learning: How will incidents, user feedback, and performance data change the system?

Before deployment, teams should create an AI use-case register, classify risk, define human decision points, and establish data and retention rules. During operation, they should monitor quality, security, drift in prompts or models, unexpected costs, and user over-reliance. After incidents, they should preserve relevant records, conduct blameless analysis, and update controls.

Challenges and Limitations

Generative AI can hallucinate facts, reproduce bias, expose confidential information, generate vulnerable code, and encourage shallow understanding. Model updates can change behaviour and reduce reproducibility. Evaluation benchmarks may not represent the organization’s domain. Vendors may provide limited visibility into training data or internal safeguards. Low-code and agentic systems can create hidden dependencies and concentration risk. Finally, productivity claims are often difficult to compare because task complexity and review effort vary.

These limitations do not imply that AI should be rejected. They imply that adoption must be evidence-based, risk-sensitive, and reversible. Small pilots with measurable objectives are preferable to uncontrolled organization-wide deployment.

Chapter Summary

AI-assisted engineering applies generative models to requirements, architecture, coding, testing, documentation, maintenance, and operations. Its value lies in augmentation, acceleration, and the generation of alternatives, but generated output is not evidence of correctness. Validation requires independent expectations, automated checks, human review, traceability, and risk-appropriate authorization.

Security controls must protect prompts, context, tools, and outputs. Provenance records support accountability, reproducibility, and legal review. Autonomous agents require bounded permissions, sandboxing, observability, approval gates, and rollback. Low-code platforms can accelerate suitable applications but require governance, portability planning, and professional engineering. Platform engineering provides self-service golden paths and policy controls, while emerging delivery models combine AI, agents, platforms, reusable services, and continuous verification.

The future role of software professionals will emphasize problem framing, system thinking, security, verification, communication, domain knowledge, and ethical judgement. Effective organizations will measure outcomes and reliability rather than merely counting generated code, and will treat AI adoption as a socio-technical transformation.

Key Terms

AI-assisted engineering: Use of AI tools to support software engineering activities while retaining human and organizational accountability.

Autonomous agent: A system that plans and performs multi-step actions through tools with limited human intervention.

Augmentation: Use of AI to extend human capability without transferring final responsibility.

Generative AI: AI that produces new text, code, images, structured data, or other artefacts from learned patterns.

Hallucination: A plausible-looking but unsupported or incorrect model output.

Human-in-the-loop: A design in which a person reviews or authorizes specified system actions.

Indirect prompt injection: Malicious or misleading instructions embedded in content that an AI system retrieves.

Low-code platform: A development platform using visual and declarative construction with limited conventional coding.

No-code platform: A platform intended to build applications primarily through configuration rather than programming.

Platform engineering: Development of internal platforms that provide reusable, governed capabilities to software teams.

Provenance: Information about an artefact’s origin, transformations, inputs, and approvals.

Prompt: An instruction or contextual input provided to a generative model.

Retrieval-augmented generation: Generation supported by retrieving relevant external or organizational information.

Sandbox: An isolated environment that limits the effects of code or agent actions.

Traceability: The ability to connect requirements, design, code, tests, decisions, and releases.

Validation: Evaluation of whether a system meets stakeholder and real-world needs.

Verification: Evaluation of whether an artefact conforms to specified properties.

Golden path: A recommended, supported route through a platform for completing a common engineering task.

Review Questions

Short Answer Questions

1. Why should AI-generated code be treated as an untrusted contribution?
2. Distinguish verification from validation in AI-assisted engineering.
3. What information can be included in a provenance record?
4. State two risks associated with low-code platforms.
5. Why should high-impact agent actions require explicit authorization?

Long Answer Questions

1. Explain how generative AI can support each major activity in the software life cycle, and identify a limitation for each activity.
2. Design a governance process for using an AI coding assistant in a financial application.
3. Compare augmentation and autonomous agents in terms of productivity, risk, accountability, and required controls.
4. Explain how platform engineering can improve the safety and consistency of AI-assisted delivery.
5. Discuss the workforce and professional implications of widespread generative AI adoption in software engineering.

Analytical / Application Questions

1. A team reports that AI increased test coverage from 60% to 90%, but production defects also increased. Analyse at least four possible explanations and propose an investigation plan.
2. Design an autonomy policy for an agent that manages cloud infrastructure. Classify at least five actions by risk and specify suitable permissions and approval gates.
3. A university wants to build a low-code student-support system using an AI assistant. Recommend whether and how it should proceed, considering privacy, accessibility, vendor dependence, testing, provenance, and human oversight.

Multiple Choice Questions with Answers

Question 1

Which statement best describes AI-assisted engineering?

A. Replacing all software engineers with autonomous models

B. Using AI to support engineering work while retaining accountability and validation

C. Generating code without requirements or testing

D. Using only visual programming tools

Correct Answer: B. Using AI to support engineering work while retaining accountability and validation

Question 2

Why is compilation insufficient evidence that AI-generated code is correct?

A. Compilers cannot detect syntax errors

B. Compilation proves business rules and security properties

C. Compilation checks limited technical constraints, not all intended behaviour

D. Compiled code cannot be tested

Correct Answer: C. Compilation checks limited technical constraints, not all intended behaviour

Question 3

Which practice provides the strongest independent check on AI-generated eligibility logic?

A. Asking the same model whether its code is correct

B. Using domain-derived expected cases based on policy

C. Increasing the prompt length

D. Removing human review

Correct Answer: B. Using domain-derived expected cases based on policy

Question 4

What is provenance primarily used to support?

A. Increasing processor speed

B. Replacing access control

C. Tracing an artefact’s origins, transformations, and approvals

D. Guaranteeing that a model never makes an error

Correct Answer: C. Tracing an artefact’s origins, transformations, and approvals

Question 5

Which is an example of indirect prompt injection?

A. A developer writes a clear requirement

B. A retrieved repository file contains instructions that manipulate an agent’s actions

C. A compiler reports a type error

D. A tester creates a boundary-value case

Correct Answer: B. A retrieved repository file contains instructions that manipulate an agent’s actions

Question 6

Which agent action normally requires the strongest authorization?

A. Searching a source repository

B. Summarizing logs

C. Creating a draft pull request

D. Deleting production data

Correct Answer: D. Deleting production data

Question 7

What is a principal benefit of platform engineering?

A. Eliminating the need for software testing

B. Providing reusable, governed paths for common engineering tasks

C. Allowing every team to bypass security policies

D. Preventing all architectural variation

Correct Answer: B. Providing reusable, governed paths for common engineering tasks

Question 8

Which statement about low-code platforms is most accurate?

A. They remove the need for engineering governance

B. They are suitable for every performance-critical system

C. They can accelerate suitable applications but may introduce lock-in and governance risks

D. They cannot integrate with external systems

Correct Answer: C. They can accelerate suitable applications but may introduce lock-in and governance risks

Question 9

A test suite has 30 defects found during testing and 10 found later. Using D = d_f ÷ (d_f + d_e), what is the defect detection rate?

A. 25%

B. 40%

C. 75%

D. obsoleted

Correct Answer: C. 75%

Question 10

Which measure is most appropriate for evaluating a future AI-assisted delivery model?

A. Lines of generated code alone

B. Number of prompts submitted alone

C. Delivery speed together with reliability, security, recovery, cost, and customer outcomes

D. Number of developers replaced

Correct Answer: C. Delivery speed together with reliability, security, recovery, cost, and customer outcomes