8. Agents, Tools, and Generative AI Applications

Introduction and Context

Large language models (LLMs) initially appeared primarily as systems for generating and transforming text. They could answer questions, summarize documents, translate language, and produce code, but their operation was generally confined to the information contained in the prompt and the knowledge encoded during training. Modern generative AI systems increasingly go beyond this passive question-and-answer model. They can invoke external tools, retrieve current information, execute code, maintain task-specific memory, coordinate multiple steps, and interact with users and enterprise systems. Such systems are commonly described as tool-using systems, AI agents, or agentic applications.

An agentic application is not simply a larger language model. It is an engineered system in which a model interprets an objective, selects or recommends actions, obtains information, uses tools, evaluates intermediate results, and either continues or requests human intervention. The model provides flexible language understanding and reasoning, while software components provide access, control, validation, persistence, and security. This distinction is central to responsible design: an LLM may suggest an action, but the surrounding system determines whether that action is permitted and how it is executed.

This chapter examines the architecture and practical development of agents, tools, workflows, memory systems, and generative AI applications. It also considers code generation, conversational systems, content creation, analytics, decision support, human oversight, governance, and emerging developments. The emphasis is on designing reliable systems rather than treating an LLM as an autonomous substitute for organizational judgment.

Learning Objectives

After completing this chapter, students should be able to:

1. Distinguish between an LLM application, a workflow, and an agent.

2. Explain tool calling, planning, memory, retrieval, and orchestration.

3. Design a controlled architecture for a tool-using generative AI system.

4. Select suitable patterns for conversational, analytical, coding, and content-generation applications.

5. Evaluate agent quality using task, safety, operational, and human-centered criteria.

6. Identify risks involving hallucination, excessive autonomy, privacy, security, bias, and accountability.

7. Propose appropriate human-oversight and governance mechanisms.

From Prompting to Action: The Agentic Idea

A conventional LLM application can be represented as:

response = model(prompt, context)

The context may include retrieved documents, conversation history, system instructions, or structured data. In an agentic application, the system instead follows an iterative loop:

observe → interpret → plan → act → evaluate → continue or stop

The model may determine that it needs to search a knowledge base, query a database, call a weather service, calculate a value, or ask the user for clarification. The action is performed through a tool whose interface is defined by software. The resulting observation is returned to the model, which decides what to do next.

An agent therefore combines several capabilities:

• Goal interpretation: converting an expressed objective into a task representation.
• Planning: deciding which steps or tools may achieve the objective.
• Tool use: invoking external functions, APIs, databases, browsers, or code interpreters.
• State management: preserving relevant task information across steps.
• Reflection or evaluation: checking whether an intermediate result is complete or plausible.
• Interaction: communicating with users, systems, or other agents.
• Termination: recognizing when the task is complete or when escalation is required.

Not every application needs all these capabilities. A fixed workflow may be safer and more predictable than a free-form agent. The choice should follow the task’s uncertainty, risk, and need for flexibility.

Important Concepts and Terminology

A tool is a callable capability exposed to a model through a defined interface. Examples include a search function, a calculator, a ticket-creation API, or a database query service. A tool schema describes the tool’s name, purpose, input parameters, data types, and constraints. Structured schemas reduce ambiguity and make validation possible.

A workflow is a predetermined sequence or graph of operations. It may contain branching, retries, approval gates, and parallel activities, but the permitted paths are designed in advance. An agent has greater discretion to choose actions dynamically. The boundary is not absolute: many production systems combine fixed workflows with agentic components.

Planning refers to decomposing a goal into actions. Plans may be explicit, such as a numbered task list, or implicit, when the model selects the next action one step at a time. ReAct-style reasoning combines internal problem interpretation with external action and observation. In production, private reasoning should not be exposed indiscriminately; systems should record concise action traces and evidence instead.

Memory is information retained beyond a single model call. Short-term memory includes the current conversation and task state. Long-term or persistent memory may contain user preferences, prior cases, or organizational knowledge. Memory is not automatically trustworthy: it requires permissions, retention rules, provenance, correction mechanisms, and protection against malicious or irrelevant information.

Retrieval-augmented generation (RAG) supplies the model with relevant external information at inference time. A typical RAG pipeline embeds documents, retrieves similar passages, applies filters or reranking, and includes selected evidence in the prompt. RAG improves access to current or private information, but it does not guarantee that the model will use the evidence correctly.

Orchestration coordinates model calls, tools, state, retries, validation, and user interaction. Guardrails are preventive or detective controls that restrict unsafe inputs, outputs, tool calls, data access, and operational behavior. Human-in-the-loop (HITL) design requires human review at specified points. Human-on-the-loop design allows automated operation with monitoring and the possibility of intervention.

Architectural Patterns for Agentic Systems

A useful architecture separates the system into layers rather than allowing the model to control everything directly.

Interaction and task layer

The interaction layer receives a user request through a chat interface, voice channel, enterprise portal, or application programming interface. It authenticates the user, identifies the session, applies rate limits, and classifies the request. Task interpretation may identify the user’s goal, relevant entities, urgency, and required authorization.

Model and orchestration layer

The orchestration layer constructs prompts, selects models, maintains state, routes tasks, and controls the loop. It may use a fast model for classification, a stronger model for complex planning, and deterministic software for calculations. The orchestrator should impose limits on the number of steps, token usage, execution time, and financial cost.

Knowledge and memory layer

This layer includes document stores, vector indexes, relational databases, conversation state, user profiles, and event logs. Retrieval should enforce access controls before documents are passed to the model. Memory should be separated by purpose: temporary task state should not automatically become permanent user memory.

Tool and execution layer

Tools may be read-only or state-changing. Read-only tools include search and data retrieval. State-changing tools create orders, send messages, modify records, or execute transactions. The latter require stronger validation, authorization, idempotency controls, transaction limits, and often human approval.

Governance and observability layer

Logs should record requests, retrieved sources, tool calls, validation results, approvals, errors, latency, and final outcomes without unnecessarily storing sensitive content. Monitoring should support auditability and incident investigation. Policy enforcement should occur outside the model wherever possible, because a prompt instruction alone is not a reliable security boundary.

Tool Use and API Design

Tool design strongly affects agent reliability. A tool should perform a narrow, understandable function and expose explicit inputs and outputs. For example, instead of giving an agent unrestricted database access, an organization might provide a function called `get_customer_order_status` with validated customer and order identifiers. Narrow tools reduce accidental misuse and simplify auditing.

Effective tool interfaces generally include:

• precise names and descriptions;
• typed and validated parameters;
• explicit error messages;
• predictable response formats;
• authorization checks;
• timeouts and retry policies;
• idempotency for operations that may be repeated;
• limits on records, monetary value, or execution scope.

Tool responses should distinguish success, absence of data, invalid input, temporary failure, and authorization failure. If all errors are returned as natural-language messages, the model may misinterpret them and continue incorrectly. Structured error codes improve orchestration.

A robust design uses a two-stage approach for consequential actions. First, the model proposes an action in structured form. Second, deterministic software validates the proposal against business rules and permissions. Where risk is significant, a human approves the action before execution. For example, an agent may prepare a refund recommendation, but a policy engine and authorized employee determine whether the refund is issued.

Planning, Memory, and Workflow Control

Planning can be implemented at several levels. In a fixed workflow, the sequence is known: classify a request, retrieve records, generate a draft, validate it, and route it for approval. In a dynamic planner, the model chooses among tools according to the situation. A hybrid approach often provides the best balance: predetermined stages constrain the process, while the model handles interpretation within each stage.

Planning systems should address failure explicitly. A plan may be incomplete, impossible, or based on false assumptions. Useful controls include:

1. Precondition checks: verify that required information and permissions exist.

2. Step limits: prevent indefinite loops and uncontrolled cost.

3. Intermediate validation: check outputs before using them in later steps.

4. Alternative paths: define what happens when a tool fails.

5. Confidence or evidence requirements: require supporting data for important conclusions.

6. Escalation: transfer ambiguous or high-risk cases to a person.

Memory improves continuity but introduces risks. Conversation history may become too long, contain contradictory statements, or include sensitive data. A practical memory policy specifies what is retained, for how long, for which purpose, and how it can be corrected or deleted. Memory entries should include provenance and timestamps. A user preference inferred from one conversation should not automatically be treated as a permanent fact.

Application Domains

Conversational and service systems

Customer-service agents can classify requests, retrieve account information, explain policies, summarize interactions, and create support tickets. A safe design limits access according to the authenticated user, cites relevant policy information, and escalates disputes, vulnerable-user situations, or regulated decisions. Conversation quality is not measured only by fluency; resolution accuracy, transfer appropriateness, response time, and customer effort are also important.

Code generation and software engineering

Generative AI can generate functions, tests, documentation, database queries, and migration suggestions. Agentic coding systems may inspect a repository, run tests, diagnose errors, and propose changes. They should operate in isolated environments, use version control, avoid exposing secrets, and require review before merging. Generated code must be tested for correctness, security, licensing concerns, maintainability, and performance. Passing tests do not prove that code is safe or semantically appropriate.

Content creation

Systems can produce marketing drafts, reports, educational materials, product descriptions, and multimedia storyboards. Effective workflows separate ideation, drafting, factual verification, style control, rights review, and publication. Human editorial judgment remains important where content affects reputation, public understanding, or professional advice. Provenance records and disclosure policies may be necessary for synthetic media.

Analytics and decision support

An analytics agent can translate a natural-language question into a database query, generate visualizations, explain trends, and compare scenarios. The system must prevent unauthorized data access and validate generated queries. It should distinguish descriptive findings from causal claims and communicate uncertainty. Decision-support systems should present evidence, assumptions, alternatives, and limitations rather than disguising recommendations as objective conclusions.

Enterprise operations and knowledge work

Agents may assist with procurement, recruitment administration, compliance review, research, scheduling, and document processing. In each case, the organization must define the agent’s authority. Administrative automation may be appropriate for low-risk repetitive tasks, whereas employment, credit, healthcare, legal, and safety-related decisions require stronger controls, explainability, review, and compliance assessment.

Evaluating Agentic Applications

Evaluation must reflect the complete system, not merely the language quality of the model. Useful dimensions include:

• Task success: whether the intended objective was achieved.
• Factual and procedural accuracy: whether outputs and actions are correct.
• Tool selection accuracy: whether the appropriate tool was selected with valid parameters.
• Efficiency: number of steps, latency, token use, and operational cost.
• Robustness: performance under ambiguous inputs, tool failures, and changing data.
• Safety: resistance to unauthorized actions, prompt injection, data leakage, and harmful outputs.
• User experience: clarity, effort, trust calibration, and ease of escalation.
• Auditability: ability to reconstruct what happened and why.

Offline evaluation uses curated test cases, adversarial examples, and historical tasks. Online evaluation may use controlled pilots, shadow mode, sampled review, and carefully selected key performance indicators. Human evaluation remains necessary for qualities such as usefulness, tone, and appropriateness, but evaluators should use explicit rubrics. A high user-satisfaction score cannot compensate for a serious privacy or authorization failure.

Detailed Applied Case: An Illustrative Procurement Assistant

Background

Consider a fictional organization, Northbridge University, which manages thousands of routine purchases through a central procurement office. Staff members frequently ask about approved suppliers, spending limits, delivery status, and required forms. The university proposes an AI procurement assistant integrated with its internal purchasing system.

Situation and Problem

The existing help desk receives repetitive requests, while procurement officers spend substantial time searching policy documents and checking order records. The university wants faster responses but does not want an automated system to make unauthorized purchases or provide incorrect compliance advice.

Analysis

The requests fall into three categories. First, policy questions can often be answered through retrieval from approved documents. Second, order-status questions require authenticated access to a procurement database. Third, new purchase requests may lead to a financial commitment and therefore require policy validation and human authorization. These categories should not be handled with identical autonomy.

The main risks are inaccurate policy interpretation, disclosure of another employee’s order information, prompt injection through uploaded documents, duplicate purchase requests, and the assumption that a confident response represents an official approval.

Approach and Solution

Northbridge designs a hybrid workflow. The assistant first authenticates the user and classifies the request. For policy questions, it retrieves current documents, displays citations, and states when the policy does not resolve the question. For order-status questions, a narrow read-only API returns only records that the user is authorized to view.

For a new purchase, the assistant collects structured information and generates a draft request. A rules engine checks spending thresholds, supplier eligibility, budget codes, and required approvals. The agent cannot directly create a financial commitment. Instead, it submits the validated draft to a procurement officer, who reviews and approves it through the existing system.

The system uses short-lived task memory, retains an audit trail of sources and actions, and applies limits to tool calls. Uploaded documents are treated as untrusted content; instructions found inside them cannot modify system permissions. Monitoring tracks retrieval quality, unauthorized-access attempts, escalation rates, and incorrect answers identified through sampling.

Outcome

In the illustrative deployment, routine policy questions are resolved more quickly, while complex or high-value requests are routed to staff with a structured summary. The system does not eliminate procurement expertise; it concentrates that expertise on exceptions, negotiation, and oversight. Its success is assessed through resolution accuracy, appropriate escalation, policy citation quality, and absence of unauthorized transactions rather than through conversational fluency alone.

Learning Points

The case demonstrates four principles. First, autonomy should be proportional to risk. Second, tools should expose limited capabilities rather than unrestricted system access. Third, deterministic validation and human approval are essential for consequential actions. Fourth, an agent is an organizational process as much as it is a model: policies, permissions, monitoring, and accountability determine its real behavior.

Risks, Limitations, and Common Mistakes

The most familiar limitation is hallucination: the model may produce an unsupported answer or invent a tool result. Retrieval reduces but does not remove this risk. Systems should require evidence, distinguish unknown from known, and test answers against authoritative sources.

Prompt injection is another major risk. Malicious instructions may appear in user input, retrieved documents, websites, emails, or tool outputs. An agent that treats all text as instructions may reveal data or misuse tools. Trusted instructions, untrusted content, and executable actions should be separated. Permissions must be enforced by software, not merely by prompts.

Excessive autonomy can cause financial loss, privacy violations, or operational disruption. Other risks include insecure generated code, data poisoning, biased recommendations, model drift, excessive dependence on a single provider, and unpredictable cost. Long agent loops may amplify a small initial error. Memory can preserve incorrect or sensitive information. Multi-agent systems may increase complexity without providing meaningful benefit.

Common implementation mistakes include starting with a highly autonomous design before defining the business process, granting broad API access, omitting approval gates, evaluating only on impressive demonstrations, failing to test rare failures, and storing all conversations indefinitely. Organizations should begin with bounded use cases and expand autonomy only when evidence supports it.

Ethical, Governance, and Professional Considerations

Agentic systems raise questions of responsibility: who is accountable when an agent makes an error, the model provider, application developer, deploying organization, or approving employee? Accountability cannot be delegated to the model. Organizations should assign owners for data, tools, policies, incidents, and final decisions.

Governance should address purpose limitation, data minimization, access control, retention, transparency, auditability, bias assessment, accessibility, and incident response. Users should know when they are interacting with an AI system and when a human is available. High-impact decisions require special attention to due process, contestability, discrimination, and documentation.

Professional practice also requires respecting confidentiality, intellectual property, software licensing, and sector-specific obligations. Human review should be meaningful rather than ceremonial: reviewers need sufficient evidence, authority to reject the recommendation, and enough time and expertise to evaluate it. Excessive automation may create automation bias, in which people accept machine recommendations without appropriate scrutiny.

Emerging Directions

Current development is moving toward smaller specialized models, multimodal agents, structured outputs, stronger retrieval, and more capable code-execution environments. Model Context Protocol-like approaches and related interface standards seek to make connections between models and tools more consistent, although interoperability does not remove governance requirements. Long-running agents are also being explored for research, software maintenance, and operational monitoring.

Another important direction is the rise of evaluation and observability platforms that trace prompts, retrieved passages, tool calls, latency, costs, and outcomes. Organizations are increasingly treating agentic applications as socio-technical systems requiring versioning, testing, red-teaming, change management, and post-deployment monitoring. The likely mature pattern is not unrestricted autonomy, but graduated autonomy: systems earn broader permissions through demonstrated reliability and remain subject to policy and human control.

Best Practices for Design and Deployment

A disciplined development process can be summarized as follows:

1. Define the business objective, acceptable risk, and success criteria before selecting a model.

2. Decide whether a deterministic workflow, an agent, or a hybrid architecture is appropriate.

3. Begin with read-only and low-risk tools; add state-changing actions gradually.

4. Use typed schemas, least-privilege authorization, validation, timeouts, and idempotency.

5. Separate trusted instructions from untrusted retrieved or user-generated content.

6. Require evidence and citations for factual or policy-sensitive answers.

7. Add approval gates for financial, legal, safety, employment, healthcare, and reputational decisions.

8. Maintain useful logs while minimizing unnecessary retention of personal or confidential data.

9. Evaluate normal cases, edge cases, adversarial cases, and tool failures.

10. Monitor production behavior and revise prompts, tools, policies, and workflows as evidence develops.

Chapter Summary

Agents extend generative AI from response generation to controlled action. They combine language models with planning, tools, memory, retrieval, orchestration, validation, and human oversight. Workflows provide predictability, while agents provide flexibility; hybrid designs are often appropriate for organizational applications. Tool schemas, least-privilege access, structured errors, step limits, and approval gates are central to reliability. Applications include conversational service, code generation, content creation, analytics, decision support, and enterprise operations. Evaluation must include task success, factual accuracy, safety, efficiency, user experience, and auditability. Hallucination, prompt injection, privacy loss, automation bias, and excessive autonomy require technical and organizational controls. Responsible deployment treats an agent as a socio-technical system whose authority, evidence, monitoring, and accountability are deliberately designed.

Key Terms

Agent: A software system that interprets goals, selects actions, uses tools, and works toward an outcome.

Tool: A callable external capability made available to a model through a defined interface.

Tool schema: A structured description of a tool’s inputs, outputs, purpose, and constraints.

Workflow: A designed sequence or graph of operations with defined control paths.

Planning: Decomposing a goal into actions or selecting the next action required for progress.

Orchestration: Coordination of model calls, tools, state, validation, retries, and user interaction.

Memory: Information retained across model calls or sessions for task continuity or personalization.

Retrieval-augmented generation: Generation supported by information retrieved from an external knowledge source.

Guardrail: A control that prevents, detects, or limits unsafe or unauthorized behavior.

Human-in-the-loop: A design in which a person reviews or approves specified system actions.

Human-on-the-loop: A design in which automation operates while humans monitor and can intervene.

Prompt injection: An attempt to manipulate system behavior through malicious or conflicting instructions in input or retrieved content.

Least privilege: Granting only the minimum access required to perform an authorized task.

Idempotency: The property that repeating an operation does not create unintended additional effects.

Structured output: A response constrained to a specified format, such as a schema or JSON object.

Agent trace: A record of relevant model decisions, tool calls, observations, validations, and outcomes.

Automation bias: The tendency to accept automated recommendations without sufficient independent evaluation.

Graduated autonomy: Incrementally expanding system permissions according to demonstrated reliability and risk controls.

Review Questions

Short Answer Questions

1. What distinguishes a workflow from an agent?

2. Why are tool schemas important in agentic applications?

3. Differentiate short-term memory from persistent memory.

4. What is the purpose of an approval gate?

5. Define prompt injection in the context of tool-using systems.

Long Answer Questions

1. Explain the major layers of a secure agentic application architecture.

2. Discuss how planning, retrieval, memory, and orchestration interact in a multi-step task.

3. Examine the use of generative AI for code generation and identify the controls required before deployment.

4. Explain why agent evaluation must go beyond language fluency and list appropriate evaluation dimensions.

5. Discuss the ethical and governance responsibilities of an organization deploying an AI agent.

Analytical / Application Questions

1. A university agent can answer policy questions and submit purchase orders. Design a hybrid workflow that limits unauthorized financial actions while preserving efficiency.

2. An analytics agent produces a confident sales forecast from incomplete data. Analyze the risks and propose technical, procedural, and communication controls.

3. A customer-service agent repeatedly follows instructions embedded in uploaded documents. Diagnose the security problem and redesign the instruction, retrieval, and tool-execution boundaries.

22. Multiple Choice Questions with Answers

Question 1

Which feature most clearly distinguishes an agent from a simple LLM question-answering application?

A. It always produces longer responses.

B. It can select actions, use tools, and evaluate intermediate results.

C. It uses a larger vocabulary.

D. It stores every conversation permanently.

Correct Answer: B. It can select actions, use tools, and evaluate intermediate results.

Question 2

Why should a state-changing tool such as a refund API normally have stronger controls than a read-only search tool?

A. State-changing tools require more screen space.

B. Read-only tools cannot return text.

C. State-changing tools can create consequential external effects.

D. Search tools are always more expensive.

Correct Answer: C. State-changing tools can create consequential external effects.

Question 3

Which architecture best limits an agent’s ability to make unauthorized purchases?

A. Give the model unrestricted database and payment access.

B. Allow the model to approve its own proposed transactions.

C. Use narrow APIs, deterministic policy checks, and human approval for consequential actions.

D. Remove all logging to improve speed.

Correct Answer: C. Use narrow APIs, deterministic policy checks, and human approval for consequential actions.

Question 4

What is the primary purpose of retrieval-augmented generation in an enterprise agent?

A. To eliminate the need for access controls.

B. To provide relevant external or current information at inference time.

C. To guarantee that every generated answer is correct.

D. To permanently retrain the model after each query.

Correct Answer: B. To provide relevant external or current information at inference time.

Question 5

An agent repeatedly calls a failing API until its token budget is exhausted. Which control most directly addresses this problem?

A. Increase the maximum number of retries indefinitely.

B. Add step limits, timeouts, and explicit failure-handling paths.

C. Remove structured error codes.

D. Store more conversation history.

Correct Answer: B. Add step limits, timeouts, and explicit failure-handling paths.

Question 6

Why should memory entries include provenance and timestamps?

A. To make the model’s responses longer.

B. To help determine the origin, relevance, and currency of retained information.

C. To avoid authenticating users.

D. To convert all memories into training data automatically.

Correct Answer: B. To help determine the origin, relevance, and currency of retained information.

Question 7

Which evaluation result would provide the strongest evidence that a procurement agent is safe to deploy?

A. Users report that its tone is friendly.

B. The agent generates creative explanations.

C. It achieves accurate resolutions while preventing unauthorized transactions and escalating high-risk cases.

D. It uses the maximum available context window.

Correct Answer: C. It achieves accurate resolutions while preventing unauthorized transactions and escalating high-risk cases.

Question 8

A malicious instruction is hidden in a retrieved webpage and tells the agent to reveal confidential records. What security principle is most relevant?

A. Treat all retrieved content as trusted system instructions.

B. Separate untrusted content from trusted instructions and enforce permissions outside the model.

C. Allow the webpage to redefine tool permissions.

D. Disable audit logging.

Correct Answer: B. Separate untrusted content from trusted instructions and enforce permissions outside the model.

Question 9

Which task is most suitable for a hybrid workflow rather than unrestricted agent autonomy?

A. Drafting alternative marketing slogans.

B. Summarizing a publicly available article.

C. Preparing a high-value financial transaction subject to policy approval.

D. Reformatting a user-provided paragraph.

Correct Answer: C. Preparing a high-value financial transaction subject to policy approval.

Question 10

What does graduated autonomy mean in agent deployment?

A. Allowing the agent to operate without any monitoring after launch.

B. Increasing permissions incrementally as reliability and controls are demonstrated.

C. Using multiple models for every task regardless of risk.

D. Replacing all human decisions with automated recommendations.

Correct Answer: B. Increasing permissions incrementally as reliability and controls are demonstrated.