7. Back-End Services and API Architecture

Introduction and Context

A modern full-stack application is not merely a collection of screens connected to a database. Its reliability, security, scalability, and ability to evolve depend substantially on the quality of its back-end services and the interfaces through which other software communicates with them. The back end coordinates business rules, persistence, authentication, integration with external systems, background work, and operational policies. Application programming interfaces (APIs) expose selected capabilities of this back end to web clients, mobile applications, partner systems, and internal services.

Back-end architecture therefore involves two related but distinct concerns. The first is the internal organization of services: how responsibilities are divided, how dependencies are supplied, how input is validated, and how failures are handled. The second is the design of communication contracts: how clients request capabilities, how responses are represented, how APIs are versioned, and how access is controlled. Poor decisions in either area create coupling, unpredictable behaviour, and expensive maintenance.

This chapter examines service-layer design and compares REST, GraphQL, remote procedure call (RPC), and real-time communication. It also addresses contracts, pagination, idempotency, rate limiting, API gateways, asynchronous processing, and the governance issues that arise when APIs become organizational assets rather than isolated programming interfaces.

Learning Objectives

By the end of this chapter, a learner should be able to:

• distinguish controllers, application services, domain services, repositories, and infrastructure adapters;
• apply dependency injection and validation in maintainable back-end systems;
• design consistent error-handling and background-processing strategies;
• compare REST, GraphQL, RPC, and real-time protocols according to context and trade-offs;
• define API contracts involving schemas, status codes, pagination, versioning, and idempotency;
• explain rate limiting, authentication, authorization, and API gateway responsibilities;
• evaluate risks involving security, reliability, observability, governance, and operational cost;
• select an appropriate architecture for a realistic application scenario.

From Request Handling to Service Design

A request entering a back-end system normally passes through several conceptual layers. The transport layer receives HTTP, WebSocket, or RPC communication. A controller or resolver translates that communication into an application command or query. The application service coordinates a use case. Domain logic enforces business rules, while repositories and adapters communicate with databases, queues, payment providers, or other external systems.

This separation is not an excuse to create numerous trivial classes. Its purpose is to isolate reasons for change. A controller should change when an external protocol changes; a business service should change when a business rule changes; a database adapter should change when persistence technology changes. If all of these concerns are placed in one endpoint function, testing and future modification become difficult.

A useful request flow is:

Client → API gateway or edge layer → authentication and rate limiting → controller/resolver → application service → domain model and repositories → infrastructure systems.

The response follows the reverse direction. Each layer should expose only the information needed by the layer above it. For example, a controller should not construct SQL queries, and a repository should not decide whether a customer is eligible for a discount.

Service-layer responsibilities

An application service represents a meaningful use case such as “place order,” “approve loan,” or “register student.” It coordinates operations, manages transaction boundaries where appropriate, and returns an application-level result. A domain service contains business logic that does not naturally belong to one entity or value object. A repository abstracts access to an aggregate or persistence boundary. Infrastructure adapters implement technical details such as database access, email delivery, object storage, and message publishing.

The service layer should not become a second database layer or a procedural collection of unrelated utilities. Its quality can be assessed by asking whether a use case can be tested without starting every external system. If it cannot, dependencies may be too tightly coupled.

Dependency injection

Dependency injection (DI) means that an object receives the services it requires rather than constructing them internally. Constructor injection is generally the clearest form:

```text

OrderService(paymentGateway, orderRepository, eventPublisher)

```

The service depends on abstractions such as `PaymentGateway`, not on a specific payment vendor. A composition root—usually the application startup module—connects those abstractions to concrete implementations.

DI supports substitution, unit testing, configuration management, and separation of concerns. A test can inject a fake payment gateway; production can inject a secure external adapter. However, indiscriminate use of a dependency-injection container can hide object creation, produce circular dependencies, and make the system difficult to understand. Dependencies should be explicit, narrowly scoped, and preferably immutable after construction.

Contracts, Validation, and Error Semantics

An API contract is an agreed description of operations, inputs, outputs, errors, security requirements, and behavioural expectations. OpenAPI documents are widely used for HTTP APIs; GraphQL schemas define types and operations; Protocol Buffers commonly define strongly typed RPC interfaces. A contract is more than a list of URLs. It is a promise about meaning and compatibility.

Validation at the boundary

Validation begins at the system boundary. Syntactic validation checks such matters as data types, required fields, formats, length, ranges, and allowed values. Semantic validation determines whether the request makes sense in the business context—for example, whether an end date follows a start date or whether a requested quantity is available.

Boundary validation should reject malformed input before it reaches domain logic. Nevertheless, domain rules must also be enforced inside the domain or application layer because services may be invoked by a queue consumer, scheduled job, or another internal process rather than through the public API. Client-side validation improves usability but is never a security control.

A robust error response should be consistent and safe. It may contain a machine-readable code, a human-readable message, a correlation identifier, and structured field errors. Internal stack traces, SQL statements, credentials, and infrastructure details must not be returned to clients. HTTP status codes should communicate broad categories: 400-series responses generally identify client or authorization problems, while 500-series responses indicate server-side failure. The precise code must reflect the contract rather than being selected mechanically.

Error handling should distinguish expected business outcomes from unexpected faults. “Insufficient credit” may be an expected domain result; a database connection failure is an operational fault. Centralized exception mapping prevents every controller from inventing a different format. Logging should record diagnostic detail on the server, with sensitive data redacted, while responses reveal only what the client needs.

Choosing an API Style

No API style is universally superior. The appropriate choice depends on interaction patterns, client diversity, latency requirements, governance capacity, and the stability of the domain.

REST

REST-style APIs expose resources through uniform HTTP semantics. A client may retrieve a representation of an order with `GET /orders/{id}`, create an order with `POST /orders`, or update a resource with `PATCH`. REST benefits from familiar HTTP methods, status codes, caching mechanisms, proxy support, and broad tooling.

Its central design challenge is deciding resource boundaries and representations. An endpoint that exposes database tables directly may leak implementation details, while an endpoint that combines too many unrelated operations becomes difficult to understand. REST can also produce over-fetching or under-fetching: a mobile client may receive more data than it needs, or need several requests to construct one screen. These issues can be mitigated through carefully designed representations, field selection, composition endpoints, or an additional API aggregation layer.

GraphQL

GraphQL exposes a typed schema through which clients specify the fields and relationships they require. It is valuable when several clients need different views of the same data or when front-end development would otherwise require frequent endpoint changes. A single query can retrieve related data, and schema introspection supports tooling.

GraphQL transfers some complexity to the server. Query depth, query cost, authorization at field level, caching, and the “N+1” data-fetching problem require deliberate design. A resolver that performs one database call per returned item can degrade performance; batching and request-scoped data loaders are common remedies. GraphQL also requires careful control of arbitrary queries to prevent resource exhaustion.

RPC

RPC models communication as invocation of operations such as `CreateInvoice` or `GetUserProfile`. gRPC, often combined with Protocol Buffers, provides compact binary serialization, generated clients, streaming, and explicit typing. It is frequently effective for internal service-to-service communication where both endpoints are controlled by one organization.

RPC can make remote calls look deceptively similar to local function calls. Network latency, partial failure, retries, deadlines, and compatibility remain real concerns. Public consumers may also prefer HTTP and JSON because of browser support and accessibility of tooling. A sensible architecture may therefore use RPC internally and a REST or GraphQL façade externally.

Real-time communication

Polling repeatedly asks whether new information exists. Server-sent events (SSE) provide one-way server-to-client streams over HTTP, while WebSockets support bidirectional communication over a long-lived connection. These approaches suit chat, collaborative editing, live dashboards, presence, and notifications.

Real-time systems require connection management, authentication renewal, reconnection, ordering, duplicate delivery handling, backpressure, and horizontal scaling. A load-balanced fleet may need a message broker or shared event infrastructure so that a message received by one instance can reach clients connected to another. Real-time delivery should not automatically be treated as guaranteed business processing; durable events and acknowledgements may be required separately.

Designing Durable API Contracts

Resource representation and schema evolution

A contract should describe stable business concepts rather than internal database columns. Names, units, nullability, precision, and time-zone conventions must be explicit. Dates and timestamps need a documented representation. Enumerations should be extended cautiously because some clients fail when they encounter unknown values.

Backward-compatible evolution typically adds optional fields or new operations without changing the meaning of existing ones. Renaming a field, changing its type, removing an enum value, or altering error semantics may break consumers. Versioning strategies include URI versions such as `/v2/orders`, media-type or header versions, and evolution within a single schema. Versioning is not a substitute for compatibility discipline. Old versions also require ownership, monitoring, documentation, and a retirement policy.

Pagination

Returning an unbounded collection creates latency, memory, and denial-of-service risks. Offset pagination uses `limit` and `offset` and is easy to understand, but records inserted or deleted while a client pages through results can cause duplicates or omissions. Cursor pagination uses a position derived from a stable ordering, often an encoded identifier and timestamp. It generally behaves better for large or changing datasets but is less convenient for jumping to an arbitrary page.

A contract should state the default and maximum page size, ordering rules, cursor opacity, and whether a next-page cursor is available. The server must enforce limits rather than trusting the client.

Idempotency and retries

An operation is idempotent when repeating it produces the same intended state as performing it once. `GET` is conventionally safe and idempotent; `PUT` can replace a resource idempotently when designed correctly. `POST` is not inherently idempotent, but an idempotency key can allow a server to associate retries with the original operation.

This matters because clients and gateways retry after timeouts, yet a timeout does not reveal whether the server completed the request. For a payment or order operation, the server can store the key, request fingerprint, result, and expiry within an appropriate scope. A repeated key with different request content should be rejected. Idempotency does not eliminate concurrency problems; transactional constraints and appropriate locking or version checks may still be necessary.

Security, Governance, and the API Gateway

Authentication establishes identity; authorization determines what an identified principal may do. Effective authorization checks resource ownership, role, scope, or policy at the service boundary. A token proving that a user is authenticated does not prove that the user may retrieve every resource of that type. Object-level authorization failures are a common API risk.

An API gateway is an intermediary that can provide routing, TLS termination, authentication integration, rate limiting, request transformation, observability, and sometimes response composition. It is useful for enforcing cross-cutting policies consistently and hiding internal topology. It may also support gradual migration, canary release, and external-to-internal protocol translation.

The gateway should not become a “god service” containing all business logic. Policies requiring domain knowledge generally belong in the service that owns the domain. A gateway also introduces an operational dependency and potential bottleneck; it must be highly available, observable, and configured with conservative timeouts.

Rate limiting protects fairness and system capacity. Fixed-window, sliding-window, token-bucket, and leaky-bucket approaches have different behaviours. A limit may be applied per IP address, user, client application, tenant, or endpoint. A useful response communicates that the limit was exceeded and, where appropriate, when a retry may succeed. Rate limiting should complement authentication, quotas, payload-size restrictions, timeouts, and circuit breakers rather than replace them.

Background Processing and Reliability

Not every task belongs in a synchronous request. Image conversion, report generation, email delivery, search indexing, and integration with slow external systems can be placed on a queue. The request handler records or publishes work, and a worker processes it later. This reduces response latency and isolates failures.

A reliable design defines the message schema, ownership, retry policy, visibility timeout, maximum attempts, and dead-letter handling. Consumers should be idempotent because messages may be delivered more than once. A transaction that updates business data and publishes an event can suffer from a dual-write problem: one action may succeed while the other fails. The transactional outbox pattern records the event in the same database transaction as the business change; a separate publisher then delivers it to the broker.

Retries should distinguish transient errors from permanent errors. Exponential backoff with jitter reduces synchronized retry storms. Poison messages need quarantine and operational review rather than infinite retries. Background processing also requires tracing across the original request, queue message, and worker so that delayed failures remain diagnosable.

An Illustrative Case: A Multi-Channel Course Registration Platform

Background

Consider a fictional university platform serving a browser application, a mobile application, administrative staff, and external learning partners. Students search courses, reserve seats, pay fees, and receive notifications. Enrolment demand increases sharply at the beginning of each term.

Situation and problem

The initial system exposes database-oriented endpoints. Controllers contain SQL, payment calls, and email logic. During registration peaks, clients receive inconsistent errors, duplicate enrolments occur after network timeouts, and administrators cannot determine whether a failure occurred in payment, seat allocation, or notification.

Analysis

The principal problems are mixed responsibilities, absence of an idempotency strategy, insufficient concurrency control, and a contract that does not distinguish business outcomes from infrastructure failure. A single synchronous request also makes email delivery unnecessarily dependent on the availability of the mail provider.

Approach

The redesigned architecture introduces an `EnrolmentService` coordinating the use case, a repository for course capacity, a payment adapter, and an event publisher. Dependencies are injected through interfaces. Input is validated at the API boundary and business rules are enforced again in the service. The enrolment endpoint accepts an idempotency key. A database constraint and transaction prevent the same student from receiving the same course twice.

The public interface uses REST because it must support browsers, mobile clients, and partner tooling. Cursor pagination is used for course search, and an OpenAPI contract is published and tested in the delivery pipeline. A gateway applies authentication, tenant-aware rate limits, and tracing. After successful enrolment, an outbox event is processed by workers that send email and update search projections. Administrative dashboards use server-sent events for near-real-time seat availability, while the authoritative enrolment state remains in the transactional service.

Outcome

The architecture separates the critical seat-allocation transaction from non-critical notifications, gives clients a safe retry mechanism, and makes failures observable by stage. It also enables a future mobile-specific representation without changing core business rules.

Learning points

The case demonstrates that API style cannot compensate for weak service boundaries. Reliability depends on the combined design of contracts, transactions, dependency management, asynchronous processing, and observability. It also illustrates why real-time presentation should not replace a durable source of truth.

Challenges, Risks, and Professional Responsibilities

A service architecture can fail through excessive decomposition. Too many small services create network calls, distributed transactions, duplicated models, and organizational overhead. A modular monolith may be a better starting point when one team owns the system and deployment independence is not yet valuable.

Other common mistakes include returning inconsistent error formats, exposing stack traces, trusting client-supplied prices or roles, using unlimited list endpoints, retrying non-idempotent operations blindly, and allowing background jobs to run without dead-letter handling. API documentation that is not tested against implementation becomes misleading. Metrics should include latency percentiles, error rates, saturation, queue age, retry counts, and business indicators such as successful enrolments—not merely request totals.

Governance includes ownership of contracts, approval of breaking changes, access review, retention of audit records, and documented service-level objectives. Personal data must be minimized, encrypted appropriately, and excluded from logs unless there is a justified and controlled need. Privacy and sector-specific regulation may require consent, deletion processes, data residency controls, and traceable access. Automated decision services should be explainable to the extent required by their context, and authorization policies should be reviewable rather than hidden in scattered code.

Emerging Directions

API platforms increasingly use schema registries, contract testing, automated client generation, and policy-as-code. OpenTelemetry-style distributed tracing connects requests across gateways, services, queues, and databases. Event-driven architectures and asynchronous APIs are becoming more common where workflows span multiple bounded contexts.

Artificial intelligence can assist with API documentation, test generation, anomaly detection, and natural-language interfaces, but generated code and schemas require human review. AI-mediated access introduces additional concerns: prompt-derived requests must still undergo authorization and validation, sensitive data must not be exposed to an unsuitable model, and auditability must be preserved. The central architectural principles remain unchanged: explicit contracts, least privilege, controlled failure, and verifiable behaviour.

Best-Practice Principles

A maintainable back end generally follows these principles:

• place business rules in the domain or application layer, not in transport handlers;
• inject abstractions and keep external adapters replaceable;
• validate at boundaries and enforce critical rules internally;
• publish stable, documented contracts and test compatibility;
• design retries, timeouts, idempotency, and pagination deliberately;
• make asynchronous consumers safe against duplicate delivery;
• use gateways for cross-cutting concerns, not as containers for all business logic;
• protect resources with object-level authorization and meaningful rate limits;
• instrument the complete request and background-processing path;
• choose REST, GraphQL, RPC, or real-time communication according to use case rather than fashion.

Chapter Summary

Back-end services transform external requests into controlled business operations. Clear separation among controllers, application services, domain logic, repositories, and infrastructure adapters improves testability and changeability, while dependency injection makes these boundaries explicit. Validation, consistent errors, authentication, authorization, and observability are essential parts of service design rather than afterthoughts.

REST offers resource-oriented HTTP interaction; GraphQL offers client-shaped queries; RPC provides strongly typed operation calls; and real-time protocols support continuous communication. Each has distinct operational and governance costs. Durable API contracts must address schemas, evolution, pagination, idempotency, and error semantics. Gateways centralize selected cross-cutting policies, while queues and workers support resilient background processing. Professional architecture balances technical quality with security, privacy, operational capacity, organizational ownership, and the needs of API consumers.

Key Terms

Dependency injection: Supplying an object’s dependencies externally rather than constructing them internally.

Application service: A component that coordinates a business use case.

Domain service: Business logic that does not naturally belong to one domain entity.

Repository: An abstraction for accessing persistent domain data.

API contract: An agreed specification of operations, data, errors, and behavioural expectations.

Validation: Checking that input is syntactically and semantically acceptable.

REST: An HTTP-oriented style that represents resources through uniform operations.

GraphQL: A typed query language and runtime allowing clients to request selected data.

RPC: A communication style that models remote interaction as operation invocation.

WebSocket: A protocol supporting persistent, bidirectional communication.

Server-sent events: A mechanism for streaming server-to-client events over HTTP.

Pagination: Dividing a large result set into bounded sections.

Idempotency: The property that repeating an operation has the same intended effect as performing it once.

API gateway: An intermediary that routes requests and applies cross-cutting policies.

Rate limiting: Controlling request frequency to protect capacity and fairness.

Circuit breaker: A resilience mechanism that temporarily stops calls to a failing dependency.

Transactional outbox: A pattern that stores an event with a business transaction before publishing it asynchronously.

Dead-letter queue: A location for messages that cannot be processed successfully after defined attempts.

Observability: The ability to understand system behaviour through logs, metrics, and traces.

Object-level authorization: Checking whether a principal may access a particular resource instance.

Review Questions

Short Answer Questions

1. What is the primary purpose of separating controllers from application services?

2. Distinguish authentication from authorization.

3. Why is cursor pagination often preferable for large, changing datasets?

4. What problem does an idempotency key address?

5. State two responsibilities commonly performed by an API gateway.

Long Answer Questions

1. Explain how dependency injection supports maintainability and testing, and discuss risks of excessive container-based injection.

2. Compare REST, GraphQL, and RPC in terms of client flexibility, typing, tooling, and operational complexity.

3. Describe a robust error-handling strategy for a public API.

4. Explain the transactional outbox pattern and its relationship to duplicate message delivery.

5. Discuss how API contracts should evolve without unnecessarily breaking consumers.

Analytical / Application Questions

1. A payment endpoint times out after the provider may have completed the transaction. Design a safe retry and idempotency strategy.

2. A GraphQL query permits deeply nested requests that overload the database. Identify the risks and propose controls.

3. Design the back-end architecture for a live collaborative dashboard, including communication protocol, scaling, authorization, reconnection, and durable state.

22. Multiple Choice Questions with Answers

Question 1

Which responsibility most appropriately belongs in an application service?

A. Parsing raw HTTP headers only

B. Coordinating the steps of a business use case

C. Rendering browser HTML exclusively

D. Managing physical network cables

Correct Answer: B. Coordinating the steps of a business use case

Question 2

What is a principal benefit of constructor-based dependency injection?

A. It makes dependencies explicit and replaceable

B. It guarantees that no runtime errors can occur

C. It removes the need for interfaces

D. It automatically makes every database transaction atomic

Correct Answer: A. It makes dependencies explicit and replaceable

Question 3

Which API style allows a client to request precisely selected fields from a typed schema?

A. REST

B. GraphQL

C. SMTP

D. FTP

Correct Answer: B. GraphQL

Question 4

Why are idempotency keys especially valuable for payment requests?

A. They compress payment data

B. They prevent all authorization checks

C. They allow safe handling of retries after uncertain outcomes

D. They eliminate the need for transaction records

Correct Answer: C. They allow safe handling of retries after uncertain outcomes

Question 5

Which pagination approach generally handles inserts and deletions during traversal more reliably?

A. Unbounded pagination

B. Offset pagination only

C. Cursor pagination with stable ordering

D. Random pagination

Correct Answer: C. Cursor pagination with stable ordering

Question 6

What is a suitable use of an API gateway?

A. Storing every domain rule in one central script

B. Applying routing, authentication integration, and rate limiting

C. Replacing all database transactions

D. Guaranteeing that external services never fail

Correct Answer: B. Applying routing, authentication integration, and rate limiting

Question 7

What does the transactional outbox pattern primarily address?

A. The dual-write problem between business data and published events

B. The visual design of API documentation

C. The encryption of browser cookies only

D. The selection of a programming language

Correct Answer: A. The dual-write problem between business data and published events

Question 8

Which practice best protects against unauthorized access to another user’s record?

A. Checking only that a token exists

B. Using a larger page size

C. Performing object-level authorization

D. Disabling all error responses

Correct Answer: C. Performing object-level authorization

Question 9

A real-time service is deployed across several instances. What commonly helps distribute events to clients connected to different instances?

A. A shared message broker or event infrastructure

B. A larger HTML template

C. A local variable in one process

D. Removing authentication

Correct Answer: A. A shared message broker or event infrastructure

Question 10

Which statement best describes a safe API evolution practice?

A. Rename public fields whenever internal database names change

B. Add compatible optional fields while monitoring consumer use

C. Return different meanings for the same status code

D. Remove old operations without a retirement policy

Correct Answer: B. Add compatible optional fields while monitoring consumer use