API Design and Implementation

Introduction

An application programming interface (API) is a defined boundary through which software components communicate. In backend development, an API exposes selected data and operations to clients such as web browsers, mobile applications, desktop programs, and other services. A well-designed API is more than a collection of URLs: it is a stable contract describing resources, representations, requests, responses, validation rules, security expectations, and failure conditions.

Representational State Transfer (REST) is a widely used architectural style for web APIs. REST commonly uses HTTP methods and status codes to work with resources represented as JSON. Although REST is not a formal protocol, it provides constraints that encourage statelessness, discoverability, scalability, and separation between clients and servers.

This chapter develops the practical foundations needed to design and implement RESTful APIs. It also introduces pagination, filtering, sorting, versioning, error formats, OpenAPI documentation, and the basic characteristics of GraphQL and RPC.

Learning Objectives

By the end of this chapter, you should be able to:

• explain resources, endpoints, representations, and HTTP methods;
• design CRUD operations using REST conventions;
• create and consume JSON request and response bodies;
• validate input and return appropriate status codes and errors;
• implement pagination, filtering, and sorting safely;
• explain API versioning and backward compatibility;
• document an API with OpenAPI;
• compare REST with GraphQL and RPC;
• apply security, reliability, and governance practices to API development.

1. APIs as Contracts

An API contract specifies what a client may request and what the server will return. The contract normally includes:

1. endpoint paths and HTTP methods;
2. request parameters, headers, and body formats;
3. response status codes and body schemas;
4. authentication and authorization requirements;
5. validation and error behavior;
6. performance or rate-limit expectations.

The client should not need to know how the server stores data. For example, a client can request a product without knowing whether the product is stored in PostgreSQL, MongoDB, or a cache. This separation allows each side to evolve independently.

An endpoint is a network-accessible operation identified by a URL path and an HTTP method. A resource is a conceptual object or collection managed by the API. In a library API, common resources might be books, authors, and members.

A representation is a transferable form of a resource. JSON is the most common representation in contemporary web APIs, but XML, CSV, and other formats are also possible.

Important terminology

2. RESTful Resource Design

REST-oriented APIs generally use nouns for resource paths and HTTP methods for actions. A collection is conventionally plural:

GET    /api/v1/books       list books
POST   /api/v1/books       create a book
GET    /api/v1/books/42    retrieve book 42
PATCH  /api/v1/books/42    partially update book 42
PUT    /api/v1/books/42    replace book 42
DELETE /api/v1/books/42    delete book 42

The path identifies the resource; the method expresses the requested operation. Paths such as /getBooks or /createNewBook are usually less consistent because they place the action in the URL rather than using HTTP semantics.

HTTP methods

• GET retrieves a resource. It should not change server state and is generally safe and idempotent.
• POST submits data for processing, often creating a subordinate resource. Repeating a POST may create multiple resources unless the API supplies an idempotency mechanism.
• PUT replaces a resource at a known URI. A carefully designed PUT is idempotent.
• PATCH applies a partial modification. Its idempotency depends on the operation and implementation.
• DELETE removes a resource. Repeating a delete commonly leaves the resource absent, although the response behavior may differ.

The terms safe and idempotent have specific meanings. Safe means that the method is intended not to modify server state. Idempotent means that repeating the same request has the same intended server-side effect as making it once; it does not require identical response bodies.

Resource relationships

Relationships can be represented in nested paths, such as /authors/7/books, when the relationship is central to the request. However, excessive nesting can make URLs difficult to use. A practical rule is to keep paths shallow and use query parameters or links for secondary relationships.

For example:

GET /api/v1/orders/381/items
GET /api/v1/items?orderId=381

Both may be valid. The first emphasizes the order-to-items relationship; the second treats items as an independently queryable resource.

3. CRUD and JSON Representations

CRUD means create, read, update, and delete. A simple book resource might be represented as:

{
  "id": 42,
  "title": "Practical Backend Development",
  "isbn": "9780000000001",
  "publishedYear": 2024,
  "available": true
}

A collection response should communicate both the data and, when useful, collection metadata:

{
  "data": [
    {
      "id": 42,
      "title": "Practical Backend Development"
    }
  ],
  "meta": {
    "page": 1,
    "pageSize": 20,
    "totalItems": 1
  }
}

JSON supports strings, numbers, Boolean values, arrays, objects, and null. JSON does not define a native date type; APIs should therefore document a consistent representation, commonly an ISO 8601 string such as 2025-03-08T10:30:00Z.

Clients should send the media type explicitly:

Content-Type: application/json
Accept: application/json

The server should not expose internal fields accidentally. Database passwords, password hashes, internal audit fields, and private tokens should never be included in ordinary resource representations.

4. Status Codes and Error Formats

HTTP status codes communicate the broad result of a request. Useful codes include:

A consistent error format makes client development easier. One suitable format is:

{
  "error": {
    "code": "VALIDATION_FAILED",
    "message": "One or more fields are invalid.",
    "details": [
      {
        "field": "publishedYear",
        "issue": "must be between 1450 and the current year"
      }
    ],
    "requestId": "req_8f31"
  }
}

The code is stable for programmatic handling, while message is readable by people. Detailed errors should help legitimate users correct input but should not disclose SQL statements, stack traces, secrets, or internal infrastructure.

A 404 should not be used to conceal every kind of failure. For example, invalid JSON is a client request problem, while a database outage is a server problem. Accurate status codes improve monitoring and troubleshooting.

5. Validation and Business Rules

Validation occurs at several layers:

1. Syntax validation checks whether the request is well formed, such as valid JSON.
2. Schema validation checks types, required properties, lengths, formats, and ranges.
3. Business validation checks domain rules, such as whether a book can be borrowed.
4. Authorization checks whether the caller may perform the operation.
5. Database constraints provide a final integrity boundary.

Validation must occur on the server even when a client performs validation. Client-side checks improve usability but cannot be trusted because requests can be modified or sent directly.

For a book-creation request, rules might include:

• title is required and has between 1 and 200 characters;
• isbn, if present, has an accepted format;
• publishedYear is an integer within a reasonable range;
• unknown fields are rejected or explicitly ignored according to the contract.

Validation should be deterministic and documented. Normalization, such as trimming surrounding whitespace, should be applied consistently. Be cautious with automatic type coercion: silently converting a string such as "true" to a Boolean can hide client defects.

Example implementation pattern

The following framework-neutral pseudocode illustrates the order of responsibilities:

function createBook(request):
    body = parseJson(request.body)
    if body is invalid:
        return 400, error("INVALID_JSON")

    errors = validateBookSchema(body)
    if errors is not empty:
        return 422, validationError(errors)

    if not currentUser.can("book:create"):
        return 403, error("FORBIDDEN")

    if repository.isbnExists(body.isbn):
        return 409, error("ISBN_ALREADY_EXISTS")

    book = repository.insert(body)
    return 201, book

The sequence matters. Authorization should not be bypassed because validation passed, and database uniqueness should not be assumed merely because the application performed a preliminary check. Concurrent requests require database-level constraints and suitable transaction handling.

6. Pagination, Filtering, and Sorting

Returning every record from a large collection can consume excessive memory, bandwidth, and processing time. Pagination limits each response to a manageable subset.

Offset pagination

A common interface is:

GET /api/v1/books?page=3&pageSize=20

If pages are one-based, the database offset is:

Here, page is the requested page number and pageSize is the number of records per page. Both should be positive integers, and the server should impose a maximum page size such as 100. For page = 3 and pageSize = 20, offset = (3 − 1) × 20 = 40.

Offset pagination is easy to understand but can become slower for very large offsets and may produce duplicates or omissions when records are inserted between requests.

Cursor pagination

Cursor pagination uses a position derived from a stable ordering:

GET /api/v1/books?limit=20&after=eyJpZCI6NDJ9

The cursor should be opaque to clients. Cursor pagination is often more stable for changing or large datasets, but it requires careful ordering and more complex client navigation. A cursor must not be treated as trusted input; it should be validated, signed, or encoded safely according to the implementation.

Filtering and sorting

Query parameters express collection selection:

GET /api/v1/books?available=true&publishedAfter=2020
GET /api/v1/books?sort=title&direction=asc

Do not concatenate arbitrary query values into SQL. Map allowed public fields to known database columns:

allowedSorts = {
    "title": "books.title",
    "publishedYear": "books.published_year"
}

Unknown filters and sort fields should produce a clear validation error or be ignored only if that behavior is documented. Stable sorting is important. If two records have equal titles, add a unique tie-breaker such as id, so that page boundaries remain predictable:

ORDER BY title ASC, id ASC

Filtering and sorting must also be authorized. An API should not allow clients to sort by confidential columns merely because those columns exist in the database.

7. Versioning and Evolution

APIs evolve because business requirements, security standards, and data models change. Versioning communicates compatibility expectations. Common approaches include:

• URI versioning: /api/v1/books;
• media-type versioning using the Accept header;
• query-parameter versioning such as ?version=1.

URI versioning is highly visible and easy for certificate-level projects to implement. It can, however, create duplicated routes. Header-based versioning keeps URLs stable but is less visible during testing and troubleshooting.

A change is generally backward compatible if existing valid client requests continue to work and existing response meanings are preserved. Adding an optional response field is often compatible, but clients that reject unknown fields may behave differently. Renaming a field, changing its type, removing an endpoint, or changing the meaning of a status code is potentially breaking.

Good evolution practices include:

• document deprecation dates and migration instructions;
• avoid changing the meaning of existing fields;
• introduce new optional fields before making them required;
• maintain contract tests for important clients;
• monitor use of old versions;
• remove obsolete versions only through a communicated policy.

Versioning is not a substitute for careful design. A new version should be created when compatibility genuinely cannot be preserved, not for every small implementation change.

8. OpenAPI Documentation

OpenAPI is a machine-readable specification for describing HTTP APIs. It can support documentation generation, client SDK generation, mock servers, validation, and testing. A minimal OpenAPI document contains an openapi version, metadata, servers, paths, operations, parameters, request bodies, responses, and schemas.

A simplified example is:

openapi: 3.0.3
info:
  title: Library API
  version: 1.0.0
paths:
  /api/v1/books/{id}:
    get:
      summary: Retrieve a book
      parameters:
        - name: id
          in: path
          required: true
          schema:
            type: integer
      responses:
        '200':
          description: Book found
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Book'
        '404':
          description: Book not found
components:
  schemas:
    Book:
      type: object
      required: [id, title]
      properties:
        id:
          type: integer
        title:
          type: string

The documentation must match the implementation. A beautifully generated specification that omits authentication, validation errors, or pagination is not a reliable contract. Treat the OpenAPI document as part of the source-controlled project and validate it in continuous integration where possible.

9. A Practical REST Implementation

A maintainable backend separates responsibilities. A typical request path is:

1. routing selects an endpoint;
2. middleware handles concerns such as authentication and request IDs;
3. a controller translates HTTP input into application commands;
4. a service applies business rules;
5. a repository communicates with storage;
6. a serializer constructs the public response.

This separation prevents controllers from becoming large collections of SQL statements and business rules. It also improves testing: service rules can be tested without starting a web server, while endpoint tests can verify HTTP behavior.

A conceptual route might look like:

router.patch("/api/v1/books/:id", authenticate, validate(updateBook),
            bookController.update)

A controller should translate errors into documented HTTP responses rather than expose raw exceptions. Unexpected exceptions should be logged with a correlation or request ID and returned as a generic server error.

Testing an endpoint

Useful API tests include:

• a successful creation returns 201 and a representation;
• malformed JSON returns 400;
• missing required fields return 422;
• an unauthenticated request returns 401;
• a user lacking permission returns 403;
• an unknown identifier returns 404;
• a duplicate unique value returns 409;
• pagination respects the maximum page size;
• sorting accepts only documented fields;
• sensitive fields are absent from responses.

Tests should include both ordinary and adversarial inputs. Contract tests are especially valuable when multiple teams consume the same API.

10. Security, Reliability, and Governance

API security begins with transport protection. Production APIs should use HTTPS so that credentials and payloads are not exposed in transit. Authentication identifies the caller; authorization determines what that caller may do. These concepts must not be confused.

Common authentication mechanisms include secure sessions, OAuth 2.0 flows, and signed tokens. Token-based systems still require careful expiration, storage, revocation, and scope management. Passwords must be hashed with an appropriate password-hashing algorithm rather than encrypted or stored in plain text.

Important controls include:

• object-level authorization on every protected resource;
• rate limiting and abuse detection;
• input validation and output filtering;
• protection against injection attacks;
• safe CORS configuration;
• secret management outside source code;
• audit logging for sensitive actions;
• dependency and vulnerability management.

Logs should support diagnosis without recording passwords, access tokens, payment details, or unnecessary personal information. Privacy regulations may require data minimization, retention controls, access rights, and breach procedures. API designers should consider whether an endpoint exposes personal data and whether the user has a lawful and authorized basis to access it.

Reliability practices include timeouts, retries only where safe, circuit breakers for dependent services, bounded request sizes, graceful degradation, and health monitoring. Retrying a non-idempotent POST can create duplicates unless the API supports an idempotency key. An idempotency key allows the server to associate repeated attempts with one logical operation, subject to an explicit retention and conflict policy.

11. REST, GraphQL, and RPC

REST, GraphQL, and RPC are different interface styles rather than interchangeable names for the same design.

GraphQL commonly exposes one endpoint and a schema through which clients request precisely the fields they need. This can reduce over-fetching, but the server must control query depth, cost, and authorization. GraphQL does not eliminate the need for validation, authentication, rate limiting, or careful database access.

RPC models communication as calling a remote procedure, for example CreateInvoice or CancelSubscription. RPC can be appropriate for command-heavy internal systems, especially when generated clients and strongly typed contracts are important. The choice should reflect client needs, domain structure, operational constraints, and team expertise.

12. Illustrative Case: Community Library API

Background

A community library wants a backend service for a web application and a mobile application. The first version stores books, members, and loans. Staff need to search books, while members need to view availability and create loan requests.

Problem

The library’s initial prototype provides a single /library endpoint that accepts arbitrary JSON. It returns database records directly, has no consistent errors, and loads all books into memory. Mobile clients experience slow responses, and a user can view another member’s private loan details by changing an identifier in the URL.

Analysis

The prototype has several defects: it does not model resources clearly, lacks object-level authorization, exposes persistence details, and has no bounded collection query. The team identifies books, members, and loans as resources. A loan is associated with both a book and a member, but private member information must not be included in public book responses.

Approach and solution

The team creates the following contract:

GET  /api/v1/books?query=&available=&page=&pageSize=&sort=
GET  /api/v1/books/{id}
POST /api/v1/loans
GET  /api/v1/members/me/loans
DELETE /api/v1/loans/{id}

Book listing uses a maximum page size of 50, validates allowed sort fields, and returns a stable ordering. A loan request accepts a book identifier and obtains the authenticated member from the security context rather than trusting a member identifier supplied by the client. The database enforces that an active loan cannot be created when the book is already unavailable.

The API returns 201 for a successful loan request, 409 if the book is already on loan, 401 if authentication is absent, and 403 if a member attempts to cancel another member’s loan. All errors use the same envelope and include a request ID. OpenAPI describes the paths, schemas, security requirements, and error responses.

Outcome

The mobile client receives only the fields it needs, large searches are bounded, and unauthorized access is rejected. Staff can use filtering and sorting without supplying database expressions. The service can add fields in version 1 without changing the existing contract, while a genuinely breaking change can be introduced under version 2.

Learning points

The case demonstrates that API quality depends on more than routing. Resource modeling, validation, authorization, database constraints, representation design, pagination, documentation, and observability must work together. Security must be designed into each resource operation, not added only as a login screen.

13. Best Practices and Limitations

A practical checklist for a certificate-level API project is:

• use consistent plural resource names;
• use HTTP methods and status codes according to their intended semantics;
• define JSON schemas and error envelopes;
• validate all untrusted input on the server;
• apply authorization at the object and operation level;
• paginate collections and cap page sizes;
• whitelist filter and sort fields;
• document endpoints with OpenAPI;
• return only fields appropriate to the caller;
• log request IDs and security-relevant events without secrets;
• test success, validation, authorization, conflict, and failure paths;
• plan compatibility before changing a public contract.

REST conventions do not solve every problem. A resource model may be awkward for complex workflows, real-time communication, bulk processing, or highly connected data. HTTP status codes also provide only broad categories; the error body must supply application-specific meaning. Finally, documentation cannot compensate for inconsistent implementation. An API is trustworthy when its behavior, documentation, security, and operational characteristics agree.

Chapter Summary

An API is a software contract between clients and servers. RESTful design organizes that contract around resources, endpoint paths, HTTP methods, representations, and stateless requests. CRUD operations commonly use GET, POST, PUT or PATCH, and DELETE. JSON provides a practical representation, while status codes and consistent error envelopes communicate outcomes.

Reliable APIs validate syntax, schemas, business rules, authorization, and database integrity. Collection endpoints should support bounded pagination, documented filtering, and safe, stable sorting. Offset pagination is simple; cursor pagination can provide more stable navigation for large changing datasets. Versioning and deprecation policies help preserve compatibility. OpenAPI makes an API understandable and supports documentation, testing, and tooling.

Security requires HTTPS, authentication, authorization, safe handling of secrets, rate limits, privacy-aware logging, and protection against injection and data exposure. GraphQL and RPC offer alternative interface models for particular requirements. Ultimately, successful API implementation combines clear resource design with testing, governance, reliability, and disciplined evolution.

Key Terms

API: A contract through which software components communicate.

REST: An architectural style emphasizing resources, representations, statelessness, and uniform interfaces.

Resource: A conceptual entity or collection exposed by an API.

Endpoint: A method and path used to access an API capability.

Representation: A transferable format of a resource, such as JSON.

CRUD: Create, read, update, and delete operations.

JSON: A text-based format for structured data.

Idempotent: Having the same intended server-side effect when the same request is repeated.

Statelessness: The requirement that each request contain the information needed for processing.

Validation: Checking that input is syntactically, structurally, and semantically acceptable.

Pagination: Dividing a large collection into smaller responses.

Cursor: An opaque position used to continue through a collection.

Filtering: Selecting collection members that satisfy conditions.

OpenAPI: A machine-readable specification for HTTP API contracts.

Authentication: Verifying the identity of a caller.

Authorization: Determining what an authenticated caller may do.

Rate limiting: Restricting request frequency or volume.

Versioning: Managing different compatible or incompatible forms of an API.

GraphQL: A typed query language and runtime in which clients select response fields.

RPC: An interface style that models communication as calls to remote procedures.

Review Questions

Short Answer Questions

1. What is the difference between an API resource and an endpoint?
2. Why should a server validate input even when the client validates it?
3. State two differences between PUT and PATCH.
4. What information should a consistent error response contain?
5. Why should sort fields be restricted to an allow-list?

Long Answer Questions

1. Explain how a RESTful API can represent CRUD operations for a collection and an individual resource. Include suitable methods, paths, and status codes.
2. Describe the layers of validation in a backend API and explain why database constraints remain important.
3. Compare offset pagination with cursor pagination, including one advantage and one limitation of each.
4. Explain how OpenAPI contributes to API development, testing, and maintenance.
5. Discuss the security and privacy responsibilities of an API developer.

Analytical / Application Questions

1. Design an API for courses and student enrollments. Propose endpoints, methods, representations, validation rules, and at least four error responses.
2. An endpoint accepts sort=price;DROP TABLE products and places the value directly into a database query. Analyze the risks and propose a safe implementation strategy.
3. A mobile client must repeatedly submit a payment request when network responses are lost. Explain how idempotency, status codes, and error handling can prevent duplicate payments.

Multiple Choice Questions with Answers

Question 1

Which HTTP method is conventionally used to retrieve a resource without intentionally changing server state?

A. GET

B. POST

C. PATCH

D. DELETE

Correct Answer: A. GET

Question 2

Which status code most appropriately indicates that a new resource was created successfully?

A. 200

B. 201

C. 204

D. 304

Correct Answer: B. 201

Question 3

For one-based pagination, what is the offset when page = 3 and pageSize = 20?

A. 20

B. 40

C. 60

D. 80

Correct Answer: B. 40

Question 4

Which practice provides the safest handling of client-provided sort fields?

A. Insert the value directly into SQL

B. Allow only numeric sort values

C. Map documented public fields to approved database columns

D. Ignore authentication for sorting requests

Correct Answer: C. Map documented public fields to approved database columns

Question 5

Which status code commonly indicates that authentication is required or has failed?

A. 401

B. 403

C. 404

D. 409

Correct Answer: A. 401

Question 6

What is the primary purpose of an OpenAPI document?

A. To replace database backups

B. To describe an API contract in a machine-readable form

C. To encrypt all API traffic

D. To store user passwords

Correct Answer: B. To describe an API contract in a machine-readable form

Question 7

Which statement best describes authorization?

A. It verifies the identity of a caller

B. It compresses a request body

C. It determines what an authenticated caller may do

D. It converts JSON into SQL

Correct Answer: C. It determines what an authenticated caller may do

Question 8

Which change is most likely to be breaking for existing API clients?

A. Adding an optional response field

B. Adding a new documented endpoint

C. Adding an optional filter

D. Changing an existing field from a number to a string

Correct Answer: D. Changing an existing field from a number to a string

Question 9

What is a major characteristic of GraphQL?

A. Clients select the fields they need from a typed schema

B. Every operation must use a separate database

C. It prohibits authentication

D. It requires all responses to be CSV

Correct Answer: A. Clients select the fields they need from a typed schema

Question 10

Why are idempotency keys useful for some POST operations?

A. They make passwords unnecessary

B. They help prevent duplicate effects when a request is retried

C. They automatically paginate responses

D. They eliminate authorization checks

Correct Answer: B. They help prevent duplicate effects when a request is retried