RESTful API Design and Documentation

Introduction and Context

A backend application rarely serves only its own user interface. Mobile applications, web clients, partner systems, automated jobs, and third-party developers may all need access to the same data and business capabilities. An application programming interface (API) provides that access through a defined contract. A well-designed API does more than expose database tables: it expresses domain resources, enforces rules, communicates outcomes through standard HTTP semantics, and evolves without unexpectedly breaking clients.

REST (Representational State Transfer) is an architectural style commonly used for HTTP APIs. REST does not prescribe one programming language or framework. Node.js frameworks such as Express and NestJS, and Python frameworks such as Django REST Framework, can implement REST principles. The important design decisions concern resource modeling, representations, status codes, validation, error handling, versioning, discoverability, and documentation.

This chapter develops a practical and analytical approach to designing maintainable REST APIs. Examples use HTTP and JSON, but the principles also apply to other representations and frameworks.

Learning Objectives

By the end of this chapter, a student should be able to:

1. Model domain concepts as REST resources and select appropriate HTTP methods.
2. Design versioned endpoints with predictable status codes and representations.
3. Implement filtering, sorting, pagination, content negotiation, validation, and idempotency.
4. Produce standardized errors and secure, useful API responses.
5. Describe an API with OpenAPI and use the description for testing and client generation.
6. Evaluate backward compatibility and plan controlled API evolution.

1. REST Foundations: Resources, Representations, and Constraints

A REST API exposes resources identified by uniform resource identifiers (URIs). A resource is a conceptual entity or collection, such as a book, order, or customer. A representation is a transferable form of that resource, commonly JSON.

For example:

GET /api/v1/books/42

may return:

{
  "id": "42",
  "title": "Distributed Systems",
  "author": "A. Kumar",
  "status": "available",
  "links": {
    "self": "/api/v1/books/42"
  }
}

The URI identifies the book; the JSON document is its current representation. A URI should generally use nouns rather than verbs. /books/42 is preferable to /getBook?id=42. The HTTP method supplies the operation.

Core REST constraints

The REST style is commonly explained through six constraints:

• Client–server separation: the user interface and data service evolve independently.
• Statelessness: each request contains the information needed to process it; the server does not rely on conversational request state stored between requests.
• Cacheability: responses state whether they may be reused by caches.
• Uniform interface: resources, representations, standard methods, and status codes provide a consistent interaction model.
• Layered system: a client need not know whether it communicates with the origin server, a gateway, or a cache.
• Code-on-demand (optional): a server may transfer executable code, although this is uncommon in JSON APIs.

Statelessness does not mean that an application has no database session or persistent data. It means that request processing does not depend on hidden conversational context such as “the last page the client viewed.” Authentication tokens, resource identifiers, and request parameters should make the request understandable on its own.

Resource modeling

A collection and an individual resource normally have different URIs:

Nesting is useful when the child has no meaningful identity outside its parent. Excessive nesting, such as /users/8/accounts/3/orders/917/items/4, can make authorization and client use difficult. A resource may also be addressed directly as /items/4 when it has an independent identity.

2. HTTP Methods, Safety, and Idempotency

HTTP methods communicate intent and carry standardized properties.

A method is safe when it is intended only for retrieval. A method is idempotent when repeating the same request has the same intended final effect as making it once. Idempotency does not require identical response bodies or status codes. For example, the first DELETE may return 204 No Content, while a repeated one may return 404 Not Found; the resource remains absent in either case.

POST is often non-idempotent: repeating POST /payments might create two payments. A client can request application-level idempotency by sending an idempotency key:

POST /api/v1/payments
Idempotency-Key: 7f1c9d...
Content-Type: application/json

The server stores the key with the resulting status and response for an appropriate retention period. A repeat with the same key and equivalent request returns the original result rather than performing the operation again. The server should reject reuse of a key with materially different request data, commonly with 409 Conflict or 422 Unprocessable Content. Keys must be scoped appropriately, authenticated users must not share an unsafe key namespace, and the persistence operation should be atomic with the business operation. An idempotency key alone is not sufficient if the database transaction can commit while the key record is lost.

PUT should replace the complete representation at the target URI. If partial updates are intended, use PATCH and document its patch format and semantics. A server should not silently interpret an incomplete PUT as a partial update.

3. Status Codes and Response Semantics

Status codes are part of the API contract. A client should be able to decide whether to retry, correct input, authenticate, or display a result from the code and response structure.

Do not use 200 OK for every outcome. Doing so forces clients to parse application-specific error fields and weakens HTTP tooling. Conversely, status codes should not reveal sensitive information. An authorization policy may intentionally return 404 for a resource that an unauthorized user must not discover.

A successful creation should commonly include:

HTTP/1.1 201 Created
Location: /api/v1/books/43
Content-Type: application/json

4. Filtering, Sorting, and Pagination

Collection endpoints need predictable controls. Query parameters are appropriate for representation preferences and collection selection:

GET /api/v1/books?author=Kumar&status=available&sort=-published_at&page=2&page_size=20

The API should document which fields are filterable and sortable. Never construct database queries by concatenating unrestricted client input. Map approved parameter names to known columns, validate data types, and impose maximum page sizes.

Pagination models

Offset pagination uses a page or offset and a limit. It is easy to understand and works well for small, relatively stable collections. However, inserts and deletions during traversal can cause duplicates or omissions, and large offsets may be expensive in some databases.

Cursor pagination returns an opaque cursor representing a position in an ordered result set:

{
  "data": [
    {"id": "91", "created_at": "2025-02-02T12:00:00Z"}
  ],
  "pagination": {
    "next_cursor": "eyJjcmVhdGVkX2F0Ijoi..."
  }
}

The cursor should not expose trusted internal information and should encode a stable ordering. A robust ordering often uses a primary timestamp plus a unique identifier as a tie-breaker. Cursor pagination is usually preferable for large or frequently changing feeds, but it is less convenient for “jump to page 20.”

A collection response should state whether the total count is exact, estimated, or omitted. Counting every row can be costly; an API should not promise an exact total unless it can provide it reliably. Pagination controls should also defend against denial-of-service risks through maximum page size, query timeouts, and rate limits.

5. Content Negotiation and Representations

Content negotiation allows a client and server to agree on representation formats. The request header Accept indicates response preferences:

Accept: application/json

The request header Content-Type identifies the format of the request body:

Content-Type: application/json

These headers have different roles. Accept: application/json does not prove that the request body is JSON; Content-Type does. A server should return 415 Unsupported Media Type for an unsupported request format and may return 406 Not Acceptable when it cannot produce any representation acceptable to the client.

If a response varies by Accept, caches need the header in the cache key. The server should send:

Vary: Accept

Media-type versioning is an alternative to URI versioning, for example:

Accept: application/vnd.library.book+json; version=2

URI versioning, such as /api/v2/books, is generally more visible and simpler for many educational and enterprise systems. Neither approach eliminates the need for compatibility discipline. The chosen strategy should be consistent and documented.

6. Versioning and Backward Compatibility

Version an API when a change cannot be safely consumed by existing clients. A major version commonly represents a breaking contract change, not merely a change in server implementation.

Usually backward-compatible changes include:

• adding a new endpoint;
• adding an optional request field;
• adding a response field when clients ignore unknown fields;
• accepting an additional media type;
• adding a new enum value only if clients handle unknown values safely.

Potentially breaking changes include:

• removing or renaming a field;
• changing a field’s type or meaning;
• making an optional field required;
• changing authorization behavior;
• changing default ordering or pagination semantics;
• removing a status code or altering error structure;
• narrowing accepted input values.

Adding a response field can still break clients that perform strict schema validation or deserialize into closed structures. Therefore “non-breaking” is contextual, not absolute.

A responsible versioning policy includes a support window, deprecation notices, migration documentation, contract tests, and telemetry showing which clients use deprecated features. A response header such as Deprecation or a documented sunset date can help, but headers do not replace direct communication with important consumers. Database migrations and API migrations should be separated where possible: expose a stable API contract while internal storage changes behind it.

7. Validation and Standardized Errors

Validation occurs at multiple boundaries:

1. Transport validation: JSON syntax, media type, body size, and required envelope.
2. Schema validation: types, formats, lengths, ranges, and allowed fields.
3. Domain validation: rules such as “a loan cannot be returned before it is issued.”
4. Authorization validation: whether the caller may perform the operation.
5. Persistence constraints: unique keys, foreign keys, and transaction conditions.

Validation should be explicit and consistent. Reject unknown fields when accidental input could hide client defects; alternatively, document that unknown fields are ignored. The policy must be deliberate.

A standardized error format prevents every endpoint from inventing a different structure. RFC 9457 Problem Details for HTTP APIs provides a useful model:

{
  "type": "https://api.example.edu/problems/validation-error",
  "title": "Request validation failed",
  "status": 422,
  "detail": "One or more fields are invalid.",
  "instance": "/api/v1/books",
  "errors": [
    {"field": "title", "code": "required", "message": "Title is required."}
  ],
  "trace_id": "01HT..."
}

The type identifies the problem class; title is a short summary; status repeats the HTTP status; detail explains the particular occurrence; and instance identifies the request or occurrence. Error messages should not expose stack traces, SQL statements, secrets, or internal hostnames. A trace identifier can connect a client-visible failure with protected server logs.

8. OpenAPI Documentation as an Executable Contract

OpenAPI is a machine-readable description of paths, operations, parameters, request bodies, responses, schemas, security requirements, and servers. It supports interactive documentation, contract testing, validation middleware, and client generation.

A compact OpenAPI fragment might be:

openapi: 3.0.3
info:
  title: Library API
  version: 1.0.0
paths:
  /api/v1/books/{id}:
    get:
      summary: Retrieve a book
      parameters:
        - name: id
          in: path
          required: true
          schema:
            type: string
      responses:
        '200':
          description: Book returned
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Book'
        '404':
          description: Book not found
          content:
            application/problem+json:
              schema:
                $ref: '#/components/schemas/Problem'
components:
  schemas:
    Book:
      type: object
      required: [id, title, status]
      properties:
        id: {type: string}
        title: {type: string, minLength: 1}
        status: {type: string, enum: [available, loaned]}
    Problem:
      type: object
      required: [type, title, status]
      properties:
        type: {type: string, format: uri}
        title: {type: string}
        status: {type: integer}

Good documentation specifies examples, authentication schemes, validation limits, pagination behavior, error responses, idempotency requirements, and deprecation status. The specification should be reviewed in the same change as implementation. “Documentation drift” occurs when the code and contract diverge; automated schema validation and consumer-driven contract tests reduce this risk.

In Node.js, OpenAPI can be maintained as YAML or generated from route metadata. In Django REST Framework, serializers express much of the schema and tools can generate OpenAPI, but generated output still requires human review. Automatic generation is convenient, not a substitute for design.

9. Implementation Architecture in Node.js and Django

A maintainable endpoint separates responsibilities:

• Router: maps method and path to a handler.
• Authentication and authorization middleware: establishes identity and permissions.
• Serializer or schema validator: validates and transforms input and output.
• Controller/view: coordinates the request and response.
• Service layer: implements domain operations and transactions.
• Repository/model layer: interacts with persistence.
• Error middleware/exception handler: maps known failures to standard responses.

A Node.js route might conceptually look like:

router.post('/api/v1/books', authenticate, validate(createBookSchema), async (req, res, next) => {
  try {
    const book = await bookService.create(req.user, req.validatedBody);
    res.status(201)
       .location(`/api/v1/books/${book.id}`)
       .type('application/json')
       .json(book);
  } catch (error) {
    next(error);
  }
});

In Django REST Framework, a serializer validates input and a viewset or API view delegates to application logic. Framework convenience must not obscure transaction boundaries. For example, checking uniqueness in application code and then inserting is subject to a race condition; the database should enforce uniqueness, with the resulting integrity error translated into a suitable API error.

Authentication is not authorization. A valid token identifies a principal, but authorization determines whether that principal may read or modify a particular resource. Object-level authorization must be performed for every relevant resource, not only at the route level.

10. Case Study: University Library API

Background

A university library offers a web portal and a mobile application. Its initial endpoint returns all books from one database query. Mobile users experience slow responses, and a planned partner application needs a stable public contract.

Problem

The original API has no version, returns 200 for validation failures, exposes database column names, provides no pagination, and creates duplicate loan records when a mobile client retries after a network timeout.

Analysis

The team identifies three separate concerns: representation design, transport reliability, and evolution. The book collection is potentially large, loans are state-changing operations, and mobile clients cannot always distinguish a lost response from a failed request.

Approach and solution

The team introduces /api/v1/books and /api/v1/loans. Book listing supports an allow-listed status filter, cursor pagination, and a stable ordering by created_at and id. A request with an invalid filter receives 422 and a Problem Details body. A successful book creation returns 201 and Location.

Loan creation accepts an Idempotency-Key. The key, authenticated user, request fingerprint, result status, and response are recorded in a durable store. A unique constraint prevents two simultaneous requests with the same key from creating two loans. The loan operation and inventory update occur in one transaction.

The OpenAPI document defines schemas, examples, security requirements, cursor behavior, and all expected error responses. Contract tests verify that the implementation accepts documented requests and returns documented response shapes. The next release adds a nullable cover_image_url field without changing existing fields. A future breaking change is planned as /api/v2, with a documented migration period.

Outcome

Mobile responses become bounded in size, retries no longer create duplicate loans, and partner developers can generate clients from the contract. Monitoring shows use of the old endpoint, enabling an evidence-based deprecation schedule.

Learning points

The case demonstrates that API quality is systemic. Pagination without stable ordering is incomplete; idempotency without durable atomic storage is unreliable; versioning without compatibility analysis is cosmetic; and documentation is most valuable when tested against implementation.

11. Security, Governance, and Ethical Considerations

REST API design is also governance design. Apply least privilege, validate authorization at object level, use TLS, protect credentials, and avoid placing secrets in URLs because URLs may be logged. Rate limiting, request-size limits, timeouts, and pagination limits reduce abuse and accidental overload.

Personal data requires purpose limitation, minimization, access controls, retention rules, and auditability. An API should return only fields needed for the client’s purpose. Error responses should not disclose whether a sensitive account exists. Logs should avoid tokens, passwords, and unnecessary personal data. Depending on jurisdiction and institutional context, privacy regulations may impose rights concerning access, correction, deletion, and processing records.

API keys and tokens should be rotatable and scoped. CORS configuration should specify trusted origins rather than allowing every origin by default. Security documentation should state authentication schemes, authorization scopes, and rate limits. Threat modeling should consider injection, broken object-level authorization, replay, denial of service, mass assignment, and excessive data exposure.

AI-assisted coding and schema generation can accelerate development, but generated routes and validators require review. An AI tool may reproduce insecure patterns, omit authorization checks, or expose personal data in prompts. Human ownership of the contract, tests, security review, and operational decisions remains essential.

12. Testing, Observability, and Operational Practice

A professional API is tested at several levels:

• unit tests for validation and domain rules;
• integration tests for database constraints and transactions;
• endpoint tests for status codes, headers, and representations;
• contract tests against OpenAPI;
• authorization tests for permitted and forbidden object access;
• resilience tests for timeouts, retries, and duplicate requests;
• performance tests for large collections and expensive filters.

Observability should include structured logs, metrics, traces, latency percentiles, error rates by endpoint and status, pagination usage, and deprecated-feature usage. Never log raw authorization headers or sensitive payloads by default. Correlation or trace IDs help connect client reports, gateway logs, and backend traces.

A retry policy must be method-aware. Retrying a failed GET is generally safer than retrying an unprotected POST. Even idempotent methods can cause load amplification if clients retry aggressively, so use bounded retries, exponential backoff, jitter, and server-provided guidance such as Retry-After where appropriate.

13. Common Design Failures and Best Practices

Common failures include verb-heavy URLs, inconsistent pluralization, undocumented defaults, unbounded lists, ambiguous PATCH behavior, leaking ORM objects, treating all errors as 200, and returning internal exception details. Another failure is designing from database tables outward; API resources should reflect client and domain needs rather than exposing storage structure.

A practical review checklist is:

1. Is each resource and relationship clearly named?
2. Are method semantics, status codes, headers, and representations documented?
3. Are authentication and object-level authorization enforced?
4. Are filters allow-listed and pagination bounded?
5. Are validation and errors consistent?
6. Is retry and idempotency behavior explicit?
7. Does OpenAPI match deployed behavior?
8. Is the proposed change backward-compatible for real clients?
9. Are sensitive data, logging, rate limits, and retention addressed?
10. Can the endpoint be monitored and retired safely?

Chapter Summary

RESTful API design uses resources, representations, uniform HTTP methods, and standard status codes to create predictable interactions. Resource-oriented URIs should be supported by clear method semantics: GET retrieves, POST commonly creates, PUT replaces, PATCH partially modifies, and DELETE removes. Safety and idempotency determine how clients and intermediaries may retry requests.

Production APIs require more than CRUD. Filtering must be allow-listed, pagination must be bounded and consistently ordered, and content negotiation must distinguish Accept from Content-Type. Validation should address transport, schema, domain, authorization, and persistence rules. Problem Details offers a consistent error model. OpenAPI turns an informal interface into a reviewable and testable contract.

Versioning should be based on compatibility impact, supported by deprecation policy, telemetry, and migration documentation. Security, privacy, observability, and operational resilience are integral to API quality. In both Node.js and Django, separation of routing, validation, domain services, persistence, and error handling supports maintainability.

Key Terms

REST: An architectural style for networked systems based on constraints including statelessness and a uniform interface.

Resource: A conceptual entity or collection identified by a URI.

Representation: A transferable format of a resource, such as JSON.

Endpoint: A network-accessible combination of a URI and HTTP operation.

Idempotent: Having the same intended final effect when the same request is repeated.

Safe method: An HTTP method intended only for retrieval and not state change.

Content negotiation: Agreement between client and server about representation formats.

Pagination: Dividing a collection response into bounded portions.

Cursor pagination: Pagination using an opaque position token.

Validation: Checking whether input satisfies syntactic, schema, domain, and persistence rules.

Problem Details: A standardized structure for communicating HTTP API errors.

OpenAPI: A machine-readable specification format for HTTP APIs.

Backward compatibility: The ability of existing clients to continue working after a change.

Deprecation: Formal notice that a feature remains available but is scheduled for replacement or removal.

Authentication: Establishing the identity of a requester.

Authorization: Determining what an authenticated requester may do.

Rate limiting: Restricting request volume over a defined policy period.

Contract test: A test verifying that implementation conforms to an agreed API contract.

Object-level authorization: Permission checking for the specific resource being accessed.

Trace ID: An identifier used to correlate a request across services and logs.

Review Questions

Short Answer Questions

1. Distinguish an API resource from its representation.
2. Why is Content-Type different from Accept?
3. Give one advantage and one limitation of cursor pagination.
4. What is the purpose of an idempotency key?
5. Name three elements commonly included in a Problem Details response.

Long Answer Questions

1. Explain the REST constraints and discuss why statelessness is useful for scalable APIs.
2. Compare POST, PUT, and PATCH, including their usual idempotency properties and appropriate uses.
3. Develop a versioning and deprecation strategy for an API used by web, mobile, and partner clients.
4. Explain how OpenAPI supports design, implementation, testing, and client development.
5. Discuss how security, privacy, validation, and observability should influence API design.

Analytical / Application Questions

1. Design the endpoints, status codes, pagination approach, and error responses for a university course-enrollment API. Justify each decision.
2. A payment client times out after submitting POST /payments and retries twice. Analyze the duplicate-payment risk and propose a transactionally sound idempotency design.
3. An API adds a required response field, changes an enum, and alters default sorting in one release. Classify the compatibility risks and propose a safer migration plan.

Multiple Choice Questions with Answers

Question 1

Which HTTP method is normally used to retrieve a representation without intending a state change?

A. GET

B. POST

C. PATCH

D. DELETE

Correct Answer: A. GET

Question 2

A successful request creates a new book resource. Which response is most appropriate?

A. 200 OK with no Location header

B. 201 Created with a Location header where practical

C. 204 No Content with an error body

D. 304 Not Modified

Correct Answer: B. 201 Created with a Location header where practical

Question 3

What is the primary purpose of an Idempotency-Key on a non-idempotent operation?

A. To compress the request body

B. To select the response media type

C. To prevent a retried request from repeating the business operation

D. To replace authentication

Correct Answer: C. To prevent a retried request from repeating the business operation

Question 4

Which header identifies the media type of a request body?

A. Accept

B. Content-Type

C. Vary

D. Location

Correct Answer: B. Content-Type

Question 5

Which status code best represents syntactically valid JSON whose values violate documented domain validation rules?

A. 201

B. 301

C. 422

D. 503

Correct Answer: C. 422

Question 6

Which pagination approach is generally better suited to a large, frequently changing feed?

A. Cursor pagination with stable ordering

B. Unlimited offset pagination

C. Random page selection without ordering

D. Returning the entire collection

Correct Answer: A. Cursor pagination with stable ordering

Question 7

Which change is most clearly backward-incompatible?

A. Adding a new optional endpoint

B. Adding an optional response field for tolerant clients

C. Removing an existing response field

D. Accepting an additional request media type

Correct Answer: C. Removing an existing response field

Question 8

What does OpenAPI primarily provide?

A. A database engine

B. A machine-readable description of an API contract

C. A replacement for authorization

D. A network encryption protocol

Correct Answer: B. A machine-readable description of an API contract

Question 9

Which statement correctly distinguishes authentication from authorization?

A. Authentication determines permissions; authorization encrypts data

B. Authentication identifies a principal; authorization determines permitted actions

C. They are two names for pagination

D. Authorization is required only for public endpoints

Correct Answer: B. Authentication identifies a principal; authorization determines permitted actions

Question 10

Why should an API allow-list sortable and filterable fields?

A. To ensure clients can modify arbitrary database queries

B. To prevent all caching

C. To control behavior, validate input, and reduce security and performance risks

D. To make every response return XML

Correct Answer: C. To control behavior, validate input, and reduce security and performance risks