Back-End Development and API Engineering

Introduction and Context

A web application is more than its visible interface. The back end receives requests, applies business rules, accesses data, enforces security, and returns responses that the front end or another client can understand. Back-end development therefore connects user interaction with reliable information processing.

In a full-stack project, the back end commonly consists of an application server, routing layer, business logic, data-access layer, database, authentication mechanism, and API interface. A well-designed back end is not merely functional; it is maintainable, secure, testable, observable, and capable of evolving without breaking existing clients.

This chapter uses REST-style APIs as the main model. The examples use JavaScript-like pseudocode and Express-style conventions, but the principles apply equally to Python, Java, PHP, C#, and other server-side platforms.

Learning Objectives

After completing this chapter, students should be able to:

1. Explain the structure and responsibilities of a server-side web application.
2. Design routes using HTTP methods and resource-oriented URLs.
3. Separate routing, business logic, data access, and error handling.
4. Implement CRUD operations for a practical resource.
5. Apply validation, authentication, authorization, and secure password handling.
6. Design consistent REST API responses and status codes.
7. Document API contracts using OpenAPI-style descriptions.
8. Test API behavior, including failure cases and authorization rules.
9. Evaluate the limitations, risks, and professional responsibilities of back-end systems.

1. The Server-Side Application Model

A client sends an HTTP request to a server. The request may contain a method, URL, headers, query parameters, path parameters, and a body. The server processes the request and returns an HTTP response containing a status code, headers, and usually a representation such as JSON.

A useful layered model is:

• Transport layer: receives HTTP requests and sends responses.
• Routing layer: maps a method and URL to a handler.
• Controller layer: translates HTTP input into application operations.
• Service layer: implements business rules and use cases.
• Repository or data-access layer: communicates with a database or external service.
• Security layer: authenticates identities and authorizes actions.
• Infrastructure layer: supplies logging, configuration, caching, queues, and monitoring.

The separation is architectural rather than absolute. Small applications may combine layers, but the responsibilities should still be distinguishable. If SQL queries, password checks, and HTTP formatting are mixed in every route handler, changes become difficult and defects become likely.

Figure: A layered structure for a maintainable web application.

HTTP and REST Fundamentals

HTTP methods communicate the intended operation:

REST is an architectural style based on resources, representations, stateless requests, and a uniform interface. A route such as GET /api/v1/tasks/42 identifies task 42. The URL should identify a resource, while the method expresses the operation. Routes such as /getTaskById are workable but less consistent with resource-oriented design.

A request should contain enough information to be understood independently of previous requests. This is the principle of statelessness. Authentication credentials or tokens may be sent on each request; the server should not depend on hidden conversational state in order to interpret it.

Idempotency is also important. An operation is idempotent when repeating it has the same intended effect as performing it once. GET, PUT, and DELETE are normally designed to be idempotent, although an implementation must preserve that property. POST is generally not idempotent because repeated requests may create multiple resources.

2. Project Structure and Request Flow

A practical project may use the following structure:

src/
  app.js
  routes/taskRoutes.js
  controllers/taskController.js
  services/taskService.js
  repositories/taskRepository.js
  middleware/auth.js
  middleware/errorHandler.js
  validators/taskSchemas.js
  tests/tasks.test.js

The application entry point should configure middleware and routes, while the server startup code should be separate where possible. This makes the application easier to test without opening a network port.

A simplified request flow is:

HTTP request
  → middleware (logging, parsing, authentication)
  → route matching
  → validation
  → controller
  → service and repository
  → response or centralized error handler

A controller should remain thin. For example:

router.post('/tasks', authenticate, validate(createTaskSchema), async (req, res, next) => {
  try {
    const task = await taskService.create(req.user.id, req.body);
    res.status(201).json(task);
  } catch (error) {
    next(error);
  }
});

The service contains business rules, such as preventing a user from creating a task in a closed project. The repository contains persistence operations. This division enables service tests that do not require a real database and repository tests that focus on data access.

3. CRUD Design for a Task API

Assume a task resource has the fields id, title, description, status, ownerId, createdAt, and updatedAt. A suitable API contract is:

Create

The server must not trust client-supplied ownership or audit fields. It should obtain the owner identity from the authenticated request and generate the identifier and timestamps itself. A valid request might be:

{
  "title": "Prepare API documentation",
  "description": "Describe task endpoints and error responses"
}

The server can default status to open. It should return 201 Created, a representation of the new task, and preferably a Location header identifying the new resource.

Read

Collection endpoints should support controlled filtering, pagination, and sorting. For example:

GET /api/v1/tasks?status=open&page=2&pageSize=20

The server should impose maximum page sizes. Otherwise a client can request an extremely large result set and consume excessive memory or database resources. A response may include metadata:

{
  "data": [ ... ],
  "pagination": { "page": 2, "pageSize": 20, "total": 63 }
}

The exact pagination strategy depends on requirements. Offset pagination is simple but can produce duplicates or omissions when data changes between requests. Cursor pagination is often more stable for large, frequently changing datasets.

Update and Delete

PUT conventionally replaces a complete representation; PATCH changes selected fields. The implementation should document which interpretation it follows. Updates should enforce ownership or another authorization rule and should validate the final state, not merely individual fields.

A successful deletion commonly returns 204 No Content. If the resource does not exist, an API may return 404 Not Found. The choice between revealing non-existence and returning a generic authorization response should be considered in relation to information disclosure.

4. Validation and Business Rules

Validation determines whether input has the required shape and acceptable values. Business rules determine whether an otherwise valid operation is allowed in the domain.

For a task, validation might require:

• title is a string between 1 and 120 characters;
• description, if present, is a string within a documented limit;
• status is one of open, in_progress, or done;
• unknown fields are rejected or explicitly ignored;
• identifiers have the correct format.

Validation should occur at the boundary, but it cannot replace database constraints or business checks. For example, checking that a project identifier is syntactically valid does not prove that the project exists.

A robust API returns structured validation errors without exposing internal implementation details:

{
  "error": {
    "code": "VALIDATION_ERROR",
    "message": "The request contains invalid fields.",
    "fields": {
      "title": "Title must contain 1 to 120 characters."
    },
    "requestId": "req-7f21"
  }
}

Never construct database queries by concatenating untrusted input. Parameterized queries or a carefully designed ORM protect against SQL injection. Output encoding, content-security policies, and appropriate response headers help reduce other injection risks.

5. Authentication and Authorization

Authentication answers “Who is the caller?” Authorization answers “What may that caller do?” Confusing these concepts produces serious security defects.

A common password workflow is:

1. Receive credentials over HTTPS.
2. Find the account by a unique identifier.
3. Compare the supplied password with a slow, salted password hash.
4. Issue a session identifier or access token.
5. On later requests, authenticate the credential and attach a user identity to the request.

Passwords must never be stored as plaintext or reversible encryption. A password-hashing algorithm such as Argon2id, bcrypt, or scrypt is designed to make guessing expensive. Salts must be unique and generated by a trusted library. Rate limiting, multi-factor authentication, secure password-reset procedures, and breach monitoring further improve account security.

Authorization can be implemented using:

• Role-based access control: permissions are assigned to roles such as administrator or member.
• Attribute-based access control: decisions use attributes such as owner, department, time, or resource status.
• Object-level authorization: every request for a specific resource checks whether the caller may access that object.

A route protected only by “the user is logged in” may still be insecure. For example, a user who changes /tasks/41 to /tasks/42 must not gain access to another user’s task. The service should enforce ownership or an explicit permission.

Tokens should have limited lifetime, appropriate audience and issuer checks, secure storage, and revocation or rotation strategies suitable for the threat model. Cookies require secure, HttpOnly, and SameSite settings where appropriate. CORS must be configured deliberately; allowing every origin with credentials is unsafe.

6. Error Handling and Reliability

Errors should be classified consistently:

Status-code conventions should be documented and applied consistently. A centralized error handler can map known application errors to safe public responses while logging diagnostic details privately.

A production error response should not expose stack traces, SQL statements, file paths, secrets, or internal service topology. Logs should contain enough context for investigation, including timestamp, severity, route, status, duration, and correlation or request ID. Sensitive values such as passwords, access tokens, and payment data must not be logged.

Reliability also requires timeouts, connection-pool limits, graceful shutdown, retry policies, and transaction boundaries. Retries must be selective: retrying a non-idempotent operation can create duplicate effects unless an idempotency key or equivalent design is used.

7. API Documentation and Contract Design

An API contract describes what clients may send, what the server returns, which errors are possible, and what security is required. Documentation should be written before or alongside implementation, not after clients have already guessed the behavior.

An OpenAPI-style description for a task endpoint might specify:

paths:
  /api/v1/tasks:
    post:
      summary: Create a task
      security:
        - bearerAuth: []
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/CreateTask'
      responses:
        '201':
          description: Task created
        '400':
          description: Invalid request
        '401':
          description: Authentication required

A useful contract defines field types, required fields, allowed values, maximum lengths, examples, authentication schemes, pagination rules, and error formats. Versioning, such as /api/v1, helps manage incompatible changes. Additive changes are usually safer than renaming or removing fields. Deprecation notices and migration periods are essential when clients are external or numerous.

8. Testing API Contracts and Integrations

Testing should examine both successful behavior and failure behavior. Important levels include:

• Unit tests: test a service or validation function in isolation.
• Integration tests: test application code with a real or test database.
• API tests: send HTTP requests and inspect status, headers, and body.
• Contract tests: verify that an implementation conforms to an agreed schema.
• End-to-end tests: exercise a realistic client-to-database workflow.

A task API test matrix should include:

Tests should not merely assert that a response exists. They should verify status code, content type, response schema, authorization behavior, persistence, and absence of sensitive data. Test data should be isolated, deterministic, and disposable. Secrets must be supplied through a secure test configuration rather than committed to source control.

9. Applied Case Study: Campus Equipment Booking API

Background

This illustrative case concerns a college media department that lends cameras, microphones, and lighting equipment to students. The original process uses spreadsheets and email. Staff cannot reliably determine availability, and students sometimes book equipment they are not authorized to borrow.

Problem Analysis

The system requires users, equipment items, bookings, and approval decisions. A booking has a time interval, borrower, item, status, and audit fields. The principal risks are overlapping bookings, unauthorized borrowing, incomplete validation, and loss of accountability.

The API might expose:

• POST /api/v1/bookings
• GET /api/v1/bookings?from=...&to=...
• GET /api/v1/bookings/{id}
• PATCH /api/v1/bookings/{id}/approval
• DELETE /api/v1/bookings/{id}

Approach and Solution

Authentication identifies the student or staff member. Authorization rules permit students to create bookings for eligible equipment, while staff may approve or reject them. The service checks that the requested interval is valid, the item is available, and the borrower has completed any required training.

Availability must be checked using a transaction or database constraint strong enough to prevent two simultaneous requests from both succeeding. A simple application-level “check, then insert” sequence can fail under concurrency because two requests may observe the same free slot. The service should use an appropriate transaction, locking strategy, or database exclusion mechanism for the selected database technology.

The booking service also validates that the end time is later than the start time. If start time is t_s and end time is t_e, the duration is:

where d is the booking duration and t_s and t_e use the same time standard and units. The condition t_e > t_s must hold. In practice, timestamps should be stored and compared consistently, preferably in UTC, while the user interface converts them for display.

The API returns 201 Created for a valid pending booking, 409 Conflict for an overlapping booking, and 403 Forbidden when the user lacks permission. An audit record stores who approved or rejected a booking and when.

Outcome

The department gains a single source of truth, predictable approval workflows, and reports on equipment utilization. Staff can identify conflicts before approval, while students receive clear status information.

Learning Points

The case demonstrates that CRUD endpoints alone are insufficient. Domain rules, concurrency control, authorization, time handling, auditability, and meaningful error responses are part of back-end engineering.

10. Governance, Ethics, and Contemporary Practice

Back-end systems process personal information and may influence access to services. Developers should collect only necessary data, define retention periods, restrict access, and support correction or deletion where legally required. Privacy regulations vary by jurisdiction, but principles such as purpose limitation, data minimization, security, and accountability are broadly relevant.

Accessibility is not limited to the front end. APIs should provide predictable errors, usable documentation, stable semantics, and machine-readable representations so that assistive technologies and diverse clients can function correctly.

Artificial intelligence may assist with code generation, test creation, log analysis, or API documentation. Generated code must be reviewed for insecure dependencies, incorrect authorization, data leakage, and license concerns. AI systems should not receive production secrets or unnecessary personal data. Human developers remain accountable for the behavior of deployed software.

Contemporary back ends increasingly use event-driven processing, containers, managed cloud databases, observability platforms, and automated deployment pipelines. These technologies do not remove fundamental responsibilities: explicit contracts, least privilege, validation, testing, monitoring, and safe change management remain essential.

Best-Practice Checklist

Before releasing an API, verify that:

• routes use consistent resource naming and HTTP semantics;
• input is validated at the boundary and business rules are enforced in services;
• passwords are hashed with a suitable adaptive algorithm;
• authentication and object-level authorization are tested;
• database queries are parameterized;
• errors use documented status codes and safe response bodies;
• pagination and rate limits protect expensive endpoints;
• logs exclude secrets and include correlation identifiers;
• API schemas, examples, and security requirements are documented;
• automated tests cover success, failure, and unauthorized cases;
• configuration and secrets are managed outside source code;
• dependencies are patched and production behavior is monitored.

Chapter Summary

Back-end development provides the server-side structure that connects clients, business rules, and persistent data. A maintainable application separates transport, routing, controllers, services, repositories, and infrastructure responsibilities. REST APIs use resource-oriented URLs, HTTP methods, representations, status codes, and stateless requests to create a predictable interface.

CRUD implementation must be accompanied by validation, business rules, transactions where necessary, and safe data access. Authentication establishes identity, whereas authorization determines permitted actions; object-level checks are essential to prevent unauthorized access. Centralized error handling, structured responses, secure logging, timeouts, and monitoring improve reliability.

API contracts should define schemas, errors, security, pagination, and versioning. Unit, integration, API, contract, and end-to-end tests provide different forms of assurance. The equipment-booking case demonstrates that professional back-end engineering includes concurrency, auditability, time consistency, and domain-specific rules. Privacy, least privilege, responsible AI use, and ongoing maintenance are technical and ethical responsibilities rather than optional additions.

Key Terms

API: An interface through which software systems exchange requests and responses.

Authentication: The process of verifying a caller’s identity.

Authorization: The process of deciding whether an authenticated caller may perform an action.

CRUD: Create, read, update, and delete operations.

Controller: A component that translates an HTTP request into an application operation and response.

Endpoint: A callable API location defined by a URL and HTTP method.

Idempotency: A property in which repeating an operation has the same intended effect as performing it once.

Middleware: Software that processes requests or responses between the server and route handler.

OpenAPI: A machine-readable specification format for describing HTTP APIs.

PATCH: An HTTP method commonly used for partial modification.

Repository: A component that abstracts persistence operations.

REST: An architectural style based on resources, representations, statelessness, and a uniform interface.

Service layer: The part of an application that implements use cases and business rules.

Status code: A numeric HTTP response indicator describing the result of a request.

Token: A credential presented by a client to demonstrate an authenticated session or authority.

Validation: Checking whether input has an acceptable structure and value.

Versioning: Managing changes to an API contract over time.

Web service: A network-accessible software service that communicates using standardized protocols.

Review Questions

Short Answer Questions

1. What are the separate purposes of authentication and authorization?
2. Why should a controller generally be kept thin?
3. What is the usual difference between PUT and PATCH?
4. Why are parameterized database queries important?
5. State two properties of a useful API contract.

Long Answer Questions

1. Explain the layers of a server-side web application and describe the responsibility of each layer.
2. Design a CRUD API for a library-book resource, including endpoints, methods, and likely status codes.
3. Discuss password storage, token handling, and object-level authorization in a secure API.
4. Explain how validation, centralized error handling, logging, and testing contribute to API reliability.
5. Discuss the concurrency and auditability challenges in the campus equipment-booking case.

Analytical / Application Questions

1. An authenticated student changes the identifier in a request URL and can view another student’s record. Diagnose the vulnerability and propose a service-layer correction.
2. Two clients simultaneously book the same camera for the same interval. Explain why a simple availability check followed by an insert may fail, and recommend a concurrency-control approach.
3. An API returns HTTP 200 with an error message for invalid input. Evaluate this design and propose a consistent error contract with suitable status codes and fields.

Multiple Choice Questions with Answers

Question 1

Which HTTP method is most commonly used to create a subordinate resource?

A. GET

B. POST

C. PATCH

D. DELETE

Correct Answer: B. POST

Question 2

What is the primary responsibility of a service layer?

A. Rendering HTML pages

B. Opening a network socket

C. Implementing business rules and use cases

D. Storing passwords as plaintext

Correct Answer: C. Implementing business rules and use cases

Question 3

A request lacks valid authentication credentials. Which response is generally appropriate?

A. 201 Created

B. 401 Unauthorized

C. 204 No Content

D. 409 Conflict

Correct Answer: B. 401 Unauthorized

Question 4

Which practice best protects stored user passwords?

A. Reversible encryption with one shared key

B. Plaintext storage in a private table

C. A unique salt and an adaptive password-hashing algorithm

D. Base64 encoding

Correct Answer: C. A unique salt and an adaptive password-hashing algorithm

Question 5

What does authorization determine?

A. Whether a caller may perform a requested action

B. Whether JSON is syntactically valid

C. Whether a server has an IP address

D. Whether a database table exists

Correct Answer: A. Whether a caller may perform a requested action

Question 6

Which status code commonly indicates that a requested resource does not exist?

A. 200

B. 201

C. 404

D. 503

Correct Answer: C. 404

Question 7

Why should an API impose a maximum page size on collection requests?

A. To prevent clients from using JSON

B. To limit resource consumption and oversized responses

C. To make authentication unnecessary

D. To convert GET requests into POST requests

Correct Answer: B. To limit resource consumption and oversized responses

Question 8

Which testing type checks whether an API implementation conforms to an agreed schema or contract?

A. Contract testing

B. Load-free testing

C. Compilation testing only

D. Manual styling review

Correct Answer: A. Contract testing

Question 9

In the booking case, what condition must hold for start time t_s and end time t_e?

A. t_s > t_e

B. t_s = t_e always

C. t_e > t_s

D. The two values must use unrelated time zones

Correct Answer: C. t_e > t_s

Question 10

Which response is most appropriate when a valid request conflicts with an existing unique or scheduling constraint?

A. 204 No Content

B. 301 Moved Permanently

C. 409 Conflict

D. 100 Continue

Correct Answer: C. 409 Conflict