Infrastructure as Code and Configuration Management

Introduction

Cloud platforms make it possible to create networks, virtual machines, databases, containers, identity policies, and monitoring services through application programming interfaces (APIs). Although this flexibility accelerates delivery, manually creating resources through web consoles or ad hoc commands creates inconsistency, weakens auditability, and makes environments difficult to reproduce. Infrastructure as Code (IaC) addresses this problem by representing infrastructure in machine-readable files that can be versioned, reviewed, tested, and applied automatically.

Configuration management complements IaC. IaC generally establishes resources and their relationships, while configuration management ensures that operating systems, applications, services, and security settings are in the desired condition. Modern DevOps teams commonly combine declarative provisioning, configuration automation, continuous integration, policy as code, and immutable infrastructure.

This chapter focuses on practical and conceptual foundations using Terraform as the principal example. The same principles apply to equivalent tools such as OpenTofu, CloudFormation, Pulumi, Ansible, Chef, Puppet, and Salt.

Learning Objectives

By the end of this chapter, a learner should be able to:

1. Explain Infrastructure as Code and distinguish declarative from imperative approaches.
2. Design a basic Terraform project using providers, resources, variables, outputs, modules, and state.
3. Explain configuration management and select an appropriate operating model.
4. Identify and manage state, dependencies, secrets, and environment differences.
5. Detect configuration drift and design suitable reconciliation processes.
6. Apply policy as code, testing, and governance controls to infrastructure delivery.
7. Evaluate immutable infrastructure and its relationship with continuous delivery.
8. Analyse the technical, operational, security, and ethical implications of automated infrastructure.

1. Infrastructure as Code as an Engineering Discipline

Infrastructure as Code is the practice of defining infrastructure in files that can be processed by tools rather than relying primarily on manual actions. An IaC repository may contain definitions for a virtual network, subnets, firewall rules, a managed database, Kubernetes clusters, DNS records, and observability services.

IaC is more than a scripting technique. It introduces software-engineering practices into infrastructure work:

• Version control: changes are recorded with authorship, history, and review.
• Repeatability: the same definition can create equivalent environments.
• Automation: provisioning can be integrated into CI/CD pipelines.
• Peer review: changes can be evaluated before affecting production.
• Testing: syntax, security, policy, and deployment behaviour can be checked.
• Documentation: code expresses intended architecture and dependencies.
• Recovery: infrastructure can be recreated after failure, subject to data and provider constraints.

The phrase “reproducible” does not mean that every deployment is physically identical. Cloud providers may assign different IP addresses, availability zones, or hardware. Reproducibility means that the same declared requirements and approved inputs produce an infrastructure arrangement consistent with those requirements.

Desired state and convergence

Most declarative IaC tools use a desired-state model. The engineer specifies what should exist, and the tool determines actions needed to approach that state. If the desired configuration is unchanged and the platform is functioning normally, applying it again should produce no unnecessary changes. This property is commonly called idempotence.

Idempotence is not absolute. Some resources have provider-generated values, time-dependent behaviour, or APIs that do not perfectly represent state. A sound design therefore treats idempotence as a target supported by stable resource definitions, lifecycle controls, and provider-specific knowledge.

2. Declarative and Imperative Approaches

An imperative approach specifies a sequence of actions. For example:

1. Create a virtual network.
2. Create a subnet.
3. Allocate an address range.
4. Create a firewall rule.
5. Launch a server.
6. Install an application.

A declarative approach specifies the required end state: a network with a particular address range, a subnet associated with it, approved firewall rules, and a server attached to the subnet. The tool determines a suitable execution order.

Neither approach is universally superior. Imperative commands are useful for one-time migration steps, debugging, and operations that cannot be represented declaratively. Declarative IaC is generally preferable for long-lived infrastructure because it creates a reviewable model of intent. In practice, teams often use a declarative tool for resource provisioning and an imperative or configuration-management tool for exceptional tasks.

3. Terraform’s Core Model

Terraform configuration is written in HashiCorp Configuration Language (HCL). A typical project contains the following elements:

• Provider: a plugin that communicates with a platform API.
• Resource: an object managed by Terraform, such as a network or database.
• Data source: information read from an existing system.
• Variable: an input parameter.
• Local value: a reusable expression within a module.
• Output: a value exposed after planning or application.
• Module: a reusable collection of related configuration.
• State: Terraform’s record of managed objects and their attributes.

A simplified configuration might be written as follows:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = var.region
}

variable "region" {
  type        = string
  description = "Deployment region"
}

resource "aws_s3_bucket" "logs" {
  bucket = var.bucket_name

  tags = {
    Environment = var.environment
    ManagedBy   = "terraform"
  }
}

output "bucket_name" {
  value = aws_s3_bucket.logs.bucket
}

The configuration expresses relationships. The provider supplies the implementation, the resource declares the bucket, variables allow controlled variation, tags support governance, and the output exposes a useful result.

The plan–apply lifecycle

A normal Terraform workflow is:

terraform init
terraform fmt -check
terraform validate
terraform plan -out=tfplan
terraform apply tfplan

init downloads providers and prepares the working directory. fmt checks canonical formatting. validate checks configuration structure and types. plan compares configuration and known state with the provider’s current information and proposes changes. apply executes an approved plan.

A plan is not a guarantee that an apply will succeed. Between planning and applying, permissions, quotas, resource availability, or external changes may change. For production, the approved plan should be stored as an artifact and applied in a controlled environment with appropriate credentials.

Dependency graphs

Terraform constructs a dependency graph. An explicit reference such as aws_vpc.main.id normally creates an implicit dependency, ensuring that the VPC is processed before a resource requiring its identifier. An explicit depends_on should be reserved for genuine dependencies that cannot be inferred from references. Excessive explicit dependencies reduce parallelism and can hide weak modelling.

4. Project Structure, Variables, and Environments

A maintainable repository separates reusable logic from environment-specific values. A common arrangement is:

infrastructure/
├── modules/
│   ├── network/
│   └── application/
├── environments/
│   ├── development/
│   ├── staging/
│   └── production/
└── README.md

Variables should have explicit types, descriptions, validation rules, and safe defaults where appropriate. For example, a variable representing a deployment tier might accept only approved values. Sensitive variables should be marked sensitive, but this does not encrypt them automatically; values may still appear in state or logs depending on the resource and operation.

Environment separation can be implemented through directories, workspaces, separate state backends, or distinct accounts and subscriptions. Separate cloud accounts or projects usually provide stronger isolation than workspaces alone. A workspace is a state-management feature, not a complete security boundary.

Modules

A module is an interface around infrastructure. A well-designed module has:

• a clear purpose;
• well-defined inputs and outputs;
• sensible defaults;
• validation and documentation;
• minimal provider assumptions;
• versioning and tests.

Modules should not conceal every detail. Excessive abstraction can make debugging difficult and may prevent consumers from using new provider capabilities. A module should represent a stable architectural concept, such as a network or service cluster, rather than merely wrapping one provider resource without adding meaningful value.

5. State Management

Terraform state maps configuration addresses to real infrastructure objects and records attributes required for planning. Without state, Terraform would have difficulty determining which remote object corresponds to a resource declaration.

State is operationally important and security-sensitive because it can contain identifiers, topology, and sometimes secrets returned by provider APIs. Local state is unsuitable for most teams because it is difficult to share safely and can be lost. A remote backend should normally provide:

• centralized access;
• encryption at rest and in transit;
• authentication and authorization;
• state locking or another concurrency control mechanism;
• versioning and recovery;
• audit logging.

Only one authorized operation should normally modify a given state at a time. State locking prevents two pipeline jobs from applying incompatible changes concurrently. Locking does not replace code review or backup.

State migration must be treated carefully. Renaming a resource in configuration can appear to Terraform as destruction and creation unless the state address is moved. Modern Terraform supports explicit moved blocks; state-move commands are also available. The objective is to preserve identity when the real object has not been replaced.

State is not the same as truth

State is a tool-managed representation, not an infallible inventory. It may be stale, incomplete, corrupted, or inconsistent with the provider. Teams should protect it, refresh it through supported workflows, and investigate discrepancies rather than editing it casually. Direct state editing is a high-risk administrative operation and should be backed up and audited.

6. Configuration Management

Configuration management maintains the desired condition of systems after infrastructure exists. It may include:

• operating-system packages and updates;
• users, groups, and file permissions;
• service configuration;
• security baselines;
• application deployment;
• logging agents and monitoring;
• scheduled jobs.

Tools such as Ansible commonly use agentless remote execution, whereas Chef and Puppet traditionally use agents that periodically enforce configuration. The choice depends on scale, network design, operating systems, compliance needs, and team expertise.

A configuration-management playbook might install a web server, deploy a controlled configuration file, enable the service, and verify that it is running. Good automation is idempotent: running it repeatedly should converge on the same result without creating duplicate users, appending repeated configuration lines, or restarting services unnecessarily.

Provisioning versus configuration

The distinction is useful but not absolute:

Cloud-native services reduce the need for traditional server configuration. Managed databases, containers, and serverless platforms shift configuration toward API objects, container images, and platform policies.

7. Immutable Infrastructure

Mutable infrastructure is changed in place. A server is created once and subsequently patched, reconfigured, and repaired. Immutable infrastructure replaces rather than modifies deployed instances. A new machine image or container image is built, tested, and deployed; the old instance is removed after traffic is safely transferred.

Immutability reduces configuration drift and creates a clearer relationship between an artifact and a deployment. It also supports rollback by retaining earlier image versions. However, it requires effective image pipelines, externalized persistent data, health checks, service discovery, and deployment strategies such as blue–green or rolling deployment.

A database should not be treated as disposable merely because application servers are immutable. Data durability, backup, replication, schema migration, and recovery objectives must be designed separately. Immutable compute with mutable, carefully governed data services is a common architecture.

8. Drift Detection and Reconciliation

Drift is a difference between the declared configuration, tool-managed state, and actual infrastructure. Causes include console changes, emergency commands, provider defaults, failed deployments, and changes made by another automation system.

A practical drift process is:

1. Detect differences through scheduled plans, provider inventory, or monitoring.
2. Classify the change as authorized, unauthorized, temporary, or erroneous.
3. Determine whether code or infrastructure should be the source of truth.
4. Review the security and operational impact.
5. Reconcile by updating code, reverting the resource, or formally importing it.
6. Record the decision and improve preventive controls.

Automatic reconciliation is appropriate for some low-risk environments but dangerous when a plan could destroy data or reverse an approved emergency change. Production reconciliation should use change windows, approvals, backups, and resource lifecycle protections.

A useful conceptual model is:

Declared configuration ──┐
                         ├─> Plan and policy checks ──> Approved change ──> Actual infrastructure
Managed state ───────────┘                                      ↑
                                                               │
                                              Drift detection ──┘

The model illustrates that state, configuration, and actual infrastructure are related but distinct. Governance is required when they disagree.

9. Policy as Code and Governance

Policy as code expresses organizational rules in machine-evaluable form. Examples include:

• storage must not be publicly readable;
• resources must contain an owner and cost-centre tag;
• production workloads must use approved regions;
• encryption must be enabled;
• databases may not use public network access;
• virtual machines must use approved image families;
• privileged identities require strong authentication.

Policy can be applied at several stages:

1. Static analysis: inspect source files before planning.
2. Plan evaluation: examine proposed resource changes.
3. Admission control: reject requests at an orchestration or cloud boundary.
4. Runtime assessment: continuously evaluate deployed resources.

Tools include Terraform validation rules, Sentinel, Open Policy Agent, Checkov, tfsec successors, and cloud-native policy services. A policy should state its scope, exception process, severity, owner, and evidence requirements. Overly rigid policies encourage unsafe workarounds; poorly enforced policies create false assurance.

Governance should distinguish preventive controls from detective controls. A policy that blocks public storage is preventive. A periodic scanner that reports public storage is detective. Mature organizations use both.

10. Security, Secrets, and Compliance

IaC repositories are high-value targets. Security practices include:

• never committing cloud access keys or passwords;
• using short-lived, workload-specific identities;
• storing secrets in a dedicated secret-management service;
• restricting who can read and modify state;
• scanning commits and artifacts;
• pinning provider and module versions;
• reviewing generated plans for sensitive changes;
• separating development and production credentials;
• logging administrative actions.

Marking a variable as sensitive controls display in some command output but does not make the underlying value safe. State backends and CI logs must be protected independently.

Compliance evidence should be generated from authoritative records: version-control history, pull requests, pipeline logs, plan artifacts, cloud audit logs, policy results, and approved exceptions. A screenshot of a console is usually weaker evidence than an immutable, timestamped pipeline record.

Ethically, automation amplifies both good and bad decisions. A flawed module can create insecure resources at scale. Engineers should use least privilege, protect customer data, test destructive behaviour, and provide a responsible process for emergency intervention. Automation must not be treated as an excuse to remove human accountability.

11. Testing and CI/CD Integration

Infrastructure pipelines should use multiple test levels:

A pull request pipeline may format and validate code, run security checks, generate a plan, and publish the plan for review. A protected deployment pipeline can then apply only the reviewed artifact, using a narrowly scoped identity.

Tests should include negative cases. For example, a module test should verify that an invalid CIDR range or unsupported region is rejected. Destructive actions should be explicitly visible and protected by approval.

12. Worked Application: A Multi-Environment Web Platform

Background

Consider an illustrative online retail company operating a web application in development, staging, and production. Initially, engineers create networks and virtual machines manually. Production contains undocumented firewall rules, staging differs from development, and an emergency console change is not reflected in source control.

Problem analysis

The company faces four risks:

1. Reproducibility risk: a failed environment cannot be rebuilt reliably.
2. Security risk: network rules differ without systematic review.
3. Operational risk: manual changes create drift.
4. Audit risk: the organization cannot show who approved each change.

Approach

The team adopts the following design:

• a version-controlled Terraform repository;
• a reusable network module;
• separate cloud accounts for production and non-production;
• remote encrypted state with locking;
• an image pipeline that builds a hardened application image;
• a deployment system that replaces application instances rather than configuring them manually;
• policy checks requiring encryption, tags, and private database networking;
• pull-request plans and protected production applies;
• scheduled drift detection.

The network module accepts a CIDR range, availability zones, and environment metadata. It returns subnet identifiers and security-group identifiers. The production environment supplies production-specific values while retaining the same module contract.

Drift event

After adoption, a security engineer temporarily adds a console firewall rule during an incident. The next scheduled plan reports an unmanaged rule. The incident record confirms that the rule is still necessary, so the engineer adds the rule to the module through a reviewed pull request and removes the manual exception. If the rule had been unauthorized, the team would have removed it after confirming that doing so would not interrupt legitimate traffic.

Outcome

The company gains reproducible environments, auditable changes, faster recovery, and earlier detection of security violations. It does not eliminate operational judgment: database migrations, incident exceptions, and provider failures still require human analysis.

Learning points

• Reuse should be controlled through module interfaces rather than copied configuration.
• Separate accounts strengthen environment isolation.
• State protection is part of security architecture.
• Immutable application servers do not eliminate data-management responsibilities.
• Drift detection is valuable only when teams have a defined response process.

13. Challenges and Limitations

IaC introduces its own complexity. Provider APIs change, modules become outdated, plans can be difficult to interpret, and large state files may slow operations. A declaration can also hide provider-specific side effects. For example, changing a property may force resource replacement, causing downtime or data loss.

Other limitations include:

• incomplete support for newly released cloud features;
• differences between plan-time assumptions and apply-time reality;
• secrets appearing in provider responses or state;
• eventual consistency in distributed cloud APIs;
• accidental privilege escalation through pipeline identities;
• overuse of modules that makes ownership unclear;
• false confidence from tests that do not exercise real provider behaviour.

Teams should read provider documentation, inspect plans, use small changes, define backup and rollback procedures, and make destructive operations explicit. “No manual changes” is not a complete operating model; emergency access, break-glass procedures, and reconciliation must be designed deliberately.

14. Contemporary Developments

Current practice is moving toward platform engineering: internal platforms provide approved modules, templates, policies, and self-service workflows to application teams. This approach hides unnecessary cloud complexity while preserving organizational guardrails.

Other developments include:

• OpenTofu as an alternative compatible with many Terraform workflows;
• infrastructure testing using ephemeral environments;
• software supply-chain security for providers and modules;
• provenance and signing of images and deployment artifacts;
• continuous compliance and cloud security posture management;
• GitOps models in which a repository drives reconciliation;
• AI-assisted generation and review of infrastructure code.

AI can suggest resources, explain plans, detect suspicious patterns, and generate tests. It must not be trusted to invent secure architecture without review. Generated code should be validated, scanned, tested, and checked against organizational policy. The authority to deploy should remain with controlled identities and approved processes.

Best-Practice Checklist

1. Keep infrastructure definitions in version control.
2. Use remote, encrypted, locked, and backed-up state.
3. Pin provider and module versions and update them deliberately.
4. Separate environments and restrict credentials by scope.
5. Require plans, peer review, and approvals for sensitive changes.
6. Use modules with clear interfaces and documented ownership.
7. Store secrets in dedicated secret-management systems.
8. Test syntax, security, policy, integration, and post-deployment health.
9. Detect drift and define whether code or an approved exception is authoritative.
10. Prefer immutable application infrastructure where operationally suitable.
11. Protect data independently through backup, replication, and recovery design.
12. Record exceptions, emergency changes, and reconciliation decisions.

Chapter Summary

Infrastructure as Code represents cloud infrastructure as versioned, reviewable, and automatable definitions. Declarative tools such as Terraform describe desired state and calculate changes through a dependency graph, while imperative tools specify an action sequence. Terraform projects are built from providers, resources, data sources, variables, outputs, modules, and state.

State enables Terraform to relate configuration to real objects, but it is sensitive, potentially stale, and not a complete source of truth. Remote backends, locking, encryption, access control, and backups are essential. Configuration management maintains operating systems and services, while immutable infrastructure reduces in-place changes by replacing tested images or instances.

Drift detection identifies disagreement between code, state, and actual infrastructure. Policy as code converts security, compliance, and architectural requirements into automated controls. Effective CI/CD combines formatting, validation, security scanning, plan review, integration testing, policy checks, controlled application, and post-deployment verification.

The professional use of IaC requires more than tool knowledge. Engineers must understand provider behaviour, identity security, data durability, governance, exception handling, and the consequences of automating decisions at scale.

Key Terms

Infrastructure as Code: Managing infrastructure through machine-readable, version-controlled definitions.

Declarative approach: Describing the desired end state rather than a precise action sequence.

Imperative approach: Specifying the ordered operations required to produce an outcome.

Idempotence: The property that repeating an operation produces the same intended result without unnecessary additional effects.

Desired state: The condition that a managed system is intended to reach and maintain.

Terraform provider: A plugin that allows Terraform to communicate with a platform or service API.

Resource: An infrastructure object managed by an IaC tool.

Data source: Information read from an existing system for use in configuration.

Module: A reusable, versioned collection of infrastructure configuration.

State: Tool-managed information linking configuration addresses to real infrastructure objects.

State locking: Concurrency control that prevents conflicting state updates.

Configuration management: Automated maintenance of operating-system, service, and application settings.

Drift: A difference between declared, managed, and actual infrastructure conditions.

Policy as code: Machine-evaluable rules that enforce governance, security, or compliance requirements.

Immutable infrastructure: Infrastructure replaced with a new version rather than modified in place.

Remote backend: A service or storage location used to centrally store and protect IaC state.

Reconciliation: The process of resolving differences between desired and actual state.

Break-glass access: Exceptional, tightly controlled emergency administrative access.

Platform engineering: Building internal platforms that provide reusable delivery capabilities and guardrails.

Review Questions

5 Short Answer Questions

1. What is Infrastructure as Code, and how does version control improve infrastructure operations?
2. State one important difference between declarative and imperative automation.
3. What purpose does Terraform state serve?
4. Define configuration drift and identify two possible causes.
5. Why should sensitive values and state files be protected separately from source code?

5 Long Answer Questions

1. Explain the Terraform plan–apply lifecycle and discuss why a plan is not an absolute guarantee of successful deployment.
2. Compare mutable and immutable infrastructure, including their effects on rollback, operations, and data management.
3. Describe how modules, variables, outputs, and separate environments contribute to maintainable IaC.
4. Explain the role of configuration management in a cloud-native architecture and distinguish it from infrastructure provisioning.
5. Design a secure state-management approach for a production Terraform platform.

3 Analytical / Application Questions

1. A production database has been changed manually through a cloud console, and the next Terraform plan proposes replacement. Analyse the risks and describe a safe investigation and reconciliation process.
2. Design a CI/CD pipeline for a reusable network module. Identify checks that should occur before planning, during plan review, and after application.
3. An organization wants to enforce encryption, approved regions, mandatory tags, and private database access. Propose a policy-as-code strategy, including preventive controls, detective controls, exceptions, and evidence collection.

Multiple Choice Questions with Answers

Question 1

Which statement best describes a declarative Infrastructure as Code approach?

A. It specifies the desired end state and allows the tool to determine actions.

B. It requires an administrator to execute every command manually.

C. It specifies only the credentials used to access a cloud platform.

D. It prohibits dependencies between infrastructure resources.

Correct Answer: A. It specifies the desired end state and allows the tool to determine actions.

Question 2

What is the primary purpose of Terraform state?

A. To store application source code

B. To map configuration resources to real infrastructure objects and record relevant attributes

C. To replace cloud-provider audit logs

D. To encrypt every secret used by an organization

Correct Answer: B. To map configuration resources to real infrastructure objects and record relevant attributes

Question 3

Which practice most directly prevents two Terraform operations from modifying the same state concurrently?

A. Using local variables

B. Formatting configuration files

C. State locking

D. Adding output values

Correct Answer: C. State locking

Question 4

A team wants application servers to be replaced with a tested image instead of patched manually. Which approach is being used?

A. Mutable infrastructure

B. Immutable infrastructure

C. Manual configuration drift

D. Unmanaged provisioning

Correct Answer: B. Immutable infrastructure

Question 5

Which activity is an example of policy as code?

A. Writing a narrative description of a network in a wiki

B. Requiring, through an automated rule, that production storage be encrypted

C. Changing a firewall rule through an unrecorded console session

D. Restarting a server after a hardware failure

Correct Answer: B. Requiring, through an automated rule, that production storage be encrypted

Question 6

What is the most appropriate response when scheduled planning detects an unexpected firewall rule?

A. Delete it immediately without investigation

B. Ignore it because state is always authoritative

C. Classify and investigate the change, then reconcile it through an approved process

D. Disable drift detection permanently

Correct Answer: C. Classify and investigate the change, then reconcile it through an approved process

Question 7

Which statement about a sensitive Terraform variable is correct?

A. It guarantees that the value is absent from state.

B. It prevents all users with cloud access from reading the value.

C. It may reduce display in command output but does not by itself secure state or logs.

D. It automatically rotates the underlying credential.

Correct Answer: C. It may reduce display in command output but does not by itself secure state or logs.

Question 8

Which item is most characteristic of configuration management rather than basic network provisioning?

A. Creating a virtual private network

B. Allocating a subnet address range

C. Installing and enabling an operating-system service

D. Creating a cloud account

Correct Answer: C. Installing and enabling an operating-system service

Question 9

Why are separate production cloud accounts generally stronger than workspaces alone for environment isolation?

A. They eliminate the need for version control.

B. They provide a stronger boundary for permissions, resources, and billing.

C. They prevent all provider API failures.

D. They make state unnecessary.

Correct Answer: B. They provide a stronger boundary for permissions, resources, and billing.

Question 10

Which sequence best represents a responsible infrastructure delivery workflow?

A. Apply first, review the result, then write configuration

B. Format and validate, run security and policy checks, review a plan, then apply an approved change

C. Make console changes, delete the repository, then import resources later

D. Store credentials in source control and run unrestricted scripts

Correct Answer: B. Format and validate, run security and policy checks, review a plan, then apply an approved change