Infrastructure as Code and Configuration Automation

Introduction

Cloud infrastructure is created through networks of interdependent resources: virtual networks, subnets, routing tables, security groups, compute instances, storage services, databases, identity policies, monitoring systems, and deployment pipelines. Creating these resources manually through a web console may be acceptable for experimentation, but it becomes unreliable at organisational scale. Manual procedures are difficult to reproduce, document, audit, review, and recover.

Infrastructure as Code (IaC) addresses this problem by representing infrastructure in machine-readable files and managing it through software-engineering practices. A desired environment can therefore be reviewed, versioned, tested, automatically deployed, and recreated consistently. Configuration automation extends this principle to the operating systems, middleware, applications, and runtime settings hosted on the infrastructure.

IaC does not mean that every infrastructure decision is automatically correct. It means that decisions are made explicit and managed systematically. Good automation combines declarative modelling, modular design, state management, version control, policy enforcement, testing, drift detection, secure credentials, and controlled deployment processes.

Learning Objectives

By the end of this chapter, you should be able to:

1. Explain the purpose and operating models of Infrastructure as Code.
2. Distinguish declarative and imperative automation.
3. Design reusable templates and modules.
4. Explain state management, drift, idempotence, and reconciliation.
5. Apply version control and environment-promotion practices.
6. Compare provisioning tools with configuration-management tools.
7. Describe policy as code, testing strategies, and secure automation.
8. Analyse an infrastructure-automation scenario and recommend an implementation approach.

1. The Foundations of Infrastructure as Code

1.1 Desired state and actual state

IaC commonly distinguishes between two states:

• Desired state: the resources and properties that should exist.
• Actual state: the resources and properties currently present in the cloud environment.

Automation evaluates the difference between these states and performs actions intended to make actual state conform to desired state. This is sometimes called reconciliation. For example, a template may specify three application servers, an encrypted storage volume, and a private database subnet. If only two servers exist, the automation should create the missing server. If a storage volume is unencrypted, the system should report or correct the non-compliance according to its governance policy.

The ideal relationship can be expressed conceptually as:

This is not a physical law or performance equation; it is a model of the automation objective. Its success depends on provider APIs, permissions, tool behaviour, resource dependencies, and the correctness of the declared configuration.

1.2 Declarative and imperative approaches

A declarative approach specifies the required result. The tool determines the sequence of API operations needed to achieve it. A declaration might state that a virtual network must contain two private subnets, without explicitly describing every API call.

An imperative approach specifies the steps to execute, such as creating a network, then creating subnets, then attaching route tables. Scripts written with cloud command-line interfaces are often imperative.

The distinction is not absolute. Declarative tools may contain procedural hooks, and imperative scripts can be designed to be idempotent. Idempotence means that applying the same operation repeatedly produces the same intended result after the first successful application. For example, “ensure a user exists” is generally idempotent, whereas “create a user” may fail or create duplicates when run repeatedly.

1.3 Benefits and trade-offs

IaC provides several important benefits:

• Consistency: equivalent environments are built from the same definitions.
• Reproducibility: environments can be recreated after failure or in another region.
• Auditability: changes have authors, timestamps, review records, and version history.
• Speed: automated deployment reduces repetitive manual work.
• Scalability: the same patterns can manage many accounts, projects, or regions.
• Documentation: code describes infrastructure more precisely than informal diagrams.
• Recovery: tested definitions support disaster recovery and controlled rebuilding.

However, IaC also introduces risks. A mistaken template can change or destroy many resources quickly. Provider APIs may behave differently across regions. State files may contain sensitive information. Poorly designed modules can hide important decisions or create excessive coupling. Automation may also reproduce an insecure design consistently. Consequently, IaC improves control only when combined with review, testing, access management, monitoring, and operational responsibility.

2. Templates, Resources, Parameters, and Dependencies

An IaC template normally contains resource declarations, input parameters, variables, outputs, metadata, and dependency relationships. A resource is a cloud object such as a subnet, load balancer, database, or identity role. Parameters allow the same template to be used with different values, such as region, environment name, or instance size.

A well-designed definition separates:

• Logic: how resources relate to one another.
• Configuration: values that vary between environments.
• Secrets: credentials and sensitive data, which should not be embedded in source files.
• Outputs: values needed by other systems, such as a load-balancer address.

Dependencies may be explicit or inferred. If a compute instance refers to a subnet, the tool can often infer that the subnet must exist first. Explicit dependencies are useful when an ordering requirement is not visible from direct references, but excessive explicit dependencies can make code difficult to maintain.

Naming and tagging are also part of infrastructure design. Standard tags may identify the owner, application, environment, cost centre, data classification, and lifecycle. These attributes support billing analysis, incident response, inventory, and automated policy enforcement.

2.1 Modularity

A module is a reusable unit of infrastructure code. For example, a network module may create a virtual network, subnets, routes, and associated security controls. A database module may expose parameters for engine type, storage size, backup retention, and network placement.

Good modules have:

1. A clear purpose.
2. A small and understandable interface.
3. Sensible secure defaults.
4. Explicit inputs and outputs.
5. Documentation and examples.
6. Versioning and compatibility rules.
7. Tests that verify important behaviour.

A module should not expose every provider-specific detail unless consumers genuinely need it. At the same time, excessive abstraction can make troubleshooting difficult. The correct level of abstraction is the one that reduces duplication without hiding important operational consequences.

A module registry may contain organisation-approved modules. Such a registry improves consistency, but it requires ownership, security review, release management, and vulnerability monitoring. A module dependency should be pinned to a known version rather than silently changing whenever a pipeline runs.

3. State Management and Drift Detection

3.1 The role of state

Many declarative IaC tools maintain a state representation that maps declared resources to real provider resources. State may contain resource identifiers, attributes, dependency information, and sometimes sensitive values. It allows the tool to determine what has changed and what actions are required.

State must be treated as critical operational data. In a team, it should normally be stored remotely in an approved backend with encryption, access control, backup, and locking. Locking prevents two deployment processes from modifying the same state simultaneously. Without locking, concurrent operations can overwrite each other or produce an inaccurate representation of infrastructure.

State-management practices include:

• Use a separate state boundary for each appropriate environment or system.
• Restrict read and write permissions.
• Encrypt state in transit and at rest.
• Enable version history and recovery where supported.
• Avoid manually editing state unless a documented recovery procedure requires it.
• Remove secrets from state where the tool and provider allow that option.
• Protect state from accidental deletion.

Separating state too aggressively can make dependencies difficult to manage; combining unrelated systems into one state can make changes slow and increase the impact of errors. The boundary should reflect ownership, lifecycle, blast radius, and deployment frequency.

3.2 Planning and applying changes

A common workflow is:

1. Author a change.
2. Format and validate the files.
3. Resolve dependencies and provider versions.
4. Generate a plan showing proposed additions, modifications, and deletions.
5. Review and approve the plan.
6. Apply the approved plan through a controlled identity.
7. Verify the resulting environment.
8. Record outputs, evidence, and operational changes.

A plan is not a substitute for review. A planned deletion of a production database may be syntactically valid but operationally unacceptable. Reviewers must understand impact, dependencies, data protection, downtime, cost, and rollback options.

3.3 Drift

Configuration drift occurs when actual infrastructure differs from the approved definition. Causes include console changes, emergency fixes, provider defaults, failed deployments, and changes made by another automation system.

Drift detection can be periodic, event-driven, or performed before deployment. Possible responses are:

• Reconcile the environment to the declared state.
• Update the declaration if the change was authorised.
• Ignore an approved externally managed attribute.
• Raise an incident when the difference is unauthorised or dangerous.

Blindly overwriting drift is unsafe. A manual change may represent an emergency security fix or a data-preservation action. Teams should define ownership and precedence: which system is authoritative for each resource and which changes require formal approval.

4. Version Control and Environment Promotion

IaC files belong in version control. A repository should contain source definitions, module references, documentation, validation rules, and tests. Sensitive values and long-lived credentials should not be committed.

A professional workflow commonly includes:

• feature branches or short-lived change branches;
• peer review through pull requests;
• automated syntax, security, and policy checks;
• a generated plan attached to the review;
• approval rules for high-risk environments;
• immutable or traceable release versions;
• deployment records linked to a commit.

Environment differences should be explicit. Common approaches include separate variable sets, separate accounts or projects, overlays, or environment-specific module composition. Copying an entire template for every environment often creates divergence. Conversely, forcing production and development to be identical may be impractical because production usually requires stronger availability, backup, monitoring, and access controls.

A promotion model moves the same reviewed artefact from development to testing and then production, changing only controlled environment inputs. This is safer than rebuilding different code separately for each environment.

5. Provisioning and Configuration Management

IaC is often divided into two related activities.

Provisioning creates or changes cloud resources: networks, virtual machines, managed databases, queues, and identity roles. Tools in this category usually interact with provider APIs and manage resource lifecycles.

Configuration management configures software inside operating systems or hosts: packages, users, services, files, certificates, runtime settings, and agents. It may be agent-based or agentless and often applies desired configuration repeatedly.

The boundary is increasingly blurred by containers, immutable machine images, managed services, and serverless platforms. A practical strategy is to choose the least complex mechanism that meets the requirement:

• Use managed services where appropriate.
• Build immutable images for stable operating-system configuration.
• Use startup configuration for limited initialisation.
• Use configuration-management tools where systems must be changed in place.
• Prefer application deployment mechanisms for application releases rather than treating application code as operating-system configuration.

A layered model is useful: foundational networking and identity, compute or managed services, host configuration, application deployment, and observability. Each layer should have defined ownership and interfaces.

6. Policy as Code

Policy as code expresses governance rules in machine-evaluable form. Examples include:

• storage must be encrypted;
• public ingress must be restricted;
• resources must have an owner tag;
• production databases must have backups;
• workloads may use only approved regions;
• privileged roles require multifactor authentication;
• container images must originate from an approved registry.

Policies can be applied at different points:

1. Authoring time: editor extensions and local validation.
2. Pull-request time: static analysis and compliance checks.
3. Plan time: inspection of proposed infrastructure.
4. Deployment time: admission or organisational controls.
5. Runtime: continuous monitoring and remediation.

Preventive controls block an unsafe change; detective controls identify violations; corrective controls remediate them. A mature programme uses all three, while allowing documented exceptions with owners, reasons, expiry dates, and compensating controls.

Policy must be designed carefully. A rule that is too broad may block legitimate work; one that is too narrow creates false assurance. Policies should be tested with compliant and non-compliant examples and should produce actionable messages.

7. Testing Infrastructure Code

Infrastructure requires testing because defects can cause outages, data loss, security exposure, or unexpected cost.

7.1 Testing levels

• Formatting and syntax tests: verify valid structure and style.
• Static analysis: identify insecure or obsolete configurations.
• Unit tests: test module logic and input validation without creating real resources.
• Plan tests: inspect the proposed resource graph and values.
• Policy tests: verify organisational requirements.
• Integration tests: deploy to an isolated environment and test actual provider behaviour.
• End-to-end tests: verify that users or dependent systems can use the resulting service.
• Destructive and recovery tests: verify deletion safeguards, backups, restoration, and rollback.

Testing should include negative cases. For example, a network module should be tested not only with valid subnet ranges but also with overlapping ranges, missing mandatory tags, invalid region combinations, and insecure public exposure.

A test environment is not automatically safe. It may still access production data, incur costs, or create public endpoints. Test accounts should have bounded permissions, budget controls, and cleanup procedures.

8. Secure Automation Practices

Automation identities are powerful and therefore require strong control. Recommended practices include:

• Use short-lived credentials or workload identity instead of static keys.
• Apply least privilege to deployment roles.
• Separate planning permissions from production-apply permissions where practical.
• Protect approval steps and require stronger approval for destructive changes.
• Store secrets in a dedicated secret-management service.
• Mask secrets in logs and avoid passing them through command-line arguments.
• Pin provider, module, and action versions.
• Verify third-party modules and software provenance.
• Generate audit logs for changes and administrative actions.
• Use separate accounts or projects to reduce blast radius.
• Include deletion protection, backups, and recovery procedures for stateful resources.

Secrets deserve special attention. A secret may appear in source code, state, plan output, logs, crash reports, or environment variables. A secure design minimises the number of locations in which a secret exists and rotates it when exposure is suspected. Encryption does not eliminate the need for access control: anyone able to decrypt the data may still misuse it.

Supply-chain risk applies to IaC modules, provider plugins, CI/CD actions, container images, and configuration packages. Organisations should maintain an approved catalogue, scan dependencies, review licences and provenance, and monitor security advisories.

9. An Applied Case: Northstar Retail Platform

The following is an illustrative case study.

Background

Northstar Retail operates an online ordering platform in two cloud regions. Development teams created resources manually, while operations staff maintained server settings through ad hoc scripts. Environments differed in network rules, backup settings, and monitoring. A recovery exercise showed that rebuilding the production environment would take several days.

Problem

The organisation needed repeatable environments, controlled changes, stronger security, and a credible disaster-recovery process. It also needed to avoid exposing customer data in development.

Analysis

An assessment identified four main problems:

1. Infrastructure had no authoritative version-controlled definition.
2. Shared credentials were used by scripts and were difficult to rotate.
3. Manual changes caused drift that was discovered only during incidents.
4. Configuration and application deployment responsibilities were unclear.

The team classified resources by lifecycle. Network and identity foundations changed infrequently and were separated from application stacks. Databases and storage were treated as stateful resources requiring backup and deletion safeguards. Stateless application resources were designed for replacement.

Approach and solution

The team established a repository containing reviewed modules for networking, identity, logging, and application services. Environment-specific values were stored separately from reusable logic. A remote, encrypted state backend with locking was introduced for each environment boundary.

A pipeline performed formatting, validation, static security analysis, policy checks, and plan generation. A pull request was required for changes. Development deployments were automatic after checks passed; production required an approved plan and two authorised reviewers. The deployment identity used short-lived federation and had no permanent access keys.

Policies required encryption, mandatory ownership tags, private database placement, approved regions, and backup configuration. A scheduled drift check compared declared and actual infrastructure. Emergency console changes were permitted only through a documented process and had to be reconciled into code afterward.

Host configuration was reduced by producing a standard immutable machine image. Application releases were handled by a deployment service rather than by modifying servers manually. Synthetic tests verified that the public endpoint could reach the application while databases remained inaccessible from the public network.

Outcome

Northstar could recreate a test environment from a known commit and could promote the same reviewed artefact through environments. Drift alerts identified unauthorised changes earlier. Recovery procedures became testable, and audit evidence was linked to repository commits and pipeline runs. The organisation did not eliminate all risk: provider outages, defective modules, and incorrect requirements remained possible. It reduced operational uncertainty by making infrastructure explicit, reviewable, and recoverable.

Learning points

• Reproducibility requires both code and controlled dependencies.
• State is an operational asset, not merely a technical detail.
• Policy checks should be combined with human review.
• Stateful resources need stronger protection than disposable resources.
• Automation should clarify ownership rather than conceal it.

10. Professional, Governance, and Operational Implications

IaC changes professional roles. Infrastructure specialists increasingly write code, review pull requests, design tests, and interpret automated plans. Developers must understand network, identity, cost, reliability, and compliance implications. Security specialists contribute policies and threat models early in the delivery process rather than only auditing completed systems.

Governance should define who may create resources, who owns modules, how exceptions are approved, how long environments may live, and how unused resources are removed. Cost governance is also important. Templates can enforce tagging, budgets, size limits, and expiration dates, but teams must still review actual usage and business value.

Automation can affect fairness and accountability. A policy that blocks deployment without a clear explanation can encourage workarounds. An automated remediation that deletes resources may cause disproportionate harm if classification is wrong. Human escalation paths, accessible logs, and reversible actions are therefore part of responsible automation.

11. Contemporary Developments and Limitations

Cloud providers increasingly offer native declarative services, cross-account deployment systems, drift monitoring, and policy engines. Platform engineering teams create internal “golden paths” that expose approved infrastructure through simpler interfaces. GitOps extends the version-control model by treating a repository as the desired source for continuously reconciled environments.

Artificial intelligence can assist with template generation, documentation, anomaly detection, and policy suggestions. However, generated code may contain insecure permissions, incorrect dependencies, or destructive changes. AI output must be reviewed, tested, and scanned like code written by any other contributor. Confidential configurations should not be submitted to an unapproved service.

Important limitations remain:

• Not all provider behaviour is fully represented declaratively.
• Some resources are difficult to import or update safely.
• External manual systems can undermine a single source of truth.
• Large state graphs can slow planning and increase coupling.
• Rollback is not always possible, especially after destructive data changes.
• Infrastructure code can become repetitive, complex, or obsolete.
• Provider-specific definitions may reduce portability.

The goal is therefore not maximum automation. The goal is controlled, understandable, secure, and recoverable automation appropriate to the service and its risks.

Best-Practice Checklist

Before approving an infrastructure change, ask:

1. Is the desired state clear and version controlled?
2. Are inputs, outputs, dependencies, and ownership documented?
3. Is the change using approved, pinned modules and providers?
4. Has the plan been reviewed for security, cost, availability, and data impact?
5. Are secrets excluded from code, logs, and unnecessary state?
6. Are policy, integration, and negative tests present?
7. Is the deployment identity least-privileged and auditable?
8. Could the change cause destructive or irreversible effects?
9. Is drift monitored and is the authority for each resource clear?
10. Can the environment be recovered from a known version?

Chapter Summary

Infrastructure as Code represents cloud infrastructure in machine-readable, version-controlled definitions. Declarative models describe the desired state, while automation tools calculate actions that move actual infrastructure toward that state. Templates become more maintainable when they separate logic, configuration, secrets, and outputs and when reusable modules have clear interfaces and secure defaults.

State management enables resource tracking and planning but introduces responsibilities for encryption, access control, locking, backup, and recovery. Drift detection identifies differences between approved and actual infrastructure; reconciliation must be governed carefully because not every manual change should be overwritten. Version control, peer review, environment promotion, and pinned dependencies provide the foundation for reliable change management.

Provisioning and configuration management address different layers of the environment, although modern immutable images and managed services blur the boundary. Policy as code enforces security, compliance, cost, and operational requirements throughout the delivery lifecycle. Testing should range from syntax and static analysis to integration, end-to-end, recovery, and negative testing.

Secure automation requires least privilege, short-lived credentials, protected state, secret management, dependency verification, audit logs, and safeguards for destructive actions. IaC improves repeatability and governance, but it does not remove the need for expertise, human review, monitoring, recovery planning, and accountability.

Key Terms

Infrastructure as Code: Managing infrastructure through machine-readable definitions and software practices.

Declarative configuration: A description of the desired result rather than a sequence of commands.

Imperative automation: Automation that specifies the actions and their execution order.

Desired state: The resources and properties that should exist.

Actual state: The resources and properties currently present in an environment.

Idempotence: The property that repeated application produces the same intended result.

Template: A structured file describing infrastructure resources and relationships.

Module: A reusable, versioned unit of infrastructure code.

State: Metadata used to associate declared resources with real infrastructure.

State locking: Preventing concurrent operations from corrupting shared state.

Configuration drift: A difference between declared and actual configuration.

Provisioning: Creating or changing infrastructure resources.

Configuration management: Applying operating-system and software settings.

Policy as code: Machine-evaluable rules for governance and compliance.

Static analysis: Examining code without deploying it to identify defects or risks.

Least privilege: Granting only the permissions required for a task.

Immutable infrastructure: Infrastructure replaced with a new version rather than modified in place.

GitOps: A delivery model in which version-controlled definitions drive continuous reconciliation.

Blast radius: The scope of systems affected by a failure or change.

Drift detection: The process of identifying differences between desired and actual state.

Review Questions

Short Answer Questions

1. What is the difference between desired state and actual state?
2. Define idempotence and give one infrastructure example.
3. Why should IaC state be protected and locked?
4. What is configuration drift?
5. State two characteristics of a well-designed module.

Long Answer Questions

1. Compare declarative and imperative automation, including their strengths and risks.
2. Explain a professional workflow for reviewing and applying an infrastructure change.
3. Describe how policy as code can support security and compliance.
4. Explain the relationship between provisioning and configuration management.
5. Discuss the security risks of automation identities, state files, and third-party modules.

Analytical / Application Questions

1. A production environment has been changed manually to address an outage, but the change now appears as drift. Analyse the appropriate response and identify the governance decisions required.
2. Design a testing strategy for a reusable network module that creates public and private subnets, routing, and security controls. Include positive and negative tests.
3. An organisation wants to share one large state file across ten applications and three environments. Evaluate the design and recommend a more suitable state-boundary strategy.

Multiple Choice Questions with Answers

Question 1

Which statement best describes declarative infrastructure automation?

A. It specifies the desired end state and lets the tool determine implementation steps.

B. It requires an operator to execute every cloud API call manually.

C. It always modifies resources in place rather than replacing them.

D. It prevents the use of version control.

Correct Answer: A. It specifies the desired end state and lets the tool determine implementation steps.

Question 2

What is the primary purpose of infrastructure state?

A. To store application source code.

B. To map declared resources to real provider resources and support change planning.

C. To replace identity and access management.

D. To guarantee that no infrastructure can ever drift.

Correct Answer: B. To map declared resources to real provider resources and support change planning.

Question 3

Which practice best protects shared IaC state?

A. Store it in an unrestricted public repository.

B. Allow multiple deployments to write it simultaneously.

C. Use an encrypted backend with access control and locking.

D. Delete it after every deployment.

Correct Answer: C. Use an encrypted backend with access control and locking.

Question 4

What is configuration drift?

A. A planned version release of a module.

B. A difference between declared infrastructure and actual infrastructure.

C. A method for encrypting secrets.

D. A test that runs without cloud resources.

Correct Answer: B. A difference between declared infrastructure and actual infrastructure.

Question 5

Which is an example of policy as code?

A. A meeting in which engineers discuss security.

B. A rule requiring all storage resources to be encrypted, evaluated automatically.

C. A manually drawn architecture diagram.

D. A password written in a deployment script.

Correct Answer: B. A rule requiring all storage resources to be encrypted, evaluated automatically.

Question 6

Which credential practice is most secure for a CI/CD deployment identity?

A. Use a permanent administrator access key.

B. Share one password among all engineers.

C. Use short-lived credentials with only required permissions.

D. Place credentials in source control for reproducibility.

Correct Answer: C. Use short-lived credentials with only required permissions.

Question 7

Which test most directly verifies behaviour against a real cloud provider?

A. A formatting check.

B. An integration test in an isolated environment.

C. A variable-naming check.

D. A documentation review.

Correct Answer: B. An integration test in an isolated environment.

Question 8

Why are immutable machine images useful?

A. They eliminate the need for versioning.

B. They encourage replacement with a known configuration instead of uncontrolled in-place changes.

C. They guarantee that applications contain no defects.

D. They allow secrets to be embedded safely.

Correct Answer: B. They encourage replacement with a known configuration instead of uncontrolled in-place changes.

Question 9

A module is intended to be reused by many teams. Which design is most appropriate?

A. Expose a clear interface, use secure defaults, and pin its version.

B. Hide all documentation and change its inputs without notice.

C. Embed long-lived credentials in the module.

D. Copy a separate untracked version for every consumer.

Correct Answer: A. Expose a clear interface, use secure defaults, and pin its version.

Question 10

What is a major limitation of automatically reconciling every detected drift?

A. It may overwrite an authorised emergency change or cause destructive effects.

B. It makes infrastructure definitions version controlled.

C. It improves auditability.

D. It prevents all provider API calls.

Correct Answer: A. It may overwrite an authorised emergency change or cause destructive effects.