Infrastructure as Code and Environment Automation

Introduction

Cloud platforms make it possible to create computing resources through consoles, command-line tools, and application programming interfaces (APIs). Although this flexibility is valuable, manually creating environments is slow, difficult to audit, and prone to inconsistency. A development server may use one operating-system image, network rule, or database setting while production uses another. Such differences can produce failures that are difficult to reproduce and diagnose.

Infrastructure as Code (IaC) addresses this problem by representing infrastructure in machine-readable files that can be stored, reviewed, tested, and executed through automation. Instead of documenting that a team should create a virtual network, database, and application service, IaC describes those resources in a form that an automation tool can create and manage.

Environment automation extends this idea across the software delivery lifecycle. Development, testing, staging, and production environments can be created from common definitions while allowing controlled differences such as capacity, domain names, and credentials. The objective is not merely faster provisioning. It is reliable, repeatable, observable, and governable change.

Learning Objectives

By the end of this chapter, you should be able to:

1. Distinguish declarative and imperative infrastructure automation.
2. Explain the role of state, idempotence, and dependency management in IaC.
3. Design repeatable cloud environments using modules, parameters, and separate configurations.
4. Identify and manage configuration drift.
5. Apply version control, peer review, testing, and approval to infrastructure changes.
6. Evaluate security, cost, reliability, and governance implications of environment automation.

1. Infrastructure as Code as an Operating Model

Infrastructure includes the resources that support an application: networks, subnets, routing rules, firewalls, load balancers, compute instances, containers, databases, storage, identity policies, monitoring, and DNS. IaC treats these resources as managed artefacts rather than as objects created only through a graphical console.

A typical IaC workflow is:

1. An engineer changes a configuration file.
2. The change is committed to version control.
3. Automated checks validate syntax, security, policy, and expected impact.
4. A plan or preview shows the proposed resource changes.
5. An authorized reviewer approves the change.
6. A CI/CD system applies it using controlled credentials.
7. Results, logs, and state are stored for audit and future operations.

This workflow creates a connection between infrastructure change and software-engineering practices. Infrastructure can have a history, reviewers, automated tests, release controls, and rollback procedures.

IaC does not mean that every operational action must be encoded immediately. Emergency actions, experimentation, and provider-specific tasks may still occur outside the primary workflow. However, important long-lived infrastructure should ultimately be represented in the authoritative configuration. Otherwise, manually created resources become undocumented dependencies.

2. Declarative and Imperative Automation

Declarative automation

A declarative definition specifies the desired end state. For example, it may state that a virtual network must exist with a particular address range, that three application instances must run, and that a database must be encrypted. The automation engine determines which create, update, or delete operations are required to reach that state.

Declarative tools commonly compare the desired configuration with the current state. This permits a plan such as:

• create one network;
• create two subnets;
• update a firewall rule;
• leave the existing database unchanged.

The engineer describes what should exist rather than every procedural step required to create it.

Imperative automation

Imperative automation specifies a sequence of actions. A script might create a network, wait for it to become available, create a subnet, attach a route table, and then create a server. The author controls the order and detailed commands.

Imperative approaches are useful when:

• an API exposes an operation but no mature declarative resource model exists;
• a temporary migration must run once;
• an algorithm depends on complex runtime decisions;
• an existing operational script is being automated.

They can become difficult to maintain when scripts are rerun, partially fail, or encounter resources that already exist.

The distinction is not absolute. A declarative tool may call imperative provider APIs internally, and an imperative script may attempt to enforce a desired state. The important question is whether the user expresses an end state or a procedure.

Idempotence and convergence

An operation is idempotent when applying it repeatedly produces the same intended result as applying it once, assuming the relevant external conditions have not changed. For example, “ensure that a storage bucket named logs-prod exists with encryption enabled” is closer to an idempotent operation than “create a bucket named logs-prod,” because the latter may fail when the bucket already exists.

Declarative systems aim for convergence: repeated evaluation moves the actual environment toward the desired state. Convergence is not guaranteed automatically. Non-deterministic names, timestamps, unmanaged manual changes, external scripts, and provider limitations can prevent stable results.

3. Core IaC Concepts

Resources, attributes, and dependencies

A resource is a managed infrastructure object. Attributes describe its properties, such as a subnet’s address range or a database’s storage size. Dependencies describe ordering or relationships. A compute service may depend on a network and identity role; a DNS record may depend on a load balancer address.

Good IaC makes dependencies explicit where possible. Implicit dependencies based on naming conventions or timing are fragile. A dependency graph allows an engine to create independent resources in parallel while respecting required order.

State

Many IaC tools maintain state: a record of resources they believe correspond to configuration objects. State allows the tool to compare desired configuration with the current environment and identify updates or deletions.

State is operationally important because it may contain resource identifiers and, in some tools, sensitive values. It should therefore be stored in an access-controlled remote backend, encrypted at rest, protected from simultaneous writes, and backed up. State locking helps prevent two deployments from modifying the same state concurrently.

State is not necessarily the complete truth about the cloud environment. Provider APIs, manually created resources, delayed updates, and failed operations can cause discrepancies. Teams should understand how to refresh, import, reconcile, and recover state safely.

Modules and reuse

A module is a reusable unit of infrastructure configuration. A network module might accept a name, address range, availability zones, and tags, then create the required subnets and routes. Modules reduce duplication and encourage organizational standards.

Reuse must be balanced against complexity. A module with dozens of switches may be harder to understand than several focused modules. Interfaces should be documented, defaults should be safe, and breaking changes should be versioned.

Parameters and environment-specific values

Common structure should be separated from environment-specific values. For example, all environments may use the same network pattern, while development has smaller capacity and production has higher availability.

Suitable parameters include:

• region or availability zone selection;
• instance or service size;
• replica count;
• domain name;
• retention period;
• network ranges;
• feature flags that have been approved for infrastructure use.

Secrets should not be committed in plain text. They should be retrieved at deployment time from a secrets manager or supplied through a protected CI/CD variable. Encryption does not remove the need for access control and rotation.

4. Provisioning Repeatable Cloud Environments

Repeatability means that the same version of infrastructure code, with the same declared inputs and compatible provider versions, produces materially equivalent environments. “Equivalent” does not necessarily mean identical: resource identifiers and public IP addresses may differ, but security controls, topology, and service behavior should meet the same requirements.

Environment patterns

Three common patterns are used:

1. Separate configurations: Each environment has its own files. This is simple but can duplicate logic.
2. Shared modules with separate variable files: Common structure is reused while values differ. This often provides a useful balance.
3. Environment composition: A higher-level configuration composes versioned modules for each environment. This is suitable for larger organizations but requires disciplined interfaces.

A basic environment definition may include a network module, an identity module, a data module, and an application-platform module. The application deployment itself may be managed by a separate pipeline, but the platform must expose stable interfaces such as namespaces, service accounts, databases, or queues.

Immutable and mutable infrastructure

Mutable infrastructure is modified in place. For example, an administrator may patch a running server and restart a service. This can be efficient, but the server may acquire undocumented differences.

Immutable infrastructure replaces rather than modifies deployed units. A new machine image or container image is built, tested, and deployed; the old version is then removed. Immutable approaches reduce drift within individual instances, although they do not eliminate drift in surrounding cloud resources.

A practical cloud design often combines both approaches. Infrastructure definitions manage networks and managed services, while immutable images or containers manage application hosts.

Provisioning lifecycle

A robust provisioning lifecycle includes:

• Define: express resources, policies, and relationships as code.
• Validate: check syntax, formatting, provider compatibility, security policies, and naming.
• Plan: generate a proposed change set.
• Review: assess correctness, risk, cost, and compliance.
• Apply: execute through an authenticated, logged automation identity.
• Verify: test reachability, security controls, service health, and expected outputs.
• Record: retain the code revision, plan, executor, result, and relevant evidence.

Provisioning should also be designed for failure. Partial creation, quota exhaustion, API throttling, and invalid permissions can interrupt a deployment. Teams need retry behavior, safe re-execution, clear error messages, and documented recovery procedures.

5. Configuration Drift

Configuration drift is the divergence between the approved intended configuration and the actual environment. Drift may arise from:

• manual console changes;
• emergency fixes;
• provider defaults or automatic upgrades;
• undocumented scripts;
• failed or partially completed deployments;
• changes to external dependencies;
• differences between environment definitions.

Drift can be classified as:

• Resource drift: a resource is added, removed, or replaced unexpectedly.
• Attribute drift: a managed resource exists but a property differs, such as an open firewall port.
• Dependency drift: a relationship changes, such as a route pointing to an unexpected target.
• Tooling drift: provider, module, plugin, or operating-system versions change.

Detecting drift

Drift detection may compare the IaC state and configuration with live provider data. Scheduled plans, policy scanners, cloud configuration services, and inventory queries can identify differences. Monitoring should prioritize high-risk resources such as identity policies, public network access, encryption settings, and backup configuration.

Detection must distinguish intentional change from unauthorized or accidental change. A plan showing a difference is evidence for investigation, not automatic proof that the live resource should be overwritten.

Responding to drift

There are four typical responses:

1. Reconcile: restore the live environment to the approved configuration.
2. Adopt: import an intentional existing change into IaC and review it.
3. Document: record a temporary exception with an owner and expiry date.
4. Remove: delete an unapproved resource when risk and dependencies have been assessed.

A dangerous practice is automatic correction without safeguards. An IaC tool may plan to destroy a production database if its state is incorrect or if a resource was renamed. Destructive operations require explicit review, backups where appropriate, and tested recovery procedures.

6. Versioning and Review of Infrastructure Changes

Infrastructure code belongs in a version-control system. A useful repository records:

• source files and modules;
• provider and module version constraints;
• environment inputs, excluding secrets;
• validation and security policies;
• deployment workflows;
• documentation and ownership information.

A commit should represent a coherent change. Pull requests or merge requests should show the code difference and the generated infrastructure plan. Reviewers should ask:

• What resources will be created, changed, or destroyed?
• Is the dependency graph correct?
• Does the change expose data or services unnecessarily?
• Will it increase cost or reduce availability?
• Are credentials and sensitive outputs protected?
• Is the change reversible or supported by a recovery plan?
• Does it comply with organizational policy?

Version pinning and upgrades

Provider and module versions should be constrained to known-compatible ranges. Uncontrolled upgrades may change defaults or resource behavior. Pinning improves reproducibility, but old versions can contain defects or vulnerabilities. Upgrades should therefore be deliberate, tested, and documented rather than permanently avoided.

Testing infrastructure

Infrastructure testing occurs at several levels:

• Static validation: syntax, formatting, type checks, and reference validity.
• Policy testing: rules such as “storage must be encrypted” or “public SSH access is prohibited.”
• Plan testing: inspection of proposed resource changes.
• Integration testing: deployment into an isolated account or project and verification of behavior.
• Operational testing: backup restoration, failover, monitoring, and access-control tests.

Tests should cover both positive and negative conditions. It is not enough to prove that an environment can be created; teams should test that unsafe configurations are rejected.

7. Governance, Security, and Responsible Automation

IaC increases control but also increases the impact of mistakes. A repository or pipeline with broad credentials can create, modify, or destroy significant resources. Least privilege should be applied to both humans and automation identities. Production deployment roles should be separated from planning roles where practical.

Important controls include:

• protected branches and mandatory review;
• short-lived credentials and workload identity;
• encrypted state and restricted state access;
• secret scanning and prevention of secret commits;
• policy-as-code checks;
• approval gates for production and destructive actions;
• audit logs that record actor, time, revision, and result;
• cost budgets, quotas, and tagging standards;
• separation of development and production accounts or projects.

Compliance requirements may affect data residency, retention, encryption, access logging, segregation of duties, and evidence preservation. Automation should make compliant behavior easier, not hide accountability behind a pipeline. Human approval remains important for high-impact changes, especially where automated decisions can affect customer data or service availability.

Artificial intelligence can assist with code suggestions, plan summaries, anomaly detection, and policy explanations. However, generated infrastructure must be treated as untrusted until validated. AI systems may invent unsupported resource attributes, expose sensitive configuration in prompts, or recommend overly broad permissions. Organizations should retain human review, test generated changes, and avoid sending confidential state or credentials to unauthorized services.

8. Worked Example: A Repeatable Web Environment

Consider a small web application requiring a virtual network, two private application subnets, a public load balancer, an encrypted database, and centralized logging. The team wants development and production environments to have the same architecture but different capacities.

The shared module defines the topology and security rules. Inputs include environment name, region, database size, and desired application replica count. Development uses one application replica and a small database. Production uses at least two replicas, a larger database, automated backups, and stricter deletion protection.

A proposed production change increases the application replica count from two to three and changes the database backup retention from seven to thirty days. Before applying it, the pipeline:

1. checks formatting and syntax;
2. verifies that database encryption is enabled;
3. confirms that no public database route exists;
4. generates a plan showing one application scaling change and one retention change;
5. estimates the cost impact;
6. requests review from an application owner and an infrastructure owner;
7. applies the approved plan using a production deployment identity;
8. verifies health checks, backup settings, and log delivery.

If an engineer manually opens a database port to the internet during troubleshooting, the next scheduled drift scan detects the attribute difference. The security team investigates the event. If the change is not approved, the port is closed through the controlled workflow, and the incident is recorded. If the requirement is legitimate, the rule is redesigned to permit access through a private administrative path rather than leaving a public exception.

This example demonstrates that IaC is more than resource creation. It connects architecture, security, review, operations, and evidence.

9. Illustrative Case Study: Northstar Retail

The following is a fictional illustrative case.

Background

Northstar Retail operated an online store in one cloud region. Engineers manually created test environments, while production was maintained through a mixture of console changes and shell scripts. Releases frequently passed testing but failed in production because network rules, environment variables, and database settings differed.

Problem

The organization identified three major risks: environments could not be recreated quickly, changes were difficult to audit, and manual emergency fixes were becoming permanent. A production firewall rule also remained open after a temporary incident response activity.

Analysis

An inventory exercise found that some resources were present in the cloud account but absent from scripts. Several scripts were not idempotent: rerunning them created duplicate monitoring rules or failed when a resource already existed. The team also found that credentials were stored in a private but weakly protected repository.

Approach and solution

Northstar adopted a shared-module IaC structure. Network, identity, logging, and database components were versioned separately. Environment configurations supplied approved parameters. Existing resources were imported after review rather than destroyed and recreated.

The team introduced pull requests with generated plans, mandatory security checks, encrypted remote state, and separate deployment identities. Production changes required two approvals. A scheduled drift scan ran daily, and exceptions required an owner and expiry date. Secrets moved to a managed secrets service. Standard images were rebuilt rather than manually patched.

Outcome

Test environments became reproducible, and new environments could be created from the same reviewed definitions. The open firewall rule was detected and removed. Reviewers gained visibility into planned deletions and cost increases before deployment. The organization did not eliminate all incidents, but it reduced undocumented change and improved recovery confidence.

Learning points

The case illustrates four principles. First, adoption should begin with inventory and risk reduction rather than an immediate rewrite of everything. Second, state and imports require careful management. Third, drift detection is useful only when findings have owners and response procedures. Finally, automation improves governance when it is connected to review, identity, logging, and operational testing.

10. Challenges and Limitations

IaC introduces its own complexity. Teams must learn provider behavior, state management, dependency modeling, testing, and safe handling of destructive changes. A poorly designed module can hide important details and make simple changes difficult.

Cloud providers also evolve continuously. Defaults, APIs, service limits, and pricing may change. A configuration that is valid today may require updates later. Reproducibility therefore depends on recording tool versions and periodically testing the whole provisioning process.

Rollback is more complicated than reverting a source-code commit. Reverting code may destroy a resource, lose data, or fail because the provider cannot reverse an operation. For stateful services, recovery usually requires backups, point-in-time restoration, or a migration plan rather than a simple reverse deployment.

Finally, IaC cannot correct unclear ownership or poor architecture. It can automate an insecure design very efficiently. Technical automation must therefore be combined with architecture review, security engineering, cost management, and operational responsibility.

11. Best-Practice Checklist

• Treat infrastructure definitions as production software.
• Store IaC in version control with meaningful history.
• Prefer declarative resource management where it provides reliable convergence.
• Make dependencies explicit and modules focused.
• Separate reusable structure from environment-specific inputs.
• Keep secrets out of repositories and state where possible.
• Protect, encrypt, back up, and lock shared state.
• Pin and deliberately upgrade provider and module versions.
• Generate and review plans before applying changes.
• Test syntax, policy, integration behavior, backups, and failure recovery.
• Detect drift regularly and define a response for each finding.
• Use least privilege, protected branches, approvals, and audit logs.
• Require additional safeguards for destructive or production changes.
• Record exceptions with owners and expiry dates.
• Treat AI-generated infrastructure as a draft requiring human validation.

Chapter Summary

Infrastructure as Code represents cloud infrastructure in versioned, reviewable, and executable definitions. Declarative automation expresses a desired state, while imperative automation specifies a sequence of actions; both have valid uses. Idempotence, convergence, dependencies, modules, parameters, and state are central to reliable automation.

Repeatable environments use common structures with controlled environment-specific inputs. Drift occurs when live resources diverge from approved definitions and must be detected, investigated, and reconciled safely. Version control, plan review, policy checks, testing, protected credentials, and audit records turn infrastructure changes into governed delivery activities. IaC improves speed and consistency, but it does not remove the need for architecture, security, cost control, recovery planning, or human accountability.

Key Terms

Infrastructure as Code: Managing infrastructure through machine-readable, versioned definitions.

Declarative automation: Automation that specifies the desired end state.

Imperative automation: Automation that specifies the sequence of actions to execute.

Idempotence: The property that repeated application produces the same intended result as one application.

Convergence: Movement of an actual environment toward a declared desired state.

Resource: A managed cloud object such as a network, database, or load balancer.

Dependency graph: Relationships that determine resource order and interaction.

State: Information used by an IaC tool to associate configuration objects with live resources.

Module: A reusable, encapsulated unit of infrastructure configuration.

Parameter: An input that customizes a shared configuration.

Configuration drift: Divergence between approved configuration and the live environment.

Immutable infrastructure: Infrastructure replaced with new versions rather than modified in place.

Plan: A preview of changes an automation tool proposes to make.

Policy as code: Machine-enforced rules describing required or prohibited configurations.

Remote state: Shared state stored in a managed, access-controlled backend.

State locking: A mechanism that prevents conflicting concurrent state updates.

Least privilege: Giving an identity only the permissions required for its task.

Provider: A software integration that translates IaC definitions into cloud-platform operations.

Infrastructure pipeline: An automated workflow that validates, reviews, and applies infrastructure changes.

Exception: An approved deviation from a standard, normally assigned an owner and expiry date.

Review Questions

Short Answer Questions

1. What is the principal difference between declarative and imperative automation?
2. Why is state important in many IaC tools?
3. Define configuration drift and give two possible causes.
4. What is the purpose of a plan or preview before applying infrastructure changes?
5. Why should secrets not be stored in plain-text IaC repositories?

Long Answer Questions

1. Explain how version control, peer review, policy checks, and audit logs improve infrastructure governance.
2. Compare mutable and immutable infrastructure, including one benefit and one limitation of each.
3. Describe a practical strategy for using shared modules across development, testing, and production environments.
4. Explain how an organization should investigate and respond to detected drift.
5. Discuss why rollback of infrastructure changes can be more difficult than reverting application source code.

Analytical / Application Questions

1. A deployment plan proposes deleting a production database after a resource was renamed in configuration. What checks and recovery safeguards should be completed before any action is taken?
2. A team repeatedly creates duplicate firewall rules whenever its provisioning script is rerun. Diagnose the likely automation problem and propose a more reliable design.
3. Design a review and deployment workflow for a production network change that balances delivery speed with security, cost, availability, and accountability.

Multiple Choice Questions with Answers

Question 1

Which statement best describes declarative infrastructure automation?

A. It specifies the desired end state and lets the tool determine required actions.

B. It always requires a fixed sequence of shell commands.

C. It prevents all changes outside the automation tool.

D. It stores credentials directly in configuration files.

Correct Answer: A. It specifies the desired end state and lets the tool determine required actions.

Question 2

What is configuration drift?

A. A planned upgrade of an IaC provider.

B. A difference between approved configuration and the live environment.

C. A dependency explicitly defined in a module.

D. A backup of infrastructure state.

Correct Answer: B. A difference between approved configuration and the live environment.

Question 3

Which practice best protects shared IaC state?

A. Store it in an unrestricted public bucket.

B. Allow simultaneous writes without locking.

C. Use an encrypted, access-controlled backend with locking.

D. Delete it after every deployment.

Correct Answer: C. Use an encrypted, access-controlled backend with locking.

Question 4

What is the main purpose of a module in IaC?

A. To provide a reusable unit of infrastructure configuration.

B. To replace all security testing.

C. To store production passwords in source control.

D. To guarantee that cloud prices never change.

Correct Answer: A. To provide a reusable unit of infrastructure configuration.

Question 5

An engineer manually changes a production firewall rule. What is the most appropriate first response from an IaC process?

A. Ignore the change permanently.

B. Destroy the entire environment immediately.

C. Detect and investigate the drift before reconciling or adopting it.

D. Disable all monitoring.

Correct Answer: C. Detect and investigate the drift before reconciling or adopting it.

Question 6

Which control most directly supports review of the resources that a change will create or destroy?

A. A generated infrastructure plan in a pull request.

B. A larger virtual machine size.

C. A public IP address.

D. An application log rotation setting.

Correct Answer: A. A generated infrastructure plan in a pull request.

Question 7

Why are provider and module versions commonly constrained?

A. To prevent any future security updates.

B. To improve reproducibility and control unexpected behavior changes.

C. To eliminate the need for testing.

D. To make all environments physically identical.

Correct Answer: B. To improve reproducibility and control unexpected behavior changes.

Question 8

Which approach is most appropriate for a secret required during deployment?

A. Commit it to a private repository as plain text.

B. Place it in a public configuration example.

C. Retrieve it from a managed secrets service or protected pipeline variable.

D. Print it in deployment logs for troubleshooting.

Correct Answer: C. Retrieve it from a managed secrets service or protected pipeline variable.

Question 9

What is a characteristic of immutable infrastructure?

A. Running instances are manually patched indefinitely.

B. New versions replace existing deployment units rather than modifying them in place.

C. All resources are created only through a cloud console.

D. Configuration drift becomes impossible in every resource.

Correct Answer: B. New versions replace existing deployment units rather than modifying them in place.

Question 10

Which statement about AI-generated IaC is most accurate?

A. It can be applied directly without review because generated code is deterministic.

B. It should be treated as a draft and validated for correctness, security, and policy compliance.

C. It removes the need for least-privilege permissions.

D. It guarantees that provider-specific attributes are valid.

Correct Answer: B. It should be treated as a draft and validated for correctness, security, and policy compliance.