Infrastructure as Code and Configuration Management

Introduction

Infrastructure as Code (IaC) is the practice of defining, provisioning, and managing infrastructure through machine-readable files rather than manual actions in a cloud console or on individual servers. Configuration management applies the same principle to the software and operating-system settings that make infrastructure useful: packages, users, services, files, security settings, and runtime parameters.

Together, IaC and configuration management support a central DevOps aim: infrastructure should be reproducible, reviewable, testable, and recoverable. A development team should be able to create an equivalent environment for development, testing, and production without relying on undocumented human memory. Tools such as Terraform, OpenTofu, Ansible, AWS CloudFormation, Azure Bicep, and Pulumi provide different ways to achieve this goal.

IaC is not simply “automation with scripts.” A script may execute a sequence of commands, whereas a declarative IaC system describes a desired end state and determines what actions are required to reach it. This distinction affects planning, state management, drift detection, dependency handling, and safe change control.

Learning Objectives

By the end of this chapter, you should be able to:

• distinguish declarative IaC from imperative scripting;
• explain the roles of provisioning, configuration management, and orchestration;
• describe Terraform-style resources, providers, plans, state, modules, and dependencies;
• use Ansible concepts such as inventories, playbooks, tasks, handlers, and idempotence;
• apply version control, policy checks, secrets management, and testing to infrastructure code;
• explain drift, state locking, remote state, and recovery considerations;
• select an appropriate tool or combination of tools for a practical scenario;
• evaluate the security, governance, and operational limitations of IaC.

1. The Infrastructure Automation Problem

Traditional infrastructure administration often involves manually creating virtual machines, configuring firewalls, installing packages, and recording settings in documents. This approach produces several risks:

1. Inconsistency: two servers described as identical may differ because administrators performed slightly different steps.
2. Poor traceability: it may be difficult to determine who changed a setting and why.
3. Slow recovery: rebuilding an environment depends on people remembering undocumented procedures.
4. Configuration drift: the actual environment gradually diverges from the approved design.
5. Limited scalability: manual work does not increase efficiently as the number of environments grows.

IaC addresses these risks by representing infrastructure as code stored in a repository. Changes can be reviewed through pull requests, tested automatically, approved according to policy, and applied by a controlled pipeline.

A useful distinction is:

• Provisioning: creating infrastructure resources such as networks, virtual machines, databases, load balancers, and identity roles.
• Configuration management: installing and configuring software and operating-system settings on those resources.
• Orchestration: coordinating multiple actions and systems in a particular order, often across application, infrastructure, and deployment layers.

These activities overlap but are not identical. Terraform is commonly used for provisioning cloud resources, while Ansible is commonly used for configuration management and orchestration. In practice, organizations frequently combine them.

2. Imperative and Declarative Models

An imperative program specifies the steps to perform. For example, a shell script might create a directory, install a package, edit a configuration file, and restart a service. The result depends on the current condition of the machine and on whether earlier commands succeed.

A declarative specification describes the required condition. It might state that a package must be installed, a service must be enabled and running, and a file must contain particular content. The automation engine compares the desired state with the current state and performs only the necessary actions.

Declarative automation has two important properties:

• Idempotence: applying the same desired-state definition repeatedly should produce the same final state, without unnecessary changes.
• Convergence: repeated execution should move the actual environment toward the desired state.

Idempotence is not automatic. A poorly designed command such as “append this line to a file” may add the line repeatedly. Configuration tools provide specialized resources and modules to reduce this risk, but engineers must still understand side effects.

A declarative system does not eliminate sequencing. Dependencies still exist. A database must usually exist before an application can connect to it, and a network must exist before a virtual machine can attach to it. Mature tools represent these relationships explicitly or infer them from references.

3. Terraform-Style Infrastructure as Code

Terraform and compatible tools use configuration files to describe resources managed by providers. A provider is a plugin that translates the configuration into API operations for a platform such as a cloud provider, Kubernetes, GitHub, or a monitoring service.

A simplified resource concept contains:

• a resource type;
• a local name;
• arguments describing properties;
• references to other resources;
• optional lifecycle behavior.

For example, a network resource may expose an identifier that is referenced by a compute resource. This creates an implicit dependency. An explicit dependency can be declared when the relationship cannot be inferred from an attribute reference.

The plan-and-apply workflow

A typical Terraform workflow is:

1. Write: create or modify configuration.
2. Format and validate: check style and basic configuration validity.
3. Initialize: install providers and configure the working directory.
4. Plan: compare the configuration and recorded state with the real environment, then show proposed actions.
5. Review: inspect additions, changes, and destructions.
6. Apply: execute the approved plan.
7. Record outputs: expose useful values such as an endpoint or resource identifier.

The plan is an important control point. An unexpected database replacement or firewall change should be investigated before application. Automated pipelines can generate a plan for review and require an approval before applying it.

State management

Terraform-style systems maintain state: a record connecting configuration objects to real infrastructure objects and storing relevant attributes. State allows the tool to determine what already exists and what must change.

State is operationally sensitive. It may contain resource identifiers, connection details, or values marked as sensitive. Treating the configuration repository as safe does not automatically make the state file safe.

Important state practices include:

• store state in an approved remote backend for team use;
• encrypt state at rest and in transit;
• restrict read and write permissions;
• enable state locking where supported;
• back up state and test recovery;
• avoid storing secrets in configuration or outputs unless necessary;
• use separate state boundaries for environments or trust domains.

State locking prevents two operators or pipelines from modifying the same state concurrently. Without locking, concurrent operations can overwrite one another’s records or create conflicting infrastructure changes.

Drift detection

Drift is a difference between the desired configuration and the actual infrastructure. It may result from a console change, an emergency intervention, a platform default, or an external automation system.

A refresh or plan operation can reveal drift. The correct response depends on intent:

• If the manual change was unauthorized, reapply the approved configuration.
• If the change was legitimate, update the code and then apply it through the normal process.
• If the resource is managed partly by another system, define ownership boundaries to avoid destructive conflict.

Drift detection is not perfect. Some provider attributes may be computed, normalized, eventually consistent, or unavailable through the API. Therefore, a clean plan is evidence of alignment, not an absolute proof that every operational property is correct.

Modules

A module is a reusable collection of resources, variables, outputs, and supporting logic. Modules reduce duplication and provide an interface between a platform implementation and its users.

A well-designed module should:

• have a clear purpose and ownership;
• expose only necessary variables;
• provide useful outputs;
• apply secure defaults;
• document assumptions and supported versions;
• avoid hidden side effects;
• be versioned and tested.

Modules should not become opaque “magic.” Excessive abstraction can make debugging difficult. A small organization may benefit from a few focused modules rather than a very large internal framework.

4. Ansible and Configuration Management

Ansible commonly manages machines through an inventory and YAML playbooks. It generally connects to target systems, gathers facts, and executes modules that implement operations such as package installation, file management, user creation, service control, and template rendering.

Core concepts include:

• Inventory: the hosts and groups to manage;
• Play: a mapping of hosts to tasks and variables;
• Task: one intended operation;
• Module: the implementation used by a task;
• Handler: an action triggered by a notification, often to restart a service after a configuration change;
• Role: a reusable structure containing tasks, defaults, variables, templates, files, and handlers;
• Facts: information collected about a host, such as operating system or memory.

A configuration playbook should express the target state rather than a sequence of arbitrary shell commands. For example, a package module can determine whether a package is already installed, while a raw installation command may run every time.

Ansible is often described as agentless because it usually communicates over SSH or a platform-specific remote mechanism rather than requiring a permanent management agent. “Agentless” does not mean dependency-free: the target may need an interpreter, network access, credentials, and suitable privileges.

Terraform and Ansible can be combined, but their ownership boundaries must be explicit. A common arrangement is:

1. Terraform creates the network, compute instances, identity, and managed services.
2. Ansible configures operating systems, application dependencies, service files, and runtime settings.
3. A deployment system releases the application artifact.

The same setting should not be simultaneously controlled by two tools without a deliberate integration design.

5. Version Control, Testing, and Policy

Infrastructure code should be treated as production software. A repository should normally contain the code, documentation, examples, tests, and version constraints required to reproduce the intended behavior. It should not contain credentials or unencrypted secrets.

A practical change workflow is:

1. create a branch;
2. make a focused change;
3. format and statically validate the code;
4. run security and policy checks;
5. generate a plan or test execution;
6. obtain peer review;
7. merge after required checks pass;
8. apply through a controlled pipeline;
9. monitor the resulting infrastructure.

Testing can occur at several levels:

• Syntax and validation tests: detect malformed files and invalid arguments.
• Linting: identify style problems and suspicious constructs.
• Unit tests: test module logic or templates in isolation.
• Policy tests: check organizational rules such as encryption and approved regions.
• Integration tests: create temporary resources and verify behavior.
• End-to-end tests: verify that infrastructure supports a real application path.
• Post-deployment checks: confirm health, connectivity, and security properties.

Policy as Code expresses governance rules in machine-evaluable form. Examples include “storage must be encrypted,” “public administrative ports are prohibited,” or “production resources require approved tags.” Policy checks can operate before deployment, during admission, or through continuous compliance scanning.

Policy must be carefully designed. A rule that blocks every unusual configuration may encourage bypasses. Exceptions should be explicit, time-limited, documented, and reviewed.

6. Secrets, Identity, and Governance

Secrets include passwords, API tokens, private keys, certificates, and encryption keys. They require stronger controls than ordinary configuration. Recommended practices include:

• retrieve secrets at runtime from an approved secret manager;
• use short-lived credentials where possible;
• apply least privilege to users, roles, and pipelines;
• rotate credentials and revoke unused access;
• prevent secrets from appearing in logs, plans, artifacts, and error messages;
• separate development, test, and production credentials;
• audit access and investigate unusual use.

Marking a value as “sensitive” usually affects display; it does not automatically encrypt every location in which the value may exist. Engineers must understand the storage and logging behavior of the complete toolchain.

Governance also includes resource ownership, tagging, cost allocation, data residency, retention, and change approval. Infrastructure definitions can accidentally create public data stores, excessive privileges, or resources in a jurisdiction that violates organizational requirements. Security review therefore belongs in the design and pipeline, not only after deployment.

7. Worked Example: A Reproducible Web Service Environment

Consider a small web service requiring a virtual network, a private database, two application instances, and a load balancer.

The desired design is represented as follows:

A reference from the application instance to a subnet identifier creates a dependency on the network. The database is placed in private subnets, and its firewall rule permits traffic only from the application security group. The application configuration receives the database endpoint through a secret-management mechanism rather than storing a password in the repository.

A pipeline runs formatting, validation, linting, security scanning, and policy checks. It then creates a plan. Reviewers inspect whether the plan creates the intended resources and whether any action replaces a database or opens a public port. After approval, the pipeline applies the plan using a locked remote state backend.

An Ansible role installs the required runtime, renders a service configuration, and notifies a handler only when the configuration changes. The handler restarts the service once, rather than restarting it after every unrelated task.

Suppose an administrator manually changes the database firewall to allow a temporary office address. The next scheduled plan reports a difference. The team either removes the address by reapplying the approved definition or records the business requirement in code and submits it for review. The manual change is not allowed to become an undocumented permanent dependency.

The example illustrates a division of responsibility: provisioning creates the platform, configuration management prepares hosts, and the pipeline supplies review and governance.

8. Illustrative Case Study: Northstar Retail

Background. Northstar Retail operates an online store across development, staging, and production. Its infrastructure was created manually by different administrators. Releases were slow, and production incidents were difficult to reproduce.

Problem. Development and production used different operating-system packages. A manually modified firewall rule was not recorded. During a recovery exercise, the team discovered that the database backup policy had never been applied consistently.

Analysis. An inventory identified three classes of control: cloud resources, host configuration, and operational policies. The team also found that two automation systems were editing the same firewall rules, creating ownership conflicts.

Approach. Northstar adopted Terraform for cloud resources and Ansible for host configuration. It created separate remote state files for each environment, enabled locking, and stored credentials in a secret manager. Reusable modules supplied network and database patterns. A pull-request pipeline ran validation, security scanning, policy checks, and a plan. Production required an approval. A scheduled drift plan alerted the platform team. Ownership documentation stated that Terraform controlled cloud firewall rules and that emergency changes required a follow-up code change.

Outcome. New environments became repeatable, recovery procedures were tested, and reviewers could see infrastructure changes before execution. The organization did not eliminate all incidents: provider outages, incorrect application assumptions, and incomplete monitoring remained possible. However, the causes became more visible and remediation became faster.

Learning points. IaC improves control only when code, state, credentials, policy, and operational ownership are managed together. Tool adoption alone does not create reproducibility.

9. Challenges and Limitations

IaC has significant benefits, but it also introduces risks.

• Destructive changes: changing an immutable identifier or a resource relationship may replace a resource and cause data loss.
• State corruption or loss: an unavailable or damaged state record can complicate recovery.
• Provider behavior: APIs may be eventually consistent, incomplete, or changed by vendors.
• Complexity: modules, pipelines, policies, and multiple environments require expertise.
• False confidence: a successful plan does not prove that the application is secure or healthy.
• Secret exposure: values can leak through state, logs, shell history, or debugging output.
• Over-automation: automatic remediation can repeatedly reapply an incorrect definition.
• Tool fragmentation: different teams may use incompatible conventions and ownership models.

Safe practice includes backups, small changes, explicit lifecycle settings, review of destructive operations, tested recovery, and monitoring after application. Production automation should be designed for failure, not only for the successful path.

10. Contemporary Developments and Professional Practice

Modern platform teams increasingly use internal developer platforms that expose approved infrastructure capabilities through self-service interfaces. These platforms commonly rely on IaC modules behind the scenes, allowing developers to request a database or service while preserving organizational policies.

Other developments include:

• ephemeral preview environments created for pull requests;
• continuous compliance and drift monitoring;
• policy engines integrated into CI/CD and cloud admission controls;
• infrastructure testing frameworks that create temporary environments;
• GitOps models in which a repository is the authoritative source for desired state;
• software supply-chain controls for providers, modules, collections, and actions;
• AI-assisted generation of configuration and troubleshooting suggestions.

AI can accelerate drafting and explain plans, but generated infrastructure must be reviewed, tested, and scanned. AI systems may produce insecure public access, excessive permissions, obsolete syntax, or fabricated provider arguments. Human accountability remains essential, particularly for production and regulated environments.

Best-Practice Checklist

Before applying an infrastructure change, ask:

1. Is the desired state clear and documented?
2. Is ownership of each resource unambiguous?
3. Are state storage, locking, backup, and recovery configured?
4. Are secrets excluded from source code and unsafe logs?
5. Has the change passed formatting, validation, security, and policy checks?
6. Has the plan been reviewed for replacement, deletion, exposure, and cost?
7. Are modules and dependencies version-pinned and maintained?
8. Is the change idempotent and tested in a representative environment?
9. Can drift be detected and investigated?
10. Is there a rollback or recovery procedure that has been exercised?

Chapter Summary

Infrastructure as Code represents infrastructure in machine-readable definitions so that environments can be provisioned consistently, reviewed, tested, and recovered. Declarative tools describe desired state, while the engine determines the actions needed to converge on that state. Terraform-style systems use providers, resources, plans, dependencies, modules, and state. State must be protected, locked, backed up, and treated as sensitive operational data.

Configuration management tools such as Ansible manage host and application settings through inventories, playbooks, modules, roles, handlers, and idempotent tasks. Provisioning and configuration should have clear ownership boundaries. Version control, pull requests, policy as Code, security checks, secret management, drift detection, and controlled pipelines turn automation into a dependable engineering practice.

IaC is not a guarantee of correctness. It can create destructive changes, expose secrets, or reproduce an insecure design at scale. Professional practice therefore combines declarative definitions with testing, governance, least privilege, monitoring, recovery planning, and human review.

Key Terms

Infrastructure as Code: Managing infrastructure through versioned, machine-readable definitions.

Configuration management: Maintaining desired software and operating-system settings on systems.

Declarative model: A model that specifies the desired result rather than every execution step.

Imperative model: A model that specifies commands or steps to perform.

Idempotence: The property that repeated application produces the same intended result without unnecessary changes.

Convergence: Movement of an actual system toward a defined desired state.

Provider: A plugin that connects an IaC tool to a platform or service API.

Resource: A managed infrastructure object such as a network, server, or database.

Plan: A proposed set of infrastructure changes generated before execution.

State: Data linking configuration objects to real infrastructure and recording attributes.

State locking: A mechanism preventing conflicting concurrent state operations.

Drift: Difference between declared desired state and actual infrastructure.

Module: A reusable collection of infrastructure definitions and interfaces.

Inventory: A list or grouping of systems managed by a configuration tool.

Playbook: An Ansible definition of plays and tasks for target hosts.

Handler: A conditionally triggered action, commonly used to restart a service after change.

Policy as Code: Machine-evaluable rules that enforce governance or security requirements.

Secret manager: A controlled system for storing, retrieving, and rotating sensitive values.

Least privilege: Granting only the access required for a task.

GitOps: An operational model in which version-controlled desired state drives deployment and reconciliation.

Review Questions

Short Answer Questions

1. What is the difference between provisioning and configuration management?
2. Why is state important in Terraform-style IaC?
3. Define idempotence and explain why it matters.
4. What is infrastructure drift?
5. What purpose does an Ansible handler serve?

Long Answer Questions

1. Explain the plan-and-apply workflow and identify the controls that should occur before production application.
2. Compare declarative IaC with imperative scripting, including benefits and limitations of each.
3. Describe how modules, version control, and policy checks improve infrastructure engineering.
4. Explain the security risks associated with state files and secrets, and propose mitigations.
5. Discuss how Terraform and Ansible can be combined while avoiding conflicting ownership.

Analytical / Application Questions

1. A plan proposes replacing a production database after a small configuration change. What should the engineering team investigate before approving it?
2. A manually opened firewall port repeatedly disappears after nightly automation. Analyze the likely cause and propose a governance solution.
3. Design a high-level IaC workflow for a regulated application requiring separate development, staging, and production environments, including state, approvals, testing, secrets, and drift detection.

Multiple Choice Questions with Answers

Question 1

Which statement best describes declarative infrastructure as code?

A. It specifies the desired end state and lets the tool determine required actions.

B. It requires an administrator to execute every command manually.

C. It stores infrastructure only in a binary database.

D. It prevents dependencies between resources.

Correct Answer: A. It specifies the desired end state and lets the tool determine required actions.

Question 2

What is the primary purpose of Terraform state?

A. To replace all source-code repositories.

B. To link configuration objects with real infrastructure objects and record attributes.

C. To encrypt every secret automatically.

D. To serve as an operating-system kernel.

Correct Answer: B. To link configuration objects with real infrastructure objects and record attributes.

Question 3

Which practice helps prevent two pipelines from changing the same Terraform state concurrently?

A. Disabling backups

B. Using longer resource names

C. State locking

D. Removing provider constraints

Correct Answer: C. State locking

Question 4

What is configuration drift?

A. A difference between desired configuration and actual infrastructure

B. A planned module release

C. A syntax error in YAML only

D. A provider installation process

Correct Answer: A. A difference between desired configuration and actual infrastructure

Question 5

Which Ansible feature is commonly used to restart a service only after a relevant configuration change?

A. Inventory

B. Handler

C. Fact

D. Provider

Correct Answer: B. Handler

Question 6

Which is the safest general approach for production secrets?

A. Commit them to a private repository

B. Print them in pipeline logs for auditing

C. Store and retrieve them through an approved secret manager

D. Place them in public module outputs

Correct Answer: C. Store and retrieve them through an approved secret manager

Question 7

What is a principal benefit of a reusable IaC module?

A. It eliminates the need for testing.

B. It provides a versioned and consistent interface for common infrastructure patterns.

C. It guarantees that all resources are public.

D. It prevents all provider API failures.

Correct Answer: B. It provides a versioned and consistent interface for common infrastructure patterns.

Question 8

A policy check requires all production storage to be encrypted. This is an example of:

A. Policy as Code

B. State destruction

C. Imperative deployment only

D. Manual drift creation

Correct Answer: A. Policy as Code

Question 9

Which division of responsibility is most appropriate in a common Terraform-Ansible workflow?

A. Terraform configures every application file and Ansible creates all cloud accounts.

B. Terraform provisions platform resources and Ansible configures hosts.

C. Both tools independently manage the same firewall rules.

D. Neither tool uses version control.

Correct Answer: B. Terraform provisions platform resources and Ansible configures hosts.

Question 10

Why should a destructive change shown in an infrastructure plan receive careful review?

A. It may replace or delete a resource and cause data loss or service interruption.

B. Plans cannot contain resource changes.

C. Destruction always improves availability.

D. Review disables state locking.

Correct Answer: A. It may replace or delete a resource and cause data loss or service interruption.