3. Terraform Fundamentals and HCL

Introduction and Context

Infrastructure as Code (IaC) treats infrastructure configuration as software: it is written in files, reviewed, versioned, tested, and executed through automation. Terraform is a widely used declarative IaC tool for provisioning and managing resources across cloud platforms, SaaS products, networks, and private infrastructure. Rather than describing every command required to create a system, a Terraform configuration expresses the desired end state. Terraform then compares that declaration with the known and observed state of the environment and proposes a set of actions to reconcile the two.

This chapter develops the practical and conceptual foundations required to work safely with Terraform. It introduces the Terraform command-line workflow, HashiCorp Configuration Language (HCL), providers, resources, data sources, variables, outputs, expressions, dependencies, state, execution plans, and lifecycle rules. The emphasis is not merely on syntax. Professional Terraform practice requires understanding how configuration, state, provider APIs, collaboration, and operational risk interact.

Terraform is commonly used alongside configuration-management tools such as Ansible. Terraform is generally strongest at provisioning and changing infrastructure objects—networks, instances, clusters, databases, and access policies—whereas Ansible is often used to configure operating systems and applications after those objects exist. The boundary is not absolute, but distinguishing provisioning from configuration helps prevent duplicated responsibilities and unclear ownership.

Learning Objectives

By the end of this chapter, the learner should be able to:

1. Explain Terraform’s declarative model, state mechanism, and provider architecture.

2. Use the principal Terraform CLI commands in a controlled workflow.

3. Construct HCL configurations containing providers, resources, data sources, variables, locals, and outputs.

4. Write expressions involving references, collections, conditionals, functions, and iteration.

5. Distinguish implicit and explicit dependencies and identify when each is appropriate.

6. Interpret Terraform plans, including additions, changes, destructions, and replacements.

7. Apply lifecycle rules and safeguards to reduce accidental outages.

8. Design a reviewable, secure, and maintainable Terraform workflow for professional environments.

Terraform’s Declarative Model

Terraform configuration describes desired infrastructure rather than prescribing a sequence of imperative commands. For example, a configuration can state that a virtual network must contain a subnet and that a compute instance must be attached to that subnet. Terraform determines the order in which provider API operations should occur.

This model depends on three related representations. The configuration is the intended design written by the practitioner. The state records Terraform’s understanding of objects it manages and their relevant attributes. The real infrastructure is the set of objects currently present in the provider or platform. During planning, Terraform evaluates the configuration, refreshes or reads information about managed objects, and calculates a proposed transition from the current state to the desired state.

The state file is therefore not simply a cache. It is an important mapping between configuration addresses and real-world objects. It may contain identifiers, dependency information, and sensitive values. State must be protected, backed up, and normally stored in a controlled remote backend when a team is involved. Editing state directly is an exceptional operation, not a normal configuration technique.

Terraform is idempotent in an operational sense: applying the same configuration, when the environment already matches it, should normally produce no changes. Idempotence is not guaranteed if external systems mutate resources, providers have inconsistent behavior, or configurations use unstable values. Consequently, practitioners must understand both Terraform’s desired state and the behavior of the provider APIs on which it depends.

The Terraform CLI Workflow

A normal workflow begins by creating a working directory containing one or more `.tf` files. Terraform treats all configuration files in a directory as one module; filenames are primarily a way to organize content for readers.

`terraform init` initializes the directory. It downloads required providers and modules, configures the backend, and creates or updates dependency lock information. Initialization should be repeated when provider requirements, module sources, or backend configuration changes. The dependency lock file should generally be committed so that a team uses reviewed provider versions rather than resolving different versions opportunistically.

`terraform fmt` formats Terraform files according to standard conventions. Formatting is not merely cosmetic: consistent layout makes review easier and reduces noise in version-control changes. `terraform validate` checks whether the configuration is syntactically valid and internally consistent. Validation does not prove that credentials are correct, APIs are available, or the proposed infrastructure is operationally suitable.

`terraform plan` evaluates a proposed change without applying it. A saved plan can be produced with `terraform plan -out=tfplan`; the exact saved plan can later be supplied to `terraform apply tfplan`. In controlled pipelines, this separation allows one identity or process to create a plan and another authorized process to approve and apply that exact proposal. Plans should be reviewed for unexpected replacements, deletions, public exposure, privilege changes, and cost implications.

`terraform apply` executes a plan. Without a saved plan, Terraform normally creates a new plan and requests confirmation in interactive use. In automation, approval is commonly represented by an approved pipeline rather than an unrestricted command such as `terraform apply -auto-approve`.

`terraform destroy` proposes removal of objects managed by the configuration. It is powerful and should be restricted in production environments. Other useful commands include `terraform show` for displaying a plan or state representation, `terraform output` for reading declared outputs, `terraform state list` for listing state addresses, and `terraform import` for associating existing infrastructure with a resource address. Import does not automatically create a complete configuration; configuration must still be written and reconciled.

HCL as a Configuration Language

HashiCorp Configuration Language is a declarative language designed to be readable by humans and processed by tools. A block has a type, optional labels, and a body of arguments or nested blocks. For example:

```hcl

resource "aws_s3_bucket" "logs" {

bucket = var.log_bucket_name

tags = {

Environment = var.environment

ManagedBy = "terraform"

}

}

```

Here, `resource` is the block type, `aws_s3_bucket` is the resource type, and `logs` is the local name. The address of the object is `aws_s3_bucket.logs`. The expression `var.log_bucket_name` references an input variable.

HCL supports strings, numbers, booleans, lists, sets, maps, tuples, and objects. Types matter because providers often distinguish between a single value and a collection, or between an ordered list and an unordered set. Explicit type constraints improve module interfaces and make errors easier to detect.

Comments may be written with `#`, `//`, or block-comment syntax. Interpolation is usually implicit in modern HCL: an expression can be assigned directly, as in `name = var.environment`. String templates remain useful when a value combines literal text and expressions, such as `name = "${var.project}-${var.environment}"`.

Providers, Resources, and Data Sources

A provider is a plugin that translates Terraform operations into API calls for a platform or service. Provider configuration commonly includes a region, endpoint, or authentication-related settings, although credentials should normally be supplied through environment variables, workload identity, or a secrets system rather than committed to source code.

A minimal provider requirement may appear as follows:

```hcl

terraform {

required_providers {

aws = {

source = "hashicorp/aws"

version = "~> 5.0"

}

}

}

provider "aws" {

region = var.region

}

```

The `required_providers` block identifies the provider source and acceptable version range. Pinning or constraining versions supports reproducibility, while regular upgrades are necessary for security fixes and new platform capabilities.

A resource represents an object Terraform creates, updates, or deletes. Its type and name form a unique address within a module. Resource attributes may be fixed values, variable references, or expressions. Providers can expose multiple instances through aliases, allowing one configuration to work across regions or accounts:

```hcl

provider "aws" {

alias = "secondary"

region = var.secondary_region

}

resource "aws_vpc" "replica" {

provider = aws.secondary

cidr_block = var.replica_cidr

}

```

A data source reads information that already exists or is calculated externally. It does not normally create the object it describes. For example, a data source can locate an existing image, subnet, account identity, or certificate. The distinction between a resource and a data source is operationally important: confusing them can lead to attempts to manage an object that should remain owned by another team.

```hcl

data "aws_caller_identity" "current" {}

output "account_id" {

value = data.aws_caller_identity.current.account_id

}

```

Variables, Locals, and Outputs

Input variables define a module’s interface. A variable can have a description, type, default, and sensitivity marker:

```hcl

variable "environment" {

type = string

description = "Deployment environment"

validation {

condition = contains(["dev", "stage", "prod"], var.environment)

error_message = "Environment must be dev, stage, or prod."

}

}

variable "instance_count" {

type = number

default = 2

sensitive = false

}

```

Values can be supplied through variable definition files, command-line options, environment variables, or workspace and pipeline mechanisms. Sensitive marking affects display in command output; it is not encryption. Secrets still require secure storage and access control, and sensitive values may remain in state if a resource needs them.

Local values give names to expressions used repeatedly within a module. They improve readability and reduce duplication but do not create independently managed objects:

```hcl

locals {

common_tags = {

Project = var.project

Environment = var.environment

}

}

```

Outputs publish selected values from a module. They are useful for passing information between modules, displaying endpoints, or integrating Terraform with deployment systems. Outputs should be designed carefully because they become part of the module’s public interface. An output containing a password or token should be marked sensitive and should generally be avoided unless there is a compelling integration requirement.

Expressions, Collection Processing, and Modules

Terraform expressions combine values and references. Common operators support arithmetic, comparison, logical operations, and conditional selection:

```hcl

instance_type = var.environment == "prod" ? "m6i.large" : "t3.micro"

```

Functions support transformations such as `merge`, `lookup`, `contains`, `length`, `join`, and `coalesce`. A conditional expression should have compatible result types on both branches. Complex expressions should be decomposed with locals when that improves reviewability.

The `for_each` and `count` meta-arguments create multiple resource instances. `count` is suitable when instances are essentially interchangeable and indexed numerically. `for_each` is usually safer when objects have stable logical keys:

```hcl

resource "aws_iam_user" "team" {

for_each = toset(var.team_members)

name = each.value

}

```

Changing a list used with `count` can shift indexes and cause unwanted replacements. A map used with `for_each` preserves identity by key, provided those keys are stable. The choice is therefore a modeling decision, not merely a syntactic preference.

A module is a reusable collection of Terraform configuration. A root module is the working directory where Terraform is run; child modules are called from it. Modules should expose a small, purposeful interface through variables and outputs. Reuse is valuable, but excessive abstraction can hide provider behavior and make debugging difficult. A module should represent a coherent capability rather than every possible variation of a platform resource.

Dependencies and Graph Evaluation

Terraform builds a dependency graph to determine ordering and parallelism. An implicit dependency occurs when one object references an attribute of another:

```hcl

resource "aws_subnet" "app" {

vpc_id = aws_vpc.main.id

cidr_block = var.app_cidr

}

```

The reference to `aws_vpc.main.id` tells Terraform that the subnet depends on the VPC. Terraform can therefore create the VPC first and destroy the subnet before the VPC.

An explicit dependency uses `depends_on` when the relationship is real but not represented by a value reference:

```hcl

resource "aws_iam_role_policy_attachment" "app" {

role = aws_iam_role.app.name

policy_arn = var.policy_arn

}

resource "aws_instance" "app" {

ami = data.aws_ami.application.id

instance_type = var.instance_type

depends_on = [aws_iam_role_policy_attachment.app]

}

```

Explicit dependencies should be used sparingly. Overuse reduces parallelism and can conceal poor modeling. A dependency should describe a genuine ordering requirement, not simply attempt to force Terraform to behave like a manually scripted sequence.

Reading Plans and Applying Changes Safely

A plan is both a technical proposal and a governance artifact. Terraform commonly marks actions with symbols such as `+` for creation, `~` for in-place update, `-` for destruction, and `-/+` for replacement. Replacement is more disruptive than an in-place update and deserves particular attention. A change may trigger replacement because an attribute is immutable, a resource has a `ForceNew` behavior in the provider schema, or its identity has changed.

Plan review should ask several questions. Is the affected environment correct? Are all changes intentional? Does any update expose a service publicly, broaden permissions, alter encryption, or remove data? Are there changes outside the expected application boundary? Does the plan contain a replacement of a stateful resource? Is the provider version appropriate? Are cost or quota implications understood?

Plans are not absolute guarantees. Between planning and applying, external changes may occur, credentials may expire, provider behavior may differ, or an API may reject a request. For this reason, plan and apply should occur close enough in time to remain meaningful, and production systems should use remote state locking, controlled identities, audit logs, and approval policies.

Lifecycle Rules

Lifecycle meta-arguments refine how Terraform handles a resource. `create_before_destroy = true` requests creation of a replacement before the old object is removed. This can reduce downtime but may fail when names must be unique, capacity is limited, or the platform cannot support simultaneous objects.

`prevent_destroy = true` causes Terraform to reject a plan that would destroy the resource. It is a useful guard for critical databases or storage, but it is not a substitute for backups, access controls, or a recovery plan. It can also obstruct legitimate replacement and must therefore be managed deliberately.

`ignore_changes` tells Terraform to disregard changes to selected attributes. This is appropriate when an external system is explicitly authorized to manage those attributes, such as a platform adding operational metadata. It is dangerous when used to hide drift or silence unexpected changes. Ignoring a field transfers part of its ownership outside Terraform and should be documented.

Modern Terraform also supports preconditions and postconditions for expressing assumptions about resource values. These checks can stop an operation when a security, compliance, or operational invariant is not satisfied. Conditions should be meaningful and understandable; excessive checks can make configurations brittle.

Illustrative Case Study: A Multi-Environment Service Platform

Background

A fictional organization, Northstar Analytics, operates a web service in development, staging, and production. Initially, engineers created virtual networks, security groups, and compute resources manually through a cloud console. The environments drifted: production used stronger network controls, staging contained obsolete rules, and no reliable record explained which team owned particular resources.

Situation and Problem

The platform team needed reproducible environments without allowing every developer to modify production. It also needed to preserve an existing production database while introducing Terraform gradually. A careless first attempt could have treated all existing infrastructure as new and proposed destructive changes.

Analysis

The team separated ownership boundaries. Terraform would manage network foundations, security groups, and compute services. A database team would continue managing the production database, which would be referenced through a data source or carefully imported only if ownership later changed. The team selected stable keys for environment-specific objects and established a remote backend with locking and restricted access.

Approach and Solution

The root module accepted variables for environment, region, CIDR ranges, and instance sizing. Common tags were defined in locals. Network resources used implicit references so Terraform could construct the dependency graph. Application instances used `for_each` with stable service names rather than `count` indexes. Production storage resources received `prevent_destroy = true`, while replacements of stateless compute resources used `create_before_destroy` where the provider and naming constraints allowed it.

The team first ran `fmt`, `validate`, and `plan` in a non-production account. Existing production resources were inventoried and imported only where Terraform would become the authoritative owner. Each import was followed by configuration refinement and a plan expected to show no unintended changes. Pull requests displayed plans for peer review. A protected pipeline applied approved plans using a narrowly scoped identity.

Outcome

Development and staging became reproducible, and the team could identify proposed changes before execution. Production remained protected by ownership boundaries, state locking, approval controls, and lifecycle safeguards. The organization did not eliminate all operational risk, but it converted undocumented manual procedures into a reviewable delivery process.

Learning Points

The central lesson is that Terraform adoption is a design and governance activity as well as a coding activity. Stable resource identity, explicit ownership, carefully protected state, and plan review matter as much as valid HCL. Import is a migration technique, not a replacement for understanding the infrastructure being adopted.

Challenges, Risks, and Professional Considerations

State is a major security and reliability concern. It may contain credentials, private keys, connection strings, or other sensitive attributes even when configuration marks outputs as sensitive. Remote state should use encryption at rest, access control, versioning, auditability, and locking where supported. Backend credentials should not be embedded in source code.

Drift occurs when infrastructure changes outside Terraform. Some drift is legitimate, such as an approved autoscaling action; other drift indicates an unauthorized or undocumented change. Teams should decide whether a system is Terraform-authoritative, externally authoritative, or shared with clearly defined fields. `ignore_changes` should not become a general drift-hiding mechanism.

Provider and module supply chains require attention. Use trusted sources, constrain versions, review upgrades, and maintain the lock file. Provider credentials should follow least privilege. A plan that can modify identity policies or network boundaries should receive stronger review than a change to a development tag.

Common mistakes include committing secrets, using one state file for unrelated environments, applying from an unreviewed laptop, relying on `-auto-approve`, using unstable list indexes for long-lived objects, overusing `depends_on`, and assuming `sensitive = true` encrypts data. Another mistake is treating a successful apply as evidence that the architecture is secure or compliant; Terraform confirms execution, not organizational suitability.

Terraform also raises governance questions. Who may approve production changes? Who owns state backups? How are emergency changes recorded? How are regulated data locations, encryption requirements, retention periods, and segregation of duties enforced? Policy-as-code tools and CI checks can reject public storage, unrestricted ingress, or unapproved regions before apply, but policies must be maintained and tested like software.

Current Developments and Emerging Practice

Contemporary Terraform practice increasingly emphasizes remote execution, policy checks, drift detection, reusable modules, and automated plan review. Infrastructure teams are adopting ephemeral environments for testing, while attempting to control cloud cost and prevent the accumulation of abandoned resources. Provider ecosystems continue to expand, and provider upgrades require compatibility testing rather than blind adoption.

Declarative infrastructure is also being integrated with software supply-chain controls. Module registries, signed artifacts, dependency scanning, static analysis, and identity federation reduce the risks associated with downloading and executing infrastructure plugins. AI-assisted code generation may accelerate the creation of HCL, but generated configurations still require provider-specific validation, security review, cost analysis, and human accountability. An AI system cannot determine organizational ownership or approve a destructive production plan on its own.

Best-Practice Workflow

A disciplined workflow can be summarized as follows:

1. Define ownership, environment boundaries, and backend strategy before writing resources.

2. Pin or constrain provider versions and commit the dependency lock file.

3. Format and validate every change.

4. Use variables for genuine configuration differences and locals for readable reuse.

5. Prefer implicit dependencies through meaningful references.

6. Use stable keys with `for_each` for independently identifiable objects.

7. Generate and review plans, paying special attention to replacements and destruction.

8. Apply through a controlled identity with state locking and approval procedures.

9. Protect state and treat outputs as potentially sensitive.

10. Test modules and policy rules, document exceptions, and review provider upgrades.

Chapter Summary

Terraform provides a declarative method for managing infrastructure through configuration, provider APIs, and state. HCL expresses providers, resources, data sources, variables, locals, outputs, and reusable modules. Terraform’s CLI workflow—initialize, format, validate, plan, review, and apply—supports repeatable and auditable changes. Expressions and iteration make configurations adaptable, while implicit dependencies allow Terraform to construct an execution graph. Lifecycle rules such as `create_before_destroy`, `prevent_destroy`, and `ignore_changes` must be applied with a clear understanding of ownership and operational impact. Safe Terraform practice depends on protected state, constrained credentials, version control, plan review, policy enforcement, and careful treatment of replacements, drift, and sensitive data.

Key Terms

Provider: A Terraform plugin that communicates with an external platform or service.

Resource: A managed infrastructure object that Terraform can create, update, or destroy.

Data source: A read-only configuration construct used to retrieve existing or computed information.

State: Terraform’s record of managed objects and their attributes and relationships.

Backend: The mechanism used to store state and, where supported, coordinate locking.

HCL: HashiCorp Configuration Language used to write Terraform configuration.

Root module: The working directory from which Terraform is executed.

Child module: A reusable module called by another Terraform module.

Input variable: A declared value used to parameterize a module.

Local value: A named expression evaluated within a module.

Output: A value exposed by a module for users or other modules.

Expression: A combination of literals, references, operators, and functions that produces a value.

Implicit dependency: A dependency inferred from a reference between configuration objects.

Explicit dependency: A dependency declared with `depends_on`.

Execution plan: Terraform’s proposed set of changes needed to reconcile configuration and infrastructure.

Drift: A difference between Terraform’s managed understanding and externally changed infrastructure.

Lifecycle rule: A meta-argument controlling resource replacement, destruction, or change handling.

`for_each`: A meta-argument that creates instances identified by stable keys.

Idempotence: The property that repeated application of unchanged desired state produces no additional changes.

Plan replacement: A destroy-and-create operation required when an object cannot be updated in place.

Review Questions

Short Answer Questions

1. What roles do configuration, state, and real infrastructure play in Terraform’s reconciliation model?

2. Distinguish a Terraform resource from a data source.

3. Why is a provider version constraint important?

4. What is the purpose of `terraform plan`?

5. Why should sensitive outputs and state be protected separately?

Long Answer Questions

1. Explain the Terraform CLI workflow from initialization through application, including the role of formatting, validation, and plan review.

2. Discuss how variables, locals, outputs, and modules contribute to reusable Terraform design.

3. Explain implicit and explicit dependencies, including the risks of excessive use of `depends_on`.

4. Compare `count` and `for_each` as mechanisms for managing multiple resource instances.

5. Analyze the purpose and limitations of `create_before_destroy`, `prevent_destroy`, and `ignore_changes`.

Analytical / Application Questions

1. A plan proposes to replace a production database after a variable key changes. How would you investigate the cause and what safeguards would you apply before proceeding?

2. A development team wants to store cloud credentials in a `.tfvars` file committed to its repository. Evaluate this design and propose a safer workflow.

3. An organization has manually created production resources and wishes to adopt Terraform without disruption. Design an adoption strategy using ownership boundaries, import, state protection, and plan review.

22. Multiple Choice Questions with Answers

Question 1

Which statement best describes Terraform state?

A. It is only a temporary cache used during formatting

B. It records Terraform’s mapping between configuration objects and managed infrastructure

C. It is a provider-specific source-code repository

D. It replaces the need for configuration files

Correct Answer: B. It records Terraform’s mapping between configuration objects and managed infrastructure

Question 2

What is the principal purpose of terraform plan?

A. To execute all proposed infrastructure changes immediately

B. To install operating-system packages on managed instances

C. To calculate and display proposed changes before application

D. To delete resources not found in the current directory

Correct Answer: C. To calculate and display proposed changes before application

Question 3

Which construct is most appropriate for reading the identifier of an existing cloud account without creating it?

A. A data source

B. A lifecycle block

C. A count expression

D. A local-only resource

Correct Answer: A. A data source

Question 4

Why is for_each often preferable to count for long-lived objects with meaningful identities?

A. It prevents Terraform from creating more than one object

B. It identifies instances using stable keys rather than shifting numeric indexes

C. It automatically encrypts all resource attributes

D. It removes the need for a provider

Correct Answer: B. It identifies instances using stable keys rather than shifting numeric indexes

Question 5

A subnet references aws_vpc.main.id in its vpc_id argument. What does this normally create?

A. An implicit dependency on the VPC

B. A backend lock for the subnet

C. A request to ignore VPC changes

D. A data source for the subnet

Correct Answer: A. An implicit dependency on the VPC

Question 6

What is the principal risk of using ignore_changes indiscriminately?

A. Terraform will always destroy the resource

B. It can conceal drift or changes that should have been reviewed

C. It forces every resource to use count

D. It prevents state from being stored

Correct Answer: B. It can conceal drift or changes that should have been reviewed

Question 7

A plan displays `-/+` for a resource. What does this most commonly indicate?

A. A formatting-only change

B. An in-place update with no interruption

C. A replacement involving destruction and creation

D. A provider initialization failure

Correct Answer: C. A replacement involving destruction and creation

Question 8

Which practice provides the strongest protection for collaborative production Terraform?

A. Applying from any laptop with auto-approval enabled

B. Committing credentials alongside configuration

C. Using protected remote state, locking, reviewed plans, and controlled identities

D. Keeping all environments in one unversioned local state file

Correct Answer: C. Using protected remote state, locking, reviewed plans, and controlled identities

Question 9

What does sensitive = true on a variable or output primarily do?

A. It guarantees that the value is encrypted everywhere

B. It prevents the provider from receiving the value

C. It reduces exposure in normal command output but does not replace secure state handling

D. It makes the value immutable

Correct Answer: C. It reduces exposure in normal command output but does not replace secure state handling

Question 10

A team wants a replacement resource created before the old resource is removed. Which lifecycle setting addresses this intention, subject to platform constraints?

A. prevent_destroy = true

B. create_before_destroy = true

C. ignore_changes = all

D. depends_on = false

Correct Answer: B. create_before_destroy = true