Database Administration, Performance and Reliability

Introduction and Context

A database management system (DBMS) is not reliable merely because its schema is logically correct or its queries return valid results. In production, a database must remain available, respond within acceptable limits, protect data from loss and unauthorized change, and recover predictably from failure. These responsibilities belong to database administration and operations.

Database administration connects database theory with operational engineering. It includes schema deployment, configuration management, monitoring, capacity planning, workload analysis, performance tuning, backup and restoration, replication, and high-availability design. It also requires explicit service levels and documented procedures. For AI and machine-learning systems, these concerns are especially important because training datasets, feature stores, experiment metadata, model registries, and inference records often depend on databases with demanding throughput and availability requirements.

A useful operational principle is that performance, reliability, security, and cost are interdependent. Increasing redundancy may improve availability but increase cost and write latency. Aggressive caching may reduce response time but create consistency risks. Removing indexes may improve insert performance but harm analytical queries. Good administration therefore involves measurement, trade-off analysis, controlled change, and continuous improvement.

Learning Objectives

After completing this chapter, students should be able to:

1. Explain the responsibilities and operating model of database administration.

2. Deploy schema changes safely across development, testing, and production environments.

3. Interpret major performance metrics and analyze database workloads.

4. Perform basic capacity planning and reason about scaling options.

5. Apply practical query, index, configuration, and data-design tuning techniques.

6. Design backup, restoration, replication, and high-availability strategies.

7. Define service-level indicators, service-level objectives, recovery point objectives, and recovery time objectives.

8. Evaluate operational, governance, ethical, and regulatory implications of database management.

1. The Operational Role of Database Administration

A database administrator (DBA) is responsible for maintaining the database service throughout its lifecycle. In modern organizations, this role may be distributed among DBAs, database reliability engineers, cloud engineers, security specialists, and development teams. Typical responsibilities include:

• provisioning database instances and storage;
• managing users, roles, privileges, and encryption;
• designing and deploying schemas;
• monitoring availability, latency, resource utilization, and errors;
• tuning queries, indexes, transactions, and configuration;
• implementing backup, restore, replication, and disaster recovery;
• planning capacity and lifecycle changes;
• documenting operational procedures and audit evidence.

The database should be treated as a production service, not as an isolated software component. A service-management perspective asks four questions:

1. What behavior do users and applications require?

2. How will that behavior be measured?

3. What actions occur when performance or availability deteriorates?

4. How can the organization recover when prevention fails?

Environments and change control

Most organizations separate development, testing, staging, and production environments. Development supports experimentation; testing validates functionality and performance; staging approximates production; production serves real users. Direct manual editing of production schemas is risky because it is difficult to reproduce, review, or roll back.

A controlled change normally includes a versioned migration script, an explanation of its purpose, dependency information, test evidence, an execution plan, a rollback or forward-fix plan, and an owner. Database migrations should be idempotent where practical: rerunning a migration should not create an incorrect duplicate state. Destructive operations, such as dropping a column, require particular care. A common safe pattern is to add a replacement structure, deploy application code that supports both old and new forms, backfill data, verify correctness, and only then remove the obsolete structure.

Schema deployment must account for locking and transaction duration. An operation that is logically simple may block reads or writes while it changes metadata or rewrites a table. Large data transformations should therefore be chunked, scheduled during suitable periods, and monitored. Online or concurrent index-building features can reduce blocking, but they do not eliminate resource consumption or failure risk.

2. Service Levels and Reliability Objectives

Operational requirements become useful only when they are measurable. The following terms are foundational:

• Service-level indicator (SLI): an observed measurement, such as successful request ratio or query latency.
• Service-level objective (SLO): a target for an SLI, such as 99.9% monthly availability.
• Service-level agreement (SLA): a formal commitment, often including consequences if objectives are missed.
• Recovery point objective (RPO): the maximum acceptable amount of data loss measured in time.
• Recovery time objective (RTO): the maximum acceptable time to restore service after an incident.

RPO and RTO describe different dimensions. An RPO of 15 minutes means that, after a severe failure, the organization can tolerate losing up to approximately 15 minutes of committed data. An RTO of two hours means service should be restored within two hours. Neither objective specifies how the system achieves it; implementation may involve backups, transaction-log shipping, synchronous replication, or a combination.

Availability is often represented as:

Availability = uptime / (uptime + downtime)

The quantities must be measured over the same period. If a service is unavailable for 43.2 minutes in a 30-day month, its availability is approximately 99.9%, because a 30-day month contains 43,200 minutes and 43.2/43,200 = 0.001. Availability percentages should not be interpreted without defining exclusions, maintenance policies, measurement points, and dependency failures.

Reliability engineering also distinguishes between failure prevention, failure detection, recovery, and learning. Monitoring can detect a failed replica, but only tested procedures ensure that recovery is successful. A backup that has never been restored is an assumption, not evidence.

3. Monitoring and Observability

Monitoring collects measurements that indicate whether the database is healthy. Observability goes further by helping engineers explain why a condition exists. A practical database observability system combines metrics, logs, traces, query statistics, and events.

Important metrics

Availability metrics include connection success, failed transactions, replica health, and failover events. Latency metrics include query execution time, transaction duration, lock-wait time, and connection acquisition time. Percentiles are often more informative than averages. The 95th percentile latency is the value below which 95% of observations fall; it reveals tail behavior that an average may conceal.

Throughput metrics include transactions per second, queries per second, rows processed, and bytes read or written. Resource metrics include CPU utilization, memory pressure, buffer-cache effectiveness, storage capacity, input/output operations per second, throughput, and network utilization. Database-specific metrics include active connections, deadlocks, lock waits, temporary-table creation, checkpoint activity, replication lag, and transaction-log growth.

A metric should have a defined unit, collection interval, aggregation method, and interpretation. For example, “CPU = 80%” is incomplete without knowing whether it is a five-minute average, a peak, or a per-core measurement.

Logs, traces, and alerts

Database logs can reveal authentication failures, deadlocks, long-running queries, checkpoints, replication errors, and crash recovery. Logs should be structured and correlated with application request identifiers where possible. Distributed tracing can connect a slow user request to a database span, showing whether delay arose from connection acquisition, lock waiting, execution, or network transfer.

Alerts should be actionable. An alert should identify the condition, severity, affected service, likely impact, owner, and immediate runbook. Alerting on every temporary spike creates fatigue; alerting only after users complain creates delayed detection. Thresholds should be supplemented with baselines and rate-of-change signals. For example, rapid transaction-log growth may be more urgent than a high but stable storage percentage.

4. Workload Analysis and Performance Models

Performance tuning begins with workload characterization rather than guesswork. Important questions include:

• Which queries consume the most total time?
• Which queries have the highest frequency?
• Which operations are latency-sensitive?
• Are delays caused by CPU, memory, storage, locks, network, or poor plans?
• Does the workload consist mainly of short transactions, analytical scans, or mixed activity?
• How does behavior change during peak periods?

A useful prioritization measure is total resource consumption. A query executed 100,000 times at 20 milliseconds may consume more aggregate time than a query executed once at 30 seconds. Both may matter, but they require different interventions.

Response time and queueing intuition

A request’s response time can be decomposed approximately as:

Response time = service time + waiting time + communication time

Service time is time spent executing; waiting time includes lock, CPU, I/O, or connection-pool queues; communication time includes network transfer. The equation is a diagnostic model, not an exact accounting identity in every DBMS, because activities can overlap.

As utilization approaches saturation, queues often grow rapidly. A simple utilization relationship is:

Utilization = demand / capacity

Demand and capacity must use compatible units, such as CPU-seconds per second. If a workload demands 70 CPU-seconds per second from a system with 100 CPU-seconds per second of effective capacity, utilization is 0.70, or 70%. This does not guarantee acceptable latency: lock contention, skew, bursts, and serial portions may dominate.

Execution plans

An execution plan describes how the DBMS intends to access and combine data. Administrators inspect whether predicates use suitable indexes, whether estimated and actual row counts differ substantially, whether joins use appropriate algorithms, and whether sorting or temporary materialization is excessive.

A large difference between estimated and actual cardinality can cause a poor plan. Possible causes include stale statistics, correlated columns, data skew, implicit type conversion, or predicates that the optimizer cannot estimate well. Updating statistics may help, but it should be tested because a new plan can improve one parameter range and worsen another. Parameter-sensitive workloads sometimes require query rewriting, specialized plans, or workload separation.

5. Capacity Planning

Capacity planning anticipates when current resources will no longer satisfy demand. It should consider data volume, transaction rate, concurrency, retention, growth, peak behavior, redundancy, and recovery requirements.

A basic storage estimate is:

Required storage = current data + projected growth + indexes + logs + temporary space + backup or replica overhead

These components should be measured separately. A database with 500 GB of table data may require substantially more physical capacity after indexes, free space, transaction logs, snapshots, and retained backups are included.

For linear growth, a simple projection is:

Future size = current size + (growth per period × number of periods)

This assumes growth is approximately constant. It is unsuitable when growth is seasonal, accelerating, or driven by a planned product change. Capacity plans should include a safety margin and a review trigger, such as reaching a defined percentage of usable storage or a sustained latency objective breach.

Scaling choices include:

• Vertical scaling: adding CPU, memory, or faster storage to one system.
• Read scaling: distributing read queries to replicas, subject to replication lag and consistency requirements.
• Partitioning: dividing data within a logical database according to a key or range.
• Sharding: distributing data across independent database nodes.
• Archiving: moving cold data to lower-cost storage while preserving required access and governance.
• Workload separation: isolating transactional, reporting, batch, or machine-learning workloads.

Sharding can increase capacity but also adds routing, rebalancing, cross-shard transaction, and operational complexity. It should not be selected simply because a single query is poorly indexed.

6. Performance Tuning

Tuning should follow a disciplined loop: establish a baseline, identify the dominant bottleneck, change one important factor, test under representative workload, compare results, and document the outcome. Optimizing without a baseline can improve an irrelevant metric or create regressions elsewhere.

Query and schema tuning

Common query improvements include selecting only required columns, filtering early, avoiding unnecessary repeated computation, using appropriate join predicates, and replacing inefficient procedural loops with set-based operations where suitable. However, “set-based” is not a universal guarantee of better performance; execution plans and data characteristics remain decisive.

Indexes accelerate selected access paths by maintaining additional structures, but they consume storage and impose update cost. An index is useful when it supports frequent selective predicates, joins, ordering, or covering access. Low-selectivity columns may still benefit from composite or specialized indexes depending on the DBMS and workload. Index order matters for composite indexes because leading columns commonly determine which predicates can use the structure effectively.

Over-indexing can slow inserts, updates, and deletes. Duplicate and unused indexes should be identified using workload evidence, not removed solely because they appear similar. Index maintenance also has operational cost and may require scheduling.

Normalization reduces update anomalies and redundancy. Denormalization can reduce joins and improve read performance, but introduces duplication and consistency obligations. A denormalized design should define the authoritative source, update mechanism, reconciliation procedure, and acceptable staleness.

Transactions, locks, and concurrency

Transactions provide atomicity, consistency, isolation, and durability. Isolation mechanisms differ among DBMSs, but administrators must understand dirty reads, non-repeatable reads, phantom rows, blocking, and deadlocks. Stronger isolation may improve consistency while reducing concurrency or increasing version-storage requirements.

Deadlocks occur when transactions hold resources while waiting for resources held by one another. They can be reduced through consistent access ordering, short transactions, appropriate indexes, and avoiding unnecessary user interaction inside a transaction. Applications should usually be prepared to retry safely after a deadlock or transient serialization failure, using idempotent logic where possible.

Connection pools prevent applications from creating unbounded connections, but a pool that is too large can overload the database. Pool sizing must consider database capacity, number of application instances, query duration, and workload bursts.

Configuration tuning

Configuration parameters may govern memory allocation, worker processes, checkpoints, log buffers, parallelism, connection limits, and optimizer behavior. Changing parameters without understanding their units, scope, restart requirements, and interactions is hazardous. A configuration change should be versioned, tested, monitored, and reversible.

7. Backup, Restoration, and Disaster Recovery

A backup is a recoverable copy or representation of data. Common forms include:

• Full backup: captures the complete selected database state.
• Incremental backup: captures changes since a previous backup, according to the product’s definition.
• Differential backup: captures changes since a reference full backup.
• Logical backup: exports schemas and data in logical form.
• Physical backup: copies database storage structures.
• Transaction-log or archive-log backup: preserves changes needed for point-in-time recovery.

A backup strategy must answer what is protected, how frequently it is captured, where it is stored, how long it is retained, how it is encrypted, and how restoration is verified. The 3-2-1 principle is a useful baseline: maintain at least three copies, on at least two types of storage or media, with at least one copy geographically or logically separate. It is a guideline rather than a substitute for threat analysis.

Point-in-time recovery generally requires a valid base backup plus a continuous, usable sequence of log records up to the target time. Recovery may fail because of missing logs, corruption, incompatible versions, unavailable keys, or undocumented dependencies. Restoration tests should measure both RTO and data loss against RPO.

A recovery runbook should specify authority to declare an incident, communication channels, dependency order, access credentials or secret retrieval, restore commands, validation queries, application redirection, reconciliation, and criteria for returning to normal operation. Backups must be protected from ransomware and insider threats through access control, immutability where appropriate, separation of duties, and monitoring.

8. Replication and High Availability

Replication maintains copies of data on multiple systems. It may be synchronous or asynchronous, and physical or logical. Synchronous replication can reduce acknowledged-data loss between replicas but may increase write latency and make availability depend on communication with the replica. Asynchronous replication usually offers lower write latency and greater geographic flexibility, but replicas can lag and recent acknowledged transactions may be absent after primary failure.

Replication is not the same as backup. An accidental deletion or corrupted transaction may be replicated immediately to every copy. Backups and point-in-time recovery remain necessary.

High availability (HA) is a system property achieved through redundant components, failure detection, failover, recovery, and operational readiness. A common architecture includes a primary database, one or more replicas, a health-check mechanism, a coordination or leader-election system, a connection endpoint that can move to the current primary, and tested promotion procedures.

Failover decisions require care. If the original primary is still able to accept writes but is isolated from the control system, promoting another node can create split-brain behavior. Fencing, leases, quorum, or independent termination mechanisms help ensure that only one node is authoritative. Automatic failover reduces detection time but can amplify faults if health checks are poorly designed. Manual approval may be appropriate for high-risk or ambiguous failures.

HA design should examine failure domains: process, host, rack, availability zone, region, network, storage, identity service, and operator error. Replicas located in the same failure domain do not provide full disaster protection. Read replicas also require explicit consistency expectations; users may observe stale data after a write if reads are routed to a lagging replica.

9. Illustrative Case: An ML Feature-Store Database

Background

An illustrative retail company operates an online recommendation service. Its machine-learning feature store uses a relational database for customer and product features. The service also runs nightly model-training queries and receives frequent online feature updates.

Problem

During promotional events, recommendation latency rises from a normal 80 milliseconds at the 95th percentile to more than 600 milliseconds. Training jobs sometimes exhaust available connections. The database has one primary and one asynchronous replica, but no documented RPO, RTO, or tested restoration procedure.

Analysis

Workload analysis shows that online requests are short and latency-sensitive, whereas training queries scan large historical ranges. The training workload runs on the primary, competing for I/O and buffer cache. Several queries retrieve unnecessary columns, and an index is missing on the leading columns used by the most selective online predicate. Monitoring also shows replication lag during training peaks. Backup logs reveal that full backups exist, but point-in-time recovery has not been tested.

Approach

The team first defines objectives: an online-service SLO of 99.9% monthly availability, an RPO of 15 minutes, and an RTO of one hour. It then:

1. captures query plans and establishes a workload baseline;

2. adds and tests a selective composite index;

3. rewrites online queries to return only required fields;

4. moves training reads to a suitable replica or separate analytical environment;

5. limits training concurrency and adjusts the application connection pool;

6. enables continuous transaction-log protection and stores backups separately;

7. performs a restoration and measures actual recovery time;

8. documents promotion, rollback, and communication procedures;

9. tests replica lag and failover during a controlled exercise.

Outcome

The changes reduce online tail latency and prevent training jobs from exhausting the primary’s connection pool. The organization discovers that its original backup schedule could not meet the new RPO, so log archiving is added. The failover exercise exposes an application defect: one service caches the old database endpoint. Correcting endpoint discovery becomes part of the HA solution.

Learning points

The case demonstrates that performance problems are often workload-interference problems rather than purely query problems. It also shows why a replica is not automatically a disaster-recovery solution, why objectives must be measurable, and why application behavior must be included in database failover tests.

10. Governance, Security, and Ethical Responsibilities

Database administration involves access to personal, financial, health, and behavioral data. Least privilege, role separation, strong authentication, encryption in transit and at rest, secure key management, auditing, and timely revocation are essential. Administrative access should be attributable to named identities and reviewed periodically.

Retention should be justified by legal, contractual, operational, and analytical requirements. Keeping data indefinitely increases breach impact and may conflict with deletion obligations. Test environments should use masked, synthetic, or appropriately de-identified data rather than unrestricted production copies.

AI systems create additional concerns. Feature data may encode sensitive attributes or proxies for protected characteristics. Training-data lineage, correction procedures, access controls, and reproducibility depend on reliable database metadata. Database administrators should support provenance and auditability without exposing confidential information. Automated tuning or cloud scaling tools should be governed because an optimization that reduces cost may unintentionally weaken availability, retention, or security.

11. Contemporary Developments and Best Practices

Cloud-managed databases increasingly provide automated backups, patching, replicas, scaling, and observability. These capabilities reduce routine administration but do not transfer accountability for data classification, access policy, recovery objectives, or application compatibility. Serverless and elastic systems may change cost behavior and require careful control of connection storms.

Database DevOps applies version control, automated testing, infrastructure as code, migration pipelines, policy checks, and progressive deployment to database changes. Database reliability engineering emphasizes error budgets, incident learning, runbooks, and reducing repetitive operational work. HTAP systems, distributed SQL databases, lakehouse architectures, and vector-enabled databases support new workloads, but each introduces trade-offs in consistency, cost, query behavior, and operational complexity.

Recommended practices include:

• measure before tuning;
• use representative workloads and realistic data volumes;
• review and test every production change;
• maintain tested backups independent of replicas;
• define RPO, RTO, SLOs, and ownership;
• monitor tail latency, not only averages;
• document dependencies and failure domains;
• automate repeatable procedures while retaining human approval for risky actions;
• conduct post-incident reviews focused on learning rather than blame;
• rehearse restoration and failover regularly.

Challenges and Limitations

No tuning method guarantees improvement across all workloads. Statistics become stale, data distributions change, and optimizers differ among DBMS products. Benchmarks can mislead when they omit concurrency, cache state, failures, or production-like data. High availability increases complexity and may create new failure modes. Synchronous replication can harm performance across long distances; asynchronous replication cannot guarantee zero data loss. Capacity projections are uncertain when business demand is volatile. Finally, operational controls can fail through human error, credential compromise, software defects, or undocumented dependencies. These limitations reinforce the need for testing, observability, and layered defenses.

Chapter Summary

Database administration is the disciplined operation of a database as a production service. Safe schema deployment requires versioned migrations, testing, controlled locking, and rollback or forward-fix planning. Performance management begins with workload measurement, execution-plan analysis, resource diagnosis, and carefully tested changes. Capacity planning accounts for data, indexes, logs, backups, growth, peaks, and failure-domain requirements.

Reliability requires explicit SLOs, RPOs, and RTOs, supported by monitoring, actionable alerts, documented runbooks, and tested recovery. Backups protect against data loss and corruption, while replication supports availability but does not replace backup. High-availability designs must address failover, split brain, consistency, fencing, and failure domains. Governance requires least privilege, encryption, auditing, appropriate retention, and protection of sensitive data. For AI and ML platforms, workload isolation, lineage, reproducibility, and protection against feature-data misuse are especially important.

Key Terms

Database administrator: Professional responsible for operating, securing, and maintaining databases.

Schema migration: Versioned change to database structures or constraints.

Service-level indicator (SLI): Measured attribute of service behavior.

Service-level objective (SLO): Target value for an SLI.

Recovery point objective (RPO): Maximum acceptable time-equivalent data loss.

Recovery time objective (RTO): Maximum acceptable time to restore service.

Replication: Maintenance of one or more data copies on separate systems.

Replication lag: Delay between a source database change and its application to a replica.

High availability: Capability to continue service through specified failures.

Failover: Transfer of service authority from a failed or impaired component.

Point-in-time recovery: Restoration to a selected time using backups and change records.

Execution plan: DBMS strategy for accessing and processing a query.

Cardinality estimate: Optimizer estimate of rows produced by an operation.

Index: Auxiliary data structure that can accelerate selected access paths.

Deadlock: Cycle of transactions waiting for resources held by one another.

Connection pool: Managed set of reusable database connections.

Capacity planning: Forecasting resources needed for future workload and growth.

Workload isolation: Separating competing workloads to limit interference.

Runbook: Documented operational procedure for routine or incident actions.

Fencing: Mechanism that prevents an old or isolated node from acting as primary.

Review Questions

Short Answer Questions

1. Distinguish an SLI from an SLO.

2. Why is replication not a substitute for backup?

3. What is the purpose of an execution plan?

4. Define RPO and RTO.

5. Give two risks of over-indexing a database.

Long Answer Questions

1. Explain a controlled process for deploying a potentially destructive schema migration.

2. Describe how metrics, logs, traces, and query plans contribute to database observability.

3. Compare vertical scaling, read scaling, partitioning, and sharding.

4. Explain the components of a backup and restoration strategy.

5. Discuss the design considerations for a high-availability database architecture.

Analytical / Application Questions

1. A database has 600 GB of current data and grows by 45 GB per month. Indexes require 25% of table-data volume, while logs and temporary space require an additional 120 GB. Estimate storage needed after six months before any safety margin. State your assumptions.

2. An online feature-store workload shares a primary database with a large nightly analytical scan. Propose a measurement and workload-isolation plan, and identify possible consistency trade-offs.

3. Design an incident response sequence for a primary database failure when the asynchronous replica is five minutes behind and the stated RPO is 15 minutes and RTO is one hour.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes RPO from RTO?

A. RPO measures restoration duration, while RTO measures data volume.

B. RPO specifies acceptable data loss, while RTO specifies acceptable restoration time.

C. RPO measures query latency, while RTO measures replication lag.

D. RPO specifies availability percentage, while RTO specifies backup frequency.

Correct Answer: B. RPO specifies acceptable data loss, while RTO specifies acceptable restoration time.

Question 2

A service is unavailable for 43.2 minutes in a 30-day month. What is its approximate availability?

A. 95.0%

B. 99.0%

C. 99.9%

D. 99.99%

Correct Answer: C. 99.9%

Question 3

Which action most directly helps diagnose a query whose estimated row count differs greatly from its actual row count?

A. Inspect the execution plan and update or investigate statistics.

B. Increase the application connection-pool size.

C. Move all backups to the primary server.

D. Disable transaction logging.

Correct Answer: A. Inspect the execution plan and update or investigate statistics.

Question 4

Why can excessive indexing harm write-heavy workloads?

A. Indexes prevent all concurrent reads.

B. Indexes consume storage and must be maintained during data changes.

C. Indexes always eliminate optimizer statistics.

D. Indexes make backups impossible.

Correct Answer: B. Indexes consume storage and must be maintained during data changes.

Question 5

What is a principal risk of asynchronous replication during primary failure?

A. The replica can never be promoted.

B. Recent acknowledged changes may not yet exist on the replica.

C. The primary cannot execute read queries.

D. All backups are automatically deleted.

Correct Answer: B. Recent acknowledged changes may not yet exist on the replica.

Question 6

Which practice best protects against an accidental deletion replicated to every replica?

A. Use only synchronous replication.

B. Increase the connection-pool size.

C. Maintain backups and point-in-time recovery capability.

D. Remove all database constraints.

Correct Answer: C. Maintain backups and point-in-time recovery capability.

Question 7

A workload demands 70 CPU-seconds per second from a system with 100 CPU-seconds per second of effective capacity. What is utilization?

A. 7%

B. 30%

C. 70%

D. 170%

Correct Answer: C. 70%

Question 8

Which condition is most associated with split-brain behavior during failover?

A. Two nodes believe they are authoritative and both accept writes.

B. A backup is stored in a separate region.

C. A query uses a covering index.

D. A replica is intentionally read-only.

Correct Answer: A. Two nodes believe they are authoritative and both accept writes.

Question 9

Which statement about a database runbook is most accurate?

A. It replaces the need for monitoring.

B. It documents repeatable actions, responsibilities, and validation steps.

C. It should contain only vendor marketing information.

D. It is used exclusively for schema design.

Correct Answer: B. It documents repeatable actions, responsibilities, and validation steps.

Question 10

What is the safest general approach to performance tuning?

A. Change many configuration parameters simultaneously.

B. Optimize based only on average latency.

C. Establish a baseline, change a controlled factor, and measure the result.

D. Remove all indexes before testing.

Correct Answer: C. Establish a baseline, change a controlled factor, and measure the result.