Audit Sampling and Data Analytics

Introduction

Auditors rarely examine every transaction or account balance manually. Modern audits therefore depend on two complementary approaches: carefully designed sampling and the analysis of complete populations using information technology. Audit sampling provides a defensible basis for conclusions about a population when less than 100% of items are tested. Data analytics can examine entire data populations, identify unusual relationships, and direct professional attention toward higher-risk items.

Neither approach removes the need for professional judgment. A statistically valid sample can produce a misleading conclusion if the population is incomplete, the sampling unit is inappropriate, or exceptions are not interpreted correctly. Similarly, a sophisticated analytical dashboard is not audit evidence merely because it is visually persuasive. The auditor must understand the objective, population, controls, data quality, risks, and evidence obtained.

This chapter explains statistical and non-statistical sampling, sample-size determination, evaluation and projection of errors, analytical procedures, visualisation, full-population testing, and the professional implications of artificial intelligence and automated audit tools.

Learning Objectives

By the end of this chapter, students should be able to:

1. Explain the purpose, terminology, and stages of audit sampling.
2. Distinguish statistical from non-statistical sampling and select appropriate methods.
3. Determine and interpret sample sizes using risk, materiality, expected error, and population characteristics.
4. Evaluate sample exceptions and project misstatements to a population.
5. Apply analytical procedures, visualisation, and full-population testing to identify anomalies.
6. Assess the reliability, limitations, governance, and ethical implications of audit data analytics.

1. The Audit Population and Sampling Risk

An audit population is the complete set of data from which a sample is selected and about which the auditor wishes to draw a conclusion. Examples include all sales invoices recorded during a year, all trade receivable balances at year-end, or all journal entries posted by users with elevated access rights.

A population must be relevant, complete, and appropriately defined. If the audit objective concerns unrecorded liabilities, the population of recorded purchases alone is insufficient; the auditor may need to examine subsequent cash disbursements, unmatched receiving reports, or supplier statements.

Important terms include:

• Sampling unit: The individual item selected, such as an invoice, account balance, or journal entry.
• Tolerable misstatement: The maximum misstatement the auditor is willing to accept in a population without changing the planned conclusion.
• Tolerable deviation rate: The maximum rate of control deviations acceptable to the auditor.
• Expected misstatement or deviation: The amount or rate the auditor expects to find, based on prior audits, preliminary testing, or knowledge of the client.
• Sampling risk: The risk that a conclusion based on a sample differs from the conclusion that would have been reached by examining the entire population.
• Non-sampling risk: The risk of reaching an incorrect conclusion for reasons unrelated to sampling, such as misunderstanding an audit procedure, using unreliable evidence, or failing to recognise an exception.

Sampling risk has two important forms. In tests of controls, the auditor may conclude that controls are effective when they are not, or conclude that they are ineffective when they are effective. In substantive testing, the auditor may conclude that a material misstatement does not exist when it does, or investigate an apparent misstatement that is not material to the financial statements. The first type in each pair generally creates the greater threat to audit effectiveness.

2. Statistical and Non-Statistical Sampling

Statistical sampling uses random selection and probability theory to measure sampling risk and quantify sampling precision. Non-statistical sampling uses auditor judgment to select items and evaluate results, without formally measuring sampling risk. Both can be appropriate when properly designed and performed.

A non-statistical sample is not automatically a small or careless sample. The auditor should still define the population, objective, selection method, sample size rationale, testing procedure, and evaluation criteria. Conversely, statistical sampling does not guarantee representativeness if the population is incomplete or the randomisation process is defective.

Common selection methods

Random selection gives every sampling unit a known chance of selection, commonly using a random-number generator.

Systematic selection selects every kth item after a randomly chosen starting point. If the population contains N items and the desired sample size is n, the interval is:

Here, N is the population size and n is the sample size. Systematic selection is appropriate only when periodic patterns in the population will not bias the sample.

Haphazard selection attempts to select items without conscious bias but does not provide a measurable probability of selection. It is less defensible than random selection when statistical conclusions are required.

Stratified selection divides a population into groups, or strata, and applies different procedures or sampling rates to them. High-value items may be tested 100%, while the remaining items are sampled. Stratification can improve efficiency because items within each stratum are more homogeneous.

Block selection examines a contiguous group of items. It may be useful for understanding a process but is generally weak for projecting results across a population if transactions are ordered chronologically or by location.

3. Designing an Audit Sample

Effective sampling follows a sequence rather than beginning with an arbitrary number of items.

Step 1: Define the audit objective

The objective determines the population, sampling unit, test, and evaluation. For example, a control test may ask whether each payment above a threshold was independently approved. A substantive test may ask whether recorded receivables exist and are correctly valued.

Step 2: Define the population and completeness

The auditor should reconcile the population to the accounting records and investigate missing, duplicated, or excluded items. The period, transaction type, monetary field, and system source should be documented.

Step 3: Specify the misstatement or deviation

An exception must be defined before testing. In a control test, a missing approval is usually a deviation. In a substantive test, the difference between the recorded amount and the audited amount may be a misstatement. The auditor should distinguish factual errors, projection differences, and clearly trivial differences.

Step 4: Set risk and tolerable error parameters

Sample size generally increases when:

• the risk of material misstatement or control failure increases;
• the auditor requires lower sampling risk;
• tolerable misstatement or deviation decreases;
• expected misstatement or deviation increases; or
• the population is more variable.

Population size often has a smaller effect than these factors, especially in large populations. Once a population is sufficiently large, adding more items does not increase sample size proportionally.

Step 5: Select and test the sample

All selected items should be tested using the planned procedure. Replacements should not be made merely because an item is inconvenient or difficult to audit. If an item is genuinely not applicable, the auditor should document the reason and follow the relevant sampling design.

Step 6: Evaluate results and document the conclusion

The auditor evaluates the nature and cause of exceptions, considers whether they are isolated or systematic, projects identified misstatements when appropriate, and determines whether additional work is necessary.

4. Sample Size and Statistical Reasoning

There is no universal sample-size formula for all audit situations. The correct method depends on whether the auditor is testing a rate of deviation, a monetary amount, or a difference between recorded and audited values.

Attribute sampling for controls

Attribute sampling focuses on whether a specified attribute is present. A simple approximate planning formula for a large population is:

where:

• n = approximate sample size;
• z = the standard-normal critical value associated with the desired confidence level;
• p = anticipated population deviation rate; and
• e = the desired absolute precision, expressed as a proportion.

This formula assumes a simple random sample, a sufficiently large population, independent observations, and a proportion-based objective. It is an approximation and should not replace professional sampling guidance or specialised audit software.

For a 95% confidence level, z is approximately 1.96. Suppose an auditor expects a deviation rate of 2% and wants precision of 3 percentage points:

The auditor would normally round upward to 84 items, subject to finite-population considerations, the audit objective, and the firm’s methodology. If the expected rate rises or the desired precision becomes narrower, the sample increases.

Finite-population adjustment

When the sample is a significant fraction of a small population, an approximate finite-population adjustment is:

where n is the initial sample size and N is the population size. This adjustment assumes the initial calculation was based on a large-population approximation and that sampling is without replacement.

Monetary-unit and substantive sampling

Monetary-unit sampling gives greater selection emphasis to items with larger recorded monetary amounts. It is useful when overstatement is a principal concern because each monetary unit has a chance of selection. Large or individually significant items may be tested separately.

Substantive sample sizes are influenced by materiality, assessed risk, expected misstatement, the strength of other audit evidence, and population variability. A simple normal-theory formula for estimating a mean or difference is:

where s is the estimated standard deviation of item-level differences and E is the desired margin of error in monetary units. This formula assumes an approximately stable population and a random sample. It is not appropriate without modification for highly skewed populations, unusual-value populations, or monetary-unit sampling.

Auditors frequently use specialised methods such as classical variables sampling, probability-proportional-to-size sampling, or exact binomial and hypergeometric calculations. The important principle is that sample design must match the audit assertion and error model.

5. Evaluating Exceptions and Projecting Misstatements

Finding an exception does not automatically establish that the financial statements are materially misstated. The auditor must determine the nature, cause, amount, and possible pervasiveness of the error.

For a simple random sample in which misstatements are proportionally projected, the basic projection is:

This ratio projection is suitable only when the sample is designed to support ratio projection and the items tested are reasonably representative. If the sample contains a few large items or the error pattern is unusual, a different evaluation may be necessary.

Worked example: ratio projection

A population contains recorded sales of $2,000,000. A substantive sample tests $100,000 of sales and identifies total factual misstatements of $3,000. The projected misstatement is:

If tolerable misstatement is $80,000, the point estimate alone is below tolerable misstatement. However, the auditor must also consider sampling risk, the possibility of additional misstatement, qualitative factors, and whether the errors indicate a control deficiency or fraud risk. A conclusion should not be based solely on comparing $60,000 with $80,000.

For controls, the auditor compares the observed deviation rate with the tolerable deviation rate, allowing for sampling risk. An observed rate below the tolerable rate may still be unacceptable if the confidence bound exceeds the tolerable rate. Statistical software or audit tables are commonly used to calculate an upper deviation rate.

Errors should be classified carefully:

• Known misstatement: A misstatement established by audit evidence.
• Projected misstatement: An estimate of misstatement in the population based on sample results.
• Judgmental misstatement: A difference arising from an accounting estimate or accounting policy judgment.
• Extrapolated misstatement: A projected amount extended from tested items to untested items.

The auditor should aggregate identified and projected misstatements with other audit differences and compare the total with materiality. The auditor should also communicate control deficiencies and fraud indicators even when the monetary amount is not material.

6. Audit Analytical Procedures

Analytical procedures involve evaluating financial and non-financial information through analysis of plausible relationships, trends, ratios, and expectations. They may be used during risk assessment, as substantive procedures, and near the end of the audit as an overall review.

Examples include:

• comparing current revenue with prior periods and approved budgets;
• analysing gross-margin movements by product or location;
• comparing payroll expense with headcount and average salary;
• examining days sales outstanding and ageing patterns;
• reconciling units shipped with revenue and price lists; and
• identifying unexpected manual journal entries near period end.

A useful analytical procedure begins with an independently developed expectation. The auditor then defines a threshold for investigation based on materiality, assessed risk, and the precision of the expectation. The more precise the expectation and the higher the risk, the smaller the acceptable unexplained difference should be.

For example, if 10,000 units were shipped at an expected price of $25, expected revenue is:

If recorded revenue is $270,000, the unexplained difference is $20,000. The auditor must investigate whether the difference arises from price changes, returns, timing, discounts, multiple delivery obligations, or an error. A difference is an investigation trigger, not proof of misstatement.

Analytical procedures are stronger when data is reliable, relationships are predictable, expectations are precise, and the auditor has appropriate disaggregated information. They are weaker when relationships are volatile, management estimates are significant, or the data has not been validated.

7. Data Analytics and Visualisation

Audit data analytics refers to the use of technology to inspect, transform, model, and visualise data for audit purposes. Common techniques include duplicate testing, gap analysis, Benford-style screening, stratification, outlier detection, trend analysis, network analysis, and rule-based exception testing.

A typical analytics workflow is:

1. Define the question: For example, “Are there payments split below the approval threshold?”
2. Acquire the data: Obtain relevant ledger, subledger, master-file, and access information.
3. Validate the data: Reconcile record counts and monetary totals to authoritative reports.
4. Clean and transform: Standardise dates, remove technical duplicates, and document transformations.
5. Analyse: Apply rules, statistical tests, visualisation, or machine-learning techniques.
6. Investigate exceptions: Trace unusual items to source documents and explanations.
7. Conclude and retain evidence: Preserve scripts, parameters, outputs, and review evidence.

Visualisations can reveal patterns that tables conceal. A time-series chart may show unusual year-end spikes; a scatterplot may reveal transactions inconsistent with normal price-volume relationships; and a heat map may show that exceptions are concentrated in one branch or user group. A visualisation supports understanding but does not replace vouching, confirmation, reperformance, or other appropriate procedures.

Figure: A controlled audit analytics workflow.

Full-population testing

Full-population testing examines every item against a defined rule. Examples include testing all journal entries posted on weekends, all payments exceeding a threshold, or all supplier bank-account changes. It is particularly useful when data is structured, the test is repeatable, and the risk is concentrated in identifiable characteristics.

Full-population testing eliminates sampling risk for the specific rule tested, but it does not eliminate non-sampling risk. It may fail to identify errors that do not match the rule, errors in fields excluded from the analysis, or transactions omitted from the data extract. Therefore, full-population testing is not equivalent to a complete audit of every assertion.

Anomaly detection and AI

Machine-learning tools can identify clusters, unusual sequences, or transactions unlike historical patterns. However, an anomaly is not synonymous with an error or fraud. Models may reproduce historical bias, generate false positives, or be difficult to explain. Auditors should document the model’s purpose, training data, variables, thresholds, version, validation, and human review. Confidential client data must be protected, and external generative-AI tools should not receive sensitive information without appropriate authorisation and controls.

8. Illustrative Case: Northbridge Retail Ltd.

Background

Northbridge Retail Ltd. operates 45 stores and an online channel. Sales returns increased sharply in December, while gross margin fell from 31% to 26%. Management attributes the change to promotional pricing and holiday returns.

Problem

The audit team must assess revenue occurrence, sales returns, cut-off, and inventory valuation. The sales system contains 2.4 million transaction lines, making manual examination of every line impractical.

Analysis

The team first reconciles the extracted sales population to the general ledger and obtains a separate returns file. Analytics identify: (1) a concentration of returns within three days after year-end; (2) sales invoices issued at prices below approved promotional prices; (3) multiple refunds to the same bank account; and (4) manual journals posted by two users after the reporting date but dated before year-end.

The team stratifies the population. Individually significant transactions are tested 100%. The remaining transactions are selected using random sampling. A substantive sample identifies errors of $18,000 in a tested sales amount of $600,000. The related population is $12,000,000.

The projected error is $360,000. The audit team also investigates the unusual refunds and discovers that several refunds were processed without the required independent approval. Some transactions represent genuine customer returns; others were recorded in the wrong reporting period.

Approach and outcome

The team expands testing around the identified risk, tests post-year-end cash refunds, confirms selected customer balances, and recommends an adjustment for cut-off errors. It reports a control deficiency concerning refund approval and evaluates whether the unusual user activity indicates a fraud risk. The analytics did not constitute the final evidence; they directed the team toward targeted substantive procedures.

Learning points

1. Population reconciliation is necessary before analysis.
2. Full-population analytics and sampling can be used together.
3. A projected misstatement requires consideration of sampling risk and qualitative evidence.
4. Anomaly investigation must trace transactions to reliable source evidence.
5. Control deficiencies may be important even when individual errors are not material.

9. Limitations, Governance, and Best Practice

Sampling and analytics have several limitations. A biased or incomplete population produces biased results. Poorly defined exceptions lead to inconsistent evaluation. Excessive reliance on historical relationships may overlook structural change. Automated tools may create an appearance of precision without reliable evidence. Large datasets can also encourage auditors to investigate numerous immaterial anomalies while overlooking a significant but less unusual misstatement.

Best practice includes:

• linking each test to an assertion, risk, and audit objective;
• reconciling extracted data to the accounting records;
• documenting inclusion and exclusion criteria;
• using random selection when statistical inference is intended;
• testing the reliability of reports, interfaces, and algorithms;
• separating high-value, unusual, and ordinary items where appropriate;
• retaining reproducible scripts, parameters, versions, and outputs;
• independently reviewing significant judgments and exceptions; and
• protecting confidentiality, access rights, and personal data.

Professional skepticism remains essential. Management explanations should be corroborated, not merely recorded. The auditor must also consider whether unusual patterns indicate fraud, related-party activity, override of controls, or data manipulation.

Chapter Summary

Audit sampling allows auditors to obtain evidence from less than an entire population while managing sampling and non-sampling risk. Sound design requires a clear objective, a complete population, an appropriate sampling unit, defined exceptions, defensible sample size, unbiased selection, and systematic evaluation. Statistical sampling quantifies sampling risk; non-statistical sampling depends more heavily on professional judgment but must still be rigorous.

Sample size is influenced principally by assessed risk, tolerable error, expected error, desired confidence, and population variability. Identified errors may be projected using an appropriate method, but point estimates must be considered alongside sampling risk, materiality, qualitative factors, and control implications.

Analytical procedures develop expectations and investigate significant unexplained differences. Data analytics can test complete populations, visualise patterns, identify anomalies, and improve audit focus. Nevertheless, data quality, model risk, incomplete populations, false positives, confidentiality, and human judgment remain important concerns. The strongest audits combine technology with professional skepticism and reliable corroborating evidence.

Key Terms

Audit population: The complete set of items relevant to an audit objective.

Sampling unit: The individual item selected for testing.

Sampling risk: The risk that a sample-based conclusion differs from a full-population conclusion.

Non-sampling risk: Risk of an incorrect conclusion caused by factors other than sampling.

Tolerable misstatement: Maximum acceptable misstatement in a population.

Tolerable deviation rate: Maximum acceptable rate of control deviations.

Expected misstatement: Misstatement anticipated before testing.

Statistical sampling: Sampling using random selection and probability theory.

Non-statistical sampling: Sampling based primarily on professional judgment.

Random selection: Selection in which each unit has a known chance of selection.

Systematic selection: Selection of every kth item after a random start.

Stratification: Division of a population into subpopulations for separate testing.

Attribute sampling: Sampling focused on the presence or absence of a characteristic.

Projection: Estimation of population misstatement from sample results.

Analytical procedures: Evaluation of plausible relationships and expected patterns.

Full-population testing: Testing every item against a defined rule or criterion.

Anomaly: An item or pattern that differs from an established expectation.

Data lineage: Documentation of a dataset’s source, transformations, and movement.

Professional skepticism: A questioning mind and critical assessment of audit evidence.

Review Questions

Short Answer Questions

1. What is sampling risk, and how does it differ from non-sampling risk?
2. Why must an auditor establish population completeness before selecting a sample?
3. Distinguish statistical sampling from non-statistical sampling.
4. What is the purpose of an independently developed expectation in analytical procedures?
5. Why does full-population testing not eliminate all audit risk?

Long Answer Questions

1. Explain the stages of designing and evaluating an audit sample.
2. Discuss the factors that influence substantive sample size and control-test sample size.
3. Explain how misstatements identified in a sample may be projected to a population, including the limitations of ratio projection.
4. Evaluate the strengths and weaknesses of analytical procedures as substantive audit evidence.
5. Discuss governance and ethical considerations when auditors use automated analytics and artificial intelligence.

Analytical / Application Questions

1. A population contains recorded purchases of $8,000,000. A sample tests $400,000 and identifies $10,000 of factual misstatement. Calculate the ratio-projected misstatement and explain what additional considerations are required before reaching a conclusion.
2. An auditor expects a control deviation rate of 3% and wants an approximate 95% confidence level with precision of 2 percentage points. Using n ≈ (z² × p × (1 − p)) ÷ e² and z = 1.96, calculate the approximate sample size and explain why the result is only a planning estimate.
3. Design an analytics-based audit approach for identifying potentially improper revenue cut-off at year-end. Specify the population, data validation procedures, anomaly rules, follow-up evidence, and limitations.

Multiple Choice Questions with Answers

Question 1

Which factor generally increases the required audit sample size?

A. A lower assessed risk of material misstatement

B. A higher tolerable misstatement

C. A higher expected misstatement

D. A weaker desired level of precision

Correct Answer: C. A higher expected misstatement

Question 2

A population of 500 items is tested using a desired sample of 50 items. What is the systematic sampling interval?

A. 5

B. 10

C. 50

D. 450

Correct Answer: B. 10

Question 3

Which method gives every sampling unit a known chance of selection?

A. Block selection

B. Haphazard selection

C. Random selection

D. Convenience selection

Correct Answer: C. Random selection

Question 4

A sample tests $200,000 of a population and finds $4,000 of misstatement. The population is $5,000,000. What is the ratio-projected misstatement?

A. $20,000

B. $40,000

C. $100,000

D. $1,000,000

Correct Answer: C. $100,000

Question 5

Which statement about an anomaly identified by audit analytics is correct?

A. It automatically proves fraud

B. It is evidence that no further work is necessary

C. It is a trigger for investigation and corroboration

D. It must be excluded because it was computer-generated

Correct Answer: C. It is a trigger for investigation and corroboration

Question 6

Which procedure is an example of full-population testing?

A. Selecting 40 invoices using random numbers

B. Testing every journal entry posted on a weekend

C. Selecting the first 20 transactions in a ledger

D. Reviewing only transactions chosen by management

Correct Answer: B. Testing every journal entry posted on a weekend

Question 7

In the approximate attribute-sampling formula, what does p represent?

A. Population monetary value

B. Anticipated population deviation rate

C. Audit materiality in dollars

D. The number of strata

Correct Answer: B. Anticipated population deviation rate

Question 8

Why is stratification useful in audit sampling?

A. It guarantees that no errors exist in the population

B. It allows different groups to be tested using approaches suited to their characteristics

C. It eliminates the need to define the population

D. It always produces a smaller sample than random selection

Correct Answer: B. It allows different groups to be tested using approaches suited to their characteristics

Question 9

Which is a limitation of full-population data analytics?

A. It always has greater sampling risk than a sample

B. It cannot test structured data

C. It may miss errors that do not match the chosen rule or are absent from the extract

D. It prevents auditors from investigating exceptions

Correct Answer: C. It may miss errors that do not match the chosen rule or are absent from the extract

Question 10

If expected revenue is based on 10,000 units at $25 each, what is the expectation?

A. $25,000

B. $250,000

C. $2,500,000

D. $10,025

Correct Answer: B. $250,000