Data Collection, Sampling and Preparation

Introduction

Business analytics is only as reliable as the data on which it is based. A sophisticated predictive model cannot compensate for a poorly designed survey, an unrepresentative sample, inconsistent coding, or unrecognised missing and extreme values. Data collection and preparation are therefore not preliminary clerical activities; they are methodological stages that determine the validity, fairness and usefulness of analytical conclusions.

This chapter develops a practical framework for designing surveys and observational studies, selecting samples, recognising sources of error, and preparing data for analysis. The emphasis is on decisions commonly made by business analysts: defining a population, constructing a sampling frame, choosing an appropriate sampling method, validating records, coding variables, treating missing values and assessing outliers.

Learning Objectives

By the end of this chapter, you should be able to:

1. Distinguish populations, samples, parameters and statistics.
2. Design suitable surveys and observational studies.
3. Apply probability and non-probability sampling methods.
4. Explain sampling error and important non-sampling errors.
5. Estimate a simple sample size for a population proportion.
6. Validate, code, transform and document analytical data.
7. Select appropriate approaches to missing values and outliers.
8. Evaluate data quality, ethics and governance risks in a business setting.

1. From Business Question to Data Requirement

A useful data project begins with a decision, not with an available spreadsheet. The analyst should translate the business problem into a measurable question. For example, “Why are customers leaving?” may require information about tenure, service use, complaints, price changes and satisfaction. Each concept must be converted into an operational definition.

An operational definition specifies exactly how a variable will be observed or measured. “Customer satisfaction” might be measured by a five-point rating collected within 30 days of a service interaction. “Churn” might mean no purchase for 90 consecutive days. Without such definitions, different data collectors may record the same phenomenon differently.

A population is the complete set of units about which conclusions are required. A unit may be a customer, transaction, employee, store or delivery. A sample is a subset of the population. A population value is a parameter, such as the true average delivery time μ. A value calculated from a sample is a statistic, such as the sample mean x̄.

The sampling frame is the operational list from which a sample is selected. It may be a customer database, employee register or list of stores. The target population and frame are rarely identical. Customers without email addresses, recently opened accounts or unregistered informal businesses may be omitted. This difference creates coverage risk.

2. Surveys and Observational Studies

Surveys

A survey collects information by asking people questions. It may be administered online, by telephone, in person or through a structured interview. Surveys are appropriate when the required information concerns attitudes, intentions, experiences or characteristics that cannot be observed directly.

A well-designed questionnaire follows a logical sequence:

1. Introduce the purpose, sponsor, expected duration and confidentiality arrangements.
2. Ask screening questions to establish eligibility.
3. Ask simple, relevant questions before sensitive or complex questions.
4. Group questions by topic.
5. Collect demographic questions where justified and preferably near the end.
6. Close with an opportunity for comments or contact information if appropriate.

Questions should be specific and neutral. “How satisfied are you with delivery speed?” is preferable to “How excellent was our fast delivery?” The latter is leading. Double-barrelled questions, such as “How satisfied are you with price and service?”, combine two constructs and cannot be interpreted reliably if the respondent has different views of each.

Response options must be mutually exclusive and collectively exhaustive. Age categories such as 18–24, 25–34 and 35–44 are clear; categories such as 18–25 and 25–35 overlap. A question about income may require “prefer not to say” to reduce forced inaccurate responses. A pilot test with a small group can reveal ambiguity, excessive length, technical faults and missing response options.

Reliability concerns consistency of measurement. Validity concerns whether the instrument measures the intended construct. A scale can be reliable but invalid: a faulty weighing instrument may give the same incorrect reading repeatedly. Content validity asks whether important aspects of a construct are covered; criterion validity compares responses with an appropriate external measure.

Observational studies

In an observational study, the analyst records behaviour or characteristics without assigning an intervention. Examples include measuring queue times, analysing website clicks, or comparing purchase patterns across stores. Observational data may be passive, such as automatically generated transaction logs, or active, such as trained staff recording shelf availability.

Observation is valuable when behaviour differs from what people report. However, it does not automatically establish causation. If stores with self-checkout have shorter queues, other factors—store size, staffing, customer volume or location—may explain the association. Confounding occurs when a third variable is related to both the apparent cause and outcome.

A structured observation plan should specify the units observed, time periods, locations, variables, recording rules and quality checks. Observers require training and, where possible, inter-rater agreement should be assessed. The Hawthorne effect occurs when subjects change behaviour because they know they are being observed.

3. Sampling Designs

Probability sampling

In probability sampling, every population unit has a known, non-zero probability of selection. This permits probability-based estimation of sampling uncertainty and supports generalisation when the design and implementation are sound.

Simple random sampling selects units so that each permitted sample of a given size has an equal chance of selection. It is conceptually simple but requires a complete sampling frame.

Systematic sampling selects every kth unit after a random starting point. If the population size is N and the desired sample size is n, the approximate interval is:

The frame should be checked for periodic patterns. For example, selecting every tenth transaction may be biased if transactions are ordered in a ten-day cycle.

Stratified sampling divides the population into relatively homogeneous groups called strata, such as region or customer type, and samples from each stratum. It ensures representation of important subgroups and can improve precision. Proportional allocation gives stratum h a sample size:

where n_h is the sample from stratum h, n is total sample size, N_h is the population size of stratum h and N is total population size. The formula assumes all strata are sampled proportionally. Disproportionate allocation may be used to obtain enough observations from a small but important subgroup; weighted analysis may then be needed.

Cluster sampling selects naturally occurring groups, such as branches, schools or geographic areas, and surveys units within selected clusters. It can reduce travel and administration costs. However, units in the same cluster tend to resemble one another, so cluster sampling often provides less statistical information than a simple random sample of the same number of units.

Multistage sampling combines stages—for example, selecting regions, then stores within regions, then customers within stores. It is practical for large dispersed populations but requires careful design and analysis.

Non-probability sampling

Non-probability sampling does not provide known selection probabilities. It is often cheaper and useful for exploratory research, rapid feedback or populations without a usable frame, but sampling uncertainty cannot be estimated in the same conventional way.

Convenience sampling uses accessible participants. It is quick but vulnerable to severe selection bias. Purposive or judgement sampling deliberately selects cases with relevant characteristics, such as experienced procurement managers. Quota sampling fills predetermined subgroup quotas without random selection within each quota. Snowball sampling asks participants to refer others and may help reach hidden populations, although networks can produce highly similar respondents. Volunteer or self-selection sampling is especially vulnerable to participation by people with unusually strong opinions.

The choice is not simply between “good” probability sampling and “bad” non-probability sampling. A probability sample from an outdated frame can be poor, while purposive sampling may be appropriate for interviewing expert informants. The analyst must match the design to the inference intended.

4. Sample Size and Sampling Error

Sampling error is the natural difference between a sample statistic and the population parameter caused by observing only part of the population. It is not the same as a mistake. Increasing sample size generally reduces sampling variability, but it does not correct systematic undercoverage or nonresponse bias.

For a large population, a commonly used planning formula for estimating a population proportion is:

Here n₀ is the preliminary sample size, z is the standard-normal critical value for the chosen confidence level, p is the anticipated population proportion, and e is the desired maximum margin of error expressed as a proportion. The formula assumes a probability sample, an approximately independent sample, and a proportion estimate under the usual large-sample approximation.

When p is unknown, p = 0.5 is conservative because it produces the largest required sample size for a fixed z and e. For a 95% confidence level, z is approximately 1.96. If p = 0.5 and e = 0.05:

Thus, at least 385 completed responses are commonly planned before allowing for nonresponse. If the population is finite and not large relative to the sample, a finite population correction can be applied:

where N is the population size. If 1,000 eligible customers form the population, n is approximately 278. The calculation concerns completed usable responses, so an expected response rate must be considered. With an expected response rate of 70%, the number invited is approximately 278 ÷ 0.70 ≈ 398. This planning calculation does not eliminate nonresponse bias.

5. Errors in Data Collection

Sampling errors

Coverage error occurs when the frame excludes, duplicates or incorrectly includes population units. Sampling error arises from selecting a sample rather than a census. Nonresponse error occurs when selected units do not participate or do not answer particular questions, especially when respondents differ systematically from nonrespondents.

Non-sampling errors

Non-sampling errors can occur in a census as well as a sample. Important types include:

• Measurement error: the recorded value differs from the true value because of poor wording, faulty instruments, recall problems or social desirability.
• Processing error: mistakes in data entry, coding, merging, transcription or calculation.
• Interviewer error: an interviewer influences answers or records them inconsistently.
• Response error: a participant misunderstands, forgets or intentionally misreports information.
• Specification error: the variable collected does not represent the business concept required.
• Survivorship bias: analysis includes only entities that remain observable, excluding failures or exits.

Quality control includes pilot testing, validation rules, duplicate checks, interviewer training, audit trails and comparison with trusted totals. Analysts should report the response rate, sampling design, field dates, exclusions and known limitations rather than presenting a result with false precision.

6. Data Preparation as a Reproducible Process

Data preparation converts collected data into a form suitable for analysis while preserving its meaning and provenance. A defensible workflow is:

1. Define the analytical dataset and unit of analysis.
2. Preserve an untouched raw copy.
3. Inspect structure, metadata, ranges, duplicates and relationships.
4. Validate values and business rules.
5. Standardise formats and code categorical variables.
6. Treat missing values and outliers using documented rules.
7. Transform variables only when analytically justified.
8. Recheck totals, distributions and referential integrity.
9. Save a clean dataset, data dictionary and transformation log.

The unit of analysis must be explicit. A table may contain one row per customer, order or order item. Joining an order table to an item table can duplicate order-level revenue if the analyst does not aggregate first. This is a common source of inflated totals.

Validation

Validation checks whether values conform to defined requirements. Examples include:

• Type checks: dates are stored as dates; quantities are numeric.
• Range checks: satisfaction lies between 1 and 5.
• Domain checks: country codes come from an approved list.
• Uniqueness checks: customer IDs are unique where required.
• Completeness checks: mandatory fields are present.
• Consistency checks: shipment date is not earlier than order date.
• Referential checks: every order’s customer ID exists in the customer table.

A failed validation should be investigated, not automatically overwritten. The correct response may be correction from a source system, marking the value as missing, excluding the record under a stated rule, or retaining it with a quality flag.

Coding and measurement levels

Coding assigns meaningful representations to observations. Nominal categories, such as payment method, have no inherent order. Ordinal categories, such as low, medium and high, have order but not necessarily equal distances. Interval scales have meaningful differences but no true zero, while ratio scales have a meaningful zero and support ratios.

For a nominal variable with categories such as Online, Store and Phone, labels may be retained or represented by codes such as 1, 2 and 3. The numbers are identifiers, not quantities; calculating an average of these codes has no substantive meaning. Binary variables are often coded 0 and 1, but the coding convention must be documented. One-hot or dummy encoding may be used in modelling, with care to avoid redundant categories when an intercept is included.

A data dictionary should record variable name, description, unit, type, permissible values, missing-value codes, source, transformation and sensitivity classification.

Transformation and standardisation

Transformations can improve interpretability, meet model requirements or reduce skewness. Common operations include extracting year or month from a date, aggregating transactions to customer level, converting currencies using a stated exchange-rate date, and calculating rates with a clearly defined denominator.

A frequently used standard score is:

where x_i is an observation, x̄ is the sample mean and s is the sample standard deviation. The resulting value has no physical unit and indicates how far an observation lies from the sample mean in standard-deviation units. Standardisation is useful for comparing variables on different scales, but it does not make data normally distributed and can itself be affected by extreme values.

Logarithmic transformation may reduce right skew for strictly positive variables such as revenue. Zero and negative values require a justified alternative; adding an arbitrary constant can alter interpretation. Transformations must be recorded so results can be explained and, where necessary, converted back to the original scale.

7. Missing Values

Missingness can be structural—for example, “number of children” may not apply to a business customer—or caused by refusal, system failure, loss or nonresponse. Codes such as 99, “unknown” and blank must not be treated as ordinary numerical values.

Three useful conceptual mechanisms are:

• MCAR (Missing Completely At Random): missingness is unrelated to observed and unobserved values.
• MAR (Missing At Random): after accounting for observed variables, missingness is unrelated to the missing value itself.
• MNAR (Missing Not At Random): missingness depends on the unobserved value or an unmeasured process.

These mechanisms are assumptions about the data-generating process, not labels that can always be proven from a dataset.

Simple deletion is acceptable only when its assumptions and consequences are understood. Listwise deletion removes records with missing values in any analysed variable and may reduce sample size substantially. Pairwise deletion uses available values for particular calculations but can produce different bases across statistics. Single-value imputation, such as replacing missing income with the median, is simple but understates uncertainty and can distort relationships.

More defensible approaches may include multiple imputation, model-based methods, or an explicit “not applicable” category where substantively appropriate. The method should reflect the analysis, missingness pattern and consequences. Analysts should compare results under plausible alternatives as a sensitivity analysis.

8. Outliers and Anomalous Records

An outlier is an observation unusually distant from the general pattern. It may be a data-entry error, a valid rare event, a member of a different population, or evidence of an important business phenomenon. Outlier does not mean incorrect.

For a roughly symmetric variable, a boxplot convention identifies potential outliers below:

or above:

where Q₁ and Q₃ are the first and third quartiles and IQR = Q₃ − Q₁. This is a screening rule, not a universal law. In a dataset with Q₁ = 10 and Q₃ = 18, IQR = 8 and the upper threshold is 18 + 1.5 × 8 = 30. A value of 35 should be investigated, not automatically deleted.

For approximately normal data, a z-score with absolute value greater than about 3 may be a useful alert, but the mean and standard deviation are themselves sensitive to outliers. Robust measures such as the median and IQR may be preferable for skewed data.

Possible actions are correction, retention, segmentation, transformation, winsorisation or exclusion. Exclusion requires a documented reason, and analyses with and without influential observations can reveal whether conclusions depend on them.

9. Applied Case: Improving a Retail Customer Survey

Background

The fictional retailer BrightMart believes that delivery delays are reducing repeat purchases. Management wants to estimate the proportion of recent online customers who are satisfied with delivery and identify operational factors associated with dissatisfaction.

Problem

The first proposed approach was to post a survey link on social media. This would be inexpensive but would produce a self-selected sample, likely overrepresenting customers with very positive or very negative experiences. It would also omit customers who do not follow the retailer online.

Analysis and approach

The analyst defines the population as customers who completed an online order during the previous 60 days. The sampling frame is the order-management database, deduplicated so that the unit is the customer rather than the order. Customers are stratified by delivery region and randomly selected within each region. The survey asks one clear satisfaction question on a five-point scale, records actual delivery time from the operational system, and includes an optional open comment.

A pilot identifies that “delivery was on time” is interpreted differently by customers. The final instrument defines on-time delivery as arrival on or before the promised date. A response-rate report is prepared by region, and reminder messages are limited to two to avoid pressure. Nonresponse is examined by comparing respondents and nonrespondents on available order characteristics.

Preparation rules specify that satisfaction values must be 1–5, delivery dates must be valid, and a customer cannot appear more than once in the survey file. Missing satisfaction is retained as missing rather than coded as zero. An extremely long delivery time is checked against the order record; if genuine, it is retained and flagged.

Outcome

The resulting estimate is more defensible than the social-media estimate because selection is linked to the defined population and regional representation is controlled. Management receives both the satisfaction estimate and its limitations, including nonresponse risk. Analysis shows that dissatisfaction is concentrated in two regions, where warehouse dispatch delays—not courier travel time—are the dominant operational issue.

Learning points

The case demonstrates that a credible analysis depends on defining the population, choosing a defensible frame, designing neutral questions, linking survey and operational data carefully, and distinguishing unusual valid observations from errors. The sampling plan also improves managerial action by connecting customer perceptions to measurable process data.

10. Ethics, Governance and Contemporary Practice

Data collection must respect informed participation, privacy, fairness and proportionality. Respondents should know the purpose of collection, intended uses, retention arrangements and relevant rights. Sensitive personal data require stronger access controls and a legitimate basis for processing under applicable law and organisational policy. Data minimisation means collecting only what is necessary.

Analysts should restrict access, encrypt sensitive files, remove direct identifiers where possible, and maintain an audit trail. Re-identification risk can remain even after names are removed when several attributes are combined. Automated systems and AI tools may accelerate entity matching, anomaly detection and coding, but they can reproduce biased labels, infer sensitive attributes or make opaque corrections. AI-generated transformations should be reviewed, reproducible and subject to human accountability.

Contemporary data collection increasingly combines surveys, transaction systems, sensors, web analytics and customer-service records. Combining sources can increase insight but also creates linkage errors, inconsistent definitions and consent concerns. A data catalogue, common definitions, quality ownership and version-controlled preparation code help maintain trust.

Best-Practice Checklist

Before releasing an analytical dataset, ask:

• Is the target population and unit of analysis explicit?
• Does the sampling frame cover the population adequately?
• Are selection, response and exclusion processes documented?
• Are questions neutral, specific and piloted?
• Are validation rules and data-quality exceptions recorded?
• Are codes, units and missing-value meanings documented?
• Were duplicates and join effects checked?
• Were missing values and outliers investigated rather than mechanically removed?
• Can every transformation be reproduced from the raw data?
• Are privacy, fairness, retention and access requirements satisfied?

Chapter Summary

Reliable analytics begins with a clearly defined population, measurable concepts and a collection design aligned with the intended decision. Surveys require neutral, unambiguous questions, suitable response options and pilot testing. Observational studies record behaviour without assigning treatments and therefore require caution about causal claims.

Probability sampling—simple random, systematic, stratified, cluster and multistage—supports generalisation when selection probabilities and implementation are appropriate. Non-probability methods can be practical for exploratory or specialist work but provide weaker conventional bases for statistical inference. Sampling error differs from coverage, nonresponse, measurement, processing and specification errors.

Data preparation is a documented, reproducible process involving validation, coding, transformation, missing-value treatment and outlier assessment. Missingness mechanisms affect the appropriateness of deletion or imputation. Outliers must be investigated in context because they may be errors or valuable evidence. Ethical governance requires privacy protection, transparency, data minimisation, fairness and human accountability, especially when AI tools are used.

Key Terms

Population: Complete set of units about which conclusions are required.

Sample: Subset of a population selected for study.

Parameter: Numerical characteristic of a population.

Statistic: Numerical characteristic calculated from a sample.

Sampling frame: Operational list from which units are selected.

Probability sampling: Sampling in which each unit has a known, non-zero selection probability.

Stratified sampling: Sampling separately within defined population subgroups.

Cluster sampling: Sampling naturally occurring groups and units within them.

Convenience sampling: Selecting readily accessible units.

Coverage error: Error caused by mismatch between the target population and sampling frame.

Nonresponse bias: Bias arising when respondents differ systematically from nonrespondents.

Measurement error: Difference between a recorded value and the intended true value.

Operational definition: Precise rule for measuring a concept.

Data dictionary: Documentation of variables, codes, units, sources and rules.

Validation: Checking whether data satisfy specified requirements.

Missing value: An unavailable, inapplicable or unrecorded observation.

MCAR: Missingness unrelated to observed and unobserved values.

Outlier: Observation unusually distant from the general data pattern.

Standardisation: Rescaling a value using its mean and standard deviation.

Data minimisation: Collecting only data necessary for a legitimate purpose.

Review Questions

Short Answer Questions

1. Distinguish a population parameter from a sample statistic.
2. What is the purpose of a sampling frame?
3. State two characteristics of a well-designed survey question.
4. Differentiate sampling error from measurement error.
5. Why should an outlier not be deleted automatically?

Long Answer Questions

1. Compare stratified, cluster and systematic sampling, including a suitable business application for each.
2. Explain how questionnaire wording, response options and pilot testing affect validity and reliability.
3. Describe a reproducible workflow for preparing a raw customer transaction dataset.
4. Discuss MCAR, MAR and MNAR missingness and the implications for treatment choices.
5. Explain the ethical and governance responsibilities involved in collecting and combining customer data.

Analytical / Application Questions

1. A company has 12,000 customers and wants a proportional stratified sample of 600. Region A contains 3,000 customers. Calculate the proportional sample for Region A and explain one reason management might choose disproportionate allocation instead.
2. A survey uses a social-media link to estimate customer satisfaction. Identify at least three likely sources of bias and redesign the sampling approach.
3. In a delivery dataset, Q₁ = 10 minutes and Q₃ = 18 minutes. Calculate the IQR and upper boxplot screening threshold. Explain the investigations that should occur before any record is removed.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes a parameter from a statistic?

A. A parameter describes a sample, while a statistic describes a population.

B. A parameter describes a population, while a statistic is calculated from a sample.

C. A parameter is always categorical, while a statistic is always numerical.

D. A parameter is collected by survey, while a statistic is collected by observation.

Correct Answer: B. A parameter describes a population, while a statistic is calculated from a sample.

Question 2

A researcher selects every 20th customer after choosing a random starting point. Which method is being used?

A. Cluster sampling

B. Quota sampling

C. Systematic sampling

D. Snowball sampling

Correct Answer: C. Systematic sampling

Question 3

What is a principal advantage of stratified sampling?

A. It guarantees no nonresponse.

B. It ensures important subgroups are represented.

C. It eliminates measurement error.

D. It requires no sampling frame.

Correct Answer: B. It ensures important subgroups are represented.

Question 4

For N = 1,000 and n₀ = 384.16, what is the approximate finite-population sample size using n = n₀ ÷ [1 + (n₀ − 1) ÷ N]?

A. 96

B. 192

C. 278

D. 384

Correct Answer: C. 278

Question 5

Which example is a non-sampling error?

A. Variation caused by selecting a sample instead of a census

B. A frame that omits customers without email addresses

C. A respondent misunderstanding a question

D. Random selection of 200 customers

Correct Answer: C. A respondent misunderstanding a question

Question 6

A variable coded as 1 = Online, 2 = Store and 3 = Phone should generally be treated as which measurement level?

A. Nominal

B. Ordinal

C. Interval

D. Ratio

Correct Answer: A. Nominal

Question 7

Which action is most appropriate when a delivery time appears unusually high?

A. Delete it immediately

B. Replace it with the mean automatically

C. Investigate whether it is an error or a valid event

D. Recode it as zero

Correct Answer: C. Investigate whether it is an error or a valid event

Question 8

If Q₁ = 10 and Q₃ = 18, what is the upper 1.5 × IQR screening threshold?

A. 22

B. 26

C. 30

D. 34

Correct Answer: C. 30

Question 9

Which statement about a survey posted openly on social media is most accurate?

A. It is a simple random sample of all customers.

B. It is vulnerable to self-selection bias.

C. It eliminates nonresponse bias.

D. It guarantees proportional regional representation.

Correct Answer: B. It is vulnerable to self-selection bias.

Question 10

What is the main purpose of retaining a raw data copy and a transformation log?

A. To make missing values disappear

B. To guarantee a normal distribution

C. To support reproducibility and auditability

D. To convert nominal variables into ratio variables

Correct Answer: C. To support reproducibility and auditability