Data, Measurement and Sampling

Introduction

Statistical and predictive analytics begin before modelling. An analyst must first determine what is being measured, where the data originate, how observations were selected, and whether the resulting dataset represents the population of interest. A sophisticated algorithm cannot compensate for a poorly defined variable, a biased sample, systematic missingness, or unreliable recording processes.

This chapter develops the foundations required to prepare defensible analytical datasets. It examines variable types, measurement scales, structured and unstructured data, populations and samples, probability and non-probability sampling, sampling error, bias, data quality, missingness, and practical dataset preparation. The emphasis is on business applications, where data may be drawn from transactions, customers, employees, sensors, surveys, websites, documents, and third-party platforms.

Learning objectives

By the end of this chapter, you should be able to:

• classify variables according to their nature and analytical role;
• distinguish nominal, ordinal, interval and ratio measurement scales;
• identify structured, semi-structured and unstructured data sources;
• define a population, sampling frame, sample and sampling unit;
• select and implement suitable sampling methods;
• calculate and interpret basic sampling quantities;
• identify sampling bias, non-response, measurement error and data-quality problems;
• diagnose missing-data mechanisms and select defensible treatments; and
• prepare a representative, documented dataset for statistical or predictive analysis.

1. Data as evidence for decision-making

Data are recorded observations, measurements, classifications or representations of events. Information is data interpreted in context, while evidence is information considered sufficiently reliable and relevant for a decision. In analytics, the central question is not merely “How much data do we have?” but “What do these observations justify us in concluding?”

A business dataset usually has two dimensions:

• Rows, or records, represent units such as customers, orders, employees, products or time periods.
• Columns, or variables, represent attributes measured for those units.

For example, an order table might contain order ID, customer segment, order date, quantity, unit price, discount and delivery status. The order ID identifies a record; quantity and price are measured variables; customer segment is a categorical variable; and delivery status may be an outcome variable.

A useful analytical specification defines:

1. the unit of analysis—the entity about which a conclusion will be made;
2. the target population—the complete group to which the conclusion is intended to apply;
3. the time period and geographical scope;
4. the operational definition of every variable; and
5. the intended decision or research question.

Confusing units of analysis is a common error. A customer-level retention model cannot be correctly trained from a mixture of customer, order and product rows unless the data are deliberately aggregated or linked.

2. Variables and their analytical roles

A variable is a characteristic that can take different values across observations. A constant has the same value for every observation in a defined dataset. Variables can be classified in several overlapping ways.

2.1 Qualitative and quantitative variables

Qualitative, or categorical, variables describe membership in groups. Examples include payment method, region, industry and complaint type. Their values are labels, even when numerical codes are used. Coding “online payment” as 1 and “cash” as 2 does not make the variable quantitative.

Quantitative variables represent amounts or counts for which arithmetic has meaningful interpretation. They include revenue, waiting time, distance, number of purchases and temperature.

2.2 Discrete and continuous variables

A discrete variable takes separate, countable values, commonly non-negative integers. Number of returned items is discrete. A continuous variable can, in principle, take any value within an interval. Delivery time, mass and temperature are continuous, although instruments and databases record them to finite precision.

The distinction is conceptual rather than absolute. Age recorded in completed years is discrete in the dataset, whereas exact age measured in days or seconds is closer to continuous.

2.3 Predictor, response and control variables

In supervised analytics, the response or target is the outcome to be explained or predicted. A predictor, also called a feature or explanatory variable, is used to estimate that outcome. A control variable is included to account for a factor that may otherwise confound an association.

These labels depend on the analytical design. A variable such as price may be a response in a pricing study, a predictor in a demand model, or a control variable in an evaluation of promotional effectiveness. Predictive importance does not establish causation.

2.4 Identifiers and derived variables

An identifier distinguishes records, such as customer ID or invoice number. It is usually not a meaningful predictor in its raw form. High-cardinality identifiers can cause overfitting or allow a model to memorize records.

A derived variable is calculated from one or more original fields. Examples include profit margin, customer age band, recency, or average order value. Derived variables must be documented because choices about definitions, time windows and denominators affect interpretation.

3. Measurement scales

Measurement scales determine which comparisons and statistical operations are defensible. Four classical levels are commonly used.

3.1 Nominal measurement

Nominal values have no intrinsic ranking. A region coded 1, 2 and 3 is still nominal. The code “3” is not more region-like than “1”. Suitable summaries include frequency, percentage, mode and contingency tables. The mean of nominal codes is not meaningful.

Binary variables are nominal variables with two categories, such as defective/not defective. They may be represented as 0 and 1. The coding permits useful calculations—for example, the mean of a binary variable equals the proportion coded 1—but the interpretation depends on the coding convention.

3.2 Ordinal measurement

Ordinal values have a meaningful order but unequal or unknown distances between categories. A five-point satisfaction response from “very dissatisfied” to “very satisfied” indicates rank, but the difference between adjacent responses cannot automatically be assumed equal. Medians, percentiles and rank-based methods are appropriate. Treating ordinal responses as numerical scores can be reasonable in some modelling contexts, but the assumption should be stated and tested through sensitivity analysis.

3.3 Interval measurement

Interval scales have equal units, so differences are meaningful, but zero is arbitrary rather than an absence of the measured quantity. The difference between 10 °C and 20 °C equals the difference between 20 °C and 30 °C. However, 20 °C is not twice as hot as 10 °C. Calendar years are another example: 2020 is not twice 1010 in a meaningful temporal sense.

3.4 Ratio measurement

Ratio scales have equal units and a non-arbitrary zero representing the absence of the quantity. A revenue of £200 is twice £100, and a duration of 20 minutes is twice 10 minutes. Ratio-scale data support addition, subtraction, multiplication and division, subject to the context and units.

Measurement scale is not always a property of a field name alone. “Customer score” could be nominal, ordinal, interval-like or ratio-like depending on how it was constructed. Analysts should consult the measurement protocol and data dictionary.

4. Data sources and data structures

4.1 Structured data

Structured data are organized according to a defined schema, usually in tables with rows, columns, keys and data types. Examples include enterprise resource planning records, point-of-sale transactions, payroll systems and relational databases. Structured data are comparatively easy to query, validate and join, but they may reflect only processes that an organization has chosen to record.

4.2 Semi-structured data

Semi-structured data contain organizational markers but do not conform fully to a fixed table. JSON documents, XML files, application logs and event streams are common examples. A single event may contain optional fields or nested arrays. Before analysis, analysts generally parse the structure, establish field definitions and decide how repeated or missing elements will be represented.

4.3 Unstructured data

Unstructured data lack a regular tabular schema. Examples include emails, contracts, call recordings, photographs, videos, social-media text and scanned forms. They can be highly informative but require additional processing such as optical character recognition, natural-language processing, image analysis or human coding. Transformation can introduce errors and ethical risks, including incorrect transcription, inferred sensitive attributes and context loss.

4.4 Internal and external sources

Internal sources include transaction systems, customer relationship management platforms, human-resource records and operational sensors. External sources include government statistics, market research, public websites, data brokers and partner organizations. External data require scrutiny of provenance, licensing, update frequency, population coverage and compatibility of definitions.

A data-source inventory should record owner, collection method, time coverage, granularity, refresh schedule, access restrictions, known limitations and permitted uses.

5. Population, sampling frame and sample

The population is the complete set of units relevant to a question. The target population is the population to which findings are intended to generalize. The accessible population is the portion that can actually be reached.

A sampling frame is the operational list or mechanism used to select units. It may be a customer register, employee roster, address database or stream of eligible transactions. A frame can suffer from undercoverage when eligible units are absent, duplication when units appear more than once, or outdated information when eligibility has changed.

A sample is a subset selected for analysis. The sampling unit is the basic element selected, such as a person, household, store or transaction. In cluster designs, a selected cluster may contain many ultimate units.

A parameter describes a population, whereas a statistic describes a sample. For a variable X, the population mean is commonly denoted μ and the sample mean x̄. The sample mean is an estimator of μ, not a guarantee that μ has been exactly measured.

6. Sampling methods

6.1 Simple random sampling

In simple random sampling, each eligible unit has a known, usually equal, probability of selection, and selections are made using a random mechanism. A random-number generator can select customer IDs from a complete, deduplicated frame.

For a sample mean based on a simple random sample of n observations, the standard error is commonly estimated by:

Here, s is the sample standard deviation and n is sample size. This expression assumes independent observations and is most directly applicable to simple random sampling. If sampling without replacement from a finite population of size N, a finite population correction may be used:

The correction becomes important when the sampling fraction n/N is not small, often when it exceeds approximately 5% to 10%, depending on the design and context.

6.2 Systematic sampling

Systematic sampling selects every k-th unit after a random start. If the frame has N units and the desired sample size is n, the approximate interval is:

The method is efficient, but periodic patterns in the frame can create bias. For example, selecting every seventh record may align with a weekly operational cycle.

6.3 Stratified sampling

In stratified sampling, the population is divided into non-overlapping strata, such as region or customer tier, and a sample is drawn from each stratum. Stratification improves representation and can increase precision when members within a stratum are relatively similar and strata differ from one another.

If h indexes strata, the overall estimated mean is:

where Wₕ = Nₕ/N is the population proportion in stratum h, Nₕ is the stratum population size, N is total population size, and x̄ₕ is the sample mean within stratum h. The weights must correspond to the target population, particularly when disproportionate numbers are sampled from strata.

6.4 Cluster sampling

Cluster sampling selects naturally occurring groups, such as branches, schools or geographic areas, and then surveys units within selected clusters. It reduces travel and administration costs, but units within a cluster may be similar, producing less independent information than an equally sized simple random sample.

6.5 Multistage sampling

Multistage sampling combines stages—for example, selecting regions, then branches, then customers. It is practical for geographically dispersed populations but requires design weights and variance methods that reflect selection at each stage.

6.6 Convenience, purposive and quota sampling

Non-probability methods do not provide known selection probabilities for all eligible units.

• Convenience sampling uses readily available units, such as website visitors.
• Purposive sampling deliberately selects information-rich cases, often for qualitative investigation or specialist research.
• Quota sampling seeks specified numbers in categories but selects cases non-randomly within those categories.

These methods may be appropriate when no frame exists or when exploratory insight is the aim. However, matching sample proportions to known demographics does not remove hidden selection bias within categories.

7. Sample size and precision

For estimating a population proportion under a simple random design, an approximate planning formula is:

Here, n₀ is the preliminary sample size, z is the standard-normal critical value for the desired confidence level, p is the anticipated population proportion, and e is the maximum tolerated margin of error expressed as a proportion. The formula assumes a probability sample, independent observations and a sufficiently large population. When p is unknown, p = 0.5 gives the largest required sample size for a fixed z and e.

For a 95% confidence level, z is approximately 1.96. If p = 0.5 and e = 0.05:

n₀ = (1.96² × 0.5 × 0.5) / 0.05² ≈ 384.16

Thus, approximately 385 completed observations are required under the stated assumptions. This is not a universal rule: design effects, non-response, finite populations and subgroup comparisons may require a larger initial sample.

A sample size should also reflect the decision purpose. Estimating an overall percentage, comparing two customer segments, detecting a small effect and training a complex predictive model are different tasks. Power analysis and expected event rates are important for predictive and inferential studies.

8. Bias, error and representativeness

Sampling error is random variation arising because a sample rather than the entire population is observed. It can often be quantified through standard errors or confidence intervals. Systematic bias is different: it shifts results in a direction that may not disappear simply by increasing sample size.

Important forms include:

• Undercoverage bias: eligible units are absent from the frame.
• Selection bias: inclusion probabilities differ in ways related to the outcome.
• Non-response bias: selected units do not participate, and respondents differ from non-respondents.
• Survivorship bias: analysis includes only units that remain observable or successful.
• Measurement bias: the recorded value systematically differs from the intended concept.
• Recall bias: respondents inaccurately remember past events.
• Social-desirability bias: respondents give socially acceptable rather than truthful answers.
• Processing bias: coding, joining, transcription or transformation introduces systematic error.

Representativeness is multidimensional. A sample may match the population by age and region but differ in purchasing frequency, digital access or motivation. Weighting, calibration, follow-up of non-respondents and comparison with external benchmarks can improve, but cannot always eliminate, bias.

9. Data quality assessment

Data quality should be assessed relative to intended use. Common dimensions are:

Quality checks may include range validation, format checks, duplicate detection, referential-integrity tests, reconciliation to control totals and comparison with trusted benchmarks. A negative quantity may be valid for a product return but invalid for a shipment count; rules require business context.

A data dictionary should define field name, business meaning, type, unit, valid values, missing-value codes, source, transformation history and responsible owner. Data lineage records how a value moved from collection to analytical use.

10. Missing data

Missingness may result from refusal, inapplicability, system failure, sensor interruption, conditional survey routing or deliberate suppression. Distinguish a true zero from “not recorded”, “not applicable” and “unknown”. Special codes such as 999 can be mistaken for real measurements if not converted and documented.

Three conceptual mechanisms are frequently used:

• MCAR (Missing Completely At Random): missingness is unrelated to observed and unobserved values.
• MAR (Missing At Random): after accounting for observed variables, missingness is unrelated to the unobserved value itself.
• MNAR (Missing Not At Random): missingness depends on the unobserved value or an unmeasured process.

These are assumptions about a missingness mechanism, not merely descriptions of the percentage missing. For example, income may be more often missing for high-income respondents, suggesting an MNAR concern.

Practical treatments include complete-case analysis, available-case analysis, single imputation, multiple imputation, missingness indicators and model-based methods. Complete-case analysis is simple but can reduce sample size and bias results when missingness is not MCAR. Mean imputation distorts variation and relationships and should rarely be used as a default. Multiple imputation can represent uncertainty under plausible assumptions, but it requires careful model specification and diagnostics.

Missingness itself may be predictive—for example, failure to complete an onboarding field may signal disengagement. Such a feature must be used cautiously, especially when the missingness process could change after deployment or encode protected characteristics.

11. Preparing a representative analytical dataset

A defensible workflow is iterative rather than a single cleaning step:

1. Define the estimand or decision target. State exactly what quantity, prediction or comparison is required.
2. Define the population and eligibility rules. Specify geography, time period, exclusions and unit of analysis.
3. Audit the frame and sources. Check coverage, duplicates, permissions and source definitions.
4. Choose the sampling design. Use probability sampling when generalization and measurable uncertainty are required.
5. Record selection probabilities and weights. Do not discard design information.
6. Profile the data. Examine distributions, missingness, duplicates, outliers, invalid values and subgroup coverage.
7. Resolve or document quality issues. Never silently overwrite questionable values.
8. Separate preparation from evaluation. For predictive work, split data by time or randomly as appropriate, and fit preprocessing operations on training data only to avoid leakage.
9. Assess representativeness. Compare sample distributions with population benchmarks and investigate differences.
10. Create a reproducible data pipeline. Version code, specifications and datasets; log transformations and approvals.
11. Validate the final analytical table. Confirm row grain, key uniqueness, target availability, feature timing and permitted use.

For predictive analytics, sampling and splitting must also respect the deployment context. Randomly splitting repeated observations from the same customer can place near-duplicates in both training and test sets, overstating performance. Temporal validation is usually more realistic when future prediction is the objective.

12. Case application: estimating service satisfaction

This is an illustrative case. A national telecommunications provider wants to estimate the proportion of customers who are satisfied with technical support and identify factors associated with dissatisfaction.

Background and problem

The company initially proposes emailing a survey link to all customers who contacted support during one month. Management wants a national estimate and comparisons among prepaid, contract and business customers.

Analysis of the initial design

The email frame excludes customers with invalid email addresses and customers who use assisted channels without a registered email. Survey participation is voluntary. Customers with extremely positive or negative experiences may be more motivated to respond. Therefore, the design faces undercoverage and non-response bias. A simple percentage of respondents cannot automatically be treated as a national estimate.

Improved approach

The analytics team defines the unit of analysis as a customer-support episode and, separately, creates a customer-level summary for management reporting. It constructs a frame of all eligible episodes, removes duplicates, and stratifies by customer type, region and contact channel. Within each stratum, episodes are selected using a random procedure. Oversampling of business customers is used to obtain a useful subgroup estimate, and population weights are retained for national estimates.

The questionnaire defines satisfaction using a five-category ordinal scale and records the response option “not applicable” separately from missing. A small pilot tests wording, completion time and mobile usability. Reminder contacts are sent using a standardized schedule. Response rates are monitored by stratum, and selected non-respondents are compared with respondents using administrative variables such as channel, tenure and resolution time.

Data preparation and outcome

The team validates episode IDs, checks that resolution time is non-negative, identifies repeated submissions, and links survey responses to operational records using controlled keys. It reports weighted estimates with uncertainty, provides unweighted response counts, and documents assumptions about non-response. The final predictive dataset uses only variables available before the satisfaction response, preventing leakage.

Learning points

The case demonstrates that representativeness depends on frame coverage, selection, participation, measurement and analysis—not merely sample size. Stratification can support subgroup analysis, but weights are needed when sample composition differs from the population. Clear variable definitions and timing rules are as important as statistical technique.

13. Ethical, legal and governance considerations

Data preparation involves responsibilities beyond technical accuracy. Analysts should establish a lawful and transparent basis for collection and use, apply purpose limitation, minimize unnecessary personal data, protect sensitive attributes, and control access. Depending on jurisdiction, privacy and data-protection law may require notice, consent or another lawful basis, retention limits, data-subject rights, impact assessments and safeguards for automated decisions.

Sampling can exclude vulnerable groups, while poorly designed surveys can pressure participation. Proxy variables may reproduce discrimination even when protected attributes are removed. Analysts should assess subgroup performance, document limitations, and avoid presenting a sample estimate as universal when coverage is narrow.

Artificial intelligence can assist with parsing documents, coding text and detecting anomalies, but generated labels and inferred attributes require validation. Models may amplify historical measurement and selection bias. Human review, provenance records and appropriate access controls remain necessary.

14. Best-practice checklist

Before releasing a dataset for analysis, confirm that:

• the unit of analysis and target population are explicit;
• every variable has an operational definition and appropriate scale;
• the sampling frame is assessed for coverage and duplication;
• selection probabilities, strata, clusters and weights are recorded;
• response rates and non-response patterns are examined;
• missing codes are distinguished from valid values and zeros;
• quality checks are automated where possible and exceptions reviewed;
• transformations, joins and exclusions are reproducible;
• predictive splits prevent temporal, entity and target leakage;
• privacy, licensing, fairness and retention requirements are satisfied; and
• limitations are communicated with the results.

Chapter Summary

Reliable analytics begins with a clear relationship between a business question, a population, a unit of analysis and a measurement process. Variables may be categorical or quantitative, discrete or continuous, and may serve as predictors, responses, controls, identifiers or derived features. Nominal, ordinal, interval and ratio scales determine which comparisons and calculations are meaningful.

Structured, semi-structured and unstructured sources differ in organization, processing requirements and risks. Sampling requires attention to the population, frame, unit and selection mechanism. Probability methods support measurable uncertainty, while non-probability methods may be useful for exploration but limit generalization. Sampling error can be quantified; systematic bias generally cannot be solved by simply increasing sample size.

Data quality includes accuracy, completeness, validity, consistency, timeliness, uniqueness and integrity. Missingness must be investigated rather than hidden, and treatments should reflect plausible mechanisms. A representative analytical dataset is produced through explicit definitions, appropriate sampling, documented weighting, quality control, leakage prevention, ethical governance and reproducible processes.

Key Terms

Analytical dataset: A prepared collection of records and variables used for analysis or modelling.

Bias: Systematic deviation of an estimate or process from the intended truth.

Cluster sampling: Sampling in which naturally occurring groups are selected rather than, or before, individual units.

Continuous variable: A variable that can theoretically take any value within an interval.

Data dictionary: Documentation describing fields, meanings, types, units, values and rules.

Discrete variable: A countable variable, often taking integer values.

Frame: The operational list or mechanism from which a sample is selected.

Interval scale: A scale with equal differences but no meaningful absolute zero.

Missing data: Values or records that are unavailable, unrecorded or not applicable.

Nominal scale: A scale consisting of labels without inherent order.

Non-response bias: Bias arising when participation differs systematically between respondents and non-respondents.

Ordinal scale: A scale with ordered categories but not necessarily equal distances.

Population: The complete set of units relevant to an analytical question.

Probability sampling: Sampling in which units have known selection probabilities.

Ratio scale: A scale with equal units and a meaningful zero.

Sampling error: Random variation caused by observing a sample rather than the full population.

Sampling unit: The basic entity selected in a sampling procedure.

Statistic: A numerical summary calculated from a sample.

Stratified sampling: Sampling separately within defined population groups.

Variable: A characteristic that can take different values across observations.

Review Questions

Short Answer Questions

1. What is the difference between a population, a sampling frame and a sample?
2. Why is numerical coding of nominal categories not sufficient to make them quantitative?
3. Distinguish discrete and continuous variables using one business example of each.
4. What is the difference between sampling error and systematic bias?
5. Why should “not applicable” be distinguished from an ordinary missing value?

Long Answer Questions

1. Explain the four measurement scales and discuss which summaries or operations are appropriate for each.
2. Compare simple random, stratified, cluster and systematic sampling, including one advantage and one limitation of each.
3. Discuss how undercoverage and non-response can affect the validity of a customer survey.
4. Explain MCAR, MAR and MNAR missingness and evaluate the strengths and limitations of common treatments.
5. Design a data-quality and governance process for preparing a dataset from multiple internal and external sources.

Analytical / Application Questions

1. A retailer wants to estimate satisfaction across three customer tiers. The business tier represents 10% of customers, but the research team samples equal numbers from all tiers. Explain why weighting may be required and describe an appropriate analysis plan.
2. A predictive dataset contains repeated monthly records for the same customers. Explain how a random train-test split could produce leakage and propose a more realistic validation design.
3. A survey of annual income has 28% missing values, with missingness substantially higher among respondents who report financial stress. Identify the likely concern, assess whether mean imputation is appropriate, and recommend a defensible investigation and treatment strategy.

Multiple Choice Questions with Answers

Question 1

Which measurement scale has categories with meaningful order but not necessarily equal distances?

A. Nominal

B. Ordinal

C. Interval

D. Ratio

Correct Answer: B. Ordinal

Question 2

A database of sales transactions with fixed columns, keys and defined data types is an example of:

A. Structured data

B. Unstructured data

C. Purposive data

D. Unmeasured data

Correct Answer: A. Structured data

Question 3

Which method selects every k-th record after choosing a random starting point?

A. Cluster sampling

B. Quota sampling

C. Systematic sampling

D. Purposive sampling

Correct Answer: C. Systematic sampling

Question 4

In stratified sampling, why might a researcher oversample a small subgroup?

A. To eliminate all measurement error

B. To obtain enough observations for subgroup analysis

C. To ensure the subgroup is excluded from weighting

D. To convert ordinal variables into ratio variables

Correct Answer: B. To obtain enough observations for subgroup analysis

Question 5

If p = 0.5, z = 1.96 and e = 0.05, the approximate preliminary sample size from n₀ = z²p(1 − p)/e² is:

A. 96

B. 196

C. 285

D. 385

Correct Answer: D. 385

Question 6

Which situation is an example of undercoverage bias?

A. A selected respondent gives an incorrect answer

B. A customer appears twice in the database

C. Customers without email addresses cannot enter an email survey

D. A random sample differs from the population by chance

Correct Answer: C. Customers without email addresses cannot enter an email survey

Question 7

What does a meaningful zero distinguish about a ratio scale?

A. It permits categories to be alphabetized

B. It represents the absence of the measured quantity

C. It guarantees normally distributed data

D. It prevents missing values

Correct Answer: B. It represents the absence of the measured quantity

Question 8

Which statement about mean imputation is generally correct?

A. It always removes bias

B. It preserves the original variance exactly

C. It can distort variation and relationships

D. It is required whenever data are MNAR

Correct Answer: C. It can distort variation and relationships

Question 9

A variable recording the number of products returned by an order is usually:

A. A discrete quantitative variable

B. A continuous qualitative variable

C. An interval categorical variable

D. A nominal identifier

Correct Answer: A. A discrete quantitative variable

Question 10

Why can randomly splitting repeated customer records into training and test sets be problematic?

A. It makes all variables nominal

B. It may place near-duplicate customer information in both sets

C. It guarantees undercoverage

D. It prevents calculation of a sample mean

Correct Answer: B. It may place near-duplicate customer information in both sets