Data, Measurement, Sampling and Governance

Introduction

Business analytics begins before any model is fitted or dashboard is designed. It begins with decisions about what to observe, how to measure it, whom to study, how to obtain information, and how to protect the people and organisations represented in the data. Poor decisions at this stage cannot usually be repaired by sophisticated statistical methods. A highly accurate algorithm trained on biased, incomplete, or unlawfully collected data may produce confidently wrong conclusions.

This chapter develops the foundations required to create and evaluate business data. It distinguishes major data types, explains measurement scales, compares primary and secondary sources, introduces probability and non-probability sampling, and examines bias, missingness, data quality, privacy, consent, ownership, and governance. The emphasis is practical: a business analyst should be able to defend why data were collected, judge whether they represent the target population, identify threats to validity, and establish controls for responsible use.

Learning Objectives

By the end of this chapter, you should be able to:

1. Classify data by source, structure, time, and analytical role.
2. Distinguish nominal, ordinal, interval, and ratio measurement scales.
3. Evaluate primary, secondary, observational, transactional, and digital trace data.
4. Construct and assess probability and non-probability samples.
5. Explain sampling error, bias, nonresponse, missing data, and data-quality problems.
6. Apply appropriate approaches to privacy, consent, ownership, security, and data governance.
7. Critically evaluate a business data-collection plan.

1. What Counts as Data?

Data are recorded representations of observations, events, attributes, or claims. A single recorded value is often called an observation or data value; a collection of observations is a dataset. Information is data interpreted in context, while evidence is information judged relevant and credible for a particular decision.

A useful classification considers several dimensions simultaneously.

Quantitative data are expressed numerically, but not every number supports arithmetic. A postcode, product code, or customer identifier is numeric in appearance yet categorical in meaning. Qualitative data may be text, images, audio, or categories. They can be analysed systematically through coding, classification, or natural-language methods, but their interpretation depends strongly on context.

Discrete and continuous data

Discrete data arise from counts, such as the number of orders. Continuous data can, in principle, take any value in an interval, such as delivery time or temperature. In practice, instruments and business systems record continuous quantities to finite precision. The distinction matters because counts cannot normally be negative or fractional, whereas measurement variables may require rounding and tolerance rules.

Cross-sectional, time-series, and panel data

Cross-sectional data describe units at one point or period. Time-series data record one or more variables repeatedly over time, where ordering and dependence matter. Panel data follow multiple units across multiple periods, allowing analysts to distinguish differences between units from changes within a unit. For example, annual sales for 500 stores over five years form panel data.

Metadata and data lineage

Metadata describe data: definitions, units, allowable values, source systems, timestamps, ownership, and transformation history. Data lineage records how data move and change from origin to report or model. Without metadata and lineage, analysts may confuse gross and net revenue, local and UTC time, or current and historical customer status.

2. Measurement and Operationalisation

Measurement is the assignment of values to attributes according to a rule. Operationalisation converts an abstract construct into observable variables. “Customer loyalty,” for example, might be operationalised through repeat-purchase rate, share of wallet, retention duration, or a survey scale. No single operational definition automatically captures the entire construct.

A sound measure should be relevant to the construct, reliable across repeated or equivalent observations, and valid for the intended inference. Reliability is consistency; validity concerns whether the measure represents what it claims to represent. A scale can be reliable but invalid: a consistently miscalibrated thermometer gives repeatable but inaccurate readings.

The four measurement scales

Nominal categories should be mutually exclusive and collectively exhaustive for the classification purpose, although a “not stated” category may be needed. Ordinal values have order, but the distance between adjacent categories is not necessarily equal. Thus, calculating a mean satisfaction score may be a useful convention, but it requires justification rather than automatic acceptance.

Interval data have equal units, so differences are interpretable. Zero on the Celsius scale is not the absence of temperature; therefore 20°C is not twice as hot as 10°C. Ratio data possess a meaningful zero, so statements such as “twice as much” are meaningful.

Measurement scale affects analysis. Frequencies and proportions are appropriate for nominal data; medians and percentiles are often appropriate for ordinal data; means and standard deviations can be appropriate for interval and ratio data when distributions and assumptions support them. Coding categories as 1, 2, and 3 does not transform ordinal data into ratio data.

Validity and reliability threats

Content validity asks whether important dimensions of a construct are covered. Criterion validity compares a measure with a relevant external outcome. Construct validity examines whether relationships with other variables agree with theory. Common threats include ambiguous questions, leading wording, recall failure, social desirability, inconsistent definitions, instrument drift, and changes in the business process generating the data.

3. Sources and Modes of Data Collection

Primary and secondary data

Primary data are collected for the current research purpose, such as a survey, interview, observation, or experiment. They offer control over definitions and collection procedures but may be costly, slow, and affected by response behaviour.

Secondary data were collected for another purpose. Examples include financial statements, administrative records, public statistics, syndicated market research, and open-data portals. They can be economical and broad, but the analyst must investigate definitions, population coverage, collection dates, revisions, missingness, and incentives of the original collector. A dataset is not automatically suitable merely because it is large or publicly available.

A data-source evaluation should ask:

• Who created the data, and for what purpose?
• What population and time period are covered?
• What exactly does each variable mean, including units and denominators?
• How were records selected, measured, cleaned, and revised?
• What errors, missing values, duplicates, or exclusions are known?
• Are use rights, privacy requirements, and retention limits clear?

Observational and experimental data

Observational data record naturally occurring behaviour without assigning treatments. They are valuable for monitoring and prediction, but associations may reflect confounding, selection, or reverse causality. Experimental data arise when a researcher assigns an intervention according to a design, ideally with randomisation and a defined control condition. Experiments can support stronger causal claims, though implementation, noncompliance, spillover, and external validity remain concerns.

Transactional and digital trace data

Transactional data are generated by business operations: purchases, invoices, claims, shipments, or service tickets. They often have precise timestamps and identifiers but reflect recorded transactions, not all customer activity. Digital trace data include clicks, searches, GPS signals, app events, cookies, and social interactions. They can provide detailed behavioural evidence, yet visibility is uneven: device settings, platform policies, algorithmic ranking, shared accounts, and offline behaviour affect what is observed.

Logs should not be treated as neutral reality. An event may be missing because a user did not act, because tracking failed, or because consent settings blocked collection. Analysts must distinguish “no event observed” from “event did not occur.”

4. Sampling: From Population to Evidence

The population is the full set of units about which a conclusion is intended. A target population is the conceptual population of interest; a sampling frame is the operational list or mechanism from which units can be selected. The sample is the subset actually studied. Coverage error occurs when the frame omits, duplicates, or improperly includes population units.

A sampling plan should specify the unit of analysis, target population, frame, selection method, sample size, contact or observation procedure, replacement rules, and weighting or adjustment strategy. Analysts must also distinguish the unit sampled from the unit measured. A household may be sampled while individual household members provide different responses.

Probability sampling

In probability sampling, each eligible unit has a known, non-zero probability of selection. This permits design-based estimation and, when assumptions hold, quantification of sampling uncertainty.

Simple random sampling selects units so that every sample of a given size has equal probability. It is conceptually straightforward but requires a suitable frame.

Systematic sampling selects every k-th unit after a random start, where approximately k = N/n, N is frame size and n is desired sample size. It is efficient, but periodic patterns in the frame can cause bias.

Stratified sampling divides the population into relevant strata and samples within each. It can improve representation and precision when strata are internally similar on the outcome. Disproportionate sampling requires weights when estimating population quantities.

Cluster sampling selects groups, such as branches or schools, and observes units within selected groups. It can reduce travel and administration costs, but observations within a cluster are often correlated, reducing effective information.

For a simple random sample estimating a population proportion p, the standard error of the sample proportion p̂ is approximately:

Here n is the sample size and p̂ is the observed proportion. This approximation assumes a probability sample, sufficiently large expected counts, and a sampling fraction small enough that finite-population correction can be ignored. If sampling without replacement from a substantial fraction of a finite population, use:

where N is population size. The second factor is the finite-population correction. A 95% approximate confidence interval is p̂ ± 1.96SE(p̂) under the usual large-sample normal approximation; the interval describes sampling uncertainty, not all possible sources of bias.

Worked example. A random sample of 400 customers finds that 120 used a mobile app. Thus p̂ = 120/400 = 0.30. The approximate standard error is √[0.30 × 0.70 / 400] ≈ 0.0229. The approximate 95% interval is 0.30 ± 1.96(0.0229), or approximately 0.255 to 0.345. This calculation does not correct coverage error or nonresponse bias.

Non-probability sampling

In non-probability sampling, selection probabilities are unknown or zero for some units. Convenience sampling selects accessible units; purposive sampling selects cases judged relevant; quota sampling fills category targets without random selection within categories; snowball sampling uses participant referrals; and voluntary-response sampling relies on self-selection.

These methods may be appropriate for exploratory research, rare or hidden populations, qualitative inquiry, or rapid operational feedback. However, a large non-probability sample does not automatically represent the population. Sampling error formulas based on random selection should not be presented as if they validate a convenience sample. Analysts should report recruitment channels, eligibility, response rates where meaningful, composition, exclusions, and limits on generalisation.

5. Bias, Missing Data, and Data Quality

Sampling error is random variation caused by observing a sample rather than the whole population. Bias is systematic deviation from the target quantity. Important forms include:

• Coverage bias: the frame excludes or overrepresents parts of the population.
• Nonresponse bias: participation or completion depends on characteristics related to the outcome.
• Selection bias: inclusion is related to exposure, outcome, or both.
• Measurement bias: the recorded value systematically differs from the intended value.
• Survivorship bias: analysis excludes failed, closed, or dropped-out units.
• Confirmation bias: collection or interpretation favours a prior belief.
• Algorithmic bias: automated systems produce unequal errors or outcomes because of data, design, or deployment conditions.

Missing data are not merely empty cells. A value may be missing completely at random, missing at random conditional on observed variables, or missing not at random when missingness depends on the unobserved value or an unmeasured cause. These assumptions cannot generally be proven from the observed dataset alone.

Practical responses include documenting missingness, comparing respondents and nonrespondents, using a meaningful “not applicable” category, conducting sensitivity analysis, and applying justified imputation. Mean imputation is simple but reduces variability and can distort relationships. Multiple imputation may better represent uncertainty when its model and assumptions are defensible. Deleting records can create bias and should not be an automatic default.

Data quality is multidimensional:

Quality controls include validation at entry, range and referential-integrity checks, duplicate detection, reconciliation to authoritative totals, version control, anomaly review, and clear correction logs. Cleaning must preserve an audit trail: changing a value without recording who changed it, when, why, and from what source weakens trust and reproducibility.

6. Privacy, Consent, Ownership, and Governance

Privacy concerns the appropriate use and protection of information about identifiable or identifiable people. Personal data may include direct identifiers, such as names, and indirect combinations, such as location, date, and rare occupation. Pseudonymisation replaces identifiers with codes but does not necessarily remove re-identification risk. Anonymisation aims to make identification no longer reasonably possible, but risk depends on auxiliary data and context.

A responsible collection process follows principles commonly reflected in data-protection regimes: purpose limitation, data minimisation, transparency, accuracy, storage limitation, security, and accountability. Legal requirements vary by jurisdiction and sector; organisations should obtain specialist advice rather than assume that public availability means unrestricted reuse.

Consent should be informed, specific, freely given where applicable, and documented. Consent is not always the only lawful basis for processing, and it is not a substitute for fair practice. A notice should explain what is collected, why, with whom it is shared, retention, rights, and contact points in understandable language. For sensitive data, children, employees, or vulnerable groups, stronger safeguards may be necessary.

Ownership and rights are distinct from possession. A company may store a dataset without owning every underlying right in the data. Contracts, intellectual-property law, confidentiality duties, database rights, platform terms, and data-protection law may all apply. Data licensing should specify permitted purposes, redistribution, derived products, retention, security, and deletion.

Governance is the system of decision rights, policies, standards, roles, controls, and oversight that makes data trustworthy and responsibly usable. Typical roles include a data owner accountable for business use, a data steward responsible for definitions and quality, a custodian responsible for technical controls, and an analyst responsible for appropriate interpretation. Governance should cover classification, access, lineage, quality thresholds, retention, incident response, model documentation, and audit.

Security controls should follow least privilege, strong authentication, encryption in transit and at rest, secure backups, monitoring, and tested incident procedures. Access should be granted according to role and purpose, not curiosity. Privacy-preserving techniques such as aggregation, suppression, masking, differential privacy, and secure analysis can reduce disclosure risk, but each introduces trade-offs in accuracy and utility.

AI systems intensify governance requirements. Training data may encode historical discrimination, scraped content may have unclear rights, and generated outputs may reveal confidential information or fabricate facts. Organisations should document data provenance, assess representativeness, test performance across relevant groups, monitor drift, provide human oversight for consequential decisions, and maintain an appeal or correction process. A model card or comparable record can describe intended use, limitations, evaluation data, and known risks.

7. An Integrated Data-Readiness Framework

Before analysis, an analyst can use the following sequence:

1. Define the decision. State the action, outcome, population, time horizon, and acceptable error.
2. Define constructs and variables. Specify operational definitions, units, categories, and identifiers.
3. Map sources and lineage. Record origin, transformations, owners, permissions, and refresh schedule.
4. Assess representativeness. Compare the frame and achieved sample with the target population.
5. Audit quality. Check validity, duplicates, missingness, consistency, outliers, and reconciliation totals.
6. Assess ethics and law. Confirm purpose, notice, consent or other legal basis, minimisation, access, and retention.
7. Document limitations. Separate sampling uncertainty from systematic bias and state what cannot be inferred.
8. Monitor after deployment. Recheck data quality, population changes, drift, complaints, and unequal impacts.

This framework prevents a common error: treating data preparation as a purely technical task. Data quality is partly a design property and partly a governance property. It is created by processes, incentives, definitions, and accountability.

8. Illustrative Case: A Retailer’s Personalisation Survey

Background

Illustrative retailer Northstar operates stores and an online channel. Management wants to estimate customer satisfaction and build a model to identify customers likely to stop purchasing.

Problem

The analytics team proposes emailing a satisfaction survey to customers who bought online during the previous month. Only 8% respond. The team reports an average satisfaction score of 4.6 out of 5 and plans to use the responses as representative of all customers.

Analysis

The target population is all active customers, but the frame covers only customers with an email address and a recent online purchase. Store-only customers and customers who opted out are excluded. Voluntary response creates additional selection: highly satisfied or highly dissatisfied customers may be more motivated to respond. Satisfaction is ordinal if the response categories are ordered labels, although an average score may be used as a practical summary only with a stated rationale. The survey also asks respondents to recall a purchase, creating possible recall error.

The churn model has a different outcome and should not automatically use survey respondents as its training population. Transactional data may cover more customers, but “no purchase” must be distinguished from missing transaction feeds. Privacy review is needed because linking survey answers to purchase histories creates a personal-data profile.

Approach and solution

Northstar defines the target population and separates online and store customers. It draws a stratified probability sample from an approved customer frame, oversampling store-only customers where necessary and applying documented weights. It uses multiple contact modes, records disposition outcomes, and compares respondents with nonrespondents. The survey is shortened, tested for ambiguous wording, and introduced with a clear privacy notice. Analysts report response bias as a limitation and conduct sensitivity analyses under alternative assumptions.

For churn modelling, Northstar documents the prediction date, prevents future information from entering training data, evaluates performance across customer groups, and restricts access to the linked dataset. A data owner approves the purpose, while a steward maintains definitions and lineage.

Outcome

The original 4.6 score is not used as an estimate of all-customer satisfaction. The revised design produces a more defensible estimate and reveals that store-only customers have different experiences. The churn model becomes less accurate in headline terms but more reliable for the intended population and less likely to target customers unfairly.

Learning points

A high response score is not evidence of representativeness. Sampling frame, recruitment, measurement, linkage, privacy, and modelling purpose must be considered together. Explicit documentation often changes the business question from “What did respondents say?” to “What can we responsibly infer about the target population?”

Challenges and Limitations

No sampling method eliminates all error. Probability methods may be expensive and still suffer nonresponse or poor measurement. Administrative data may be comprehensive for transactions but silent about people who never enter the system. Privacy protection can reduce granularity, while excessive collection increases risk without guaranteeing better decisions. Governance frameworks may become bureaucratic if controls are not proportionate to sensitivity and impact. Finally, representativeness is purpose-dependent: a sample adequate for estimating national sales may be inadequate for analysing a small regional subgroup.

Best-Practice Checklist

• Begin with a clearly defined population and decision.
• Use precise variable definitions, units, codes, and metadata.
• Prefer probability sampling when population estimates are required.
• Do not confuse sample size with representativeness.
• Measure and report nonresponse, exclusions, and missingness.
• Separate data cleaning from undocumented alteration.
• Match analytical claims to the design: association is not automatically causation.
• Minimise personal data and restrict access by purpose.
• Record consent, legal basis, retention, ownership, and lineage.
• Test data and AI systems for unequal error and harmful consequences.

Chapter Summary

Data can be classified by structure, source, time, origin, and measurement role. Measurement scales—nominal, ordinal, interval, and ratio—determine which comparisons and summaries are meaningful. Primary data provide purpose-specific control, whereas secondary, transactional, and digital trace data require careful evaluation of definitions, coverage, and collection incentives. Observational data support description and prediction but do not automatically establish causation.

Sampling connects a target population to observed evidence. Probability sampling provides known selection probabilities and supports design-based uncertainty calculations; non-probability methods can be useful but limit generalisation. Bias, nonresponse, missingness, and measurement error may remain even with large samples. Data quality includes accuracy, completeness, consistency, timeliness, validity, uniqueness, and integrity.

Responsible analytics requires privacy, informed and appropriate consent, clear rights and ownership, security, minimisation, lineage, and accountable governance. The analyst’s professional task is not merely to produce a number or model, but to explain what the data represent, what they do not represent, and whether the intended use is justified.

Key Terms

Anonymisation: Processing intended to prevent reasonable identification of individuals.

Bias: Systematic deviation from the target quantity or population.

Cluster sampling: Selecting groups of units and observing units within selected groups.

Consent: An informed and appropriately documented agreement to specified processing or participation.

Construct: An abstract concept, such as loyalty or trust, measured through observable indicators.

Data governance: Policies, roles, standards, controls, and oversight for responsible data use.

Data lineage: A record of data origin, movement, and transformation.

Digital trace data: Records generated by digital activity, such as clicks, searches, or location signals.

Measurement scale: The rules determining the meaning of values and allowable comparisons.

Missing data: Values not recorded, unavailable, or inapplicable for an observation.

Nonresponse bias: Bias arising when participation or completion is related to the outcome or relevant characteristics.

Operationalisation: Converting an abstract construct into measurable variables.

Panel data: Repeated observations on multiple units across time.

Pseudonymisation: Replacing identifiers with codes while retaining possible re-identification risk.

Sampling frame: The operational list or mechanism from which a sample is selected.

Sampling error: Random variation caused by studying a sample rather than the whole population.

Stratified sampling: Sampling separately within defined population strata.

Target population: The population to which an inference is intended to apply.

Review Questions

Short Answer Questions

1. Distinguish structured, semi-structured, and unstructured data, giving one example of each.
2. Why is a customer identification number not ratio-scale data, even though it is numeric?
3. Explain the difference between a sampling frame and a target population.
4. Name two causes of missing data and explain why treating all missing values as zero can be misleading.
5. What is data lineage, and why does it support trustworthy analysis?

Long Answer Questions

1. Compare primary and secondary data for a customer research project. Discuss cost, control, relevance, quality, and ethical considerations.
2. Explain nominal, ordinal, interval, and ratio scales and evaluate which descriptive statistics are appropriate for each.
3. Compare simple random, systematic, stratified, and cluster sampling. Include the circumstances in which each may be preferred.
4. Discuss how coverage error, nonresponse, measurement error, and survivorship bias can affect a business survey.
5. Explain how privacy, consent, ownership, security, and governance interact in an organisation using customer data.

Analytical / Application Questions

1. A company surveys only customers who made a recent online purchase and receive promotional emails. Identify at least four threats to representativeness or validity and propose a revised sampling and measurement plan.
2. A random sample of 625 invoices contains 175 invoices with a discount. Calculate the sample proportion and approximate standard error using SE(p̂) = √[p̂(1 − p̂)/n]. Interpret what the result says and does not say.
3. An AI churn model uses app activity as a predictor, but older customers and customers who disabled tracking appear less often in the app data. Analyse the likely data-quality, bias, privacy, and governance issues and recommend controls.

Multiple Choice Questions with Answers

Question 1

Which measurement scale supports meaningful statements such as “twice as much”?

A. Nominal

B. Ordinal

C. Interval

D. Ratio

Correct Answer: D. Ratio

Question 2

A dataset containing repeated annual sales for the same stores over five years is best described as:

A. Panel data

B. Purely nominal data

C. A simple random sample

D. Cross-sectional data from one date

Correct Answer: A. Panel data

Question 3

What is the principal advantage of probability sampling?

A. It guarantees no missing data

B. It gives eligible units known, non-zero selection probabilities

C. It always costs less than convenience sampling

D. It eliminates measurement error

Correct Answer: B. It gives eligible units known, non-zero selection probabilities

Question 4

A frame excludes customers who do not have email addresses. This is primarily an example of:

A. Coverage error

B. Random sampling error

C. Finite-population correction

D. Data encryption

Correct Answer: A. Coverage error

Question 5

In a sample of 400 customers, 120 use an app. What is the sample proportion?

A. 0.03

B. 0.12

C. 0.30

D. 3.33

Correct Answer: C. 0.30

Question 6

Which statement about a convenience sample is correct?

A. Its sampling error can always be calculated exactly

B. Every population unit has equal selection probability

C. It may be useful for exploration but limits population generalisation

D. It is necessarily larger than a probability sample

Correct Answer: C. It may be useful for exploration but limits population generalisation

Question 7

Which data-quality dimension asks whether values conform to permitted formats and rules?

A. Timeliness

B. Validity

C. Uniqueness

D. Completeness

Correct Answer: B. Validity

Question 8

Why is a “no recorded click” not necessarily evidence that a user did not click?

A. Clicks are always interval-scale variables

B. Tracking may have failed or been blocked by consent settings

C. Digital trace data cannot contain timestamps

D. A click is a nominal measurement

Correct Answer: B. Tracking may have failed or been blocked by consent settings

Question 9

Which governance principle generally reduces privacy risk?

A. Collect every potentially useful variable indefinitely

B. Grant all analysts unrestricted access

C. Use data minimisation and purpose limitation

D. Remove all metadata

Correct Answer: C. Use data minimisation and purpose limitation

Question 10

Pseudonymisation means that:

A. All re-identification risk is eliminated

B. Identifiers are replaced with codes, but re-identification may remain possible

C. Data are automatically made public

D. Data no longer require security controls

Correct Answer: B. Identifiers are replaced with codes, but re-identification may remain possible