Data Literacy, Measurement and Data Quality

Introduction

Business decisions increasingly depend on data generated by customers, employees, suppliers, transactions, sensors, platforms and public institutions. Yet data are not automatically facts, and a large dataset is not necessarily a good dataset. Data literacy is the capability to understand how data are defined, measured, generated, structured, analysed and used to support decisions. For managers, this capability includes questioning the meaning, origin, limitations and fitness of data before relying on analytical results.

Data quality is contextual. A dataset may be adequate for monitoring daily website traffic but unsuitable for estimating customer lifetime value. A survey may be valid for measuring attitudes among respondents but not for generalising to an entire market. Consequently, data quality should be judged against a clearly specified purpose, population, decision and acceptable level of risk.

This chapter develops a framework for distinguishing data types, measurement scales, populations and samples, structured and unstructured data, and the principal dimensions of data quality. It also examines provenance, bias, missingness, reliability, validity and governance.

Learning Objectives

By the end of this chapter, you should be able to:

1. Distinguish qualitative and quantitative, discrete and continuous, primary and secondary data.
2. Explain nominal, ordinal, interval and ratio measurement scales.
3. Differentiate populations, samples, sampling frames and units of analysis.
4. Compare structured, semi-structured and unstructured data.
5. Evaluate provenance, bias, missingness, reliability, validity and fitness for purpose.
6. Apply a systematic data-quality assessment to a managerial problem.
7. Identify ethical, governance and regulatory implications of data use.

1. Data as Representations of Phenomena

Data are recorded representations of entities, events, attributes or processes. An observation is a recorded instance, such as one invoice, one customer response or one machine reading. A variable is a characteristic that may take different values, such as price, region or satisfaction score. Metadata describe the data: their definitions, units, source, collection time, coding rules and permitted values.

The distinction between a phenomenon and its representation is fundamental. “Customer loyalty” is a theoretical construct; the number of repeat purchases during six months is one operational measure of it. The measure may be useful, but it is not identical to the construct. Measurement therefore involves translating a concept into observable rules.

A useful data description specifies:

• the unit of analysis, such as customer, order, employee or store;
• the variables and their operational definitions;
• the time period and geographic scope;
• the source and collection process;
• the units of measurement and coding scheme; and
• the intended decision or research question.

Without these elements, apparently precise data can be interpreted incorrectly.

2. Major Data Classifications

2.1 Qualitative and quantitative data

Qualitative data express categories, meanings, experiences or interpretations. Examples include interview transcripts, customer comments, product descriptions and organisational narratives. They may be coded into categories, but their original meaning is often textual or visual.

Quantitative data express counts or measured amounts. Examples include revenue, delivery time, defect rate and number of complaints. Quantitative data support arithmetic operations, statistical summaries and modelling, provided that the measurement scale permits the operation.

The distinction is not absolute. A qualitative response such as “very dissatisfied” can be coded numerically, but the resulting numbers do not automatically become measurements with meaningful arithmetic distances. Coding creates a representation; it does not necessarily change the underlying scale.

2.2 Discrete and continuous data

Discrete data arise from counting and usually take separate, often integer, values: number of orders, employees or defects. Continuous data arise from measurement and can, in principle, take any value within an interval: weight, duration, temperature or distance. In practice, continuous values are recorded with finite precision.

2.3 Primary and secondary data

Primary data are collected for the current investigation, for example through a designed survey, experiment or interview. Secondary data were originally collected for another purpose, such as government statistics, enterprise-resource-planning records or syndicated market research.

Primary data may offer closer alignment with the research question, but they can be expensive and affected by investigator choices. Secondary data may be economical and extensive, but definitions, coverage and collection procedures may not fit the current purpose. Reuse requires checking comparability, permissions, time period and data-generating incentives.

2.4 Administrative, transactional and sensor data

Administrative data are created through routine operations, such as payroll or claims processing. Transactional data record events such as purchases, payments and shipments. Sensor or telemetry data are generated automatically by devices. These sources can provide scale and timeliness, but automation does not guarantee accuracy. A sensor may be miscalibrated, a transaction system may omit informal activity, and an administrative field may reflect a policy rule rather than the underlying phenomenon.

3. Measurement Scales

Measurement scales determine which comparisons and calculations are defensible.

Nominal variables identify categories. A region code of 1, 2 and 3 is a label, not a quantity; calculating its average generally has no substantive meaning.

Ordinal variables provide order but not necessarily equal distances. The difference between “satisfied” and “very satisfied” cannot safely be assumed to equal the difference between “neutral” and “satisfied.” Medians and rank-based methods are often appropriate, although some analytical models treat rating scales as approximately interval when justified by design and evidence.

Interval scales have equal intervals, so differences are interpretable. Celsius temperature illustrates this: a change from 10°C to 20°C is the same size as a change from 20°C to 30°C, but 20°C is not twice as hot as 10°C because zero Celsius is not an absolute absence of temperature.

Ratio scales have an absolute or meaningful zero. A revenue of $200 is twice $100, and a delivery time of 20 minutes is twice 10 minutes. Ratio interpretations require that the zero genuinely represent absence of the measured quantity.

A measurement scale is not merely a formatting choice. It constrains summary statistics, visualisations, models and managerial conclusions. Treating category codes as numerical quantities is a common source of error.

4. Populations, Samples and Generalisation

A population is the complete set of units about which a conclusion is intended. A sample is a subset observed to learn about that population. The sampling frame is the operational list or mechanism from which the sample is selected. The unit of analysis is the entity represented by each observation.

For example, a retailer may want to understand all customers who purchased during 2025. The population is those customers; the customer database is a possible sampling frame; selected customers form the sample; and the customer is the unit of analysis. If the database excludes cash purchasers, the frame does not fully cover the population.

A population parameter describes the population, whereas a sample statistic describes the sample. Sampling error is the difference arising because only part of the population is observed. It should not be confused with non-sampling errors such as inaccurate measurement, non-response, coverage gaps or data-entry mistakes.

Probability sampling gives units a known, non-zero chance of selection. Simple random, stratified and cluster sampling are examples. Non-probability sampling includes convenience, purposive and snowball sampling. Non-probability methods may be appropriate for exploratory or hard-to-reach populations, but conventional margins of error cannot automatically justify generalisation from them.

A larger sample reduces some forms of random sampling variation, but it does not correct systematic bias. A very large survey of only loyal customers may produce a precise estimate of the loyal-customer subgroup while remaining inaccurate for the whole market.

5. Structured, Semi-Structured and Unstructured Data

Structured data conform to a predefined schema, commonly rows and columns in a relational database. They are relatively easy to query, validate and aggregate. Examples include invoice tables and inventory records.

Semi-structured data do not use a rigid tabular schema but contain tags, keys or other organisational markers. JSON, XML, event logs and many application-programming-interface responses are examples. Their structure can evolve, but schema validation and version control remain important.

Unstructured data lack a fixed tabular organisation. Text documents, images, audio, video, emails and social-media content are common examples. They may be analysed using natural-language processing, computer vision or other techniques, but interpretation depends heavily on context, annotation quality and algorithmic assumptions.

The categories concern data organisation, not data value. A structured database can be biased or incomplete, while an unstructured interview can contain information unavailable in transaction records. Modern organisations frequently use a multimodal approach that links structured outcomes with text, image or sensor evidence.

6. Data Provenance and the Data-Generating Process

Provenance is the documented history of data: where they originated, how they were collected, transformed, combined, stored and accessed. A provenance record should answer questions such as:

• Who created or supplied the data?
• For what purpose and under what definitions?
• When and where were they collected?
• What instruments, prompts, software or sampling procedures were used?
• Which transformations, imputations, exclusions or aggregations were applied?
• Who changed the data, and when?
• What permissions and restrictions apply?

A data lineage diagram can connect source systems to transformations, analytical tables and reports. Versioning and reproducible scripts are preferable to undocumented manual edits. Provenance is especially important when data are purchased, generated by platforms or used in high-impact decisions.

The data-generating process matters because observed values reflect both the underlying phenomenon and the mechanism that records it. A decline in complaints may indicate improved service, reduced customer access, a changed complaint form or discouragement from reporting. Analysts must distinguish a change in the world from a change in measurement.

7. Dimensions of Data Quality

Quality is multidimensional. Common dimensions include:

These dimensions can conflict. Real-time data may be timely but initially less validated. Highly complete data may include many low-quality fields. A single quality score can hide important weaknesses; a quality profile is often more informative.

7.1 Missingness

Missing data occur when a value expected by the design is absent. Missingness may result from refusal, inapplicability, system failure, skipped questions, linkage failure or deliberate suppression.

Three conceptual mechanisms are commonly distinguished. Missing completely at random means missingness is unrelated to observed or unobserved values. Missing at random means missingness may depend on observed variables after those variables are included in the analysis. Missing not at random means missingness depends on the unobserved value itself or on unmeasured factors. These are assumptions about a process, not labels that can be proven merely by inspecting a blank field.

Appropriate responses include clarifying whether “not applicable,” “unknown” and “not asked” are distinct states; reporting missingness by subgroup and time; investigating patterns; collecting the value again where feasible; and using justified methods such as complete-case analysis, weighting or imputation. Imputation does not create observed facts. It supplies estimated values and should be documented, sensitivity-tested and separated conceptually from direct observations.

7.2 Bias

Bias is systematic deviation from the target quantity or population. Sources include coverage bias, selection bias, non-response bias, response bias, measurement bias, survivorship bias, omitted-variable bias and algorithmic bias.

Bias can enter before analysis. A customer survey distributed only through a premium mobile application excludes customers without access. A performance measure based only on completed cases creates survivorship bias. A predictive model trained on historical hiring decisions may reproduce past discrimination, even if protected attributes are removed, because proxy variables remain.

Bias assessment should examine who is represented, who is missing, how variables were defined, whether incentives influenced reporting, and whether errors differ across groups. Disaggregation, external benchmarks, alternative measures and sensitivity analyses can reveal unequal performance.

7.3 Reliability and validity

Reliability concerns consistency or dependability of measurement. A scale that gives similar results under stable conditions is reliable, although it may consistently measure the wrong construct. Validity concerns whether the measure supports the intended interpretation and use.

Important forms of validity include content validity, construct validity, criterion-related validity and ecological or external validity. A customer-satisfaction questionnaire may have content validity if it covers relevant service dimensions; construct validity if it behaves as theory predicts; and criterion validity if it relates appropriately to retention or repeat purchase. External validity concerns whether results transfer to other settings or populations.

For a set of k items, internal consistency is sometimes summarised by Cronbach’s alpha:

Here, sᵢ² is the variance of item i, and sₜ² is the variance of the total score formed by the items. Alpha assumes, among other considerations, that items are intended to measure a common construct; a high value does not prove unidimensionality or validity. It can also increase simply because many similar items are included. Test–retest reliability, inter-rater agreement and measurement-error studies may be more appropriate depending on the design.

8. Fitness for Purpose

Fitness for purpose asks whether data are adequate for a specified decision. A practical assessment should identify:

1. The decision and consequences of error.
2. The target population and unit of analysis.
3. Required variables, precision, timeliness and granularity.
4. Acceptable levels of missingness, bias and measurement error.
5. Legal, ethical and security constraints.
6. Validation tests and escalation rules.

For example, approximate location data may be fit for regional advertising but not for emergency dispatch. A three-month-old inventory file may be fit for historical analysis but not for same-day replenishment. Fitness is therefore relational, not an intrinsic label permanently attached to a dataset.

A useful decision is sometimes to collect better data, combine sources, narrow the claim, postpone the decision or proceed with explicit uncertainty. Data literacy includes knowing when not to analyse.

9. Worked Application: Diagnosing a Customer-Churn Dataset

Suppose a subscription company wants to identify customers at risk of cancelling. Its analyst receives a table containing customer ID, plan, monthly fee, login count, support contacts and cancellation status.

First, the unit of analysis must be established. If each row is a customer-month, repeated rows are expected; if each row is a customer, duplicate IDs may indicate an error. The target population must also be defined: all active customers at the start of a specified observation period, not merely customers who interacted with the support team.

Next, the analyst checks definitions and time order. Login count must be measured before cancellation, not after the customer has already left. Support contacts may reflect dissatisfaction, but they may also reflect a technical problem affecting only a particular channel. The cancellation field requires a definition that distinguishes voluntary cancellation, involuntary payment failure and temporary suspension.

The analyst then profiles missingness. If monthly fee is missing mainly for annual-plan customers, treating missing values as zero would systematically distort comparisons. If cancellation status is missing for customers transferred to another system, the apparent churn rate may be biased downward.

Validation checks include unique-key rules, permitted plan codes, non-negative fees, plausible login ranges and reconciliation with billing totals. A sample of records is traced to source systems. The team compares churn rates by acquisition channel and customer segment to identify coverage and selection problems.

If the model is ultimately used to allocate retention offers, fairness and operational consequences must be considered. A high-risk score may reflect historical service inequality rather than customer propensity alone. The company should test whether prediction quality and offer allocation differ across relevant groups, document the model’s limitations and provide a route for human review.

The appropriate conclusion may be narrower than “the model predicts churn.” It may be: “Within customers represented in the billing and usage systems, and for the defined three-month horizon, the variables provide useful indicators of observed cancellation, subject to missingness and channel-coverage limitations.” This statement is less dramatic but more defensible.

10. Governance, Ethics and Regulation

Data governance assigns responsibilities for ownership, stewardship, access, quality, retention and acceptable use. A governance framework should include data dictionaries, classification rules, access controls, audit logs, retention schedules, incident procedures and quality monitoring.

Ethical data use requires purpose limitation, proportionality, transparency, fairness and respect for autonomy. Consent may be relevant, but consent alone does not justify unlimited reuse or unfair treatment. Organisations should minimise unnecessary collection, protect sensitive information and assess re-identification risk, especially when combining datasets.

Privacy regulations vary by jurisdiction, but recurring principles include lawful and transparent processing, data minimisation, accuracy, security, rights of individuals and accountability. Managers should obtain specialist legal advice for applicable requirements rather than assuming that de-identification eliminates all risk.

Artificial intelligence intensifies these issues. Automated systems can amplify historical bias, exploit proxy variables, obscure provenance and create false confidence through polished outputs. Generative AI may produce plausible but unsupported summaries, while synthetic data may preserve or distort the properties of the source data. Human oversight should focus on evidence, uncertainty, impact and contestability, not merely on whether a system produces a prediction.

11. Best Practices for Managers and Analysts

A disciplined data-quality workflow includes:

1. Define the decision, target population and unit of analysis.
2. Create a data dictionary before interpreting results.
3. Document provenance, transformations and version history.
4. Profile distributions, duplicates, ranges, missingness and subgroup coverage.
5. Validate against business rules, source records and external benchmarks.
6. Investigate discrepancies rather than silently correcting them.
7. Separate observed, derived, imputed and estimated values.
8. Test whether errors and missingness differ across groups or time.
9. Match methods and claims to the measurement scale and sampling design.
10. Report limitations, uncertainty and fitness for purpose.
11. Protect access and retain only what is justified.
12. Monitor quality after deployment because systems and populations change.

Chapter Summary

Data literacy requires understanding how data represent phenomena and how definitions, collection processes and transformations shape evidence. Data may be qualitative or quantitative, discrete or continuous, primary or secondary, and structured, semi-structured or unstructured. Measurement scales—nominal, ordinal, interval and ratio—determine which comparisons and calculations are meaningful.

Populations define the intended scope of inference, while samples, sampling frames and units of analysis determine how observations relate to that scope. Data provenance documents origin and transformation. Quality is multidimensional and includes accuracy, completeness, consistency, timeliness, uniqueness, validity, accessibility and integrity.

Missingness and bias can systematically alter conclusions. Reliability concerns consistency; validity concerns the defensibility of interpretations and uses. Neither is guaranteed by a high numerical score or a large dataset. Fitness for purpose is the central managerial test: data should be judged against the decision, population, risk, timing and acceptable uncertainty. Ethical governance requires transparent provenance, privacy protection, fairness, accountability and appropriate human oversight.

Key Terms

Data literacy: Ability to understand, evaluate and use data responsibly.

Variable: A characteristic recorded for units of analysis.

Metadata: Information describing data definitions, structure, origin and use.

Nominal scale: Measurement scale consisting of unordered categories.

Ordinal scale: Ordered categories without known equal intervals.

Interval scale: Equal-interval scale without a meaningful zero.

Ratio scale: Equal-interval scale with a meaningful zero.

Population: Complete set of units targeted by an inference.

Sample: Observed subset of a population.

Sampling frame: Operational list or mechanism used to select a sample.

Unit of analysis: Entity represented by an observation.

Structured data: Data organised according to a predefined schema.

Provenance: Documented history of data origin and transformation.

Missingness: Absence of an expected data value.

Bias: Systematic deviation from a target quantity or population.

Reliability: Consistency or dependability of measurement.

Validity: Degree to which evidence supports an intended interpretation or use.

Fitness for purpose: Adequacy of data for a specified decision.

Data governance: Policies and responsibilities governing data quality, access and use.

Review Questions

Short Answer Questions

1. Why is coding an ordinal response numerically not sufficient to make it a ratio-scale variable?
2. Distinguish a population, a sample and a sampling frame.
3. Give one example each of structured, semi-structured and unstructured data.
4. How do reliability and validity differ?
5. Why can a large sample still produce a biased conclusion?

Long Answer Questions

1. Explain the four measurement scales and discuss their implications for managerial analysis.
2. Compare primary and secondary data, including their principal advantages and risks.
3. Discuss the major dimensions of data quality and explain why a single quality score may be inadequate.
4. Explain how provenance supports reproducibility, accountability and data governance.
5. Discuss missingness mechanisms and appropriate managerial responses to missing data.

Analytical / Application Questions

1. A retailer surveys only customers who used its mobile application and concludes that all customers are highly satisfied. Diagnose the likely coverage and selection problems, and propose a better design.
2. An organisation uses historical promotion decisions to train an employee-promotion model. Identify possible sources of bias and design an evaluation plan.
3. A churn dataset contains customer-level records, but some customers appear several times and cancellation status is missing for transferred accounts. Specify the checks needed before modelling and explain how the limitations should affect the conclusions.

Multiple Choice Questions with Answers

Question 1

Which measurement scale supports ordering but does not necessarily provide equal intervals between categories?

A. Nominal

B. Ordinal

C. Interval

D. Ratio

Correct Answer: B. Ordinal

Question 2

A company wants to generalise from selected customers to all customers. Which term describes the complete set about which it intends to conclude?

A. Sample

B. Sampling frame

C. Population

D. Variable

Correct Answer: C. Population

Question 3

Which example is semi-structured data?

A. A relational invoice table with fixed columns

B. A JSON event record containing keys and values

C. A recorded customer interview

D. A warehouse temperature measurement

Correct Answer: B. A JSON event record containing keys and values

Question 4

Which statement about a large sample is correct?

A. It automatically eliminates systematic bias

B. It guarantees valid measurement

C. It can reduce random sampling variation but not necessarily bias

D. It makes missing data irrelevant

Correct Answer: C. It can reduce random sampling variation but not necessarily bias

Question 5

What is data provenance primarily concerned with?

A. The visual colour scheme of a dashboard

B. The documented origin and transformation history of data

C. The average value of every variable

D. The number of rows in a dataset

Correct Answer: B. The documented origin and transformation history of data

Question 6

A database field contains negative product quantities even though the business rule permits only zero or positive quantities. Which quality dimension is most directly violated?

A. Validity

B. Timeliness

C. Accessibility

D. Uniqueness

Correct Answer: A. Validity

Question 7

A measure produces nearly identical results each time, but it consistently measures employee tenure instead of job satisfaction. The measure is best described as:

A. Valid but unreliable

B. Reliable but invalid for the intended construct

C. Neither reliable nor consistent

D. Automatically fit for purpose

Correct Answer: B. Reliable but invalid for the intended construct

Question 8

Why should “not applicable” and “unknown” generally be stored as distinct statuses?

A. They always have the same statistical meaning

B. They distinguish different reasons for the absence of a value

C. They convert nominal data into ratio data

D. They remove all non-response bias

Correct Answer: B. They distinguish different reasons for the absence of a value

Question 9

Which situation is an example of survivorship bias?

A. Including both active and cancelled customers

B. Analysing only firms that remained in business while studying business failure

C. Randomly selecting customers from a complete list

D. Recording a transaction in two currencies

Correct Answer: B. Analysing only firms that remained in business while studying business failure

Question 10

Fitness for purpose means that data:

A. Are always complete and error-free

B. Are suitable for a specified decision, population and level of risk

C. Come exclusively from primary research

D. Contain only quantitative variables

Correct Answer: B. Are suitable for a specified decision, population and level of risk