Data, Measurement and Sampling

Introduction

Accounting and finance dissertations depend on evidence. Whether a study examines audit quality, financial reporting quality, capital structure, budgeting behaviour, investment decisions, fraud risk, sustainability reporting, or the performance of financial institutions, its conclusions are only as credible as the data on which they are based. Data must therefore be defined, measured, sampled, collected, cleaned, documented and protected systematically.

This chapter focuses on the practical and methodological foundations required to construct a reliable dataset. It explains how abstract concepts are converted into measurable variables, how questionnaires and other instruments are designed, how probability and non-probability samples are selected, and how primary and secondary data are managed. It also addresses validity, reliability, missing data, data quality, ethics, confidentiality and reproducibility.

Learning Objectives

By the end of this chapter, students should be able to:

1. Distinguish concepts, constructs, variables, indicators and observations.
2. Classify variables and select appropriate measurement scales.
3. Operationalise accounting and finance concepts into defensible measures.
4. Evaluate reliability and validity of research instruments.
5. Calculate and interpret a basic sample size for a proportion.
6. Compare probability and non-probability sampling methods.
7. Design procedures for acquiring, cleaning, coding and documenting data.
8. Address missing data, outliers, confidentiality and research governance.
9. Critically assess the quality and limitations of primary and secondary data.

1. From Research Concepts to Variables

A concept is a broad idea, such as profitability, audit quality or investor confidence. A construct is a theoretically specified concept that may not be directly observable. For example, financial literacy is a construct inferred from knowledge, attitudes and behaviours. A variable is an attribute that can take different values across units of analysis. An observation is the recorded value of a variable for one unit, such as the leverage ratio of one listed company in one year.

The unit of analysis must be established before measurement begins. It may be an individual investor, firm, bank branch, annual report, transaction, country or firm-year observation. Confusing the unit of analysis creates serious errors. For example, a study of firms cannot claim that its findings directly describe individual managers unless individual-level evidence has also been collected.

A useful sequence is:

Research question → construct → operational definition → variable or indicator → instrument item or data field → recorded observation.

1.1 Variables in accounting and finance research

Common classifications include:

A variable can be treated differently depending on the research design. Firm size may be measured continuously using total assets, transformed using a logarithm, or grouped into size categories. The choice should be justified by theory, data quality and the planned analysis rather than made solely for convenience.

1.2 Operationalisation

Operationalisation specifies exactly how a construct will be observed. Consider the construct audit committee effectiveness. Possible indicators include the committee’s independence, financial expertise, meeting frequency and attendance. A composite index might assign one point for each criterion satisfied. Such an index is useful only if the criteria are theoretically defensible, consistently coded and transparently reported.

A variable definition should identify:

• name and label;
• conceptual meaning;
• unit of analysis;
• source;
• measurement scale and unit;
• coding rules;
• time period;
• treatment of missing or unavailable values;
• expected direction or relationship, where relevant.

A measurement matrix is a practical design tool.

For profitability, the ratio is:

where average total assets are commonly calculated as:

These measures assume that the accounting figures are comparable across firms and periods. Differences in accounting policies, fiscal years, inflation, mergers or exceptional items may weaken comparability.

2. Measurement Scales and Coding

Measurement scales determine what comparisons and statistical operations are meaningful.

• Nominal: categories without inherent order, such as industry or ownership type.
• Ordinal: ordered categories where distances are not necessarily equal, such as agreement from strongly disagree to strongly agree.
• Interval: numerical values with equal intervals but no meaningful absolute zero, such as a temperature scale.
• Ratio: equal intervals and a meaningful zero, such as revenue, assets or number of employees.

A numerical code does not automatically make a variable quantitative. Coding male as 1 and female as 2 creates labels, not a meaningful arithmetic scale. Similarly, a five-point agreement item is ordinal even though researchers sometimes calculate means for a multi-item scale. That practice requires justification and should be accompanied by appropriate robustness checks.

Coding should be mutually exclusive and collectively exhaustive wherever possible. For example, an industry classification must not assign one company to two incompatible categories, and the categories should include an “other” or “not classified” option when justified. Codes for “don’t know,” “refused” and “not applicable” should be distinct from valid numerical responses and should not be silently treated as zero.

2.1 Composite measures and indices

A composite measure combines several indicators. Suppose financial literacy is measured by five scored questions, where each correct answer receives 1 and each incorrect answer receives 0. The unweighted score is:

where each xᵢ is 0 or 1 and L ranges from 0 to 5. A percentage score may be computed as:

Equal weighting is not automatically appropriate. Weighting should reflect theory, validated instruments or an explicitly defended methodological choice. Researchers should report the items, coding, handling of missing responses and evidence that the indicators represent the same construct.

3. Designing Measurement Instruments

The principal instruments used in accounting and finance research include questionnaires, interview schedules, observation protocols, coding sheets and data-extraction templates.

A well-designed questionnaire normally follows this order:

1. introduction, purpose and consent information;
2. eligibility screening;
3. central research questions;
4. sensitive questions, placed carefully;
5. demographic or organisational classification items;
6. closing message and contact information.

Questions should be clear, specific and relevant. Avoid double-barrelled questions such as “The finance department is efficient and ethical,” because respondents may agree with one part and reject the other. Avoid leading wording, unexplained technical terms, excessive recall periods and assumptions that respondents possess information they cannot reasonably know.

A Likert item should express one proposition. For example: “Management communicates financial risks clearly.” Response options may range from 1 = strongly disagree to 5 = strongly agree. If negatively worded items are used, reverse coding must be documented. For a five-point item with original score x, the reverse-coded score is:

This formula applies only to a 1–5 scale, where xᵣ is the reversed score. It must not be used unchanged for another scale.

3.1 Pilot testing

A pilot test evaluates comprehension, completion time, routing, response options and technical operation. Pilot participants should resemble the target population, although pilot observations are normally excluded from the final analysis if the instrument is changed substantially. Cognitive interviewing can reveal how respondents interpret terms such as “investment risk,” “independence” or “material information.”

3.2 Reliability

Reliability concerns consistency of measurement. It does not by itself establish that an instrument measures the intended construct.

For a multi-item scale, internal consistency is often assessed using Cronbach’s alpha:

where k is the number of items, sᵢ² is the variance of item i, and sₜ² is the variance of the total score formed by summing the items. Alpha assumes, among other conditions, that items measure a common underlying construct and that item errors are not problematically correlated. A high alpha can result from many repetitive items and is not proof of validity. Researchers should consider item content, dimensionality and, where appropriate, alternative reliability evidence such as test–retest reliability or omega.

Test–retest reliability examines stability by administering the same instrument on two occasions, provided the construct is expected to remain stable. Inter-rater reliability is important when annual reports, disclosures or interview transcripts are coded by more than one researcher. Agreement should be assessed using a suitable statistic rather than relying only on percentage agreement, particularly when category prevalence is unbalanced.

3.3 Validity

• Content validity: the instrument adequately covers the relevant domain.
• Face validity: items appear relevant and understandable, though this is weak evidence alone.
• Construct validity: the measure behaves as theory predicts. Convergent validity concerns association with related measures; discriminant validity concerns distinction from different constructs.
• Criterion validity: scores correspond to an accepted external criterion.
• Internal validity: the design supports credible conclusions about relationships within the study.
• External validity: findings can reasonably transfer to the intended population or contexts.

Validity is cumulative. A carefully worded questionnaire cannot repair a biased sample, and a representative sample cannot compensate for a measure that does not capture the construct.

4. Sampling: Population, Frame and Sample

The target population is the complete group about which the researcher wishes to draw conclusions. The sampling frame is the operational list or procedure from which units are selected. The sample is the subset actually studied. Frame coverage error occurs when eligible units are missing, duplicated or incorrectly included in the frame.

Sampling error is the difference between a sample estimate and the corresponding population value arising because only part of the population is observed. Non-sampling errors include non-response, inaccurate responses, interviewer effects, coding errors and coverage problems. Increasing sample size does not automatically eliminate non-sampling error.

4.1 Probability sampling

In probability sampling, every eligible unit has a known, non-zero probability of selection. This supports statistical inference when the design is properly implemented.

Simple random sampling gives each unit an equal selection probability. It requires a complete frame and is conceptually straightforward.

Systematic sampling selects every mth unit after a random start. If population size is N and desired sample size is n, the interval is commonly:

Systematic sampling can be efficient, but periodic patterns in the frame may create bias.

Stratified sampling divides the population into relatively homogeneous strata, such as industry or firm size, and samples within each stratum. Proportional allocation is:

where nₕ is the sample size in stratum h, n is the total sample size, Nₕ is the population size of stratum h and N is the total population size. Disproportionate allocation may be appropriate when small strata are substantively important, but weights may then be required for population estimates.

Cluster sampling selects naturally occurring groups, such as branches, regions or companies, and studies units within selected clusters. It may reduce cost but often increases sampling variance because units within clusters resemble one another.

Multistage sampling combines stages, for example selecting regions, then firms, then employees. Each stage must be documented, including selection probabilities and any weighting procedure.

4.2 Non-probability sampling

Non-probability samples do not provide known selection probabilities for all eligible units. They may be appropriate for exploratory, qualitative or hard-to-reach research, but claims of statistical generalisation must be cautious.

• Convenience sampling: accessible participants are selected.
• Purposive sampling: cases are selected because they possess relevant characteristics or information.
• Quota sampling: researchers fill specified category quotas without random selection within categories.
• Snowball sampling: participants recruit or identify other participants.
• Expert sampling: individuals with specialised knowledge are selected.

Non-probability sampling is not automatically poor research. Its credibility depends on a clear rationale, transparent inclusion criteria, attention to diversity and an appropriately limited claim. It is inappropriate to describe a convenience sample as representative merely because it is large.

4.3 Sample size for a proportion

For a large population and a simple random sample, an initial sample size for estimating a population proportion is:

where n₀ is the initial sample size, z is the standard-normal critical value for the desired confidence level, p is the anticipated population proportion, and e is the tolerated margin of error expressed as a proportion. The formula assumes probability sampling, an adequately specified frame, an independent response structure and the intended confidence interpretation.

When p is unknown, p = 0.5 is commonly used because it produces the largest value of p(1 − p) and therefore a conservative sample size. For a 95% confidence level, z is approximately 1.96. If e = 0.05 and p = 0.5:

The result is rounded upward to 385. This is the number of completed responses under the stated assumptions, not necessarily the number to invite. If the anticipated response rate is r, the number invited may be estimated as:

At a 60% response rate, approximately 385 ÷ 0.60 = 641.67, so at least 642 invitations would be needed, subject to practical and design considerations.

For a finite population of size N, a finite-population correction may be applied:

This formula is relevant when the sample is a substantial fraction of a known finite population. Sample size should also consider study design, subgroup analysis, expected effect size, statistical power, model complexity and available resources. A formula alone does not establish adequacy.

5. Acquiring Primary and Secondary Data

5.1 Primary data

Primary data are collected specifically for the current study. Methods include surveys, interviews, experiments, observations and researcher-administered assessments. Strengths include alignment with the research question and control over definitions. Limitations include cost, non-response, social desirability, recall error and researcher influence.

A primary-data protocol should specify recruitment, eligibility, consent, administration mode, storage, withdrawal procedures and quality checks. Response rates should be reported transparently. A high response rate does not prove representativeness, and a low response rate does not automatically invalidate a study; the pattern and consequences of non-response must be examined.

5.2 Secondary data

Secondary data were collected previously for another purpose. Sources include annual reports, regulatory filings, stock-exchange data, commercial databases, government statistics and institutional records. Researchers should assess:

• provenance and authority;
• definitions and accounting standards;
• completeness and missingness;
• time coverage and revisions;
• consistency across firms and years;
• treatment of restatements, mergers and currency changes;
• accessibility and licensing conditions;
• whether the source measures the construct required.

A reported number is not automatically comparable merely because it has the same label. For example, “operating income” may be defined differently across databases or reporting regimes. A data dictionary should record the original field name, transformed field, source page or table, extraction date and transformation rule.

5.3 Data extraction and audit trails

For document-based research, use a structured extraction form. Each observation should be traceable to a source location. Store the raw file separately from the cleaned dataset. Maintain a change log recording who made a change, when, why and how it was verified. Version control is especially useful for code, questionnaires and data dictionaries.

6. Data Cleaning and Preparation

Cleaning is not an opportunity to remove inconvenient observations. It is a transparent process of identifying errors and applying pre-specified rules.

A defensible workflow is:

1. preserve an immutable raw copy;
2. inspect variable names, types and ranges;
3. verify identifiers and remove only documented duplicates;
4. check coding and units;
5. identify missing, impossible and inconsistent values;
6. investigate outliers against original sources;
7. apply transformations with a recorded rationale;
8. generate descriptive summaries and validation reports;
9. save the cleaned file and syntax or code used;
10. protect the final dataset and document exclusions.

6.1 Missing data

Missingness may be classified as:

• MCAR: missing completely at random;
• MAR: missingness depends on observed information after relevant variables are considered;
• MNAR: missingness depends partly on unobserved or missing values themselves.

These are assumptions about the missingness mechanism, not labels that can be established merely by looking at blank cells. Deleting every record with a missing value may reduce statistical power and bias estimates. Options include complete-case analysis, available-case analysis, single imputation and multiple imputation. The appropriate choice depends on the design, variables, missingness pattern and analysis model. The method and sensitivity analysis should be reported.

6.2 Outliers and transformations

An outlier may be a data-entry error, a legitimate extreme observation or evidence of an unusual event. It should be investigated rather than automatically deleted. Possible responses include correction from the original source, retaining and analysing it, using a justified transformation, robust methods or a pre-specified winsorisation rule. Any alteration must be disclosed.

Financial variables are often right-skewed. A logarithmic transformation may be used when theoretically and statistically appropriate. If x is positive, the natural logarithm is written ln(x). Zero and negative values require a carefully justified alternative; adding an arbitrary constant can alter interpretation and should not be done silently.

7. Data Governance, Ethics and Security

Research data management protects participants, institutions and the integrity of the project. Core principles include lawfulness, fairness, purpose limitation, data minimisation, accuracy, storage limitation, integrity, confidentiality and accountability. Applicable institutional policies and data-protection laws must be followed; requirements vary by jurisdiction.

Informed consent should explain the purpose, procedures, risks, benefits, voluntary nature of participation, withdrawal arrangements, data recipients and retention period. Confidentiality means limiting access and protecting identity; anonymity means identity cannot reasonably be linked to the data. A promise of anonymity is inappropriate if identifiers are retained.

Security controls include:

• role-based access and strong authentication;
• encryption in transit and at rest where appropriate;
• pseudonymisation and separate storage of the re-identification key;
• secure institutional storage rather than personal devices;
• regular backups and recovery testing;
• retention and secure deletion schedules;
• incident reporting procedures;
• restrictions on exporting confidential data to public AI tools.

Generative AI may assist with questionnaire wording, code explanation or data-management templates, but confidential raw data should not be entered into an external system without explicit institutional approval. AI-generated classifications, transcriptions or synthetic values require human verification and documentation. AI does not replace researcher responsibility for accuracy, consent, privacy or reproducibility.

8. Illustrative Case: A Study of Audit Committee Effectiveness

Background

A student proposes to examine whether audit committee effectiveness is associated with the quality of financial reporting among listed companies. The intended population is all non-financial companies listed on a national stock exchange during 2022–2024.

Problem

The initial proposal defines effectiveness as “a strong audit committee” and plans to survey available finance managers. This definition is vague, the sample is potentially biased, and the proposed survey does not match the firm-level unit of analysis.

Analysis

The researcher refines the construct using observable indicators: committee independence, financial expertise, meeting frequency and attendance. The unit becomes the firm-year. Annual reports provide the committee variables, while audited financial statements provide reporting measures. The researcher recognises that publicly available reports may omit attendance data and that reporting quality is not identical to the absence of an audit qualification.

Approach and solution

A population list is constructed from exchange records. Firms with incomplete listing histories are retained only for the years in which they meet the eligibility criteria. A coding manual defines each indicator and specifies how ambiguous disclosures are handled. Two coders independently code a pilot set of reports, discuss disagreements and revise the manual before full coding.

If the researcher wants broad population inference, stratified probability sampling by industry and firm size is considered. If the population is small enough, a census of eligible firm-years is preferred. Source documents are downloaded, named consistently and stored in a read-only raw folder. A separate extraction file records company identifier, year, page number, original wording, coded value and coder initials.

Outcome

The revised design improves construct clarity, aligns the unit of analysis with the research question, makes source verification possible and reduces inconsistent coding. It does not eliminate limitations: disclosure quality may itself be related to governance, missing information may be systematic, and associations cannot automatically be interpreted as causal effects.

Learning points

1. A broad concept must be converted into explicit indicators.
2. The sampling unit and unit of analysis must be compatible.
3. A coding manual is an instrument and requires pilot testing.
4. Secondary data require provenance and comparability checks.
5. Transparent limitations strengthen, rather than weaken, a dissertation.

9. Best-Practice Checklist

Before collecting data, confirm that:

• the research question, unit of analysis and population are aligned;
• each major construct has a defensible operational definition;
• the instrument has been reviewed and piloted;
• sampling eligibility and selection procedures are documented;
• consent and ethical approval requirements are satisfied;
• variables, codes, units and missing-value rules are listed in a data dictionary;
• raw data will be preserved separately from processed data;
• quality checks and source verification procedures are specified;
• access, retention, backup and deletion arrangements are secure;
• the final report will disclose exclusions, transformations and limitations.

Chapter Summary

Reliable accounting and finance research begins with a clear chain from construct to operational measure and recorded observation. Variables must be defined in terms of their meaning, unit, scale, source and coding. Instruments require attention to wording, pilot testing, reliability and several forms of validity. Probability sampling supports defensible population inference when the frame and selection process are sound; non-probability sampling can be valuable when its purpose and limits are explicit.

Sample-size formulas provide planning guidance under stated assumptions, but design effects, non-response, subgroup analysis and non-sampling error must also be considered. Primary and secondary data require different quality assessments, and all datasets should be preserved, cleaned transparently and documented through a data dictionary and audit trail. Missing values and outliers should be investigated rather than mechanically deleted. Finally, ethical governance, privacy, secure storage and responsible use of technology are integral parts of research quality, not administrative afterthoughts.

Key Terms

Concept: A broad idea examined in research.

Construct: A theoretically specified idea that may not be directly observable.

Variable: An attribute that can take different values across observations.

Indicator: An observable measure used to represent a construct.

Operationalisation: Defining how a construct will be measured.

Unit of analysis: The entity about which conclusions are drawn.

Measurement scale: The classification of values as nominal, ordinal, interval or ratio.

Reliability: Consistency or stability of measurement.

Validity: The extent to which a measure supports the intended interpretation.

Sampling frame: The list or procedure from which a sample is selected.

Probability sampling: Sampling in which eligible units have known, non-zero selection probabilities.

Non-probability sampling: Sampling without known selection probabilities for all eligible units.

Sampling error: Error arising because a sample rather than the full population is observed.

Non-response bias: Bias caused when respondents differ systematically from non-respondents.

Composite measure: A score formed by combining several indicators.

Data dictionary: A document defining variables, codes, units, sources and rules.

Missing data: Values not recorded or unavailable for an observation.

Outlier: An unusually extreme observation relative to the data or model.

Pseudonymisation: Replacing direct identifiers while retaining a separately protected re-identification possibility.

Audit trail: A documented record of data sources, changes, decisions and processing steps.

Review Questions

Short Answer Questions

1. Distinguish a construct, variable and indicator, using an accounting example.
2. What is the difference between a sampling frame and a target population?
3. State two reasons why a high Cronbach’s alpha does not prove validity.
4. Why should “refused” and “not applicable” be coded separately from zero?
5. Identify two security controls suitable for confidential dissertation data.

Long Answer Questions

1. Explain how a researcher could operationalise audit committee effectiveness and evaluate the resulting measures.
2. Compare simple random, stratified, cluster and systematic sampling, including one advantage and limitation of each.
3. Discuss the differences between reliability, content validity, construct validity and external validity.
4. Explain a complete workflow for acquiring, cleaning, documenting and storing secondary financial data.
5. Discuss ethical and governance issues arising when researchers use confidential financial or personal data.

Analytical / Application Questions

1. A researcher wants 95% confidence and a 5% margin of error for a population proportion, with no prior estimate of p. Calculate the initial sample size and explain why p = 0.5 is used.
2. A questionnaire measures investor confidence using six items, but two items are negatively worded and 18% of responses are missing on one item. Design a procedure for reverse coding, reliability assessment and missing-data analysis.
3. A database reports revenue in local currency for firms operating in different countries and years. Identify at least five comparability checks required before using the variable in a panel analysis.

Multiple Choice Questions with Answers

Question 1

Which statement best describes operationalisation?

A. Selecting participants without a sampling frame

B. Defining how an abstract construct will be measured

C. Deleting observations with missing values

D. Calculating the response rate

Correct Answer: B. Defining how an abstract construct will be measured

Question 2

A variable identifying whether a company is family-owned or non-family-owned is measured on which scale?

A. Nominal

B. Ordinal

C. Interval

D. Ratio

Correct Answer: A. Nominal

Question 3

What is the principal purpose of pilot testing a questionnaire?

A. To guarantee a representative sample

B. To eliminate all sampling error

C. To identify comprehension, routing and measurement problems

D. To replace ethical approval

Correct Answer: C. To identify comprehension, routing and measurement problems

Question 4

In the large-population proportion formula, using p = 0.5 when no prior estimate exists is conservative because it:

A. Produces the smallest possible sample

B. Maximises p × (1 − p)

C. Eliminates non-response bias

D. Makes the population finite

Correct Answer: B. Maximises p × (1 − p)

Question 5

Which sampling method divides a population into groups and samples within each group?

A. Snowball sampling

B. Convenience sampling

C. Stratified sampling

D. Cluster-free sampling

Correct Answer: C. Stratified sampling

Question 6

A sampling frame that omits small firms creates which problem?

A. Coverage error

B. Reverse coding

C. Internal consistency

D. Criterion validity

Correct Answer: A. Coverage error

Question 7

Which statement about Cronbach’s alpha is most accurate?

A. It proves that a scale is valid

B. It assesses one aspect of internal consistency

C. It is used only for nominal variables

D. It measures population coverage

Correct Answer: B. It assesses one aspect of internal consistency

Question 8

What is the most defensible first response to an extreme financial observation?

A. Delete it automatically

B. Replace it with the mean

C. Investigate it against the original source

D. Hide it from the data dictionary

Correct Answer: C. Investigate it against the original source

Question 9

Which practice best preserves reproducibility?

A. Overwriting the raw dataset after cleaning

B. Keeping a raw copy, processing code and change log

C. Relying on memory for coding decisions

D. Reporting only the final sample size

Correct Answer: B. Keeping a raw copy, processing code and change log

Question 10

At a 60% response rate, approximately how many invitations are needed to obtain 385 completed responses?

A. 231

B. 385

C. 542

D. 642

Correct Answer: D. 642