Surveys, Secondary Data and Digital Evidence

Introduction

Business research rarely depends on a single source of evidence. Managers may need to understand customer preferences through a survey, examine historical sales records, compare performance with government statistics, and interpret digital traces such as search activity or online reviews. Each source can be useful, but none is automatically trustworthy or suitable for every research question.

This chapter develops a practical framework for collecting primary survey data and evaluating secondary and digital evidence. The central managerial issue is fitness for purpose: whether a source is sufficiently relevant, accurate, timely, complete, credible, ethically obtained and well documented for the decision at hand. At master's level, researchers must move beyond asking whether data are available. They must ask how the data were generated, whose interests shaped them, what population they represent, what errors they contain and whether the evidence supports the intended inference.

Learning objectives

By the end of this chapter, readers should be able to:

1. Design and administer a survey aligned with a research question.
2. Distinguish primary, secondary, administrative, open and digital-trace data.
3. Evaluate data sources for relevance, accuracy, provenance, completeness, timeliness and fitness for purpose.
4. Critically assess organizational databases, open data, social-media data and other digital evidence.
5. Integrate heterogeneous sources while recognizing coverage, measurement, selection and ethical risks.
6. Make defensible managerial decisions about data acquisition, documentation and use.

1. Evidence architecture in business research

Primary and secondary data

Primary data are collected specifically for the present investigation. A customer-satisfaction survey, interview study or experiment is primary research when the researcher defines the population, measures and collection process. Primary data provide control over operationalization and sampling, but they require time, money and careful fieldwork.

Secondary data were originally collected for another purpose but are reused for the present study. Examples include annual reports, industry statistics, census data, prior surveys and transaction records. Secondary data can be economical and longitudinal, but their definitions, coding and sampling decisions may not match the new research question.

The distinction concerns purpose, not format. A spreadsheet can contain primary or secondary data; so can a database, text archive or dashboard. A firm's customer database is usually secondary for a researcher studying churn, even though the firm originally collected the records as part of routine operations.

A source-to-inference chain

Evidence should be evaluated as a chain:

Figure: Every managerial inference depends on choices made before analysis.

Errors can enter at each link. A vague construct produces poor measurement; a convenience sample produces coverage error; a database may contain duplicate accounts; an algorithm may convert activity into a biased proxy. Thus, sophisticated statistical analysis cannot repair fundamentally unsuitable evidence.

2. Designing primary survey research

A survey is a systematic procedure for asking standardized questions of a defined set of respondents. It is especially useful for measuring attitudes, perceptions, intentions, self-reported behavior and characteristics that are not available in organizational records.

From construct to questionnaire

Survey design begins with the research question and constructs. A construct such as perceived service quality must be translated into observable indicators. A good operational definition identifies what will be measured, for whom, during what period and using what response scale.

A questionnaire normally contains: an introduction and consent statement; screening questions; substantive measures; demographic or classification items; and a closing message. Questions should be specific, neutral and comprehensible. Double-barrelled questions, such as “How satisfied are you with price and delivery?”, prevent respondents from expressing different views about the two elements. Leading questions, undefined technical terms, excessive recall periods and socially sensitive wording can create measurement error.

Response scales must match the construct. A frequency question may use “never” to “daily,” whereas agreement with a statement may use a balanced Likert-type scale. The researcher should specify whether scale points are ordered categories or treated as approximately interval measures during analysis. “Not applicable” and “don’t know” should be available when substantively meaningful, rather than forcing an arbitrary answer.

A multi-item scale should normally be based on a theoretical definition, reviewed for content validity and piloted with members of the target population. Internal consistency can be summarized using Cronbach’s alpha, but a high value does not prove validity and can result from redundant items. Construct validity also requires considering convergent evidence, discriminant evidence and the possibility that respondents interpret items differently.

Sampling and fieldwork

The target population is the complete group to which the study intends to generalize. The sampling frame is the operational list or mechanism from which units are selected. A gap between the two creates coverage error. Probability sampling methods—simple random, systematic, stratified and cluster sampling—give a defensible basis for generalization when selection probabilities are known. Non-probability methods—convenience, purposive, quota and snowball sampling—may be appropriate for exploratory or hard-to-reach populations, but they limit statistical generalization.

Sample size is not determined by a universal rule. It depends on the precision required, population variability, design, expected response rate, subgroup analysis and study resources. For a simple proportion estimate under a large-population approximation, a commonly used planning expression is:

Here, n₀ is the initial required sample size, z is the standard-normal critical value for the desired confidence level, p is the anticipated population proportion and e is the tolerated margin of sampling error. The expression assumes a probability sample, independent observations and a proportion estimate. When p is unknown, 0.5 gives the largest variance and therefore a conservative sample size. For a finite population of size N, a finite-population adjustment may be used:

These formulas address sampling precision, not nonresponse bias, poor questions or coverage error. If an anticipated response rate is r, the number invited should be approximately n/r, with r expressed as a proportion. For example, a required 400 completed questionnaires and an anticipated response rate of 0.40 imply 1,000 invitations. This is a planning calculation, not a guarantee.

Fieldwork protocols should specify recruitment, reminders, eligibility, duplicate prevention, interviewer training and treatment of partial responses. Pilot testing should examine completion time, comprehension, missingness, routing logic and technical performance. Researchers should retain a version-controlled questionnaire and fieldwork log.

Survey quality and nonresponse

Survey quality involves total survey error rather than response rate alone. Important components include coverage error, sampling error, nonresponse error, measurement error, processing error and specification error. A high response rate can still produce biased results if the questions are poor or the sampling frame excludes an important group. Conversely, a lower response rate may be less damaging when respondents and nonrespondents are demonstrably similar on relevant characteristics.

Researchers should compare early and late respondents where appropriate, examine response patterns, report the denominator and document recruitment. Weighting can adjust unequal selection probabilities or known demographic differences, but it depends on credible auxiliary information and cannot fully correct unobserved differences.

3. Evaluating secondary and organizational data

Secondary research begins with source discovery and metadata, not with downloading a file. The researcher should identify the original purpose, population, unit of analysis, collection period, definitions, transformations, missing-value codes, revision history and access restrictions.

Organizational databases

Organizational data include enterprise-resource-planning records, customer-relationship-management systems, point-of-sale data, web analytics, call-centre logs, accounting systems and human-resource records. Their advantages are scale, operational relevance and the possibility of longitudinal analysis. Their limitations include changing business rules, inconsistent data entry, duplicated entities, missing fields, system migration and incentives that shape what gets recorded.

A database record is not necessarily a direct observation of the underlying construct. “Customer churn” may be inferred from an account-closure code; “engagement” may be represented by clicks; “employee performance” may be represented by manager ratings. Researchers must inspect data dictionaries and understand the data-generating process.

A practical audit includes: defining the unit of analysis; checking uniqueness of identifiers; profiling ranges and distributions; reconciling totals with authoritative reports; tracing transformations through an extract-transform-load pipeline; documenting missingness; and testing whether historical definitions changed. A customer table and transaction table, for example, may require a one-to-many join. An incorrect join can multiply revenue and produce apparently precise but false results.

Open data

Open data are data made available for public access and reuse, often by governments, international organizations, research institutions or businesses. They can support benchmarking, market analysis, policy evaluation and location decisions. Examples include demographic indicators, transport flows, business registrations, economic statistics and environmental measures.

Openness does not guarantee quality. A dataset may be technically downloadable but poorly documented, revised without notice, aggregated too coarsely or subject to political and institutional measurement choices. Researchers should check the publisher’s authority, methodology, geographic and temporal coverage, licensing conditions, version date, revision policy, suppression rules and reproducibility of downloads. Official statistics may be highly credible while still reflecting definitions that differ from a firm's managerial concept.

Social-media data

Social-media evidence includes posts, comments, reactions, public profiles, network connections and platform-generated engagement metrics. It can provide rapid insight into discourse, complaints, brand associations and emerging issues. However, users are not a representative sample of customers, platform algorithms shape visibility, automated accounts may distort activity, and deleted or private content is not necessarily observable.

The meaning of a post is contextual. A positive reaction may indicate agreement, irony, politeness or low-effort interaction. Sentiment-analysis systems can misclassify dialect, sarcasm and multilingual content. Social-media metrics should therefore be treated as indicators of platform activity or expressed discourse, not direct measures of population attitudes unless validated against an appropriate benchmark.

Digital-trace data

Digital-trace data are records generated as people, devices or systems interact with digital services. Examples include clickstreams, search queries, GPS traces, app events, sensor readings, digital payments and server logs. They are often passive, high-volume and time-stamped, enabling behavioral and near-real-time analysis.

Their principal strength—observing behavior rather than asking about it—also creates interpretive risks. A click may reflect curiosity rather than purchase intention; a location ping may represent a device rather than a person; absence of a log may mean a technical failure rather than absence of behavior. Digital traces are shaped by interface design, tracking permissions, device ownership, platform access and algorithmic recommendation. They commonly measure activity proxies rather than psychological constructs.

4. A framework for source evaluation

A source should be evaluated against the intended inference, not ranked absolutely. The following dimensions are particularly important.

Provenance means the documented history of a dataset: origin, collection process, transformations, custodians, versions and uses. Good provenance permits an independent researcher to understand how a value came to exist and to reproduce or challenge the processing. A dashboard without a visible source, definition or update date has weak provenance even if it looks authoritative.

Accuracy differs from precision. A system may record values consistently but systematically measure the wrong thing. Completeness is not simply the percentage of non-missing cells; a dataset can be complete at the record level yet omit an entire customer segment. Timeliness is decision-dependent: a five-year historical series may be ideal for trend analysis but unsuitable for pricing today.

A source assessment should produce a written decision: use as primary evidence, use for triangulation, use only descriptively, require adjustment or reject. Triangulation means comparing different sources, methods or perspectives. Agreement increases confidence only when sources are genuinely independent; two dashboards copied from the same original file do not provide independent confirmation.

5. Integration, analysis and responsible use

Combining sources requires harmonization. Researchers should align identifiers, units, time zones, geographic boundaries, category definitions and observation periods. A data dictionary should record original and harmonized variables. Any imputation, deduplication, filtering, weighting or aggregation must be logged.

Linkage across datasets can increase analytical value but also increase re-identification risk. Deterministic linkage uses an exact identifier; probabilistic linkage estimates whether records refer to the same entity based on partially matching fields. False matches and missed matches can both bias results. Researchers should validate linkage quality on a sample and avoid retaining unnecessary identifiers.

Digital and organizational data often create missing-not-at-random problems. For instance, complaint data contain more information about people who complain than those who remain silent. A model trained on such data may predict complaint behavior rather than dissatisfaction in the whole customer base. Managers should distinguish descriptive association from causal evidence. A rise in searches after an advertising campaign may be consistent with an effect, but seasonality, competitors and concurrent events may provide alternative explanations.

Ethical governance includes informed consent where required, purpose limitation, data minimization, confidentiality, secure storage, controlled access and transparent communication. Public availability does not automatically make collection ethically harmless. Sensitive location, health, financial or employment data require heightened safeguards. Researchers should assess legal requirements such as data-protection law, contractual platform restrictions, copyright and sector-specific regulation. Automated sentiment or risk scoring should be tested for disparate error rates and should not be treated as neutral merely because it is computational.

Artificial intelligence can assist with transcription, classification, anomaly detection and source discovery, but generated labels and summaries are themselves data products requiring validation. A responsible workflow retains the original evidence, documents model versions and prompts where relevant, tests performance on representative cases and provides human review for consequential decisions. AI-generated interpretations should not replace provenance or methodological judgment.

6. Illustrative case: assessing evidence for a loyalty-program decision

Background and problem

This is an illustrative case. A national retailer observes declining repeat purchases and considers redesigning its loyalty programme. The research team has four sources: a customer survey, CRM transactions, public census data and social-media comments. Executives request a single “customer loyalty score.”

Analysis

The survey measures satisfaction, switching intentions and reasons for inactivity, but the invitation is sent only to customers with valid email addresses. The team therefore recognizes coverage and nonresponse risks. CRM data contain three years of transactions, but a membership-system change created duplicate customer IDs during the first six months. Public census data provide neighborhood income and age distributions, useful for market context but not direct measures of individual loyalty. Social-media comments reveal complaints about rewards expiry, but contributors are self-selected and platform users do not represent all customers.

The team audits the CRM by reconciling monthly revenue totals with finance reports and creates a crosswalk for the ID migration. It defines repeat purchase consistently as at least two completed transactions in a 90-day period, rather than using an unstable “active customer” flag. The survey is piloted, stratified by recent purchase frequency and weighted cautiously to known customer characteristics. Social-media material is coded for themes and used diagnostically, not as a population estimate.

Approach and outcome

Instead of a single unqualified score, the team produces an evidence matrix. Transaction data establish observed purchasing patterns; the survey explains attitudes and barriers; census data support geographic segmentation; social-media analysis identifies emerging complaints. The retailer discovers that reward expiry is prominent online and in survey responses, but the strongest behavioral predictor of inactivity is a complex redemption process. It simplifies redemption and tests the change in selected stores using a pre-specified evaluation plan.

Learning points

The case demonstrates that triangulation is more valuable than forced aggregation. Each source answers a different question, and the quality judgment depends on the inference. The team also avoids treating an accessible digital signal as representative, repairs documented database problems before analysis, and links managerial action to evidence that can actually test the proposed intervention.

7. Best-practice checklist

Before collecting or using data, researchers should:

• define the decision, population, construct and intended inference;
• document why each source is relevant;
• obtain a codebook, provenance record and licence or consent basis;
• pilot surveys and test data extraction procedures;
• inspect coverage, missingness, duplicates, outliers and definition changes;
• separate observed behavior from inferred attitudes or intentions;
• harmonize variables and preserve raw data;
• triangulate without assuming non-independent sources confirm one another;
• report limitations and uncertainty to decision-makers;
• apply security, privacy, fairness and retention controls; and
• maintain a reproducible research log with versions, transformations and decisions.

Chapter Summary

Business research combines primary survey data with secondary, organizational, open and digital evidence. Surveys offer control over measurement but are vulnerable to coverage, nonresponse and measurement error. Organizational databases provide scale and operational detail, yet their fields are products of business processes and may change over time. Open data can support benchmarking but require scrutiny of definitions, revisions, licensing and institutional provenance. Social-media and digital-trace data can reveal timely behavior and discourse, but they are shaped by platform access, algorithms, self-selection and imperfect proxies.

A rigorous researcher evaluates relevance, accuracy, provenance, completeness, timeliness, consistency, credibility, legality and overall fitness for purpose. Sound practice includes metadata review, pilot testing, database auditing, harmonization, triangulation, transparent documentation and ethical governance. The aim is not to find a universally best source but to match evidence to an inference while making uncertainty visible.

Key Terms

Primary data: Data collected specifically for the current investigation.

Secondary data: Data originally collected for another purpose and reused.

Survey: A standardized procedure for collecting information from respondents.

Target population: The complete group to which a study intends to generalize.

Sampling frame: The operational list or mechanism from which a sample is selected.

Coverage error: Bias caused when the sampling frame excludes or inadequately represents the target population.

Nonresponse error: Bias arising when those who respond differ systematically from those who do not.

Measurement error: Difference between the measured value and the intended construct.

Organizational database: A structured operational repository maintained by an organization.

Open data: Data made available for public access and reuse under stated conditions.

Social-media data: Posts, interactions, profiles or metrics generated on social platforms.

Digital-trace data: Records produced through interactions with digital systems or devices.

Provenance: The documented origin, custody, transformation and version history of data.

Accuracy: The degree to which data represent the phenomenon they are intended to measure.

Completeness: The extent to which required units, variables and periods are present.

Timeliness: The suitability of data’s recency for the decision or research purpose.

Triangulation: Comparison of evidence from different sources, methods or perspectives.

Fitness for purpose: Overall suitability of evidence for a specified inference or decision.

Data minimization: Collecting and retaining only data necessary for a legitimate purpose.

Review Questions

Short Answer Questions

1. Distinguish primary data from secondary data and give one business example of each.
2. What is the difference between a target population and a sampling frame?
3. Why does public availability not guarantee that social-media data are representative?
4. Define provenance and identify two items that a provenance record should contain.
5. Name four dimensions used to evaluate fitness for purpose.

Long Answer Questions

1. Explain the principal sources of error in survey research and discuss how a researcher can reduce them.
2. Critically evaluate the strengths and weaknesses of organizational databases for customer research.
3. Compare open data, social-media data and digital-trace data as sources of managerial evidence.
4. Explain why a high response rate does not by itself establish survey quality.
5. Discuss the ethical and governance requirements for linking organizational and digital-trace data.

Analytical / Application Questions

1. A retailer has a customer database with changing customer IDs, a voluntary online survey and public neighborhood statistics. Design an evidence-evaluation plan that addresses provenance, linkage, coverage and fitness for purpose.
2. A firm observes that online searches for its product increased after an advertising campaign. Identify at least three alternative explanations and propose additional evidence needed before claiming a causal effect.
3. A survey requires 400 completed responses and the expected response rate is 40%. Calculate the number of invitations required, state the assumption, and explain why this calculation does not eliminate nonresponse bias.

Multiple Choice Questions with Answers

Question 1

Which statement best defines secondary data?

A. Data collected specifically for the current research question

B. Data originally collected for another purpose and reused

C. Data collected only through interviews

D. Data that have no documented source

Correct Answer: B. Data originally collected for another purpose and reused

Question 2

What is the main purpose of a sampling frame?

A. To provide the operational basis for selecting units

B. To calculate a respondent’s attitude score

C. To replace the research question

D. To guarantee a 100% response rate

Correct Answer: A. To provide the operational basis for selecting units

Question 3

A database field labelled “active customer” changes definition after a system migration. Which quality issue is most directly indicated?

A. Improved representativeness

B. Definition inconsistency

C. Random sampling error

D. Increased survey validity

Correct Answer: B. Definition inconsistency

Question 4

Which feature is central to data provenance?

A. The visual attractiveness of a dashboard

B. The number of decimal places displayed

C. The history of data origin and transformations

D. The absence of all missing values

Correct Answer: C. The history of data origin and transformations

Question 5

Why should social-media engagement not automatically be treated as population opinion?

A. All social-media data are necessarily inaccurate

B. Platform users and visible activity may be systematically selective

C. Social-media data cannot contain timestamps

D. Engagement metrics are always qualitative

Correct Answer: B. Platform users and visible activity may be systematically selective

Question 6

A study needs 400 completed surveys and expects a 0.40 response rate. Approximately how many invitations are required?

A. 160

B. 400

C. 800

D. 1,000

Correct Answer: D. 1,000

Question 7

Which statement about the sample-size formula for a population proportion is correct?

A. It automatically corrects measurement error

B. It assumes a probability-sampling context for its usual precision interpretation

C. It guarantees that nonrespondents resemble respondents

D. It makes the sampling frame unnecessary

Correct Answer: B. It assumes a probability-sampling context for its usual precision interpretation

Question 8

What is triangulation?

A. Deleting all sources except the largest dataset

B. Comparing evidence from different sources or methods

C. Converting every variable to a monetary value

D. Using only social-media observations

Correct Answer: B. Comparing evidence from different sources or methods

Question 9

Which example best illustrates a digital-trace proxy problem?

A. A purchase transaction directly records the purchased item

B. A click is interpreted as definite purchase intention

C. A codebook documents variable definitions

D. A survey includes a consent statement

Correct Answer: B. A click is interpreted as definite purchase intention

Question 10

Which practice best supports responsible use of linked customer and digital data?

A. Retaining every identifier indefinitely

B. Ignoring platform terms because data are visible

C. Applying purpose limitation, access controls and documented linkage checks

D. Treating an algorithmic score as unbiased by default

Correct Answer: C. Applying purpose limitation, access controls and documented linkage checks