Data Preparation, Analysis and Insight Communication

Introduction

Marketing research creates value only when raw responses are converted into trustworthy evidence and then into decisions. A questionnaire may produce thousands of records, but unprocessed records are not yet information. They may contain missing answers, inconsistent coding, duplicate cases, extreme values, careless responses and measurement errors. Analysis is therefore not merely the mechanical production of tables and charts. It is a disciplined process of checking data quality, selecting appropriate methods, interpreting uncertainty and communicating implications without overstating what the evidence supports.

For product managers, this process connects customer evidence to decisions such as segment selection, feature prioritisation, pricing, positioning, campaign evaluation and retention planning. The central principle is traceability: every reported insight should be traceable to a defined question, a documented dataset, an appropriate analytical method and a defensible interpretation.

Learning objectives

By the end of this chapter, you should be able to:

• prepare, code, clean and document marketing research data;
• calculate and interpret descriptive statistics and cross-tabulations;
• distinguish association, correlation and causation;
• apply and interpret hypothesis tests, confidence intervals and effect sizes;
• explain correlation and simple regression, including their assumptions;
• select visualisations that match data types and analytical purposes;
• convert statistical findings into actionable product and advertising recommendations;
• design a clear research report or dashboard while protecting respondents’ rights.

1. From raw responses to analysis-ready data

A dataset is a structured representation of observations. In a customer survey, a row commonly represents one respondent and a column represents a variable. A variable may be categorical, ordinal, interval-like or ratio-scaled, although the measurement scale should be determined by how the construct was measured rather than by the appearance of the numbers.

1.1 The data preparation workflow

A professional workflow usually follows these stages:

1. Define the unit of analysis. Decide whether the record represents a person, household, transaction, advertisement impression, product review or account. Mixing units can produce invalid conclusions.
2. Create a data dictionary. Record variable name, question wording, response options, coding, measurement level, missing-value codes and permitted range.
3. Import and preserve the source. Keep a read-only copy of the original file and perform cleaning on a documented working copy.
4. Check structural integrity. Confirm row counts, column names, data types, IDs and whether records were truncated or duplicated during export.
5. Validate values and logic. Check ranges, skip patterns and internal consistency. For example, a respondent who reports never using an app should not have a weekly app-use frequency of 20.
6. Handle missing and invalid data. Distinguish a true non-response, “not applicable,” “prefer not to say” and an invalid response.
7. Code variables carefully. Use meaningful labels and retain the original wording. Reverse-code negatively worded scale items before constructing a composite score.
8. Identify duplicates and unusual cases. Duplicate records may arise from repeated submissions; unusual values may be genuine or erroneous.
9. Construct derived variables. Examples include total satisfaction, usage frequency bands, campaign-exposed status and customer lifetime-value categories. Document every transformation.
10. Freeze and audit the analysis file. Record the cleaning date, analyst, exclusions and software or code version.

1.2 Missing data

Missingness should be analysed, not silently erased. A useful first report gives both the number and percentage missing for each important variable. The percentage missing for variable X is:

The denominator should be the number of eligible cases, not necessarily the total sample, because skip logic may legitimately make a question inapplicable.

Three conceptual mechanisms are commonly discussed. MCAR (missing completely at random) means missingness is unrelated to observed or unobserved values. MAR (missing at random) means missingness can be explained by observed variables. MNAR (missing not at random) means missingness remains related to the unobserved value itself. These mechanisms cannot always be proven from the dataset, so the analyst should conduct sensitivity checks and state assumptions.

Simple deletion can reduce sample size and introduce bias. Mean imputation is easy but reduces variation and can distort relationships. More advanced approaches, such as multiple imputation, may be appropriate when missingness is substantial and assumptions are defensible. For a diploma-level project, a transparent approach—reporting the extent of missingness, using a justified rule and comparing results with and without imputation—is often more valuable than an unexplained sophisticated method.

1.3 Outliers and data quality

An outlier is an observation unusually distant from the rest of the data. It may indicate an entry error, an unusual but valid customer, fraud, bot activity or a special market segment. Do not delete an outlier merely because it weakens a desired result. First verify its source, compare it with the research design and conduct an analysis with and without it. Report the effect of the decision.

For a quantitative variable, the interquartile range is:

where Q₁ is the first quartile and Q₃ is the third quartile. A common exploratory flag is a value below Q₁ − 1.5×IQR or above Q₃ + 1.5×IQR. This is a screening rule, not proof that a case is erroneous.

2. Descriptive statistics: describing the evidence

Descriptive statistics summarise what was observed. They do not by themselves establish generalisability, causation or statistical significance.

2.1 Frequencies and proportions

For a category with frequency f and total valid frequency n, the percentage is:

Always state whether percentages use all cases, valid cases or weighted cases. Percentages in a table should normally sum to approximately 100%; small differences may result from rounding.

2.2 Measures of central tendency

The arithmetic mean is:

where xᵢ is observation i and n is the number of observations. The mean is useful for quantitative variables and approximately interval-level scales, but it is sensitive to extreme values.

The median is the middle ordered observation, or the average of the two middle observations when n is even. It is more resistant to skew and is often preferable for income, spending and waiting time. The mode is the most frequently occurring value and is especially useful for nominal categories.

For Likert-type items, reporting the distribution across response categories is often more informative than reporting only a mean. A mean satisfaction score of 3.4 could represent moderate agreement by everyone or a polarised split between very dissatisfied and very satisfied customers.

2.3 Dispersion

The sample variance and sample standard deviation are:

The n − 1 denominator is used when estimating population variability from a sample. Standard deviation is expressed in the same units as the variable. A large standard deviation means observations are widely dispersed around the mean; it does not necessarily indicate measurement error.

The coefficient of variation is sometimes used for positive ratio-scale variables:

It is dimensionless, but it is not appropriate when the mean is zero or near zero, or when zero has no meaningful interpretation.

Worked example: descriptive summary

Suppose five customers spend 10, 12, 13, 15 and 20 currency units. The mean is 14. The deviations from the mean are −4, −2, −1, 1 and 6; their squared deviations sum to 58. Thus the sample variance is 58 ÷ 4 = 14.5 and the sample standard deviation is approximately 3.81 currency units. The median is 13. Because the mean exceeds the median, the largest value is pulling the average upward. A report should therefore provide both mean and median, rather than implying that 14 describes every customer.

3. Cross-tabulation and segmentation

A cross-tabulation compares the distribution of one categorical variable across the categories of another. For example, a product manager might compare trial conversion by acquisition channel and customer segment.

The analyst must choose the correct denominator. Row percentages answer: “Within this row group, how are cases distributed?” Column percentages answer: “Within this column group, how are cases distributed?” Confusing them can reverse the interpretation.

A cross-tab is descriptive. A visible difference may reflect sampling variation, confounding variables or unequal group sizes. For a formal test of association between categorical variables, the chi-square statistic is:

Oᵢ is the observed frequency and Eᵢ is the expected frequency in cell i under the assumption of independence. For a two-way table, the expected count in a cell is:

The chi-square approximation is less reliable when expected counts are very small. Categories may need to be combined only when that is substantively defensible; otherwise an exact test or another method may be preferable. Statistical significance does not indicate the strength or practical importance of an association. Cramér’s V can be reported as an effect-size measure for a contingency table, but its interpretation depends on table dimensions and context.

4. Hypothesis testing and uncertainty

A hypothesis test evaluates whether sample evidence is inconsistent with a null model. The null hypothesis, H₀, commonly states that there is no difference or association. The alternative hypothesis, H₁, states the effect or relationship the research seeks to detect.

A p-value is the probability, assuming H₀ and the model assumptions are correct, of observing a result at least as extreme as the one obtained. It is not the probability that H₀ is true, nor is it the probability that the result will replicate.

A significance level α is selected before testing, often 0.05. If p ≤ α, the result is described as statistically significant under that decision rule; if p > α, the analyst does not reject H₀. “Failing to reject” is not proof that there is no effect.

A confidence interval gives a range of values compatible with the data and method. For a sample mean, a commonly used interval is:

where t* is a critical value determined by the chosen confidence level and degrees of freedom, s is sample standard deviation and n is sample size. The formula assumes independent observations and an appropriate sampling distribution; for small samples, severe skew or non-probability samples, interpretation requires caution. A 95% confidence procedure captures the true parameter in 95% of repeated samples under its assumptions; it is not strictly correct to say there is a 95% probability that one fixed parameter lies in this particular interval.

Practical significance and errors

A statistically significant difference may be too small to affect product decisions. Conversely, a meaningful difference may fail to reach significance in a small sample. Analysts should report the estimated effect, confidence interval, sample size and practical benchmark.

A Type I error rejects a true H₀; a Type II error fails to reject a false H₀. Statistical power—the probability of detecting an effect of a specified size under a chosen design—increases with larger samples, larger true effects and lower measurement noise. Testing many outcomes or segments increases the chance of false positives; pre-specification, correction procedures and transparent reporting reduce this risk.

5. Correlation and regression

Correlation measures the direction and strength of a linear association between two quantitative variables. Pearson’s sample correlation is:

The value lies between −1 and +1. A positive value indicates that larger values of one variable tend to accompany larger values of the other; a negative value indicates the opposite. Pearson correlation is sensitive to outliers and detects linear, not every possible, association. Spearman’s rank correlation may be suitable for ordinal data or monotonic relationships when normality is doubtful.

Correlation does not establish causation. A third variable may influence both variables, the direction may be reversed, or the relationship may be coincidental. For example, advertising exposure and purchase may correlate because high-intent customers are more likely to encounter or click advertisements.

Simple linear regression predicts an outcome Y from a predictor X:

Here Ŷ is the predicted outcome, a is the intercept, b is the slope and X is the predictor. The slope represents the expected change in Y for a one-unit increase in X, within the observed range and model context. The residual is:

The coefficient of determination is:

SSres is the sum of squared residuals and SStot is the total sum of squares around the mean. R² is the proportion of observed variation in Y explained by the fitted linear model in the sample; it is not proof of causation or a guarantee of predictive performance on new data.

Important regression checks include linearity, independent observations, constant residual variance, plausible residual distribution for inference, influential cases and—when there are multiple predictors—multicollinearity. Extrapolation beyond the observed X range is risky. A regression model should be interpreted as an evidence-supported approximation, not as an automatic decision rule.

6. Visualisation for reasoning and communication

A visualisation should answer a stated question. Use a bar chart for category comparisons, a histogram for the distribution of a quantitative variable, a box plot for distribution and outlier comparison, a line chart for ordered time observations, a scatterplot for two quantitative variables and a heat map for compact matrix patterns. Pie charts are usually inferior when there are many categories or small differences.

Effective charts:

• use an informative title that states the comparison;
• label axes, units, time period and sample size;
• use a zero baseline for bar-chart magnitude comparisons unless a justified alternative is clearly indicated;
• avoid three-dimensional effects and unnecessary decoration;
• use colour consistently and provide accessible contrast;
• distinguish counts from percentages;
• show uncertainty where it matters;
• annotate the decision-relevant pattern without hiding contrary evidence.

A dashboard is not simply a collection of charts. It should have a defined audience, decision purpose, refresh frequency, metric definitions, filters and ownership. Each metric requires a numerator, denominator, time window and inclusion rule. For example, “conversion rate” could mean purchasers divided by unique visitors, sessions or qualified leads. Without a definition, dashboard precision is misleading.

A useful insight statement combines evidence, interpretation and action: “Among valid respondents, new users exposed to the tutorial had a 12 percentage-point higher activation rate than those not exposed; because the comparison was observational, test the tutorial with random assignment before scaling.” This is stronger than “the tutorial works,” which overstates the design.

7. From findings to actionable insight

An insight is not a surprising number. It is an evidence-based interpretation that reduces uncertainty about a decision. A practical translation process is:

1. Restate the decision and the population of interest.
2. Identify the relevant metric and comparison.
3. Quantify the size and uncertainty of the pattern.
4. Check alternative explanations, sampling limitations and data quality.
5. Segment only when the segment is large enough and strategically meaningful.
6. Convert the finding into a specific recommendation.
7. State the confidence level and propose a next action, such as an experiment, interview or additional data collection.

A recommendation should specify the target user, product or advertising intervention, expected mechanism, success metric, time frame and decision rule. For instance, “Improve onboarding” is vague. “Test a two-step onboarding checklist for first-week trial users and target a five-percentage-point increase in activation without reducing day-seven retention” is operational.

8. Reporting and dashboard structure

A professional report commonly includes:

• an executive summary with the decision, principal findings and recommendations;
• research purpose, questions and design;
• sample description, fieldwork dates and limitations;
• data preparation and exclusions;
• methods and assumptions;
• results with tables and visualisations;
• interpretation and implications;
• recommended actions, owners and measures;
• technical appendix containing questionnaire, coding, detailed tables and analysis notes.

The executive summary should be written after the analysis is stable, but it should answer the manager’s questions first: What happened? How certain are we? Why does it matter? What should happen next?

9. Illustrative case: evaluating a mobile-app onboarding campaign

Background

This fictional case concerns LaunchLoop, a subscription productivity app. The product team introduced an optional onboarding tutorial and wanted to know whether it improved activation, defined as completing three key actions within seven days.

Problem and preparation

The team exported 2,400 trial records. After removing 36 exact duplicate IDs and 24 records created by automated test accounts, 2,340 eligible records remained. Tutorial exposure was recorded as a binary variable. The analyst checked that activation dates occurred after signup, identified 18 records with impossible negative time values and corrected them from the event log rather than deleting them. Missing device type was retained as “unknown” because it was not needed for the primary comparison.

Analysis

Among 1,200 tutorial-exposed users, 696 activated, giving 58.0%. Among 1,140 unexposed users, 570 activated, giving 50.0%. The difference was 8 percentage points. A cross-tabulation showed that the pattern was larger on Android than iOS. A chi-square test indicated evidence of association at the pre-specified 5% level, but the observational design did not establish that the tutorial caused activation. Users who chose the tutorial may already have been more motivated.

The team fitted a logistic regression as a supplementary analysis, controlling for device type, acquisition channel and prior account history. The adjusted result remained positive but smaller. Because the dashboard displayed only the unadjusted comparison, the analyst added a methodological note and a separate adjusted view.

Approach and solution

The product team recommended a randomised experiment: eligible new users would be randomly assigned to the tutorial or current onboarding, with activation as the primary outcome and day-seven retention as a guardrail. The dashboard was redesigned to show sample size, activation definition, exposure group, confidence intervals and experiment status.

Outcome

The experiment later found a 6 percentage-point activation improvement, with no material reduction in retention. The tutorial was rolled out gradually, and the team monitored effects by device and acquisition channel.

Learning points

The case demonstrates that data cleaning affects the denominator, descriptive comparisons should precede modelling, observational association should not be called causal, and a good insight leads to a testable product action. It also illustrates why dashboards must disclose definitions and uncertainty rather than presenting a single attractive percentage.

10. Ethics, governance and contemporary practice

Research data should be collected and analysed for a legitimate purpose, with informed consent where required, appropriate access controls and data minimisation. Personally identifiable information should be separated from analytical data when possible. Reports should avoid re-identification through tiny segments, especially when combining demographic and behavioural variables. Sensitive attributes require heightened care, and automated decisions should be checked for disparate impact.

Analysts must not manufacture significance by trying many analyses and reporting only favourable results, suppress inconvenient cases, manipulate chart scales or describe correlation as causation. Maintain an audit trail of transformations, exclusions and analysis choices. Follow applicable privacy and data-protection law, institutional policy, contractual restrictions and platform terms; legal requirements vary by jurisdiction.

Modern analytics tools can automate coding suggestions, anomaly detection, chart creation and natural-language summaries. Generative AI can help draft syntax or explain an output, but it should not receive confidential respondent data without approved safeguards. Analysts remain responsible for checking calculations, hallucinated interpretations, leakage of personal data, biased classifications and unsupported causal claims. Reproducible scripts, version control and peer review are preferable to opaque point-and-click workflows for consequential decisions.

Challenges and limitations

Research findings are constrained by sampling error, non-response, coverage gaps, measurement error, leading questions, common-method bias, small subgroups and changing market conditions. A large sample does not repair a biased sample. Statistical significance does not compensate for poor construct validity. Segment comparisons can become unstable when many groups are examined. Models trained on historical behaviour may reproduce past discrimination or fail when customer behaviour changes. These limitations should be stated in proportion to their likely effect on the decision, not hidden in a technical appendix.

Best-practice checklist

Before presenting results, verify that:

• the unit of analysis and target population are explicit;
• variables, missing codes and derived measures are documented;
• exclusions have a defensible rule and are reported;
• denominators and percentages are correct;
• the method matches the measurement scale and research design;
• assumptions, uncertainty and effect size accompany p-values;
• charts are labelled, accessible and not misleading;
• causal language is reserved for designs that support it;
• recommendations identify an owner, metric and next step;
• privacy, fairness and data-security requirements are addressed.

Chapter Summary

Data preparation is the foundation of credible marketing analysis. Analysts should define the unit of analysis, document variables, validate values, distinguish types of missingness, investigate outliers and preserve an audit trail. Descriptive statistics summarise distributions, while cross-tabulations reveal patterns across categorical groups; both require careful denominators and cautious interpretation. Hypothesis tests use a null model to assess evidence, but p-values do not measure practical importance or prove causation. Confidence intervals, effect sizes and sample sizes provide a fuller account of uncertainty.

Correlation describes linear association, whereas regression models an outcome as a function of predictors under stated assumptions. Neither automatically establishes causation. Visualisations and dashboards should be designed around decisions, clearly defined metrics and honest communication of uncertainty. The final professional task is to translate evidence into a specific, testable recommendation while respecting privacy, fairness, reproducibility and the limitations of the research design.

Key Terms

Data dictionary: A record of variable names, meanings, codes, ranges and measurement information.

Unit of analysis: The entity represented by one observation or row in a dataset.

Missing data: Values that were not recorded, are inapplicable or are otherwise unavailable.

Outlier: An observation unusually distant from the rest of a distribution.

Mean: The arithmetic average of quantitative observations.

Median: The middle value after observations are ordered.

Standard deviation: A measure of dispersion around the mean, expressed in the variable’s units.

Cross-tabulation: A table showing the joint distribution of categorical variables.

Expected frequency: The count predicted in a table cell if the variables are independent.

Chi-square test: A test commonly used to assess association between categorical variables.

Null hypothesis: A statement representing no difference, effect or association under a specified model.

P-value: The probability, under the null model, of a result at least as extreme as that observed.

Confidence interval: An interval produced by a procedure designed to capture a population parameter at a stated long-run rate.

Effect size: A measure of the magnitude of a difference or relationship.

Correlation: A numerical description of the direction and strength of association between variables.

Regression: A model relating an outcome to one or more predictors.

Residual: The observed outcome minus its fitted or predicted value.

Dashboard: A structured visual display of defined metrics for monitoring or decision-making.

Actionable insight: An evidence-based interpretation linked to a specific decision or intervention.

Review Questions

Short Answer Questions

1. Why should the original research file be preserved before cleaning?
2. Distinguish row percentages from column percentages in a cross-tabulation.
3. What does a p-value represent, and what does it not represent?
4. Why does correlation not establish causation?
5. Name three features of a trustworthy marketing dashboard.

Long Answer Questions

1. Explain a complete workflow for preparing survey data, including missing values, invalid records and outliers.
2. Compare mean, median, mode and standard deviation, and explain when each is useful in marketing research.
3. Explain chi-square testing, including observed frequencies, expected frequencies and the importance of practical significance.
4. Describe the assumptions and interpretation of a simple linear regression model.
5. Explain how an analyst should communicate uncertainty and limitations in an executive research report.

Analytical / Application Questions

1. A campaign dashboard reports a 10% higher conversion rate for exposed customers. Design an analysis plan that distinguishes descriptive association from a credible causal claim.
2. A survey shows an average satisfaction score of 3.6, but the response distribution is strongly polarised. What additional analyses and visualisations would you provide before recommending a product change?
3. Using the LaunchLoop case, propose an experiment, primary metric, guardrail metric, segmentation plan and decision rule for scaling the onboarding tutorial.

Multiple Choice Questions with Answers

Question 1

Which action best protects the integrity of a marketing analysis?

A. Replace every missing value with zero

B. Delete all observations with unusual values

C. Preserve the source file and document transformations

D. Report only statistically significant results

Correct Answer: C. Preserve the source file and document transformations

Question 2

Five customers spend 10, 12, 13, 15 and 20 currency units. What is the median spending value?

A. 12

B. 13

C. 14

D. 15

Correct Answer: B. 13

Question 3

What does a row percentage in a cross-tabulation describe?

A. The distribution within a row group

B. The distribution within a column group

C. The probability that the null hypothesis is true

D. The standard deviation of the row

Correct Answer: A. The distribution within a row group

Question 4

In a chi-square test of independence, expected cell frequencies are calculated from:

A. The observed value minus the mean

B. The row total and column total divided by the grand total

C. The sample standard deviation multiplied by the mean

D. The correlation coefficient squared

Correct Answer: B. The row total and column total divided by the grand total

Question 5

A p-value is best described as:

A. The probability that the research hypothesis is true

B. The probability that the sample was collected without bias

C. The probability, under the null model, of a result at least as extreme as observed

D. The percentage of variation explained by a regression

Correct Answer: C. The probability, under the null model, of a result at least as extreme as observed

Question 6

Which statement about a 95% confidence interval is most accurate?

A. There is a 95% probability that the fixed population parameter is inside this computed interval

B. The method captures the true parameter in about 95% of repeated samples under its assumptions

C. It proves that the sample is representative

D. It guarantees that the result is practically important

Correct Answer: B. The method captures the true parameter in about 95% of repeated samples under its assumptions

Question 7

A Pearson correlation of −0.80 indicates:

A. A strong negative linear association

B. A weak positive linear association

C. That one variable causes the other to fall

D. That 80% of respondents gave negative answers

Correct Answer: A. A strong negative linear association

Question 8

In the regression equation Ŷ = a + bX, what does b represent?

A. The predicted value when X is zero

B. The residual for an observation

C. The expected change in Ŷ for a one-unit increase in X

D. The proportion of missing data

Correct Answer: C. The expected change in Ŷ for a one-unit increase in X

Question 9

Which visualisation is generally most suitable for examining the relationship between two quantitative variables?

A. Scatterplot

B. Pie chart

C. Stacked categorical frequency table only

D. Single-value dashboard card

Correct Answer: A. Scatterplot

Question 10

Why was random assignment recommended in the LaunchLoop case?

A. To guarantee every customer activates

B. To reduce confounding and support a causal comparison

C. To eliminate the need for a sample-size calculation

D. To replace the need for defining activation

Correct Answer: B. To reduce confounding and support a causal comparison