Descriptive and Diagnostic Analytics

Introduction and Context

Business analytics helps organisations convert data into evidence for action. Descriptive analytics answers “What happened?” by summarising historical or current performance. Diagnostic analytics answers “Why did it happen?” by examining patterns, differences, relationships and operational causes. Together, they form the foundation for more advanced predictive and prescriptive analytics.

A sales manager may first observe that monthly revenue declined by 8%. Descriptive analysis quantifies the decline by product, region and customer segment. Diagnostic analysis then investigates whether the cause was lower demand, stock-outs, pricing changes, delivery failures or customer churn. The quality of later decisions depends heavily on the accuracy and discipline of these foundational stages.

This chapter presents practical methods for summarising performance and investigating causes: aggregation, segmentation, benchmarking, variance analysis, trend analysis, drill-down analysis and root-cause frameworks.

Learning Objectives

By the end of this chapter, you should be able to:

1. Distinguish descriptive analytics from diagnostic analytics.
2. Aggregate business data using appropriate measures and levels of detail.
3. Segment customers, products, transactions or operations for meaningful comparison.
4. Apply benchmarking, variance analysis and trend analysis.
5. Use drill-down techniques to move from an overall result to operational detail.
6. Investigate causes using structured root-cause frameworks.
7. Interpret analytical results cautiously, recognising data-quality and causality limitations.
8. Communicate findings in a way that supports managerial decisions.

1. Descriptive and Diagnostic Analytics in the Analytics Lifecycle

Descriptive analytics typically uses historical data, reports, dashboards, tables and visualisations. Common outputs include total sales, average order value, defect rates, customer counts and monthly trends. It establishes a reliable factual baseline.

Diagnostic analytics extends this work by comparing groups, examining changes over time and searching for plausible explanations. It may combine internal data with operational records, customer feedback, weather, competitor information or process data.

The boundary between descriptive and diagnostic analytics is practical rather than absolute. A grouped report can be descriptive, while the act of comparing groups and testing explanations becomes diagnostic.

2. Measurement Foundations

Good analysis begins with a clear definition of the business question, population, time period and measures. A metric is a quantified measure of performance, while a dimension is an attribute used to group or filter data, such as region, product category, channel or month.

2.1 Counts, totals and rates

A count records the number of events or entities. A total adds a quantity such as revenue or units sold. A rate relates a numerator to an appropriate denominator.

The numerator and denominator must refer to compatible populations and time periods. For example, a conversion rate may be calculated as:

If 240 purchases result from 6,000 qualified visits, the conversion rate is 4%. It would be misleading to divide purchases by total website sessions if many sessions were automated traffic or unrelated visits.

2.2 Mean, median and weighted average

The arithmetic mean for n observations is:

Here, xᵢ is an observation and n is the number of observations. The mean is appropriate when values are measured on a meaningful numerical scale and extreme values do not distort interpretation.

The median is the middle value after observations are ordered. It is often more representative for skewed measures such as income, delivery time or customer spending.

A weighted average is:

where xᵢ is a value and wᵢ is its weight. The weights must be non-negative and represent the relative contribution of observations. For example, an overall customer satisfaction score should normally weight branch scores by the number of responses, not simply average branch percentages.

2.3 Ratios and aggregation errors

Some metrics cannot be safely averaged. Suppose Branch A has 90 satisfied customers out of 100 responses and Branch B has 45 out of 50. The branch satisfaction rates are 90% and 90%, and the combined rate is:

If branches had different response volumes, averaging percentages equally would produce a different and potentially incorrect result. Analysts should aggregate the underlying numerators and denominators whenever possible.

3. Aggregation and the Level of Detail

Aggregation combines records into a higher-level summary. Transaction-level sales may be aggregated by day, store, region, product or customer. The appropriate level depends on the decision being made.

A chief executive may need quarterly revenue by business unit, whereas a warehouse supervisor may need hourly pick errors by process stage. Excessive aggregation can hide important differences; excessive detail can create noise and slow interpretation.

A useful approach is to define a hierarchy, such as:

Enterprise → Region → Branch → Department → Transaction

Analysts can move upward for summary reporting or downward for investigation. Aggregation should preserve the meaning of measures. Additive measures such as units sold can usually be summed across products and periods. Semi-additive measures such as account balance can be summed across accounts but not across time without care. Non-additive measures such as ratios should generally be recalculated from their components.

Worked example: sales aggregation

A retailer records the following weekly sales:

The regional summary shows equal revenue, but the product mix differs. North obtains 60% of revenue from Product A, while South obtains approximately 53.3% from Product B. A single company total would hide this operationally relevant distinction.

4. Segmentation

Segmentation divides a population into meaningful groups whose behaviour or performance may differ. Common segmentation dimensions include:

• customer type, industry or size;
• geography and sales channel;
• product category or service plan;
• order value, frequency or recency;
• operational process, shift or supplier; and
• demographic or behavioural characteristics, where lawful and appropriate.

Effective segments should be relevant to the decision, sufficiently large for reliable interpretation and defined consistently. A segment should not be created merely because it produces an interesting chart.

Descriptive versus diagnostic segmentation

Descriptive segmentation reports what differs. Diagnostic segmentation asks why the difference may exist. For example, a delivery dashboard may show that rural customers receive orders 1.5 days later than urban customers. Investigation might reveal longer carrier routes, fewer local distribution centres or different order cut-off times.

Segmentation can also reveal Simpson’s paradox: an aggregate relationship may reverse when data are separated into meaningful groups. For instance, a product may have a higher overall return rate because it is sold disproportionately in a high-return customer segment, even though it performs better within every segment. This is why analysts should examine relevant strata before drawing conclusions.

5. Benchmarking

Benchmarking compares performance against a reference point. Common benchmarks include:

1. historical performance, such as last month;
2. budget or target;
3. another branch, region or product;
4. an industry standard; and
5. a best-performing internal unit.

The benchmark must be comparable. A large urban store should not be judged directly against a small rural store without considering traffic, product mix and operating conditions.

Two useful measures are absolute difference and percentage difference:

If actual delivery time is 4.4 days and the target is 4.0 days, the absolute difference is 0.4 days and the percentage difference is 10%. Percentage differences require a non-zero benchmark and should be interpreted carefully when the benchmark is very small.

Benchmarking supports learning, but it does not by itself establish cause. A branch may outperform another because of customer composition, staffing, location or temporary conditions.

6. Variance Analysis

Variance analysis examines the difference between an actual result and a planned, expected or comparative result. It is widely used in budgeting, operations, sales and quality management.

For a cost, a positive variance may be unfavourable because actual cost exceeds plan. For revenue, the same positive variance may be favourable. Therefore, analysts should label variances as favourable or unfavourable according to the business objective rather than relying only on the sign.

Worked example: revenue variance

A department budgeted revenue of $250,000 but achieved $230,000.

Revenue is therefore $20,000 below budget, or 8% unfavourable. Diagnostic analysis should then separate possible volume and price effects. A decline may result from fewer units sold, a lower average selling price, a change in product mix or a combination.

Variance analysis is most useful when thresholds are established in advance. Investigating every tiny fluctuation consumes resources, while ignoring a material variance delays corrective action.

7. Trend Analysis

Trend analysis studies the direction and pattern of a measure over time. It may identify growth, decline, seasonality, cycles, sudden changes or unusual observations.

A simple percentage change between an initial value and a later value is:

If monthly complaints rise from 800 to 920, the change is 15%. This describes the change but does not prove that service quality deteriorated; reporting volume, customer numbers or recording practices may also have changed.

For a measure observed over several periods, the compound growth rate is:

Here, n is the number of periods and the beginning value must be positive. If revenue grows from $100,000 to $121,000 over two years, CAGR is 10% per year because 1.21^(1/2) − 1 = 0.10.

Trend charts should show consistent time intervals and clearly identify unusual events, such as a price change, system migration or public campaign. Analysts should distinguish a genuine trend from seasonal variation. Comparing December with November may be less informative than comparing December with previous Decembers.

8. Drill-Down Analysis

Drill-down moves from a high-level result to increasingly detailed dimensions. It is a structured way to locate where a problem is concentrated.

For example:

1. Total customer complaints increased.
2. Complaints are concentrated in the online channel.
3. The increase is concentrated in the delivery category.
4. Most delivery complaints arise from two distribution centres.
5. The centres share a carrier introduced six weeks earlier.

Drill-down should be guided by materiality and business logic, not by endless filtering. At each level, the analyst should ask whether the difference is large, reliable, operationally meaningful and actionable.

Drill-down is different from merely viewing more detail. It should preserve definitions and denominators. A region with many complaints may simply have more customers. Complaint rate per 1,000 orders may provide a fairer comparison than complaint count alone.

9. Root-Cause Analysis Frameworks

Diagnostic analytics identifies plausible explanations, but a correlation or pattern is not automatically a root cause. Root-cause analysis uses structured methods to move from symptoms to underlying process conditions.

9.1 The five whys

The five whys repeatedly asks why a problem occurred. The number five is not mandatory; questioning should stop when the team reaches a controllable process cause supported by evidence.

Problem: Orders were shipped late.

• Why? Orders waited for picking.
• Why? The picking queue was unusually large.
• Why? A promotion generated more orders than forecast.
• Why? The forecast excluded recent campaign data.
• Why? Marketing and operations used separate planning systems.

The final statement is a hypothesis requiring verification. A useful corrective action may involve integrated planning rather than simply telling staff to work faster.

9.2 Fishbone or cause-and-effect analysis

A fishbone diagram organises possible causes into categories such as People, Process, Technology, Materials, Measurement and Environment. It encourages a team to consider multiple explanations rather than blaming individuals prematurely.

9.3 Pareto analysis

A Pareto analysis ranks causes by their contribution to a problem. It often reveals that a small number of causes account for a substantial proportion of incidents. The familiar “80/20” idea is a heuristic, not a universal law; the data must determine the distribution.

9.4 Process mapping

Process mapping traces activities, decisions, handoffs and controls. It is particularly valuable when the apparent problem occurs at one stage but originates earlier. A map may reveal duplicate data entry, unclear ownership, approval bottlenecks or missing exception handling.

10. A Diagnostic Workflow

A disciplined workflow can be expressed as follows:

1. Define the decision problem. State the business question, outcome measure, population and time period.
2. Validate the data. Check missing values, duplicates, inconsistent categories, time zones, units and changes in data collection.
3. Establish the baseline. Produce totals, rates, distributions and relevant trends.
4. Compare meaningfully. Use targets, prior periods, peer groups or benchmarks with comparable definitions.
5. Segment and drill down. Identify where the pattern is concentrated.
6. Generate hypotheses. Use process knowledge, interviews and operational records.
7. Test explanations. Check whether evidence is consistent across time, groups and data sources.
8. Recommend action. Link the finding to an owner, intervention and measurable outcome.
9. Monitor results. Determine whether the intervention changed the relevant metric without causing unintended effects.

This workflow separates observation from interpretation. It also reduces confirmation bias, in which analysts search only for evidence supporting an initial belief.

Applied Case: Diagnosing a Decline in Online Orders

The following is an illustrative case.

Background

Northstar Home, an online retailer, reports that January revenue was $1.08 million, compared with $1.20 million in December. Management initially concludes that demand has weakened.

Problem definition

The analytics team defines the problem as: “Why did January revenue fall relative to December, and which operational response is justified?” Revenue alone is insufficient, so the team also examines orders, average order value, website conversion, product availability and delivery performance.

Descriptive analysis

Average order value is calculated as revenue divided by orders. January revenue equals $9,600 × $112.50 = $1,080,000, confirming the source figures.

Segmentation and drill-down

The team segments results by product category and finds that furniture revenue is stable, while small appliances declined by 22%. Further drill-down shows that three high-selling appliance models were unavailable for much of January. Website conversion fell most sharply for searches involving those models.

A second comparison shows that January traffic was only 2% lower than December. This makes a general demand collapse less plausible. The evidence instead points toward product availability and substitution difficulty.

Diagnostic approach

The team applies a fishbone framework. Potential causes include:

• Process: replenishment orders were placed using a delayed weekly report.
• Technology: inventory alerts did not include goods reserved for pending orders.
• Measurement: the stock-out report excluded one fulfilment centre.
• People: promotional demand assumptions were not communicated to procurement.
• External: a supplier experienced a temporary manufacturing delay.

Supplier records confirm a delay, while internal process analysis shows that the retailer had no automatic alert for projected stock-outs. The supplier delay was the immediate cause; weak monitoring was the underlying process vulnerability.

Solution and outcome

Northstar introduces daily availability monitoring, an alert for projected stock-outs and a cross-functional review before major promotions. It also adds a substitution recommendation to product pages. In the following month, appliance availability improves and conversion returns to 3.9%.

The improvement cannot be attributed solely to one intervention without a stronger evaluation design, because seasonality and other changes may also have contributed. Nevertheless, the combined evidence supports a practical corrective action.

Learning points

1. Revenue should be decomposed into drivers such as order volume and order value.
2. Traffic and conversion provide useful context for interpreting sales.
3. Drill-down identifies where a decline is concentrated.
4. Immediate causes and systemic causes may differ.
5. A diagnostic conclusion should state the strength and limits of the evidence.

11. Data Quality, Ethics and Governance

Diagnostic conclusions are only as credible as the data and definitions behind them. Important checks include completeness, accuracy, consistency, timeliness, uniqueness and validity. Analysts should document metric definitions, filters, exclusions, refresh dates and known limitations.

Segmentation can create ethical risks. Customer attributes such as age, gender, ethnicity, disability or location may act as sensitive or proxy variables. Organisations should use only relevant and lawful data, limit access, protect confidentiality and assess whether decisions produce unfair disadvantage. Small segments may also create re-identification risk.

Analysts must avoid presenting association as causation. A higher cancellation rate among a customer group may reflect price, service channel, contract type or another confounding factor. Diagnostic analysis can produce hypotheses, but causal claims require stronger evidence, such as controlled experiments, natural experiments or well-designed longitudinal analysis.

Dashboards should provide context rather than encourage surveillance or blame. Thresholds, alerts and performance rankings should be reviewed for unintended consequences, such as employees avoiding difficult customers to protect a metric.

12. Technology and Contemporary Practice

Business intelligence platforms now support interactive filtering, automated aggregation, anomaly alerts and natural-language queries. Machine learning can assist with clustering, anomaly detection and text classification of complaints. However, automation does not remove the need for metric governance, human review or causal reasoning.

Generative AI may help summarise a dashboard or suggest possible explanations, but its outputs must be checked against source data. It may invent relationships, overlook denominator problems or reproduce bias in historical records. A responsible workflow retains an auditable link between the published finding, the underlying data and the decision taken.

Best Practices

• Begin with a clearly defined decision, not a collection of available charts.
• Use numerators and denominators appropriate to the population.
• Choose mean, median, rate or total according to the data distribution and decision context.
• Compare like with like and document benchmark definitions.
• Decompose aggregate results before explaining them.
• Separate observed facts, hypotheses and confirmed causes.
• Use multiple data sources where a process explanation is important.
• Report uncertainty and limitations honestly.
• Protect personal data and examine fairness in segmentation.
• Connect every major finding to an action, owner and follow-up measure.

Chapter Summary

Descriptive analytics summarises what has happened through measures, aggregation, segmentation, benchmarking and trend analysis. Diagnostic analytics investigates why results differ by using variance analysis, meaningful comparisons, drill-down and root-cause frameworks. Reliable analysis depends on sound definitions, appropriate denominators, valid aggregation and careful attention to time, context and data quality. A fall in performance should be decomposed into its drivers before a cause is claimed. Root-cause methods such as the five whys, fishbone analysis, Pareto analysis and process mapping help connect patterns to operational explanations. Ethical governance is essential when data are personal, sensitive or used to evaluate employees and customers. The ultimate purpose of these techniques is not merely to describe performance, but to support evidence-based and accountable improvement.

Key Terms

Aggregation: Combining detailed records into a higher-level summary.

Benchmark: A reference point used for comparison.

CAGR: Compound annual growth rate over multiple periods.

Diagnostic analytics: Analysis concerned with explaining why an outcome occurred.

Dimension: An attribute used to group, filter or describe data.

Drill-down: Moving from a summary result to increasingly detailed information.

Favourable variance: A difference that improves performance relative to the objective.

Fishbone analysis: A structured cause-and-effect method for organising possible causes.

Mean: The arithmetic average of numerical observations.

Median: The middle ordered observation.

Metric: A quantified measure used to assess performance.

Pareto analysis: Ranking causes or categories by their contribution to a result.

Rate: A numerator expressed relative to an appropriate denominator.

Root cause: An underlying condition whose correction can prevent recurrence.

Segmentation: Dividing a population into meaningful groups.

Simpson’s paradox: A relationship in aggregated data that changes or reverses after segmentation.

Trend analysis: Examination of direction and pattern over time.

Variance: The difference between an actual and a reference value.

Review Questions

Short Answer Questions

1. What is the principal difference between descriptive and diagnostic analytics?
2. Why should analysts avoid averaging percentages when the underlying group sizes differ?
3. Define benchmarking and give two examples of valid benchmarks.
4. What is the purpose of drill-down analysis?
5. Name two root-cause analysis frameworks and describe their uses.

Long Answer Questions

1. Explain how aggregation, segmentation and benchmarking can be combined to evaluate regional sales performance.
2. Discuss the assumptions and limitations involved in using trend analysis for business decisions.
3. Explain variance analysis and illustrate how a manager should interpret a negative revenue variance.
4. Describe a complete diagnostic workflow from problem definition to monitoring corrective action.
5. Evaluate the ethical and governance risks associated with customer segmentation.

Analytical / Application Questions

1. A company’s revenue rises from $500,000 to $540,000 while orders rise from 5,000 to 6,000. Calculate the percentage change in revenue and the average order value in each period. What diagnostic questions should management ask?
2. A branch has 30 complaints from 1,000 orders, while another has 45 complaints from 3,000 orders. Which branch has the higher complaint rate, and why is comparing counts alone misleading?
3. A retailer observes declining conversion after introducing a new website. Design a diagnostic investigation using segmentation, benchmarking, drill-down and at least one root-cause framework.

Multiple Choice Questions with Answers

Question 1

Which question is primarily addressed by descriptive analytics?

A. What happened?

B. What will happen next year?

C. Which decision maximises profit?

D. What is the optimal resource allocation?

Correct Answer: A. What happened?

Question 2

A business records 80 purchases from 2,000 qualified visits. What is the conversion rate?

A. 2%

B. 4%

C. 8%

D. 25%

Correct Answer: B. 4%

Question 3

Which measure is generally most appropriate for a highly skewed distribution of customer spending?

A. Mean

B. Median

C. Total only

D. Maximum only

Correct Answer: B. Median

Question 4

Actual revenue is $230,000 and budgeted revenue is $250,000. What is the percentage variance?

A. −4%

B. −8%

C. 8%

D. 20%

Correct Answer: B. −8%

Question 5

What is the main purpose of segmentation?

A. To remove all variation from data

B. To divide a population into meaningful comparison groups

C. To guarantee causal conclusions

D. To replace data validation

Correct Answer: B. To divide a population into meaningful comparison groups

Question 6

Which method moves from an overall performance result to increasingly detailed operational information?

A. Drill-down analysis

B. Random sampling

C. Data deletion

D. Encryption

Correct Answer: A. Drill-down analysis

Question 7

In root-cause analysis, the five whys is primarily used to:

A. Calculate five performance ratios

B. Repeatedly question why a problem occurred

C. Rank five competitors

D. Create a five-period forecast

Correct Answer: B. Repeatedly question why a problem occurred

Question 8

Why can complaint counts be misleading when comparing branches?

A. Counts cannot be stored digitally

B. Branches may process different numbers of orders

C. Complaints never have a denominator

D. Counts always exaggerate small branches

Correct Answer: B. Branches may process different numbers of orders

Question 9

Which statement about correlation and causation is correct?

A. A correlation always proves causation

B. Causation can be assumed when two trends move together

C. A relationship may be influenced by confounding factors

D. Diagnostic analytics never uses comparisons

Correct Answer: C. A relationship may be influenced by confounding factors

Question 10

Which is an appropriate governance practice for analytics dashboards?

A. Hide metric definitions to simplify use

B. Publish results without refresh dates

C. Document definitions, filters and known limitations

D. Use sensitive attributes without review

Correct Answer: C. Document definitions, filters and known limitations