Statistics and Data Analytics

Introduction

Information technology produces data continuously: web requests, transaction logs, sensor readings, network events, user interactions, and software measurements. Statistics provides the methods for turning these observations into understandable evidence. Data analytics applies those methods to describe what has happened, identify patterns, estimate unknown quantities, test claims, and support decisions.

Statistics does not eliminate uncertainty. Rather, it measures and communicates uncertainty so that technical and managerial decisions are more defensible. In IT, statistical reasoning is used to monitor system performance, evaluate user experience, detect anomalies, compare software releases, forecast demand, and assess whether an apparent relationship is likely to be meaningful.

A central discipline is distinguishing association from causation. Two variables may change together because one causes the other, because a third variable affects both, or simply because the apparent pattern occurred by chance. Responsible analytics therefore combines mathematical methods with careful study design, domain knowledge, ethical judgment, and transparent communication.

Learning Objectives

By the end of this chapter, you should be able to:

1. Distinguish populations, samples, parameters, and statistics.
2. Summarise data using tables, graphs, measures of centre, and measures of spread.
3. Explain sampling methods and sources of sampling and measurement bias.
4. Calculate and interpret covariance, correlation, and simple linear regression.
5. Construct and interpret basic confidence intervals and hypothesis tests.
6. Explain why association does not by itself establish causation.
7. Apply statistical reasoning to an IT analytics case while recognising ethical and practical limitations.

1. Statistical Thinking in IT

A population is the complete set of units about which a question is asked. It may be all users of an application, all packets handled by a router during a period, or all software builds released by a team. A sample is a subset observed for analysis. A numerical feature measured on each unit is a variable.

A population characteristic is a parameter, commonly denoted by μ for a population mean or σ for a population standard deviation. A quantity calculated from a sample is a statistic, commonly denoted by x̄ for a sample mean or s for a sample standard deviation. Sampling is useful because observing an entire population may be expensive, slow, or impossible.

Variables may be:

• Categorical, such as operating system or support category.
• Numerical, such as response time or monthly expenditure.
• Discrete, taking countable values, such as number of failed logins.
• Continuous, measured on a continuum, such as latency in milliseconds.

The measurement scale matters. Nominal categories have labels without order; ordinal categories have order but not necessarily equal intervals; interval measurements have meaningful differences but no natural zero; ratio measurements have a meaningful zero and support ratios. For example, 20 ms is twice 10 ms, whereas a temperature measured in Celsius cannot generally be interpreted in that way.

A typical analytics workflow is:

1. Define the decision or research question.
2. Identify the population and variables.
3. Obtain data through an appropriate sampling or experimental design.
4. Clean and validate the data without obscuring important anomalies.
5. Summarise and visualise the data.
6. Model patterns and quantify uncertainty.
7. Interpret results in context.
8. Communicate limitations and take an appropriate action.

2. Organising and Visualising Data

A frequency table records how often values or categories occur. For numerical data with many distinct values, observations can be grouped into intervals. A histogram displays frequencies for numerical intervals; adjacent bars normally touch because the variable is measured on a continuous or ordered scale. A bar chart displays category counts and normally uses separated bars. A box plot shows the minimum, first quartile, median, third quartile, and maximum, often highlighting outliers. A scatter plot displays paired numerical observations and is useful for examining association.

Good visualisation practice includes clear units, meaningful titles, labelled axes, an honest scale, and an indication of sample size where relevant. Truncated axes can exaggerate differences, while excessive aggregation can hide important subgroups. A graph should expose structure rather than merely decorate a report.

Measures of centre

For observations x₁, x₂, …, xₙ, the arithmetic sample mean is:

Here n is the number of observations and xᵢ is observation i. The mean is appropriate for numerical data when addition and division are meaningful. It is sensitive to extreme values.

The median is the middle ordered value. If n is even, it is commonly the average of the two middle values. The mode is the most frequent value or category. The median is more resistant than the mean to skewness and outliers.

Measures of spread

The range is maximum minus minimum. The interquartile range is:

where Q₁ and Q₃ are the first and third quartiles. The IQR describes the spread of the middle 50% of observations.

The sample variance is:

The sample standard deviation is s = √s². The denominator n − 1, rather than n, is used when estimating a population variance from a sample; this is Bessel’s correction. Variance has squared units, while standard deviation has the same units as the original variable.

The coefficient of variation, when the mean is positive and a ratio interpretation is meaningful, is:

It is often reported as a percentage, 100s/x̄%, to compare relative variability across measurements with different scales. It should not be used uncritically when the mean is near zero or when negative values have substantive meaning.

Worked Example 1: response times

Suppose five API response times, in milliseconds, are 100, 110, 110, 120, and 160. The mean is:

The median is 110 ms, and the range is 60 ms. The deviations from the mean are −20, −10, −10, 0, and 40 ms. Their squared deviations sum to 2,600 ms², so:

The mean exceeds the median because the 160 ms observation pulls the mean upward. Reporting only the mean would hide this skew; reporting the median and a spread measure gives a more useful picture.

3. Sampling and Data Quality

A simple random sample gives each population member an equal known chance of selection. Stratified sampling divides the population into meaningful groups and samples within each group, which can improve representation. Systematic sampling selects every kth unit after a random starting point. Cluster sampling selects groups or clusters and observes units within them. A convenience sample uses easily available units and is often vulnerable to bias.

Important sources of error include:

• Undercoverage: some population groups have little or no chance of selection.
• Nonresponse: selected units do not provide data, and responders differ from nonresponders.
• Response bias: participants provide inaccurate or socially desirable answers.
• Measurement error: instruments, definitions, or logging systems record values inaccurately.
• Survivorship bias: analysis includes only units that remain observable or successful.
• Selection bias: inclusion is related to the outcome being studied.

A large sample does not automatically remove bias. Increasing sample size mainly reduces random sampling variability; it does not correct a systematically unrepresentative design. Data cleaning should address duplicate records, impossible values, missingness, inconsistent units, and timestamp problems. Deleting outliers solely because they are inconvenient can remove genuine failures and create misleadingly optimistic results.

Missing data should be classified and reported. If missingness is related to the outcome—for example, dissatisfied users abandon a survey—complete-case analysis may be biased. Imputation can be useful, but the method and assumptions must be documented.

4. Association: Covariance, Correlation, and Regression

For paired observations (xᵢ, yᵢ), sample covariance is:

Positive covariance indicates that large x values tend to occur with large y values, while negative covariance indicates an opposite tendency. Covariance depends on measurement units, so its magnitude is difficult to compare across datasets.

The Pearson sample correlation coefficient is:

Provided both variables have nonzero variation, −1 ≤ r ≤ 1. Values near 1 indicate strong positive linear association; values near −1 indicate strong negative linear association; values near 0 indicate little linear association. Correlation does not measure every possible relationship: a curved relationship can have a correlation near zero. Correlation is also sensitive to outliers and does not establish causation.

For a simple linear regression model:

β₀ is the population intercept, β₁ is the population slope, and εᵢ is an error term. The fitted least-squares line is:

where b₁ = sₓᵧ/sₓ² and b₀ = ȳ − b₁x̄. The slope describes the predicted change in y for a one-unit increase in x within the observed and substantively appropriate range. The coefficient of determination is R² = r² in simple linear regression; it describes the proportion of sample variation in y explained by the fitted linear relationship, not the proportion caused by x.

Regression assumptions for conventional inference include linearity, independent errors, constant error variance, and—especially for small-sample confidence intervals and tests—approximately normal errors. Diagnostics should examine residual plots, influential observations, and whether the model is appropriate. Extrapolating beyond the observed x-range is risky.

5. Estimation and Confidence Intervals

A point estimate uses a sample statistic to estimate a population parameter. An interval estimate gives a range of plausible parameter values under stated assumptions.

If the population standard deviation is known and the sampling distribution is appropriate, a confidence interval for a population mean is:

where σ is the population standard deviation, n is sample size, and z* is the standard-normal critical value. In practice σ is usually unknown, so the t interval is used:

where s is the sample standard deviation and t* is selected from the t distribution with n − 1 degrees of freedom. These formulas assume independent observations and a population that is normal or a sufficiently large sample for the sampling mean to be approximately normal. Strong skewness, dependence, or clustering may require alternative methods.

A 95% confidence interval is produced by a method that, over repeated samples under the same conditions, captures the true parameter approximately 95% of the time. It is not technically correct to say that the fixed parameter has a 95% probability of lying in one already computed frequentist interval.

Worked Example 2: estimating mean latency

A random sample of 25 requests has mean latency 200 ms and sample standard deviation 40 ms. Using an approximate 95% critical value t* = 2.064 for 24 degrees of freedom:

The interval is approximately (183.5 ms, 216.5 ms). Under the sampling assumptions, this method gives intervals containing the population mean in about 95% of repeated samples. It does not imply that 95% of individual requests lie in this interval.

For a population proportion, with sample proportion p̂ and sufficiently large expected counts, an approximate interval is:

Small samples or proportions near 0 or 1 may require exact or improved methods.

6. Hypothesis Testing

Hypothesis testing evaluates whether sample evidence is sufficiently inconsistent with a specified null model. The null hypothesis, H₀, usually represents no difference, no association, or a stated benchmark. The alternative hypothesis, H₁ or Hₐ, represents the claim being investigated.

A test statistic measures departure from H₀. The p-value is the probability, assuming H₀ is true and the test assumptions hold, of obtaining a result at least as extreme as the observed result in the direction specified by H₁. It is not the probability that H₀ is true.

A significance level α is selected before analysis. If p ≤ α, the result is described as statistically significant and H₀ is rejected. If p > α, H₀ is not rejected; this is not proof that H₀ is true. A Type I error rejects a true H₀, with long-run probability α under the testing procedure. A Type II error fails to reject a false H₀. Power is 1 − β, the probability of detecting a specified effect when it exists.

Statistical significance differs from practical significance. A tiny improvement may be statistically significant with millions of observations but irrelevant to users or operating cost. Conversely, a valuable effect may fail to reach significance in a small study. Report effect size, uncertainty, sample size, and operational consequences together.

For comparing two independent means, a common test uses:

This Welch test does not require equal population variances and uses an approximate degrees-of-freedom calculation. Independence and an appropriate outcome scale remain important. For paired measurements, analyse within-pair differences rather than treating paired observations as independent.

Multiple testing increases the chance of false discoveries. If many metrics, user segments, or time windows are examined, analysts should pre-specify primary outcomes, adjust procedures where appropriate, and clearly distinguish exploratory from confirmatory findings.

7. Association Is Not Causation

An association is a statistical relationship; causation means that changing one factor would produce a change in another under specified conditions. Causal interpretation requires more than correlation. Three common explanations for an observed association are:

1. X causes Y.
2. Y causes X or feedback exists.
3. A confounding variable Z influences both X and Y.

For example, users who enable push notifications may retain the application longer. This does not prove that notifications cause retention: highly engaged users may be more likely both to enable notifications and to remain active.

A randomised experiment supports causal inference because random assignment tends, in expectation, to balance measured and unmeasured confounders. In an A/B test, eligible users are randomly assigned to control and treatment groups, outcomes are defined in advance, exposure is monitored, and analysis follows the assignment principle. Important safeguards include adequate sample size, stable treatment delivery, avoidance of interference between users, and monitoring for harm.

When randomisation is impossible, observational studies may use matching, stratification, regression adjustment, difference-in-differences, instrumental variables, or regression discontinuity. These methods depend on assumptions that must be justified; sophisticated modelling cannot automatically remove unmeasured confounding.

8. Illustrative Case: Reducing E-Commerce Checkout Failure

Background. An online retailer observes that 8% of checkout attempts fail. The engineering team proposes a redesigned payment page and expects fewer failures.

Problem. A before-and-after comparison shows failure falling from 8.0% in March to 6.5% in April after the redesign. The team initially claims a 1.5 percentage-point causal improvement.

Analysis. The comparison may be affected by changes in payment-provider availability, traffic mix, mobile-device share, promotions, or seasonal demand. The data team defines one primary outcome—completed payment within 10 minutes of checkout initiation—and checks logging completeness. It stratifies descriptive results by device and region, but avoids presenting subgroup patterns as causal without sufficient design.

Approach. Eligible users are randomly assigned to the existing page or redesigned page for two weeks. Assignment is at the user level to reduce contamination, while payment-provider outages are recorded. The analysis follows intention to treat: users are analysed according to assigned version. The team reports failure proportions, the difference in proportions, a confidence interval, and operational metrics such as page load time, support contacts, and revenue per visitor.

Suppose the control group has 10,000 users with 800 failures (8.0%), while treatment has 10,000 users with 650 failures (6.5%). The estimated difference is −1.5 percentage points. The approximate standard error for two independent proportions is:

Using p₁ = 0.080, p₂ = 0.065, and both sample sizes 10,000 gives SE ≈ 0.00368. An approximate 95% interval for treatment minus control is −0.015 ± 1.96(0.00368), or approximately (−0.0222, −0.0078). The interval suggests a reduction between about 0.78 and 2.22 percentage points, subject to the approximation and experiment assumptions.

Outcome. The redesign is adopted because the primary outcome improves and page-load and revenue metrics do not deteriorate. Monitoring continues after deployment because effects can change with new devices, providers, or user behaviour.

Learning points. The observational before-after pattern was useful for generating a hypothesis but not sufficient for causal attribution. Random assignment, pre-specified outcomes, uncertainty intervals, and guardrail metrics produced stronger evidence. The result still applies most directly to the studied population and period, not automatically to every future market.

9. Ethics, Governance, and Contemporary Analytics

IT analytics often involves personal, behavioural, or sensitive data. Organisations should collect only data necessary for a legitimate purpose, provide appropriate notice, restrict access, protect identifiers, define retention periods, and comply with applicable privacy and data-protection law. De-identification reduces risk but does not guarantee anonymity, especially when datasets can be linked.

Fairness requires checking whether measurement, sampling, model error, or deployment decisions disadvantage groups. A model can have high overall accuracy while performing poorly for a minority group. Analysts should document data provenance, transformations, exclusions, model versions, and known limitations. Automated decisions affecting access, employment, credit, education, or safety require meaningful human oversight and avenues for challenge where applicable.

Modern platforms support streaming dashboards, anomaly detection, experimentation systems, and machine-learning models. These technologies increase scale but do not replace statistical design. Data drift, feedback loops, alert fatigue, concept drift, and repeated monitoring can produce misleading conclusions. AI-generated analyses should be independently checked for definitions, arithmetic, sampling assumptions, privacy risks, and unsupported causal claims.

10. Best Practices and Limitations

Effective IT analytics follows several principles:

• Start with a precise question and decision criterion.
• Define the unit of analysis and avoid mixing levels, such as users and sessions.
• Inspect distributions and missingness before modelling.
• Use visualisations appropriate to variable type.
• Report centre, spread, sample size, and uncertainty.
• Distinguish exploratory analysis from confirmatory testing.
• Prefer randomisation for causal questions when feasible.
• Examine practical importance, not only p-values.
• Test robustness to plausible definitions and outliers.
• Document assumptions, code, data versions, and limitations.
• Protect privacy and assess disparate impacts.

Statistics has limitations. Samples may not represent future users, measurements may be imperfect, observations may not be independent, and models simplify reality. A statistically sound result can still be operationally irrelevant if the wrong outcome was measured. Conversely, an unusual observation may be a critical security incident rather than noise. Statistical reasoning is therefore a structured aid to judgment, not a substitute for technical knowledge or ethical responsibility.

Chapter Summary

Statistics enables IT professionals to convert data into evidence. Descriptive statistics organise observations through tables, graphs, measures of centre, and measures of spread. Means are sensitive to extreme values, whereas medians and IQRs are more resistant. Sampling design determines how far results can be generalised, and large samples do not correct systematic bias.

Covariance and Pearson correlation describe association, while regression models a conditional linear relationship under stated assumptions. Confidence intervals quantify estimation uncertainty, and hypothesis tests evaluate evidence against a null model. P-values do not measure the probability that a hypothesis is true, and statistical significance does not guarantee practical importance.

Causal claims require appropriate design, especially randomisation for experiments. Confounding, reverse causality, selection effects, and measurement problems can make associations misleading. In professional IT analytics, privacy, fairness, reproducibility, governance, and clear communication are as important as calculation. Sound practice combines mathematical accuracy with domain context and cautious interpretation.

Key Terms

Population: Complete set of units about which a study asks a question.

Sample: Subset of a population that is observed.

Parameter: Numerical characteristic of a population.

Statistic: Numerical quantity calculated from a sample.

Variable: Measured characteristic of an observation.

Mean: Arithmetic average of numerical observations.

Median: Middle ordered observation, or average of the two middle observations.

Variance: Average squared deviation measure, with sample variance using n − 1.

Standard deviation: Square root of variance, expressed in original units.

Interquartile range: Difference between the third and first quartiles.

Outlier: Observation unusually distant from the rest of a dataset.

Sampling bias: Systematic distortion caused by an unrepresentative sampling process.

Covariance: Unit-dependent measure of joint variation between two variables.

Correlation: Standardised measure of linear association between two variables.

Regression: Model relating an outcome to one or more explanatory variables.

Confidence interval: Interval produced by a method designed to capture a parameter at a stated long-run rate.

Null hypothesis: Baseline claim evaluated by a statistical test.

P-value: Probability, under the null model, of results at least as extreme as observed.

Confounding: Distortion of an association by a variable related to both measured variables.

Causation: Relationship in which changing one factor produces a change in another under defined conditions.

Review Questions

5 Short Answer Questions

1. Distinguish a population parameter from a sample statistic.
2. Why can the median be preferable to the mean for highly skewed response times?
3. What information does the interquartile range provide?
4. Define a p-value without interpreting it as the probability that the null hypothesis is true.
5. Why does correlation not establish causation?

5 Long Answer Questions

1. Explain the main probability and non-probability sampling methods and discuss the biases associated with convenience sampling.
2. Compare histograms, bar charts, box plots, and scatter plots, including the type of data appropriate for each.
3. Explain the assumptions and interpretation of simple linear regression in an IT performance study.
4. Describe confidence intervals, Type I errors, Type II errors, power, and practical significance.
5. Discuss privacy, fairness, transparency, and reproducibility responsibilities in data analytics.

3 Analytical / Application Questions

1. A monitoring team records response times of 80, 90, 95, 100, and 235 ms. Calculate the mean, median, range, and sample standard deviation, and explain which summary best represents a typical request.
2. A product team observes that users who receive more notifications remain active longer. Identify at least two possible confounders and design an experiment capable of testing whether notifications cause increased retention.
3. An A/B test reports a statistically significant 0.2% reduction in checkout failures but increases average page-load time by 900 ms. Recommend what additional analyses and decision criteria should be used before deployment.

Multiple Choice Questions with Answers

Question 1

Which measure is generally most resistant to an extreme outlier?

A. Mean

B. Median

C. Variance

D. Standard deviation

Correct Answer: B. Median

Question 2

A sample contains response times 10, 20, and 30 ms. What is the sample mean?

A. 10 ms

B. 15 ms

C. 20 ms

D. 30 ms

Correct Answer: C. 20 ms

Question 3

What does the interquartile range measure?

A. The total number of observations

B. The spread of the middle 50% of observations

C. The average of all observations

D. The largest observation only

Correct Answer: B. The spread of the middle 50% of observations

Question 4

Which sampling method divides a population into groups and samples within each group?

A. Stratified sampling

B. Convenience sampling

C. Cluster-free sampling

D. Unstructured sampling

Correct Answer: A. Stratified sampling

Question 5

Which statement about Pearson correlation is correct?

A. It proves that one variable causes another

B. It measures linear association and lies between −1 and 1

C. It can only be positive

D. It is unaffected by outliers

Correct Answer: B. It measures linear association and lies between −1 and 1

Question 6

A p-value is best described as the probability of obtaining results at least as extreme as observed, assuming:

A. The alternative hypothesis is true

B. The sample mean equals zero

C. The null hypothesis and test assumptions hold

D. The result is practically important

Correct Answer: C. The null hypothesis and test assumptions hold

Question 7

If a 95% confidence interval for a mean is (183.5, 216.5) ms, what is the interval's point estimate?

A. 16.5 ms

B. 183.5 ms

C. 200 ms

D. 216.5 ms

Correct Answer: C. 200 ms

Question 8

Which design provides the strongest basis for a causal claim when properly implemented?

A. A convenience survey

B. A before-and-after comparison without controls

C. A randomised controlled experiment

D. A correlation calculated from existing logs

Correct Answer: C. A randomised controlled experiment

Question 9

In the regression equation ŷ = b₀ + b₁x, what does b₁ represent?

A. The predicted value when x is zero

B. The change in predicted y for a one-unit increase in x

C. The sample size

D. The residual variance only

Correct Answer: B. The change in predicted y for a one-unit increase in x

Question 10

Why can a large sample still produce a biased result?

A. Large samples always have zero variance

B. Sampling bias is not necessarily removed by increasing sample size

C. Large samples cannot contain outliers

D. A large sample prevents measurement error

Correct Answer: B. Sampling bias is not necessarily removed by increasing sample size