Random Variables, Expectation, and Moments

Introduction

Probability models describe uncertainty through events, but many scientific and machine-learning applications require numerical summaries. A random variable provides the bridge: it assigns a numerical value to each outcome of a random experiment. Examples include the number of failed inference requests, model latency, prediction error, insurance loss, sensor voltage, and daily demand.

Once a random variable has been defined, its probability mass function, probability density function, or cumulative distribution function describes its behavior. Expectation and moments then summarize location, variability, asymmetry, and tail behavior. Covariance quantifies linear dependence between variables, while transformations and generating functions enable new distributions and aggregate quantities to be analyzed systematically.

These concepts underpin statistical estimation, regression, probabilistic machine learning, risk analysis, simulation, and decision-making under uncertainty.

Learning Objectives

After studying this chapter, students should be able to:

1. Distinguish discrete, continuous, and mixed random variables.

2. Use probability mass, density, and cumulative distribution functions.

3. Compute expectations using direct definitions and the law of the unconscious statistician.

4. Derive variance, raw moments, central moments, skewness, and kurtosis.

5. Calculate covariance and correlation for jointly distributed variables.

6. determine distributions and moments of transformed random variables.

7. Use probability-generating, moment-generating, cumulant-generating, and characteristic functions.

8. Apply random-variable models critically in AI and operational settings.

1. Random Variables

Let the sample space of an experiment be denoted by \(\Omega\). A real-valued random variable is a measurable function

\[

X:\Omega\rightarrow\mathbb{R}.

\]

Thus, randomness belongs to the experimental outcome \(\omega\), while \(X(\omega)\) is the numerical value assigned to that outcome. Measurability ensures that events such as \(\{X\leq x\}\) have defined probabilities.

1.1 Discrete random variables

A random variable is discrete if its possible values form a finite or countably infinite set. Examples include:

• the number of classification errors in a test set;
• the number of customers arriving in an hour;
• the number of retraining attempts before convergence.

Its probability mass function (PMF) is

\[

p_X(x)=P(X=x).

\]

A valid PMF satisfies

\[

p_X(x)\geq 0,\qquad \sum_x p_X(x)=1,

\]

where the sum is over the support of \(X\).

1.2 Continuous random variables

A random variable is continuous in the usual density-based sense if there is a nonnegative function \(f_X\) such that

\[

P(a<X\leq b)=\int_a^b f_X(x)\,dx

\]

and

\[

\int_{-\infty}^{\infty}f_X(x)\,dx=1.

\]

The function \(f_X\) is the probability density function (PDF). A density value is not itself a probability. For a continuous random variable,

\[

P(X=x)=0

\]

for every individual value \(x\), even when \(f_X(x)>0\).

If \(X\) represents a physical quantity, the density has reciprocal units. For example, if \(X\) is measured in seconds, \(f_X(x)\) has units of seconds\(^{-1}\), making the integral dimensionless.

1.3 Mixed random variables

Not every random variable is purely discrete or absolutely continuous. A service may have zero waiting time with positive probability but otherwise have a continuous waiting-time distribution. Such a variable has a point mass at zero and a density over positive values. Mixed distributions are naturally represented through their cumulative distribution functions.

2. Cumulative Distribution Functions

The cumulative distribution function (CDF) of any real-valued random variable is

\[

F_X(x)=P(X\leq x).

\]

Every CDF satisfies:

1. \(F_X\) is nondecreasing.

2. \(F_X\) is right-continuous.

3. \(\lim_{x\to-\infty}F_X(x)=0\).

4. \(\lim_{x\to\infty}F_X(x)=1\).

Interval probabilities follow from

\[

P(a<X\leq b)=F_X(b)-F_X(a).

\]

For a discrete variable,

\[

F_X(x)=\sum_{u\leq x}p_X(u).

\]

A jump at \(x\) has magnitude

\[

F_X(x)-F_X(x^-)=P(X=x),

\]

where \(F_X(x^-)=\lim_{t\uparrow x}F_X(t)\).

For an absolutely continuous distribution, there is an integrable density \(f_X\) satisfying

\[

F_X(x)=\int_{-\infty}^{x}f_X(u)\,du.

\]

Consequently,

\[

f_X(x)=F_X'(x)

\]

where the derivative exists, which is almost everywhere. The requirement of absolute continuity is important: differentiability properties alone do not generally justify reconstructing a CDF by integrating its derivative, and not every continuous distribution possesses a PDF.

Worked Example 1: A discrete model

Suppose the number \(X\) of incorrect predictions in a small batch has PMF

\[

P(X=0)=0.2,\quad P(X=1)=0.5,\quad P(X=2)=0.3.

\]

The CDF is

\[

F_X(x)=

\begin{cases}

0, & x<0,\\

0.2, & 0\leq x<1,\\

0.7, & 1\leq x<2,\\

1, & x\geq2.

\end{cases}

\]

Therefore, \(P(X>0)=1-F_X(0)=0.8\).

Worked Example 2: A continuous model

Let inference delay \(T\) have exponential density

\[

f_T(t)=0.25e^{-0.25t},\qquad t\geq0,

\]

where time is measured in minutes. Its CDF is

\[

F_T(t)=1-e^{-0.25t},\qquad t\geq0.

\]

Thus,

\[

P(2<T\leq6)=F_T(6)-F_T(2)

=e^{-0.5}-e^{-1.5}\approx0.3834.

\]

3. Expectation

The expected value is the probability-weighted average of a random variable. For a discrete variable,

\[

E[X]=\sum_x x p_X(x),

\]

provided the sum is absolutely convergent. For a continuous variable with density,

\[

E[X]=\int_{-\infty}^{\infty}x f_X(x)\,dx,

\]

provided \(E[|X|]<\infty\).

Expectation describes a long-run average under repeated sampling from a stable probability model. It need not be a possible value. For example, the expected outcome of a fair die is \(3.5\), although no roll produces \(3.5\).

3.1 Expectation of a function

For a function \(g\), the law of the unconscious statistician gives

\[

E[g(X)]=\sum_x g(x)p_X(x)

\]

for a discrete variable, and

\[

E[g(X)]=\int_{-\infty}^{\infty}g(x)f_X(x)\,dx

\]

for an absolutely continuous variable. A separate derivation of the distribution of \(g(X)\) is therefore unnecessary when only its expectation is required.

3.2 Linearity of expectation

If the relevant expectations exist, then

\[

E[aX+bY+c]=aE[X]+bE[Y]+c.

\]

Independence is not required. More generally,

\[

E\left[\sum_{i=1}^n a_iX_i\right]=\sum_{i=1}^n a_iE[X_i].

\]

Worked Example 3: Mean and expected cost

For the PMF in Worked Example 1,

\[

E[X]=(0)(0.2)+(1)(0.5)+(2)(0.3)=1.1.

\]

If each incorrect prediction incurs a review cost of $12 and every batch has a fixed processing cost of $5, then \(C=12X+5\). Hence,

\[

E[C]=12E[X]+5=12(1.1)+5=\$18.20.

\]

4. Variance and Moments

Expectation alone does not describe uncertainty. Two variables may have equal means but very different dispersion or tail behavior.

4.1 Variance

For a variable with a finite second moment, variance is

\[

\operatorname{Var}(X)=E[(X-\mu)^2],\qquad \mu=E[X].

\]

Expanding the square gives the computational identity

\[

\operatorname{Var}(X)=E[X^2]-\{E[X]\}^2.

\]

The derivation is

\[

\begin{aligned}

E[(X-\mu)^2]

&=E[X^2-2\mu X+\mu^2]\\

&=E[X^2]-2\mu E[X]+\mu^2\\

&=E[X^2]-\mu^2.

\end{aligned}

\]

Variance is expressed in squared units. The standard deviation

\[

\sigma_X=\sqrt{\operatorname{Var}(X)}

\]

has the same units as \(X\).

For constants \(a\) and \(b\),

\[

\operatorname{Var}(aX+b)=a^2\operatorname{Var}(X).

\]

Adding a constant changes location but not dispersion.

For the discrete model in Worked Example 1,

\[

E[X^2]=(0^2)(0.2)+(1^2)(0.5)+(2^2)(0.3)=1.7.

\]

Therefore,

\[

\operatorname{Var}(X)=1.7-(1.1)^2=0.49,

\]

and \(\sigma_X=0.7\).

4.2 Raw and central moments

The \(r\)th raw moment, when finite, is

\[

\mu_r'=E[X^r].

\]

The \(r\)th central moment is

\[

\mu_r=E[(X-E[X])^r].

\]

Important cases include:

• \(\mu_1'=E[X]\), the mean;
• \(\mu_2'=E[X^2]\), the second raw moment;
• \(\mu_2=\operatorname{Var}(X)\), the variance.

Raw and central moments have units raised to the corresponding power. Standardized moments remove units.

4.3 Skewness and kurtosis

If the necessary moments exist and \(\sigma>0\), population skewness is

\[

\gamma_1=\frac{E[(X-\mu)^3]}{\sigma^3}.

\]

Positive skewness usually indicates a longer right tail, while negative skewness indicates a longer left tail. Skewness is not a complete description of distributional shape.

Population kurtosis is

\[

\beta_2=\frac{E[(X-\mu)^4]}{\sigma^4}.

\]

Excess kurtosis is \(\gamma_2=\beta_2-3\), so a normal distribution has excess kurtosis zero. Kurtosis is influenced strongly by extreme observations and is often interpreted as sensitivity to tail values rather than merely “peakedness.”

4.4 Existence of moments

Not every distribution has finite moments. A Cauchy random variable has neither a finite mean nor a finite variance. Heavy-tailed models may possess a mean but not a variance. Algebraic manipulation of nonexistent moments is invalid; moment existence must be checked before formulas are applied.

5. Joint Random Variables, Covariance, and Correlation

For random variables \(X\) and \(Y\), a joint PMF or PDF describes their simultaneous behavior. Marginal distributions are obtained by summing or integrating out the other variable. In the continuous case,

\[

f_X(x)=\int_{-\infty}^{\infty}f_{X,Y}(x,y)\,dy.

\]

If the required expectation exists, covariance is

\[

\operatorname{Cov}(X,Y)=E[(X-E[X])(Y-E[Y])].

\]

Equivalently,

\[

\operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y].

\]

Covariance is positive when large values of the variables tend to occur together, negative when large values of one tend to accompany small values of the other, and zero when there is no linear association.

The variance of a linear combination is

\[

\operatorname{Var}(aX+bY)=a^2\operatorname{Var}(X)+b^2\operatorname{Var}(Y)+2ab\operatorname{Cov}(X,Y).

\]

For \(n\) variables,

\[

\operatorname{Var}\left(\sum_{i=1}^{n}a_iX_i\right)

=\sum_{i=1}^{n}a_i^2\operatorname{Var}(X_i)

+2\sum_{i<j}a_ia_j\operatorname{Cov}(X_i,X_j).

\]

If \(X\) and \(Y\) are independent and have finite second moments, then their covariance is zero. The converse is generally false: zero covariance does not imply independence. An important special case is a jointly normal pair, for which zero covariance does imply independence.

Correlation standardizes covariance:

\[

\rho_{X,Y}=\frac{\operatorname{Cov}(X,Y)}{\sigma_X\sigma_Y},

\]

provided both standard deviations are positive and finite. Correlation lies between \(-1\) and \(1\), is dimensionless, and measures linear—not general—dependence.

Worked Example 4: Fraud and alert indicators

Let \(X\) indicate whether a transaction is fraudulent and \(Y\) indicate whether an automated system raises an alert. Suppose

\[

P(X=1)=0.10,\quad P(Y=1)=0.20,\quad P(X=1,Y=1)=0.08.

\]

Because \(X\) and \(Y\) are indicators,

\[

E[X]=0.10,\quad E[Y]=0.20,\quad E[XY]=0.08.

\]

Thus,

\[

\operatorname{Cov}(X,Y)=0.08-(0.10)(0.20)=0.06.

\]

Also,

\[

\operatorname{Var}(X)=0.1(0.9)=0.09,

\quad

\operatorname{Var}(Y)=0.2(0.8)=0.16.

\]

Therefore,

\[

\rho_{X,Y}=\frac{0.06}{\sqrt{0.09}\sqrt{0.16}}

=\frac{0.06}{(0.3)(0.4)}=0.5.

\]

The positive correlation indicates linear association, but it does not by itself measure precision, recall, causal effect, or fairness.

5.1 Covariance matrices

For a random vector \(\mathbf{X}\in\mathbb{R}^d\) with mean vector \(\boldsymbol{\mu}\), the covariance matrix is

\[

\boldsymbol{\Sigma}=E[(\mathbf{X}-\boldsymbol{\mu})(\mathbf{X}-\boldsymbol{\mu})^T].

\]

Its diagonal elements are variances and its off-diagonal elements are covariances. A valid covariance matrix is symmetric and positive semidefinite:

\[

\mathbf{a}^T\boldsymbol{\Sigma}\mathbf{a}\geq0

\]

for every vector \(\mathbf{a}\). Covariance matrices are central to principal component analysis, Gaussian models, portfolio analysis, Kalman filters, and uncertainty quantification.

6. Transformations of Random Variables

A transformation creates a new random variable \(Y=g(X)\). Transformations appear in feature engineering, normalization, calibration, logarithmic modeling, and simulation.

6.1 Discrete transformations

For discrete \(X\),

\[

P(Y=y)=\sum_{x:g(x)=y}P(X=x).

\]

All original values mapping to the same transformed value must be included.

6.2 Continuous monotone transformations

Suppose \(X\) has density \(f_X\), and \(Y=g(X)\), where \(g\) is differentiable and strictly monotone over the support of \(X\). If \(x=g^{-1}(y)\), then

\[

f_Y(y)=f_X(g^{-1}(y))\left|\frac{d}{dy}g^{-1}(y)\right|

\]

for \(y\) in the transformed support.

For a non-one-to-one differentiable transformation, contributions from all inverse branches must be added:

\[

f_Y(y)=\sum_{x_i:g(x_i)=y}\frac{f_X(x_i)}{|g'(x_i)|},

\]

provided the relevant roots are regular, meaning \(g'(x_i)\neq0\).

Worked Example 5: Affine transformation

Let \(X\sim\operatorname{Uniform}(0,1)\) and define \(Y=10+5X\). Then \(Y\) has support \((10,15)\). Since \(x=(y-10)/5\),

\[

f_Y(y)=1\left|\frac{1}{5}\right|=\frac{1}{5},\qquad 10<y<15.

\]

Thus \(Y\sim\operatorname{Uniform}(10,15)\). Its moments may also be obtained directly:

\[

E[Y]=10+5E[X]=12.5,

\]

\[

\operatorname{Var}(Y)=25\operatorname{Var}(X)=\frac{25}{12}.

\]

6.3 Multivariate transformations

For an invertible differentiable transformation \(\mathbf{Y}=g(\mathbf{X})\),

\[

f_{\mathbf{Y}}(\mathbf{y})

=f_{\mathbf{X}}(g^{-1}(\mathbf{y}))

\left|\det J_{g^{-1}}(\mathbf{y})\right|,

\]

where \(J_{g^{-1}}\) is the Jacobian matrix of the inverse transformation. The determinant corrects for changes in volume. This result underlies normalizing flows and other density-transformation methods in modern generative AI.

7. Generating Functions

Generating functions encode distributions or moments in functional form.

7.1 Probability-generating function

For a nonnegative integer-valued random variable,

\[

G_X(s)=E[s^X]=\sum_{k=0}^{\infty}P(X=k)s^k,

\]

for values of \(s\) where the series converges. Its derivatives yield factorial moments:

\[

G_X'(1)=E[X],

\]

\[

G_X''(1)=E[X(X-1)].

\]

Therefore,

\[

\operatorname{Var}(X)=G_X''(1)+G_X'(1)-\{G_X'(1)\}^2,

\]

when the derivatives and moments exist.

7.2 Moment-generating function

The moment-generating function (MGF) is

\[

M_X(t)=E[e^{tX}].

\]

If the MGF is finite on an open interval containing zero, then it determines the distribution uniquely and

\[

M_X^{(r)}(0)=E[X^r]

\]

for moments justified by differentiation under the expectation. Some distributions have finite moments but no finite MGF away from zero, so the MGF is not universally available.

For independent \(X\) and \(Y\),

\[

M_{X+Y}(t)=M_X(t)M_Y(t)

\]

where the MGFs exist.

For a Bernoulli variable with success probability \(p\),

\[

M_X(t)=(1-p)+pe^t.

\]

Differentiating gives \(M_X'(0)=p\) and \(M_X''(0)=p\), so

\[

\operatorname{Var}(X)=p-p^2=p(1-p).

\]

7.3 Cumulant-generating function

The cumulant-generating function is

\[

K_X(t)=\log M_X(t)

\]

where the MGF is positive and finite. Its first two derivatives at zero are

\[

K_X'(0)=E[X],\qquad K_X''(0)=\operatorname{Var}(X).

\]

For independent variables, cumulant-generating functions add. Higher derivatives produce higher-order cumulants, which are useful in approximation theory and asymptotic statistics.

7.4 Characteristic function

The characteristic function is

\[

\phi_X(t)=E[e^{itX}],

\]

where \(i^2=-1\). Unlike the MGF, it exists for every probability distribution because \(|e^{itX}|=1\). Characteristic functions uniquely determine distributions and are fundamental in proofs of convergence, including versions of the central limit theorem.

8. Applied Case: Reliability of an AI Inference Service

This is an illustrative case.

Background

An organization deploys an image-classification model to edge devices. An inference attempt succeeds with probability \(p=0.8\). Failed attempts are retried, and the engineering team initially assumes attempts are independent with a constant success probability.

Problem

Each request requires 40 milliseconds of base processing, and every failed attempt before success adds 15 milliseconds. The team must estimate mean latency, latency variability, and compliance with a 70-millisecond service-level threshold.

Analysis

Let \(X\) be the number of failures before the first success. Then \(X\) follows a geometric distribution on \(\{0,1,2,\ldots\}\):

\[

P(X=k)=p(1-p)^k=0.8(0.2)^k.

\]

For this parameterization,

\[

E[X]=\frac{1-p}{p}=\frac{0.2}{0.8}=0.25,

\]

and

\[

\operatorname{Var}(X)=\frac{1-p}{p^2}

=\frac{0.2}{0.8^2}=0.3125.

\]

Latency is the transformed variable

\[

T=40+15X

\]

in milliseconds. Therefore,

\[

E[T]=40+15(0.25)=43.75\text{ ms},

\]

and

\[

\operatorname{Var}(T)=15^2(0.3125)=70.3125\text{ ms}^2.

\]

The standard deviation is

\[

\sigma_T=\sqrt{70.3125}\approx8.39\text{ ms}.

\]

The event \(T>70\) is equivalent to

\[

40+15X>70,

\]

so \(X\geq3\). For the geometric model,

\[

P(X\geq3)=(1-p)^3=(0.2)^3=0.008.

\]

Thus, the estimated probability of exceeding 70 milliseconds is 0.8%.

The 99th percentile is 70 milliseconds because

\[

P(T\leq55)=P(X\leq1)=1-(0.2)^2=0.96,

\]

while

\[

P(T\leq70)=P(X\leq2)=1-(0.2)^3=0.992.

\]

Approach and Outcome

The engineering team may provision the service using an expected latency of 43.75 milliseconds, but capacity planning should also use tail probabilities rather than the mean alone. Under the model, 99.2% of requests complete within 70 milliseconds.

However, the result depends on independent attempts and a constant success probability. Network congestion, device overheating, or corrupted inputs may create dependent failures and heavier latency tails. The model should therefore be validated against production telemetry and stress conditions.

Learning Points

1. An operational quantity can be represented as a transformation of a standard random variable.

2. Mean latency alone does not establish service reliability.

3. Variance and quantiles provide complementary information.

4. Model assumptions must be tested before probabilities are used in service guarantees.

9. Practical Challenges, Governance, and Best Practices

9.1 Model selection and validation

A mathematically convenient distribution may not match observed data. Histograms, empirical CDFs, quantile plots, goodness-of-fit procedures, and out-of-sample checks should be used to assess fit. Discrete, continuous, censored, and mixed observations must be distinguished correctly.

9.2 Sensitivity to tails and outliers

Means, variances, skewness, and kurtosis can be unstable under heavy tails or contamination. Robust summaries such as medians, quantiles, median absolute deviation, and trimmed means may be preferable. Infinite theoretical moments cannot be repaired merely by obtaining finite sample estimates.

9.3 Dependence

Assuming independence simplifies variance and generating-function calculations but may seriously underestimate risk. Temporal dependence, shared infrastructure, repeated users, and common environmental conditions frequently induce correlation in AI data.

9.4 Numerical computation

For large or extreme values, direct evaluation of probabilities can underflow or overflow. Log-probability calculations, stable scientific libraries, simulation, and automatic differentiation are common tools. Monte Carlo methods estimate expectations by sample averages, but accuracy depends on sample size, variance, and dependence among simulated observations.

9.5 Ethical and governance implications

Expected performance can conceal uneven risk across demographic or operational groups. An average false-negative rate, latency, or loss should be supplemented with subgroup distributions and tail measures. Data collection practices must respect privacy and applicable regulation. Probabilistic claims should communicate assumptions, uncertainty, and validation limits rather than presenting model outputs as certainty.

9.6 Contemporary AI applications

Modern probabilistic AI relies heavily on transformations and moments. Variational inference uses expectations to optimize distributional approximations. The reparameterization technique expresses a random variable as a differentiable transformation of parameter-free noise, permitting gradient-based optimization under appropriate regularity conditions. Normalizing flows use invertible transformations and Jacobian determinants to create flexible probability densities. Uncertainty-aware systems increasingly report predictive distributions rather than single predictions.

Recommended professional practice includes checking support and units, verifying normalization, testing moment existence, retaining covariance terms unless independence is justified, evaluating tail behavior, and documenting modeling assumptions.

Chapter Summary

A random variable maps experimental outcomes to numerical values. Discrete variables are described by PMFs, absolutely continuous variables by PDFs, and all real-valued random variables by CDFs. A PDF is recovered from an absolutely continuous CDF by differentiation almost everywhere.

Expectation provides a probability-weighted average, while variance and higher moments describe dispersion and shape. Covariance and correlation measure linear association, although zero correlation does not generally imply independence. Transformations require careful treatment of support, inverse branches, and Jacobian factors. Probability-generating, moment-generating, cumulant-generating, and characteristic functions encode distributions and facilitate moment and sum calculations.

In AI and operational analytics, these methods support uncertainty quantification, latency modeling, simulation, risk assessment, and probabilistic learning. Their conclusions remain conditional on model assumptions, moment existence, data quality, dependence structure, and responsible interpretation.

Key Terms

Random Variable: A measurable numerical function defined on a sample space.

Support: The set of values a random variable can take with positive mass or density relevance.

Probability Mass Function: A function assigning probabilities to the values of a discrete random variable.

Probability Density Function: A nonnegative function whose integral over a region gives probability for an absolutely continuous variable.

Cumulative Distribution Function: The function \(F_X(x)=P(X\leq x)\).

Absolute Continuity: A property ensuring that a CDF can be represented as the integral of a density.

Expectation: The probability-weighted average of a random variable.

Variance: The expected squared deviation from the mean.

Standard Deviation: The positive square root of variance.

Raw Moment: An expectation of the form \(E[X^r]\).

Central Moment: An expectation of the form \(E[(X-E[X])^r]\).

Skewness: A standardized third central moment measuring asymmetry.

Kurtosis: A standardized fourth central moment sensitive to extreme deviations.

Covariance: A measure of linear co-variation between two variables.

Correlation: Standardized covariance with values from \(-1\) to \(1\).

Jacobian: A derivative matrix whose determinant adjusts density under multivariate transformation.

Probability-Generating Function: A generating function for a nonnegative integer-valued random variable.

Moment-Generating Function: The function \(M_X(t)=E[e^{tX}]\), when finite.

Characteristic Function: The always-existing function \(\phi_X(t)=E[e^{itX}]\).

Cumulant: A quantity obtained from derivatives of the logarithm of the MGF.

Review Questions

Short Answer Questions

1. Distinguish a PMF from a PDF.

2. State the four defining properties of a CDF.

3. Why is a density value not generally a probability?

4. State the relationship between variance and the first two raw moments.

5. Under what condition does zero covariance imply independence for a pair of normal variables?

Long Answer Questions

1. Explain discrete, absolutely continuous, and mixed random variables, giving an example of each.

2. Derive the formula \(\operatorname{Var}(X)=E[X^2]-E[X]^2\) and explain its units.

3. Compare covariance and correlation, including their interpretations and limitations.

4. Derive the density transformation formula for a strictly monotone function and explain the role of the absolute derivative.

5. Compare probability-generating, moment-generating, cumulant-generating, and characteristic functions.

Analytical / Application Questions

1. A model makes \(X\) errors with probabilities \(P(X=0)=0.3\), \(P(X=1)=0.4\), and \(P(X=2)=0.3\). Compute the mean, variance, and expected cost if the cost is \(C=20X+10\).

2. Two model losses have variances 9 and 16 and covariance 6. Compute the variance of their difference and interpret the role of covariance.

3. Let \(X\sim\operatorname{Uniform}(0,2)\) and \(Y=X^2\). Derive the CDF and PDF of \(Y\), clearly identifying its support.

Multiple Choice Questions with Answers

Question 1

Which statement correctly describes a probability density function f_X(x)?

A. f_X(x) equals P(X=x) for every continuous random variable.

B. f_X(x) must be nonnegative and integrate to 1 over the real line.

C. f_X(x) must be no greater than 1 at every point.

D. f_X(x) must be a nondecreasing function.

Correct Answer: B. f_X(x) must be nonnegative and integrate to 1 over the real line.

Question 2

Under which condition can a CDF F_X be represented as F_X(x)=∫_{−∞}^x f_X(u)du for some density f_X?

A. F_X is absolutely continuous.

B. F_X is merely right-continuous.

C. X has a finite mean.

D. X has countable support.

Correct Answer: A. F_X is absolutely continuous.

Question 3

If E[X]=4 and E[X²]=20, what is Var(X)?

A. 4

B. 8

C. 16

D. 20

Correct Answer: A. 4

Question 4

A random variable X has variance 3. If Y=5X−2, what is Var(Y)?

A. 13

B. 15

C. 75

D. 225

Correct Answer: C. 75

Question 5

If X and Y are independent and have finite second moments, which conclusion must hold?

A. E[X]=E[Y]

B. Cov(X,Y)=0

C. Var(X)=Var(Y)

D. X+Y is normally distributed

Correct Answer: B. Cov(X,Y)=0

Question 6

Suppose Var(X)=4, Var(Y)=9, and Cov(X,Y)=2. What is Var(X+Y)?

A. 11

B. 13

C. 15

D. 17

Correct Answer: D. 17

Question 7

For a nonnegative integer-valued variable X, which derivative of its probability-generating function equals E[X]?

A. G_X(0)

B. G_X'(0)

C. G_X'(1)

D. G_X''(1)

Correct Answer: C. G_X'(1)

Question 8

Which generating function exists for every probability distribution?

A. Moment-generating function

B. Probability-generating function

C. Cumulant-generating function

D. Characteristic function

Correct Answer: D. Characteristic function

Question 9

If X is uniformly distributed on (0,1) and Y=10+5X, what is the density of Y on (10,15)?

A. 1/10

B. 1/5

C. 1

D. 5

Correct Answer: B. 1/5

Question 10

An inference system has latency T=40+15X milliseconds, where X is the number of failures before success and each independent attempt succeeds with probability 0.8. What is P(T>70)?

A. 0.008

B. 0.032

C. 0.040

D. 0.200

Correct Answer: A. 0.008