36-220 Exam 1 Guide
36-220 Exam 1
0 / 14
Jump to a module ↓

    Everything on Exam 1, from zero

    This assumes you have never been to a single class. Every symbol is defined before it is used, every formula on your sheet is explained in plain English, and every worked example is a real problem from your worksheets or homework. Work top to bottom.

    00

    10 minutes

    Start here

    Your course is 36-220, Engineering Statistics and Quality Control. Exam 1 covers lectures from Friday of Week 1 through Monday of Week 5. That is ten lectures. Here is the whole territory in one sentence:

    The arc of the course so far

    You learn to describe uncertainty with probability, package a random outcome into a random variable, describe that variable with a distribution, summarize the distribution with moments, recognize four standard distributions by name, then run the whole machine backwards: given data, estimate the unknown parameter that generated it, and describe how good that estimate is.

    The one picture the whole course hangs on

    Your professor drew this on day one and again in lecture 4M. If you understand it, you understand what every question is secretly asking.

    Population true, fixed, unknown Data random Statistic random too experiment summarize statistical inference
    Left to right is the real world happening to you. Right to left, in blue, is the thing you are being trained to do.
    • Population. The complete set of things you care about, described by numbers called parameters. The true average resistance of every resistor the factory will ever make. These numbers are real, fixed, and you will never know them.
    • Experiment. You sample some data from that population. This is where randomness enters: run it again and you get different numbers.
    • Statistic. A number you compute from the data. The average of your 36 resistors, say. Because the data were random, the statistic is random too. That single sentence is the deepest idea on this exam.
    • Inference. Reasoning backwards from your one statistic to a claim about the population parameter you can never see.

    What is on the exam

    TopicLectureModule here
    Unconditional probability, sets, 2×2 tables1F2
    Conditional probability, trees, Bayes2M3
    Random variables, p.m.f.s and p.d.f.s2W4
    Moments: expected value, variance, standard deviation2F5
    Binomial and Poisson distributions3W6
    Normal and exponential distributions3F6
    Likelihood and maximum likelihood estimation4M7
    Bias, variance, mean-squared error4W8
    Statistics, sampling distributions, sufficiency4F9
    Central limit theorem5M10

    What is not on the exam

    Do not study these

    Two of the lectures in your folder are past the cutoff. 5W (The Fundamental Equation of Statistical Inference) and 5F (Introduction to Hypothesis Testing) are Exam 2 material — the Exam 2 study guide says its coverage starts at "Wednesday of Week 5."

    So you can ignore: null and alternative hypotheses, H₀ / Hₑ, level of a test, Type I error, p-values, power, rejection regions, confidence intervals, and the equation $F_Y(y;\theta) - q = 0$. If one of those words appears in your notes, skip it.

    Also skip

    Python. Your study guide says there will be no direct coding — at most you might be shown a codelet and asked what quantity it represents. Worksheet 4W is entirely a coding exercise; it is not reproduced here. Everywhere the lecture notes used norm.cdf or minimize, this guide shows you the setup that comes before the code, because that is the part you can be asked to produce.

    How to spend the time

    The full path is about eight hours. If you have less, here is the triage, in the order that buys the most points per minute:

    If you haveDo these modules
    8 hoursAll of them, in order.
    5 hours1, 2, 3, 4, 5, 6, 7, then 12. Skim 9 and 10.
    3 hours1, then 6 (distributions), then 7 (MLE), then 3 (Bayes). These four carry the most marks.
    1 hourModule 11, the formula-sheet decoder, plus the trap list at the end of module 12.

    Tick the box at the bottom of each module as you finish it. Progress is saved in this browser.

    01

    20 minutes

    The language

    Most of the difficulty in this course is notation, not mathematics. Read this module once carefully and the formula sheet stops looking like hieroglyphics.

    The single most important distinction: capital vs lowercase

    Your professor underlines this repeatedly. It is the difference between a machine and a number that fell out of the machine.

    Capital $X$

    A random variable. A rule, a procedure, a thing that has not happened yet. It has a distribution, an expected value, a variance. You can ask "what is the probability that $X$ lands between 1 and 2?"

    lowercase $x$

    A specific value the random variable took, or could take. A plain number. It has no distribution. Once you have rolled the die and seen a 4, that 4 is $x$.

    So $\mathbb{P}(X = x)$ reads: "the probability that the random variable $X$ comes out equal to the particular number $x$." And $p_X(x)$ is a function: you feed it a number $x$, it hands back the probability that $X$ equals it. The subscript $X$ just labels which random variable the function belongs to.

    Say it out loud

    $p_X(x)$ — "pee sub ex of ex". $f_X(x)$ — "eff sub ex of ex". $F_X(x)$ — "capital eff sub ex of ex". The capital/lowercase distinction between $f$ and $F$ matters as much as the one between $X$ and $x$.

    Decorations on letters

    SymbolNameWhat it means
    $\bar{X}$"X bar"The sample mean: add up your $n$ data and divide by $n$. Capital, so it is a random variable.
    $\hat{\theta}$"theta hat"An estimate of $\theta$. The hat always means "this is our guess at the thing underneath, computed from data." Never the truth itself.
    $\hat{\theta}_{MLE}$"theta hat em-ell-ee"The estimate produced specifically by the maximum likelihood recipe.
    $A^c$"A complement", "not A"Everything in the sample space that is not in $A$.
    $X_i$"X sub i"The $i$-th data point out of $n$. So $X_1, X_2, \ldots, X_n$ is your whole dataset.
    $\mathbf{x}$ or $\vec{x}$"the data vector"Shorthand for the whole collection $\{x_i\}_{i=1}^{n}$ at once.

    Greek letters, and which are parameters

    Greek letters are almost always parameters: fixed, true, unknown numbers that control a distribution's shape and location. They are not random. Your professor writes this explicitly in 2W: "values of $\theta$ are fixed (not random)."

    SymbolSayRole
    $\Omega$"omega", capitalThe sample space: the set of every possible outcome of an experiment.
    $\theta$"theta"The generic name for "whatever parameter we are trying to estimate." Also the specific parameter of the exponential distribution.
    $\mu$"mew"The population mean. The true average. $\mathbb{E}(X)=\mu$.
    $\sigma$"sigma", lowercaseThe population standard deviation. A width.
    $\sigma^2$"sigma squared"The population variance. $\mathbb{V}(X)=\sigma^2$.
    $\lambda$"lambda"The parameter of the Poisson distribution: the expected count per window.
    $\Sigma$"sigma", capitalNot a parameter. This is the summation operator: add things up.
    $\Pi$"pi", capitalThe product operator: multiply things together. Central to likelihood.
    $\Gamma$"gamma", capitalThe gamma distribution (and the gamma function). You only need to recognize the name.
    Summation and product
    $$\sum_{i=1}^{n} X_i = X_1 + X_2 + \cdots + X_n \qquad\qquad \prod_{i=1}^{n} X_i = X_1 \cdot X_2 \cdots X_n$$
    • $i$the counter. It starts at the number below the symbol and climbs to the number above.
    • $n$how many data points you have.

    When you use it: $\sum$ appears everywhere. $\prod$ appears in exactly one place on this exam — the likelihood function — and the first thing you do is take a logarithm to turn it back into a $\sum$.

    Set symbols

    SymbolSayMeans
    $A \cup B$"A union B"A or B (or both). The bigger region.
    $A \cap B$"A intersect B"A and B, simultaneously. The overlap.
    $A^c$"A complement"Not A.
    $A \subset B$"A is a subset of B"Every outcome in $A$ is also in $B$. If $A$ happens, $B$ automatically happened.
    $\varnothing$"the empty set", "the null set"Nothing. $A \cap B = \varnothing$ means $A$ and $B$ can never happen together.
    $x \in [0,1]$"x is in the interval zero to one"Square brackets include the endpoints; round brackets exclude them. $(0,1]$ excludes 0, includes 1.
    $\mathbb{R}$"the reals"The number line.

    Probability and distribution symbols

    SymbolNameWhat it does
    $\mathbb{P}(A)$probability of AA number between 0 and 1.
    $\mathbb{P}(A \mid B)$"probability of A given B"The probability of $A$ in a world where you already know $B$ happened.
    $p_X(x)$probability mass function (p.m.f.)Discrete variables. Returns an actual probability.
    $f_X(x)$probability density function (p.d.f.)Continuous variables. Returns a density, not a probability. You must integrate it to get a probability.
    $F_X(x)$cumulative distribution function (c.d.f.)$\mathbb{P}(X \le x)$ — the probability of landing at or below $x$. Works for both kinds.
    $F_X^{-1}(q)$inverse c.d.f., quantile functionYou give it a probability $q$, it gives you back the value $x$ that has that much probability below it.
    $\mathbb{E}(X)$expected value of XThe probability-weighted average of $X$. A single number.
    $\mathbb{V}(X)$variance of XHow spread out $X$ is, in squared units.
    $\mathcal{L}(\theta \mid \mathbf{x})$likelihoodHow plausible the parameter value $\theta$ is, given the data you saw.
    $\ell(\theta \mid \mathbf{x})$log-likelihood$\ln$ of the above. Script lowercase ell.

    Relationship symbols

    SymbolSayMeans
    $X \sim \mathcal{N}(\mu,\sigma^2)$"X is distributed as...""$X$ is sampled from a normal distribution with parameters $\mu$ and $\sigma^2$."
    $X \stackrel{d}{\to} Y$"converges in distribution to"As $n$ grows, the shape of $X$'s distribution looks more and more like $Y$'s. This is the central limit theorem's verb.
    $\dot\sim$ or $\approx$"is approximately distributed as"Close enough for the sample size you have.
    i.i.d."eye-eye-dee"Independent and identically distributed. Every data point came from the same distribution, and none of them influenced any other. Almost every formula in the back half of this course assumes it.
    The three-layer model — memorize this
    1. Parameters ($\theta$, $\mu$, $\sigma$, $\lambda$, $p$): fixed, true, unknown. Not random.
    2. Data ($X_1,\ldots,X_n$): random. Change if you rerun the experiment.
    3. Statistics ($\bar{X}$, $S_X$, $\hat{\theta}$): computed from data, therefore also random, therefore they have their own distributions (called sampling distributions).

    Half the exam is testing whether you know which layer a given symbol lives on.

    Trap

    $\sigma$ and $S_X$ look interchangeable but are not. $\sigma$ is the population standard deviation — a parameter, a fixed truth. $S_X$ is the sample standard deviation — a statistic you computed from your $n$ numbers. And $\mathrm{SE}(\bar{X}) = \sigma/\sqrt{n}$ is a third thing entirely: the standard deviation of the statistic $\bar{X}$, not of the data. Module 9 pulls these apart properly.

    02

    40 minutes · lecture 1F

    Unconditional probability

    "Unconditional" means: the probability of an event, full stop, with no side information given to you. Everything in this module is about counting and combining regions.

    What probability is

    Your professor gives a specific definition: the long-run relative frequency of the occurrence of an event. Flip a coin a million times; the fraction that came up heads is the probability of heads. This is called the frequentist interpretation, and it is the one this course uses. (The alternative, the Bayesian paradigm, treats probability as a degree of belief. You only need to know the word.)

    Experiment, outcome, sample space, event

    • Experiment: the act of sampling data from a population. Flipping two coins.
    • Sample space $\Omega$: the set of all possible outcomes. For two coins, $\Omega = \{HH, HT, TH, TT\}$.
    • Simple event: one single element of $\Omega$. $HH$ is a simple event.
    • Compound event: a set of several simple events grouped by some description. "At least one tails" is the compound event $\{HT, TH, TT\}$.

    Lecture 1F — three sample spaces

    • (a)Two coins. $\Omega = \{HH, HT, TH, TT\}$ — four simple events.
    • (b)Free throws until she misses. With $H$ = hit and $M$ = miss: $\Omega = \{M, HM, HHM, HHHM, \ldots\}$. The number of simple events can be infinite — even uncountably so.
    • (c)Reading any two of three books. If order does not matter, $\Omega = \{AB, AC, BC\}$. If it does, $\Omega = \{AB, BA, AC, CA, BC, CB\}$. Always ask whether order matters — it changes the answer.

    Combining events

    A B Ω A ∪ B A or B A B A ∩ B A and B A Ω Aᶜ not A A B A ∩ B = ∅ disjoint
    The whole rectangle is the sample space $\Omega$. Two events are disjoint (also called mutually exclusive) when they cannot both happen.

    Two identities from your sheet that fall straight out of the picture:

    • $A \cup A^c = \Omega$ — an event either happens or it doesn't, and that covers everything.
    • $A \cap B = \varnothing \iff A$ and $B$ are mutually exclusive.

    The algebra laws on your sheet

    These are on the formula sheet and you will rarely use them directly, but they turn ugly event descriptions into manageable ones.

    Distributive and associative laws
    $$A \cap (B \cup C) = (A\cap B) \cup (A \cap C) \qquad A \cup (B \cap C) = (A\cup B) \cap (A \cup C)$$ $$A \cup (B \cup C) = (A\cup B) \cup C \qquad A \cap (B \cap C) = (A \cap B) \cap C$$

    In words: $\cap$ and $\cup$ distribute over each other exactly the way $\times$ distributes over $+$ in ordinary algebra, and grouping does not matter when the operator is the same. When you use it: when a problem describes an event with a nested "and/or" and you need to break it into disjoint pieces you can add.

    De Morgan's laws
    $$(A \cap B)^c = A^c \cup B^c \qquad\qquad (A \cup B)^c = A^c \cap B^c$$

    First law in words: "not (both A and B)" is the same as "either not-A or not-B." If it is not true that both alarms fired, then at least one of them failed.

    Second law in words: "not (A or B)" is the same as "neither A nor B." If neither device detected smoke, then device A failed and device B failed.

    When you use it: whenever a question says "neither", "none", or "at least one." The second law is the engine behind the single most useful trick in this course: $\mathbb{P}(\text{at least one}) = 1 - \mathbb{P}(\text{none})$.

    The axioms

    Events alone are not enough — a sample space tells you what can happen but nothing about how often. You bolt on a probability measure $\mathbb{P}(\cdot)$: a function from subsets of $\Omega$ to the real line obeying three rules. Your professor calls them self-evident, which is why they are axioms and not theorems.

    The three axioms of probability
    $$\mathbb{P}(\Omega) = 1 \qquad\quad 0 \le \mathbb{P}(A) \le 1 \;\; \forall A \in \Omega \qquad\quad \mathbb{P}\!\left(\bigcup_{i=1}^n A_i\right) = \sum_{i=1}^{n}\mathbb{P}(A_i) \;\; \text{if disjoint}$$
    • 1stThe experiment must produce some outcome. Total probability is 1.
    • 2ndNo probability is negative or above 1. ($\forall$ means "for all".)
    • 3rdIf events cannot overlap, the probability that one of them happens is just the sum. The word "disjoint" is doing all the work — drop it and the rule is false.

    When you use it: the third axiom is why you can add up p.m.f. values over a range, and why decision-tree branches add.

    The rules you actually compute with

    Complement rule
    $$\mathbb{P}(A) = 1 - \mathbb{P}(A^c)$$

    When you use it: constantly. Any time the event you want is messy but its opposite is clean. "At least one," "at most," "more than" — all flip.

    Additive law
    $$\mathbb{P}(A \cup B) = \mathbb{P}(A) + \mathbb{P}(B) - \mathbb{P}(A\cap B)$$
    • $-\,\mathbb{P}(A\cap B)$corrects for double-counting: the overlap got added once by $\mathbb{P}(A)$ and once by $\mathbb{P}(B)$.

    When you use it: any "or" question. If the events happen to be disjoint the last term is zero and you are back to the third axiom.

    Neither-nor
    $$\mathbb{P}(A^c \cap B^c) = 1 - \mathbb{P}(A \cup B)$$

    This is De Morgan plus the complement rule in one line. When you use it: a problem gives you "the probability of missing both" and wants something else — run this backwards to recover $\mathbb{P}(A\cup B)$, then use the additive law.

    Independence

    Independence
    $$\mathbb{P}(A \cap B) = \mathbb{P}(A)\,\mathbb{P}(B) \quad \iff \quad A \text{ and } B \text{ are independent}$$

    Equivalently $\mathbb{P}(A\mid B) = \mathbb{P}(A)$ and $\mathbb{P}(B\mid A) = \mathbb{P}(B)$. If any one of the three holds, all three hold. In words: knowing that $B$ happened tells you nothing about whether $A$ happened.

    When you use it: only when the problem says so, or when the physical setup obviously implies it (separate coin flips, separate relays, separate wafers). You may not assume it.

    The classic trap: disjoint is not independent

    They sound similar and are close to opposites. Lecture 2M proves it. Suppose $A$ and $B$ are disjoint with $\mathbb{P}(A)>0$ and $\mathbb{P}(B)>0$. Then $\mathbb{P}(A\cap B)=0$, so

    $$\mathbb{P}(A\mid B) = \frac{\mathbb{P}(A\cap B)}{\mathbb{P}(B)} = \frac{0}{\mathbb{P}(B)} = 0 \ne \mathbb{P}(A)$$

    Knowing $B$ happened tells you a great deal: it tells you $A$ definitely did not. So disjoint events are maximally dependent, never independent.

    The 2×2 table

    This is the tool your professor introduces for "simple probability problems," and it is the fastest way to handle any question about two events. Build a grid: columns $A$ and $A^c$, rows $B$ and $B^c$. The four interior cells are the four possible joint outcomes; they sum to 1. The row and column totals are the marginal probabilities.

    $A$$A^c$
    $B$$\mathbb{P}(A\cap B)$$\mathbb{P}(A^c\cap B)$$\mathbb{P}(B)$
    $B^c$$\mathbb{P}(A\cap B^c)$$\mathbb{P}(A^c\cap B^c)$$\mathbb{P}(B^c)$
    $\mathbb{P}(A)$$\mathbb{P}(A^c)$1

    How to use it: write in whatever the problem gives you, then fill the rest by subtraction — each row must hit its row total, each column its column total, and the four interior cells must sum to 1. Once the grid is complete, every question about $A$ and $B$ is a matter of reading off and adding cells.

    Lecture 1F, worked in class

    You are given $\mathbb{P}(A)=0.5$, $\mathbb{P}(B)=0.3$, $\mathbb{P}(A\cap B)=0.1$. What is the probability that either $A$ or $B$ occurs, but not both?

    Show solution

    "Either one but not both" is the event $(A\cap B^c) \cup (A^c \cap B)$ — two cells of the grid, and they are disjoint, so you add them. Fill the table:

    $A$$A^c$
    $B$0.10.20.3
    $B^c$0.40.30.7
    0.50.51

    Start with the given 0.1 in the top-left. The $B$ row must total 0.3, so the cell beside it is $0.3-0.1=0.2$. The $A$ column must total 0.5, so the cell below is $0.5-0.1=0.4$. The last cell is whatever makes everything sum to 1: $0.3$.

    Now read off the two "exactly one" cells: $0.4 + 0.2$.

    0.6

    Homework 1, Question 2 — free throws

    A player shoots two free throws. $\mathbb{P}(\text{first made}) = 0.8$, $\mathbb{P}(\text{second made}) = 0.6$.

    1. What is the probability the player makes both? (If it cannot be determined, say so.)
    2. Now additionally $\mathbb{P}(\text{miss both}) = 0.1$. Now what is $\mathbb{P}(\text{both})$?
    3. Now instead: $\mathbb{P}(\text{first made})=0.8$, $\mathbb{P}(\text{miss both})=0.1$, $\mathbb{P}(\text{exactly one})=0.3$. Now what is $\mathbb{P}(\text{both})$?
    4. Now: $\mathbb{P}(\text{both})=0.4$, $\mathbb{P}(\text{miss both})=0.2$. One point per made shot — what is the average number of points per trip to the line?
    Show solution

    Let $A$ = "makes the first", $B$ = "makes the second".

    (a) You want $\mathbb{P}(A\cap B)$. You are given only $\mathbb{P}(A)$ and $\mathbb{P}(B)$. Two marginals do not determine an intersection — you would need independence, and nothing says the shots are independent (a player who makes the first may be more relaxed on the second).

    Cannot be determined

    (b) "Miss both" is $A^c \cap B^c$, so $\mathbb{P}(A^c\cap B^c)=0.1$. Use the neither-nor identity:

    $$\mathbb{P}(A\cup B) = 1 - \mathbb{P}(A^c\cap B^c) = 1 - 0.1 = 0.9$$

    Then the additive law, rearranged:

    $$\mathbb{P}(A\cap B) = \mathbb{P}(A)+\mathbb{P}(B)-\mathbb{P}(A\cup B) = 0.8 + 0.6 - 0.9$$

    0.5

    (c) Again $\mathbb{P}(A\cup B)=1-0.1=0.9$. But now split $A\cup B$ into two disjoint pieces — "exactly one" and "both":

    $$\mathbb{P}(A\cup B) = \mathbb{P}(\text{exactly one}) + \mathbb{P}(\text{both}) \;\Rightarrow\; 0.9 = 0.3 + \mathbb{P}(\text{both})$$

    0.6

    Sanity check: $\mathbb{P}(B) = \mathbb{P}(A\cup B) - \mathbb{P}(A) + \mathbb{P}(A\cap B) = 0.9-0.8+0.6 = 0.7$, which is a legal probability. Good.

    (d) Let $X$ = points scored, so $X \in \{0,1,2\}$. You are given $\mathbb{P}(X=2)=0.4$ and $\mathbb{P}(X=0)=0.2$, so $\mathbb{P}(X=1) = 1-0.4-0.2 = 0.4$. "Average number of points" is the expected value (module 5):

    $$\mathbb{E}(X) = 0(0.2) + 1(0.4) + 2(0.4) = 1.2$$

    1.2 points

    Homework 2, Question 3 — smoke detector

    A system uses devices A and B. Given smoke is present, $\mathbb{P}(A)=0.95$, $\mathbb{P}(B)=0.8$, $\mathbb{P}(A\cap B)=0.78$.

    1. Find $\mathbb{P}(A \cup B)$.
    2. What is the probability A fails to detect, given B does detect?
    Show solution

    (a) Straight additive law: $0.95 + 0.80 - 0.78$.

    0.97

    (b) You want $\mathbb{P}(A^c \mid B)$. Use the conditional definition (module 3) and get the numerator by subtraction — the $B$ row splits into "B and A" plus "B and not A":

    $$\mathbb{P}(A^c\cap B) = \mathbb{P}(B) - \mathbb{P}(A\cap B) = 0.80 - 0.78 = 0.02$$

    $$\mathbb{P}(A^c\mid B) = \frac{0.02}{0.80} = 0.025$$

    0.025

    Note $\mathbb{P}(A^c)=0.05$ but $\mathbb{P}(A^c\mid B)=0.025$. Knowing B fired makes A's failure less likely. The events are not independent.

    Homework 1, Question 5 — two dice, order matters

    Roll a fair six-sided die twice, recording both results in order. Let $A$ = first roll is 2 or 6; $B$ = second roll is 1 or 5; $C$ = the sum is $>7$.

    1. List the simple events in $A\cap B$.
    2. List $A \cap C$.
    3. List $A\cap B\cap C$.
    4. Find $\mathbb{P}(A\cap B)$ by counting.
    Show solution

    $\Omega$ has $6\times 6 = 36$ equally likely ordered pairs.

    (a) First $\in\{2,6\}$ and second $\in\{1,5\}$ — every combination:

    (2,1), (2,5), (6,1), (6,5)

    (b) First $\in \{2,6\}$ and sum $>7$. If the first is 2, you need the second $>5$, so only 6. If the first is 6, you need the second $>1$, so 2 through 6.

    (2,6), (6,2), (6,3), (6,4), (6,5), (6,6)

    (c) Take the four pairs from (a) and keep those with sum $>7$: sums are 3, 7, 7, 11. Note that $7$ is not $>7$ — the inequality is strict.

    (6,5) only

    (d) 4 favourable simple events out of 36, each equally likely:

    $$\mathbb{P}(A\cap B) = \frac{4}{36} = \frac{1}{9}$$

    1/9

    Cross-check with independence: the two rolls genuinely are independent, and $\mathbb{P}(A)=2/6$, $\mathbb{P}(B)=2/6$, so $\mathbb{P}(A)\mathbb{P}(B) = 4/36$. Matches.

    Homework 1, Question 4 — complements

    1. In a sample space $\Omega$, what is the complement of $A\cap B$?
    2. What is the complement of $A \cup B$?
    3. If $A\cup B = \Omega$ and $A \cap B = B$, what must be true about $A$?
    Show solution

    (a) De Morgan: $(A\cap B)^c = A^c \cup B^c$. In words, "not both" means "at least one of them failed."

    (b) $(A\cup B)^c = A^c \cap B^c$. "Neither."

    (c) Read the second condition first. $A\cap B = B$ says the overlap is all of $B$, which means every outcome in $B$ is also in $A$: $B \subset A$. But then $A \cup B$ adds nothing new to $A$, so $A\cup B = A$. And you were told $A\cup B=\Omega$.

    $A = \Omega$

    Homework 1, Question 3 — flipping until the first head

    Flip a fair coin, recording H or T each time.

    1. Write the sample space for "flip until the first heads."
    2. Assuming independence and a fair coin, what is the probability you need four or more tosses to see the first head?
    3. What is the probability the first head appears on an even flip? You may use $\mathbb{P}(A)[1+\mathbb{P}(B)+\mathbb{P}(B)^2+\cdots] = \dfrac{\mathbb{P}(A)}{1-\mathbb{P}(B)}$.
    Show solution

    (a) $\Omega = \{H,\; TH,\; TTH,\; TTTH,\; \ldots\}$ — infinitely many simple events.

    (b) Flip the event. "Four or more tosses needed" happens exactly when the first three tosses are all tails — nothing about the fourth matters. By independence:

    $$\mathbb{P}(TTT) = \left(\tfrac12\right)^3 = \tfrac18$$

    1/8

    (c) "First head on flip 2, or 4, or 6, ..." — these are disjoint, so add:

    $$\mathbb{P} = \left(\tfrac12\right)^2 + \left(\tfrac12\right)^4 + \left(\tfrac12\right)^6 + \cdots = \tfrac14\left[1 + \tfrac14 + \left(\tfrac14\right)^2+\cdots\right]$$

    Apply the given identity with $\mathbb{P}(A)=\tfrac14$ and $\mathbb{P}(B)=\tfrac14$:

    $$= \frac{1/4}{1-1/4} = \frac{1/4}{3/4}$$

    1/3

    Sanity check: the odd flips must then carry probability $2/3$. That is reasonable — flip 1 alone already has probability $1/2$.

    Traps in this module
    • Assuming independence that was not given. HW1 Q2(a) is a trap set on purpose. If the problem does not say independent and the physical setup does not force it, you cannot multiply.
    • Forgetting the $-\mathbb{P}(A\cap B)$ in the additive law. Only drop it if the events are disjoint.
    • Strict vs non-strict inequalities. "$>7$" excludes 7. "At least 3" includes 3. Read carefully — for discrete variables this changes the answer.
    • Order. Ask every time whether $(2,6)$ and $(6,2)$ count as different outcomes.
    03

    45 minutes · lecture 2M

    Conditional probability

    Conditioning means someone handed you a piece of information. The universe shrinks to the region where that information is true, and all probabilities are rescaled inside it.

    Definition of conditional probability
    $$\mathbb{P}(A \mid B) = \frac{\mathbb{P}(A \cap B)}{\mathbb{P}(B)}$$
    • $A\mid B$"$A$ given $B$". The condition goes to the right of the bar.
    • numeratorthe part of $A$ that survives inside $B$.
    • denominatorthe new total. You are no longer dividing by 1; you are dividing by however much of the world $B$ occupies.

    The picture: specifying a condition shrinks the sample space down to $B$. Inside that smaller world, $A$'s share is the overlap divided by $B$'s size. When you use it: any time a question contains "given," "if," "among those who," or "of the ones that tested positive."

    Causality is not implied

    Your professor stresses this. $\mathbb{P}(A\mid B)$ does not say $B$ caused $A$. It says nothing but: you have been told $B$ occurred, and nothing more.

    $\mathbb{P}(A \mid B)$ is not $\mathbb{P}(B \mid A)$

    Lecture 2M's example makes the point vividly. Let $F$ = "speaks French" and $B$ = "lives in Brussels."

    Roughly $4\times10^8$ French speakers out of $8\times 10^9$ people, so $\mathbb{P}(F) \approx \tfrac{1}{20}$. About $10^6$ people live in Brussels, so $\mathbb{P}(B)\approx \tfrac{1}{8000}$.

    About half of Brussels speaks French: $\mathbb{P}(F\mid B) \approx \tfrac12$. But running Bayes the other way, $\mathbb{P}(B\mid F) = \dfrac{\mathbb{P}(F\mid B)\mathbb{P}(B)}{\mathbb{P}(F)} = \dfrac{\tfrac12 \cdot \tfrac{1}{8000}}{\tfrac{1}{20}} = \tfrac{1}{800}$.

    So $\mathbb{P}(F \mid B) \approx 0.5$ while $\mathbb{P}(B\mid F) \approx 0.00125$. Same two events, answers differing by a factor of 400. Half of Brussels speaks French; almost no French speakers live in Brussels. Both are true and they are different statements.

    The four laws

    Multiplicative law
    $$\mathbb{P}(A\cap B) = \mathbb{P}(A\mid B)\,\mathbb{P}(B) = \mathbb{P}(B \mid A)\,\mathbb{P}(A)$$

    Just the definition rearranged. Note it works in either order — use whichever conditional the problem handed you.

    When you use it: to build an intersection out of a conditional. "First the bowl is chosen, then given that bowl a white ball is drawn" is exactly $\mathbb{P}(\text{bowl})\times\mathbb{P}(\text{white}\mid\text{bowl})$. Every branch of a decision tree is one application of this law.

    Law of total probability (L.o.T.P.)
    $$\mathbb{P}(A) = \sum_{i=1}^{n}\mathbb{P}(A\mid B_i)\,\mathbb{P}(B_i) \qquad \text{if the } B_i \text{ are disjoint and exhaustive}$$ $$\text{two-case version:}\qquad \mathbb{P}(A) = \mathbb{P}(A\mid B)\mathbb{P}(B) + \mathbb{P}(A\mid B^c)\mathbb{P}(B^c)$$
    • disjointno two $B_i$ can happen together.
    • exhaustiveone of them must happen — together they fill all of $\Omega$.

    Why it works: the event $A$ can be sliced as $A = (A\cap B)\cup(A\cap B^c)$, and those two slices are disjoint, so their probabilities add. Then expand each with the multiplicative law.

    When you use it: whenever you need an unconditional probability but the problem only gave you conditionals. It is almost always the first step, and it is almost always what builds the denominator for Bayes.

    Bayes' rule
    $$\mathbb{P}(A\mid B) = \frac{\mathbb{P}(B\mid A)\,\mathbb{P}(A)}{\mathbb{P}(B)}$$

    What it is for: reversing a conditional. The problem tells you $\mathbb{P}(\text{evidence}\mid\text{cause})$ and asks for $\mathbb{P}(\text{cause}\mid\text{evidence})$.

    The standard recipe, which covers nearly every Bayes question on this exam:

    1. Name the events. Put the thing you are told after the bar.
    2. Write down the given conditionals and the prior.
    3. Build the denominator with L.o.T.P.
    4. Divide.

    Decision trees

    Your study guide names decision trees explicitly. They are just the multiplicative law and L.o.T.P. drawn as a picture. Here is lecture 2M's nitrates problem as a tree.

    0.3 0.7 N nitrates Nᶜ none 0.9 0.1 0.1 0.9 R Rᶜ R Rᶜ 0.3×0.9 = 0.27 0.3×0.1 = 0.03 0.7×0.1 = 0.07 0.7×0.9 = 0.63 ∑ = 0.34
    Multiply along a path to get the probability of that whole path. Add across the paths that satisfy your event. The four leaves always sum to 1.

    Worksheet 2M & lecture 2M — nitrates

    If a water specimen contains nitrates, a solution turns it red 90% of the time; if there are no nitrates, it turns red 10% of the time. Nitrates exist in 30% of specimens.

    1. What is the probability a randomly selected specimen turns red?
    2. What is the probability a specimen contains nitrates, given it turned red?
    Show solution

    Step 1 — name the events and translate. Let $N$ = "contains nitrates", $R$ = "turns red". The phrase "if it contains nitrates, it turns red 90% of the time" means the condition is $N$ and the outcome is $R$:

    $$\mathbb{P}(R\mid N)=0.9 \qquad \mathbb{P}(R\mid N^c)=0.1 \qquad \mathbb{P}(N)=0.3$$

    (a) You want an unconditional probability from conditionals — that is L.o.T.P. The partition is $\{N, N^c\}$, which is disjoint and exhaustive:

    $$\mathbb{P}(R) = \mathbb{P}(R\mid N)\mathbb{P}(N) + \mathbb{P}(R\mid N^c)\mathbb{P}(N^c) = (0.9)(0.3) + (0.1)(0.7) = 0.27 + 0.07$$

    0.34

    (b) Now you want $\mathbb{P}(N\mid R)$ — the reverse of what you were given. Bayes:

    $$\mathbb{P}(N\mid R) = \frac{\mathbb{P}(R\mid N)\mathbb{P}(N)}{\mathbb{P}(R)} = \frac{(0.9)(0.3)}{0.34} = \frac{0.27}{0.34}$$

    27/34 ≈ 0.794

    Read the tree: of the total 0.34 worth of red specimens, 0.27 came from the nitrate branch. The answer is just that share. Notice the test looks impressive (90% sensitive) but a red result still only makes nitrates about 79% likely — because 70% of specimens had no nitrates to begin with, and 10% of those turn red too.

    Lecture 2M — relays in parallel

    Three relays sit in parallel between points A and B. Current flows if any one of them is operational. Each relay works with probability 0.9, independently.

    1. What is the probability current flows from A to B?
    2. Given current is flowing, what is the probability relay 1 is operational?
    Show solution

    Let $C$ = "current flows" and let $1, 2, 3$ denote "relay $i$ is closed", each with probability 0.9.

    (a) Current flows if relay 1 or 2 or 3 works, so $C = 1\cup 2\cup 3$. Apply the additive law twice, nesting:

    $$\mathbb{P}(1\cup 2) = \mathbb{P}(1)+\mathbb{P}(2)-\mathbb{P}(1\cap 2) = 0.9+0.9-0.9^2 = 0.99$$

    $$\mathbb{P}(C) = \mathbb{P}\big((1\cup2)\cup 3\big) = 0.99 + 0.9 - (0.99)(0.9) = 0.999$$

    0.999

    Much faster route

    Current fails only if all three relays fail. By independence, $\mathbb{P}(\text{all fail}) = (0.1)^3 = 0.001$, so $\mathbb{P}(C) = 1-0.001 = 0.999$. This is the complement trick, and on the exam it saves you two minutes.

    (b) You want $\mathbb{P}(1\mid C)$. The key observation: if relay 1 is operational, current necessarily flows, so the event $1$ is a subset of $C$, which means $1\cap C = 1$.

    $$\mathbb{P}(1\mid C) = \frac{\mathbb{P}(1\cap C)}{\mathbb{P}(C)} = \frac{\mathbb{P}(1)}{\mathbb{P}(C)} = \frac{0.9}{0.999}$$

    ≈ 0.901

    Homework 2, Question 1 — three bowls

    Bowl 1 has 1 white and 3 black balls; Bowl 2 has 2 white and 2 black; Bowl 3 has 3 white and 1 black. You pick a bowl at random, then draw two balls from it without replacement.

    1. What is the probability of drawing two white balls?
    2. Given you drew one white and one black, what is the probability they came from Bowl 1?
    Show solution

    Each bowl is chosen with probability $1/3$. "Without replacement" means the second draw's probability depends on the first — the denominator drops from 4 to 3.

    (a) Compute $\mathbb{P}(WW\mid \text{bowl } i)$ for each bowl by chaining the two draws:

    BowlContents$\mathbb{P}(WW\mid\text{bowl})$
    11W 3B$\frac14\cdot\frac03 = 0$
    22W 2B$\frac24\cdot\frac13 = \frac16$
    33W 1B$\frac34\cdot\frac23 = \frac12$

    Bowl 1 gives zero because after taking its only white ball there are none left. Now L.o.T.P. over the three bowls:

    $$\mathbb{P}(WW) = \tfrac13\left(0 + \tfrac16 + \tfrac12\right) = \tfrac13\cdot\tfrac23$$

    2/9

    (b) Let $D$ = "one white and one black." Careful: this can happen as $WB$ or $BW$, so add both orders.

    Bowl$\mathbb{P}(D\mid\text{bowl})$
    1$\frac14\cdot\frac33 + \frac34\cdot\frac13 = \frac14+\frac14 = \frac12$
    2$\frac24\cdot\frac23 + \frac24\cdot\frac23 = \frac13+\frac13 = \frac23$
    3$\frac34\cdot\frac13 + \frac14\cdot\frac33 = \frac14+\frac14 = \frac12$

    $$\mathbb{P}(D) = \tfrac13\left(\tfrac12+\tfrac23+\tfrac12\right) = \tfrac13 \cdot \tfrac53 = \tfrac59$$

    Bayes:

    $$\mathbb{P}(\text{Bowl 1}\mid D) = \frac{\mathbb{P}(D\mid \text{B1})\,\mathbb{P}(\text{B1})}{\mathbb{P}(D)} = \frac{\tfrac12\cdot\tfrac13}{\tfrac59} = \frac{1/6}{5/9} = \frac16\cdot\frac95$$

    3/10

    Below the prior of $1/3$, which makes sense: Bowl 2 is the most likely producer of a mixed pair, so seeing one shifts belief away from Bowls 1 and 3.

    Homework 2, Question 2 — the guessing student

    A student answers a four-choice question. $\mathbb{P}(\text{knows the answer}) = 0.8$, so $\mathbb{P}(\text{guesses}) = 0.2$. If guessing, she gets it right with probability 0.5.

    1. Given she got it right, what is the probability she knew it?
    2. What is the probability she both did not know it and got it right?
    Show solution

    Let $K$ = "knows", $G = K^c$ = "guesses", $R$ = "answers correctly". The unstated given: if she knows the answer she gets it right, so $\mathbb{P}(R\mid K)=1$.

    $$\mathbb{P}(R) = \mathbb{P}(R\mid K)\mathbb{P}(K) + \mathbb{P}(R\mid G)\mathbb{P}(G) = (1)(0.8) + (0.5)(0.2) = 0.9$$

    (a) $$\mathbb{P}(K\mid R) = \frac{(1)(0.8)}{0.9}$$

    8/9 ≈ 0.889

    (b) This one is not conditional — it asks for an intersection. Multiplicative law:

    $$\mathbb{P}(G\cap R) = \mathbb{P}(R\mid G)\mathbb{P}(G) = (0.5)(0.2)$$

    0.1

    Watch the wording. "Both A and B" is an intersection; "A given B" is conditional. Part (b) is the second leaf of the tree, not a ratio.

    Homework 2, Question 4 — the blood-typing race

    An accident victim will die in 10 minutes without type A Rh-positive blood from a single donor. Typing a donor takes 2 minutes and the transfusion takes 2 minutes. 40% of available donors are a match. Only one typing kit exists, and it processes one donor at a time. What is the probability the victim is saved?

    Show solution

    The hard part is the timing, not the probability. Work out how many donors you can afford to test.

    If you test $k$ donors before finding a match, that consumes $2k$ minutes, and then the transfusion needs 2 more. So you need

    $$2k + 2 \le 10 \quad\Longrightarrow\quad k \le 4$$

    You get four attempts. The victim is saved if at least one of the first four donors is a match. Flip it with the complement rule — "at least one match" is the opposite of "all four fail":

    $$\mathbb{P}(\text{saved}) = 1 - \mathbb{P}(\text{first four all fail}) = 1 - (0.6)^4 = 1 - 0.1296$$

    0.8704

    The independence here is physical: whether one donor matches says nothing about the next. Note this is also a binomial "at least one" — see module 6.

    Traps in this module
    • Flipping the bar. "The probability a specimen contains nitrates if it turns red" is $\mathbb{P}(N\mid R)$, not $\mathbb{P}(R \mid N)$. Underline the thing you are told and put it on the right.
    • Forgetting the denominator. Bayes has three ingredients; the denominator almost always needs L.o.T.P. built first.
    • Non-exhaustive partitions. L.o.T.P. only works if the $B_i$ cover everything. Three bowls, all equally likely — fine. Two of three bowls — not fine.
    • "Both" vs "given." "Didn't know it and got it right" is an intersection (multiply). "Knew it given right" is conditional (divide).
    • Both orders. "One white and one black" includes $WB$ and $BW$. Forgetting the second order halves your answer.
    04

    40 minutes · lecture 2W

    Random variables and distributions

    Sample-space probability works fine for three coin flips and collapses for five hundred. Random variables are the machinery that lets you stop listing outcomes and start doing algebra.

    What a random variable is

    Random variable
    $$X : \Omega \to \mathbb{R}$$

    A function that maps events in the sample space to the real number line. That is the whole definition, and your study guide asks for it in exactly those words.

    The point your professor underlines: a random variable carries no probability information. It is a labelling scheme. Defining $X$ = "number of heads in two flips" does not assume the coin is fair — it does not assume anything. Probability arrives separately, when you attach a distribution.

    Ω HH HT TH TT 0 1 2 x X = the number of heads
    Two outcomes, $HT$ and $TH$, both map to 1. That is fine — a function may be many-to-one. What it may not be is one-to-many.
    A consequence you will need in module 9

    Functions of random variables are themselves random variables. So $X^2$ is a random variable, $3X+1$ is a random variable, and — crucially — the sample mean $\bar{X}$ is a random variable, because it is a function of $X_1,\ldots,X_n$. This is why statistics have distributions.

    Two kinds of random variable

    Discrete

    The outcomes are separated values you could list: $\{0,1,2\}$, or all the integers $\mathbb{Z}$. Number of heads, number of flaws, number of draws.

    Continuous

    The outcomes fill an interval: $[0,2]$, or all of $\mathbb{R}$. Waiting time, resistance, height.

    p.m.f. versus p.d.f.

    A probability distribution is a function that takes a coordinate $x$ and returns either a probability (discrete) or a probability density (continuous). The word "distribution" simply signals that the function sums or integrates to 1 over its domain.

    DiscreteContinuous
    Nameprobability mass function (p.m.f.)probability density function (p.d.f.)
    Symbol$p_X(x)$$f_X(x)$
    Range of values$0 \le p_X(x) \le 1$$f_X(x) \ge 0$ — may exceed 1
    Totals to 1$\sum_{\text{all }x} p_X(x) = 1$$\int_{\text{all }x} f_X(x)\,dx = 1$
    Probability of a range$\mathbb{P}(a\le X\le b) = \sum_{x\in[a,b]} p_X(x)$$\mathbb{P}(a\le X\le b) = \int_a^b f_X(x)\,dx$
    Strict vs non-strictMatters. $\mathbb{P}(aDoes not matter. $\mathbb{P}(X=a)=0$ for any single point.
    Why a density can exceed 1

    $f_X(x)$ is not a probability. It is probability per unit of $x$, like grams per cubic centimetre. A very narrow distribution can be very tall. Only the area under it is a probability, and area is what must total 1. This is also why $\mathbb{P}(X=a)=0$ for a continuous variable: the area of a line with zero width is zero. So for continuous variables, $<$ and $\le$ are interchangeable; for discrete ones they absolutely are not.

    Cumulative distribution functions

    c.d.f. and its inverse
    $$F_X(x) = \mathbb{P}(X\le x) = \sum_{y\le x}p_X(y) \quad\text{or}\quad \int_{-\infty}^{x} f_X(y)\,dy$$ $$f_X(x) = \frac{d}{dx}F_X(x) \qquad\qquad F_X^{-1}(q) = \text{the value } x \text{ with } q \text{ probability below it}$$
    • $F_X(x)$"how much probability has accumulated at or below $x$." Always climbs from 0 to 1, never decreases.
    • $f = F'$the density is the slope of the c.d.f. Steep c.d.f. means dense region.
    • $F^{-1}(q)$the quantile function. Feed it $q=0.5$ and it returns the median. For discrete variables it is defined as $\inf\{x : F_X(x)\ge q\}$ — the smallest $x$ whose accumulated probability has reached $q$.

    When you use it: "what is the probability of observing less than 15?" is $F_X(15)$ directly. "More than 15" is $1-F_X(15)$. Your professor's stated convention: always rewrite a probability question as $1 - \mathbb{P}(X\le\cdot)$ rather than $\mathbb{P}(X>\cdot)$, because the c.d.f. is the object you actually have.

    Families and parameters

    Distributions are parameterized: constants (collectively called $\theta$) set their shape, location, and sometimes their domain. The exponential p.d.f. $f_X(x)=\frac1\theta e^{-x/\theta}$ is not one curve; it is a whole family of curves, one per value of $\theta$. A family of distributions means all the shapes a parameterized form can take.

    Two things your professor flags: the parameter values are fixed, not random; and estimating them is the entire point of statistical inference. Your study guide asks you to understand "at a conceptual level how changing $\theta$ changes the shape and/or location of a p.m.f. or p.d.f." — module 6 does that for each of the four named families.

    Conditional probabilities from a distribution

    Nothing new — the conditional definition still holds, you just compute each piece from the distribution:

    $$\mathbb{P}(X < 1 \mid X < 2) = \frac{\mathbb{P}(X<1 \;\cap\; X<2)}{\mathbb{P}(X<2)} = \frac{\mathbb{P}(X<1)}{\mathbb{P}(X<2)}$$

    The simplification happens because $\{X<1\}$ is a subset of $\{X<2\}$: if $X$ is below 1 it is automatically below 2, so the intersection is just the smaller event.

    Worksheet 2W

    Given the p.d.f. $f_X(x) = 3x^2$ on the domain $x\in[0,1]$, what is $\mathbb{P}(X>0.5)$?

    Show solution

    Probability for a continuous variable is area, so integrate the density over the region you want:

    $$\mathbb{P}(X>0.5) = \int_{0.5}^{1} 3x^2\,dx = \Big[x^3\Big]_{0.5}^{1} = 1 - (0.5)^3 = 1 - \tfrac18$$

    7/8 = 0.875

    Note the upper limit is 1, not $\infty$ — the density is zero outside $[0,1]$. Also note $f_X(1)=3$, a density well above 1, which is fine.

    Lecture 2W — two quick ones

    (i) Given the p.m.f. below, find the missing value, then find $\mathbb{P}(1\le X < 3)$.

    $x$0123
    $p_X(x)$0.20?0.350.15

    (ii) Given $f_X(x) = cx^2$ on $x\in[0,1]$, find $c$.

    Show solution

    (i) Masses must sum to 1:

    $$p_X(1) = 1 - 0.20 - 0.35 - 0.15 = 0.30$$

    For $\mathbb{P}(1\le X<3)$, ask which masses lie in the range. The inequality includes 1 and excludes 3, so it is $x=1$ and $x=2$:

    $$0.30 + 0.35 = 0.65$$

    $p_X(1)=0.30$,   $\mathbb{P}(1\le X<3)=0.65$

    Interpretation, in your professor's words: "the next time we run the experiment governed by this distribution, there is a 65% chance we observe an outcome of $X=1$ or $X=2$."

    (ii) Densities integrate to 1:

    $$\int_0^1 cx^2\,dx = c\left[\tfrac{x^3}{3}\right]_0^1 = \tfrac{c}{3} = 1 \;\Longrightarrow\; c = 3$$

    c = 3

    Homework 2, Question 8 — normalizing and conditioning

    $f_X(x) = cx^2$ for $0\le x\le 3$, and 0 otherwise.

    1. Find $c$.
    2. Find $\mathbb{P}(X<1)$ and interpret it for someone who has never taken a probability class.
    3. Find $\mathbb{P}(X<1\mid X<2)$ and interpret it.
    Show solution

    (a) $$\int_0^3 cx^2\,dx = c\left[\tfrac{x^3}{3}\right]_0^3 = c\cdot 9 = 1 \;\Longrightarrow\; c=\tfrac19$$

    c = 1/9

    (b) $$\mathbb{P}(X<1) = \int_0^1 \tfrac19 x^2\,dx = \tfrac19\cdot\tfrac13 = \tfrac{1}{27} \approx 0.037$$

    1/27 ≈ 0.037

    Interpretation: if we run this experiment once more, there is about a 3.7% chance the value we observe is below 1.

    (c) First get the conditioning event's probability:

    $$\mathbb{P}(X<2) = \int_0^2 \tfrac19 x^2\,dx = \tfrac19\cdot\tfrac83 = \tfrac{8}{27}$$

    Since $\{X<1\}\subset\{X<2\}$, the intersection collapses:

    $$\mathbb{P}(X<1\mid X<2) = \frac{1/27}{8/27} = \frac18$$

    1/8 = 0.125

    Interpretation: among the runs that land below 2, one in eight lands below 1. Knowing the value is small makes a very small value more plausible — the probability rose from 3.7% to 12.5%.

    Homework 2, Question 5 — building a p.m.f. from scratch

    An urn holds 5 balls: 2 red and 3 black. Let $X$ be the number of draws, without replacement, until you have collected at least one red and at least one black.

    1. Write the p.m.f. of $X$ as a table.
    2. Compute $\mathbb{E}(X)$.
    3. Compute $\sigma$.
    Show solution

    (a) First, what values can $X$ take? You stop as soon as you hold both colours, so the earliest is 2. The latest is 4 — if you draw $BBB$ you have exhausted the black balls, so the fourth draw is certainly red. So $X\in\{2,3,4\}$.

    $X=2$: the first two draws differ.

    $$\mathbb{P}(RB) + \mathbb{P}(BR) = \tfrac25\cdot\tfrac34 + \tfrac35\cdot\tfrac24 = \tfrac{6}{20}+\tfrac{6}{20} = \tfrac{12}{20}$$

    $X=3$: the first two match, the third breaks the streak. Two sub-cases.

    • $RR$ then black: $\mathbb{P}(RR) = \tfrac25\cdot\tfrac14 = \tfrac{2}{20}$. Only blacks remain, so the third is certainly black. Contributes $\tfrac{2}{20}$.
    • $BB$ then red: $\mathbb{P}(BB) = \tfrac35\cdot\tfrac24 = \tfrac{6}{20}$. Now 2 red and 1 black remain, so red comes with probability $\tfrac23$. Contributes $\tfrac{6}{20}\cdot\tfrac23 = \tfrac{4}{20}$.

    $$\mathbb{P}(X=3) = \tfrac{2}{20}+\tfrac{4}{20} = \tfrac{6}{20}$$

    $X=4$: the only route is $BBB$ then a forced red.

    $$\mathbb{P}(BBB) = \tfrac35\cdot\tfrac24\cdot\tfrac13 = \tfrac{6}{60} = \tfrac{2}{20}$$

    $x$234
    $p_X(x)$12/20 = 0.66/20 = 0.32/20 = 0.1

    Check: $0.6+0.3+0.1 = 1$. Always do this.

    (b) $$\mathbb{E}(X) = 2(0.6)+3(0.3)+4(0.1) = 1.2+0.9+0.4 = 2.5$$

    E(X) = 2.5 draws

    (c) Use the shortcut formula (module 5):

    $$\mathbb{E}(X^2) = 4(0.6)+9(0.3)+16(0.1) = 2.4+2.7+1.6 = 6.7$$

    $$\mathbb{V}(X) = \mathbb{E}(X^2)-\big(\mathbb{E}(X)\big)^2 = 6.7 - 6.25 = 0.45$$

    σ = √0.45 ≈ 0.671

    Homework 2, Question 7 — a mixed distribution

    $$h_X(x) = \begin{cases}0.25 & x=-3\\ c & x\in[-1,1]\\ 0.25 & x = 2\\ 0 & \text{otherwise}\end{cases}$$

    This is partly discrete (at $-3$ and $2$) and partly continuous (over $[-1,1]$).

    1. Find $c$.
    2. Find $\mathbb{E}(X)$.
    Show solution

    The principle: total probability is still 1. Sum the discrete masses, integrate the continuous density, set the whole thing equal to 1.

    (a) $$0.25 + \int_{-1}^{1} c\,dx + 0.25 = 1 \;\Longrightarrow\; 0.5 + 2c = 1 \;\Longrightarrow\; c = 0.25$$

    c = 0.25

    (b) Same split: sum $x\cdot p$ over the masses, integrate $x\cdot f$ over the continuous part.

    $$\mathbb{E}(X) = (-3)(0.25) + \int_{-1}^{1}x(0.25)\,dx + (2)(0.25)$$

    The integral is zero by symmetry — a flat density on an interval centred at 0:

    $$\int_{-1}^1 0.25x\,dx = 0.25\left[\tfrac{x^2}{2}\right]_{-1}^{1} = 0.25\left(\tfrac12 - \tfrac12\right) = 0$$

    $$\mathbb{E}(X) = -0.75 + 0 + 0.5$$

    E(X) = −0.25

    Traps in this module
    • Treating $f_X(x)$ as a probability. It is a density. It can exceed 1. You must integrate.
    • Integrating over the wrong limits. Clip to the domain. $\int_{0.5}^{1}$, not $\int_{0.5}^{\infty}$, when the support is $[0,1]$.
    • Getting sloppy with $<$ vs $\le$ on a p.m.f. For discrete variables this changes the answer; for continuous ones it never does.
    • Forgetting to verify the total. Any p.m.f. you build should sum to exactly 1. It is a free check and it catches most arithmetic slips.
    05

    35 minutes · lecture 2F

    Moments: expected value and variance

    A distribution is a whole function. Moments are how you compress it into two or three informative numbers: where it sits, and how wide it is.

    Moments about the origin

    The $k$-th moment about the origin
    $$\mu_k' = \mathbb{E}(X^k) = \sum_{\text{all }x} x^k\, p_X(x) \qquad\text{or}\qquad \int_{\text{all }x} x^k f_X(x)\,dx$$
    • $k=1$gives $\mathbb{E}(X)$, the mean. This is the one you use most.
    • $k=2$gives $\mathbb{E}(X^2)$, which you need for the variance shortcut. Note the square goes on $x$, not on $p_X(x)$.
    • $X$ capitalthe moment is a property of the random variable, so the argument is capitalized.
    Law of the Unconscious Statistician (LOTUS)
    $$\mathbb{E}\big(g(X)\big) = \sum_{x} g(x)\,p_X(x) \qquad\text{or}\qquad \int_x g(x)\,f_X(x)\,dx$$

    To find the expected value of any function of $X$, weight $g(x)$ by the distribution of $X$ and add up. You do not need to work out the distribution of $g(X)$ first. That shortcut is why the name teases you for using it unconsciously.

    When you use it: $g(x)=x$ gives the mean; $g(x)=x^2$ gives the second moment; $g(x)=(x-\mu)^2$ gives the variance directly. Every moment formula on your sheet is one instance of LOTUS.

    Expected value
    $$\mathbb{E}(X) = \sum_x x\,p_X(x) \qquad\text{or}\qquad \int_x x\,f_X(x)\,dx \;\;=\;\; \mu$$

    In your professor's phrasing: the probability-weighted average for a datum. Each possible value is counted in proportion to how likely it is. If you sample one value from this distribution, on average it will be $\mathbb{E}(X)$.

    Note: $\mathbb{E}(X)=\mu$, the true population mean. The expected value is the parameter $\mu$ — not an estimate of it.

    Properties of the expected-value operator

    RuleMeaning
    $\mathbb{E}(a) = a$The expected value of a constant is that constant. Nothing random about it.
    $\mathbb{E}(aX) = a\,\mathbb{E}(X)$Constants pull straight out.
    $\mathbb{E}(aX+b) = a\,\mathbb{E}(X)+b$Expectation is linear. Scale and shift pass right through.
    $\mathbb{E}\!\left(\sum_{i=1}^n X_i\right) = \sum_{i=1}^n \mathbb{E}(X_i)$The expectation of a sum is the sum of expectations. This needs no assumptions at all — not even independence.
    $\mathbb{E}(XY) \ne \mathbb{E}(X)\mathbb{E}(Y)$In general. Equality holds only if $X$ and $Y$ are independent.

    Moments about the mean, and variance

    The $k$-th central moment
    $$\mu_k = \mathbb{E}\big((X-\mu)^k\big)$$

    Same idea, but measured from the distribution's own centre rather than from zero. When people say "the moments of $X$," this is usually what they mean.

    ($\mu_3/\sigma^3$ measures skewness. Your professor explicitly says you will not be doing that here.)

    Variance — and the shortcut formula
    $$\mathbb{V}(X) = \sigma^2 = \mu_2 = \mathbb{E}\big((X-\mu)^2\big) = \underbrace{\mathbb{E}(X^2) - \big(\mathbb{E}(X)\big)^2}_{\text{the shortcut}}$$
    • $\mathbb{E}(X^2)$square each value first, then average.
    • $(\mathbb{E}(X))^2$average first, then square. These are different numbers, and their gap is the variance.

    Always use the shortcut. Computing $\mathbb{E}((X-\mu)^2)$ directly means subtracting $\mu$ from every value before squaring; the shortcut needs only two sums. Every variance problem on this exam is faster this way.

    Standard deviation
    $$\sigma_X = \sqrt{\mathbb{V}(X)}$$

    Variance is in squared units, which makes it hard to interpret. Taking the square root puts you back in the units of $X$, and gives a usable notion of the distribution's width: the range over which you generally expect to see data.

    Rule of thumb from lecture: nearly all observed data lie within three standard deviations of the mean. $\mathbb{P}(\mu - 3\sigma \le X \le \mu+3\sigma)\approx 1$.

    What happens to E and V under transformation

    Linear transformations and sums
    $$\mathbb{E}(aX+b) = a\,\mathbb{E}(X)+b \qquad\qquad \mathbb{V}(aX+b) = a^2\,\mathbb{V}(X) \qquad \sigma_{aX+b} = |a|\,\sigma_X$$ $$\mathbb{V}\!\left(\sum_{i=1}^{n}X_i\right) = \sum_{i=1}^{n}\mathbb{V}(X_i) \qquad \textbf{only if the } X_i \textbf{ are independent}$$
    • $a^2$variance is in squared units, so a scale factor enters squared. Doubling every value quadruples the variance.
    • $+b$vanishes from the variance. Sliding the whole distribution sideways does not change its width.
    • $|a|$absolute value, because a standard deviation is never negative.

    The independence condition on the last line is the one people drop. Expectations of sums always add; variances of sums only add when the terms are independent. Module 9 uses this exact rule to derive $\mathbb{V}(\bar{X}) = \sigma^2/n$.

    Worksheet 2F

    Given the p.m.f. below, what is the standard deviation?

    $x$012
    $p_X(x)$0.40.40.2
    Show solution

    Three lines, every time.

    $$\mathbb{E}(X) = 0(0.4)+1(0.4)+2(0.2) = 0.8$$

    $$\mathbb{E}(X^2) = 0^2(0.4)+1^2(0.4)+2^2(0.2) = 0.4+0.8 = 1.2$$

    $$\mathbb{V}(X) = 1.2 - (0.8)^2 = 1.2-0.64 = 0.56$$

    σ = √0.56 ≈ 0.748

    Notice the squares land on the $x$ values, not on the probabilities. $2^2(0.2) = 0.8$, not $2(0.2^2)$.

    Lecture 2F — the card game

    You draw one card from a 52-card deck. A jack or queen pays 10 dollars; a king or ace pays 4 dollars; anything else costs you 5 dollars. What is the expected monetary gain from one play?

    Show solution

    There are 13 ranks, each equally likely. Group them by payoff:

    • 2 through 10 — 9 ranks — lose 5
    • J, Q — 2 ranks — win 10
    • K, A — 2 ranks — win 4

    Define $X$ = dollars gained, and write its p.m.f.:

    $x$−5+10+4
    $p_X(x)$9/132/132/13

    $$\mathbb{E}(X) = -5\left(\tfrac{9}{13}\right) + 10\left(\tfrac{2}{13}\right) + 4\left(\tfrac{2}{13}\right) = \frac{-45+20+8}{13}$$

    −17/13 ≈ −1.31 dollars per play

    The lesson is the modelling step, not the arithmetic: define a random variable for the quantity you actually care about (money), build its p.m.f. by grouping outcomes with the same value, then apply the definition.

    Lecture 2F — expected value of a continuous distribution

    If $X$ has p.d.f. $f_X(x) = \theta x^{\theta-1}$ on $x\in[0,1]$ with $\theta>0$, what is $\mathbb{E}(X)$, and how would you interpret it?

    Show solution

    $$\mathbb{E}(X) = \int_0^1 x\cdot\theta x^{\theta-1}\,dx = \theta\int_0^1 x^{\theta}\,dx$$

    Pull the constant $\theta$ out, then combine the powers: $x \cdot x^{\theta-1} = x^{\theta}$. Now integrate by the power rule:

    $$= \theta\left[\frac{x^{\theta+1}}{\theta+1}\right]_0^1 = \frac{\theta}{\theta+1}(1 - 0)$$

    E(X) = θ/(θ+1)

    Interpretation: if we sample a single datum from this distribution, on average its value will be $\theta/(\theta+1)$.

    Sanity check the behaviour: as $\theta$ grows, $\theta/(\theta+1)\to 1$, meaning large $\theta$ piles the density up near $x=1$. As $\theta\to 0$, the mean goes to 0. That is exactly the "how does $\theta$ change the shape" question your study guide asks about.

    Homework 2, Question 6 — working backwards from E(X)

    A partially known p.m.f.: $p_X(1)=0.3$, $p_X(3)=0.3$, and $p_X(0)$, $p_X(2)$ unknown. You also know $\mathbb{E}(X)=1.8$.

    1. Find the two missing values.
    2. Find $\sigma$.
    3. Let $Y = 3X$. Find $\mathbb{E}(Y)$ and $\mathbb{V}(Y)$.
    Show solution

    (a) Two unknowns need two equations. Call them $a = p_X(0)$ and $b = p_X(2)$.

    Equation 1 — masses sum to 1:

    $$a + 0.3 + b + 0.3 = 1 \;\Longrightarrow\; a+b = 0.4$$

    Equation 2 — the given expected value:

    $$0(a) + 1(0.3) + 2(b) + 3(0.3) = 1.8 \;\Longrightarrow\; 1.2 + 2b = 1.8 \;\Longrightarrow\; b = 0.3$$

    p(0) = 0.1,   p(2) = 0.3

    (b) $$\mathbb{E}(X^2) = 0(0.1)+1(0.3)+4(0.3)+9(0.3) = 0.3+1.2+2.7 = 4.2$$

    $$\mathbb{V}(X) = 4.2 - (1.8)^2 = 4.2-3.24 = 0.96$$

    σ = √0.96 ≈ 0.980

    (c) Use the transformation rules rather than rebuilding the p.m.f. of $Y$. Here $a=3$, $b=0$:

    $$\mathbb{E}(Y) = 3\,\mathbb{E}(X) = 3(1.8) = 5.4 \qquad \mathbb{V}(Y) = 3^2\,\mathbb{V}(X) = 9(0.96) = 8.64$$

    E(Y) = 5.4,   V(Y) = 8.64

    The common error is $\mathbb{V}(Y)=3(0.96)$. The scale factor enters squared.

    Traps in this module
    • $\mathbb{E}(X^2)$ versus $(\mathbb{E}(X))^2$. These are never equal (unless the variance is zero). Mixing them up is the single most common error in this course.
    • Squaring the probability instead of the value. In $\mathbb{E}(X^2)=\sum x^2 p_X(x)$, the exponent is on $x$ only.
    • $\mathbb{V}(aX+b) = a\mathbb{V}(X)$. No — it is $a^2$. And the $b$ disappears entirely.
    • Adding variances without independence. Expectations always add; variances only add for independent terms.
    • Forgetting the square root when the question asks for the standard deviation, not the variance.
    06

    70 minutes · lectures 3W, 3F

    Binomial, Poisson, normal, exponential

    Four named families. For each one you need three things: what physical situation it models, how its parameters change its shape, and how to compute a probability from it.

    Which distribution is this problem about?

    The question says...UseBecause
    a fixed number of trials, count the successesBinomial$n$ is set in advance; each trial succeeds or fails
    count events in a window of time/space, no upper limitPoissonthere is no "number of trials" to fix
    waiting time until the next eventExponentialcontinuous, positive, right-skewed
    a measurement that clusters symmetrically around a centreNormalbell shaped, unbounded in both directions
    The give-away words

    "Out of 36 resistors," "in four rolls," "three flips" → binomial. "Per minute," "per wafer," "per year" → Poisson. "How long until" → exponential. "Average resistance," "mean weight" → normal.


    Binomial

    Start with a Bernoulli process: a sequence of $n$ trials $Y_1,\ldots,Y_n$, each one a 0 or a 1, each with the same success probability $p$. The binomial random variable is the running total:

    $$X = \sum_{i=1}^{n} Y_i \;=\; \text{the number of successes}$$

    The four conditions for a binomial experiment
    1. The number of trials, $n$, is fixed in advance.
    2. Each trial has exactly two possible outcomes: success (1) or failure (0).
    3. The success probability $p$ stays constant across all trials.
    4. Trial outcomes are independent of each other.
    Binomial — $X \sim \mathrm{Binom}(k,p)$
    $$p_X(x) = \binom{k}{x}p^x(1-p)^{k-x}, \qquad x\in\{0,1,\ldots,k\},\;\; p\in[0,1]$$ $$\binom{k}{x} = \frac{k!}{x!\,(k-x)!} \qquad \mathbb{E}(X) = kp \qquad \mathbb{V}(X) = kp(1-p) \qquad \sigma = \sqrt{kp(1-p)}$$
    • $k$the number of trials. Your formula sheet calls it $k$; lecture called it $n$. Same thing — see the warning below.
    • $x$the number of successes you are asking about.
    • $p^x$the probability of getting those $x$ successes.
    • $(1-p)^{k-x}$the probability that the other $k-x$ trials all fail.
    • $\binom{k}{x}$"$k$ choose $x$" — the number of different orders in which those $x$ successes could occur. Without it you would be computing the probability of one specific sequence.
    • $k!$"$k$ factorial" $= k(k-1)\cdots 3\cdot 2\cdot 1$. And $0! = 1$.

    Shape: $p$ shifts the mass left or right (low $p$ piles it near 0); $k$ stretches it and makes it more symmetric. At $p=0.5$ it is perfectly symmetric.

    The $k$ versus $n$ warning — read this twice

    Your formula sheet writes $X\sim\mathrm{Binom}(k,p)$ with $k$ trials, because it reserves $n$ for the number of data points you collected. So if you flip a coin 3 times and record the number of heads, then repeat that whole experiment 6 times:

    • $k = 3$ — trials within one datum
    • $n = 6$ — how many data you have
    • and the sufficient statistic is $X_+ = \sum_{i=1}^n X_i \sim \mathrm{Binom}(nk, p) = \mathrm{Binom}(18,p)$ — see module 9.

    Relatives of the binomial

    NameWhat is fixedWhat is random
    Bernoullia binomial with $k=1$a single 0 or 1
    Binomialthe number of trials $k$the number of successes
    Negative binomialthe number of successes $s$the number of failures
    Geometrica negative binomial with $s=1$failures before the first success
    Hypergeometriclike binomial, but sampling without replacement from a finite setsuccesses (probabilities change each draw)

    You need to recognize these names and know what distinguishes them. Only the binomial, Bernoulli, Poisson, normal and exponential have formulas on your sheet.

    The "at least one" trick — memorize this

    $$\mathbb{P}(X \ge 1) = 1 - \mathbb{P}(X=0) = 1 - \binom{k}{0}p^0(1-p)^{k} = 1-(1-p)^{k}$$

    Because $\binom{k}{0}=1$ and $p^0=1$. It appears in nearly every binomial question on this exam.

    Worksheet 3W

    An unfair coin shows heads with probability $p=0.4$. Flip it three times. What is the probability of observing at least one tail?

    Show solution

    Flip the event. "At least one tail" fails only when every flip is a head.

    $$\mathbb{P}(\ge 1\text{ tail}) = 1 - \mathbb{P}(\text{no tails}) = 1 - \mathbb{P}(\text{3 heads}) = 1 - (0.4)^3 = 1 - 0.064$$

    0.936 = 117/125

    Formally, let $X$ = number of tails with $X\sim\mathrm{Binom}(3, 0.6)$; then $\mathbb{P}(X\ge1)=1-\binom{3}{0}(0.6)^0(0.4)^3$. Same number, but the shortcut is faster.

    Lecture 3W — the Chevalier de Méré's question

    Are these two probabilities the same? (a) at least one 6 in four rolls of a fair die; (b) at least one double-6 in 24 rolls of two fair dice.

    Show solution

    First, identify the distribution. The number of rolls is fixed in advance, so this is binomial.

    (a) $X$ = number of 6s, $X\sim\mathrm{Binom}(4, 1/6)$:

    $$\mathbb{P}(X\ge1) = 1-\mathbb{P}(X=0) = 1 - \binom{4}{0}\left(\tfrac16\right)^0\left(1-\tfrac16\right)^{4} = 1-\left(\tfrac56\right)^4 \approx 0.518$$

    (b) Two dice give 36 equally likely pairs, so a double-6 has probability $1/36$. $X\sim\mathrm{Binom}(24, 1/36)$:

    $$\mathbb{P}(X\ge1) = 1-\left(1-\tfrac1{36}\right)^{24} \approx 0.491$$

    No — 0.518 vs 0.491

    A real historical bet turned on this narrow gap. The point for you: identifying $k$ and $p$ correctly is the whole problem.

    Homework 3, Question 1 — best of five

    Teams A and B play until one has won three games. $p$ is the probability A wins any individual game, and games are independent. What is the probability A wins the series?

    Show solution

    The structure: A can win in exactly 3, 4, or 5 games. Those are disjoint events, so their probabilities add.

    1. Wins in 3. A must win all three: $p^3$.
    2. Wins in 4. A wins exactly 2 of the first 3, then wins game 4. The first part is binomial, the second is a single independent game: $$\binom{3}{2}p^2(1-p)^{1}\cdot p = 3p^3(1-p)$$
    3. Wins in 5. A wins exactly 2 of the first 4, then wins game 5: $$\binom{4}{2}p^2(1-p)^{2}\cdot p = 6p^3(1-p)^2$$

    Add and factor out $p^3$:

    $$\mathbb{P} = p^3\left[1 + 3(1-p) + 6(1-p)^2\right] = p^3\left[1 + 3 - 3p + 6 - 12p + 6p^2\right]$$

    P = p³(6p² − 15p + 10)

    Check at $p=0.5$: $0.125(1.5-7.5+10)=0.125(4)=0.5$. Even teams split the series half the time. Check at $p=1$: $6-15+10=1$. Both correct.

    Why the last game is separate. You cannot write "A wins 3 of 4 games" for the 4-game case, because that would include sequences where A already clinched in 3 and a fourth game was never played. Forcing A to win the final game is what makes the cases disjoint.


    Poisson

    Some counting situations have no fixed number of trials. Lecture 3W's example: deaths per year among Prussian soldiers from horse kicks. It is discrete, but there is no theoretical upper bound, so it cannot come from a Bernoulli process. You have moved from a discrete Bernoulli process to a continuous Poisson process in time or space.

    Poisson — $X\sim\mathrm{Pois}(\lambda)$
    $$p_X(x) = \frac{\lambda^x e^{-\lambda}}{x!}, \qquad x\in\{0,1,2,\ldots\},\;\; \lambda>0$$ $$\mathbb{E}(X) = \mathbb{V}(X) = \lambda \qquad\qquad \sigma = \sqrt{\lambda}$$
    • $\lambda$the expected count per observational window. Not a rate — a count. This distinction is the trap below.
    • $x!$factorial in the denominator, which is why the mass falls off fast for large $x$.
    • $\mathbb{E}=\mathbb{V}$unique among your four distributions. Mean and variance are the same number.

    Shape: $\lambda$ is the only parameter, so it controls both location and spread at once. Small $\lambda$ gives a shape crammed against 0 and heavily right-skewed; large $\lambda$ becomes roughly symmetric.

    The one you will use constantly

    $$\mathbb{P}(X=0) = \frac{\lambda^0 e^{-\lambda}}{0!} = e^{-\lambda}$$

    Because $\lambda^0=1$ and $0!=1$. Combined with the complement rule, this handles every "at least one event" Poisson question.

    The rate-versus-λ trap

    If a problem says "10 calls per minute" and asks about a window of $\Delta t$ minutes, then $\lambda$ is not 10. You must convert:

    $$\lambda = \left(10\;\tfrac{\text{calls}}{\text{min}}\right)\times(\Delta t\;\text{min}) = 10\,\Delta t \;\text{calls}$$

    $\lambda$ must be a dimensionless count for the window you are actually observing. Always check the units cancel.

    Lecture 3W — phone calls

    Calls arrive at a long-term rate of 10 per minute. You record counts in windows of width $\Delta t$. What is the largest $\Delta t$ for which the probability of seeing no calls in a window is at least 0.5?

    Show solution

    Step 1 — convert the rate into $\lambda$.

    $$\lambda = 10\,\Delta t$$

    Step 2 — write the probability of zero.

    $$\mathbb{P}(X=0) = e^{-\lambda} = e^{-10\Delta t}$$

    Step 3 — impose the condition and solve.

    $$e^{-10\Delta t}\ \ge\ \tfrac12 \;\Longrightarrow\; -10\Delta t \ \ge\ \ln\left(\tfrac12\right) \;\Longrightarrow\; 10\Delta t\ \le\ \ln 2$$

    $$\Delta t \le \frac{\ln 2}{10}$$

    Δt ≤ 0.0693 minutes

    Watch the inequality: multiplying by $-1$ flips it. And $-\ln(1/2)=\ln 2$.

    Homework 3, Question 2 — silicon wafers

    4% of wafers have no apparent flaws. Let $X$ be the number of flaws on a randomly chosen wafer, modelled as Poisson.

    1. What is $\lambda$?
    2. What is the probability of more than two flaws on one wafer?
    3. What are the expected number of flaws and the standard deviation?
    Show solution

    (a) "No apparent flaws" means $X=0$, and you are told that happens 4% of the time. So work backwards from the zero-count formula:

    $$\mathbb{P}(X=0) = e^{-\lambda} = 0.04 \;\Longrightarrow\; -\lambda = \ln(0.04) \;\Longrightarrow\; \lambda = \ln(25)$$

    λ = ln 25 ≈ 3.219

    (b) "More than two" means $X\ge3$. Complement it — three terms is much less work than an infinite sum:

    $$\mathbb{P}(X>2) = 1 - \left[p_X(0)+p_X(1)+p_X(2)\right] = 1 - e^{-\lambda}\left[1 + \lambda + \frac{\lambda^2}{2}\right]$$

    With $e^{-\lambda}=0.04$ and $\lambda=3.219$:

    $$= 1 - 0.04\left[1 + 3.219 + 5.181\right] = 1 - 0.04(9.400) = 1-0.376$$

    ≈ 0.624

    Notice $e^{-\lambda}$ was handed to you as 0.04 — substituting it saves you from exponentiating anything.

    (c) For a Poisson, $\mathbb{E}(X)=\mathbb{V}(X)=\lambda$:

    E(X) ≈ 3.219 flaws,   σ = √3.219 ≈ 1.794


    Normal (Gaussian)

    Your professor gives three reasons this distribution dominates statistics:

    1. Data are often approximately normally distributed.
    2. Many non-normal distributions are close to normal in shape.
    3. The central limit theorem: for a large sample, the distribution of $\bar{X}$ is approximately normal regardless of what the data's own distribution is. (Module 10.)
    Normal — $X\sim\mathcal{N}(\mu,\sigma^2)$
    $$f_X(x) = \frac{1}{\sqrt{2\pi\sigma^2}}\,e^{-(x-\mu)^2/(2\sigma^2)}, \qquad x,\mu\in(-\infty,\infty),\;\;\sigma>0$$ $$\mathbb{E}(X) = \mu \qquad\qquad \mathbb{V}(X) = \sigma^2$$
    • $\mu$the location. Slides the whole bell left or right. It is the centre, the peak, the mean, and the median all at once.
    • $\sigma$the scale. Larger $\sigma$ means a wider, flatter bell; smaller means narrower and taller.
    • $(x-\mu)^2$squared, so the curve is perfectly symmetric about $\mu$ — your study guide asks you to know this explicitly.
    • $\frac{1}{\sqrt{2\pi\sigma^2}}$the normalizing constant, present only to make the area equal 1.

    Special feature: for this family the parameters are the moments. $\mu$ is the mean; $\sigma^2$ is the variance. That is not true of the others.

    μ−3σ μ−2σ μ−σ μ μ+σ μ+2σ μ+3σ 68.3% 95.4% within ±2σ
    The empirical rule. These three numbers are on your formula sheet and are the only way to get a normal probability by hand.
    Useful normal probabilities — memorize the top row
    $$\mathbb{P}(\mu-\sigma\le X\le \mu+\sigma)\approx 0.683 \qquad \mathbb{P}(\mu-2\sigma\le X\le\mu+2\sigma)\approx 0.954 \qquad \mathbb{P}(\mu-3\sigma\le X\le \mu+3\sigma)\approx 0.997$$ $$\text{central } 90\%:\ \mu\pm1.645\sigma \qquad 95\%:\ \mu\pm1.960\sigma \qquad 99\%:\ \mu\pm 2.576\sigma$$

    Why you need these: the normal p.d.f. has no closed-form antiderivative. You cannot integrate it by hand. So every hand-computable normal question on this exam must reduce to one of these six numbers plus the symmetry of the curve.

    The symmetry moves: if $\mathbb{P}(\mu-\sigma\le X\le\mu+\sigma)=0.683$, then the two tails together hold $1-0.683=0.317$, and by symmetry each tail holds $0.317/2 = 0.1585$. Halving the leftover is the move that unlocks most of these problems.

    Where the computer would take over

    In lecture your professor writes $\mathbb{P}(X>15)$ as $1-\mathbb{P}(X\le 15) = 1 - F_X(15)$ and then reaches for code, because that c.d.f. value cannot be found by hand. On the exam you are expected to get to exactly that point and stop, or to be handed numbers that land on the memorized values. Setting it up correctly is the graded part.

    Homework 3, Question 3 — sampling until you exceed a threshold

    Sample repeatedly from $\mathcal{N}(0,1)$ until you observe a value with absolute value greater than $a$; then stop.

    1. If $a=1$, what is the probability you stop after only one or two samples?
    2. Separately: for a single draw $X$ from $\mathcal{N}(0,1)$, what is $\mathbb{P}(X>2\mid X>1)$?
    Show solution

    Here $\mu=0$ and $\sigma=1$, so "$|X|>1$" is exactly "outside $\mu\pm\sigma$".

    (a) First, the per-draw stopping probability:

    $$\mathbb{P}(|X|>1) = 1 - \mathbb{P}(-1\le X\le 1) = 1-0.683 = 0.317$$

    You stop within two draws unless both draws fail to exceed the threshold. Draws are independent:

    $$\mathbb{P}(\text{stop by draw 2}) = 1 - (0.683)^2 = 1 - 0.4665$$

    ≈ 0.5335

    Equivalently $0.317 + (0.683)(0.317)$ — stop on the first, or miss then stop on the second. Same number.

    (b) Turn it into a ratio of unconditional probabilities. Since $\{X>2\}\subset\{X>1\}$:

    $$\mathbb{P}(X>2\mid X>1) = \frac{\mathbb{P}(X>2)}{\mathbb{P}(X>1)}$$

    Get each tail by halving the leftover:

    $$\mathbb{P}(X>1) = \frac{1-0.683}{2} = 0.1585 \qquad \mathbb{P}(X>2) = \frac{1-0.954}{2} = 0.023$$

    $$\frac{0.023}{0.1585} \approx 0.145$$

    ≈ 0.145


    Exponential

    Picture a Poisson process — people passing through a doorway at a steady average rate. The number in any given minute is Poisson. But the waiting time from now until the next person arrives is continuous, and it is exponential. The two distributions describe the same process from two angles.

    Exponential — $X\sim\mathrm{Exp}(\theta)$
    $$f_X(x) = \frac{1}{\theta}e^{-x/\theta}, \qquad x\ge 0,\;\;\theta>0$$ $$\mathbb{E}(X) = \theta \qquad \mathbb{V}(X) = \theta^2 \qquad \sigma = \theta$$
    • $\theta$the mean waiting time. Larger $\theta$ stretches the curve out; smaller $\theta$ crams it against zero.
    • $x\ge0$waiting times cannot be negative. The density is zero for $x<0$.
    • $\sigma=\theta$The mean equals the standard deviation. Distinctive to this family, and a quick way to recognize it.

    Shape: always a decaying curve, highest at $x=0$, right-skewed. Unlike the normal it is never symmetric. An exponential is a special case of the more general gamma distribution.

    0 θ 2θ 3θ 4θ 63.2% skewed right, never symmetric
    Because $F_X(\theta)=1-e^{-1}\approx 0.632$, about 63% of the probability sits below the mean. That asymmetry is why the median and the mean differ here and not for the normal.
    Derive the exponential c.d.f. once, then never integrate again

    $$F_X(x) = \int_0^x \frac1\theta e^{-t/\theta}\,dt = \Big[-e^{-t/\theta}\Big]_0^{x} = 1 - e^{-x/\theta}$$

    So $\mathbb{P}(X\le x) = 1-e^{-x/\theta}$ and $\mathbb{P}(X> x) = e^{-x/\theta}$. Every exponential probability question becomes a one-line substitution. Note $\tfrac1\theta$ and the chain rule cancel exactly, which is the only reason the antiderivative is clean.

    Worksheet 3F

    With $\theta=2$, what is $\mathbb{P}(X<2)$? (The distribution is skewed, so the answer will not be exactly 1/2 — but it will not be far.)

    Show solution

    $$\mathbb{P}(X<2) = \int_0^2 \tfrac12 e^{-x/2}\,dx = \Big[-e^{-x/2}\Big]_0^2 = -e^{-1} - (-e^{0}) = 1 - e^{-1}$$

    1 − e⁻¹ ≈ 0.632

    This is the general fact that $\mathbb{P}(X<\theta) = 1-e^{-1}$ for any exponential: the probability of landing below the mean is always about 63.2%. That is what the skew does.

    Lecture 3F — two exponential examples

    (i) With $\theta=1$, find $\mathbb{P}(1\le X\le 2)$ and interpret it.
    (ii) For what value of $\theta$ is $\mathbb{P}(X\le 1) = 0.5$?

    Show solution

    (i) With $\theta=1$ the density is just $e^{-x}$:

    $$\mathbb{P}(1\le X\le 2) = \int_1^2 e^{-x}\,dx = \Big[-e^{-x}\Big]_1^2 = e^{-1}-e^{-2} \approx 0.368 - 0.135$$

    ≈ 0.233

    Interpretation: the next time we run this experiment, there is a 23.3% chance of observing a value between 1 and 2.

    (ii) You are solving for the parameter now, not the probability. Use the c.d.f.:

    $$1 - e^{-1/\theta} = \tfrac12 \;\Longrightarrow\; e^{-1/\theta} = \tfrac12 \;\Longrightarrow\; -\tfrac1\theta = \ln\left(\tfrac12\right) \;\Longrightarrow\; \tfrac1\theta = \ln 2$$

    θ = 1/ln2 ≈ 1.443

    This $\theta$ is the value whose median is 1. Since the median is always below the mean for an exponential, $\theta>1$ as expected.


    The four families side by side

    BinomialPoissonNormalExponential
    Typediscretediscretecontinuouscontinuous
    Support$\{0,\ldots,k\}$$\{0,1,2,\ldots\}$$(-\infty,\infty)$$[0,\infty)$
    Parameters$k,\;p$$\lambda$$\mu,\;\sigma^2$$\theta$
    $\mathbb{E}(X)$$kp$$\lambda$$\mu$$\theta$
    $\mathbb{V}(X)$$kp(1-p)$$\lambda$$\sigma^2$$\theta^2$
    $\sigma$$\sqrt{kp(1-p)}$$\sqrt{\lambda}$$\sigma$$\theta$
    Modelssuccesses in fixed trialscounts in a windowsymmetric measurementswaiting times
    Shapesymmetric at $p=0.5$, skewed otherwiseright-skewed, symmetric as $\lambda$ growsalways symmetricalways right-skewed
    Traps in this module
    • Dropping $\binom{k}{x}$. Without it you have the probability of one specific ordering, not of "$x$ successes in any order."
    • Using a rate as $\lambda$. Multiply by the window length first.
    • Trying to integrate the normal p.d.f. You cannot. Reduce to the empirical rule or stop at the c.d.f.
    • Forgetting the symmetry halving. One tail of a normal holds half the leftover, not all of it.
    • Mixing up $\lambda$ and $\theta$. Poisson $\lambda$ is a count; exponential $\theta$ is a mean waiting time. If events arrive at rate $\lambda$ per unit time, the mean waiting time is $1/\lambda$.
    • $\mathbb{V}=\lambda$, not $\lambda^2$, for a Poisson. And $\mathbb{V}=\theta^2$, not $\theta$, for an exponential.
    07

    55 minutes · lectures 4M, 4W

    Likelihood and maximum likelihood estimation

    This is where the course turns around. Until now you were given a distribution and asked for a probability. Now you are given data and asked for the distribution.

    The setup

    You ran an experiment and got $n$ i.i.d. data points $X_1,\ldots,X_n$, sampled from a distribution whose form you know but whose parameter $\theta$ you do not. You want a point estimate $\hat{\theta}$: a single best guess at $\theta$, computed from the data. Maximum likelihood is the standard algorithm for producing one.

    What the likelihood function is

    Your professor's definition, in words: a function that quantifies how likely a value of $\theta$ is, given the data we observe.

    Likelihood function for i.i.d. data
    $$\mathcal{L}(\theta\mid \mathbf{x}) = \prod_{i=1}^{n} p_X(x_i;\theta) \;\;\text{(discrete)} \qquad\qquad \mathcal{L}(\theta\mid\mathbf{x}) = \prod_{i=1}^{n} f_X(x_i;\theta) \;\;\text{(continuous)}$$
    • $\prod$multiply, because the data are independent: the probability of seeing all of them is the product of the individual probabilities. This is exactly where the i.i.d. assumption earns its keep.
    • $;\theta$the semicolon marks $\theta$ as a parameter of the distribution, not a second variable.
    • $\theta\mid\mathbf{x}$the notation is deliberate: the data are fixed (you already saw them) and $\theta$ is the variable. That inversion is the whole idea.

    Read it this way: plug in a candidate $\theta$ and the function returns a number saying how plausible the data you actually got would have been under that $\theta$. The best $\theta$ is the one that makes the observed data least surprising.

    μ = −2 μ = 1 μ = 4 three data points, three candidate values of μ density ≈ 0.01 at each point density ≈ 0.4 at each point
    For $\mu=-2$ or $\mu=4$, the density at each data point is tiny, so the product is around $10^{-6}$. For $\mu=1$ it is about $0.4^3\approx0.06$. That is thousands of times larger, so $\mu=1$ is far more likely given this data.

    Why take a logarithm

    Log-likelihood
    $$\ell(\theta\mid\mathbf{x}) = \ln \mathcal{L}(\theta\mid\mathbf{x})$$

    Two reasons, both essential:

    1. Logs turn products into sums. $\ln(\prod) = \sum \ln$. Differentiating a product of $n$ terms is miserable; differentiating a sum is trivial.
    2. $\ln$ is strictly increasing, so it does not move the location of the maximum. Whatever $\theta$ maximizes $\mathcal{L}$ also maximizes $\ell$. You lose nothing.

    Your professor writes $\ell$ as a script lowercase ell. Do not confuse it with the number 1 or with $\mathcal{L}$.

    Log rules you will need — have these cold
    $\ln(AB) = \ln A + \ln B$splits the product into a sum
    $\ln(A/B) = \ln A - \ln B$and $\ln(1/A) = -\ln A$
    $\ln(A^n) = n\ln A$exponents come down in front — used constantly
    $\ln(e^a) = a$kills every exponential in one step
    $\dfrac{d}{d\theta}\ln\theta = \dfrac1\theta$and $\dfrac{d}{d\theta}\theta^{-1} = -\theta^{-2}$

    The five-step recipe

    Deriving an MLE analytically
    $$\hat{\theta}_{MLE} \;\text{maximizes}\; \mathcal{L}, \;\text{ i.e. the value of }\theta\text{ for which }\; \frac{d\ell}{d\theta} = 0$$
    1. Write down $\mathcal{L}(\theta\mid\mathbf{x}) = \prod_{i=1}^n f_X(x_i;\theta)$ and simplify it — collect the constants, gather exponents, get it into the form $(\text{stuff})^n \times (\text{stuff})^{\sum}$.
    2. Take the log: $\ell = \ln\mathcal{L}$. Apply the log rules until no products or powers remain.
    3. Differentiate with respect to $\theta$. Any term without a $\theta$ in it differentiates to zero — drop it and move on.
    4. Set $\ell'(\theta\mid\mathbf{x}) = 0$.
    5. Solve for $\theta$. That is $\hat{\theta}_{MLE}$.

    Technically you should check the second derivative to confirm it is a maximum and not a minimum. Your professor notes this and then says you need not do it on homework; a question will tell you if it wants the check.

    Lecture 4M — the exponential MLE

    You sample $n$ i.i.d. data from $f_X(x;\theta) = \frac1\theta e^{-x/\theta}$, $x\ge0$, $\theta>0$. Find $\hat{\theta}_{MLE}$.

    Show solution

    1. The likelihood. Multiply $n$ copies of the density, one per data point:

    $$\mathcal{L}(\theta\mid\mathbf{x}) = \prod_{i=1}^{n}\frac1\theta e^{-x_i/\theta} = \frac{1}{\theta^n}\,e^{-\sum_{i=1}^n x_i/\theta}$$

    The $n$ copies of $1/\theta$ multiply into $\theta^{-n}$. The exponents add, because $e^a e^b = e^{a+b}$.

    2. Take the log.

    $$\ell(\theta\mid\mathbf{x}) = \ln\!\left(\theta^{-n}\right) + \ln\!\left(e^{-\sum x_i/\theta}\right) = -n\ln\theta - \frac{1}{\theta}\sum_{i=1}^{n}x_i$$

    3. Differentiate. Treat $\sum x_i$ as a constant — the data are fixed.

    $$\ell'(\theta\mid\mathbf{x}) = -\frac{n}{\theta} + \frac{1}{\theta^2}\sum_{i=1}^{n}x_i$$

    The second term: $-\tfrac1\theta \sum x_i = -\theta^{-1}\sum x_i$, whose derivative is $+\theta^{-2}\sum x_i$.

    4. Set to zero.

    $$\frac{1}{\theta^2}\sum_{i=1}^n x_i = \frac{n}{\theta}$$

    5. Solve. Multiply both sides by $\theta^2$ and divide by $n$:

    $$\theta = \frac1n\sum_{i=1}^{n}x_i$$

    θ̂MLE = X̄

    Capitalized, because once you regard it as a rule applied to random data it is itself a random variable. And it is deeply sensible: $\mathbb{E}(X)=\theta$ for an exponential, so estimating $\theta$ by the sample mean is exactly what intuition suggests.

    Lecture 4M — the Bernoulli MLE, plus invariance

    You sample $n$ i.i.d. data from $p_X(x) = p^x(1-p)^{1-x}$, $x\in\{0,1\}$, $p\in[0,1]$. Find the MLE for $p$. Then for $\sqrt{p}$ and for $p^2$.

    Show solution

    1. $$\mathcal{L}(p\mid\mathbf{x}) = \prod_{i=1}^{n}p^{x_i}(1-p)^{1-x_i} = p^{\sum x_i}\,(1-p)^{\,n-\sum x_i}$$

    Multiplying powers of the same base adds the exponents: $\sum x_i$ for the first, and $\sum(1-x_i) = n - \sum x_i$ for the second.

    2. $$\ell = \left(\sum x_i\right)\ln p + \left(n - \sum x_i\right)\ln(1-p)$$

    3. $$\ell'(p) = \frac{\sum x_i}{p} - \frac{n-\sum x_i}{1-p}$$

    The minus sign in the second term comes from the chain rule on $\ln(1-p)$.

    4 & 5. Set to zero and cross-multiply:

    $$\frac{1-p}{p} = \frac{n-\sum x_i}{\sum x_i} \;\Longrightarrow\; \frac1p - 1 = \frac{n}{\sum x_i} - 1 \;\Longrightarrow\; p = \frac1n\sum_{i=1}^n x_i$$

    p̂MLE = X̄

    Now the invariance property. No new derivation needed — just substitute:

    (√p)̂MLE = √X̄

    (p²)̂MLE = X̄²

    The invariance property
    $$\text{If } \hat{\theta}_{MLE} \text{ is the MLE for } \theta, \text{ then } \widehat{g(\theta)}_{MLE} = g\!\left(\hat{\theta}_{MLE}\right)$$

    In words: to get the MLE of any function of $\theta$, take the MLE of $\theta$ and push it through the function. You never redo the derivation.

    When you use it: whenever a question asks for the MLE of the mean, the variance, the standard deviation, or anything else built out of $\theta$. Your study guide names this property explicitly, so expect it.

    Worksheet 4M — a single datum

    You sample one datum from $f_X(x;\theta) = \theta x^{\theta-1}$, $x\in[0,1]$, $\theta>0$. What is the MLE for $\theta$?

    Hint: one datum means no products and no summations, and no $n$ will appear in the answer.

    Show solution

    With $n=1$ the likelihood is just the density itself: $\mathcal{L}(\theta\mid x) = \theta x^{\theta-1}$.

    Take the log. Use $\ln(AB)=\ln A + \ln B$ and then $\ln(A^n)=n\ln A$:

    $$\ell = \ln\theta + (\theta-1)\ln x$$

    Differentiate with respect to $\theta$. Here $\ln x$ is a constant — the datum is fixed — and $(\theta-1)\ln x$ is linear in $\theta$:

    $$\frac{d\ell}{d\theta} = \frac{1}{\theta} + \ln x$$

    Set to zero and solve.

    $$\frac1\theta = -\ln x \;\Longrightarrow\; \theta = \frac{-1}{\ln x}$$

    θ̂MLE = −1 / ln(x)

    Sanity check the sign: $x\in[0,1]$, so $\ln x \le 0$, so $-1/\ln x > 0$. The estimate respects the constraint $\theta>0$, as it must.

    Homework 3, Question 5 — the full four-part MLE problem

    $f_X(x) = \theta(1-x)^{\theta-1}$ on $x\in[0,1]$, $\theta>0$.

    1. Write down $\mathcal{L}(\theta\mid\mathbf{x})$ and simplify as far as possible.
    2. Compute the log-likelihood $\ell(\theta\mid\mathbf{x})$.
    3. Derive the MLE for $\theta$.
    4. The mean of this distribution is $1/(1+\theta)$. What is the MLE for the mean?
    Show solution

    (a) Multiply $n$ copies and pull out the $\theta$s:

    $$\mathcal{L}(\theta\mid\mathbf{x}) = \prod_{i=1}^{n}\theta(1-x_i)^{\theta-1} = \theta^n\prod_{i=1}^{n}(1-x_i)^{\theta-1} = \theta^n\left[\prod_{i=1}^{n}(1-x_i)\right]^{\theta-1}$$

    The last step works because every factor carries the same exponent $\theta-1$.

    (b) $$\ell = \ln\left(\theta^n\right) + (\theta-1)\ln\left[\prod_{i=1}^n(1-x_i)\right] = n\ln\theta + (\theta-1)\sum_{i=1}^{n}\ln(1-x_i)$$

    The log of a product becomes a sum of logs — this is the step that makes the derivative easy.

    (c) Differentiate. The sum is a constant with respect to $\theta$:

    $$\frac{d\ell}{d\theta} = \frac{n}{\theta} + \sum_{i=1}^{n}\ln(1-x_i) = 0$$

    $$\frac{n}{\theta} = -\sum_{i=1}^n \ln(1-x_i)$$

    θ̂MLE = −n / ∑ ln(1 − xi)

    Again check the sign: each $x_i\in[0,1]$ so $1-x_i\le1$ so $\ln(1-x_i)\le 0$, making the sum negative and the estimate positive.

    (d) Pure invariance. The mean is $g(\theta) = 1/(1+\theta)$, so:

    mean̂MLE = 1 / (1 + θ̂MLE)

    When you cannot do it by hand

    Lecture 4W's point: some distributions resist analytic differentiation. The gamma density contains the gamma function $\Gamma(\alpha)$, and differentiating with respect to $\alpha$ is, in your professor's words, something "we don't even dare" do. In those cases you hand the problem to a numerical optimizer, which searches for the $\theta$ minimizing $-\ell$.

    What you need from this for the exam

    Only the concept: MLEs can be found analytically (differentiate) or numerically (optimize), an optimizer works even when analysis would too, and minimizing the negative log-likelihood is the same as maximizing the likelihood. No code required.

    How MLEs behave as $n$ grows

    Your study guide asks you to "understand their asymptotic behavior." The three facts from lecture 4W:

    1. With a finite sample, MLEs are not exactly equal to the true parameter values. That is not a bug — it is the randomness of sampling. They do approach the truth as $n\to\infty$.
    2. The MLE always has the lowest mean-squared error given infinite data, though not necessarily when $n$ is small.
    3. As $n\to\infty$, the sampling distribution of $\hat{\theta}_{MLE}$ becomes normal, centred at $\theta$, with variance as small as or smaller than any other estimator's.

    In short: the MLE is not always the absolute best estimator for a parameter, but it often is, and it is almost never a bad one.

    Traps in this module
    • Differentiating $\mathcal{L}$ instead of $\ell$. Always log first. Always.
    • Forgetting the product. $\mathcal{L}$ has $n$ factors, not one — unless the question says a single datum.
    • Treating the data as variable. The $x_i$ are fixed numbers. $\sum x_i$ and $\sum\ln(1-x_i)$ are constants when you differentiate with respect to $\theta$.
    • Keeping terms that have no $\theta$. They differentiate to zero. Drop them early to save time.
    • Sign errors on $\ln(1-p)$ or $\ln(1-x)$. The chain rule contributes a $-1$.
    • Re-deriving when invariance would do. If you already have $\hat{\theta}_{MLE}$, the MLE of any function of $\theta$ is one substitution away.
    08

    20 minutes · lecture 4W

    Bias, variance, and mean-squared error

    You now have a way to produce an estimate. The natural next question is how to tell a good estimator from a bad one. Your study guide asks for this "at the conceptual level," so understanding beats memorizing.

    The dartboard

    Think of each dart as one run of the whole experiment: collect $n$ data, compute $\hat{\theta}$. The bullseye is the true $\theta$. Repeating the experiment scatters darts.

    low bias, low variance the ideal low bias, high variance right on average, unreliable high bias, low variance consistently wrong high bias, high variance the worst case
    Red dot = the true $\theta$. Blue dots = estimates from repeated experiments. Bias is accuracy (where the cluster sits). Variance is precision (how tight it is).
    Bias, variance, and mean-squared error
    $$B(\hat{\theta}) = \mathbb{E}(\hat{\theta}) - \theta \qquad\qquad \mathbb{V}(\hat{\theta}) \qquad\qquad \mathrm{MSE} = B(\hat{\theta})^2 + \mathbb{V}(\hat{\theta})$$
    • $\mathbb{E}(\hat{\theta})$the average estimate you would get across infinitely many repeats of the experiment. Since $\hat{\theta}$ is a random variable, it has an expected value.
    • $B(\hat{\theta})$bias, or accuracy. How far the centre of the cluster sits from the bullseye. $B=0$ means unbiased: right on average.
    • $\mathbb{V}(\hat{\theta})$variance, or precision. How tightly the darts group, regardless of where.
    • $\mathrm{MSE}$the two combined into one number. Bias is squared; variance is not — bias is in the units of $\theta$ and must be squared to match variance's units.

    The decision rule: given several competing estimators, choose the one with the lowest MSE. This is why a slightly biased estimator with tiny variance can beat an unbiased one with huge variance.

    Lecture 4F — the sample mean is unbiased

    Show that $\bar{X}$ is an unbiased estimator of $\mu$.

    Show solution

    $$\mathbb{E}(\bar{X}) = \mathbb{E}\!\left(\frac1n\sum_{i=1}^{n}X_i\right) = \frac1n\,\mathbb{E}\!\left(\sum_{i=1}^n X_i\right) = \frac1n\sum_{i=1}^{n}\mathbb{E}(X_i)$$

    Each $X_i$ is drawn from the same distribution (that is the "identically distributed" part of i.i.d.), so every $\mathbb{E}(X_i)=\mu$:

    $$= \frac1n\sum_{i=1}^{n}\mu = \frac1n\cdot n\mu = \mu$$

    $$B(\bar{X}) = \mathbb{E}(\bar{X}) - \mu = 0$$

    Unbiased — and for every distribution, not just some

    Note this derivation used only linearity of expectation, which needs no independence. Independence enters when you compute the variance — module 9.

    Homework 4, Question 2 — a biased MLE

    Data come from a Uniform$(0,\theta)$ distribution, flat over $[0,\theta]$. The MLE for $\theta$ is the largest observed value. You are told the expected value of the largest of $n$ data is $n\theta/(n+1)$. Write down the bias of this MLE, and confirm it goes to zero as $n\to\infty$.

    Show solution

    Apply the definition directly. Here $\hat{\theta} = \max(x_1,\ldots,x_n)$ and $\mathbb{E}(\hat{\theta}) = \dfrac{n\theta}{n+1}$:

    $$B(\hat{\theta}) = \mathbb{E}(\hat{\theta}) - \theta = \frac{n\theta}{n+1} - \theta = \frac{n\theta - \theta(n+1)}{n+1} = \frac{n\theta - n\theta - \theta}{n+1}$$

    B(θ̂) = −θ/(n+1)

    The check: as $n\to\infty$, the denominator grows without bound, so $B\to 0$. The estimator is biased for any finite sample but becomes unbiased in the limit — exactly the asymptotic behaviour lecture 4W describes for MLEs.

    Why the bias is negative, intuitively. The largest value you observe can never exceed $\theta$, and will almost surely fall a little short of it. So this estimator systematically underestimates. That is a bias you could see coming without any algebra, and the sign of your answer should match it.

    Traps in this module
    • $\mathrm{MSE} = B + \mathbb{V}$. No — the bias is squared.
    • Thinking unbiased means best. An unbiased estimator with enormous variance can have a worse MSE than a slightly biased, very precise one.
    • Bias is about the estimator, not your one estimate. $B(\hat\theta)$ describes the average over infinitely many repeats. A single estimate being off does not make an estimator biased.
    • Forgetting the sign. Bias can be negative, and the sign tells you the direction of the systematic error.
    09

    45 minutes · lecture 4F

    Statistics and sampling distributions

    The word "statistic" has been used loosely for four weeks. Here is what it actually means, and why every statistic secretly carries a distribution of its own.

    What a statistic is

    Statistic
    $$Y = g(X_1, X_2, \ldots, X_n)$$

    A function of the observed data. That is the entire definition. An MLE is a statistic, defined by an algorithm. So is the sample mean, the median, the maximum, the sample standard deviation.

    Why we use them — your professor gives two halves, and the word connecting them matters: (a) to reduce the dimensionality of the data from $n$ down to 1, while (b) retaining information about $\theta$. Doing only (a) is easy and useless.

    A bad statistic

    $$X_1 + \tan(X_2) - \pi X_{57}^{2}$$

    Perfectly legal — it is a function of the data. But it is not clearly informative about any parameter, so it fails half (b).

    Good statistics

    $$\bar{X} = \frac1n\sum_{i=1}^{n}X_i \qquad S_X = \sqrt{\frac{1}{n-1}\sum_{i=1}^n (X_i-\bar{X})^2}$$

    One targets the population mean $\mu$; the other targets the population standard deviation $\sigma$.

    Sample mean and sample standard deviation
    $$\bar{X} = \frac1n\sum_{i=1}^{n}X_i \qquad\qquad S_X = \left[\frac{1}{n-1}\sum_{i=1}^{n}(X_i - \bar{X})^2\right]^{1/2}$$
    • $\bar{X}$your estimate of the population mean $\mu$.
    • $S_X$your estimate of the population standard deviation $\sigma$. Compare it with the population version $\mathbb{V}(X)=\int(x-\mu)^2f_X(x)\,dx$: same idea, but the integral over the true density is replaced by an average over your actual data, and $\mu$ by $\bar X$.
    • $n-1$not $n$. Because you used the data once already to compute $\bar{X}$, dividing by $n$ would systematically underestimate the spread. The $n-1$ corrects that bias.

    The key idea: statistics are random variables

    Read this three times

    A statistic is a function of the data. The data are random. So the statistic is random. Which means it has its own distribution — called its sampling distribution — and its own mean and standard deviation.

    The sampling distribution is the p.m.f. or p.d.f. you would see if you repeated the entire experiment over and over, computing the statistic each time, and made a histogram of the results.

    Standard error
    $$\mathrm{SE}(Y) = \sqrt{\mathbb{V}(Y)}$$

    The standard deviation of the sampling distribution. It measures the width of the statistic's distribution, not the data's.

    The distinction that trips people up: $\sigma$ is how spread out the individual data are. $\mathrm{SE}$ is how spread out your estimate is across hypothetical repeats of the experiment. They are different quantities with different sizes.

    Why any of this matters

    Point estimates do not need sampling distributions — you can compute an MLE without one. Sampling distributions are needed for the two things that come after: hypothesis tests and confidence intervals. Those are Exam 2. What Exam 1 asks is that you can identify the right sampling distribution for a problem described in words.

    The two results you must be able to derive

    Lecture 4F — the expected value and variance of the sample mean

    You sample $n$ i.i.d. data from some distribution with mean $\mu$ and variance $\sigma^2$. Find $\mathbb{E}(\bar{X})$, $\mathbb{V}(\bar{X})$, and $\mathrm{SE}(\bar{X})$.

    Show solution

    Expected value. Pull the constant $1/n$ out, then distribute the expectation over the sum:

    $$\mathbb{E}(\bar{X}) = \mathbb{E}\!\left(\frac1n\sum_{i=1}^n X_i\right) = \frac1n\mathbb{E}\!\left(\sum_{i=1}^n X_i\right) = \frac1n\sum_{i=1}^{n}\mathbb{E}(X_i) = \frac1n\sum_{i=1}^n \mu = \frac1n \cdot n\mu$$

    E(X̄) = μ

    Your professor's note: this step is true for any set of $X_i$ — linearity of expectation needs no assumptions. And the conclusion holds for every distribution: the expected value of the sample mean is the true mean.

    Variance. Now the constant comes out squared:

    $$\mathbb{V}(\bar{X}) = \mathbb{V}\!\left(\frac1n\sum_{i=1}^n X_i\right) = \frac{1}{n^2}\mathbb{V}\!\left(\sum_{i=1}^n X_i\right) = \frac{1}{n^2}\sum_{i=1}^{n}\mathbb{V}(X_i) = \frac{1}{n^2}\sum_{i=1}^n \sigma^2 = \frac{n\sigma^2}{n^2}$$

    V(X̄) = σ²/n

    The middle step is only valid for independent data. This is the one place in the derivation where independence is load-bearing — your professor marks it explicitly.

    Standard error.

    SE(X̄) = σ/√n

    What this means: more data means less uncertainty about the true mean. But the $\sqrt{n}$ is important — to halve your uncertainty you need four times the data.

    Sufficiency

    A sufficient statistic for a parameter $\theta$ captures all the information about $\theta$ contained in the sample. Two consequences:

    • Computing additional statistics beyond the sufficient one will not improve your inference about $\theta$.
    • No single statistic can contain more information about $\theta$ than the sufficient one does.

    So the sufficient statistic is the optimal summary. For each distribution you work with, you need to know which statistic is sufficient and what its sampling distribution is.

    The master table

    This is lecture 4F page 3, and it is the densest single page of the course. Every entry is also on your formula sheet.

    Data distributionInferringSufficient statisticIts sampling distribution
    $X\sim\mathrm{Binom}(k,p)$$p$$X_+ = \sum_{i=1}^{n}X_i$$X_+ \sim \mathrm{Binom}(nk,\,p)$
    $X\sim\mathrm{Pois}(\lambda)$$\lambda$$X_+ = \sum_{i=1}^n X_i$$X_+\sim\mathrm{Pois}(n\lambda)$
    $X\sim\mathcal{N}(\mu,\sigma^2)$, $\sigma$ known$\mu$$\bar{X}$$\bar{X}\sim\mathcal{N}(\mu,\;\sigma^2/n)$
    $X\sim\mathcal{N}(\mu,\sigma^2)$, $\sigma$ unknown$\mu$$T = \dfrac{\bar{X}-\mu}{S_X/\sqrt{n}}$$T\sim t(n-1)$
    $X\sim\mathcal{N}(\mu,\sigma^2)$$\sigma^2$$W = \dfrac{(n-1)S_X^2}{\sigma^2}$$W \sim \chi^2(n-1)$
    $X\sim\mathrm{Exp}(\theta)$$\theta$$\bar{X}$$\bar{X}\sim\Gamma(n,\;\theta/n)$

    The $t$ distribution

    Has one parameter: the degrees of freedom, here $n-1$. It appears whenever you had to estimate $\sigma$ with $S_X$ rather than knowing it. It looks like a normal with heavier tails, reflecting the extra uncertainty, and approaches the normal as $n$ grows.

    $\chi^2$ and $\Gamma$

    Chi-squared, also parameterized by degrees of freedom, governs inference about a variance. Gamma has a shape parameter $n$ and a scale parameter $\theta/n$; the exponential is the special case with shape 1. You need to recognize the names and read the parameters off the table.

    Reading the binomial row carefully

    $\mathrm{Binom}(nk,p)$ has two numbers multiplied: $n$ data points, each from an experiment with $k$ trials. Six repetitions of a three-flip experiment give $nk = 18$ total flips. If a problem only ever mentions one number, look hard at which one it is.

    Worksheet 4F

    You flip a coin three times and record the number of heads. You repeat the experiment five more times. The data are:

    2   2   1   2   3   1

    Compute the observed value of the relevant sufficient statistic for the binomial probability $p$, and write down the sampling distribution for that statistic.

    Show solution

    Step 1 — identify $n$ and $k$. Each datum is the number of heads in $k=3$ flips, so each $X_i\sim\mathrm{Binom}(3,p)$. You have six data values, so $n=6$.

    Step 2 — the sufficient statistic for $p$ is the total.

    $$X_+ = \sum_{i=1}^{6}X_i = 2+2+1+2+3+1 = 11$$

    Step 3 — read its sampling distribution off the table. $X_+\sim\mathrm{Binom}(nk, p)$ with $nk = 6\times 3 = 18$.

    X₊ = 11,   and   X₊ ~ Binom(18, p)

    The logic is physical: you flipped the coin 18 times in total and saw 11 heads. Whether those flips came in groups of three is irrelevant to what they tell you about $p$ — which is exactly what "sufficient" means. And $p$ stays as a symbol; you are describing the distribution, not estimating the parameter.

    Traps in this module
    • $\sigma$ vs $S_X$ vs SE. Population spread, sample estimate of that spread, and spread of the statistic. Three different things.
    • $\mathbb{V}(\bar X) = \sigma^2/n$, not $\sigma/n$ and not $\sigma^2/\sqrt{n}$. The square root appears only when you convert to standard error.
    • $n$ vs $k$ in the binomial row.
    • Choosing the $t$ row when $\sigma$ was given. If the problem hands you the population standard deviation, use the normal row. Only reach for $t$ when you had to compute $S_X$ from data.
    • Using $n$ instead of $n-1$ in $S_X$ and in the degrees of freedom.
    10

    30 minutes · lecture 5M

    The central limit theorem

    The master table in module 9 covers four distributions. The world contains rather more than four. The central limit theorem is what you use when you do not know the sampling distribution and cannot look it up.

    The problem it solves

    Suppose your data come from $f_X(x) = \theta x^{\theta-1}$ on $[0,1]$. You want to infer the population mean. That distribution is not in the table, and there is no tidy sampling distribution for $\bar X$ to look up. You are stuck — unless the sample is reasonably large.

    The central limit theorem
    $$\text{as } n\to\infty: \qquad \bar{X} \;\stackrel{d}{\longrightarrow}\; Y \sim \mathcal{N}\!\left(\mu,\; \frac{\sigma^2}{n}\right)$$
    • $\stackrel{d}{\to}$"converges in distribution" — how statisticians say the sampling distribution looks more and more like a normal as $n$ grows.
    • $\mu$the true population mean of the individual data. The sample mean centres on it.
    • $\sigma^2/n$the variance you derived in module 9. The CLT adds the shape; the mean and variance were already known.

    The extraordinary part: the right-hand side does not mention the distribution the data came from. Whatever it was — skewed, discrete, bizarre — the sample mean goes normal.

    The rule of thumb

    By convention you need $n \gtrsim 30$ to apply the CLT "safely." Your professor's caveat: there is nothing magical about 30. For some distributions the CLT works fine with fewer; for others it works poorly even with more. It is a convention, not a theorem.

    Three conditions and one non-condition

    You need

    • The data are i.i.d.
    • $n$ is large enough (the $\gtrsim 30$ convention).
    • You are inferring the population mean $\mu$, and only the mean.

    You do not need

    To know or identify the distribution the individual data came from. Your professor states this as the crucial contrast: with the CLT you never have to identify the data-generating distribution; with simulation you do.

    Two practical rules

    When $\sigma$ is unknown
    $$\bar{X}\;\dot\sim\;\mathcal{N}\!\left(\mu,\;\frac{S_X^2}{n}\right) \qquad \mathrm{SE}(\bar{X}) \approx \frac{S_X}{\sqrt{n}}$$

    In real problems you almost never know $\sigma$. You still use the CLT — you simply plug the sample standard deviation $S_X$ in where $\sigma$ would go.

    When you know the exact sampling distribution, use it

    Your professor writes this out and underlines it. The CLT is an approximation and a fallback. If the data are normal with known $\sigma$, then $\bar X\sim\mathcal{N}(\mu,\sigma^2/n)$ is exactly true for any $n$ — no CLT needed, no $n\ge30$ requirement. Reach for the table in module 9 first; reach for the CLT only when the table fails you.

    The third option: simulation

    If you cannot look up the sampling distribution and cannot use the CLT — because you are not inferring the mean, or because $n$ is too small — you can estimate the sampling distribution by simulation. The algorithm, conceptually:

    1. Assume a distribution for each datum, and values for its parameters.
    2. Assume a sample size $n$.
    3. Fix the random-number seed so the result is reproducible.
    4. Choose how many datasets to simulate.
    5. Set up storage for the results.
    6. Loop: generate a dataset, compute the statistic, store the value.
    7. Process the stored values — that collection is your estimated sampling distribution.

    The cost of this route is the assumption in step 1: unlike the CLT, simulation requires you to commit to a distribution for the individual data. (If you cannot even do that, there is a method called the bootstrap. It is not on this exam.)

    Lecture 5M

    You sample $n=100$ i.i.d. data from an unknown distribution with mean $\mu=20$ and variance $\sigma^2 = 4$. What is $\mathbb{P}(19.8\le \bar{X}\le 20.2)$?

    Show solution

    Step 1 — check you may use the CLT. The distribution is unknown, so the table is no help. But $n=100\ge30$, the data are i.i.d., and the question is about the mean. CLT applies.

    Step 2 — write the approximate sampling distribution.

    $$\bar{X}\;\dot\sim\;\mathcal{N}\!\left(20,\;\frac{4}{100}\right) \qquad \mathrm{SE}(\bar X) = \frac{\sigma}{\sqrt{n}} = \frac{2}{10} = 0.2$$

    Step 3 — express the probability with c.d.f.s. The range $[19.8, 20.2]$ is exactly $\mu\pm0.2 = \mu\pm 1\,\mathrm{SE}$:

    $$\mathbb{P}(19.8\le\bar X\le 20.2) = F_{\bar X}(20.2) - F_{\bar X}(19.8)$$

    Since this is one standard error either side of the mean, the empirical rule gives it directly:

    ≈ 0.683

    In lecture your professor evaluates the two c.d.f. terms numerically. On the exam the numbers will land on 1, 2, or 3 standard errors — or you will be asked to stop at the c.d.f. expression.

    Worksheet 5M

    You sample $n=64$ data from a distribution whose population mean is $\mu = 10$. The sample standard deviation is $S_X = 4$. What is $\mathbb{P}(\bar X \le 9)$, at least approximately?

    Show solution

    Step 1 — $\sigma$ is unknown, so substitute $S_X$.

    $$\mathrm{SE}(\bar X) \approx \frac{S_X}{\sqrt{n}} = \frac{4}{\sqrt{64}} = \frac48 = 0.5$$

    Step 2 — the CLT sampling distribution. $n=64\ge30$:

    $$\bar{X}\;\dot\sim\;\mathcal{N}(10,\; 0.5^2)$$

    Step 3 — how many standard errors below the mean is 9?

    $$\frac{9-10}{0.5} = -2 \qquad\text{so } 9 = \mu - 2\,\mathrm{SE}$$

    Step 4 — use symmetry. The central $\pm2$ region holds 0.954, so the two tails together hold $1-0.954 = 0.046$, and the lower tail alone holds half of that:

    $$\mathbb{P}(\bar X\le 9) \approx \frac{0.046}{2}$$

    ≈ 0.023

    The worksheet asked for this as code. The setup above — identify SE, count standard errors, halve the leftover — is the part that would be graded on a no-code exam.

    Homework 4, Question 1 — resistors

    Resistors have average resistance 100 ohms with standard deviation 10 ohms. You randomly select $n=36$.

    1. What is the sampling distribution of $\bar X$? Name it and give its parameter values.
    2. What is the estimated probability that the measured average resistance falls between 99 and 101 ohms?
    Show solution

    (a) You are not told the distribution of an individual resistor, so this is a CLT problem. Check: $n=36\ge30$, i.i.d. (randomly selected), inferring the mean. Then

    $$\mathbb{E}(\bar X) = \mu = 100 \qquad \mathbb{V}(\bar X) = \frac{\sigma^2}{n} = \frac{100}{36} = 2.78$$

    X̄ ˙~ Normal(μ = 100, σ²/n = 100/36 ≈ 2.78)

    Give the variance, not the standard deviation, when the distribution is written $\mathcal{N}(\mu,\sigma^2)$. The standard error is $\sqrt{2.78} = 10/6 \approx 1.667$.

    (b) Express it in standard errors:

    $$\frac{101-100}{1.667} = 0.6 \qquad\text{so the range is } \mu \pm 0.6\,\mathrm{SE}$$

    $$\mathbb{P}(99\le\bar X\le101) = F_{\bar X}(101) - F_{\bar X}(99)$$

    ≈ 0.451

    0.6 is not one of the memorized values, which is exactly why the homework told you to use two lines of Python. On a no-code exam, the setup above is the answer — identify the distribution, compute the standard error, and write the probability as a difference of c.d.f.s. Getting to that line is what earns the marks.

    Traps in this module
    • Applying the CLT to a single datum. It governs $\bar X$, not $X$. One resistor is not approximately normal just because 36 of them average out that way.
    • Using $\sigma$ where $\sigma/\sqrt n$ belongs. The sampling distribution is much narrower than the data distribution. This is the most common CLT error.
    • Using the CLT when an exact answer exists. Normal data with known $\sigma$ gives an exact result for any $n$.
    • Applying it to something other than a mean. The CLT says nothing about the sample maximum, the variance, or the median.
    • Forgetting to halve. A one-sided tail is half the leftover, not all of it.
    11

    20 minutes

    The formula sheet, line by line

    Every line on your sheet, in the sheet's own order, with what it does and when to reach for it. Read this last, the night before. During the exam you want to find a formula by recognizing the situation, not by scanning.

    Basic probability

    On the sheetWhat it is / when you reach for it
    $A\cup B$, $A\cap B$, $A^c$, $A\subset B$or · and · not · is contained in
    distributive & associative lawsRearranging nested and/or descriptions into disjoint pieces you can add.
    $(A\cap B)^c = A^c\cup B^c$
    $(A\cup B)^c = A^c\cap B^c$
    De Morgan. The engine behind every "at least one" and "neither" problem.
    $\mathbb{P}(\Omega)=1$; $0\le\mathbb{P}(A)\le1$; disjoint sumThe three axioms. The third is why p.m.f. values may be added over a range.
    $\mathbb{P}(A^c\cap B^c) = 1-\mathbb{P}(A\cup B)$"Neither." Use it backwards when the problem gives you $\mathbb{P}(\text{miss both})$.

    Conditional probability

    On the sheetWhat it is / when you reach for it
    $\mathbb{P}(A\mid B) = \mathbb{P}(A\cap B)/\mathbb{P}(B)$The definition. Any "given", "if", "among those".
    $\mathbb{P}(A\mid B)=\mathbb{P}(A)$ iff independent
    $\mathbb{P}(A\cap B)=\mathbb{P}(A)\mathbb{P}(B)$ iff independent
    Two equivalent tests for independence. Use only when stated or physically obvious.
    $\mathbb{P}(A\cap B) = \mathbb{P}(A\mid B)\mathbb{P}(B)$Multiplicative rule. Each branch of a decision tree.
    $\mathbb{P}(A\cup B) = \mathbb{P}(A)+\mathbb{P}(B)-\mathbb{P}(A\cap B)$Additive rule. Any "or".
    $\mathbb{P}(A) = 1-\mathbb{P}(A^c)$The complement trick. Use it more than you think you should.
    $\mathbb{P}(A)=\sum_i\mathbb{P}(A\mid B_i)\mathbb{P}(B_i)$L.o.T.P. Getting an unconditional probability out of conditionals. Usually builds the Bayes denominator. The $B_i$ must be disjoint and exhaustive.
    $\mathbb{P}(A\mid B) = \mathbb{P}(B\mid A)\mathbb{P}(A)/\mathbb{P}(B)$Bayes' rule. Reversing a conditional.

    Probability mass functions — $p_X(x)$

    On the sheetWhat it is / when you reach for it
    $0\le p_X(x)\le 1$, $\sum_x p_X(x)=1$The two defining properties. The sum condition is how you find a missing table value.
    $F_X(x) = \mathbb{P}(X\le x) = \sum_{y\le x}p_Y(y)$The c.d.f.: accumulated probability at or below $x$.
    $F_X^{-1}(q) = \inf\{x: F_X(x)\ge q\}$The quantile function. The smallest $x$ whose accumulated probability reaches $q$.
    $\mathbb{P}(a\le X\le b)$ vs $\mathbb{P}(aTwo separate lines on your sheet because for discrete variables they differ. Check every inequality.

    Probability density functions — $f_X(x)$

    On the sheetWhat it is / when you reach for it
    $f_X(x)\ge0$, $\int_x f_X(x)\,dx = 1$Note it is not bounded above by 1. The integral condition is how you solve for a normalizing constant $c$.
    $F_X(x) = \int_{y\le x} f_Y(y)\,dy$The c.d.f. as accumulated area.
    $f_X(x) = \frac{d}{dx}F_X(x)$Density is the slope of the c.d.f. Use it to go backwards from a c.d.f.
    $\mathbb{P}(aOnly one line here, unlike the p.m.f. section — because $\mathbb{P}(X=a)=0$, so the inequality type does not matter.

    Characterizing distributions

    On the sheetWhat it is / when you reach for it
    $\mathbb{E}(g(X)) = \sum_x g(x)p_X(x)$ or $\int g(x)f_X(x)dx$LOTUS. The master formula — every line below is a special case.
    $\mathbb{E}(X) = \sum_x x\,p_X(x)$ or $\int x f_X(x)dx$The mean. $g(x)=x$.
    $\mathbb{V}(X) = \sigma^2 = \mathbb{E}(X^2)-(\mathbb{E}(X))^2$The shortcut formula. Always use this rather than the definition.
    $\sigma = \sqrt{\mathbb{V}(X)}$Do not forget the square root when asked for a standard deviation.
    $\mathbb{E}(aX+b) = a\mathbb{E}(X)+b$Linear transformations of a random variable.
    $\mathbb{V}(aX+b) = a^2\mathbb{V}(X)$$a$ squared, and the $b$ vanishes.
    $\mathbb{E}(\sum X_i) = \sum\mathbb{E}(X_i)$Always true — no assumptions needed.
    $\mathbb{V}(\sum X_i)=\sum\mathbb{V}(X_i)$"For independent data only" — the sheet says so explicitly. This caveat is load-bearing.

    Statistics

    On the sheetWhat it is / when you reach for it
    $Y = g(X_1,\ldots,X_n)$The definition of a statistic: a function of the observed data.
    i.i.d.Independent and identically distributed.
    $\bar{X} = \frac1n\sum X_i$Sample mean.
    $S_X = \left[\frac{1}{n-1}\sum(X_i-\bar X)^2\right]^{1/2}$Sample standard deviation. Note the $n-1$.
    $Y$ drawn from its sampling distributionStatistics are random variables and have distributions of their own.
    $\mathrm{SE}(Y) = \sqrt{\mathbb{V}(Y)}$Standard error: the standard deviation of the statistic, not of the data.
    $\mathbb{E}(\bar X) = \mu$True for every distribution. The sample mean is unbiased.
    $\mathbb{V}(\bar X) = \sigma^2/n$So $\mathrm{SE}(\bar X) = \sigma/\sqrt n$. Four times the data halves the uncertainty.

    Likelihood and MLE

    On the sheetWhat it is / when you reach for it
    $\mathcal{L}(\theta\mid\mathbf{x}) = \prod_i p_X(x_i;\theta)$ or $\prod_i f_X(x_i;\theta)$Step 1. The product is legitimate because the data are independent.
    $\ell(\theta\mid\mathbf{x}) = \ln\mathcal{L}(\theta\mid\mathbf{x})$Step 2. Turns the product into a sum without moving the maximum.
    $\hat\theta_{MLE}$: the $\theta$ with $d\ell/d\theta = 0$Steps 3–5. Differentiate, set to zero, solve.
    $\widehat{g(\theta)}_{MLE} = g(\hat\theta_{MLE})$Invariance. Never re-derive — substitute.

    Estimator properties

    On the sheetWhat it is / when you reach for it
    $B(\hat\theta) = \mathbb{E}(\hat\theta)-\theta$Bias = accuracy. Zero means unbiased.
    $\mathbb{V}(\hat\theta)$Variance = precision.
    $\mathrm{MSE} = B(\hat\theta)^2 + \mathbb{V}(\hat\theta)$Bias squared plus variance. Choose the estimator with the lowest MSE.

    The four distributions

    FamilyLine on the sheetNote
    Binomial
    $X\sim\mathrm{Binom}(k,p)$
    $p_X(x)=\binom{k}{x}p^x(1-p)^{k-x}$$x\in\{0,\ldots,k\}$. The coefficient counts orderings.
    $\binom{k}{x} = k!/[x!(k-x)!]$And $0!=1$, so $\binom{k}{0}=1$.
    $\mathbb{E}=kp$; $\mathbb{V}=kp(1-p)$; Bernoulli is $k=1$$\sigma = \sqrt{kp(1-p)}$.
    $X_+ = \sum X_i \sim \mathrm{Binom}(nk,p)$Sufficient statistic. $n$ data × $k$ trials each.
    Poisson
    $X\sim\mathrm{Pois}(\lambda)$
    $p_X(x) = \lambda^x e^{-\lambda}/x!$$\mathbb{P}(X=0)=e^{-\lambda}$, used constantly.
    $\mathbb{E}=\mathbb{V}=\lambda$; $\sigma=\sqrt\lambda$Mean equals variance — unique to Poisson.
    $X_+ = \sum X_i\sim\mathrm{Pois}(n\lambda)$Sufficient statistic for $\lambda$.
    Normal
    $X\sim\mathcal{N}(\mu,\sigma^2)$
    $f_X(x)=\frac{1}{\sqrt{2\pi\sigma^2}}e^{-(x-\mu)^2/(2\sigma^2)}$Symmetric about $\mu$. No closed-form integral.
    $\mathbb{E}=\mu$; $\mathbb{V}=\sigma^2$The parameters are the moments.
    $\sigma$ known: $\bar X\sim\mathcal{N}(\mu,\sigma^2/n)$
    $\sigma$ unknown: $T=\frac{\bar X-\mu}{S_X/\sqrt n}\sim t(n-1)$
    Which row you use depends on whether $\sigma$ was handed to you.
    $\mathbb{P}(\mu\pm i\sigma)\approx 0.683,\,0.954,\,0.997$The only way to get a normal probability by hand. Halve the leftover for one tail.
    Exponential
    $X\sim\mathrm{Exp}(\theta)$
    $f_X(x)=\frac1\theta e^{-x/\theta}$, $x\ge0$Derive once: $F_X(x)=1-e^{-x/\theta}$.
    $\mathbb{E}=\theta$; $\mathbb{V}=\theta^2$; $\sigma=\theta$Mean equals standard deviation.
    $\bar X\sim\Gamma(n,\theta/n)$Sufficient statistic for $\theta$; shape $n$, scale $\theta/n$.
    Three things that are not on the sheet but you will need
    1. $F_X(x) = 1-e^{-x/\theta}$ for the exponential. Derive it in 15 seconds or memorize it.
    2. $\mathbb{P}(\text{at least one}) = 1 - \mathbb{P}(\text{none})$. Not written anywhere, used everywhere.
    3. The log rules: $\ln(AB)=\ln A+\ln B$, $\ln(A^n)=n\ln A$, $\ln(e^a)=a$. Every MLE derivation needs them.
    12

    45 minutes

    Exam simulation

    Twelve fresh problems spanning every module, in roughly the order the material was taught. None of them repeat a homework or worksheet question. Work each one on paper before opening the solution — reading a solution you have not attempted teaches you almost nothing.

    1 · module 2

    In a factory, 60% of units pass inspection A, 50% pass inspection B, and 35% pass both.

    1. What is the probability a unit passes at least one inspection?
    2. Exactly one?
    3. Neither?
    4. Are the two inspections independent?
    Show solution

    (a) $\mathbb{P}(A\cup B) = 0.60+0.50-0.35 = 0.75$

    (b) "Exactly one" is $A\cup B$ minus the overlap: $0.75-0.35 = 0.40$. Or build the table: $\mathbb{P}(A\cap B^c) = 0.60-0.35 = 0.25$ and $\mathbb{P}(A^c\cap B)=0.50-0.35=0.15$, summing to 0.40.

    (c) $\mathbb{P}(A^c\cap B^c) = 1-\mathbb{P}(A\cup B) = 0.25$

    (d) Test it: $\mathbb{P}(A)\mathbb{P}(B) = (0.6)(0.5) = 0.30$, but $\mathbb{P}(A\cap B)=0.35$.

    0.75 · 0.40 · 0.25 · not independent

    2 · module 3

    2% of chips coming off a line are defective. A sensor flags 95% of defective chips, and also flags 4% of good chips. A chip is flagged. What is the probability it is actually defective?

    Show solution

    Let $D$ = defective, $F$ = flagged. Given: $\mathbb{P}(D)=0.02$, $\mathbb{P}(F\mid D)=0.95$, $\mathbb{P}(F\mid D^c)=0.04$. You want $\mathbb{P}(D\mid F)$.

    Denominator by L.o.T.P.:

    $$\mathbb{P}(F) = (0.95)(0.02) + (0.04)(0.98) = 0.019 + 0.0392 = 0.0582$$

    Bayes:

    $$\mathbb{P}(D\mid F) = \frac{0.019}{0.0582}$$

    ≈ 0.326

    A flagged chip is still probably fine. The sensor is good, but defects are rare, so the 4% false-alarm rate applied to 98% of chips produces more false flags than true ones. Recognizing this pattern is what Bayes questions are testing.

    3 · module 4 & 5

    $f_X(x) = c(1+x)$ on $x\in[0,2]$, zero elsewhere.

    1. Find $c$.
    2. Find $\mathbb{E}(X)$.
    3. Find $\mathbb{P}(X>1 \mid X>0.5)$.
    Show solution

    (a) $$\int_0^2 c(1+x)\,dx = c\left[x+\tfrac{x^2}{2}\right]_0^2 = c(2+2) = 4c = 1$$

    c = 1/4

    (b) $$\mathbb{E}(X) = \int_0^2 x\cdot\tfrac14(1+x)\,dx = \tfrac14\int_0^2 (x+x^2)\,dx = \tfrac14\left[\tfrac{x^2}{2}+\tfrac{x^3}{3}\right]_0^2 = \tfrac14\left(2+\tfrac83\right) = \tfrac14\cdot\tfrac{14}{3}$$

    E(X) = 7/6 ≈ 1.167

    Above the midpoint of 1, which is right — the density increases with $x$, so mass piles toward the upper end.

    (c) Since $\{X>1\}\subset\{X>0.5\}$, the conditional is a plain ratio.

    $$\mathbb{P}(X>1) = \tfrac14\left[x+\tfrac{x^2}{2}\right]_1^2 = \tfrac14\left(4 - 1.5\right) = \tfrac58$$

    $$\mathbb{P}(X>0.5) = \tfrac14\left[x+\tfrac{x^2}{2}\right]_{0.5}^{2} = \tfrac14\left(4 - 0.625\right) = \tfrac{27}{32}$$

    $$\frac{5/8}{27/32} = \frac58\cdot\frac{32}{27}$$

    20/27 ≈ 0.741

    4 · module 5

    $X$ has $p_X(1)=0.2$, $p_X(2)=0.5$, $p_X(4)=0.3$. Define $Y = 5-2X$. Find $\mathbb{E}(Y)$, $\mathbb{V}(Y)$ and $\sigma_Y$.

    Show solution

    Do $X$ first, then transform — do not rebuild a p.m.f. for $Y$.

    $$\mathbb{E}(X) = 1(0.2)+2(0.5)+4(0.3) = 0.2+1.0+1.2 = 2.4$$

    $$\mathbb{E}(X^2) = 1(0.2)+4(0.5)+16(0.3) = 0.2+2.0+4.8 = 7.0$$

    $$\mathbb{V}(X) = 7.0-(2.4)^2 = 7.0-5.76 = 1.24$$

    Now apply the rules with $a=-2$, $b=5$:

    $$\mathbb{E}(Y) = -2(2.4)+5 = 0.2 \qquad \mathbb{V}(Y) = (-2)^2(1.24) = 4.96 \qquad \sigma_Y = |{-2}|\sqrt{1.24}$$

    E(Y) = 0.2 · V(Y) = 4.96 · σY ≈ 2.227

    The negative $a$ flips the distribution but the variance is unaffected by the sign, because $a$ enters squared.

    5 · module 6

    A machine produces parts that are defective 8% of the time, independently. You inspect 12 parts.

    1. What is the probability exactly 2 are defective?
    2. At least one?
    3. What are $\mathbb{E}(X)$ and $\sigma$ for the number of defectives?
    Show solution

    The number of trials is fixed at 12, so $X\sim\mathrm{Binom}(12, 0.08)$.

    (a) $$p_X(2) = \binom{12}{2}(0.08)^2(0.92)^{10} = 66(0.0064)(0.4344)$$

    ≈ 0.183

    $\binom{12}{2} = \frac{12!}{2!\,10!} = \frac{12\cdot11}{2} = 66$.

    (b) $$\mathbb{P}(X\ge1) = 1-\mathbb{P}(X=0) = 1-(0.92)^{12} = 1-0.3677$$

    ≈ 0.632

    (c) $$\mathbb{E}(X) = kp = 12(0.08) = 0.96 \qquad \mathbb{V}(X) = 12(0.08)(0.92) = 0.8832$$

    E(X) = 0.96 · σ ≈ 0.940

    6 · module 6

    Defects occur along an optical fiber at an average rate of 0.3 per meter.

    1. What is the probability a 5-metre span has no defects?
    2. More than one defect?
    3. How long a span gives a 90% chance of containing at least one defect?
    Show solution

    Counts in a window with no upper bound — Poisson. Convert the rate first.

    (a) For a 5-metre window, $\lambda = 0.3\times 5 = 1.5$:

    $$\mathbb{P}(X=0) = e^{-1.5}$$

    ≈ 0.223

    (b) $$\mathbb{P}(X>1) = 1 - p_X(0)-p_X(1) = 1 - e^{-1.5} - 1.5e^{-1.5} = 1 - e^{-1.5}(2.5) = 1-0.558$$

    ≈ 0.442

    (c) For a span of length $L$, $\lambda = 0.3L$. Set up and solve:

    $$\mathbb{P}(X\ge1) = 1-e^{-0.3L} = 0.9 \;\Longrightarrow\; e^{-0.3L} = 0.1 \;\Longrightarrow\; 0.3L = \ln 10$$

    L = ln(10)/0.3 ≈ 7.68 metres

    7 · module 6

    Bolt lengths are normally distributed with $\mu = 50$ mm and $\sigma^2 = 4$ mm².

    1. $\mathbb{P}(46\le X\le 54)$?
    2. $\mathbb{P}(X>54)$?
    3. $\mathbb{P}(X>54\mid X>52)$?
    4. Give the interval containing the central 95% of bolts.
    Show solution

    First: $\sigma^2 = 4$, so $\sigma = 2$. That step is worth a mark on its own.

    (a) $46 = \mu-2\sigma$ and $54 = \mu+2\sigma$, so this is the central $\pm2\sigma$ band.

    ≈ 0.954

    (b) Leftover $= 1-0.954 = 0.046$, split evenly between the two tails.

    ≈ 0.023

    (c) $52 = \mu+\sigma$, so $\mathbb{P}(X>52) = (1-0.683)/2 = 0.1585$. Since $\{X>54\}\subset\{X>52\}$:

    $$\frac{0.023}{0.1585}$$

    ≈ 0.145

    (d) $\mu\pm1.960\sigma = 50\pm 1.960(2) = 50\pm3.92$

    [46.08, 53.92] mm

    8 · module 6

    A component's lifetime is exponential with mean 500 hours.

    1. Probability it fails within 200 hours?
    2. Probability it lasts beyond 1000 hours?
    3. What is $\sigma$?
    4. What mean lifetime would give a 90% chance of surviving 200 hours?
    Show solution

    "Mean 500 hours" means $\theta = 500$. Use $F_X(x) = 1-e^{-x/\theta}$.

    (a) $$\mathbb{P}(X\le 200) = 1-e^{-200/500} = 1-e^{-0.4}$$

    ≈ 0.330

    (b) $$\mathbb{P}(X>1000) = e^{-1000/500} = e^{-2}$$

    ≈ 0.135

    (c) For an exponential the standard deviation equals the mean.

    σ = 500 hours

    (d) Now solve for the parameter:

    $$e^{-200/\theta} = 0.9 \;\Longrightarrow\; -\frac{200}{\theta} = \ln(0.9) \;\Longrightarrow\; \theta = \frac{200}{-\ln 0.9} = \frac{200}{0.1054}$$

    θ ≈ 1898 hours

    9 · module 7 — a distribution you have not seen

    You sample $n$ i.i.d. data from $f_X(x;\theta) = \theta^2 x\, e^{-\theta x}$, for $x>0$ and $\theta>0$.

    1. Write and simplify the likelihood.
    2. Take the log-likelihood.
    3. Derive $\hat\theta_{MLE}$.
    4. The mean of this distribution is $2/\theta$. What is the MLE of the mean?
    Show solution

    (a) $$\mathcal{L}(\theta\mid\mathbf{x}) = \prod_{i=1}^{n}\theta^2 x_i e^{-\theta x_i} = \theta^{2n}\left(\prod_{i=1}^n x_i\right)e^{-\theta\sum_{i=1}^n x_i}$$

    Three groups: the $\theta^2$ terms multiply to $\theta^{2n}$; the $x_i$ collect into a product; the exponents add.

    (b) $$\ell = 2n\ln\theta + \sum_{i=1}^{n}\ln x_i - \theta\sum_{i=1}^{n}x_i$$

    (c) Differentiate. The middle term has no $\theta$, so it disappears:

    $$\frac{d\ell}{d\theta} = \frac{2n}{\theta} - \sum_{i=1}^{n}x_i = 0 \;\Longrightarrow\; \theta = \frac{2n}{\sum x_i}$$

    θ̂MLE = 2n / ∑xi = 2/X̄

    (d) Invariance: push $\hat\theta_{MLE}$ through $g(\theta)=2/\theta$.

    $$\widehat{\text{mean}} = \frac{2}{\hat\theta_{MLE}} = \frac{2}{2/\bar X}$$

    = X̄

    A satisfying result: the MLE of the population mean turns out to be the sample mean. The "spot the term with no $\theta$ and drop it" move in (c) is the one that makes these problems fast.

    10 · module 8

    You have $n=4$ i.i.d. data with population mean $\mu$ and variance $\sigma^2$. Consider three estimators of $\mu$:

    $$\hat\theta_1 = \bar X \qquad \hat\theta_2 = \frac{X_1+X_2}{2} \qquad \hat\theta_3 = 0.9\,\bar X$$

    Find the bias, variance, and MSE of each. Which would you choose?

    Show solution
    $\mathbb{E}(\hat\theta)$BiasVarianceMSE
    $\hat\theta_1$$\mu$0$\sigma^2/4$$0.25\sigma^2$
    $\hat\theta_2$$\mu$0$\sigma^2/2$$0.50\sigma^2$
    $\hat\theta_3$$0.9\mu$$-0.1\mu$$0.81\sigma^2/4$$0.01\mu^2 + 0.2025\sigma^2$

    Where each entry comes from. $\hat\theta_2$ is a sample mean of only two data, so its variance is $\sigma^2/2$ — throwing away half your data doubles the variance. For $\hat\theta_3$, use $\mathbb{E}(aX)=a\mathbb{E}(X)$ and $\mathbb{V}(aX)=a^2\mathbb{V}(X)$ with $a=0.9$, giving $0.81\times\sigma^2/4 = 0.2025\sigma^2$.

    Which to choose. $\hat\theta_1$ beats $\hat\theta_2$ always: same bias, half the variance. Comparing $\hat\theta_1$ and $\hat\theta_3$:

    $$0.01\mu^2 + 0.2025\sigma^2 < 0.25\sigma^2 \;\iff\; 0.01\mu^2 < 0.0475\sigma^2 \;\iff\; |\mu| < 2.18\,\sigma$$

    θ̂₃ wins when |μ| < 2.18σ; otherwise θ̂₁

    This is the whole point of MSE: a deliberately biased estimator can beat an unbiased one when it buys enough precision. In practice you would not use $\hat\theta_3$ because the condition depends on the $\mu$ you do not know — but the arithmetic shows why "unbiased" is not automatically "best."

    11 · module 9

    You record the number of calls to a help desk in each of $n=8$ one-hour windows: 3, 5, 2, 4, 6, 3, 4, 5. Assume each count is Poisson.

    1. What is the sufficient statistic for $\lambda$, and its observed value?
    2. What is its sampling distribution?
    3. Derive the MLE for $\lambda$ and evaluate it.
    Show solution

    (a) For a Poisson, the sufficient statistic for $\lambda$ is the total:

    $$X_+ = \sum_{i=1}^{8}x_i = 3+5+2+4+6+3+4+5 = 32$$

    (b) From the master table, $X_+\sim\mathrm{Pois}(n\lambda)$:

    X₊ = 32,   X₊ ~ Pois(8λ)

    (c) Run the five steps.

    $$\mathcal{L}(\lambda\mid\mathbf{x}) = \prod_{i=1}^{n}\frac{\lambda^{x_i}e^{-\lambda}}{x_i!} = \frac{\lambda^{\sum x_i}e^{-n\lambda}}{\prod x_i!}$$

    $$\ell = \left(\sum x_i\right)\ln\lambda - n\lambda - \ln\left(\prod x_i!\right)$$

    The last term carries no $\lambda$, so it dies on differentiation:

    $$\frac{d\ell}{d\lambda} = \frac{\sum x_i}{\lambda} - n = 0 \;\Longrightarrow\; \hat\lambda_{MLE} = \frac1n\sum x_i = \bar X$$

    $$\hat\lambda = \frac{32}{8}$$

    λ̂MLE = 4 calls per hour

    12 · module 10

    The time to complete a task has an unknown distribution with $\mu = 45$ seconds and $\sigma = 12$ seconds. You time $n=64$ tasks.

    1. What is the sampling distribution of $\bar X$? Justify why you may use it.
    2. Find $\mathbb{P}(\bar X > 48)$.
    3. How many tasks would you need to time to get the standard error down to 1 second?
    Show solution

    (a) The distribution of a single task time is unknown, so the master table does not apply. But the data are i.i.d., $n = 64 \ge 30$, and you are inferring the mean — so the CLT applies.

    $$\mathrm{SE}(\bar X) = \frac{\sigma}{\sqrt n} = \frac{12}{8} = 1.5$$

    X̄ ˙~ Normal(45, 1.5² = 2.25)

    (b) $48 = 45 + 2(1.5) = \mu + 2\,\mathrm{SE}$, so this is the upper $2\sigma$ tail of the sampling distribution:

    $$\mathbb{P}(\bar X > 48) = \frac{1-0.954}{2}$$

    ≈ 0.023

    (c) $$\frac{12}{\sqrt n} \le 1 \;\Longrightarrow\; \sqrt n \ge 12 \;\Longrightarrow\; n \ge 144$$

    n = 144

    Going from $\mathrm{SE}=1.5$ to $\mathrm{SE}=1$ costs you more than twice the data. That is the $\sqrt n$ at work.


    The master trap list

    Read this on the way into the exam
    1. Assuming independence you were not given. Two marginal probabilities do not determine an intersection.
    2. Disjoint is not independent. Disjoint events are maximally dependent.
    3. Flipping the conditional bar. Underline what you were told and put it on the right of the bar.
    4. $\mathbb{E}(X^2)\ne(\mathbb{E}(X))^2$. Their difference is the variance.
    5. $\mathbb{V}(aX+b)=a^2\mathbb{V}(X)$, and the $b$ vanishes.
    6. Variances only add for independent terms. Expectations always add.
    7. A density is not a probability. Integrate it.
    8. $<$ vs $\le$ matters for p.m.f.s and never for p.d.f.s.
    9. Convert a rate into $\lambda$ by multiplying by the window size.
    10. Poisson $\mathbb{V}=\lambda$; exponential $\mathbb{V}=\theta^2$. Do not swap them.
    11. One normal tail is half the leftover. Always halve.
    12. $\sigma^2=4$ means $\sigma=2$. Check which one the question handed you.
    13. Log before differentiating, every single time.
    14. Data are constants inside an MLE derivative. $\sum x_i$ does not vary with $\theta$.
    15. Use invariance instead of re-deriving.
    16. $\mathrm{MSE}=B^2+\mathbb{V}$, with the bias squared.
    17. $\mathbb{V}(\bar X)=\sigma^2/n$ — the sampling distribution is far narrower than the data.
    18. $n$ vs $k$ in $\mathrm{Binom}(nk,p)$.
    19. Exact beats approximate. Check the master table before reaching for the CLT.
    20. Take the square root when asked for a standard deviation.

    The final hour

    Be able to write these from memory

    • The additive law and the complement rule
    • $\mathbb{P}(A\mid B)$, L.o.T.P., Bayes
    • $\mathbb{V}(X)=\mathbb{E}(X^2)-(\mathbb{E}(X))^2$
    • The four p.m.f.s / p.d.f.s and their $\mathbb{E}$ and $\mathbb{V}$
    • $F_X(x)=1-e^{-x/\theta}$ for the exponential
    • 68 / 95 / 99.7, and 1.645 / 1.960 / 2.576
    • The five MLE steps
    • $\mathbb{E}(\bar X)=\mu$, $\mathbb{V}(\bar X)=\sigma^2/n$
    • $\mathrm{MSE}=B^2+\mathbb{V}$
    • The CLT and its three conditions

    Be able to do these cold

    • Fill a 2×2 table from partial information
    • Draw a decision tree and run Bayes off it
    • Normalize a p.d.f. by solving for $c$
    • Compute $\mathbb{E}$, $\mathbb{E}(X^2)$, $\mathbb{V}$, $\sigma$ from a p.m.f.
    • Recognize which of the four distributions a story describes
    • Turn "at least one" into $1-\mathbb{P}(\text{none})$
    • Derive an MLE for a p.d.f. you have never seen
    • Name a sufficient statistic and its sampling distribution
    • Decide whether to use the table or the CLT
    During the exam
    1. Name the random variable first. "Let $X$ = the number of defectives in 12 parts." Half of all errors are really translation errors.
    2. Write down what you were given in symbols before computing anything. It shows work and it catches flipped conditionals.
    3. Check the sign and the range. A probability above 1, a negative variance, or a $\hat\theta$ that violates $\theta>0$ means go back.
    4. Verify totals. A p.m.f. sums to 1. The four cells of a 2×2 table sum to 1. The leaves of a tree sum to 1.
    5. Show the setup even when you cannot finish. On a no-code exam, correctly writing $F_{\bar X}(101)-F_{\bar X}(99)$ with the right parameters is the answer.
    13

    30 minutes · extra practice

    Extra practice: 2×2 tables and Bayes

    Two more problems of each type from the first two exam-simulation questions. Each type opens with the method as a checklist, then two problems with every step written out.


    Type 1 — the 2×2 table

    How to recognize it: two events, and the question asks about "at least one," "exactly one," "both," "neither," or whether they're independent.

    The method
    1. Name the two events with letters. Write down every probability you're given, in symbols.
    2. Draw the empty grid. Columns $A$ and $A^c$, rows $B$ and $B^c$, with a totals row and column. The bottom-right corner is always 1.
    3. Write in what you know. $\mathbb{P}(A)$ goes at the bottom of the $A$ column. "Both" goes top-left. "Neither" goes bottom-right.
    4. Fill the rest by subtraction. Every row adds to its row total. Every column adds to its column total. Keep going until all four cells are full.
    5. Read off the answer using the guide below.
    $A$$A^c$total
    $B$bothonly $B$$\mathbb{P}(B)$
    $B^c$only $A$neither$\mathbb{P}(B^c)$
    total$\mathbb{P}(A)$$\mathbb{P}(A^c)$1
    The question saysAnswer
    "both," "A and B"the top-left cell
    "neither"the bottom-right cell
    "exactly one," "one but not both"add the other two cells: only $A$ + only $B$
    "at least one," "A or B"everything except neither: $1 - \text{neither}$
    "B given A," "among those with A"top-left ÷ the $A$ column total
    "are they independent?"check whether $\mathbb{P}(A)\times\mathbb{P}(B)$ equals the top-left cell

    13.1 · 2×2 table

    In a class, 45% of students take calculus, 30% take statistics, and 20% take both.

    1. What fraction take at least one of the two?
    2. Exactly one?
    3. Neither?
    4. Among students who take calculus, what fraction also take statistics?
    5. Are taking calculus and taking statistics independent?
    Show solution

    Step 1 — name the events. $C$ = takes calculus, $S$ = takes statistics.

    $$\mathbb{P}(C) = 0.45 \qquad \mathbb{P}(S) = 0.30 \qquad \mathbb{P}(C\cap S) = 0.20$$

    Step 2 — write in what you know. "Both" = 0.20 goes top-left. $\mathbb{P}(C)$ = 0.45 goes at the bottom of the $C$ column. $\mathbb{P}(S)$ = 0.30 goes at the end of the $S$ row.

    Step 3 — fill by subtraction, one cell at a time:

    • Only $S$: the $S$ row totals 0.30, and 0.20 is already used. $\;0.30 - 0.20 = 0.10$
    • Only $C$: the $C$ column totals 0.45, and 0.20 is already used. $\;0.45 - 0.20 = 0.25$
    • $C^c$ column total: $\;1 - 0.45 = 0.55$
    • Neither: the $C^c$ column totals 0.55, and 0.10 is already used. $\;0.55 - 0.10 = 0.45$
    $C$$C^c$total
    $S$0.200.100.30
    $S^c$0.250.450.70
    total0.450.551

    Check: $0.20 + 0.10 + 0.25 + 0.45 = 1.00$ ✓

    Step 4 — read off each answer.

    (a) At least one = everything except neither: $\;1 - 0.45$

    0.55

    (b) Exactly one = only $C$ + only $S$: $\;0.25 + 0.10$

    0.35

    (c) Neither = bottom-right cell.

    0.45

    (d) "Among students who take calculus" means shrink the world to the $C$ column. That column totals 0.45. The part of it that also takes statistics is the top-left cell, 0.20.

    $$\mathbb{P}(S\mid C) = \frac{\mathbb{P}(C\cap S)}{\mathbb{P}(C)} = \frac{0.20}{0.45}$$

    4/9 ≈ 0.444

    (e) Multiply the two totals and compare with the top-left cell:

    $$\mathbb{P}(C)\times\mathbb{P}(S) = 0.45\times 0.30 = 0.135 \qquad\text{but}\qquad \mathbb{P}(C\cap S) = 0.20$$

    Not independent

    Part (d) says the same thing a second way: 30% of all students take statistics, but 44% of calculus students do. Knowing someone takes calculus changes the odds, so the events are linked.

    13.2 · 2×2 table, working backwards

    A machine has two components, A and B. The probability component A fails is 0.15. The probability that neither fails is 0.78. The probability that both fail is 0.05.

    1. What is the probability at least one fails?
    2. What is the probability component B fails?
    3. What is the probability exactly one fails?
    4. Are the two failures independent?
    Show solution

    What makes this one harder: you are not given $\mathbb{P}(B)$. You get "neither" and "both" instead, and have to work out the rest.

    Step 1 — name the events. $A$ = component A fails, $B$ = component B fails.

    $$\mathbb{P}(A) = 0.15 \qquad \mathbb{P}(A^c\cap B^c) = 0.78 \qquad \mathbb{P}(A\cap B) = 0.05$$

    Step 2 — write in what you know. "Both" = 0.05 goes top-left. "Neither" = 0.78 goes bottom-right. $\mathbb{P}(A)$ = 0.15 goes at the bottom of the $A$ column.

    Step 3 — fill by subtraction. Start with the $A$ column, because you know its total:

    • Only $A$: the $A$ column totals 0.15, and 0.05 is used. $\;0.15 - 0.05 = 0.10$
    • $A^c$ column total: $\;1 - 0.15 = 0.85$
    • Only $B$: the $A^c$ column totals 0.85, and 0.78 is used. $\;0.85 - 0.78 = 0.07$
    • $B$ row total: add across the top row. $\;0.05 + 0.07 = 0.12$
    • $B^c$ row total: add across the bottom row. $\;0.10 + 0.78 = 0.88$
    $A$ fails$A$ workstotal
    $B$ fails0.050.070.12
    $B$ works0.100.780.88
    total0.150.851

    Check: $0.05 + 0.07 + 0.10 + 0.78 = 1.00$ ✓

    Step 4 — read off each answer.

    (a) At least one = everything except neither: $\;1 - 0.78$

    0.22

    (b) The total of the $B$ row, which you just built.

    0.12

    (c) Exactly one = only $A$ + only $B$: $\;0.10 + 0.07$

    0.17

    (d) $$\mathbb{P}(A)\times\mathbb{P}(B) = 0.15\times 0.12 = 0.018 \qquad\text{but}\qquad \mathbb{P}(A\cap B) = 0.05$$

    Not independent

    Both components fail together almost three times as often as independence would predict. In a real machine that points to a shared cause, like a power surge that takes out both.


    Type 2 — Bayes' rule

    How to recognize it: the problem tells you "if X, then Y happens this often," and then asks the reverse: "given Y happened, how likely is X?"

    The method
    1. Name the events. One is the cause (defective, has the condition, which machine). The other is the evidence (flagged, tests positive, is defective).
    2. Translate every sentence into symbols. "If it's defective, the sensor flags it 95% of the time" becomes $\mathbb{P}(\text{flag}\mid\text{defective}) = 0.95$. Whatever follows "if" goes after the bar.
    3. Build the Bayes table below. One row per possible cause.
    4. Read off the answer. The total of the product column is the unconditional probability. Each product divided by that total is the Bayes answer.

    The Bayes table does the law of total probability and Bayes' rule in one grid. It works for two causes or ten.

    CausePrior× Likelihood= ProductProduct ÷ total
    what could be true$\mathbb{P}(\text{cause})$$\mathbb{P}(\text{evidence}\mid\text{cause})$$\mathbb{P}(\text{cause}\cap\text{evidence})$$\mathbb{P}(\text{cause}\mid\text{evidence})$
    total1$\mathbb{P}(\text{evidence})$1
    What each column is
    • The Product column is the multiplicative rule. It answers "both" questions.
    • The total of the Product column is the law of total probability. It answers "what fraction overall" questions.
    • The last column is Bayes' rule. It answers "given" questions.

    13.3 · Bayes, two causes

    A condition affects 1% of people. A screening test catches it 90% of the time when it's present, and falsely flags 5% of people who don't have it.

    1. What fraction of all people test positive?
    2. Someone tests positive. What's the probability they actually have the condition?
    3. Someone tests negative. What's the probability they have the condition anyway?
    Show solution

    Step 1 — name the events. $D$ = has the condition (the cause). $T$ = tests positive (the evidence), so $T^c$ = tests negative.

    Step 2 — translate each sentence.

    The problem saysIn symbols
    "affects 1% of people"$\mathbb{P}(D) = 0.01$
    "catches it 90% of the time when it's present"$\mathbb{P}(T\mid D) = 0.90$
    "falsely flags 5% of people who don't have it"$\mathbb{P}(T\mid D^c) = 0.05$

    Step 3 — the Bayes table, with "tests positive" as the evidence:

    CausePrior× $\mathbb{P}(T\mid\text{cause})$= Product
    has it, $D$0.010.900.0090
    doesn't, $D^c$0.990.050.0495
    total10.0585

    The 0.99 comes from $\mathbb{P}(D^c) = 1 - 0.01$.

    (a) The total of the product column.

    P(T) = 0.0585

    (b) Your row's product divided by the total:

    $$\mathbb{P}(D\mid T) = \frac{0.0090}{0.0585}$$

    ≈ 0.154

    Why so low? Count 10,000 people
    • 100 have the condition. The test catches 90 of them.
    • 9,900 don't. The test falsely flags 5% of them: 495 people.
    • So 90 + 495 = 585 people test positive, and only 90 of them actually have it.

    $90 / 585 = 0.154$. The false alarms outnumber the real cases because so few people have the condition to begin with. This is the same pattern as the defective-chip problem in module 12.

    (c) The evidence is now "tests negative," so you need new likelihoods. Each one is a complement within the same condition:

    $$\mathbb{P}(T^c\mid D) = 1 - \mathbb{P}(T\mid D) = 1 - 0.90 = 0.10$$

    $$\mathbb{P}(T^c\mid D^c) = 1 - \mathbb{P}(T\mid D^c) = 1 - 0.05 = 0.95$$

    CausePrior× $\mathbb{P}(T^c\mid\text{cause})$= Product
    has it, $D$0.010.100.0010
    doesn't, $D^c$0.990.950.9405
    total10.9415

    $$\mathbb{P}(D\mid T^c) = \frac{0.0010}{0.9415}$$

    ≈ 0.001

    Check: $\mathbb{P}(T^c) = 0.9415 = 1 - 0.0585$ ✓. A negative result is very reassuring (about 1 in 1,000), even though a positive result was not very alarming.

    The trap in (c): $\mathbb{P}(T^c\mid D)$ is $1 - \mathbb{P}(T\mid D)$. The condition $D$ stays the same and only the part before the bar flips. It is not $1 - \mathbb{P}(T\mid D^c)$.

    13.4 · Bayes, three causes

    A factory's parts come from three machines. Machine 1 makes 50% of the parts, machine 2 makes 30%, and machine 3 makes 20%. Their defect rates are 1%, 3%, and 6%.

    1. What fraction of all parts are defective?
    2. A part is defective. What's the probability it came from machine 3?
    3. What's the probability a randomly chosen part is both from machine 3 and defective?
    4. A part is defective. Which machine most likely made it?
    Show solution

    Step 1 — name the events. $M_1, M_2, M_3$ = which machine made the part (the causes). $D$ = the part is defective (the evidence).

    Step 2 — translate. "Machine 1 makes 50%" gives $\mathbb{P}(M_1) = 0.50$. "A defect rate of 1%" for machine 1 means if it came from machine 1, it's defective 1% of the time: $\mathbb{P}(D\mid M_1) = 0.01$.

    Step 3 — the Bayes table. Three causes means three rows:

    MachinePrior× $\mathbb{P}(D\mid M)$= Product÷ total
    10.500.010.0050.192
    20.300.030.0090.346
    30.200.060.0120.462
    total1.000.0261.000

    The last column adds to 1 ✓ — a defective part came from some machine.

    (a) The total of the product column. This is the law of total probability with three branches:

    $$\mathbb{P}(D) = (0.50)(0.01) + (0.30)(0.03) + (0.20)(0.06) = 0.005 + 0.009 + 0.012$$

    0.026

    (b) Machine 3's product divided by the total:

    $$\mathbb{P}(M_3\mid D) = \frac{0.012}{0.026}$$

    ≈ 0.462

    (c) "Both" is an intersection, not a conditional. Read machine 3's product and don't divide:

    $$\mathbb{P}(M_3\cap D) = \mathbb{P}(D\mid M_3)\,\mathbb{P}(M_3) = (0.06)(0.20)$$

    0.012

    Compare (b) and (c). Same two events, very different answers. (b) says "given it's defective," so you shrink the world to defective parts and divide by 0.026. (c) says "both," so you stay in the whole world and don't divide.

    (d) The largest value in the last column.

    Machine 3 (46%)

    Machine 3 makes the fewest parts, but its defect rate is so high that it produces almost half of all defects. Machine 1 makes half the parts but only 19% of the defects. The answer depends on both the prior and the likelihood, which is why you multiply them.