Probability Theory — Comprehensive Study Notes Summary & Study Notes

📘 Overview

Probability theory studies uncertainty and long-run frequencies of outcomes from a random experiment. A random experiment satisfies: (i) all distinct outcomes are known ahead of time, (ii) an individual trial's outcome is not known a priori, and (iii) the experiment can be repeated in principle. The set of all possible outcomes is the sample space $S$; each outcome is a sample point or elementary event.

📐 Sample Space, Events, and Set Operations

An event is any subset of $S$. Important set operations and notations:

• Complement: $A^{c}=S\setminus A$ — event that $A$ does not occur.
• Union: $A\cup B$ — event that $A$ or $B$ (or both) occur.
• Intersection: $A\cap B$ — event that both $A$ and $B$ occur.
• Disjoint / Mutually exclusive: $A\cap B=\varnothing$.

Key identities (De Morgan and algebraic laws):

• $(A\cup B)^{c}=A^{c}\cap B^{c}$ and $(A\cap B)^{c}=A^{c}\cup B^{c}$.
• Associative, commutative, distributive laws hold for unions/intersections.

🧾 s-algebras and Measurable Spaces

An s-algebra (sigma-field) $\mathcal{F}$ is a collection of subsets of $S$ closed under complementation and countable unions. A measurable space is the pair $(S,\mathcal{F})$. A measure $m$ on $(S,\mathcal{F})$ is a nonnegative, countably additive set function. A probability measure $P$ satisfies $P(S)=1$ and $P:\mathcal{F}\to[0,1]$.

🎯 Definitions of Probability

• Classical (equiprobable): If there are $n$ equally likely outcomes and $n_A$ favorable to event $A$, then $Pr(A)=\frac{n_A}{n}$.
• Relative frequency (frequentist): If $r_n(A)$ occurrences of $A$ in $n$ trials and the limit $\lim_{n\to\infty} \frac{r_n(A)}{n}=P$ exists, then $P$ is the probability of $A$.
• Subjective: Probability expresses a degree of belief; used in Bayesian inference to update prior beliefs.

⚖️ Axiomatic Approach (Kolmogorov axioms)

Let $P$ be a function on $\mathcal{F}$.

• Axiom 1: $P(A)\ge 0$ for all $A\in\mathcal{F}$.
• Axiom 2: $P(S)=1$.
• Axiom 3: For countable disjoint $A_i$, $P\big(\bigcup_{i} A_i\big)=\sum_i P(A_i)$.

Consequences and useful results:

• $P(\varnothing)=0$.
• $0\le P(A)\le 1$ for any $A$.
• Monotonicity: If $A\subseteq B$ then $P(A)\le P(B)$.
• Complement rule: $P(A^{c})=1-P(A)$.
• Addition rule: $P(A\cup B)=P(A)+P(B)-P(A\cap B)$; extends to inclusion–exclusion for more events.

🔢 Counting Techniques (for finite problems)

• Rule of product: If one operation has $n_1$ choices and another $n_2$, both together have $n_1n_2$ choices.
• Permutation of $n$ distinct objects: $n!$.
• Permutation (r from n): $P(n,r)=\frac{n!}{(n-r)!}$.
• Combination (order ignored): $\displaystyle {n\choose r}=\frac{n!}{r!(n-r)!}$. These are essential for enumerating sample points when outcomes are finite and equally likely.

🔁 Conditional Probability and Multiplication Rule

• Conditional probability: $Pr(A\mid B)=\frac{Pr(A\cap B)}{Pr(B)}$ for $Pr(B)>0$.
• Multiplication rule for two events: $Pr(A\cap B)=Pr(A),Pr(B\mid A)=Pr(B),Pr(A\mid B)$.
• Extended to $k$ events: $Pr(A_1\cap\cdots\cap A_k)=Pr(A_1)Pr(A_2\mid A_1)Pr(A_3\mid A_1\cap A_2)\cdots$.

Useful conditional identities:

• Law of total probability: If ${B_i}$ is a partition, $Pr(A)=\sum_i Pr(B_i)Pr(A\mid B_i)$.
• Bayes' theorem: $Pr(B_j\mid A)=\dfrac{Pr(B_j)Pr(A\mid B_j)}{\sum_i Pr(B_i)Pr(A\mid B_i)}$.

🔗 Independence

• Two events $A$ and $B$ are independent iff $Pr(A\cap B)=Pr(A)Pr(B)$ (equivalently $Pr(A\mid B)=Pr(A)$ when $Pr(B)>0$).
• Pairwise independence does not imply mutual independence for three or more events. For mutual independence of $A,B,C$ we require: $Pr(A\cap B)=Pr(A)Pr(B)$, $Pr(A\cap C)=Pr(A)Pr(C)$, $Pr(B\cap C)=Pr(B)Pr(C)$, and $Pr(A\cap B\cap C)=Pr(A)Pr(B)Pr(C)$.

➕ Addition and Inclusion–Exclusion

• For two events: $Pr(A\cup B)=Pr(A)+Pr(B)-Pr(A\cap B)$.
• For three events: $Pr(A\cup B\cup C)=Pr(A)+Pr(B)+Pr(C)-Pr(A\cap B)-Pr(A\cap C)-Pr(B\cap C)+Pr(A\cap B\cap C)$.

✅ Practical Examples (summary of approaches)

• Divisibility or counting problems: enumerate sample space with counting rules and apply classical ratio $\frac{\text{favorable}}{\text{total}}$.
• Urn problems, drawing without replacement: use combinations or conditional probabilities reflecting changing counts.
• Repeated trials with stopping rules (e.g., coin toss until first head): model sample space as sequences and sum geometric-type probabilities.

🧾 Key Formulas (reference)

• Classical probability: $Pr(A)=\frac{n_A}{n}$.
• Conditional: $Pr(A\mid B)=\frac{Pr(A\cap B)}{Pr(B)}$.
• Multiplication: $Pr(A\cap B)=Pr(A)Pr(B\mid A)$.
• Bayes: $Pr(A\mid B)=\frac{Pr(A)Pr(B\mid A)}{Pr(B)}$.
• Permutations: $n!$, $P(n,r)=\frac{n!}{(n-r)!}$.
• Combinations: $\displaystyle {n\choose r}=\frac{n!}{r!(n-r)!}$.

These notes summarize the main concepts and tools from basic probability theory useful for exercises and economic applications: event algebra, axioms, conditional probability, independence, counting methods, and Bayes' rule.

📝 Note on User Request

The original user input requested a flashcard, but this response provides comprehensive study notes instead (as requested in the overall instruction). If you still want a single flashcard or a deck later, please ask.

This short note acknowledges the request and clarifies the delivered material.