$\sigma$-Algebra

We begin with some definitions:

Definition

Let $\lbrace A_n \rbrace$ be a sequence of sets, the set of all points $\omega \in \Omega$ (where $\Omega$ is the reference set) that belong to $A_n$ for infinitely many values of $n$ is known as the limit supremum of the sequence and is defined by

Similarly, the set of all points that belong to $A_n$ for all but a finite number of values of $n$ is known as the limit inferior of the sequence $\lbrace A_n \rbrace$ and is denoted by

If $\displaystyle{\lim_{n \rightarrow +\infty} \sup A_n = \lim_{n \rightarrow +\infty} \inf A_n}$, we say the limit exists and denote $\displaystyle{\lim_{n \rightarrow +\infty}A_n}$ to be its limit.\

Corollary

Given the definition of $\lim \sup$ and $\lim \inf$ last lecture, we now give some examples:

• Let $A_n = \lbrace n \rbrace, n \in \mathbb{N}$, then by definition we have

• Let $A_n = \lbrace (-1)^n \rbrace, n \in \mathbb{N}$, then by definition we have

Definition

• If $A_n \subset A_{n+1}, n \in \mathbb{N}$, we say $\lbrace A_n \rbrace$ is \textit{non-decreasing}, then $\lim_{n \rightarrow \infty} A_n = \bigcup_{n=1}^{\infty} A_n $.

• If $A_{n+1} \subset A_n, n \in \mathbb{N}$, we say $\lbrace A_n \rbrace$ is \textit{non-increasing}, then $\lim_{n \rightarrow \infty} A_n = \bigcap_{n=1}^{\infty} A_n$.

Definition

Let $X$ be a space (i.e a non-empty set) and $\mathcal{F}$ be a collection of subsets of $X$ (here $\mathcal{F}$ is the collection of subsets of $X$ which we are going to measure). $\mathcal{F}$ is called a $\sigma$-algebra of subsets of $X$ if:

• $X \in \mathcal{F}$

• If $A \in \mathcal{F}$, then $A^C := X \backslash A \in \mathcal{F}$. (Closed Under Taking Complement)

• If a series of subsets $\lbrace A_n, n \geq 1 \rbrace \in \mathcal{F}$, then $\displaystyle{\bigcup_{n=1}^{\infty}} A_n \in \mathcal{F}$. (Closed Under Countable Union)

Based on the definition, the following propositions also hold.

Corollary

The definition of $\sigma$-algebra leads to:

• $\varnothing \in \mathcal{F}$

• $X \in \mathcal{F}$

• If a series of subsets $\lbrace A_n , n \geq 1 \rbrace \in \mathcal{F}$, then $\displaystyle{\bigcap_{n=1}^{\infty} }A_n \in \mathcal{F}$. (Closed Under Countable Intersection)

• If $A_1, A_2, \cdots, A_N \in \mathcal{F}$, then $\displaystyle{\bigcap_{n=1}^{N} }A_n \in \mathcal{F}$ and $\displaystyle{\bigcup_{n=1}^{N} }A_n \in \mathcal{F}$. (Closed Under Taking Finite Union or Intersection)

• If $A,B \in \mathcal{F}$, then $A \backslash B, B \backslash A \in \mathcal{F}$

We say $\mathcal{F}_1 $ is bigger than $\mathcal{F}_2$, if $\mathcal{F}_1 \subset \mathcal{F}_2$. In this case, for any space $X$, we may conclude that $\lbrace \varnothing, X \rbrace $ is the smallest $\sigma$-algebra and $2^X$ is the biggest $\sigma$-algebra. A $\sigma$-algebra can also be generated by a collection of sets.

Definition

Let $X$ be a space and $\mathcal{C}$ be a collection of subsets of $X$, then the $\sigma$-algebra generated by $\mathcal{C}$, denoted by $\sigma(\mathcal{C})$, is such that

• $\sigma(\mathcal{C}$) is itself a $\sigma$-algebra with $\mathcal{C}\subset \sigma(\mathcal{C})$

• If $\mathcal{F}’$ is a $\sigma$-algebra with $\mathcal{C} \subset \mathcal{F}’$, then $\sigma(\mathcal{C}) \subset \mathcal{F}’$. i.e $\sigma(\mathcal{C})$ is the smallest $\sigma$-algebra that is a super set of $\mathcal{C}$.

Again, we have the following propositions:

Corollary

• $\sigma(\mathcal{C}) = \bigcap { \mathcal{F} : \mathcal{C} \subset \mathcal{F} }$ for all $\sigma$-algebra $\mathcal{F}$

• If $\mathcal{C}$ itself is a $\sigma$-algebra, then $\sigma(\mathcal{C}) = \mathcal{C}$

• If $\mathcal{C}_1, \mathcal{C}_2$ are 2 collections of subsets of $X$ with $\mathcal{C}_1 \subset \mathcal{C}_2$, then $\sigma(\mathcal{C}_1) \subset \sigma(\mathcal{C}_2) $

Borel $\sigma$-Algebra

An important example of $\sigma$-algebra on $\mathbb{R}$ (of subsets of $\mathbb{R})$ is called a Borel $\sigma$-Algebra.

Definition

The Borel $\sigma$-algebra of $\mathbb{R}$ is defined by

Also we need to know that the generator of $\mathfrak{B}{\mathbb{R}}$ is not unique. By standard definition, we give

But we still have the following proposition:

Corollary

I shall illustrate why those are indeed equal. Let’s just take a look at \textcircled{2}, we need to show

We could indeed prove $L.H.S \subset R.H.S$ and $R.H.S \subset L.H.S$.

$\bullet$ To show that $L.H.S \subset R.H.S$, we need to show

and we have

$\bullet$ Similarly,

And that’s why they are indeed equal.

Definition

A set $G$ is called a Borel Set if $G \in \mathfrak{B}_{\mathbb{R}}$.

In fact, any set produced by countable operations are also Borel Sets.

Corollary

A set with a single element (or singletons) $\lbrace x \rbrace$ is also a Borel Set.

Proof

For a singleton $x \in \mathbb{R}$, we have

QED

Measures

Given a space $X$ and a $\sigma$-algebra $\mathcal{F}$ of subsets of $X$, $(X,\mathcal{F})$ is called a measurable space.

Definition

Given a measure space $(X,\mathcal{F})$, define

is a non-negative set function, then $\mu$ is a measure if

• $\mu \lbrace \varnothing \rbrace = 0$;

• If $\lbrace A_n, n \geq 1 \rbrace \subset \mathcal{F}$ such that $A_n$’s are (pairwise) disjoint, then

this is known as countable additivity.

We say that $\mu$ is finite if $\mu(X) < +\infty$, we say that $\mu$ is a \textit{probability measure} if $\mu(X) = 1$.

We say $\mu$ is \textit{$\sigma$-finite} if $\exists \lbrace A_n , n \geq 1\rbrace \subset \mathcal{F}$ such that $\displaystyle{X = \bigcup_{n=1}^{\infty} A_n}$ with $\mu(A_n) < +\infty$.

We call the triple $(X,\mathcal{F},\mu)$ a measure space.

Here are a few examples of measures on $(\mathbb{R},\mathfrak{B}_{\mathbb{R}})$:

• A measure $\mu_1$ given by

We say $\mu_1$ is the counting measure.

• Let $x_0 \in \mathbb{R}$, a measure $\mu_2$ is such that

We say $\mu_2$ is the probability measure.

We now give some propositions for the measure space. Below, let $(X,\mathcal{F}, \mu)$ be a measure space.

Corollary (Finite Additivity)

If $A_1,A_2,\cdots,A_N \in \mathcal{F}$ are disjoint, then

Corollary (Monotoncity)

Given $A,B \in \mathcal{F}$, if $A \subset B$, then $\mu(A) \leq \mu(B)$.

Proof Note that since $\mu(A),\mu(B)$ could both be infinite, so taking $\mu(B)-\mu(A)$ is incorrect. Indeed, we may construct $B = A \bigcup (B \backslash A)$, where

QED

Corollary (Countable / Finite Subadditivity)

If $\lbrace A_n, n \geq 1 \rbrace \subset \mathcal{F}$, then

Note that if $A_n$’s are disjoint, then it should be “$=$”.

Proof Let $B_1 = A_1$, and $B_n := A_n \backslash \left( \bigcup_{i=1}^{n-1} A_i \right)$ for every $n \geq 2$, then we know that $\lbrace B_n, n \geq 1 \rbrace \subset \mathcal{F}$ and $B_n$’s are disjoint. More importantly, by our construction we have $\bigcup_{n=1}^{\infty} B_n = \bigcup_{n=1}^{\infty} A_n$. Thus by countable additivity, we have

QED

Corollary (Coutinuity From Below)

Given $\lbrace A_n , n\geq 1\rbrace \subset \mathcal{F}$ such that $A_n \subset A_{n+1}$ for every $n$, then

Proof Set $B_1 = A_1$, $B_n = A_n \backslash A_{n-1}$ for all $n \geq 2$. By definition we have $\lbrace B_n, n \geq 1 \rbrace \subset \mathcal{F}$ is disjoint, and we have $\bigcup_{n=1}^{\infty} B_n = \bigcup_{n=1}^{\infty} A_n$; also for each $N \geq 1$, $\bigcup_{n=1}^N B_n = A_n$, which means

QED

Corollary (Continuity From Above)

Given $\lbrace A_n, n \geq 1 \rbrace \in \mathcal{F}$ such that $A_{n+1} \subset A_n$ for every $n \geq 1$, if $\mu(A_1) < +\infty$, then

Proof Set $B_n = A_1 \backslash A_n$, then we know that the sequence $\lbrace B_n , n\geq 1 \rbrace$ is increasing, then we have

\(\bigcup_{n=1}^{\infty} B_n = A_1 \backslash \bigcap_{n=1}^{\infty} A_n\) Thus,

Given that the measure of $A_n$ is always finite, so $\mu(A_1 \backslash A_n ) = \mu(A_1) - \mu(A_n)$ and

Thus, by equation $(*)$,

\(\mu(A_1) - \mu \left( \bigcap_{n=1}^{\infty} A_n \right) = \lim_{n \rightarrow \infty} \left(\mu(A_1) - \mu(A_n)\right)\) that is

QED

The reason why we would introduce the definition of a $\sigma$-algebra, measure is that, probability, is also a measure.

An introduction to sample spaces and probability axioms

Definition

A random (or a statistical) experiment is an experiment with the following:

• All outcomes of the experiment are known in advance;

• Any performance of the experiment results in an outcome is not known i advance;

• The experiment can be repeated under identical conditions.

Definition

The sample space of a statistical experiment is a pair $(\Omega, \mathcal{F})$ where $\Omega$ is the set of all possible outcomes of the experiment and $\mathcal{F}$ is a $\sigma$ field of the subsets of $\Omega$.

Here is an example to illustrate this: Suppose we are tossing a coin, and we denote $H$ (head) and $T$ (tail) to be the only two possible outcomes. Then we have $\Omega := \lbrace H,T \rbrace$ and $\mathcal{F} := \lbrace { H } , { T }, { H,T } , \varnothing \rbrace$. Element of $\Omega$ are called the sample points and any set $A \in \mathcal{F}$ is called an event.

Definition (Kolmogorov’s Axioms)

Let $(\Omega,\mathcal{F})$ be a sample space and a set function $\mathbb{P} : \mathcal{F} \longrightarrow \mathbb{R}$ is called a probability measure (or simply probability) if it satisfies:

• $\mathbb{P}(A) \geq 0$ for all $A \in \mathcal{F}$;

• $\mathbb{P}(\Omega) = 1$;

• Let ${ A_j }$ be a sequence of disjoint sets where $A_j \in \mathcal{F}$, then $\displaystyle{\mathbb{P} \left( \bigcup_{i=1}^{\infty} A_i \right) = \sum_{i =1}^{\infty} \mathbb{P}(A_i)}$.

Corollary

$\mathbb{P}$ is monotone and substrative, i.e if $A,B \in \mathcal{F}$ with $A \subset B$, then $\mathbb{P}(A) \leq \mathbb{P}(B)$.

Corollary

If $A \in \mathcal{F}$, then $\mathbb{P}(A) = 1 - \mathbb{P}(A^C)$.

Now we will introduce an important theorem in probability:

Theorem

(The principle of inclusion and exclusion)

Let $A_1,A_2,\cdots,A_n \in \mathcal{F}$, then

Proof We will first consider some special cases, let’s say $n=2$, then we have

Similarly, consider $n=3$, then we have

So when $n=2$ or $n=3$, it is already proven according to the construction we did using Venn diagram. Now we perform the induction on $n$, assume it holds for some number $N (N>3)$, then say

Denote $\left[ \bigcup_{i=1}^N A_i \right] = B$, then we have

QED

The principle of inclusion and exculsion also leads us to another theorem:

Theorem (Bonferroni’s Inequality)

Given $n$ events $A_1,A_2,\cdots,A_n \in \mathcal{F}, (n>1)$, then

Proof The proof is basically the same idea as above, reader should try to prove this by their own.

QED

Theorem (Boole’s Inequality)

Suppose $A,B \in \mathcal{F}$, then

Proof

QED

Corollary

Let ${ A_j }_{j=1}^{\infty}$ be a sequence of events, then

Proof This proof can be achieved by induction, using the results in theorem 4.

QED

Theorem (The implicative rule)

If $A_1,A_2,A_3 \in \mathcal{F}$ and $A_1 \cap A_2 \subset A_3$, i.e $A_1,A_2$ implies $A_3$, then $\mathbb{P}(A_3^C) \leq \mathbb{P}(A_1^C) + \mathbb{P}(A_2^C)$.

Proof By monotoncity of probability, we have $\mathbb{P}(A_1 \cap A_2) \leq \mathbb{P}(A_3)$, i.e $\mathbb{P}((A_1 \cap A_2)^C) \geq \mathbb{P}(A_3^C)$, that is $\mathbb{P}(A_1^C \cap A_2^C) \geq \mathbb{P}(A_3^C)$, by inclusion and exculsion formula, we have $\mathbb{P}(A_1^C) + \mathbb{P}(A_2^C) - P(A_1^C \cap A_2^C) \geq \mathbb{P}(A_3^C)$, given that the probability is always non-negative, we then conclude that $\mathbb{P}(A_3^C) \leq \mathbb{P}(A_1^C) + \mathbb{P}(A_2^C)$.

QED

Theorem (Continuity of $\mathbb{P}$)

Let ${ A_n }$ be a sequence of non-decreasing events in $\mathcal{F}$, i.e $A_n \subset A_{n+1}$, then

Proof Let $A = \bigcup_{j=1}^{\infty} A_j$, then $A$ can be written as $A = A_n \bigcup \left( \bigcup_{j=n}^{\infty} A_{j+1} \backslash A_j \right)$, by $\sigma$ additivity, we also have

Now taking $n \to \infty$:

QED

Theorem (Continuity of $\mathbb{P}$)

Let ${ A_n }$ be a sequence of non-increasing events in $\mathcal{F}$, i.e $A_{n+1} \subset A_n$, then

Proof This can be done by taking $B_n = A_n^C$.

QED

Recommended citation: Jiajun Zhang, (2024) Basic Measure Theory and Probability Axioms