Uniform Distribution

Definition A random variable $X$ is said to have a uniform distribution on the interval $[a,b]$, if its probability density function is given by

And we denote by $X \sim Uniform[a,b]$.

Based on that, we may derive the distribution function $F(x)$ to be

Corollary

Suppose $X \sim U[a,b]$, then for all positive integer $k$,

and

Proof

Left as an exercise to the reader.

QED

Furthermore, we can derive the moment generating function of $X \sim Uniform[a,b]$, we have

Exponential Distribution

The exponential distribution is one of the widely used continuous distributions. It is often used to model the time elapsed between events. We will now mathematically define the exponential distribution, and derive its mean and expected value. Then we will develop the intuition for the distribution and discuss several interesting properties that it has.

Definition

A continuous random variable $X$ is said to have an exponential distribution with parameter $\lambda >0$, shown as $X \sim Exponential(\lambda)$, if its PDF is given by

We may also derive the distribution function of $X \sim Exponential(\lambda)$:

We can also find its expected value using integration by parts:

Now let’s find $\mathbf{Var}(X)$. Since

Thus we obtain

Corollary

If $X \sim Exponential(\lambda)$, then

Proof

This follows trivially from the arguments above.

QED

To get some intuition for this interpretation of the exponential distribution, suppose you are waiting for an event to happen. For example, you are at a store and are waiting for the next customer. In each millisecond, the probability that a new customer enters the store is very small. You can imagine that, in each millisecond, a coin (with a very small $\mathbb{P}(H)$ ) is tossed, and if it lands heads a new customers enters. If you toss a coin every millisecond, the time until a new customer arrives approximately follows an exponential distribution.

The above interpretation of the exponential is useful in better understanding the properties of the exponential distribution. The most important of these properties is that the exponential distribution is memoryless. To see this, think of an exponential random variable in the sense of tossing a lot of coins until observing the first heads. If we toss the coin several times and do not observe a heads, from now on it is like we start all over again. In other words, the failed coin tosses do not impact the distribution of waiting time from now on. The reason for this is that the coin tosses are independent. We can state this formally as follows:

To see why this is true, we have

Normal (Gaussian) Distribution

The normal distribution is by far the most important probability distribution. One of the main reasons for that is the Central Limit Theorem (CLT) that we will discuss later. To give you an idea, the CLT states that if you add a large number of random variables, the distribution of the sum will be approximately normal under certain conditions. The importance of this result comes from the fact that many random variables in real life can be expressed as the sum of a large number of random variables and, by the CLT, we can argue that distribution of the sum should be normal. The CLT is one of the most important results in probability and we will discuss it later on. Here, we will introduce normal random variables.

We first define the standard normal random variable. We will then see that we can obtain other normal random variables by scaling and shifting a standard normal random variable.

Definition

A continuous random variable $Z$ is said to be a standard normal (standard Gaussian) random variable, shown as $Z \sim N(0,1)$, if its PDF is given by

And the total area under the curve is $1$. This is because of the fact that

Now we will study the expected value and the variance of a standard normal variable.

Corollary

If $Z \sim N(0,1)$, then $\mathbb{E} Z = 0, \mathbf{Var}(Z) = 1$.

Proof

Left as an exercise for the reader.

QED

To find the distribution function of the standard normal distribution, we need to integrate the PDF function. In particular we define

And based on that, here are some properties of the $\Phi$ function:

• $\lim_{x \to +\infty} \Phi(x) = 1, \lim_{x \to -\infty} \Phi(x) = 0$;

• $\Phi(0) = \displaystyle{\frac{1}{2}}$;

• $\Phi(-x) = 1 - \Phi(x), \text{for all $x \in \mathbb{R}$}$.

Indeed, $\Phi(x)$ is a distribution function.

In general, we can obtain any normal random variable variable by shifting and scaling a standard normal random variable. In particular, define

Then

And we say that $X$ is a normal random variable with mean $\mu$ and variance $\sigma^2$. We write $X \sim N(\mu,\sigma^2)$.

Definition

If $Z$ is a standard normal random variable and $X = \sigma Z +\mu$, then $X$ is a normal random variable with mean $\mu$ and variance $\sigma^2$, i.e

Conversely, if $X \sim N(\mu,\sigma^2)$, then the random variable defined by $\displaystyle{Z = \frac{X-\mu}{\sigma}}$ is a standard random variable, so

And we may differentiate $F_X(x)$ to find the PDF of $X$:

Corollary

If $X$ is a normal random variable with mean $\mu$ and variance $\sigma^2$, $X \sim N(\mu,\sigma^2)$, then

An important property of the normal distribution is that a linear transformation of a normal random variable is itself a normal random variable.

Theorem

If $X \sim N(\mu_X,\sigma_X^2)$ and $Y = aX+b$ where $a,b \in \mathbb{R}$, then

Proof

This follows trivially from the arguments shown above.

QED

Gamma Distribution

The gamma distribution is another widely used distribution. Its importance is largely due to its relation to exponential and normal distributions. Before introducing the gamma random variable, we need to introduce the gamma function.

Definition

The gamma function, shown by $\Gamma(x)$ is an extension of the factorial function to real and complex numbers. If $n \in {1,2,3,\cdots}$, then

More generally, for any positive real number $\alpha$, define

Note that when $\alpha =1$, we have

Using the change of variable $x = \lambda y$, the following equation is also very useful:

More properties of a gamma function follows:

• $\Gamma(\alpha+1) = \alpha\Gamma(\alpha)$;

• $\displaystyle{\Gamma\left( \frac{1}{2} \right) = \sqrt{\pi} }$.

Now we may define the gamma distribution:

Definition

A continuous random variable $X$ is said to have a gamma distribution with parameters $\alpha>0$ and $\lambda>0$, shown as $X \sim Gamma(\alpha,\lambda)$, if its PDF is given by

If $\alpha =1$, we get $Gamma(1,\lambda) = Exponential(\lambda)$, which is a special case. More generally, if we take the sum of $n$ independent $Exponential(\lambda)$ random variables, we will get a $Gamma(n,\lambda)$ random variable. It can be proved by using moment generating functions.

Corollary

If $X \sim Gamma(\alpha,\lambda)$, then

Weibull Distribution

The Weibull distribution is widely used in engineering practice due to its versatility. It was originally proposed for the interpretation of fatigue data, but now its use has been extended to many other engineering problems. In particular, it is widely used in the field of life phenomena as the distribution of the lifetime of some object, especially when the “weakest link” model is appropriate for the object. That is, consider an object consisting of many parts, and suppose that the object experiences death (failure) when any of its parts fail. It has been shown (both theoretically and empirically) that under these conditions a Weibull distribution provides a close approximation to the distribution of the lifetime of the item.

Definition

A continuous random variable is said to have a Weibull distribution if its distribution function is given by

By differentiating the function above, we get the PDF of a Weibull random variable with parameters $\nu,\alpha,\beta$:

Cauchy Distribution

Definition

A continuous random variable is said to have a Cauchy Distribution with parameter $\theta \in \mathbb{R}$, if its PDF is given by

Suppose that a narrow beam flashlight is spun around its center, which is located a unit distance from the $x$-axis. Consider the point $X$ at which the beam interests the $X$ axis when the flashlight has stopped spinning, as shown in figure $3.6$.

The point $X$ is determined by the angle $\theta$ between the flashlight and the $y$-axis, which from the physical situation, appears to be uniformly distributed between $-\pi/2$ and $\pi/2$/ The distribution function of $X$ is thus given by

Where we know that

Hence the density function of $X$ is given by

and we see that $X$ has the Cauchy distribution.

Recommended citation: Jiajun Zhang, (2024) Some Continuous Distributions