beamer-tu-logo

Lecture 11: Useful distributions (part II)

Gamma distribution gamma(α,β)

Let α > 0 and β > 0 be two constants. The gamma distribution with shape parameter α and scale parameter β has pdf

f (x) = \{ \begin{matrix}\frac{1}{Γ(α)β^{α}} x^{α−1} e^{−x/β} & x > 0 \\ 0 & x ≤ 0 \end{matrix}

where

Γ(α) = ∫_{0}^{∞} x^{α−1} e^{−x} dx

is called the gamma function,

Γ(1/2) = \sqrt{π}, Γ(α + 1) = αΓ(α), Γ(n) = (n − 1)!, n = 1,2,... If α < 1, then the pdf of gamma(α,β) is decreasing in x and unbounded at x = 0. If α ≥ 1, then the pdf of gamma(α,β) is bounded, increasing in x < (α − 1)β, and decreasing in x > (α − 1)β. Previously, we showed that, if X ∼ gamma(α,β), then

UW-Madison (Statistics)  Stat 609 Lecture 11  2014  1 / 13

beamer-tu-logo

M_{X}(t) = (1 − βt)^{−α} , t < 1/β
E(X^{n}) = α(α + 1)··· (α + n − 1)β^{n}, n = 1,2,...

In particular,

E(X) = αβ, E(X^{2}) = α(α + 1)β^{2}, Var(X) = αβ^{2}

From the mfg, the chf is ϕ_{X}(t) = (1 − i̇βt)^{−α} , t ∈ R

Some gamma pdf curves

beamer-tu-logo

Chi-square distribution

The gamma distribution gamma(α,β) with β = 2 and α = k/2 for a positive integer k is called the chi-square distribution with k degrees of freedom, and its pdf is

f (x) = \{ \begin{matrix}\frac{1}{Γ(k/2)2^{k/2}} x^{(k/2)−1} e^{−x/2} & x ≥ 0 \\ 0 & x < 0 \end{matrix}

If X ∼ f , then

E(X) = k, Var(X) = 2k,

and the mgf M_{X}(t) and chf ϕ_{X}(t) are given by

M_{X}(t) = (1 − 2t)^{−k/2} , t < 1/2, ϕ_{X}(t) = (1 − 2i̇t)^{−k/2} , t ∈ R

The chi-square distribution is closely related to the normal distribution. In Example 2.1.9, we showed that if X ∼ N(0,1), then X^{2} ∼ chi-square with 1 degree of freedom. A similar result about the chi-square distribution with k degrees of freedom will be introduced later.

UW-Madison (Statistics)  Stat 609 Lecture 11  2014  3 / 13

beamer-tu-logo

Exponential distribution exponential(β)

A special case of the gamma distribution with α = 1 is called the exponential distribution, i.e., gamma(1,β) = exponential(β). The exponential(β) distribution has pdf

f (x) = \{ \begin{matrix}\frac{1}{β}e^{−x/β} & x ≥ 0 \\ 0 & x < 0 \end{matrix}

For any x > 0,

P(X ≤ x) = \frac{1}{β} ∫_{0}^{x} e^{−t/β} dt = e^{−t/β} |_{x}^{0} = 1 − e^{−x/β}

Hence, the cdf of exponential(β) has an explicit form

F(x) = \{ \begin{matrix}1 − e^{−x/β} & x ≥ 0 \\ 0 & x < 0 \end{matrix}

An example of X ∼ exponential(β) is the lifetime.

beamer-tu-logo

Since exponential(β) = gamma(1,β), if X ∼ exponential(β),

E(X) = β, Var(X) = β^{2}, M_{X}(t) = \frac{1}{1 − βt} , t < \frac{1}{β} ,

ϕ_{X}(t) = \frac{1}{1 − i̇βt} , t ∈ R

The exponential distribution has a memoryless property, i.e.,

P(X > s|X > t) = P(X > s − t)

for any s > t ≥ 0, because

P(X > s|X > t) = \frac{P(X > s,X > t)}{P(X > t)} = \frac{P(X > s)}{P(X > t)}
= \frac{1 − F(s)}{1 − F(t)} = \frac{e^{−s/β}}{e^{−t/β}} = e^{−(s−t)/β}
= 1 − F(s − t) = P(X > s − t)

That means, if X is the lifetime of a product, then the lifetime of a product that has been used t hours has the same distribution as that of a new product (new = used).

UW-Madison (Statistics)  Stat 609 Lecture 11  2014  5 / 13

beamer-tu-logo

Two parameter exponential distribution exponential(μ,β)

If X ∼ exponential(β) and μ ∈ R is a constant, then the distribution of Y = X + μ is called the two parameter exponential distribution and denoted by exponential(μ,β). Its pdf and cdf are (by transformation)

f (x) = \{ \begin{matrix}\frac{1}{β}e^{−(x−μ)/β} & x ≥ μ \\ 0 & x < μ \end{matrix} F(x) = \{ \begin{matrix}1 − e^{−(x−μ)/β} & x ≥ μ \\ 0 & x < μ \end{matrix}

and, if Y ∼ exponential(μ,β),

E(Y )=μ +β, Var(Y )=β^{2}, M_{Y}(t)= \frac{e^{μt}}{1 − βt} , t < \frac{1}{β} , ϕ_{Y}(t)= \frac{e^{i̇μt}}{1 − i̇βt} , t ∈ R

Double exponential distribution double-exponential(μ,σ)

By reflecting the pdf of exponential(μ,σ) around μ, we obtain the double-exponential(μ,σ) pdf that is symmetric about μ:

f (x) = \{ \begin{matrix}\frac{1}{2σ} e^{−(x−μ)/σ} & x ≥ μ \\ \frac{1}{2σ} e^{(x−μ)/σ} & x < μ \end{matrix} = \frac{1}{2σ} e^{−|x−μ|/σ} , x ∈ R

beamer-tu-logo

Two parameter exponential distribution exponential(μ,β)

If X ∼ exponential(β) and μ ∈ R is a constant, then the distribution of Y = X + μ is called the two parameter exponential distribution and denoted by exponential(μ,β). Its pdf and cdf are (by transformation)

f (x) = \{ \begin{matrix}\frac{1}{β}e^{−(x−μ)/β} & x ≥ μ \\ 0 & x < μ \end{matrix} F(x) = \{ \begin{matrix}1 − e^{−(x−μ)/β} & x ≥ μ \\ 0 & x < μ \end{matrix}

and, if Y ∼ exponential(μ,β),

E(Y )=μ +β, Var(Y )=β^{2}, M_{Y}(t)= \frac{e^{μt}}{1 − βt} , t < \frac{1}{β} , ϕ_{Y}(t)= \frac{e^{i̇μt}}{1 − i̇βt} , t ∈ R

Double exponential distribution double-exponential(μ,σ)

By reflecting the pdf of exponential(μ,σ) around μ, we obtain the double-exponential(μ,σ) pdf that is symmetric about μ:

f (x) = \{ \begin{matrix}\frac{1}{2σ} e^{−(x−μ)/σ} & x ≥ μ \\ \frac{1}{2σ} e^{(x−μ)/σ} & x < μ \end{matrix} = \frac{1}{2σ} e^{−|x−μ|/σ} , x ∈ R

beamer-tu-logo

This pdf is not bell-shaped; in fact, it has a peak (a non-differentiable point) at x = μ. Its cdf is

F(x) = \{ \begin{matrix}1 − \frac{1}{2}e^{−(x−μ)/σ} & x ≥ μ \\ \frac{1}{2}e^{(x−μ)/σ} & x < μ \end{matrix}

If X ∼ double-exponential(μ,σ), then Z = (X − μ)/σ ∼ double-exponential(0,1). If Z = (X − μ)/σ ∼ double-exponential(0,1), then

E(Z) = \frac{1}{2} ∫_{−∞}^{∞} xe^{−|x|} dx = 0

because xe^{−|x|} is an odd function, and

Var(Z) = E(Z^{2}) = \frac{1}{2} ∫_{−∞}^{∞} x^{2}e^{−|x|} dx = ∫_{0}^{∞} x^{2}e^{−x} dx = Γ(3) = 2

If X ∼ double-exponential(μ,σ), then X = σZ + μ, Z = (X − μ)/σ ∼ double-exponential(0,1), and

E(X) = E(σZ +μ) = μ, Var(X) = Var(σZ +μ) = σ^{2}Var(Z) = 2σ^{2}

UW-Madison (Statistics)  Stat 609 Lecture 11  2014  7 / 13

beamer-tu-logo

Cauchy distribution Cauchy(μ,σ)

For constants μ ∈ R and σ > 0, the Cauchy(μ,σ) distribution has pdf

f (x) = \frac{σ}{π[σ^{2} + (x − μ)^{2}]} , x ∈ R

The Cauchy pdf is bell-shaped and symmetric about μ. Its biggest difference from the normal pdf is that, if X ∼ Cauchy(μ,σ), then E|X| = ∞ (and hence E|X|^{n} = ∞, n = 2,3,...) The mgf of X is ∞ except at 0 and the chf of X is e^{i̇μ+σ|t|}, t ∈ R. μ still measures the center of Cauchy(μ,σ) although the expectation does not exist: μ is the median in the sense that P(X < μ) = \frac{1}{2}.

Standard normal and Cauchy pdf curves

beamer-tu-logo

Logistic distribution logistic(μ,σ)

For constants μ ∈ R and σ > 0, the logistic(μ,σ) distribution has pdf

f (x) = \frac{e^{−(x−μ)/σ}}{σ[1 + e^{−(x−μ)/σ} ]^{2}} , x ∈ R

This pdf is again bell-shaped and symmetric about μ. The cdf of logistic(μ,σ) has a close form:

F(x) = ∫_{−∞}^{x} f (t)dt = \frac{1}{1 + e^{−(x−μ)/σ}} , x ∈ R

By symmetry, E(X) = μ if X ∼ logistic(μ,σ). The variance of X ∼ logistic(μ,σ) is not easy to obtain, but we give the result here: Var(X) = σ^{2}π^{2}/3.

Pareto distribution pareto(α,β)

For constants α > 0 and β > 0, the pareto(α,σ) distribution has pdf

f (x) = \{ \begin{matrix}αβ^{α} x^{−(α+1)} & x > β \\ 0 & x ≤ β \end{matrix}

UW-Madison (Statistics)  Stat 609 Lecture 11  2014  9 / 13

beamer-tu-logo

Logistic distribution logistic(μ,σ)

For constants μ ∈ R and σ > 0, the logistic(μ,σ) distribution has pdf

f (x) = \frac{e^{−(x−μ)/σ}}{σ[1 + e^{−(x−μ)/σ} ]^{2}} , x ∈ R

This pdf is again bell-shaped and symmetric about μ. The cdf of logistic(μ,σ) has a close form:

F(x) = ∫_{−∞}^{x} f (t)dt = \frac{1}{1 + e^{−(x−μ)/σ}} , x ∈ R

By symmetry, E(X) = μ if X ∼ logistic(μ,σ). The variance of X ∼ logistic(μ,σ) is not easy to obtain, but we give the result here: Var(X) = σ^{2}π^{2}/3.

Pareto distribution pareto(α,β)

For constants α > 0 and β > 0, the pareto(α,σ) distribution has pdf

f (x) = \{ \begin{matrix}αβ^{α} x^{−(α+1)} & x > β \\ 0 & x ≤ β \end{matrix}

UW-Madison (Statistics)  Stat 609 Lecture 11  2014  9 / 13

beamer-tu-logo

First, f is indeed a pdf, because

∫_{−∞}^{∞} f (x)dx = αβ^{α} ∫_{β}^{∞} x^{−(α+1)} dx = β^{α} x^{−α} |_{∞}^{β} = β^{α} β^{−α} = 1

Using a similar argument, we can obtain the cdf of pareto(α,β) as

F(x) = \{ \begin{matrix}1 − ( \frac{β}{x})^{α} & x > β \\ 0 & x ≤ β \end{matrix}

Since the integral ∫_{β}^{∞} x^{−t} dx is finite iff t > 1, E(X) = ∞ if α ≤ 1 when X ∼ pareto(α,β); if α > 1, then

E(X) = αβ^{α} ∫_{β}^{∞} x^{−α} dx = \frac{αβ^{α}}{α − 1} x^{−(α−1)} |_{∞}^{β} = \frac{αβ^{α}}{α − 1} β^{−(α−1)} = \frac{αβ}{α − 1}

Similarly, Var(X) = ∞ if α ≤ 2; and if α > 2,

E(X^{2}) = αβ^{α} ∫_{β}^{∞} x^{−α+1} dx = \frac{αβ^{α}}{α − 2} x^{−α+2} |_{∞}^{β} = \frac{αβ^{α}}{α − 2} β^{−α+2} = \frac{αβ^{2}}{α − 2}

Var(X) = E(X^{2}) − [E(X)]^{2} = \frac{αβ^{2}}{α − 2} − \frac{α^{2}β^{2}}{(α − 1)^{2}} = \frac{αβ^{2}}{(α − 1)^{2}(α − 2)}

UW-Madison (Statistics)  Stat 609 Lecture 11  2014  10 / 13

beamer-tu-logo

Weibull distribution Weibull(γ,β)

For constants γ > 0 and β > 0, if X ∼ exponential(β), then Y = X^{1/γ} ∼ Weibull(γ,β) with pdf

f (x) = \{ \begin{matrix}\frac{γ}{β}x^{γ−1} e^{−x^{γ}/β} & x > 0 \\ 0 & x ≤ 0 \end{matrix}

An example of Y ∼ Weibull(γ,β) is lifetime or failure time. If Y ∼ Weibull(γ,β), then X = Y^{γ} ∼ exponential(β) and

E(Y ) = E(X^{1/γ}) = \frac{1}{β} ∫_{0}^{∞} x^{1/γ}e^{−x/β} dx
= β^{1/γ}∫_{0}^{∞} u^{1/γ}e^{u}du = β^{1/γ}Γ( \frac{1}{γ} + 1)

Similarly, we can obtain that

Var(Y ) = β^{2/γ}\{ Γ( \frac{2}{γ} + 1) − [Γ( \frac{1}{γ} + 1) ]^{2}\}

beamer-tu-logo

Beta distribution beta(α,β)

For constants α > 0 and β > 0, the beta(α,β) distribution has pdf

f (x) = \{ \begin{matrix}\frac{Γ(α+β)}{Γ(α)Γ(β)} x^{α−1} (1 − x)^{β−1} & 0 < x < 1 \\ 0 & otherwise \end{matrix}

This is a pdf because

∫_{0}^{1}x^{α−1} (1 − x)^{β−1} dx = \frac{Γ(α)Γ(β)}{Γ(α + β)}

Since Γ(2) = Γ(1) = 1, beta(1,1) is the same as uniform(0,1). If X ∼ beta(α,β), then 1 − X ∼ beta(β,α). The pdf of beta(α,β) can be increasing (α > 1,β = 1), decreasing (α = 1,β > 1), U-shaped (α < 1,β < 1), or unimodal (α > 1,β > 1). If α = β, then the pdf of beta(α,β) is symmetric about \frac{1}{2}. For any r > 0, if X ∼ beta(α,β), then

E(X^{r}) = \frac{Γ(α + β)}{Γ(α)Γ(β)} ∫_{0}^{1}x^{r+α−1} (1 − x)^{β−1} dx = \frac{Γ(α + β)Γ(r + α)}{Γ(α)Γ(r + α + β)}

In particular,

E(X) = \frac{α}{α + β} , E(X^{2}) = \frac{α(α + 1)}{(α + β)(α + β + 1)}

Then

Var(X) = \frac{α(α + 1)}{(α + β)(α + β + 1)} − \frac{α^{2}}{(α + β)^{2}} = \frac{αβ}{(α + β)^{2}(α + β + 1)}

Some other continuous distributions will be introduced in Chapter 5.

beamer-tu-logo UW-Madison (Statistics)  Stat 609 Lecture 11  2014  13 / 13

Since Γ(2) = Γ(1) = 1, beta(1,1) is the same as uniform(0,1). If X ∼ beta(α,β), then 1 − X ∼ beta(β,α). The pdf of beta(α,β) can be increasing (α > 1,β = 1), decreasing (α = 1,β > 1), U-shaped (α < 1,β < 1), or unimodal (α > 1,β > 1). If α = β, then the pdf of beta(α,β) is symmetric about \frac{1}{2}. For any r > 0, if X ∼ beta(α,β), then

E(X^{r}) = \frac{Γ(α + β)}{Γ(α)Γ(β)} ∫_{0}^{1}x^{r+α−1} (1 − x)^{β−1} dx = \frac{Γ(α + β)Γ(r + α)}{Γ(α)Γ(r + α + β)}

In particular,

E(X) = \frac{α}{α + β} , E(X^{2}) = \frac{α(α + 1)}{(α + β)(α + β + 1)}

Then

Var(X) = \frac{α(α + 1)}{(α + β)(α + β + 1)} − \frac{α^{2}}{(α + β)^{2}} = \frac{αβ}{(α + β)^{2}(α + β + 1)}

Some other continuous distributions will be introduced in Chapter 5.

beamer-tu-logo UW-Madison (Statistics)  Stat 609 Lecture 11  2014  13 / 13