beamer-tu-logo

Chapter 3: Common Families of Distributions Lecture 10: Useful distributions (part I)

Binomial distribution binomial(n,p)

Let n be a positive integer and p ∈ [0,1]. The binomial pmf with size n and probability p is

f (x) = \{ \begin{matrix}\binom{n}{x}p^{x}(1 − p)^{n−x} & x = 0,1,...,n \\ 0 & otherwise, \end{matrix}

We have previously obtained that, if X ∼ f , then E(X) = np; Var(X) = np(1 − p); mgf M_{X}(t) = (pe^{t}+ 1 − p)^{n}, chf ϕ_{X}(t) = (pe^{i̇t}+ 1 − p)^{n}, t ∈ R. In the special case of n = 1, the binomial distribution is also called the Bernoulli distribution with probability p. If X_{1},...,X_{n} are n Bernoulli random variables from n independent Bernoulli trials with the same probability p, then X = ∑ _{i=1}^{n}X_{i} has the binomial distribution with size n and parameter p.

UW-Madison (Statistics)  Stat 609 Lecture 10  2014  1 / 12

beamer-tu-logo

Poisson distribution Poisson(λ)

The Poisson distribution with parameter λ > 0 has pmf

f (x) = \{ \begin{matrix}\frac{λ^{x}e^{−λ}}{x!} & x = 0,1,2,... \\ 0 & otherwise, \end{matrix}

The mgf has been obtained previously, which is e^{λ(e^{t}−1)} , t ∈ R. The chf is then e^{λ(e^{i̇t}−1)} , t ∈ R. If X ∼ f , then

E(X) = \frac{d}{dt}e^{λ(e^{t}−1)} |_{t=0} = λe^{t}e^{λ(e^{t}−1)} |_{t=0} = λ

E(X^{2}) = λ \frac{d}{dt}e^{t}e^{λ(e^{t}−1)} |_{t=0}= λ [e^{t}e^{λ(e^{t}−1)} + e^{t}λe^{t}e^{λ(e^{t}−1)} ] |_{t=0}= λ +λ^{2}

Hence,

Var(X) = E(X^{2}) − [E(X)]^{2} = λ + λ^{2} − λ^{2} = λ

Example 3.2.4 (Waiting time)

As an example of a waiting-for-occurrence application, consider a telephone operator who, on the average, handles 5 calls per 3 minutes.

UW-Madison (Statistics)  Stat 609 Lecture 10  2014  2 / 12

beamer-tu-logo

Poisson distribution Poisson(λ)

The Poisson distribution with parameter λ > 0 has pmf

f (x) = \{ \begin{matrix}\frac{λ^{x}e^{−λ}}{x!} & x = 0,1,2,... \\ 0 & otherwise, \end{matrix}

The mgf has been obtained previously, which is e^{λ(e^{t}−1)} , t ∈ R. The chf is then e^{λ(e^{i̇t}−1)} , t ∈ R. If X ∼ f , then

E(X) = \frac{d}{dt}e^{λ(e^{t}−1)} |_{t=0} = λe^{t}e^{λ(e^{t}−1)} |_{t=0} = λ

E(X^{2}) = λ \frac{d}{dt}e^{t}e^{λ(e^{t}−1)} |_{t=0}= λ [e^{t}e^{λ(e^{t}−1)} + e^{t}λe^{t}e^{λ(e^{t}−1)} ] |_{t=0}= λ +λ^{2}

Hence,

Var(X) = E(X^{2}) − [E(X)]^{2} = λ + λ^{2} − λ^{2} = λ

Example 3.2.4 (Waiting time)

As an example of a waiting-for-occurrence application, consider a telephone operator who, on the average, handles 5 calls per 3 minutes.

UW-Madison (Statistics)  Stat 609 Lecture 10  2014  2 / 12

beamer-tu-logo

Let X = number of calls in a minute. If X follows a Poisson distribution, what is λ? Since E(X) = average of calls in a minute, λ = E(X) = 5/3. What is the probability that there will be no calls in the next minute?

P(X = 0) = \frac{e^{−5/3} (5/3)^{0}}{0!} = e^{−5/3} = 0.189

What is the probability that there will be at least two calls?

P(X ≥ 2) = 1−P(X = 0)−P(X = 1) = 1−0.189− \frac{e^{−5/3} (5/3)^{1}}{1!} = 0.496

Hypergeometric distribution hypergeometric(K ,M,N)

Let K , M, and N be positive integers with M ≤ N. The hypergeometric distribution has pmf

f (x) = \{ \begin{matrix}\binom{M}{x}\binom{N−M}{K −x} ; \binom{N}{K} & x = 0,1,...,min\{K ,M\}, M − N + K ≤ x \\ 0 & otherwise \end{matrix}

If we select K balls at random without replacement from a box filled with M red balls and N − M green balls, then

f (x) = P(exactly x of the balls are red )

UW-Madison (Statistics)  Stat 609 Lecture 10  2014  3 / 12

beamer-tu-logo

Let X = number of calls in a minute. If X follows a Poisson distribution, what is λ? Since E(X) = average of calls in a minute, λ = E(X) = 5/3. What is the probability that there will be no calls in the next minute?

P(X = 0) = \frac{e^{−5/3} (5/3)^{0}}{0!} = e^{−5/3} = 0.189

What is the probability that there will be at least two calls?

P(X ≥ 2) = 1−P(X = 0)−P(X = 1) = 1−0.189− \frac{e^{−5/3} (5/3)^{1}}{1!} = 0.496

Hypergeometric distribution hypergeometric(K ,M,N)

Let K , M, and N be positive integers with M ≤ N. The hypergeometric distribution has pmf

f (x) = \{ \begin{matrix}\binom{M}{x}\binom{N−M}{K −x} ; \binom{N}{K} & x = 0,1,...,min\{K ,M\}, M − N + K ≤ x \\ 0 & otherwise \end{matrix}

If we select K balls at random without replacement from a box filled with M red balls and N − M green balls, then

f (x) = P(exactly x of the balls are red )

UW-Madison (Statistics)  Stat 609 Lecture 10  2014  3 / 12

beamer-tu-logo

Typically, K < min{M,N} and then

f (x) = \{ \begin{matrix}\binom{M}{x}\binom{N−M}{K −x} ; \binom{N}{K} & x = 0,1,...,K , \\ 0 & otherwise \end{matrix}

Thus

∑_{x=0}^{K} \binom{M}{x}\binom{N−M}{K −x} ; \binom{N}{K} = 1 (not trivial to verify)

If X ∼ f , then

E(X) = ∑_{x=0}^{K}x \binom{M}{x}\binom{N−M}{K −x} ; \binom{N}{K} = ∑_{x=1}^{K} \frac{KM\binom{M−1}{x−1}\binom{N−M}{K −x}}{N\binom{N−1}{K −1}}

= \frac{MK}{N} ∑_{y=0}^{K −1} \binom{M−1}{y}\binom{(N−1)−(M−1)}{K −1−y} ; \binom{N−1}{K −1} = \frac{MK}{N}

A similar but more lengthy argument leads to

Var(X) = \frac{KM(N − M)(N − K )}{N^{2}(N − 1)}

UW-Madison (Statistics)  Stat 609 Lecture 10  2014  4 / 12

beamer-tu-logo

Negative binomial negative-binomial(r,p)

Let r be a positive integer and p ∈ [0,1]. The negative binomial pmf is

f (x) = \{ \begin{matrix}\binom{x−1}{r−1}p^{r}(1 − p)^{x−r} & x = r,r + 1,... \\ 0 & otherwise, \end{matrix}

In a sequence of independent Bernoulli trials with probability p, if X is the number of trials needed to have the rth success (1 in the Bernoulli trial), then X ∼ f . If Y = X − r (the number of 0’s before the rth 1), then

P(Y = y) = \binom{r + y − 1}{y} p^{r}(1 − p)^{y}, y = 0,1,2,...

This is also called the negative binomial distribution. The special case of negative binomial distribution with r = 1 is called the geometric distribution:

f (x) = \{ \begin{matrix}p(1 − p)^{x−1} & x = 1,2,... \\ 0 & otherwise, \end{matrix}
UW-Madison (Statistics)  Stat 609 Lecture 10  2014  5 / 12

beamer-tu-logo

The expectation of the negative binomial distribution (2nd definition) is

E(Y ) = = ∑_{y=0}^{∞}y\binom{r + y − 1}{y} p^{r}(1 − p)^{y} = ∑_{y=1}^{∞} \frac{(r + y − 1)!}{(y − 1)!(r − 1)!} p^{r}(1 − p)^{y}

= ∑_{y=1}^{∞}r\binom{r + y − 1}{y − 1} p^{r}(1 − p)^{y} = ∑_{z=0}^{∞}r\binom{r + z}{z} p^{r}(1 − p)^{z+1}

= \frac{r(1 − p)}{p} ∑_{z=0}^{∞} \binom{(r + 1) + (z − 1)}{z} p^{r+1}(1 − p)^{z}
= \frac{r(1 − p)}{p}

The expectation of the negative binomial distribution (1st definition) is

E(X) = E(Y ) + r = \frac{r(1 − p)}{p} + r = \frac{r}{p}

A similar calculation shows

Var(Y ) = Var(X) = \frac{r(1 − p)}{p^{2}}

beamer-tu-logo

Uniform distribution uniform(a,b)

The uniform distribution on the interval [a,b], where a and b are real numbers with a < b, has pdf

f (x) = \{ \begin{matrix}\frac{1}{b−a} & a ≤ x ≤ b \\ 0 & otherwise \end{matrix}

If X ∼ f , then

E(X) = \frac{1}{b − a} ∫_{a}^{b}xdx = \frac{x^{2}}{2(b − a)} |_{a}^{b} = \frac{b^{2} − a^{2}}{2(b − a)} = \frac{a + b}{2}
E(X^{2}) = \frac{1}{b − a} ∫_{a}^{b}x^{2}dx = \frac{x^{3}}{3(b − a)} |_{a}^{b} = \frac{b^{3} − a^{3}}{2(b − a)} = \frac{b^{2} + a^{2} + ab}{3}
Var(X) = E(X^{2}) − [E(X)]^{2} = \frac{b^{2} + a^{2} + ab}{3} − \frac{(a + b)^{2}}{4} = \frac{(b − a)^{2}}{12}
M_{X}(t) = \frac{1}{b − a} ∫_{a}^{b}e^{tx}dx = \frac{e^{tb}− e^{ta}}{t(b − a)} , t ∈ R

beamer-tu-logo

Normal distribution N(μ,σ^{2})

The normal distribution plays a central role in statistics. The normal distributions and distributions associated with it are very tractable analytically. The normal distribution has the familiar bell shape, whose symmetry makes it an appealing choice for many population models. Under the Central Limit Theorem (Chapter 5), the normal distribution can be used to approximate a large variety of distributions in large samples.

Let μ ∈ R and σ > 0 be two constants. The normal distribution N(μ,σ^{2}) has pdf

f (x) = \frac{1}{\sqrt{2π}σ} e^{−(x−μ)^{2}/(2σ^{2})} , x ∈ R

When μ = 0 and σ = 1, N(0,1) is called the standard normal distribution. The following is a figure of N(0,1) pdf curve.

UW-Madison (Statistics)  Stat 609 Lecture 10  2014  8 / 12

beamer-tu-logo

Standard normal pdf curve

We first need to show that f is indeed a pdf. By the transformation z = (x − μ)/σ,

\frac{1}{\sqrt{2π}σ} ∫_{−∞}^{∞} e^{−(x−μ)^{2}/(2σ^{2})} dx = \frac{1}{\sqrt{2π}} ∫_{−∞}^{∞} e^{−z^{2}/2} dz = \frac{2}{\sqrt{2π}} ∫_{0}^{∞} e^{−z^{2}/2} dz

because e^{−z^{2}/2} is symmetric.

UW-Madison (Statistics)  Stat 609 Lecture 10  2014  9 / 12

beamer-tu-logo

Note that

( ∫_{0}^{∞} e^{−z^{2}/2} dz) ^{2} = ( ∫_{0}^{∞} e^{−z^{2}/2} dz) ( ∫_{0}^{∞} e^{−u^{2}/2} du)
= ∫_{0}^{∞} ∫_{0}^{∞} e^{−(z^{2}+u^{2})/2} dudz
= ∫_{0}^{∞} ∫_{0}^{π/2} re^{−r^{2}/2} dθdr  z = r cos θ, u = r sinθ
= \frac{π}{2} ∫_{0}^{∞} re^{−r^{2}/2} dr = \frac{π}{2} e^{−r^{2}/2} |_{∞}^{0} = \frac{π}{2}

Thus,

\frac{1}{\sqrt{2π}σ} ∫_{−∞}^{∞} e^{−(x−μ)^{2}/(2σ^{2})} dx = \frac{2}{\sqrt{2π}} ∫_{0}^{∞} e^{−z^{2}/2} dz = \frac{2}{\sqrt{2π}} \sqrt{\frac{π}{2}} = 1

A consequence of this result is

\sqrt{\frac{π}{2}} = ∫_{0}^{∞} e^{−z^{2}/2} dz = \frac{1}{\sqrt{2}} ∫_{0}^{∞} w^{−1/2} e^{−w} dw = \frac{1}{\sqrt{2}}Γ(1/2) (w = z^{2}/2)

Hence

Γ(1/2) = \sqrt{π}
UW-Madison (Statistics)  Stat 609 Lecture 10  2014  10 / 12

beamer-tu-logo

Properties of normal distributions

X ∼ N(μ,σ^{2}) iff Z = (X − μ)/σ ∼ N(0,1) (by transformation). If Z ∼ N(0,1), then for any t ∈ R, its mgf is

M_{Z}(t) = \frac{1}{\sqrt{2π}} ∫_{−∞}^{∞} e^{tz}e^{−z^{2}/2} dz = \frac{e^{t^{2}/2}}{\sqrt{2π}} ∫_{−∞}^{∞} e^{−(z−t)^{2}/2} dz = e^{t^{2}/2}

If X ∼ N(μ,σ^{2}), then Z = (X − μ)/σ ∼ N(0,1) and the mgf of X is

M_{X}(t) = M_{σZ+μ} (t) = e^{μt}M_{Z}(σt) = e^{μt+σ^{2}t^{2}/2}, t ∈ R If X ∼ N(μ,σ^{2}), then its chf is ϕ_{X}(t) = e^{i̇μt−σ^{2}t^{2}/2} , t ∈ R. If X ∼ N(0,σ^{2}), by differentiating M_{X}(t) = e^{σ^{2}t^{2}/2}, we obtain that

E(X^{r}) = \{ \begin{matrix}(r − 1)(r − 3)··· 3 · 1σ^{r} & when r is an even integer \\ 0 & when r is an odd integer \end{matrix}

If Z ∼ N(0,1), then E(Z) = 0 and Var(Z) = E(Z^{2}) = 1. If X ∼ N(μ,σ^{2}), then E(X) = μ and Var(X) = σ^{2}. If X ∼ N(μ,σ^{2}), the distribution of Y = e^{X} is called the log-normal distribution with E(Y ) = e^{μ+σ^{2}/2} and Var(Y ) = e^{2(μ+σ^{2})} −e^{2μ+σ^{2}} .

UW-Madison (Statistics)  Stat 609 Lecture 10  2014  11 / 12

beamer-tu-logo

The cdf of the standard normal is called the standard normal cdf and denoted by

Φ(x) = \frac{1}{\sqrt{2π}} ∫_{−∞}^{x} e^{−t^{2}/2} dt, x ∈ R

But this cdf does not have a close form. If X ∼ N(μ,σ^{2}), then its cdf is

F_{X}(x) = P ( \frac{X − μ}{σ} ≤ \frac{x − μ}{σ} ) = P ( Z ≤ \frac{x − μ}{σ} ) = Φ ( \frac{x − μ}{σ} )

We can then use the standard normal cdf to calculate probabilities related to all normal random variables. The pdf of N(μ,σ^{2}) is a bell shaped curve that is symmetric about μ, maximized at μ, and changes from concave to convex at μ ± σ. When X ∼ N(μ,σ^{2}),

P(|X − μ| ≤ σ) = Φ(1) − Φ(−1) = 0.6826
P(|X − μ| ≤ 2σ) = Φ(2) − Φ(−2) = 0.9544
P(|X − μ| ≤ σ) = Φ(3) − Φ(−3) = 0.9974
UW-Madison (Statistics)  Stat 609 Lecture 10  2014  12 / 12