beamer-tu-logo

Lecture 3: Random variables and their distributions

Definition 1.4.1.

A random variable X is a function from S into a subset of R such that for any Borel set B ⊂ R {X ∈ B} = {ω ∈ S : X(ω) ∈ B} is an event.

It is convenient to deal with a summary variable than the elements in the original sample space. The range of a random variables is simpler than S. In some problems, there is a natural random variable; e.g., the number of accidents, number of successes, etc. In other cases, we may define a random variable according to our interests.

Example 1.4.3. (Toss a coin 3 times)

outcome  hhh hht hth thh tth tht htt ttt
X: number of heads 3  2 2 2 1 1 1 0

The range of X, {0,1,2,3}, is simpler than S. X treats hht, hth, thh the same, and tth, tht, htt the same.

UW-Madison (Statistics)  Stat 609 Lecture 3  2014  1 / 13

beamer-tu-logo

Definition 1.4.1.

A random variable X is a function from S into a subset of R such that for any Borel set B ⊂ R {X ∈ B} = {ω ∈ S : X(ω) ∈ B} is an event.

It is convenient to deal with a summary variable than the elements in the original sample space. The range of a random variables is simpler than S. In some problems, there is a natural random variable; e.g., the number of accidents, number of successes, etc. In other cases, we may define a random variable according to our interests.

Example 1.4.3. (Toss a coin 3 times)

outcome  hhh hht hth thh tth tht htt ttt
X: number of heads 3  2 2 2 1 1 1 0

The range of X, {0,1,2,3}, is simpler than S. X treats hht, hth, thh the same, and tth, tht, htt the same.

UW-Madison (Statistics)  Stat 609 Lecture 3  2014  1 / 13

beamer-tu-logo

Definition 1.4.1.

A random variable X is a function from S into a subset of R such that for any Borel set B ⊂ R {X ∈ B} = {ω ∈ S : X(ω) ∈ B} is an event.

It is convenient to deal with a summary variable than the elements in the original sample space. The range of a random variables is simpler than S. In some problems, there is a natural random variable; e.g., the number of accidents, number of successes, etc. In other cases, we may define a random variable according to our interests.

Example 1.4.3. (Toss a coin 3 times)

outcome  hhh hht hth thh tth tht htt ttt
X: number of heads 3  2 2 2 1 1 1 0

The range of X, {0,1,2,3}, is simpler than S. X treats hht, hth, thh the same, and tth, tht, htt the same.

UW-Madison (Statistics)  Stat 609 Lecture 3  2014  1 / 13

beamer-tu-logo

Induced probability

The induced probability of X is

P_{X}(B) = P(X ∈ B) = P({ω ∈ S : X(ω) ∈ B})

The probability P_{X} is called the distribution of X.

Example 1.4.3 (continued)

If the coin is fair, then

x  0 1 2 3
P(X = x) \frac{1}{8} \frac{3}{8} \frac{3}{8} \frac{1}{8}

A table like this effective for X taking finite many values.

Definition 1.5.1.

The cumulative distribution function (cdf) of a random variable X, denoted by F_{X}(x), is defined by

F_{X}(x) = P_{X}(X ≤ x), x ∈ R

Even if X may not take all real numbers, the cdf is defined for all x ∈ R

UW-Madison (Statistics)  Stat 609 Lecture 3  2014  2 / 13

beamer-tu-logo

Induced probability

The induced probability of X is

P_{X}(B) = P(X ∈ B) = P({ω ∈ S : X(ω) ∈ B})

The probability P_{X} is called the distribution of X.

Example 1.4.3 (continued)

If the coin is fair, then

x  0 1 2 3
P(X = x) \frac{1}{8} \frac{3}{8} \frac{3}{8} \frac{1}{8}

A table like this effective for X taking finite many values.

Definition 1.5.1.

The cumulative distribution function (cdf) of a random variable X, denoted by F_{X}(x), is defined by

F_{X}(x) = P_{X}(X ≤ x), x ∈ R

Even if X may not take all real numbers, the cdf is defined for all x ∈ R

UW-Madison (Statistics)  Stat 609 Lecture 3  2014  2 / 13

beamer-tu-logo

Induced probability

The induced probability of X is

P_{X}(B) = P(X ∈ B) = P({ω ∈ S : X(ω) ∈ B})

The probability P_{X} is called the distribution of X.

Example 1.4.3 (continued)

If the coin is fair, then

x  0 1 2 3
P(X = x) \frac{1}{8} \frac{3}{8} \frac{3}{8} \frac{1}{8}

A table like this effective for X taking finite many values.

Definition 1.5.1.

The cumulative distribution function (cdf) of a random variable X, denoted by F_{X}(x), is defined by

F_{X}(x) = P_{X}(X ≤ x), x ∈ R

Even if X may not take all real numbers, the cdf is defined for all x ∈ R

UW-Madison (Statistics)  Stat 609 Lecture 3  2014  2 / 13

beamer-tu-logo

Example 1.4.3 (continued)

The cdf of X is a step function

F_{X}(x) = \{ \begin{matrix}0 & −∞ < x < 0 \\ 0.125 & 0 ≤ x < 1 \\ 0.5 & 1 ≤ x < 2 \\ 0.875 & 2 ≤ x < 3 \\ 1 & 3 ≤ x < ∞ \end{matrix}

beamer-tu-logo

Theorem 1.5.3.

The function F(x) is a cdf iff a. lim_{x→−∞} F(x) = 0 and lim_{x→∞} F(x) = 1; b. F(x) is nondecreasing in x; c. F(x) is right-continuous: lim_{ϵ→0,ϵ>0} F(x + ϵ) = F(x) for any x ∈ R.

This theorem says that if F is the cdf of a random variable X, then F satisfies a-c (this is easy to prove); if F satisfies a-c, then there exists a random variable X such that the cdf of X is F (this is not easy to prove).

Definition 1.5.7.

A random variable X is continuous if F_{X}(x) is continuous in x. A random variable X is discrete if F_{X}(x) is a step function of x.

There are random variables that are neither discrete nor continuous; e.g., a mixture of the two types.

beamer-tu-logo

Theorem 1.5.3.

The function F(x) is a cdf iff a. lim_{x→−∞} F(x) = 0 and lim_{x→∞} F(x) = 1; b. F(x) is nondecreasing in x; c. F(x) is right-continuous: lim_{ϵ→0,ϵ>0} F(x + ϵ) = F(x) for any x ∈ R.

This theorem says that if F is the cdf of a random variable X, then F satisfies a-c (this is easy to prove); if F satisfies a-c, then there exists a random variable X such that the cdf of X is F (this is not easy to prove).

Definition 1.5.7.

A random variable X is continuous if F_{X}(x) is continuous in x. A random variable X is discrete if F_{X}(x) is a step function of x.

There are random variables that are neither discrete nor continuous; e.g., a mixture of the two types.

beamer-tu-logo

Theorem 1.5.3.

The function F(x) is a cdf iff a. lim_{x→−∞} F(x) = 0 and lim_{x→∞} F(x) = 1; b. F(x) is nondecreasing in x; c. F(x) is right-continuous: lim_{ϵ→0,ϵ>0} F(x + ϵ) = F(x) for any x ∈ R.

This theorem says that if F is the cdf of a random variable X, then F satisfies a-c (this is easy to prove); if F satisfies a-c, then there exists a random variable X such that the cdf of X is F (this is not easy to prove).

Definition 1.5.7.

A random variable X is continuous if F_{X}(x) is continuous in x. A random variable X is discrete if F_{X}(x) is a step function of x.

There are random variables that are neither discrete nor continuous; e.g., a mixture of the two types.

beamer-tu-logo

Example 1.5.5 (logistic distribution)

An example of a continuous cdf is

F(x) = \frac{1}{1 + e^{−x}} , x ∈ R

It satisfies a-c in Theorem 1.5.3, because

lim_{x→−∞} \frac{1}{1 + e^{−x}} = 0,  lim_{x→∞} \frac{1}{1 + e^{−x}} = 1

F is continuous everywhere; in fact, it is differentiable everywhere with

\frac{d}{dx}F(x) = F′(x) = \frac{e^{−x}}{(1 + e^{−x} )^{2}} > 0 x ∈ R

Hence, F is strictly increasing.

Example

Let c be a constant and

G_{c}(x) = \{ \begin{matrix}0 & x < 0 \\ 1 − ce^{−3x} & x ≥ 0 \end{matrix}

For what values of c, G_{c} is a cdf?

UW-Madison (Statistics)  Stat 609 Lecture 3  2014  5 / 13

beamer-tu-logo

Example 1.5.5 (logistic distribution)

An example of a continuous cdf is

F(x) = \frac{1}{1 + e^{−x}} , x ∈ R

It satisfies a-c in Theorem 1.5.3, because

lim_{x→−∞} \frac{1}{1 + e^{−x}} = 0,  lim_{x→∞} \frac{1}{1 + e^{−x}} = 1

F is continuous everywhere; in fact, it is differentiable everywhere with

\frac{d}{dx}F(x) = F′(x) = \frac{e^{−x}}{(1 + e^{−x} )^{2}} > 0 x ∈ R

Hence, F is strictly increasing.

Example

Let c be a constant and

G_{c}(x) = \{ \begin{matrix}0 & x < 0 \\ 1 − ce^{−3x} & x ≥ 0 \end{matrix}

For what values of c, G_{c} is a cdf?

UW-Madison (Statistics)  Stat 609 Lecture 3  2014  5 / 13

beamer-tu-logo

Clearly,

lim_{x→−∞} G_{c}(x) = 0,  lim_{x→∞} G_{c}(x) = 1
G′_{c}(x) = \{ \begin{matrix}0 & x < 0 \\ 3ce^{−3x} & x > 0 \end{matrix}

If c < 0, G_{c} is decreasing. If c > 1, G_{c}(0) = 1 − c < 0. If 0 ≤ c ≤ 1, then G_{c} is nondecreasing. G_{c} is always right-continuous.

Hence, G_{c} is a cdf iff 0 ≤ c ≤ 1. If c = 1, then G_{c} is continuous. If c = 0, then G_{c} is a special discrete cdf. If 0 < c < 1, then G_{c} is neither continuous nor discrete.

Definition 1.5.8/Theorem 1.5.10.

Two random variables X and Y with cdf’s F_{X} and F_{Y} respectively are identically distributed iff F_{X}(x) = F_{Y}(x) for any x ∈ R, which is equivalent to P(X ∈ B) = P(Y ∈ B) for any Borel set B.

beamer-tu-logo

Clearly,

lim_{x→−∞} G_{c}(x) = 0,  lim_{x→∞} G_{c}(x) = 1
G′_{c}(x) = \{ \begin{matrix}0 & x < 0 \\ 3ce^{−3x} & x > 0 \end{matrix}

If c < 0, G_{c} is decreasing. If c > 1, G_{c}(0) = 1 − c < 0. If 0 ≤ c ≤ 1, then G_{c} is nondecreasing. G_{c} is always right-continuous.

Hence, G_{c} is a cdf iff 0 ≤ c ≤ 1. If c = 1, then G_{c} is continuous. If c = 0, then G_{c} is a special discrete cdf. If 0 < c < 1, then G_{c} is neither continuous nor discrete.

Definition 1.5.8/Theorem 1.5.10.

Two random variables X and Y with cdf’s F_{X} and F_{Y} respectively are identically distributed iff F_{X}(x) = F_{Y}(x) for any x ∈ R, which is equivalent to P(X ∈ B) = P(Y ∈ B) for any Borel set B.

beamer-tu-logo

Note that two identically distributed random variables are not necessarily equal (see Example 1.5.9).

The cdf’s can be used to calculate various probabilities related to random variables but sometimes it is more convenient to use another function to calculate probabilities.

Definition 1.6.1.

The probability mass function (pmf) of a discrete random variable X is

f_{X}(x) = P(X = x) x ∈ R

For discrete X, f_{X}(x) > 0 for only countably many x’s.

f_{X}(x) is the size of the jump in the cdf F_{X} at x.

For any Borel set A, P(X ∈ A) = ∑_{k∈A}f_{X}(k) and F_{X}(x) = P(X ≤ x) = ∑_{k≤b} f_{X}(k)

How to obtain the pmf for a discrete random variable X?

List all values X can take (with positive probabilities), x_{1},x_{2},... Find P(X = x_{i}), i = 1,2,...

UW-Madison (Statistics)  Stat 609 Lecture 3  2014  7 / 13

beamer-tu-logo

Note that two identically distributed random variables are not necessarily equal (see Example 1.5.9).

The cdf’s can be used to calculate various probabilities related to random variables but sometimes it is more convenient to use another function to calculate probabilities.

Definition 1.6.1.

The probability mass function (pmf) of a discrete random variable X is

f_{X}(x) = P(X = x) x ∈ R

For discrete X, f_{X}(x) > 0 for only countably many x’s.

f_{X}(x) is the size of the jump in the cdf F_{X} at x.

For any Borel set A, P(X ∈ A) = ∑_{k∈A}f_{X}(k) and F_{X}(x) = P(X ≤ x) = ∑_{k≤b} f_{X}(k)

How to obtain the pmf for a discrete random variable X?

List all values X can take (with positive probabilities), x_{1},x_{2},... Find P(X = x_{i}), i = 1,2,...

UW-Madison (Statistics)  Stat 609 Lecture 3  2014  7 / 13

beamer-tu-logo

Note that two identically distributed random variables are not necessarily equal (see Example 1.5.9).

The cdf’s can be used to calculate various probabilities related to random variables but sometimes it is more convenient to use another function to calculate probabilities.

Definition 1.6.1.

The probability mass function (pmf) of a discrete random variable X is

f_{X}(x) = P(X = x) x ∈ R

For discrete X, f_{X}(x) > 0 for only countably many x’s.

f_{X}(x) is the size of the jump in the cdf F_{X} at x.

For any Borel set A, P(X ∈ A) = ∑_{k∈A}f_{X}(k) and F_{X}(x) = P(X ≤ x) = ∑_{k≤b} f_{X}(k)

How to obtain the pmf for a discrete random variable X?

List all values X can take (with positive probabilities), x_{1},x_{2},... Find P(X = x_{i}), i = 1,2,...

UW-Madison (Statistics)  Stat 609 Lecture 3  2014  7 / 13

beamer-tu-logo

Characteristics of a pmf

For any pmf f , f (x) ≥ 0, x ∈ R and there is {x_{1},x_{2},...} such that

f (x_{1}) + f (x_{2}) + ···= 1.
If p_{i} > 0, i = 1,2,..., satisfy p_{1} + p_{2} + ···= 1, then f (x_{i}) = p_{i},
i = 1,2,..., is a pmf, where x_{1},x_{2},... can be arbitrary.

How to define something similar to a pmf for a continuous X?

Defining f_{X}(x) = P(X = x) does not work for a continuous X. For any ϵ > 0,

P(X = x) ≤ P(x − ϵ < X ≤ x) = F_{X}(x) − F_{X}(x − ϵ)

Since F_{X} is continuous at x, we obtain

P(X = x) ≤ lim_{ϵ→0}[F_{X}(x) − F_{X}(x − ϵ)] = 0

Hence, P(X = x) = 0 for any x ∈ R (compare this to a discrete X). For a discrete X, we have

F_{X}(x) = ∑_{k≤x} f_{X}(k)

beamer-tu-logo

Characteristics of a pmf

For any pmf f , f (x) ≥ 0, x ∈ R and there is {x_{1},x_{2},...} such that

f (x_{1}) + f (x_{2}) + ···= 1.
If p_{i} > 0, i = 1,2,..., satisfy p_{1} + p_{2} + ···= 1, then f (x_{i}) = p_{i},
i = 1,2,..., is a pmf, where x_{1},x_{2},... can be arbitrary.

How to define something similar to a pmf for a continuous X?

Defining f_{X}(x) = P(X = x) does not work for a continuous X. For any ϵ > 0,

P(X = x) ≤ P(x − ϵ < X ≤ x) = F_{X}(x) − F_{X}(x − ϵ)

Since F_{X} is continuous at x, we obtain

P(X = x) ≤ lim_{ϵ→0}[F_{X}(x) − F_{X}(x − ϵ)] = 0

Hence, P(X = x) = 0 for any x ∈ R (compare this to a discrete X). For a discrete X, we have

F_{X}(x) = ∑_{k≤x} f_{X}(k)

beamer-tu-logo

Definition 1.6.3.

The probability density function (pdf) of a continuous random variable X (if it exists) is the function f_{X}(x) satisfying

F_{X}(x) = ∫_{−∞}^{x} f_{X}(t)dt x ∈ R

Unlike the discrete case, a pdf of a continuous X may not exist. Unlike the cdf, f_{X} is not unique. For a continuous X, it is convenient to use the pdf to calculate probabilities If F_{X} is differentiable, then f_{X}(x) = F′_{X}(x) = \frac{d}{dx}F_{X}(x). A continuous random variable has a pdf iff its cdf is absolutely continuous. If f is a pdf, the set {x : f (x) > 0} is called its support.

Theorem 1.6.5.

A function f (x) is a pdf iff a. f (x) ≥ 0 for all x; b. ∫_{−∞}^{∞} f (x)dx = 1.

UW-Madison (Statistics)  Stat 609 Lecture 3  2014  9 / 13

beamer-tu-logo

Definition 1.6.3.

The probability density function (pdf) of a continuous random variable X (if it exists) is the function f_{X}(x) satisfying

F_{X}(x) = ∫_{−∞}^{x} f_{X}(t)dt x ∈ R

Unlike the discrete case, a pdf of a continuous X may not exist. Unlike the cdf, f_{X} is not unique. For a continuous X, it is convenient to use the pdf to calculate probabilities If F_{X} is differentiable, then f_{X}(x) = F′_{X}(x) = \frac{d}{dx}F_{X}(x). A continuous random variable has a pdf iff its cdf is absolutely continuous. If f is a pdf, the set {x : f (x) > 0} is called its support.

Theorem 1.6.5.

A function f (x) is a pdf iff a. f (x) ≥ 0 for all x; b. ∫_{−∞}^{∞} f (x)dx = 1.

UW-Madison (Statistics)  Stat 609 Lecture 3  2014  9 / 13

beamer-tu-logo

Definition 1.6.3.

The probability density function (pdf) of a continuous random variable X (if it exists) is the function f_{X}(x) satisfying

F_{X}(x) = ∫_{−∞}^{x} f_{X}(t)dt x ∈ R

Unlike the discrete case, a pdf of a continuous X may not exist. Unlike the cdf, f_{X} is not unique. For a continuous X, it is convenient to use the pdf to calculate probabilities If F_{X} is differentiable, then f_{X}(x) = F′_{X}(x) = \frac{d}{dx}F_{X}(x). A continuous random variable has a pdf iff its cdf is absolutely continuous. If f is a pdf, the set {x : f (x) > 0} is called its support.

Theorem 1.6.5.

A function f (x) is a pdf iff a. f (x) ≥ 0 for all x; b. ∫_{−∞}^{∞} f (x)dx = 1.

UW-Madison (Statistics)  Stat 609 Lecture 3  2014  9 / 13

beamer-tu-logo

Calculation of probabilities

For a continuous X with pdf f_{X},

P(X ∈ A) = ∫_{A}f_{X}(x)dx

For example, P(a < X < b) = P(a < X ≤ b) = P(a ≤ X < b)

= P(a ≤ X ≤ b) = F_{X}(b) − F_{X}(a) = ∫_{a}^{b}f_{X}(t)dt

which is the area under the curve f_{X}(t).

beamer-tu-logo

Example (finding the cdf, given a pdf)

Suppose that

f_{X}(x) = \{ \begin{matrix}1 + x & −1 ≤ x < 0 \\ 1 − x & 0 ≤ x < 1 \\ 0 & otherwise \end{matrix}

F_{X}(x) =? Obviously, F_{X}(x) = 0 when x < −1. For −1 ≤ x < 0,

F_{X}(x) = ∫_{−∞}^{x} f_{X}(t)dt = ∫_{−1}^{x} (1 + t)dt = \frac{t^{2}}{2} + t|_{t=−1}^{x} = \frac{x^{2}}{2} + x + \frac{1}{2}

For 0 ≤ x < 1,

F_{X}(x) = ∫_{−∞}^{x} f_{X}(t)dt = ∫_{−1}^{0} (1 + t)dt + ∫_{0}^{x}(1 − t)dt = \frac{1}{2} − \frac{x^{2}}{2} + x

For x ≥ 1,

F_{X}(x) = ∫_{−∞}^{x} f_{X}(t)dt = ∫_{−1}^{0} (1 + t)dt + ∫_{0}^{1}(1 − t)dt = \frac{1}{2} + \frac{1}{2} = 1

UW-Madison (Statistics)  Stat 609 Lecture 3  2014  11 / 13

beamer-tu-logo

Finding the pdf, given a cdf

If F_{X} is differentiable everywhere, then f_{X} = F′_{X}. What if F_{X} is differentiable except for some (at most countably many) points? f_{X}(x) = F′_{X}(x) for x at which F_{X} is differentiable; f_{X}(x) can be any c ≥ 0 for x at which F_{X} is not differentiable.

Example

F_{X}(x) = \{ \begin{matrix}0 & x < 0 \\ x/2 & 0 ≤ x < 1/2 \\ x^{2} & 1/2 ≤ x < 1 \\ 1 & x ≥ 1 \end{matrix}

f_{X}(x) =? F_{X} is differentiable except at 0, 1/2 and 1.

f_{X}(x) = \{ \begin{matrix}0 & x < 0 \\ 1/2 & 0 ≤ x < 1/2 \\ 2x & 1/2 ≤ x < 1 \\ 0 & x ≥ 1 \end{matrix}

beamer-tu-logo

Finding the pdf, given a cdf

If F_{X} is differentiable everywhere, then f_{X} = F′_{X}. What if F_{X} is differentiable except for some (at most countably many) points? f_{X}(x) = F′_{X}(x) for x at which F_{X} is differentiable; f_{X}(x) can be any c ≥ 0 for x at which F_{X} is not differentiable.

Example

F_{X}(x) = \{ \begin{matrix}0 & x < 0 \\ x/2 & 0 ≤ x < 1/2 \\ x^{2} & 1/2 ≤ x < 1 \\ 1 & x ≥ 1 \end{matrix}

f_{X}(x) =? F_{X} is differentiable except at 0, 1/2 and 1.

f_{X}(x) = \{ \begin{matrix}0 & x < 0 \\ 1/2 & 0 ≤ x < 1/2 \\ 2x & 1/2 ≤ x < 1 \\ 0 & x ≥ 1 \end{matrix}

beamer-tu-logo

Example.

What values of θ and β will make f (x) a pdf?

f (x) = βe^{θx−|x|} x ∈ R

If θ ≥ 1, then lim_{x→∞} e^{θx−|x|} = ∞; if θ ≤ −1, then lim_{x→−∞} e^{θx−|x|} = ∞; hence, θ has to be in (−1,1). For θ ∈ (−1,1),

∫_{−∞}^{∞} e^{θx−|x|} dx = ∫_{−∞}^{0} e^{θx+x}dx + ∫_{0}^{∞} e^{θx−x} dx = \frac{1}{1 + θ} + \frac{1}{1 − θ} = \frac{2}{1 − θ^{2}}

Hence, in orde to have a pdf, we must have β = (1 − θ^{2})/2. What is the cdf of f ? For x ≤ 0,

F(x) = ∫_{−∞}^{x} f (t)dt = \frac{1 − θ^{2}}{2} ∫_{−∞}^{x} e^{θt+t}dt = \frac{1 − θ}{2} e^{(1+θ)x}

For x > 0,

F(x) = ∫_{−∞}^{x} f (t)dt = \frac{1 − θ^{2}}{2} [∫_{−∞}^{0} e^{θt+t}dt + ∫_{0}^{x} e^{θt−t}dt] = 1− \frac{1 + θ}{2} e^{(1−θ)x}

UW-Madison (Statistics)  Stat 609 Lecture 3  2014  13 / 13