beamer-tu-logo

Chapter 4: Multiple Random Variables Lecture 14: Joint and marginal distributions

So far we focus on the distribution of a single random variable. In applications we need to consider a set of random variables jointly. In some cases we study relationships among random variables.

Definition 4.1.1.

For an integer n, X = (X_{1},...,X_{n}) is called an n-dimensional random vector iff each X_{i} is a random variable.

Joint cdf’s

The joint cdf of an n-dimensional random vector X is a function F_{X} in R^{n} such that

F_{X}(x_{1},...,x_{n}) = P ( ⋂_{i=1}^{n}{X_{i} ≤ x_{i}})  x_{i} ∈ R
= P(X_{1} ≤ x_{1},...,X_{n} ≤ x_{n}) i = 1,...,n
UW-Madison (Statistics)  Stat 609 Lecture 14  2014  1 / 13

beamer-tu-logo

Properties of cdf’s

F_{X} is nondecreasing and right-continuous in any of its n arguments. For any i = 1,...,n and fixed x_{1},...,x_{i−1} ,x_{i+1},...,x_{n},

lim_{x_{i}→−∞} F(x_{1},...,x_{i−1} ,x_{i},x_{i+1},...,x_{n}) = 0

lim_{x_{i}→∞, i=1,...,n} F(x_{1},...,x_{n}) = 1

Marginal cdf’s

For a random vector X = (X_{1},...,X_{n}) and any i, the cdf of X_{i} is called the marginal cdf of X_{i} and is equal to

F_{X_{i}}(x_{i}) = lim_{x_{j}→∞, j=1,...,i−1,i+1,...,n} F_{X}(x_{1},...,x_{n})

Knowing the joint cdf F_{X} we can obtain n marginal cdf’s, but in general, knowing F_{X_{1}},...,F_{X_{n}} is not enough to determine the joint cdf F_{X}.

Similar to the univariate case, we mainly consider two types of random vectors, discrete random vectors and continuous random vectors.

UW-Madison (Statistics)  Stat 609 Lecture 14  2014  2 / 13

beamer-tu-logo

Properties of cdf’s

F_{X} is nondecreasing and right-continuous in any of its n arguments. For any i = 1,...,n and fixed x_{1},...,x_{i−1} ,x_{i+1},...,x_{n},

lim_{x_{i}→−∞} F(x_{1},...,x_{i−1} ,x_{i},x_{i+1},...,x_{n}) = 0

lim_{x_{i}→∞, i=1,...,n} F(x_{1},...,x_{n}) = 1

Marginal cdf’s

For a random vector X = (X_{1},...,X_{n}) and any i, the cdf of X_{i} is called the marginal cdf of X_{i} and is equal to

F_{X_{i}}(x_{i}) = lim_{x_{j}→∞, j=1,...,i−1,i+1,...,n} F_{X}(x_{1},...,x_{n})

Knowing the joint cdf F_{X} we can obtain n marginal cdf’s, but in general, knowing F_{X_{1}},...,F_{X_{n}} is not enough to determine the joint cdf F_{X}.

Similar to the univariate case, we mainly consider two types of random vectors, discrete random vectors and continuous random vectors.

UW-Madison (Statistics)  Stat 609 Lecture 14  2014  2 / 13

beamer-tu-logo

Properties of cdf’s

F_{X} is nondecreasing and right-continuous in any of its n arguments. For any i = 1,...,n and fixed x_{1},...,x_{i−1} ,x_{i+1},...,x_{n},

lim_{x_{i}→−∞} F(x_{1},...,x_{i−1} ,x_{i},x_{i+1},...,x_{n}) = 0

lim_{x_{i}→∞, i=1,...,n} F(x_{1},...,x_{n}) = 1

Marginal cdf’s

For a random vector X = (X_{1},...,X_{n}) and any i, the cdf of X_{i} is called the marginal cdf of X_{i} and is equal to

F_{X_{i}}(x_{i}) = lim_{x_{j}→∞, j=1,...,i−1,i+1,...,n} F_{X}(x_{1},...,x_{n})

Knowing the joint cdf F_{X} we can obtain n marginal cdf’s, but in general, knowing F_{X_{1}},...,F_{X_{n}} is not enough to determine the joint cdf F_{X}.

Similar to the univariate case, we mainly consider two types of random vectors, discrete random vectors and continuous random vectors.

UW-Madison (Statistics)  Stat 609 Lecture 14  2014  2 / 13

beamer-tu-logo

Definition 4.1.3 (Discrete joint pmf)

A random vector X = (X_{1},...,X_{n}) is discrete iff each X_{i} is discrete. The joint pmf of X is

f_{X}(x_{1},...,x_{n}) = P(X_{1} = x_{1},...,X_{n} = x_{n}),

which is positive only for countably many (x_{1},...,x_{n}) ∈ R^{n}.

For any event A ⊂ R^{n},

P(X ∈ A) = ∑_{(x_{1},...,x_{n})∈A}f_{X}(x_{1},...,x_{n})

For any i, the marginal pmf of X_{i} is

f_{X_{i}}(x_{i}) = ∑_{x_{1},...,x_{i−1} ,x_{i+1},...,x_{n}}f_{X}(x_{1},...,x_{n})

For any function g(x_{1},...,x_{n}), the expected value of g(X_{1},...,X_{n}) is

E[g(X_{1},...,X_{n})] = ∑_{x_{1},...,x_{n}}g(x_{1},...,x_{n})f_{X}(x_{1},...,x_{n})

If n = 2 and each X_{i} takes finitely many values, then the joint and marginal pmf’s can be listed in a 2 × 2 table.

UW-Madison (Statistics)  Stat 609 Lecture 14  2014  3 / 13

beamer-tu-logo

Definition 4.1.3 (Discrete joint pmf)

A random vector X = (X_{1},...,X_{n}) is discrete iff each X_{i} is discrete. The joint pmf of X is

f_{X}(x_{1},...,x_{n}) = P(X_{1} = x_{1},...,X_{n} = x_{n}),

which is positive only for countably many (x_{1},...,x_{n}) ∈ R^{n}.

For any event A ⊂ R^{n},

P(X ∈ A) = ∑_{(x_{1},...,x_{n})∈A}f_{X}(x_{1},...,x_{n})

For any i, the marginal pmf of X_{i} is

f_{X_{i}}(x_{i}) = ∑_{x_{1},...,x_{i−1} ,x_{i+1},...,x_{n}}f_{X}(x_{1},...,x_{n})

For any function g(x_{1},...,x_{n}), the expected value of g(X_{1},...,X_{n}) is

E[g(X_{1},...,X_{n})] = ∑_{x_{1},...,x_{n}}g(x_{1},...,x_{n})f_{X}(x_{1},...,x_{n})

If n = 2 and each X_{i} takes finitely many values, then the joint and marginal pmf’s can be listed in a 2 × 2 table.

UW-Madison (Statistics)  Stat 609 Lecture 14  2014  3 / 13

beamer-tu-logo

Values and joint and marginal pmf’s of a 2-dimensional random vector

value x_{21} x_{22} ··· x_{2c} marginal
x_{11}  p_{11} p_{12} ··· p_{1c}  p_{1·}
x_{12}  p_{21} p_{22} ··· p_{2c}  p_{2·}
··· ··· ··· ··· ··· ···
x_{1r} p_{r1} p_{r2} ··· p_{rc} p_{r·}
marginal p_{·1} p_{·2} ··· p_{·c}  1
p_{ij} = P(X_{1} = i,X_{2} = j), p_{i·}= P(X_{1} = i), p_{·j}(X_{2} = j).

Example: Multinomial distribution

An experiment has r possible outcomes A_{1},...,A_{r} with P(A_{i}) = p_{i}, i = 1,...,r, p_{1} + ··· + p_{r} = 1. We independently repeat the experiment n times. If X_{i} is the number of times A_{i} is the result in n experiments, i = 1,...,r, then X = (X_{1},...,X_{r}) has joint pmf

P(X_{1} = x_{1},...,X_{r} = x_{r}) = \frac{n!}{x_{1}!··· x_{r}!}p_{1}^{x_{1}} ··· p_{r}^{x_{r}} 0 ≤ x_{j}, ∑_{i=1}^{r}x_{i} = n

beamer-tu-logo

Values and joint and marginal pmf’s of a 2-dimensional random vector

value x_{21} x_{22} ··· x_{2c} marginal
x_{11}  p_{11} p_{12} ··· p_{1c}  p_{1·}
x_{12}  p_{21} p_{22} ··· p_{2c}  p_{2·}
··· ··· ··· ··· ··· ···
x_{1r} p_{r1} p_{r2} ··· p_{rc} p_{r·}
marginal p_{·1} p_{·2} ··· p_{·c}  1
p_{ij} = P(X_{1} = i,X_{2} = j), p_{i·}= P(X_{1} = i), p_{·j}(X_{2} = j).

Example: Multinomial distribution

An experiment has r possible outcomes A_{1},...,A_{r} with P(A_{i}) = p_{i}, i = 1,...,r, p_{1} + ··· + p_{r} = 1. We independently repeat the experiment n times. If X_{i} is the number of times A_{i} is the result in n experiments, i = 1,...,r, then X = (X_{1},...,X_{r}) has joint pmf

P(X_{1} = x_{1},...,X_{r} = x_{r}) = \frac{n!}{x_{1}!··· x_{r}!}p_{1}^{x_{1}} ··· p_{r}^{x_{r}} 0 ≤ x_{j}, ∑_{i=1}^{r}x_{i} = n

beamer-tu-logo

Multivariate hypergeometric distribution

A bag contains N balls with r different colors. N_{i} = the number of balls for color i, N_{1} + ··· + N_{r} = N. We randomly select n balls from the bag. If X_{i} is the number of selected balls having color i, i = 1,...,r, then X = (X_{1},...,X_{r}) has joint pmf

P(X_{1} = x_{1},...,X_{r} = x_{r}) = \binom{N_{1}}{x_{1}}··· \binom{N_{r}}{x_{r}} ; \binom{N}{n} 0 ≤ x_{j}, ∑_{i=1}^{r}x_{i} = n

What are the marginal pmf’s for the multinomial and multivariate hypergeometric distributions?

Definition 4.1.10 (continuous pdf)

A random vector X = (X_{1},...,X_{n}) has a continuous joint pdf if there exists a nonnegative function f_{X} on R^{n} such that

P(X ∈ A) = ∫ ··· ∫ _{(x_{1},...,x_{n})∈A}f_{X}(x_{1},...,x_{n})dx_{1}··· dx_{n}

beamer-tu-logo

Multivariate hypergeometric distribution

A bag contains N balls with r different colors. N_{i} = the number of balls for color i, N_{1} + ··· + N_{r} = N. We randomly select n balls from the bag. If X_{i} is the number of selected balls having color i, i = 1,...,r, then X = (X_{1},...,X_{r}) has joint pmf

P(X_{1} = x_{1},...,X_{r} = x_{r}) = \binom{N_{1}}{x_{1}}··· \binom{N_{r}}{x_{r}} ; \binom{N}{n} 0 ≤ x_{j}, ∑_{i=1}^{r}x_{i} = n

What are the marginal pmf’s for the multinomial and multivariate hypergeometric distributions?

Definition 4.1.10 (continuous pdf)

A random vector X = (X_{1},...,X_{n}) has a continuous joint pdf if there exists a nonnegative function f_{X} on R^{n} such that

P(X ∈ A) = ∫ ··· ∫ _{(x_{1},...,x_{n})∈A}f_{X}(x_{1},...,x_{n})dx_{1}··· dx_{n}

beamer-tu-logo

Multivariate hypergeometric distribution

A bag contains N balls with r different colors. N_{i} = the number of balls for color i, N_{1} + ··· + N_{r} = N. We randomly select n balls from the bag. If X_{i} is the number of selected balls having color i, i = 1,...,r, then X = (X_{1},...,X_{r}) has joint pmf

P(X_{1} = x_{1},...,X_{r} = x_{r}) = \binom{N_{1}}{x_{1}}··· \binom{N_{r}}{x_{r}} ; \binom{N}{n} 0 ≤ x_{j}, ∑_{i=1}^{r}x_{i} = n

What are the marginal pmf’s for the multinomial and multivariate hypergeometric distributions?

Definition 4.1.10 (continuous pdf)

A random vector X = (X_{1},...,X_{n}) has a continuous joint pdf if there exists a nonnegative function f_{X} on R^{n} such that

P(X ∈ A) = ∫ ··· ∫ _{(x_{1},...,x_{n})∈A}f_{X}(x_{1},...,x_{n})dx_{1}··· dx_{n}

beamer-tu-logo

If X = (X_{1},...,X_{n}) has joint pdf f_{X}, a short notation is

P(X ∈ A) = ∫_{A}f_{X}(x)dx x = (x_{1},...,x_{n})

If the joint pdf f_{X} exists, then

F_{X}(x_{1},...,x_{n}) = ∫_{−∞}^{x_{1}} ··· ∫_{−∞}^{x_{n}}f_{X}(t_{1},...,t_{n})dt_{1}··· dt_{n}

If the joint cdf F_{X} is differentiable, then the joint f_{X} exists and

f_{X}(x_{1},...,x_{n}) = \frac{∂^{n}F_{X}(x_{1},...,x_{n})}{∂x_{1}··· ∂x_{n}}, (x_{1},...,x_{n}) ∈ R^{n}

If the joint pdf f_{X} exists, then the ith margianl pdf f_{X_{i}} exists and

f_{X_{i}}(x_{i}) = ∫_{−∞}^{∞} ··· ∫_{−∞}^{∞} f_{X}(x_{1},...,x_{n})dx_{1}··· dx_{i−1} dx_{i+1}··· dx_{n}

If the joint pdf f_{X} exists, then for any function g(x_{1},...,x_{n}), the expected value of g(X_{1},...,X_{n}) is

E[g(X_{1},...,X_{n})] = ∫_{−∞}^{∞} ··· ∫_{−∞}^{∞} g(x_{1},...,x_{n})f_{X}(x_{1},...,x_{n})dx_{1}··· dx_{n}

beamer-tu-logo

Example

Suppose that a 2-dimensional random vector (X,Y ) has pdf

f (x,y) = \{ \begin{matrix}Ce^{−(2x+3y)} & x ≥ 0,y ≥ 0 \\ 0 & otherwise \end{matrix}

What should C be? Since

∫_{−∞}^{∞} ∫_{−∞}^{∞} f (x,y)dxdy = ∫_{0}^{∞} ∫_{0}^{∞} Ce^{−(2x+3y)} dxdy
= C ∫_{0}^{∞} e^{−2x} dx ∫_{0}^{∞} e^{−3y} dy
= C ( \frac{e^{−2x}}{2} |_{∞}^{0} \frac{e^{−3y}}{3} |_{∞}^{0} ) = \frac{C}{6}

must be 1, we obtain that C = 6. The joint cdf of (X,Y ) is

F(x,y) = \{ \begin{matrix}(1 − e^{−2x} )(1 − e^{−3y} ) & x ≥ 0,y ≥ 0 \\ 0 & otherwise \end{matrix}

UW-Madison (Statistics)  Stat 609 Lecture 14  2014  7 / 13

beamer-tu-logo

This is because, for x > 0 and y > 0,

F(x,y) = ∫_{0}^{∞} ∫_{0}^{∞} 6e^{−(2x+3y)} dxdy = 6∫_{0}^{∞} e^{−2x} dx ∫_{0}^{∞} 6e^{−3y} dy
= (1 − e^{−2x} )(1 − e^{−3y} )

Calculate P(2X + 3Y ≤ 6). It is not convenient to use the joint cdf. Using the joint pdf, we obtain

P(2X + 3Y ≤ 6) = ∫_{2x+3y≤6} f (x,y)dxdy

beamer-tu-logo

= 6∫_{0}^{3}[∫_{0}^{(6−2x)/3} e^{−(2x+3y)} dy] dx = 6∫_{0}^{3}e^{−2x} [−\frac{e^{−3y}}{3} |_{0}^{(6−2x)/3} ] dx
= 6∫_{0}^{3}e^{−2x} [ \frac{1}{3} − \frac{e^{−(6−2x)}}{3} ] dx = 2∫_{0}^{3}(e^{−2x} − e^{−6} )dx
= −e^{−2x} |_{0}^{3} − 2e^{−6} × 3 = 1 − e^{−6} − 6e^{−6}
= 1 − 7e^{−6}

Example 4.1.12.

Suppose that a 2-dimensional random vector (X,Y ) has pdf

f (x,y) = \{ \begin{matrix}e^{−y} & 0 < x < y \\ 0 & otherwise \end{matrix}

We want to calculate P(X + Y ≥ 1).

beamer-tu-logo

= 6∫_{0}^{3}[∫_{0}^{(6−2x)/3} e^{−(2x+3y)} dy] dx = 6∫_{0}^{3}e^{−2x} [−\frac{e^{−3y}}{3} |_{0}^{(6−2x)/3} ] dx
= 6∫_{0}^{3}e^{−2x} [ \frac{1}{3} − \frac{e^{−(6−2x)}}{3} ] dx = 2∫_{0}^{3}(e^{−2x} − e^{−6} )dx
= −e^{−2x} |_{0}^{3} − 2e^{−6} × 3 = 1 − e^{−6} − 6e^{−6}
= 1 − 7e^{−6}

Example 4.1.12.

Suppose that a 2-dimensional random vector (X,Y ) has pdf

f (x,y) = \{ \begin{matrix}e^{−y} & 0 < x < y \\ 0 & otherwise \end{matrix}

We want to calculate P(X + Y ≥ 1).

beamer-tu-logo

Note that

P(X + Y < 1) = ∫_{x+y<1} f (x,y)dxdy = ∫_{0}^{1/2} ( ∫_{x}^{1−x} e^{−y} dy) dx
= ∫_{0}^{1/2}(e^{−x} − e^{−(1−x)} )dx = 1 − e^{−1/2} − e^{−1/2} + e^{−1}
= 1 + e^{−1} − 2e^{−1/2}

Hence,

P(X + Y ≥ 1) = 1 − P(X + Y < 1) = 2e^{−1/2} − e^{−1}

beamer-tu-logo

Example

Let

f (x,y) = \{ \begin{matrix}\frac{x^{α−1} (y−x)^{β−1} e^{−y}}{Γ(α)Γ(β)} & 0 < x < y \\ 0 & otherwise \end{matrix}

where α > 0 and β > 0 are constants. We want to show this is a pdf and find its two marginal pdf’s. For x > 0, consider

f_{X}(x) = ∫_{−∞}^{∞} f (x,y)dy = ∫_{x}^{∞} \frac{x^{α−1} (y − x)^{β−1} e^{−y}}{Γ(α)Γ(β)} dy
= \frac{x^{α−1}}{Γ(α)Γ(β)} ∫_{0}^{∞} u^{β−1} e^{−(u+x)} du  y − x = u
= \frac{x^{α−1}}{Γ(α)} e^{−x}

This is the pdf of Gamma(α,1), which also shows that f (x,y) is a pdf. The other marginal pdf is, for y > 0,

UW-Madison (Statistics)  Stat 609 Lecture 14  2014  11 / 13

beamer-tu-logo

f_{Y}(y) = ∫_{−∞}^{∞} f (x,y)dx = ∫_{0}^{y} \frac{x^{α−1} (y − x)^{β−1} e^{−y}}{Γ(α)Γ(β)} dx
= \frac{e^{−y}}{Γ(α)Γ(β)} ∫_{0}^{1}(ty)^{α−1} (y − ty)^{β−1} ydt  x = ty
= \frac{y^{α+β−1} e^{−y}}{Γ(α)Γ(β)} ∫_{0}^{1}t^{α−1} (1 − t)^{β−1} dt = \frac{y^{α+β−1} e^{−y}}{Γ(α + β)}

which is the pdf of Gamma(α + β,1).

Example

Let F(x) and G(y) be univariate cdf’s with pdf’s f (x) and g(y), respectively, and α be a constant with |α| ≤ 1. We want to show that

H(x,y) = F(x)G(y){1 + α[1 − F(x)][1 − G(y)]} (x,y) ∈ R^{2}

is a 2-dimensional cdf with pdf

h(x,y) = f (x)g(y){1 + α[2F(x) − 1][2G(y) − 1]} (x,y) ∈ R^{2}

beamer-tu-logo

Since |α[2F(x) − 1][2G(y) − 1]| ≤ |[2F(x) − 1][2G(y) − 1]| ≤ 1

we know that h(x,y) ≥ 0. For any (x,y) ∈ R^{2},

∫_{−∞}^{x} ∫_{−∞}^{y} h(u,v)dvdu
= ∫_{−∞}^{x} ∫_{−∞}^{y} f (u)g(v){1 + α[2F(u) − 1][2G(v) − 1]}dvdu
= F(x)G(y) + α ∫_{−∞}^{x} f (u)[2F(u) − 1]du ∫_{−∞}^{y} g(v)[2G(v) − 1]dv
= F(x)G(y) + α[2F^{2}(x) − F(x)][G^{2}(y) − G(y)]
= F(x)G(y){1 + α[1 − F(x)][1 − G(y)]}
= H(x,y)

This shows that h(x,y) is a pdf of H(x,y), if H(x,y) is a cdf. H(x,y) is indeed a cdf, because

lim_{x→∞,y→∞} H(x,y) = lim_{x→∞,y→∞} F(x)G(y) = 1
UW-Madison (Statistics)  Stat 609 Lecture 14  2014  13 / 13