beamer-tu-logo

Lecture 19: Multivariate transformation

In Chapter 2, we considered a single transformation of a single random variable. We now consider a vector of transformations of a random vector.

The pdf of a multivariate transformation

Let X be a k-dimensional random vector with a joint pdf f_{X} and let Y = g(X), where g is a Borel function from R^{k} to R^{k} (so that Y is a k-dimensional random vector). Let A_{1},...,A_{m} be disjoint sets in B^{k} such that P(X ∈ A_{1} ∪ ··· ∪ A_{m}) = 1 and g on A_{j} is one-to-one with a nonvanishing Jacobian, i.e., the determinant |∂g(x)/∂x| ≠ 0 on A_{j}, j = 1,...,m. Then Y has the following joint pdf

f_{Y}(x) = ∑_{j=1}^{m}| |∂h_{j}(x)/∂x||f_{X}(h_{j}(x)),

where h_{j} is the inverse function of g on A_{j}, and ||∂h_{j}(x)/∂x|| is the absolute value of the determinant of the matrix ∂h_{j}(x)/∂x, j = 1,...,m.

beamer-tu-logo

Example

Let X = (X_{1},X_{2}) be a 2-dimensional random vector with joint pdf f_{X}. Consider first the transformation u = g(x_{1},x_{2}) = (u_{1},u_{2}) = (x_{1},x_{1} + x_{2}).

\frac{∂u}{∂x} = ( \begin{matrix}\frac{∂u_{1}}{∂x_{1}} & \frac{∂u_{1}}{∂x_{2}} \\ \frac{∂u_{2}}{∂x_{1}} & \frac{∂u_{2}}{∂x_{2}} \end{matrix} ) = ( \begin{matrix}1 & 0 \\ 1 & 1 \end{matrix} )  | \frac{∂u}{∂x} | = 1  | \frac{∂x}{∂u} | = 1

Using the transformation theorem, the joint pdf of U = g(X) is

f_{U}(u_{1},u_{2}) = f_{X}(u_{1},u_{2} − u_{1}),

The marginal pdf of U_{2} = X_{1} + X_{2} is then

f_{U_{2}}(u_{2}) = ∫_{−∞}^{∞} f_{X}(u_{1},u_{2} − u_{1})du_{1}.

The same result has been derived previously. Next, consider the transformation v = g(x_{1},x_{2}) = (v_{1},v_{2}) = (x_{1}/x_{2},x_{2}).

\frac{∂v}{∂x} = ( \begin{matrix}\frac{∂v_{1}}{∂x_{1}} & \frac{∂v_{1}}{∂x_{2}} \\ \frac{∂v_{2}}{∂x_{1}} & \frac{∂v_{2}}{∂x_{2}} \end{matrix} ) = ( \begin{matrix}\frac{1}{x_{2}} & −\frac{x_{1}}{x_{2}^{2}} \\ 0 & 1 \end{matrix} )  | \frac{∂v}{∂x} | = \frac{1}{x_{2}}  | \frac{∂x}{∂v} | = v_{2}

beamer-tu-logo

Using the transformation theorem, the joint pdf of V = g(X) is

f_{V}(v_{1},v_{2}) = |v_{2}|f_{X}(v_{1}v_{2},v_{2}),

The marginal pdf of V_{1} = X_{1}/X_{2} is

f_{V_{1}}(v_{1}) = ∫_{−∞}^{∞} |v_{2}|f_{X}(v_{1}v_{2},v_{2})dv_{2}.

In particular, if X_{1} and X_{2} are independent, then

f_{V_{1}}(z) = ∫_{−∞}^{∞} |v_{2}|f_{X_{1}}(v_{1}v_{2})f_{X_{2}}(v_{2})dv_{2}

Finally, consider the transformation w =g(x_{1},x_{2})=(w_{1},w_{2})=(x_{1}x_{2},x_{2}).

\frac{∂w}{∂x} = ( \begin{matrix}\frac{∂w_{1}}{∂x_{1}} & \frac{∂w_{1}}{∂x_{2}} \\ \frac{∂w_{2}}{∂x_{1}} & \frac{∂w_{2}}{∂x_{2}} \end{matrix} ) = ( \begin{matrix}x_{2} & x_{1} \\ 0 & 1 \end{matrix} )  | \frac{∂w}{∂x} | = x_{2}  | \frac{∂x}{∂w} | = \frac{1}{w_{2}}

Using the transformation theorem, the joint pdf of W = g(X) is

f_{W}(w_{1},w_{2}) = |w_{2}|^{−1} f_{X}(w_{1}w_{2}^{−1} ,w_{2}),

The marginal pdf of W_{1} = X_{1}X_{2} is

UW-Madison (Statistics)  Stat 609 Lecture 19  2014  3 / 11

beamer-tu-logo

f_{W_{1}}(w_{1}) = ∫_{−∞}^{∞} |w_{2}|^{−1} f_{X}(w_{1}w_{2}^{−1} ,w_{2})dw_{2}.

In particular, if X_{1} and X_{2} are independent, then f_{W_{1}}(w_{1}) = ∫_{−∞}^{∞} |w_{2}|^{−1} f_{X_{1}}(w_{1}w_{2}^{−1} )f_{X_{2}}(w_{2})dw_{2}

A number of results can be derived from these results.

Ratio of two normal random variables

If X_{1} and X_{2} are independent and both have the normal distribution N(0,1), then, the pdf of V_{1} = X_{1}/X_{2} is

f_{V_{1}}(v_{1}) = ∫_{−∞}^{∞} |v_{2}|\frac{e^{−(v_{1}v_{2})^{2}/2}}{\sqrt{2π}} \frac{e^{−v_{2}^{2}/2}}{\sqrt{2π}} dv_{2} = \frac{1}{2π} ∫_{−∞}^{∞} |v_{2}|e^{−(1+v_{1}^{2})v_{2}^{2}/2} dv_{2}
= \frac{1}{π} ∫_{0}^{∞} v_{2}e^{−(1+v_{1}^{2})v_{2}^{2}/2} dv_{2} = \frac{1}{π} ∫_{0}^{∞} e^{−(1+v_{1}^{2})s} ds
= \frac{1}{π(1 + v_{1}^{2})}

beamer-tu-logo

f_{W_{1}}(w_{1}) = ∫_{−∞}^{∞} |w_{2}|^{−1} f_{X}(w_{1}w_{2}^{−1} ,w_{2})dw_{2}.

In particular, if X_{1} and X_{2} are independent, then f_{W_{1}}(w_{1}) = ∫_{−∞}^{∞} |w_{2}|^{−1} f_{X_{1}}(w_{1}w_{2}^{−1} )f_{X_{2}}(w_{2})dw_{2}

A number of results can be derived from these results.

Ratio of two normal random variables

If X_{1} and X_{2} are independent and both have the normal distribution N(0,1), then, the pdf of V_{1} = X_{1}/X_{2} is

f_{V_{1}}(v_{1}) = ∫_{−∞}^{∞} |v_{2}|\frac{e^{−(v_{1}v_{2})^{2}/2}}{\sqrt{2π}} \frac{e^{−v_{2}^{2}/2}}{\sqrt{2π}} dv_{2} = \frac{1}{2π} ∫_{−∞}^{∞} |v_{2}|e^{−(1+v_{1}^{2})v_{2}^{2}/2} dv_{2}
= \frac{1}{π} ∫_{0}^{∞} v_{2}e^{−(1+v_{1}^{2})v_{2}^{2}/2} dv_{2} = \frac{1}{π} ∫_{0}^{∞} e^{−(1+v_{1}^{2})s} ds
= \frac{1}{π(1 + v_{1}^{2})}

beamer-tu-logo

Ratio of chi-square random variables and F-distribution

Let X_{1} and X_{2} be independent random variables having the chi-square distributions with degrees of freedom n_{1} and n_{2}, respectively. By the transformation theorem, the p.d.f. of Z = X_{1}/X_{2} is

f_{Z}(z) = ∫_{0}^{∞} x_{2} \frac{(zx_{2})^{n_{1}/2−1} e^{−zx_{2}/2}}{2^{n_{1}/2}Γ(n_{1}/2)} \frac{x_{2}^{n_{2}/2−1} e^{−x_{2}/2}}{2^{n_{2}/2}Γ(n_{2}/2)} dx_{2}
= \frac{z^{n_{1}/2−1}}{2^{(n_{1}+n_{2})/2}Γ(n_{1}/2)Γ(n_{2}/2)} ∫_{0}^{∞} x_{2}^{(n_{1}+n_{2})/2−1} e^{−(1+z)x_{2}/2} dx_{2}
= \frac{Γ[(n_{1} + n_{2})/2]}{Γ(n_{1}/2)Γ(n_{2}/2)} \frac{z^{n_{1}/2−1}}{(1 + z)^{(n_{1}+n_{2})/2}} ,  z > 0

where the last equality follows from the fact that

\frac{1}{2^{(n_{1}+n_{2})/2}Γ[(n_{1} + n_{2})/2]} x_{2}^{(n_{1}+n_{2})/2−1} e^{−x_{2}/2} x_{2} > 0

is the p.d.f. of the chi-square distribution with degrees of freedom n_{1} + n_{2}.

beamer-tu-logo

Making another transformation Y = (n_{2}/n_{1})Z = (X_{1}/n_{1})/(X_{2}/n_{2}) and applying the (univariate) transformation theorem, we obtain that the pdf of Y is

\frac{n_{1}^{n_{1}/2}n_{2}^{n_{2}/2}Γ[(n_{1} + n_{2})/2]}{Γ(n_{1}/2)Γ(n_{2}/2)} \frac{y^{n_{1}/2−1}}{(n_{1}z + n_{2})^{(n_{1}+n_{2})/2}} , y > 0

which is the pdf of the well-known F-distribution with degrees of freedom n_{1} and n_{2}.

t-distribution

Let U_{1} and U_{2} be independent random variables, U_{1} ∼ N(0,1) and U_{2} has the chi-square distribution with degrees of freedom n. What is the distribution of T = U_{1}/\sqrt{U_{2}/n}? Let X_{1} = U_{1}^{2} and X_{2} = U_{2}. Then X_{1} and X_{2} are independent, because U_{1} and U_{2} are independent. By a result obtained previously, X_{1} has the chi-square distribution with degree of freedom 1. From the previous proof, Y = X_{1}/(X_{2}/n) has the F-distribution with degrees of freedom 1 and n.

UW-Madison (Statistics)  Stat 609 Lecture 19  2014  6 / 11

beamer-tu-logo

Making another transformation Y = (n_{2}/n_{1})Z = (X_{1}/n_{1})/(X_{2}/n_{2}) and applying the (univariate) transformation theorem, we obtain that the pdf of Y is

\frac{n_{1}^{n_{1}/2}n_{2}^{n_{2}/2}Γ[(n_{1} + n_{2})/2]}{Γ(n_{1}/2)Γ(n_{2}/2)} \frac{y^{n_{1}/2−1}}{(n_{1}z + n_{2})^{(n_{1}+n_{2})/2}} , y > 0

which is the pdf of the well-known F-distribution with degrees of freedom n_{1} and n_{2}.

t-distribution

Let U_{1} and U_{2} be independent random variables, U_{1} ∼ N(0,1) and U_{2} has the chi-square distribution with degrees of freedom n. What is the distribution of T = U_{1}/\sqrt{U_{2}/n}? Let X_{1} = U_{1}^{2} and X_{2} = U_{2}. Then X_{1} and X_{2} are independent, because U_{1} and U_{2} are independent. By a result obtained previously, X_{1} has the chi-square distribution with degree of freedom 1. From the previous proof, Y = X_{1}/(X_{2}/n) has the F-distribution with degrees of freedom 1 and n.

UW-Madison (Statistics)  Stat 609 Lecture 19  2014  6 / 11

beamer-tu-logo

The pdf of Y is

\frac{n^{n/2}Γ[(n + 1)/2]y^{−1/2}}{\sqrt{π}Γ(n/2)(n + y)^{(n+1)/2}} y > 0

Applying the univariate transformation theorem, we obtain that the pdf of W = \sqrt{Y} is (dy = 2wdw when y = w^{2})

\frac{2n^{n/2}Γ[(n + 1)/2]}{\sqrt{π}Γ(n/2)(n + w^{2})^{(n+1)/2}} w > 0

Note that

T = \{ \begin{matrix}W & U_{1} ≥ 0 \\ −W & U_{1} < 0. \end{matrix}

and

P(T < −t) = P(T > t). t > 0.

Hence the pdf of T is

\frac{n^{n/2}Γ[(n + 1)/2]}{\sqrt{π}Γ(n/2)(n + t^{2})^{(n+1)/2}} t ∈ R

This is the pdf of the well-known t-distribution with degrees of freedom n.

UW-Madison (Statistics)  Stat 609 Lecture 19  2014  7 / 11

beamer-tu-logo

Example 4.3.3 (product of two beta random variables)

Let X ∼ beta(α,β) and Y ∼ beta(α + β,γ) be independent so that the joint pdf is

f_{X,Y}(y,x) = \frac{Γ(α + β)}{Γ(α)Γ(β)} x^{α−1} (1 − x)^{β−1} \frac{Γ(α + β + γ)}{Γ(α + β)Γ(γ)} y^{α+β−1} (1 − y)^{γ−1}

0 < x < 1, 0 < y < 1

Consider U = XY and V = X. An application of the early result for product gives that

f_{U}(u) = ∫_{−∞}^{∞} |v|^{−1} f (uv^{−1} ,v)dv

But we need to be careful when the pdf is 0 in some intervals. Note that the range of (U,V ) is the region 0 < u < v < 1 (why?). Thus,

f_{U}(u) = \frac{Γ(α + β + γ)}{Γ(α)Γ(β)Γ(γ)} ∫_{u}^{1}v^{α−1} (1 − v)^{β−1} ( \frac{u}{v})^{α+β−1} ( 1 − \frac{u}{v}) ^{γ−1} \frac{1}{v}dv

for 0 < u < 1 (f_{U}(u) = 0 when u < 0 or u > 1.)

UW-Madison (Statistics)  Stat 609 Lecture 19  2014  8 / 11

beamer-tu-logo

Making the change of variable s = (u/v − u)/(1 − u) so that dy = −u/[v^{2}(1 − u)], s = 0 when v = 1 and s = 1 when v = u, we obtain that

f_{U}(u) = \frac{Γ(α + β + γ)}{Γ(α)Γ(β)Γ(γ)} u^{α−1} (1 − u)^{β+γ−1} ∫_{u}^{1}( \frac{u}{v} − u) ^{β−1} ( 1 − \frac{u}{v}) ^{γ−1} \frac{u}{v^{2}}dv
= \frac{Γ(α + β + γ)}{Γ(α)Γ(β)Γ(γ)} u^{α−1} (1 − u)^{β+γ−1} ∫_{0}^{1}s^{β−1} (1 − s)^{γ−1} ds
= \frac{Γ(α + β + γ)}{Γ(α)Γ(β)Γ(γ)} u^{α−1} (1 − u)^{β+γ−1} \frac{Γ(β)Γ(γ)}{Γ(β + γ)}
= \frac{Γ(α + β + γ)}{Γ(α)Γ(β + γ)} u^{α−1} (1 − u)^{β+γ−1}  0 < u < 1

Thus, the product XY ∼ beta(α,β + γ).

Example

If X_{i} ∼ N(0,1), i = 1,...,n, are independent, we want to show that, for every i = 1,...,n, the random variables U = X_{1}^{2} + ··· + X_{n}^{2} and V_{i} = X_{i}^{2}/U are independent, and we want to find the pdf of V_{i}.

UW-Madison (Statistics)  Stat 609 Lecture 19  2014  9 / 11

beamer-tu-logo

The joint pdf of X_{1},...,X_{n} is

f_{X}(x_{1},...,x_{n}) = \frac{1}{(\sqrt{2π})^{n}} exp( −\frac{1}{2} ∑_{i=1}^{n}x_{i}^{2}) x_{i} ∈ R, i = 1,...,n

For Y_{i} = X_{i}^{2}, i = 1,...,n, an application of the transformation theorem shows that the joint pdf of Y_{1},...,Y_{n} is

f_{Y}(y_{1},...,y_{n}) = \frac{1}{2^{n/2}\sqrt{y_{1}··· y_{n}}} exp( −\frac{1}{2} ∑_{i=1}^{n}y_{i}) y_{i} > 0, i = 1,...,n.

Let U = Y_{1} + ··· + Y_{n} and V_{i} = Y_{i}/U, i = 1,...,n. Then Y_{i} = UV_{i}, i = 1,...,n − 1, and Y_{n} = UV_{n} = U(1 − V_{1} − ··· − V_{n−1} ).

| \frac{∂(y_{1},...,y_{n})}{∂(u,v_{1},...,v_{n−1} )}| = | \begin{matrix}v_{1} & u & 0 & ··· & 0 \\ v_{2} & 0 & u & ··· & 0 \\ ··· & ··· & ··· & ··· & ··· \\ v_{n−1} & 0 & 0 & ··· & u \\ v_{n} & 0 & 0 & ··· & 0 \end{matrix}| = v_{n}u^{n−1}

where v_{n} = 1 − v_{1} − ··· − v_{n−1} .

UW-Madison (Statistics)  Stat 609 Lecture 19  2014  10 / 11

beamer-tu-logo

The joint pdf for U and V_{1},...,V_{n−1} is

f_{Y}(uv_{1},...,uv_{n})v_{n}u^{n−1} = \frac{1}{2^{n/2}} u^{n/2−1} e^{−u/2} \sqrt{\frac{1 − v_{1} − ··· − v_{n−1}}{v_{1}··· v_{n−1}}}

Therefore, we know that U and (V_{1},...,V_{n−1} ) are independent and, hence, U and V_{i} are independent for any i = 1,...,n − 1. Since V_{n} = 1 − V_{1} − ··· − V_{n−1} , U and V_{n} are independent. If W = U − X_{i}^{2}, then

V_{i} = \frac{X_{i}^{2}}{U} = \frac{X_{i}^{2}}{W + X_{i}^{2}} = \frac{X_{i}^{2}/W}{1 + X_{i}^{2}/W}

Previously we showed that Z = X_{i}^{2}/W has pdf

\frac{Γ(n/2)}{Γ(1/2)Γ((n − 1)/2)} \frac{z^{−1/2}}{(1 + z)^{(n−1)/2}} , z > 0

Then the transformation V_{i} = Z/(1 + Z) gives the following pdf for V_{i}:

\frac{Γ(n/2)}{Γ(1/2)Γ((n − 1)/2)} v^{−1/2} (1 − v)^{(n−3)/2} , 0 < v < 1

which is beta(1/2,(n − 1)/2).

UW-Madison (Statistics)  Stat 609 Lecture 19  2014  11 / 11