beamer-tu-logo

Chapter 2: Transformations and Expectations Lecture 4: Transformations

For a random variable X, we are often interested in a transformation, Y = g(X), which is also a random variable. Here g is a function from the space for X to a new space. For an event A in the Y -space,

P(Y ∈ A) = P(g(X) ∈ A) = P(X ∈ g^{−1} (A))

where

g^{−1} (A) = {x : g(x) ∈ A}.

Here, g^{−1} is not the inverse function and g^{−1} (A) is an event.

Example: g(x) = x^{2}

If A = \{t\}, a single point with t ≥ 0, then g^{−1} (\{t\}) = \{−\sqrt{t},\sqrt{t}\}, since g(−\sqrt{t}) = (−\sqrt{t})^{2} = t and g(\sqrt{t}) = (\sqrt{t})^{2} = t. If A = [a,b] with 0 ≤ a ≤ b, then g^{−1} (A) = [−\sqrt{b},−\sqrt{a}] ∪ [\sqrt{a},\sqrt{b}].

UW-Madison (Statistics)  Stat 609 Lecture 4  2014  1 / 11

beamer-tu-logo

Chapter 2: Transformations and Expectations Lecture 4: Transformations

For a random variable X, we are often interested in a transformation, Y = g(X), which is also a random variable. Here g is a function from the space for X to a new space. For an event A in the Y -space,

P(Y ∈ A) = P(g(X) ∈ A) = P(X ∈ g^{−1} (A))

where

g^{−1} (A) = {x : g(x) ∈ A}.

Here, g^{−1} is not the inverse function and g^{−1} (A) is an event.

Example: g(x) = x^{2}

If A = \{t\}, a single point with t ≥ 0, then g^{−1} (\{t\}) = \{−\sqrt{t},\sqrt{t}\}, since g(−\sqrt{t}) = (−\sqrt{t})^{2} = t and g(\sqrt{t}) = (\sqrt{t})^{2} = t. If A = [a,b] with 0 ≤ a ≤ b, then g^{−1} (A) = [−\sqrt{b},−\sqrt{a}] ∪ [\sqrt{a},\sqrt{b}].

UW-Madison (Statistics)  Stat 609 Lecture 4  2014  1 / 11

beamer-tu-logo

How to obtain F_{Y} (or f_{Y}) using F_{X} (or f_{X})?

If X is discrete, then Y is also discrete. The pmf for Y is

f_{Y}(y) = P(Y = y) = P(g(X) = y)

= ∑_{x: g(x)=y}P(X = x) = ∑_{x: x∈g^{−1} ({y})} f_{X}(x).

For discrete X, we often can directly work out a formula for f_{Y}.

Example

Suppose X has the following distribution

\begin{matrix}x & −2 & −1 & 0 & 1 & 2 & 3 \\ f_{X}(x) & 0.1 & 0.2 & 0.1 & 0.2 & 0.3 & 0.1 \end{matrix}

Let Y = X^{2}. Then Y can take values 0, 1, 4, and 9 and the distribution of Y is

y  0 1 4 9
f_{Y}(y) 0.1 0.4 0.4 0.1

UW-Madison (Statistics)  Stat 609 Lecture 4  2014  2 / 11

beamer-tu-logo

How to obtain F_{Y} (or f_{Y}) using F_{X} (or f_{X})?

If X is discrete, then Y is also discrete. The pmf for Y is

f_{Y}(y) = P(Y = y) = P(g(X) = y)

= ∑_{x: g(x)=y}P(X = x) = ∑_{x: x∈g^{−1} ({y})} f_{X}(x).

For discrete X, we often can directly work out a formula for f_{Y}.

Example

Suppose X has the following distribution

\begin{matrix}x & −2 & −1 & 0 & 1 & 2 & 3 \\ f_{X}(x) & 0.1 & 0.2 & 0.1 & 0.2 & 0.3 & 0.1 \end{matrix}

Let Y = X^{2}. Then Y can take values 0, 1, 4, and 9 and the distribution of Y is

y  0 1 4 9
f_{Y}(y) 0.1 0.4 0.4 0.1

UW-Madison (Statistics)  Stat 609 Lecture 4  2014  2 / 11

beamer-tu-logo

Continuous random variables

If X is continuous with a pdf f_{X}, then

F_{Y}(y) = P(Y ≤ y) = P(g(X) ≤ y) = ∫_{{x: g(x)≤y}} f_{X}(x)dx

In general, the region {x : g(x) ≤ y} may be difficult to identify. Sometimes it is possible to find simple formulas for the cdf and pdf of Y in terms of the cdf and pdf of X.

Example: Y = X^{2}

If y ≤ 0, F_{Y}(y) = 0 and f_{Y}(y) = 0. If y > 0,

F_{Y}(y) = ∫_{\{x: x^{2}≤y\}} f_{X}(x)dx = ∫_{\{x: −\sqrt{y}≤x≤\sqrt{y}\}} f_{X}(x)dx
= ∫_{−\sqrt{y}}^{√ \overline{y}} f_{X}(x)dx = F_{X}(\sqrt{y}) − F_{X}(−\sqrt{y})
f_{Y}(y) = \frac{d}{dy}F_{Y}(y) = f_{X}(\sqrt{y})\frac{1}{2\sqrt{y}} + f_{X}(−\sqrt{y})\frac{1}{2\sqrt{y}}

beamer-tu-logo

Continuous random variables

If X is continuous with a pdf f_{X}, then

F_{Y}(y) = P(Y ≤ y) = P(g(X) ≤ y) = ∫_{{x: g(x)≤y}} f_{X}(x)dx

In general, the region {x : g(x) ≤ y} may be difficult to identify. Sometimes it is possible to find simple formulas for the cdf and pdf of Y in terms of the cdf and pdf of X.

Example: Y = X^{2}

If y ≤ 0, F_{Y}(y) = 0 and f_{Y}(y) = 0. If y > 0,

F_{Y}(y) = ∫_{\{x: x^{2}≤y\}} f_{X}(x)dx = ∫_{\{x: −\sqrt{y}≤x≤\sqrt{y}\}} f_{X}(x)dx
= ∫_{−\sqrt{y}}^{√ \overline{y}} f_{X}(x)dx = F_{X}(\sqrt{y}) − F_{X}(−\sqrt{y})
f_{Y}(y) = \frac{d}{dy}F_{Y}(y) = f_{X}(\sqrt{y})\frac{1}{2\sqrt{y}} + f_{X}(−\sqrt{y})\frac{1}{2\sqrt{y}}

beamer-tu-logo

Example 2.1.2: Y = sin^{2}(X)

Suppose that X has the following pdf:

f_{X}(x) = \{ \begin{matrix}\frac{1}{2π} & 0 < x < 2π \\ 0 & otherwise \end{matrix}

For 0 < y < 1, if x_{j}, j = 1,2, are solutions to sin^{2}(x) = y, 0 < x < π, then

F_{Y}(y) = P(Y ≤ y) = 2P(X ≤ x_{1}) + 2P(x_{2} ≤ X ≤ π) = 2arcsin(\sqrt{y})/π

beamer-tu-logo

If g is strictly monotone, then the inverse function g^{−1} exists and

\{x : g(x) ≤ y\} = \{ \begin{matrix}\{x : x ≤ g^{−1} (y)\} & if g is increasing \\ \{x : x ≥ g^{−1} (y)\} & if g is decreasing \end{matrix}

Theorem 2.1.3/Theorem 2.1.5

Let X be a continuous random variable and Y = g(X) with range Y . a. If g is increasing, then F_{Y}(y) = F_{X}(g^{−1} (y)), y ∈ Y . b. If g is decreasing, then F_{Y}(y) = 1 − F_{X}(g^{−1} (y)), y ∈ Y . c. If f_{X} is continuous and g is continuously differentiable, then

f_{Y}(y) = \{ \begin{matrix}f_{X}(g^{−1} (y))| \frac{d}{dy}g^{−1} (y)| & y ∈ Y \\ 0 & otherwise \end{matrix}

Example 2.1.5.

Suppose that Y = g(X), g(x) = 1/x, and X has the pdf

f_{X}(x) = [(n − 1)!β^{n}]^{−1} x^{n−1} e^{−x/β} , x > 0

Then Y = (0,∞), g^{−1} (y) = 1/y, \frac{d}{dy}g^{−1} (y) = −1/y^{2}, and

beamer-tu-logo

\{x : g(x) ≤ y\} = \{ \begin{matrix}\{x : x ≤ g^{−1} (y)\} & if g is increasing \\ \{x : x ≥ g^{−1} (y)\} & if g is decreasing \end{matrix}

Theorem 2.1.3/Theorem 2.1.5

Let X be a continuous random variable and Y = g(X) with range Y . a. If g is increasing, then F_{Y}(y) = F_{X}(g^{−1} (y)), y ∈ Y . b. If g is decreasing, then F_{Y}(y) = 1 − F_{X}(g^{−1} (y)), y ∈ Y . c. If f_{X} is continuous and g is continuously differentiable, then

f_{Y}(y) = \{ \begin{matrix}f_{X}(g^{−1} (y))| \frac{d}{dy}g^{−1} (y)| & y ∈ Y \\ 0 & otherwise \end{matrix}

Example 2.1.5.

Suppose that Y = g(X), g(x) = 1/x, and X has the pdf

f_{X}(x) = [(n − 1)!β^{n}]^{−1} x^{n−1} e^{−x/β} , x > 0

Then Y = (0,∞), g^{−1} (y) = 1/y, \frac{d}{dy}g^{−1} (y) = −1/y^{2}, and

beamer-tu-logo

\{x : g(x) ≤ y\} = \{ \begin{matrix}\{x : x ≤ g^{−1} (y)\} & if g is increasing \\ \{x : x ≥ g^{−1} (y)\} & if g is decreasing \end{matrix}

Theorem 2.1.3/Theorem 2.1.5

Let X be a continuous random variable and Y = g(X) with range Y . a. If g is increasing, then F_{Y}(y) = F_{X}(g^{−1} (y)), y ∈ Y . b. If g is decreasing, then F_{Y}(y) = 1 − F_{X}(g^{−1} (y)), y ∈ Y . c. If f_{X} is continuous and g is continuously differentiable, then

f_{Y}(y) = \{ \begin{matrix}f_{X}(g^{−1} (y))| \frac{d}{dy}g^{−1} (y)| & y ∈ Y \\ 0 & otherwise \end{matrix}

Example 2.1.5.

Suppose that Y = g(X), g(x) = 1/x, and X has the pdf

f_{X}(x) = [(n − 1)!β^{n}]^{−1} x^{n−1} e^{−x/β} , x > 0

Then Y = (0,∞), g^{−1} (y) = 1/y, \frac{d}{dy}g^{−1} (y) = −1/y^{2}, and

beamer-tu-logo

Many useful functions are not monotone. The next theorem extends Theorem 2.1.5 to the case where g is piecewise monotone.

Theorem 2.1.8.

Let X be a continuous random variable with pdf f_{X}. Suppose that there are disjoint A_{1},...,A_{k} such that P(X ∈ ∪_{t=0}^{k}A_{t}) = 1, f_{X} is continuous on each A_{t}, t = 1,...,k, and there are functions g_{1}(x),...,g_{k}(x) defined on A_{1},...,A_{k}, respectively, satisfying i. g(x) = g_{t}(x) for x ∈ A_{t}; ii. g_{t}(x) is strictly monotone on A_{t}; iii. the set Y = {y : y = g_{t}(x) for some x ∈ A_{t}} is the same for each t; iv. g_{t}^{−1} (y) has a continuous derivative on Y for each t. Then Y has the pdf

f_{Y}(y) = \{ \begin{matrix}∑_{t=1}^{k}f_{X}(g_{t}^{−1} (y))| \frac{d}{dy}g_{t}^{−1} (y)| & y ∈ Y \\ 0 & otherwise \end{matrix}

beamer-tu-logo

Many useful functions are not monotone. The next theorem extends Theorem 2.1.5 to the case where g is piecewise monotone.

Theorem 2.1.8.

Let X be a continuous random variable with pdf f_{X}. Suppose that there are disjoint A_{1},...,A_{k} such that P(X ∈ ∪_{t=0}^{k}A_{t}) = 1, f_{X} is continuous on each A_{t}, t = 1,...,k, and there are functions g_{1}(x),...,g_{k}(x) defined on A_{1},...,A_{k}, respectively, satisfying i. g(x) = g_{t}(x) for x ∈ A_{t}; ii. g_{t}(x) is strictly monotone on A_{t}; iii. the set Y = {y : y = g_{t}(x) for some x ∈ A_{t}} is the same for each t; iv. g_{t}^{−1} (y) has a continuous derivative on Y for each t. Then Y has the pdf

f_{Y}(y) = \{ \begin{matrix}∑_{t=1}^{k}f_{X}(g_{t}^{−1} (y))| \frac{d}{dy}g_{t}^{−1} (y)| & y ∈ Y \\ 0 & otherwise \end{matrix}

beamer-tu-logo

Example 2.1.9 (Y = X^{2})

Let A_{1} = (−∞,0), A_{2} = (0,∞), Y = (0,∞). On A_{1}, g_{1}(x) = x^{2} is decreasing, g_{1}^{−1} (y) = −\sqrt{y}; On A_{2}, g_{2}(x) = x^{2} is increasing, g_{2}^{−1} (y) = \sqrt{y}. By Theorem 2.1.8,

f_{Y}(y) = f_{X}(−\sqrt{y})|−\frac{1}{2\sqrt{y}} | + f_{X}(\sqrt{y})| \frac{1}{2\sqrt{y}} | y > 0

We obtained this previously. As a special case, consider

f_{X}(x) = \frac{1}{\sqrt{2π}} e^{−x^{2}/2} , x ∈ R (the standard normal)

Then, for y > 0,

f_{Y}(y) = \frac{1}{\sqrt{2π}} e^{−(−\sqrt{y})^{2}/2} |− \frac{1}{2\sqrt{y}} |+ \frac{1}{\sqrt{2π}} e^{−(\sqrt{y})^{2}/2} | \frac{1}{2\sqrt{y}} | = \frac{1}{\sqrt{2πy}} e^{−y/2} ,

which is the chi-square pdf with 1 degree of freedom.

UW-Madison (Statistics)  Stat 609 Lecture 4  2014  7 / 11

beamer-tu-logo

Theorem 2.1.10

Let X have continuous cdf F_{X} and Y = F_{X}(X). Then Y is uniformly distributed on (0,1), that is, P(Y ≤ y) = y, 0 < y < 1.

This result is useful for generating random variables. If we want to generate a random value X from a cdf F_{X}, we need only generate a uniform random number Y (from the uniform distribution on (0,1)) and solve for X in the equation F_{X}(X) = Y .

The inverse of a cdf

If F_{X} is strictly increasing, then its inverse F_{X}^{−1} is well defined by

F_{X}^{−1} (y) = x iff F_{X}(x) = y

If F_{X} is flat on an interval (x_{1},x_{2}] (see the figure), then we define

F_{X}^{−1} (y) = inf{x : F_{X}(x) ≥ y}, 0 ≤ y ≤ 1
F_{X}^{−1} is increasing in y.
F_{X}(F_{X}^{−1} (y)) = y when F_{X} is continuous at F_{X}^{−1} (y).
F_{X}^{−1} (y) ≤ x iff y ≤ F_{X}(x).
UW-Madison (Statistics)  Stat 609 Lecture 4  2014  8 / 11

beamer-tu-logo

Theorem 2.1.10

Let X have continuous cdf F_{X} and Y = F_{X}(X). Then Y is uniformly distributed on (0,1), that is, P(Y ≤ y) = y, 0 < y < 1.

This result is useful for generating random variables. If we want to generate a random value X from a cdf F_{X}, we need only generate a uniform random number Y (from the uniform distribution on (0,1)) and solve for X in the equation F_{X}(X) = Y .

The inverse of a cdf

If F_{X} is strictly increasing, then its inverse F_{X}^{−1} is well defined by

F_{X}^{−1} (y) = x iff F_{X}(x) = y

If F_{X} is flat on an interval (x_{1},x_{2}] (see the figure), then we define

F_{X}^{−1} (y) = inf{x : F_{X}(x) ≥ y}, 0 ≤ y ≤ 1
F_{X}^{−1} is increasing in y.
F_{X}(F_{X}^{−1} (y)) = y when F_{X} is continuous at F_{X}^{−1} (y).
F_{X}^{−1} (y) ≤ x iff y ≤ F_{X}(x).
UW-Madison (Statistics)  Stat 609 Lecture 4  2014  8 / 11

beamer-tu-logo

Theorem 2.1.10

Let X have continuous cdf F_{X} and Y = F_{X}(X). Then Y is uniformly distributed on (0,1), that is, P(Y ≤ y) = y, 0 < y < 1.

This result is useful for generating random variables. If we want to generate a random value X from a cdf F_{X}, we need only generate a uniform random number Y (from the uniform distribution on (0,1)) and solve for X in the equation F_{X}(X) = Y .

The inverse of a cdf

If F_{X} is strictly increasing, then its inverse F_{X}^{−1} is well defined by

F_{X}^{−1} (y) = x iff F_{X}(x) = y

If F_{X} is flat on an interval (x_{1},x_{2}] (see the figure), then we define

F_{X}^{−1} (y) = inf{x : F_{X}(x) ≥ y}, 0 ≤ y ≤ 1
F_{X}^{−1} is increasing in y.
F_{X}(F_{X}^{−1} (y)) = y when F_{X} is continuous at F_{X}^{−1} (y).
F_{X}^{−1} (y) ≤ x iff y ≤ F_{X}(x).
UW-Madison (Statistics)  Stat 609 Lecture 4  2014  8 / 11

beamer-tu-logo

In part b of the figure, F_{X}^{−1} (y) = x_{1}.

Proof of Theorem 2.1.10

For 0 < y < 1,

P(Y ≤ y) = P(F_{X}(X) ≤ y)
= P(F_{X}^{−1} (F_{X}(X)) ≤ F_{X}^{−1} (y)) (F_{X}^{−1} is increasing)
= P(X ≤ F_{X}^{−1} (y))  (P(F_{X}^{−1} (F_{X}(X)) < X) = 0)

= F_{X}(F_{X}^{−1} (y))

= y  (continuity of F_{X})
UW-Madison (Statistics)  Stat 609 Lecture 4  2014  9 / 11

beamer-tu-logo

In part b of the figure, F_{X}^{−1} (y) = x_{1}.

Proof of Theorem 2.1.10

For 0 < y < 1,

P(Y ≤ y) = P(F_{X}(X) ≤ y)
= P(F_{X}^{−1} (F_{X}(X)) ≤ F_{X}^{−1} (y)) (F_{X}^{−1} is increasing)
= P(X ≤ F_{X}^{−1} (y))  (P(F_{X}^{−1} (F_{X}(X)) < X) = 0)

= F_{X}(F_{X}^{−1} (y))

= y  (continuity of F_{X})
UW-Madison (Statistics)  Stat 609 Lecture 4  2014  9 / 11

beamer-tu-logo

Probability integral transform

If U is uniformly distributed on (0,1) and F is a cdf on R, then the random variable X = F^{−1} (U) has cdf F. We use the property F^{−1} (y) ≤ x iff y ≤ F(x).

If y ≤ F(x), then x ≥ inf{t : F(t) ≥ y} = F^{−1} (y).

If F^{−1} (y) = inf{t : F(t) ≥ y} ≤ x, then F(F^{−1} (y)) ≤ F(x). But

F(F^{−1} (y)) \begin{matrix}= y & if F is continuous at y \\ > y & if F has a jump at y \end{matrix}

Thus, we must have y ≤ F^{−1} (y) ≤ x. Then, for any x ∈ R,

P(X ≤ x) = P(F^{−1} (U) ≤ x) = P(U ≤ F(x)) = F(x)

Symmetric random variables

A random variable X is symmetric if −X has the same distribution as X. Since −X is a simple transformation, if X has pdf f_{X}, then X is symmetric iff f_{X}(x) = f_{X}(−x), x ∈ R, i.e., f_{X} is a symmetric function.

UW-Madison (Statistics)  Stat 609 Lecture 4  2014  10 / 11

beamer-tu-logo

Probability integral transform

If U is uniformly distributed on (0,1) and F is a cdf on R, then the random variable X = F^{−1} (U) has cdf F. We use the property F^{−1} (y) ≤ x iff y ≤ F(x).

If y ≤ F(x), then x ≥ inf{t : F(t) ≥ y} = F^{−1} (y).

If F^{−1} (y) = inf{t : F(t) ≥ y} ≤ x, then F(F^{−1} (y)) ≤ F(x). But

F(F^{−1} (y)) \begin{matrix}= y & if F is continuous at y \\ > y & if F has a jump at y \end{matrix}

Thus, we must have y ≤ F^{−1} (y) ≤ x. Then, for any x ∈ R,

P(X ≤ x) = P(F^{−1} (U) ≤ x) = P(U ≤ F(x)) = F(x)

Symmetric random variables

A random variable X is symmetric if −X has the same distribution as X. Since −X is a simple transformation, if X has pdf f_{X}, then X is symmetric iff f_{X}(x) = f_{X}(−x), x ∈ R, i.e., f_{X} is a symmetric function.

UW-Madison (Statistics)  Stat 609 Lecture 4  2014  10 / 11

beamer-tu-logo

f_{X} is symmetric iff F_{X}(−x) = 1 − F_{X}(x), x ∈ R

If F_{X}(−x) = 1 − F_{X}(x), x ∈ R, then differentiating on both sizes gives −f_{X}(−x) = −f_{X}(x), x ∈ R. If f_{X} is symmetric, then

F_{X}(−x) =∫_{−∞}^{−x} f_{X}(t)dt = −∫_{−∞}^{−x} f_{X}(−t)d(−t) =∫_{x}^{∞} f_{X}(s)ds = 1−F_{X}(x)

f_{X} is symmetric iff F_{X}(−x) = \frac{1}{2} − ∫_{0}^{x} f_{X}(t)dt, x > 0

If F_{X}(−x) = \frac{1}{2} − ∫_{0}^{x} f_{X}(t)dt, x > 0, then differentiating on both sizes gives −f_{X}(−x) = −f_{X}(x), x ∈ R. If f_{X} is symmetric, then

F_{X}(−x) = ∫_{−∞}^{−x} f_{X}(t)dt = ∫_{−∞}^{0} f_{X}(t)dt − ∫_{−x}^{0} f_{X}(t)dt = \frac{1}{2} − ∫_{0}^{x} f_{X}(s)ds

beamer-tu-logo

f_{X} is symmetric iff F_{X}(−x) = 1 − F_{X}(x), x ∈ R

If F_{X}(−x) = 1 − F_{X}(x), x ∈ R, then differentiating on both sizes gives −f_{X}(−x) = −f_{X}(x), x ∈ R. If f_{X} is symmetric, then

F_{X}(−x) =∫_{−∞}^{−x} f_{X}(t)dt = −∫_{−∞}^{−x} f_{X}(−t)d(−t) =∫_{x}^{∞} f_{X}(s)ds = 1−F_{X}(x)

f_{X} is symmetric iff F_{X}(−x) = \frac{1}{2} − ∫_{0}^{x} f_{X}(t)dt, x > 0

If F_{X}(−x) = \frac{1}{2} − ∫_{0}^{x} f_{X}(t)dt, x > 0, then differentiating on both sizes gives −f_{X}(−x) = −f_{X}(x), x ∈ R. If f_{X} is symmetric, then

F_{X}(−x) = ∫_{−∞}^{−x} f_{X}(t)dt = ∫_{−∞}^{0} f_{X}(t)dt − ∫_{−x}^{0} f_{X}(t)dt = \frac{1}{2} − ∫_{0}^{x} f_{X}(s)ds