beamer-tu-logo

Lecture 8: Differentiating under an integral sign

When can we switch the differentiation and integration? If the range of the integral is finite, this switch is usually valid.

Theorem 2.4.1 (Leibnitz’s rule)

If f (x,θ), a(θ), and b(θ) are differentiable with respect to θ, then

\frac{d}{dθ} ∫_{a(θ)}^{b(θ)}f (x,θ)dx = f (b(θ),θ)b′(θ)−f (a(θ),θ)a′(θ)+∫_{a(θ)}^{b(θ)} \frac{∂f (x,θ)}{∂θ}dx

In particular, if a(θ) = a and b(θ) = b are constants, then

\frac{d}{dθ} ∫_{a}^{b}f (x,θ)dx = ∫_{a}^{b} \frac{∂f (x,θ)}{∂θ}dx

If the range of integration is infinite, problems can arise. Whether the order of differentiation and integration can be switched is the same as whether limit and integration can be interchanged, because differentiation is a special kind of limit.

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  1 / 12

beamer-tu-logo

Lecture 8: Differentiating under an integral sign

When can we switch the differentiation and integration? If the range of the integral is finite, this switch is usually valid.

Theorem 2.4.1 (Leibnitz’s rule)

If f (x,θ), a(θ), and b(θ) are differentiable with respect to θ, then

\frac{d}{dθ} ∫_{a(θ)}^{b(θ)}f (x,θ)dx = f (b(θ),θ)b′(θ)−f (a(θ),θ)a′(θ)+∫_{a(θ)}^{b(θ)} \frac{∂f (x,θ)}{∂θ}dx

In particular, if a(θ) = a and b(θ) = b are constants, then

\frac{d}{dθ} ∫_{a}^{b}f (x,θ)dx = ∫_{a}^{b} \frac{∂f (x,θ)}{∂θ}dx

If the range of integration is infinite, problems can arise. Whether the order of differentiation and integration can be switched is the same as whether limit and integration can be interchanged, because differentiation is a special kind of limit.

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  1 / 12

beamer-tu-logo

Lecture 8: Differentiating under an integral sign

When can we switch the differentiation and integration? If the range of the integral is finite, this switch is usually valid.

Theorem 2.4.1 (Leibnitz’s rule)

If f (x,θ), a(θ), and b(θ) are differentiable with respect to θ, then

\frac{d}{dθ} ∫_{a(θ)}^{b(θ)}f (x,θ)dx = f (b(θ),θ)b′(θ)−f (a(θ),θ)a′(θ)+∫_{a(θ)}^{b(θ)} \frac{∂f (x,θ)}{∂θ}dx

In particular, if a(θ) = a and b(θ) = b are constants, then

\frac{d}{dθ} ∫_{a}^{b}f (x,θ)dx = ∫_{a}^{b} \frac{∂f (x,θ)}{∂θ}dx

If the range of integration is infinite, problems can arise. Whether the order of differentiation and integration can be switched is the same as whether limit and integration can be interchanged, because differentiation is a special kind of limit.

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  1 / 12

beamer-tu-logo

Interchange of integration and limit

Note that

∫_{−∞}^{∞} \frac{∂f (x,θ)}{∂θ}dx = ∫_{−∞}^{∞} lim_{δ→0} \frac{f (x,θ + δ) − f (x,θ)}{δ} dx

Hence, the interchange of differentiation and integration means whether this is equal to

\frac{d}{dθ} ∫_{−∞}^{∞} f (x,θ)dx = lim_{δ→0}∫_{−∞}^{∞} \frac{f (x,θ + δ) − f (x,θ)}{δ} dx

An example of invalid interchanging integration and limit

Consider

f (x,δ) = \{ \begin{matrix}δ^{−1} & 0 < x < δ \\ 0 & otherwise \end{matrix}

Then, lim_{δ→0} f (x,δ) = 0 for any x and, hence,

0 = ∫_{−∞}^{∞} lim_{δ→0}f (x,δ)dx ≠ lim_{δ→0}∫_{−∞}^{∞} f (x,δ)dx = lim_{δ→0}∫_{0}^{δ} δ^{−1} dx = 1

beamer-tu-logo

Interchange of integration and limit

Note that

∫_{−∞}^{∞} \frac{∂f (x,θ)}{∂θ}dx = ∫_{−∞}^{∞} lim_{δ→0} \frac{f (x,θ + δ) − f (x,θ)}{δ} dx

Hence, the interchange of differentiation and integration means whether this is equal to

\frac{d}{dθ} ∫_{−∞}^{∞} f (x,θ)dx = lim_{δ→0}∫_{−∞}^{∞} \frac{f (x,θ + δ) − f (x,θ)}{δ} dx

An example of invalid interchanging integration and limit

Consider

f (x,δ) = \{ \begin{matrix}δ^{−1} & 0 < x < δ \\ 0 & otherwise \end{matrix}

Then, lim_{δ→0} f (x,δ) = 0 for any x and, hence,

0 = ∫_{−∞}^{∞} lim_{δ→0}f (x,δ)dx ≠ lim_{δ→0}∫_{−∞}^{∞} f (x,δ)dx = lim_{δ→0}∫_{0}^{δ} δ^{−1} dx = 1

beamer-tu-logo

We now give some sufficient conditions under which interchanging integration and limit (or differentiation) is valid. The proof of the main result is technical and out of the scope of this course.

Theorem 2.4.2 (Lebesgue’s dominated convergence theorem)

Suppose that the function h(x,y) is continuous at y_{0} for each x, and there exists a function g(x) satisfying

(i) |h(x,y)| ≤ g(x) for all x and y in a neighborhood of y_{0}; (ii) ∫_{−∞}^{∞} g(x)dx < ∞. Then,

lim_{y→y_{0}} ∫_{−∞}^{∞} h(x,y)dx = ∫_{−∞}^{∞} lim_{y→y_{0}} h(x,y)dx

The key condition in this theorem is the existence of a dominating function g(x) with a finite integral. Applying Theorem 2.4.2 to the difference [f (x,θ + δ) − f (x,θ)]/δ, we obtain the next result.

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  3 / 12

beamer-tu-logo

We now give some sufficient conditions under which interchanging integration and limit (or differentiation) is valid. The proof of the main result is technical and out of the scope of this course.

Theorem 2.4.2 (Lebesgue’s dominated convergence theorem)

Suppose that the function h(x,y) is continuous at y_{0} for each x, and there exists a function g(x) satisfying

(i) |h(x,y)| ≤ g(x) for all x and y in a neighborhood of y_{0}; (ii) ∫_{−∞}^{∞} g(x)dx < ∞. Then,

lim_{y→y_{0}} ∫_{−∞}^{∞} h(x,y)dx = ∫_{−∞}^{∞} lim_{y→y_{0}} h(x,y)dx

The key condition in this theorem is the existence of a dominating function g(x) with a finite integral. Applying Theorem 2.4.2 to the difference [f (x,θ + δ) − f (x,θ)]/δ, we obtain the next result.

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  3 / 12

beamer-tu-logo

We now give some sufficient conditions under which interchanging integration and limit (or differentiation) is valid. The proof of the main result is technical and out of the scope of this course.

Theorem 2.4.2 (Lebesgue’s dominated convergence theorem)

Suppose that the function h(x,y) is continuous at y_{0} for each x, and there exists a function g(x) satisfying

(i) |h(x,y)| ≤ g(x) for all x and y in a neighborhood of y_{0}; (ii) ∫_{−∞}^{∞} g(x)dx < ∞. Then,

lim_{y→y_{0}} ∫_{−∞}^{∞} h(x,y)dx = ∫_{−∞}^{∞} lim_{y→y_{0}} h(x,y)dx

The key condition in this theorem is the existence of a dominating function g(x) with a finite integral. Applying Theorem 2.4.2 to the difference [f (x,θ + δ) − f (x,θ)]/δ, we obtain the next result.

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  3 / 12

beamer-tu-logo

Theorem 2.4.3.

Suppose that f (x,θ) is differentiable at θ = θ_{0}, i.e.,

lim_{δ→0} \frac{f (x,θ_{0} + δ) − f (x,θ_{0})}{δ} = \frac{∂f (x,θ)}{∂θ}|_{θ=θ_{0}}

exists for every x, and there exists a function g(x,θ_{0}) and a constant δ_{0} > 0 such that

(i) | \frac{f (x,θ_{0}+δ)−f (x,θ_{0})}{δ} | ≤ g(x,θ_{0}) for all x and |δ| ≤ δ_{0}; (ii) ∫_{−∞}^{∞} g(x,θ_{0})dx < ∞. Then

\frac{d}{dθ} ∫_{−∞}^{∞} f (x,θ)dx|_{θ=θ_{0}} = ∫_{−∞}^{∞} \frac{∂f (x,θ)}{∂θ}|_{θ=θ_{0}}dx

This result is for θ at a particular value, θ_{0}. Typically f (x,θ) is differentiable at all θ, and

\frac{f (x,θ_{0} + δ) − f (x,θ_{0})}{δ} = \frac{∂f (x,θ)}{∂θ}|_{θ=θ_{0}+δ^{∗}(x)}

for some δ^{∗}(x) with |δ^{∗}(x)| ≤ δ_{0}, which leads to the next result.

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  4 / 12

beamer-tu-logo

Theorem 2.4.3.

Suppose that f (x,θ) is differentiable at θ = θ_{0}, i.e.,

lim_{δ→0} \frac{f (x,θ_{0} + δ) − f (x,θ_{0})}{δ} = \frac{∂f (x,θ)}{∂θ}|_{θ=θ_{0}}

exists for every x, and there exists a function g(x,θ_{0}) and a constant δ_{0} > 0 such that

(i) | \frac{f (x,θ_{0}+δ)−f (x,θ_{0})}{δ} | ≤ g(x,θ_{0}) for all x and |δ| ≤ δ_{0}; (ii) ∫_{−∞}^{∞} g(x,θ_{0})dx < ∞. Then

\frac{d}{dθ} ∫_{−∞}^{∞} f (x,θ)dx|_{θ=θ_{0}} = ∫_{−∞}^{∞} \frac{∂f (x,θ)}{∂θ}|_{θ=θ_{0}}dx

This result is for θ at a particular value, θ_{0}. Typically f (x,θ) is differentiable at all θ, and

\frac{f (x,θ_{0} + δ) − f (x,θ_{0})}{δ} = \frac{∂f (x,θ)}{∂θ}|_{θ=θ_{0}+δ^{∗}(x)}

for some δ^{∗}(x) with |δ^{∗}(x)| ≤ δ_{0}, which leads to the next result.

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  4 / 12

beamer-tu-logo

Corollary 2.4.4.

Suppose that f (x,θ) is differentiable in θ and there exists a function g(x,θ) such that

(i) | \frac{∂f (x,ϑ)}{∂ϑ} | ≤ g(x,θ) for all x and ϑ such that |ϑ − θ| ≤ δ_{0}; (ii) ∫_{−∞}^{∞} g(x,θ)dx < ∞ for all θ. Then

\frac{d}{dθ} ∫_{−∞}^{∞} f (x,θ)dx = ∫_{−∞}^{∞} \frac{∂f (x,θ)}{∂θ}dx

Example 2.4.5.

Let X have the exponential pdf

f (x) = λ^{−1} e^{−x/λ} , x > 0

where λ > 0 is a constant. We want to prove a recursion relation

E(X^{n+1}) = λE(X^{n}) + λ \frac{d}{dλ} E(X^{n}), n = 1,2,...

beamer-tu-logo

Corollary 2.4.4.

Suppose that f (x,θ) is differentiable in θ and there exists a function g(x,θ) such that

(i) | \frac{∂f (x,ϑ)}{∂ϑ} | ≤ g(x,θ) for all x and ϑ such that |ϑ − θ| ≤ δ_{0}; (ii) ∫_{−∞}^{∞} g(x,θ)dx < ∞ for all θ. Then

\frac{d}{dθ} ∫_{−∞}^{∞} f (x,θ)dx = ∫_{−∞}^{∞} \frac{∂f (x,θ)}{∂θ}dx

Example 2.4.5.

Let X have the exponential pdf

f (x) = λ^{−1} e^{−x/λ} , x > 0

where λ > 0 is a constant. We want to prove a recursion relation

E(X^{n+1}) = λE(X^{n}) + λ \frac{d}{dλ} E(X^{n}), n = 1,2,...

beamer-tu-logo

If we could move the differentiation inside the integral, we would have

\frac{d}{dλ} E(X^{n}) = \frac{d}{dλ} ∫_{0}^{∞} \frac{x^{n}}{λ}e^{−x/λ} dx = ∫_{0}^{∞} \frac{∂}{∂λ} ( \frac{x^{n}}{λ}e^{−x/λ} ) dx
= ∫_{0}^{∞} \frac{x^{n}}{λ^{2}} ( \frac{x}{λ} − 1) e^{−x/λ} dx = \frac{1}{λ^{2}} E(X^{n+1}) − \frac{1}{λ} E(X^{n})

which is the result we want to show. To justify the interchange of integration and differentiation, we take

g(x,λ) = \frac{x^{n}e^{−x/(λ+δ_{0})}}{(λ − δ_{0})^{2}} ( \frac{x}{λ − δ_{0}} + 1)

Then

| \frac{∂}{∂ξ} ( \frac{x^{n}}{ξ}e^{−x/ξ} ) | ≤ \frac{x^{n}e^{−x/ξ}}{ξ^{2}} | \frac{x}{ξ} + 1| ≤ g(x,λ), |ξ − λ| ≤ δ_{0}

and we can apply Corollary 2.4.4.

In the proof of Theorem 2.3.7 (differentiating mgf to obtain moments), we interchanged differentiation and integration without justification. We now provide a proof.

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  6 / 12

beamer-tu-logo

\frac{d}{dλ} E(X^{n}) = \frac{d}{dλ} ∫_{0}^{∞} \frac{x^{n}}{λ}e^{−x/λ} dx = ∫_{0}^{∞} \frac{∂}{∂λ} ( \frac{x^{n}}{λ}e^{−x/λ} ) dx
= ∫_{0}^{∞} \frac{x^{n}}{λ^{2}} ( \frac{x}{λ} − 1) e^{−x/λ} dx = \frac{1}{λ^{2}} E(X^{n+1}) − \frac{1}{λ} E(X^{n})

which is the result we want to show. To justify the interchange of integration and differentiation, we take

g(x,λ) = \frac{x^{n}e^{−x/(λ+δ_{0})}}{(λ − δ_{0})^{2}} ( \frac{x}{λ − δ_{0}} + 1)

Then

| \frac{∂}{∂ξ} ( \frac{x^{n}}{ξ}e^{−x/ξ} ) | ≤ \frac{x^{n}e^{−x/ξ}}{ξ^{2}} | \frac{x}{ξ} + 1| ≤ g(x,λ), |ξ − λ| ≤ δ_{0}

and we can apply Corollary 2.4.4.

In the proof of Theorem 2.3.7 (differentiating mgf to obtain moments), we interchanged differentiation and integration without justification. We now provide a proof.

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  6 / 12

beamer-tu-logo

Completion of the proof of Theorem 2.3.7.

Assuming that M_{X}(t) exists for t ∈ [−h,h], h > 0, we want to show

\frac{d}{dt}M_{X}(t) = \frac{d}{dt}∫_{−∞}^{∞} e^{tx}f_{X}(x)dx = ∫_{−∞}^{∞} \frac{∂}{∂t}e^{tx}f_{X}(x)dx

It is enough to show (why?)

\frac{d}{dt}∫_{0}^{∞} e^{tx}f_{X}(x)dx = ∫_{0}^{∞} \frac{∂}{∂t}e^{tx}f_{X}(x)dx

For t ∈ (−h/2,h/2) and x > 0,

| \frac{∂}{∂t}e^{tx}f_{X}(x)| = |xe^{tx}f_{X}(x)| ≤ xe^{hx/2}f_{X}(x) = g(x)

For sufficiently large x > 0, x ≤ e^{hx/2} and, hence,

∫_{0}^{∞} g(x)dx ≤ ∫_{0}^{∞} e^{hx}f_{X}(x)dx = M_{X}(h) < ∞

An application of Corollary 2.4.4 proves the result.

We now turn to the question of when we can interchange differentiation and infinite summation.

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  7 / 12

beamer-tu-logo

Completion of the proof of Theorem 2.3.7.

Assuming that M_{X}(t) exists for t ∈ [−h,h], h > 0, we want to show

\frac{d}{dt}M_{X}(t) = \frac{d}{dt}∫_{−∞}^{∞} e^{tx}f_{X}(x)dx = ∫_{−∞}^{∞} \frac{∂}{∂t}e^{tx}f_{X}(x)dx

It is enough to show (why?)

\frac{d}{dt}∫_{0}^{∞} e^{tx}f_{X}(x)dx = ∫_{0}^{∞} \frac{∂}{∂t}e^{tx}f_{X}(x)dx

For t ∈ (−h/2,h/2) and x > 0,

| \frac{∂}{∂t}e^{tx}f_{X}(x)| = |xe^{tx}f_{X}(x)| ≤ xe^{hx/2}f_{X}(x) = g(x)

For sufficiently large x > 0, x ≤ e^{hx/2} and, hence,

∫_{0}^{∞} g(x)dx ≤ ∫_{0}^{∞} e^{hx}f_{X}(x)dx = M_{X}(h) < ∞

An application of Corollary 2.4.4 proves the result.

We now turn to the question of when we can interchange differentiation and infinite summation.

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  7 / 12

beamer-tu-logo

Theorem 2.4.8.

Suppose that the series ∑ _{x=0}^{∞}h(θ,x) converges for all θ in an interval (a,b) ⊂ R and

(i) \frac{∂}{∂θ} h(θ,x) is continuous in θ for each x; (ii) ∑ _{x=0}^{∞} \frac{∂}{∂θ} h(θ,x) converges uniformly on every closed bounded subinterval of (a,b). Then

\frac{d}{dθ} ∑_{x=0}^{∞}h(θ,x) = ∑_{x=0}^{∞} \frac{∂}{∂θ} h(θ,x)

This theorem is from calculus (or mathematical analysis) and is not easy to apply because we need to show the uniform convergence of an infinite series.

It is more convenient to apply the following dominated convergence theorem for infinite sums.

Theorem 2.4.2A (dominated convergence theorem)

Suppose that the function h(x,y) is continuous at y_{0} for each x, and there exists a function g(x) satisfying

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  8 / 12

beamer-tu-logo

Theorem 2.4.8.

Suppose that the series ∑ _{x=0}^{∞}h(θ,x) converges for all θ in an interval (a,b) ⊂ R and

(i) \frac{∂}{∂θ} h(θ,x) is continuous in θ for each x; (ii) ∑ _{x=0}^{∞} \frac{∂}{∂θ} h(θ,x) converges uniformly on every closed bounded subinterval of (a,b). Then

\frac{d}{dθ} ∑_{x=0}^{∞}h(θ,x) = ∑_{x=0}^{∞} \frac{∂}{∂θ} h(θ,x)

This theorem is from calculus (or mathematical analysis) and is not easy to apply because we need to show the uniform convergence of an infinite series.

It is more convenient to apply the following dominated convergence theorem for infinite sums.

Theorem 2.4.2A (dominated convergence theorem)

Suppose that the function h(x,y) is continuous at y_{0} for each x, and there exists a function g(x) satisfying

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  8 / 12

beamer-tu-logo

(i) |h(x,y)| ≤ g(x) for all x and y in a neighborhood of y_{0}; (ii) ∑_{x} g(x) < ∞. Then,

lim_{y→y_{0}} ∑_{x} h(x,y) = ∑_{x} lim_{y→y_{0}} h(x,y)

If h(x,θ) is differentiable in θ and there is a function g(x,θ) such that (i) | \frac{∂h(x,ϑ)}{∂ϑ} | ≤ g(x,θ) for all x and ϑ such that |ϑ − θ| ≤ δ_{0}; (ii) ∑_{x} g(x,θ) < ∞ for all θ. Then

\frac{d}{dθ} ∑_{x} h(x,θ) = ∑_{x} \frac{∂h(x,θ)}{∂θ}

Example 2.4.7.

Let X be a discrete random variable having pmf

f_{X}(x) = P(X = x) = θ(1 − θ)^{x}, x = 0,1,2,...

where θ ∈ (0,1) is a fixed constant. We want to derive E(X). Note that

∑_{x=0}^{∞}θ(1 − θ)^{x} = 1

beamer-tu-logo

(i) |h(x,y)| ≤ g(x) for all x and y in a neighborhood of y_{0}; (ii) ∑_{x} g(x) < ∞. Then,

lim_{y→y_{0}} ∑_{x} h(x,y) = ∑_{x} lim_{y→y_{0}} h(x,y)

If h(x,θ) is differentiable in θ and there is a function g(x,θ) such that (i) | \frac{∂h(x,ϑ)}{∂ϑ} | ≤ g(x,θ) for all x and ϑ such that |ϑ − θ| ≤ δ_{0}; (ii) ∑_{x} g(x,θ) < ∞ for all θ. Then

\frac{d}{dθ} ∑_{x} h(x,θ) = ∑_{x} \frac{∂h(x,θ)}{∂θ}

Example 2.4.7.

Let X be a discrete random variable having pmf

f_{X}(x) = P(X = x) = θ(1 − θ)^{x}, x = 0,1,2,...

where θ ∈ (0,1) is a fixed constant. We want to derive E(X). Note that

∑_{x=0}^{∞}θ(1 − θ)^{x} = 1

beamer-tu-logo

If interchange of summation and differentiation is justified,

0 = \frac{d}{dθ} ∑_{x=0}^{∞}θ(1 − θ)^{x} = ∑_{x=0}^{∞} \frac{∂}{∂θ} θ(1 − θ)^{x}
= ∑_{x=0}^{∞}[(1 − θ)^{x} − θx(1 − θ)^{x−1} ]
= \frac{1}{θ} ∑_{x=0}^{∞}θ(1 − θ)^{x} − \frac{1}{1 − θ} ∑_{x=0}^{∞}xθ(1 − θ)^{x}
= \frac{1}{θ} − \frac{1}{1 − θ} E(X)

Thus, E(X) = (1 − θ)/θ. To justify the interchange, let h(x,θ) = θ(1 − θ)^{x} and then

\frac{∂}{∂θ} h(θ,x) = (1 − θ)^{x} − θx(1 − θ)^{x−1}

Consider θ ∈ [c,d] ⊂ (0,1).

|(1 − θ)^{x} − θx(1 − θ)^{x−1} | ≤ (1 − c)^{x} + dx(1 − c)^{x−1} = g(x)

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  10 / 12

beamer-tu-logo

Since

∑_{x=0}^{∞}[(1 − c)^{x} + dx(1 − c)^{x−1} ] < ∞

the exchange of sum and differentiation is justified.

Monotone convergence theorem

In some cases, the following result is more convenient to apply.

Suppose that functions g_{n}(x), n = 1,2,..., x ∈ R, satisfying that 0 ≤ g_{n}(x) ≤ g_{n+1}(x) for all n and x and lim_{n→∞} g_{n}(x) = g(x) for all x. Then

lim_{n→∞}∫_{−∞}^{∞} g_{n}(x)dx = ∫_{−∞}^{∞} lim_{n→∞}g_{n}(x)dx = ∫_{−∞}^{∞} g(x)dx

which holds even if g(x) is not integrable (i.e., both sides are infinity); or

lim_{n→∞} ∑_{x} g_{n}(x) = ∑_{x} lim_{n→∞}g_{n}(x) = ∑_{x} g(x)

which holds even if both sides are infinity.

If ∫_{−∞}^{∞} g(x)dx < ∞ or ∑_{x} g(x) < ∞, then this result is a direct consequence of the dominated convergence theorem.

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  11 / 12

beamer-tu-logo

Since

∑_{x=0}^{∞}[(1 − c)^{x} + dx(1 − c)^{x−1} ] < ∞

the exchange of sum and differentiation is justified.

Monotone convergence theorem

In some cases, the following result is more convenient to apply.

Suppose that functions g_{n}(x), n = 1,2,..., x ∈ R, satisfying that 0 ≤ g_{n}(x) ≤ g_{n+1}(x) for all n and x and lim_{n→∞} g_{n}(x) = g(x) for all x. Then

lim_{n→∞}∫_{−∞}^{∞} g_{n}(x)dx = ∫_{−∞}^{∞} lim_{n→∞}g_{n}(x)dx = ∫_{−∞}^{∞} g(x)dx

which holds even if g(x) is not integrable (i.e., both sides are infinity); or

lim_{n→∞} ∑_{x} g_{n}(x) = ∑_{x} lim_{n→∞}g_{n}(x) = ∑_{x} g(x)

which holds even if both sides are infinity.

If ∫_{−∞}^{∞} g(x)dx < ∞ or ∑_{x} g(x) < ∞, then this result is a direct consequence of the dominated convergence theorem.

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  11 / 12

beamer-tu-logo

Suppose now that ∫_{−∞}^{∞} g(x)dx = ∞. Then

lim_{M→∞} ∫_{−∞}^{∞} g(x)I_{{|x|≤M,g(x)≤M}} dx = ∞

where I_{A} is the indicator function of A. For each M > 0, ∫_{−∞}^{∞} g(x)I_{{|x|≤M,g(x)≤M}} dx ≤ 2M^{2} < ∞ and g_{n}(x)I_{{|x|≤M,g(x)≤M}} is monotone and converges to g(x)I_{{|x|≤M,g(x)≤M}} . Then

lim_{n→∞}∫_{−∞}^{∞} g_{n}(x)dx ≥ lim_{n→∞}∫_{−∞}^{∞} g_{n}(x)I_{{|x|≤M,g(x)≤M}} dx
= ∫_{−∞}^{∞} g(x)I_{{|x|≤M,g(x)≤M}} dx

for every M. Since the right side goes to ∞,

lim_{n→∞}∫_{−∞}^{∞} g_{n}(x)dx = ∞

The proof for the case of ∑_{x} g(x) = ∞ is similar.

UW-Madison (Statistics)  Stat 609 Lecture 8  2014  12 / 12