beamer-tu-logo

Lecture 9: Characteristic functions

The mgf’s are useful, but they sometimes are not finite. Another function that can characterise a distribution is the so-called characteristic function, which is always well defined but it is a complex function.

Definition (characteristic functions).

The characteristic function (chf) of the distribution of a random variable X is defined as

ϕ_{X}(t) = E(e^{i̇tX}) = E[cos (tX)] + i̇E[sin(tX)], t ∈ R

where i̇= \sqrt{−1} and e^{i̇s} = cos (s) + i̇sin(s).

If the mgf M_{X}(t) of X is finite in a neighborhood of 0, then the chf of X is ϕ_{X}(t) = M_{X}(i̇t), t ∈ R. If Y = aX + b for constants a and b, then the chf of Y in terms of the chf ϕ_{X}(t) of X is ϕ_{Y}(t) = e^{i̇bt}ϕ_{X}(at), t ∈ R. A chf is uniformly continuous in R.

UW-Madison (Statistics)  Stat 609 Lecture 9  2014  1 / 11

beamer-tu-logo

Lecture 9: Characteristic functions

The mgf’s are useful, but they sometimes are not finite. Another function that can characterise a distribution is the so-called characteristic function, which is always well defined but it is a complex function.

Definition (characteristic functions).

The characteristic function (chf) of the distribution of a random variable X is defined as

ϕ_{X}(t) = E(e^{i̇tX}) = E[cos (tX)] + i̇E[sin(tX)], t ∈ R

where i̇= \sqrt{−1} and e^{i̇s} = cos (s) + i̇sin(s).

If the mgf M_{X}(t) of X is finite in a neighborhood of 0, then the chf of X is ϕ_{X}(t) = M_{X}(i̇t), t ∈ R. If Y = aX + b for constants a and b, then the chf of Y in terms of the chf ϕ_{X}(t) of X is ϕ_{Y}(t) = e^{i̇bt}ϕ_{X}(at), t ∈ R. A chf is uniformly continuous in R.

UW-Madison (Statistics)  Stat 609 Lecture 9  2014  1 / 11

beamer-tu-logo

Theorem C1.

If a random variable X has finite E|X|^{r} for a positive integer r, then

\frac{d^{r}ϕ_{X}(t)}{dt^{r}}|_{t=0} = i̇^{r}E(X^{r})

Proof.

We give a proof for the case where X has pdf f_{X}(x). If we can exchange the differentiation and integration, then

\frac{d^{r}ϕ_{X}(t)}{dt^{r}}= \frac{d^{r}}{dt^{r}}∫_{−∞}^{∞} e^{i̇tx}f_{X}(x)dx = ∫_{−∞}^{∞} \frac{d^{r}}{dt^{r}}e^{i̇tx}f_{X}(x)dx = ∫_{−∞}^{∞} i̇^{r}x^{r}e^{i̇tx}f_{X}(x)dx

and the result follows by setting t = 0 at the beginning and end of the above expression. The exchange is justified by the dominated convergence theorem, since

| \frac{d^{r}}{dt^{r}}e^{i̇tx}f_{X}(x)| = |i̇^{r}x^{r}e^{i̇tx}|f_{X}(x) ≤ |x|^{r}f_{X}(x)

and

E|X|^{r} = ∫_{−∞}^{∞} |x|^{r}f_{X}(x)dx < ∞
UW-Madison (Statistics)  Stat 609 Lecture 9  2014  2 / 11

beamer-tu-logo

Thus, like the mgf, the chf can be used to calculate the moments, but we have to first know the existence of moments.

Example.

Let X be a random variable with pdf

f_{X}(x) = \frac{1}{π(1 + x^{2})}, x ∈ R

From the Fourier transformation,

∫_{0}^{∞} \frac{cos (tx)}{1 + x^{2}}dx = \frac{π}{2} e^{−t} , t > 0

we obtain that, when t > 0,

ϕ_{X}(t) = E[cos (tX)] + i̇E[sin(tX)]
= ∫_{−∞}^{∞} \frac{cos (tx)}{π(1 + x^{2})}dx + i̇∫_{−∞}^{∞} \frac{sin(tx)}{π(1 + x^{2})}dx
= \frac{2}{π} ∫_{−∞}^{∞} \frac{cos (tx)}{1 + x^{2}}dx = e^{−t}

beamer-tu-logo

Thus, like the mgf, the chf can be used to calculate the moments, but we have to first know the existence of moments.

Example.

Let X be a random variable with pdf

f_{X}(x) = \frac{1}{π(1 + x^{2})}, x ∈ R

From the Fourier transformation,

∫_{0}^{∞} \frac{cos (tx)}{1 + x^{2}}dx = \frac{π}{2} e^{−t} , t > 0

we obtain that, when t > 0,

ϕ_{X}(t) = E[cos (tX)] + i̇E[sin(tX)]
= ∫_{−∞}^{∞} \frac{cos (tx)}{π(1 + x^{2})}dx + i̇∫_{−∞}^{∞} \frac{sin(tx)}{π(1 + x^{2})}dx
= \frac{2}{π} ∫_{−∞}^{∞} \frac{cos (tx)}{1 + x^{2}}dx = e^{−t}

beamer-tu-logo

For t < 0,

ϕ_{X}(t) = E[cos (tX)] + i̇E[sin(tX)] = E[cos (−tX)] = e^{t} = e^{−|t|}

When t = 0, ϕ_{X}(0) = 1 for any chf, and hence we obtain that

ϕ_{X}(t) = e^{−|t|} t ∈ R

Note that this chf is not differentiable at t = 0 and previously we showed that the expectation of X does not exist.

Uniqueness and inversion

Similar to the mgf, the chf characterizes the distribution in the sense that there is a one-to-one correspondence between chf and cdf. We first establish an inversion formula.

Theorem C2

Let X be a random variable with cdf F and chf ϕ. For any real numbers y < u,

P(y < X < u) + \frac{P(X = y) + P(X = u)}{2}= lim_{T →∞} ∫_{−T}^{T} \frac{e^{−i̇ty} − e^{−i̇tu}}{2πi̇t} ϕ(t)dt

UW-Madison (Statistics)  Stat 609 Lecture 9  2014  4 / 11

beamer-tu-logo

For t < 0,

ϕ_{X}(t) = E[cos (tX)] + i̇E[sin(tX)] = E[cos (−tX)] = e^{t} = e^{−|t|}

When t = 0, ϕ_{X}(0) = 1 for any chf, and hence we obtain that

ϕ_{X}(t) = e^{−|t|} t ∈ R

Note that this chf is not differentiable at t = 0 and previously we showed that the expectation of X does not exist.

Uniqueness and inversion

Similar to the mgf, the chf characterizes the distribution in the sense that there is a one-to-one correspondence between chf and cdf. We first establish an inversion formula.

Theorem C2

Let X be a random variable with cdf F and chf ϕ. For any real numbers y < u,

P(y < X < u) + \frac{P(X = y) + P(X = u)}{2}= lim_{T →∞} ∫_{−T}^{T} \frac{e^{−i̇ty} − e^{−i̇tu}}{2πi̇t} ϕ(t)dt

UW-Madison (Statistics)  Stat 609 Lecture 9  2014  4 / 11

beamer-tu-logo

Proof.

By definition,

∫_{−T}^{T} \frac{e^{−i̇ty} − e^{−i̇tu}}{2πi̇t} ϕ(t)dt = ∫_{−T}^{T} \frac{e^{−i̇ty} − e^{−i̇tu}}{2πi̇t} E(e^{i̇tX})dt
= E ∫_{−T}^{T} \frac{e^{−i̇ty} − e^{−i̇tu}}{2πi̇t} e^{i̇tX}dt
= E[g_{T}(u,y,X)]

where the switch of E and ∫_{−T}^{T} is justified by the fact that

| \frac{e^{−i̇ty} − e^{−i̇tu}}{i̇t} e^{i̇tX}| = |∫_{y}^{u}e^{−i̇ts} ds| ≤ (u − y)

and

g_{T}(u,y,x) = ∫_{−T}^{T} \frac{e^{i̇t(x−y)} − e^{i̇t(x−u)}}{2πi̇t} dt
= \frac{1}{π} ∫_{0}^{T} \frac{sin(t(x − y)) − sin(t(x − u))}{t} dt

because \frac{sin(t(x−y))−sin(t(x−u))}{t} is even and \frac{cos (t(x−y))−cos (t(x−u))}{t} is odd.

UW-Madison (Statistics)  Stat 609 Lecture 9  2014  5 / 11

beamer-tu-logo

By Fourier transformation,

∫_{0}^{∞} \frac{sin(st)}{t}dt = \frac{π}{2} sign(s)

Hence

lim_{T →∞} g_{T}(u,y,x) = \{ \begin{matrix}−\frac{1}{2} − (−\frac{1}{2}) = 0 & x < y \\ 0 − (−\frac{1}{2}) = \frac{1}{2} & x = y \\ \frac{1}{2} − (−\frac{1}{2}) = 1 & y < x < u \\ \frac{1}{2} − 0 = \frac{1}{2} & x = u \\ \frac{1}{2} − \frac{1}{2} = 0 & x > u \end{matrix}

Also,

0 ≤ sign(s)∫_{0}^{T} \frac{sin(st)}{t}dt ≤ ∫_{0}^{π} \frac{sint}{t}dt T > 0

which implies that

|g_{T}(u,y,x)| ≤ \frac{2}{π} ∫_{0}^{π} \frac{sint}{t}dt T > 0

By the dominated convergence theorem,

UW-Madison (Statistics)  Stat 609 Lecture 9  2014  6 / 11

beamer-tu-logo

lim_{T →∞} E [g_{T}(u,y,X)] = E [ lim_{T →∞} g_{T}(u,y,X)]
= E [I(y < X < u) + \frac{1}{2}I(X = y) + \frac{1}{2}I(X = u)]
= P(y < X < u) + \frac{P(X = y) + P(X = u)}{2}

This proves the theorem.

Theorem C3.

Let F be a cdf with chf ϕ.

(i) If F is continuous at y and u with y < u, then F(u) − F(y) = lim_{T →∞} ∫_{−T}^{T} \frac{e^{−i̇ty} − e^{−i̇tu}}{2πi̇t} ϕ(t)dt

(ii) If F is continuous at x, then

F(u) = lim_{y→−∞} lim_{T →∞} ∫_{−T}^{T} \frac{e^{−i̇ty} − e^{−i̇tu}}{2πi̇t} ϕ(t)dt

(iii) There is a one-to-one correspondence between cdf and chf.

UW-Madison (Statistics)  Stat 609 Lecture 9  2014  7 / 11

beamer-tu-logo

lim_{T →∞} E [g_{T}(u,y,X)] = E [ lim_{T →∞} g_{T}(u,y,X)]
= E [I(y < X < u) + \frac{1}{2}I(X = y) + \frac{1}{2}I(X = u)]
= P(y < X < u) + \frac{P(X = y) + P(X = u)}{2}

This proves the theorem.

Theorem C3.

Let F be a cdf with chf ϕ.

(i) If F is continuous at y and u with y < u, then F(u) − F(y) = lim_{T →∞} ∫_{−T}^{T} \frac{e^{−i̇ty} − e^{−i̇tu}}{2πi̇t} ϕ(t)dt

(ii) If F is continuous at x, then

F(u) = lim_{y→−∞} lim_{T →∞} ∫_{−T}^{T} \frac{e^{−i̇ty} − e^{−i̇tu}}{2πi̇t} ϕ(t)dt

(iii) There is a one-to-one correspondence between cdf and chf.

UW-Madison (Statistics)  Stat 609 Lecture 9  2014  7 / 11

beamer-tu-logo

Proof.

(i) is obvious. (ii)-(iii) follow from the fact that any cdf can only have countably many discontinuous points and, hence, it is determined by the F values at continuity points of F.

If a cdf is continuous, then Theorem 3C(ii) gives an inversion formula. For a discrete cdf, we have the following result whose proof is similar to that of Theorem 2C and is omitted.

Theorem C4.

Let F be a discrete cdf with chf ϕ.

(i) If F has a jump at x, then

F(x) − F(x−) = lim_{T →∞} \frac{1}{2T} ∫_{−T}^{T} e^{−i̇tx} ϕ(t)dt

(ii) If D is the set of all discontinuous points of F, then

∑_{x∈D}[F(x) − F(x−)] = lim_{T →∞} \frac{1}{2T} ∫_{−T}^{T} |ϕ(t)|^{2}dt

beamer-tu-logo

Proof.

(i) is obvious. (ii)-(iii) follow from the fact that any cdf can only have countably many discontinuous points and, hence, it is determined by the F values at continuity points of F.

If a cdf is continuous, then Theorem 3C(ii) gives an inversion formula. For a discrete cdf, we have the following result whose proof is similar to that of Theorem 2C and is omitted.

Theorem C4.

Let F be a discrete cdf with chf ϕ.

(i) If F has a jump at x, then

F(x) − F(x−) = lim_{T →∞} \frac{1}{2T} ∫_{−T}^{T} e^{−i̇tx} ϕ(t)dt

(ii) If D is the set of all discontinuous points of F, then

∑_{x∈D}[F(x) − F(x−)] = lim_{T →∞} \frac{1}{2T} ∫_{−T}^{T} |ϕ(t)|^{2}dt

beamer-tu-logo

Theorem 4C(ii) gives a sufficient and necessary condition for a cdf to be continuous. We know that e^{−|t|} is a chf, and

∫_{−T}^{T} (e^{−|t|})^{2}dt ≤ ∫_{−∞}^{∞} e^{−2|t|} dt = 2∫_{0}^{∞} e^{−2t} dt = 1 T > 0

Hence, the corresponding cdf must be continuous. In fact, in this case we know that the cdf corresponding to e^{−|t|} has a pdf according to the following result, which gives a sufficient condition for a cdf having a pdf.

Theorem C5.

Let ϕ(t) be a chf satisfying ∫_{−∞}^{∞} |ϕ(t)|dt < ∞. Then the corresponding cdf F has derivative

F′(x) = \frac{1}{2π} ∫_{−∞}^{∞} e^{−i̇tx} ϕ(t)dt

Proof.

Let x − δ and x + δ be continuity points of F.

UW-Madison (Statistics)  Stat 609 Lecture 9  2014  9 / 11

beamer-tu-logo

Theorem 4C(ii) gives a sufficient and necessary condition for a cdf to be continuous. We know that e^{−|t|} is a chf, and

∫_{−T}^{T} (e^{−|t|})^{2}dt ≤ ∫_{−∞}^{∞} e^{−2|t|} dt = 2∫_{0}^{∞} e^{−2t} dt = 1 T > 0

Hence, the corresponding cdf must be continuous. In fact, in this case we know that the cdf corresponding to e^{−|t|} has a pdf according to the following result, which gives a sufficient condition for a cdf having a pdf.

Theorem C5.

Let ϕ(t) be a chf satisfying ∫_{−∞}^{∞} |ϕ(t)|dt < ∞. Then the corresponding cdf F has derivative

F′(x) = \frac{1}{2π} ∫_{−∞}^{∞} e^{−i̇tx} ϕ(t)dt

Proof.

Let x − δ and x + δ be continuity points of F.

UW-Madison (Statistics)  Stat 609 Lecture 9  2014  9 / 11

beamer-tu-logo

By Theorem C2,

F(x + δ) − F(x − δ) = lim_{T →∞} ∫_{−T}^{T} \frac{e^{−i̇t(x−δ)} − e^{−i̇t(x+δ)}}{2πi̇t} ϕ(t)dt
= lim_{T →∞} ∫_{−T}^{T} \frac{sin(tδ)e^{−i̇tx}}{πt} ϕ(t)dt
= ∫_{−∞}^{∞} \frac{sin(tδ)e^{−i̇tx}}{πt} ϕ(t)dt

where the last equality follows from the fact that

| \frac{sin(tδ)e^{−i̇tx}}{πtδ} ϕ(t)| ≤ |ϕ(t)|

which is integrable. Also, from the previous bound we can apply the dominated convergence theorem to obtain that

F′(x) = lim_{δ→0} \frac{F(x + δ) − F(x − δ)}{2δ} = lim_{δ→0}∫_{−∞}^{∞} \frac{sin(tδ)e^{−i̇tx}}{2πtδ} ϕ(t)dt
= ∫_{−∞}^{∞} lim_{δ→0} \frac{sin(tδ)e^{−i̇tx}}{2πtδ} ϕ(t)dt = \frac{1}{2π} ∫_{−∞}^{∞} e^{−i̇tx} ϕ(t)dt

UW-Madison (Statistics)  Stat 609 Lecture 9  2014  10 / 11

beamer-tu-logo

Theorem C6.

A random variable X is symmetric about 0 iff its chf ϕ_{X} is real-valued.

Proof.

If X and −X have the same distribution, then ϕ_{X}(t) = ϕ_{−X} (t). But ϕ_{−X} (t) = ϕ_{X}(−t). Then ϕ_{X}(t) = ϕ_{X}(−t). Note that sin(−tX) = −sin(tX) and cos (tX) = cos (−tX). Hence E[sin(tX)] = 0 and, thus, ϕ_{X} is real-valued. If ϕ_{X} is real-valued, then ϕ_{X}(t) = E[cos (tX)] and ϕ_{−X} (t) = ϕ_{X}(−t) = ϕ_{X}(t). By Theorem C3, X and −X must have the same distribution.

Theorem C7.

Suppose that X_{1},X_{2},... is a sequence of random variables with chf’s ϕ_{X_{n}}(t), and lim_{n→∞} ϕ_{X_{n}}(t) = ϕ(t), t ∈ R, where ϕ(t) is continuous in t. Then ϕ must be the chf of a random variable X and for all x at which F_{X}(x) is continuous,

lim_{n→∞}F_{X_{n}}(x) = F_{X}(x)
UW-Madison (Statistics)  Stat 609 Lecture 9  2014  11 / 11

beamer-tu-logo

Theorem C6.

A random variable X is symmetric about 0 iff its chf ϕ_{X} is real-valued.

Proof.

If X and −X have the same distribution, then ϕ_{X}(t) = ϕ_{−X} (t). But ϕ_{−X} (t) = ϕ_{X}(−t). Then ϕ_{X}(t) = ϕ_{X}(−t). Note that sin(−tX) = −sin(tX) and cos (tX) = cos (−tX). Hence E[sin(tX)] = 0 and, thus, ϕ_{X} is real-valued. If ϕ_{X} is real-valued, then ϕ_{X}(t) = E[cos (tX)] and ϕ_{−X} (t) = ϕ_{X}(−t) = ϕ_{X}(t). By Theorem C3, X and −X must have the same distribution.

Theorem C7.

Suppose that X_{1},X_{2},... is a sequence of random variables with chf’s ϕ_{X_{n}}(t), and lim_{n→∞} ϕ_{X_{n}}(t) = ϕ(t), t ∈ R, where ϕ(t) is continuous in t. Then ϕ must be the chf of a random variable X and for all x at which F_{X}(x) is continuous,

lim_{n→∞}F_{X_{n}}(x) = F_{X}(x)
UW-Madison (Statistics)  Stat 609 Lecture 9  2014  11 / 11