% This is a simple LaTex sample document that gives a submission format
%   for IEEE PAMI-TC conference submissions.  Use at your own risk.

% Make two column format for LaTex 2e.
\documentclass[11pt,twocolumn]{article} %,twocolumn
%\usepackage[dvips]{graphicx,graphics}
%\usepackage{times,amsmath,amsfonts}
\usepackage[dvips]{graphicx,graphics,color}
\usepackage{times,amsmath,amsfonts}

% Use following instead for LaTex 2.09 (may need some other mods as well).
% \documentstyle[times,twocolumn]{article}

% Set dimensions of columns, gap between columns, and paragraph indent



\setlength{\textheight}{10in} \setlength{\textwidth}{6.875in}
\setlength{\columnsep}{0.3125in} \setlength{\topmargin}{-0.5in}
\setlength{\headheight}{0in} \setlength{\headsep}{0in}
\setlength{\parindent}{1pc}
\setlength{\oddsidemargin}{-.2in}  % Centers text.
\setlength{\evensidemargin}{-.2in}

% Add the period after section numbers.  Adjust spacing.
\newcommand{\Section}[1]{\vspace{-8pt}\section{\hskip -1em.~~#1}\vspace{-3pt}}
\newcommand{\SubSection}[1]{\vspace{-3pt}\subsection{\hskip -1em.~~#1}
        \vspace{-3pt}}
\newcommand{\bq}{\begin{eqnarray*}}
\newcommand{\eq}{\end{eqnarray*}}
\newcommand {\diff}[1] {\frac{\partial}{\partial #1}}
\newcommand{\jacob}[3]{\frac{\partial^2 #3}{\partial #1 \partial #2}}
\newcommand{\der}[2]{\frac{\partial #2}{\partial #1}}
\begin{document}

% Don't want date printed
%\date{January 23, 2003}

% Make title bold and 14 pt font (Latex default is non-bold, 16pt)
\title{Stat 312: Lecture 12\\
Errors in hypothesis testing}

% For single author (just remove % characters)
\author{Moo K. Chung\\
mchung@stat.wisc.edu}
\date{Oct 20, 2004}
% For two authors (default example)
\maketitle
\thispagestyle{empty}



{\bf Type I error}: you reject $H_0$ when $H_0$ is true.
$P(\mbox{Type I error}) = P(\mbox{reject } H_0|H_0 \mbox{ true})
=\alpha.$ The resulting $\alpha$ is called the {\em significance
level} of the test and the corresponding test is called a {\em level
$\alpha$ test}. We will use test procedures that give
$\alpha$ less than a specified level (0.05 or 0.01 usually).\\

I believe that dogs are as smart as people. Assume IQ of a dog
follows $X_i \sim N(\mu, 10^2)$. IQ of 10 dogs are measured: 30,
25, 70, 110, 40, 80, 50, 60, 100, 60. We want to test if dogs are
as smart as people by testing
$$H_0: \mu = 100 \mbox{ vs. } H_1: \mu < 100.$$
\begin{enumerate}
\item[(1)]
Let $\bar X$ be a test statistic and $R=(-\infty,90)$ to be a
rejection region. Let's compute the probability of making Type I
error based on this testing procedure. Under the assumption $H_0$ is
true,
$$X_i \sim N(100,10^2).$$
Under this condition, $\bar X \sim N(100,10)$ and
$$\alpha = P(\bar X < 90).$$
\begin{verbatim}
>help(pnorm)
...
 Usage:
... pnorm(q, mean=0, sd=1,...)
...
> pnorm(90,100,sqrt(10))
[1] 0.0007827011
\end{verbatim}
By using this test procedure, it is highly unlikely to make Type I
error.
\item[(2)]
Let's see what happens when we change the test precedure. When
$R=(-\infty,95)$ with the same test statistic $\bar X$,

 $\alpha = P(\bar X < 95).$
\begin{verbatim}
> pnorm(95,100,sqrt(10))
[1] 0.05692315
\end{verbatim}
\item[(3)]
When $R=(-\infty,99)$, $\alpha = P(\bar X < 99).$
\begin{verbatim}
> pnorm(99,100,sqrt(10))
[1] 0.3759148 \end{verbatim} The test procedure based on rejecting
$H_0$ if $\bar X < 99$ is producing large Type I error. Why?
\end{enumerate}
\begin{figure}
\centering
%\renewcommand{\baselinestretch}{1}
\includegraphics[scale=0.8]{lecture12-1.ps}
\caption{Errors in performing hypothesis testing based on test
procedure if $\bar X < 90$, reject $H_0: \mu = 100$. Left figure:
type I error, right figure: type II error.}
\end{figure}


{\bf Type II error}: you do not reject $H_0$ when H_0$ is false. \bq
\beta &=& P(\mbox{Type II error})\\
&=& P(\mbox{do not reject } H_0|H_0 \mbox{ false })\\
&=& 1- P(\mbox{reject }H_0|H_1 \mbox{ true}) \eq

With the fixed test procedure of rejecting $H_0$ if $\bar X < 90$,
let us compute $\beta$. When $\mu=90$ is true,
\begin{verbatim}
> 1-pnorm(90,90,sqrt(10))
[1] 0.5
> 1-pnorm(90,95,sqrt(10))
[1] 0.9430769
> 1-pnorm(90,99,sqrt(10))
[1] 0.9977867
\end{verbatim}

Now let us see how $\beta$ changes when we change the rejection
region. Assume $\mu =90$ is true.
\begin{verbatim}
> a<- -20:20
> a<- a + 90
%Type II error --> right figure
> plot(a,1-pnorm(a,90,sqrt(10)),type='l')
%Type I error --> left figure
> plot(a,pnorm(a,100,sqrt(10)),type='l')

\end{verbatim}

{\em Review problems} Exercise 8.1., 8.3., 8.5.\\

\end{document}