The published literature is filled with methodological errors.
Some of these errors are small and unlikely to change the broad conclusions of the paper. Some of these errors are much larger and lead us to fundamentally question the conclusions.
These errors survive through peer review! The former editor of BMJ (a major medical journal) Richard Smith said “Publication is not the end of the peer review process but a part of it.”
Finding these errors helps to improve science, but it is challenging. If it was easy, then the errors would not survive peer review!
We will learn how to critically read and evaluate the use of statistics in published research. We will do this by developing various “statistical reasoning devices” and apply these devices to papers. These are (for example) general questions that you could ask of any paper and help to reflect upon your understanding of what was done and potentially highlight issues within the paper. For example, two deceptively simple questions “what groups does this paper compare?” and “why does this provide evidence for their broad conclusions”? Sometimes the answers to these questions are obvious. However, sometimes when used properly, these questions illuminate fundamental problems in the statistical reasoning of the paper.
Perhaps you can develop a device for us to use 1) in class and 2) in Karl and Auden’s research!
While critically evaluating published work is itself worthwhile, we also believe that developing these skills will also help you create more convincing evidence in your own work.
Stat 340 or Stat 333 or Stat 310 or consent of instructor.
This course requires a level of statistical maturity that is usually gained by taking two or more statistics courses at a university level. In order to code this into the system, the requirements are Stat 340 or Stat 333 or Stat 310, since you would have needed to complete two courses in order to get there. If you have gained statistical maturity another way, please seek instructor consent.
This class meets in person 2 times a week for 75 minutes.
We will read research papers that “use data” to “do science.” Primarily, we will focus on medical research papers because they tend to be shorter and follow a standard structure.
For each class, you will read a paper beforehand; these are typically around 10 pages, dense with statistical and medical jargon that will require looking up things as you read. In class, we will discuss the paper in the context of the devices that we are currently working on.
For each paper that we read, you will be asked to respond in some way. Different ways that we will have you respond:
We will discuss their limitations. The course assignments will be to read assigned papers and reflect critically upon them.
After taking this course, a student should have the ability to
The majority of the text will be from medical journals.
We will provide parts of Bjørn Andersen’s Methodological Errors in Medical Research (1990) via pdf in canvas.
Homework will be in the form of quizzes before each new paper discussion. If you’ve read the paper, the questions should be relatively straightforward to answer.
Types of studies, Causal inference from observational studies, statistical reviewing devices, Comparability of groups, Outcome measurement (surrogate outcomes and multiple outcomes), Types of biases and their implications on study results, Effective usage of graphics in studies
You are permitted and encouraged to talk to other students, the TA, or Karl about homework. However, you may not present other people’s work as your own. It is not acceptable for one student to use language written by another student.
By virtue of enrollment, each student agrees to uphold the high academic standards of the University of Wisconsin-Madison; academic misconduct is behavior that negatively impacts the integrity of the institution. Cheating, fabrication, plagiarism, unauthorized collaboration, and helping others commit these previously listed acts are examples of misconduct which may result in disciplinary action. Examples of disciplinary action include, but is not limited to, failure on the assignment/course, written reprimand, disciplinary probation, suspension, or expulsion. For detailed information, please see here.
The members of the faculty of the Department of Statistics at UW-Madison uphold the highest ethical standards of teaching, data, and research. They expect their students to uphold the same standards of ethical conduct. Standards of ethical conduct in data analysis and data privacy are detailed on the ASA website, and include:
By registering for this course, you are implicitly agreeing to conduct yourself with the utmost integrity throughout the semester.
Diversity is a source of strength, creativity, and innovation for UW-Madison. We value the contributions of each person and respect the profound ways their identity, culture, background, experience, status, abilities, and opinion enrich the university community. We commit ourselves to the pursuit of excellence in teaching, research, outreach, and diversity as inextricably linked goals.
The University of Wisconsin-Madison fulfills its public mission by creating a welcoming and inclusive community for people from every background–people who as students, faculty, and staff serve Wisconsin and the world.
The University of Wisconsin-Madison supports the right of all enrolled students to a full and equal educational opportunity. The Americans with Disabilities Act (ADA), Wisconsin State Statute (36.12), and UW-Madison policy (Faculty Document 1071) require that students with disabilities be reasonably accommodated in instruction and campus life. Reasonable accommodations for students with disabilities is a shared faculty and student responsibility. Students are expected to inform me of their need for instructional accommodations by the end of the third week of the semester, or as soon as possible after a disability has been incurred or recognized. I will work either directly with the student or in coordination with the McBurney Center to identify and provide reasonable instructional accommodations. Disability information, including instructional accommodations as part of a student’s educational record, is confidential and protected under FERPA.
If you have a complaint about a TA or course instructor, you should feel free to discuss the matter directly with the TA or instructor. If the complaint is about the TA and you do not feel comfortable discussing it with him or her, you should discuss it with the course instructor. Complaints about mistakes in grading should be resolved with the instructor in the great majority of cases. If the complaint is about the instructor (other than ordinary grading questions) and you do not feel comfortable discussing it with him or her, contact the Director of Undergraduate Studies, Professor Cecile Ane, cecile dot ane at wisc dot edu.
If your complaint concerns sexual harassment, please see campus resources listed here. In particular, there are a number of options to speak to someone confidentially.
If you have concerns about climate or bias in this class, or if you wish to report an incident of bias or hate that has occurred in any statistics class, you may contact the Chair of the Statistics Department Climate & Diversity Committee, Professor Karl Rohe (karlrohe at stat dot wisc dot edu). If you would prefer someone who is not the instructor for this course, you may contact XXXXXX (XXXX) who is another member of the Statistics Department Climate & Diversity Committee. You may also use the University’s bias incident reporting system, which you can reach here.
See here.