Sebastian Raschka
Sebastian Raschka
Assistant Professor of Statistics @ UW-Madison

STAT 479 -- Machine Learning (Fall 2018)

Table of Contents


Project Award Winners

Teaching this class was a pleasure, and I am especially happy about how awesome the class projects turned out. Listed below are the winners of the three award categories as determined by ~210 votes. Congratulations!

Course Logistics

When

Where

Instructors

Office Hours

Course Description

Credits: 3

Course Description:

This course will cover the key concepts of machine learning, including classification, regression analysis, clustering, and dimensionality reduction. Students will learn about the fundamental mathematical concepts underlying machine learning algorithms, but this course will equally focus on the practical use of machine learning algorithms using open source libraries from the Python programming ecosystem. Students are expected to participate in an individual, final class project (the topic can be flexibly chosen by the students within the scope of the material taught in this class) and apply the learned concepts to real-world problem-solving.

Along with introducing of the concepts of machine learning, the lectures will provide a refresher on relevant concepts from calculus and linear algebra – a calculus background (e.g., Math 221) and a linear algebra background (e.g., Math 340) is recommended. While this course will also provide an introduction to the basics of the Python programming language for machine learning, it is highly recommended that students are familiar with basic programming and have completed an introductory programming class.

Learning Outcomes:

Course Prerequisites: Consent of instructor.

Course Audience: Students majoring in math or statistics or those wishing to take additional statistics courses.

Credits: 3

Resources

Python Machine Learning, 2nd Edition (highly recommended)

Elements of Statistical Learning (recommended)

Illustrated Guide to Python (recommended)

This book will not be coverered in class. However, some readers asked me for good Python resources as preparation for this class, and this is one of the resources I would recommend. However, there are many other Python learning resources available online.

For instance, another great book is Allen Downey’s Think Python 2e (free PDF available at https://greenteapress.com/wp/think-python-2e/).

Depending on your preferred learning style, also consider learning Python interactively instead/or in addition of reading a Python book. A great interactive resource for learning Python is Codecademy: https://www.codecademy.com. In particular, there is a free, < 10 hr interactive course: https://www.codecademy.com/learn/learn-python.

Class Project

Overview

The goal of working on a class project is three-fold. First, it will provide you with the opportunity to apply the concepts learned in this class creatively, which helps you with understanding material more deeply. Second, designing and working on a unique project in a team which is something that you will encounter, if you haven’t already, rather sooner than later in life, and this course project helps with preparing for that. Third, along with the opportunity to practice and the satisfaction of working creatively, students can use this project to enhance their portfolio or resume.

Note about grading

There is no “perfect project.” While you are encouraged to be ambitious, the most important aspect of this project is your learning experience. Hence, you don’t want to pick something that is too easy for you, but similarly, you don’t want to choose a project where you are not certain that is out of the scope of this class. The project proposal is not graded by how exciting your project is but based on whether you follow the objectives of the project proposal, project presentation, and project report. For instance, if your project ends up being unsuccessful – for example, if you choose to design a classifier and it doesn’t achieve the desired accuracy – it will not negatively affect your grade as long as you are honest, describe the potential issues well, and suggest improvements or further experiments. Again, the objective of this project is to provide you with hands-on practice and an opportunity to learn.

The project consists of 3 parts: a project proposal, a short project presentation, and a project report. The expectations for each part will be discussed in the following sections.

1) Project Proposal

The main purpose of the project proposal is to receive feedback from the TAs/the instructor regarding whether your project is feasible and whether it is within the scope of this class. Also, the project proposal offers a chance to receive useful feedback and suggestions on your project.

For this project, you will be working in a team consisting of three students. The members of each team will be randomly assigned by the TAs/instructor. If you have any concerns working with someone in your group, please talk to a TA or the instructor for accommodations.

Proposal Format:

Introduction:

Motivation:

Evaluation:

Resources:

Contributions:

You are expected to share the workload evenly, and every group member is expected to participate in both the experiments and writing. (As a group, you only need to submit one proposal and one report, though. So you need to work together and coordinate your efforts.)

It is crucial that you talk to each other regularly!!! Schedule regular meetings and/or use online communication tools (e.g., Gitter, Slack, or email) to stay in touch with your group members throughout the semester regarding the process of your project.

Modifications to the Proposal. After you have received feedback from the TAs/the instructor and your project proposal has been graded, you are advised to stick to the project outline in the proposal as closely as possible. However, if there is a concept introduced in a later lecture (for instance, a machine learning algorithm that you think is more appropriate then the one you proposed), you have the option to modify your proposal, but you are not penalized if you don’t. If you wish to update your project outline, talk to a TA first.

2) Project Presentation

During the last three lectures, you will be presenting your project to the class. The presentation is “free form” but should cover the following:

The presentation should be 8-10 minutes long, plus 2 minutes will be reserved for questions. All members of the group should participate in the presentation.

The voting card should be filled out as follows:

  1. Title of the Presentation, x/10, y/10, z/10
  2. Title of the Presentation, x/10, y/10, z/10 …

where

The awards will be computed based on the highest number of points for each category. However, one project can only receive one of the prizes.

Each of the three cards handed in will provide 3 bonus points towards your project report grade (9 pts in total).

3) Project Report

The project report is expected to be 6-8 pages long (excluding references) and should contain the follwing sections:

  1. Introduction
  2. Related Work
  3. Proposed Method
  4. Experiments
  5. Results and Discussion
  6. Conclusions
  7. Contributions

More details are provided in the LaTeX report template at https://github.com/rasbt/stat479-machine-learning-fs18.

Also, you are required to submit all the code, computations, and experiments you developed and conducted for this project. Note that the quality of code will not have any influence on your grad and will merely serve as a basis to establish that the report contains original and “real” results.

Optional: Sharing your Project

You are encouraged to share your project/final project report online after you completed the course – for example, via GitHub or on a personal website online. Besides, I would be happy to write a blog article summarizing each project in a few sentences, including a link to your project website (if applicable). However, note that your project will only be included with your explicit consent, and if you don’t want to share your project online, that’s totally fine.

Grading

The final grade will be computed using the following weighted grading scheme:

Other Important Course Information

RULES, RIGHTS & RESPONSIBILITIES

See the Guides’s Rules, Rights and Responsibilities

ACADEMIC INTEGRITY

By enrolling in this course, each student assumes the responsibilities of an active participant in UW-Madison’s community of scholars in which everyone’s academic work and behavior are held to the highest academic integrity standards. Academic misconduct compromises the integrity of the university. Cheating, fabrication, plagiarism, unauthorized collaboration, and helping others commit these acts are examples of academic misconduct, which can result in disciplinary action. This includes but is not limited to failure on the assignment/course, disciplinary probation, or suspension. Substantial or repeated cases of misconduct will be forwarded to the Office of Student Conduct & Community Standards for additional review. For more information, refer to studentconduct.wiscweb.wisc.edu/academic-integrity/.

ACCOMMODATIONS FOR STUDENTS WITH DISABILITIES

McBurney Disability Resource Center syllabus statement: “The University of Wisconsin-Madison supports the right of all enrolled students to a full and equal educational opportunity. The Americans with Disabilities Act (ADA), Wisconsin State Statute (36.12), and UW-Madison policy (Faculty Document 1071) require that students with disabilities be reasonably accommodated in instruction and campus life. Reasonable accommodations for students with disabilities is a shared faculty and student responsibility. Students are expected to inform faculty [me] of their need for instructional accommodations by the end of the third week of the semester, or as soon as possible after a disability has been incurred or recognized. Faculty [I], will work either directly with the student [you] or in coordination with the McBurney Center to identify and provide reasonable instructional accommodations. Disability information, including instructional accommodations as part of a student’s educational record, is confidential and protected under FERPA.” http://mcburney.wisc.edu/facstaffother/faculty/syllabus.php

DIVERSITY & INCLUSION

Institutional statement on diversity: “Diversity is a source of strength, creativity, and innovation for UW-Madison. We value the contributions of each person and respect the profound ways their identity, culture, background, experience, status, abilities, and opinion enrich the university community. We commit ourselves to the pursuit of excellence in teaching, research, outreach, and diversity as inextricably linked goals.

The University of Wisconsin-Madison fulfills its public mission by creating a welcoming and inclusive community for people from every background – people who as students, faculty, and staff serve Wisconsin and the world.” https://diversity.wisc.edu/

Schedule

Note that this is a tentative schedule subject to changes.

Below is a list of topics we aim to cover. However, we will take our time, and it is more important to build a good understanding of the core concepts and the field in general rather than covering one more algorithm. Keep in mind that a good foundation will enable you to study and understand additional algorithms if the need arises.



Topics

Part I: Introduction

Part II: Computational Foundations

Part III: Tree-Based Methods

Part IV: Evaluation

Part V: Dimensionality Reduction

Part VI: Bayesian Learning

Part VII: Regression and Unsupervised Learning

The following topics will be covered in the beginning of the Deep Learning class next Spring.

Part VIII: Introduction to Artificial Neural Networks


Calendar

Date Event Description Lecture Material Announcements
Part I: Introduction
Thu,
Sep 06
Day 1 ● Course Overview
● L01: ML Intro & Overview
[L01 Intro - Notes]
[L01 Intro - Slides]
Tue,
Sep 11
Day 2 ● L01 Cont'd
Thu,
Sep 13
Day 3 ● L02: Intro to Supervised Learning: KNN [L02 KNN - Notes]
[L02 KNN - Slides]
[L02 KNN - Demo]
Part II: Computational Foundations
Tue,
Sep 18
Day 4 ● L03: Using Python [L03 Python - Notes] Problem Set 1
Available
Thu,
Sep 20
Day 5 ● L04: Python's Scientific Computing Stack [L04 - Notes (ipynb)]
[L04 - Notes (pdf)]
Tue,
Sep 25
Day 6 ● L05: Data Preprocessing and
Machine Learning with Scikit-Learn
[L05 - Slides]
[L05 - Notes (ipynb)]
Part III: Tree-Based Methods
Thu,
Sep 27
Day 7 ● L05 Cont'd
● L06: Decision Trees
[L06 Trees - Slides]
[L06 Trees - Notes]
[L06 Trees - Demo]
Tue,
Oct 02
Day 8 ● HW1 Discussion
● L06 Cont'd
Problem Set 1
Due
Thu,
Oct 04
Day 9 ● L06 Cont'd
● L07: Ensemble Methods
[L07 Ensembles - Notes]
[L07 Ensembles - Slides]
Tue,
Oct 09
Day 10 ● L07 Cont'd
Thu,
Oct 11
Day 11 ● L07 Cont'd
Tue,
Oct 16
Day 12 ● L08: Model Eval. 1 - Overfitting [L08 Model Eval 1 - Slides]
[L08 Model Eval 1 - Notes]
Thu,
Oct 18
Day 13 Midterm
Exam
Midterm
Exam
Tue,
Oct 23
Day 14 ● L08 Cont'd Problem Set 2
Available
Thu,
Oct 25
Day 15 ● L09: Model Eval. 2 - Conf. Intervals [L09 Model Eval 2 - Slides]
[L09 Model Eval 2 - Notes]
[L09 Model Eval 2 - Code]
Project
Proposal
Due
Tue,
Oct 30
Day 16 ● L09 Cont'd
Thu,
Nov 1
Day 17 ● L10: Model Eval. 3 - Cross-Validation [L10 Model Eval 3 - Notes]
[L10 Model Eval 3 - Slides]
[L10 Model Eval 3 - Code]
Tue,
Nov 6
Day 18 ● L10 Cont'd
Thu,
Nov 8
Day 19 ● L11: Model Eval. 4 - Algorithm Selection [L11 Model Eval 5 - Notes]
[L11 Model Eval 4 - Slides]
[L11 Model Eval 4 - Code]
Problem Set 2
Due
Tue,
Nov 13
Day 20 ● L11 Cont'd
Thu,
Nov 15
Day 21 ● L12: Model Eval. 5 - Perform. Metrics [L12 Model Eval 5 - Slides]
Tue,
Nov 20
Day 22 ● L12 Cont'd
● L13: Feature Selection
[L13 Feat. Select. - Slides]
[L13 Feat. Select. - Code]
Problem Set 3
Available
Thu,
Nov 22
-- No class: Thanksgiving, university closed
Tue,
Nov 27
Day 23 ● L13 Cont'd
Thu,
Nov 29
Day 24 ● L14: Feature Extraction [L14 Feat. Extract. - Slides]
[L14 Feat. Extract. - Code]
Problem Set 3
Due on Dec 3
Part X: Wrapping it Up -- Final Exam and Class Project
Tue,
Dec 04
Day 25 Project Presentations
Thu,
Dec 06
Day 26 Project Presentations
Tue,
Dec 11
Day 27 Project Presentations [Project Report Template] Project
Report
Due
Thu,
Dec 13
-- No Class: Official UW Study Day
Thu,
Dec 20
Final Exam
7:45 am - 9:45 am
Ingraham Hall 19