CSE 417A: Fall 2014

Washington University in St. Louis
Department of Computer Science and Engineering

CSE 417A: Introduction to Machine Learning

Fall 2014

ANNOUNCEMENTS

Homework 6 is out. Due December 2nd in lecture.

Here are general instructions for how to use the SVN repository and the files we provide.

We will use Piazza for discussions and questions. This will serve as a permanent announcement linking to the Piazza page.

Welcome to CSE 417A: Introduction to Machine Learning! We will meet on Tuesdays and Thursdays from 10:00-11:30 AM in McDonnell 162. The first meeting will be on the 26th of August.

OVERVIEW

This course is a broad introduction to machine learning, covering supervised learning, unsupervised learning, decision-making under uncertainty, and reinforcement learning. Topics that will be covered include generative and discriminative techniques for classification (likely including regression, Naive Bayes, decision trees, neural networks, nearest-neighbor methods, support vector machines, and boosting), clustering and dimensionality reduction, dynamic programming, and temporal difference methods. Note that there is some overlap with topics in the 500-level courses on Artificial Intelligence and Machine Learning, but the material covered in this class will be at a more elementary level.

STAFF

Instructor: Sanmay Das
Office: Jolley 510
Office hours: Tuesdays from 2-3 PM, Thursdays from 11:30AM-12:30PM, and by appointment.

TAs: There are several TAs for the class. Justin Peabody will coordinate the TAs and grading, and hold occasional recitation sessions. Ignacio de Erausquin, Mark Heimann, Nick Kolkin, Benjamin (Benjy) Roberts and Charles (Chip) Schaff will all hold regular office hours (times and places TBA), grade homeworks, and answer questions on Piazza. TA office hours will be held in Urbauer 114, the ACM Lounge. The complete office hour schedule is as follows (Sanmay's office hours will be in Jolley 510):

Mondays	3-5 (Nick), 5-7 (Mark)
Tuesdays	2-3 (Sanmay)
Wednesdays	10-12 (Ignacio), 4-6 (Chip)
Thursdays	11:30-12:30 (Sanmay)
Fridays	11:30-1:30 (Benjy)

POLICIES

Detailed policies are in the official syllabus. A few points to highlight: please read and understand the collaboration policy. Homeworks will typically be due at the beginning of lecture on the day specified -- for any requests for extensions, you must go through Engineering Student Services and have someone from ESS send me an official email explaining why you need an extension. I will not grant extensions without such an email. There will be two in-class exams, each covering approximately half the course material, and no separate final exam.

Course Syllabus

TEXTBOOKS

There are two primary textbooks for this class.

LFD: Learning From Data, Abu-Mostafa, Magdon-Ismail, and Lin.

AIMA: Artificial Intelligence: A Modern Approach, Russell & Norvig.

PREREQUISITES

CSE 241 and ESE 326 (or Math 320) or equivalents; Linear algebra and multi-variable calculus. If you do not have a basic background in CS through data structures and algorithms, or if you are not comfortable with calculus and probability, you may have a hard time in this class.

PROGRAMMING

Programming tasks should be done in Matlab. There will be Matlab help sessions on Friday during the first week of class.

SCHEDULE

Date	Topics	Readings	Extras
Aug 26	Introduction. Course policies. Course overview.	Slides, LFD 1.1 and 1.2
Aug 28	The perceptron learning rule, and the perceptron convergence theorem	LFD 1.1.2, Problem 1.3
Sep 2	Generalizing outside the training set. Error and noise.	LFD 1.3, 1.4	HW1 out (due Sep 9)
Sep 4	Infinite hypothesis spaces. Growth functions and the VC-dimension.	LFD 2.1 (minus the "safe skip" portion)
Sep 9	The VC generalization bound.	LFD 2.2	HW1 due
Sep 11	The Bias-Variance Tradeoff.	LFD 2.3	HW2 out (due Sep 18)
Sep 16	Linear models: The Pocket algorithm, linear regression.	LFD 3.1, 3.2
Sep 18	Linear regression continued; logistic regression.	LFD 3.2, 3.3	HW2 due
Sep 23	Gradient descent.	LFD 3.3
Sep 25	Nonlinear transformations; overfitting.	LFD 3.4, started 4.1	HW3 out. SVN instructions.
Sep 30	Overfitting and regularization.	LFD 4.1, started 4.2	Malik Magdon-Ismail's slides on overfitting
Oct 2	Regularization and validation.	LFD 4.1, 4.2	HW3 due. HW4 out.
Oct 7	Validation; Occam's razor	LFD 4.2, started 5.1	Malik Magdon-Ismail's slides on validation
Oct 9	Three learning principles	LFD Chapter 5
Oct 14	Decision Trees	AIMA 18.3	HW4 due.
Oct 16	In-class exam #1
Oct 21	Midterm discussion. Decision trees contd.	AIMA 18.3
Oct 23	Decision trees: overfitting and pruning.	AIMA 18.3
Oct 28	Bagging decision trees. Artificial neural networks.	CiML 11.1, AIMA 18.7
Oct 30	ANNs contd.	AIMA 18.7	HW5 out
Nov 4	Nonparametric models	AIMA 18.8
Nov 6	Nonparametric models, contd.	AIMA 18.8
Nov 11	No class (Sanmay at INFORMS)
Nov 13	Support vector machines	AIMA 18.9	HW5 due
Nov 18	Boosting	AIMA 18.10. Also, this short exposition	HW6 out
Nov 20	Guest lecture on unsupervised learning by Prof. Kilian Weinberger
Nov 25	Introduction to MDPs.	AIMA 17.1
Dec 2	A little more on MDPs. Exploration vs. exploitation (not on exam). Wrap-up	AIMA 17.2.1	HW6 due
Dec 4	In-class exam #2

HOMEWORKS

HW1: Out September 2, Due September 9.
HW2: Out September 11, Due September 18.
HW3: Out September 25, Due October 2. SVN instructions
HW4: Out October 2, Due ~~October 9~~ October 14.
HW5: Out October 30, Due November 13.
HW6: Out November 18, Due December 2.