CISC 859 Pattern Recognition, Fall 2017

Instructor: Dorothea Blostein, 720 Goodwin Hall, 533-6537,

Lectures: Mondays and Thursday 10:00-11:15. Starting Nov 13, class now meets in Walter Light 212

Office hours: Mondays and Thursday 11:30-12:30 in Goodwin 720


The exam is 1:00-4:00 on Friday December 8 in Goodwin 254. (I expect that most students will finish the exam in less than two hours.)

Review session is at 10AM on Thursday December 7 in Ontario Hall 207.

The exam tests your mastery of material covered in the assignments. It will include major questions on the following two topics:

The remainder of the exam will be shorter questions on various topics, similar to the following assignment problems. (Starred problems were mentioned above.) The assignment problems that are not listed above include problems from the textbook that ask you to prove theorems. Working on such problems is essential for getting deep understanding of the course material, but these types of problems take too long to be included on an exam. Other assignment problems are omitted from the above list because they are review or are too detailed. I wrote problem [1] of assignment 4 in brackets because I do not expect you to memorize the sum-of-squared-error criterion or the answer to this question; you should be prepared to answer general questions about clustering.

The exam is done without aids: closed-book and no calculators. Doing this allows me to ask simple questions such as "Describe what curse of dimensionality means" or "Give an example of training on the test data: describe a series of training&testing steps where this problem occurs". I will of course provide a copy of the Anderson Math Grammar with the exam, along with the derivation sequence from "expression" to "letter" (top left of course reader page 87).
I realize that being without a calculator might make you prone to simple math errors. I will not deduct marks for obvious errors like 2*3=5, provided that you are clearly showing your work so I can see that you are correct in how you are attempting to find the answer.

Textbook and Course Reader

Pattern Classification, Second Edition, by Duda, Hart, and Stork, Wiley 2001.
This book is well established as the standard reference book in pattern recognition. Available from Queen’s bookstore; the publisher offers an e-book version.

The CISC859 course reader contains my notes about the course material. Available from Queen’s bookstore. If their stock runs out, the bookstore does not automatically print more copies – you must ask them to print another copy for you.

Course description and prerequisites

CISC859 is an introductory pattern recognition course geared toward students who have some background in computer science. The course material is relevant to many areas of research, including data mining, artificial intelligence, computer vision and signal processing. CISC859 has been taken by graduate students from computing, electrical engineering, mechanical engineering, geology, chemistry, and mathematics. Here is a course overview.

Familiarity with the following subjects is helpful. Students missing this background have successfully taken the course by doing some extra reading.

Marking Scheme; Information about Assignments, Presentation, Project

The marking scheme is as follows.

32% Assignments and exam

38% Study of a pattern recognition topic, with oral presentation and written report. Here is a list of suggested topics; or choose your own topic. 30% Digit recognizer project. Here is a project description and here are sample programs for doing image I/O in C and Java, as well as digit images for classifier training and testing.

Assignments and Schedule

This schedule may be adjusted slightly during the term.

Pattern Recognition Resources

International Association for Pattern Recognition

IAPR is the International Association for Pattern Recognition. IAPR maintains this list of internship positions in pattern recognition for undergraduate and graduate students.
The IAPR education committee provides researcher/student resources for three areas of core technology (symbolic PR; statistical PR; machine learning) and two broad families of application areas (1D signal analysis; 2D image analysis). For each area, they provide links to tutorials and surveys; explanations; online demos; datasets; books; code.
I also recommend taking at look at the information provided by the Technical Committees of the IAPR including:

Software Environments and Tools

Toolbox by Duda, Hart, Stork written in MATLAB. Read this Introduction to the DHS toolbox by graduate student Nawei Chen and me; it illustrates some of the basic pattern recognition ideas we discuss in class. You are not required to buy or use this toolbox. You can use other software tools such as the ones listed below.

Python scikit machine learning and data visualization

R is a free software environment for statistical computing and graphics. A past CISC 859 student writes: "R implements a lot of the concepts that you discuss in the course, and provides many built-in datasets (like the IRIS dataset and a car manufacturer dataset), so it's a great way to get some fast and easy experience with classifiers from a practical perspective. Coupled with the more theoretical perspective provided by the lectures, I found it to be great one-two punch."

Weka tutorial recommended by a former CISC859 student.


Classification in Matlab

OpenCV (Open Source Computer Vision Library) has C++, C, Python and Java interfaces and supports Windows, Linux, Mac OS, iOS and Android.

Mirage is a publicly available Java-based tool for exploratory data analysis written by Tin Kam Ho at Bell Labs; she now works at IBM Watson. Tin is one of the world's top researchers in statistical pattern recognition. This tool offers excellent support for exploratory analysis and visualization of large data sets.

Comparisons of these tools

Computer Vision Resources

CVonline, a compendium of computer vision. Covers many topics, such as Hidden Markov Models (HMMs).
Supplemental information with CVonline: online and hardcopy books, datasets for research and student projects, software packages

Video lectures for an introductory course on computer vision. Topics include flat part recognition, deformable part recognition, range data and stereo data 3D part recognition, detecting & tracking objects in video,and behaviour recognition

CVDICT: Dictionary of Computer Vision and Image Processing.

Other Resources

An extensive list of links for pattern recognition and statistics.

Document Layout Interpretation and its Application lists research groups, conferences, data sets, software, and bibliographies.