Home Projects Teaching Publications Software


Machine Learning: Pattern Recognition

Here are the lecture notes

Cover illustration

I teach the Machine Learning: Pattern Recognition master's course at the UvA. This is a challenging course for master's students, which provides a solid introduction to Machine Learning and Pattern Recognition, with a focus on probabilistic modelling. The book of the course is Chris Bishop's Pattern Recognition and Machine Learning, and we cover pretty much all of the topics in the book with this course, with occasional side-excursions to cover additional material. I also wrote some additional lecture notes to spend some more time on the more basic aspects of the course. You are encouraged to use a recent version of acrobat reader to read those, as the PDF contains animations which some PDF readers cannot render.

The course consists of a weekly two-hour lecture, a two-hour exercise session and a two-hour lab. The exercise sessions follow the lectures and reinforce the material by providing exercises that prove or illustrate items seen in the preceding lecture. The computer labs provide more hands-on interaction with the material. Examples of labs include: implementing logistic regression, the E.M. algorithm for mixtures of Gaussians, an email spam filter, face recognition using principal component analysis, etc.

Prof. Dariu Gavrila provides two guest lectures where the application of pedestrian detection from a moving vehicle is used to highlight such issues as feature selection, very high accuracy recognition, dealing with very large datasets, real-time constraints, etc.

Here are the slides to the lectures:

Lecture 1: Introduction
Introductory material and practical stuff.
Lecture 2: Training and Testing
This covers overfitting and generalisation, regularisation, etc.
Lecture 3: Linear discriminants
Simple classifiers
Lecture 4: Bayesian decision theory
Probabilistic modelling, and making decisions based on probabilities.
Lecture 5: Graphical Models
Bayesian Networks, Markov Random Fields and algorithms for efficient inference with these models.
Lecture 6: Expectation Maximisation
Learning in Graphical Models
Lecture 7: Sequential Data
Hidden Markov Models and Kalman Filters (but no CRFs, for now)
Lecture 8: Neural Networks
Multilayer Perceptrons, and techniques to train them
Lecture 9: Kernel Methods
Gaussian Processes and Support Vector Machines
Lecture 10: Dimensionality Reduction
Linear and non-linear dimensionality reduction: Principal Component Analysis, Factor analysis, autoencoders, locally linear embedding, etc.
Lecture 11: Sampling
Basic sampling algorithms and Markov-chain Monte Carlo.
Lecture 12: Combining Models
Bagging, boosting, etc.

Student supervision

Ph.D. Students

I am supervising the following PhD students at the moment:

The Ph.D. students who are now doctors

M.Sc. Students

MSc students that I am currently supervising

Nourolhoda Alemi
Hoda is investigating how active learning can be applied to object detection, when very large datasets of unlabelled images are available, but labelling is labour-intensive
Cem Özalp
Cem is working on recognising emotions in (non-violent) interpersonal touch, as measured by pressure sensors. Volunteers touch each other's lower arm, with the touch conveying different emotions.
Kirstin Rieping (together with Ben Kröse)
Kirstin is investigating how feature extraction and unsupervised models can be combined. Automatic feature generation is combined with a probabilistic LSA model on large datasets of human behaviour as measured by simple environmental sensors in a Minimum Description Length setting. The hope is that the features found in this unsupervised model will also prove to be useful for human interpretation of the data.

Students that whose M.Sc. projects I have supervised in the past

Tjeerd van Dijk
Gesture Recognition using a Time-of-Flight Camera and Hidden Markov Models
Jeroen Kools
Recognizing Non-verbal Social Behavior with Accelerometers
Wouter Josemans
fall detection of elderly people with combined 2D and 3D cameras
Bram Stoeller
Recognizing individuals by appearance across non-overlapping stereo cameras
Domenic Vossen
Detecting Speaker Status in a Social Setting with a Single Triaxial Accelerometer
Silvia-Laura Pintea
Orientation Estimation from ceiling-mounted cameras
Nimrod Raiman
Move and I will tell you who you are: detecting deceptive roles in low-quality [video] data
Yanxia Zhang
Multi-people Tracking using Graph Representation with Ceiling-Mounted Video Cameras