Machine Learning: Pattern Recognition
Here are the lecture notes
I teach the Machine Learning: Pattern Recognition master's course at the UvA. This is a challenging course for master's students, which provides a solid introduction to Machine Learning and Pattern Recognition, with a focus on probabilistic modelling. The book of the course is Chris Bishop's Pattern Recognition and Machine Learning, and we cover pretty much all of the topics in the book with this course, with occasional side-excursions to cover additional material. I also wrote some additional lecture notes to spend some more time on the more basic aspects of the course. You are encouraged to use a recent version of acrobat reader to read those, as the PDF contains animations which some PDF readers cannot render.
The course consists of a weekly two-hour lecture, a two-hour exercise session and a two-hour lab. The exercise sessions follow the lectures and reinforce the material by providing exercises that prove or illustrate items seen in the preceding lecture. The computer labs provide more hands-on interaction with the material. Examples of labs include: implementing logistic regression, the E.M. algorithm for mixtures of Gaussians, an email spam filter, face recognition using principal component analysis, etc.
Prof. Dariu Gavrila provides two guest lectures where the application of pedestrian detection from a moving vehicle is used to highlight such issues as feature selection, very high accuracy recognition, dealing with very large datasets, real-time constraints, etc.
Here are the slides to the lectures:
- Lecture 1: Introduction
- Introductory material and practical stuff.
- Lecture 2: Training and Testing
- This covers overfitting and generalisation, regularisation, etc.
- Lecture 3: Linear discriminants
- Simple classifiers
- Lecture 4: Bayesian decision theory
- Probabilistic modelling, and making decisions based on probabilities.
- Lecture 5: Graphical Models
- Bayesian Networks, Markov Random Fields and algorithms for efficient inference with these models.
- Lecture 6: Expectation Maximisation
- Learning in Graphical Models
- Lecture 7: Sequential Data
- Hidden Markov Models and Kalman Filters (but no CRFs, for now)
- Lecture 8: Neural Networks
- Multilayer Perceptrons, and techniques to train them
- Lecture 9: Kernel Methods
- Gaussian Processes and Support Vector Machines
- Lecture 10: Dimensionality Reduction
- Linear and non-linear dimensionality reduction: Principal Component Analysis, Factor analysis, autoencoders, locally linear embedding, etc.
- Lecture 11: Sampling
- Basic sampling algorithms and Markov-chain Monte Carlo.
- Lecture 12: Combining Models
- Bagging, boosting, etc.
Student supervision
Ph.D. Students
I am supervising the following PhD students at the moment:
- Deepak Viswanathan
- Saskia Robben
- Ahmed Nait Aicha
- Ninghang Hu
The Ph.D. students who are now doctors
- Athanasios "Thanasis" Noulas
- Tim van Kasteren
- Julian Kooij
M.Sc. Students
MSc students that I am currently supervising
- Nourolhoda Alemi
- Hoda is investigating how active learning can be applied to object detection, when very large datasets of unlabelled images are available, but labelling is labour-intensive
- Cem Özalp
- Cem is working on recognising emotions in (non-violent) interpersonal touch, as measured by pressure sensors. Volunteers touch each other's lower arm, with the touch conveying different emotions.
- Kirstin Rieping (together with Ben Kröse)
- Kirstin is investigating how feature extraction and unsupervised models can be combined. Automatic feature generation is combined with a probabilistic LSA model on large datasets of human behaviour as measured by simple environmental sensors in a Minimum Description Length setting. The hope is that the features found in this unsupervised model will also prove to be useful for human interpretation of the data.
Students that whose M.Sc. projects I have supervised in the past
- Tjeerd van Dijk
- Gesture Recognition using a Time-of-Flight Camera and Hidden Markov Models
- Jeroen Kools
- Recognizing Non-verbal Social Behavior with Accelerometers
- Wouter Josemans
- fall detection of elderly people with combined 2D and 3D cameras
- Bram Stoeller
- Recognizing individuals by appearance across non-overlapping stereo cameras
- Domenic Vossen
- Detecting Speaker Status in a Social Setting with a Single Triaxial Accelerometer
- Silvia-Laura Pintea
- Orientation Estimation from ceiling-mounted cameras
- Nimrod Raiman
- Move and I will tell you who you are: detecting deceptive roles in low-quality [video] data
- Yanxia Zhang
- Multi-people Tracking using Graph Representation with Ceiling-Mounted Video Cameras