Lectures will be in English, with most slides and reading material in English. The final exam may be done in English or Italian.
Course Content
The course is organized around a number of technical and theoretical topics related to the Fundamentals of Machine Learning. Please see the Course Program for more details.
Bishop, Christopher M. Pattern Recognition and Machine Learning. Springer, 2006.
MacKay, David. Information Theory, Inference and Learning Algorithms. Cambridge University Press, 2003.
Deisenroth, Marc Peter, A. Aldo Faisal, and Cheng Soon Ong. Mathematics for machine learning. Cambridge University Press, 2020.
Learning Objectives
The objective of this course is to furnish the concepts and tools most important to the study and application of Machine Learning:
- Knowledge and understanding of the mathematical and statistical foundations of contemporary machine learning.
- Knowledge and understanding of the basic theoretical models for regression, classification, and density estimation.
- Knowledge and understanding of the theoretical foundations of neural networks for classification and regression.
- Knowledge and understanding of contemporary Deep Learning models and techniques.
At the end of this course, the student will be able to analyze real-world machine learning problems and design and apply state-of-the-art solutions to them using best practices and advanced software/hardware tools.
Prerequisites
Understanding of basic probability theory, statistics, and multivariate calculus.
Good mastery of at least one high-level programming language, preferably Python.
Teaching Methods
Lectures and laboratory exercises.
Further information
Type of Assessment
The final grade consists of a combination of:
- Laboratory and homework exercises; and
- An oral presentation of a recent scientific article in which the student demonstrates their mastery of the course material.
Course program
- Foundations of the Foundations: probability theory and statistics for machine learning, probability distributions, basics of information theory, Bayesian versus frequentist interpretations, linear models for regression, linear models for classification, the bias-variance decomposition, overfitting and underfitting, model regularization, probabilistic generative models, probabilistic discriminative models, Maximum Likelihood Estimation (MLE), Maximum a Posteriori (MAP) inference, Bayesian inference.
- Machine Learning: Support Vector Machines (SVMs), kernel machines, graphical models, decision trees, ensemble methods, boosting, bagging, Bayesian model averaging, random forests, Expectation Maximization (EM), mixture density estimation.
- Deep Learning: connectionist models, Hebbian learning rules, the perceptron, neural networks, Stochastic Gradient Descent (SGD), the Backpropagation algorithm, the Multilayer Perceptron (MLP), vanishing and exploding gradients, model size and regularization, network regularization.
- Special Topics and Applications: Long Short-term Memory Networks (LSTMs), natural language processing and language models, Convolutional Neural Networks (CNNs), object recognition, Generative Adversarial Networks (GANs), self-supervised learning, continual learning, domain adaptation, transfer learning.
- Tools, Techniques, and Best Practices: numerical programming, visualization, model diagnostics and monitoring training, MLOps, scikit-learn, PyTorch.