LEARNING OUTCOMES

The primary objective is to provide the students with the basic language and tools of machine learning, with particular emphasis on the supervised case. The approach is based on a formulation of the problem of machine learning as an inverse stochastic problem. The students will also need to know some of the best known algorithms, including both statistical and computational properties.

AIMS AND LEARNING OUTCOMES

At the end of the course, the student will have:

• a good understanding of the basic notions of machine learning and of the related basic mathematical tools; 
• a good comprehension of the basic concepts and techniques of convex optimization
• a good knowledge of the statistical and computational  properties of some well known machine learning   algorithms;
• some ability to implement machine learning algorithms on synthetic and real data sets.

PREREQUISITES

Calculus 1 and 2, probability and linear algebra. 

TEACHING METHODS

Classes using  blackboard

SYLLABUS/CONTENT

• introduction to supervised statistical learning as a regression problem (random design): expected error,  learning algorithms and their consistency;
• convcentration inequalities in Hilbert spaces ;
• reproducing kernel Hilbert spaces ;
• review of the basic theory  of linear operators and spectral theorem for compact operators;
• classical supervised learning algorithms  (regularized least squares, Support Vector Machines, etc.) and their consistency;
• introduction to manifold learning;
• introduction to  deep learning;
• first order methods for smooth functions: gradient method
• first order methods for nonsmooth functions: proximal gradient method, applications to sparsity (Lasso and Elastic Net);
• stochastic gradient methods;
• approximate algorithms for large scale problems.

RECOMMENDED READING/BIBLIOGRAPHY

• L. Rosasco, Introductory Machine Learning Notes, University of Genoa,  (http://lcsl.mit.edu/courses/ml/1718/MLNotes.pdf)
• Steinwart, Ingo, Christmann, Andreas, Support vector machines, Springer, ISBN 978-0-387-77241-7
• Cucker, Felipe, Zhou, Ding-Xuan, Learning theory: an approximation theory viewpoint, Cambridge University Press 2007, ISBN  978-0-521-86559-3
• Boyd, Vandenberghe, Convex Optimization, Cambridge University Press, 2004, ISBN 0 521 83378 7