Machine Learning - Département de Physique de l'Ecole Normale supérieure

Why choose this program? Program, courses and calendar Master ICFP 1st year (2023-2024) 2023-2024 Calendar and timetable Courses Internships Program, courses and calendar Master ICFP 1st year (2022-2023) 2022-2023 Calendar and timetable Courses Internships Program, courses and calendar Master ICFP 2nd year 2023-2024 First Semester Second Semester "Physics Now" Seminars 2019/2020 "Physics Now" Seminars 2020/2021 "Physics Now" Seminars 2021/2022 "Physics Now" Seminars 2023/2024 "Physics Now" Seminars 2022/2023 Internships 2023 2024 calendar Program, courses and calendar Master ICFP 2nd year 2022-2023 First Semester Second Semester "Physics Now" Seminars 2019/2020 "Physics Now" Seminars 2020/2021 "Physics Now" Seminars 2021/2022 "Physics Now" Seminars 2022/2023 Internships 2022 2023 calendar Admission and Language Requirements How to apply Information for students Information for International Students Fees and Scholarships Careers Master Intranet Archives Erreur ( A supprimer) Program, courses and calendar Master ICFP 1st year (2021-2022) Program, courses and calendar Master ICFP 1st year (2021-2022) Program, courses and calendar Master ICFP 2nd year 2021-2022 Program, courses and calendar Master ICFP 2nd year 2020-2021 Program, courses and calendar Master ICFP 2nd year 2019-2020 Archives 2015-2016 Archives 2016-2017 Archives 2017-2018 Archives 2018-2019 Archives 2019-2020 Archives 2020-2021

Machine Learning

Quick links

Next student seminar :
Access to the program

Here you can find information about your internships:
Experimental Internship - Undergraduate program
Master ICFP first year Internship

News : ICFP Research seminars
November 14 - 18, 2022 :
All information about the program

Contact us - Student support and Graduate School office :
Tél : 01 44 32 35 60
enseignement@phys.ens.fr

Quick links

Next student seminar :
Access to the program

Here you can find information about your internships:
Experimental Internship - Undergraduate program
Master ICFP first year Internship

News : ICFP Research seminars
November 14 - 18, 2022 :
All information about the program

Contact us - Student support and Graduate School office :
Tél : 01 44 32 35 60
enseignement@phys.ens.fr

Enseignants : Francis Bach, Lenaic Chizat

Chargé de TD :

ECTS : 3

Langue d’enseignement : Anglais

Description :

Statistical machine learning is a growing discipline at the intersection of computer science and applied mathematics (probability / statistics, optimization, etc.) and which increasingly plays an important role in many other scientific disciplines.
Unlike a course on traditional statistics, statistical machine learning is particularly focused on the analysis of data in high dimension, as well as the efficiency of algorithms to process the large amount of data encountered in multiple application areas such as image or sound analysis, natural language processing, bioinformatics or finance.
The objective of this class is to present the main theories and algorithms in statistical machine learning, with simple proofs of the most important results. The practical sessions will lead to simple implementations of the algorithms seen in class.

( ) Evaluation: practical sessions to finish at home + written in-class exam

( ) Prerequisites: (1) Fluency in linear algebra and differential calculus, (2) one probability class, (3) basic notions in Python

( ) Syllabus and sessions

10 January: Introduction to supervised learning (loss, risk, over-fitting and capacity control + cross-validation, Bayes predictor for classification and regression -17 January: Least-squares regression (all aspects, from linear algebra to statistical guarantees and L2 regularization + practical session)
24 January: Statistical ML without optimization (learning theory, from finite number of hypothesis to Rademacher / covering numbers)
31 January: Local averaging techniques (K-nearest neighbor, Nadaraya-Watson regression: algorithms + statistical analysis + practical session)
7 February: Empirical risk minimization (logistic regression, loss-based supervised learning, probabilistic interpretation through maximum likelihood)
14 February: Convex optimization (gradient descent + nonsmooth + stochastic versions + practical session (logistic regression))
21 Feb: holidays
28 February: Model selection (feature selection, L1 regularization and high-dimensional inference + practical session)
6 March: Kernels (positive-definite kernels and reproducing kernel Hilbert spaces + practical session)
13 March: Neural networks (from one-hidden layer to deep networks + practical session)
20 March: Unsupervised learning (K-means and PCA (potentially with kernels) + mixture models (potentially EM) + practical session)
27 March: review
3 April: exam

Quick links

Next student seminar :
Access to the program

Here you can find information about your internships:
Experimental Internship - Undergraduate program
Master ICFP first year Internship

News : ICFP Research seminars
November 14 - 18, 2022 :
All information about the program

Contact us - Student support and Graduate School office :
Tél : 01 44 32 35 60
enseignement@phys.ens.fr