This is a course of mathematics exploring foundational and theorical concepts underlying the development and applications of intelligent systems and deep learning algorithms. The emphasis of the course will be theoretical aspects. The aim of the course is to provide the necessary background to start a graduate research project in this emerging area of investigation.

Topics of the course include: statistical learning theory, Support Vector Machines, geometry of high-dimensional data, manifold learning, dimensionality reduction, expressive power of neural networks, generalization in neural networks, convolutional neural networks.

This is class is targeted to graduate students interested in mastering theoretical tools underlying machine learning and data science.
Even though algorithmic aspects of the topics will not be ignored and exploration of algorithmic issues will be assigned for individual or group projects, this course will not duplicate existing courses on machine learning or data science offered in the Computer Science Department that are focused on algorithmic implementation and computation.

Prerequisite

Students attending this course are expected to have a solid background in linear algebra, undergraduate real analysis (MATH 4331-4332) and basic probability.

Useful background resources:

Review on : Lebesgue Integration (by Prof. C.Heil)

Review on : Banach and Hilbert spaces (by Prof. C.Heil)

Homework and student assignments

Student evaluation is based on two assignments: (i) class participation and (ii) final project.

(i) Class participation: Every week, I will assign simple proofs or numerical tests or literature searches.

(ii) Final project: It requires the critical reading of one or more fundamental research papers in an area closely related to topics of the course. I will select the papers in coordination with the students and will provide assistance during the senester. Please, note that they will be mathematically challenging papers that it would be unrealistic to critically read in a few day but will require a serious study. I will set up several deadlines during the semester to verify the completion of a number of intermediate objectives finalized to the preparation of a written report and a 15-to-20 min in-class presentation.
 - Suggested topics with related papers are listed here. This file will be periodically updated.
 - Please follow the deadlines and instructions given in this document for the preparation of your project: Instructions and timeline.
 - Please, use the following Latex template to submit your reports: Report Latex template
 - Delivery of the final project report is scheduled by April 7. Presentation instructions and presentation calendar is given here (to be updated).

List of lecture notes (to be updated):

Bibliographical References (to be updated)

• This course brings together mathematical tools not usually presented in a single course, for the purpose of solving problems arising in different fields related to the analysis of data. I will be selecting material from several sources including:
• 1. Mathematics of Data Science, by A. S. Bandeira, A. Singer, T. Strohmer available free online here.
• 2. Foundations of Data Science, by Blum, Hopcroft and Kannan’s available free online at https://www.cs.cornell.edu/jeh/book.pdf. It includes material on the Curse of Dimensionality and various topics in machine learning.
• 3. The Elements of Statistical Learning by Hastie, Tibshirani and Friedman, Springer 2017. The authors have made this book freely available on the website https://web.stanford.edu/~hastie/ElemStatLearn/printings/ESLII_print12_toc.pdf This classical treatise covers a broad range of topics in statistical learning theory and neural networks.
• 4. Understanding Machine Learning: From Theory to Algorithms by Shai Shalev-Shwartz and Shai Ben-David, Cambridge University Press 2014. Authors made the book available for personal use here.
• 5. Deep learning theory lecture notes by Matus Telgarski, available here Thses notes include a detailed analysis of the approximation properties of neural networks.
• 6. Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges by Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković, which is available here. Related material can be found at the following webpage
• 7. Deep Learning with PyTorch by Stevens, Antiga and Viehmann. The authors have made this book freely available here It is a practial manual to implement deep learning algorithms in Pytorch - for those students more interested into the numerical/applied side.