This is a course of mathematics exploring foundational and theorical concepts underlying the development and applications of intelligent systems and deep learning
algorithms. One major emphasis of this course is the connection between topics from classical and advanced signal processing on one hand and deep neural networks on the other hand.
For instance, convolution operators underpin the design and development of convolutional neural networks; multiresolution analysis underlies several neural network designs such as the
Inception module; manifold learning and sparse approximations provide powerful theoretical tools for the analysis and interpretation of deep learning architectures.
Topics of the course include: Fourier transform and convolution, multiresolution analysis, sparse approximations, manifold learning, statistical learning theory,
dimensionality reduction and spectral clustering, convolutional neural networks.
This is class is targeted to graduate students interested in mastering theoretical tools underlying machine learning and data science. Even though algorithmic aspects of the topics will not be ignored and exploration of algorithmic issues will be assigned for
individual or group projects, this course will not duplicate existing courses on machine learning or data science offered in
the Computer Science Department that are focused on algorithmic implementation and computation.
Prerequisite
Students attending this course are expected to have a solid background in linear algebra, undergraduate real analysis (MATH 4331-4332) and basic probability.
Student evaluation is based on two assignments: (i) presenting selected material from book Deep Learning with PyTorch (see below) and (ii) final project.
(i) Every week, one or two students will be in charge of presenting in class selected material from the book Deep Learning with PyTorch
(ii) The final project will be based on the critical reading of one (or possibly more than one) research paper. I recommend to form a team of 2 students for this project.
The topic of the final project need to be relevant to the class and need to receive my prior approval. Students need to form a team and submit a proposal of their project
by FRI March 19. My detailed instructions for the proposal and final project are given here.
Delivery of the final project and brief in-class presentation are tentatively scheduled between April 23-29. Presentation instructions, list and calendar are here.
This course brings together mathematical tools not usually presented in a single course, for the purpose of solving problems arising in different fields related to the analysis of data sets.
I will be selecting material from several sources:
1. The Mathematics of Signal Processing by Damelin and Miller, Cambridge University Press ISBN-13: 978-1107601048.
This is a mathematically rigorous book covering topics from advances and modern signal processing that useful for practitioners in data-driven fields such as imaging and time series.
2. Foundations of Data Science, by Blum, Hopcroft and Kannan’s available free online at
https://www.cs.cornell.edu/jeh/book2016June9.pdf.
It includes material on the Curse of Dimensionality and various topics in machine learning.
3. The Elements of Statistical Learning by Hastie, Tibshirani and Friedman, Springer 2017. The authors have made this book freely available on the website
https://web.stanford.edu/~hastie/ElemStatLearn/printings/ESLII_print12_toc.pdf
This classical treatise covers a broad range of topics in statistical learning theory and neural networks.
4. Deep Learning with PyTorch by Stevens, Antiga and Viehmann. The authors have made this book freely available on the website
https://pytorch.org/assets/deep-learning/Deep-Learning-with-PyTorch.pdf
It is a practial manual to implement deep learning algorithms in Pytorch - for those students more interested into the numerical/applied side.
5. Additional notes and reference papers will be provided by the instructor.