Project IV: Deep learning and generative AI
Nabil Iqbal
Fall 2026 - Spring 2027
I asked ChatGPT "draw an image in the style of Picasso of a group of students training a diffusion model." And it did! I know that we're used to this sort of thing by now but its honestly kind of absurd that this is possible. Even the formulas are reasonable. If you want to learn the mathematics behind how this somewhat tacky image was generated, take this project.
Artificial intelligence and deep learning are having a fundamental impact on all layers of society. A particularly dramatic example comes from generative AI, wherein very simple mathematical principles -- when applied to large datasets -- have resulted in the creation of entities (e.g. large language models like ChatGPT) which appear to talk, respond as a human, and reason about the world. How is this possible?
In this project we will study the principles and applications of generative artificial intelligence. We will understand the mathematics of this, which involves the formal problem of creating new samples from a dataset given a large set of existing samples. We will also discuss the implementation in code, which uses neural networks to parametrize the functions which must be constructed to solve the above problem. Finally, each student will construct their own individual generative AI model to solve a slightly different problem which we will choose from the literature. This project will be extremely hands-on, in that theoretical work will be supplemented by Python programming using the standard deep learning package PyTorch: it is essential that students taking this project have very strong Python skills.
To the left you see the basic MNIST dataset, one of the simplest datasets in machine learning. We will teach a computer to generate new digits given a set of existing ones.
Project topics
A rough outline of the topics we will study is:
-
Basic neural networks -- the multilayer perceptron, convolutional neural networks, the transformer architecture, and their training (as a tool to solve optimization problems)
-
Mathematics of sampling and datasets
-
Different models of generative AI, e.g. the variational autoencoder, diffusion models, flow matching, and the autoregressive transformer
-
In the second half, students will build their own generative AI model. A few possible ideas include:
-
Generating basic images such as the MNIST digits above
-
Creating text (a sort of very basic ChatGPT -- more of a kind of glorified autocomplete)
-
Creating plausible molecules from a dataset of existing ones (currently something with industrial application)
-
Many other things are possible: let me know if you are curious about something not listed above.
-
Mode of Operation and Evidence of Learning for the individual project
This will involve both understanding the mathematics of neural networks and generative AI as well as programming in Python. Students will demonstrate their understanding by discussing the theory, performing simple derivations, and making sure their ideas are correctly implemented in code (i.e. that the neural networks they construct do manage to do what they are supposed to). They will also be expected to clearly communicate their understanding in both written and oral forms.
Prerequisites:
-
A strong ability to program in Python and understanding of applied probability.
Co-requisites:
-
MATH4267: Deep Learning and Artificial Intelligence is highly recommended but perhaps not strictly necessary as we will skip directly to more advanced topics (which is possible due to various historical quirks of the field). But please contact me if you want to take this project without the co-requisite.
References
-
CS231N: a computer science course in machine learning at Stanford (covers things slightly more CS-y and less math-y than we will). Also its about computer vision and not generative AI, but its good to get a flavor of what we will do.
-
Deep Learning by Goodfellow, Bengio, Courville: a great starting textbook, though the field has moved very quickly.
Email: Nabil Iqbal