Harvard Machine Learning Theory

We are a research group focused on building towards a theory of modern machine learning. We are interested in both experimental and theoretical approaches that advance our understanding.

Key topics include: generalization, over-parameterization, representation learning in artificial and natural networks, robustness, dynamics of SGD, and relations to kernel methods.

We also run a research-level seminar series on recent advances in the field. Join the seminar mailing list for talk announcements.

In Spring 2021, Boaz Barak taught Harvard CS 229br, a graduate level course on recent advances and open questions in the theory of machine learning and specifically deep learning.

Opportunities: We are looking for graduate students and postdocs. See opportunities section below. Announcements on positions will also be posted on social media.

People

Researchers

Avatar

Demba Ba

Faculty

Avatar

Yamini Bansal

PhD Student

Avatar

Boaz Barak

Faculty

Avatar

Chi-Ning Chou

PhD Student

Avatar

Ben Edelman

PhD Student

Avatar

Gal Kaplun

PhD Student

Avatar

Sharon Qian

PhD Student

Avatar

Yonadav Shavit

PhD Student

Avatar

Nikhil Vyas

PhD Student

Avatar

Tristan Yang

Undergraduate

Avatar

Fred Zhang

PhD Student

Affiliated

Avatar

Na Li

Faculty

Emeritus

Avatar

Preetum Nakkiran

PhD Student

Coming Soon

Avatar

Sitan Chen

Faculty

Recent Publications

By our group and its members.

(This list is not comprehensive. Also we’re sometimes slow in updates, see individual homepages and the arXiv for the latest publications.)

Inductive Biases and Variable Creation in Self-Attention Mechanisms

For Self-supervised Learning, Rationality Implies Generalization, Provably

The Deep Bootstrap: Good Online Learners are Good Offline Generalizers

Distributional Generalization: A New Kind of Generalization

Learning From Strategic Agents: Accuracy, Improvement, and Causality

Deep Double Descent: Where Bigger Models and More Data Hurt

SGD on Neural Networks Learns Functions of Increasing Complexity

More Data Can Hurt for Linear Regression: Sample-wise Double Descent

Computational Limitations in Robust Classification and Win-Win Results

Minnorm training: an algorithm for training over-parameterized deep neural networks

Adversarial Robustness May Be at Odds With Simplicity

On the Information Bottleneck Theory of Deep Learning

Recent & Upcoming Talks

Large neural networks perform extremely well in practice, providing the backbone of modern machine learning. The goal of this talk is …

This talk will introduce two new tools for summarizing a probability distribution more effectively than independent sampling or …

Seminar Calendar

Join the mailing list for talk announcements.

Opportunities

We are looking for graduate students and postdocs in the theory of machine learning.

For graduate students we have openings in Computer Science, Electrical Engineering,applied mathematics or statistics degrees. If you are applying for graduate studies in CS interested in the theory of machine learning, please mark both “Machine Learning” and “Theory of Computation” as areas of interest. Please also list the names of faculty you want to work with on your application. ML theory group faculty include Demba Ba (Electrical Engineering and Bioengineering), Boaz Barak and Sham Kakade (Computer Science), Cengiz Pehlevan (Applied Mathematics), and Lucas Janson (Statistics). There are also ML theory affiliated faculty in all of the above departments and more. All of us are also open to the possibilities of co-advising students, including across different departments and schools.

Other opportunities include:

Follow us on social media for announcements of more opportunities.