Margaret Li

Gates Center, Box 352355 · 3800 E Stevens Way NE · Seattle, WA 98195
margsli@cs.washington.edu

Hi! I'm a final year PhD student at the University of Washington with Luke Zettlemoyer. I work on NLP, and I'm specifically interested in LLM scaling and efficiency, as well as conditional compute methods and modular architectures.

Prior to my PhD, I was a Research Engineer at FAIR (then Facebook AI Research) in New York. During that time, I primarily worked on open domain dialogue, but also explored RL for agents in text-based games, text-to-multimodal adaptation, and mental health applications.

I am currently on the academic job market, seeking openings for tenure-track research professorship.


Education

University of Washington

PhD
Department of Computer Science and Engineering
Advisor: Luke Zettlemoyer
September 2020 - May 2027 (expected)

University of Pennsylvania

Bachelor of Applied Sciences
Computer Science
Minor: Mathematics
August 2013 - May 2017

Selected Publications

A full list of my publications may be found on my Google Scholar profile.

Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models.

Margaret Li*, Suchin Gururangan*, Tim Dettmers, Mike Lewis, Tim Althoff, Noah A. Smith, Luke Zettlemoyer
Paper PDF

Scaling Expert Language Models with Unsupervised Domain Discovery

Suchin Gururangan*, Margaret Li*, Mike Lewis, Tim Althoff, Noah A. Smith, Luke Zettlemoyer
Paper PDF

Artificial Hivemind: The open-ended homogeneity of language models (and beyond)

Liwei Jiang, Yuanjun Chai, Margaret Li, Mickel Liu, Raymond Fok, Nouha Dziri, Yulia Tsvetkov, Maarten Sap, Yejin Choi
Paper PDF

Predicting Vs. Acting: A trade-off between world modeling & agent modeling

Margaret Li*, Weijia Shi*, Artidoro Pagnoni, Peter West, Ari Holtzman
Paper PDF

Byte latent transformer: Patches scale better than tokens

Artidoro Pagnoni, Ramakanth Pasunuru, Pedro Rodriguez, John Nguyen, Benjamin Muller, Margaret Li, Chunting Zhou, Lili Yu, Jason E Weston, Luke Zettlemoyer, Gargi Ghosh, Mike Lewis, Ari Holtzman, Srini Iyer.
Paper PDF

(Mis)Fitting Scaling Laws: A survey of scaling law fitting techniques in deep learning

Margaret Li*, Sneha Kudugunta*, Luke Zettlemoyerr
Paper PDF

Data Scarcity and Model Sparsity: Mixtures-of-experts overfit more to repeated data

Atindra Jha*, Margaret Li*, Jure Leskovec, Percy Liang, Luke Zettlemoyer
Paper PDF

Slicing and dicing: Configuring optimal mixtures of experts

Margaret Li, Sneha Kudugunta, Danielle Rothermel, Luke Zettlemoyer
Paper PDF


Employment Experience

Research Scientist Intern

FAIR at Meta

May 2026 - September 2026

Visiting Researcher

FAIR at Meta

October 2022 - October 2024

Research Scientist Intern

FAIR

May 2022 - September 2022

Research Scholar

Grid (now Lightning) AI Research

September 2020 - June 2021

Research Engineer

Facebook AI Research (FAIR)

August 2017 - September 2020

Software Engineer Intern

Facebook

May 2016 - August 2016

Software Engineer Intern

Point.io

January 2015 - May 2015

Data Analyst Intern

CreditEase

May 2014 - August 2014

Teaching Experience

Head TA / TA

University of Washington
Data Science Capstone
Autumn 2023; Autumn 2022

Invited Lecturer

Program for Algorithmic and Combinatorial Thinking

Zero-Knowledge Proofs; Introduction to Reinforcement Learning

July 2017; July 2019

Teaching Assistant, Interim Instructor

Center for Talented Youth

Fundamentals of Computer Science

July 2017

Teaching Assistant

AwesomeMath

Combinatorics I & II

June 2017

Teaching Assistant

University of Pennsylvania

CIS160 Mathematical Foundations of Computer Science
(Discrete Maths & Graph Theory)

August 2015 - May 2017

Grader and TA

Art of Problem Solving

Various courses

January 2013 - January 2017

Instructor

HuaXia Chinese school

AMC8 & MathCounts Competition Math

August 2012 - May 2013

Private Tutor

Various students

Various subjects: AP subject tests, SAT & SATII prep, ESL, Secondary Math, Primary Math & English

Intermittently 2009 - 2014

Failure Resume

It's easy for me to say "failure is normal" and "we all have imposter syndrome," but that isn't reflected in the curation of CVs and LinkedIn posts. So I've made what I wish I saw more of -- a 'failure' resume -- in an attempt to provide a little bit of practical grounding for these platitudes. I hope this helps you see my supposed failures, and yours, in a kinder light. So here are a few of the greatest hits of my career-related rejections and such since undergraduate.

There are plenty more for all other aspects of my life too, and I'm happy to chat about those, or anything listed below, in person.

Rejected SWE Internship Applications: 100+

I transferred from Bioengineering to CS during undergrad and had never coded before. With less experience and coursework than many of my peers, I applied to > 100 internships during my undergraduate years and got rejected from them all -- except for Facebook, Google, and a startup my final summer.

Rejection from FAIR

I had 0 ML experience coming out of university, not even coursework. When I tried to join FAIR, I was rejected initially in favor of another candidate who had ML research experience. A research scientist later accepted me on their team, and I'm super grateful to have gotten that opportunity despite there being more qualified engineers. But I was team-less for nearly 3 months, longer than anyone else I knew.

Poor work performance

I struggled to ramp up on both engineering and machine learning knowledge at the same time, because of the aforementioned lack of experience in ...both. So I took longer to get my (mandatory) first promotion than anyone else I knew, including a negative rating the cycle before I did get the promotion.

Rejected Grad School Applications: 5 (or, n-1 out of n)

I didn't get any kind of outreach or interview, didn't even get explicitly rejected. Got ghosted by a professor who had asked me to email them -- no hard feelings to that person, who is definitely a very busy person with a crowded inbox! But I did take it as negative personal feedback then.

Rejected Fellowship Applications: 20+

I spent about 5 weeks of my first two years doing nothing but applying, and got exactly one interview over the course of 2 years. One fellowship gave me wildly opposite feedback the second year I applied compared to the first year, which just goes to show that the application readers are inconsistent and the entire process is super noisy.

Paper Rejections: 12

Most got in on resubmission, but it didn't make the first rejection pleasant. Some choice phrases I've seen in my reviews are "lacks novelty," "not very interesting," "not significant," and "not convincing." The papers I am most proud of are often the ones I've had the most trouble submitting.

Research dry spells / stuck-ness: ~3 years, and a continuing process

I went almost 3 years from 2019 - 2022 without completing a single first-author paper. I was doing research! I just got no positive results I felt were worth publishing. (In retrospect, I should have written up my negative results, some of which I still think are really fascinating.) It turns out that research is full of setbacks, and some may stretch on for seemingly forever, but they do eventually end.


Life (Outside of Research)

I enjoy running slowly with Race Condition Running (yeah it is a pun), and have embraced my new identity as a chronic middle-of-the-pack marathoner. The PNW made me take up hiking and snowboarding, or just walking around looking like I could go hiking or snowboarding. I also spend time walking to every bubble tea shop within 5 miles, singing mostly on-key, rewatching/rereading the Lord of the Rings trilogy, and listening to people on Youtube talk about anything. I'm a prolific window-eater and actual eater (my Beli profile), when I find the time between boba runs.

I'm always interested in learning about other people's hobbies, so if you meet me, please share yours!