Margaret Li

Gates Center, Box 352355 · 3800 E Stevens Way NE · Seattle, WA 98195
margsli@cs.washington.edu

Hi! I'm a final year PhD student at University of Washington with Luke Zettlemoyer. I work on NLP, and I'm specifically interested in LLM scaling and efficiency, as well as conditional compute methods and modular architectures.

Prior to my PhD, I was a Research Engineer at FAIR (then Facebook AI Research) in New York. During that time, I primarily worked on open domain dialogue, but also explored RL for agents in text-based games, text-to-multimodal adaptation, and mental health applications.

I am currently on the academic job market, seeking openings for tenure-track research professorship.


Experience

Research Scientist Intern

FAIR at Meta

May 2026 - September 2026

Visiting Researcher

FAIR at Meta

October 2022 - October 2024

Research Scientist Intern

FAIR

May 2022 - September 2022

Research Scholar

Grid (now Lightning) AI Research

September 2020 - June 2021

Research Engineer

Facebook AI Research (FAIR)

August 2017 - September 2020

Software Engineer Intern

Facebook

May 2016 - August 2016

Software Engineer Intern

Point.io

January 2015 - May 2015

Data Analyst Intern

CreditEase

May 2014 - August 2014

Teaching

Head TA / TA

University of Washington
CSE481DS Data Science Capstone
Autumn 2023; Autumn 2022

Invited Lecturer

Program for Algorithmic and Combinatorial Thinking

Zero-Knowledge Proofs; Introduction to Reinforcement Learning

July 2017; July 2019

Teaching Assistant, Interim Instructor

Center for Talented Youth

Fundamentals of Computer Science

July 2017

Teaching Assistant

AwesomeMath

Combinatorics I & II

June 2017

Teaching Assistant

University of Pennsylvania

CIS160 Mathematical Foundations of Computer Science
(Discrete Maths & Graph Theory)

August 2015 - May 2017

Grader and TA

Art of Problem Solving

Various courses

January 2013 - January 2017

Instructor

HuaXia Chinese school

AMC8 & MathCounts Competition Math

August 2012 - May 2013

Private Tutor

Various students

Various subjects: AP subject tests, SAT & SATII prep, ESL, Secondary Math, Primary Math & English

Intermittently 2009 - 2014

Education

University of Washington

PhD
Department of Computer Science and Engineering
ML / NLP
Advisor: Luke Zettlemoyer
September 2020 - now

University of Pennsylvania

Bachelor of Applied Sciences
Computer Science
Minor: Mathematics
August 2013 - May 2017

Homeschooled / International Academy of Beijing

August 2009 - May 2013

Publications

Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models

Terra Blevins, Tomasz Limisiewicz, Suchin Gururangan, Margaret Li, Hila Gonen, Noah A. Smith, Luke Zettlemoyer
arXiv PDF

Under review

In-Context Pretraining: Language Modeling Beyond Document Boundaries

Weijia Shi, Sewon Min, Maria Lomeli, Chunting Zhou, Margaret Li, Gergely Szilvasy, Rich James, Xi Victoria Lin, Noah A. Smith, Luke Zettlemoyer, Scott Yih, Mike Lewis
arXiv PDF

ICLR 2024

Scaling Expert Language Models with Unsupervised Domain Discovery

Suchin Gururangan*, Margaret Li*, Mike Lewis, Tim Althoff, Noah A. Smith, Luke Zettlemoyer
arXiv PDF

JMLR

Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models.

Margaret Li*, Suchin Gururangan*, Tim Dettmers, Mike Lewis, Tim Althoff, Noah A. Smith, Luke Zettlemoyer
arXiv PDF

Under review

Overconfidence in the Face of Ambiguity with Adversarial Data.

Margaret Li*, Julian Michael*
ACL PDF

NAACL 2022 DADC Workshop Best Paper

Don't Sweep your Learning Rate under the Rug: A Closer Look at Cross-modal Transfer of Pretrained Transformers.

Danielle Rothermel, Margaret Li, Tim Rocktäschel, Jakob Foerster
arXiv PDF

arXiv preprint

Recipes for Safety in Open-domain Chatbots.

Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, Emily Dinan
arXiv PDF

NAACL 2021

How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds.

Prithviraj Ammanabrolu, Jack Urbanek, Margaret Li, Arthur Szlam, Tim Rocktäschel, Jason Weston.
arXiv PDF

AAAI 2021 Games Workshop

I love your chain mail! Making knights smile in a fantasy game world.

Shrimai Prabhumoye*, Margaret Li*, Emily Dinan, Jack Urbanek, Douwe Kiela, Jason Weston, Arthur Szlam.
arXiv PDF

NeurIPS 2020 WORDPLAY Workshop

Don't say that! Making inconsistent dialogue unlikely with unlikelihood training.

Margaret Li, Stephen Roller, Ilia Kulikov, Sean Welleck, Spencer Poff, Emily Dinan, Y-Lan Boureau, Kyunghyun Cho, and Jason Weston.
arXiv PDF

ACL 2020

Conversational Agents: Current Progress, Open Problems, and Future Directions.

Stephen Roller*, Y-Lan Boureau*, Jason Weston*, Antoine Bordes, Emily Dinan, Angela Fan, David Gunning, Da Ju, Margaret Li, Spencer Poff, Pratik Ringshia, Kurt Shuster, Eric Michael Smith, Arthur Szlam, Jack Urbanek, Mary Williamson.
arXiv PDF

arXiv preprint

Acute-eval: Improved dialogue evaluation with optimized questions and multi-turn comparisons.

Margaret Li, Jason Weston, and Stephen Roller.
arXiv PDF

NeurIPS 2019 Conversational AI Workshop

Towards empathetic open-domain conversation models: A new benchmark and dataset.

Hannah Rashkin, Eric M. Smith, Margaret Li, and Y-Lan Boureau.
arXiv PDF

ACL 2019

Failure Resume

We've all heard "failure is normal" and "we all have imposter syndrome," but that isn't reflected in the curation of CVs and LinkedIn posts. So I've made what I wish I saw more of -- a 'failure' resume -- in an attempt to provide a little bit of practical grounding for these platitudes. I hope this helps you see my supposed failures, and yours, in a kinder light. So here are a few of the greatest hits of my career-related rejections and such since undergraduate.

There are plenty more for all other aspects of my life too, and I'm happy to chat about those, or anything listed below, in person.

Rejected SWE Internship Applications: 100+

I transferred into CS from Bioengineering during undergrad and had never coded before. With less experience and coursework than many of my peers, I applied to > 100 internships during my undergraduate years and got rejected from them all -- except for Facebook, Google, and a startup my final summer.

Rejection from FAIR

I had 0 ML experience coming out of university, and when I tried to join FAIR, I was rejected the first time. A research scientist later accepted me on their team, and I'm super grateful to have gotten that opportunity despite there being more qualified engineers (who maybe had at least taken a Machine Learning course for their degree). But I was team-less for nearly 3 months, longer than anyone else I knew.

Poor work performance

I struggled to ramp up on both engineering and machine learning knowledge at the same time, because of the aforementioned lack of experience in ...both. So I took longer to get my (mandatory) first promotion than anyone else I knew, including a negative rating the cycle before I did get the promotion.

Rejected Grad School Applications: 5 (or, n-1 out of n)

I didn't get any kind of outreach or interview, didn't even get explicitly rejected. Got ghosted by a professor who had asked me to email them -- no hard feelings to that person, who is definitely very busy! But I did take it as negative personal feedback then.

Rejected Fellowship Applications: 20+

I spent about 4-5 weeks of my life doing nothing but applying, and got exactly one interview over the course of 2 years. One fellowship, notably, gave me wildly opposite feedback the second year I applied compared to the first year, which just goes to show that the application readers aren't perfect and the process is super noisy.

Paper Rejections: 12

Most got in on resubmission, but it didn't make the first rejection pleasant. Some choice phrases I've seen in my reviews are "not novel," "not interesting," "not significant," and "not convincing." The papers I am most proud of are often the ones I've had the most trouble submitting.

Research dry spells / Unproductivity: ~3 years, and a continuing process

I went almost 3 years from 2019 - 2022 without completing a single first-author paper. I was doing research! I just got no results I felt were worth publishing. It turns out that research is full of setbacks, and some may stretch on for seemingly forever, but they do eventually end.


Life (Outside of Research)

I enjoy running slowly with Race Condition Running (yeah it is a pun), and am a now-chronic marathoner. I also spend time walking to every bubble tea shop within 5 miles, singing mostly on-key, rewatching/rereading the Lord of the Rings trilogy, and listening to people on Youtube talk about anything. I'm a prolific window-eater and actual eater (my Beli profile), when I find the time between boba runs.

I've decided to embrace life in the PNW by taking up hiking and snowboarding, or just walking around looking like I could go hiking or snowboarding. I'm always interested in learning about other people's hobbies, so if you meet me, please share yours!