Margaret Li

Gates Center, Box 352355 · 3800 E Stevens Way NE · Seattle, WA 98195
margsli@cs.washington.edu

Hi! I'm a PhD student at University of Washington with Luke Zettlemoyer and Tim Althoff. I work on NLP and Dialogue. I'm specifically interested in efficiently training and generating from large language models, conditional compute methods, controllability and safety, and applications to mental health. I'm concurrently a visiting researcher on the FAIR team at Meta.

Previously, I was a Research Engineer at FAIR (then Facebook AI Research) in New York. During that time, I primarily worked on open domain dialogue, but also explored RL for agents in text-based games, text-to-multimodal adaptation, and mental health applications.


Experience

Visiting Researcher

FAIR at Meta

October 2022 - now

Research Scientist Intern

FAIR

May 2022 - September 2022

Research Scholar

Grid AI Research

September 2020 - June 2021

Research Engineer

Facebook AI Research (FAIR)

August 2017 - September 2020

Software Engineer Intern

Facebook

May 2016 - August 2016

Software Engineer Intern

Point.io

January 2015 - May 2015

Data Analyst Intern

CreditEase

May 2014 - August 2014

Teaching

Head TA / TA

University of Washington
CSE481DS Data Science Capstone
Autumn 2023; Autumn 2022

Invited Lecturer

Program for Algorithmic and Combinatorial Thinking

Zero-Knowledge Proofs; Introduction to Reinforcement Learning

July 2017; July 2019

Teaching Assistant, Interim Instructor

Center for Talented Youth

Fundamentals of Computer Science

July 2017

Teaching Assistant

AwesomeMath

Combinatorics I & II

June 2017

Teaching Assistant

University of Pennsylvania

CIS160 Mathematical Foundations of Computer Science
(Discrete Maths & Graph Theory)

August 2015 - May 2017

Grader and TA

Art of Problem Solving

Various courses

January 2013 - January 2017

Instructor

HuaXia Chinese school

AMC8 & MathCounts Competition Math

August 2012 - May 2013

Private Tutor

Various students

Various subjects: AP subject tests, SAT & SATII prep, ESL, Secondary Math, Primary Math & English

Intermittently 2009 - 2014

Education

University of Washington

PhD
Department of Computer Science and Engineering
ML / NLP
Advisors: Luke Zettlemoyer and Tim Althoff
September 2020 - now

University of Pennsylvania

Bachelor of Applied Sciences
Computer Science
Minor: Mathematics
August 2013 - May 2017

Homeschooled / International Academy of Beijing

August 2009 - May 2013

Publications

Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models

Terra Blevins, Tomasz Limisiewicz, Suchin Gururangan, Margaret Li, Hila Gonen, Noah A. Smith, Luke Zettlemoyer
arXiv PDF

Under review

In-Context Pretraining: Language Modeling Beyond Document Boundaries

Weijia Shi, Sewon Min, Maria Lomeli, Chunting Zhou, Margaret Li, Gergely Szilvasy, Rich James, Xi Victoria Lin, Noah A. Smith, Luke Zettlemoyer, Scott Yih, Mike Lewis
arXiv PDF

ICLR 2024

Scaling Expert Language Models with Unsupervised Domain Discovery

Suchin Gururangan*, Margaret Li*, Mike Lewis, Tim Althoff, Noah A. Smith, Luke Zettlemoyer
arXiv PDF

JMLR

Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models.

Margaret Li*, Suchin Gururangan*, Tim Dettmers, Mike Lewis, Tim Althoff, Noah A. Smith, Luke Zettlemoyer
arXiv PDF

Under review

Overconfidence in the Face of Ambiguity with Adversarial Data.

Margaret Li*, Julian Michael*
ACL PDF

NAACL 2022 DADC Workshop Best Paper

Don't Sweep your Learning Rate under the Rug: A Closer Look at Cross-modal Transfer of Pretrained Transformers.

Danielle Rothermel, Margaret Li, Tim Rocktäschel, Jakob Foerster
arXiv PDF

arXiv preprint

Recipes for Safety in Open-domain Chatbots.

Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, Emily Dinan
arXiv PDF

NAACL 2021

How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds.

Prithviraj Ammanabrolu, Jack Urbanek, Margaret Li, Arthur Szlam, Tim Rocktäschel, Jason Weston.
arXiv PDF

AAAI 2021 Games Workshop

I love your chain mail! Making knights smile in a fantasy game world.

Shrimai Prabhumoye*, Margaret Li*, Emily Dinan, Jack Urbanek, Douwe Kiela, Jason Weston, Arthur Szlam.
arXiv PDF

NeurIPS 2020 WORDPLAY Workshop

Don't say that! Making inconsistent dialogue unlikely with unlikelihood training.

Margaret Li, Stephen Roller, Ilia Kulikov, Sean Welleck, Spencer Poff, Emily Dinan, Y-Lan Boureau, Kyunghyun Cho, and Jason Weston.
arXiv PDF

ACL 2020

Conversational Agents: Current Progress, Open Problems, and Future Directions.

Stephen Roller*, Y-Lan Boureau*, Jason Weston*, Antoine Bordes, Emily Dinan, Angela Fan, David Gunning, Da Ju, Margaret Li, Spencer Poff, Pratik Ringshia, Kurt Shuster, Eric Michael Smith, Arthur Szlam, Jack Urbanek, Mary Williamson.
arXiv PDF

arXiv preprint

Acute-eval: Improved dialogue evaluation with optimized questions and multi-turn comparisons.

Margaret Li, Jason Weston, and Stephen Roller.
arXiv PDF

NeurIPS 2019 Conversational AI Workshop

Towards empathetic open-domain conversation models: A new benchmark and dataset.

Hannah Rashkin, Eric M. Smith, Margaret Li, and Y-Lan Boureau.
arXiv PDF

ACL 2019

Failure Resume

People have told me "failure is normal" and "we all have imposter syndrome," but that isn't reflected in the curation of CVs and LinkedIn posts. So I've made what I wish I saw more of -- a 'failure' resume -- in an attempt to provide a little bit of practical grounding for these platitudes. I hope this helps you see my supposed failures, and yours, in a kinder light. So here are a few of the greatest hits of my career-related rejections and such since undergraduate.

There are plenty more for all other aspects of my life too, and I'm happy to chat about those, or anything listed below, in detail. Especially if I know you personally, feel free to reach out anytime.

Rejected SWE Internship Applications: 100+

I transferred into CS from Bioengineering during undergrad and had never coded before. With less experience and coursework than many of my peers, I applied to > 100 internships during my undergraduate years and got rejected from them all -- except for Facebook, Google, and a startup my final summer.

Rejection from FAIR

I had 0 ML experience coming out of university, and when I tried to join FAIR, I was rejected the first time. A research scientist later accepted me on their team, and I'm super grateful to have gotten that opportunity despite there being more qualified engineers (who maybe had at least taken a Machine Learning course for their degree). But I was team-less for nearly 3 months, longer than anyone else I knew.

Poor work performance

I struggled to ramp up on both engineering and machine learning knowledge at the same time, because of the aforementioned lack of experience in ...both. So I took longer to get my (mandatory) first promotion than anyone else I knew, including a negative rating the cycle before I did get the promotion.

Rejected Grad School Applications: 5 (or, n-1 out of n)

I didn't get any kind of outreach or interview, didn't even get explicitly rejected. Got ghosted by a professor who had asked me to email them -- no hard feelings to that person, who is definitely very busy! But I did take it as negative personal feedback then.

Rejected Fellowship Applications: 20+

I spent about 4-5 weeks of my life doing nothing but applying, and got exactly one interview over the course of 2 years. One fellowship, notably, gave me wildly opposite feedback the second year I applied compared to the first year, which just goes to show that the application readers aren't perfect and the process is super noisy.

Rejected Papers: 10

Most got in on resubmission, but it didn't make the first rejection pleasant. Some key phrases I've gotten are "not novel," "not interesting," "not significant," and "not convincing." The papers I am most proud of are the ones I've had the most trouble submitting.

Research dry spells Unproductivity: ~3 years, and constantly

I went almost 3 years from 2019 - 2022 without writing a single first-author paper. I was doing research! I just got no results I felt were worth publishing.


Interests

When I'm not doing research or teaching, I enjoy walking to every bubble tea shop within 5 miles, singing mostly on-key, rewatching/rereading the Lord of the Rings trilogy, working out in the most stereotypically bb fitness class ever, and listening to people on Youtube talk about anything. I also spend far too much time online window-shopping and window-eating, which is exactly what it sounds like, in an attempt to limit my actual shopping and eating (the latter is a lost cause).

I've decided to do my best to fit in while living in the PNW by taking up hiking and snowboarding, or just walking around with hiking and snowboarding equipment. I'm always interested in new hobbies, so if you have a beginner- and klutz-friendly one to share, let me know :).