Margaret Li

Gates Center, Box 352355 · 3800 E Stevens Way NE · Seattle, WA 98195
margsli@cs.washington.edu

Hi! I'm a PhD student at University of Washington with Luke Zettlemoyer and Tim Althoff. I work on NLP and Dialogue. I'm specifically interested in efficiently training and generating from large language models, controllability and safety, and applications to mental health.

Previously, I was a Research Engineer at Facebook AI Research (now Meta AI) in New York, mostly working on open domain dialogue.


Experience

Visiting Student Researcher

Meta AI -- FAIR

October 2022 - now

Research Scientist Intern

Meta AI -- FAIR

May 2022 - September 2022

Research Scholar

Grid AI Research

September 2020 - June 2021

Research Engineer

Facebook AI Research (now Meta AI)

August 2017 - September 2020

Software Engineer Intern

Facebook

May 2016 - August 2016

Software Engineer Intern

Point.io

January 2015 - May 2015

Data Analyst Intern

CreditEase

May 2014 - August 2014

Teaching

Head TA

University of Washington

CSE481DS Data Science Capstone

Autumn 2022

Invited Lecturer

Program for Algorithmic and Combinatorial Thinking

Zero-Knowledge Proofs; Introduction to Reinforcement Learning

July 2017; July 2019

Teaching Assistant, Interim Instructor

Center for Talented Youth

Fundamentals of Computer Science

July 2017

Teaching Assistant

AwesomeMath

Combinatorics I & II

June 2017

Teaching Assistant

University of Pennsylvania

CIS160 Mathematical Foundations of Computer Science
(Discrete Maths & Graph Theory)

August 2015 - May 2017

Grader and TA

Art of Problem Solving

Various courses

January 2013 - January 2017

Instructor

HuaXia Chinese school

AMC8 & MathCounts Competition Math

August 2012 - May 2013

Private Tutor

Various students

Various subjects: AP subject tests, SAT & SATII prep, ESL, Secondary Math

Intermittently 2009 - 2014

Education

University of Washington

PhD
Department of Computer Science and Engineering
ML / NLP
Advisors: Luke Zettlemoyer and Tim Althoff
September 2020 - now

University of Pennsylvania

Bachelor of Applied Sciences
Computer Science
Minor: Mathematics
August 2013 - May 2017

Homeschooled / International Academy of Beijing

August 2009 - May 2013

Publications

Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models.

Margaret Li*, Suchin Gururangan*, Tim Dettmers, Mike Lewis, Tim Althoff, Noah A. Smith, Luke Zettlemoyer
arXiv PDF

Under review

Overconfidence in the Face of Ambiguity with Adversarial Data.

Margaret Li*, Julian Michael*
ACL PDF

NAACL 2022 DADC Workshop Best Paper

Don't Sweep your Learning Rate under the Rug: A Closer Look at Cross-modal Transfer of Pretrained Transformers.

Danielle Rothermel, Margaret Li, Tim Rocktäschel, Jakob Foerster
arXiv PDF

arXiv preprint

Recipes for Safety in Open-domain Chatbots.

Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, Emily Dinan
arXiv PDF

NAACL 2021

How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds.

Prithviraj Ammanabrolu, Jack Urbanek, Margaret Li, Arthur Szlam, Tim Rocktäschel, Jason Weston.
arXiv PDF

AAAI 2021 Games Workshop

I love your chain mail! Making knights smile in a fantasy game world.

Shrimai Prabhumoye*, Margaret Li*, Emily Dinan, Jack Urbanek, Douwe Kiela, Jason Weston, Arthur Szlam.
arXiv PDF

NeurIPS 2020 WORDPLAY Workshop

Don't say that! Making inconsistent dialogue unlikely with unlikelihood training.

Margaret Li, Stephen Roller, Ilia Kulikov, Sean Welleck, Spencer Poff, Emily Dinan, Y-Lan Boureau, Kyunghyun Cho, and Jason Weston.
arXiv PDF

ACL 2020

Conversational Agents: Current Progress, Open Problems, and Future Directions.

Stephen Roller*, Y-Lan Boureau*, Jason Weston*, Antoine Bordes, Emily Dinan, Angela Fan, David Gunning, Da Ju, Margaret Li, Spencer Poff, Pratik Ringshia, Kurt Shuster, Eric Michael Smith, Arthur Szlam, Jack Urbanek, Mary Williamson.
arXiv PDF

arXiv preprint

Acute-eval: Improved dialogue evaluation with optimized questions and multi-turn comparisons.

Margaret Li, Jason Weston, and Stephen Roller.
arXiv PDF

NeurIPS 2019 Conversational AI Workshop

Towards empathetic open-domain conversation models: A new benchmark and dataset.

Hannah Rashkin, Eric M. Smith, Margaret Li, and Y-Lan Boureau.
arXiv PDF

ACL 2019

Fails

For all the generalities I've read and heard about failure being ok and imposter syndrome being universal, I so rarely get to see behind the curation of CVs and posts. So I've made what I wish I saw more of -- a 'failure' resume. But I do hope that you'll try to see my supposed failures, and yours, as vital and constructive building blocks of life. So here's an excerpt (such a small slice) of my career-related rejections and such since undergraduate.

There are plenty more for all other aspects of my life too, and I'm happy to chat about those, or anything listed below, in detail. Especially if I know you personally, feel free to reach out anytime to talk about what you're going through.

Rejected SWE Internship Applications: 100+

I transferred into CS from Bioengineering during undergrad and had never coded before. With less experience and coursework than many of my peers, I applied to > 100 internships during my undergraduate years and got rejected from them all -- except for Facebook, Google, and a startup my final summer.

Rejection from FAIR

I had 0 ML experience coming out of university, and when I tried to join FAIR, I was rejected the first time. A research scientist later accepted me on their team, and I'm super grateful to have gotten that opportunity despite there being more qualified engineers (who maybe had at least taken a Machine Learning course for their degree). But I was team-less for nearly 3 months, longer than anyone else I knew.

Poor work performance

I struggled to ramp up on both engineering and machine learning knowledge at the same time, because of the aforementioned lack of experience in ...both. So I took longer to get my (mandatory) first promotion than anyone else I knew, including a negative rating the cycle before I did get the promotion.

Rejected Grad School Applications: 5 (or, n-1 out of n)

I didn't get any kind of outreach or interview, didn't even get explicitly rejected. Got ghosted by a professor who had asked me to email them -- no hard feelings to that person, who is definitely very busy! But I did take it as negative personal feedback then.

Rejected Fellowship Applications: 20+

I spent about 4-5 weeks of my life doing nothing but applying, and got exactly one interview over the course of 2 years. One fellowship, notably, gave me wildly opposite feedback the second year I applied compared to the first year, which just goes to show that the application readers aren't perfect and the process is super noisy.

Rejected Papers: 6(?)

Most got in on resubmission, but it didn't make the first rejection pleasant. Rebuttals can be extremely demoralizing, and reviewers can state their opinions as fact sometimes.

Research dry spell: ~3 years

I went almost 3 years from 2019 - 2022 without writing a single first-author paper. I was doing research! I just got no results I felt were worth publishing.


Interests

When I'm not doing research or teaching, I enjoy walking to every bubble tea shop within 5 miles, singing mostly on-key, rewatching/rereading the Lord of the Rings trilogy, lifting heavy things and putting them down, and listening to people on Youtube talk about absolutely anything. I also spend far too much time online window-shopping and window-eating, which is exactly what it sounds like, in an attempt to limit my actual shopping and eating (the latter is a lost cause).

I've decided to do my best to fit in while living in the PNW by taking up hiking and snowboarding, or just walking around with hiking and snowboarding equipment. I'm always interested in new hobbies, so if you have a beginner- and klutz-friendly one to share, let me know :).