Calendar
Behavior of large language models
- Mon Jan 3
- Introduction
- Lecture
- Percy Liang
- Why does this course exist?
- Language models
- Overview of the course
- Wed Jan 5
- Capabilities
- Lecture Discussion
- Percy Liang
- Adaptation framework
- Perplexity
- Prompting
- Overview of GPT-3 tasks
- Discussion paper:
- Mon Jan 10
- Harms I
- Lecture Discussion
- Rishi Bommasani
- Performance disparities
- Social bias and stereotypes
- Discussion paper:
- Wed Jan 12
- Harms II
- Lecture Discussion
- Percy Liang
- Toxicity
- Disinformation
- Discussion paper:
Data behind large language models
- Mon Jan 17
- (no class - MLK day)
- Wed Jan 19
- Data
- Lecture Discussion
- Percy Liang
- Scrutiny of existing datasets
- Data documentation
- Data ecosystems
- Discussion paper:
- Mon Jan 24
- Security and privacy
- Lecture Discussion
- Tatsunori Hashimoto
- Data poisoning
- Differential privacy
- Discussion paper:
- Wed Jan 26
- Legal considerations
- Lecture Discussion
- Percy Liang
- Copyright law
- Fair use
- Discussion paper:
Building large language models
- Mon Jan 31
- Modeling
- Lecture Discussion
- Percy Liang
- Tokenization
- RNNs, Transformers
- Discussion paper:
- Wed Feb 2
- Training
- Lecture Discussion
- Percy Liang
- Objective functions
- Stability
- Debugging
- Discussion paper:
- Mon Feb 7
- Parallelism
- Lecture Discussion
- Christopher RĂ©
- Data parallelism
- Model parallelism
- Pipeline parallelism
- Discussion paper:
- Wed Feb 9
- Scaling laws
- Lecture Discussion
- Tatsunori Hashimoto
- Scaling laws
- Discussion paper:
- Mon Feb 14
- Modular architectures
- Lecture Discussion
- Percy Liang
- Mixture of experts
- Memory-augmented (retrieval) models
- Discussion paper:
- Wed Feb 16
- Adaptation
- Lecture Discussion
- Sang Michael Xie
- Probing
- Fine-tuning
- Lightweight fine-tuning
- Discussion paper:
- Mon Feb 21
- no class (Presidents’ Day)
- Wed Feb 23
- Environmental impact
- Lecture Discussion
- Percy Liang
- Training and inference costs
- Carbon emissions
- Discussion paper:
Beyond large language models
- Mon Feb 28
- Guest lecture
- Lecture Discussion
- Maxwell Nye
- Wed Mar 2
- Guest lecture
- Lecture Discussion
- Margaret Mitchell and Stella Biderman
- Mon Mar 7
- Project presentations
- Student Presentations
- Wed Mar 9
- Project presentations
- Student presentations