Calendar
Week 1
- Jan 9
- LectureIntroduction
- Percy Liang
- Jan 11
- SpeakerFlashAttention
- Tri Dao (Stanford)
- Reading materials
- The Illustrated Transformer
- Attention is All You Need (Optional, but recommended!)
- Making Deep Learning Go Brrrr From First Principles.
- NVIDIA deep learning performance guide. (Optional)
Week 2
- Jan 16
- HolidayMLK Day
- Jan 18
- SpeakerModel Training
- Ce Zhang
Week 3
- Jan 23
- LectureScaling Laws and Emergent Behavior
- Tatsu Hashimoto
- Jan 25
- SpeakerPathways Language Model (PaLM) and Model Scaling
- Aakanksha Chowdhery (Google Brain)
Week 4
- Jan 30
- SpeakerMechanistic Interpretability – Reverse Engineering Learned Algorithms from Transformers
- Stella Biderman (EleutherAI, Booz Allen Hamilton)
- Feb 1
- SpeakerA data-centric view on reliable generalization
- Ludwig Schmidt
Week 5
- Feb 6
- LectureSecurity + Privacy
- Tatsu Hashimoto
- Feb 8
- SpeakerBuilding ML Models like Open-Source Software
- Colin Raffel (UNC, Hugging Face)
Week 6
- Feb 13
- LectureUnlocking the RNA Universe
- Raphael Townshend (Atomic AI)
- Feb 15
- SpeakerFoundation Model Ethics
- Rob Reich (Stanford)
Week 7
- Feb 20
- HolidayPresident’s Day
- Feb 22
- SpeakerPoisoning web-scale datasets is practical
- Nicholas Carlini (Google Brain)
Week 8
- Feb 27
- SpeakerCompression for AGI
- Jack Rae (OpenAI)
- Mar 1
- SpeakerTalk TBD
- Susan Zhang (Meta)
Week 9
- Mar 6
- SpeakerTalk TBD
- Yejin Choi (University of Washington, Allen Institute for AI)
- Mar 8
- SpeakerTalk TBD
- Jared Kaplan (Anthropic)
Week 10
- Mar 13
- Speaker or Final Presentations (TBD)
- TBD
- Mar 15
- Final Presentations