Welcome to Stat 8105: GenAI!

📚 Course Information

• Course: Stat 8105, Fall 2025

• Instructor: Jie Ding

• Teaching Assistant: Fangqiao Tian

• Institution: School of Statistics, University of Minnesota Twin Cities

🙏 Acknowledgments

We acknowledge An Luo for their valuable contributions as a TA in the earlier version of this course during Stat 8931, Fall 2024.

Contents:

• Introduction
  • Quick Survey
  • About the Stat 8105 Course (Fall 2025)
  • Philosophy of Course Arrangement
  • How to Use This Course and Materials
  • Course Survey
• 1. Quick Study of Deep Learning
  • Optimization Algorithms
  • Supervise Learning
  • Unsupervised Learning
  • Semisupervised Learning
  • Reinforcement Learning
  • References
• 2. Large Language Modeling
  • What the Blackbox Is
  • Decoding from A Trained Model
  • Tokenization and Vocabulary
  • References
• 3. Training LLMs from Scratch
  • Introduction
  • Transformer Model Architecture
  • Various Model Architectures
  • Data Curation
  • Evaluating Your Bespoke LLM
  • References
• 4. Fine-Tuning LLMs
  • Overview of Using LLM in Practice
  • Fine-Tuning Techniques
  • Instruction Fine-Tuning
  • Application Case: LLM-based Recommender Systems
  • Application Case: LLM for Tabular Data Analysis
  • The Curse of Catastrophic Forgetting
  • Reference
• 5. Human Value Alignment
  • (Recall) Overview of Using LLM in Practice
  • Human Value Alignment Formulation
  • Reward Models
  • Implementation of Model Alignment
  • Direct Preference Optimization (DPO): Using Implicit Reward Function
  • Multi-Dimensional Human Value Alignment
  • References
• 6. Efficient Training of Large Models
  • Infrastructure for Large Model Training
  • Parallelism Schemes for LLM Training
  • Computation Optimizations
  • Memory Optimizations
• 7. Diffusion Models
  • 1. Variational Autoencoders (VAEs)
  • 2. Denoising Diffusion Probabilisitc Model (DDPM)
  • 3. Score-Matching Langevin Dynamics (SMLD)
  • 4. Stochastic Differential Equations (SDE)
  • 5. Diffusion Model in Practice: Stable Diffusion
• Agentic AI
• 8. Retrieval Augmented Generation
  • Overview of RAG
  • Indexing
  • Retrieval
  • Generation
• 9. Efficient Deployment Strategies
  • Knowledge Distillation
  • Optimizing Memory and Precision for Efficient LLM Deployment
  • Model Pruning
  • Flash Attention
  • Key-Value Cache
  • Model Compiling and Serving
• 10. AI Safty
  • Poisoning Backdoor Attacks
  • Model Stealing Attacks and Defenses
  • Watermarking
  • Safety in RAG Systems
  • References