The new transformer. Some crazy RL application. Basically every DL paper that doesn’t have at least 2 lemmas.

Youre gonna see topics like:

• new model architectures/techniques
• reinforcement learning
• scaling laws & empirical observations

list of papers

• [ ] "S4 Transformer" - Gu, Goel, Re
• [ ] "Mamba: Linear-Time Sequence Modeling with Selective State Spaces" - Gu, Tri Dao
• [ ] "Measuring the Intrinsic Dimension of Objective Landscapes" - Rosanne Liu and co
• [ ] "DR3: VALUE-BASED DEEP REINFORCEMENT LEARNING REQUIRES EXPLICIT REGULARIZATION" - Tengyu, Sergey, etc.
• [ ] "A Parametric Class of Approximate Gradient Updates for Policy Optimization" - Gummadi, Kumar, Wen, Schuurmans
• [ ] "UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models" - Zhao et al
• [x] Finite-time Lyapunov exponents of deep neural networks - Storm et al
• [x] "Benchmarking Neural Network Training Algorithms" - Naman and Google friends

paper summaries/notes