# Paper - Dynamics of Transient Structure in In-Context Linear Regression Transformers

> Source: https://ollybritton.com/notes/papers/paper-dynamics-of-transient-structure-in-in-context-linear-regression-transformers/ · Updated: 2025-11-13 · Tags: uni, notes

- **Full title**: *Dynamics of Transient Structure in In-Context Linear Regression Transformers*
- **Author(s)**: Liam Carroll, Jesse Hoogland, Matthew Farrugia-Roberts, Daniel Murfet
- **Year**: 2025
- **Link**: http://arxiv.org/abs/2501.17745
- **Relevant for**:
	- [Course - Theories of Deep Learning MT25](https://ollybritton.com/notes/uni/part-c/mt25/theories-of-deep-learning/)
- **Links to**:
	- [redacted](https://ollybritton.com/404)

### Summary
- in-context linear regression
	- given some task vector $t \in \mathbb R^D$, sample a fixed length distribution of pairs $(x_i, y_i)$ where $x_i$ is normally distributed and $y_i \sim \mathcal N(y_i \mid t^\top x_i, \sigma^2)$
	- vary the "task diversity" by picking different 

### Flashcards

---
Olly Britton — https://ollybritton.com. Machine-readable index: https://ollybritton.com/llms.txt