Paper review
1.[Paper review] DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
Review of DeepSeek-v2
2026년 9월 24일
2.[Paper review] Fast Transformer Decoding: One Write-Head is All You Need
Review of MQA
2026년 9월 25일
3.[Paper review] GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Review of GQA
2026년 9월 26일
4.[Paper Review] Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
Review of Linear attention
2026년 9월 29일
5.[Paper Review] ROFORMER: Enhanced Transformer with Rotary Position Embedding
Review of Roformer, which using RoPE as position embedding
2026년 9월 30일
6.[Paper review] Scaling Laws for Neural Language Models
Review of Scaling Laws which shows that the loss mainly depends on number of hyperparmeter, dataset size and compute.
4일 전