Seyed Morteza Emadi

Articles by Seyed Morteza Emadi

Academic · 1 min

Exact Attention Sensitivity and the Geometry of Transformer Stability

arXiv:2602.18849v1 Announce Type: new Abstract: Despite powering modern AI, transformers remain mysteriously brittle to train. We develop a stability theory that explains why pre-LayerNorm works, …

Seyed Morteza Emadi

7 views Feb 25

Academic · 1 min

Rank-Aware Spectral Bounds on Attention Logits for Stable Low-Precision Training

arXiv:2602.18851v1 Announce Type: new Abstract: Attention scores in transformers are bilinear forms $S_{ij} = x_i^\top M x_j / \sqrt{d_h}$ whose maximum magnitude governs overflow risk …

Seyed Morteza Emadi

8 views Feb 25

Seyed Morteza Emadi

Articles by Seyed Morteza Emadi

Exact Attention Sensitivity and the Geometry of Transformer Stability

Rank-Aware Spectral Bounds on Attention Logits for Stable Low-Precision Training

JCG, PC

HSOLLC Co., Ltd.