Seyed Morteza Emadi

Seyed Morteza Emadi의 전문분석자료

Academic · 약 1분

Exact Attention Sensitivity and the Geometry of Transformer Stability

arXiv:2602.18849v1 Announce Type: new Abstract: Despite powering modern AI, transformers remain mysteriously brittle to train. We develop a stability theory that explains why pre-LayerNorm works, …

Seyed Morteza Emadi

조회수 23회 02.25

Academic · 약 1분

Rank-Aware Spectral Bounds on Attention Logits for Stable Low-Precision Training

arXiv:2602.18851v1 Announce Type: new Abstract: Attention scores in transformers are bilinear forms $S_{ij} = x_i^\top M x_j / \sqrt{d_h}$ whose maximum magnitude governs overflow risk …

Seyed Morteza Emadi

조회수 24회 02.25