Intrinsic Lorentz Neural Network
arXiv:2602.23981v1 Announce Type: new Abstract: Real-world data frequently exhibit latent hierarchical structures, which can be naturally represented by hyperbolic geometry. Although recent hyperbolic neural networks have demonstrated promising results, many existing architectures remain partially intrinsic, mixing Euclidean operations with hyperbolic ones or relying on extrinsic parameterizations. To address it, we propose the \emph{Intrinsic Lorentz Neural Network} (ILNN), a fully intrinsic hyperbolic architecture that conducts all computations within the Lorentz model. At its core, the network introduces a novel \emph{point-to-hyperplane} fully connected layer (FC), replacing traditional Euclidean affine logits with closed-form hyperbolic distances from features to learned Lorentz hyperplanes, thereby ensuring that the resulting geometric decision functions respect the inherent curvature. Around this fundamental layer, we design intrinsic modules: GyroLBN, a Lorentz batch normaliza
arXiv:2602.23981v1 Announce Type: new Abstract: Real-world data frequently exhibit latent hierarchical structures, which can be naturally represented by hyperbolic geometry. Although recent hyperbolic neural networks have demonstrated promising results, many existing architectures remain partially intrinsic, mixing Euclidean operations with hyperbolic ones or relying on extrinsic parameterizations. To address it, we propose the \emph{Intrinsic Lorentz Neural Network} (ILNN), a fully intrinsic hyperbolic architecture that conducts all computations within the Lorentz model. At its core, the network introduces a novel \emph{point-to-hyperplane} fully connected layer (FC), replacing traditional Euclidean affine logits with closed-form hyperbolic distances from features to learned Lorentz hyperplanes, thereby ensuring that the resulting geometric decision functions respect the inherent curvature. Around this fundamental layer, we design intrinsic modules: GyroLBN, a Lorentz batch normalization that couples gyro-centering with gyro-scaling, consistently outperforming both LBN and GyroBN while reducing training time. We additionally proposed a gyro-additive bias for the FC output, a Lorentz patch-concatenation operator that aligns the expected log-radius across feature blocks via a digamma-based scale, and a Lorentz dropout layer. Extensive experiments conducted on CIFAR-10/100 and two genomic benchmarks (TEB and GUE) illustrate that ILNN achieves state-of-the-art performance and computational cost among hyperbolic models and consistently surpasses strong Euclidean baselines. The code is available at \href{https://github.com/Longchentong/ILNN}{\textcolor{magenta}{this url}}.
Executive Summary
The Intrinsic Lorentz Neural Network (ILNN) is a novel, fully intrinsic hyperbolic architecture that leverages the Lorentz model for hyperbolic geometry. ILNN replaces traditional Euclidean operations with closed-form hyperbolic distances, ensuring geometric decision functions respect the inherent curvature. The network introduces a point-to-hyperplane fully connected layer, Lorentz batch normalization, and a gyro-additive bias. Extensive experiments demonstrate ILNN achieves state-of-the-art performance and computational cost among hyperbolic models, surpassing strong Euclidean baselines. The code is available for further research and development.
Key Points
- ▸ ILNN is a fully intrinsic hyperbolic architecture that leverages the Lorentz model.
- ▸ ILNN replaces traditional Euclidean operations with closed-form hyperbolic distances.
- ▸ ILNN introduces a novel point-to-hyperplane fully connected layer and Lorentz batch normalization.
Merits
Strength in Hyperbolic Geometry
ILNN's use of Lorentz model enables the accurate representation of hyperbolic geometry, which is essential for modeling real-world data with latent hierarchical structures.
Improved Performance
ILNN achieves state-of-the-art performance and computational cost among hyperbolic models, demonstrating its effectiveness in various applications.
Demerits
Limited Generalizability
ILNN's performance may be limited to specific domains, such as image classification and genomic analysis, and may not generalize well to other applications.
Computational Complexity
ILNN's use of Lorentz model and novel point-to-hyperplane fully connected layer may increase computational complexity, making it less suitable for resource-constrained environments.
Expert Commentary
The Intrinsic Lorentz Neural Network (ILNN) is a significant contribution to the field of hyperbolic neural networks. By leveraging the Lorentz model and introducing novel components, ILNN achieves state-of-the-art performance and computational cost among hyperbolic models. However, its limited generalizability and increased computational complexity may be concerns in certain applications. Further research is needed to fully explore ILNN's potential and to address its limitations. Nevertheless, ILNN has the potential to revolutionize the way we approach machine learning, particularly in domains where data exhibit latent hierarchical structures.
Recommendations
- ✓ Future research should focus on exploring ILNN's generalizability to other domains and applications.
- ✓ Developing more efficient and effective methods for computing hyperbolic distances and Lorentz transformations will be essential for scaling ILNN to larger datasets and more complex models.