Skip to main content
Academic

Benchmarking GNN Models on Molecular Regression Tasks with CKA-Based Representation Analysis

arXiv:2602.20573v1 Announce Type: new Abstract: Molecules are commonly represented as SMILES strings, which can be readily converted to fixed-size molecular fingerprints. These fingerprints serve as feature vectors to train ML/DL models for molecular property prediction tasks in the field of computational chemistry, drug discovery, biochemistry, and materials science. Recent research has demonstrated that SMILES can be used to construct molecular graphs where atoms are nodes ($V$) and bonds are edges ($E$). These graphs can subsequently be used to train geometric DL models like GNN. GNN learns the inherent structural relationships within a molecule rather than depending on fixed-size fingerprints. Although GNN are powerful aggregators, their efficacy on smaller datasets and inductive biases across different architectures is less studied. In our present study, we performed a systematic benchmarking of four different GNN architectures across a diverse domain of datasets (physical chemis

R
Rajan, Ishaan Gupta
· · 1 min read · 6 views

arXiv:2602.20573v1 Announce Type: new Abstract: Molecules are commonly represented as SMILES strings, which can be readily converted to fixed-size molecular fingerprints. These fingerprints serve as feature vectors to train ML/DL models for molecular property prediction tasks in the field of computational chemistry, drug discovery, biochemistry, and materials science. Recent research has demonstrated that SMILES can be used to construct molecular graphs where atoms are nodes ($V$) and bonds are edges ($E$). These graphs can subsequently be used to train geometric DL models like GNN. GNN learns the inherent structural relationships within a molecule rather than depending on fixed-size fingerprints. Although GNN are powerful aggregators, their efficacy on smaller datasets and inductive biases across different architectures is less studied. In our present study, we performed a systematic benchmarking of four different GNN architectures across a diverse domain of datasets (physical chemistry, biological, and analytical). Additionally, we have also implemented a hierarchical fusion (GNN+FP) framework for target prediction. We observed that the fusion framework consistently outperforms or matches the performance of standalone GNN (RMSE improvement > $7\%$) and baseline models. Further, we investigated the representational similarity using centered kernel alignment (CKA) between GNN and fingerprint embeddings and found that they occupy highly independent latent spaces (CKA $\le0.46$). The cross-architectural CKA score suggests a high convergence between isotopic models like GCN, GraphSAGE and GIN (CKA $\geq0.88$), with GAT learning moderately independent representation (CKA $0.55-0.80$).

Executive Summary

This article presents a benchmarking study of four Graph Neural Network (GNN) architectures on molecular regression tasks, utilizing a hierarchical fusion framework that combines GNN with fingerprint (FP) embeddings. The results demonstrate the fusion framework's superior performance, outperforming standalone GNN and baseline models. The study also employs centered kernel alignment (CKA) to analyze the representational similarity between GNN and fingerprint embeddings, revealing highly independent latent spaces and high convergence between isotopic models.

Key Points

  • Benchmarking of four GNN architectures on molecular regression tasks
  • Implementation of a hierarchical fusion framework (GNN+FP) for target prediction
  • Investigation of representational similarity using centered kernel alignment (CKA) between GNN and fingerprint embeddings

Merits

Comprehensive Benchmarking

The study provides a thorough comparison of different GNN architectures, offering valuable insights into their performance on various molecular regression tasks.

Demerits

Limited Dataset Scope

The study's focus on a specific domain of datasets may limit the generalizability of the findings to other areas of application.

Expert Commentary

The article presents a rigorous evaluation of GNN architectures on molecular regression tasks, highlighting the benefits of combining GNN with fingerprint embeddings. The use of CKA to analyze representational similarity provides valuable insights into the underlying mechanisms of GNN models. However, the study's scope and generalizability could be further enhanced by exploring a broader range of datasets and applications. Overall, the study contributes significantly to the understanding of GNN models in molecular property prediction and has important implications for the development of more accurate and reliable models.

Recommendations

  • Future studies should investigate the applicability of the hierarchical fusion framework to other areas of molecular property prediction
  • The development of more diverse and comprehensive datasets is necessary to further evaluate the performance and generalizability of GNN models

Sources