Skip to main content
Academic

A Fusion of context-aware based BanglaBERT and Two-Layer Stacked LSTM Framework for Multi-Label Cyberbullying Detection

arXiv:2602.22449v1 Announce Type: new Abstract: Cyberbullying has become a serious and growing concern in todays virtual world. When left unnoticed, it can have adverse consequences for social and mental health. Researchers have explored various types of cyberbullying, but most approaches use single-label classification, assuming that each comment contains only one type of abuse. In reality, a single comment may include overlapping forms such as threats, hate speech, and harassment. Therefore, multilabel detection is both realistic and essential. However, multilabel cyberbullying detection has received limited attention, especially in low-resource languages like Bangla, where robust pre-trained models are scarce. Developing a generalized model with moderate accuracy remains challenging. Transformers offer strong contextual understanding but may miss sequential dependencies, while LSTM models capture temporal flow but lack semantic depth. To address these limitations, we propose a fusi

arXiv:2602.22449v1 Announce Type: new Abstract: Cyberbullying has become a serious and growing concern in todays virtual world. When left unnoticed, it can have adverse consequences for social and mental health. Researchers have explored various types of cyberbullying, but most approaches use single-label classification, assuming that each comment contains only one type of abuse. In reality, a single comment may include overlapping forms such as threats, hate speech, and harassment. Therefore, multilabel detection is both realistic and essential. However, multilabel cyberbullying detection has received limited attention, especially in low-resource languages like Bangla, where robust pre-trained models are scarce. Developing a generalized model with moderate accuracy remains challenging. Transformers offer strong contextual understanding but may miss sequential dependencies, while LSTM models capture temporal flow but lack semantic depth. To address these limitations, we propose a fusion architecture that combines BanglaBERT-Large with a two-layer stacked LSTM. We analyze their behavior to jointly model context and sequence. The model is fine-tuned and evaluated on a publicly available multilabel Bangla cyberbullying dataset covering cyberbully, sexual harassment, threat, and spam. We apply different sampling strategies to address class imbalance. Evaluation uses multiple metrics, including accuracy, precision, recall, F1-score, Hamming loss, Cohens kappa, and AUC-ROC. We employ 5-fold cross-validation to assess the generalization of the architecture.

Executive Summary

This article proposes a fusion architecture for multilabel cyberbullying detection in the Bangla language, leveraging the strengths of context-aware BanglaBERT and two-layer stacked LSTM models. The authors address the limitations of these models by analyzing their behavior to jointly model context and sequence. The proposed model is fine-tuned and evaluated on a publicly available multilabel Bangla cyberbullying dataset, demonstrating improved accuracy and generalization using 5-fold cross-validation. The authors also employ different sampling strategies to address class imbalance. The findings have significant implications for the development of effective multilabel cyberbullying detection systems in low-resource languages.

Key Points

  • The article proposes a fusion architecture combining BanglaBERT-Large and a two-layer stacked LSTM for multilabel cyberbullying detection.
  • The authors address the limitations of individual models by jointly modeling context and sequence.
  • The proposed model is fine-tuned and evaluated on a publicly available multilabel Bangla cyberbullying dataset.

Merits

Strength

The authors' use of a fusion architecture effectively addresses the limitations of individual models, leading to improved accuracy and generalization.

Methodological Innovation

The proposed model's ability to jointly model context and sequence offers a novel approach to multilabel cyberbullying detection.

Demerits

Limitation

The article does not provide a thorough comparison with existing models, making it difficult to assess the proposed model's superiority.

Expert Commentary

The article's primary contribution lies in its innovative fusion architecture, which effectively addresses the limitations of individual models. However, a more thorough comparison with existing models would strengthen the article's arguments. The proposed model's ability to jointly model context and sequence offers a promising approach to multilabel cyberbullying detection. Nonetheless, the article's findings should be replicated and extended to more diverse datasets to further validate the proposed model's generalizability.

Recommendations

  • Future studies should investigate the application of the proposed model to other low-resource languages and datasets.
  • The authors should conduct a more comprehensive comparison with existing models to demonstrate the proposed model's superiority.

Sources