SEAHateCheck: Functional Tests for Detecting Hate Speech in Low-Resource Languages of Southeast Asia
arXiv:2603.16070v1 Announce Type: new Abstract: Hate speech detection relies heavily on linguistic resources, which are primarily available in high-resource languages such as English and Chinese, creating barriers for researchers and platforms developing tools for low-resource languages in Southeast Asia, where diverse socio-linguistic contexts complicate online hate moderation. To address this, we introduce SEAHateCheck, a pioneering dataset tailored to Indonesia, Thailand, the Philippines, and Vietnam, covering Indonesian, Tagalog, Thai, and Vietnamese. Building on HateCheck's functional testing framework and refining SGHateCheck's methods, SEAHateCheck provides culturally relevant test cases, augmented by large language models and validated by local experts for accuracy. Experiments with state-of-the-art and multilingual models revealed limitations in detecting hate speech in specific low-resource languages. In particular, Tagalog test cases showed the lowest model accuracy, likely
arXiv:2603.16070v1 Announce Type: new Abstract: Hate speech detection relies heavily on linguistic resources, which are primarily available in high-resource languages such as English and Chinese, creating barriers for researchers and platforms developing tools for low-resource languages in Southeast Asia, where diverse socio-linguistic contexts complicate online hate moderation. To address this, we introduce SEAHateCheck, a pioneering dataset tailored to Indonesia, Thailand, the Philippines, and Vietnam, covering Indonesian, Tagalog, Thai, and Vietnamese. Building on HateCheck's functional testing framework and refining SGHateCheck's methods, SEAHateCheck provides culturally relevant test cases, augmented by large language models and validated by local experts for accuracy. Experiments with state-of-the-art and multilingual models revealed limitations in detecting hate speech in specific low-resource languages. In particular, Tagalog test cases showed the lowest model accuracy, likely due to linguistic complexity and limited training data. In contrast, slang-based functional tests proved the hardest, as models struggled with culturally nuanced expressions. The diagnostic insights of SEAHateCheck further exposed model weaknesses in implicit hate detection and models' struggles with counter-speech expression. As the first functional test suite for these Southeast Asian languages, this work equips researchers with a robust benchmark, advancing the development of practical, culturally attuned hate speech detection tools for inclusive online content moderation.
Executive Summary
SEAHateCheck introduces a groundbreaking functional test suite tailored to low-resource languages in Southeast Asia—Indonesian, Tagalog, Thai, and Vietnamese—addressing a critical gap in hate speech detection resources. Leveraging the HateCheck framework and refining prior methodologies, the dataset incorporates culturally relevant test cases validated by local experts and augmented with large language models. Experimental results highlight significant model limitations, particularly in Tagalog due to linguistic complexity and data scarcity, and in slang-based expressions where cultural nuance confounded AI performance. These findings expose gaps in implicit hate detection and counter-speech recognition, offering a vital benchmark for advancing culturally attuned moderation tools. SEAHateCheck represents a pivotal step toward equitable content moderation in under-resourced linguistic environments.
Key Points
- ▸ First functional test suite for low-resource Southeast Asian languages
- ▸ Validation by local experts enhances cultural relevance and accuracy
- ▸ Model performance disparities reveal linguistic and cultural challenges in hate detection
Merits
Cultural Relevance
SEAHateCheck incorporates culturally specific test cases validated by local experts, ensuring applicability to real-world regional contexts.
Benchmark Value
As the first standardized test suite for these languages, it provides researchers with a critical evaluation framework for improving detection models.
Demerits
Data Limitation
The effectiveness of Tagalog detection is hindered by insufficient training data, indicating a persistent barrier to scalable AI deployment in low-resource languages.
Complexity Challenge
Slang-based tests expose persistent difficulties in modeling culturally nuanced expressions, suggesting a need for more sophisticated linguistic processing techniques.
Expert Commentary
SEAHateCheck marks a significant advancement in bridging the divide between linguistic equity and AI-driven content moderation. The authors’ integration of culturally validated testing frameworks within a functional paradigm demonstrates a nuanced understanding of the challenges inherent in low-resource environments. While the Tagalog accuracy gap is disconcerting, it is not surprising given the documented scarcity of annotated datasets in Southeast Asian languages—a systemic issue exacerbated by colonial-era linguistic hierarchies and underinvestment in local AI infrastructure. The counter-speech detection anomalies further illuminate a critical blind spot in current AI models: the inability to recognize nuanced expressions of dissent or solidarity, which are often linguistically encoded. This work does not merely identify limitations; it catalyzes a broader conversation on the ethical imperatives of model transparency and data justice. The path forward demands collaborative data-sharing initiatives, community-driven annotation frameworks, and incentivized research funding for underrepresented linguistic domains. Without such interventions, AI moderation tools risk perpetuating marginalization under the guise of neutrality.
Recommendations
- ✓ 1. Launch collaborative annotation projects with local universities and civil society organizations to expand annotated datasets for Tagalog and other low-resource languages.
- ✓ 2. Develop open-source frameworks for counter-speech recognition that incorporate linguistic pragmatics and sociolinguistic context as predictive variables.
- ✓ 3. Advocate for policy incentives at regional level to fund AI research centered on linguistic equity and underrepresented languages.