DariMis: Harm-Aware Modeling for Dari Misinformation Detection on YouTube
arXiv:2603.22977v1 Announce Type: new Abstract: Dari, the primary language of Afghanistan, is spoken by tens of millions of people yet remains largely absent from the misinformation detection literature. We address this gap with DariMis, the first manually annotated dataset of 9,224 Dari-language YouTube videos, labeled across two dimensions: Information Type (Misinformation, Partly True, True) and Harm Level (Low, Medium, High). A central empirical finding is that these dimensions are structurally coupled, not independent: 55.9 percent of Misinformation carries at least Medium harm potential, compared with only 1.0 percent of True content. This enables Information Type classifiers to function as implicit harm-triage filters in content moderation pipelines. We further propose a pair-input encoding strategy that represents the video title and description as separate BERT segment inputs, explicitly modeling the semantic relationship between headline claims and body content, a key sign
arXiv:2603.22977v1 Announce Type: new Abstract: Dari, the primary language of Afghanistan, is spoken by tens of millions of people yet remains largely absent from the misinformation detection literature. We address this gap with DariMis, the first manually annotated dataset of 9,224 Dari-language YouTube videos, labeled across two dimensions: Information Type (Misinformation, Partly True, True) and Harm Level (Low, Medium, High). A central empirical finding is that these dimensions are structurally coupled, not independent: 55.9 percent of Misinformation carries at least Medium harm potential, compared with only 1.0 percent of True content. This enables Information Type classifiers to function as implicit harm-triage filters in content moderation pipelines. We further propose a pair-input encoding strategy that represents the video title and description as separate BERT segment inputs, explicitly modeling the semantic relationship between headline claims and body content, a key signal of misleading information. An ablation study against single-field concatenation shows that pair-input encoding yields a 7.0 percentage point gain in Misinformation recall (60.1 percent to 67.1 percent), the safety-critical minority class, despite modest overall macro F1 differences (0.09 percentage points). We benchmark a Dari/Farsi-specialized model (ParsBERT) against XLM-RoBERTa-base; ParsBERT achieves the best test performance with accuracy of 76.60 percent and macro F1 of 72.77 percent. Bootstrap 95 percent confidence intervals are reported for all metrics, and we discuss both the practical significance and statistical limitations of the results.
Executive Summary
The article introduces DariMis, a pioneering dataset and analysis framework for detecting misinformation in Dari-language YouTube content. With 9,224 annotated videos across misinformation classification and harm levels, it fills a critical gap in linguistic coverage. The study reveals a structural coupling between misinformation and harm potential, enabling information-type classifiers to serve dual roles as harm-triage filters. The pair-input encoding strategy—using separate BERT segments for title and description—demonstrates measurable gains in recall for the minority class of misinformation, despite minimal impact on overall macro F1. ParsBERT outperforms XLM-RoBERTa-base in Dari/Farsi-specific benchmarks, offering localized model efficacy. The work bridges linguistic diversity and content moderation needs with empirical rigor.
Key Points
- ▸ First annotated Dari misinformation dataset (9,224 videos)
- ▸ Structural coupling between misinformation and harm potential (55.9% of misinformation carries medium/high harm)
- ▸ Pair-input encoding improves misinformation recall by 7 percentage points
Merits
Linguistic Representation
Addresses a critical void in misinformation detection literature by including Dari, a language spoken by millions in Afghanistan.
Methodological Innovation
Pair-input encoding introduces a nuanced modeling strategy that better captures semantic relationships between headline claims and body text, improving detection sensitivity.
Benchmarking
ParsBERT’s superior performance on Dari/Farsi content validates localized model adaptability, offering practical relevance for regional content moderation.
Demerits
Generalizability Concern
Results may not generalize beyond Dari/Farsi contexts due to dataset specificity; external validation on other regional languages is absent.
Statistical Limitation
Confidence intervals reported, yet the modest macro F1 differences (0.09%) suggest limited incremental gains may affect scalability or real-world impact assessment.
Expert Commentary
This work represents a significant step forward in the intersection of linguistic diversity and computational content moderation. The authors rightly identify the critical underrepresentation of Dari in existing literature and respond with a meticulously annotated dataset that enables empirical validation of coupled phenomena between misinformation and harm. The pair-input encoding strategy is particularly noteworthy—it reflects a sophisticated understanding of how semantic disjointness between title and body content often signals misleading information. While the macro F1 gains are modest, the 7-point increase in misinformation recall is statistically and practically meaningful, especially given the safety-critical nature of minority class detection. ParsBERT’s outperformance underscores the value of fine-tuned, region-specific models over generic cross-lingual embeddings. Importantly, the paper acknowledges statistical limitations with transparency, avoiding overstatement. This is a model of how empirical research should be conducted in low-resource language domains: rigorous annotation, thoughtful modeling, and candid assessment of constraints. It sets a precedent for similar efforts in other underrepresented languages.
Recommendations
- ✓ Extend DariMis annotation to include additional Dari content types (e.g., audio/video transcripts, user comments) for richer multimodal analysis.
- ✓ Develop open-source toolkits for adapting the pair-input encoding strategy to other low-resource languages, facilitating replication and extension.
Sources
Original: arXiv - cs.CL
