Mind the Gap: Pitfalls of LLM Alignment with Asian Public Opinion
arXiv:2603.06264v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly being deployed in multilingual, multicultural settings, yet their reliance on predominantly English-centric training data risks misalignment with the diverse cultural values of different societies. In this paper, we present a comprehensive, multilingual audit of the cultural alignment of contemporary LLMs including GPT-4o-Mini, Gemini-2.5-Flash, Llama 3.2, Mistral and Gemma 3 across India, East Asia and Southeast Asia. Our study specifically focuses on the sensitive domain of religion as the prism for broader alignment. To facilitate this, we conduct a multi-faceted analysis of every LLM's internal representations, using log-probs/logits, to compare the model's opinion distributions against ground-truth public attitudes. We find that while the popular models generally align with public opinion on broad social issues, they consistently fail to accurately represent religious viewpoints, especia
arXiv:2603.06264v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly being deployed in multilingual, multicultural settings, yet their reliance on predominantly English-centric training data risks misalignment with the diverse cultural values of different societies. In this paper, we present a comprehensive, multilingual audit of the cultural alignment of contemporary LLMs including GPT-4o-Mini, Gemini-2.5-Flash, Llama 3.2, Mistral and Gemma 3 across India, East Asia and Southeast Asia. Our study specifically focuses on the sensitive domain of religion as the prism for broader alignment. To facilitate this, we conduct a multi-faceted analysis of every LLM's internal representations, using log-probs/logits, to compare the model's opinion distributions against ground-truth public attitudes. We find that while the popular models generally align with public opinion on broad social issues, they consistently fail to accurately represent religious viewpoints, especially those of minority groups, often amplifying negative stereotypes. Lightweight interventions, such as demographic priming and native language prompting, partially mitigate but do not eliminate these cultural gaps. We further show that downstream evaluations on bias benchmarks (such as CrowS-Pairs, IndiBias, ThaiCLI, KoBBQ) reveal persistent harms and under-representation in sensitive contexts. Our findings underscore the urgent need for systematic, regionally grounded audits to ensure equitable global deployment of LLMs.
Executive Summary
This article examines the cultural alignment of Large Language Models (LLMs) with Asian public opinion, focusing on the sensitive domain of religion. The study finds that popular LLMs generally align with public opinion on broad social issues but fail to accurately represent religious viewpoints, especially those of minority groups. The authors propose lightweight interventions to mitigate these cultural gaps, but emphasize the need for systematic, regionally grounded audits to ensure equitable global deployment of LLMs.
Key Points
- ▸ LLMs are increasingly being deployed in multilingual, multicultural settings
- ▸ The reliance on English-centric training data risks misalignment with diverse cultural values
- ▸ The study finds that LLMs consistently fail to accurately represent religious viewpoints, especially those of minority groups
Merits
Comprehensive Multilingual Audit
The study conducts a comprehensive, multilingual audit of contemporary LLMs across India, East Asia, and Southeast Asia, providing a nuanced understanding of the cultural alignment of LLMs.
Demerits
Limited Scope of Interventions
The proposed lightweight interventions, such as demographic priming and native language prompting, may not be sufficient to eliminate the cultural gaps, and more robust solutions may be required.
Expert Commentary
The study's findings underscore the importance of considering the cultural context in which LLMs are deployed. The authors' proposal for systematic, regionally grounded audits is a crucial step towards ensuring that LLMs are developed and deployed in a way that is respectful of diverse cultural values and minimizes the risk of harm. However, more research is needed to develop effective solutions to address the cultural gaps identified in the study, and to ensure that LLMs are developed and deployed in a way that is fair, transparent, and accountable.
Recommendations
- ✓ Conducting further research on the development of culturally sensitive LLMs
- ✓ Establishing regulatory frameworks to ensure the equitable deployment of LLMs