FENCE: A Financial and Multimodal Jailbreak Detection Dataset
arXiv:2602.18154v1 Announce Type: new Abstract: Jailbreaking poses a significant risk to the deployment of Large Language Models (LLMs) and Vision Language Models (VLMs). VLMs are particularly vulnerable because they process both text and images, creating broader attack surfaces. However, available resources for jailbreak detection are scarce, particularly in finance. To address this gap, we present FENCE, a bilingual (Korean-English) multimodal dataset for training and evaluating jailbreak detectors in financial applications. FENCE emphasizes domain realism through finance-relevant queries paired with image-grounded threats. Experiments with commercial and open-source VLMs reveal consistent vulnerabilities, with GPT-4o showing measurable attack success rates and open-source models displaying greater exposure. A baseline detector trained on FENCE achieves 99 percent in-distribution accuracy and maintains strong performance on external benchmarks, underscoring the dataset's robustness
arXiv:2602.18154v1 Announce Type: new Abstract: Jailbreaking poses a significant risk to the deployment of Large Language Models (LLMs) and Vision Language Models (VLMs). VLMs are particularly vulnerable because they process both text and images, creating broader attack surfaces. However, available resources for jailbreak detection are scarce, particularly in finance. To address this gap, we present FENCE, a bilingual (Korean-English) multimodal dataset for training and evaluating jailbreak detectors in financial applications. FENCE emphasizes domain realism through finance-relevant queries paired with image-grounded threats. Experiments with commercial and open-source VLMs reveal consistent vulnerabilities, with GPT-4o showing measurable attack success rates and open-source models displaying greater exposure. A baseline detector trained on FENCE achieves 99 percent in-distribution accuracy and maintains strong performance on external benchmarks, underscoring the dataset's robustness for training reliable detection models. FENCE provides a focused resource for advancing multimodal jailbreak detection in finance and for supporting safer, more reliable AI systems in sensitive domains. Warning: This paper includes example data that may be offensive.
Executive Summary
The article introduces FENCE, a bilingual multimodal dataset for training and evaluating jailbreak detectors in financial applications. FENCE aims to address the scarcity of resources for jailbreak detection, particularly in finance. The dataset focuses on domain realism through finance-relevant queries paired with image-grounded threats. Experiments with commercial and open-source Vision Language Models reveal consistent vulnerabilities, and a baseline detector trained on FENCE achieves high accuracy. The dataset provides a valuable resource for advancing multimodal jailbreak detection in finance and supporting safer AI systems.
Key Points
- ▸ FENCE is a bilingual multimodal dataset for jailbreak detection in financial applications
- ▸ The dataset emphasizes domain realism through finance-relevant queries and image-grounded threats
- ▸ Experiments reveal consistent vulnerabilities in commercial and open-source Vision Language Models
Merits
Comprehensive Dataset
FENCE provides a focused resource for advancing multimodal jailbreak detection in finance, addressing a significant gap in available resources.
Demerits
Limited Scope
The dataset's focus on finance and bilingual (Korean-English) nature may limit its applicability to other domains and languages.
Expert Commentary
The introduction of FENCE marks a significant step forward in addressing the vulnerabilities of Large Language Models and Vision Language Models. The dataset's emphasis on domain realism and multimodal threats highlights the complexity of jailbreak detection in financial applications. As AI systems become increasingly ubiquitous, the development of robust detection models is crucial for ensuring the security and reliability of these systems. The high accuracy achieved by the baseline detector trained on FENCE underscores the dataset's potential for advancing multimodal jailbreak detection.
Recommendations
- ✓ Further research should focus on expanding FENCE to include other languages and domains, enhancing its applicability and generalizability
- ✓ The development of more sophisticated detection models that can effectively utilize the FENCE dataset should be prioritized
