CheckIfExist: Detecting Citation Hallucinations in the Era of AI-Generated Content
arXiv:2602.15871v1 Announce Type: new Abstract: The proliferation of large language models (LLMs) in academic workflows has introduced unprecedented challenges to bibliographic integrity, particularly through reference hallucination -- the generation of plausible but non-existent citations. Recent investigations have documented the presence of AI-hallucinated citations even in papers accepted at premier machine learning conferences such as NeurIPS and ICLR, underscoring the urgency of automated verification mechanisms. This paper presents "CheckIfExist", an open-source web-based tool designed to provide immediate verification of bibliographic references through multi-source validation against CrossRef, Semantic Scholar, and OpenAlex scholarly databases. While existing reference management tools offer bibliographic organization capabilities, they do not provide real-time validation of citation authenticity. Commercial hallucination detection services, though increasingly available, oft
arXiv:2602.15871v1 Announce Type: new Abstract: The proliferation of large language models (LLMs) in academic workflows has introduced unprecedented challenges to bibliographic integrity, particularly through reference hallucination -- the generation of plausible but non-existent citations. Recent investigations have documented the presence of AI-hallucinated citations even in papers accepted at premier machine learning conferences such as NeurIPS and ICLR, underscoring the urgency of automated verification mechanisms. This paper presents "CheckIfExist", an open-source web-based tool designed to provide immediate verification of bibliographic references through multi-source validation against CrossRef, Semantic Scholar, and OpenAlex scholarly databases. While existing reference management tools offer bibliographic organization capabilities, they do not provide real-time validation of citation authenticity. Commercial hallucination detection services, though increasingly available, often impose restrictive usage limits on free tiers or require substantial subscription fees. The proposed tool fills this gap by employing a cascading validation architecture with string similarity algorithms to compute multi-dimensional match confidence scores, delivering instant feedback on reference authenticity. The system supports both single-reference verification and batch processing of BibTeX entries through a unified interface, returning validated APA citations and exportable BibTeX records within seconds.
Executive Summary
In 'CheckIfExist', the authors propose an open-source web-based tool designed to verify bibliographic references against multiple scholarly databases, addressing the pressing issue of AI-generated citation hallucinations. Leveraging a cascading validation architecture with string similarity algorithms, the tool computes multi-dimensional match confidence scores to deliver instant feedback on reference authenticity. The system supports single-reference verification and batch processing of BibTeX entries, providing validated APA citations and exportable BibTeX records within seconds. This innovative solution fills a critical gap in reference management tools, offering a necessary safeguard against the proliferation of AI-hallucinated citations.
Key Points
- ▸ CheckIfExist is an open-source web-based tool for verifying bibliographic references against multiple scholarly databases.
- ▸ The tool employs a cascading validation architecture with string similarity algorithms to compute multi-dimensional match confidence scores.
- ▸ CheckIfExist supports both single-reference verification and batch processing of BibTeX entries.
Merits
Comprehensive validation capabilities
CheckIfExist's multi-source validation against CrossRef, Semantic Scholar, and OpenAlex databases ensures a high level of accuracy in verifying bibliographic references.
User-friendly interface
The tool's unified interface enables users to easily verify single references or batch process BibTeX entries, facilitating efficient reference management.
Open-source and accessible
As an open-source tool, CheckIfExist can be freely accessed and adapted by researchers, promoting collaboration and innovation in the academic community.
Demerits
Scalability limitations
The tool's performance may be affected by the volume of references to be verified, potentially leading to delays or errors in batch processing.
Dependency on database accuracy
The accuracy of CheckIfExist's results relies on the accuracy of the underlying scholarly databases, which may contain errors or inconsistencies.
Limited support for non-standard citation formats
The tool's current implementation may not support non-standard citation formats, potentially limiting its applicability in certain research domains.
Expert Commentary
The introduction of CheckIfExist represents a crucial step towards addressing the pressing issue of AI-generated citation hallucinations in academic publishing. By leveraging a cascading validation architecture and string similarity algorithms, the tool offers a reliable and efficient means of verifying bibliographic references against multiple scholarly databases. While the tool's scalability and dependency on database accuracy are potential limitations, its open-source nature and user-friendly interface make it an attractive solution for researchers seeking to ensure the integrity of their citations. As the use of AI-generated content continues to grow in academic publishing, tools like CheckIfExist will play a vital role in maintaining the accuracy and reliability of published research.
Recommendations
- ✓ Researchers should adopt CheckIfExist as a standard tool for verifying bibliographic references in academic publishing.
- ✓ Journal editors and publishers should consider integrating CheckIfExist into their submission and review processes to ensure the accuracy and reliability of published findings.
