A Dataset for Named Entity Recognition and Relation Extraction from Art-historical Image Descriptions
arXiv:2602.19133v1 Announce Type: new Abstract: This paper introduces FRAME (Fine-grained Recognition of Art-historical Metadata and Entities), a manually annotated dataset of art-historical image descriptions for Named Entity Recognition (NER) and Relation Extraction (RE). Descriptions were collected from museum catalogs, auction listings, open-access platforms, and scholarly databases, then filtered to ensure that each text focuses on a single artwork and contains explicit statements about its material, composition, or iconography. FRAME provides stand-off annotations in three layers: a metadata layer for object-level properties, a content layer for depicted subjects and motifs, and a co-reference layer linking repeated mentions. Across layers, entity spans are labeled with 37 types and connected by typed RE links between mentions. Entity types are aligned with Wikidata to support Named Entity Linking (NEL) and downstream knowledge-graph construction. The dataset is released as UIMA
arXiv:2602.19133v1 Announce Type: new Abstract: This paper introduces FRAME (Fine-grained Recognition of Art-historical Metadata and Entities), a manually annotated dataset of art-historical image descriptions for Named Entity Recognition (NER) and Relation Extraction (RE). Descriptions were collected from museum catalogs, auction listings, open-access platforms, and scholarly databases, then filtered to ensure that each text focuses on a single artwork and contains explicit statements about its material, composition, or iconography. FRAME provides stand-off annotations in three layers: a metadata layer for object-level properties, a content layer for depicted subjects and motifs, and a co-reference layer linking repeated mentions. Across layers, entity spans are labeled with 37 types and connected by typed RE links between mentions. Entity types are aligned with Wikidata to support Named Entity Linking (NEL) and downstream knowledge-graph construction. The dataset is released as UIMA XMI Common Analysis Structure (CAS) files with accompanying images and bibliographic metadata, and can be used to benchmark and fine-tune NER and RE systems, including zero- and few-shot setups with Large Language Models (LLMs).
Executive Summary
The article introduces FRAME, a meticulously annotated dataset designed for Named Entity Recognition (NER) and Relation Extraction (RE) within the domain of art-historical image descriptions. Collected from diverse sources such as museum catalogs and scholarly databases, the dataset focuses on single-artwork descriptions, providing annotations across three layers: metadata, content, and co-reference. The dataset includes 37 entity types aligned with Wikidata, facilitating Named Entity Linking (NEL) and knowledge-graph construction. Released in UIMA XMI CAS format, FRAME is intended to benchmark and fine-tune NER and RE systems, including applications with Large Language Models (LLMs).
Key Points
- ▸ Introduction of FRAME dataset for NER and RE in art-historical image descriptions.
- ▸ Annotations across three layers: metadata, content, and co-reference.
- ▸ Alignment with Wikidata for NEL and knowledge-graph construction.
- ▸ Dataset released in UIMA XMI CAS format for benchmarking and fine-tuning.
- ▸ Potential applications with zero- and few-shot setups with LLMs.
Merits
Comprehensive Annotation
The dataset provides detailed annotations across multiple layers, ensuring a rich and nuanced understanding of art-historical descriptions.
Alignment with Wikidata
The alignment with Wikidata enhances the dataset's utility for Named Entity Linking and knowledge-graph construction, making it more versatile for various applications.
Diverse Data Sources
The data is collected from a variety of reputable sources, ensuring a broad and representative sample of art-historical descriptions.
Demerits
Limited Scope
The dataset focuses solely on single-artwork descriptions, which may limit its applicability to more complex or multi-artwork contexts.
Annotation Complexity
The three-layer annotation structure, while comprehensive, may introduce complexity that could be challenging for some users to navigate.
Potential Bias
The reliance on specific sources for data collection may introduce biases that could affect the generalizability of the dataset.
Expert Commentary
The introduction of the FRAME dataset represents a significant advancement in the field of art-historical NLP. By providing a meticulously annotated dataset that spans multiple layers of annotation, the authors have created a valuable resource for researchers and practitioners. The alignment with Wikidata is particularly noteworthy, as it facilitates the integration of the dataset with broader knowledge-graph initiatives. However, the focus on single-artwork descriptions may limit the dataset's applicability in more complex scenarios. Additionally, the three-layer annotation structure, while comprehensive, may present a steep learning curve for some users. Despite these limitations, the dataset's potential to enhance NER and RE systems in the art-historical domain is substantial. The practical implications of this work are evident in the improved automation of metadata extraction and knowledge organization, which can streamline the digitization and preservation of cultural heritage. Policy-wise, the dataset underscores the importance of AI in cultural heritage preservation, potentially influencing future policies in this area.
Recommendations
- ✓ Expand the dataset to include multi-artwork descriptions to broaden its applicability.
- ✓ Provide additional documentation and tutorials to help users navigate the complex annotation structure.
