When Meaning Gets Lost in Translation: Evaluating Semantic Alignment of LLM-Generated Dublin Core Metadata for Korean Cultural Heritage

1 Introduction and Problem Statement

Cultural heritage collections hold irreplaceable records of human experience — artefacts, manuscripts, oral histories, and objects that embody the knowledge, identity, and values of specific communities. The metadata attached to these objects is not neutral: it shapes how communities are represented, how histories are interpreted, and whose voices are heard in the archive. In this sense, metadata for cultural heritage functions as a site of meaning-making, extending beyond technical data organization.

The rapid adoption of LLMs for automated Dublin Core metadata generation introduces a fundamental tension. While automation offers efficiency gains, it risks imposing dominant cultural frameworks on materials that require nuanced, community-specific description. Foka et al. (2025) demonstrate that every stage of the AI pipeline can inherit and amplify historical biases embedded in existing records[4], and Foka & Griffin (2024) argue that AI applied to cultural heritage may perpetuate colonial or exclusionary worldviews at scale[3]. Birhane (2021) further grounds this concern in a relational ethics framework, showing how algorithmic systems disproportionately harm marginalized communities when designed without adequate cultural consideration[1].

Korean cultural heritage presents a particularly compelling case. Rich in historically specific terminology — dynastic period names, region-based classification systems, and community-defined subject vocabularies encoded in the Korean Subject Headings (KSSH) — Korean heritage objects require metadata that preserves local meaning. When LLMs default to Western-centric classification norms, this meaning is not merely translated: it is flattened and lost. This poster introduces a framework to identify, measure, and mitigate this loss, and presents the research design currently being implemented.

Although the empirical case is Korean, the underlying dynamic is not unique to it: any cultural or linguistic community whose descriptive vocabulary is underrepresented in LLM training distributions faces a comparable risk of dominant-culture flattening. This positions the present study as both a Korea-specific analysis and a transferable diagnostic approach for the wider Dublin Core community’s engagement with Meaning-Driven AI.

2 Background and Related Work

Prior work on AI-generated Dublin Core metadata reveals a consistent pattern: LLMs perform well on factual surface elements but poorly on semantically rich fields. Kim et al. (2023) evaluated ChatGPT-generated Dublin Core records for 90 Korean monographs and found satisfactory completeness (0.87) but substantially lower accuracy (0.71), with the weakest performance concentrated in Subject and Description — the very elements most critical for cultural heritage discovery and contextual interpretation[5].

Brzustowicz et al. (2026) extend this line of inquiry to community archives, finding that ChatGPT 4o generated records with approximately 70% element accuracy but significant inconsistencies in schema conformance and source interpretation[2]. Critically, their study introduces qualitative dimensions — bias, meaning, and context — that go beyond standard completeness and accuracy metrics, pointing toward the kind of meaning-level evaluation this poster pursues.

On the ethical side, Foka & Griffin (2024) establish that bias in cultural heritage collections is inherent — rooted in historical practices of digitization, selection, and description — and that AI systems may amplify rather than neutralize it[3]. Zavalina & Burke (2021) further demonstrate the difficulty of producing high-quality Dublin Core metadata even among trained graduate students, reinforcing that metadata creation is a skilled, culturally situated practice that AI cannot yet fully replicate[7].

To improve conceptual rigor, semantic flattening is defined here as an unintended, generation-time replacement of culturally or historically specific terminology with a generalized or dominant-culture equivalent, distinguished from three adjacent concepts. Unlike semantic drift, which describes the gradual reinterpretation of a term’s meaning through use over time, semantic flattening is a synchronic compression occurring within a single act of AI generation. Unlike cultural bias, which denotes systemic favoritism embedded in training data or model behavior at a structural level, semantic flattening is defined at the level of observable metadata output rather than as a claim about underlying model mechanisms. And unlike metadata homogenization — the deliberate, institution-driven standardization of vocabularies to support interoperability — semantic flattening is an unintended by-product of automated generation rather than a chosen cataloging strategy, and concerns loss of content-level specificity rather than structural convergence.

3 Research Questions

RQ1. To what extent do LLM-generated Dublin Core records for Korean cultural heritage objects align with expert-curated records in terms of semantic meaning, beyond syntactic accuracy?

RQ2. Which Dublin Core elements exhibit the greatest semantic drift — specifically the loss of culturally specific terms in favor of dominant-culture defaults — in AI-generated outputs?

4 Methodology

This study employs a two-phase comparative evaluation design. Data collection and analysis are currently underway; the research design is described below.

5 Illustrative Example

To ground the framework in a concrete case ahead of full pilot results, this section presents a real, citable object: the National Treasure Celadon Prunus Vase with Inlaid Cloud and Crane Design (Goryeo dynasty, 12th century; sanggam inlay technique), held at the Kansong Art Museum, Seoul. Its official designation is government-standardized under the Korea Heritage Service’s English Designation Rules for Cultural Heritage Names, and can be treated as an expert-curated dc:subject baseline: “Celadon; Goryeo dynasty; Maebyeong (prunus vase); Cloud-and-crane design; Sanggam inlay technique.” A preliminary zero-shot generation, produced without access to this official designation, rendered the same object as “a green ceramic vase with bird and cloud motifs, believed to be from East Asia” — retaining general object categorization while omitting the dynastic period, the object-type term (maebyeong), and the technique-specific term (sanggam). This single-item comparison illustrates the intended unit of analysis for the full pilot; the complete dataset will be constructed from structured records obtainable through the e-Museum (emuseum.go.kr) open API of the National Museum of Korea and affiliated institutions.

6 Work in Progress and Expected Findings

This study is currently in the data collection and pilot design phase. Quantitative evaluation results are not yet available and will be reported in subsequent publications. However, the theoretical basis for anticipating semantic flattening is well grounded in existing literature. Kim et al. (2023) demonstrate that LLM performance is systematically weakest on Subject and Description elements — the fields in which culturally specific Korean terminology is most densely represented[5]. Brzustowicz et al. (2026) similarly find that AI-generated records exhibit meaningful inconsistencies in how primary source material is interpreted, suggesting that culturally embedded meaning is among the first aspects to be lost in automated generation[2].

Based on these patterns in the literature, we anticipate that LLM-generated records for Joseon-era artefacts will show lower semantic fidelity than records for intangible heritage items, given the greater terminological distance between period-specific Korean classifications and the Western-centric training data of general-purpose LLMs. We also expect semantic flattening to be most pronounced in dc:subject and dc:coverage — the elements that carry the heaviest burden of cultural and geographic specificity. A secondary pattern of interest, to be explored as an additional dimension in the full study, concerns whether AI-generated descriptions of materials related to historically marginalized communities reproduce dominant-culture framing — an ethical representativeness dimension that extends the core semantic drift analysis toward the broader meaning alignment goals of DCMI 2026.

7 Research Plan and Future Directions

The full study is planned for completion in 2026–2027. Evaluation will be conducted by a panel of three domain experts to enable inter-rater reliability assessment, addressing a key methodological constraint of single-reviewer designs. LLMs will be evaluated under zero-shot, few-shot, and chain-of-thought prompting conditions, allowing systematic comparison across models and prompting strategies.

The completed study aims to deliver three contributions. First, it will produce an empirically validated, two-dimensional meaning alignment evaluation framework applicable to non-Western cultural heritage contexts beyond Korea. Second, it will generate a comparative analysis of semantic drift patterns across models and prompting conditions, providing evidence-based guidance for practitioners selecting LLMs for heritage metadata workflows. Third, in direct response to the design question raised by RQ2, it will formalize human-in-the-loop metadata curation principles that explicitly prioritize cultural meaning — a concrete methodological step toward metadata systems that align with human values as called for by DCMI 2026.

8 Conclusion

Cultural heritage metadata carries meaning that belongs to communities — it is not a neutral technical artifact. This poster argues that the automation of Dublin Core generation for Korean cultural heritage objects risks producing semantic flattening: the substitution of community-specific cultural meaning with globally dominant classification defaults. This is precisely the failure mode that the DCMI 2026 theme of Meaning-Driven AI seeks to address.

While quantitative results are forthcoming, the theoretical framework and research design presented here offer a replicable foundation for evaluating AI-generated metadata not only for accuracy, but for the human values it encodes or omits. As AI systems become more deeply embedded in the management of cultural memory, ensuring that metadata remains a genuinely meaning-driven practice constitutes a technical and ethical obligation. This research-in-progress presents a structured approach toward that objective.

References

1. [1] A. Birhane, Algorithmic injustice: A relational ethics approach. in Patterns, vol. 2, no. 2, pp. 100205, 2021. https://doi.org/10.1016/j.patter.2021.100205.
2. [2] R. Brzustowicz, A. Marchetti, and C. Stevenson, 'Appears to be about': An evaluation of AI-generated metadata quality for community archives. in Information Research, vol. 31, no. 1, 2026. https://doi.org/10.47989/ir3162419.
3. [3] A. Foka and G. Griffin, AI, cultural heritage, and bias: Some key queries that arise from the use of GenAI. in Heritage, vol. 7, no. 11, pp. 6125-6136, 2024. https://doi.org/10.3390/heritage7110287.
4. [4] A. Foka, G. Griffin, D. Ortiz Pablo, P. Rajkowska, and S. Badri, Tracing the bias loop: AI, cultural heritage and bias-mitigating in practice. in AI & Society, vol. 40, pp. 5835-5847, 2025. https://doi.org/10.1007/s00146-025-02349-z.
5. [5] S. Kim, H. Lee, and Y. Lee, Quality evaluation of automatically generated metadata using ChatGPT: Focusing on Dublin Core for Korean monographs. in Journal of the Korean Society for Information Management, vol. 40, no. 2, pp. 183-209, 2023. https://doi.org/10.3743/KOSIM.2023.40.2.183.
6. [6] UNESCO, Recommendation on the ethics of artificial intelligence. United Nations Educational, Scientific and Cultural Organization, 2021. https://unesdoc.unesco.org/ark:/48223/pf0000381137.
7. [7] O. L. Zavalina and M. Burke, Assessing skill building in metadata instruction: Quality evaluation of Dublin Core metadata records created by graduate students. in Journal of Education for Library and Information Science, vol. 62, no. 4, pp. 423-442, 2021. https://doi.org/10.3138/jelis.62-4-2020-0083.