When Metadata Is Not Enough: Repository Interoperability in a National Library Open-Source Discovery Platform

2.1 Quality and Completeness of Metadata in Digital Repositories

The quality of metadata is a fundamental factor in the success of information retrieval systems and web-scale discovery services. High-quality metadata allows information sources to be effectively indexed, discovered, and utilized within a distributed digital repository ecosystem. One of the most widely studied dimensions of metadata quality is metadata completeness, which refers to the extent to which the required metadata elements are available and well-filled. Within the framework of classical metadata quality, completeness is seen as a minimum prerequisite for a record to function optimally within an information system (Bruce & Hillmann, 2004)[7].

In recent decades, metadata completeness has been increasingly used as a quantitative indicator to assess system quality, primarily because it is relatively easy to measure and can be applied across metadata schemas. Asok et al. (2024) emphasize that complete metadata is an essential foundation for open science practices, particularly in supporting the visibility, interoperability, and reuse of research data[1]. Rasuli et al. also showed that metadata completeness contributes to increased visibility and user engagement, for example, in institutional repositories and library collection automation (2025)[22].

Various approaches have been developed to systematically measure the completeness of metadata. Ochoa and Duval (2009) introduced the automated evaluation of metadata quality using element-presence-based metrics[20], while Király and Büchler (2019 developed a weighted approach that considers the relative importance of each metadata element[13]. Another approach considers the hierarchical structure of metadata, where an element is considered complete only if all its sub-elements are filled (Margaritopoulos et al., 2012)[18]. Although these approaches provide a strong quantitative basis for assessing the descriptive quality of metadata, most still view metadata as a static entity evaluated at a single point in time.

Beside completeness, other quality dimensions such as accuracy and consistency are also widely discussed in the literature. However, several studies indicate that incompleteness, format inconsistency, and filling errors remain common issues in institutional repositories, even in stable systems (Çolak & Eroğlu, 2025)[8]. This finding indicates that the quality of metadata depends not only on the schema or guidelines used, but also on the practices of ongoing metadata management and maintenance.

2.2 Repository Interoperability and the Sustainability of Metadata Harvesting

In a large-scale aggregation environment, the quality of metadata cannot be separated from the issues of interoperability and operational sustainability. Web-scale discovery services rely on the ability of repository systems to provide metadata that can be consistently harvested thru interoperability protocols such as OAI-PMH. In this context, interoperability is not only determined by the completeness of metadata elements, but also by the stability of the endpoint, format compliance, and the continuity of system maintenance.

Global aggregators like CORE indicate that the heterogeneity of repository system platforms, variations in institutional technical capacity, and differences in metadata management practices are major challenges in cross-system metadata integration (Knoth et al., 2023)[14]. Manocci's study highlights that many repositories listed in national or international registries are not always sustainably harvestable due to URL changes, server failures, or a lack of technical maintenance (2022)[17]. This condition indicates that interoperability is a dynamic process susceptible to degradation over time.

Sustainable harvesting is an important indicator in assessing the interoperability readiness of repositories. Outdated metadata or repositories that are no longer responsive to harvesting requests can cause information to become obsolete, out of sync, or even disappear from the aggregator's index. This phenomenon is often associated with the concept of metadata decay, which is the decline in the quality and relevance of metadata due to a lack of long-term maintenance. Although the dimension of timeliness has long been recognized as part of metadata quality, this aspect is relatively rarely operationalized systematically in repository evaluations (Kumar & Harinarayana, 2024)[16].

Some recent studies are beginning to emphasize the importance of combining metadata quality evaluation with operational sustainability indicators. Baglioni et al. (2025) highlight the need for a registry-based interoperability approach that not only records the existence of repositories but also validates the technical readiness and sustainability of metadata integration[3]. A similar approach is also seen in studies on repository registries and national aggregation networks, which emphasize the importance of technical coordination and sustainable maintenance to ensure the stability of discovery services (Knoth et al., 2025)[15].

4.1 Mapping of IOS Registration Field Metadata Categories

The Indonesian One Search (IOS) registration dataset contains metadata fields that not only function as a description of the repository's identity but also track the technical readiness and operational status of its integration into the national retrieval system. Due to the heterogeneous nature of the field characters, category mapping is necessary to ensure that each attribute can be read consistently within the interoperability evaluation framework. In this study, the entire field is understood thru four functional categories: (1) administrative registration metadata, (2) interoperability technical metadata, (3) operational harvesting metadata, and (4) system administration metadata. The four components form a sequence that represents the repository integration cycle: from identity at registration, technical access for metadata exchange, activity trail of harvesting and indexing, to workflow control and change auditing.

4.2 Data Preprocessing

Data preprocessing in this study is a fundamental step to ensure that the Indonesian One Search (IOS) dataset is in a suitable condition for analysis. The raw dataset obtained from the IOS registration and harvesting system exhibits variations in format, the presence of null values, and heterogeneity in metadata structure, which could potentially affect the accuracy of quantitative analysis. Pre-processing focuses on four main aspects: data cleaning, format consistency, semantic completeness, and mapping of analytical indicators. This series of steps ensures that each metadata field can be interpreted consistently within the framework of repository interoperability evaluation. With systematically processed datasets, further analysis can more accurately reflect the institution's metadata conditions within the Indonesian One Search ecosystem, while also supporting the validity and comparability of research results.

4.3 Analysis of Metadata Completeness: Distribution, Dimensions, and Systemic Patterns

The distribution of health metadata scores indicates a concentration of repositories in the mid-to-low score range, with a small percentage of repositories at a very high level of completeness. This uneven distribution pattern reflects what is referred to in the literature as metadata quality skewness, which is the condition where most repositories only meet minimum metadata requirements, while more mature metadata practices are only found in a limited group of institutions (Park & Tosaka, 2015)[21].

Some studies confirm that this type of distribution pattern is common in open and decentralized institutional repository ecosystems. Stvilia et al. (2017) explain that without continuous audit and feedback mechanisms, metadata tends to evolve unevenly, depending on the local capacity of the managing institution[24]. In the national WSDS context, this means that the existence of repositories with high metadata scores does not automatically improve the overall quality of the system, as the majority of repositories remain at a minimum level of contribution.

This distribution also strengthens the argument that interoperability assessment cannot be based solely on average values, but must consider the spread and concentration of scores. This kind of distributional approach has been recommended in the evaluation of large-scale information systems to avoid conclusions biased toward high-capacity institutions (Batini, Cappiello & Francalanci, 2016)[4]. Thus, the histogram of iOS metadata scores not only describes the technical condition of the metadata but also reveals structural disparities in the interoperability readiness of repositories.

The boxplot visualization shows a sharp difference between the basic, administrative, technical, and harvesting metadata dimensions. The basic metadata dimension shows a relatively high and stable level of completeness, while the technical and harvesting dimensions exhibit a low median with greater variation. This pattern indicates that metadata practices in repositories are more focused on resource description rather than supporting operational interoperability.

This phenomenon aligns with findings in metadata management studies that distinguish between descriptive sufficiency and functional interoperability. Gilliland (2016) emphasizes that descriptive metadata is often prioritized because it is directly related to the visibility and legitimacy of collections, while technical and operational metadata is considered a less institutionally focused supporting layer[11]. As a result, although the repository appears "complete" descriptively, its capacity to actively participate in the aggregation system remains limited.

Additionally, the low completeness in the technical and harvesting dimensions can also be interpreted as an indicator of weak integration sustainability. According to Higgins (2018), long-term interoperability in digital systems is highly dependent on process metadata, including the recording of harvesting and update activities[12]. When this type of metadata is not managed consistently, aggregation systems risk functional degradation even if descriptive metadata remains available.

The differences between dimensions shown by the boxplot also support the multidimensional evaluation approach in this study. As stated by Zeng and Qin (2016), the quality of metadata cannot be reduced to a single indicator, but must be read as a combination of interconnected dimensions[26]. This visualization strengthens the argument that the completeness of administrative metadata is not sufficient to represent the interoperability readiness of repositories within the national WSDS.

These two visualizations together show that the quality of repository metadata within the Indonesian OneSearch ecosystem is uneven and highly dependent on the dimensions being evaluated. While basic metadata is relatively stable and complete, the technical and operational metadata that underpins long-term interoperability still shows structural weaknesses. This finding reinforces the view that WSDS interoperability cannot be evaluated solely thru the completeness of descriptive metadata, but rather requires a multidimensional approach that considers score distribution and the operational dynamics of the repository.

4.4 Dynamics of Repository Harvesting Activity: Patterns, Challenges, and Sustainability

The distribution of sustainability harvesting classes shows that most repositories in the Indonesian OneSearch ecosystem fall into the "never harvested" category, while the proportion of repositories actively harvested in the last year is very small. This pattern indicates that the integration of repositories into the national WSDS is often administrative, without sustained operational connectivity. In other words, the existence of a repository within a registration system does not automatically transform into an active metadata flow within the discovery index.

The dominance of the "never harvested" and "dormant" categories reflects a phenomenon known in repository aggregation literature as structural harvesting failure, which is a systemic and recurring integration failure, not just a temporary technical glitch. Mannocci, Baglioni, and Manghi (2022) show that at a large aggregation scale, many repositories become inaccessible due to endpoint changes, lack of system maintenance, or the absence of periodic monitoring mechanisms[17]. In the context of iOS, this pattern indicates a gap between the registration process and the sustainable integration of metadata.

From an interoperability perspective, these findings confirm that harvesting sustainability is a more sensitive indicator than the completeness of registration metadata in assessing repository readiness. Repositories can have complete registration metadata, but still not contribute to WSDS if harvesting activity is not sustained. This condition strengthens the argument that interoperability needs to be understood as a dynamic process that depends on technical and operational continuity, rather than as a static status at a single point in time (Higgins, 2018)[12].

If analyzed by repository type, there is a noticeable difference in the sustainability patterns of harvesting. The journal repository shows a relatively better proportion of harvesting activity compared to ETD repositories, datasets, and other types. This pattern reflects that repositories with regular publication cycles and a strong visibility orientation tend to have greater institutional incentives to maintain their technical interoperability. Conversely, ETD and dataset repositories are dominated by the never harvested and dormant categories. This indicates that although these repositories play a crucial role in the knowledge and open science ecosystem, managing their interoperability is often not a long-term priority. Repository management literature indicates that repositories with irregular content flows and fragmented management responsibilities are more prone to experiencing a decline in metadata integration quality (Park & Tosaka, 2015)[21].

The low sustainability of harvesting on dataset repositories also underscores the complexity of research data interoperability. Batini, Cappiello, and Francalanci (2016) emphasize that dataset interoperability demands higher metadata investment and technical maintenance compared to text-based repositories[4]. Without adequate policy and resource support, this type of repository tends to fail to maintain metadata integration within the national aggregation system.

Both visualizations together show that the main interoperability challenge for repositories in IOS does not lie in the registration stage, but rather in the post-registration phase. Low harvesting sustainability indicates the risk of metadata decay, which is a condition where metadata remains structurally present but loses its operational functionality within the discovery system (Gilliland, 2016)[11]. In the long term, this condition can lead to institutional representation imbalances and reduce the reliability of WSDS as a national retrieval service.

The variation in sustainability across repository types also confirms that interoperability is contextual and influenced by domain characteristics, institutional practices, and technical governance. Therefore, the national WSDS evaluation needs to move beyond a uniform approach and consider monitoring and intervention mechanisms tailored to the type of repository. This finding reinforces the need for sustainable harvesting indicators as an integral part of repository interoperability evaluation, not merely a supplement to registration metadata analysis.

4.5 Integrating Metadata Completeness, Harvesting Sustainability, and Interoperability Readiness

This density visualization shows that most repositories within the Indonesian OneSearch ecosystem are concentrated in a combination of low to medium metadata completeness with very low harvesting sustainability. The highest density is in areas with a harvesting sustainability value approaching zero, although the completeness level of basic metadata varies. This pattern indicates that the completeness of registration metadata is not automatically directly proportional to the sustainability of repository integration into aggregation systems, a phenomenon that has been widely reported in large-scale repository interoperability studies (Park & Tosaka, 2015; Mannocci, Baglioni & Manghi, 2022)[17, 21].

The low repository density in areas with a combination of high metadata completeness and high harvesting sustainability indicates that ideal interoperability conditions are relatively rare. This finding supports the view that descriptive metadata is often prioritized for initial legitimacy and visibility purposes, while the technical and operational metadata that underpins sustainable harvesting receives less institutional attention (Gilliland, 2016; Zeng & Qin, 2016)[11, 26]. Thus, structurally complete metadata does not necessarily reflect a repository's readiness to actively participate in the web-scale discovery services ecosystem.

The distribution of repositories with varying levels of metadata completeness but still at a low harvesting level also indicates a structural decoupling between metadata quality and the operational performance of interoperability. Global repository aggregation literature indicates that the sustainability of harvesting is more influenced by technical and governance factors, such as the stability of the OAI-PMH endpoint, repository software maintenance, and human resource sustainability, than by the mere completeness of registration metadata (Knoth et al., 2023; Higgins, 2018)[12, 14].

From a system evaluation perspective, this density pattern strengthens the argument that repository interoperability is a dynamic process susceptible to degradation if not accompanied by continuous monitoring and maintenance. The condition where metadata remains available but is no longer actively harvested reflects the phenomenon of metadata decay, which directly impacts the timeliness and reliability of national discovery indexes (Higgins, 2018; Mannocci et al., 2022)[12, 17]. Therefore, this visualization confirms the need to integrate metadata completeness indicators with sustainable harvesting indicators in assessing repository interoperability readiness.

This scatterplot visualization shows that the relationship between metadata completeness and harvesting sustainability is weak and non-linear. The slightly upward sloping regression line indicates a positive trend, but its very low slope confirms that increasing the completeness of basic metadata only contributes marginally to the sustainability of harvesting activities. In other words, repositories with more complete metadata do not consistently show a higher level of sustainable harvesting.

The wide distribution of points across various levels of metadata completeness, including repositories with high completeness but low harvesting values, suggests that metadata completeness is not the primary predictor of operational interoperability. This finding aligns with the literature emphasizing that the quality of descriptive metadata often serves as an initial prerequisite, but is not sufficient to guaranty long-term technical integration success in aggregation and discovery systems (Park & Tosaka, 2015; Zeng & Qin, 2016)[21, 26].

The variation in dot color based on repository type shows that the relationship between metadata completeness and harvesting sustainability is also contextual and domain-specific. Journal repositories tend to appear at a higher harvesting rate compared to other repository types, although they don't always have the highest metadata completeness. This pattern indicates that institutional factors—such as publication rhythm, visibility demands, and management support—have a greater influence on harvesting sustainability than metadata completeness alone. Studies on global repository aggregation show that repositories with regular content update cycles are better able to maintain technical interoperability, regardless of variations in the quality of their descriptive metadata (Knoth et al., 2023)[14].

Conversely, the dataset and ETD repositories show a more inconsistent distribution, with many points at low harvesting levels despite varying metadata completeness. This condition reinforces the finding that the interoperability of this type of repository is highly dependent on the sustainability of technical maintenance and data management policies, not just on filling in registration metadata (Higgins, 2018)[12]. In the national WSDS context, this confirms that the interoperability improvement strategy cannot focus solely on enhancing metadata completeness, but needs to be directed toward strengthening operational harvesting mechanisms.

Conceptually, this visualization supports the argument that repository interoperability is the result of a multidimensional interaction between metadata, technology, and institutional practices. The weak relationship between metadata completeness and harvesting sustainability indicates a structural decoupling between descriptive readiness and operational performance. Therefore, a comprehensive evaluation of WSDS interoperability needs to integrate sustainable harvesting indicators and institutional context as key components of repository readiness assessment.

This boxplot visualization clearly shows differences in metadata capacity across repository types, confirming that interoperability readiness is not uniform within the Indonesian OneSearch ecosystem. Journal and ETD repositories show a relatively higher median metadata capacity compared to book, dataset, and other categories of repositories. This pattern indicates that repositories with a strong academic orientation and a more structured content flow tend to have more mature metadata management practices.

The high metadata capacity in journal repositories can be attributed to the demands of visibility, standardization, and the sustainability of scientific publications. Repository management literature indicates that journal repositories generally have stricter metadata policies and more stable integration with global aggregation and indexing systems, thus consistently supporting higher metadata capacity (Knoth et al., 2023; Park & Tosaka, 2015)[14, 21]. This explains why journal repositories, despite not always having the highest metadata completeness, still demonstrate better interoperability readiness.

ETD repositories occupy a middle position with considerable variation, reflecting the heterogeneity of management practices at the institutional level. Some ETD repositories exhibit metadata capabilities approaching those of journal repositories, while others are at a lower level. This variation aligns with the finding that ETD management is highly dependent on institutional policies and the continuity of technical support, resulting in uneven quality of the generated metadata (Higgins, 2018; Gilliland, 2016)[11, 12].

Conversely, the "other" dataset and category repositories show a lower median metadata capacity with a wide range of values. This condition indicates that although dataset repositories have significant potential in the context of open science, their metadata practices and interoperability are not yet fully established. Batini, Cappiello, and Francalanci (2016) emphasize that research data interoperability requires complex and sustained metadata investment; without adequate policy and resource support, dataset repositories are likely to experience metadata capacity limitations as reflected in this visualization[4].

The differences in metadata capacity between these repository types strengthen the argument that repository interoperability needs to be understood as a systemic capability, not merely the result of basic metadata completeness or simple harvesting activities. Metadata capacity serves as an integrative indicator, capturing the structural, technical, and operational readiness of the repository more comprehensively. Thus, this visualization bridges the previous completeness-sustainability relationship analysis toward the interoperability readiness classification of repositories in the next stage.

This heatmap shows that the interoperability readiness of repositories within the Indonesian OneSearch ecosystem varies significantly across repository types and tends to be concentrated at medium to low levels of readiness. Most repositories fall into the "Ready but not active" category, while the proportion of repositories that are truly in the "Well integrated" category is relatively limited and dominated by journal repositories. This pattern shows that although many repositories have met basic administrative and technical prerequisites, only a small fraction have been able to consistently maintain operational integration.

Journal repositories hold the most prominent position in the Well-integrated class, indicating that the combination of metadata capacity, harvesting sustainability, and technical governance is more mature compared to other repository types. International literature confirms that journal repositories generally have more stable interoperability standards because they are tied to the global scientific publishing ecosystem, including indexers, aggregators, and open access initiatives (Björk, 2017; Knoth et al., 2023)[5, 14]. Thus, the interoperability readiness of journal repositories is not only influenced by metadata, but also by external pressures and institutional incentives.

Conversely, the dataset and ETD repositories are dominated by the Ready but not active and Partially ready classes. This reflects that although the repository already has metadata elements and basic infrastructure, the sustainability of integration and operational maintenance remains a major challenge. Studies on research data management show that dataset interoperability requires more complex coordination of metadata, policies, and technical resources compared to text-based repositories, which is why many dataset repositories fail to achieve full integration (Whyte & Wilson, 2017; Borgman, 2015)[6, 25].

The "Not ready" category that still appears for some repository types, especially the "other" category, indicates structural gaps in interoperability readiness. The presence of repositories in this class indicates that the registration of repositories in the national WSDS has not been fully followed by continuous validation and mentoring mechanisms. The literature on discovery system evaluation emphasizes that without continuous interoperability governance, aggregation systems risk becoming mere lists of repositories rather than reliable retrieval infrastructures (Deodato, 2015; NISO, 2014)[10, 19].

Overall, this visualization confirms that repository interoperability readiness is tiered and highly influenced by domain context and institutional practices. This heatmap also reinforces the findings from the previous visualization that interoperability cannot be reduced to a single indicator, but is rather the result of integration between metadata capacity, harvesting sustainability, and system governance. Therefore, the interoperability readiness classification becomes an important instrument for the strategic and policy-oriented evaluation of the national WSDS.