Enhancing Discovery with AI: Volume Extraction and Summary Statements for Holdings Metadata

Mass digitization has enabled libraries and other cultural heritage institutions to identify collection gaps, support shared collection development, and highlight areas for future digitization that would enhance user discovery of resources. However, serials [1] present unique challenges for collection gap analysis due to varying practices in serials management—particularly in binding and inconsistencies in holdings metadata across institutions.

Unlike monographs, which typically have one bibliographic record describing a single item, serials bibliographic metadata represents the entire publication issued under a given title. Item-specific information—such as volume-level details and overall coverage—is recorded separately in holdings metadata, often summarized through statements that indicate the volumes held and any gaps or missing issues. While two international standards exist for recording holdings summary statements [2],[3], in practice, implementations frequently vary due to local policies. These policies are influenced by factors such as the library system in use, the availability of serials cataloging expertise, and the institution's strategic priorities. As a result, inconsistent holdings metadata presents significant challenges in aggregated environments like HathiTrust [4], where users rely on volume-level information submitted by individual institutions. These records typically lack a comprehensive summary statement for the entire serial title, making it difficult to identify missing volumes or issues. Consequently, both users and institutions face barriers in understanding the full extent of digitized coverage for a given serial.

While Artificial Intelligence (AI) and related technologies are increasingly tested and applied in cultural heritage institutions for metadata creation and enhancement, most efforts to date have focused on bibliographic metadata, not holdings metadata. This paper explores the potential of AI—specifically Google's Gemini model [5] and BERT model [6]—to train and extract serials volume information from the full text of digitized serial content and generate summary statements that include information about missing volumes. This approach aims to improve user understanding of digitized serial coverage and support libraries in making informed decisions about collection development and future digitization priorities.

The test sample was randomly selected from HathiTrust. Using the search term "Annual report" in the HathiTrust search interface, a serial title with digitized content contributed by multiple institutions was chosen. This particular title had 17 digitized bound volumes, which were downloaded in both .txt (11.8 MB total) and .pdf (385.1 MB total) formats. The corresponding MARC metadata was also retrieved using the HathiTrust Bibliographic API service [7]. For this serial title, four institutions contributed digitized volumes, each applying different practices for recording volume information. These variations in descriptive practices are summarized in Table 1 below.

Two Python scripts were developed for this testing, as illustrated in Figure 1. The first script was designed to process all 17 downloaded files, identify title pages and extract volume information using a trained Gemini 1.5-pro-002 AI model. It works by extracting text from each identified title pages by applying a structured Gemini prompt to do the tasks. Based on the model's response, the script generates structured output summarizing the volume information for each file. These summaries are saved in a designated output folder for further analysis.

The second script aggregates the extracted volume information produced by the first python script, generating a summary CSV file that includes a per-file breakdown and the complete volume range. During this process, any missing volumes are also identified. Using the full range and the missing volume data, the script generated a summary statement, which was then added to an existing MARC.XML record following the cataloging standard. This enriched MARC metadata holdings information provides clear, consolidated digitized volume information for a serial title.

A labeled data set and python script were also developed to train a BERT base model for detecting title pages within annual report volumes as illustrated in Figure 2. BERT was chosen as a Large Language Model that is easy to run and can work with smaller amounts of data, and is able to perform classification tasks with complicated text structures through its bidirectional context learning [8].

The labeled data set contained texts from a random selection of historical digitized annual reports. Title pages of annual reports, which contain date information and appear throughout the volumes, are labeled as 1, and other text pages from the volumes, such as report information, organization member lists, schedules, and financial statements are labeled 0. 100 title pages were selected manually and 80 non-title pages with varying length of texts were selected manually, attempting to include the different kinds of information and texts on non-title pages.

A crossvalidation test was conducted using the labeled data, with splitting 80% of the texts into training data and 20% into test data. The training data was used to train a BERT base uncased model, which generated predictions for which elements of texts in the test data were title pages. The predictions were evaluated against the ground truth of the test data.

Following the training and evaluation, the BERT model was applied to five randomly chosen historical volumes of annual reports. These volumes were split into individual pages using the OCR page delimiter through regular expressions. The model was applied to these pages to generate predictions and probabilities of each page being a title page.

The results of title page identification, volume information extraction, summary statement generation, and metadata enhancement using AI technologies were encouraging. The Gemini API performed reliably with the designed prompt and effectively supported batch processing of digitized serial content.

The first Python script produced a structured list of volume information for each digitized item. The second script consolidated these outputs into a single summary statement, identifying duplicate and missing volumes. The final summary expressed the full digitized range as "first volume - last volume," with any gaps clearly noted. This summary was programmatically integrated into the existing MARC.XML metadata, as illustrated in Figure 3. The enhanced metadata enables more accurate and accessible representation of digitized serial holdings.

However, the process also raised important questions that merit further discussion—particularly in the areas of volume disambiguation, edge cases in OCR or AI misidentification, and long-term maintenance of AI-enhanced metadata.

Through the initial cross-validation work, the BERT model was evaluated for its title page predictions against "ground truth" manually labeled test data. The model had an accuracy score of .8889, and a 0.9000 score for precision and recall. The model had strong accuracy and predictive ability. Moving forward, the model can be tested and continually evaluated on a larger data set.

The BERT model was used to detect title pages from five new annual reports contributed from different institutions within the HathiTrust Digital Library. These volumes are from the Ontario Historical Society digitized by Indiana University (ID: inu.30000117897003), the Board of Trade of the City of LaCrosse, WI digitized by the New York Public Library (ID: nyp.33433020893230), The Archaeological Institute of America digitized by Princeton University (ID: njp.32101067642437), The Michigan Historical Commission digitized by the New York Public Library (ID: nyp.33433091140131), and the American Seaman's Friend Society digitized by Harvard University (ID: hvd.32044100865286).

The BERT model produced probabilities for identifying title pages from the complete digitized serial volumes. For each volume, it identified the title pages through the probability of prediction accuracy. The model was able to correctly label all title pages within these volumes. Pages with a determined title page probability of above 0.9850 captured almost all title pages, with a few false positives included for the American Seamen's Friend Society volume. An example of correct labels and a false positive is presented in Table 2. Duplicate title pages are detected as well. This establishes a strong confidence threshold for detecting title pages using this model. Detected title pages reveal further information about the annual report years within the volumes.

The model detections confirmed the publication year range for volumes from three titles, the Archaeological Institute, LaCrosse Board of Trade, and Ontario Historical Society. The model reveals that the volume of the Michigan Historical Commission digitized by the New York Public Library only contains one report, May 28 to December 31, 1913, published in January of 1914. However, this volume was labeled as 'v.3-4 1913-1916.'

The model also reveals that the American Seaman's Friend Society volume from Harvard University contains reports from the American Bethel Society from 1860, Western Seamen's Friend Society in 1860 and 1861, and Boston Seaman's Society in 1860. Specific date information for these additional reports included in the volume are not mentioned in the metadata. The metadata includes these titles in the 580 field. The BERT model is successful in detecting title pages within historical annual reports, and should be improved through more data, evaluation, and application to other serial publications. The predictions of the model can improve metadata, confirming, correcting, and adding to information on included publication years and specific reports within historical volumes.

The current approaches of using Gemini and BERT models and selected digitized content for the research were tailored for annual reports—among the simplest forms of serial publications. These reports are typically issued once per year, with the publication year commonly serving as the volume designation. While the script using a Gemini prompt includes logic to handle noise and duplication (e.g., "Some PDFs may contain duplicate title pages for the same year. Only include each year once"), the core prompt was optimized for a specific use case, with instructions such as: "Identify all title pages that follow the structure of an 'Annual Report' or similar government publication," and "From each matching title page, extract the year (as a four-digit number)." The BERT model learns to recognize title pages from multiple irregular examples, but only draws from historical annual report volumes. Further coding to handle outputs could also work to remove duplicates and noise to find the correct date information.

To adapt the approaches for broader batch processing of various serial types, the prompt and the process must be scalable and flexible. Two possible directions for enhancing scalability are:

• Incorporating Bibliographic Metadata for Publication Patterns — information already available in serial bibliographic records can guide and refine the extraction process. Specifically, control field 008 position 18 indicates publication frequency and position 19 indicates regularity, and datafield 310 provides the current publication frequency. These MARC fields in bibliographic metadata can help define expectations for how volumes are structured and labeled, enabling more intelligent filtering and pattern recognition across different serial types.
• Training the Model on Title Page Structures — a second path is training the AI models to recognize a variety of title page structures beyond annual reports. By building a training dataset of labeled title pages—representing diverse serial formats, government documents, journals, and other periodic publications—the model can generalize more effectively to new inputs. This would support broader applicability for title page detection and volume extraction in batch workflows.

The current method demonstrates that AI can be effectively applied to assist in enhancing holdings metadata, beyond bibliographic metadata creation, that have not been addressed in recent years. At the same time, it reveals new opportunities for scaling these methods across various types of serial publications. Achieving this may require metadata-driven strategies and enhancements to AI models that account for the complexity and variability inherent in digitized serials.

This project was supported by the Berthold Family Research Fund for Information Access and Discovery.