Automated Classification of Chinese Books: A Large Language Model Approach to Knowledge Transfer and Domain Adaptation

3.1 Theoretical Basis and Research Question

From an information science perspective, book subject indexing is fundamentally a mapping from natural language semantic space onto a structured KOS[14]: it assigns a document, based on its explicit attributes and latent semantic content, to the corresponding category within that system. This process requires catalogers or automated systems to possess strong semantic recognition capabilities and reliable disciplinary judgment. While general-purpose LLMs offer broad semantic understanding, a substantial distributional gap exists between general-domain corpora and the rigorously hierarchical structure of CLC. Moreover, the evaluation result has demonstrated that zero-shot or few-shot indexing with general LLMs is not viable[15], with characteristic failures including difficulty distinguishing deep-level category boundaries, coarse-grained assignments, and category hallucination.

The central methodological challenge, therefore, is how to leverage the semantic competence and broad world knowledge of LLMs to improve their performance on indexing tasks. This study adopts transfer learning as its theoretical foundation. Transfer learning works by reusing general features learned in a source domain to support task completion in a target domain, thereby mitigating limitations arising from annotation scarcity or task complexity. This domain adaptation process utilizes specific fine-tuning and expert-annotated data to adjust model parameters. By calibrating internal attention mechanisms, it enables precise mapping from document semantics to the vast KOS, such as CLC. Accordingly, this study addresses the following research question: to what extent can domain knowledge-enhanced supervised fine-tuning improve hierarchical alignment and fine-grained category recognition in CLC-based indexing?

3.2 Knowledge-Augmented SFT Framework

Based on the transfer learning perspective outlined above, this study proposes a Knowledge-Augmented SFT Framework for CLC-Based Subject Indexing. The framework consists of three components: (1) bibliographic semantic data, (2) CLC hierarchical knowledge, and (3) LoRA-based supervised fine-tuning. By integrating domain knowledge into the adaptation process, the framework aims to improve hierarchical alignment and fine-grained category recognition.

This study adopts SFT as the primary domain adaptation pathway, enabling parameter-level alignment between general semantic representations and the disciplinary logic embedded in the CLC. Given the availability of expert-annotated bibliographic records and the need for efficient local deployment, a lightweight LoRA-based adaptation strategy is employed to balance indexing accuracy and computational cost.

4.1 Data Collection and Preprocessing

(1) Data Collection. Two data sources were used. The first comprises 32,837 CNMARC bibliographic records, from which titles, abstracts, and series information were extracted as semantic inputs. The second is the complete CLC (5th edition) dataset, yielding 53,326 class notation–category name pairs. These datasets jointly form the training and evaluation corpus for this study. For consistency, all bibliographic notations were validated against the CLC 5th edition, and the classification task was restricted to the standardized CLC category space.

(2) Dataset Construction. Bibliographic and category data were converted into structured fine-tuning instances (see Table 1). This dual-data approach maps document semantics to notations, while using category definitions to internalize the CLC's hierarchical logic. To optimize throughput, simplified task instructions were used to reduce per-sample token length, as empirical observation confirmed this does not degrade indexing performance.

Note: Original records are in Chinese; English translations are provided for illustrative purposes.

(3) Dataset Partitioning. Stratified sampling was applied to maintain consistent disciplinary distribution. To address long-tail distribution, categories with a frequency below five were assigned exclusively to the training set, while remaining categories followed a 9:1 split. For multi-label records, training data was decomposed into independent single-label instances to refine semantic correspondence, while original multi-label annotations were retained for testing to ensure realistic evaluation. The final dataset comprises 36,786 training instances and 2,019 validation instances.

4.2 Experimental Design

To identify the optimal trade-off between computational cost and indexing accuracy, this study develops a progressive technical framework consisting of three interrelated controlled experiments. These experiments form a hierarchical path where each stage establishes foundational parameters for the subsequent phase.

(1) Hyperparameter Sensitivity Experiment. This stage evaluates core SFT and LoRA hyperparameters, including rank ($r$), scaling factor ($\alpha$), learning rate, batch size, and gradient accumulation steps, to derive a stable baseline configuration. By establishing this unified control scheme, the study ensures rigorous variable control for subsequent model selection and knowledge injection.

(2) Backbone Model Comparative Experiment. Using the fixed baseline from Experiment 1, this study assesses the impact of model scale and architecture on domain adaptation. By comparing representative open-source LLMs under identical configurations, we quantify knowledge transfer efficiency across CLC long-tail distributions to guide model selection for resource-constrained library environments.

(3) Knowledge Injection Ablation Experiment. Built upon the optimal configuration and backbone from previous stages, this experiment isolates the contribution of dataset construction. We compare fine-tuning on bibliographic metadata alone versus a dual-data approach incorporating CLC notation-category name pairs. The aim is to quantify the role of CLC ontological knowledge in mitigating hallucinations and improving precision for deep-level category recognition.

4.3 Experimental Configuration

Experiments were conducted on the AutoDL platform (Ubuntu 22.04) using Python 3.12, PyTorch 2.8.0, and CUDA 12.8. Fine-tuning utilized LLaMA-Factory (v0.9.5.dev0) on an RTX 5090 (32 GB) GPU, incorporating LoRA for efficiency, 8-bit BitsAndBytes for quantization, and PyTorch SDPA for long-text acceleration.

To ensure deep Chinese semantic understanding and professional knowledge representation, we selected the Qwen model family, specifically Qwen 2.5-1.5B-Instruct, Qwen 2.5-7B-Instruct, and Qwen 3-8B. These instruction-tuned variants were chosen for their robust alignment and instruction-following capabilities, which eliminate the need for large-scale alignment from scratch while enhancing training efficiency. This selection spans multiple parameter scales, enabling a systematic investigation of the relationship between model size and indexing performance.

4.4 Evaluation Metrics

Evaluating CLC indexing requires addressing both classification accuracy and hierarchical coherence. This study utilizes precision, recall, and F1 scores through a layer-by-layer evaluation framework extending to Level 10. Instead of a global exact-match criterion, this hierarchical strategy quantifies performance degradation as depth increases, capturing the inherent complexity of XMLC tasks. This approach provides an objective assessment of logic internalization by isolating stable upper-level performance from specific errors at deeper subcategory levels.

5.1 Hyperparameter Sensitivity Results

To balance experimental rigor with training efficiency, this experiment was conducted as a pilot study. Smaller-scale models exhibit sufficient sensitivity gradients in hyperparameter response to serve as proxies for full-scale experiments. Qwen 2.5-1.5B-Instruct was selected as the test backbone to rapidly identify optimal configurations. Table 2 reports results from three representative runs out of ten, with corresponding configurations detailed in Table 3.

Based on the hyperparameter sensitivity analysis, the configuration from Experiment 02 was adopted as the unified baseline for all subsequent experiments, employing a learning rate of $5\times {10}^{-5}$, an effective batch size of 8, max samples >50,000, reasoning enabled = ture, and LoRA parameters $r$ = 8 and $\alpha$ = 16.

(1) Impact of sample scale on deep-level recognition. Expanding training samples from 10,000 to 100,000 (Exp 01 vs. Exp 02) yielded a 10-15% F1 gain at Levels 1-5. Critically, larger data volume activated recognition of long-tail categories, raising Level 10 F1 from 0% to 17.99%. This confirms that absolute data volume is a prerequisite for extracting differentiating semantic features in high-density XMLC tasks. This result suggests that large-scale hierarchical classification systems such as the CLC require sufficient representational coverage of bibliographic evidence to support fine-grained conceptual differentiation. In knowledge organization terms, deeper category boundaries are not defined by surface lexical cues alone, but by cumulative distinctions among disciplinary domains, subfields, and cataloging conventions. Therefore, a diverse and adequately sized set of training instances is necessary for the model to approximate the classificatory judgment required to distinguish semantically adjacent categories across the CLC hierarchy.

(2) Influence of learning dynamics. Learning efficiency was found highly sensitive to effective batch size and update frequency. An excessively large effective batch size, such as 192 led to premature convergence toward coarse-grained patterns, resulting in degraded classification performance. From a knowledge organization perspective, such learning dynamics favor superficial regularities while failing to support the gradual formation of stable conceptual boundaries. For large-scale hierarchical systems like CLC, effective domain adaptation depends on fine-grained, incremental parameter updates that allow the model to progressively differentiate closely related categories across levels, rather than relying on shallow pattern memorization.

5.2 Backbone Model Performance Comparison Results

With the fine-tuning strategy and training corpus held constant, this experiment evaluates model adaptability across parameter scales for CLC hierarchical classification (see Table 3).

Experimental results exhibit a clear scale dependency in model capacity. Qwen 3-8B demonstrated superior precision across all levels, with F1 scores exceeding 90% at Levels 1 and 2, suggesting that larger parameter spaces better encode latent disciplinary signals. Conversely, the 1.5B model showed lower precision despite high shallow-level recall, reflecting insufficient capacity for fine-grained discrimination at semantically ambiguous boundaries.

Performance degrades nonlinearly with increasing depth, driven by a progressive dilution of semantic distinctiveness. Beyond Level 6, the narrowing of intra-class semantic distance and the intensification of inter-class overlap exponentially increase discriminative difficulty in the extreme label space. While Qwen 2.5-1.5B precision falls below 30% at Levels 7–10, Qwen 3-8B maintains precision above 37% at Level 10, underscoring the necessity of parameter capacity for capturing fine-grained professional context.

Although Qwen 3-8B achieves a superior balance with an F1 of 77.32% at Level 5, diminishing marginal returns relative to the 7B variant are notable. Given the exacting standards of library cataloging, Qwen 2.5-7B offers the most balanced trade-off between computational cost and indexing accuracy for practical deployment.

5.3 Knowledge Injection Ablation Results

Qwen 3-8B was selected for the ablation study due to its superior performance in Experiment 2. This strategy aims to transition the model from statistical association to logically guided classification by utilizing CLC's category definitions as training constraints.

Instead of uniformly injecting the full CLC dataset, this study employs a sampling strategy focused on deep-level categories with extended notation paths. This approach leverages the hierarchical nesting of CLC; by processing deep-level notations, the model repeatedly activates the semantic prefixes representing their corresponding upper-level classes. This repetitive activation reinforces the model's internal representation of the overall hierarchical structure through the inherent inclusion relationships within the notation strings.

The enhanced model achieved consistent performance gains across all classification levels (Table 4). Improvements at Levels 1-3 were modest, indicating that broad disciplinary distinctions are largely internalized during large-scale pre-training. The most substantial and structurally meaningful gains were observed at levels 4–6, where F1 scores increased steadily by 2.69% to 7.62%. These levels correspond to the core strata of the CLC hierarchy, where broad disciplinary classes are systematically differentiated into more specialized subject domains. The introduction of CLC category knowledge therefore helped reinforce the model's sensitivity to hierarchical relations and conceptual boundaries within the classification system. However, at Levels 8–10, the gains became increasingly constrained by semantic sparsity and conceptual fragmentation. This indicates that parameter-efficient fine-tuning alone cannot fully reproduce the deepest classificatory decision logic of the CLC, where expert cataloging judgment and highly specific domain knowledge remain essential.

5.4 Qualitative Error Analysis

To complement the quantitative evaluation, we conducted a brief qualitative analysis of representative successful and erroneous classifications produced by the best-performing models. The cases reveal distinct patterns across the CLC hierarchy. Successful classifications typically occur when bibliographic resources contain clear disciplinary signals, enabling accurate alignment with both core classes and auxiliary notations. In contrast, shallow errors often arise in interdisciplinary works, where models prioritize dominant semantic themes over cataloging conventions. Most errors occur at intermediate levels of the hierarchy, where models correctly identify the general subject area but fail to follow the precise classificatory path or assign auxiliary subdivisions. Deep-level errors are concentrated in highly specialized categories with sparse training examples, indicating limitations in fine-grained hierarchical reasoning. Overall, the results suggest that knowledge-augmented fine-tuning substantially improves CLC adaptation, while intermediate and deep-level classification remains challenging for automated subject indexing.

6.1 Discussion

This study explores an efficient pathway to automated Chinese book classification through LoRA-based SFT of open-source LLMs. Compared with direct API-based use of large proprietary models, the local fine-tuning approach offers substantial advantages in professional specificity and data privacy protection; after targeted adaptation, models can deeply internalize the classification logic of CLC. The best-performing model, Qwen 3-8B, achieves a peak global F1 score of 93.68% at Level 1 and maintains 80.55% at Level 5. While the current state of the technique is not yet sufficient to replace human review entirely, it holds considerable promise as an assistive tool for substantially reducing the repetitive workload of cataloging staff.

Across the three controlled experiments, several consistent patterns emerge. Larger parameter models provide richer semantic representation space, enabling more accurate capture of latent disciplinary signals in bibliographic abstracts, though the relationship between model scale and indexing performance is nonlinear. The hyperparameter sensitivity experiment reveals that XMLC tasks are strongly dependent on data volume, with sample scale being decisive for activating recognition of long-tail categories. The knowledge injection experiment demonstrates that CLC ontological content plays a key role in supporting fine-grained semantic discrimination[16], particularly at deeper classification levels where the introduction of CLC category data substantially improves differentiation between semantically adjacent disciplinary boundaries. Taken together, these findings suggest that improving automated indexing depends not only on scaling model size but critically on deep alignment between high-quality domain knowledge and general semantic representations.

6.2 Conclusion

In summary, LoRA-based SFT provides a viable approach to automated Chinese book subject indexing that balances accuracy with cost. Dual-data fine-tuning combining bibliographic semantic data with CLC category knowledge is an effective method for improving LLM-based indexing performance. By incorporating a domain knowledge dataset encoding category definitions and hierarchical relationships, the SFT approach demonstrates strong knowledge internalization when processing CLC's complex hierarchical nesting. The fine-tuning procedure developed here exhibits robust knowledge transfer in deep classification tasks and clearly captures the characteristic performance degradation as semantic distinctiveness decreases across levels, reflecting the representational bottlenecks that remain for LLMs at extremely fine-grained classification depths. Low computational requirements make this approach a viable indexing support tool for small to medium-sized libraries and community documentation centers.

Several limitations merit acknowledgment. Compared with RAG approaches that enable retrieval-based reasoning, the SFT pathway operates as a black box, and its performance is highly sensitive to precise hyperparameter tuning, introducing a degree of interpretability limitation. Dataset construction and computational resource management during fine-tuning also retain a degree of technical complexity that could impede large-scale deployment. Finally, the generalizability of this approach to international indexing schemes with differing structures, including DDC and LCSH, remains to be investigated, as the current study focus is limited to the CLC system.