The research uses the FAIR principle from the perspective of libraries to construct a reliable data selection and evaluation model for the study of the relationship between science and technology, in order to solve the problems of insufficient utilization of other data and unclear application logic, which mainly rely on paper and patent data in current research. Three issues are discussing in the paper: the format and interaction methods of multimodal data, the construction of indicator system, and the evaluation process. The result shows that the index construction method based on metadata interoperability is beneficial for the fusion of metadata layer, and use ontology to achieve semantic fusion, and identified scientific and technological relationships by multimodal data jointly.