MatQnA: A Benchmark Dataset for Multi-modal Large Language Models in Materials Characterization and Analysis

This repository hosts the MatQnA dataset, a multi-modal benchmark dataset presented in the paper MatQnA: A Benchmark Dataset for Multi-modal Large Language Models in Materials Characterization and Analysis.

MatQnA is specifically designed to evaluate the capabilities of AI models in the specialized field of materials characterization and analysis. It includes data from ten mainstream characterization methods, such as X-ray Photoelectron Spectroscopy (XPS), X-ray Diffraction (XRD), Scanning Electron Microscopy (SEM), and Transmission Electron Microscopy (TEM).

The dataset comprises high-quality question-answer pairs, incorporating both multiple-choice and subjective questions, developed using a hybrid approach combining LLMs with human-in-the-loop validation. It serves as a crucial resource for systematically validating and advancing multi-modal AI models in scientific research scenarios related to materials.