This report presents the results of introducing and operating the generative AI–based chat system “AskDona” on the support site of the supercomputer Fugaku. The introduction and operation of the generative AI chat system, AskDona, were jointly conducted by GFLOPS Inc. (hereafter, “our company”) and the RIKEN Center for Computational Science (hereafter, R-CCS). In this report, we refer to this series of activities as “the Project.” Through the Project, our company and R-CCS aim to publicly share the insights gained and thereby promote the social implementation of generative AI technologies.

R-CCS operates a dedicated support site for users of the supercomputer Fugaku to provide high-quality assistance. Fugaku is used extensively throughout the year by many users, and the number of users continues to increase. According to the Fugaku Annual Report, which summarizes various activities and information related to Fugaku, the number of users in FY2023 was approximately 3,800, and the average number of daily login users (active users) was about 395. These figures represent increases of more than 10% compared with FY2022 (users: approx. 2,916; active users: approx. 355).

Users of Fugaku can obtain high-quality support through the Fugaku Support Site. However, the traditional support system at R-CCS has been primarily based on human operators, and as the number of users has grown in recent years, the workload of support personnel has increased, making the efficiency of support operations an urgent issue.

Several factors contributed to the growing support burden, the most significant of which was the sheer volume of information related to Fugaku. To solve computational science problems using Fugaku, users must possess accurate operational knowledge. To that end, extensive technical information is provided in the form of manuals, user guides, and other documents distributed as PDFs or HTML files. The total volume of such material reaches tens of thousands of pages. When users have questions, it is often difficult for them to locate the necessary information scattered across these vast materials, making self-resolution challenging and leading to frequent submission of support tickets. There was a clear need for a service that could automatically provide appropriate answers aligned with the user’s question intent—something beyond simple keyword search. In addition, R-CCS regularly extracts commonly asked questions from the numerous inquiries received and publishes their answers as summarized FAQ articles on the Fugaku Support Site. To enable users to utilize these valuable FAQ articles together with the manuals and guides in an integrated, cross-referenced manner, an improved information-access environment was also required.

Given this background, R-CCS began exploring the introduction of a generative AI–based automatic response system grounded in RAG (Retrieval-Augmented Generation) using data from the Fugaku Support Site, as well as the implementation of a more advanced search function capable of fuzzy and similarity-based retrieval, offering significantly greater convenience than a conventional simple search.

Our company provides the generative AI chat system AskDona, which is built on our proprietary RAG solution and leverages our strengths in generative AI technologies and data analysis. To participate in the Project, our company first underwent a “pre-deployment technical evaluation” in May 2024 to verify that the system satisfied R-CCS’s technical requirements. The initial version of AskDona (dona-rag-1.0) achieved perfect performance—answering all test questions correctly (100% accuracy) against the R-CCS requirement of at least 80% accuracy—and was selected for the Project based on its ability to deliver high-accuracy responses even when processing large-scale datasets.

The results reported in this document cover the period from July 9, 2024—the date AskDona was deployed on the Fugaku Support Site—through June 2025. These results were obtained with the cooperation of R-CCS as part of our joint evaluation activities. R-CCS promotes the public disclosure of insights gained through this initiative from the perspective of contributing to the advancement of Japanese science, technology, and AI utilization.

Interview Article: The Future of Supercomputer “Fugaku” and Generative AI Utilization as Envisioned by the RIKEN Center for Computational Science (R-CCS)

The structure of this report is as follows.

Chapter 2 examines changes in user behavior after the introduction of generative AI, analyzing both quantitative and qualitative aspects of how users’ question patterns and self-resolution processes have evolved. In particular, we demonstrate, using real-world usage data, that question patterns have shifted in quality, with increases in “composite queries” that require cross-referencing multiple documents. Chapter 3 clarifies the concrete impact of these usage trends on the support operations of R-CCS, focusing on both the reduction of workload for human support staff and the maintenance of support quality. Chapter 4 then addresses the technical solutions to the challenges identified in Chapter 2, presenting comparative evaluations that demonstrate the effectiveness of next-generation RAG architectures. Specifically, we compare the response accuracy of AskDona’s latest RAG architecture with several standard RAG systems using a set of questions that includes composite queries, thereby identifying the technical requirements needed for real-world operational environments.