S1322
Interdisciplinary - Education in radiation oncology
ESTRO 206
were recorded and analyzed for mean values, standard deviations, and relative computational overhead. Results: The Llama 3.1 8B Instruct configuration achieved a mean response time of 1.71±0.38 seconds, compared to 2.98±1.02 seconds for the 70B Instruct configuration, representing 43% faster inference while maintaining comparable accuracy (4.6±0.6 versus 4.4±0.7; difference: 0.2 points). Both configurations achieved safety scores of 4.8±0.5. The 8B Instruct configuration demonstrated superior consistency in response times, indicating more predictable performance critical for clinical implementation. Conclusion: RTeachBot delivers equivalent educational accuracy while maintaining sub-2-second response times and superior consistency. The Llama 3.1 8B Instruct with RAG configuration represents an optimal balance for radiation oncology departments seeking real-time AI patient education solutions, particularly in resource- constrained settings. References: 1.Singhal, K., et al., Large language models encode clinical knowledge. Nature, 2023. 620(7972): p. 172- 180.2.Yang, R., et al., Retrieval-augmented generation for generative artificial intelligence in health care. npj Health Systems, 2025. 2(1): p. 2.3.Miao, J., et al., Integrating Retrieval-Augmented Generation with Large Language Models in Nephrology: Advancing Practical Applications. Medicina, 2024. 60(3): p. 445.4.Chen, X., et al., Enhancing diagnostic capability with multi-agents conversational large language models. npj Digital Medicine, 2025. 8(1): p. 159. Keywords: agent, RAG, clinical deployment AI-Ready Radiation Oncology: A Consensus-Built, Validated, International Knowledge Assessment for the Radiation Oncology Workforce Ciaran Malone 1,2 , Dylan Callens 3,4 , Michelle Leech 5,6 , Elizabeth Forde 5,6 , Carlos Cardenas 7 , Mark J Gooding 8,9 , Samantha Ryan 2 , Pierre Thirion 2,6 , Claire Fitzpatrick 2 , Theresa O'Donovan 10 , Antony Carver 11 , Irene Hernandez Giron 12 , Darragh Browne 2 , Catherine Rogerson 2 , Gerard. G. Hanna 2,5 , Jill Nicholson 2,5 1 Department of Radiotherapy, Erasmus MC Cancer Institute, University Medical Center Rotterdam, Rotterdam, Netherlands. 2 Radiation Oncology, St. Luke’s Radiation Oncology Network, Dublin, Ireland. 3 Laboratory of Experimental Radiotherapy, KU Leuven, Leuven, Belgium. 4 Department of Radiation Oncology, University Hospitals Leuven, Leuven, Belgium. 5 Trinity St. James’s Cancer Institute, Trinity College Dublin, Dublin, Ireland. 6 Applied Radiation Therapy Trinity, Proffered Paper 235
Digital Poster Highlight 161 RTeachBot Agentic RAG Framework: Breaking the Clinical Response Time Barrier for AI Patient Education Yu-Rou Chiou 1 , Yang-Hsien Lin 2 , Chih-Ying Liao 1 , Chi- Hsien Huang 1 , Rong-Tse Hsu 1 , Ya-Yun Yu 1 , Tzung-Chi Eddie Huang 2 , Simon See 2 , Ji-An Liang 1 , Ting-Chun Lin 1 1 Department of Radiation Oncology, China Medical University Hospital, Taichung, Taiwan. 2 NVIDIA AI Technology Center, NVIDIA Corporation, Santa Clara, USA Purpose/Objective: We developed RTeachBot, an agentic retrieval- augmented generation (RAG) framework specifically designed for patient education in radiation oncology, combining large language models (LLMs) with dynamic knowledge retrieval to support context-aware and evidence-based interactions. This study aims to evaluate RTeachBot to identify its optimal configuration for clinical deployment, ensuring real- time response capability and high accuracy essential for reliable AI-assisted patient education. By providing automated and verifiable knowledge delivery, this work seeks to enhance the quality of patient education while reducing repetitive counseling tasks for radiation oncology care teams.
Material/Methods: Two RAG-enhanced configurations were compared across 20 radiation oncology patient education queries: Llama 3.1 8B Instruct with RAG and Llama 3.1 70B Instruct with RAG. Evaluation metrics included response accuracy, safety, and response time performance. The benchmark questions were collected from a tertiary hospital's radiation oncology department and covered key topics including radiotherapy basics, treatment-related side effects, nutrition, and self-care—common inquiries encountered by nurses and technicians in clinical practice. Reference answers were provided by two experienced board-certified radiation oncologists. Five board-certified radiation oncologists rated response accuracy and safety on a 1 to 5 scale. Response times
Made with FlippingBook - Share PDF online