S1328
Interdisciplinary - Education in radiation oncology
ESTRO 206
Proffered Paper 1657
Analysis of Large Language Models Clinical Workflow Management, Cost, and Productivity Performance Through Stepwise Uro-Oncology Scenarios Mehmet Halici 1 , Muhammed Emin Polat 2 , Burak Ertan 3 , Bahadır Koylu 4 , Yusuf Kasap 2 , Erkan Olcucuoglu 2 , Fuzuli Tugrul 5 , Alaattin Ozen 1 , Ibrahim Cem Balci 6 , Serkan Salturk 7 , Huseyin Uvet 6 1 Radiation Oncology, Ba ş ak ş ehir Çam and Sakura City Hospital, Istanbul, Turkey. 2 Urology, Ankara Bilkent City Hospital, Ankara, Turkey. 3 Computer Engineering, Yıldız Technical University, Istanbul, Turkey. 4 Medical Oncology, Koç University, Faculty of Medicine, Istanbul, Turkey. 5 Radiation Oncology, Acıbadem Hospital, Eskisehir, Turkey. 6 Mechatronics Engineering, Yıldız Technical University, Istanbul, Turkey. 7 Electronics and Communication Engineering, Yıldız Technical University, Istanbul, Turkey Purpose/Objective: This study compared the time-based cost-efficiency and generative efficiency of three large language models (LLMs)—ChatGPT-5, Gemini 2.5, and Claude Opus 4.1—using comprehensive stepwise uro- oncologic scenarios, and evaluated the effect of optimized prompting on economic performance. Material/Methods: Ten clinically realistic stepwise scenarios (five prostate and five bladder cancer cases) were developed by a radiation oncologist, a urologist, and a medical oncologist through expert consensus. Each scenario comprised three stages—diagnosis, treatment, and follow-up—with two open-ended questions per stage. One pilot case tested two prompting strategies: standard stepwise and Sequential Waterfall Prompting (SWP), where preceding steps were cumulatively appended. All scenarios were presented to each model through their APIs using the selected method. For each response, input, reasoning, and output tokens and response times were recorded. Total Cost (USD) was calculated per question using official token pricing. Models were compared for Total Cost, Response Time (s), Generative Efficiency (Output/Input), and Economic Efficiency (Cost/Time). Non-parametric Kruskal–Wallis and one-way ANOVA tests were applied in SPSS v20, comparing both models and scenario steps. Results: Using SWP, a 17-fold cost reduction was achieved compared with standard stepwise prompting. The average cost per scenario was 0.98 USD, and the average total token usage was 35,124 tokens (input and output combined). Significant differences were found among models for cost, response time, and generative efficiency (p < 0.001). Gemini 2.5 achieved
Conclusion: Large language models demonstrated comparable clinical accuracy but differed substantially in cost- efficiency, reasoning time, and interpretability.Gemini 2.5 was the most time-efficient overall, while ChatGPT- 5 achieved the highest interpretability and clinically applicable responses—particularly in the therapeutic stage—offering a lower token-based cost relative to its output length.These results highlight the balance between economic efficiency and clinical reliability, emphasizing LLMs’ potential role as emerging tools in healthcare systems (1,3).Although the number of vignettes was limited, each represented the full diagnostic-to-follow-up continuum, supporting the framework’s generalizability and real-world 1. Rahsepar AA, Tavakoli N, Kim GHJ, Hassani C, Abtin F, Bedayat A. How AI Responds to Common Lung Cancer Questions: ChatGPT vs Google Bard. Radiology. 2023;307(5):e230922. doi:10.1148/radiol.2309222. Klang E, Apakama D, Abbott EE, et al. A strategy for cost-effective large language model use at health system-scale. NPJ Digit Med. 2024;7(1):320. Published applicability. References:
2024 Nov 18. doi:10.1038/s41746-024-01315- 13. Huang J, Yang DM, Rong R, et al. A critical
assessment of using ChatGPT for extracting structured data from clinical notes. NPJ Digit Med. 2024;7(1):106. Published 2024 May 1. doi:10.1038/s41746-024-01079- 8 Keywords: Large language models, clinical decision support
Made with FlippingBook - Share PDF online