S1334
Interdisciplinary - Education in radiation oncology
ESTRO 206
using semi-structured interviews with 16 participants: eight clinicians across specialties and eight AI experts with experience in model development and/or implementation. Interview questions covered prior AI exposure and information required before using an AI- CDSS. After, participants reviewed three reporting standards: the Model Card (Mitchell et al., 2020), TRIPOD-AI (Collins et al., 2024), and the Model Facts label (Sendak et al., 2020). Each is filled-out according to an existing colon-cancer prediction model. Interview recordings were transcribed verbatim and analyzed using codebook thematic analysis to identify perspectives offered by the participants. Results: Four themes were identified: (i) Clinicians require clear insight into the model training data, including data provenance, size, in- and exclusion criteria, and class imbalance, to judge model applicability compared to their patient; (ii) Performance information should include clinically meaningful metrics and thresholds, not only a global indicator such as AUC; (iii) Explicit communication of model limitations should be provided in the form of warnings about inappropriate use, which is necessary to prevent misuse; (iv) Information should be provided in a layered, customizable format and integrated within existing clinical software. Participants agreed that current reporting standards each provide value, but none are sufficient in their current form. A combined and clinician-centered reporting strategy aligned with workflow demands was viewed as necessary. Conclusion: Improving adoption of AI-CDSS requires reporting practices designed for clinical usability, emphasizing transparency, clarity and alignment with workflow realities. Communication about training data, performance, and limitations can enhance clinicians understanding. Co-creation with clinical end-users throughout the development and implementation of an AI model appears central for creating AI-CDSS that are both technically sound and workable in practice. Further research should investigate structured approaches to reporting validation and performance metrics and assess how information provision influences appropriate use of AI-CDSS. References: Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. Published online April 16, 2024:e078378. doi:10.1136/bmj-2023-078378 Mitchell M, Wu S, Zaldivar A, et al. Model cards for model reporting. In: Association for Computing Machinery, Inc; 2019:220-229. doi:10.1145/3287560.3287596 Sendak MP, Gao M, Brajer N, Balu S. Presenting machine learning model information to clinical end users with model facts
1. Felitti VJ, Anda RF, Nordenberg D, Williamson DF, Spitz AM, Edwards V, Koss MP, & Marks JS. Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults. Am J Prev Med. 1998: 14(4): 245-58. 2. Saripalli AL, Ross DH, Murphy E, Gomez K, Thilges S, & Harkenrider MM. Prevalence of trauma history and symptoms in patients who have received vaginal brachytherapy as part of their endometrial cancer treatment. Gynecologic Oncology. 2024: 185: 68-74. 3. Clark KR, & Sonsiadekk JS. Trauma-Informed Care in Medical Imaging and Radiation Therapy to Reduce Retraumatization. Radiologic Technology. 2023: 95(1): 26-35. 4. SAMHSA’s Concept of Trauma and Guidance for a Trauma-Informed Approach. 2014.
Keywords patient centred care
Digital Poster Highlight 2819 Understanding clinicians’ informational needs for AI-driven clinical decision support systems: Qualitative interview study Simone M.B. Mingels 1 , Hannah S Piehl 1 , Madeline S Therrien 1 , Ekaterina Akhmad 1 , Anniek R van Hienen 1 , Johan P.A. van Soest 1,2 , Laura Hochstenbach 3 , Andre L.A.J. Dekker 1 , Olga C Damman 4 , Rianne R.R. Fijten 1 1 Department of Radiation Oncology (Maastro), GROW Research Institute for Oncology and Reproduction, Maastricht University Medical Centre+, Maastricht, Netherlands. 2 Brightlands Institute for Smart Society (BISS), Faculty of Science and Engineering, Maastricht University, Heerlen, Netherlands. 3 Department of Health Services Research, Care and Public Health Research Institute (CAPHRI), Maastricht University, Maastricht, Netherlands. 4 Department of Public & Occupational Health and Amsterdam Public Health research institute, section Quality of Care, Amsterdam UMC location VUmc, Amsterdam, Netherlands Purpose/Objective: Artificial Intelligence-driven Clinical Decision Support Systems (AI-CDSS) offer opportunities for improved personalization of cancer care. Yet, clinical uptake remains limited. Clinicians continue to express concerns related to transparency, interpretability and the potential for inappropriate or unsafe use. This study aimed to (1) identify the informational needs and preferences of clinicians to support appropriate use of AI-CDSS, and (2) compare these with AI experts’ views on what information should be communicated to ensure safe deployment in clinical workflows. Material/Methods: We conducted a qualitative description design study
Made with FlippingBook - Share PDF online