S1585
Physics - Autosegmentation
ESTRO 2026
A literature search was performed in PubMed between January 1st 2021 and June 30th 2025 for primary research studies that undertook clinician-based qualitative auto-contour assessment. All papers published were reviewed by one reviewer and a second reviewer reviewed 10% of these to confirm accuracy. Data extracted included the type and design of each assessment, the definitions for specific ratings, the number of raters and approach to assessment (e.g. blinded), as well as an explicit check on inter-rater agreement where applicable. Results: The search generated 1045 papers, with 132 meeting the criteria for inclusion. 126 different scales were in use, with only 3 scales being repeated verbatim in other papers. Only 13 papers gave a formal reference for the source of the scale and only 2 others described their rationale for their scale. Table 1 presents nine distinct clinician-based assessment criteria with examples. The most common theme was degree of editing, in 100 papers. These papers frequently gave no, or contrasting definitions for major and minor edits and the number of scoring options varied substantially. Scales used had between 2 and 100 points on them, with 4-point scales being the most popular (Table 2).29 papers reported appraisal by a single reviewer and 9 did not specify their number of reviewers. Blinding to the source of the contour or original manual contours was not explicitly mentioned in 73 papers. Of 94 papers using multiple raters, only 28 displayed scores from individual raters. A formal statistical consistency check (e.g. kappa) was only performed 15 times.
Conclusion: Our new delineation method shifts the paradigm of manual target delineation to parametric delineation. We validated the method across contouring practices of different physicians within our institution. A model- based approach to CTV delineation is promising for improving the consistency and efficiency of target delineation in clinical practice. Keywords: Glioma, CTV, Modeling
Poster Discussion 3548
A systematic review of the clinical assessment methods used to evaluate auto-contouring systems for radiotherapy; a call for standardised practice Katherine Mackay 1,2 , Jessica Little 1 , David Bernstein 3,4 , Alexandra Taylor 2,4 , Ajay Aggarwal 5,6 1 Department of Clinical Oncology, Imperial College Healthcare NHS Trust, London, United Kingdom. 2 Department of Radiotherapy, The Royal Marsden Hospital NHS Foundation Trust, London, United Kingdom. 3 Department of Physics, The Royal Marsden Hospital NHS Foundation Trust, London, United Kingdom. 4 Radiotherapy and Imaging, The Institute of Cancer Research, London, United Kingdom. 5 Department of Oncology, Guy's and St Thomas' NHS Foundation Trust, London, United Kingdom. 6 Faculty of Public Health and Policy, London School of Hygiene and Tropical Medicine, London, United Kingdom Purpose/Objective: It is recommended that both quantitative and qualitative clinician-based assessments should be performed to evaluate the safety and quality of radiotherapy auto-contouring systems1. For clinician- based assessment, guidelines focus on the importance of using multiple raters and considering inter-rater reliability, to reduce bias and subjectivity1,2 but no single recommended qualitative framework exists. As large-scale, prospective trials of AI auto-contouring, (e.g. ARCHERY trial3) progress, there is a necessity to define robust and consistent clinician assessments. The purpose of this systematic review is to identify current methods in the literature to assess auto- contouring. Material/Methods:
Made with FlippingBook - Share PDF online