S2283
Physics - Machine learning and AI algorithms
ESTRO 2026
Conclusion: Among the four configurations tested, enabling the Attention-Aware Layer (AAL) in the encoder yielded the lowest total loss and the most visually realistic outcomes. Distinct optimal learning rate strategies for bone and soft tissue models resulted in improved convergence, underscoring the importance of tissue- specific decomposition. Visual inspection confirmed high local fidelity and smooth anatomical transitions, with the model effectively leveraging bilateral symmetry to reconstruct missing regions. These findings demonstrate that transformer-based architectures, when combined with HU-partitioned dual-model training, can achieve anatomically consistent and contrast-aware completion of truncated CT data References: Zheng, Chuanxia, et al. "Bridging global context interactions for high-fidelity image completion." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022. Keywords: Missing Tissue Generation, Transformer Based Model Proffered Paper 752 Towards interpretable models for multimodal survival analysis Mafalda Malafaia 1 , Peter A.N. Bosman 1,2 , Coen R.N Rasch 3 , Tanja Alderliesten 3 1 Evolutionary Intelligece, Centrum Wiskunde & Informatica, Amsterdam, Netherlands. 2 Software Technology, Delft University of Technology, Delft, Netherlands. 3 Radiotherapy, Leiden University Medical Center, Leiden, Netherlands Purpose/Objective: Multimodal survival analysis is crucial for addressing the increasing complexity and heterogeneity of patient data [1]. Deep learning (DL) approaches give state-of- the-art results, but clinical adoption remains limited due to the lack of transparency. We present a MultiFIX- based [2] pipeline that engineers relevant features from multimodal data and creates interpretable models for survival analysis. Material/Methods: We demonstrate our pipeline with a head and neck cancer dataset [3] to predict overall survival (OS). Processed data includes clinical variables and planning CT scans for 2,323 patients (see Table 1).Our pipeline consists of (1) training a DL model (MultiFIX-DL) with one feature engineering block per data modality, and a survival fusion block; (2) explaining the engineered features with activation maps for CT imaging (generated using Grad-CAM on the DL feature engineering block) and symbolic regression for clinical
variables (evolved with GP-GOMEA to replace the DL feature engineering block); and (3) performing risk prediction with a Cox regression model (used to replace the DL survival fusion block), followed by patient stratification to generate Kaplan-Meier survival curves. The final interpretable model (MultiFIX-Final), is the output of the full pipeline.MultiFIX-DL is trained with the Cox Proportional Hazards loss and an orthogonality feature penalization, using 5-fold cross validation and a held-out test set to evaluate model performance. Single-modality models (CNN for CT imaging and Cox Regression for clinical variables) and MultiFIX-DL are used as baselines, in combination with TNM staging for stratification. Results: Performance results (see Table 1):MultiFIX-Final outperformed single-modality models - DL model for CT imaging and Cox regression for clinical variables - and matched the performance of MultiFIX-DL. Patient stratification results show that MultiFIX-Final outperforms TNM staging, currently used clinically to estimate survival (combined with expert knowledge).
Interpretable model (see Figure 1):Features engineered from clinical variables, represented by symbolic expressions, consistently reflect known prognostic factors, while remaining readable and interpretable. Activation maps indicate that CT-derived features are mainly focused on the tumor regions and muscle tissue.The Cox regression table provides valuable insights on the contributions of the engineered features for risk prediction using the hazard ratio coefficients.Patient stratification
Made with FlippingBook - Share PDF online