Advancing Explainable AI for Clinical Decision Support: A Multimodal Evaluation Framework

Abstract

The increasing integration of Artificial Intelligence (AI) into Clinical Decision Support Systems (CDSS) is constrained by the limited transparency of model predictions, which undermines clinician trust and slows adoption in safety-critical settings. To address this barrier, we propose a unified multimodal evaluation framework for eXplainable AI (XAI) and empirically assess the behavior of the explanations in four clinically relevant modalities: chest radiography for pathology detection, Electroencephalography (EEG)-based epilepsy decision support using High-Frequency Oscillation (HFO) evidence, multimodal emotion recognition for psychological decision support and prediction of Alzheimer's disease based on Electronic Health Records (EHR). The framework evaluates widely used explanation mechanisms, including Gradient-weighted Class Activation Mapping (Grad-CAM), Integrated Gradients (IG), SHapley Additive exPlanations (SHAP)-style feature attribution, and attention-based interpretation, using modality-appropriate criteria that emphasize reliability, robustness, and clinical plausibility rather than visualization quality alone. The results show that Grad-CAM provides stable region-level localization in chest X-ray prediction. In contrast, EEG-based epilepsy decision support requires interpretability grounded in domain-specific biomarkers and time-frequency structure rather than generic saliency. In multimodal emotion recognition, fusion improves performance, but the contribution of each modality varies by emotional state, highlighting the need for interpretable fusion analyses. For Alzheimer's prediction, a tuned CatBoost model achieves strong discrimination, and feature-level analyses identify clinically significant drivers, including Mini-Mental State Examination (MMSE)-related measures. Cross-modal synthesis demonstrates that explanation effectiveness is inherently task- and modality-dependent and that explanation instability and susceptibility to spurious cues remain recurring risks across settings, consistent with concerns about the faithfulness of saliency explanations. In general, the proposed framework supports the practical implementation of XAI in healthcare by providing modality-aligned guidance for selecting and validating explanations within clinical workflows.