Adversarially Robust Cross-Modal Medical Image Retrieval via Self-Supervised Deep Hashing and Large Language Model Agents - Hristopher Gell; Rennis Karlsson; Enzo M. Koskinen (International Journal of Clinical and Translational Medicine, 2026)
Cross-modal medical image retrieval supports clinical decision-making by enabling semantically grounded queries across heterogeneous data sources, yet its deployment in real-world settings faces substantial challenges from adversarial perturbations, modality gaps, and governance demands. This paper advances a system-level framework that integrates self-supervised deep hashing with large language model (LLM) agents to achieve adversarially robust, semantically precise retrieval. The architecture couples a self-supervised hashing backbone that learns modality-agnostic binary codes from unpaired radiological reports and imaging studies with an LLM-based semantic mediator that reformulates queries, validates retrieved candidates, and injects domain constraints. A threat model encompassing modality-specific gradient-based perturbations, linguistic prompt injection, and distributional drift is formally characterized. Defense mechanisms are woven throughout the retrieval pipeline, including adversarial hashing via margin-scalable constraints, randomized smoothing for certified robustness, and prompt sanitization layers within the LLM agent. The discussion emphasizes structural trade-offs among retrieval latency, bit-width efficiency, and robustness guarantees. Governance implications are analyzed with regard to fairness across patient subpopulations, audit trails for retrieval decisions, energy sustainability of dual-stage architectures, and compliance with evolving regulatory frameworks for AI-enabled medical devices. By treating robustness and governance not as afterthoughts but as design constraints embedded within the self-supervised training loop and the agent orchestration, the framework aims to bridge the gap between laboratory validation and trustworthy clinical deployment. This systems-oriented synthesis highlights open challenges including continual adaptation under domain shift, scalable federated hash learning, and the need for standardized benchmark protocols that jointly evaluate retrieval accuracy, adversarial resilience, and fairness metrics in cross-modal medical search.