← All publications

conference · 2026 · Accepted

Lesion-Aware AI for Mammography: Multi-Dataset Pretraining with ROI-Guided Contrastive Learning and Clinical Image Retrieval

Adarsh Bhandary Panambur · Tri-Thien Nguyen · Siming Bayer · Andreas Maier

DOI ↗
01 / SUMMARY

What this paper does

Global contrastive pretraining lets a mammography model align on breast density or view rather than on the lesion, because those cues are enough to match an image to its report. This work adds region-of-interest guidance to multi-dataset contrastive pretraining, so localized pathology has to contribute to the representation, and then tests the embeddings on clinical image retrieval — finding visually and diagnostically similar prior cases. Accepted as a scientific exhibit at ECR 2026; quantitative results are not yet publicly available.

02 / METHOD

How

Region-of-interest guidance is incorporated into a contrastive image-text pretraining objective across multiple mammography datasets, and the resulting embeddings are evaluated for clinical image retrieval.

03 / FINDINGS

Reported results

  • Accepted as an ECR 2026 scientific exhibit (DOI 10.26044/ecr2026/C-15780). Quantitative results are not yet publicly available.
04 / WHY IT MATTERS

Relevance

Retrieval is a use case radiologists already have — find me a similar prior case — and it is a harder test of a representation than a single classification head.

05 / LIMITATION

What it does not show

Conference exhibit; no peer-reviewed full paper and no public numbers yet.

06 / REFERENCE

Citation

A. Bhandary Panambur, T.-T. Nguyen, S. Bayer, A. Maier. Lesion-Aware AI for Mammography: Multi-Dataset Pretraining with ROI-Guided Contrastive Learning and Clinical Image Retrieval. ECR 2026, C-15780. doi:10.26044/ecr2026/C-15780

View original abstract

Contrastive image-text pretraining for mammography usually aligns a whole image with a whole report, which lets a model succeed on global cues while ignoring the lesion. This work introduces region-of-interest guidance into multi-dataset contrastive pretraining so that localized pathology contributes directly to the learned representation, and evaluates the resulting embeddings for clinical image retrieval.

Related work

Connected publications

BE-WISE architecture: pre-contrast, post-contrast and subtraction MRI volumes stacked into a (32, 3, 224, 224) tensor, encoded by a shared Swin-Tiny slice encoder into (32, 768) embeddings, then split into a slice-level lesion predictor producing a per-slice probability curve and a breast-level classifier producing normal, benign or malignant

Bildverarbeitung für die Medizin (BVM) 2026, 10–17

BE-WISE: Breast MRI Evaluation with Weakly-Informed Slice-Level Explanation

Published

Breast MRI models usually need lesions outlined voxel by voxel, which does not scale. BE-WISE asks how far one annotated slice per exam gets you. Pre-contrast, post-contrast and subtraction volumes are stacked and encoded slice by slice with a shared Swin-Tiny transformer. One branch predicts a per-slice lesion probability against a Gaussian target centred on the radiologist's slice; the other pools the same embeddings by attention into a normal / benign / malignant decision. On the multicenter ODELIA dataset this reached test AUC 0.8683 and an ODELIA score of 0.7098, against 0.7968 and 0.5695 for a medical slice transformer baseline.

Explainable AIMedical ImagingDetection / Localization
1 min read → Paper ↗
BE-WISER architecture: pre-contrast, post-contrast and subtraction inputs go through a SwinV2-Tiny slice encoder into slice embeddings, which split into a breast-level classifier module with ABMIL attention, feature aggregation, global head and global logits, and a weak slice-level lesion localization branch with a slice head, slice logits and per-slice class probabilities over normal, benign and malignant

Research Square preprint (in review)

BE-WISER: Class-Aware Weak Slice-Level Supervision for Breast MRI

Preprint · in review

BE-WISE could say which slice contained something; BE-WISER says what that something is. Each slice inside the Gaussian region around the radiologist's marked slice inherits the exam's class — benign or malignant — while the rest are labelled normal, so the slice head becomes three-class instead of binary. On the latest multicenter ODELIA release (926 / 230 / 292 examinations from six institutions) this reached breast-level AUC 0.864 and an ODELIA score of 0.699, against 0.807 and 0.591 for a breast-level baseline. Localization also improved: Hit@1 rose from 0.554 to 0.632 and the mean slice distance fell from 2.838 to 2.505 slices.

Explainable AIDetection / LocalizationMedical Imaging
1 min read → Paper ↗
Three-stage architecture: an input mammogram through a SigLIP vision encoder and linear projection into visual tokens, a text instruction through a tokenizer into text token embeddings, both combined into a Gemma 3 4B model with 34 transformer layers generating a report autoregressively, then attention weights extracted per token, aggregated across heads and layers and bicubically interpolated to produce one heatmap per generated token

Manuscript in preparation — no venue yet

Explainable Vision-Language Models for Mammography Report Generation

Manuscript in preparation · not peer reviewed

This is the follow-up to MammoBLIP, and it asks a different question: not how good the report is, but whether you can see where each word came from. Domain-adapted SigLIP vision encoders are paired with 4B-parameter LLMs (Gemma 3 and MedGemma), and attention is aggregated across heads and layers to produce one heatmap per generated token. The best of six configurations reached ROUGE-1 0.877 and ROUGE-2 0.843, up 0.127 and 0.143 on MammoBLIP. The more interesting finding is negative: attention often lands on the right microcalcification cluster while the diagnosis is still wrong.

Vision-Language ModelsLLM / GenAIExplainable AI
1 min read →