NHR@FAU · Erlangen National High Performance Computing Center
Pattern Recognition Lab, FAU Erlangen-Nürnberg

ADARSH BHANDARY PANAMBUR

Research Scientist · Medical AI · Multimodal AI · AI Systems

I work across representation learning, multimodal and foundation models, LLM and VLM applications, and scalable GPU inference — and on getting those methods close enough to clinical imaging data to be tested properly.

Portrait of Adarsh Bhandary Panambur

Erlangen, Germany

SELF-SUPERVISED LEARNING

Representation learning, DINO-style pretraining, transfer learning, foundation-model research.

VISION-LANGUAGE MODELS

Medical image-text learning, CLIP-style models, multimodal reasoning, report generation.

LARGE LANGUAGE MODELS

Local LLMs, foundation-model APIs, RAG, LLM evaluation, applied GenAI systems.

MULTIMODAL MEDICAL AI

Mammography, breast MRI, detection, localization, explainability, clinical validation.

AI SYSTEMS + HPC

GPU inference, model serving, distributed and HPC workflows, research software.

01 / AI SYSTEMS

AI SYSTEMS / APPLIED GENAI

I build the layer between a model and something people can actually use: local LLM and VLM runtimes, embeddings and retrieval, RAG, agent workflows, foundation-model APIs, and GPU inference on HPC infrastructure.

Architecture and capabilities only — project and partner details stay confidential.

  1. 01 LOCAL MODELS Self-hosted LLM and VLM runtimes
  2. 02 EMBEDDINGS / RETRIEVAL Vector search over domain corpora
  3. 03 RAG Grounded, citable generation
  4. 04 LLMs / VLMs Open-weight and API-backed models
  5. 05 AGENTS / WORKFLOWS Tool use, multi-step orchestration
  6. 06 MODEL APIs Foundation-model services
  7. 07 GPU INFERENCE Batching, serving, throughput
  8. 08 APPLICATIONS Research software people use

MODELING

PyTorchHugging FaceTransformersPEFT / LoRA

GENAI

Local LLM deploymentRAGVector retrievalAgentic workflowsLLM evaluation

SERVING

GPU inferenceModel servingSGLangQuantized runtimes

INFRASTRUCTURE

SLURMHPC clustersDistributed trainingContainersReproducible environments

WHAT I AM WORKING ON NOW

System

Local model runtimes

Deploying and evaluating LLM and VLM runtimes for privacy-aware, GPU-backed research applications.

Applied AI

Retrieval and grounded generation

Prototyping retrieval pipelines that connect domain knowledge with foundation-model APIs and locally deployed models.

Application

Agent workflows and research software

Building tool-using agent workflows, model-backed interfaces, and reproducible research software around open-source AI frameworks.

Infrastructure

Scalable inference infrastructure

Designing training, serving, and inference workflows for GPU clusters and supercomputing environments.

02 / THE THREAD

FROM DATA TO INTELLIGENCE

One line runs through everything I have worked on: keep the information that matters in a difficult image, learn a representation from it, connect that representation to language and reasoning, then make it run and hold up on someone else's data.

  1. 01

    DATA

    High-resolution mammography, breast MRI, CT and cell imaging — curated and standardized.

  2. 02

    REPRESENTATION LEARNING

    Self-supervised pretraining on the target cohort rather than on ImageNet alone.

  3. 03

    VISION / LANGUAGE

    Aligning image regions with radiology language so models can be described, not just scored.

  4. 04

    REASONING

    Structured findings, BI-RADS categories, generated reports and grounded evidence.

  5. 05

    AI SYSTEMS

    Distributed training, GPU inference and serving that make the above reproducible.

  6. 06

    CLINICAL APPLICATION

    Retrospective evaluation with radiologists on real, external cohorts.

03 / FEATURED RESEARCH

FOUR PIECES OF WORK

Learning representations without labels, generating language from images, supervising a 3D model from one annotated slice, and letting a dataset shape its own training objective.

Three-panel diagram: DINO self-supervised pretraining with a student and teacher ViT-S, conversion of six attention heads into binary masks and erased images, and transfer learning with attention-head selection by validation F1

01 / SELF-SUPERVISED REPRESENTATION LEARNING

Attention-guided erasing for enhanced transfer learning in breast abnormality classification

Use a self-supervised model's own attention to choose which pixels an augmentation is allowed to destroy — then check the four tasks where that helps and the one where it does not.

4 / 5
Significant gains
05
Classification tasks
1 min read PaperDOI
MammoBLIP pipeline: five public dataset sources feed a standardized preprocessing step; a mammogram goes through a frozen EVA-CLIP image encoder and the paired report through a tokenizer, both into the MammoBLIP transformer, whose projected embeddings drive a frozen BioMedLM to produce a generated radiology report

02 / VISION-LANGUAGE MODELS

MammoBLIP: End-to-End Mammography Report Generation with Vision-Language Models and Public Multi-Institutional Datasets

Freeze a vision foundation model, freeze a biomedical LLM, train only the small bridge between them — and see how close the generated report gets to the reference.

65.36
BLEU overall
81,076
Mammograms curated
1 min read DOI
DITL framework diagram: whole-image and ROI mammography inputs feed a three-stage circle of self-supervised pretraining without labels, dataset property extraction with labels, and dataset-informed transfer learning; a feature-space scatter and a nearest-neighbour analysis panel show difficulty level, k-nearest and k-farthest features

04 / ADAPTIVE TRAINING OBJECTIVES

Re-thinking Mammography Transfer Learning: The Dataset-Informed Transfer Learning (DITL) Framework for Breast Cancer Screening and Lesion Diagnosis

Take the three quantities I previously hand-tuned — sample difficulty, triplet margin, neighbour count — and derive all of them from the data's own self-supervised feature space instead.

0.609
VinDr density macro F1
0
Loss hyperparameters to tune
1 min read PaperarXiv
Full publication archive

04 / TRAJECTORY

HOW THE WORK EVOLVED

Each step here was driven by a limitation in the previous one. The order matters more than the count of papers.

  1. 2019 — 2022

    CLASSICAL / DEEP LEARNING

    CNN classifiers for calcification patches and unstained blood cells, with augmentation studies rather than architecture chasing.

    BVM 2022 · IJLH 2023

  2. 2022 — 2023

    TRANSFER LEARNING

    Reusing abnormality classifiers as initialization for harder targets, including luminal subtype prediction from whole mammograms.

    arXiv:2301.09282

  3. 2023 — 2025

    SELF-SUPERVISED LEARNING

    DINO pretraining on the target cohort, then using its attention as a task-aware augmentation policy instead of a segmentation label.

    BVM 2024 · IJCARS 2025

  4. 2024 — 2025

    MULTIMODAL AI

    Contrast-enhanced mammography, neighbourhood-informed losses, and multi-view multi-modal detection that mirrors how radiologists read.

    SPIE MI 2024 · Deep-Breath MICCAI 2025

  5. 2025

    VISION-LANGUAGE MODELS

    End-to-end mammography report generation from a frozen vision encoder and a frozen biomedical LLM joined by a small trained bridge.

    IEEE NSS/MIC/RTSD 2025

  6. 2025 — 2026

    FOUNDATION MODELS

    Objectives that adapt to a dataset's own structure, and lesion-aware contrastive pretraining across multiple public cohorts.

    arXiv:2607.26043 · ECR 2026

  7. 2026

    LLMs / RAG / AGENTIC AI

    Domain-adapted SigLIP encoders with 4B-parameter LLMs for grounded report generation; retrieval, agents and local models in applied work.

    NHR@FAU · manuscript in preparation

  8. 2026 — present

    AI SYSTEMS / HPC

    GPU inference, model serving, and reproducible distributed workflows so the research above runs at cluster scale.

    NHR@FAU

05 / SKILLS

WHAT I WORK WITH

Grouped by what they are for, not sorted by how impressive they sound.

REPRESENTATION & VISION

Self-Supervised LearningDINO / Vision TransformersTransfer LearningDetectionLocalizationExplainability

MULTIMODAL & FOUNDATION MODELS

Vision-Language ModelsCLIP / SigLIPImage-Text LearningMedical VLMsReport GenerationFoundation Models

GENERATIVE AI

LLMsRAGEmbeddingsAgentsFoundation-Model APIsLocal LLM Deployment

AI SYSTEMS

PyTorchHugging FaceGPU InferenceSLURMHPCDistributed TrainingModel ServingContainers

MEDICAL AI

MammographyBreast MRIClinical ValidationMedical Image AnalysisMultimodal Clinical Data

ENGINEERING & DATA

PythonC++NumPy / pandasscikit-learnSimpleITKMONAIDockerGit

06 / EXPERIENCE

WHERE THE WORK HAPPENED

Medical devices, then clinical AI inside a manufacturer, then a hospital cohort, then a supercomputing centre. Each move added a constraint the previous one did not have.

Full CV
  1. AI systems + applied GenAI

    June 2026 — Present

    Erlangen, Germany

    Research Scientist

    NHR@FAU · Erlangen National High Performance Computing Center

    • Develop AI solutions built on locally deployed LLMs and VLMs, foundation-model APIs, embeddings and retrieval.
    • Build tool-using agent workflows and research software on open-source AI frameworks.
    • Engineer training, serving and inference workflows for GPU clusters and supercomputing infrastructure.
    2 more on the CV →
  2. Medical AI research

    May 2020 — Present

    Erlangen, Germany

    PhD Researcher

    Pattern Recognition Lab, FAU Erlangen-Nürnberg

    • Developed self-supervised and transfer-learning methods for mammography classification, detection and localization.
    • Designed vision-language models for mammography report generation over a curated 81,076-image multi-institutional public dataset.
    • Built weakly supervised breast MRI models that localize lesion evidence from a single annotated slice per examination.
    3 more on the CV →
  3. Clinical validation

    July 2024 — September 2024

    Chengdu, China

    Guest Researcher

    West China Hospital, Sichuan University

    • Evaluated a Siemens Healthineers breast MRI AI research prototype on a retrospective external cohort.
    • Worked with radiologists on lesion annotation quality and reading-workflow questions.
    • Reported the study as an ECR 2025 scientific exhibit (DOI 10.26044/ecr2025/C-21110).
  4. Mammography AI

    May 2020 — February 2024

    Erlangen, Germany

    Doctoral Researcher

    Siemens Healthineers

    • Contributed to retrospective evaluation of AI-assisted mammography reading in a Vietnamese screening cohort (ECR 2024, ECR 2025).
    • Developed deep-learning methods for categorizing mammography positioning artifacts.
    • Worked on imaging AI within regulatory and clinical-workflow constraints.
  5. Deep learning

    October 2017 — March 2020

    Erlangen, Germany

    Working Student and Master's Thesis

    Siemens Healthineers

    • Built MATLAB prototypes for compliance-based task prioritization in clinical workflows.
    • Prepared software specifications and use-case survey reports for blockchain applications in healthcare.
    • Evaluated IoT technologies for healthcare workflow optimization.
    2 more on the CV →
  6. Medical devices

    March 2015 — February 2016

    Bangalore, India

    Senior Engineer (Medical Devices)

    Transcaal Engineers India Pvt. Ltd.

    • Delivered medical-device QA services across five Indian states.
    • Led and mentored a biomedical engineering team.
  7. Medical devices

    September 2014 — February 2015

    Bangalore, India

    Calibration Engineer

    Transcaal Engineers India Pvt. Ltd.

    • Performed preventive maintenance and compliance testing of medical devices using Fluke biomedical equipment.

EDUCATION

PhD Research Student - AI / Medical Imaging

Friedrich-Alexander-Universität Erlangen-Nürnberg

May 2020 - Present

M.Sc. Medical Data Engineering

Friedrich-Alexander-Universität Erlangen-Nürnberg

April 2016 - March 2020

B.E. Medical Electronics

Dr. Ambedkar Institute of Technology, Bangalore

August 2010 - June 2014

07 / SELECTED PAPERS

PUBLICATIONS

23 entries across mammography, breast MRI, representation learning and vision-language models — each with its DOI or preprint link.

Full archive
  1. 01

    Bildverarbeitung für die Medizin (BVM) 2026, 10–17 · Published

    BE-WISE: Breast MRI Evaluation with Weakly-Informed Slice-Level Explanation

    Explainable AIMedical ImagingDetection / Localization

    0.8683Test AUC

  2. 02

    Manuscript in preparation — no venue yet · 2026 · not peer reviewed

    Explainable Vision-Language Models for Mammography Report Generation

    Vision-Language ModelsLLM / GenAIExplainable AI

    0.877ROUGE-1

  3. 03

    arXiv preprint (accepted at the International Conference on Computer Vision and Image Processing, CVIP 2025 — in print) · 2026 · CVIP 2025 in print · not peer reviewed

    Re-thinking Mammography Transfer Learning: The Dataset-Informed Transfer Learning (DITL) Framework for Breast Cancer Screening and Lesion Diagnosis

    Foundation ModelsSelf-Supervised LearningMedical Imaging

    0.609VinDr density macro F1

  4. 04

    International Journal of Computer Assisted Radiology and Surgery, 20(3), 433–440 · 2025 · Published · open access (CC BY 4.0)

    Attention-guided erasing for enhanced transfer learning in breast abnormality classification

    Self-Supervised LearningMedical ImagingExplainable AI

    4 / 5Significant gains

  5. 05

    2025 IEEE Nuclear Science Symposium, Medical Imaging Conference and Room Temperature Semiconductor Detector Conference (NSS/MIC/RTSD), Yokohama · Published

    MammoBLIP: End-to-End Mammography Report Generation with Vision-Language Models and Public Multi-Institutional Datasets

    Vision-Language ModelsLLM / GenAIMedical Imaging

    65.36BLEU overall

09 / CONTACT

GET IN TOUCH.

I am interested in research collaborations and roles across multimodal AI, vision-language models, applied GenAI systems and medical imaging. Email is the fastest way to reach me.