ImageCLEF 2026
MultimodalReasoning

Building on the success of previous iterations, the 2026 edition shifts focus toward a updated high-quality dataset from 2025 academic questions and fully transparent open-source development.

Evaluate the next generation of frontier multimodal models on challenging vision-based exam problems.

High-Quality Datasets

Updated reasoning-intensive data from 2025 academic questions, focusing on diagrams, charts, and technical schematics.

Open-Source Solutions

Promoting transparency with fully open-source architectures and reproducible systems.

Latest News

Jan 20, 2026

Competition Information Release

Official announcement and website launch for the 2026 edition.

Mar 07, 2026

OpenQA Train & Development Dataset Released

April 17, 2026

OpenQA Train & Development Dataset Released

Important Dates

26 Jan 2026

Registration opens for all ImageCLEF tasks

07 Mar 2026

(OpenQA) Train & Development dataset release

17 Apr 2026

(OpenQA+MCQ) Test dataset release

23 Apr 2026

Registration closes for all ImageCLEF tasks

10 May 2026

Deadline for submitting participant runs

11 May 2026

Late registration directly to AI4Media platform

28th May 2026

Submission of Participant Papers [CEUR-WS]

30th June

Notification of Acceptance for Participant Papers [CEUR-WS]

06 July

Camera Ready Copy of Participant Papers [CEUR-WS]

All dates follow 23:59 Anywhere on Earth (AoE).

Competition Tasks

The competition tasks are designed to evaluate the performance of multimodal models on challenging vision-based exam problems.

1

Multiple-Choice Question Answering (MCQ)

Given an image of a question with three to five possible answer options, the solution must select the single correct answer.

Classification
2

Open Question Answering (OpenQA)

Given an image of a question without predefined answer options, the solution must generate a free-form textual answer.

Generative

Data & Resources

Competition Guidelines

VM Environment

GPU
A40 (40GB)
Command
bash inference.sh

Submissions must be reproducible and run within these constraints for the private test set evaluation.

Categories

To promote and encourage efficient solutions, the competition has two categories:

  • Tiny Models ≤ 7B parameters.
  • Normal Models ≥ 8B parameters.

Rules & Constraints

  • Open-weights/source models only.
  • Pre-trained VLM models allowed.
  • External data allowed (must be disclosed).
  • Proprietary models (e.g. GPT-4 API) NOT allowed.

Evaluation & Submission

Leaderboard Phase

Submit JSON predictions only.

Max 20 submissions/day, 200 total.

Final Evaluation

  • Public GitHub repository
  • Model weights
  • Training dataset (optional)
  • Execution command

Competition results

Visual MCQ — leaderboard

Overall accuracy and per-language breakdown

# Participant Overall EN BG ZH HR IT SR

Textual MCQ — leaderboard

Overall accuracy and per-language breakdown

# Participant Overall EN BG ZH HR IT SR

Visual OpenQA — leaderboard

Generation quality metrics across languages · ranked by COMET overall

# Team COMET BG ZH HR EN IT SR BLEU ROUGE-L METEOR
COMET per language Additional metrics

Textual OpenQA — leaderboard

Generation quality metrics across languages · ranked by COMET overall

# Team COMET BG ZH HR EN IT SR BLEU ROUGE-L METEOR
COMET per language Additional metrics

Recommended Citations

Papers MUST include references to the ImageCLEF 2026 overview paper and the ImageCLEF 2026 Multimodal Reasoning overview paper.

ImageCLEF 2026 Lab Overview

LNCS
@inproceedings{ImageCLEF2026,
title = {Overview of ImageCLEF 2026: Multimodal Challenges in Medicine, Science, Agritech, and Security},
author = {Bogdan Ionescu and Henning M{\"u}ller and Dan{-}Cristian Stanciu and Andrei Radu and Radu{-}George Bolborici and Marian Negru and Alexandru{-}Florin Ene and Vlad{-}Mihai Vasilescu and Ana-Antonia Nicolae and Liviu{-}Daniel \c{S}tefan and Mihai{-}Gabriel Constantin and Mihai Dogariu and Alexandra{-}Georgiana Andrei and Hendrik Damm and Tabea M. G. Pakull and Asma {Ben Abacha} and Alba {Garc\'ia Seco de Herrera} and Christoph M. Friedrich and Raphael Br{\"u}ngel and Lea Reinartz and Henning Sch{\"a}fer and Cynthia Sabrina Schmidt and Benjamin Bracke and Praveen Nath and Bahad{\i}r Ery{\i}lmaz and Maja Hjuler and Diandra Fabre and Claire Lemaire and Benjamin Lecouteux and Didier Schwab and Dimitar Dimitrov and Ming Shan Hee and Momina Ahsan and Sarfraz Ahmad and Dimitrina Zlatkova and Georgi Pachov and Zhuohan Xie and Preslav Nakov and Ivan Koychev and Juampablo E. {Heras Rivera} and Daniel K. Low and Wen{-}wai Yim and Jacob Ruzevick and Dan Child and Mehmet Kurt and Zhaoyi Sun and Fei Xia and Meliha Yetisgen and Ahmedkhan Radzhabov and Yuri Prokopchuk and Vassili Kovalev and Dzmitry Karpenka and Steven A. Hicks and Sushant Gautam and Michael A. Riegler and Vajira Thambawita and P\r{a}l Halvorsen and Mohammad {El Sakka} and Josiane Mothe and Alexandra B\u{a}icoianu and Corneliu{-}Nicolae Florea and Mihai Ivanovici},
booktitle = {Experimental IR Meets Multilinguality, Multimodality, and Interaction},
series = {Proceedings of the Seventeenth International Conference of the CLEF Association (CLEF 2026)},
year = {2026},
month = {September 21--24},
address = {Jena, Germany},
publisher = {Springer Lecture Notes in Computer Science LNCS},
}

ImageCLEF Multimodal Reasoning Task Overview

CEUR-WS
@inproceedings{ImageCLEFMultimodalReasoningTaskOverview2026,
  title = {{O}verview of the {I}mage{CLEF} 2026 {T}ask on {M}ultimodal {R}easoning},
  author = {Dimitrov, Dimitar and Hee, Ming Shan and Ahsan, Momina and Ahmad, Sarfraz and Zlatkova, Dimitrina and Pachov, Georgi and Xie, Zhuohan and Nakov, Preslav and Koychev, Ivan},
  booktitle = {CLEF 2026 Working Notes},
  series = {CEUR Workshop Proceedings},
  year = {2026},
  month = {September 21--24},
  address = {Jena, Germany},
  publisher = {CEUR-WS.org},
}

Organizers

Dimitar Dimitrov

Dimitar Dimitrov*

Sofia University "St. Kliment Ohridski", Bulgaria

Momina Ahsan

Momina Ahsan*

MBZUAI, UAE

Ming Shan Hee

Ming Shan Hee*

MBZUAI, UAE

Zhuohan Xie

Zhuohan Xie

MBZUAI, UAE

Sarfraz Ahmad

Sarfraz Ahmad

MBZUAI, UAE

Dimitrina Zlatkova

Dimitrina Zlatkova

Sofia University "St. Kliment Ohridski", Bulgaria

Georgi Pachov

Georgi Pachov

Sofia University "St. Kliment Ohridski", Bulgaria

Ivan Koychev

Ivan Koychev

Sofia University "St. Kliment Ohridski", Bulgaria

Preslav Nakov

Preslav Nakov

MBZUAI, UAE

* Competition Co-Leads