FinMMEval Lab 2026

Motivation

Global finance increasingly demands multilingual reasoning across documents such as English financial filings and foreign-language news. However, most existing benchmarks are monolingual. This task introduces the first cross-lingual financial reasoning benchmark, testing a model's ability to integrate multilingual textual signals and perform analytical reasoning grounded in authentic financial data.

Difficulty Tiers

Data

Combination of the PolyFiQA-Easy and PolyFiQA-Expert datasets, which combine U.S. SEC filings with multilingual news (English, Chinese, Japanese, Spanish, Greek). Each tier includes 172 Q&A instances (344 total).

Data & Submission

Historical data for backtesting, validation, and model training: MBZUAI/finmmeval-lab-clef2026.

To participate, submit your endpoint via the Agent Market Arena page (Google Form): Agent Market Arena.

At present, we do not enforce a hard submission cap for Task 3. Teams may update their submitted endpoint as needed before the endpoint submission deadline, but should avoid unnecessary rapid resubmission.

The endpoint submission deadline is the deadline for submitting or updating your endpoint. It is not the end of the Task 3 evaluation period.

Submitting the Google Form registers or updates your endpoint, but it may not appear on the leaderboard immediately. Organizers first verify submitted endpoints and confirm the final endpoint list.

Daily Workflow

• Each endpoint receives one request per day and must return one action: BUY, HOLD, or SELL.
• Position mapping: BUY -> long, HOLD -> flat, SELL -> short.
• Execution rule: each new action fully replaces the previous day’s position.
• Price convention: daily close price.

Scheduling & Request Policy

• Daily process starts at 00:00 UTC.
• Requests are sent progressively after start time.
• Official Task 3 performance is computed over a common evaluation window for all accepted endpoints, rather than starting separately from each team’s individual form submission date.
• Submitted endpoints will continue to be called daily after the endpoint submission deadline for the official Task 3 evaluation window. We expect this window to run through late June or early July, aligned with the final lab reporting schedule.
• Teams do not need to keep endpoints online for the full day, but should start them shortly before 00:00 UTC and keep them available for several hours to allow for queued requests, retries, and temporary network delays.
• Per-request timeout is 3 minutes.
• If a request fails (timeout, server error, or invalid response), the action defaults to HOLD.

Request / Response Format

Input is a JSON object containing date, price, news, symbol, momentum, history_price, and optional 10k/10q (object or null). The symbol varies by asset (e.g., TSLA, BTC).

{
  "recommended_action": "BUY"
}

Valid actions: BUY, HOLD, SELL.

Input Samples

Sample A (TSLA):

Note: 10k and 10q can each be an object or null.

{
  "date": "2025-01-15",
  "price": {"TSLA": 250.50},
  "news": {"TSLA": ["Tesla announces new production milestone"]},
  "symbol": ["TSLA"],
  "momentum": {"TSLA": "bullish"},
  "10k": {"TSLA": ["[SEC 10-K Filing - 2025-01-15]\nSummary..."]},
  "10q": {"TSLA": ["[SEC 10-Q Filing - 2025-01-15]\nSummary..."]},
  "history_price": {"TSLA": [
    {"date": "2025-01-12", "price": 249.80},
    ...,
    {"date": "2025-01-13", "price": 250.50},
    {"date": "2025-01-14", "price": 250.30}
  ]}
}

Sample B (BTC):

{
  "date": "2025-01-15",
  "price": {"BTC": 67890.50},
  "news": {"BTC": ["Bitcoin ETF inflows remain strong"]},
  "symbol": ["BTC"],
  "momentum": {"BTC": "neutral"},
  "10k": null,
  "10q": null,
  "history_price": {"BTC": [
    {"date": "2025-01-12", "price": 67580.00},
    ...,
    {"date": "2025-01-13", "price": 67720.00},
    {"date": "2025-01-14", "price": 67810.00}
  ]}
}

cURL Sample

curl -X POST "<your_endpoint>" \
  -H "Content-Type: application/json" \
  -d '{
    "date": "2025-01-15",
    "price": {"TSLA": 250.50},
    "news": {"TSLA": ["Tesla announces new production milestone"]},
    "symbol": ["TSLA"],
    "momentum": {"TSLA": "bullish"},
    "10k": {"TSLA": ["[SEC 10-K Filing - 2025-01-15]\nSummary..."]},
    "10q": {"TSLA": ["[SEC 10-Q Filing - 2025-01-15]\nSummary..."]},
    "history_price": {"TSLA": [
      {"date": "2025-01-12", "price": 249.80},
      ...,
      {"date": "2025-01-13", "price": 250.50},
      {"date": "2025-01-14", "price": 250.30}
    ]}
  }'

Data & Submission

Historical data for backtesting, validation, and model training: MBZUAI/finmmeval-lab-clef2026.

To participate, submit your endpoint via the Agent Market Arena page (Google Form): https://huggingface.co/spaces/TheFinAI/Agent-Market-Arena.

Reference endpoint example (FastAPI): examples/simple_trading_api.py.