FinMMEval Lab 2026

Motivation

Global finance increasingly demands multilingual reasoning across documents such as English financial filings and foreign-language news. However, most existing benchmarks are monolingual. This task introduces the first cross-lingual financial reasoning benchmark, testing a model's ability to integrate multilingual textual signals and perform analytical reasoning grounded in authentic financial data.

Difficulty Tiers

Data

Combination of the PolyFiQA-Easy and PolyFiQA-Expert datasets, which combine U.S. SEC filings with multilingual news (English, Chinese, Japanese, Spanish, Greek). Each tier includes 172 Q&A instances (344 total).

Data & Submission

Historical data for backtesting, validation, and model training: TheFinAI/CLEF_Task3_Trading.

To participate, submit your endpoint via the Agent Market Arena page (Google Form): Agent Market Arena.

Daily Workflow

• Each endpoint receives one request per day and must return one action: BUY, HOLD, or SELL.
• Position mapping: BUY -> long, HOLD -> flat, SELL -> short.
• Execution rule: each new action fully replaces the previous day’s position.
• Price convention: daily close price.

Scheduling & Request Policy

• Daily process starts at 00:00 UTC.
• Requests are sent progressively after start time.
• Per-request timeout is 3 minutes.
• If a request fails (timeout, server error, or invalid response), the action defaults to HOLD.

Request / Response Format

Input is a JSON object containing date, price, news, symbol, momentum, history_price, and optional 10k/10q (object or null). The symbol varies by asset (e.g., TSLA, BTC).

{
  "recommended_action": "BUY"
}

Valid actions: BUY, HOLD, SELL.

Input Samples

Sample A (TSLA):

Note: 10k and 10q can each be an object or null.

{
  "date": "2025-01-15",
  "price": {"TSLA": 250.50},
  "news": {"TSLA": ["Tesla announces new production milestone"]},
  "symbol": ["TSLA"],
  "momentum": {"TSLA": "bullish"},
  "10k": {"TSLA": ["[SEC 10-K Filing - 2025-01-15]\nSummary..."]},
  "10q": {"TSLA": ["[SEC 10-Q Filing - 2025-01-15]\nSummary..."]},
  "history_price": {"TSLA": [
    {"date": "2025-01-12", "price": 249.80},
    ...,
    {"date": "2025-01-13", "price": 250.50},
    {"date": "2025-01-14", "price": 250.30}
  ]}
}

Sample B (BTC):

{
  "date": "2025-01-15",
  "price": {"BTC": 67890.50},
  "news": {"BTC": ["Bitcoin ETF inflows remain strong"]},
  "symbol": ["BTC"],
  "momentum": {"BTC": "neutral"},
  "10k": null,
  "10q": null,
  "history_price": {"BTC": [
    {"date": "2025-01-12", "price": 67580.00},
    ...,
    {"date": "2025-01-13", "price": 67720.00},
    {"date": "2025-01-14", "price": 67810.00}
  ]}
}

cURL Sample

curl -X POST "<your_endpoint>" \
  -H "Content-Type: application/json" \
  -d '{
    "date": "2025-01-15",
    "price": {"TSLA": 250.50},
    "news": {"TSLA": ["Tesla announces new production milestone"]},
    "symbol": ["TSLA"],
    "momentum": {"TSLA": "bullish"},
    "10k": {"TSLA": ["[SEC 10-K Filing - 2025-01-15]\nSummary..."]},
    "10q": {"TSLA": ["[SEC 10-Q Filing - 2025-01-15]\nSummary..."]},
    "history_price": {"TSLA": [
      {"date": "2025-01-12", "price": 249.80},
      ...,
      {"date": "2025-01-13", "price": 250.50},
      {"date": "2025-01-14", "price": 250.30}
    ]}
  }'

Data & Submission

Historical data for backtesting, validation, and model training: TheFinAI/CLEF_Task3_Trading.

To participate, submit your endpoint via the Agent Market Arena page (Google Form): https://huggingface.co/spaces/TheFinAI/Agent-Market-Arena.

Reference endpoint example (FastAPI): examples/simple_trading_api.py.