ICBCBench-Leaderboard

Sleeping

App Files Files Community

Leonnel1220 commited on 26 days ago

Commit

5148820

verified ·

1 Parent(s): 10e99ae

Upload folder using huggingface_hub

Browse files

Files changed (19) hide show

.gitignore +4 -8
README.md +103 -10
app.py +1 -2
create_leaderboard.py +44 -53
data/leaderboard.csv +20 -46
main_results.tex +44 -0
objective_public_CalibErr.tex +31 -0
requirements.txt +1 -0
results_domain_specific.tex +41 -0
results_private_set.tex +44 -0
results_subset_en.tex +47 -0
results_subset_zh.tex +47 -0
tabs/data_viewer_side_by_side_tab.py +71 -33
tabs/data_viewer_tab.py +75 -50
tabs/leaderboard_tab.py +349 -147
tabs/shared_data.py +67 -8
utils/merge_raw_data.py +82 -48
utils/parse_icbcbench_tex.py +219 -0
utils/rank_leaderboard.py +214 -84

.gitignore CHANGED Viewed

@@ -1,10 +1,6 @@
 __pycache__/
-*.py[cod]
-*$py.class
-.env
-.venv
-env/
-venv/
-ENV/
 .DS_Store
-*.log# trigger

+.venv/
 __pycache__/
+*.pyc
 .DS_Store
+*.egg-info/
+.ipynb_checkpoints/

README.md CHANGED Viewed

@@ -1,7 +1,7 @@
 ---
-title: DeepResearch Bench
-emoji: 🔍
-colorFrom: blue
 colorTo: gray
 sdk: docker
 app_file: app.py
@@ -9,20 +9,113 @@ pinned: false
 license: apache-2.0
 ---
-# DeepResearch Bench Leaderboard
-A comprehensive benchmark and leaderboard for evaluating Deep Research Agents.
 ## Overview
-This Hugging Face Space hosts the interactive leaderboard for the DeepResearch Bench project.
-Visit our [project website](https://deepresearch-bench.github.io) for more information.
-### Local Development
 ```bash
 pip install -r requirements.txt
 python app.py
 ```
-## Hugging Face Space Details
-- SDK: Docker

 ---
+title: ICBCBench Leaderboard
+emoji: 🏦
+colorFrom: red
 colorTo: gray
 sdk: docker
 app_file: app.py
 license: apache-2.0
 ---
+# ICBCBench Leaderboard
+Interactive leaderboard for **ICBCBench: An Industry Consortium Benchmark for Financial Deep Research**.
 ## Overview
+ICBCBench adopts a dual-track evaluation paradigm:
+- **Objective Track**: Closed-ended financial questions with verifiable answers, judged by GPT-5.4.
+- **Subjective Track**: Long-form financial research reports, evaluated by Gemini-3.1-Pro-Preview using expert-aligned rubrics, citation consistency, and source quality.
+The public subset contains 120 tasks across Global (English) and Chinese markets.
+- 📄 Paper: <https://arxiv.org/abs/2606.17458>
+- 🤗 HF Space: <https://huggingface.co/spaces/DeepFin-Intelligence/ICBCBench-Leaderboard>
+- 💻 Code: <https://github.com/ICBCBench/ICBCBench>
+- 📊 Dataset: <https://huggingface.co/datasets/ICBCBench/ICBCBench>
+## Repository Layout
+```
+├── app.py                          # Gradio entry point
+├── create_leaderboard.py           # Main Gradio Blocks UI
+├── tabs/
+│   ├── leaderboard_tab.py          # Leaderboard rendering
+│   ├── data_viewer_tab.py          # Single-model report viewer
+│   ├── data_viewer_side_by_side_tab.py  # Side-by-side comparison
+│   └── shared_data.py              # Lazy data loading
+├── utils/
+│   ├── rank_leaderboard.py         # Aggregate raw results -> leaderboard.csv
+│   └── merge_raw_data.py           # Merge raw data + scores -> data_viewer.jsonl
+└── data/
+    ├── leaderboard.csv             # Aggregated leaderboard (generated)
+    ├── data_viewer.jsonl           # Per-sample viewer data (generated)
+    ├── raw_data/                   # Per-model prompts + articles
+    └── raw_results/                # Per-model evaluation scores
+```
+## Data Format
+### `data/leaderboard.csv`
+| Column | Description |
+|--------|-------------|
+| `model` | Model / system name |
+| `overall` | Weighted overall score across EN & ZH, Objective & Subjective |
+| `objective_en` | Objective score on Global (English) tasks |
+| `objective_zh` | Objective score on Chinese tasks |
+| `objective_avg` | Average objective score across both language tracks |
+| `subjective_en` | Subjective report score on Global (English) tasks |
+| `subjective_zh` | Subjective report score on Chinese tasks |
+| `subjective_avg` | Average subjective score across both language tracks |
+| `citation_score` | Citation consistency score (0-100) |
+| `source_quality` | Source authority & timeliness score (0-100) |
+| `rmsce` | Root Mean Square Calibration Error for objective confidence (lower is better) |
+### `data/raw_results/<model>/results.jsonl`
+Each line should be a JSON object:
+```json
+{
+  "id": "1",
+  "language": "en",
+  "track": "objective",
+  "score": 0.85,
+  "correct": true,
+  "confidence": 0.90
+}
+```
+or for subjective tasks:
+```json
+{
+  "id": "41",
+  "language": "zh",
+  "track": "subjective",
+  "score": 0.72,
+  "expert_score": 0.74,
+  "citation_score": 0.68,
+  "source_quality_score": 0.70
+}
+```
+Scores can be either 0-1 ratios or 0-100 percentages.
+## Local Development
 ```bash
 pip install -r requirements.txt
+python utils/rank_leaderboard.py      # generate data/leaderboard.csv
+python utils/merge_raw_data.py        # generate data/data_viewer.jsonl
 python app.py
 ```
+> **Note**: The `data/raw_results/` directory in this repository currently contains legacy DeepResearch-Bench-format `race_result.txt` files. Before running `rank_leaderboard.py` for ICBCBench, replace these with ICBCBench `results.jsonl` files as described above. The checked-in `data/leaderboard.csv` is a sample ICBCBench leaderboard and will be overwritten if you rerun the script without updating `raw_results/`.
+Then open <http://localhost:7860>.
+## Citation
+```bibtex
+@article{li2026icbcbench,
+  author    = {Weiya Li and Zhiwei Tang and Yizhou He and Chenghao Wang and Liang Feng and Xiao Sun and Dongrui Liu and Zichen Wen and Hu Wei and Jinghang Wang and Yi Luo and Li Guo and Linfeng Zhang},
+  title     = {ICBCBench: An Industry Consortium Benchmark for Financial Deep Research},
+  journal   = {arXiv preprint arXiv:2606.17458},
+  year      = {2026},
+}
+```

app.py CHANGED Viewed

@@ -1,7 +1,7 @@
 #!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 """
-DeepResearch Bench HF Space 入口文件
 """
 from __future__ import annotations
@@ -13,5 +13,4 @@ if __name__ == "__main__":
         server_name="0.0.0.0",
         server_port=7860,
         share=False,
-        show_api=False,
     )

 #!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 """
+ICBCBench HF Space entrypoint.
 """
 from __future__ import annotations
         server_name="0.0.0.0",
         server_port=7860,
         share=False,
     )

create_leaderboard.py CHANGED Viewed

@@ -1,7 +1,8 @@
 #!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 """
-Gradio UI – v2.1  (Leaderboard · Data Viewer · Prompt-to-Leaderboard)
 """
 from __future__ import annotations
@@ -10,11 +11,11 @@ from datetime import datetime
 import pandas as pd
 import gradio as gr
-# ---- Tab 组件 ----
 from tabs.leaderboard_tab import create_leaderboard_tab
-from tabs.leaderboard_tab_gpt55 import create_leaderboard_tab_gpt55
-from tabs.data_viewer_tab import create_data_viewer_tab
-from tabs.data_viewer_side_by_side_tab import create_data_viewer_side_by_side_tab
 def get_leaderboard_info():
     leaderboard_path = Path(__file__).parent / "data" / "leaderboard.csv"
@@ -27,7 +28,8 @@ def get_leaderboard_info():
             return model_count, last_update
         except Exception:
             pass
-    return 21, "02 August 2025"
 model_count, last_update = get_leaderboard_info()
@@ -35,28 +37,27 @@ model_count, last_update = get_leaderboard_info()
 # UI
 # ---------------------------------------------------------------------------
-with gr.Blocks(title="DeepResearch Bench") as demo:
-    # ========= 全局 CSS（仅作用于自定义标题 & 简介） =========
     gr.HTML("""
     <style>
       .title-block{
-        /* 渐变文字效果 - 改进版 */
-        background: linear-gradient(to right, #009CFF, #823AFF);
-        background: -webkit-linear-gradient(to right, #009CFF, #823AFF);
-        background: -moz-linear-gradient(to right, #009CFF, #823AFF);
         -webkit-background-clip: text;
         -webkit-text-fill-color: transparent;
         background-clip: text;
         color: transparent;
         text-align: center;
         font-size: 2rem;
         font-weight: 700;
         margin: 0 0 1rem 0;
         padding-bottom: 0.2rem;
-        display: inline-block; /* 重要：确保渐变效果正常 */
-        width: 100%; /* 确保居中对齐 */
       }
       .intro-block{
         text-align:center;
@@ -72,46 +73,36 @@ with gr.Blocks(title="DeepResearch Bench") as demo:
     </style>
     """)
-    # ========= 顶部标题 & 简介（不使用 Markdown 标题语法） =========
     gr.HTML(f"""
     <div class="title-block">
-      DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents
     </div>
     <div class="intro-block">
-      The research aims to comprehensively evaluate the capabilities of Deep Research Agents.<br>
-      <a href="https://github.com/Ayanami0730/deep_research_bench" target="_blank">Code</a> |
-      <a href="https://deepresearch-bench.github.io" target="_blank">Website</a> |
-      <a href="https://arxiv.org/abs/2506.11763" target="_blank">Paper</a> |
-      <a href="#" target="_blank">Eval Dataset</a> |
       Total models: {model_count} | Last Update: {last_update}<br>
       <small style="color: #666; font-size: 0.9em;">
-        Leaderboard tab — Race judge: GPT-5.5 | Fact-check: GPT-5.4-mini<br>
-        Gemini-2.5 Eval tab — Race judge: gemini-2.5-pro | Fact-check: gemini-2.5-flash
       </small>
     </div>
     """)
-    # ========= 主 Tabs =========
     with gr.Tabs():
-        create_leaderboard_tab_gpt55()     # 🏆 Leaderboard
-        create_leaderboard_tab()           # 🏆 Leaderboard (Gemini-2.5 Eval)
-        sbs_on_load, sbs_outputs = create_data_viewer_side_by_side_tab()
-        dv_on_load, dv_outputs = create_data_viewer_tab()
-        with gr.Tab("💬Prompt-to-Leaderboard"):
-            gr.Markdown(
-                """
-🚧 **Prompt-to-Leaderboard** module not implemented yet.
-Planned: inspect how individual prompts affect overall model ranking.
-"""
-            )
-    demo.load(fn=dv_on_load, outputs=dv_outputs)
-    demo.load(fn=sbs_on_load, outputs=sbs_outputs)
-    # ========= Citation 板块 =========
     gr.HTML("""
     <style>
       .citation-block {
@@ -158,26 +149,26 @@ Planned: inspect how individual prompts affect overall model ranking.
         background-color: #198754;
       }
     </style>
     <div class="citation-block">
       <div class="citation-title">📚 Citation</div>
       <div class="citation-content" id="citation-text">
         <button class="copy-btn" onclick="copyCitation()">Copy</button>
-@article{du2025deepresearch,
-  author    = {Mingxuan Du and Benfeng Xu and Chiwei Zhu and Xiaorui Wang and Zhendong Mao},
-  title     = {DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents},
-  journal   = {arXiv preprint},
-  year      = {2025},
 }</div>
     </div>
     <script>
     function copyCitation() {
-      const citationText = `@article{du2025deepresearch,
-  author    = {Mingxuan Du and Benfeng Xu and Chiwei Zhu and Xiaorui Wang and Zhendong Mao},
-  title     = {DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents},
-  journal   = {arXiv preprint},
-  year      = {2025},
 }`;
       navigator.clipboard.writeText(citationText).then(function() {
         const btn = document.querySelector('.copy-btn');

 #!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 """
+Gradio UI – ICBCBench Leaderboard
+Dual-track (Objective / Subjective) leaderboard for financial deep research.
 """
 from __future__ import annotations
 import pandas as pd
 import gradio as gr
+# ---- Tab components ----
 from tabs.leaderboard_tab import create_leaderboard_tab
+# from tabs.data_viewer_tab import create_data_viewer_tab
+# from tabs.data_viewer_side_by_side_tab import create_data_viewer_side_by_side_tab
 def get_leaderboard_info():
     leaderboard_path = Path(__file__).parent / "data" / "leaderboard.csv"
             return model_count, last_update
         except Exception:
             pass
+    return 0, "Not yet updated"
 model_count, last_update = get_leaderboard_info()
 # UI
 # ---------------------------------------------------------------------------
+with gr.Blocks(title="ICBCBench Leaderboard") as demo:
+    # ========= Global CSS =========
     gr.HTML("""
     <style>
       .title-block{
+        background: linear-gradient(to right, #C41E3A, #8B0000);
+        background: -webkit-linear-gradient(to right, #C41E3A, #8B0000);
+        background: -moz-linear-gradient(to right, #C41E3A, #8B0000);
         -webkit-background-clip: text;
         -webkit-text-fill-color: transparent;
         background-clip: text;
         color: transparent;
         text-align: center;
         font-size: 2rem;
         font-weight: 700;
         margin: 0 0 1rem 0;
         padding-bottom: 0.2rem;
+        display: inline-block;
+        width: 100%;
       }
       .intro-block{
         text-align:center;
     </style>
     """)
+    # ========= Header =========
     gr.HTML(f"""
     <div class="title-block">
+      ICBCBench: An Industry Consortium Benchmark for Financial Deep Research
     </div>
     <div class="intro-block">
+      ICBCBench evaluates Deep Research Agents on financial tasks through a dual-track paradigm:<br>
+      <strong>Objective</strong> (verifiable answers) and <strong>Subjective</strong> (expert-aligned report quality).<br>
+      <a href="https://huggingface.co/spaces/DeepFin-Intelligence/ICBCBench-Leaderboard" target="_blank">HF Space</a> |
+      <a href="https://arxiv.org/abs/2606.17458" target="_blank">Paper</a> |
+      <a href="https://github.com/ICBCBench/ICBCBench" target="_blank">Code</a> |
+      <a href="https://huggingface.co/datasets/ICBCBench/ICBCBench" target="_blank">Dataset</a> |
       Total models: {model_count} | Last Update: {last_update}<br>
       <small style="color: #666; font-size: 0.9em;">
+        Objective judge: GPT-5.4 &nbsp;|&nbsp; Subjective judge: Gemini-3.1-Pro-Preview
       </small>
     </div>
     """)
+    # ========= Main Tabs =========
     with gr.Tabs():
+        create_leaderboard_tab()             # 🏆 Leaderboard
+        # sbs_on_load, sbs_outputs = create_data_viewer_side_by_side_tab()  # ⚔️ Side-by-Side
+        # dv_on_load, dv_outputs = create_data_viewer_tab()                 # 🔍 Data Viewer
+    # demo.load(fn=dv_on_load, outputs=dv_outputs)
+    # demo.load(fn=sbs_on_load, outputs=sbs_outputs)
+    # ========= Citation =========
     gr.HTML("""
     <style>
       .citation-block {
         background-color: #198754;
       }
     </style>
     <div class="citation-block">
       <div class="citation-title">📚 Citation</div>
       <div class="citation-content" id="citation-text">
         <button class="copy-btn" onclick="copyCitation()">Copy</button>
+@article{li2026icbcbench,
+  author    = {Weiya Li and Zhiwei Tang and Yizhou He and Chenghao Wang and Liang Feng and Xiao Sun and Dongrui Liu and Zichen Wen and Hu Wei and Jinghang Wang and Yi Luo and Li Guo and Linfeng Zhang},
+  title     = {ICBCBench: An Industry Consortium Benchmark for Financial Deep Research},
+  journal   = {arXiv preprint arXiv:2606.17458},
+  year      = {2026},
 }</div>
     </div>
     <script>
     function copyCitation() {
+      const citationText = `@article{li2026icbcbench,
+  author    = {Weiya Li and Zhiwei Tang and Yizhou He and Chenghao Wang and Liang Feng and Xiao Sun and Dongrui Liu and Zichen Wen and Hu Wei and Jinghang Wang and Yi Luo and Li Guo and Linfeng Zhang},
+  title     = {ICBCBench: An Industry Consortium Benchmark for Financial Deep Research},
+  journal   = {arXiv preprint arXiv:2606.17458},
+  year      = {2026},
 }`;
       navigator.clipboard.writeText(citationText).then(function() {
         const btn = document.querySelector('.copy-btn');

data/leaderboard.csv CHANGED Viewed

@@ -1,46 +1,20 @@
-model,overall_score,comprehensiveness,insight,instruction_following,readability,citation_accuracy,effective_citations
-qianfan_deepresearch_0430,58.03,59.48,61.48,53.87,54.34,-,-
-ZTE-Nebula-DeepResearch-V20260519,57.27,58.37,59.76,54.06,54.66,-,-
-Link,57.08,58.24,59.74,53.24,55.05,-,-
-zhipu_deep_research,57.06,58.15,60.14,53.47,53.88,-,-
-xiaoyi,57.00,58.58,59.38,53.58,53.99,-,-
-WhaleCloud-DocChain,56.81,57.13,59.30,53.98,54.97,-,-
-cellcog-max,56.67,57.40,60.01,53.25,53.21,-,-
-1688AILab-DeepResearch-0428,56.53,57.32,59.27,53.51,53.36,-,-
-octen-deepresearch-0508,56.31,56.89,59.00,53.39,53.83,-,-
-grep-v5,56.23,56.82,58.92,53.38,53.44,-,-
-nvidia-aiq-nemotron-gpt52-updated,55.95,56.90,58.49,52.89,53.43,-,-
-1688AILab-DeepResearch-0325,55.39,55.48,57.59,53.38,53.50,-,-
-ms_deepresearch_gpt52mixqwen35_09_edit_restart09_think_medium,55.31,56.76,56.79,53.10,52.28,-,-
-drb_cellcog,55.31,55.41,58.21,52.50,53.12,-,-
-deepinsight,55.24,55.66,58.70,52.53,50.94,-,-
-ms_deepresearch,54.97,56.45,56.22,53.25,51.71,-,-
-TrajectoryKit,54.92,54.10,57.90,52.91,52.72,-,-
-onyx,54.54,54.67,56.43,53.08,52.02,-,-
-deepsynth,54.22,54.23,56.09,52.86,51.81,-,-
-deepdog,53.52,53.14,56.10,51.83,51.18,-,-
-RecallRadar,53.19,53.91,53.53,52.18,52.38,-,-
-MindDR-V1.5,52.54,51.54,55.30,50.45,51.26,-,-
-tavily-research,52.44,52.84,53.59,51.92,49.21,-,-
-thinkdepthai-deepresearch,52.43,52.02,53.88,52.04,50.12,-,-
-salesforce-air-deep-research,50.65,50.00,51.09,50.77,50.32,-,-
-gensee-search-gpt-5,50.60,50.06,50.76,51.31,49.72,32.94,21.06
-gemini-2.5-pro-deepresearch,49.71,49.51,49.45,50.12,50.00,78.30,165.34
-langchain-open-deep-research-gpt-5,49.33,49.80,47.34,51.05,48.99,34.74,22.44
-openai-deepresearch,46.45,46.46,43.73,49.39,47.22,75.01,39.79
-raaa-deep-research,46.13,43.77,48.34,47.21,43.78,-,-
-dr-tulu,45.49,44.08,44.65,49.56,42.30,-,-
-claude-research,45.00,45.34,42.79,47.58,44.66,-,-
-kimi-researcher,44.64,44.96,41.97,47.14,45.59,-,-
-doubao-deepresearch,44.34,44.84,40.56,47.95,44.69,52.86,52.62
-langchain-open-deep-research,43.44,42.97,39.17,48.09,45.22,49.10,29.49
-nvidia-aiq-research-assistant,40.52,37.98,38.39,44.59,42.63,-,-
-tongyi-deepresearch-30B-A3B,40.46,39.46,34.44,46.22,44.27,-,-
-perplexity-Research,40.46,39.10,35.65,46.11,43.08,82.63,31.20
-grok-deeper-search,38.22,36.08,30.89,46.59,42.17,73.08,8.58
-sonar-reasoning-pro,37.76,34.96,31.65,44.93,42.42,45.19,9.39
-sonar-reasoning,37.75,34.73,32.59,44.42,42.39,52.58,13.37
-claude-3-7-sonnet-with-search,36.63,35.95,31.29,44.05,36.07,87.32,24.51
-sonar-pro,36.19,33.92,29.69,43.39,41.07,79.72,16.75
-gemini-2.5-pro-preview-05-06,31.90,31.75,24.61,40.24,32.76,-,-
-gpt-4o-search-preview,30.74,27.81,20.44,41.01,37.60,86.63,5.05

+model,overall,objective_en,objective_zh,objective_avg,subjective_en,subjective_zh,subjective_avg,expert_avg,citation_score,source_quality,rmsce
+OpenClaw(+GPT-5.5),59.09,50.00,67.50,58.75,59.60,59.25,59.42,74.28,-,-,46.23
+DeerFlow(+GPT-5.5),58.76,52.50,60.00,56.25,64.85,57.67,61.26,76.57,-,-,47.83
+Gemini-deep-research,58.23,50.00,52.50,51.25,64.77,65.69,65.23,71.79,56.94,12.86,50.77
+OpenClaw(+DeepSeek-V4-Pro),54.54,37.50,57.50,47.50,65.79,57.36,61.58,76.97,-,-,56.57
+DeerFlow(+DeepSeek-V4-Pro),51.57,27.50,55.00,41.25,65.71,58.08,61.89,77.36,-,-,64.52
+OpenAI-o3-deep-research,51.24,37.50,32.50,35.00,71.84,63.12,67.48,72.60,74.47,15.49,61.11
+MiroThinker,48.63,52.50,45.00,48.75,53.15,43.88,48.52,60.64,-,-,55.57
+Kimi-deep-research,46.16,35.00,35.00,35.00,60.19,54.44,57.31,71.64,-,-,62.94
+GPT-5.5,43.75,27.50,27.50,27.50,62.69,57.33,60.01,75.02,-,-,52.32
+Jina-deepsearch,42.73,37.50,35.00,36.25,47.51,50.89,49.20,52.52,46.36,27.54,65.67
+Claude-opus-4-7,42.38,25.00,20.00,22.50,63.71,60.83,62.27,77.83,-,-,39.24
+Doubao-deep-research,40.76,37.50,20.00,28.75,52.93,52.61,52.77,65.97,-,-,71.30
+Perplexity-deep-research,39.26,22.50,22.50,22.50,63.17,48.85,56.01,70.02,-,-,55.65
+Kimi-k2.5,38.48,17.50,10.00,13.75,64.81,61.60,63.20,79.01,-,-,77.62
+Gemini-3.1-pro-preview,38.20,22.50,12.50,17.50,59.53,58.25,58.89,73.62,-,-,75.60
+DeepSeek-V4-Pro,31.17,5.00,15.00,10.00,49.09,55.59,52.34,65.42,-,-,76.12
+Grok-3-deepsearch,30.46,10.00,5.00,7.50,56.43,50.40,53.41,66.77,-,-,84.70
+Qwen-deep-research,29.97,2.50,17.50,10.00,51.59,48.25,49.92,62.40,-,-,83.37
+Tongyi-deepresearch-30b-a3b,24.09,2.50,5.00,3.75,46.69,42.16,44.42,55.53,-,-,87.92

main_results.tex ADDED Viewed

	@@ -0,0 +1,44 @@

+\begin{table*}[t]
+\centering
+\small
+\setlength{\tabcolsep}{5pt}
+\renewcommand{\arraystretch}{1.08}
+\caption{\textbf{Main results on ICBCBench.} We report performance on global (EN) and Chinese (ZH) market scenarios. Objective evaluates closed-ended tasks, while Subjective assesses open-ended financial research reports. The best and second-best scores are highlighted in \textbf{bold} and \underline{underline}, respectively. Higher is better for all metrics.}
+\label{tab:main_results}
+\begin{tabular}{lcccccc}
+\toprule
+\multirow{2}{*}{\textbf{System}} & \multicolumn{3}{c}{\textbf{Global (EN)}} & \multicolumn{3}{c}{\textbf{Chinese (ZH)}} \\
+\cmidrule(lr){2-4} \cmidrule(lr){5-7}
+& \textbf{Objective} & \textbf{Subjective} & \textbf{Overall} & \textbf{Objective} & \textbf{Subjective} & \textbf{Overall} \\
+\midrule
+\rowcolor{gray!20}
+\multicolumn{7}{c}{\textbf{\textit{Closed}}} \\
+Gemini-deep-research                     &           50.00 &           64.77 & \underline{57.38} &           52.50 &  \textbf{65.69} & \underline{59.09} \\
+OpenAI-o3-deep-research                  &           37.50 &  \textbf{71.84} &           54.67 &           32.50 & \underline{63.12} &           47.81 \\
+Kimi-deep-research                       &           35.00 &           60.19 &           47.59 &           35.00 &           54.44 &           44.72 \\
+Doubao-deep-research                     &           37.50 &           52.93 &           45.22 &           20.00 &           52.61 &           36.30 \\
+GPT-5.5                                  &           27.50 &           62.69 &           45.09 &           27.50 &           57.33 &           42.41 \\
+Claude-opus-4-7                          &           25.00 &           63.71 &           44.36 &           20.00 &           60.83 &           40.41 \\
+Perplexity-deep-research                 &           22.50 &           63.17 &           42.84 &           22.50 &           48.85 &           35.67 \\
+Gemini-3.1-pro-preview                   &           22.50 &           59.53 &           41.02 &           12.50 &           58.25 &           35.38 \\
+Grok-3-deepsearch                        &           10.00 &           56.43 &           33.22 &            5.00 &           50.40 &           27.70 \\
+Qwen-deep-research                       &            2.50 &           51.59 &           27.05 &           17.50 &           48.25 &           32.88 \\
+\midrule
+\rowcolor{gray!20}
+\multicolumn{7}{c}{\textbf{\textit{Open}}} \\
+DeerFlow(+GPT-5.5)                       &  \textbf{52.50} &           64.85 &  \textbf{58.67} & \underline{60.00} &           57.67 &           58.84 \\
+OpenClaw(+GPT-5.5)                       &           50.00 &           59.60 &           54.80 &  \textbf{67.50} &           59.25 &  \textbf{63.38} \\
+MiroThinker                              &  \textbf{52.50} &           53.15 &           52.83 &           45.00 &           43.88 &           44.44 \\
+OpenClaw(+DeepSeek-V4-Pro)               &           37.50 & \underline{65.79} &           51.65 &           57.50 &           57.36 &           57.43 \\
+DeerFlow(+DeepSeek-V4-Pro)               &           27.50 &           65.71 &           46.60 &           55.00 &           58.08 &           56.54 \\
+Jina-deepsearch                          &           37.50 &           47.51 &           42.50 &           35.00 &           50.89 &           42.95 \\
+Kimi-k2.5                                &           17.50 &           64.81 &           41.16 &           10.00 &           61.60 &           35.80 \\
+DeepSeek-V4-Pro                          &            5.00 &           49.09 &           27.05 &           15.00 &           55.59 &           35.30 \\
+Tongyi-deepresearch-30b-a3b              &            2.50 &           46.69 &           24.59 &            5.00 &           42.16 &           23.58 \\
+% OpenClaw(+DeepSeek-V4-Flash)             &              -- &              -- &              -- &              -- &              -- &              -- \\
+% DeerFlow(+DeepSeek-V4-Flash)             &              -- &              -- &              -- &              -- &              -- &              -- \\
+\bottomrule
+\end{tabular}
+\end{table*}

objective_public_CalibErr.tex ADDED Viewed

	@@ -0,0 +1,31 @@

+\begin{table}[t]
+\centering
+\small
+\caption{Objective Evaluation Results on Public Subset (All Languages). Higher Accuracy and lower Calibration Error are better.}
+\label{tab:objective_public_CalibErr}
+\begin{tabular}{lcc}
+\toprule
+\textbf{Model} & \textbf{Accuracy (\%)} & \textbf{Calibration Error (\%)} \\
+\midrule
+OpenClaw(+GPT-5.5) & 58.75 & 46.23 \\
+DeerFlow(+GPT-5.5) & 56.25 & 47.83 \\
+Gemini-deep-research & 51.25 & 50.77 \\
+MiroThinker & 48.75 & 55.57 \\
+OpenClaw(+DeepSeek-V4-Pro) & 47.50 & 56.57 \\
+DeerFlow(+DeepSeek-V4-Pro) & 41.25 & 64.52 \\
+Jina-deepsearch & 36.25 & 65.67 \\
+OpenAI-o3-deep-research & 35.00 & 61.11 \\
+Kimi-deep-research & 35.00 & 62.94 \\
+Doubao-deep-research & 28.75 & 71.30 \\
+GPT-5.5 & 27.50 & 52.32 \\
+Claude-opus-4-7 & 22.50 & 39.24 \\
+Perplexity-deep-research & 22.50 & 55.65 \\
+Gemini-3.1-pro-preview & 17.50 & 75.60 \\
+Kimi-k2.5 & 13.75 & 77.62 \\
+DeepSeek-V4-Pro & 10.00 & 76.12 \\
+Qwen-deep-research & 10.00 & 83.37 \\
+Grok-3-deepsearch & 7.50 & 84.70 \\
+Tongyi-deepresearch-30b-a3b & 3.75 & 87.92 \\
+\bottomrule
+\end{tabular}
+\end{table}

requirements.txt CHANGED Viewed

@@ -1,4 +1,5 @@
 gradio>=5.31.0,<6.0.0
 pandas>=1.5
 numpy
 plotly

 gradio>=5.31.0,<6.0.0
+huggingface_hub>=0.25.0
 pandas>=1.5
 numpy
 plotly

results_domain_specific.tex ADDED Viewed

	@@ -0,0 +1,41 @@

+\begin{table*}[t]
+\centering
+\small
+\setlength{\tabcolsep}{6pt}
+\renewcommand{\arraystretch}{1.1}
+\caption{\textbf{Domain-specific objective performance across major financial sectors.} This table reports objective task accuracy across Banking, Capital Markets, Insurance, and Other Financial Services. The results reveal that Open-Agentic frameworks consistently outperform closed-source proprietary models across all domains, highlighting their robust factual extraction and tool-use capabilities in specialized financial contexts. The best and second-best scores are highlighted in \textbf{bold} and \underline{underline}.}
+\label{tab:results_domain_specific}
+\begin{tabular}{lccccc}
+\toprule
+\textbf{System} & \textbf{Banking} & \textbf{Capital Markets} & \textbf{Insurance} & \textbf{Others} & \textbf{Average} \\
+\midrule
+\rowcolor{gray!20}
+\multicolumn{6}{c}{\textbf{\textit{Closed}}} \\
+Gemini-deep-research          & \underline{53.57} & \underline{66.67} & 40.00 & 25.00 & 46.31 \\
+Kimi-deep-research            & 28.57 & 50.00 & 30.00 & 25.00 & 33.39 \\
+OpenAI-o3-deep-research       & 42.86 & 33.33 & 30.00 & 25.00 & 32.80 \\
+GPT-5.5                       & 25.00 & 20.83 & 35.00 & 37.50 & 29.58 \\
+Doubao-deep-research          & 35.71 & 29.17 & 25.00 & 12.50 & 25.60 \\
+Perplexity-deep-research      & 17.86 & 25.00 & 25.00 & 25.00 & 23.21 \\
+Claude-opus-4-7               & 28.57 & 12.50 & 30.00 & 12.50 & 20.89 \\
+Gemini-3.1-pro-preview        & 21.43 & 16.67 & 15.00 & 12.50 & 16.40 \\
+Grok-3-deepsearch             & 7.14 & 8.33 & 5.00 & 12.50 & 8.24 \\
+Qwen-deep-research            & 10.71 & 20.83 & 0.00 & 0.00 & 7.89 \\
+\rowcolor{gray!20}
+\multicolumn{6}{c}{\textbf{\textit{Open}}} \\
+OpenClaw(+GPT-5.5)            & \textbf{57.14} & 62.50 & \underline{50.00} & \textbf{75.00} & \textbf{61.16} \\
+DeerFlow(+GPT-5.5)            & 46.43 & \underline{66.67} & \textbf{55.00} & \underline{62.50} & \underline{57.65} \\
+MiroThinker                   & \underline{53.57} & 58.33 & 35.00 & 37.50 & 46.10 \\
+OpenClaw(+DeepSeek-V4-Pro)    & 42.86 & \underline{66.67} & 35.00 & 37.50 & 45.51 \\
+DeerFlow(+DeepSeek-V4-Pro)    & 21.43 & \textbf{70.83} & 25.00 & \underline{62.50} & 44.94 \\
+Jina-deepsearch               & 39.29 & 41.67 & 30.00 & 25.00 & 33.99 \\
+Kimi-k2.5                     & 14.29 & 12.50 & 20.00 & 0.00 & 11.70 \\
+DeepSeek-V4-Pro               & 10.71 & 4.17 & 15.00 & 12.50 & 10.60 \\
+Tongyi-deepresearch-30b-a3b   & 3.57 & 4.17 & 5.00 & 0.00 & 3.18 \\
+\bottomrule
+\end{tabular}
+\end{table*}

results_private_set.tex ADDED Viewed

	@@ -0,0 +1,44 @@

+\begin{table*}[t]
+\centering
+\small
+\setlength{\tabcolsep}{5pt}
+\renewcommand{\arraystretch}{1.08}
+\caption{\textbf{Generalization performance on the private hold-out set of ICBCBench.} The private set is not publicly released and is designed to evaluate model generalization and prevent benchmark overfitting. Results are reported on both global (EN) and Chinese (ZH) scenarios across objective and subjective tasks. The best and second-best scores are highlighted in \textbf{bold} and \underline{underline}, respectively. Higher is better for all metrics.}
+\label{tab:results_private_set}
+\begin{tabular}{lcccccc}
+\toprule
+\multirow{2}{*}{\textbf{System}} & \multicolumn{3}{c}{\textbf{Global (EN)}} & \multicolumn{3}{c}{\textbf{Chinese (ZH)}} \\
+\cmidrule(lr){2-4} \cmidrule(lr){5-7}
+& \textbf{Objective} & \textbf{Subjective} & \textbf{Overall} & \textbf{Objective} & \textbf{Subjective} & \textbf{Overall} \\
+\midrule
+\rowcolor{gray!20}
+\multicolumn{7}{c}{\textbf{\textit{Closed}}} \\
+Gemini-deep-research                     & \underline{75.00} &           64.68 &           69.84 &           45.00 & \underline{63.19} &           54.09 \\
+OpenAI-o3-deep-research                  &           55.00 &  \textbf{69.05} &           62.02 &           35.00 &           61.86 &           48.43 \\
+Kimi-deep-research                       &           55.00 &           59.57 &           57.28 &           40.00 &           55.24 &           47.62 \\
+Doubao-deep-research                     &           40.00 &           47.17 &           43.59 &           25.00 &           49.47 &           37.23 \\
+Perplexity-deep-research                 &           20.00 &           60.53 &           40.27 &           30.00 &           45.64 &           37.82 \\
+GPT-5.5                                  &           20.00 &           57.00 &           38.50 &           20.00 &           53.93 &           36.97 \\
+Claude-opus-4-7                          &            5.00 &           62.57 &           33.78 &           20.00 &           58.38 &           39.19 \\
+Gemini-3.1-pro-preview                   &           10.00 &           56.43 &           33.22 &           25.00 &           55.40 &           40.20 \\
+Grok-3-deepsearch                        &           10.00 &           53.03 &           31.52 &           15.00 &           41.89 &           28.45 \\
+Qwen-deep-research                       &           10.00 &           48.30 &           29.15 &           20.00 &           45.18 &           32.59 \\
+\midrule
+\rowcolor{gray!20}
+\multicolumn{7}{c}{\textbf{\textit{Open}}} \\
+OpenClaw(+DeepSeek-V4-Pro)               &  \textbf{85.00} &           64.83 &  \textbf{74.91} & \underline{55.00} &           56.09 & \underline{55.55} \\
+DeerFlow(+DeepSeek-V4-Pro)               & \underline{75.00} & \underline{68.63} & \underline{71.81} &           40.00 &           56.27 &           48.14 \\
+OpenClaw(+GPT-5.5)                       &           70.00 &           61.63 &           65.81 &  \textbf{60.00} &           57.24 &  \textbf{58.62} \\
+DeerFlow(+GPT-5.5)                       &           65.00 &           59.80 &           62.40 &           45.00 &           57.67 &           51.34 \\
+MiroThinker                              &           65.00 &           43.30 &           54.15 &           40.00 &           36.18 &           38.09 \\
+Jina-deepsearch                          &           20.00 &           48.60 &           34.30 &           10.00 &           45.37 &           27.68 \\
+Kimi-k2.5                                &            5.00 &           62.63 &           33.81 &           20.00 &  \textbf{64.16} &           42.08 \\
+Tongyi-deepresearch-30b-a3b              &            0.00 &           46.27 &           23.14 &            5.00 &           36.91 &           20.95 \\
+DeepSeek-V4-Pro                          &            5.00 &           22.50 &           13.75 &           20.00 &           49.47 &           34.73 \\
+% OpenClaw(+DeepSeek-V4-Flash)             &              -- &              -- &              -- &              -- &              -- &              -- \\
+% DeerFlow(+DeepSeek-V4-Flash)             &              -- &              -- &              -- &              -- &              -- &              -- \\
+\bottomrule
+\end{tabular}
+\end{table*}

results_subset_en.tex ADDED Viewed

	@@ -0,0 +1,47 @@

+\begin{table*}[t]
+\centering
+\small
+\setlength{\tabcolsep}{5pt}
+\renewcommand{\arraystretch}{1.1}
+\caption{\textbf{Performance on English (EN) tasks in ICBCBench.} We report objective and subjective results for global market scenarios. Objective includes text-only and aggregated scores (All), while Subjective evaluates report quality via Expert Rubrics, Citation Consistency, and Source Quality. The Overall score is the arithmetic mean of the Objective and Subjective scores. The best and second-best scores are highlighted in \textbf{bold} and \underline{underline}, respectively. Higher is better for all metrics.}
+\label{tab:results_subset_en}
+\begin{tabular}{lcccccc}
+\toprule
+\multirow{2}{*}{\textbf{System}}
+& \multicolumn{2}{c}{\textbf{Objective}}
+& \multicolumn{3}{c}{\textbf{Subjective}}
+& \multirow{2}{*}{\textbf{Overall}} \\
+\cmidrule(lr){2-3} \cmidrule(lr){4-6}
+& \textbf{Text-Only} & \textbf{All} & \textbf{Expert} & \textbf{Citation} & \textbf{Source} & \\
+\midrule
+\rowcolor{gray!30}
+\multicolumn{7}{c}{\textbf{\textit{Closed}}} \\
+Gemini-deep-research          & \underline{57.14} &           50.00 &           72.23 & \underline{56.94} &           12.86 & \underline{57.38} \\
+OpenAI-o3-deep-research       &           37.14 &           37.50 &           78.55 &  \textbf{74.47} & \underline{15.49} &           54.67 \\
+Kimi-deep-research            &           40.00 &           35.00 &           75.23 &            -- &            -- &           47.59 \\
+Doubao-deep-research          &           42.86 &           37.50 &           66.17 &            -- &            -- &           45.22 \\
+GPT-5.5                       &           20.00 &           27.50 &           78.37 &            -- &            -- &           45.09 \\
+Claude-opus-4-7               &           17.14 &           25.00 &           79.63 &            -- &            -- &           44.36 \\
+Perplexity-deep-research      &           25.71 &           22.50 &           78.97 &            -- &            -- &           42.84 \\
+Gemini-3.1-pro-preview        &           20.00 &           22.50 &           74.42 &            -- &            -- &           41.02 \\
+Grok-3-deepsearch             &            8.57 &           10.00 &           70.53 &            -- &            -- &           33.22 \\
+Qwen-deep-research            &            2.86 &            2.50 &           64.48 &            -- &            -- &           27.05 \\
+\midrule
+\rowcolor{gray!30}
+\multicolumn{7}{c}{\textbf{\textit{Open}}} \\
+DeerFlow(+GPT-5.5)            & \underline{57.14} &  \textbf{52.50} &           81.07 &            -- &            -- &  \textbf{58.67} \\
+OpenClaw(+GPT-5.5)            &           54.29 &           50.00 &           74.50 &            -- &            -- &           54.80 \\
+MiroThinker                   &  \textbf{60.00} &  \textbf{52.50} &           66.43 &            -- &            -- &           52.83 \\
+OpenClaw(+DeepSeek-V4-Pro)    &           40.00 &           37.50 &  \textbf{82.23} &            -- &            -- &           51.65 \\
+DeerFlow(+DeepSeek-V4-Pro)    &           31.43 &           27.50 & \underline{82.13} &            -- &            -- &           46.60 \\
+Jina-deepsearch               &           34.29 &           37.50 &           50.15 &           46.36 &  \textbf{27.54} &           42.50 \\
+Kimi-k2.5                     &           14.29 &           17.50 &           81.02 &            -- &            -- &           41.16 \\
+DeepSeek-V4-Pro               &            5.71 &            5.00 &           61.37 &            -- &            -- &           27.05 \\
+Tongyi-deepresearch-30b-a3b   &            2.86 &            2.50 &           58.37 &            -- &            -- &           24.59 \\
+% OpenClaw(+DeepSeek-V4-Flash)  &              -- &              -- &              -- &              -- &              -- &              -- \\
+% DeerFlow(+DeepSeek-V4-Flash)  &              -- &              -- &              -- &              -- &              -- &              -- \\
+\bottomrule
+\end{tabular}
+\end{table*}

results_subset_zh.tex ADDED Viewed

	@@ -0,0 +1,47 @@

+\begin{table*}[t]
+\centering
+\small
+\setlength{\tabcolsep}{5pt}
+\renewcommand{\arraystretch}{1.1}
+\caption{\textbf{Performance on Chinese (ZH) tasks in ICBCBench.} We report objective and subjective results for domestic market scenarios. Objective includes text-only and aggregated scores (All), while Subjective evaluates report quality via Expert Rubrics, Citation Consistency, and Source Quality. The Overall score is the arithmetic mean of the Objective and Subjective scores. The best and second-best scores are highlighted in \textbf{bold} and \underline{underline}, respectively. Higher is better for all metrics.}
+\label{tab:results_subset_zh}
+\begin{tabular}{lcccccc}
+\toprule
+\multirow{2}{*}{\textbf{System}}
+& \multicolumn{2}{c}{\textbf{Objective}}
+& \multicolumn{3}{c}{\textbf{Subjective}}
+& \multirow{2}{*}{\textbf{Overall}} \\
+\cmidrule(lr){2-3} \cmidrule(lr){4-6}
+& \textbf{Text-Only} & \textbf{All} & \textbf{Expert} & \textbf{Citation} & \textbf{Source} & \\
+\midrule
+\rowcolor{gray!30}
+\multicolumn{7}{c}{\textbf{\textit{Closed}}} \\
+Gemini-deep-research          &           61.76 &           52.50 &           71.35 & \underline{73.16} &           12.97 & \underline{59.09} \\
+OpenAI-o3-deep-research       &           38.24 &           32.50 &           66.65 &  \textbf{79.35} & \underline{18.65} &           47.81 \\
+Kimi-deep-research            &           41.18 &           35.00 &           68.05 &            -- &            -- &           44.72 \\
+GPT-5.5                       &           29.41 &           27.50 &           71.67 &            -- &            -- &           42.41 \\
+Claude-opus-4-7               &           23.53 &           20.00 & \underline{76.03} &            -- &            -- &           40.41 \\
+Doubao-deep-research          &           23.53 &           20.00 &           65.77 &            -- &            -- &           36.30 \\
+Perplexity-deep-research      &           26.47 &           22.50 &           61.07 &            -- &            -- &           35.67 \\
+Gemini-3.1-pro-preview        &           14.71 &           12.50 &           72.82 &            -- &            -- &           35.38 \\
+Qwen-deep-research            &           20.59 &           17.50 &           60.32 &            -- &            -- &           32.88 \\
+Grok-3-deepsearch             &            5.88 &            5.00 &           63.00 &            -- &            -- &           27.70 \\
+\midrule
+\rowcolor{gray!30}
+\multicolumn{7}{c}{\textbf{\textit{Open}}} \\
+OpenClaw(+GPT-5.5)            &  \textbf{70.59} &  \textbf{67.50} &           74.07 &            -- &            -- &  \textbf{63.38} \\
+DeerFlow(+GPT-5.5)            & \underline{64.71} & \underline{60.00} &           72.08 &            -- &            -- &           58.84 \\
+OpenClaw(+DeepSeek-V4-Pro)    &           61.76 &           57.50 &           71.70 &            -- &            -- &           57.43 \\
+DeerFlow(+DeepSeek-V4-Pro)    &           61.76 &           55.00 &           72.60 &            -- &            -- &           56.54 \\
+MiroThinker                   &           52.94 &           45.00 &           54.85 &            -- &            -- &           44.44 \\
+Jina-deepsearch               &           41.18 &           35.00 &           54.88 &           38.02 &  \textbf{31.83} &           42.95 \\
+Kimi-k2.5                     &           11.76 &           10.00 &  \textbf{77.00} &            -- &            -- &           35.80 \\
+DeepSeek-V4-Pro               &           17.65 &           15.00 &           69.48 &            -- &            -- &           35.30 \\
+Tongyi-deepresearch-30b-a3b   &            5.88 &            5.00 &           52.70 &            -- &            -- &           23.58 \\
+% OpenClaw(+DeepSeek-V4-Flash)  &              -- &              -- &              -- &              -- &              -- &              -- \\
+% DeerFlow(+DeepSeek-V4-Flash)  &              -- &              -- &              -- &              -- &              -- &              -- \\
+\bottomrule
+\end{tabular}
+\end{table*}

tabs/data_viewer_side_by_side_tab.py CHANGED Viewed

@@ -1,7 +1,7 @@
 #!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 """
-Data-Viewer Side-by-Side tab
 """
 import gradio as gr
@@ -10,6 +10,7 @@ import re
 from tabs.shared_data import get_entries_for_task, get_index
 def make_user_task_markdown(item_id, prompt):
     return f"""### User Task 🎯
@@ -17,24 +18,27 @@ def make_user_task_markdown(item_id, prompt):
 **Description:** {prompt}"""
 def make_article_markdown(article: str) -> str:
     if article and isinstance(article, str):
         processed_article = re.sub(r'\n{2,}', '\n\n', article)
         table_pattern = r'(\|[^\n]*\n(?:[\|\s\-:]+\n)?(?:\|[^\n]*\n)*)'
         tables = []
         def replace_table(match):
             tables.append(match.group(1))
             return f'__TABLE_PLACEHOLDER_{len(tables)-1}__'
         processed_article = re.sub(table_pattern, replace_table, processed_article)
         processed_article = re.sub(r'(?<!\n)\*\s*\*\*([^*]+?)\*\*:', r'\n\n* **\1**:', processed_article)
         processed_article = re.sub(r'\*\s*\*\*([^*]+?)\*\*:\s*([^*]*?)\s*\*\s*\*\*', r'* **\1**: \2\n  * **', processed_article)
-        processed_article = re.sub(r'(?<!\n)\[\d+[^]]*\]\*\s*\*\*', r'\n\n* **', processed_article)
         lines = processed_article.split('\n')
         result_lines = []
         for i, line in enumerate(lines):
             result_lines.append(line)
-            if (i < len(lines) - 1 and
-                line.strip() and
                 lines[i + 1].strip() and
                 not line.strip().startswith('*') and
                 not lines[i + 1].strip().startswith('*') and
@@ -50,21 +54,63 @@ def make_article_markdown(article: str) -> str:
 {processed_article}"""
-def make_scores_html(overall, comprehensiveness, insight, instruction, readability):
-    scores_data = [
-        ("Overall<br>Score", overall),
-        ("Comprehen-<br>siveness", comprehensiveness),
-        ("Insight<br>Score", insight),
-        ("Instruction<br>Following", instruction),
-        ("Readability<br>Score", readability)
-    ]
     html_items_str = ""
     for title, score in scores_data:
-        score_value = score if score is not None else "N/A"
         html_items_str += f"""
         <div style="text-align: center; padding: 10px 3px; flex-grow: 1; flex-basis: 19%; min-width: 0;">
             <h4 style="margin: 0 0 5px 0; font-size: 1em; color: #4a4a4a; font-weight: 600; line-height: 1.2;">{title}</h4>
-            <p style="margin: 0; font-size: 1.1em; font-weight: bold; color: #333;">{score_value}</p>
         </div>
         """
     return f"""
@@ -74,14 +120,15 @@ def make_scores_html(overall, comprehensiveness, insight, instruction, readabili
     </div>
 </div>"""
 # ---------- 生成 Tab ----------
 def create_data_viewer_side_by_side_tab():
     with gr.Tab("⚔️Side-by-Side Viewer"):
         gr.HTML(
             """<style>
             .card{background:#fff;border:1px solid #e0e0e0;border-radius:8px;padding:22px 24px;margin:18px 0;box-shadow:0 2px 4px rgba(0,0,0,.06);}
-            .scrollable-sm{max-height:180px;overflow-y:auto;} /* 稍微减小任务区高度 */
-            .scrollable-lg{max-height:550px;overflow-y:auto;} /* 调整文章区高度 */
             .card p{color:#424242 !important;line-height:1.75;margin:0 0 14px 0;text-align:justify;}
             .card ul,.card ol{margin:12px 0 12px 24px;color:#424242 !important;}
             .card li{margin:4px 0;color:#424242 !important;}
@@ -112,7 +159,7 @@ def create_data_viewer_side_by_side_tab():
         def fetch_side_by_side_data(selected_task_display, model_a_name, model_b_name):
             empty_article = make_article_markdown("")
-            empty_scores = make_scores_html(None, None, None, None, None)
             if not selected_task_display:
                 no_task_msg = "请选择一个任务。"
@@ -125,10 +172,7 @@ def create_data_viewer_side_by_side_tab():
                 (task.get("prompt", "") for task in index.get("tasks", []) if str(task.get("id")) == item_id_str),
                 "任务描述未找到。"
             )
-            user_task_md_content = make_user_task_markdown(
-                item_id_str,
-                task_prompt
-            )
             outputs_a = [make_article_markdown("模型A未选择或数据未找到"), empty_scores]
             outputs_b = [make_article_markdown("模型B未选择或数据未找到"), empty_scores]
@@ -138,20 +182,14 @@ def create_data_viewer_side_by_side_tab():
             if model_a_name:
                 entry_a = entries.get(model_a_name)
                 if entry_a:
-                    outputs_a[0] = make_article_markdown(entry_a["article"])
-                    outputs_a[1] = make_scores_html(
-                        entry_a["overall_score"], entry_a["comprehensiveness_score"],
-                        entry_a["insight_score"], entry_a["instruction_following_score"],
-                        entry_a["readability_score"])
             if model_b_name:
                 entry_b = entries.get(model_b_name)
                 if entry_b:
-                    outputs_b[0] = make_article_markdown(entry_b["article"])
-                    outputs_b[1] = make_scores_html(
-                        entry_b["overall_score"], entry_b["comprehensiveness_score"],
-                        entry_b["insight_score"], entry_b["instruction_following_score"],
-                        entry_b["readability_score"])
             return user_task_md_content, outputs_a[0], outputs_a[1], outputs_b[0], outputs_b[1]
@@ -164,7 +202,7 @@ def create_data_viewer_side_by_side_tab():
         def on_load():
             index = get_index()
             empty_article = make_article_markdown("")
-            empty_scores = make_scores_html(None, None, None, None, None)
             all_models = index.get("models", [])
             tasks = index.get("tasks", [])
             if not all_models or not tasks:
@@ -195,4 +233,4 @@ def create_data_viewer_side_by_side_tab():
         model_b_dd.change(fetch_side_by_side_data, inputs=[task_dd, model_a_dd, model_b_dd],
                           outputs=[user_task_display_md, article_a_md, scores_a_html, article_b_md, scores_b_html])
-        return on_load, [task_dd, model_a_dd, model_b_dd, user_task_display_md, article_a_md, scores_a_html, article_b_md, scores_b_html]

 #!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 """
+Data-Viewer Side-by-Side tab for ICBCBench.
 """
 import gradio as gr
 from tabs.shared_data import get_entries_for_task, get_index
 def make_user_task_markdown(item_id, prompt):
     return f"""### User Task 🎯
 **Description:** {prompt}"""
 def make_article_markdown(article: str) -> str:
     if article and isinstance(article, str):
         processed_article = re.sub(r'\n{2,}', '\n\n', article)
         table_pattern = r'(\|[^\n]*\n(?:[\|\s\-:]+\n)?(?:\|[^\n]*\n)*)'
         tables = []
         def replace_table(match):
             tables.append(match.group(1))
             return f'__TABLE_PLACEHOLDER_{len(tables)-1}__'
         processed_article = re.sub(table_pattern, replace_table, processed_article)
         processed_article = re.sub(r'(?<!\n)\*\s*\*\*([^*]+?)\*\*:', r'\n\n* **\1**:', processed_article)
         processed_article = re.sub(r'\*\s*\*\*([^*]+?)\*\*:\s*([^*]*?)\s*\*\s*\*\*', r'* **\1**: \2\n  * **', processed_article)
+        processed_article = re.sub(r'(?<!\n)\[\d+[^\]]*\]\*\s*\*\*', r'\n\n* **', processed_article)
         lines = processed_article.split('\n')
         result_lines = []
         for i, line in enumerate(lines):
             result_lines.append(line)
+            if (i < len(lines) - 1 and
+                line.strip() and
                 lines[i + 1].strip() and
                 not line.strip().startswith('*') and
                 not lines[i + 1].strip().startswith('*') and
 {processed_article}"""
+def make_scores_html(entry: dict) -> str:
+    """Build score cards for ICBCBench side-by-side viewer."""
+    track = entry.get("track", "subjective")
+    overall = entry.get("overall_score")
+    objective = entry.get("objective_score")
+    subjective = entry.get("subjective_score")
+    expert = entry.get("expert_score")
+    citation = entry.get("citation_score")
+    source = entry.get("source_quality_score")
+    confidence = entry.get("confidence")
+    correct = entry.get("correct")
+    comp = entry.get("comprehensiveness_score")
+    insight = entry.get("insight_score")
+    inst = entry.get("instruction_following_score")
+    read = entry.get("readability_score")
+    def fmt(val):
+        if val is None:
+            return "N/A"
+        try:
+            return f"{float(val):.2f}"
+        except (TypeError, ValueError):
+            return str(val)
+    if track == "objective":
+        scores_data = [
+            ("Overall<br>Score", fmt(overall)),
+            ("Objective<br>Score", fmt(objective)),
+            ("Confidence", fmt(confidence)),
+            ("Correct", "Yes" if correct is True else ("No" if correct is False else "N/A")),
+        ]
+    else:
+        scores_data = [
+            ("Overall<br>Score", fmt(overall)),
+            ("Subjective<br>Score", fmt(subjective)),
+            ("Expert<br>Score", fmt(expert)),
+            ("Citation", fmt(citation)),
+            ("Source<br>Quality", fmt(source)),
+        ]
+        if subjective is None and any(v is not None for v in [comp, insight, inst, read]):
+            scores_data = [
+                ("Overall<br>Score", fmt(overall)),
+                ("Comprehen-<br>siveness", fmt(comp)),
+                ("Insight<br>Score", fmt(insight)),
+                ("Instruction<br>Following", fmt(inst)),
+                ("Readability<br>Score", fmt(read)),
+            ]
     html_items_str = ""
     for title, score in scores_data:
         html_items_str += f"""
         <div style="text-align: center; padding: 10px 3px; flex-grow: 1; flex-basis: 19%; min-width: 0;">
             <h4 style="margin: 0 0 5px 0; font-size: 1em; color: #4a4a4a; font-weight: 600; line-height: 1.2;">{title}</h4>
+            <p style="margin: 0; font-size: 1.1em; font-weight: bold; color: #333;">{score}</p>
         </div>
         """
     return f"""
     </div>
 </div>"""
 # ---------- 生成 Tab ----------
 def create_data_viewer_side_by_side_tab():
     with gr.Tab("⚔️Side-by-Side Viewer"):
         gr.HTML(
             """<style>
             .card{background:#fff;border:1px solid #e0e0e0;border-radius:8px;padding:22px 24px;margin:18px 0;box-shadow:0 2px 4px rgba(0,0,0,.06);}
+            .scrollable-sm{max-height:180px;overflow-y:auto;}
+            .scrollable-lg{max-height:550px;overflow-y:auto;}
             .card p{color:#424242 !important;line-height:1.75;margin:0 0 14px 0;text-align:justify;}
             .card ul,.card ol{margin:12px 0 12px 24px;color:#424242 !important;}
             .card li{margin:4px 0;color:#424242 !important;}
         def fetch_side_by_side_data(selected_task_display, model_a_name, model_b_name):
             empty_article = make_article_markdown("")
+            empty_scores = make_scores_html({})
             if not selected_task_display:
                 no_task_msg = "请选择一个任务。"
                 (task.get("prompt", "") for task in index.get("tasks", []) if str(task.get("id")) == item_id_str),
                 "任务描述未找到。"
             )
+            user_task_md_content = make_user_task_markdown(item_id_str, task_prompt)
             outputs_a = [make_article_markdown("模型A未选择或数据未找到"), empty_scores]
             outputs_b = [make_article_markdown("模型B未选择或数据未找到"), empty_scores]
             if model_a_name:
                 entry_a = entries.get(model_a_name)
                 if entry_a:
+                    outputs_a[0] = make_article_markdown(entry_a.get("article", ""))
+                    outputs_a[1] = make_scores_html(entry_a)
             if model_b_name:
                 entry_b = entries.get(model_b_name)
                 if entry_b:
+                    outputs_b[0] = make_article_markdown(entry_b.get("article", ""))
+                    outputs_b[1] = make_scores_html(entry_b)
             return user_task_md_content, outputs_a[0], outputs_a[1], outputs_b[0], outputs_b[1]
         def on_load():
             index = get_index()
             empty_article = make_article_markdown("")
+            empty_scores = make_scores_html({})
             all_models = index.get("models", [])
             tasks = index.get("tasks", [])
             if not all_models or not tasks:
         model_b_dd.change(fetch_side_by_side_data, inputs=[task_dd, model_a_dd, model_b_dd],
                           outputs=[user_task_display_md, article_a_md, scores_a_html, article_b_md, scores_b_html])
+        return on_load, [task_dd, model_a_dd, model_b_dd, user_task_display_md, article_a_md, scores_a_html, article_b_md, scores_b_html]

tabs/data_viewer_tab.py CHANGED Viewed

@@ -1,7 +1,7 @@
 #!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 """
-Data-Viewer tab ---- 美化·修正版
 """
 import gradio as gr
@@ -10,6 +10,7 @@ import re
 from tabs.shared_data import get_entry, get_index
 def make_user_task_markdown(item_id, prompt):
     return f"""### User Task 🎯
@@ -17,81 +18,109 @@ def make_user_task_markdown(item_id, prompt):
 **Description:** {prompt}"""
 def make_article_markdown(article: str) -> str:
     if article and isinstance(article, str):
-        # 首先，标准化已经存在的多个换行符
         processed_article = re.sub(r'\n{2,}', '\n\n', article)
-        # 保护表格区域
         table_pattern = r'(\|[^\n]*\n(?:[\|\s\-:]+\n)?(?:\|[^\n]*\n)*)'
         tables = []
         def replace_table(match):
             tables.append(match.group(1))
             return f'__TABLE_PLACEHOLDER_{len(tables)-1}__'
         processed_article = re.sub(table_pattern, replace_table, processed_article)
-        # 处理列表格式：识别 * ** 模式并确保前面有换行
-        # 匹配模式：* **标题:** 内容
         processed_article = re.sub(r'(?<!\n)\*\s*\*\*([^*]+?)\*\*:', r'\n\n* **\1**:', processed_article)
-        # 处理嵌套列表：识别 * ** 后跟 * ** 的模式
         processed_article = re.sub(r'\*\s*\*\*([^*]+?)\*\*:\s*([^*]*?)\s*\*\s*\*\*', r'* **\1**: \2\n  * **', processed_article)
-        # 在引用标记前确保有适当的换行
         processed_article = re.sub(r'(?<!\n)\[\d+[^\]]*\]\*\s*\*\*', r'\n\n* **', processed_article)
-        # 处理其他孤立的换行符（避免破坏我们刚创建的格式）
-        # 但要小心不要影响列表结构
         lines = processed_article.split('\n')
         result_lines = []
         for i, line in enumerate(lines):
             result_lines.append(line)
-            # 如果当前行不为空，下一行也不为空，且都不是列表项，则添加空行
-            if (i < len(lines) - 1 and
-                line.strip() and
                 lines[i + 1].strip() and
                 not line.strip().startswith('*') and
                 not lines[i + 1].strip().startswith('*') and
                 not line.strip().startswith('#')):
-                # 检查是否已经是双换行
                 if i + 1 < len(lines) and lines[i + 1].strip():
-                    result_lines.append('')  # 添加空行
         processed_article = '\n'.join(result_lines)
-        # 恢复表格
         for i, table in enumerate(tables):
             processed_article = processed_article.replace(f'__TABLE_PLACEHOLDER_{i}__', table)
     else:
         processed_article = article if article is not None else ""
     return f"""### Generated Article 📖
 {processed_article}"""
-def make_scores_html(overall, comprehensiveness, insight, instruction, readability):
-    scores_data = [
-        ("Overall Score", overall),
-        ("Comprehensiveness Score", comprehensiveness),
-        ("Insight Score", insight),
-        ("Instruction-Following Score", instruction),
-        ("Readability Score", readability)
-    ]
     html_items_str = ""
     for title, score in scores_data:
-        score_value = score if score is not None else "N/A"
         html_items_str += f"""
         <div style="text-align: center; padding: 8px 5px; flex-grow: 1; flex-basis: 0;">
-            <h4 style="margin: 0 0 6px 0; font-size: 1.2em; color: #4a4a4a; font-weight: 600;">{title}</h4>
-            <p style="margin: 0; font-size: 1.2em; font-weight: bold; color: #333;">{score_value}</p>
         </div>
         """
-    # Outer container styled to mimic the .card class from the main CSS block
     return f"""
 <div style="background:#fff; border:1px solid #e0e0e0; border-radius:8px; padding: 18px 15px; margin:18px 0; box-shadow:0 2px 4px rgba(0,0,0,.06);">
     <div style="display: flex; justify-content: space-between; align-items: flex-start;">
@@ -99,6 +128,7 @@ def make_scores_html(overall, comprehensiveness, insight, instruction, readabili
     </div>
 </div>"""
 # ---------- 生成 Tab ----------
 def create_data_viewer_tab():
     with gr.Tab("🔍Data Viewer"):
@@ -107,7 +137,7 @@ def create_data_viewer_tab():
 <style>
 .card{background:#fff;border:1px solid #e0e0e0;border-radius:8px;padding:22px 24px;margin:18px 0;box-shadow:0 2px 4px rgba(0,0,0,.06);}
 .scrollable-sm{max-height:260px;overflow-y:auto;}
-.scrollable-lg{max-height:700px;overflow-y:auto;} /* 调整高度为分数区域腾出空间 */
 .card p{color:#424242 !important;line-height:1.75;margin:0 0 14px 0;text-align:justify;}
 .card ul,.card ol{margin:12px 0 12px 24px;color:#424242 !important;}
 .card li{margin:4px 0;color:#424242 !important;}
@@ -135,7 +165,7 @@ def create_data_viewer_tab():
             for task in tasks:
                 item_id = str(task["id"])
                 prompt = task.get("prompt", "")
-                limit = 30 if int(item_id) <= 50 else 60
                 preview = prompt[:limit] + ("…" if len(prompt) > limit else "")
                 choices.append(f"{item_id}. {preview}")
             return choices
@@ -151,14 +181,9 @@ def create_data_viewer_tab():
                 err = f"未找到模型 **{model}** 对应任务 **{item_id}** 的内容或分数。"
                 return make_user_task_markdown(item_id, err), make_article_markdown(err), ""
-            prompt = entry["prompt"]
-            article = entry["article"]
-            overall = entry["overall_score"]
-            comprehensiveness = entry["comprehensiveness_score"]
-            insight = entry["insight_score"]
-            instruction = entry["instruction_following_score"]
-            readability = entry["readability_score"]
-            scores_content = make_scores_html(overall, comprehensiveness, insight, instruction, readability)
             return make_user_task_markdown(item_id, prompt), make_article_markdown(article), scores_content
         def on_load():

 #!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 """
+Data-Viewer tab for ICBCBench.
 """
 import gradio as gr
 from tabs.shared_data import get_entry, get_index
 def make_user_task_markdown(item_id, prompt):
     return f"""### User Task 🎯
 **Description:** {prompt}"""
 def make_article_markdown(article: str) -> str:
     if article and isinstance(article, str):
         processed_article = re.sub(r'\n{2,}', '\n\n', article)
         table_pattern = r'(\|[^\n]*\n(?:[\|\s\-:]+\n)?(?:\|[^\n]*\n)*)'
         tables = []
         def replace_table(match):
             tables.append(match.group(1))
             return f'__TABLE_PLACEHOLDER_{len(tables)-1}__'
         processed_article = re.sub(table_pattern, replace_table, processed_article)
         processed_article = re.sub(r'(?<!\n)\*\s*\*\*([^*]+?)\*\*:', r'\n\n* **\1**:', processed_article)
         processed_article = re.sub(r'\*\s*\*\*([^*]+?)\*\*:\s*([^*]*?)\s*\*\s*\*\*', r'* **\1**: \2\n  * **', processed_article)
         processed_article = re.sub(r'(?<!\n)\[\d+[^\]]*\]\*\s*\*\*', r'\n\n* **', processed_article)
         lines = processed_article.split('\n')
         result_lines = []
         for i, line in enumerate(lines):
             result_lines.append(line)
+            if (i < len(lines) - 1 and
+                line.strip() and
                 lines[i + 1].strip() and
                 not line.strip().startswith('*') and
                 not lines[i + 1].strip().startswith('*') and
                 not line.strip().startswith('#')):
                 if i + 1 < len(lines) and lines[i + 1].strip():
+                    result_lines.append('')
         processed_article = '\n'.join(result_lines)
         for i, table in enumerate(tables):
             processed_article = processed_article.replace(f'__TABLE_PLACEHOLDER_{i}__', table)
     else:
         processed_article = article if article is not None else ""
     return f"""### Generated Article 📖
 {processed_article}"""
+def make_scores_html(entry: dict) -> str:
+    """Build score cards for ICBCBench data viewer."""
+    track = entry.get("track", "subjective")
+    # ICBCBench fields
+    overall = entry.get("overall_score")
+    objective = entry.get("objective_score")
+    subjective = entry.get("subjective_score")
+    expert = entry.get("expert_score")
+    citation = entry.get("citation_score")
+    source = entry.get("source_quality_score")
+    confidence = entry.get("confidence")
+    correct = entry.get("correct")
+    # Legacy DeepResearch Bench fields
+    comp = entry.get("comprehensiveness_score")
+    insight = entry.get("insight_score")
+    inst = entry.get("instruction_following_score")
+    read = entry.get("readability_score")
+    def fmt(val):
+        if val is None:
+            return "N/A"
+        try:
+            return f"{float(val):.2f}"
+        except (TypeError, ValueError):
+            return str(val)
+    if track == "objective":
+        scores_data = [
+            ("Overall<br>Score", fmt(overall)),
+            ("Objective<br>Score", fmt(objective)),
+            ("Confidence", fmt(confidence)),
+            ("Correct", "Yes" if correct is True else ("No" if correct is False else "N/A")),
+        ]
+    else:
+        scores_data = [
+            ("Overall<br>Score", fmt(overall)),
+            ("Subjective<br>Score", fmt(subjective)),
+            ("Expert<br>Score", fmt(expert)),
+            ("Citation", fmt(citation)),
+            ("Source<br>Quality", fmt(source)),
+        ]
+        # Add legacy dimensions if ICBCBench fields not available
+        if subjective is None and any(v is not None for v in [comp, insight, inst, read]):
+            scores_data = [
+                ("Overall<br>Score", fmt(overall)),
+                ("Comprehen-<br>siveness", fmt(comp)),
+                ("Insight<br>Score", fmt(insight)),
+                ("Instruction<br>Following", fmt(inst)),
+                ("Readability<br>Score", fmt(read)),
+            ]
     html_items_str = ""
     for title, score in scores_data:
         html_items_str += f"""
         <div style="text-align: center; padding: 8px 5px; flex-grow: 1; flex-basis: 0;">
+            <h4 style="margin: 0 0 6px 0; font-size: 1.1em; color: #4a4a4a; font-weight: 600;">{title}</h4>
+            <p style="margin: 0; font-size: 1.2em; font-weight: bold; color: #333;">{score}</p>
         </div>
         """
     return f"""
 <div style="background:#fff; border:1px solid #e0e0e0; border-radius:8px; padding: 18px 15px; margin:18px 0; box-shadow:0 2px 4px rgba(0,0,0,.06);">
     <div style="display: flex; justify-content: space-between; align-items: flex-start;">
     </div>
 </div>"""
 # ---------- 生成 Tab ----------
 def create_data_viewer_tab():
     with gr.Tab("🔍Data Viewer"):
 <style>
 .card{background:#fff;border:1px solid #e0e0e0;border-radius:8px;padding:22px 24px;margin:18px 0;box-shadow:0 2px 4px rgba(0,0,0,.06);}
 .scrollable-sm{max-height:260px;overflow-y:auto;}
+.scrollable-lg{max-height:700px;overflow-y:auto;}
 .card p{color:#424242 !important;line-height:1.75;margin:0 0 14px 0;text-align:justify;}
 .card ul,.card ol{margin:12px 0 12px 24px;color:#424242 !important;}
 .card li{margin:4px 0;color:#424242 !important;}
             for task in tasks:
                 item_id = str(task["id"])
                 prompt = task.get("prompt", "")
+                limit = 60
                 preview = prompt[:limit] + ("…" if len(prompt) > limit else "")
                 choices.append(f"{item_id}. {preview}")
             return choices
                 err = f"未找到模型 **{model}** 对应任务 **{item_id}** 的内容或分数。"
                 return make_user_task_markdown(item_id, err), make_article_markdown(err), ""
+            prompt = entry.get("prompt", "")
+            article = entry.get("article", "")
+            scores_content = make_scores_html(entry)
             return make_user_task_markdown(item_id, prompt), make_article_markdown(article), scores_content
         def on_load():

tabs/leaderboard_tab.py CHANGED Viewed

@@ -9,294 +9,494 @@ BASE_DIR = Path(__file__).resolve().parent.parent
 DATA_PATH = BASE_DIR / "data" / "leaderboard.csv"
 # 用于标注的常量
-CATEGORY_TO_HIGHLIGHT = "Deep Research Agent"
 HIGHLIGHT_EMOJI = "🚀"
-# 列名重命名映射
 COLUMN_RENAME_MAP = {
-    'overall_score': 'overall',
-    'comprehensiveness': 'comp.',
-    'insight': 'insight',
-    'instruction_following': 'inst.',
-    'readability': 'read.',
-    'citation_accuracy': 'c.acc.',
-    'effective_citations': 'eff.c.'
 }
 MODEL_DISPLAY_NAMES = {
-    "gensee-search-gpt-5": "langchain-open-deep-research(GPT-5,with gensee search)",
-    "langchain-open-deep-research-gpt-5": "langchain-open-deep-research(GPT-5,with Tavily)",
-    "qianfan_deepresearch_0430": "DuMate/Qianfan-DeepResearch",
     "onyx": "Onyx Deep Research",
-    "raaa-deep-research": "UESTC-MBSE-RAAA-DeepResearch",
-    "deepsynth": "Bodhi Deep Research",
-    "xiaoyi": "Xiaoyi DeepResearch 6.0",
-    "nvidia-aiq-nemotron-gpt52-updated": "nvidia-aiq (Nemotron 3, GPT 5.2)",
-    "deepinsight": "CMCC-DeepInsight",
     "drb_cellcog": "Cellcog",
-    "cellcog-max": "Cellcog Max",
     "RecallRadar": "RecallRadar Intelligence",
-    "ms_deepresearch": "MS-Agent Agentic Insight v2\uff08Qwen3.5-Plus\u3001GPT 5\uff09",
-    "TrajectoryKit": "TrajectoryKit (GPT-OSS, GPT5.4)",
-    "1688AILab-DeepResearch-0428": "1688AILab-DeepResearch",
     "zhipu_deep_research": "Zhipu Deep Research",
-    "grep-v5": "Grep Deep Research",
-    "MindDR-V1.5": "LiAuto Mind DeepResearch 1.5",
-    "deepdog": "Deep Dog 1",
-    "Link": "iFlow-Researcher",
     "octen-deepresearch-0508": "Octen Deep Research",
-    "ZTE-Nebula-DeepResearch-V20260519": "ZTE-Nebula-DeepResearch",
-    "WhaleCloud-DocChain": "WhaleCloud-DocChain"
 }
 # 模型分类映射
 MODEL_CATEGORIES = {
-    "Deep Research Agent": [
-        "gemini-2.5-pro-deepresearch",
         "openai-deepresearch",
-        "gensee-search-gpt-5",
         "langchain-open-deep-research-gpt-5",
-        "cellcog",
         "salesforce-air-deep-research",
         "thinkdepthai-deepresearch",
         "tavily-research",
-        "qianfan_deepresearch_0430",
         "onyx",
         "raaa-deep-research",
         "deepsynth",
-        "xiaoyi",
         "nvidia-aiq-nemotron-gpt52-updated",
         "deepinsight",
         "drb_cellcog",
-        "cellcog-max",
         "RecallRadar",
         "ms_deepresearch",
         "TrajectoryKit",
-        "1688AILab-DeepResearch-0428",
         "zhipu_deep_research",
-        "ms_deepresearch_gpt52mixqwen35_09_edit_restart09_think_medium",
-        "grep-v5",
         "MindDR-V1.5",
         "deepdog",
         "Link",
         "octen-deepresearch-0508",
         "ZTE-Nebula-DeepResearch-V20260519",
-        "WhaleCloud-DocChain"
     ],
-    "LLM with Search": [
-    ]
 }
-# 模型链接映射（目前都设置为空，可以后续添加具体链接）
 MODEL_LINKS = {
-    # Deep Research Agent
     "gemini-2.5-pro-deepresearch": "https://gemini.google/overview/deep-research/",
-    "openai-deepresearch": "https://openai.com/zh-Hans-CN/index/introducing-deep-research/",
-    "gensee-search-gpt-5": "https://github.com/GenseeAI/open_deep_research",
     "langchain-open-deep-research-gpt-5": "https://github.com/langchain-ai/open_deep_research",
-    "cellcog": "https://www.cellcog.ai/",
-    "salesforce-air-deep-research": "https://github.com/SalesforceAIResearch/enterprise-deep-research ",
     "thinkdepthai-deepresearch": "https://github.com/thinkdepthai/Deep_Research",
     "tavily-research": "https://deepresearch.tavily.com",
-    "qianfan_deepresearch_0430": "https://console.bce.baidu.com/qianfan/studio/officialApp/deepResearch/69a72b3f-a18c-2f02-9ba0-007beb95b315",
     "onyx": "https://github.com/onyx-dot-app/onyx",
-    "raaa-deep-research": "https://github.com/wee235929-cmyk/RequirementAgent",
     "deepsynth": "https://www.publicissapient.com/platforms/bodhi",
-    "xiaoyi": "https://xiaoyi.huawei.com/chat/research",
-    "nvidia-aiq-nemotron-gpt52-updated": "https://github.com/NVIDIA-AI-Blueprints/aiq/tree/drb1",
-    "deepinsight": "http://81.70.174.140:3000/",
-    "drb_cellcog": "https://www.cellcog.ai/",
-    "cellcog-max": "https://www.cellcog.ai/",
-    "RecallRadar": "https://getrecallradar.com",
     "ms_deepresearch": "https://github.com/modelscope/ms-agent",
     "TrajectoryKit": "https://github.com/KabakaWilliam/trajectorykit",
-    "1688AILab-DeepResearch-0428": "https://air.1688.com/kapp/1688-ai-app/pages/home",
     "zhipu_deep_research": "https://research-hb.zhipuai-infra.cn/",
-    "ms_deepresearch_gpt52mixqwen35_09_edit_restart09_think_medium": "https://github.com/modelscope/ms-agent",
-    "grep-v5": "https://grep.ai",
     "MindDR-V1.5": "https://www.lixiang.com/tech/mindgpt",
     "deepdog": "https://github.com/beneadie/DeepDog_1",
     "Link": "https://iflow.cn/public/iflow-researcher.html",
-    "octen-deepresearch-0508": "https://octen.ai/platform/deep-research",
     "ZTE-Nebula-DeepResearch-V20260519": "https://github.com/Adlik/ZTE-Nebula-DeepResearch",
     "WhaleCloud-DocChain": "https://lab.hjcloud.com/llmdoc/login?redirect=%2Fkagent",
-    # LLM with Search
 }
 # 模型许可证类型映射
 MODEL_LICENSE_TYPE = {
-    # Deep Research Agent
-    "gemini-2.5-pro-deepresearch": "Proprietary",
     "openai-deepresearch": "Proprietary",
-    "gensee-search-gpt-5": "MIT",
     "langchain-open-deep-research-gpt-5": "MIT",
-    "cellcog": "Proprietary",
-    "salesforce-air-deep-research": "Apache-2.0 license",
     "thinkdepthai-deepresearch": "MIT",
     "tavily-research": "Proprietary",
-    "qianfan_deepresearch_0430": "Proprietary",
     "onyx": "MIT (partial EE)",
     "raaa-deep-research": "Proprietary",
     "deepsynth": "Proprietary",
-    "xiaoyi": "Proprietary",
-    "nvidia-aiq-nemotron-gpt52-updated": "Apache-2.0 license",
     "deepinsight": "Proprietary",
     "drb_cellcog": "Proprietary",
-    "cellcog-max": "Proprietary",
     "RecallRadar": "Proprietary",
-    "ms_deepresearch": "Apache-2.0 license",
     "TrajectoryKit": "MIT",
-    "1688AILab-DeepResearch-0428": "Proprietary",
     "zhipu_deep_research": "Proprietary",
-    "ms_deepresearch_gpt52mixqwen35_09_edit_restart09_think_medium": "Apache-2.0 license",
-    "grep-v5": "Proprietary",
     "MindDR-V1.5": "Proprietary",
     "deepdog": "MIT",
     "Link": "TBD",
     "octen-deepresearch-0508": "Proprietary",
     "ZTE-Nebula-DeepResearch-V20260519": "Proprietary",
     "WhaleCloud-DocChain": "Proprietary",
-    # LLM with Search
 }
 def load_leaderboard() -> pd.DataFrame:
     if not DATA_PATH.exists():
         raise FileNotFoundError(
             f"Leaderboard file not found: {DATA_PATH}.\n"
-            "→ 先运行 rank_leaderboard.py 生成 data/leaderboard.csv"
         )
     df = pd.read_csv(DATA_PATH)
     df.columns = [c.strip() for c in df.columns]
     def get_category(model_name):
         for category, models in MODEL_CATEGORIES.items():
             if model_name in models:
                 return category
         return "Others"
     def get_license_type(model_name):
-        return MODEL_LICENSE_TYPE.get(model_name, "Unknown")
     df['category'] = df['model'].apply(get_category)
     df['license_type'] = df['model'].apply(get_license_type)
     return df
 def make_ranked(df: pd.DataFrame) -> pd.DataFrame:
-    ranked = df.sort_values(by=['overall_score', 'comprehensiveness', 'insight', 'instruction_following', 'readability'], ascending=False).reset_index(drop=True)
-    # 使用 min 方法处理并列排名：同分同名次，下一名跳过（如 1,1,3）
-    ranked.insert(0, "Rank", ranked['overall_score'].rank(method='min', ascending=False).astype(int))
     medal_styles = {
-        1: {'icon': '🥇', 'color': '#FFD700'},  # 金色
-        2: {'icon': '🥈', 'color': '#C0C0C0'},  # 暗银色
-        3: {'icon': '🥉', 'color': '#CD7F32'}   # 铜色
     }
     def format_rank_with_medal(rank_num):
         if rank_num in medal_styles:
             style = medal_styles[rank_num]
-            # 数字加粗+颜色，奖牌图标放大
             return (
                 f'<span style="font-weight: bold; color: {style["color"]};">{rank_num}</span> '
                 f'<span style="font-size: 1.5em;">{style["icon"]}</span>'
             )
-        else:
-            return str(rank_num)
     ranked['Rank'] = ranked['Rank'].apply(format_rank_with_medal)
     # 重命名列名为简写形式
     ranked = ranked.rename(columns=COLUMN_RENAME_MAP)
-    # 格式化数值列为两位小数，但跳过包含"-"的值
-    numeric_columns = ['overall', 'comp.', 'insight', 'inst.', 'read.', 'c.acc.', 'eff.c.']
     for col in numeric_columns:
         if col in ranked.columns:
-            # 只对数值进行round操作，保持"-"不变
             ranked[col] = ranked[col].apply(
                 lambda x: round(float(x), 2) if x != "-" and pd.notna(x) else x
             )
     # 为模型添加链接和高亮样式
     def format_model_name(row):
         model_name = row['model']
-        display_model_name = MODEL_DISPLAY_NAMES.get(model_name, model_name)
         link = MODEL_LINKS.get(model_name, "")
-        # 根据类别决定是否高亮
         if row['category'] == CATEGORY_TO_HIGHLIGHT:
-            display_name = f'<span style="color: #823AFF;">{HIGHLIGHT_EMOJI} {display_model_name}</span>'
         else:
             display_name = display_model_name
-        # 如果有链接，包装成<a>标签
         if link and link.strip():
             return f'<a href="{link}" target="_blank" style="text-decoration: none;">{display_name}</a>'
-        else:
-            # 没有链接时，为将来添加链接做准备（可以添加点击事件等）
-            return f'<span class="model-name" data-model="{model_name}">{display_name}</span>'
     ranked['model'] = ranked.apply(format_model_name, axis=1)
     return ranked
 def filter_data(search_text: str, selected_categories: list):
     df = load_leaderboard()
     if search_text.strip():
         df = df[df['model'].str.contains(search_text.strip(), case=False, na=False)]
     if selected_categories:
         df = df[df['category'].isin(selected_categories)]
-    ranked_df = make_ranked(df)
-    return ranked_df
 def create_leaderboard_tab():
-    with gr.Tab("🏆Leaderboard (Gemini-2.5 Eval)"):
         with gr.Row():
             with gr.Column(scale=1):
                 search_box = gr.Textbox(
-                    label="Model Search",
                     placeholder="Entering model name to search...",
                     value=""
                 )
             with gr.Column(scale=2):
                 category_checkboxes = gr.CheckboxGroup(
                     label="Model Categories",
-                    choices=list(MODEL_CATEGORIES.keys()),
-                    value=list(MODEL_CATEGORIES.keys())
                 )
-        # 初始化数据（不使用样式）
-        initial_df = filter_data("", list(MODEL_CATEGORIES.keys()))
-        # 获取列数据类型，将 model 列和 Rank 列设置为 html
         column_count = len(initial_df.columns)
         datatypes = ["str"] * column_count
         model_col_index = initial_df.columns.get_loc('model')
         rank_col_index = initial_df.columns.get_loc('Rank')
         datatypes[model_col_index] = "html"
-        datatypes[rank_col_index] = "html"  # Rank 列也需要渲染 HTML
-        # 创建 Dataframe 组件
         table = gr.Dataframe(
             value=initial_df,
-            datatype=datatypes,  # 设置数据类型，model 列为 html
-            max_height=600,  # 设置表格最大高度
-            show_label=False,  # 不显示标签
-            elem_id="leaderboard_table",  # 添加元素ID
-            interactive=False,  # 禁用编辑功能
-            wrap=True,  # 启用自动换行，超长文本会自动换行显示
-            column_widths=["80px", "380px", "100px", "100px", "100px", "100px", "100px", "100px", "100px", "180px", "150px"]  # 设置各列宽度，model列设置为450px以完整显示长名称
         )
         def update_display(search_text, selected_categories):
-            df = filter_data(search_text, selected_categories)
-            return df
-        # 绑定搜索框和复选框的变化事件
         search_box.change(
             fn=update_display,
             inputs=[search_box, category_checkboxes],
@@ -307,24 +507,26 @@ def create_leaderboard_tab():
             inputs=[search_box, category_checkboxes],
             outputs=table
         )
         # 在底部添加说明
         with gr.Row():
             gr.Markdown(f"""
             ### 📊 Column Descriptions
             - **Rank**: Model ranking based on overall score
-            - **model**: Model name (<span style="color: #823AFF;">{HIGHLIGHT_EMOJI} = {CATEGORY_TO_HIGHLIGHT}</span>)
-            - **overall**: Overall Score (weighted average of all metrics)
-            - **comp.**: Comprehensiveness - How thorough and complete the research is
-            - **insight**: Insight Quality - Depth and value of analysis
-            - **inst.**: Instruction Following - Adherence to user instructions
-            - **read.**: Readability - Clarity and organization of content
-            - **c.acc.**: Citation Accuracy - Correctness of references
-            - **eff.c.**: Effective Citations - Relevance and quality of sources
             - **category**: Model category
             - **license_type**: The software license type of the model/service
-            💡 **Tip**: Model names are clickable when links are available. Visit the GitHub repositories for more details!
             """)
     return search_box

 DATA_PATH = BASE_DIR / "data" / "leaderboard.csv"
 # 用于标注的常量
+CATEGORY_TO_HIGHLIGHT = "Closed-source DRA"
 HIGHLIGHT_EMOJI = "🚀"
+# 列名重命名映射（CSV column -> display column）
 COLUMN_RENAME_MAP = {
+    'overall': 'overall',
+    'objective_en': 'obj.EN',
+    'objective_zh': 'obj.ZH',
+    'objective_avg': 'obj.avg',
+    'subjective_en': 'subj.EN',
+    'subjective_zh': 'subj.ZH',
+    'subjective_avg': 'subj.avg',
+    'expert_avg': 'expert',
+    'citation_score': 'citation',
+    'source_quality': 'source',
+    'rmsce': 'RMSCE',
 }
+# 模型显示名称（ICBCBench 参赛模型）
 MODEL_DISPLAY_NAMES = {
+    "deerflow-gpt55": "DeerFlow (+GPT-5.5)",
+    "DeerFlow(+GPT-5.5)": "DeerFlow (+GPT-5.5)",
+    "openclaw-gpt55": "OpenClaw (+GPT-5.5)",
+    "OpenClaw(+GPT-5.5)": "OpenClaw (+GPT-5.5)",
+    "openclaw-deepseek-v4-pro": "OpenClaw (+DeepSeek-V4-Pro)",
+    "OpenClaw(+DeepSeek-V4-Pro)": "OpenClaw (+DeepSeek-V4-Pro)",
+    "deerflow-deepseek-v4-pro": "DeerFlow (+DeepSeek-V4-Pro)",
+    "DeerFlow(+DeepSeek-V4-Pro)": "DeerFlow (+DeepSeek-V4-Pro)",
+    "gemini-deep-research": "Gemini Deep Research",
+    "Gemini-deep-research": "Gemini Deep Research",
+    "openai-o3-deep-research": "OpenAI o3 Deep Research",
+    "OpenAI-o3-deep-research": "OpenAI o3 Deep Research",
+    "kimi-deep-research": "Kimi Deep Research",
+    "doubao-deep-research": "Doubao Deep Research",
+    "gpt-5.5": "GPT-5.5",
+    "GPT-5.5": "GPT-5.5",
+    "claude-opus-4-7": "Claude Opus 4.7",
+    "Claude-opus-4-7": "Claude Opus 4.7",
+    "perplexity-deep-research": "Perplexity Deep Research",
+    "Perplexity-deep-research": "Perplexity Deep Research",
+    "gemini-3.1-pro-preview": "Gemini 3.1 Pro Preview",
+    "Gemini-3.1-pro-preview": "Gemini 3.1 Pro Preview",
+    "grok-3-deepsearch": "Grok 3 DeepSearch",
+    "Grok-3-deepsearch": "Grok 3 DeepSearch",
+    "qwen-deep-research": "Qwen Deep Research",
+    "Qwen-deep-research": "Qwen Deep Research",
+    "openai-deepresearch": "OpenAI Deep Research",
+    "gemini-2.5-pro-deepresearch": "Gemini 2.5 Pro Deep Research",
+    "perplexity-Research": "Perplexity Research",
+    "grok-deeper-search": "Grok Deeper Search",
+    "mirothinker": "MiroThinker",
+    "MiroThinker": "MiroThinker",
+    "jina-deepsearch": "Jina DeepSearch",
+    "Jina-deepsearch": "Jina DeepSearch",
+    "kimi-k2.5": "Kimi K2.5",
+    "Kimi-k2.5": "Kimi K2.5",
+    "deepseek-v4-pro": "DeepSeek V4 Pro",
+    "DeepSeek-V4-Pro": "DeepSeek V4 Pro",
+    "tongyi-deepresearch-30b-a3b": "Tongyi Deep Research 30B-A3B",
+    "Tongyi-deepresearch-30b-a3b": "Tongyi Deep Research 30B-A3B",
+    "langchain-open-deep-research": "LangChain Open Deep Research",
+    "langchain-open-deep-research-gpt-5": "LangChain Open Deep Research (GPT-5)",
+    "salesforce-air-deep-research": "Salesforce AIR Deep Research",
+    "thinkdepthai-deepresearch": "ThinkDepthAI Deep Research",
+    "tavily-research": "Tavily Research",
     "onyx": "Onyx Deep Research",
+    "raaa-deep-research": "RAAA Deep Research",
+    "deepsynth": "Deepsynth / Bodhi",
+    "xiaoyi_research_agent_0304": "Xiaoyi Deep Research",
+    "nvidia-aiq-nemotron-gpt52-updated": "NVIDIA AIQ (Nemotron)",
+    "deepinsight": "CMCC DeepInsight",
     "drb_cellcog": "Cellcog",
+    "drb_cellcog_max": "Cellcog Max",
     "RecallRadar": "RecallRadar Intelligence",
+    "ms_deepresearch": "MS-Agent Agentic Insight",
+    "TrajectoryKit": "TrajectoryKit",
+    "1688AILab-DeepResearch-0325": "1688AI Lab Deep Research",
     "zhipu_deep_research": "Zhipu Deep Research",
+    "grep-v4": "Grep Deep Research",
+    "MindDR-V1.5": "LiAuto Mind DeepResearch",
+    "deepdog": "Deep Dog",
+    "Link": "iFlow Researcher",
     "octen-deepresearch-0508": "Octen Deep Research",
+    "ZTE-Nebula-DeepResearch-V20260519": "ZTE Nebula DeepResearch",
+    "WhaleCloud-DocChain": "WhaleCloud DocChain",
+    "qianfan_deepresearch_0430": "Baidu Qianfan DeepResearch",
+    "doubao-deepresearch": "Doubao Deep Research",
+    "kimi-researcher": "Kimi Researcher",
+    "claude-research": "Claude Research",
+    "dr-tulu": "DR-Tulu",
+    "nvidia-aiq-research-assistant": "NVIDIA AIQ Research Assistant",
+    "tongyi-deepresearch-30B-A3B": "Tongyi Deep Research",
+    "sonar-reasoning-pro": "Sonar Reasoning Pro",
+    "sonar-reasoning": "Sonar Reasoning",
+    "sonar-pro": "Sonar Pro",
+    "claude-3-7-sonnet-with-search": "Claude 3.7 Sonnet with Search",
+    "gpt-4o-search-preview": "GPT-4o Search Preview",
 }
 # 模型分类映射
 MODEL_CATEGORIES = {
+    "Closed-source DRA": [
+        "gemini-deep-research",
+        "Gemini-deep-research",
+        "openai-o3-deep-research",
+        "OpenAI-o3-deep-research",
+        "kimi-deep-research",
+        "Kimi-deep-research",
+        "doubao-deep-research",
+        "Doubao-deep-research",
+        "gpt-5.5",
+        "GPT-5.5",
+        "claude-opus-4-7",
+        "Claude-opus-4-7",
+        "perplexity-deep-research",
+        "Perplexity-deep-research",
+        "gemini-3.1-pro-preview",
+        "Gemini-3.1-pro-preview",
+        "grok-3-deepsearch",
+        "Grok-3-deepsearch",
+        "qwen-deep-research",
+        "Qwen-deep-research",
+        # Legacy DeepResearch Bench models
         "openai-deepresearch",
+        "gemini-2.5-pro-deepresearch",
+        "perplexity-Research",
+        "grok-deeper-search",
+        "Grok-deeper-search",
+        "claude-3-7-sonnet-with-search",
+        "gpt-4o-search-preview",
+    ],
+    "Open Agentic Framework": [
+        "deerflow-gpt55",
+        "DeerFlow(+GPT-5.5)",
+        "openclaw-gpt55",
+        "OpenClaw(+GPT-5.5)",
+        "openclaw-deepseek-v4-pro",
+        "OpenClaw(+DeepSeek-V4-Pro)",
+        "deerflow-deepseek-v4-pro",
+        "DeerFlow(+DeepSeek-V4-Pro)",
+        "mirothinker",
+        "MiroThinker",
+        "jina-deepsearch",
+        "Jina-deepsearch",
+        # Legacy DeepResearch Bench models
+        "langchain-open-deep-research",
         "langchain-open-deep-research-gpt-5",
         "salesforce-air-deep-research",
         "thinkdepthai-deepresearch",
         "tavily-research",
         "onyx",
         "raaa-deep-research",
         "deepsynth",
+        "xiaoyi_research_agent_0304",
         "nvidia-aiq-nemotron-gpt52-updated",
         "deepinsight",
         "drb_cellcog",
+        "drb_cellcog_max",
         "RecallRadar",
         "ms_deepresearch",
         "TrajectoryKit",
+        "1688AILab-DeepResearch-0325",
         "zhipu_deep_research",
+        "grep-v4",
         "MindDR-V1.5",
         "deepdog",
         "Link",
         "octen-deepresearch-0508",
         "ZTE-Nebula-DeepResearch-V20260519",
+        "WhaleCloud-DocChain",
+        "qianfan_deepresearch_0430",
+    ],
+    "Open-weight LLM": [
+        "kimi-k2.5",
+        "Kimi-k2.5",
+        "deepseek-v4-pro",
+        "DeepSeek-V4-Pro",
+        "tongyi-deepresearch-30b-a3b",
+        "Tongyi-deepresearch-30b-a3b",
+        # Legacy DeepResearch Bench models
+        "doubao-deepresearch",
+        "kimi-researcher",
+        "claude-research",
+        "dr-tulu",
+        "nvidia-aiq-research-assistant",
+        "tongyi-deepresearch-30B-A3B",
+        "sonar-reasoning-pro",
+        "sonar-reasoning",
+        "sonar-pro",
+        "claude-3-7-sonnet-with-search",
+        "gpt-4o-search-preview",
     ],
 }
+# 模型链接映射
 MODEL_LINKS = {
+    "gemini-deep-research": "https://gemini.google/overview/deep-research/",
+    "Gemini-deep-research": "https://gemini.google/overview/deep-research/",
+    "openai-o3-deep-research": "https://openai.com/index/introducing-deep-research/",
+    "OpenAI-o3-deep-research": "https://openai.com/index/introducing-deep-research/",
+    "kimi-deep-research": "https://kimi.moonshot.cn/",
+    "Kimi-deep-research": "https://kimi.moonshot.cn/",
+    "doubao-deep-research": "https://www.doubao.com/chat/",
+    "Doubao-deep-research": "https://www.doubao.com/chat/",
+    "gpt-5.5": "https://openai.com/",
+    "GPT-5.5": "https://openai.com/",
+    "claude-opus-4-7": "https://www.anthropic.com/",
+    "Claude-opus-4-7": "https://www.anthropic.com/",
+    "perplexity-deep-research": "https://www.perplexity.ai/",
+    "Perplexity-deep-research": "https://www.perplexity.ai/",
+    "gemini-3.1-pro-preview": "https://gemini.google/",
+    "Gemini-3.1-pro-preview": "https://gemini.google/",
+    "grok-3-deepsearch": "https://x.ai/grok",
+    "Grok-3-deepsearch": "https://x.ai/grok",
+    "qwen-deep-research": "https://tongyi.aliyun.com/",
+    "Qwen-deep-research": "https://tongyi.aliyun.com/",
+    "openai-deepresearch": "https://openai.com/index/introducing-deep-research/",
     "gemini-2.5-pro-deepresearch": "https://gemini.google/overview/deep-research/",
+    "perplexity-Research": "https://www.perplexity.ai/",
+    "grok-deeper-search": "https://x.ai/grok",
+    "mirothinker": "https://miro.com/",
+    "MiroThinker": "https://miro.com/",
+    "jina-deepsearch": "https://jina.ai/",
+    "Jina-deepsearch": "https://jina.ai/",
+    "kimi-k2.5": "https://kimi.moonshot.cn/",
+    "Kimi-k2.5": "https://kimi.moonshot.cn/",
+    "deepseek-v4-pro": "https://www.deepseek.com/",
+    "DeepSeek-V4-Pro": "https://www.deepseek.com/",
+    "tongyi-deepresearch-30b-a3b": "https://tongyi.aliyun.com/",
+    "Tongyi-deepresearch-30b-a3b": "https://tongyi.aliyun.com/",
+    "langchain-open-deep-research": "https://github.com/langchain-ai/open_deep_research",
     "langchain-open-deep-research-gpt-5": "https://github.com/langchain-ai/open_deep_research",
+    "salesforce-air-deep-research": "https://github.com/SalesforceAIResearch/enterprise-deep-research",
     "thinkdepthai-deepresearch": "https://github.com/thinkdepthai/Deep_Research",
     "tavily-research": "https://deepresearch.tavily.com",
     "onyx": "https://github.com/onyx-dot-app/onyx",
     "deepsynth": "https://www.publicissapient.com/platforms/bodhi",
     "ms_deepresearch": "https://github.com/modelscope/ms-agent",
     "TrajectoryKit": "https://github.com/KabakaWilliam/trajectorykit",
     "zhipu_deep_research": "https://research-hb.zhipuai-infra.cn/",
     "MindDR-V1.5": "https://www.lixiang.com/tech/mindgpt",
     "deepdog": "https://github.com/beneadie/DeepDog_1",
     "Link": "https://iflow.cn/public/iflow-researcher.html",
     "ZTE-Nebula-DeepResearch-V20260519": "https://github.com/Adlik/ZTE-Nebula-DeepResearch",
     "WhaleCloud-DocChain": "https://lab.hjcloud.com/llmdoc/login?redirect=%2Fkagent",
+    "qianfan_deepresearch_0430": "https://console.bce.baidu.com/qianfan/studio/officialApp/deepResearch/69a72b3f-a18c-2f02-9ba0-007beb95b315",
+    "doubao-deepresearch": "https://www.doubao.com/chat/",
+    "kimi-researcher": "https://kimi.moonshot.cn/",
+    "claude-research": "https://www.anthropic.com/",
+    "tongyi-deepresearch-30B-A3B": "https://tongyi.aliyun.com/",
+    "sonar-reasoning-pro": "https://perplexity.ai/",
+    "sonar-reasoning": "https://perplexity.ai/",
+    "sonar-pro": "https://perplexity.ai/",
+    "claude-3-7-sonnet-with-search": "https://www.anthropic.com/",
+    "gpt-4o-search-preview": "https://openai.com/",
 }
 # 模型许可证类型映射
 MODEL_LICENSE_TYPE = {
+    "gemini-deep-research": "Proprietary",
+    "Gemini-deep-research": "Proprietary",
+    "openai-o3-deep-research": "Proprietary",
+    "OpenAI-o3-deep-research": "Proprietary",
+    "kimi-deep-research": "Proprietary",
+    "Kimi-deep-research": "Proprietary",
+    "doubao-deep-research": "Proprietary",
+    "Doubao-deep-research": "Proprietary",
+    "gpt-5.5": "Proprietary",
+    "GPT-5.5": "Proprietary",
+    "claude-opus-4-7": "Proprietary",
+    "Claude-opus-4-7": "Proprietary",
+    "perplexity-deep-research": "Proprietary",
+    "Perplexity-deep-research": "Proprietary",
+    "gemini-3.1-pro-preview": "Proprietary",
+    "Gemini-3.1-pro-preview": "Proprietary",
+    "grok-3-deepsearch": "Proprietary",
+    "Grok-3-deepsearch": "Proprietary",
+    "qwen-deep-research": "Proprietary",
+    "Qwen-deep-research": "Proprietary",
+    "deerflow-gpt55": "Apache-2.0",
+    "DeerFlow(+GPT-5.5)": "Apache-2.0",
+    "openclaw-gpt55": "Apache-2.0",
+    "OpenClaw(+GPT-5.5)": "Apache-2.0",
+    "openclaw-deepseek-v4-pro": "Apache-2.0",
+    "OpenClaw(+DeepSeek-V4-Pro)": "Apache-2.0",
+    "deerflow-deepseek-v4-pro": "Apache-2.0",
+    "DeerFlow(+DeepSeek-V4-Pro)": "Apache-2.0",
+    "mirothinker": "Apache-2.0",
+    "MiroThinker": "Apache-2.0",
+    "jina-deepsearch": "Apache-2.0",
+    "Jina-deepsearch": "Apache-2.0",
+    "kimi-k2.5": "Proprietary",
+    "Kimi-k2.5": "Proprietary",
+    "deepseek-v4-pro": "MIT",
+    "DeepSeek-V4-Pro": "MIT",
+    "tongyi-deepresearch-30b-a3b": "Proprietary",
+    "Tongyi-deepresearch-30b-a3b": "Proprietary",
     "openai-deepresearch": "Proprietary",
+    "gemini-2.5-pro-deepresearch": "Proprietary",
+    "perplexity-Research": "Proprietary",
+    "grok-deeper-search": "Proprietary",
+    "langchain-open-deep-research": "MIT",
     "langchain-open-deep-research-gpt-5": "MIT",
+    "salesforce-air-deep-research": "Apache-2.0",
     "thinkdepthai-deepresearch": "MIT",
     "tavily-research": "Proprietary",
     "onyx": "MIT (partial EE)",
     "raaa-deep-research": "Proprietary",
     "deepsynth": "Proprietary",
+    "xiaoyi_research_agent_0304": "Proprietary",
+    "nvidia-aiq-nemotron-gpt52-updated": "Apache-2.0",
     "deepinsight": "Proprietary",
     "drb_cellcog": "Proprietary",
+    "drb_cellcog_max": "Proprietary",
     "RecallRadar": "Proprietary",
+    "ms_deepresearch": "Apache-2.0",
     "TrajectoryKit": "MIT",
+    "1688AILab-DeepResearch-0325": "Proprietary",
     "zhipu_deep_research": "Proprietary",
+    "grep-v4": "Proprietary",
     "MindDR-V1.5": "Proprietary",
     "deepdog": "MIT",
     "Link": "TBD",
     "octen-deepresearch-0508": "Proprietary",
     "ZTE-Nebula-DeepResearch-V20260519": "Proprietary",
     "WhaleCloud-DocChain": "Proprietary",
+    "qianfan_deepresearch_0430": "Proprietary",
+    "doubao-deepresearch": "Proprietary",
+    "kimi-researcher": "Proprietary",
+    "claude-research": "Proprietary",
+    "dr-tulu": "Proprietary",
+    "nvidia-aiq-research-assistant": "Apache-2.0",
+    "tongyi-deepresearch-30B-A3B": "Proprietary",
+    "sonar-reasoning-pro": "Proprietary",
+    "sonar-reasoning": "Proprietary",
+    "sonar-pro": "Proprietary",
+    "claude-3-7-sonnet-with-search": "Proprietary",
+    "gpt-4o-search-preview": "Proprietary",
 }
 def load_leaderboard() -> pd.DataFrame:
     if not DATA_PATH.exists():
         raise FileNotFoundError(
             f"Leaderboard file not found: {DATA_PATH}.\n"
+            "→ 先运行 utils/rank_leaderboard.py 生成 data/leaderboard.csv"
         )
     df = pd.read_csv(DATA_PATH)
     df.columns = [c.strip() for c in df.columns]
     def get_category(model_name):
         for category, models in MODEL_CATEGORIES.items():
             if model_name in models:
                 return category
+        # Fallback to case-insensitive matching
+        for category, models in MODEL_CATEGORIES.items():
+            if model_name.lower() in {m.lower() for m in models}:
+                return category
         return "Others"
     def get_license_type(model_name):
+        lic = MODEL_LICENSE_TYPE.get(model_name)
+        if lic is None:
+            lic = MODEL_LICENSE_TYPE.get(model_name.lower())
+        return lic if lic is not None else "Unknown"
     df['category'] = df['model'].apply(get_category)
     df['license_type'] = df['model'].apply(get_license_type)
     return df
 def make_ranked(df: pd.DataFrame) -> pd.DataFrame:
+    tiebreak_cols = ['objective_avg', 'subjective_avg', 'objective_en', 'subjective_en']
+    sort_cols = ['overall'] + [c for c in tiebreak_cols if c in df.columns]
+    ranked = df.sort_values(by=sort_cols, ascending=False).reset_index(drop=True)
+    ranked.insert(0, "Rank", ranked['overall'].rank(method='min', ascending=False).astype(int))
     medal_styles = {
+        1: {'icon': '🥇', 'color': '#FFD700'},
+        2: {'icon': '🥈', 'color': '#C0C0C0'},
+        3: {'icon': '🥉', 'color': '#CD7F32'}
     }
     def format_rank_with_medal(rank_num):
         if rank_num in medal_styles:
             style = medal_styles[rank_num]
             return (
                 f'<span style="font-weight: bold; color: {style["color"]};">{rank_num}</span> '
                 f'<span style="font-size: 1.5em;">{style["icon"]}</span>'
             )
+        return str(rank_num)
     ranked['Rank'] = ranked['Rank'].apply(format_rank_with_medal)
     # 重命名列名为简写形式
     ranked = ranked.rename(columns=COLUMN_RENAME_MAP)
+    # 格式化数值列为两位小数，跳过包含"-"的值
+    numeric_columns = list(COLUMN_RENAME_MAP.values())
     for col in numeric_columns:
         if col in ranked.columns:
             ranked[col] = ranked[col].apply(
                 lambda x: round(float(x), 2) if x != "-" and pd.notna(x) else x
             )
     # 为模型添加链接和高亮样式
     def format_model_name(row):
         model_name = row['model']
+        display_model_name = MODEL_DISPLAY_NAMES.get(model_name)
+        if display_model_name is None:
+            display_model_name = MODEL_DISPLAY_NAMES.get(model_name.lower(), model_name)
         link = MODEL_LINKS.get(model_name, "")
+        if not link:
+            link = MODEL_LINKS.get(model_name.lower(), "")
         if row['category'] == CATEGORY_TO_HIGHLIGHT:
+            display_name = f'<span style="color: #C41E3A;">{HIGHLIGHT_EMOJI} {display_model_name}</span>'
         else:
             display_name = display_model_name
         if link and link.strip():
             return f'<a href="{link}" target="_blank" style="text-decoration: none;">{display_name}</a>'
+        return f'<span class="model-name" data-model="{model_name}">{display_name}</span>'
     ranked['model'] = ranked.apply(format_model_name, axis=1)
     return ranked
 def filter_data(search_text: str, selected_categories: list):
     df = load_leaderboard()
     if search_text.strip():
         df = df[df['model'].str.contains(search_text.strip(), case=False, na=False)]
     if selected_categories:
         df = df[df['category'].isin(selected_categories)]
+    return make_ranked(df)
 def create_leaderboard_tab():
+    with gr.Tab("🏆Leaderboard"):
         with gr.Row():
             with gr.Column(scale=1):
                 search_box = gr.Textbox(
+                    label="Model Search",
                     placeholder="Entering model name to search...",
                     value=""
                 )
             with gr.Column(scale=2):
                 category_checkboxes = gr.CheckboxGroup(
                     label="Model Categories",
+                    choices=list(MODEL_CATEGORIES.keys()) + ["Others"],
+                    value=list(MODEL_CATEGORIES.keys()) + ["Others"]
                 )
+        all_categories = list(MODEL_CATEGORIES.keys()) + ["Others"]
+        initial_df = filter_data("", all_categories)
         column_count = len(initial_df.columns)
         datatypes = ["str"] * column_count
         model_col_index = initial_df.columns.get_loc('model')
         rank_col_index = initial_df.columns.get_loc('Rank')
         datatypes[model_col_index] = "html"
+        datatypes[rank_col_index] = "html"
+        # 根据列数动态设置列宽
+        base_widths = ["80px", "380px"]  # Rank, model
+        metric_width = "95px"
+        info_widths = ["180px", "150px"]  # category, license_type
+        metric_cols = column_count - 4  # Rank + model + category + license_type
+        column_widths = base_widths + [metric_width] * metric_cols + info_widths
         table = gr.Dataframe(
             value=initial_df,
+            datatype=datatypes,
+            max_height=600,
+            show_label=False,
+            elem_id="leaderboard_table",
+            interactive=False,
+            wrap=True,
+            column_widths=column_widths
         )
         def update_display(search_text, selected_categories):
+            return filter_data(search_text, selected_categories)
         search_box.change(
             fn=update_display,
             inputs=[search_box, category_checkboxes],
             inputs=[search_box, category_checkboxes],
             outputs=table
         )
         # 在底部添加说明
         with gr.Row():
             gr.Markdown(f"""
             ### 📊 Column Descriptions
             - **Rank**: Model ranking based on overall score
+            - **model**: Model name (<span style="color: #C41E3A;">{HIGHLIGHT_EMOJI} = {CATEGORY_TO_HIGHLIGHT}</span>)
+            - **overall**: Average of EN and ZH overall scores (mean of Objective & Subjective per language)
+            - **obj.EN / obj.ZH**: Objective scores on Global (English) / Chinese tasks
+            - **obj.avg**: Average objective score across both language tracks
+            - **subj.EN / subj.ZH**: Subjective report scores on Global (English) / Chinese tasks
+            - **subj.avg**: Average subjective score across both language tracks
+            - **expert**: Average expert rubric score across EN & ZH subjective tasks (0-100)
+            - **citation**: Citation consistency score (auxiliary, scaled 0-100)
+            - **source**: Source quality score (authority & timeliness, scaled 0-100)
+            - **RMSCE**: Root Mean Square Calibration Error for objective confidence (lower is better)
             - **category**: Model category
             - **license_type**: The software license type of the model/service
+            💡 **Tip**: Model names are clickable when links are available. Visit the project repositories for more details!
             """)
     return search_box

tabs/shared_data.py CHANGED Viewed

@@ -22,8 +22,16 @@ DATA_VIEWER_FILE = (
 )
 DATA_VIEWER_INDEX_FILE = BASE_DIR / "data" / "data_viewer_index.json"
 _REQUIRED_COLS = [
-    "model_name", "id", "prompt", "article", "overall_score",
     "comprehensiveness_score", "insight_score",
     "instruction_following_score", "readability_score",
 ]
@@ -32,6 +40,56 @@ _cache: pd.DataFrame | None = None
 _index_cache: dict | None = None
 def get_data() -> pd.DataFrame:
     global _cache
     if _cache is not None:
@@ -45,13 +103,13 @@ def get_data() -> pd.DataFrame:
                 if not line:
                     continue
                 try:
-                    records.append(json.loads(line))
                 except json.JSONDecodeError:
                     continue
     df = pd.DataFrame(records)
     if df.empty or not all(c in df.columns for c in _REQUIRED_COLS):
-        _cache = pd.DataFrame(columns=_REQUIRED_COLS)
     else:
         df["id"] = df["id"].astype(str)
         _cache = df
@@ -81,6 +139,7 @@ def get_index() -> dict:
                     item = json.loads(line)
                 except json.JSONDecodeError:
                     continue
                 model = item.get("model_name")
                 item_id = str(item.get("id"))
                 prompt = item.get("prompt") or ""
@@ -93,7 +152,7 @@ def get_index() -> dict:
         "models": sorted(models),
         "tasks": [
             {"id": item_id, "prompt": tasks[item_id]}
-            for item_id in sorted(tasks, key=lambda value: int(value))
         ],
     }
     return _index_cache
@@ -112,7 +171,7 @@ def get_entry(model_name: str, item_id: str) -> dict | None:
             fh.seek(offset)
             line = fh.read(length).decode("utf-8")
         try:
-            item = json.loads(line)
             if item.get("model_name") == model_name and str(item.get("id")) == item_id:
                 return item
         except json.JSONDecodeError:
@@ -123,7 +182,7 @@ def get_entry(model_name: str, item_id: str) -> dict | None:
             if not line.strip():
                 continue
             try:
-                item = json.loads(line)
             except json.JSONDecodeError:
                 continue
             if item.get("model_name") == model_name and str(item.get("id")) == item_id:
@@ -148,7 +207,7 @@ def get_entries_for_task(item_id: str, model_names: set[str]) -> dict[str, dict]
             for model, (offset, length) in locations.items():
                 fh.seek(offset)
                 try:
-                    item = json.loads(fh.read(length).decode("utf-8"))
                     if item.get("model_name") == model and str(item.get("id")) == item_id:
                         found[model] = item
                 except json.JSONDecodeError:
@@ -162,7 +221,7 @@ def get_entries_for_task(item_id: str, model_names: set[str]) -> dict[str, dict]
             if not line.strip():
                 continue
             try:
-                item = json.loads(line)
             except json.JSONDecodeError:
                 continue
             model = item.get("model_name")

 )
 DATA_VIEWER_INDEX_FILE = BASE_DIR / "data" / "data_viewer_index.json"
+# ICBCBench data viewer schema: dual-track scores + optional legacy DeepResearch Bench fields
 _REQUIRED_COLS = [
+    "model_name", "id", "prompt", "article",
+    "overall_score", "track",  # track: "objective" or "subjective"
+]
+_OPTIONAL_SCORE_COLS = [
+    "objective_score", "subjective_score",
+    "expert_score", "citation_score", "source_quality_score",
+    "confidence", "correct",
+    # Legacy DeepResearch Bench fields (kept for backward compatibility)
     "comprehensiveness_score", "insight_score",
     "instruction_following_score", "readability_score",
 ]
 _index_cache: dict | None = None
+def _normalize_record(item: dict) -> dict:
+    """Ensure every record has the ICBCBench score fields, falling back to legacy fields."""
+    record = dict(item)
+    # Determine track
+    if "track" not in record or not record["track"]:
+        # Heuristic: if article is present and non-empty, treat as subjective
+        article = record.get("article") or ""
+        record["track"] = "subjective" if isinstance(article, str) and article.strip() else "objective"
+    # Normalize scores to 0-100 scale if they look like ratios
+    for key in ["overall_score", "objective_score", "subjective_score",
+                "expert_score", "citation_score", "source_quality_score",
+                "comprehensiveness_score", "insight_score",
+                "instruction_following_score", "readability_score"]:
+        val = record.get(key)
+        if isinstance(val, (int, float)) and val is not None and val <= 1.0:
+            record[key] = val * 100
+    # If the new ICBCBench fields are missing but legacy fields exist, synthesize them.
+    if record.get("subjective_score") is None:
+        legacy = []
+        for k in ["comprehensiveness_score", "insight_score",
+                  "instruction_following_score", "readability_score"]:
+            val = record.get(k)
+            if val is None:
+                continue
+            try:
+                legacy.append(float(val))
+            except (TypeError, ValueError):
+                pass
+        if legacy:
+            record["subjective_score"] = sum(legacy) / len(legacy)
+    if record.get("overall_score") is None:
+        candidates = []
+        for k in ["objective_score", "subjective_score"]:
+            val = record.get(k)
+            if val is None:
+                continue
+            try:
+                candidates.append(float(val))
+            except (TypeError, ValueError):
+                pass
+        if candidates:
+            record["overall_score"] = sum(candidates) / len(candidates)
+    return record
 def get_data() -> pd.DataFrame:
     global _cache
     if _cache is not None:
                 if not line:
                     continue
                 try:
+                    records.append(_normalize_record(json.loads(line)))
                 except json.JSONDecodeError:
                     continue
     df = pd.DataFrame(records)
     if df.empty or not all(c in df.columns for c in _REQUIRED_COLS):
+        _cache = pd.DataFrame(columns=_REQUIRED_COLS + _OPTIONAL_SCORE_COLS)
     else:
         df["id"] = df["id"].astype(str)
         _cache = df
                     item = json.loads(line)
                 except json.JSONDecodeError:
                     continue
+                item = _normalize_record(item)
                 model = item.get("model_name")
                 item_id = str(item.get("id"))
                 prompt = item.get("prompt") or ""
         "models": sorted(models),
         "tasks": [
             {"id": item_id, "prompt": tasks[item_id]}
+            for item_id in sorted(tasks, key=lambda value: int(value) if value.isdigit() else value)
         ],
     }
     return _index_cache
             fh.seek(offset)
             line = fh.read(length).decode("utf-8")
         try:
+            item = _normalize_record(json.loads(line))
             if item.get("model_name") == model_name and str(item.get("id")) == item_id:
                 return item
         except json.JSONDecodeError:
             if not line.strip():
                 continue
             try:
+                item = _normalize_record(json.loads(line))
             except json.JSONDecodeError:
                 continue
             if item.get("model_name") == model_name and str(item.get("id")) == item_id:
             for model, (offset, length) in locations.items():
                 fh.seek(offset)
                 try:
+                    item = _normalize_record(json.loads(fh.read(length).decode("utf-8")))
                     if item.get("model_name") == model and str(item.get("id")) == item_id:
                         found[model] = item
                 except json.JSONDecodeError:
             if not line.strip():
                 continue
             try:
+                item = _normalize_record(json.loads(line))
             except json.JSONDecodeError:
                 continue
             model = item.get("model_name")

utils/merge_raw_data.py CHANGED Viewed

@@ -1,8 +1,20 @@
 #!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 import json
-import os
 from pathlib import Path
 EXCLUDED_SLUGS = {
@@ -11,44 +23,62 @@ EXCLUDED_SLUGS = {
 }
 def load_scores_for_model(model_results_dir: Path):
     scores_by_id = {}
-    raw_results_file = model_results_dir / "raw_results.jsonl"
-    if not raw_results_file.exists():
-        print(f"警告: 未找到模型 {model_results_dir.name} 的结果文件: {raw_results_file}")
         return scores_by_id
-    print(f"  正在从 {model_results_dir.name}/raw_results.jsonl 加载分数...")
-    with open(raw_results_file, 'r', encoding='utf-8') as f:
         for i, line in enumerate(f):
             try:
                 data = json.loads(line.strip())
                 article_id = str(data.get('id'))
                 if not article_id:
-                    print(f"    警告: {model_results_dir.name} 第 {i+1} 行缺少ID，已跳过。")
                     continue
-                overall_score_raw = data.get('overall_score', 0.0)
-                overall_score_scaled = overall_score_raw * 100
-                comprehensiveness_score_raw = data.get('comprehensiveness', 0.0)
-                insight_score_raw = data.get('insight', 0.0)
-                instruction_score_raw = data.get('instruction_following', 0.0)
-                readability_score_raw = data.get('readability', 0.0)
-                scores_by_id[article_id] = {
-                    'overall_score': f"{overall_score_scaled:.2f}",
-                    'comprehensiveness_score': f"{comprehensiveness_score_raw * 100:.2f}",
-                    'insight_score': f"{insight_score_raw * 100:.2f}",
-                    'instruction_following_score': f"{instruction_score_raw * 100:.2f}",
-                    'readability_score': f"{readability_score_raw * 100:.2f}"
                 }
             except json.JSONDecodeError as e:
-                print(f"    错误: 解析JSON时出错 (文件: {model_results_dir.name}, 行: {i+1}): {e}")
             except Exception as e:
-                print(f"    错误: 处理数据时出错 (文件: {model_results_dir.name}, 行: {i+1}): {e}")
-    print(f"  为模型 {model_results_dir.name} 加载了 {len(scores_by_id)}篇文章的分数")
     return scores_by_id
@@ -57,73 +87,77 @@ def merge_jsonl_files():
     raw_data_dir = project_root / "data" / "raw_data"
     raw_results_dir = project_root / "data" / "raw_results"
     output_file = project_root / "data" / "data_viewer.jsonl"
     input_files = list(raw_data_dir.glob("*.jsonl"))
     print(f"在 {raw_data_dir} 中找到 {len(input_files)} 个模型JSONL文件")
     if not input_files:
         print("未找到任何原始数据文件，已退出。")
         return
-    with open(output_file, 'w', encoding='utf-8') as f:
-        pass
     all_merged_data = []
     for raw_data_file in input_files:
         model_name = raw_data_file.stem
         if model_name in EXCLUDED_SLUGS:
             print(f"跳过隐藏模型: {model_name}")
             continue
         print(f"正在处理原始数据文件: {raw_data_file.name} (模型: {model_name})")
         model_results_dir = raw_results_dir / model_name
         if not model_results_dir.exists():
             print(f"  警告: 未找到模型 {model_name} 对应的结果文件夹: {model_results_dir}")
             continue
         scores_for_current_model = load_scores_for_model(model_results_dir)
-        processed_articles_count = 0
         with open(raw_data_file, 'r', encoding='utf-8') as f_raw:
             for i, line in enumerate(f_raw):
                 try:
                     article_data = json.loads(line.strip())
                     article_id = str(article_data.get('id'))
                     if not article_id:
-                        print(f"  警告: {raw_data_file.name} 第 {i+1} 行缺少ID，已跳过。")
                         continue
                     article_scores = scores_for_current_model.get(article_id, {})
-                    if not article_scores:
-                        print(f"  警告: 模型 {model_name} 的文章ID {article_id} 未在结果文件中找到分数。")
                     merged_item = {
                         'model_name': model_name,
                         'id': article_id,
                         'prompt': article_data.get('prompt'),
                         'article': article_data.get('article'),
                         'overall_score': article_scores.get('overall_score'),
                         'comprehensiveness_score': article_scores.get('comprehensiveness_score'),
                         'insight_score': article_scores.get('insight_score'),
                         'instruction_following_score': article_scores.get('instruction_following_score'),
-                        'readability_score': article_scores.get('readability_score')
                     }
                     all_merged_data.append(merged_item)
-                    processed_articles_count += 1
                 except json.JSONDecodeError as e:
-                    print(f"  错误: 解析原始数据JSON时出错 (文件: {raw_data_file.name}, 行: {i+1}): {e}")
                 except Exception as e:
-                    print(f"  错误: 处理原始数据时出错 (文件: {raw_data_file.name}, 行: {i+1}): {e}")
-        print(f"  为模型 {model_name} 处理了 {processed_articles_count} 篇文章数据。")
     with open(output_file, 'w', encoding='utf-8') as f_out:
         for item in all_merged_data:
             f_out.write(json.dumps(item, ensure_ascii=False) + '\n')
     print(f"\n成功合并并保存到: {output_file}, 共 {len(all_merged_data)} 条记录")
 if __name__ == "__main__":
     merge_jsonl_files()
-    print("所有文件处理完成！")

 #!/usr/bin/env python3
 # -*- coding: utf-8 -*-
+"""
+Merge per-model raw data and raw result files into data/data_viewer.jsonl.
+Supports ICBCBench dual-track format (objective / subjective) and legacy
+DeepResearch Bench format.
+Expected raw_data/<model>.jsonl fields:
+    id, prompt, article (for subjective), answer (for objective), language, track
+Expected raw_results/<model>/results.jsonl fields:
+    id, score, overall_score, track, language,
+    objective_score / subjective_score / expert_score / citation_score / source_quality_score,
+    confidence, correct
+"""
 import json
 from pathlib import Path
 EXCLUDED_SLUGS = {
 }
+def _norm_score(val):
+    """Normalize scores to 0-100 scale."""
+    if val is None:
+        return None
+    try:
+        val = float(val)
+    except (TypeError, ValueError):
+        return val
+    return val * 100 if val <= 1.0 else val
 def load_scores_for_model(model_results_dir: Path):
+    """Load per-article scores from results.jsonl (ICBCBench) or raw_results.jsonl (legacy)."""
     scores_by_id = {}
+    # Try ICBCBench results.jsonl first.
+    results_file = model_results_dir / "results.jsonl"
+    if not results_file.exists():
+        results_file = model_results_dir / "raw_results.jsonl"
+    if not results_file.exists():
+        print(f"  警告: 未找到模型 {model_results_dir.name} 的结果文件")
         return scores_by_id
+    print(f"  正在从 {model_results_dir.name}/{results_file.name} 加载分数...")
+    with open(results_file, 'r', encoding='utf-8') as f:
         for i, line in enumerate(f):
             try:
                 data = json.loads(line.strip())
                 article_id = str(data.get('id'))
                 if not article_id:
                     continue
+                scores = {
+                    'overall_score': _norm_score(data.get('overall_score', data.get('score'))),
+                    'track': data.get('track', 'subjective'),
+                    'objective_score': _norm_score(data.get('objective_score')),
+                    'subjective_score': _norm_score(data.get('subjective_score')),
+                    'expert_score': _norm_score(data.get('expert_score')),
+                    'citation_score': _norm_score(data.get('citation_score')),
+                    'source_quality_score': _norm_score(data.get('source_quality_score')),
+                    'confidence': _norm_score(data.get('confidence')),
+                    'correct': data.get('correct'),
+                    # Legacy fields
+                    'comprehensiveness_score': _norm_score(data.get('comprehensiveness')),
+                    'insight_score': _norm_score(data.get('insight')),
+                    'instruction_following_score': _norm_score(data.get('instruction_following')),
+                    'readability_score': _norm_score(data.get('readability')),
                 }
+                scores_by_id[article_id] = scores
             except json.JSONDecodeError as e:
+                print(f"    错误: 解析JSON时出错 ({model_results_dir.name}, 行 {i+1}): {e}")
             except Exception as e:
+                print(f"    错误: 处理数据时出错 ({model_results_dir.name}, 行 {i+1}): {e}")
+    print(f"  为模型 {model_results_dir.name} 加载了 {len(scores_by_id)} 条结果")
     return scores_by_id
     raw_data_dir = project_root / "data" / "raw_data"
     raw_results_dir = project_root / "data" / "raw_results"
     output_file = project_root / "data" / "data_viewer.jsonl"
     input_files = list(raw_data_dir.glob("*.jsonl"))
     print(f"在 {raw_data_dir} 中找到 {len(input_files)} 个模型JSONL文件")
     if not input_files:
         print("未找到任何原始数据文件，已退出。")
         return
     all_merged_data = []
     for raw_data_file in input_files:
         model_name = raw_data_file.stem
         if model_name in EXCLUDED_SLUGS:
             print(f"跳过隐藏模型: {model_name}")
             continue
         print(f"正在处理原始数据文件: {raw_data_file.name} (模型: {model_name})")
         model_results_dir = raw_results_dir / model_name
         if not model_results_dir.exists():
             print(f"  警告: 未找到模型 {model_name} 对应的结果文件夹: {model_results_dir}")
             continue
         scores_for_current_model = load_scores_for_model(model_results_dir)
+        processed_count = 0
         with open(raw_data_file, 'r', encoding='utf-8') as f_raw:
             for i, line in enumerate(f_raw):
                 try:
                     article_data = json.loads(line.strip())
                     article_id = str(article_data.get('id'))
                     if not article_id:
                         continue
                     article_scores = scores_for_current_model.get(article_id, {})
                     merged_item = {
                         'model_name': model_name,
                         'id': article_id,
                         'prompt': article_data.get('prompt'),
                         'article': article_data.get('article'),
+                        'track': article_data.get('track') or article_scores.get('track') or 'subjective',
+                        'language': article_data.get('language') or 'en',
                         'overall_score': article_scores.get('overall_score'),
+                        'objective_score': article_scores.get('objective_score'),
+                        'subjective_score': article_scores.get('subjective_score'),
+                        'expert_score': article_scores.get('expert_score'),
+                        'citation_score': article_scores.get('citation_score'),
+                        'source_quality_score': article_scores.get('source_quality_score'),
+                        'confidence': article_scores.get('confidence'),
+                        'correct': article_scores.get('correct'),
                         'comprehensiveness_score': article_scores.get('comprehensiveness_score'),
                         'insight_score': article_scores.get('insight_score'),
                         'instruction_following_score': article_scores.get('instruction_following_score'),
+                        'readability_score': article_scores.get('readability_score'),
                     }
                     all_merged_data.append(merged_item)
+                    processed_count += 1
                 except json.JSONDecodeError as e:
+                    print(f"  错误: 解析原始数据JSON时出错 ({raw_data_file.name}, 行 {i+1}): {e}")
                 except Exception as e:
+                    print(f"  错误: 处理原始数据时出错 ({raw_data_file.name}, 行 {i+1}): {e}")
+        print(f"  为模型 {model_name} 处理了 {processed_count} 条数据。")
     with open(output_file, 'w', encoding='utf-8') as f_out:
         for item in all_merged_data:
             f_out.write(json.dumps(item, ensure_ascii=False) + '\n')
     print(f"\n成功合并并保存到: {output_file}, 共 {len(all_merged_data)} 条记录")
 if __name__ == "__main__":
     merge_jsonl_files()
+    print("所有文件处理完成！")

utils/parse_icbcbench_tex.py ADDED Viewed

	@@ -0,0 +1,219 @@

+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+"""
+Parse ICBCBench LaTeX result tables and generate data/leaderboard.csv.
+Reads:
+  - main_results.tex            EN/ZH Objective/Subjective/Overall
+  - results_subset_en.tex       EN Expert/Citation/Source details
+  - results_subset_zh.tex       ZH Expert/Citation/Source details
+  - objective_public_CalibErr.tex  All-language Accuracy / Calibration Error
+"""
+from __future__ import annotations
+import re
+import csv
+from pathlib import Path
+from collections import defaultdict
+PROJECT_ROOT = Path(__file__).resolve().parent.parent
+def parse_main_results(tex_file: Path) -> dict[str, dict]:
+    """Parse main_results.tex and return {model: {objective_en, subjective_en, overall_en, objective_zh, subjective_zh, overall_zh}}."""
+    text = tex_file.read_text(encoding="utf-8")
+    rows = {}
+    for line in text.splitlines():
+        line = line.strip()
+        if not line or line.startswith("\\") or line.startswith("%"):
+            continue
+        # Match lines like:
+        # Gemini-deep-research                     &           50.00 &           64.77 & \underline{57.38} &           52.50 &  \textbf{65.69} & \underline{59.09} \\
+        parts = [p.strip() for p in line.split("&")]
+        if len(parts) != 7:
+            continue
+        model = parts[0].strip()
+        values = []
+        for p in parts[1:]:
+            # Strip LaTeX formatting commands but keep their arguments
+            val = re.sub(r"\\(textbf|underline|rowcolor)\{([^}]*)\}", r"\2", p)
+            # Drop structural LaTeX commands and trailing \
+            val = re.sub(r"\\(multicolumn|cmidrule|toprule|midrule|bottomrule).*", "", val)
+            val = val.replace("\\", "").replace("{", "").replace("}", "").strip()
+            if val in ("", "--"):
+                values.append(None)
+            else:
+                try:
+                    values.append(float(val))
+                except ValueError:
+                    values.append(None)
+        if all(v is not None for v in values):
+            rows[model] = {
+                "objective_en": values[0],
+                "subjective_en": values[1],
+                "overall_en": values[2],
+                "objective_zh": values[3],
+                "subjective_zh": values[4],
+                "overall_zh": values[5],
+            }
+    return rows
+def parse_subset(tex_file: Path) -> dict[str, dict]:
+    """Parse results_subset_en.tex or results_subset_zh.tex.
+    Returns {model: {objective_text, objective_all, expert, citation, source, overall}}.
+    """
+    text = tex_file.read_text(encoding="utf-8")
+    rows = {}
+    for line in text.splitlines():
+        line = line.strip()
+        if not line or line.startswith("\\") or line.startswith("%"):
+            continue
+        parts = [p.strip() for p in line.split("&")]
+        if len(parts) != 7:
+            continue
+        model = parts[0].strip()
+        values = []
+        for p in parts[1:]:
+            val = re.sub(r"\\(textbf|underline|rowcolor)\{([^}]*)\}", r"\2", p)
+            val = re.sub(r"\\(multicolumn|cmidrule|toprule|midrule|bottomrule).*", "", val)
+            val = val.replace("\\", "").replace("{", "").replace("}", "").strip()
+            if val in ("", "--"):
+                values.append(None)
+            else:
+                try:
+                    values.append(float(val))
+                except ValueError:
+                    values.append(None)
+        if values[-1] is not None:  # overall is required
+            rows[model] = {
+                "objective_text": values[0],
+                "objective_all": values[1],
+                "expert": values[2],
+                "citation": values[3],
+                "source": values[4],
+                "overall": values[5],
+            }
+    return rows
+def parse_calibration(tex_file: Path) -> dict[str, dict]:
+    """Parse objective_public_CalibErr.tex. Returns {model: {accuracy, calibration_error}}."""
+    text = tex_file.read_text(encoding="utf-8")
+    rows = {}
+    for line in text.splitlines():
+        line = line.strip()
+        if not line or line.startswith("\\") or line.startswith("%"):
+            continue
+        parts = [p.strip() for p in line.split("&")]
+        if len(parts) != 3:
+            continue
+        model = parts[0].strip()
+        values = []
+        for p in parts[1:]:
+            val = p.replace("\\", "").strip()
+            if val in ("", "--"):
+                values.append(None)
+            else:
+                try:
+                    values.append(float(val))
+                except ValueError:
+                    values.append(None)
+        if all(v is not None for v in values):
+            rows[model] = {"accuracy": values[0], "calibration_error": values[1]}
+    return rows
+def merge_scores(
+    main: dict[str, dict],
+    en_detail: dict[str, dict],
+    zh_detail: dict[str, dict],
+    calib: dict[str, dict],
+) -> list[dict]:
+    models = sorted(main.keys())
+    results = []
+    for model in models:
+        m = main[model]
+        en = en_detail.get(model, {})
+        zh = zh_detail.get(model, {})
+        cal = calib.get(model, {})
+        # Aggregate citation / source from available detailed tables.
+        # Prefer EN values when both exist, otherwise use whichever is available.
+        citation = en.get("citation") if en.get("citation") is not None else zh.get("citation")
+        source = en.get("source") if en.get("source") is not None else zh.get("source")
+        # Expert score: average of EN and ZH expert if both exist.
+        experts = [v for v in [en.get("expert"), zh.get("expert")] if v is not None]
+        expert_avg = sum(experts) / len(experts) if experts else None
+        objective_avg = (m["objective_en"] + m["objective_zh"]) / 2
+        subjective_avg = (m["subjective_en"] + m["subjective_zh"]) / 2
+        overall = (m["overall_en"] + m["overall_zh"]) / 2
+        results.append({
+            "model": model,
+            "overall": overall,
+            "objective_en": m["objective_en"],
+            "objective_zh": m["objective_zh"],
+            "objective_avg": objective_avg,
+            "subjective_en": m["subjective_en"],
+            "subjective_zh": m["subjective_zh"],
+            "subjective_avg": subjective_avg,
+            "expert_avg": expert_avg,
+            "citation_score": citation,
+            "source_quality": source,
+            "rmsce": cal.get("calibration_error"),
+        })
+    # Sort by overall descending, then objective_avg, then subjective_avg.
+    results.sort(
+        key=lambda x: (x["overall"], x["objective_avg"], x["subjective_avg"]),
+        reverse=True,
+    )
+    return results
+def write_leaderboard(results: list[dict], output_file: Path):
+    fieldnames = [
+        "model", "overall", "objective_en", "objective_zh", "objective_avg",
+        "subjective_en", "subjective_zh", "subjective_avg",
+        "expert_avg", "citation_score", "source_quality", "rmsce",
+    ]
+    with open(output_file, "w", newline="", encoding="utf-8") as f:
+        writer = csv.DictWriter(f, fieldnames=fieldnames)
+        writer.writeheader()
+        for r in results:
+            row = {k: r[k] for k in fieldnames}
+            for k in fieldnames:
+                if k == "model":
+                    continue
+                val = row[k]
+                row[k] = f"{val:.2f}" if val is not None else "-"
+            writer.writerow(row)
+    print(f"Wrote {len(results)} models to {output_file}")
+def main():
+    main_file = PROJECT_ROOT / "main_results.tex"
+    en_file = PROJECT_ROOT / "results_subset_en.tex"
+    zh_file = PROJECT_ROOT / "results_subset_zh.tex"
+    calib_file = PROJECT_ROOT / "objective_public_CalibErr.tex"
+    output_file = PROJECT_ROOT / "data" / "leaderboard.csv"
+    main = parse_main_results(main_file)
+    en_detail = parse_subset(en_file)
+    zh_detail = parse_subset(zh_file)
+    calib = parse_calibration(calib_file)
+    print(f"Parsed {len(main)} models from main results")
+    print(f"Parsed {len(en_detail)} EN detailed rows")
+    print(f"Parsed {len(zh_detail)} ZH detailed rows")
+    print(f"Parsed {len(calib)} calibration rows")
+    results = merge_scores(main, en_detail, zh_detail, calib)
+    write_leaderboard(results, output_file)
+if __name__ == "__main__":
+    main()

utils/rank_leaderboard.py CHANGED Viewed

@@ -1,8 +1,18 @@
 #!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 import json
-import os
 import csv
 from pathlib import Path
 from collections import defaultdict
@@ -14,9 +24,8 @@ EXCLUDED_SLUGS = {
 def parse_race_result(race_result_file):
-    """解析race_result.txt文件获取各维度分数"""
     scores = {}
     with open(race_result_file, 'r', encoding='utf-8') as f:
         for line in f:
             line = line.strip()
@@ -24,7 +33,6 @@ def parse_race_result(race_result_file):
                 key, value = line.split(':', 1)
                 key = key.strip()
                 value = float(value.strip())
                 if key == 'Comprehensiveness':
                     scores['comprehensiveness'] = value * 100
                 elif key == 'Insight':
@@ -35,17 +43,14 @@ def parse_race_result(race_result_file):
                     scores['readability'] = value * 100
                 elif key == 'Overall Score':
                     scores['overall_score'] = value * 100
     return scores
 def parse_fact_result(fact_result_file):
-    """解析fact_result.txt文件获取引用相关指标"""
     citation_scores = {}
     if not fact_result_file.exists():
         return citation_scores
     with open(fact_result_file, 'r', encoding='utf-8') as f:
         for line in f:
             line = line.strip()
@@ -53,78 +58,196 @@ def parse_fact_result(fact_result_file):
                 key, value = line.split(':', 1)
                 key = key.strip()
                 value = float(value.strip())
                 if key == 'valid_rate':
                     citation_scores['citation_accuracy'] = value * 100
-                elif key == 'total_valid_citations':
-                    citation_scores['effective_citations'] = value
                 elif key == 'supported_per_task':
                     citation_scores['effective_citations'] = value
     return citation_scores
 def process_model_data(model_dir):
-    """处理单个模型文件夹的数据"""
     model_name = model_dir.name
     race_result_file = model_dir / "race_result.txt"
-    if not race_result_file.exists():
-        print(f"警告: 模型 {model_name} 的文件夹中未找到 race_result.txt")
-        return None
-    print(f"正在处理模型: {model_name}")
-    try:
-        scores = parse_race_result(race_result_file)
-        if not scores:
-            print(f"  - 警告: 未能解析到有效分数")
             return None
-        # 查找对应的fact_result.txt文件
-        project_root = Path(__file__).parent.parent
-        fact_results_dir = project_root / "data" / "fact_results"
-        fact_result_file = fact_results_dir / model_name / "fact_result.txt"
-        citation_scores = parse_fact_result(fact_result_file)
-        if citation_scores:
-            print(f"  - 总分: {scores['overall_score']:.2f}, 引用准确率: {citation_scores.get('citation_accuracy', 'N/A'):.2f}%, 有效引用数: {citation_scores.get('effective_citations', 'N/A')}")
-        else:
-            print(f"  - 总分: {scores['overall_score']:.2f}, 引用数据: 未找到")
-        result = {
-            'model': model_name,
-            'overall_score': scores['overall_score'],
-            'comprehensiveness': scores['comprehensiveness'],
-            'insight': scores['insight'],
-            'instruction_following': scores['instruction_following'],
-            'readability': scores['readability'],
-            'citation_accuracy': citation_scores.get('citation_accuracy', None),
-            'effective_citations': citation_scores.get('effective_citations', None)
-        }
-        return result
-    except Exception as e:
-        print(f"  - 错误: 处理文件时出错: {e}")
-        return None
 def rank_leaderboard():
-    """计算排行榜并保存到CSV"""
     project_root = Path(__file__).parent.parent
     input_dir = project_root / "data" / "raw_results"
     output_file = project_root / "data" / "leaderboard.csv"
     model_dirs = [d for d in input_dir.iterdir() if d.is_dir() and d.name not in EXCLUDED_SLUGS]
-    print(f"找到 {len(model_dirs)} 个模型文件夹（已排除 {len(EXCLUDED_SLUGS)} 个隐藏模型）")
     if not model_dirs:
-        print("未找到任何模型文件夹")
         return
     model_results = []
     for model_dir in model_dirs:
         try:
@@ -132,37 +255,44 @@ def rank_leaderboard():
             if result:
                 model_results.append(result)
         except Exception as e:
-            print(f"处理文件夹 {model_dir.name} 时出错: {e}")
             continue
-    # 按overall_score排序，同分时按comprehensiveness、insight、instruction_following、readability依次排序
-    model_results.sort(key=lambda x: (x['overall_score'], x['comprehensiveness'], x['insight'], x['instruction_following'], x['readability']), reverse=True)
-    # 写入CSV文件
     with open(output_file, 'w', newline='', encoding='utf-8') as csvfile:
-        fieldnames = ['model', 'overall_score', 'comprehensiveness', 'insight', 'instruction_following', 'readability', 'citation_accuracy', 'effective_citations']
         writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
         writer.writeheader()
         for result in model_results:
-            # 格式化数值，对于None值使用"-"
-            row = {
-                'model': result['model'],
-                'overall_score': f"{result['overall_score']:.2f}",
-                'comprehensiveness': f"{result['comprehensiveness']:.2f}",
-                'insight': f"{result['insight']:.2f}",
-                'instruction_following': f"{result['instruction_following']:.2f}",
-                'readability': f"{result['readability']:.2f}",
-                'citation_accuracy': f"{result['citation_accuracy']:.2f}" if result['citation_accuracy'] is not None else "-",
-                'effective_citations': f"{result['effective_citations']:.2f}" if result['effective_citations'] is not None else "-"
-            }
             writer.writerow(row)
-    print(f"\n排行榜已保存到: {output_file}")
-    print(f"共处理了 {len(model_results)} 个模型")
 if __name__ == "__main__":
     rank_leaderboard()
-    print("排行榜计算完成！")

 #!/usr/bin/env python3
 # -*- coding: utf-8 -*-
+"""
+Rank leaderboard for ICBCBench.
+Reads per-model result files from data/raw_results/ and writes data/leaderboard.csv.
+Expected input formats:
+  1. ICBCBench JSONL: data/raw_results/<model>/results.jsonl
+     Each line contains {"language": "en"|"zh", "track": "objective"|"subjective",
+                        "score": float, "citation_score": float (optional),
+                        "source_quality_score": float (optional), ...}
+  2. Legacy DeepResearch Bench race_result.txt + fact_result.txt (kept for compatibility).
+"""
 import json
 import csv
 from pathlib import Path
 from collections import defaultdict
 def parse_race_result(race_result_file):
+    """Legacy DeepResearch Bench: parse race_result.txt."""
     scores = {}
     with open(race_result_file, 'r', encoding='utf-8') as f:
         for line in f:
             line = line.strip()
                 key, value = line.split(':', 1)
                 key = key.strip()
                 value = float(value.strip())
                 if key == 'Comprehensiveness':
                     scores['comprehensiveness'] = value * 100
                 elif key == 'Insight':
                     scores['readability'] = value * 100
                 elif key == 'Overall Score':
                     scores['overall_score'] = value * 100
     return scores
 def parse_fact_result(fact_result_file):
+    """Legacy DeepResearch Bench: parse fact_result.txt."""
     citation_scores = {}
     if not fact_result_file.exists():
         return citation_scores
     with open(fact_result_file, 'r', encoding='utf-8') as f:
         for line in f:
             line = line.strip()
                 key, value = line.split(':', 1)
                 key = key.strip()
                 value = float(value.strip())
                 if key == 'valid_rate':
                     citation_scores['citation_accuracy'] = value * 100
                 elif key == 'supported_per_task':
                     citation_scores['effective_citations'] = value
     return citation_scores
+def parse_icbcbench_jsonl(results_file):
+    """Parse ICBCBench results.jsonl and aggregate scores."""
+    scores_by_lang_track = defaultdict(list)
+    citation_scores = []
+    source_quality_scores = []
+    confidence_pairs = []  # (correct, confidence)
+    with open(results_file, 'r', encoding='utf-8') as f:
+        for line in f:
+            line = line.strip()
+            if not line:
+                continue
+            try:
+                item = json.loads(line)
+            except json.JSONDecodeError:
+                continue
+            language = (item.get("language") or item.get("lang") or "en").lower()
+            track = (item.get("track") or "subjective").lower()
+            score = item.get("score")
+            if score is not None:
+                try:
+                    score = float(score)
+                    if score <= 1.0:
+                        score *= 100
+                    scores_by_lang_track[(language, track)].append(score)
+                except (TypeError, ValueError):
+                    pass
+            if "citation_score" in item and item["citation_score"] is not None:
+                try:
+                    val = float(item["citation_score"])
+                    if val <= 1.0:
+                        val *= 100
+                    citation_scores.append(val)
+                except (TypeError, ValueError):
+                    pass
+            if "source_quality_score" in item and item["source_quality_score"] is not None:
+                try:
+                    val = float(item["source_quality_score"])
+                    if val <= 1.0:
+                        val *= 100
+                    source_quality_scores.append(val)
+                except (TypeError, ValueError):
+                    pass
+            if track == "objective":
+                correct = item.get("correct")
+                confidence = item.get("confidence")
+                if correct is not None and confidence is not None:
+                    try:
+                        confidence = float(confidence)
+                        if confidence <= 1.0:
+                            confidence *= 100
+                        confidence_pairs.append((bool(correct), confidence))
+                    except (TypeError, ValueError):
+                        pass
+    def avg(vals):
+        return sum(vals) / len(vals) if vals else None
+    def compute_rmsce(pairs):
+        """Compute RMSCE from (correct, confidence) pairs using 10-sample bins."""
+        if not pairs:
+            return None
+        pairs = sorted(pairs, key=lambda x: x[1])
+        n = len(pairs)
+        if n == 0:
+            return None
+        bin_size = max(1, min(10, n))
+        num_bins = max(1, n // bin_size)
+        rmsce = 0.0
+        count = 0
+        for i in range(num_bins):
+            start = i * bin_size
+            end = start + bin_size if i < num_bins - 1 else n
+            bin_pairs = pairs[start:end]
+            acc = sum(1 for c, _ in bin_pairs if c) / len(bin_pairs)
+            conf = sum(conf for _, conf in bin_pairs) / len(bin_pairs)
+            rmsce += len(bin_pairs) * (acc - conf) ** 2
+            count += len(bin_pairs)
+        return (rmsce / count) ** 0.5 if count > 0 else None
+    result = {
+        'objective_en': avg(scores_by_lang_track.get(("en", "objective"), [])),
+        'objective_zh': avg(scores_by_lang_track.get(("zh", "objective"), [])),
+        'subjective_en': avg(scores_by_lang_track.get(("en", "subjective"), [])),
+        'subjective_zh': avg(scores_by_lang_track.get(("zh", "subjective"), [])),
+        'citation_score': avg(citation_scores),
+        'source_quality': avg(source_quality_scores),
+        'rmsce': compute_rmsce(confidence_pairs),
+    }
+    # Compute averages and overall.
+    obj_scores = [v for v in [result['objective_en'], result['objective_zh']] if v is not None]
+    subj_scores = [v for v in [result['subjective_en'], result['subjective_zh']] if v is not None]
+    result['objective_avg'] = avg(obj_scores)
+    result['subjective_avg'] = avg(subj_scores)
+    # Overall = weighted average of available objective and subjective averages.
+    overall_parts = []
+    if result['objective_avg'] is not None:
+        overall_parts.append(result['objective_avg'])
+    if result['subjective_avg'] is not None:
+        overall_parts.append(result['subjective_avg'])
+    result['overall'] = avg(overall_parts)
+    return result
 def process_model_data(model_dir):
+    """Process a single model directory and return aggregated scores."""
     model_name = model_dir.name
+    results_file = model_dir / "results.jsonl"
     race_result_file = model_dir / "race_result.txt"
+    print(f"Processing model: {model_name}")
+    # Prefer ICBCBench JSONL format.
+    if results_file.exists():
+        try:
+            result = parse_icbcbench_jsonl(results_file)
+            result['model'] = model_name
+            print(f"  - overall: {result['overall']:.2f}" if result['overall'] else "  - overall: N/A")
+            return result
+        except Exception as e:
+            print(f"  - error parsing results.jsonl: {e}")
             return None
+    # Fallback to legacy DeepResearch Bench format.
+    if race_result_file.exists():
+        try:
+            scores = parse_race_result(race_result_file)
+            if not scores:
+                print(f"  - warning: no valid scores in race_result.txt")
+                return None
+            project_root = Path(__file__).parent.parent
+            fact_result_file = project_root / "data" / "fact_results" / model_name / "fact_result.txt"
+            citation_scores = parse_fact_result(fact_result_file)
+            result = {
+                'model': model_name,
+                'overall': scores['overall_score'],
+                'objective_en': None,
+                'objective_zh': None,
+                'objective_avg': None,
+                'subjective_en': None,
+                'subjective_zh': None,
+                'subjective_avg': scores['overall_score'],
+                'citation_score': citation_scores.get('citation_accuracy'),
+                'source_quality': None,
+                'rmsce': None,
+            }
+            print(f"  - overall (legacy): {result['overall']:.2f}")
+            return result
+        except Exception as e:
+            print(f"  - error: {e}")
+            return None
+    print(f"  - warning: no results.jsonl or race_result.txt found")
+    return None
 def rank_leaderboard():
+    """Compute leaderboard and save to CSV."""
     project_root = Path(__file__).parent.parent
     input_dir = project_root / "data" / "raw_results"
     output_file = project_root / "data" / "leaderboard.csv"
+    if not input_dir.exists():
+        print(f"Input directory not found: {input_dir}")
+        return
     model_dirs = [d for d in input_dir.iterdir() if d.is_dir() and d.name not in EXCLUDED_SLUGS]
+    print(f"Found {len(model_dirs)} model directories (excluded {len(EXCLUDED_SLUGS)})")
     if not model_dirs:
+        print("No model data found.")
         return
     model_results = []
     for model_dir in model_dirs:
         try:
             if result:
                 model_results.append(result)
         except Exception as e:
+            print(f"Error processing {model_dir.name}: {e}")
             continue
+    # Sort by overall, then objective_avg, then subjective_avg.
+    model_results.sort(
+        key=lambda x: (
+            x['overall'] if x['overall'] is not None else -1,
+            x['objective_avg'] if x['objective_avg'] is not None else -1,
+            x['subjective_avg'] if x['subjective_avg'] is not None else -1,
+        ),
+        reverse=True
+    )
+    fieldnames = [
+        'model', 'overall', 'objective_en', 'objective_zh',
+        'objective_avg', 'subjective_en', 'subjective_zh',
+        'subjective_avg', 'citation_score', 'source_quality', 'rmsce'
+    ]
     with open(output_file, 'w', newline='', encoding='utf-8') as csvfile:
         writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
         writer.writeheader()
         for result in model_results:
+            row = {k: result.get(k) for k in fieldnames}
+            for k in fieldnames:
+                if k == 'model':
+                    continue
+                val = row[k]
+                if val is None:
+                    row[k] = "-"
+                else:
+                    row[k] = f"{float(val):.2f}"
             writer.writerow(row)
+    print(f"\nLeaderboard saved to: {output_file}")
+    print(f"Total models: {len(model_results)}")
 if __name__ == "__main__":
     rank_leaderboard()
+    print("Done!")