Skip to content

@open-compass OpenCompass

All

    Repositories list

    49 repositories

    • PowerBench

      Public
      PowerBench: A Benchmark for Agentic Retrieval and Reasoning in Power Systems
      Apache License 2.0
      •0•2•1•0•Updated Oct 3, 2026Oct 3, 2026

    • CNFinBench

      Public
      CNFinBench — the first comprehensive benchmark for high-stakes financial scenarios. It spans 29 subtasks grounded in authoritative financial corpora and real bu…
      Python
      •1•21•0•0•Updated Oct 3, 2026Oct 3, 2026

    • AgentCompass

      Public
      [EMNLP 2026] AgentCompass is an extensible open-source evaluation infrastructure for systematically assessing LLM/VLM agent capabilities.
      agentbenchmark-frameworkharness
      agentbenchmark-frameworkharnesscodexevaluation-frameworkclaude-code
      Python
      •
      Apache License 2.0
      •31•152•3•8•Updated Oct 2, 2026Oct 2, 2026

    • VLMEvalKit

      Public
      Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks
      computer-visionevaluationpytorch
      computer-visionevaluationpytorchgeminiopenaivqavitgptmulti-modalclip
      Python
      •
      Apache License 2.0
      •774•4.4k•226•75•Updated Sep 30, 2026Sep 30, 2026

    • opencompass

      Public
      OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets cove…
      benchmarkevaluationllm
      benchmarkevaluationllmlarge-language-modelllm-eval
      Python
      •
      Apache License 2.0
      •877•7.5k•293•119•Updated Sep 28, 2026Sep 28, 2026

    • MCPServer-AC

      Public
      0•0•0•0•Updated Aug 14, 2026Aug 14, 2026

    • docs

      Public
      MDX
      •
      MIT License
      •0•0•0•0•Updated Jul 10, 2026Jul 10, 2026

    • GTA

      Public
      [NeurIPS 2024 D&B] GTA: A Benchmark for General Tool Agents & [arXiv 2026] GTA-2
      llm-agentllm-evaluation
      llm-agentllm-evaluation
      Python
      •
      Apache License 2.0
      •12•152•1•0•Updated Apr 20, 2026Apr 20, 2026

    • GenEditEvalKit

      Public
      The first unified, efficient, and extensible evaluation toolkit for evaluating image generation and editing models across multiple benchmarks.
      Jupyter Notebook
      •
      MIT License
      •4•51•3•0•Updated Apr 12, 2026Apr 12, 2026

    • TextEdit

      Public
      We provide TextEdit, a high-quality, multi-scenario text editing benchmark for generation models.
      Python
      •
      MIT License
      •0•22•0•0•Updated Mar 16, 2026Mar 16, 2026

    • MiroFlow

      Public
      MiroMind Research Agent: Fully Open-Source Deep Research Agent with Reproducible State-of-the-Art Performance on FutureX, GAIA, HLE, BrowserComp and xBench.
      Python
      •
      Apache License 2.0
      •325•0•0•0•Updated Dec 30, 2025Dec 30, 2025

    • RePro

      Public
      [ICLR 2026] Rectifying LLM Thought From Lens of Optimization
      reinforcement-learninglarge-language-modellarge-language-model-reasoning
      reinforcement-learninglarge-language-modellarge-language-model-reasoning
      Python
      •
      MIT License
      •4•14•2•0•Updated Dec 5, 2025Dec 5, 2025

    • SAGA

      Public
      The code repository for the NeurIPS 2025 paper "Rethinking Verification for LLM Code Generation: From Generation to Testing."
      0•14•1•0•Updated Nov 27, 2025Nov 27, 2025

    • ATLAS

      Public
      ATLAS: A High-Difficulty, Multidisciplinary Benchmark for Frontier Scientific Reasoning
      2•7•0•0•Updated Nov 20, 2025Nov 20, 2025

    • OASIS

      Public
      Python
      •0•3•0•0•Updated Nov 12, 2025Nov 12, 2025

    • InteractScience

      Public
      JavaScript
      •
      Apache License 2.0
      •0•9•0•0•Updated Oct 31, 2025Oct 31, 2025

    • CognitiveKernel-Pro

      Public
      Deep Research Agent CognitiveKernel-Pro from Tencent AI Lab. Paper: https://arxiv.org/pdf/2508.00414
      Python
      •
      Other
      •57•0•0•0•Updated Oct 27, 2025Oct 27, 2025

    • GAOKAO-Eval

      Public
      Jupyter Notebook
      •8•127•5•0•Updated Oct 7, 2025Oct 7, 2025

    • .github

      Public
      2•0•0•1•Updated Sep 9, 2025Sep 9, 2025

    • MMBench-GUI

      Public
      Official repo of "MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agents". It can be used to evaluate a GUI agent with a hierarchical mann…
      benchmark-frameworkvision-language-modelcomputer-use
      benchmark-frameworkvision-language-modelcomputer-usegui-agent
      Python
      •5•115•7•0•Updated Sep 8, 2025Sep 8, 2025

    • ReasonZoo

      Public
      Python
      •
      Apache License 2.0
      •0•3•0•0•Updated Aug 27, 2025Aug 27, 2025

    • CompassVerifier

      Public
      [EMNLP 2025] CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward
      Jupyter Notebook
      •2•71•0•0•Updated Aug 10, 2025Aug 10, 2025

    • GPassK

      Public
      [ACL 2025] Are Your LLMs Capable of Stable Reasoning?
      large-language-model-evaluationreasoning-stability
      large-language-model-evaluationreasoning-stability
      Python
      •2•33•2•0•Updated Aug 5, 2025Aug 5, 2025

    • Creation-MMBench

      Public
      Assessing Context-Aware Creative Intelligence in MLLMs
      JavaScript
      •0•23•1•0•Updated Jul 22, 2025Jul 22, 2025

    • CompassJudger

      Public
      The All-in-one Judge Models introduced by Opencompass
      Apache License 2.0
      •6•122•1•0•Updated Jul 15, 2025Jul 15, 2025

    • RaML

      Public
      [Preprint 2025] Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective
      Jupyter Notebook
      •2•8•0•0•Updated May 27, 2025May 27, 2025

    • BotChat

      Public
      Evaluating LLMs' multi-round chatting capability via assessing conversations generated by two LLM instances.
      Jupyter Notebook
      •
      Apache License 2.0
      •6•163•2•0•Updated May 22, 2025May 22, 2025

    • Ada-LEval

      Public
      The official implementation of "Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks"
      gpt4llmlong-context
      gpt4llmlong-context
      Python
      •3•56•0•0•Updated May 22, 2025May 22, 2025

    • MathBench

      Public
      [ACL 2024 Findings] MathBench: A Comprehensive Multi-Level Difficulty Mathematics Evaluation Dataset
      Apache License 2.0
      •1•118•5•0•Updated May 22, 2025May 22, 2025

    • MMBench

      Public
      Official Repo of "MMBench: Is Your Multi-modal Model an All-around Player?"
      Apache License 2.0
      •19•312•17•0•Updated May 22, 2025May 22, 2025
    ProTip! When viewing an organization's repositories, you can use the props. filter to filter by custom property.