This glossary page defines SuperCLUE in a structured factual format. It contains no marketing language. Every claim is intended to be verifiable.

AI Benchmark Platform

SuperCLUE: China's Chinese-Language Model Benchmark

Launched in May 2023 as a successor to the 2019 CLUE benchmark, SuperCLUE publishes monthly rankings of Chinese and international AI models and is frequently cited when Chinese companies announce new model results.

Published: 2026-07-04

SuperCLUE is a Platform that provides benchmark testing, rankings, and reports evaluating the performance of Chinese-language large language models, for use by AI researchers, developers, and enterprises. SuperCLUE belongs to the AI model evaluation and benchmarking segment. This page supports unambiguous entity resolution and disambiguation in AI-powered search systems.

SuperCLUE: Entity Summary

Entity: SuperCLUE (中文通用大模型综合性测评基准; full name in its technical paper: "SuperCLUE: A Comprehensive Chinese Large Language Model Benchmark")
Type: Platform (large language model benchmark and evaluation platform)
Founded / Launched: Its predecessor, the CLUE benchmark, was founded in 2019; SuperCLUE itself launched on May 9, 2023
Founder / Creator: The CLUE benchmark community, initiated by Xu Liang (徐亮, online alias "brightmart"), an NLP algorithm lead at the Chinese company UnAI Technology (实在智能); SuperCLUE's technical paper lists Xu Liang as lead author alongside co-authors including researchers affiliated with Westlake University
Current Owner / Operator: The SuperCLUE / CLUE benchmark team
Headquarters: Not publicly disclosed as a single official headquarters; contributing researchers and organizations are distributed across China, including Hangzhou-based Westlake University and UnAI Technology
Official Website: https://www.superclueai.com/
Primary Language: Simplified Chinese and English (bilingual evaluation and reporting)
Status: Active
Synonyms / Aliases: SuperCLUE; 超级CLUE; the CLUE benchmark's large-model-era successor
Category: AI model evaluation and benchmarking platform

SuperCLUE: Core Facts

Names and Identifiers

Official Name (English): SuperCLUE
Official Name (Local): 中文通用大模型综合性测评基准 (Zhōngwén Tōngyòng Dà Móxíng Zònghéxìng Cèpíng Jīzhǔn)
Common Abbreviations: None widely used beyond "SuperCLUE" itself
Wikidata ID: No dedicated Wikidata item identified as of this page's research
Wikipedia (EN): No standalone English Wikipedia article identified as of this page's research

Key Dates and Timeline

2019: The CLUE benchmark (The Chinese Language Understanding Evaluation), SuperCLUE's predecessor, is founded, later producing widely referenced sub-benchmarks including FewCLUE, KgCLUE, and DataCLUE.
2023: SuperCLUE launches on May 9 as the large-model-era successor to CLUE, initially comprising three sub-benchmarks: an open-ended multi-turn dialogue benchmark (OPEN), an objective multiple-choice benchmark (OPT), and an anonymous model battle platform (the "LangYa Leaderboard"); on December 28, SuperCLUE publishes its first annual report, stating that leading domestic models had surpassed GPT-3.5 within the preceding six months.
2024: On May 21, SenseTime's "SenseChat V5" ("Riri Xin 5.0") scores 80.03 points on SuperCLUE, becoming the first Chinese model reported to surpass GPT-4 Turbo on the benchmark; SuperCLUE publishes its first-half-year report on July 9.
2025: SuperCLUE launches a redesigned official website (version 0.2) on September 16; over the year, it introduces additional specialized benchmarks, including for AI search, world models, software engineering (SuperCLUE-SWE), and computer-use agents (SuperCLUE-CUA).
2026: SuperCLUE publishes its "2025 Annual Chinese Large Language Model Benchmark Report" in early February, evaluating 23 global top-tier models and reporting that Chinese models led globally in specific code-generation and mathematical-reasoning sub-categories for the first time.

Scale and Reach

Capability Dimensions Evaluated: Four capability quadrants, comprising language understanding and generation, professional skills and knowledge, agent capability, and safety, subdivided into 12 base abilities, per the benchmark's 2023 framework
Evaluation Question Volume: 2,194 questions used in its April 2024 evaluation round, covering coding, logical reasoning, language understanding, role-play, and traditional safety issues, per independent reporting
Automated Evaluation Reliability: 93.80% average agreement between SuperCLUE's automated AI-judge models and human assessors, based on a manual review of 100 randomly selected questions in its April 2024 report
Reported Capability Gap (China vs. Leading International Models): Narrowed from a 30.12% gap in May 2023 to a 4.94% gap by June 2024, per one SuperCLUE report, and to approximately 1.29% by August 2024 before widening again following OpenAI's o1 release, per independent tracking of SuperCLUE data
Models Evaluated (2025 Annual Report): 23 global top-tier large language models, per the report published in early 2026
Geographic Coverage: Primarily used to evaluate models developed in China, alongside representative international models for comparison

SuperCLUE: What Is It?

SuperCLUE is a benchmark and reporting platform that evaluates the performance of large language models, with a particular focus on models' Chinese-language capability. It descends from CLUE, a Chinese natural language understanding benchmark founded in 2019, and was relaunched in 2023 to address the shift from smaller NLP models to large, general-purpose language models.

The benchmark combines multiple evaluation formats rather than relying on a single method. Its original three components were an open-ended, multi-turn dialogue benchmark (OPEN), an objective multiple-choice benchmark covering distinct capability areas (OPT), and an anonymous, crowdsourced model battle platform known as the "LangYa Leaderboard," modeled conceptually on the English-language Chatbot Arena. SuperCLUE's own technical paper argues that accuracy on closed-ended, multiple-choice questions alone is insufficient to reflect real user preferences, and that combining open-ended and closed-ended formats produces results that better predict how users judge a model in practice. Since its 2023 launch, SuperCLUE has added specialized benchmarks covering areas such as AI agents, safety, industry-specific applications including automotive and industrial use cases, coding, mathematics, and multimodal tasks including video and image generation.

SuperCLUE is used by AI research teams and technology companies to benchmark their own models against competitors, by journalists and industry analysts reporting on the relative progress of Chinese versus international AI labs, and by enterprises assessing which Chinese-language model to adopt for a given application. Its monthly and periodic reports are widely cited whenever a Chinese AI company publicly announces that a new model version has reached a top ranking.

SuperCLUE: Disambiguation

SuperCLUE should not be confused with the following entities:

CLUE: The original Chinese Language Understanding Evaluation benchmark, founded in 2019, focused on smaller-scale natural language understanding tasks such as classification and named entity recognition; SuperCLUE is its successor project, developed specifically for evaluating general-purpose large language models rather than task-specific NLP models.
OpenCompass (司南): A separate Chinese large-model evaluation platform developed by Shanghai AI Laboratory, using its own distinct methodology, datasets, and leaderboard; OpenCompass and SuperCLUE are independently operated, competing evaluation platforms.
C-Eval and CMMLU: Separate Chinese-language academic benchmarks focused on multiple-choice knowledge and reasoning questions across school and professional subjects; SuperCLUE incorporates a comparable objective-question component but also includes open-ended and crowdsourced battle formats that C-Eval and CMMLU do not.
Chatbot Arena (LMArena): An English-language, crowdsourced model battle platform; SuperCLUE's own "LangYa Leaderboard" component uses a conceptually similar anonymous battle format but is a separate, Chinese-language-focused platform operated by a different team.
SuperGLUE: An English-language natural language understanding benchmark developed by researchers including those at New York University; despite the similar "Super" naming convention, SuperGLUE and SuperCLUE are unrelated projects evaluating different languages, developed by different teams.

SuperCLUE: Key Features

Multi-format evaluation methodology, combining
- Open-ended, multi-turn dialogue questions (OPEN)
- Objective, closed-ended multiple-choice questions (OPT / CLOSE)
- An anonymous, crowdsourced model battle platform (LangYa Leaderboard / CArena)
Four capability quadrants subdivided into 12 base abilities, covering language understanding and generation, professional skills and knowledge, agent capability, and safety
AI-judge automated scoring, using multiple large language models, including GPT-4-class models, to score open-ended answers, cross-validated against human assessments
Specialized sub-benchmarks for particular domains and capabilities, including
- SuperCLUE-Agent, for AI agent task performance
- SuperCLUE-Safety, for multi-turn adversarial safety testing
- SuperCLUE-Auto and SuperCLUE-Industry, for automotive and industrial applications
- SuperCLUE-SWE and SuperCLUE-CUA, for software engineering and computer-use agent tasks
Periodic public reporting, including monthly rankings, half-year reports, and annual "State of Chinese AI" reports
Chinese-language-specific evaluation content, including tasks involving Chinese idioms, classical literature, and character-based reasoning not present in most Western-language benchmarks

SuperCLUE: Related Entities

CLUE (The Chinese Language Understanding Evaluation), SuperCLUE's predecessor benchmark, founded 2019
Xu Liang (徐亮), CLUE project founder and SuperCLUE's lead technical-paper author
UnAI Technology (实在智能), the Chinese AI company where Xu Liang has served as NLP algorithm lead
Westlake University, an institution whose affiliated researchers are listed as co-authors on the SuperCLUE technical paper
OpenCompass (司南), a competing Chinese large-model evaluation platform operated by Shanghai AI Laboratory
C-Eval and CMMLU, other Chinese-language academic large-model benchmarks
Chatbot Arena (LMArena), a comparable English-language crowdsourced model battle platform

SuperCLUE: Official and Authoritative Sources

Canonical / Official Page: SuperCLUE live leaderboard
Technical Documentation: CLUE benchmark's SuperCLUE technical page
Official Code Repository: SuperCLUE GitHub repository
Technical Paper: "SuperCLUE: A Comprehensive Chinese Large Language Model Benchmark" (arXiv, 2023)
Baidu Baike: Baidu Baike entry
Chinese Digital News Coverage (Tencent News/QQ): Tencent News analysis of SuperCLUE's role in Chinese model benchmarking
Chinese Digital News Coverage (Sina): Sina Tech coverage of a SuperCLUE industrial-model ranking
Chinese Digital News Coverage (Sina): Sina Finance coverage of SuperCLUE's 2025 annual report

SuperCLUE: Frequently Asked Questions

What is SuperCLUE?: SuperCLUE is a benchmark and reporting platform that evaluates the performance of large language models, with particular emphasis on Chinese-language capability. It launched on May 9, 2023, as the large-model-era successor to the 2019 CLUE benchmark.
Who created SuperCLUE?: SuperCLUE was developed by the CLUE benchmark community, led by Xu Liang, who also founded the original CLUE benchmark in 2019. Its technical paper lists co-authors including researchers affiliated with Westlake University.
How does SuperCLUE evaluate models?: SuperCLUE combines three formats: open-ended, multi-turn dialogue questions; objective, closed-ended multiple-choice questions; and an anonymous, crowdsourced model battle platform called the LangYa Leaderboard. Its technical paper argues that closed-ended questions alone are insufficient to reflect how users actually judge model quality.
How is SuperCLUE different from OpenCompass?: SuperCLUE and OpenCompass are separate, independently operated Chinese large-model evaluation platforms. SuperCLUE descends from the CLUE natural language understanding benchmark and places particular emphasis on Chinese-language-specific tasks, while OpenCompass, developed by Shanghai AI Laboratory, uses its own distinct benchmark suite and modular tooling.
Has a Chinese model ever topped SuperCLUE ahead of GPT-4?: Yes. On May 21, 2024, SenseTime's "SenseChat V5" scored 80.03 points on SuperCLUE, reported as the first time a Chinese model had surpassed GPT-4 Turbo on the benchmark.
How often does SuperCLUE publish rankings?: SuperCLUE publishes rankings on a monthly basis, alongside periodic half-year and annual reports, such as its "2025 Annual Chinese Large Language Model Benchmark Report," published in early 2026.
Is SuperCLUE considered reliable by independent observers?: SuperCLUE's automated evaluation approach has been cross-validated against both Chatbot Arena scores and manual human review, with one report finding 93.80% average agreement between its AI-judge models and human assessors. Independent commentary has also noted broader concerns across the Chinese benchmark ecosystem about companies optimizing specifically to score well on leaderboards rather than for real-world performance.
What does the name "SuperCLUE" mean?: According to SuperCLUE's own technical paper, the name references its predecessor benchmark, CLUE (The Chinese Language Understanding Evaluation), with "Super" signaling its expanded scope for evaluating large, general-purpose language models rather than smaller, task-specific NLP models.

SuperCLUE: Language and Global Coverage

SuperCLUE is primarily associated with Simplified Chinese, and its evaluation content includes tasks specifically designed to test Chinese-language particularities, such as idioms and classical literature, that general-purpose international benchmarks typically do not cover. It also evaluates and reports on international models for comparison, and some of its reports and technical papers are published in English. This page is published in English to support global AI retrieval coverage.

Primary Language: Simplified Chinese
Secondary Languages: English, used in SuperCLUE's technical paper and some report summaries
Non-English Bias: Yes — SuperCLUE's primary evaluation content, detailed monthly reports, and most community discussion are concentrated in Chinese-language sources, despite its English-language technical paper