The Safety–Openness Pareto Frontier: Technical Report

Authors: Jinge Wang, Weibing Wang, Liang Fang, Brian Tse

Published: September 21, 2026

AI-translated English version

Introduction

Should frontier AI models become more open or more closed? This has remained an active debate in AI governance. The case for openness is that it encourages technical innovation and knowledge sharing, broadens global access, and enables independent researchers to audit and reproduce model behavior. Transparency about training methods and data is also a basic prerequisite for external scrutiny; open weights can help prevent power from concentrating in a few companies and make it easier for the safety community to conduct research. The case for caution is that unrestricted openness may become dangerous as models approach practical thresholds in sensitive areas such as cyber offense and biological risk. Model safeguards can also be fine-tuned away once weights are released, whereas proprietary deployment preserves at least one layer of control.

This debate has long lacked a common measuring stick: different parties often mean very different things when they call a model "open" or "safe." The platform's new Safety–Openness Pareto Frontier page places both dimensions on a single chart. It quantifies nearly one hundred frontier models under monitoring, placing the Openness Index on the x-axis and the Safety Score on the y-axis, and marks the Pareto frontier—the current best boundary of safety–openness trade-offs. The analysis asks three concrete questions: Must greater safety come at the expense of openness? Are there models that perform well on both? And where does the industry's current best trade-off boundary lie?

What is the Pareto frontier? Originating in economics, the Pareto frontier is the set of best trade-off solutions in a multi-objective optimization problem, also known as the Pareto-optimal boundary. No solution on this frontier can be improved in one objective without worsening at least one other objective. In this report, the two objectives are a model's safety and openness. For any model on the frontier, there is no alternative that is strictly better on both dimensions. Put simply, moving up and to the left along the frontier trades openness for safety; moving down and to the right trades safety for openness; the models that balance both objectives best lie toward the upper right.

This article is the companion technical report. It first explains what the chart shows, then defines the two scoring systems, describes the available interactive controls, and finally presents the data sources, interpretation of the current results, and limitations.

What the Chart Shows

Figure 1: The safety–openness Pareto frontier (interactive chart, using the same data source and update cycle as the platform page). Each colored dot is a frontier AI model, colored by company (the legend at the bottom is clickable). The gray dashed line is the Pareto frontier; frontier models are labeled directly, and hovering over any dot opens the model's full profile.

Each point represents one frontier AI model, positioned by two coordinates:

The x-axis is the Openness Index (0–100; further right means more open). It measures openness across model-weight availability, training transparency, and model disclosure, combining the Artificial Analysis Openness Index ("AA Openness Index") with the platform's model-disclosure research. The algorithm is described in the methodology.
The y-axis is the Safety Score (0–100; higher means safer). This is the platform Safety Score—a weighted blend of base, jailbreak, and tamper safety scores, as defined in the methodology—and directly measures evaluated safety behavior.

The gray dashed line in the chart is the Pareto frontier. In Figure 1, the frontier consists of two groups meeting end to end. On the left are the highly safe, fully disclosed proprietary models Claude Opus 4.8 and Claude Sonnet 4.5. Moving right are the open-weight models Qwen 3.5 397B Reasoning, GLM 5, and GLM 4.5, which differ in both disclosure practices and overall openness.

The chart has three notable features:

First, proprietary models cluster on the left. Their AA openness scores are generally very low.
Second, the frontier slopes downward from upper left to lower right. Along this boundary, openness and safety are negatively correlated, and the region where both are high remains sparse.
Third, the frontier is formed by fully disclosed proprietary models at one end and open-weight models at the other. Claude represents the safe proprietary end, while GLM and DeepSeek represent high openness.

Methodology

X-axis: Openness Index

The platform Openness Index builds on the AA Openness Index and adds a parallel "model disclosure" dimension:

Platform Openness Index = (AA raw score + disclosure raw score) / 23 × 100

AA raw score (0–18): Converted from the AA Openness Index (0–100) as AA Openness Index × 18 / 100. Its four components have the following ranges: model availability 0–6, methodology transparency 0–6, post-training data transparency 0–3, and pre-training data transparency 0–3. For models missing from the AA Openness Index, proprietary models monitored by this platform are all accessible via API, so model availability receives the minimum raw score of 1 and the other AA components are set to 0; a proprietary model with genuinely no API access remains 0. Open-weight models with no AA match are treated as missing data and are not included.
Disclosure raw score (0–5): The platform's model-disclosure dimension, consisting of five binary items defined below.

We use the AA Openness Index as the foundation for two reasons: it is a mature, publicly available, continuously updated dataset covering more than two hundred models, and it lets this project build on an existing consensus measure.

AA alone, however, is not enough. It measures how open the model itself is—weight availability and transparency about training methods and data—but not how much information the company publishes around the model. Openness should also cover complementary information such as capability evaluation results, safety evaluation results, model specifications, risk-management policies, and system prompts. Two proprietary models may receive the same AA score even though one publishes a detailed system card and formal behavioral specification while the other discloses almost nothing; that difference matters greatly to policymakers and safety researchers. The disclosure dimension fills this gap.

AA Openness Index

The AA Openness Index measures how open the model itself is. It combines four components into a raw score of 0–18 (the AA website normalizes this to 0–100):

Component	Range	What it measures
Model availability	0–6	Availability of weights and inference code, and license terms (downloadability, commercial use, usage restrictions)
Methodology transparency	0–6	Disclosure of training methods, architecture design, and related development information
Post-training data transparency	0–3	Disclosure of sources and composition of data used for alignment and fine-tuning
Pre-training data transparency	0–3	Disclosure of sources and composition of pre-training data

See the AA methodology page for detailed scoring criteria.

Model Disclosure Score

The model-disclosure dimension contains five sub-items. Each model is assessed through AI-assisted research followed by human review. The research covers 128 models from 14 companies, and each sub-item is scored 0 or 1, for a total of 0–5.

Sub-item	Meaning	Models passing
Capability evals	Whether the company publishes model-specific capability evaluation results	116/128
Safety evals	Whether the company publishes model-specific safety evaluation results	49/128
Model spec	Whether a written document defines the model's behavioral logic, values, and safety boundaries	24/128
Risk management	Whether the company has a written model-risk management policy covering risk identification, assessment, and mitigation	53/128
System prompt	Whether the company publishes the default system prompt	15/128

Several scoring boundaries require further explanation:

Capability evals and safety evals are separate items. Publishing benchmark scores has become an industry norm at model release (91% of models pass), but only 38% publish safety evaluation results. Keeping the two items separate makes this gap explicit.
Model spec counts only substantive written behavioral specifications—documents that define behavioral logic, values, and safety boundaries. Current positive examples are the Anthropic Constitution, the OpenAI Model Spec, and Alibaba's Model Specification. Technical specifications such as parameter count, architecture, and context length do not count (that information is already part of the AA openness score), nor do user policies, employee codes of conduct, or generic company-level AI principles pages. Under this rule, 24 of the 128 models pass. If an enterprise-wide specification was published after a model's release, that model receives 0.
System prompt counts only an officially stated default system prompt known to be used in the real product. Prompts marked as "recommended," "strongly recommended," or Quick Start examples in developer documentation do not count. Positive examples include the DeepSeek model card's statement, "In the official DeepSeek web/app, we use the same system prompt," and MiniMax-M2.7's README label "Default system prompt." By contrast, the system message in Moonshot's K2 model-card Chat Completion example is annotated "the system prompt is a good default," alongside a recommended temperature; because this is advisory language rather than confirmation of real product use, it does not count.
Risk management asks whether the company has a written risk-management policy for its own models. A policy that covers model-risk identification, assessment, and mitigation is sufficient; it need not meet the level of a frontier risk-grading framework such as the Preparedness Framework, Responsible Scaling Policy, or Frontier Safety Framework. Two restrictions apply. Timeliness: the policy must predate the model's release, and models do not receive credit for policies published later. Relevance: risk management must target a specific model or model family, not a cloud platform or downstream product. For example, ByteDance's Volcano Engine Generative AI Security White Paper is aimed at cloud customers and never mentions the Seed/Doubao model line, so it does not count.

The precise definition, evidence hierarchy, and exclusions for each item are given in Appendix A; representative boundary cases are listed in Appendix B. Research evidence is limited to official channels. Third-party evaluations, leaks, and reverse-engineered materials are never counted.

The item design draws on two international evaluations—the FLI AI Safety Index (Summer 2026) and the Stanford FMTI (December 2025)—which include corresponding indicators for system prompts, behavioral specifications, risk management, and capability and safety evaluations. The difference is that those international indices evaluate companies and cover broad governance and accountability topics, whereas this platform's Openness Index makes model-by-model judgments and is not limited to a safety perspective.

Y-axis: Safety Score

The y-axis answers "How safe is this model?" It uses the platform Safety Score (0–100), consistent with the Risk Index: a weighted blend of the Base Safety Score (the average of non-red-team safety benchmarks), the Jailbreak Safety Score (the average of red-team safety benchmarks), and the Tamper Safety Score (0 for open-weight models and 100 for proprietary models), weighted 0.4 : 0.4 : 0.2. The loss-of-control domain does not have separate jailbreak or tamper components; its safety-benchmark average is included directly. The Safety Score is a behavioral measure based on benchmark performance, such as refusal of harmful requests and aligned behavior; it does not infer safety from capability. Higher is better, and the ideal region is the upper right.

Models without safety-benchmark data are not shown.

How to Use the Chart

The chart supports two groups of switches and a sortable data table. This section explains what each option means and how to use it.

Figure 2: The two groups of switches above the chart—minimum capability score and release date. This interactive example matches the appearance of the switches on the platform page.

Minimum Capability Score

This filter selects models by capability level. The Capability Score is the average of capability scores across cyber offense, biological risks, chemical risks, harmful manipulation, and loss-of-control (0–100), with thresholds at 35, 40, 45, 50, and 55.

Smaller or older models may score far below the current capability frontier, so even a model on the safety–openness Pareto frontier may not be useful to a user seeking frontier-level capability. The capability threshold therefore helps readers focus on the capability tier they care about. More importantly, it reveals who remains on the frontier among the strongest models: only 22 models remain at ≥50 and 14 at ≥55.

The thresholds are not arbitrary. They are rounded cut points chosen from the capability-score distribution so that each tier retains a sufficient sample, corresponding to roughly 58, 51, 36, 22, and 14 models.

Release Date

This filter selects models by release date: last two years, last year, or last six months. It reveals how the frontier moves over time. The last-six-month view shows the current generation's trade-off landscape; comparing it with "all time" reveals whether the industry is moving toward greater safety and openness or retreating toward closure.

When either capability or release date changes, the axis scales remain fixed and points filtered out simply leave the view. This makes the before-and-after states directly comparable: it is easy to see which points disappeared and how the Pareto frontier shifted.

Legend, Hover Tooltips, and Data Table

The legend at the bottom colors points by company. Clicking a company name hides or shows all of its points. Together with the "select all / invert" buttons, readers can focus on any combination of companies.
Hovering over a point opens a tooltip with the model's full profile. The first section gives the model name and company. The second shows its Safety Score, Capability Score, and overall Openness Index. The third breaks the Openness Index into its two components: AA openness (0–18) and model disclosure (0–5). The fourth lists the five model-disclosure items individually.
The data table below the chart shares the chart's filter state: any filter applied to the scatter plot also applies to the table. The table itself cannot be filtered, but any column can be used for sorting (the default is descending Openness Index). Its five disclosure columns make it possible to verify each model's item-level results.

Data Sources

The chart combines two sources with different update cycles.

AA Openness Index data (current snapshot: September 8, 2026): AA does not provide a public API for the Openness Index. We obtain the full dataset by extracting the flight data embedded in its model pages (covering 227 models on the AA side), then map it to platform models using a manually maintained model-mapping table. Mapping applies fuzzy handling to different inference variants of the same model (suffixes such as -reasoning, -nothinking, and -high).

Model-disclosure data (last reviewed September 21, 2026): Produced through AI-assisted research and human review, covering 128 platform models across 15 company series: Claude (Anthropic), GPT (OpenAI), Gemini (Google DeepMind), Grok (xAI), Qwen (Alibaba Qwen series), Kimi (Moonshot AI), GLM (Zhipu AI), MiniMax, Hunyuan (Tencent), DeepSeek, Llama (Meta), MiMo (Xiaomi), Doubao (ByteDance), Baidu, and an "other companies" collection.

The two datasets cover different model sets. Disclosure research covers 128 models, but the Pareto chart requires both a platform Safety Score and disclosure data. At present, 62 models satisfy all conditions and form the full scatter set in Figure 1; all 128 researched models are included in the model-disclosure analysis below.

Results

The following results use data as of September 21, 2026 and proceed from chart comparisons to disclosure-score distribution, item pass rates, and company-level patterns.

Comparing the Distribution Under Different Filters

First comparison: all models versus Capability Score ≥45.

Figure 3: Left, all capability scores (the same view as Figure 1); right, Capability Score ≥45 (36 models remain). Both charts use identical axis scales, so point positions correspond one-to-one. Hover over a point or click a company in the legend for details.

Raising the capability threshold from "all" to 45 reduces the chart from 62 to 36 models and reveals two patterns.

First, the right end of the frontier moves upward. The most open frontier model changes from GLM 4.5 (openness 52.2, safety 41.6) to GLM 5 (openness 43.5, safety 55.7): openness decreases, but safety improves. The models filtered out may have lower safety scores because their capabilities have not yet reached the platform's risk threshold and therefore may not have received the same level of safety hardening. If stronger new models can retain GLM 4.5's level of openness, they could extend the frontier farther toward the upper right.
Second, the left side of the frontier remains stable. Claude Opus 4.8, the safest model at 85.8, has a Capability Score of 62.3; Claude Sonnet 4.5, close behind at 85.4, scores 48.2. Raising the threshold does not affect these leading proprietary models—the strongest safety level does not require low capability.

Second comparison: all release dates versus the last six months.

Figure 4: Left, all release dates; right, models released in the last six months (12 models remain). The axis scales are identical. "Last six months" is rolling relative to the page-view date, so this count will grow slowly as new models are released.

In the full view (Figure 4, left), the frontier runs from the upper left—Safety Scores around 85—down toward the right, where the most open model has a safety score of only about 41–42. Along the boundary, openness and safety are negatively correlated. In the last-six-months view (Figure 4, right), 12 newer models remain. The distribution has not deteriorated: safety scores span roughly 40–86, and the right end of the frontier sits at about 51, above the right endpoint in the full view. In other words, models released in the last six months have not retreated on the openness–safety trade-off; the right endpoint has improved slightly. The sample is small, however, so this observation should be treated as provisional.

Analyzing the Model Disclosure Score

The Model Disclosure Score is this platform's novel extension to the Openness Index and reveals several useful patterns.

Overall Distribution of Disclosure Scores

Figure 5: Distribution of disclosure scores across the 128 models (maximum 5 points; mean 2.01). Interactive chart using the manually reviewed snapshot from September 21, 2026 (the same snapshot as Figures 5–7). Hover over a bar to see the number and percentage of models in each score band.

Among the 128 models, 10 score 5 points, 14 score 4, 14 score 3, 26 score 2, 57 score 1, and 7 score 0, for a mean of about 2.01—less than half the maximum. The distribution is concentrated at 1–2 points: most of the 57 models with 1 point pass only capability evals, while the 26 models with 2 points typically pass capability evals plus either safety evals or risk management. In other words, a typical frontier model publishes benchmark scores but not safety evaluation results, has no written behavioral specification or risk-management policy, and does not publish its default system prompt.

Pass Rates for the Five Items

Figure 6: Pass rate for each of the five disclosure items, sorted from highest to lowest. Hover over a bar to see the number of passing models.

Sorted by pass rate, the five items vary widely in how established they are across the industry. Capability evals (91%) are the only genuine industry norm, reflecting the standard practice of publishing benchmark scores at model release. The other four items all pass for fewer than half of the models. Risk management (41%) and safety evals (38%) occupy the middle: roughly 60% of companies lack a written model-risk management policy, while the 38% safety-evaluation pass rate trails capability evals by more than 50 percentage points. Model spec (19%) and system prompt (12%) remain minority practices. Only three written behavioral specifications currently qualify—the Anthropic Constitution, OpenAI Model Spec, and Alibaba's Model Specification—and only selected models from Anthropic, xAI, MiniMax, DeepSeek, and Meta publish default system prompts.

Patterns by Model Family

Figure 7: Average disclosure score by model family (family names appear to the right of the horizontal bars; red means average ≥4, blue means 2.5–4, and gray means <2.5; the dashed line marks the overall mean of 2.01). Hover over a bar to see the number of models.

Anthropic leads with a perfect average of 5.0 and is the only company where every model passes all five items. Its ten models benefit from a comprehensive disclosure system that includes the Constitution, the Responsible Scaling Policy, public system-prompt documentation, and model-by-model system cards.
All 13 OpenAI models score consistently between 3 and 4 points, supported by two family-wide documents: the Model Spec and the Preparedness Framework. Their losses are also consistent: system prompts are never published.
Google DeepMind (3.0) and Meta (2.33) form the second tier. Google publishes evaluation results and the Frontier Safety Framework, but has no model-spec document and does not publish default system prompts. Meta's Llama 4 Maverick scores 4 points, benefiting naturally from its open-weight release, while Llama 3.x models remain at 2 points because Meta's Frontier AI Framework was published after those models.
xAI averages 1.57 but has the largest internal spread. Grok 4 scores 4 points with its model card and public default-prompt repository; Grok 4.20 scores 2 points with its model card and risk-management framework; older models score only 1 point.
Chinese companies cluster between 0.6 and 2.0 points, with meaningful internal differences and a few bright spots. Tencent Hunyuan passes risk management for every model thanks to written research such as the Large Model Safety and Ethics Research Report 2024. Moonshot AI (1.4) earns all of its points from capability evals. qwen3.7-max is the only Chinese model to receive the model-spec point through Alibaba's Model Specification. Leading models from DeepSeek, Zhipu AI, and MiniMax mostly score between 1 and 3 points.

Limitations

Disclosure items are binary 0/1 judgments. They record whether information is made public, not the quality, depth, or reproducibility of that information. A hundreds-of-page system card and a short safety statement both count as 1. The judgment rules are given in Appendix A, but some boundary cases still require reviewer judgment; representative examples appear in Appendix B.
Openness is not unconditionally better. The Openness Index only measures openness and does not imply that more openness is always preferable. Excessive openness can make it easier for attackers to exploit model vulnerabilities and may increase security risks.
Disclosure evidence may be shared within a model family. Because some evidence is applied family-wide, models from the same family often receive the same disclosure score, appearing as vertical columns of points at the same x-coordinate. This is a limitation of the current evaluation system and does not mean their disclosure practices are identical.
The Safety Score covers only benchmarked safety dimensions. It is a fixed-weight composite of the platform's current safety benchmarks; safety capabilities not included in those benchmarks are outside the y-axis, and extrapolating benchmark results to real-world safety involves uncertainty.
Only part of the model landscape is covered. Disclosure research covers 128 models, the AA Openness Index covers 227, and the platform's capability and safety scores cover 70 models; only 62 satisfy all conditions at once. The Pareto chart shows only these 62 models.

Appendix A: Complete Judgment Rules for Model Disclosure

The following are the complete judgment rules for model disclosure (version September 15, 2026; six rules):

System prompt: Advisory language such as "recommended," "strongly recommended," or Quick Start examples is always scored 0; only an officially stated default system prompt counts as 1.
Evidence priority: Model-specific evidence takes precedence over family-wide or system-shared evidence.
Model spec is limited to substantive written behavioral documents ("model constitutions," values documents, or behavioral specifications) that define behavioral logic, values, and safety boundaries. Positive examples include the Anthropic Constitution, OpenAI Model Spec, and Alibaba's Model Specification. Technical specifications such as parameter count, architecture, context length, training-data volume, and knowledge cutoff do not count. Exclusions are acceptable-use policies or terms of service, safety/risk-framework documents (which belong to the risk-management item), employee codes of conduct, and generic company-level AI-principles pages. Timeliness rule: if the document was published after the model's release, the model receives 0.
Risk management asks whether the company has a written risk-management policy for its own (frontier) models. A written policy covering model-risk identification, assessment, and mitigation is sufficient; it need not include frontier risk levels, capability thresholds, or coverage of frontier risks such as biological or loss-of-control risks (frameworks such as the Preparedness Framework, Responsible Scaling Policy, and Frontier Safety Framework are sufficient but not necessary). Still excluded are infrastructure-security programs for model assets or content moderation, corporate strategy or ESG content that does not address model risk management, joint initiatives, and verbal statements.
Timeliness and entity rules for risk management: (i) the policy must predate the model's release, and later policies do not add points; (ii) risk management must target a specific model or model family rather than a cloud platform or downstream product; (iii) ESG or securities-compliance disclosure channels may count if their content qualifies, regardless of channel; (iv) a values document under the model-spec item must be a standalone written document—a one-sentence values statement inside a model card is insufficient.
Scope and evidence: (i) the assessment is limited to frontier models from mainstream companies; (ii) generic company-level AI-principles pages do not count as values documents; (iii) for system prompts, an officially stated default prompt counts whether it is used in the API or a web chatbot; (iv) non-open models are assessed as proprietary models; (v) items with disputed evidence strength remain conservatively scored.

Appendix B: Representative Boundary Cases

The following cases show how the judgment rules are applied to specific models and are intended to keep future reviews consistent:

ByteDance / Volcano Engine white paper (risk management = 0): The Generative AI Security White Paper, published by Volcano Engine for its cloud-platform customers, never mentions the Seed/Doubao model line. Under the relevance rule in Appendix A, rule 5(ii), all 14 models receive 0.
Alibaba Bailian Security White Paper (risk management = 1 for some models): Chapter 4 of the Alibaba Cloud Bailian Security White Paper, "Full-Lifecycle Security Governance for Tongyi Large Models," covers model-risk management, and Alibaba Cloud is the developer of the Tongyi model line—fundamentally different from the ByteDance case. Applying the timeliness rule in Appendix A, rule 5(i), only models released after October 14, 2025 (qwen3-max-thinking, qwen3.7-max, qwen3.5-plus, and qwen3.5-27b) receive 1; earlier models receive 0.
Values statements embedded in Meta model cards (model spec = 0): A one-sentence values statement in a model card does not constitute a values document, which must be a standalone written document. All six Llama models receive 0.
Alibaba's Model Specification and the open-weight Qwen line (model spec = 0): The Model Specification (April 2026) postdates all 17 open-weight Qwen models, so those models receive 0 under the timeliness rule; only qwen3.7-max, released after the document, receives 1.
MiniMax ESG/prospectus disclosures (risk management = 1 for some models): ESG and securities-compliance disclosure channels may count if their content qualifies. Applying the timeliness rule, only m2.7—which postdates the prospectus dated December 31, 2025—and m3 receive 1.
Moonshot's "good default" wording (system prompt = 0): The system message in the K2 model-card Chat Completion example is annotated "the system prompt is a good default," alongside a recommended temperature. This is advisory language for deployers, not confirmation that Moonshot's online product actually uses the prompt. It therefore differs from the DeepSeek model card's statement, "In the official DeepSeek web/app, we use the same system prompt," which explicitly confirms real product use and receives 1.
Baidu Qianfan white paper (risk management = 0): The Qianfan Large Model Platform Security White Paper covers platform-infrastructure security and content compliance; its "model security" section concerns encryption and theft prevention for model assets, not model-risk management, and therefore fails rule 4.

Disclaimer: This project provides a reference for the relative safety and openness of models and does not constitute a recommendation for any particular use.

Table of Contents