FUSE framework finds newer LLMs retain dangerous capabilities

Historically, AI safety research has grappled with the 'alignment tax' hypothesis, which posits that optimizing for safety may degrade model utility or that capability gains outpace safety interventions. Prior evaluation benchmarks often treated safety as a unidimensional metric or assumed monotonic improvement with scale. However, recent literature in mechanistic interpretability and red-teaming has increasingly documented 'non-monotonic' behaviors, where intermediate model sizes or versions exhibit higher failure rates than both smaller and larger counterparts. This pattern mirrors findings in complex systems engineering where increased system complexity introduces emergent failure modes not present in simpler iterations. The FUSE framework's separation of knowledge, defense, and harm aligns with evolving academic consensus that safety is multi-dimensional. Previous governance frameworks have struggled to standardize these metrics, leading to fragmented reporting where providers define safety boundaries differently. This lack of standardized, orthogonal evaluation has historically obscured whether safety improvements are robust or merely artifacts of specific benchmark overfitting.

On September 3, 2026, researchers published a preprint on arXiv introducing FUSE, a modular evaluation framework designed to assess dangerous capabilities in large language models (LLMs). According to the paper, the authors argue that current safety evaluations are too fragmented to effectively govern AI risks, proposing instead a unified protocol that tests models across three orthogonal dimensions: Knowledge ($K$), Defense ($D$), and Harm ($H$). The study evaluated 12 commercial LLMs from four distinct model families using a chemical-biological (CB) module, alongside a cyber-security pilot to demonstrate protocol transferability.

The core controversy stems from the study's temporal analysis of capability evolution. The authors report that dangerous capabilities have not monotonically declined as models have scaled. Specifically, the paper states that newer models demonstrate deeper knowledge of hazardous domains while only partially improving their defense mechanisms. This finding directly challenges the prevailing industry assumption that scaling and alignment progress uniformly translate into increased safety. The research indicates that models with comparable knowledge levels can exhibit sharply divergent refusal resilience, and that strong defenders do not necessarily generate less harmful content when they do comply with unsafe requests.

Furthermore, the FUSE framework reveals distinct family-level patterns that separate Claude, DeepSeek, and GPT models, suggesting that safety profiles are heavily influenced by specific training methodologies rather than scale alone. The authors contend that this non-monotonic safety trend undermines governance efforts, as regulators and deployers cannot rely on model versioning as a proxy for risk reduction. While commercial providers have implicitly positioned newer releases as safer through iterative alignment, the FUSE data suggests these gains are uneven and domain-dependent. The paper establishes reliability via cross-validation but acknowledges the evaluation is currently instantiated primarily through chemical-biological and cyber modules, leaving open questions about generalizability to other hazard domains. The release has highlighted a potential gap between capability scaling and safety alignment, complicating the narrative that larger, newer models are inherently more benign.

Regulators and enterprise deployers relying on model versioning as a safety proxy face elevated risk as FUSE demonstrates newer models may possess deeper hazardous knowledge without commensurate defenses. Commercial LLM providers must now defend safety claims with multidimensional evidence rather than scaling narratives, potentially slowing release cadences for additional red-teaming. The 12-model evaluation covering four families suggests this is a systemic issue, not vendor-specific. Governance frameworks assuming monotonic safety improvement require immediate revision to account for orthogonal K/D/H dynamics. Research funding may shift toward mechanistic defense alignment over pure capability scaling. Users in high-stakes domains (bio, cyber) cannot assume latest models are safest.