Researchers introduce MiJaBench exposing demographic safety gaps in LLMs
Is this a scandal?
No longer — the story is resolved: noise 52/100 · state: Case Closed · 0 source items across 0 platforms · peaked at 57/100 on Jun 23, 2026. — as of , measured by the SCAND.Ai noise pipeline.
Incident ID: SCAND-161892 · see the AI Controversy Index
Cite this incident
"Researchers introduce MiJaBench exposing demographic safety gaps in LLMs." SCAND.Ai incident SCAND-161892, noise 52/100 as of June 23, 2026. https://scand.ai/scandal/researchers-expose-selective-llm-safety-gaps-with-mijabenchWhy It Matters
The discovery of a 'Selective Safety Trap' challenges the reliability of current LLM alignment, showing that safety protocols protect some groups while leaving others vulnerable to identical attacks.
Key Points
- The new MiJaBench framework contains 43,961 controlled jailbreaking prompts across 16 minority groups in English and Portuguese.
- Safety guardrail defense rates varied by up to 42% within individual LLMs depending entirely on the demographic group targeted.
- The safety disparities persist across model sizes, architectures, and languages, and are actually amplified by model scaling.
- Applying targeted Direct Preference Optimization (DPO) allowed a 1B-parameter baseline model to achieve zero-shot safety generalization to unseen demographics.
Researchers have introduced MiJaBench, a bilingual adversarial benchmark designed to systematically evaluate safety disparities across different demographic groups in Large Language Models (LLMs). The study, which analyzed 14 state-of-the-art models, revealed a systemic 'Selective Safety Trap' where models successfully block harmful prompts targeting certain demographics but remain highly vulnerable to identical attacks targeting others. Defense rates within the same model fluctuated by up to 42% depending solely on the targeted minority group, a disparity that persisted across various model architectures and languages. The authors demonstrated that current alignment techniques tend to learn group-specific safeguards rather than generalized harm prevention. To address these vulnerabilities, researchers proposed targeted direct preference optimization (DPO) on a baseline model, achieving zero-shot safety generalizations to unseen demographics and complex jailbreak strategies.
Researchers just exposed a major flaw in how AI models are kept safe, calling it the 'Selective Safety Trap.' Think of it like a security guard who rigorously checks IDs for some guests but lets others walk right in. By testing 14 major AI models against a new benchmark called MiJaBench, they found that safety guardrails fluctuate by up to 42% depending on which minority group is mentioned in an adversarial prompt. This means today's AI models are not actually learning a general rule of 'do no harm,' but are instead memorizing specific safety rules for specific groups.
Sides
Critics
Argues that current LLM safety evaluations create a dangerous illusion of universal protection by masking deep demographic inequalities in guardrail enforcement.
Defenders
No defenders identified
Neutral
Providers of the 14 evaluated state-of-the-art LLMs whose models exhibited safety disparities under the selective safety trap testing.
Noise Level
Forecast
AI developers and evaluators will likely transition away from aggregate safety metrics toward more granular, demographic-specific benchmarks like MiJaBench. This will push frontier model providers to adopt generalized alignment techniques, such as DPO, to ensure uniform safety guardrails before deploying public API updates.
Based on current signals. Events may develop differently.
Timeline
Selective Safety Trap paper published
Researchers release MiJaBench, exposing that LLM safety defenses vary up to 42% depending on the demographic group target.
Join the Discussion
Discuss this story
Community comments coming in a future update
Be the first to share your perspective. Subscribe to comment.