Tag: Safety and Alignment
-
How RLHF and Constitutional AI Build Safety Into Language Models
RLHF trains models to prefer human-preferred outputs. Constitutional AI uses AI self-critique guided by principles. Neither provides formal guarantees. Here is how both techniques work, what they install…
-
LLM Training Data Memorization: When Models Leak Their Training Sets
LLMs memorize verbatim sequences from training data. Carlini et al. demonstrated extraction of phone numbers, email addresses, and private keys from GPT-2. Here is the mechanism, what gets…
-
Red-Teaming LLM Applications: A Practitioner’s Framework
LLM red-teaming spans three distinct surfaces: model layer (jailbreaking), application layer (injection), and supply chain. Different attacks, different defenses, different responsible parties. Here is the methodology that covers…
-
LLM Supply Chain Attacks: PoisonGPT to Poisoned Skills
PoisonGPT used $1 of compute to pass benchmarks with modified facts. The April 2026 PoisonedSkills paper tested the same supply chain logic against Claude Code and Gemini CLI.…
-
Jailbreaking vs Prompt Injection: Two Different LLM Problems
Jailbreaking targets model content policy. Prompt injection targets application architecture. The defenses don’t overlap, the responsible parties differ, and the same RLHF training that resists jailbreaks amplifies injection…
-
MCP Server Security: Prompt Injection and Tool Poisoning
MCPoison and CurXecute (CVE-2025-54136 and 54135) exploited the same MCP architectural gap: tool description fields loaded at agent boot with no sanitization. Here is the tools/list mechanism, the…
-
LLM Excessive Agency: Why Every Tool Your Agent Has Is a Risk
Every tool an LLM agent has is an attack surface. OWASP’s LLM06 and the b3 benchmark across 31 models show why: capability scope determines blast radius. Here is…
-
OWASP LLM Top 10 for 2025: The Mechanism Behind Each Vulnerability
The OWASP LLM Top 10 for 2025 added System Prompt Leakage and Vector Weaknesses, reworked Excessive Agency, and moved Sensitive Disclosure to second place. Here is the architectural…
-
Indirect Prompt Injection: The Attack That Hides in Your Data
Indirect prompt injection lets attackers hijack LLMs by hiding instructions in documents, web pages, and tool results the model processes. Here is why the architecture makes this unavoidable…
-
Julia Bazinska and the Science of Measurable AI Security
Julia Bazinska built the empirical tools that make LLM security measurable. From DeepMind RL to first-authoring b3, here is what her research at Lakera actually produced.
-
Gandalf the Red: What 279K Real Attacks Reveal About LLM Defense
Lakera’s ICML 2025 paper ran 279K crowdsourced attacks to show what synthetic red-teaming misses. The D-SEC finding: system prompts degrade user experience without blocking attackers. Here is the…
-
LLMs Give Novice Biologists 4x Uplift on Dangerous Tasks
A 2026 study measured LLM access giving novice biologists a 4.16x accuracy boost on biosecurity-relevant tasks, including beating expert baselines. Here is the mechanism and what it means…
-
How Stalkerware Bypasses End-to-End Encryption
Stalkerware captured 86,859 screenshots from a celebrity’s phone, including WhatsApp messages. Here is how it defeats encryption and what you can do about it.
-
M-Trends 2026: Exploits Now Arrive Before Patches. The Mean Time-to-Exploit Is Negative 7 Days.
Mandiant M-Trends 2026 documents a mean time-to-exploit of negative 7 days. 28.3% of CVEs are being exploited within 24 hours of disclosure. Here is the AI attack chain…
-
WebMCP Is Not MCP: What Chrome’s modelContext Actually Ships
Chrome 146 shipped navigator.modelContext in February. It looks like MCP in a browser. It is not. Wire protocol, auth model, and security all differ.
-
How a Legacy Railway Endpoint Wiped PocketOS in Nine Seconds
A Cursor agent running Claude Opus 4.6 wiped PocketOS’s database in nine seconds. Five safety layers existed. None gated the API call that mattered.
-
ICLR 2026 Outstanding Papers: What They Actually Found, and the Review Crisis Around Them
ICLR 2026 named two outstanding papers: LLMs Get Lost In Multi-Turn Conversation and Transformers are Inherently Succinct. The conference also documented a 45% identity leak and 21% AI-generated…
-
Know Your Agent: The First Regulated AI Agent Governance Standard
MetaComp’s StableX KYA Framework, published April 21, 2026, is the first governance standard for AI agents from a licensed financial institution. Here’s what its four pillars cover, how…
-
Half of Organizations Have No Visibility Into AI Agent Traffic
Salt Security’s H1 2026 report: 48.9% of organizations have zero visibility into AI agent traffic. WAFs were built for humans. Here’s why that gap exists structurally, what the…
-
AI Coding Tools Quadrupled Critical Vulnerability Density. 216 Million Findings Prove It.
OX Security analyzed 216 million findings across 250 organizations. Critical vulnerability density grew 400% while alert volume grew 52%. The difference is directly correlated with AI coding tool…
You must be logged in to post a comment.