Tag: Safety and Alignment
-
Undetectable Backdoors: What Model Scanning Cannot Catch
Microsoft shipped a backdoor scanner in February. An ICML 2026 proof says a class of backdoors is undetectable. Both are right. Here is the gap.
-
Governance Decay: How Compaction Deletes Agent Safety Rules
A June 2026 benchmark shows context compaction doesn’t fade an agent’s safety rules, it deletes them outright, and attackers can force the drop.
-
How Researchers Actually Read an LLM’s Mind
Sparse autoencoders and circuit tracing gave us wiring diagrams of LLMs. Then simple probes started winning. The honest state of interpretability in 2026.
-
Do AI Political Ads Need a Disclaimer? Florida’s First Test
A Florida campaign won an order stopping a super PAC mailer under the state’s AI ad disclaimer law. The PAC’s defense: Photoshop, not generative AI.
-
Is ChatGPT a Product? The Lawsuits Forcing an Answer
A Florida filing against OpenAI joins Raine and Garcia in testing whether an LLM is a product that can be defective. The definition decides who pays.
-
Embedding Inversion: Your Vector Database Isn’t Anonymous
Vec2Text recovers 92% of text from embeddings alone. How inversion attacks work, why geometry predicts risk, and what actually defends a RAG pipeline.
-
The Jailbreak Hiding in Your JSON Schema
A CCS 2026 paper hides jailbreaks in JSON schemas, hitting 94-99% success against GPT-5 and Gemini. Why prompt filters never see it.
-
Ghost Vectors: Deleted Embeddings Stay Recoverable
Researchers tested three vector databases and found deleted embeddings stay intact on disk, recoverable at rates that break GDPR and HIPAA.
-
How an Export Law Built for Chips Took Down Fable 5
A private Commerce Department letter used a dormant 2018 export authority to shut down Fable 5 and Mythos 5 worldwide. Here is the mechanism.
-
MCP Goes Stateless on July 28. Its Poisoning Problem Stays
MCP’s July 28 spec removes sessions and adds response caching. That solves scaling headaches and quietly widens the window for tool poisoning.
-
ShareLock Splits Malicious Prompts Across AI Agent Tools
Researchers used Shamir’s secret sharing to hide prompt-injection payloads across MCP tools, beating detectors with a 90%+ success rate.
-
The Annotation Underground: Who Trains AI for So Little
Every frontier model, Claude included, runs on labelers paid under $2 an hour in countries picked for weak labor law. The full wage chain, documented.
-
The Anchor Problem in AI Agent Delegation Chains
OAuth anchors one-hop agent delegation. At the third hop, no deployed protocol can prove which human authorized the action. Inside the 2026 standards race.
-
MITRE ATLAS: The ATT&CK Framework for AI Systems
MITRE ATLAS provides the shared vocabulary for AI security threat intelligence: 14 tactic categories, techniques like AML.T0018 (Backdoor ML Model) and AML.T0043 (Craft Adversarial Data), and a crosswalk…
-
Neural Backdoor Attacks: From BadNets to LLM Trojans
Gu et al.’s BadNets (2017) installed hidden triggers via training poisoning. By 2023, instruction-following backdoors target RLHF pipelines directly. Rare-word triggers, weight poisoning, and universal adversarial triggers all…
-
LLM Watermarking: How Models Embed Detection Signals in Their Outputs
Kirchenbauer’s green-red token list (ICML 2023), Aaronson’s EMS, and Kuditipudi’s ITS scheme all embed detectable statistical signals into LLM outputs. But Zhang et al. proved no watermark is…
-
Differential Privacy for LLMs: The Training Privacy Guarantee
Differential privacy provides the only formal guarantee against LLM training data leakage. DP-SGD’s four steps, the Moments Accountant, Renyi DP, and the epsilon values that actually mean something…
-
Multiagent LLM Security: When Your Agent Talks to a Malicious Agent
When LLMs call other LLMs as tools, injection attacks jump the boundary. ConVerse (2026) found 88% privacy violations and 60% security breaches in plausible agent-to-agent discourse. Here is…
-
LLMail-Inject: What 208K Attacks Against an Email Agent Found
Microsoft Research’s LLMail-Inject challenge: 839 participants, 208,095 unique attacks against a simulated email agent with production defenses. The finding: adaptive attackers breach even well-designed defense stacks. Here is…
-
Adversarial Machine Learning: From Szegedy to LLM Attacks
Szegedy (2014) showed deep networks could be fooled by imperceptible perturbations. FGSM, PGD, and C&W followed. By 2025, the same mathematical framework governs jailbreaks, poisoning, and memorization extraction.…
You must be logged in to post a comment.