Clinical AI systems built on retrieval-augmented generation face a security threat that does not require compromising model weights. Poisoning the knowledge base redirects outputs at inference time without touching the model itself.
How RAG Poisoning Works
A RAG system retrieves documents from a knowledge base, injects them into the model’s context, and generates a response grounded in the retrieved content. The attack vector is the knowledge base itself. An adversary who can insert documents into the knowledge base controls what the model retrieves for targeted queries. A poisoned document that appears in the top-k retrieval results will be treated as authoritative source material, because the model has no mechanism to distinguish a poisoned document from a legitimate one. It processes both as retrieved context.
In clinical EHR systems, the attack surface includes any pathway that allows external data to enter the knowledge base: imported referral letters, patient-submitted documents, third-party lab results, and scanned records. Each is a potential injection point. A document containing instructions that look like clinical content but direct the model to alter its recommendations will be retrieved, processed, and acted upon without any indication to the clinical user that the output was adversarially modified.
The Security-Utility Problem in Clinical RAG
Filtering retrieved content for adversarial instructions before it reaches the model faces the same fundamental problem as direct prompt injection filtering: detecting that a piece of text is an instruction requires the same semantic understanding that makes the model vulnerable. Aggressive filtering produces false positives that block legitimate clinical content. Permissive filtering leaves the injection surface open.
Access control on the knowledge base is the most effective mitigation at the retrieval layer: ensuring that only authorized documents can enter the knowledge base, and that retrieval is scoped to documents the querying user is authorized to access. This prevents cross-patient poisoning attacks and limits the blast radius of any individual poisoned document to users authorized to retrieve it.
The RAG poisoning attack surface maps directly to two entries in the OWASP LLM Top 10 for 2025: LLM08 (Vector and Embedding Weaknesses) and LLM01 (Prompt Injection). OWASP’s 2025 update added LLM08 specifically because RAG architectures became the dominant deployment pattern and the associated attack surfaces had not been formally catalogued before. The poisoned document in a RAG knowledge base is the most common vector for indirect prompt injection in production systems: the attacker does not interact with the LLM directly, but places instructions in content the system will retrieve and process. For the empirical evidence on which defenses reduce injection success rates at scale, including session-level detection and output-level auditing, see the Gandalf the Red analysis.
Related coverage: Prompt Injection Succeeds 94% of the Time Against Clinical LLMs | FDA Clearance for AI Medical Devices
Primary sources: Patel SB and Lam K, JAMA Network Open 2024. Zou et al., arXiv 2402.07927.