A RAG system does not fail only because of the model. It fails because of inherited permissions, broad data sources, accidental logging, weak prompts, connected tools and lack of human review.
Baseline question: can the user receive information they should not see even if the document exists somewhere?
Risks to map
- Connected data sources and sensitivity classification.
- Effective permissions, not just expected permissions.
- Indexed content that should no longer be available.
- Logs storing prompts, answers or documents.
- Tools the agent can execute.
- Providers retaining or processing data.
Minimum guardrails
Before scaling, define data scope, permission controls, redaction, safe logging, abuse monitoring, human review for critical actions and a process to remove compromised sources.