Many implementations retrieve broadly and rely on filtering later. From a security perspective, that can be uncomfortable. Once data is touched during retrieval, it’s hard to say the system never accessed something it shouldn’t have.
In practice, organizations often have layered access: parent-level policies, branch-level rules, roles, and documents that are only valid for certain periods. If that structure isn’t modeled explicitly, enforcing boundaries becomes inconsistent.
My approach that helped was treating access rules as a gate before retrieval. Only documents that already meet tenant scope, role visibility, and policy constraints are eligible for similarity search.
This reduces the risk of accidental exposure and avoids relying on prompts or model behavior as a security control.
The model becomes a consumer of already-approved context, rather than part of the enforcement path.
It’s less about stronger AI and more about clearer boundaries in the data model.
Interested in how others think about this trade-off in production.
— Arty