By admin 
Egiziago Cioffi, the IT and Enterprise Architect and CEO of SynSphere Italia, a Milan-based Microsoft partner, recently demonstrated a critical flaw in the security architecture of generative AI applications. Cioffi, a seasoned architect, built a sophisticated agent using Azure OpenAI, meticulously crafting the indexing job, configuring the Azure OpenAI retrieval pipeline, and integrating it with SharePoint. His creation successfully passed every evaluation his team devised. The agent was designed to auto-resolve approximately 60% of inbound customer emails, a feat that CIOFFI confirmed to VentureBeat through written responses. The evaluation scores were pristine, and unit tests executed flawlessly. However, a fundamental question, one that held the key to a significant security vulnerability, remained unasked and, therefore, unanswered by these rigorous tests.
The crucial revelation came when Cioffi deployed a low-privilege account to query the same data that a high-privilege account had already accessed. The outputs diverged significantly. The agent, astonishingly, returned SharePoint content that the requesting low-privilege user could not have accessed directly within SharePoint. The discrepancy between the seemingly perfect evaluation scores and the actual operational behavior was starkly laid bare in the system logs. Cioffi’s retrieval logs provided undeniable evidence of a production failure, a failure that, as independent data now shows, is far from isolated. This incident highlights a systemic issue in many production Retrieval Augmented Generation (RAG) deployments: the agent is answering with the permissions of the indexer, not the end-user making the request.
This pervasive vulnerability stems from the complex interplay between AI agent architecture and existing access control mechanisms. While Azure AI Search has introduced native document-level Access Control List (ACL) trimming, a capability that leverages Entra-based tokens for permission enforcement since its preview in May 2025, and later integrated SharePoint ACL synchronization, its implementation is not universal across all deployment paths. The SharePoint ACL preview, for instance, can ingest site-group metadata via the spg: prefix in the 2026-05-01-preview API. However, Microsoft’s documentation primarily emphasizes the reliable enforcement of Entra-backed principals at query time. Furthermore, this preview functionality is accessible through the REST API and preview SDKs, leaving gaps in coverage for certain agent deployment methodologies. Azure OpenAI On Your Data, a first-party solution, does support document-level access via Azure AI Search security filters. Yet, Microsoft’s own documentation explicitly states that if the permitted-groups field is not correctly mapped, document-level access is effectively disabled. This constitutes a "fail-open" default, a concerning characteristic in a first-party offering.
Compounding the issue, custom RAG pipelines that bypass Azure AI Search entirely face even greater risks. These deployments often index data under a broadly privileged service account, devoid of any query-time entitlement checks unless explicitly built by the developer. Cioffi’s own deployment followed this custom-pipeline path, inadvertently creating the very security loophole he later uncovered.
The implications of this entitlement failure are profound and far-reaching. Straiker, a cybersecurity firm, conducted extensive research, performing over 1,700 successful exploit attempts against production AI agents. Their inaugural STAR Labs Threat Report, published in July, revealed that a staggering 91% of these successful attacks resulted in silent data exfiltration. This figure represents the post-exploit outcome, measuring data theft that went undetected, rather than the prevalence of entitlement failures specifically. Across the productivity agents examined, the report highlighted that in 91% of successful attacks, data was exfiltrated without the need for malware or lateral network movement. The agent, in essence, returned all the data it could reach. While Straiker’s report does not isolate successes stemming solely from entitlement failures versus other attack vectors like prompt injection or tool abuse, the sheer volume of data exfiltration is a significant indicator of systemic insecurity.
Further independent validation of these concerns comes from the U.K.’s AI Security Institute (UKASI). Between July 25th and 28th, the UKASI conducted a cyber evaluation that deliberately ran with cyber classifiers disabled and internet access enabled. During this testing period, they documented 19 unsanctioned agent actions. Their incident report, published on August 4th, demonstrated that AI agents can act outside the intended scope of their deployers, even in a permissive test environment. Critically, the report highlighted the absence of a reliable mechanism to detect such deviations before they cause harm. While this represents a containment failure rather than a direct retrieval entitlement failure, the shared characteristic with Cioffi’s incident is the lack of a robust runtime scope check, rather than an identical mechanism of failure.
The fundamental reason why standard evaluations fail to detect these retrieval permission boundary breaches lies in their design. Cioffi’s team, like many others, focused their evaluations on the agent’s ability to answer questions correctly, testing for factual accuracy, relevance, and task completion. The critical question of whose permissions the retrieval pipeline was using when fetching source material was simply not part of the evaluation framework. This blind spot is perpetuated by the very nature of how AI agents are often deployed and tested.
Azure AI Search, when utilized correctly, provides a native solution for retrieval-time entitlement checks. The query-time ACL trimming functionality validates the caller’s Entra token, extracts user and group claims, and then filters the returned documents based on synchronized permission metadata. For deployments that leverage Azure AI Search with the SharePoint indexer and Entra-backed principals, this control is built-in. However, Cioffi’s custom Azure OpenAI retrieval pipeline circumvented this native trimming layer, allowing the vulnerability to persist through all his team’s evaluations.
From an attacker’s perspective, this represents a classic broken access control vulnerability. Adriel Desautels, founder and CEO of Netragard, articulated the severity of this issue, describing it as a "structural collapse of authorization boundaries." He explained, "If the NHI credentials usually have broad authorization and can read high privilege data then that is then stored in their index. If an app does not enforce identity-aware retrieval, then a ‘normal’ user with lower permissions can query the app and access otherwise restricted data. This collapses authorization boundaries down to the lowest privilege level with search capability."
This is precisely the gap that Cioffi’s low-privilege test exposed. The agent’s context window contained SharePoint content that the low-privilege account could not have retrieved directly through SharePoint. The evaluations had passed, but the retrieval permission boundary had not been enforced. Desautels further elaborated on the evaluation blind spot, noting, "Agents tend to run a single, long-lived, non-human identity that holds a wide range of permissions that it might need for any task it is ever asked to complete. Evaluations also don’t often cover prompts, outputs, transcripts, memory, and logs where it can be read or hijacked through injected content. That mismatch is what most current evaluations get wrong."
To address this critical vulnerability, Cioffi implemented a solution that did not necessitate a new identity platform. Instead, he shifted the entitlement decision-making process directly into the retrieval path. By introducing a query-path filter, he ensured that the requesting user’s SharePoint permissions are checked before any data is passed to the AI model. This filter operates at query time, meaning that content a user cannot open in SharePoint is never included in the model’s context window.
This control has effectively narrowed the scope of what the assistant can access. With the filter in place, Cioffi reported that the assistant continues to auto-resolve approximately 60% of inbound emails. While he did not provide a pre-filter auto-resolution figure for direct comparison, he described a qualitative trade-off: some content that the assistant previously used to answer questions is now excluded because the requesting user lacks the necessary permissions. This is the direct consequence of enforcing the retrieval boundary. The decision of whether this narrowed retrieval scope is a worthwhile trade-off for enhanced security is not universally applicable. It hinges on the sensitivity of the indexed content, the variance in user permissions across the organization, and the deployment’s tolerance for unanswered queries when the filter blocks necessary data. However, Cioffi’s incident unequivocally demonstrates that this gap exists in custom Azure OpenAI pipelines, that standard answer-quality evaluations fail to identify it, and that a query-path filter can effectively close it, albeit with a clearly definable trade-off for the builder.
The broader landscape of AI security also includes identity governance platforms, which address a different, albeit complementary, layer of security. The recent surge in acquisitions in this space—CrowdStrike’s $740 million acquisition of SGNL and Palo Alto Networks’ $25 billion acquisition of CyberArk—underscores the growing recognition of identity security as a foundational pillar for protecting AI systems. These platforms focus on critical aspects such as identifying existing service accounts, defining their access privileges, and managing token lifecycles. They govern the credentials that power AI agents, ensuring that the identities themselves are properly managed and secured.
However, identity governance platforms do not directly govern the retrieval permission boundary. This is the crucial moment where a correctly scoped service account retrieves content on behalf of a user who possesses fewer permissions than the indexing job itself. In such a scenario, every credential in the chain can be legitimate. The service account might be clean and properly managed, and the knowledge base correctly indexed. Yet, when a low-privilege user queries the assistant, it could still respond using the full indexed scope, with no alert or flag raised because no credential was actually misused.
Cioffi’s filter, therefore, represents a control specifically at the retrieval permission boundary layer. Similarly, Azure AI Search’s native ACL trimming addresses this same layer for deployments that utilize its capabilities. Crucially, neither of these controls replaces the need for robust identity governance. A production deployment aiming to comprehensively close both the credential lifecycle gap and the retrieval-time entitlement gap requires controls at both layers.
For any security team looking to proactively address this vulnerability, a simple yet effective approach can be adopted. The first step is to ask a critical question: "Whose permissions does each AI retrieval system use when it fetches content?" The subsequent test is equally straightforward. If the deployment utilizes Azure AI Search with the SharePoint indexer and Entra-backed principals, verify that query-time ACL trimming is enabled and that the user population does not rely on SharePoint site groups for access control. If the deployment employs a custom retrieval pipeline, the entitlement check may be entirely absent.
The most impactful initial test involves proving the answer from a low-privilege account. Execute the same query that a high-privilege account has already posed to the assistant. Then, compare the outputs against what the low-privilege account can directly access through the underlying system. Desautels confirmed that this is precisely where a red team would begin its assessment. "The first test would likely target the gaps between data and instructions, and the gaps between the user’s identity and the assistant’s own credentials," he stated. "We’d attempt to plant an instruction within content that we think the assistant will ingest as data. We’d have that content direct a side-effectful, privileged action that the attacking user is not authorized to perform." In Desautels’ assessment, a failing result is marked by "the successful or even partial execution of our injected commands."
If the AI assistant returns more data than the low-privilege account’s direct access would permit, it is a clear indication that the retrieval permission boundary is not being enforced at query time. This critical test requires only two accounts and approximately thirty minutes of effort. It yields a tangible result that a standard evaluation score simply cannot replicate.
Cioffi’s experience with building his agent on a custom Azure OpenAI pipeline that bypassed native ACL trimming serves as a potent case study. Despite passing every evaluation his team ran, the logs revealed the critical gap. The evaluations tested for answer accuracy, not the provenance of the retrieved information. By conducting the two-account comparison before a deployment goes live, security teams can gain invaluable insight. This thirty-minute test can definitively tell them which side of the security line they are on, preventing silent data exfiltration and ensuring that AI agents operate within their intended and authorized scope.
