Aug 11: AI assistants are increasingly integrated into workplace systems and everyday digital workflows, and their ability to access information through existing user permissions is creating a new area of concern for privacy and security.
AI Governance Architect and Data Protection Leader Vivek Kumar is exploring this concern in his ongoing research and in his Privacy-Aware AI Risk Architecture methodology — a practitioner framework for assessing privacy and security risk at the AI connector layer, published on SSRN and submitted as practitioner input to public consultations held by data protection regulators, including the UK's Information Commissioner's Office, France's CNIL, and Singapore's Infocomm Media Development Authority .
The topic came to light from Kumar's observation that discussions around enterprise AI governance often focus on the AI model or application, while paying less attention to the connectors that determine what information the system can actually access. Enterprise AI assistants can be connected to email, calendars, documents, HR systems, financial records and other business systems through permissions and APIs.
"What intrigued me was the gap in how we assess AI systems. We naturally focus on the model itself, but when an AI assistant is connected to enterprise systems, the connector determines what information the AI can actually reach," said Kumar.
His PAARA methodology focuses on this connector layer and examines questions around the data an AI system can reach, the reason for that access, how access is constrained, how potential misuse can be detected and who accepts the resulting risk.
For Kumar, the issue is not necessarily that AI assistants are fundamentally malicious. The concern is how AI can amplify existing access privileges. A user may already have legitimate access to certain information, but an AI assistant can make it easier to retrieve, combine and synthesize information across connected sources.
"This is where insider-risk amplification becomes important. The individual may have legitimate access and may not have any malicious intent. But AI can reduce the friction involved in accessing and working with information that is already available within those permissions," Kumar explained.
This understanding also shapes Kumar's view on how people should approach AI assistants. Rather than discouraging their use, he believes organizations and individuals need to become more conscious of the access and permissions behind the technology.
"When we use an AI assistant, the question should not only be 'What can this AI do?' but also 'What information can this AI reach through the connections it has?' Understanding those boundaries is an important part of using AI responsibly."
His research therefore focuses on a more deliberate approach to AI adoption — one that considers the systems connected to an AI assistant, the permissions provided to those connections, and the information that can potentially be retrieved through them.
Kumar does not see avoiding AI altogether as a practical alternative in an increasingly technology-integrated environment. Instead, he supports responsible integration.
"AI is already becoming part of how organizations and individuals work, and it can provide significant value. The answer is not to reject it, but to establish appropriate boundaries around its use. Organizations need to understand what systems are connected to AI, what permissions those connections provide and what data can be retrieved. Individuals also need to be conscious of the information they provide to AI systems."
The larger objective of Kumar's work is to bring the connector layer into the conversation around AI privacy and security. PAARA incorporates connector inventory and scope mapping, permission-risk assessment, retrieval boundaries, prompt-injection threat modeling, and insider-risk amplification into its methodology.
"The goal should be responsible integration rather than unrestricted integration. AI should remain a useful tool, but its convenience needs to operate within clearly understood privacy, security and governance boundaries," Kumar concludes.
