Agentic AI governance: A Computer Weekly Downtime Upload podcast

For Rashmi Shetty, vice president of enterprise AI at Capital One, governance is the next phase of large-scale artificial intelligence (AI) deployments, once organisations have established a technical blueprint and a data strategy. She says the bank is committed to protecting the privacy of consumers, which means giving consumers confidence in Capital One’s ability to safeguard their personal and financial information. As the bank expands its data-enabled tech and AI capabilities, she says: “We believe it's important to have a privacy-focused mindset and build an environment where consumer data is safe, secure, and under the consumer's control.” When deploying AI models, one of the decisions IT leaders need to make is how and when to use publicly available large language models (LLMs). It is generally safer to limit usage only to models that run internally. But these can limit the flexibility of having an LLM that has a greater understanding from knowledge accessed from the wider internet. Even when models are run internally, inside the corporate network, whenever a public LLM is accessed , any queries submitted to it will be incorporated into the model’s understanding of the world, and is potentially available to anyone. If sensitive data is included in that query, this constitutes a data leak, which is why some organisations are focusing purely on internally run models and have chosen to ban the use of public LLMs. However, rather than limiting the use of AI models only to the ones that can be run internally inside the company’s network, Shetty sees an opportunity to use governance to control how publicly available LLMs can be directed to work towards a specific business outcome. Capital One has worked on modernising its tech stacks and data ecosystem. Alongside these efforts, Shetty says it has also modernised and recalibrated risk and compliance and policy management based on a modern data ecosystem. Among the main areas of concern for organisations looking at AI governance is personally identifiable information (PII) and ensuring there are General Data Protection Regulation (GDPR) controls around that data. As enterprise AI is deployed, there will be opportunities for standalone AI systems to collaborate in order to streamline business processes. The challenge in governance for agentic AI systems is ensuring each AI system required to run an end-to-end business process only has access to the data it needs to finish its bit of the task. Shetty says this involves managing how data flows along an agentic trajectory, where data is passed between the different agentic AI systems that need to be accessed to achieve the specific task. She says that an agentic AI governance framework should define nuanced, atomic policies across this trajectory. At Capital One, the governance framework has been developed on top of the bank’s platform strategy and it is being updated to support agentic AI. “We are bringing it together for managing agency for our multi-agentic system,” Shetty adds. This governance framework provides the guardrails for protecting personally identifiable information and GDPR controls. In effect, the governance framework defines what the agentic system that is powered by LLMs is allowed to do in terms of the tools and data it can access. The overall goal is a greater level of autonomy as Shetty explains: “You're taking a highly stochastic, probabilistic system, which can make dynamic decisions and trying to define a deterministic boundary using a safety and governance mechanism in order to increase your velocity towards autonomous decision-making.”