Governing autonomous AI agents: Controls must be part of the architecture

Autonomous AI agents do more than produce answers. They may trigger workflows, interact with other systems, or take action before a person reviews the result. In an interview with AI News, Marinela Profi, SAS Global Market Strategy Lead for AI Agents and Generative AI, argues that this authority changes how such systems must be deployed and governed.

Speed must include the ability to stop

Profi says the familiar “ship fast and iterate” approach becomes risky when software can act. Moving quickly with autonomous systems must include observability, constraints, interruption, and recovery from the start. Before deployment, teams should define an agent’s authority, the potential blast radius of failure, and whether its actions can be reversed.

Citing SAS’s Data & AI Impact report, Profi says reported trust stands at 76 percent for generative AI and 66 percent for agentic AI. The same report says 89 percent of agents deployed in production are already acting rather than merely assisting, while more than half operate with limited or no human approval. These figures are findings reported by SAS, not independently measured by this article.

Governance as executable architecture

In Profi’s account, the most consequential debt often sits around the model: missing lineage between data, model, decision, and action; permissions granted too broadly; logs that record outputs but not invoked tools; or policies that remain documents rather than controls. If an auditor or customer later asks why a decision was made, which data informed it, and who was accountable, the architecture should be able to reconstruct the chain.

She recommends translating policy into executable controls: which data an agent may access, which actions require approval, what thresholds trigger escalation, what must be logged, and which models are allowed for each use case. In this framing, governance is not only a restriction; it is reusable infrastructure that can make later deployments faster and safer.

Test the system, not only the model

A proof of concept usually shows that an agent can perform a task. Production introduces stale data, permission changes, tool failures, and unexpected interactions. Evaluation therefore needs to extend beyond answer accuracy to the full decision-and-action chain: Was the agent authorised, did it use the right data, invoke the correct tools, and stay within its boundaries?

Profi warns against the “one intelligent layer over everything” pattern. Connecting a language model to all enterprise data and tools can look attractive on a roadmap, but a probabilistic component should not become the control plane for the entire organisation. Agents should operate inside an architecture of data, decisioning, and controls, with authority bounded at the agent or task level.

Human supervision without a universal bottleneck

The boundary between human and machine should move according to risk, reversibility, uncertainty, and consequence. Low-risk, reversible actions may be automated; exceptions can be escalated; and high-impact decisions involving health, credit, employment, or public services may still require explicit human approval.

Profi favours a “human-on-the-loop” design: people establish authority, boundaries, and escalation criteria; machines operate within them; and people retain visibility and the ability to intervene. Her conclusion is concise: execution can be delegated to AI, but accountability cannot.