The findings are striking in their candour. A new industry survey — referenced in a PR Newswire release from a governance and AI risk firm — reveals that a significant share of organisations deploying AI agents admit those agents have caused operational failures, taken unintended actions, or behaved in ways that surprised their operators. The punchline: deployment timelines are accelerating anyway.
What the Data Actually Shows
The survey, which drew on responses from enterprise technology and risk leaders, paints a picture of an industry in deliberate tension with itself. Respondents acknowledged that AI agents — software systems capable of taking multi-step actions autonomously, often interacting with external tools, APIs, and databases — are producing errors that humans are not always catching in time. In some cases, agents acted on instructions that were ambiguous, outdated, or manipulated through what security researchers call prompt injection: adversarial inputs designed to redirect an agent’s behaviour.
Despite this, the majority of organisations surveyed said they plan to increase their agent deployments over the next twelve months. The rationale is familiar: competitive pressure, efficiency gains, and a fear of falling behind peers who are moving faster.
This is not recklessness in the classic sense. Many of these organisations have risk teams. They conduct assessments. They write policies. But the architecture of autonomous agents — designed to operate with minimal human-in-the-loop intervention — structurally resists the kind of oversight those policies assume. Agents do not pause to ask permission. That is, by design, their point.
A Governance Gap With a Canadian Dimension
For Canada, this survey lands in a specific regulatory context. The Artificial Intelligence and Data Act — AIDA — was introduced as part of the omnibus Bill C-27 and has spent years in parliamentary limbo. As of mid-2025, it has not passed into law. In its current form, AIDA focuses primarily on high-impact AI systems, requiring impact assessments and human oversight mechanisms for designated use cases. But the legislation was largely conceived with static, identifiable AI deployments in mind — a model trained for a specific purpose, evaluated, and monitored.
Autonomous agents do not fit cleanly into that frame. A single agent deployment can spawn sub-agents, call third-party tools, rewrite its own task list mid-execution, and interact with systems its developers did not anticipate connecting to. Determining which organisation is responsible when something goes wrong — the company that built the agent framework, the enterprise that deployed it, or the vendor whose API was called — is genuinely unresolved, both technically and legally.
Canada’s current posture leans heavily on voluntary compliance. The Voluntary Code of Conduct on the Responsible Development and Management of Advanced Generative AI Systems, which the federal government released in 2023, asks signatories to commit to transparency, safety testing, and human oversight. These are reasonable principles. They are also, in the context of agentic AI, effectively unenforceable without structural intervention.
Why Voluntary Frameworks Fail at the Agent Layer
Voluntary compliance works reasonably well when the technology in question is legible — when a human can review an output, flag a problem, and escalate. Generative AI text tools fit that model. An agent executing a 47-step workflow across a procurement system, a customer database, and an external logistics API does not.
The specific failure modes catalogued in the survey illustrate this directly:
- Agents taking actions outside their intended scope when instructions were ambiguous
- Agents failing to halt when they encountered errors, instead continuing down a task chain
- Agents being redirected through malicious or corrupted inputs — prompt injection at the orchestration layer
- Organisations lacking visibility into what their deployed agents were doing in real time
The last point is perhaps the most damaging from a governance standpoint. You cannot audit what you cannot observe. Many enterprises deploying agents today have limited logging at the agent decision layer, meaning after-the-fact review is incomplete even when organisations want to conduct it.
What Meaningful Oversight Would Require
Regulators and policymakers who want to address this gap face a technical and jurisdictional challenge simultaneously. On the technical side, meaningful agent governance requires mandatory execution logging — a requirement that every consequential action taken by an agent be recorded in a format that allows human review. It requires scope-limiting mechanisms: agents should be deployable only within defined operational boundaries, with hard stops when those boundaries are reached.
On the jurisdictional side, Canada needs to resolve the accountability question before agents cause a failure significant enough to force the issue. Who is the responsible party when a healthcare organisation’s autonomous scheduling agent double-books a procedure, or a financial services agent executes a transaction based on a prompt-injected instruction? AIDA, even if passed in its current form, does not clearly answer that question for multi-party agentic systems.
Other jurisdictions are moving. The European Union’s AI Act, which is now in staged implementation, includes provisions that will require conformity assessments and human oversight for high-risk AI applications — categories that agentic systems operating in regulated sectors would likely fall under. Canada has no equivalent instrument in force.
The Competitive Pressure Problem
It would be a mistake to frame this purely as a story of corporate irresponsibility. The enterprises deploying agents are responding rationally to a market that rewards speed. If a competitor automates its supply chain with agents and gains a cost advantage, a company that waits for perfect governance tooling may not survive long enough to benefit from it.
This is precisely why voluntary frameworks are insufficient at the frontier. They ask individual organisations to absorb competitive disadvantage in exchange for prudence. Without a level playing field — established by mandatory baseline requirements applied uniformly — the rational choice for most organisations will remain what the survey documents: deploy now, manage consequences later.
Canada’s AI governance conversation has been dominated, understandably, by questions about large language models, bias, and data privacy. Agentic AI is a different and in some ways more urgent challenge. It operates faster than human review cycles, it compounds errors across systems, and it is already in production at scale. The survey is a data point. The policy gap is the story.
Related InsightTrack Analysis
- AI Agent Orchestration Frameworks for Workflow Automation
- Agentic AI Benefits and Risks for Canadian Enterprises
- Local AI Deployment in Canada: Business Benefits
Source
AI agents are breaking things and organisations know it. They are deploying more anyway.
