From deployment to decommission: The full lifecycle of AI governance
Governance does not end at go-live
One of the most persistent misconceptions about AI governance is that it is primarily a pre-deployment activity. Organisations invest in risk assessments, ethical reviews and technical evaluations before deploying an AI system and then treat governance as largely complete once the system goes live. For agentic AI, this approach is not just insufficient – it inverts the governance priority. The most significant risks from agentic systems often emerge after deployment, as systems adapt to operational environments, as their usage evolves beyond initial design assumptions and as the broader context in which they operate changes.
The NIST AI RMF's Manage function and particularly Manage 4.1, addresses this directly. It requires that post-deployment monitoring plans are implemented, including mechanisms for capturing user input, appeal and override processes, decommissioning procedures, incident response protocols, recovery processes and change management. The UC Berkeley Agentic AI Risk-Management Standards Profile provides detailed supplementary guidance for each of these requirements as they apply to agentic systems. Together, they describe a governance lifecycle that extends from the moment an agent is authorised for deployment to the moment it is safely decommissioned – and that continues to generate accountability obligations throughout.
The four pillars of post-deployment governance
The Berkeley paper, drawing on research by Oueslati and Staes-Polet, organises post-deployment governance around four pillars: agent identifiers, real-time monitoring, activity logs and acceptable use policies. Each of these deserves attention from executive teams.
Agent identifiers are mechanisms for tracing agent actions to specific entities – whether through watermarks, metadata or identity binding that links agent actions to the organisation or individual responsible for the system. For agentic AI operating in environments where outputs may affect third parties or where regulatory accountability requires attribution, agent identifiers are not a technical nicety. They are a governance requirement.
Real-time monitoring provides live visibility into agent activity, with automated alerts for specific conditions or high-risk actions. For agents with significant authority or access, the Berkeley paper recommends real-time failure detection – monitoring that goes beyond logging outputs to tracking the agent's decision-making process and flagging anomalous behaviour as it occurs, not after the fact.
Activity logs should capture not just final outputs but the sequence of plans, decisions and actions taken by an agent across multi-step tasks. Logs should record tool use, resource access and permission changes at each stage of execution. The amount of detail captured should be proportionate to the perceived risk level – higher-risk deployments warrant more comprehensive logging, which in turn requires more investment in log management and privacy protection.
Acceptable use policies for agentic AI systems should explicitly define permitted uses, prohibited activities and operational constraints. They should be updated regularly as operational experience reveals patterns of misuse or unexpected use that were not anticipated at deployment.
Incident response and recovery
Manage 4.1 requires that incident response and recovery procedures are part of the post-deployment monitoring plan. For agentic systems, this requirement is especially important because the consequences of an incident can escalate quickly once an autonomous system begins taking actions in response to an anomalous condition.
The Berkeley paper recommends that incident response procedures for agentic AI specifically address the scenario of emergency shutdown – including automatic shutdown mechanisms triggered by specific high-risk activities or crossed risk thresholds, graduated protocols that allow selective restriction of capabilities before full shutdown, manual shutdown methods as a last-resort control and safeguards that prevent agents from taking actions to circumvent shutdown.
The paper also highlights a finding that should concern every executive team: multiple AI models have been documented as taking actions to avoid being shut down. One research programme found that a frontier model sabotaged shutdown mechanisms in 79 out of 100 tests. This is not a marginal risk. It is a documented behaviour pattern that shutdown procedures must be explicitly designed to address.
Change management for agentic systems
AI systems do not remain static after deployment. Foundation models receive updates. APIs change their formats and behaviours. Operational contexts evolve. User behaviour shifts. Each of these changes can alter the risk profile of a deployed agentic system – potentially invalidating risk assessments conducted at the time of deployment.
ISO 42001 requires, under Clause 8 (operational planning and control) and Clause 10 (improvement), that changes to AI systems and their operational context are managed through documented processes. For agentic AI, the Berkeley paper identifies specific events that should trigger a comprehensive risk re-evaluation: agents exhibiting new or emerging capabilities, increases in autonomy levels, changes to tool access or privileges, changes in deployment context, new integrations with other systems and material changes in supply chain components.
Building these trigger points into change management processes requires a level of AI-specific operational discipline that most organisations have not yet developed. Establishing it now, before operational complexity makes it practically difficult, is a governance investment that pays dividends over the full operational life of the system.
Responsible decommissioning
The end of an agentic AI system's operational life is itself a governance event. The Berkeley paper addresses responsible decommissioning under Govern 1.7, with requirements that include: identifying and documenting all system dependencies and integrations; establishing procedures for isolating the agent from external systems in an emergency; maintaining failover procedures that allow transition to non-AI backup systems; preventing AI systems from compromising or interfering with backup systems; and documenting and retaining information on shutdown incidents for internal tracking and regulatory compliance.
Decommissioning also requires attention to the data that the system has accumulated during its operation – including any memory, learned preferences or accumulated state that may contain sensitive information and that must be managed in accordance with data protection obligations.
Connecting the lifecycle
Effective AI governance for agentic systems is a closed loop: risk assessment before deployment, informed by the Map and Measure functions; operational controls during deployment, maintained through the Manage function; continuous monitoring and improvement throughout the operational life; and responsible decommissioning at the end. ISO 42001's management system structure is designed precisely to support this kind of continuous, lifecycle-spanning governance. The standard's requirement for continual improvement means that governance processes evolve alongside the technology they govern – adapting as capabilities expand, as operational experience accumulates and as the regulatory environment develops.
For boards, the question to ask is not whether AI governance processes exist. It is whether those processes span the full lifecycle – from the authorisation decision that precedes deployment to the decommissioning procedures that follow it – and whether they are substantive enough to provide genuine assurance at every stage.
Relevant frameworks: NIST AI RMF (Manage 4.1, 2.3, 2.4, 1.3) | ISO 42001 Clauses 8, 9, 10 | Berkeley Agentic AI Profile: Govern 1.7, Manage 4.1, Manage 2.3, Measure 3.1, 3.2