TL;DR

Generative Artificial Intelligence (GenAI) and Agentic AI introduce risks that are categorically different from traditional cybersecurity threats — systems that can cause harm while working exactly as instructed, through authorized access, under normal operating conditions. Existing cybersecurity frameworks are necessary but not nearly sufficient to govern these risks. This brief explains why, using realistic failure scenarios drawn from water operations. And the two papers that follow dive into the factors that make GenAI adoption compelling for the water utilities, and how engineering principles developed at a DOE Natonal Lab and embraced by the AWWA can help utilities build the guardrails (i.e., safety mechanisms ) they need to keep themselves and their customers safe and secure.

Introduction

A little more than a decade ago, cybersecurity leaders Josh Corman, Nick Percoco, and Beau Woods issued a declaration to critical infrastructure operators who were waiting for someone else to handle the growing digital threat landscape. Their message was blunt: “I am the Cavalry.” Stop waiting for the cavalry to arrive. You are the cavalry. The message resonated, and in the years since, the best-run utilities have internalized it, building cultures where cybersecurity is everyone’s job, not just IT’s.

We are at a similar inflection point now, but the stakes have shifted. Cybersecurity was fundamentally about keeping bad actors out of systems that were otherwise working as designed, or detecting their movements once in. GenAI introduces something qualitatively different: systems that can cause harm while working exactly as instructed, through authorized access, under normal operating conditions. When your AI-enhanced decision support system optimizes pump schedules in a way that slowly fatigues pipe joints, or when it hallucinates a chemical dosing recommendation with the same confidence it delivers an accurate one, the failure isn’t a breach. It’s a gap in governance. And governance, unlike perimeter defense, cannot be purchased off the shelf or bolted on after the fact.

Which is why, for 2026 and beyond, a more apt mantra might be: I am the Guardrails.

The metaphor is deliberate. A guardrail doesn’t stop you from driving. It keeps you on the road when things go sideways: when the fog rolls in, when the curve is sharper than you expected, when the road itself is still being built. That is precisely the situation water utilities now face with GenAI. The technology is powerful, it is arriving fast, and the road it’s traveling has no lane markings yet. Your job is not to refuse the journey — with the wellbeing of your utility and the customers you serve in mind, it is to make sure you can navigate it safely.

This is the first in a three-paper series written for utility and public works directors, board members, engineers, IT and OT managers, and the regulators who oversee them. This paper focuses on the diagnostic: why GenAI risks are categorically different from traditional cyber threats, why existing frameworks are inadequate, and what new risk categories demand your attention. The second paper makes the economic case for adoption and explains why the forces driving GenAI into water utilities are so compelling. The third delivers the governance framework water utilities need to manage these risks safely.

The Regulatory Context: A Governance Vacuum

The instinct in regulated industries is to wait for guidance from regulatory agencies. In the water sector, that instinct has often served utilities well. In 2018, more than a decade after the electric sector implemented mandatory security controls, Congressional action made the Environmental Protection Agency (EPA) the implementor and AWWA stepped in to provide support for America’s Water Infrastructure Act (AWIA). A sector-specific implementation of the NIST Cybersecurity Framework, the AWIA cyber risk management tool gives utilities a structured process for assessing risks and building resilience.

AWIA moved the sector forward but was built primarily for a world where the primary digital threat was unauthorized access to systems operating under human control.

GenAI breaks that assumption. It introduces authorized systems that can drift, hallucinate, degrade, and optimize toward unintended consequences — none of which register as anomalies under traditional cybersecurity monitoring.

The current administration’s AI policy framework is explicitly designed to minimize regulatory burden on AI developers in the name of maintaining American competitiveness. The European Union AI Act classifies critical infrastructure AI as “high-risk” and imposes governance requirements, but its implementation timeline, jurisdictional reach, and enforcement mechanisms offer little near-term help to a utility operator in Fresno, Tacoma, or Milwaukee. State legislatures are beginning to act: Colorado, California, Illinois, and others have passed or proposed AI-related legislation, but the patchwork is inconsistent and largely focused on consumer-facing applications rather than operational technology (OT) in critical infrastructure.

The result is a governance vacuum, and water utilities are sitting squarely in the middle of it. They are being courted by AI vendors offering transformative capabilities — real-time predictive maintenance, intelligent SCADA augmentation, digital twins of retired operators’ institutional knowledge — while operating under frameworks that have nothing to say about what happens when those capabilities fail, drift, or are compromised in ways that look nothing like a traditional cyberattack.

Existing Cyber Frameworks: Necessary but Not Nearly Sufficient

Traditional cybersecurity posits that defense is about perimeter protection and access control; that a system is safe when it is working as designed; that unauthorized access is the primary threat; and that detection should focus on anomalies and intrusions. But when it comes to GenAI-enhanced systems, the main shortcoming of cyber frameworks is that they don’t address goal misalignment. A system pursues its objectives correctly, but those objectives are subtly wrong.

A robust defense against GenAI threats requires governing the decision-making process itself — something well outside the bounds of current cybersecurity frameworks, policies, and tools. Industrial security veteran and instructor Jason Christopher explains the difference precisely:

The control point isn’t just access control or detection anymore. It’s governance over how decisions are generated, validated, and constrained in real operations. Security teams are used to thinking about adversaries. AI forces us to also think about authorized systems producing unsafe outcomes. It’s an entirely different discipline.

The cybersecurity community has embraced GenAI security so quickly that most people think cyber professionals are the ones best equipped to bring security to GenAI. They are, at least for defending the new attack surfaces that come with it. But they are not the right tribe for the safety engineering skills most needed.

New Risks to Have on Your Radar

Most of these are not things utilities have had to know about, pay attention to, or devise plans to mitigate in the past. But they must be managed now if utilities are to safely realize the benefits of GenAI:

• Hallucination as a distinct risk category. This is the purest example of a risk that has no analog in traditional cyber frameworks. The system isn’t compromised or misaligned — it’s just confidently wrong.
• Model drift and degradation over time. No attacker is needed to trigger this. The model quietly becomes less accurate as the world changes around it, and traditional monitoring methods don’t notice it.
• Data governance as a prerequisite that must be done well and in advance. Bad data in, bad decisions out — but unlike a traditional data quality problem, GenAI amplifies errors with false confidence.
• Sensitive data exposure. Sending infrastructure vulnerability data and system topology to cloud-based AI services creates an attack surface that didn’t exist before GenAI adoption.
• The explainability problem for regulated utilities. A compliance risk that didn’t exist until you put a black-box system between your operators and their regulatory obligations. No prior technology created this particular bind.
• Frontier model vulnerability. As of early 2026, frontier models remain highly susceptible to jailbreaking. The pattern is consistent: a new model drops, bad actors crack it quickly, and post techniques publicly. Guardrails keep failing because they are designed to appease fears rather than fix vulnerabilities.

Water Sector Scenarios

Here are two illustrations of things that may go wrong even with solid cyber defenses in place. Note: neither is a cybersecurity failure. No unauthorized access occurs. No intrusion detection system fires off an alarm. These are failures of governance and operations for which existing cyber frameworks have nothing to say.

Scenario 1: Catastrophic Optimization

An AI system optimizes pump operations to minimize energy costs, saving $2M in electricity over its first twelve months. Its optimization strategy involves running pumps in sequences that create pressure transients that were individually within normal operating parameters but created a cumulative fatigue pattern that no one previously had a reason to monitor for — establishing a novel failure mode. These transients slowly fatigue pipe joints in a way that had not happened under human operation.

In year two, a major main break occurs; resultant floods cause $100M in damage. Questions arise immediately: who’s liable — the utility that deployed it? The AI vendor? The utility’s engineer who approved it? Under current frameworks, the utility bears all the risk, and that engineer’s career suddenly looks much less promising, to put it mildly.

Scenario 2: A Cybersecurity Cascade

A utility connects sensors to a GenAI system for predictive maintenance. The system has API connections to cloud services, third-party analytics, and remote access for vendor support. Attackers compromise the GenAI system. They don’t shut anything down — they just subtly corrupt the AI’s recommendations. Over months, the AI decision support system guides operators to decisions that degrade system resilience.

Then the attackers unleash the real attack: they shut down power to some pumps and further corrupt AI recommendations at the same moment. The system fails in ways operators can’t diagnose because their muscle memory has deteriorated. This is cognitive offloading — they’ve been relying on the AI to think for them, and now without it, they are lost.

The Insurance Gap

Beyond the regulatory void, existing insurance policies — including cyber insurance — were not written for GenAI risks. No framework exists to assign liability, and no entity is rushing to fix this problem. This is the most concrete proof that utilities adopting GenAI solutions are operating without a financial safety net.

When the AI makes a bad recommendation and causes a failure, how do you determine whether it was a technical error, a data quality problem, or a deliberate attack? Today we are very far from having the ability to do this with any confidence. The absence of attribution capability compounds the liability problem enormously.

What Comes Next

Understanding why existing frameworks fall short is the first step. The second paper in this series makes the economic case for adoption — because the forces driving GenAI into water utilities are powerful and largely irresistible, which makes governance more urgent, not less. The third paper delivers a practical governance framework built on Cyber-Informed Engineering (CIE) principles, an emerging engineering discipline in the water sector, applied to AI safety challenges that cybersecurity was not designed to address.

The goal across all three papers is not to make you afraid of GenAI. It’s to help you more fully understand the risks and prepare to mitigate them before you deploy it.

Andy Bochman is Resilience Strategic Lead at West Yost Associates. He previously served as Senior Grid Strategist at Idaho National Laboratory, where he co-developed the Cyber-informed Engineering (CIE) methodology. He is the co-author of Countering Cyber Sabotage (CRC Press, 2021).

Comments and questions: resilience@westyost.com

TL;DR

Water utilities face a convergence of crises — workforce retirement, aging infrastructure, capital backlogs measured in trillions, and intensifying climate extremes— that make GenAI adoption feel less like a choice and more like a necessity. Building an effective governance framework requires confronting how these pressures are shaping decisions; if you don’t fully acknowledge the drivers, you can’t make intentional choices about adoption.

Introduction

The first paper in this series explained why GenAI risks are categorically different from traditional cybersecurity threats and why existing frameworks are poorly equipped to address them. But before we can build governance structures adequate to those risks, we must reckon honestly with the forces that are driving GenAI adoption in water utilities.

For many utility leaders, GenAI adoption will not feel optional. The promised efficiencies are too compelling, the workforce crisis too acute, and the infrastructure backlog too deep for any responsible utility leader to simply sit this transformational moment out. That is not an argument for reckless deployment — it is an argument for urgency about governance. You cannot govern what you have not honestly confronted.

This paper examines the operational and economic pressures that are accelerating GenAI adoption across the water sector, the genuine capabilities GenAI offers, and the human factors that complicate any deployment. The third paper in this series will provide the governance framework utilities need to deploy GenAI safely.

The Workforce Crisis

Number one among the pressures driving GenAI adoption is the fact that an aging workforce is draining vital expertise from utilities with every passing year. It is a workforce that, while not necessarily taking its knowledge to the grave, is at least taking it to the golf course, the beach, or a retirement village. The brutal fact is that many organizations see 30–40% of their workforce retiring in the next few years, compounded by significant hiring and retention challenges that mean similarly skilled or experienced replacements will be difficult to find. This convergence may portend:

• According to the EPA and Bookings, field inspection and operations capacity could fall by 30–50% as the retirement wave crests
• even as the amount of infrastructure requiring inspection is increasing due to its advanced age.
• Institutional knowledge evaporating as seasoned professionals walk out the door. The person who knows why an important pump station behaves oddly, or why a certain valve configuration exists, or what happened in the 1993 freeze that explains the current setup, is no longer accessible.
• With fewer qualified people diagnosing problems, outages linger longer — translating into the hard financial costs associated with emergency repair versus planned maintenance.

The Water Research Foundation’s Project 5321 report, published in September 2024, had multiple callouts envisaging AI agents as a promising pathway for “preserving institutional knowledge.” It also noted that in smaller utilities, “staff are already stretched thin, and vital knowledge often leaves with retiring employees.” And this: “Early pilots show that Large Language Models (LLMs) can preserve institutional knowledge and enable real-time insights, helping even the smallest utilities modernize workflows and compete on a more level playing field.”

Small and mid-size utilities will likely be more motivated to embrace GenAI solutions even more rapidly than their larger peers. If a high percentage of utilities serving 10,000 or fewer people can’t afford formal security governance, the adoption pressure may hit them hardest while they enjoy the least protection. This asymmetry — maximum pressure, minimum capacity — deserves the water sector’s focused attention.

The Infrastructure Backlog

On top of the worrying personnel issues, there is the fact that infrastructure is getting older faster than it can be replaced or upgraded. Water systems with pipes from the 1920s–1960s, treatment plants now reaching their design capacity much earlier than originally projected, pump stations running around the clock that were meant for only intermittent use. As the capital backlog is measured in the trillions across US water infrastructure, when someone offers you an AI system that can:

• Monitor hundreds or thousands of sensors in real-time, versus monthly field checks
• Predict pipe failures six months out based on pressure transients, flow patterns, and water chemistry
• Optimize pump schedules to reduce energy costs 20–30%
• Flag anomalies that would take a 20-year veteran to notice
• Provide 24/7 “expert” guidance to junior operators

…not only will turning to GenAI feel like the biggest no-brainer, but utility directors who don’t adopt it may come to be seen as irresponsible to their ratepayers. That is the uncomfortable reality utility leaders must navigate: the pressure to adopt is real, and it is not coming only from vendors.

The Control Room of the Near Future

There is one more thing the AI salesman may tell you — and on this point, they may well be right. Your current control room relies on SCADA screens showing current state; when something looks odd, it triggers a call to a senior technician. Hopefully they are available. But here is what AI will enable in the near future:

• A view on not only current state, but a probabilistic six-hour forecast of system behavior.
• Something close to instant context: “This pressure drop pattern matches 23 previous incidents, where 18 were frozen services, 3 were main breaks, 2 were submerged meter vaults.”
• Access to a “digital twin” of a retired expert troubleshooter that can explain why the 1997 configuration prevents a certain failure mode.
• What-if scenarios: “If I switch to the backup pump now versus waiting two hours, what are the cascade risks?”
• Optimization and safety recommendations delivered with confidence intervals and risk scores — a genuine decision support tool for operators of every experience level.

These are not marketing fantasies. Early deployments in analogous sectors: aviation, healthcare, and large-scale manufacturing, have demonstrated similar capabilities. The water sector will get there too. The question is not whether these capabilities will arrive, but whether the governance structures to manage them responsibly will arrive first.

The Regulatory Accelerant

One additional dynamic deserves mention. Some utilities have already attempted to ban GenAI from their operations. Their hands may be forced in time, as AI capabilities become embedded in newer versions of existing SCADA, asset management, and predictive maintenance systems they already use. The choice may not be whether to deploy AI, but whether to deploy it knowingly or unknowingly.

Moreover, if the Environmental Protection Agency (EPA) or the Department of Homeland Security (DHS) issue guidance that effectively discourages or restricts frontier cloud AI in OT-adjacent contexts — which sounds plausible given the current direction of critical infrastructure cyber policy — approaches that run customizable AIs locally on utility-controlled infrastructure may shift from a technical preference to a compliance requirement. Utilities that have done the governance work will be better positioned regardless of which regulatory direction the wind blows.

The Human Side of Deployment

The economic and operational case for GenAI adoption is compelling. But do not forget the human side of new technology deployments. Adoption — fast or slow — requires gaining operator trust and significant retraining, amid resistance from some. Operators who have spent careers developing expertise will not automatically welcome systems that seem to second-guess their judgment.

This is not merely a change management problem. It has safety implications. The cognitive offloading risk — in which operators grow so reliant on AI guidance that their own situational awareness and manual skills atrophy — is well-documented in aviation, healthcare, and military contexts. Water utilities must learn from those sectors rather than repeat their mistakes. Maintaining genuine human expertise alongside AI assistance is not optional overhead; it is a core safety requirement.

What Comes Next

The economic case is strong. The operational pressures are real. The adoption trajectory is largely set. The only remaining question is whether water utilities will build the governance capacity to manage GenAI safely before or after something goes seriously wrong.

The third paper in this series provides the answer: a practical governance framework built on Cyber-Informed Engineering (CIE) principles — an emerging engineering discipline in the water sector— applied to AI safety challenges that cybersecurity was not designed to address. It covers foundational prerequisites, procurement strategy, engineered controls, design simplification, planned resilience, and the governance gaps that agentic AI is already beginning to open.

Andy Bochman is Resilience Strategic Lead at West Yost Associates. He previously served as Senior Grid Strategist at Idaho National Laboratory, where he co-developed the Cyber-informed Engineering (CIE) methodology. He is the co-author of Countering Cyber Sabotage (CRC Press, 2021).

Comments and questions: resilience@westyost.com

TL;DR

Water utilities must build their own AI safety governance now, before deployment, because neither the federal government nor the AI industry will do it for them. This paper provides a practical framework built on Cyber-informed Engineering (CIE) principles — an emerging engineering discipline in the water sector— covering foundational prerequisites, the three most critical CIE principles for AI governance, agentic AI risks, and a standing caution about operational dependency that utility leaders must internalize before they flip the switch.

Introduction

The first paper in this series explained why GenAI risks are categorically different from traditional cybersecurity threats. The second made the economic case that these pressures are largely irresistible. This third and final paper delivers the governance framework — practical, actionable, and built on engineering principles that already exist in the water sector.

Let us begin with the immortal risk management observation of cybersecurity legend  Dr. Dan Geer: “The wellspring of risk is dependency.”

Geer’s observation should serve as a standing caution throughout everything that follows: do not let your human-based operational capabilities atrophy even as GenAI technologies begin to prove their worth. That caution is not a reason to avoid deployment’ it is the most important design constraint you will bring to it.

Though many may not know it, some water utilities still retain the ability to operate in fully manual or close-to-manual mode for extended periods. That is especially good news now that GenAI-based solutions leveraging Claude, Gemini, OpenAI, and others — including some built on open-source models — are knocking at the door. Water’s close cousin, the electric sector, digitized its operations more briskly and in so doing became wholly dependent on computers, networks, and the cloud to perform its operational functions. Their manual operations capability left the barn a long time ago. Yours has not. Protect it.

Six Foundational Prerequisites

As with building a house, you need to start with a solid foundation. Here are six foundational prerequisites, in order of urgency:

1. Build governance capacity. If your organization lacks it today, get it. For smaller utilities, this means bringing in external expertise and ensuring those voices are embedded in governance and decision-making processes. A governance structure that exists only on paper protects no one.
2. Get your data house in order. GenAI is only as good as what it can reach. Data must be scrubbed for accuracy, organized into repositories accessible to AI systems, and structured to yield insights across assets, teams, and time frames. This is not optional prep work — it is the single biggest determinant of whether your AI deployment succeeds or becomes an expensive hallucination engine.
3. Get procurement right. Determine what questions you will ask vendors, what contractual protections you will demand, and what testing you will require before any system touches production. If your vendor cannot explain how their model was trained, what happens when it fails, and who bears liability when it does, walk away.
4. Pilot relentlessly. Run AI-enhanced systems in advisory mode first, with recurrent red teaming, adversarial testing, and honest evaluation against ground truth. Do not promote any system to production until it has been stress-tested under conditions designed to make it fail.
5. Plan for failure modes. Before deployment, know how you will execute Plans B and C. Know how you will operate in a degraded mode for an extended duration. If you cannot answer these questions before you flip the switch, you are not ready to flip the switch. As prominent cybersecurity expert Josh Corman is known to say: “If you can’t protect it, don’t connect it.”
6. Build AI-specific incident response. When the AI makes a bad recommendation and causes a failure, how do you diagnose it? How do you recover? How do you determine whether it was a technical error, a data quality problem, or an attack? How do you prevent recurrence? These are not questions you want to be answering for the first time during a crisis.

Building from Engineering Principles That Already Exist in the Water Sector

The good news is that water utilities do not need to invent a governance framework for GenAI from scratch. The intellectual foundation already exists, as evidenced by the definitive book on CIE published in 2025: Building Cyber Resilience in the Water Sector. Developed at Idaho National Laboratory and now being adopted across the water sector with support from the Department of Energy (DOE) and the AWWA, CIE includes twelve principles for embedding security and resilience into engineered systems from the earliest design phases forward.

CIE was built for a world where sophisticated adversaries are assumed to already have access to your networks — where the question isn’t whether your systems can be compromised, but what happens when they are. That assumption maps directly onto the challenge of governing GenAI in water operations. The question isn’t whether your AI systems will produce a flawed recommendation, hallucinate a maintenance history, or drift from their training baseline. They will. The question is what happens to your operations, your infrastructure, and your community when they do, and how your organization responds.

Though there are twelve CIE principles, three in particular provide the structural backbone for an AI safety governance framework that water utilities can build today.

CIE Principle: Engineered Controls

In the CIE framework, engineered controls are physical or mechanical safeguards that function independently of digital systems. They cannot be hacked, corrupted, or persuaded. A pressure relief valve does not care what the SCADA system thinks the pressure is. A check valve prevents backflow whether or not the AI monitoring system is online. These are deterministic protections: they operate on the laws of physics, not the recommendations of algorithms.

When applied to AI safety governance, this principle demands that utilities identify every critical function where an AI system might influence operations and ask a deceptively simple question:

What is the engineered control that prevents catastrophe if the AI is wrong?

If an AI system optimizes chemical dosing, there must be a physical upper bound — a maximum feed rate that cannot be exceeded regardless of what the algorithm recommends. If an AI system manages pump scheduling, there must be mechanical pressure limits that prevent the kind of transient-induced fatigue described in the scenarios from the first paper in this series. If an AI system advises on valve configurations, the consequence of the worst possible recommendation must be survivable without human intervention.

This is not about distrusting the technology. It is about designing systems where trust is not required for safety. The most critical functions in a water system — maintaining safe pressure, preventing contamination, ensuring minimum flows — should be protected by engineered controls that are entirely indifferent to whether the AI is performing brilliantly or has become untrustworthy.

In practice, this means that before any GenAI system is granted the ability to influence a physical process, engineers must map the consequence space of its potential recommendations and ensure that hard physical limits bound the worst outcomes. No software guardrail, no matter how sophisticated, substitutes for a mechanical one on a function where failure means public harm.

CIE Principle: Design Simplification

This CIE principle holds that simpler systems are inherently more resilient, more auditable, and harder to attack. Complexity is the enemy of security because every additional component, connection, and dependency creates a potential failure point and an additional surface for exploitation.

Applied to AI safety governance, design simplification becomes the most counterintuitive — and most important — discipline a utility can adopt. The AI vendor’s pitch is, by nature, additive: more sensors, more data streams, more integration points, more cloud connectivity, more analytical capability layered on top of existing operations. Each addition may be individually justified. But the cumulative effect is an exponential increase in system complexity, and with it, an exponential increase in the number of ways things can fail in ways no one anticipated.

Design simplification for AI governance means resisting the temptation to connect everything to everything. It means asking, for each proposed integration: what is the minimum viable architecture that delivers the operational benefit while preserving the ability to understand, audit, and override every decision the system makes? It means keeping AI advisory systems decisively separate from control systems, so that a corrupted recommendation cannot propagate directly into a physical action without a human checkpoint. It means limiting the number of third-party API connections, cloud dependencies, and data pathways, because each one is a potential vector for data exfiltration, model corruption, or cascading failure.

Most importantly, it means preserving operational simplicity at the human layer. When an AI system makes operations more complex for operators to understand — when the reasoning behind a recommendation is opaque, when the number of variables in play exceeds human comprehension, when the operator’s role shifts from decision-maker to button-pusher — the system has become less resilient, not more, regardless of how sophisticated the algorithm is.

If your operators cannot explain why the system is doing what it is doing, the system is too complex for safe deployment in a safety-critical environment.

CIE Principle: Planned Resilience

The “Planned Resilience with No Assumed Security” principle may be the most directly applicable to AI safety governance. It requires organizations to plan for the failure of every security measure and every digital system, and to ensure that critical functions can continue when those systems are unavailable, unreliable, or are actively working against you. As Sarah Freeman, Chief Engineering at MITRE’s Cyber Threat Intelligence and Modeling group, says it best:

Resilient systems (i.e., those that can endure disruption) do not succeed solely because they are digitally secure. They succeed because they are engineered to remain controllable when digital systems fail, degrade, or behave unexpectedly.

For water utilities adopting GenAI, planned resilience translates into three concrete operational requirements.

Maintain manual operational capability

Maintain manual operational capability for every critical function that AI touches. This is where the water sector’s legendary conservatism becomes a genuine strategic advantage. Many utilities still retain the ability to run their systems in manual or semi-manual mode for extended periods — a capability that their counterparts in more aggressively digitized sectors have in many cases lost. That capability must be preserved deliberately and exercised regularly, not allowed to atrophy as AI systems prove their value and operators grow comfortable relying on them.

A graduated stress testing framework captures this well: A Day without SCADA. A Week without Power. A Month without AI. These are not hypothetical disaster scenarios. They are training exercises that should be conducted routinely, with increasing duration and complexity, to ensure that your organization can function when its most sophisticated tools are unavailable. The cognitive offloading risk is real and well-documented in aviation, healthcare, and military contexts. Water utilities must learn from those sectors rather than repeat their mistakes.

Build AI-specific incident response

Build incident response plans specifically designed for AI failure modes. Traditional incident response assumes you can identify the failure, isolate the affected system, and restore normal operations. But AI failures can be subtle, cumulative, and difficult to diagnose. When the AI has been providing slightly degraded recommendations for months, what does “normal operations” even mean? When operators have been following AI guidance that has been slowly eroding system resilience, how do you establish a baseline to recover to? These questions demand answers before deployment, not discovery during a crisis.

Structure procurement around failure

Structure procurement and vendor relationships around the assumption of failure. Contracts must specify what happens when the AI system produces harmful recommendations. They must require transparency into model architecture, training data, and update processes. They must guarantee the utility’s right to audit, override, and disconnect. And they must address the insurance and liability questions that no current framework adequately covers: when the AI is wrong and the consequence is a $50 million main break, who pays?

Filling the Governance Gaps for GenAI and Agentic AI

Beyond these three principles, utilities must confront several governance gaps that existing frameworks leave wide open.

New attack surfaces

When GenAI systems are connected to OT, they create attack vectors that didn’t previously exist. Adversarial attacks on AI models — prompt injection, data poisoning, and model inversion — are categorically distinct from traditional cyber threats and require different controls. This is precisely where CIE’s full twelve-principle framework plays a critical role.

AI-specific failure attribution

When an AI system makes a consequential wrong recommendation, utilities need a clear process to determine whether the failure was a technical error, a data issue, a model drift problem, or a deliberate attack. Without that diagnostic capability, you cannot fix what broke, and you cannot determine who bears responsibility. Today we are very far from having the ability to do this reliably.

Third-party dependency risk

Routing utility-critical decisions through third-party AI systems creates an unacceptable single point of failure. When those systems are unavailable or compromised — and they will be — the consequences cascade through your operations. Plan accordingly.

Agentic AI in OT environments

AI agents that directly issue control signals to OT — adjusting pump schedules, modifying chemical dosing, or reconfiguring valve positions without human intervention — represent a categorically different governance challenge than AI systems that draft reports or summarize data. The accountability and oversight requirements must be proportionally more rigorous.

A qualified human operator must always be in the loop for any AI action that can affect public health or safety.

The International Society for Automation’s authoritative briefing “AI Risks to Critical Infrastructure” states this clearly:

“Generative AI is not permissible for autonomous control in high-consequence control systems. However, it may be acceptable in these systems where a human in the command loop supplies the ultimate decision on an action. The indeterministic nature of generative LLM models makes the variability of GenAI responses unacceptable for autonomous actions.”

Also be on the lookout for the Water Research Foundation’s Project 5394, titled “Evaluating Scalability, Reproducibility, & Impact of GenAI & Agentic AI in the Water and Wastewater Sector,” which will focus on how utilities may overcome barriers to adoption of GenAI and agentic AI, including “actionable guardrails.”

The Guardrails Are Yours to Build

The three CIE principles — engineered controls, design simplification, and planned resilience — do not constitute the entire governance framework a utility needs. Nine additional CIE principles, from consequence-focused design to cybersecurity culture, each have direct applications to AI safety governance. But these three establish the essential posture: protect the critical functions with physics, not just software; keep the system simple enough to understand and override; and plan for the day when everything fails.

The deeper insight of CIE, and the reason it translates so powerfully to AI safety governance, is that it treats security and resilience as engineering problems first and technology problems second. You do not solve an engineering problem by buying a better firewall, and you do not solve an AI governance problem by buying a better AI. You solve it by understanding the consequences, simplifying the design, and building in the ability to survive the worst case. Water engineers have been doing exactly this for more than a century. The challenge now is to apply those same disciplines to the newest and most powerful tools arriving at the plant gate.

The economics are irresistible. The technology is here. The regulatory cavalry is not coming. The vendors will tell you what their systems can do; it is your job to determine what happens when they don’t function as advertised. The insurance industry hasn’t caught up. The liability frameworks don’t exist. The only entity with both the obligation and the ability to protect your ratepayers is you.

Resilience is now one of water utilities’ most critical assets. You are the one who must build it and maintain it. And when it comes to GenAI, remember:You are the guardrails.

Andy Bochman is Resilience Strategic Lead at West Yost Associates. He previously served as Senior Grid Strategist at Idaho National Laboratory, where he co-developed the Cyber-informed Engineering (CIE) methodology. He is the co-author of Countering Cyber Sabotage (CRC Press, 2021).

Comments and questions: resilience@westyost.com