Who is accountable when AI-generated code causes a failure?

Accountability is not one thing

Leaders often use "responsible" to mean four different things. Separating them prevents two unhelpful answers: blaming the engineer who clicked Accept, or asserting that the tool vendor automatically owns every consequence.

The table means that several parties can have duties at once. A cloud or model vendor may need to investigate an outage. A systems integrator may have made the faulty change. A customer may have misconfigured the service. The company that operates the affected service must still take immediate protective action because it controls the live system and customer relationship. Sorting out ultimate legal allocation can continue after containment.

A practical chain of accountability

Board or delegated executive sets risk appetite → product or business owner accepts the intended use and impact → service owner accepts the production release and recovery duty → engineers and reviewers perform the assigned work → security, privacy, and legal functions set or verify controls → vendors supply agreed capabilities and support

Scale the approval process to the consequences of the change. The service owner should be a real person with authority to delay a release, disable a feature, and request the people or budget needed to recover. The business owner decides whether the system should take the relevant risk at all. Security, privacy, legal, finance, and compliance have decision rights when the failure mode touches their domain. A RACI matrix can record who is Responsible, Accountable, Consulted, and Informed, but do not place “AI” in an accountable column. It has no authority or duty to act outside the system’s programmed behavior.

NIST’s AI Risk Management Framework treats governance as cross-cutting and calls for risk management throughout the AI lifecycle. It also emphasizes documented measurement, independent review where useful, and monitoring as risks evolve. Those are useful governance principles even when a coding assistant itself is not a regulated AI system. NIST AI RMF Core.

Why an AI tool cannot own accountability

Accountability requires agency in the organizational sense: authority to make or reverse a decision, access to relevant records and resources, a duty to report, and an entity that can be disciplined, sued, insured, or regulated. A code assistant has none of these on its own. It produces an output within a human-designed workflow. Calling it the “author” of a bug can be useful shorthand for technical investigation, but it should never terminate the inquiry into who designed, approved, and operated that workflow.

This does not mean the human user is always individually at fault. A junior engineer may reasonably use an approved assistant in a process that made independent review impossible, rewarded unsafe speed, lacked test environments, or silently allowed privileged deployment. In that case, management and system design are central to the professional and operational review. Conversely, an engineer who bypasses required controls or conceals a known risk may have a separate performance or disciplinary issue. Establish the roles and escalation path before an incident, rather than deciding them to fit the person involved after a failure.

Do not confuse tool output with a root cause. The causal chain may include an unsafe prompt, an inaccurate generated diff, a missing test, inattentive review, an overly broad deployment permission, a weak feature flag, an incompatible dependency, or a vendor outage. The useful post-incident question is, “Which control should have prevented or limited this failure, and who owned that control?” It produces an improvement. “Which person used AI?” often produces only a scapegoat.

Assign release ownership and approval by risk

Every change needs a named change owner, but the approval depth should rise with the possible impact. A one-line internal documentation fix and an AI-generated database migration should not travel through the same release path. Define risk tiers in advance, including a default rule for a change whose impact is uncertain.

The word “AI-generated” should be a risk signal when it changes the review burden, not a new release class that bypasses ordinary engineering. A small generated change can be dangerous when it affects authorization or a shared library. A large generated refactor can be relatively routine when it is isolated, well tested, and reversible. Risk is driven by blast radius, reversibility, data sensitivity, external commitments, privilege, detectability, and the quality of existing controls.

NIST’s Secure Software Development Framework is especially direct on ownership. It recommends defining software-development roles, assigning code ownership, and requiring provenance and integrity mechanisms for third-party components. NIST SP 800-218. An AI coding workflow should meet that standard of evidence rather than inventing a lower standard because the first draft came from a model.

Review evidence worth retaining

Keep evidence that explains what changed and why a person believed it was safe to release. The aim is operational learning and defensible decision-making, not a surveillance file about individual developers.

• The change request, intended user effect, risk tier, and named service owner.
• The proposed and merged diffs, review comments, approver identity, and any approved exceptions.
• The AI-assisted workflow when material: tool and version, prompt or task description, output or patch reference, model settings if available, and any known limitation. Do not log secrets, sensitive source, or personal data merely to prove AI use.
• Automated test results, manual test notes, security scans, performance checks, and results for both normal and denied or error paths.
• Dependency, container, and build provenance, including an SBOM when the risk or procurement requirement warrants it.
• Deployment artifacts, environment and configuration changes, feature flags, release time, canary results, monitoring dashboard, and tested rollback steps.
• Incident timeline, decision log, customer or regulator communications, vendor ticket, and corrective-action owner if a failure occurs.

For AI assistance, retaining the exact prompt is not always possible or appropriate. A clean alternative is to preserve the reviewed patch and a minimal provenance record showing that an approved tool was used for a described task. If the assistant had access to confidential code or production data, record the approved data path and access scope, not the sensitive material itself. The evidence must help a later reviewer reconstruct the decision without creating a second security or privacy incident.

Legal liability and vendor obligations

There is no universal rule that assigns legal liability to the person who typed an AI prompt, the model provider, or the company that deployed the code. The result may involve contract, negligence, consumer protection, privacy, cybersecurity, sector-specific rules, intellectual property, employment law, and product-liability doctrines. Different jurisdictions use different tests and remedies. Treat this section as a governance guide, not legal advice for a particular dispute.

In the EU, Directive (EU) 2024/2853 clarifies that software, including AI systems, is a product for the directive’s product-liability purposes. It generally treats a software developer or AI-system provider as a manufacturer, but has specific rules and exceptions, including an exception for free and open-source software developed or supplied outside commercial activity. The new directive applies to products placed on the market or put into service from 9 December 2026; as of this answer’s verification date, EU countries still have until that date to transpose it. Directive overview, European Commission implementation timetable. That is an example of why a leader should not make a blanket assertion that AI code is outside existing liability law.

The EU AI Act has separate, risk-based obligations. Its high-risk rules may impose documentation, log, monitoring, human-oversight, and incident-related obligations on providers and deployers. Not every coding assistant or internal use case is high-risk, so classify the actual system and role rather than assuming the label “AI” decides the answer. EU AI Act high-risk obligations overview, deployer duties in Article 26.

What contracts should allocate

Contracts cannot substitute for operational controls, but they decide how commercial parties cooperate and who bears defined costs when something goes wrong. Have counsel tailor the points below to the transaction, governing law, sector, and bargaining position.

• Define the service, permitted use, performance and support commitments, planned and emergency change process, and a clear allocation of customer and vendor responsibilities.
• Address security practices, vulnerability disclosure, incident notice timing, investigation cooperation, evidence preservation, audit rights, subcontractors, data processing, and data location where relevant.
• Specify remedies, service credits, warranties or disclaimers, limits of liability, carve-outs, indemnities, duty to defend where applicable, and how third-party intellectual-property or confidentiality claims are handled.
• Require an exit and continuity plan: data return or deletion, exportable logs and artifacts, model or service retirement notice, transition assistance, and access to evidence during an incident.
• Identify the contract hierarchy. A marketing assurance, online terms, data-processing addendum, order form, and negotiated enterprise agreement may not say the same thing.

Check what a limitation or indemnity clause actually covers and how it operates under the applicable law. In US commercial arrangements, for example, enforceability of risk-allocation clauses varies with the jurisdiction and transaction. Cornell Legal Information Institute on hold-harmless clauses. Preserve the executed versions, not just a link to current click-through terms.

Vendor responsibility is real but bounded

A vendor should be held to the promises and legal duties that attach to its role. That may include providing secure service, accurate documentation, vulnerability notices, incident support, data-handling commitments, or an agreed remedy. If a vendor controls a model update or hosted execution environment, its logs and cooperation can be indispensable in finding the cause. Require a support route that works during an incident and test it in a tabletop exercise.

However, a model provider usually does not know the customer’s business purpose, local access controls, deployment configuration, custom code, or the individual release decision. A customer cannot treat vendor terms as a substitute for code review, authorization tests, staging, logging, monitoring, or rollback. Conversely, a vendor cannot use a general disclaimer as a reason to conceal a material service incident or ignore a negotiated support obligation. The contract and applicable law determine the commercial allocation; the live operator still must protect users first.

Open-source code and component accountability

Treat third-party dependencies, copied snippets, generated code that becomes a dependency, containers, and build tools as software supply-chain inputs. The fact that an AI proposed the import or pasted the snippet does not change the organization’s responsibility to inventory, review, update, and govern what it releases. CISA describes an SBOM as a formal record of component details and supply-chain relationships. Its current minimum-elements guidance includes component names, versions, identifiers, and hashes, which help connect a deployed artifact to a later advisory or recall. CISA SBOM resources, CISA Minimum Elements for an SBOM.

Maintain a component policy that covers approved sources, license review, security advisories, update ownership, end-of-life, provenance, and emergency patching. Scan for known vulnerabilities but do not mistake a clean scan for proof of safety. Verify a component’s function, permissions, maintainers, integrity, and compatibility with the deployment. NIST’s supply-chain guidance and SSDF both support managing component provenance and integrity as part of secure development. NIST SP 800-161 Rev. 1, NIST SP 800-218.

Do not assume that AI output is either “open source” or free of license and intellectual-property questions. An output may be newly generated, may resemble known code, or may bring in an identifiable dependency or notice. Those are different cases. Put license, attribution, and provenance review on the escalation path for material code, especially when the code is shipped externally or used in a regulated, safety-critical, or high-value product.

Incident response after an AI-assisted failure

The incident response should look much like any serious software failure, except that the evidence package may include AI workflow and supplier records. Avoid arguing about blame while users remain exposed.

1. Contain and protect. Disable the feature, roll back, revoke exposed credentials, restrict access, or use a safe fallback. Assign an incident commander with authority to make time-sensitive calls.
2. Preserve facts. Record the timeline, deployed artifact hashes, configurations, logs, feature flags, approvals, test evidence, AI-tool provenance where material, affected data, and vendor communications. Coordinate with legal counsel before deleting or altering evidence that may be relevant to an investigation.
3. Assess impact and obligations. Identify affected users, data, money, safety, availability, contractual commitments, and jurisdictions. Bring in privacy, security, legal, compliance, communications, and the vendor according to the prepared playbook.
4. Recover and verify. Use a tested rollback or correction, validate the fix in an environment representative of production, monitor for recurrence, and communicate with affected parties under the applicable obligations and agreed terms.
5. Learn and assign corrective actions. Run a blameless technical review plus an accountable management review. Identify the failed control, owner, due date, acceptance evidence, and follow-up audit. Do not close the incident merely because the immediate patch worked.

NIST’s current incident-response profile treats response as part of cybersecurity risk management and says lessons learned and root-cause analysis should improve future risk management. NIST SP 800-61r3. That is the right objective for an AI-generated defect: repair the technical flaw and improve the decision system that allowed it through.

Example

Setup. A team uses an approved coding assistant to accelerate a change to a customer-export service. The generated patch moves an authorization check after a cached lookup. Reviewers see passing happy-path tests, but no test asserts that a caller cannot retrieve another customer’s export. The change is deployed behind a feature flag and an alert identifies cross-tenant access attempts.

Action. The service owner disables the flag and the incident commander preserves the release artifact, access logs, test record, review comments, and the minimal AI-assistance provenance permitted by policy. Security and privacy assess exposure while the developers correct the check, add negative authorization tests, and verify cache key isolation. The accountable executive decides whether to notify customers or authorities after legal and privacy review. Procurement invokes the vendor-support process only if the investigation shows a hosted tool, model behavior, or service failure materially contributed. The organization does not wait for the tool vendor to decide whether to contain the service.

Takeaway. Operational accountability belonged to the operator and named service owner because they controlled the release and feature flag. The developers and reviewers had professional responsibilities defined by their roles. The vendor’s obligations depend on the evidence, agreement, and applicable law. The corrective action is not “ban AI” or “blame the person who accepted the patch.” It is to enforce authorization-test gates, risk-tiered review, release evidence, and a rollback-ready deployment path.

Insurance and leadership decisions

Insurance transfers only covered financial risk. It does not restore a service, satisfy a regulator’s operational expectations, or remove the need for a decision owner. Coverage depends on the actual policy, endorsements, exclusions, limits, retentions, claim facts, and relevant law. A legal liability finding alone does not establish that a policy responds. Cornell Legal Information Institute on liability-insurance coverage.

Ask counsel, risk management, and the broker to map likely AI-assisted software-failure scenarios against cyber, technology errors and omissions, general liability, directors and officers, professional liability, crime, and other potentially relevant policies. Confirm insured entities, territorial scope, notice deadlines, consent-to-incur-costs requirements, defense arrangements, exclusions, and whether contractual indemnities change the analysis. Include the insurer’s breach coach or incident-response process in tabletop exercises if the policy requires or provides it.

For leaders, the practical answer is to fund the controls before trying to transfer the loss after an incident. Set a risk appetite, make service ownership explicit, require evidence proportional to impact, and rehearse the incident path. Review the accountability map whenever you add a coding assistant, change a vendor, widen autonomous deployment permissions, enter a new market, or connect code generation to sensitive data or critical infrastructure.

Decision checklist

Before approving production AI-assisted code, a leader should be able to answer yes to each of these questions:

• Is there one named service owner with authority to stop or roll back the release?
• Has the business owner accepted the intended use and material failure modes?
• Does the change have a risk tier, reviewer, test evidence, deployment record, and rollback or compensating-control plan appropriate to its impact?
• Are authorization, privacy, security, dependency, and negative-path tests included where the change touches those boundaries?
• Can the team identify the deployed artifact and relevant third-party components, and obtain needed vendor assistance during an incident?
• Do the contract, data terms, and insurance program have an owner who has reviewed their fit for this service and jurisdiction?
• Is the incident commander, communications path, evidence-preservation process, and corrective-action tracker ready before the failure happens?

If any answer is no, the organization has identified a governance gap. Fixing that gap is more valuable than trying to assign accountability to the AI after something breaks.

Limits and legal caution

This answer provides general leadership and risk-management guidance, not legal, tax, insurance, compliance, or professional advice. Liability and notification duties are highly fact-specific and can change with jurisdiction, contract terms, sector, affected individuals, product type, and the date a product was placed on the market. Get qualified local advice early in a serious incident, preserve evidence, and do not rely on a generic vendor assurance or a generic policy summary.