Risks of AI-Generated Legislation in the Public Policy Process

Governments are already using AI to help write laws. Not in some hypothetical future: right now, in federal agencies and city halls, across drafting stages most citizens never see. This piece looks at how AI actually touches legislation, what breaks at each point it gets inserted, and what it would take to catch problems before they harden into statute.

An OECD report counted close to 200 examples of governments using AI across their functions worldwide, including regulatory design and policy evaluation. In one country, the federal government plans to use one AI model to speed up deregulation, including drafting supporting text for rulemaking. At the local level, cities are already using AI to draft resolutions, summarize what constituents are saying, and back up decisions with data.

"AI-generated legislation" isn't one function. It covers language drafting, policy simulation, spotting inconsistencies across legal codes, even writing committee notes. Each of those is a separate door into the bill-writing process, and each carries its own version of risk. Treating AI as either a menace or a shortcut misses the point, and it's also the wrong place to start. The tools aren't going away. What matters is whether the risks get named precisely enough that citizens and institutions can push for real safeguards, instead of either banning the tools outright or shrugging and letting them run unchecked.

Why hallucinations in legal drafting are structurally different from hallucinations elsewhere

In 2023, a lawyer submitted court filings written with an AI chatbot that cited cases that never happened. Fake case names, fake docket numbers, quoted passages that didn't exist anywhere in the legal record. That case set the baseline for what AI failure looks like in a legal context: not a garbled sentence, but a confident, fully formed fabrication that reads exactly like the real thing.

Legislation doesn't work like a search result you can double-check and toss aside. A phrasing error, or a fabricated precedent quietly baked into a bill's text, survives committee review, gets voted on, and then shapes how the law gets enforced. The mistake doesn't just sit there. It compounds at every stage downstream, because each stage assumes the one before it did its job. A subtle misstatement can shift what a law actually means, opening gaps nobody intended or loopholes nobody meant to leave.

There's a staleness problem too. AI models cite outdated statutes with total confidence, and in a fast-moving policy area like tech regulation or healthcare, "outdated" might mean last session, not last decade.

What makes the 2023 filing so instructive is who missed it. Lawyers are trained specifically to read legal language critically, and they still didn't catch the fabrications before filing. So what chance does legislative staff have? They face that same blind spot, minus the professional accountability structure, bar associations, malpractice exposure, judges who can sanction you, that at least gives lawyers a reason to double-check. The danger is that AI gets things wrong in ways people can't spot. It's that AI produces text that sounds exactly as authoritative whether it's right or completely invented, with no built-in flag to tell the difference.

How training-data bias shapes legislative language in ways drafters cannot see

Bias in AI drafting doesn't announce itself. The choices baked into an AI model's training data show up in the choices baked into its output, and neither the reader of the final draft, nor often the person who ran the tool, can trace where those choices came from.

Take the agenda-setting phase first. When AI manages large volumes of public input, it risks quietly excluding underrepresented groups, or getting misused to manufacture the appearance of public opinion that isn't really there. Move to policy formulation, and if AI is simulating impacts or drafting actual language, whatever got excluded earlier gets imported straight into the bill.

Layer on top of that a transparency problem. When a policymaker's decision traces back to an AI recommendation and nothing else, they often can't explain the reasoning behind it beyond pointing at the algorithm. The audit trail runs straight into a wall. Research into digital tools used for citizen participation has found a real weakness in how these tools communicate accountability information back to citizens, specifically around how decisions get made, carried out, and evaluated. Civic tech was already opaque before AI drafting showed up. AI just makes it worse.

Two layers are worth separating here. One is bias in what gets drafted: whose interests end up centered in the language. The other is opacity in how the draft got produced: whether anyone can reconstruct the reasoning after the fact. Neither shows up if all you do is read the finished bill. That's the uncomfortable part, and it's exactly why reading the bill can't be the whole safeguard.

The accountability gap that opens when a machine participates in writing law

One international parliamentary body's 2024 guidelines on AI in parliaments named transparency, accountability, and public trust as the central governance challenges tied to AI use inside parliaments. Legislators and staff who work with these systems directly have raised the same concern. This is an inside critique, coming from the institutions using the tools themselves, not from outsiders guessing at what might go wrong.

Democratic legitimacy rests partly on citizens knowing how laws get made and who answers for what ends up in them. When AI takes part in drafting without anyone disclosing it, that understanding erodes quietly, even if nothing else about the process looks different. Joanna Bryson, an AI ethicist at the Hertie School in Berlin, points to what might be called the fragility problem: outages, a sudden change to the underlying model, or leverage held by a vendor can all shift outcomes if AI sits at the center of a democratic process. Her standard for what's needed is blunt. Systems have to be auditable and owned, so there's "someone that can be held accountable who did what, when."

There's a security angle too. Draft bills are sensitive documents, often confidential well before public release, and pasting that language into a public chatbot risks exposing government text to outside model training nobody signed off on. The accountability gap has a security problem sitting right next to it.

These events are already documented facts. It's happening now, in legislative chambers, without anything close to the public debate the stakes call for. The problem lies in how AI takes part in writing a law. It's that it took part without disclosure, without an accountability structure attached, and without any way for citizens to know it happened or push back on it.

What happens to public trust when people learn AI helped write their laws

An August 2025 survey found just 29% of Britons trust their government to use AI to carry out some of its tasks. That's a low floor to begin with, before anyone even raises the question of undisclosed AI involvement in writing actual legislation. Surveys across multiple countries have consistently found governments more distrusted than trusted. AI adoption doesn't land on neutral ground. It lands on top of a deficit that already exists.

A deeper worry sits underneath those numbers. Governments risk treating AI as a string of small technical fixes rather than a real structural shift, patchwork solutions that skip over any serious thinking about what this means for the long-term legitimacy of public participation. Take the UK government's "Consult" tool, which summarizes public comment and maps recurring themes. On the surface, that looks like a routine efficiency upgrade, nothing more than faster paperwork. But once that summary feeds into a consequential decision, the filtering method itself becomes part of the decision. Whatever the tool decided not to surface never reaches the people voting on the bill. Ruth Fox, director of the Hansard Society, has noted that public officials are already alive to the questions this raises about human validation.

Trust doesn't come back just because someone pulls AI out after the fact. It gets built or broken during the process itself, while the drafting is happening. That's exactly why safeguards need to sit inside the process from the start, not get bolted on once the damage is already visible.

Where the regulatory response stands, and where it falls short

Lawmakers are not sitting still. As of March 2026, legislators across 45 states had introduced 1,561 AI-related bills, according to MultiState, already surpassing the total volume from 2024. On public sector AI specifically, CDT counted 50 bills introduced across 20 states in 2025, with 15 signed into law, up from 43 bills and 12 laws the year before.

Risk management was the fastest-growing category: 25 proposed bills, ten of which became law, aimed at requiring public agencies to put safeguards in place, things like acceptable use policies, impact assessments, notice and disclosure rules, and mandates for human oversight. Many public sector AI bills from 2024 called for reporting, pilot projects, task forces, the kind of measures that sound like action but don't force anyone to do anything differently. Six 2025 bills would have created real risk management requirements for AI tools that state agencies procure. None of them passed.

A few transparency proposals stalled too: Georgia's HB 147 would have required annual inventories of AI tools, North Carolina's SB 747 called for a one-time inventory, and Alaska's SB 2 proposed its own form of AI inventory requirements. At the federal level, a December 2025 executive order titled "Ensuring a National Policy Framework for Artificial Intelligence" pushes for a framework described as minimally burdensome, and it created an AI Litigation Task Force specifically to challenge state AI laws. That adds a preemption fight on top of an already fragmented state-by-state landscape. One regional bloc's AI law, adopted in 2024, sorts AI systems into four risk tiers and bans the ones judged unacceptable, making it the most structurally thorough model currently in force anywhere, though it binds no jurisdiction outside its own.

So there's plenty of legislative activity. But volume isn't depth, and most of what's on the books addresses AI in government broadly, in the abstract, as a category. AI's specific role in drafting actual legislative language remains, for the most part, untouched. The gap that matters most is also the one almost nobody is legislating toward.

What meaningful safeguards for AI-drafted legislation actually require

Disclosure has to be the floor, not a goal to work toward eventually. Citizens and legislators need a clear, reliable way to know when AI took part in drafting a bill. Watermarking, provenance tracking, and metadata that records how a document was produced are the technical mechanisms already under active discussion, including in the bipartisan federal COPIED Act and California's AI Transparency Act.

Human validation needs to be a formal requirement, not an informal habit some offices happen to follow. Ruth Fox's point about officials worrying AI could become a "default crutch" points straight at this: if review is optional, it eventually stops happening. Nobody skips a mandatory step on purpose, but an optional one gets skipped the first busy week it's inconvenient. Bryson's standard for auditability applies here too: systems traceable back to a specific accountable person at a specific point in time, rather than a black box run by a vendor with no obligation to explain itself.

There is also process to consider. CDT's AI Governance Checklist for Elected Officials lays out a practical template already circulating: public inventories of AI tools in use, advisory councils that include actual members of the public, real feedback mechanisms, and notices written in plain language rather than legal or technical jargon. Bellevue, Washington built its own AI Policy and Guidelines covering procurement, bias reduction, data privacy, periodic review, community engagement, and public records. That's proof this kind of framework can actually run inside a local government, not just sound good on paper.

The OECD's April 2025 paper, Tackling Civic Participation Challenges with Emerging Technologies: Beyond the Hype, frames the standard about as cleanly as it can be framed: AI tools, if governed transparently and kept aligned with participatory values, can support more inclusive civic engagement. Everything rides on that one small word, "if." Safeguards aren't a reason to throw these tools out. They're the condition that makes using them legitimate in the first place.

How citizens can use the same tools, and demand accountability for how government uses them

The legislative process already gives citizens a few ways in: committee hearings, contacting a representative directly, submitting public comment. All useful, but all reactive. They respond to something government already decided to do, and none of them generate anything on their own.

AI tools can flip that script. The same capabilities that create risk inside government, simplifying dense legal text, translating across languages, opening up dialogue on civic platforms, create real opportunity when citizens use them directly, as long as transparency governs how they're built and used. Citizen initiative mechanisms, available in a number of states, let voters collect signatures and put a proposed law or constitutional amendment straight on the ballot. That's a structural cousin to the verified co-signature model now emerging in civic tech more broadly.

The accountability gap running through this whole piece cuts both ways. Citizens who understand how AI can introduce bias, opacity, and outright fabrication into government drafting are simply better equipped to question legislation built that way. Knowing the risk is itself a form of power, maybe the most durable one on offer here.

That's a structural cousin to collaborative models now emerging in civic tech. That's the model worth building toward: disclosed, auditable, grounded in real community verification instead of a process nobody outside the building can see into. The risks of AI-generated legislation don't argue for backing away from citizen engagement. They argue for engaging harder, with tools that earn legitimacy by being honest about what the AI did and what a human actually checked.