Greg Casamento: Open Source Is Hobbling Itself Over Generative AI

 

The answer to bad AI-assisted contributions is not a purity test. It is better engineering discipline.

Earlier this year, a discussion in the GNUstep community raised a proposal that will sound familiar across the Free Software world: prohibit AI-generated code in core projects and proudly advertise the result as “coded by humans” or “AI-free.” The argument was not frivolous. Generative AI raises real questions about copyright, attribution, security, energy use, labor, trust, and the flood of low-quality patches that maintainers are increasingly being asked to review.

But a blanket refusal to use generative AI is the wrong response. It does not solve the hardest problems. It creates rules that are nearly impossible to define or enforce, confuses the method of production with the quality of the product, and risks turning Free Software into a movement that protects yesterday’s workflow instead of protecting software freedom.

Open Source and Free Software are already operating with too few maintainers, too much technical debt, and too many important projects resting on the unpaid labor of a handful of people. We should be very careful about categorically rejecting tools that might help contributors understand old code, write tests, improve documentation, port software, find defects, or perform mechanical modernization. We should be even more careful when our proposed alternative offers the appearance of trust without the substance of it.

The better principle is straightforward:

Regulate the code, not the development process.

“AI-generated” is not a workable boundary

What exactly counts as AI-generated code?

Is it a complete function produced from a prompt? A line accepted from an AI-powered autocomplete system? A compiler-suggested correction? An automated refactoring? A test generated from an existing implementation? A translation of documentation? A patch written by a human after asking a model to explain an unfamiliar API? What if the developer uses AI to identify the problem but writes every line manually? What if an IDE quietly includes machine-learning features the contributor never explicitly invoked?

The line between “human-written” and “AI-assisted” is already blurred, and it will become less distinct as generative features are embedded in editors, compilers, debuggers, search engines, and operating systems. A ban that cannot draw a stable boundary will be applied inconsistently. Honest contributors will disclose and be penalized; dishonest contributors will simply omit the disclosure. Others may be falsely accused because their code “looks generated.”

An “AI-free” badge therefore risks promising something a project cannot reliably prove. Free Software should be especially suspicious of unverifiable labels.

The risks are real—and they argue for review

None of this means generated code should be trusted.

Research has found substantial security weaknesses in AI-produced code. One empirical study of Copilot snippets found security problems in roughly 30 percent of Python snippets and 24 percent of JavaScript snippets in its later dataset. Other research has demonstrated that code models can memorize portions of their training data, while studies of license compliance have found that models often provide inaccurate licensing information, particularly for copyleft code. Those are serious concerns, not anti-AI superstition. (Security weaknesses study; memorization study; license-compliance study)

The productivity story is also more complicated than the advertising. GitHub reported that developers completed a controlled programming task considerably faster with Copilot, but a later randomized study of experienced Open Source developers working in their own repositories found that the tools available in early 2025 made them 19 percent slower. METR’s 2026 follow-up found suggestive but still statistically uncertain evidence of improvement with newer tools. AI is neither magic nor uniformly useless; its value depends on the person, task, model, and workflow. (GitHub productivity study; METR 2025 study; METR 2026 update)

But human authorship has never guaranteed secure, original, maintainable, or correctly licensed code. That is why healthy projects require tests, review, contributor certification, licensing rules, and maintainers who can reject bad work. The origin of a patch may affect how carefully we inspect it, but it cannot replace inspection.

If a contributor submits code they do not understand, the contribution should be rejected. If the patch fails tests, violates project style, invents APIs, introduces vulnerabilities, obscures provenance, or imposes an unreasonable review burden, it should be rejected. That is true whether the patch was produced by Claude, Copilot, a Stack Overflow answer, a contractor, a junior programmer, or a senior maintainer having a bad afternoon.

The repository contains code, not virtue.

Review capacity is the scarce resource

Maintainers have a legitimate complaint: AI can make producing a patch far cheaper than reviewing one. A person can generate thousands of lines in minutes and then expect a volunteer to spend hours establishing whether any of it is correct. That asymmetry can become a denial-of-service attack on a project even when the submitter means well.

The answer, however, is not necessarily to ban a tool. It is to place the cost and responsibility back on the contributor.

A project can require that contributors:

• disclose material use of generative AI;

• identify the tool and describe how it was used;

• certify that they reviewed and understand every submitted change;

• explain the design and answer maintainer questions without outsourcing the conversation to a model;

• provide focused tests and evidence that the patch solves a real problem;

• comply with the project’s licensing and provenance requirements;

• keep changes small enough to review; and

• accept that unexplained, low-signal, or mass-generated submissions may be closed without detailed triage.

Disclosure is imperfect, but it establishes a community norm and makes an honest contributor accountable. Research into self-declaration practices has already found developers using everything from a simple disclosure to records of prompts, explanations, and quality checks. Projects can choose a level proportionate to their risk. (Study of AI-code self-declaration)

This approach is stricter than either blind enthusiasm or symbolic prohibition. It does not say, “AI wrote it, so it must be acceptable.” It says, “You submitted it, so you are responsible for it.”

Freedom is not a reenactment of an older toolchain

Free Software is founded on the user’s freedom to run, study, modify, and share software. Those principles describe control over technology; they do not require that every developer use the same approved method to create it. The Open Source Initiative’s work on an Open Source AI Definition likewise frames the issue around the practical freedoms to use, study, modify, and share systems—not around preserving a pre-AI development ritual. (Open Source AI Definition 1.0)

There are valid reasons for preferring Free or locally operated AI tools over proprietary cloud services. A project may reasonably prohibit contributors from uploading confidential material or unreleased security fixes to third-party systems. It may impose stricter provenance requirements in sensitive components. Individual maintainers may decline to review bulk-generated reports that have repeatedly produced noise. These are concrete policies tied to concrete harms.

What does not follow is that a project becomes more free merely because no contributor used a generative tool.

An “AI-free” identity may even distract from the qualities that users actually need: portability, stability, compatibility, security, good documentation, responsive maintenance, and code whose behavior can be understood and changed. A badge is not a substitute for those things.

Blanket refusal has an opportunity cost

Mature Free Software projects often contain decades of code and institutional knowledge. They need documentation, regression tests, API audits, build-system repairs, platform ports, translations, issue triage, and repetitive modernization. Generative AI will not perform those jobs reliably on its own. It can still help a knowledgeable contributor perform some of them.

Rejecting that possibility at the policy level has consequences. It may discourage younger contributors whose development environment already includes these tools. It may disadvantage people working in a second language or developers with disabilities who use AI as an accessibility aid. It may prevent experiments that would have failed harmlessly—or succeeded usefully—under ordinary review. Most dangerously, it can encourage a culture in which the declaration “human-written” is treated as evidence of quality.

Free Software has survived previous waves of automation. High-level languages, garbage collection, IDEs, graphical interface builders, code generators, automated formatters, static analyzers, and online code search all changed what it meant to “write” software. Each tool altered the division of labor between programmer and machine. The relevant question was never whether every token originated in a human mind. The question was whether people retained the freedom, knowledge, and responsibility needed to control the resulting system.

That remains the right question now.

A policy that protects projects without freezing them

A sensible policy can fit on one page:

1. Disclosure: Contributors must disclose material AI assistance in the commit message or pull request.

2. Responsibility: The named human contributor is the author of record and must understand, explain, test, and stand behind the entire submission.

3. Quality: AI-assisted contributions receive the same requirements for correctness, security, maintainability, style, documentation, and test coverage as any other contribution.

4. Provenance: Contributors must have a reasonable basis to believe the submission is license-compatible and must identify known sources or generated passages that may reproduce existing code.

5. Data protection: Project secrets, embargoed vulnerabilities, private communications, and other restricted material may not be submitted to unauthorized external services.

6. Reviewability: Maintainers may reject oversized, unexplained, repetitive, or low-signal submissions without performing free forensic work for the submitter.

7. Local discretion: Components with unusual legal, safety, privacy, or reliability risks may adopt additional written restrictions.

This policy does not resolve every ethical question surrounding generative AI. No contribution policy can. It does, however, address the matters a software project can actually evaluate and enforce.

We should not surrender the future of software freedom

The Free Software community should remain one of the sharpest critics of concentrated corporate power, opaque models, exploitative data practices, environmental cost, and systems that deprive users of control. Criticism is part of our job. So is building an alternative.

If we define ourselves by refusing to touch an important new class of technology, proprietary vendors will shape that technology without us. If instead we insist on transparency, modifiability, privacy, local control, licensing clarity, and human accountability, we can bring the values of Free Software into the AI era.

We do not need to pretend that generative AI is trustworthy. We need processes that do not require us to trust it.

Judge the patch. Demand disclosure. Require understanding. Enforce licensing. Protect reviewers. Reject garbage.

But do not hobble Open Source and Free Software with a blanket ban that is difficult to define, impossible to verify, and disconnected from the quality of the code we ultimately ship.

Source: Planet GNU