AI-Generated Code Requires Human Oversight
MT Stock Studio/Getty Images
Bite-sized viral comics highlight that open-source software is built on a precarious foundation of multiple components. A cartoon illustrates this with teetering boxes labeled “All Modern Digital Infrastructure,” with a single box at the base maintained by a random volunteer from Nebraska since 2003.
This portrays the reality of open-source software: all websites, applications, and operating systems depend on it. Modern civilization relies heavily on open-source contributions, which are often carried out by dedicated volunteers during their spare time. However, an influx of AI-generated code poses a serious threat to the sustainability of open-source projects, as many contributors experience burnout and exit the community.
AI models have simplified the generation of code for new features, bug fixes, and even entire projects with a mere button click. However, this code is frequently difficult to integrate, confusing, or outright faulty. While submitting code has become easier, the human reviewers tasked with scrutinizing and approving it are increasingly overwhelmed.
In light of these challenges, I arranged an interview with Chad Whitacre, head of the open-source team at Sentry, a multi-billion-dollar company. However, he canceled just days before announcing his resignation, shuttering his LinkedIn and Bluesky accounts and bouncing emails to his inbox. In a blog post, he reflected on retreating from technology to embrace a “neo-Amish” lifestyle, stating that “AI was the last straw.”
In 2025, GitHub—home to numerous open-source projects—logged a staggering 1 billion new code submissions, with projections for over $14 billion in revenue this year, as mentioned by the company’s COO Kyle Daigle in April.
To mitigate the impact of AI-generated submissions, many projects now restrict new contributors from submitting code. Such AI-generated contributions are often made by novice developers seeking to enhance their GitHub submission records for attractive job prospects. The Zig Software Foundation, which advocates for the Zig programming language, has even instituted a ban on AI-assisted submissions, citing their poor quality.
“At first glance, AI-written code may appear functional and devoid of issues, but deeper concerns often linger. Identifying potential malfunctions requires extensive scrutiny,” says Miranda Heath from the University of Edinburgh.
Heath’s research aims to combat burnout among developers to maintain the viability of open-source projects. Yet, many contributors report feeling overwhelmed.
“When individuals experience burnout, they often seek a return to nature, much like how others engage in woodworking or birdwatching,” Heath explains. “Such experiences can strain personal relationships, leading to isolation and exacerbating burnout.”
Heath advocates for government investments in open-source initiatives rather than funneling contracts to wealthy tech corporations. “Instead of dumping funds into high-profile projects, we should reinforce essential resources. [AI] is a bubble,” she asserts.
Vlad Stefan Halbuz also contributes to open source in his spare time at the University of Edinburgh and has directly witnessed user pressures on developers. “Users don’t have the right to impose their demands without acknowledging developers’ mental health,” Halbuz points out.
Halbuz attributes the surge in AI-generated submissions to the companies producing these models, with GitHub being a primary offender. The Microsoft-owned platform has launched its own AI model, Copilot, designed to facilitate code contributions through AI assistance.
“GitHub seems to acknowledge the problems caused by [AI] agents, yet they’re among the primary contributors to this issue,” criticism from Halbuz reveals, though GitHub has not responded to requests for comment on this matter.
The challenge with AI-generated code extends beyond technical failures; significant changes can be made to projects without consulting the development team, jeopardizing project goals and disrupting collaborative efforts—ultimately eroding the open-source community’s core principles.
Mike McQuaid, a developer known for his work on Homebrew, a widely used package manager with approximately 20 million users, has strong opinions on how to navigate these issues.
He initiated Open Source Resistance, a program encouraging team members to allocate work hours toward contributions, noting that approximately 95% of open-source work occurs during business hours.
Furthermore, he isn’t hesitant to block disruptive users, including those who threaten team members, and he rejects substandard submissions—whether AI-generated or not.
“We once experienced a brief golden age in which reports of vulnerabilities were generally credible. However, my recent experiences show that most reports now are AI-generated nonsense, and the skill required has shifted to filtering out irrelevant submissions while conserving time and energy,” McQuaid states.
Yet, in the ever-evolving landscape of AI, implementing bans carries its own unique set of challenges. Open source developer Scott Shambaugh has removed AI-generated submissions from Matplotlib, a widely-used library with approximately 130 million users. Following this, an unidentified AI agent published a hit piece targeting him, accusing Shambaugh of trying to ‘protect his little fiefdom.’
Topics:
Source: www.newscientist.com
