| Vol. MCMLXXXIV Issue 42

BANISHING GRADIENTS

America's Loss Function

Study Finds 90% Of AI-Generated Code Is Just Stack Overflow With Extra Steps

Researchers discover coding assistants are 'basically expensive autocomplete'

Harold Finch Jr. (Industry Analyst) · · 4 min read
Code on a computer screen
Photo: Unsplash

CAMBRIDGE, MA — A groundbreaking study published this week by researchers at MIT has determined that approximately 90% of code generated by popular AI coding assistants is, statistically speaking, “just Stack Overflow with extra steps and a subscription fee.”

The research, which analyzed over 10 million code completions from leading AI programming tools, found that the overwhelming majority of AI-generated code either directly matches existing Stack Overflow answers or represents what researchers termed “vibes-based interpolation between multiple Stack Overflow answers.”

“We were expecting to find novel approaches, creative solutions, perhaps even a glimpse of machine creativity,” said lead researcher Dr. Patricia Hwang. “Instead, we found a $20-per-month service that’s essentially doing a really good job of what every developer has been doing for free since 2008: Googling things and copying the first answer that looks plausible.”

The study identified several distinct categories of AI-generated code:

  • Direct Copies (34%): Code that is character-for-character identical to the top-voted Stack Overflow answer, but with the variable names changed to sound “more enterprise.”

  • Confident Nonsense (23%): Code that looks syntactically correct and is delivered with extreme confidence but does not actually work, similar to an answer posted at 3 AM by someone who “thinks they remember how to do this.”

  • Franken-solutions (21%): Code that combines fragments from multiple Stack Overflow answers in ways that technically function but would make any senior developer “deeply uncomfortable.”

  • Actually Useful (12%): Code that meaningfully improves upon existing solutions, which researchers noted “could also be explained by the training data including blog posts by people who were already pretty good at this.”

  • Miscellaneous Chaos (10%): Code that appears to have been generated through what researchers can only describe as “statistical hallucination and vibes.”

The findings have sparked debate in the software development community.

“This is devastating news,” said Marcus Torres, a developer who has been using AI coding tools extensively. “I was under the impression that the AI was understanding my problems and crafting elegant solutions. You’re telling me it’s basically doing a really expensive Google search? I could have hired an intern for this. At least the intern would buy their own coffee.”

Representatives from major AI coding companies disputed the study’s methodology.

“Our systems don’t ‘copy’ from Stack Overflow,” said a spokesperson for one leading provider. “They analyze patterns across vast datasets to synthesize novel solutions. The fact that those solutions happen to match Stack Overflow is simply because Stack Overflow already contains the correct answers. Our AI is independently arriving at the same conclusions through advanced reasoning.”

When asked if this was meaningfully different from “a really expensive search engine,” the spokesperson paused for approximately 17 seconds before responding, “Our investors don’t see it that way.”

The study did note one area where AI coding tools significantly outperformed simple Stack Overflow searches: the ability to generate code comments that are technically accurate but entirely unhelpful.

“The AI is remarkably good at adding comments like ‘increments the counter’ above a line of code that increments a counter,” Dr. Hwang observed. “It’s almost impressive how consistently it manages to add documentation that explains what the code does rather than why it does it.”

Stack Overflow itself has reportedly requested that AI companies “send over a few bucks” for what the organization’s CEO described as “essentially licensing our entire content library to build products that are making us obsolete.”

At press time, a junior developer had asked an AI assistant to explain why their code wasn’t working and received a 500-word response that concluded with “This should work. If it doesn’t, try restarting your computer.”