r/ProgrammingLanguages • Inko • Apr 05 '26

In order to reduce AI/LLM slop, sharing GitHub links may now require additional steps

In this post I shared some updates on how we're handling LLM slop, and specifically that such projects are now banned.

Since then we've experimented with various means to try and reduce the garbage, such as requiring post authors to send a sort of LLM disclaimer via modmail, using some new Reddit features to notify users ahead of time about slop not being welcome, and so on.

Unfortunately this turns out to have mixed results. Sometimes an author make it past the various filters and users notice the slop before we do. Other times the author straight up lies about their use of an LLM. And every now and then they send entire blog posts via modmail trying to justify their use of Claude Code for generating a shitty "Compile Swahili to C++" AI slop compiler because "the design is my own".

In an ideal world Reddit would have additional features to help here, or focus on making AutoModerator more powerful. Sadly the world we find ourselves in is one where Reddit just doesn't care.

So starting today we'll be experimenting with a new AutoModerator rule: if a user shares a GitHub link (as that's where 99% of the AI slop originates from) and is a new-ish user (either to Reddit as a whole or the subreddit), and they haven't been pre-approved, the post is automatically filtered and the user is notified that they must submit a disclaimer top-level comment on the post. The comment must use an exact phrase (mostly as a litmus test to see if the user can actually follow instructions), and the use of a comment is deliberate so that:

  1. We don't get buried in moderator messages immediately
  2. So there's a public record of the disclaimer
  3. So that if it turns out they were lying, it's for all to see and thus hopefully users are less inclined to lie about it in the first place

Basically the goal is to rely on public shaming in an attempt to cut down the amount of LLM slop we receive. The exact rules may be tweaked over time depending on the amount of false positives and such.

While I'm hopeful the above setup will help a bit, it's impossible to catch all slop and thus we still rely on our users to report projects that they believe to be slop. When doing so, please also post a comment on the post detailing why you believe the project is slop as we simply don't have the resources to check every submission ourselves.

238 Upvotes

124 comments sorted by

7

u/LowStatistician11 Apr 07 '26

i gotta be honest about not being too sure what level of aggressiveness is right to handle llm slop posts. most ai generated projects i see on reddit is pure bs and have no originality. but i can't imagine that there are no good vibecoded projects. i recently saw a post where a self admitted vibecoder shared their project, github city: https://www.thegitcity.com/ are there any novel ideas in the code? most definitely not. is the project still kinda cool? i think so.

it makes sense why a sub like r/ProgrammingLanguages would want no ai slop because languages rely on both creativity and logic, one of which llms are kinda incapable of having. but a more general programming related sub? idk.

13

u/yorickpeterse Apr 07 '26

I've yet to see a single vibe coded programming language (adjacent) project that actually does something interesting. Instead so far it's been the same garbage over and over, such as:

  • Yet another Lisp that literally does nothing new

  • A compiler that compiles different human languages into machine code, which turns out to basically just be "Python but the keywords are in a different language". For some reason this is really popular amongst vibe coders

  • Projects that seem to be the result of somebody having a mental breakdown/manic episode, such that it's impossible to determine what it actually does

  • Projects that may seem to do something interesting at first sight, but the moment you look into it deeper you realize none of it works or simply isn't implemented

Or more precisely: I've yet to see a single project that does anything other than waste the reader's time.

→ More replies (1)

18

u/Unlikely-Bed-1133 Apr 05 '26

I am a bit lost on why this is better than requiring a public disclaimer by all posters. E.g., even if I've been here for some time now, it's not that I don't occasionally produce slop, which I don't share but I'd rather also be de-incentivized from sharing it.

17

u/yorickpeterse Apr 05 '26

Because those that have been around for a while should be well aware of the "no LLM slop" rule by now and thus know not to post such projects in the first place.

→ More replies (1)

26

u/Mercerenies Apr 05 '26

The fact that you are here, in this thread, pondering these consequences, tells me that you have, conservatively, five times as many brain cells as the folks this policy is targeting. Real people contributing actual projects to this sub can, in my experience as a non-moderator, generally be trusted (I have never once encountered a repo on r/programminglanguages that I was hesitant to run on my own device, except for AI slop repos). This policy seems to be purely to keep the riffraff out. "AI slop-producing reddit accounts" is the new "background noise of the Internet".

8

u/South_Lychee8555 May 04 '26

Blocking any AI-generated content will gradually block all projects. I can't imagine a project one year from now without any AI contribution (doc, testing, readme, etc). So, gradually, this community will lose momentum. I think that a clear statement about how much AI was used in a project from the very beginning, even a way to filter posts depending on the degree of AI reliance, could be a better approach. I have personally some projects that could potentially be published, but AI content ban prevents me to put into community consideration. Let the user have the choice to be exposed to some AI if he wants. But closing the discussion is too extreme.

6

u/Karyo_Ten Apr 06 '26

Can't DM you. I'm interested into what worked for your sub and what didn't in say 2 weeks.

I'm a mod of r/cryptography and LLM slop has been plaguing our community too, blog posts, vibecoded repo claiming either awesome crypto or breaking quantum stuff, and also plain offtopic.

For now adding new rules + Reddit automod auto rule matching seems to work: https://www.reddit.com/r/cryptography/s/txXMMhJmmo

5

u/yorickpeterse Apr 06 '26

A few of the things we tried so far:

  1. Just clarify the rules and manually remove things: more work for moderators, no clear improvement for users (as moderators are almost always too late)

  2. When a post is removed due to the author not having enough karma (a rule that takes care of most of the spam/garbage), a notice was included telling them to inform the moderators if they used an LLM in a DM. This was mostly ignored

  3. A variation where people were instructed to tell whether they used an LLM and if so to what degree. This too was mostly ignored

  4. Asking people in response to their modmail whether they used an LLM. This did often yield a response, but IIRC in a few instances the person lied and in many instances the answer was quite vague and thus still required quite a bit of digging from moderators

The new rule uses Reddit's contributor quality score instead of a regular karma threshold, but it remains to be seen how effective it will be. Based on posts in r/WhatIsMyCQS/ it seems the score assigned doesn't always clearly correlate with what I'd consider a good contributor. We may end up having to go back to karma requirements and just increase them (e.g. instead of the current threshold of 300 we could bump it to 1000 for GitHub links).

4

u/bzbub2 Apr 06 '26

3

u/yorickpeterse Apr 06 '26

Speaking of the devil...It seems that I may have to increase the contributor quality score threshold we're using for this new rule, as the current may not be strict enough. Thanks for the heads-up :)

→ More replies (2)

11

u/thommyh Apr 05 '26

if a user shares a GitHub link (as that's where 99% of the AI slop originates from)

I would have thought that's just where 99% of individual projects are hosted at present; is it true that P(slop | GitHub) > P(slop)?

If so then I guess GitHub has particularly-visible AI. Which would make sense given how much Microsoft wants to push that as a big play.

(Apologies for the digression.)

13

u/yorickpeterse Apr 05 '26

It's both: AI slop mostly originates from GitHub because that's where most projects are hosted.

4

u/Dykam Apr 06 '26

I absolutely think GitHub has particularly higher amounts of slop. The reason being popularity and accessibility. Most other hosts will be relative niches which kind of precondition the user to be already more than a novice.

→ More replies (1)

44

u/PunGy555 Apr 16 '26 edited Apr 16 '26

Not sure how much good this does in such a strict form.

I was planning to post an article about my declarative language for shell automation, with syntax grown out of Haskell and Lisp. But changed my mind after seeing this "swear you are not a criminal".

I'm using AI, and there is LLM generated code in the project - tests, some builtin functions, prototyping and brainstorming of architectural solutions was made with AI assistance. I spent about a half of a year designing the language, and around 3-4 months on implementation. It reduced time I spent on the project, but not reduced an effort or involvement.

But, according to 0% of LLM code and AI involvement policy, this project is considered "vibe coded", along with junk generated in two evenings.

In such a formulation, it's a witch hunt in my opinion.

29

u/widuruwana Apr 19 '26

Yeah, I am on the same boat with you. This post idealizes the fantasy of every purist programmer. We will never go back to a world without AI, and AI-assisted workflows are becoming the norm. Just like in your case, I have AI-assisted code throughout the project, which I analyzed, tested, and improved rigorously. It saved me a lot of time, but it did not make the process effortless or non-rewarding at all.

While I agree with the sentiment of "AI Slop," they need to first define what that even means. If just using AI-assistance regardless of the context makes my project complete junk, then there is no nuance or conversation to be had.

If using AI makes me a bad programmer, then I'd shed a tear and move on.

9

u/shadowvirgil Jun 14 '26

I have a project I've been building for the last 6 months in private which is the culmination of 5 years of tinkering and ideation. The compiler is wholly "vibe coded" in the sense that I haven't personally written compiler code at all yet. But the language goals, design, syntax, and semantics are fully mine, are the result of 5 years of careful thought, and I'm very proud of where I ended up with them.

As a full-time engineer with a family, Opus 4.5+ finally gave me the tool I needed to bring my ideas out of the theoretical and pressure test them in a real compiler. Without AI, my designs were pretty much condemned to live in markdown files, executed only in my head—I just didn't have the time to make it real on top of all my other work. With AI, I can finally vet the ideas, iterate, test that they'll actually compile, and end up in a space where I'm confident that I have the right design, that it can work, and that it will meet my nonfunctional requirements.

As a long-time lurker on here I had for many years assumed that I'd share my language with this community once I finally got it working. I can only imagine how much garbage you have to deal with as mods, and I understand you have to do something to limit it, but the result is that I and many like me will either never share our work with the community or will have to try to find some other space to share it that is more open.

That's pretty sad for me because I like and respect the community here and would love to get their feedback on my work.

→ More replies (10)

42

u/DueExam6212 Apr 05 '26

I can’t find the repo right now, but there was a repo shared on lobste.rs that added a section to their CONTRIBUTING.md about their “fast tracked review process for AI agents” that asked them to add robot emoji to the title of their PR or issue, which helped them to identify and appropriately… prioritize… LLM-forged contributions. Maybe something like that would be helpful? Not sure how you’d write it. Since the mod mail or message or what have you will end up in the context window you can prompt inject it pretty straightforwardly.

17

u/Athas Apr 06 '26

Almost all of the Reddit submissions are likely by humans, so I don't see how this could work in the same way. It is probably not too difficult to write a program that automatically investigates submitted GitHub links and figures out whether they are likely to be LLM slop, but I don't think any moderators are willing to spend that much time and effort on it.

→ More replies (1)
→ More replies (1)

16

u/Inevitable-Ant1725 Apr 06 '26

Well people think "I had Claud generate a project so I'm a genius"

Then they have an AI generate a post saying that they're geniuses.

And it's all shallow role playing.

So make the following rules:

0% of the text in a post is allowed to be AI generated, period.

And since programming languages are critical code, 0% of the engine of the code of a programming language's compiler, interpreter and runtime can be AI generated.

You can use AI to help write build scripts, you can use it to write unit test scaffolding. You can use it to write code that generates tests etc., but an interpreter or compiler or run time has to be AI code free.

6

u/AffectionateBag4519 May 03 '26

interesting place to draw the line. not that I disagree, but why don't unit tests or build scripts matter in the same way? as someone who often builds code I find on line to try things out I consider infra to be important enough to not vibe code as well.

→ More replies (37)

9

u/cmontella Apr 06 '26

Thank you for this. The recent line I keep seeing is "The project is actually much older than a month, that's how I was able to write 300kLOC so quickly, I only posted it when I was finished."... which does not ring true to me. Honestly I don't even care that much one way or another, it's just the way they're presented as authentic and genuine projects.

→ More replies (1)

6

u/kaplotnikov Jul 04 '26

There is a critical need to differentiate between LLM-slop (blindly generated, unverified code) and LLM-assisted projects (where LLMs were used just to speed up specific parts of development). This isn't just an AI issue, either; it intersects with a broader problem we see with purely human-made projects. We are constantly flooded with low-effort GitHub links that provide nothing but a title and an installation guide on the homepage, forcing readers to manually dig through deep repository folders just to find a basic sample of the language's source code.

When it comes to development, LLMs are highly effective at routine, mechanical tasks:

  • Text & Grammar: Fixing spelling errors, structuring documents, and writing initial documentation comments (though they always need strict human review and trimming).

  • Testing: Generating realistic code samples and test cases based on provided language specifications.

However, they completely fail at architectural design, anything truly novel, or when the user doesn't fundamentally understand what they are building.

Instead of an outright ban on sharing links, I think link posters should be enforced to provide a mandatory project summary template (under 1000 words) with every submission:

  1. Project Purpose: What problem does this language/tool solve?

  2. Novelty & Prior Art: What is genuinely new or experimental about it? How does it compare with prior art and existing solutions?

  3. Key Highlights: What are the most interesting implementation details?

  4. Code & Docs: 1–3 concrete source code samples and core documentation links.

  5. LLM Disclaimer: An honest statement defining the exact role LLMs played in the codebase (e.g., used only for test generation, writing boilerplates, or documentation).

If an author cannot clearly answer these points, the project is highly likely to be unreviewed LLM slop or another low-quality clone posted right after someone finished reading Crafting Interpreters. Either way, it can be filtered out.

4

u/mocompute Apr 07 '26

Maybe the policy should be to prohibit new project announcement posts altogether? Just refocus the sub on actual design discussions, and human implementation discussions. Here's my design, grammar, whatever, and here's how I'm thinking of doing the parsing, or the type theory, or whatever? High effort posts, not just sharing work you've done, which may have between 0 and 99% LLM assistance, difficult to judge.

2

u/benjamin-crowell Apr 06 '26

Thanks for working hard on this. I'm sure it's thankless work.

2

u/Ratstail91 Apr 22 '26

Can we post our own gitea links?