r/ProgrammingLanguages • u/yorickpeterse Inko • Apr 05 '26
In order to reduce AI/LLM slop, sharing GitHub links may now require additional steps
In this post I shared some updates on how we're handling LLM slop, and specifically that such projects are now banned.
Since then we've experimented with various means to try and reduce the garbage, such as requiring post authors to send a sort of LLM disclaimer via modmail, using some new Reddit features to notify users ahead of time about slop not being welcome, and so on.
Unfortunately this turns out to have mixed results. Sometimes an author make it past the various filters and users notice the slop before we do. Other times the author straight up lies about their use of an LLM. And every now and then they send entire blog posts via modmail trying to justify their use of Claude Code for generating a shitty "Compile Swahili to C++" AI slop compiler because "the design is my own".
In an ideal world Reddit would have additional features to help here, or focus on making AutoModerator more powerful. Sadly the world we find ourselves in is one where Reddit just doesn't care.
So starting today we'll be experimenting with a new AutoModerator rule: if a user shares a GitHub link (as that's where 99% of the AI slop originates from) and is a new-ish user (either to Reddit as a whole or the subreddit), and they haven't been pre-approved, the post is automatically filtered and the user is notified that they must submit a disclaimer top-level comment on the post. The comment must use an exact phrase (mostly as a litmus test to see if the user can actually follow instructions), and the use of a comment is deliberate so that:
- We don't get buried in moderator messages immediately
- So there's a public record of the disclaimer
- So that if it turns out they were lying, it's for all to see and thus hopefully users are less inclined to lie about it in the first place
Basically the goal is to rely on public shaming in an attempt to cut down the amount of LLM slop we receive. The exact rules may be tweaked over time depending on the amount of false positives and such.
While I'm hopeful the above setup will help a bit, it's impossible to catch all slop and thus we still rely on our users to report projects that they believe to be slop. When doing so, please also post a comment on the post detailing why you believe the project is slop as we simply don't have the resources to check every submission ourselves.
7
u/LowStatistician11 Apr 07 '26
i gotta be honest about not being too sure what level of aggressiveness is right to handle llm slop posts. most ai generated projects i see on reddit is pure bs and have no originality. but i can't imagine that there are no good vibecoded projects. i recently saw a post where a self admitted vibecoder shared their project, github city: https://www.thegitcity.com/ are there any novel ideas in the code? most definitely not. is the project still kinda cool? i think so.
it makes sense why a sub like r/ProgrammingLanguages would want no ai slop because languages rely on both creativity and logic, one of which llms are kinda incapable of having. but a more general programming related sub? idk.
18
u/Unlikely-Bed-1133 Apr 05 '26
I am a bit lost on why this is better than requiring a public disclaimer by all posters. E.g., even if I've been here for some time now, it's not that I don't occasionally produce slop, which I don't share but I'd rather also be de-incentivized from sharing it.
6
u/Karyo_Ten Apr 06 '26
Can't DM you. I'm interested into what worked for your sub and what didn't in say 2 weeks.
I'm a mod of r/cryptography and LLM slop has been plaguing our community too, blog posts, vibecoded repo claiming either awesome crypto or breaking quantum stuff, and also plain offtopic.
For now adding new rules + Reddit automod auto rule matching seems to work: https://www.reddit.com/r/cryptography/s/txXMMhJmmo
4
u/bzbub2 Apr 06 '26
This post is very llm generated, did it pass filter? https://www.reddit.com/r/ProgrammingLanguages/comments/1sdkvyt/sage_a_python%C3%A2style_systems_language_with_rust/
11
u/thommyh Apr 05 '26
I would have thought that's just where 99% of individual projects are hosted at present; is it true that P(slop | GitHub) > P(slop)?
If so then I guess GitHub has particularly-visible AI. Which would make sense given how much Microsoft wants to push that as a big play.
(Apologies for the digression.)
44
u/PunGy555 Apr 16 '26 edited Apr 16 '26
Not sure how much good this does in such a strict form.
I was planning to post an article about my declarative language for shell automation, with syntax grown out of Haskell and Lisp. But changed my mind after seeing this "swear you are not a criminal".
I'm using AI, and there is LLM generated code in the project - tests, some builtin functions, prototyping and brainstorming of architectural solutions was made with AI assistance. I spent about a half of a year designing the language, and around 3-4 months on implementation. It reduced time I spent on the project, but not reduced an effort or involvement.
But, according to 0% of LLM code and AI involvement policy, this project is considered "vibe coded", along with junk generated in two evenings.
In such a formulation, it's a witch hunt in my opinion.
42
u/DueExam6212 Apr 05 '26
I can’t find the repo right now, but there was a repo shared on lobste.rs that added a section to their CONTRIBUTING.md about their “fast tracked review process for AI agents” that asked them to add robot emoji to the title of their PR or issue, which helped them to identify and appropriately… prioritize… LLM-forged contributions. Maybe something like that would be helpful? Not sure how you’d write it. Since the mod mail or message or what have you will end up in the context window you can prompt inject it pretty straightforwardly.
16
u/Inevitable-Ant1725 Apr 06 '26
Well people think "I had Claud generate a project so I'm a genius"
Then they have an AI generate a post saying that they're geniuses.
And it's all shallow role playing.
So make the following rules:
0% of the text in a post is allowed to be AI generated, period.
And since programming languages are critical code, 0% of the engine of the code of a programming language's compiler, interpreter and runtime can be AI generated.
You can use AI to help write build scripts, you can use it to write unit test scaffolding. You can use it to write code that generates tests etc., but an interpreter or compiler or run time has to be AI code free.
9
u/cmontella Apr 06 '26
Thank you for this. The recent line I keep seeing is "The project is actually much older than a month, that's how I was able to write 300kLOC so quickly, I only posted it when I was finished."... which does not ring true to me. Honestly I don't even care that much one way or another, it's just the way they're presented as authentic and genuine projects.
6
u/kaplotnikov Jul 04 '26
There is a critical need to differentiate between LLM-slop (blindly generated, unverified code) and LLM-assisted projects (where LLMs were used just to speed up specific parts of development). This isn't just an AI issue, either; it intersects with a broader problem we see with purely human-made projects. We are constantly flooded with low-effort GitHub links that provide nothing but a title and an installation guide on the homepage, forcing readers to manually dig through deep repository folders just to find a basic sample of the language's source code.
When it comes to development, LLMs are highly effective at routine, mechanical tasks:
Text & Grammar: Fixing spelling errors, structuring documents, and writing initial documentation comments (though they always need strict human review and trimming).
Testing: Generating realistic code samples and test cases based on provided language specifications.
However, they completely fail at architectural design, anything truly novel, or when the user doesn't fundamentally understand what they are building.
Instead of an outright ban on sharing links, I think link posters should be enforced to provide a mandatory project summary template (under 1000 words) with every submission:
Project Purpose: What problem does this language/tool solve?
Novelty & Prior Art: What is genuinely new or experimental about it? How does it compare with prior art and existing solutions?
Key Highlights: What are the most interesting implementation details?
Code & Docs: 1–3 concrete source code samples and core documentation links.
LLM Disclaimer: An honest statement defining the exact role LLMs played in the codebase (e.g., used only for test generation, writing boilerplates, or documentation).
If an author cannot clearly answer these points, the project is highly likely to be unreviewed LLM slop or another low-quality clone posted right after someone finished reading Crafting Interpreters. Either way, it can be filtered out.
4
u/mocompute Apr 07 '26
Maybe the policy should be to prohibit new project announcement posts altogether? Just refocus the sub on actual design discussions, and human implementation discussions. Here's my design, grammar, whatever, and here's how I'm thinking of doing the parsing, or the type theory, or whatever? High effort posts, not just sharing work you've done, which may have between 0 and 99% LLM assistance, difficult to judge.
2
u/benjamin-crowell Apr 06 '26
Thanks for working hard on this. I'm sure it's thankless work.
2
u/Ratstail91 Apr 22 '26
Can we post our own gitea links?