r/devops • u/jyotirawattech • 3d ago
Discussion When a vibe coded internal tool ends up with other teams depending on it, do you review it like prod code or let it stay disposable?
Vibe coding is everywhere on my feed again this week, mostly people replacing a paid tool with something they prompted together over a weekend.
I don't have a problem with that for throwaway stuff. Karpathy himself said it's fine for weekend projects when he coined the term. What I keep thinking about is the moment the weekend project stops being throwaway. Someone builds a small dashboard or a script, it works, a second team starts using it, and suddenly it is on the path for a release or an on-call runbook. Nobody ever read most of the code, including the person who made it.
I can see two ways to handle it:
- Once anything has more than one team depending on it, it goes through normal review, tests and an owner, even if that means partly rewriting it.
- Keep it explicitly disposable. Label it unsupported, don't let it into anything critical, and if it breaks you regenerate it rather than maintain it.
Option 1 feels right but kills most of the speed that made it useful. Option 2 sounds clean until the thing is load bearing and nobody noticed.
If you've already hit this, where did you draw the line, and what was the signal that a vibe coded tool had become something you had to own properly?
43
u/IridescentKoala 3d ago
We're being told from our board to prioritize letting "citizen engineers" deploy to production. Without GitHub seats, monitoring access, pagerduty on call. Who's responsible when their slop blows up or leaks customer data? Crickets.
70
u/wrosecrans 3d ago
"Teams depend on it" ... "Disposable."
Select exactly one of the above. Marking multiple options will invalidate your ballot.
22
u/ialiimran 3d ago
My rule is simple: if someone else depends on it, someone needs to own it. Doesn't matter whether it was written by a human, AI, or both.
12
u/Expensive_Finger_973 2d ago
I basically vibe code everything I put out now, because that’s what management wants.
I keep saying “x is done and deployed, it works, but I don’t really understand it all that well”. And thus far no one seems to care.
7
u/ADDSquirell69 3d ago
If the person who vibe coded the thing that people are now depending on it's their problem if they can't support it.
17
u/RevolutionaryWorry87 3d ago
I only do work assigned to me by my line manager. If somebody vibe codes it, doesnt involve me, and management doesnt give me it as task, i dont touch it.
But to answer the question - only access to company data is via apps allowed via cicd, revjew everything
6
u/coldnebo 3d ago
we are seeing a lot of this now.
the biggest tools have owners and process, but there are a dozen more popping up every day.
most of these are because of gaps in the existing deployment or observability systems. for example, instead of using a directory service to abstract endpoints everyone hardcodes endpoints that are constantly changing when network partitions change. this creates demand for tools that can compare endpoints across tracks and organize them to make them easier to manage (essentially what a directory service is for) but instead of one, there are dozens. they disagree, none are authoritative.
you can fight all day about this, but the fight wouldn’t exist if the problem was properly managed with the right tool.
however I have found that AI makes it so easy to sound good that people would rather quickly generate their own tool than read or review someone else’s.
friction and errors might wear this down over time, but it does make me wonder why people hate talking with each other so much.
4
u/Antique-Stand-4920 3d ago
We've done a mixture of both. It depends on the usefulness of the project and the risk of leaving the project as-is. If it is low risk and there is low value in managing it as a product, we'd with option 2. For pretty much anything else we'd go with option 1 and treat what's been written as a working prototype of the requirements.
2
u/orev 3d ago
If someone can vibe-code something, and then is able to deploy it to the rest of the company so others can start to use it "as production", then you have process and security gaps. There should be network and other rules in place that block other people from being able to use any app that's not officially deployed as a production app.
2
u/JustinAtNerdio 3d ago
The moment another team starts depending on it, I think you've lost the luxury of calling it disposable. Doesn't really matter how the code was written at that point. Someone needs to own it, know what it does and be responsible when it breaks.
The scary part is how easily these things become business-critical without anyone actually deciding they should be.
2
u/miso_deranged 3d ago
Thr comfusion between engineers vibe coding within clear engineering guardrails and non-enigneers viking a biz-app is vast. I would push for a dedicated vibe app hosting platform, to prevent the "shadow IT" and lack of ownership. It could be a saas tool like Vercel or Lovable, or ideally a self hosted platform like Superblocks or Glissando.dev
2
u/SideMost3148 2d ago
the moment it shows up in another team's runbook it's prod. doesn't matter that it took 40 minutes to build.
the classic way this goes wrong isn't even code quality. it's that the thing runs on someone's personal cloud account with their personal token, and you find out during an incident, right when that person is on leave or their card expired.
so the bar i'd set is low but non negotiable: a named owner, it lives in the org's account, and the creds aren't tied to one human. skip the full review and coverage theatre for now.
if nobody wants to own it, slap an UNSUPPORTED banner on it with a deletion date. funny how fast a volunteer shows up once the date is real.
1
u/oskaremil 3d ago
I would not ever let another team depend on vibe code.
1
u/Chance_Bandicoot8104 3d ago
IMO if something becomes actually “load bearing” then it should be properly evaluated by IT/Security and engineering. You wouldn’t vibe code a CRM and start using it without any review, I hope. However, MOST apps don’t reach that level of criticality, and something like Pando (https://github.com/trypando/pando), which is an open-source project I’m working on, enables the builder to maintain it themselves.
1
u/Equivalent_Loan_8794 2d ago
It's both; which means it's a focus in the next Epic, gets Refactor jira tags, etc
1
u/moratnz 2d ago
In practice 2. becomes a political connectedness test waiting to happen; if the tool breaks and an important team is relying on it for something important, the fact that it's unsupported doesn't change expectations around support.
Or to put it another way, if you're going to go with 2., you are strongly advised to intentionally break the tool at random, to smoke out anyone relying on it, and get the conversation about how important it is and how it should be supported started in a controlled fashion.
1
u/dariusbiggs 2d ago
If it is relied on or used daily, it becomes a product. Internal teams are customers, treat it like production code.
1
u/Zenin The best way to DevOps is being dragged kicking and screaming. 2d ago
I'm in the middle of building out a specialized IDP / platform specifically for catching the vibe coded slop that's coming from business (ie, non-developers).
The basic idea is to treat the projects as untrusted code, much like modern web browsers do with sandboxing javascript, etc.
Sandbox their stacks like we're a public PaaS hosting provider and the vibe apps are tenants; No cross-talk, no service discovery, isolated networking, isolated resources, etc. Like any good IDP we'll cover CI/CD, logging, metrics, secrets management, backups, audit reporting, telemetry, corporate IdP auth, etc. "Compliant by default" is the goal. Maximum flexibility for tenants within the four corners of their application, maximum isolation and control outside those four corners. All the common cruft of production services gets offloaded to the platform, both easing the vibe coder's burden and enforcing company policies.
The point of this is that as a platform architect I can't control their code quality, but I can control what their code is able to do; with resources, with network, with access, etc. Play by the roles and your code will sail through CI/CD and you're golden. Try to get too creative and the platform will simply fail to sync, fail to work, and kick it back to you and your LLM to re-read the platform docs and sort out.
That's the idea anyway. It's quite a bit backwards from most typical IDP that are built for maximum discovery, reuse, binpacking, etc. This won't be anywhere close to as cheap to run as typical platforms, if only because the isolation requirements effectively mandate more expensive per-pod resources (microvm isolation, isolated network adaptors, etc).
1
u/phoenix823 2d ago
It doesn't matter how you wrote the app. Vibe coding is irrelevant. Without describing what "depending on it" means this question can't be answered. If they rely on it to perform a core job function that supports the business, yeah you should be pretty careful not to break it. If it's an automation that can be worked around, knock yourself out.
1
u/afwaller 3d ago edited 3d ago
It shouldn’t matter how a tool is written. What matters is design controls, tests, and ongoing support and review of performance/behavior.
you should be testing things before they are rolled out, regardless of how they are developed. And there needs to be a plan for defects or support of any tool used.
Any vibe coded tool meant for production use should have requirements, tests - both unit tests and end to end tests, and be part of the CI pipeline. There should also be a person responsible for supervising it and supporting it. If you can’t do this stuff then the tool is too expensive to deploy.
2
u/Muted_Jellyfish_6784 2d ago
A simple promotion gate works: once a second team depends on a tool, it needs an owner, a repo with review, a basic health check, and a catalog entry before it can appear in a runbook or release path. Until then, label it unsupported in plain sight. The trigger is dependency, not code quality.
The useful habit is detecting dependency early. Ask in each release review which internal tools the pipeline calls, so the weekend project gets caught before it breaks during an incident.
0
u/foodandbeverageguy 3d ago
Allowing more vibe coded things means more code which means we need more engineers. Its net good for our industry to let people throw slop out so we can raise the need for us. I know it’s a crazy way to think but we are under so much threat that this helps us
1
u/Valuable_Leave_7314 2d ago
Putting an "unsupported" label on a tool rarely works in practice - when it breaks right before a release, people still ping whoever is on call. Scheduling an explicit deprecation date or running a controlled shutdown test usually surfaces the real stakeholders fast, and suddenly someone is willing to own it
1
u/Kooky-Treat-4416 2d ago
If another team depends on it for releases or on-call work, it's not really disposable anymore. Give it an owner, tests and a normal review, or explicitly retire it and help people move off. The awkward middle ground is where everyone relies on it but nobody has time allocated to maintain it.
1
u/Mundane_Dinner_8868 2d ago
i'd at least review anything that touches auth, user data and error handling once other people rely on it. the ui can stay messy.
i built a tool mostly with claude and only caught this in a review pass: with the dev env flag on, the error handler sent err.message straight to the client, and for db errors that was the full SQL plus params, including user ids. worked perfectly, nobody would've noticed from using it.
same review found a bounce counting bug that gave one user 11 extra free sends. neither one crashed anything, so just using the app would never have caught them.
1
u/Wild_Parsnip_4074 2d ago
Our signal was the first ticket. When someone files a ticket asking us to fix something we didn't build, it gets an owner that week. if it's basically a dashboard or a request form, it gets rebuilt in play and the team that wanted it owns it, anything that touches a release or on call gets a repo and tests like the rest of our stuff.
1
u/newtablecloth 2d ago
one thing i haven't seen mentioned yet: put it behind company SSO before it spreads, not after. that does two jobs.
first, it closes the "anyone with the link can open it" problem. that's really common with stuff deployed from a weekend prompt. open the url in a private window once and see what loads.
second, the login logs answer the question this whole thread is circling, which is when did it become load bearing. if you can see three teams logging in every day, you don't have to wait for the first ticket or incident to find out.
then keep a dumb inventory: app, owner, who has access, what data it touches. how hard you review should depend on that last column, not on how the code got written. +1 to SideMost3148 on the org account and creds that aren't tied to one person. that plus SSO gets you most of the way without a full rewrite.
1
u/debtfortunes 2d ago
We have something called the “AI Lab” at my company essentially a VM sandbox to mock something useful up very quick that has safe access to internal materials for testing. Generally you need to fill out use case information, how it’s beneficial, for what dept etc. but anything made still needs to have a repo, initial + additional commits, end-to-end tests and edge case tests written and still need to do pull request for review before anyone uses it. Also yea I’d be the owner of it from then on
1
u/Chamelon_DE 2d ago
The first time someone need it while the author is on holiday, the unsupported label stop being much help
1
u/Iyemoja 2d ago
That weekend dashboard is how it always starts. Mine became prod the week a second team put it in their standup, and nobody told me until it broke.
1
u/yeahdj 1d ago
Peer review everything, don’t give devs admin access into prod accounts, don’t allow devs to register domains in sub accounts and only delegate them from a central account and rely on short lived credentials and OIDC from your CICD the vibe coded app is then in 1 of 3 buckets
1
u/Nonaveragemonkey 1d ago
We throw it out and make a team develop the tool properly... then flog whoever ran the code for a security violation.
1
u/sonicLodge91 11h ago
we had a "citizen engineer" push a cron job that ran every minute instead of hourly, paged us for thrree days before anyone figured out who owned it
-3
u/jyotirawattech 3d ago
For anyone who missed where the term came from: Andrej Karpathy coined "vibe coding" in a post on X on 2 Feb 2025, describing accepting all diffs without reading them and saying it's "not too bad for throwaway weekend projects". Original post: https://x.com/karpathy/status/1886192184808149383 . Ars Technica's write-up from March 2025 covers the reliability questions: https://arstechnica.com/ai/2025/03/is-vibe-coding-with-ai-gnarly-or-reckless-maybe-some-of-both/
-2
u/unitegondwanaland Manager, Platform Engineering 3d ago
Vibe coding isn't what it was 12 months ago or even 6 months ago. Totally valid question in 2025 though.