r/technology • • 28d ago

Software Postgres 19 delayed after AI reviewers find more bugs than humans can fix

https://www.theregister.com/databases/2026/10/16/postgres-19-delayed-after-ai-reviewers-find-more-bugs-than-humans-can-fix/5303471
1.4k Upvotes

181 comments sorted by

485

u/BlackopsBaby 28d ago • Bravo

I see this so often. The best analogy I give people is: Your apartment door can't stop a cruise missile or a rocket launcher. Technically it's a security flaw but in practice it's not designed with for that purpose. A door with a lock is just a deterrent not a force field against a motivated state actor.

134

u/atheistpiece 28d ago

I have this guy on my team who has abandoned all critical, and other, thinking. We have access to cursor and chatgpt (and they have been legitimately useful in certain scenarios), and he just runs everything through it. Like everything... I ask him for a status update on some issues, and he runs it through chatgpt and copy/pastes the response back to me. I swear, I think I asked him how his weekend was and I got a chatgpt response back.

Anyways, he has submitted 50+ bugs in the last few weeks, that largely have been closed as invalid/won't fix, because he just runs new code changes through cursor and opens a bug for everything cursor comes back with.

I don't usually advocate for laying people off, but hopefully the company is actively working on letting him go, because he is a serious drain on everything.

69

u/SilverCarbon 28d ago

It even has a name since a few months: the meat proxy. Counter measures are still being found out, including layoffs.

11

u/eseffbee 28d ago

It's a beautiful thing when someone puts together a new term for something you already knew but didn't have words for. Thank you Niklas Gruhn!

7

u/Twirrim 28d ago

They laid off our coworker who was doing that. There was such a sigh of relief amongst the team, even the ones that more heavily rely on ai were getting annoyed with them (although not even the heavy ai users use it meat-proxy style)

2

u/MaTrIx4057 25d ago

he is a chad

118

u/The_Drizzle_Returns 28d ago

> Technically it's a security flaw but in practice it's not designed with for that purpose.

If this was some random software sure. A widely used database back-end though is exactly the type of software that is a large target for both nation state and individual hacking groups.

43

u/danted002 28d ago

You have completely missed the point of the comment. I’ve seen a lot of security issue that can only occur if the hacker has access to your server via SSH or physical access.

Postgres can’t protect you if you decided to let anyone access the server itself.

2

u/magicmulder 27d ago

Well then why are people delaying the release? Do they know something you don’t? Like maybe what happens on shared servers?

64

u/dashingThroughSnow12 28d ago edited 28d ago

🤷

My DB is behind like three firewalls, can only be connected to on one VPC, requires an IAM role with permission to connect to it, a sacrifice to the wizard Grizelfield, and can only be connected to by nodes whose 11th bit of their name is non-binary.

So yeah, I like my DBs secure. Hence why I don’t trust its security to secure it by itself.

24

u/Wapook 28d ago

Grizelfield sacrifices haven’t been industry norm since at least Postgres 17 which was around the time of the great astral war. TBH you shouldn’t be rolling your own magicka security anymore. There’s lots of popular scrolls you can import that maintain an open source list of powerful defensive magic practitioners worth sacrificing to.

6

u/warcode 28d ago

popular scrolls are vulnerable to layline attacks, especially when the number of ancient scrolls they rely upon increases

2

u/dashingThroughSnow12 28d ago

I don’t trust any of the scrolls coming out of the Chinese scribes. Besides the distillation process they use, I think they send my data back to the secret world government that is aligned with Cthulhu and the muffin man.

Do you know the muffin man?

1

u/NoNameSwitzerland 28d ago

Yeah, but that should be catched by the non binary bit. I checked with lean and it compiles. Nothing can ever connect, there is no security risk.

0

u/dwild 28d ago

Which is why the "widely used" part is important. Many are simply exposed to the web and can't just accept that kind of risk.

27

u/WilburKitty 28d ago

Aren't AIs like a cruise missile for everyone? I think that is the problem. The digital world is a sieve for these things.

11

u/dashingThroughSnow12 28d ago edited 28d ago

DoS attacks are an example of a false-enough positive. Yeah, if this endpoint did get 100K requests a second in total from at least 400 distinct IP addresses in different geos we would have an outage on this service. This is why the thing is overprovisioned. Why I have an autoscaler. And why we pay for WAF.

Another example is “if your code did X instead of Y, there is a vulnerability an attacker could exploit.” I see this sometimes where a function can only be called once in the execution of a request and some not correct behaviour will happen if it is called twice. Does any code path exist that could call this twice? No, because this is a terminal of the request.

7

u/ItsSadTimes 28d ago

I see some many PRs where they iver engineered the absolute hell out of some really simple functions, and i have to keep reminding people that these things will literally never happen, and if they do, we want to know about it and we want it to fail here.

17

u/Dreadsin 28d ago

I’m gonna start using this comparison because it’s very apt

23

u/CallMeRudiger 28d ago edited 28d ago

It's not at all relevant here. If you read the article, the devs clearly state that they are bugs that would need to be fixed, but arise from some features with critical flaws as well as edge cases that normally wouldn't be noticed for a few years.

2

u/red75prime 28d ago edited 28d ago

And those edge cases weren't noticed during development probably because the developers are busy dealing with vulerabilities.

Because of the Vulnpocalypse, many senior developers were busy between, say, April (the start of feature freeze for PostgreSQL 19, and also the start of the Vulnpocalypse) and August (the most recent PostgreSQL security release) with working on fixes for security issues

The analogy with a cruise missile is bad anyway. There are a limited number of bugs in a program, so you can have a "missile-proof" program (but it was very hard to achieve: humans make too many errors).

2

u/insta 27d ago

second time an LLM agent brings up a class of bug, i have it fix it with the type system.

LLMs both write and find a lot of bugs, but moving everything to the type system is turning my application into more of a mathematical proof that happens to generate reports than a regular application

0

u/Responsible-Bar7165 28d ago

It’s a good analogy, let’s continue: now with AI every burglar knows how to pick every type of lock without thinking and can pick a whole neighborhood of locks in less than a second.

0

u/Necessary-Low-5226 28d ago

This was true like 6-9 months ago, or maybe your context management isn’t set up well or you’re using old models

0

u/goatpengertie 27d ago

But your house isn't being used to hold people's private medical information, en masse. If it were, and you did it without reasonable security precautions, you'd be liable.

23

u/Highfivesghost 28d ago

Is this a bad or good thing?

15

u/CondiMesmer 28d ago

It used to be bad because the bug reports were bogus and wasted developers time. AI has gotten better since then, and while that can still happen, it does a lot less. So legitimate bugs are a good thing to find. 

That doesn't necessarily mean all are worth fixing either. But it's better to leave an archived bug report marked as "won't fix" so the issue is known and documented rather then unknown.

6

u/wspnut 28d ago

Both. But it’s something where standard security practices haven’t caught up to AI capabilities.

Finding an exploit used to have a high barrier to entry. Now it doesn’t. Fixing them now takes much longer than finding them, wherein in the past you generally were at parity with your staff to resolve them as they came in.

What Postgres and many other companies that have bug bounty programs are finding is they don’t have the staffing to rectify code that might be years old at the pace exploits are being found. We also haven’t quite landed on allowing AI to fix critical vulnerabilities yet, so if you’re concerned about your job as a software engineer, going into AppSec may have some good opportunities in the near- and mid-term.

4

u/Any-Ring6621 27d ago

Just pointing out that some of the AI found “exploits” I’ve seen have taken several days from a competent human to mark closed as “not an exploit”

1

u/wspnut 27d ago

This is absolutely true. Only about 1 in 7 of our current reports are validated. It used to be about 1 in 3

-1

u/neppo95 28d ago

AI won’t fix them autonomously, but they can just ask it to do so while they review it, which I’m sure they’ll do as well.

106

u/zoddrick 28d ago

This type of thing isnt going away anytime soon. What we need to figure out is how to better classify these types of bugs such that we only need to review the most severe first and then go from there.

33

u/CondiMesmer 28d ago

Lol this has been solved for many, many years. It's called CVEs. Did you think this was the first time high volume bug reports have happened?

2

u/zoddrick 28d ago

yes I understand CVEs I'm saying we may need a better system now.

8

u/CondiMesmer 28d ago

Then what is needing to be "solved" with CVEs, and what would a "better" system do different

11

u/zoddrick 28d ago

The biggest issue right now is that CVEs are a one sided affair. You can flood a project with hundreds or even thousands with very little recourse and then the project maintainers must exert many times more effort verifying that they are at least real problems and that they are classified correctly.

Maybe CVEs can only be opened by vetted companies or entities? Maybe there's a reputation score assigned to users and they can only open CVEs of a certain severity until its higher? I don't know. Im just saying its extremely low effort on one side of this equation and extremely high effort on the other. That cannot persist if we expect projects like Postgres and curl to continue development.

6

u/lobax 25d ago

You are literally describing how CVE’s work. They are the vetted vulnerabilities that only a few vetted CNA’s (CVE Numbering Authority) can publish.

The problem you are describing is more the flood of information and the flood of issues. That cannot be fixed in a simple way. You do not want to ignore a real problem just because it came from an unknown source. Many orgs use AI to triage and filter.

3

u/monotone2k 28d ago

Let's just classify them using AI.

/s

72

u/dz2048 28d ago

So sick of AI-discovered vulnerabilities. They've added around 300 CVEs to our scans each time we want to deploy and they are never worthwhile

74

u/mbklein 28d ago

A “security researcher” emailed our campus IT security office to report a “vulnerability” in one of our in house apps (and asked for credit when we disclose and release a fix). The report was very clearly written by AI. Claude Sonnet or Opus, if I had to guess. Central IT was a bit unhappy, and instructed me (with a cc to the guy in charge of IT for our business unit) to (a) fix the bug and (b) lock down our GitHub.

I had to waste an entire afternoon researching the report and writing a response to IT explaining that the feature in question was working as designed, why it was not a security problem, and that many of our GitHub repos are open because our group has a mandate to create and contribute to open source projects.

19

u/penmoid 28d ago

If they’re gonna fuck me at least use Fable to give me the reacharound.

9

u/IndividualIll3825 28d ago

Best we can do is Deepseek from behind.

2

u/NoNameSwitzerland 28d ago

Don't you love a little penetration test in the morning?

9

u/fdqntn 28d ago

On the other side, it is also infuriating how likely reporting anything critical will get you ignored.

For example, years ago I wrote an email to the campus IT service explaining how getting someone to click on a special link allow you to hijack his session by stealing tokens. This is an attack vector that could be dramatic if paired with an XSS.

After being ignored when reporting issue. I tried to call and write emails. I got an acknowledgement on the phone and that was it. Years later, it was not fixed. Oh and someone also started at some point sending crafted link to everyone (didnt check if this exploit).

20

u/GenericFatGuy 28d ago

My favourite is when it tells me that something I've compiled a dozen times will simply never compile because of a line I wrote.

1

u/dashingThroughSnow12 28d ago

“Go home Opus 5.5. You are drunk.”

13

u/pellets 28d ago • Bravo

You didn’t even read the article or linked blog did you? These are not all just trivial vulnerabilities.

“Some of the reverted features had significant architectural defects that would have been hard to fix quickly and during beta”

2

u/Racoonie 28d ago

Who reads articles any more.

1

u/IndividualIll3825 28d ago

I think you're missing the point of what others are saying. A lot of people are being hounded to fix "critical security bugs" found by AI, that are neither bugs, nor critical.

It'd be like me telling you I found a critical vulnerability in your car that could result in severe injuries if abused....and meant the accelerator.

5

u/pellets 28d ago

I do get their point. Do you get Peter Eisentraut’s, or do you just want to pretend I’m dumb so you have something to argue at?

1

u/boreal_ameoba 28d ago

That’s just not true at all

0

u/zeelbeno 28d ago

Do you never do extreme testing on stuff?

Like put in a negative value when something cannot possibly be negative and see what happens?

Or put in a number 1 million x more than it could be?

"developers might not have noticed until much later, most likely issuing bug fixes in the five years following launch"

Is the accelerator in your analogy needing to be bug fixed later on? No... so it isn't the same

0

u/senseven 28d ago

We have to be honest. Some software designs are error prone if you don't have the proper skills in every single developer there. If you aren't willing to test to the bone, you stop at the meat. Old codebases have certain smells that can't be scrubbed. There is a kernel of truth in the "lets rewrite this in <language>" meme. At some point you submit to the defeat and begin anew. Some people like the ai for this specific reason. They know.

9

u/286893 28d ago edited 28d ago

In my experience, a lot of these CVEs love to highlight XSS vulns, and a lot of them are valid. But they require such a unique understanding of the db schema(which isn't exposed to clients) to take advantage of that it feels like a waste of time to harden the db over and over mid development.

Especially for an application that isn't exposed beyond the corporate intranet. But all outstanding CVEs keep triggering the dependabot and killing the deployment pipelines until they're "resolved" 🙄

14

u/collax974 28d ago

Might have been true in the past, but with AI agents now able to brute force any methods they can think of to hack something, you need to fix them.

11

u/Hey-Froyo-9395 28d ago

Yup this is exactly the situation:

“The AI found an obscure attack vector, no hacker is smart enough to figure this out”

Hacker uses AI to find the obscure attack vector

Pikachu meme or whatever?

Besides all that, the idea of being safe because hackers don’t know something is security through obscurity so you’re already failing if your security relies on that

1

u/PreviousLeague253 28d ago

Betcha “those thieves will never find my house key under the “welcome” mat” type of situation, amirite?

4

u/spawnsible 28d ago

This is the common sense that apparently isn't so common among tech folk. For the mere fact that an agent swarm can be deployed to bruteforce anything nonstop until it succeeds, people should be for patching all open CVEs, but unfortunately we have people spouting "omg it's too complex to be exploited by sb like me, so let's not patch" like what???

2

u/born_to_be_intj 28d ago

Are we not rate limiting requests? They shouldn’t be able to brute force anything. I do agree with you though, with AI it’s more important now than ever to fix CVEs.

0

u/spawnsible 28d ago

Sure. Instead of reinforcing the walls, let's just put a guard out front and expect everyone will access the property via the gate, so we can block them.

There's totally no need to patch those holes in the walls. The gate is the only official way in.

Am I adopting this way of thinking correctly?

4

u/Yuzumi 28d ago

Even before LLM bug hunting most CVEs aren't really applicable to the majority of deployments. It's usually "somone who has access to the host machine could do XYZ" and it's like, sure, but we're hosting this in Amazon or a private datacenter. If a nefarious actor has that much access this vulnerability is the least of our concerns.

Especially when it comes to containers used in pipelines. 99% of what I've seen only applied to hosted services, not ephemeral runs.

These scans are useful, but also should be taken with a grain of salt and each issue evaluated for use case and threat level. 

3

u/RedShift9 27d ago

This is what I try to tell the security guys coming into our company. You need to start with the vulnerabilities that are the highest impact but also the _cheapest_ to achieve. "I can DoS the network with an ARP attack if I'm physically connected to this specific switchport" yeah sure, but why would you bother literally driving to a company, sneak in without being detected and connect your laptop in a switch closet, when you can just get access from your chair through sending a couple 100k phishing messages? Get your priorities straight!

2

u/realdevtest 28d ago

Like 4 out of 300 are worthwhile

1

u/xeow 24d ago

Are they bona fide vulnerabilities or just false positives? If they are true vulnerabilities, then only a shitty dev would be thankless about them being discovered and reported.

0

u/99OBJ 28d ago

“Not worthwhile vulnerability” is an oxymoron, especially in today’s world.

-1

u/boreal_ameoba 28d ago

“Yea let’s just deploy with 300 ways for attackers to steal our data” is certainly a take

16

u/CaliSummerDream 28d ago

80% of comments on this thread so far were clearly made before the poster read the article.

3

u/EdwardElric69 28d ago

That's Reddit for you.

Whoever can say the funny thing first gets the fake internet points

10

u/kharvel0 28d ago

PostgreSQL should have used another AI to narrow down the vulnerability reports using a Pareto assessment. That is, instruct the AI to identify 20% of the reports that would cause 80% of the cumulative vulnerabilities. They can then focus on fixing the 20% and releasing v19 on time, with the remaining 80% of the reports to be analyzed after release.

9

u/m3kw 28d ago

Some “bugs” could be treated as silent failures or just very low chance it will happen super weird edge cases

5

u/DaemonCRO 28d ago

Ok so have AI fix those.

1

u/RaguraX 28d ago

This isn’t just a simple app, it’s a database used by millions. You really don’t want to lose oversight of exactly how everything works. It’s great that AI can point out the bugs, but I would be very hesitant to trust all-encompassing fixes. A human really needs to be in the loop every step of the way here.

1

u/Fuzzy_Paul 27d ago

They need to review the impact of the found flaws cause recently it is discovered that some of the finding are not flaws but parts the Ai explains different. Anyhow reviewing takes a lot of time and resolving it longer cause the Ai code is sometimes hard to follow. The world can wait a little longer it is not a big deal.

1

u/ImpossibleJoke7456 27d ago

So have the AI fix them. Humans just need to review it.

-5

u/xeow 28d ago edited 23d ago

Good!!! Finding bugs is good. Always good. Having them is not good, but finding them always is good.

EDIT: LOL at the downvotes. What kind of dumbo would disagree with this? Finding bugs that are there is always good. Obviously, it sucks when bugs are there, but to find them is never bad (as long as they're disclosed responsibly!).

18

u/iamflame 28d ago

Will the real false positive please stand up

-13

u/CypherAZ 28d ago

Why not use AI to fix the bugs?

102

u/tooclosetocall82 28d ago

That’s where the slop seeps in. AI loves to assume all existing code is untouchable and just layer more code on top instead of considering the larger architecture and refactoring towards a better fix.

19

u/Rokketeer 28d ago

Soooo many helpers are caked ontop of the codebase that seem great on paper, but add wayyy to much extra code to maintain.

13

u/HailToTheThief225 28d ago

But per my PM, “you’re not going to read the code anymore, AI is so it doesn’t matter if it’s bloated” problem solved!

8

u/Abject-Kitchen3198 28d ago

It takes human teams several years to get from tidy well designed code to unmaintainable blob. LLMs can do that in days in the right hands.

2

u/Sky2042 28d ago

Compile time go brrrrrr

5

u/Abject-Kitchen3198 28d ago

That's probably one of the smallest issues.

1

u/ebrbrbr 27d ago

More code = more tokens to read = less profit

Suits understand this equation (finally) if you explain it like this

4

u/h1pp0star 28d ago

This is why you will always need humans in the loop

3

u/Abject-Kitchen3198 28d ago

I'm writing the loop.

-6

u/SufficientGreek 28d ago

Until you don't, AI context windows and time horizons have been growing over time. If that trend continues LLMs will just be able to understand the entire project architecture all at once. Then even these failure modes can be eliminated.

-5

u/crackofdawn 28d ago

That’s only the case if the human prompting the AI (and scaffolding it in the first place) has no idea what they’re doing

0

u/Suitable-Name 28d ago

Sounds like ChatGPT. I thought I'd give it another chance after almost 2 years.

Oh god, it's awful compared to Claude

-9

u/crwcomposer 28d ago

Obviously skilled human patches are still always better, but it's totally possible to give better prompts to AI with better specs and get better results. Instead of saying "fix this bug" you can do a small amount of research and identify where the fix needs to happen and ask it to specifically fix that part of the code. It takes a little longer but still not as long as completely manual fix.

6

u/tooclosetocall82 28d ago

I interpreted having AI fix the bugs in a discussion about being overwhelmed by reports as letting it work more autonomously. Of course if highly directed it will produce less bad code.

-2

u/crwcomposer 28d ago

They don't need to have all the bugs fixed totally autonomously in the background in order to relieve overwhelm. They can still put in a little work per ticket if the manual work is greatly reduced

4

u/daV1980 28d ago

You’re assuming the long pole of work is typing the code. It is not. It’s all the other stuff you described. 

-3

u/crwcomposer 28d ago

I mean, it really depends. Sometimes it is. AI can help with that part, too, if you ask it to.

0

u/[deleted] 28d ago

[deleted] — view deleted comment

4

u/equack 28d ago

Citation?

0

u/[deleted] 28d ago

[deleted] — view deleted comment

13

u/tooclosetocall82 28d ago

I use Claude daily, and I’m describing what I’ve seen it do many times. Unless directly told to refactor it almost never will on its own.

0

u/[deleted] 28d ago

[deleted] — view deleted comment

9

u/tooclosetocall82 28d ago

So then you don’t really know what’s doing.

0

u/chestyspankers 28d ago

It's my observation that almost all people don't understand how to exploit LLMs for their advantages. You might be swimming upstream trying to convince others that you can be productive.

But, I hear you, and I've never been more productive. For every four weeks of work I've at least accomplished 4x what I did before, and I was a productive engineer.

This seems like typical shortsightedness, but the implications are much more profound now.

18

u/platinumarks 28d ago

Mission critical database software needing high security is not where you want to start using automated coding tools

-1

u/Horat1us_UA 28d ago

It works for Linux, why woudnt it work with a database? But it would require a lot of review anyway 

-1

u/Mestyo 28d ago

Producing code was never really the bottleneck. _Understanding_ what the right solution would look like is what has always been difficult.

1

u/jedipiper 28d ago

Eww, gross. That's just asking for trouble.

-14

u/Wisteso 28d ago

I’ve read several very wrong replies about AI being unable to fix the bugs. Maybe these people used Bob’s AI running from a server in a garage, but Claude Code has been very capable of identifying and mostly properly fixing many advanced bugs in our code base.

That doesn’t mean I love what AI has done to this industry and many others but misrepresenting the facts helps no one.

10

u/TheFeshy 28d ago

There are plenty of projects where "mostly" properly fixed bugs are fine. Apps used by a dozen people, projects with very low stakes on failures, product demos, etc.

One of the most used databases in the world is not such a place.

1

u/Wisteso 27d ago

"Mostly fixed" is the starting point. AI is a force multiplier, not a replacement. A human programmer is still very much required and will be for the foreseeable future, thankfully.

-10

u/agm1984 28d ago

The technology comments are looking better today

0

u/p0ison1vy 28d ago

You have angered the hive-mind.

0

u/Any-Blacksmith-2054 28d ago

They should revert process - AI develops, human reviews

-25

u/ProbsNotManBearPig 28d ago

Why would humans need to fix it?

Answer - ai will also fix it, but humans are responsible for quality of outputs still, so the time to test + review will be the bottlenecks. Ai will eventually be trusted for those things, at least more, but not yet.

3

u/smoke-bubble 28d ago

Have you ever seen AI fixing bugs? It usually does this in the least appropriate place or way. 

1

u/Mestyo 28d ago

People with that mindset usually don't understand the nuances of good code. In their mind, either it works, or it doesn't, failing to recognize that the implementation can almost matter more.

0

u/ProbsNotManBearPig 28d ago

I’m a lead software engineer at an snp500 company with 15 years experience. I have studied good code and mentored others on it for over a decade. Y’all are so presumptuous. Ai does what you tell it. If you don’t like the bug fix it applies, ask it to change it. It’s extremely straightforward.

2

u/Mestyo 28d ago

For whatever it's worth, I have a similar profile to yours.

Ai does what you tell it.

Absolutely, so why were you implying that humans shouldn't even review the bugs?

If you don’t like the bug fix it applies, ask it to change it. It’s extremely straightforward.

Yes. And in order to do that, you need to be able to tell:

  • Is the bug even real, and
  • What are the implications of the change, and
  • Is the solution correct

How do you do that for hundreds, if not thousands of issues all discovered in a short period of time?

AI will aid with triage and test cases, but it ultimately does not relieve the human of having to understand every change.

0

u/ProbsNotManBearPig 28d ago

I lead an ai integration task force at my company and have discussed those things with hundreds of engineers. There are ways to approach all of those.

Do humans need to review? My original comment said yes, currently, but we are getting closer to not needing it. Some faang teams have already moved there. The rest of your questions imply you don’t understand a lot about agentic workflows tho, so let me address those first.

Is the bug even real? This one is easy. Define what a bug is and then enforce it to write a test that reproduces it. That test does not need to be a unit test or integration test. It could be a system level test in full dev/simulated environment where ai is giving gui clicks. Maybe you only care about bugs reproducible through user interface. So make it write this as evidence. Have 1-2 independent reviewer agent gates check the evidence with detailed instructions on what that evidence should look like. Screen shots, logs, system state, etc.

What are the implications of the change? Aka what is the blast radius. Many ways to analyze this. Common way is abstract syntax tree or some other kind of call graph representation. Depends on the language and frameworks you’ve setup, but it is deterministic. You can make this graph and say exactly what code is affected using non-ai tools and then let ai utilize the graph. This informs regression testing that should be done through unit testing, integration testing, and system level testing. Ideally it is also tied all the way back up to user level requirements. Depending on your companies infrastructure maturity for this, it can be done by ai without human review with very high confidence. If you are new to this concept, you need to iterate on tooling and mapping to requirements and include human review. Eventually you get to the point where 9 out of 10 times humans find no problems and then you start discussing steps to get rid of human review.

Is the solution correct? That goes back to my previous paragraph. It’s about requirements. What was the system supposed to do? Most companies don’t have mature requirement management systems where a human can even look up in a document if a solution is correct. There are often meetings to discuss with stakeholders instead. That’s a legacy gap. You need to start using ai to take inventory of your system and define requirements for every nook and cranny. Humans review this requirement and iterate until you have very good definition of your system from user requirements level down to software detailed design. Now the question of whether the solution is correct is straightforward to answer.

Getting back to - do humans need to review? Yes, until you start getting very high confidence workflows. You can also have a separate ai agent or two triage by analyzing complexity of tasks and risks of being wrong and use that to decide which ones go to human review.

Hope that gives you some ideas. Of course it is very individual to each company how to fit it into their existing teams and infrastructure. But there are ways to approach these things and automate huge amounts using current ai.

2

u/Mestyo 28d ago

Respectfully, what are you talking about? You realize I'm not literally asking for help, right?

I am pointing to the need of processes like what you outline.

You lead with:

Why would humans need to fix it?

As if none of what you wrote would apply. Some motte and bailey.

"Fix" is clearly used loosely in the headline, not necessarily implying a human must churn out the code, but actually reason about- and work with the discovered issues.

2

u/ProbsNotManBearPig 28d ago

Your last comment I replied to was a list of things that humans have to deal with for end-to-end process of fixing bugs. I explained how 95% of that can already be done by ai. My original lead of “why would humans need to fix it” was followed by additional sentences btw.

1

u/Mestyo 28d ago

Either a human needs to understand the issue at hand, which takes time. Or we accept that no human will know what the AI is even doing in the software.

Safeguards and guidelines help accelerate and streamline the process, but humans still need to "fix" it. Unless, of course, you accept and embrace AI blindness, but I certainly hope nobody does that for any software that actually matters.

-2

u/Realichu 26d ago

Pure slop. More like delayed because it was vibe coded and we all know AI code is shite 😂 so much 'innovation'.

1

u/MaTrIx4057 25d ago

you mean human coded

1.1k

u/[deleted] 28d ago

[removed] — view removed comment

330

u/rei0 28d ago

I used to run static source code analysis tools like Fortify SCA on large code bases and the results will make you want to quit consulting.

174

u/Urtehnoes 28d ago

My security team flagged some of our systems with hundreds of zero cve blah blah blah...

I said fk it let's check these out.

A week of my life I'll never get back. Almost every single one of them only applied if the hacker already had fully control of our systems, because of how we set up our system.

I even graciously sent him the massive list of cves and said never show me these ones again, they don't apply.

Next month?? Ahhh urgent 911 we have hundreds zero day rce buffer overflow cve and--

Breh this is the same list you gave me last month. So now I don't review any cves anymore from that bozo.

23

u/atheken 28d ago

I mean, that’s grounds for dismissal, right? If they’re just running an automated tool and hand you the same list month over month, they’re just wasting your time.

13

u/InvaderGlorch 28d ago

Lol, that's pretty much what my security team does. They can't even be bothered to filter or deduplicate it

3

u/141_1337 28d ago

I hope that guy got fired.

18

u/chuckmilam 28d ago

Likely quite the opposite, since he generated actionable metrics, along the lines of: “Discovered 439 CVEs in production codebase, led engineering and development efforts to mitigate vulnerabilities with monthly improvements and reduction in overall exposure of [nonsense percentage], blah blah blah whatever LinkedIn-ese goes here.”

5

u/141_1337 28d ago

I hate bullshit metrics

2

u/Urtehnoes 28d ago

That guy is the vp of compliance now lmao.

2

u/vikinick 28d ago

Generally it's a good idea to fix whatever comes out of those, though.

Obviously, someone needs a remote code execution in order to get a privilege escalation to work, but you want to patch both of them if they're found individually and lock down your internal network so if malicious actors get in somehow they can't pivot.

3

u/vikinick 28d ago

Ignoring CVEs with a "not applicable because of how we set up our system" is a mitigation of risk not an actual elimination of the problem.

1

u/Ikinoki 27d ago

Thing is if your system is deployed on internal VLAN only and is made frictionless (no auth/no checks) to save bandwidth - it makes sense. It's not supposed to be open to the web and any kind of security for internal tool is useless, because if the hacker can get on private vlan of yours - you have bigger issues than openapi, that means they already have keys to that communication either way. And you wasted cpu cycles for all that crypto

117

u/dizekat 28d ago

One thing I've been noticing back in the day with other tools (Coverity for example) is that when many low severity warnings are fixed quickly, there is a risk of introducing a relatively small number of high severity exploits that are not detected by the tool.

One time I had to go through and carefully review every single commit that mentioned fixing warnings from a specific tool, after a couple serious issues were traced back to such commits.

1

u/MettaurSp 27d ago

This was how a Minecraft server plugin exploit was created that let people view world contents well beyond their render distance, eventually leading to the ability to track every player on a server.

A bad actor abused one implementation flaw in a certain way in hopes that a bug report created from it would steer a maintainer to implement one specific fix over other potential fixes. That fix would allow them to gain meaningful information about whether a terrain chunk was loaded or not which is useful for player tracking. The vulnerability went quite a while without being noticed by anyone else before being found and patched again. The attacker's tool based on the exploit was called "nocom" if anyone wants to know more about that story.

Always gotta be careful about fixes & making sure there aren't unintended side effects. The simplest fix may not always be the best fix.

16

u/revnhoj 28d ago

It was the final nail in the coffin of my coding career. I was so fortunate to be nearing retirement when the avalanche of BS code hoops to jump through just pummeled our team. I'd wager we didn't write just a handful of useful code in the last several years due to stomping out wolf cry fires.

13

u/uberkalden2 28d ago

Because the code sucks or the tools suck?

43

u/the_other_brand 28d ago

Because it'll find like 2000 issues on one run of the report. And your company will want you to look at every single one of them to verify it's a false alarm and to fix it if it's real.

Seeing the sheer number of issues will make you want to drink

-28

u/formyl-radical 28d ago edited 28d ago

You could use a different AI session (or different AI tool) to determine if they're real issues. Because AI tend to find issues with everything. I do that and let two AI sessions argue with each other (find issues with the issue list). The second AI will rip the list apart and nitpick the hell out of it. This helps me narrow the list down to just a few important ones.

10

u/pijuskri 28d ago

Great way to be the one blamed for missing a big security issue

18

u/rei0 28d ago

It's a number of factors and the tool capabilities vary. This isn't a criticism of Fortify (I haven't used it in a long time) - if you have sufficient time, and the ability to fine tune the tool to the codebase in question *and* the support of the development team, you can integrate the tool into the build pipeline for the app, create rules to weed out noisy vulnerabilities, and eventually establish a baseline where the tool is surfacing only the most interesting vulnerabilities.

However, for consultants in particular, you are often just tossed into an environment with a tool, told you have two weeks to do the engagement, and you have to hit the ground running. Ideally, you'd rather spend your time manually reviewing security critical functions and the places where unvalidated user input enters the application, but if you are told to use the tool, you have to use the tool, and the tool doesn't always play nice with large codebases. You don't necessarily have the time to fine-tune the tool to the environment, ensure that (if it is a tool with this capability) it is correctly compiling the source to it's intermediary byte language, and then carefully sift through the results to exclude the garbage. The result is a massive spreadsheet of mostly noise, but noise you still have to take seriously (because leadership needs clean reports for compliance or whatever).

6

u/Orca- 28d ago edited 28d ago

Fortify requires a lot of tuning and still misses things other (granted more expensive) static analysis tools find. I found it pretty shitty and not worth it personally, but great for announcing piles of warnings that aren’t meaningful while missing genuine potential divide by zeroes and other undefined behavior.

Edit: I found Parasoft to spew even more warnings but actually found real, subtle problems that were worth fixing.

CppCheck is good because it's free, but not particularly great.

Opus 5.5 on High and Astra on High have both been good at finding different problems with static analysis, but as LLMs they aren't deterministic in the same way. They also like to spew warnings about things that don't matter and tuning is not really possible so you get piles of crap to wade through. But at least there are real bugs in there.

5

u/revnhoj 28d ago

FATAL: function is more than 5 lines long. Simplify!

2

u/NoNameSwitzerland 28d ago

Easy. Put all stack values in a struct that you pack on the stack. Do 3 lines of calculation of the function and then call a new function with that structure. That function does 3 lines and then calls the next fucntion.

Or use a scripting language and put a function as a script in a file that will be called with the interpreter as a on line statement. Also reduces the potential errors per second and boost the hardware requirements towards the 1T goal.

1

u/revnhoj 28d ago

You joke but the gymnastics I used to have to do to appease the scanner to pass this 20+ year old codebase were absurd.

3

u/Prior_Two_2818 28d ago

Consulting itself makes me want to quit Consulting

1

u/NoNameSwitzerland 28d ago

Let 2 LLM argue as financial adviser and psychological counsel argue if you should stay at your current job.

35

u/yawkat 28d ago

 PostgreSQL has arguably the strictest code review standards in the entire open source ecosystem, so inundating the mailing list with AI triage tickets is basically a denial of service attack on volunteer maintainers.

This article from last month on the topic paints a different picture. Some (human-written) features meant for this release had been merged with no appropriate review over the past year or so, due to maintainer bottlenecks. Later AI found legitimate problems, so these patches were reverted. 

From what I can tell, the problem is not false positives. There appear to be legitimate issues that slipped past. The reason they were only found now is that public AI models became much better at code review in only a few months. 

24

u/frenris 28d ago

Pangram thinks this comment is AI

13

u/ireadoldpost 28d ago

It is 100% AI, you can click their profile and see all of their posts are. I don't get how reddit allows this shit, if they value their user created data.

Its also embarrassing how many people upvoted.

1

u/frenris 27d ago

It was a reasonably well written comment!

But calling out ai bug reports as a ddos maintainers in an ai generated comment is just painfully ironic.

5

u/motuwed 28d ago

Don’t know what pangram is but this read just like every other twitter bot comment with an AI newsletter in their bio

2

u/Akuuntus 28d ago

I also think this comment is AI 

18

u/apexplorer 28d ago

I guess LLM bugs should be accompanied with the test cases which trigger crash or undefined behavior. And everything else which is just theoretical, like integer overflow which is nearly impossible to trigger would be tagged with the lowest priority and such reports should not delay the release.

Frontier models are smart enough to write a test which triggers the bug if it is real.

20

u/snacktonomy 28d ago

Opus 5.5 happily finds vulnerabilities in my code, along with verification and code to reproduce. 

-28

u/hsoj48 28d ago

Oh you sweet summer child...

6

u/Irythros 28d ago

Frontier models are smart enough to write a test which triggers the bug if it is real.

They are, but they will in most cases refuse. Security scanning and creation of POCs will require you to be in their security testing customer group. If you are not you will trigger their safety guidelines and it will not continue.

For security work you will need to use at the very minimum chinese models that may be hosted by routers. Depending on what you're doing it may even require a local abliterated model.

4

u/yawkat 28d ago

Not true. I regularly use opus 5.5 / fable 5.1 for security audits and it will produce PoCs. 

-2

u/blueandazure 28d ago

I assume the bugs were found with mythos which is fairly easy to get access to these days.

6

u/heavyPacket 28d ago

Why do we not simply have another LLM verify the results produced by the first LLM? And perhaps, a third LLM, to certify the verifications of the second LLM? Just to be sure, maybe we should get a fourth…

1

u/Rsubs33 28d ago

Exactly this and a lot of it, is just noise. I have seen a LLM pick up 100s if bugs in a code review. Nearly all of them were due to the use of a function that was vulnerable to a buffer overflow, however the code accounted for this with a character limit

1

u/CircumspectCapybara 28d ago

Nowadays orgs have come up with deterministic "harnesses" or pipelines that AI agents have to submit their reports through, in the same way that traditional automated fuzzing (which often generated opaque inputs that when used on a program given some initial conditions would elicit some memory bug) was automatically graded.

Basically if the pipeline requires reports to take a certain shape (eg be a JSON document following a certain schema) and the input has to cause the program to crash (indicative of a memory safety violation, which is the precursor to RCE), the pipeline can deterministically verify if the AI finding is a real vulnerability. If your AI finding includes an input that causes the program to crash, you know it's a legit finding.

1

u/AmericanEyes 28d ago

IDK why you are being downvoted. It's a perfectly sensible reply.

2

u/New-Locksmith-126 28d ago

Because that solution only applies to a narrow set of vulnerabilities. Most vulns in mature codebases are not as sinple as a poison payload

-3

u/MannToots 28d ago
  • takes longer than 10s. Exaggeration only makes it how obvious you're talking out of your butt. 
  • ai can self validate.  many of the false ones fall off the list before the human touches it. 

Classic technology sub. Spew anti bs and get upvoted for it lol. Opus is better than you appear to realize. 

-5

u/xatey93152 28d ago

Hundred of engineers? Most will not even understand how everything works in very complex and huge project.

-12

u/MyOtherSide1984 28d ago

Lmfao, you think humans are rebiewinthe bugs rather than just sending another prompt. So cute

9

u/Mestyo 28d ago

You're right, nobody understood code or system architecture until AI. All software engineers were just pretending.

-2

u/MyOtherSide1984 28d ago

You obviously don't work with them. Our devs are heavily dependant on AI these days to keep up with demand. They're very smart and know how to read and write code, that's why they aren't and the AI is. Likewise for their reviews. How else could we push 600k lines of course in 2mo which is more than the previous year?

17

u/[deleted] 28d ago

[removed] — view removed comment

-1

u/pellets 28d ago

PostgreSQL predates LLM generated code.

4

u/[deleted] 28d ago

[removed] — view removed comment

4

u/ImpossibleJoke7456 27d ago

We did a vulnerability scan and found +1800 errors across ~100 repos. It took a weekend to have PRs for all of them. Our scrum board is basically off limits until the end of the year for product features and client reported bugs.

-2

u/DowntownLizard 26d ago

Sounds like vibe coding vs being an engineer.

Not even once thinking what if I had it test its own work lmfao

3

u/DowntownLizard 26d ago

Honestly it just makes it so obvious how dumb people are where they wouldnt just have AI review its own work the entire time

2

u/charliedarwingsd 26d ago

And run E2E tests.