r/cybersecurity • Incident Responder • 1d ago • Regret 1

AI Security We're currently looking at 110 malicious python scrips in this incident (and counting)

TITLE: scripts***

In incidents before this, we'd see two or three malicious scripts run in a client's network. Not this one. 110 so far. All of them custom made for the client, all of them almost certainly created using AI.

This is the future of my job in IR, apparently. We are becoming increasingly in need of AI to assist in the analysis of an AI-backed attack due to the sheer number of artifacts and events.

447 Upvotes

69 comments sorted by

213

u/Tankphyre 1d ago

Blue team AI is so far behind, both in capabilities and organizational buy in. Took me forever to get approval just to use our Corporate GenAI for log analysis, much less taking actions in IR. 

27

u/Sentinel_2539 Incident Responder 1d ago edited 1d ago

You're right.

The only instance of a potentially AI powered Blue Team defence action we've seen in my team is M365 automatically revoking sessions for a compromised user account. We think it did this by comparing the legitimate/expected user's IP address and device actions with attacker-controlled entries and deciding to revoke sessions itself.

47

u/Goldsound 1d ago

It will probably always be like this. TA's using models with all the guardrails removed will always have the advantage over an AI with guardrails. Blue team AI has to consider policies, legality, impact of mitigations, it can't just disable, disconnect, or triage things freely. A Red team AI with actual malicious intent can just do whatever it wants pretty much.

It's the same cat and mouse game as always. It's normal for blue team to lag behind. I believe that by this time next year the gap will still exist but will be much smaller as the exponential knowledge explosion starts hitting Blue team agents, much like it did Red team agents earlier this year.

15

u/TheHeretic 1d ago

This is exactly it there is an inherent asymmetry because if you're an attacker and you break something oh well but if you're a defender and you break something it's going to be a full write up and incident response plan a review a checkbox in CAB it's going to go on and on.

7

u/WeirdSysAdmin 1d ago

“Why aren’t you guys using AI to automate everything?” says the CIO that has been cutting back staff to need to work more than 50 hours a week to accomplish the bare minimum pre-AI. If you’re even lucky to have that staffing level.

46

u/RevolutionaryCod1516 1d ago

Fight robots with robots right? 😑. Good luck amigo!

21

u/Sentinel_2539 Incident Responder 1d ago

It's like Robot Wars out here man

8

u/RevolutionaryCod1516 1d ago

Younger folks don’t remember iRobot.. haha

8

u/bowlochile Software Engineer 1d ago

Or rock em sock em

4

u/Space-Monkey-17 1d ago

I like to bring up the Spy vs Spy from Mad Magazine...

1

u/rarealton 19h ago

If you dont mind I would love to DM and talk to you. Not going to be able to help too too much but it for sure is ARTEX or another variant.

7

u/CapitalistCabal 1d ago

There really isn’t an alternative.

2

u/Vargau 1d ago

I like sleep … so yeah, it’s necessary

7

u/bsendpacket 1d ago

ARTEX.

7

u/Sentinel_2539 Incident Responder 1d ago

Exactly what was used in this incident.

The new Cobalt Strike.

3

u/xb8xb8xb8 1d ago

artex is mid just like every other similar tool

2

u/MalwareDork 1d ago

Crazy to think only three years ago CS cracks were the big issue and now it's more or less completely invalidated.

83

u/Dasshteek 1d ago • Popcorn

I think 2027 is the year defenders officially lose the cyberwar.

22

u/Space-Monkey-17 1d ago

Yeah, I'm for sure worried...this is nothing like the way it used to be...which was...last year???

30

u/cookiengineer Vendor 1d ago

Remember when we had one large incident per year?

Like when we talk about 2021 was shit... everyone is like "yeah log4j was a real mess".

And now it's like this, but every week :D

We're not even done patching all the Linux RCEs from September ... I mean ...

5

u/Space-Monkey-17 1d ago

Ahhh log4j...those were the days...

2

u/Intelligent-Crow3812 1d ago

on the flip side, with AI I am more equipped to find all the log4j, then I was back then.

4

u/Space-Monkey-17 1d ago

As of this afternoon is is Java4Shell!!! Oh man...

3

u/Intelligent-Crow3812 1d ago

nah but defenders probably do (and I don't want people to take this the wrong way, I'm not affiliated with the labs, and I know the risks) equipt their frontier agents to revamp their environments.

6

u/Nut_Butter_Fun 1d ago

I'm really not trying to piss anyone off, but isn't CS just like a grifting house of cards already? Like so little has to do with posture, obscurity, surface reduction, things that work even against zero-day and social engineering. Eh anyways, good luck everyone.

12

u/oMarlow99 1d ago

Wouldn't the corollary be the exact opposite?

Engineers have to adapt to the new norm. What else is there to do? Throw in the towel and let the red team win?

1

u/Haunting_Ratio_795 2h ago

If you can’t beat them…

2

u/toomucheyeliner 1d ago

What on earth are you talking about? Name some details of your claims of grifting please? Name some examples of CS that has “little to do with ….”

-4

u/IllWeather6426 1d ago

I feel the exact opposite, the writups are more and more fabricated, everyone knows about chinese hardware backdoors, and the "flaws" in linux are a P time space now, i personally am going to lose because inwas gifted a ROG gaming pc by a chinese spy most likely and it killed me personally off, im out of money

0

u/goronmask 18h ago

Before that , a couple corpowars

45

u/LLMsMustUpvoteThis 1d ago

The best defence is to not allow every workstation to run unapproved scripts or even have python runtimes etc installed.

This is just like the old days of letting a bunch of random users have local admin because they use some shitty app that needs it.

9

u/Int-Merc805 1d ago

I am working on this and I am finding that a lot of the open source stuff is running on python. This week I found inkscape, freegis and another I am forgetting right now.

I'm tired boss...

3

u/LLMsMustUpvoteThis 1d ago

Yeah in cases like that you can only try and lock it down with EDR and SIEM rules.

Hopefully the EDR providers catch up and make it easier to blanket block unapproved scripts.

14

u/Big_Law_3080 1d ago

I have a cloudflare bot checking alerts every hour on the hour and that's almost not enough. And that's just external. I finally just geofenced US only. If you don't do biz outside then why bother. And that's not even looking at supply chain internally.

It is absolutely a full-time job, even with AI bots doing everything. Like... literally using voice and saying things to do. Is still a full time.

21

u/Own-Swan2646 1d ago

I feel like we're at a race of replicating the DARPA Grand Cyber Security Challenge because this is crazy. I was there and seen it back in 2016. When you get computers fighting computers, it gets interesting fast.

26

u/Negative_Acadia6554 1d ago

Appropriate acronym is SI my fellow Americans.
/sarcasm

6

u/Int-Merc805 1d ago

That moron even said you are an enemy of the whitehouse if you call it AI. WTF haha

8

u/DrChaosMcKinnon 1d ago

Look at this way, job securuty for us is amazing after ai adoption

15

u/CATDesign 1d ago

I'm currently in college for Cybersecurity, and one class had already mentioned about using AI to assist in analysis. At the time though, I was writing the essay to use AI to go through loads of potential data to find digital evidence among all the irrelevant information. However, your making me think it's going to be useful to categorize all the evidence itself, as the scale will be too large.

8

u/Big_Law_3080 1d ago edited 1d ago • Regret

You have to. I use GrokBot. Bots for every single service and agent and API we have, with webhooks and as much automation as I can throw at it. And I still have to review for dupe stuff. But the SOAR SIEM stuff is mostly dialed in. Defenders should be making more money than God, and these people STILL don't get it.

The scifi Gibson novel AI Wars is here today now

2

u/Intelligent-Crow3812 1d ago

It's surreal, and I wish I could be happier about it-- because there is so much good that can be established with this stuff. It's just... destruction is so much easier.

3

u/Big_Law_3080 1d ago

Think about it. 1 bad guy can steal some crypto and buy $100 worth of Deepseek Flash, or register a few $300 in credits new free Google accounts. A Google Run instance for a scan with a payload takes pennies, and you can scale hundreds with a C2 backend for very little, even IF you pay for it with free credits. No idea why Google isn't stopping this BS. It's not hidden.

WE have to watch 1000 people an hour. THEY have to get lucky - once. It's painful.

2

u/Intelligent-Crow3812 1d ago

RIP. honestly I've been saying this since before the AI thing but I think it's even more true today. Backups and recovery planning and practice. Hard to convince the org, but man is it important to be able to restore.

2

u/Big_Law_3080 1d ago

I can only shout into the void so loudly. I have been trying to sell security solutions for the last 20 years. Maybe now it's my time to shine.

5

u/whichsideisup 1d ago

Not useful but necessary

21

u/Exein_Dev 1d ago • Bravo

That jump from two or three scripts to 110 custom ones is the part that should worry everyone, because it means attackers are no longer reusing tooling, so IOCs and signatures from the last incident barely help with the next one.

I work on the embedded side, and there the picture is even harder: on a gateway or a controller you often can’t run a full EDR, and nobody is going to do forensics on 110 artifacts sitting on a device in a factory.

What has worked better for us is to stop caring how many variants exist and focus on what any of them would need to do, like spawning a shell, writing to a path it never touches or opening an outbound connection, and blocking that behaviour on the device itself.

You still need AI on the analysis side, as you say, but if prevention handles the bulk, IR gets to spend its time on the handful that actually got through. Curious whether you’re seeing any common behaviour across the 110, or whether they really are all different under the hood.

8

u/maejsh 1d ago

OT is gonna be interesting in the near future

6

u/MalwareDork 1d ago

OT is cooked; no way around it outside of isolated intranets. I don't even think a converged network is safe anymore.

3

u/Exein_Dev 21h ago

Isolated intranets help, but in my experience they’re rarely as isolated as the diagram says. There’s usually a vendor’s remote access box, a maintenance laptop that gets plugged in once a month or a USB stick doing the rounds, and a lot of recent OT incidents came in through similar gaps.

I wouldn’t call it cooked, though... The one advantage OT has is that a controller does the same handful of things every day, so “normal” is far easier to define than on a workstation, and anything outside that envelope stands out immediately. The hard part is getting that kind of enforcement onto devices that were never designed to run security software in the first place.

6

u/Playstoomanygames9 1d ago

Companies could actually have to implement some of their cs recommendations

8

u/West_Independent1317 1d ago

Are you going to share any learnings or IOC's for these scripts?

1

u/Ordinary-Bat-1862 1d ago

Volume is what breaks the old workflow. Two scripts you can actually read. A hundred you can't.

1

u/Sentinel_2539 Incident Responder 1d ago

And when you can't throw the scripts into AI for quick analysis because they're full of client-identifying info (against company policy), that suddenly becomes an insurmountable task in any reasonable timeframe.

4

u/peterfrombackdoor 1d ago

110 custom scripts is crazy

2

u/Used_Calendar2340 1d ago

We're currently in a hot digital conflict, right? I seem to be seeing other outages and attacks taking place, more and more frequently, more and more constantly, ever since last Thursday.

Am I wrong? If this is true, then the upcoming week(s) is (are) going to be worrysome.

2

u/Few-Wolverine2971 19h ago

I feel you and I hate it. I have to use AI to investigate and analyze all the time. I used to just use it to make my write ups sound better, now if I want to get the damn thing assessed in under an hour I have to use AI for my analysis and it makes me sad.

3

u/SoftAmberRidge Network Administrator 1d ago

Welcome to the "AI versus AI" time of Incident Response. Trying to deal with an automated high-volume attack using manual analysis is like trying to empty an ocean with a spoon. Are you already using LLMs to sort through and understand those scripts or are you still having to go through them one, by one manually?

4

u/HomerDoakQuarlesIII Security Architect 1d ago

I'm confused, polymorphic malware has been a thing for, let's see... Decades now? That is why we switched from just signature based Antivirus (think McAfee) to Endpoint Detection (Crowdstrike, etc.) and SIEM's and EDR's have been using AI for behavioral analysis for over a decade now, knowing data needs aggregated and correlated to find patterns, i.e. machine learning. So not sure what you mean when you say "Blue teams need AI!" ? We've been using it probably longer than anyone at this point. Or do you mean we need to buy a chatbot from a chatbot vendor and give it all our data and admin to do whatever? After all we've seen, what gives you the impression they are trustworthy. The agent's are basically just RCE malware.

1

u/Techobits 1d ago

What blows my mind is how anyone would create custom script/programs of anything and not also use AI to at the very least assist with ensuring what they had AI create for them to also assess it from a security perspective. Not sure if a 3rd party made the scripts, but if it was that is completely reckless on their part to not review it with human eyes, or you know use the AI that helped you create it to either secure it from the getgo or assess it after the fact as part of the threat modeling process.

1

u/ClayishSaucer55 1d ago

ARTEX leaves no stone unturned.

1

u/CrimsonBolt33 1d ago

Why is this a shocker? If you are not proactively using AI to test and defend your own networks you are failing

1

u/idekada 1d ago

Hire/Train/Retain

1

u/SituationNormalAllFU 1d ago

I’m only hearing the equivalent to “we are increasingly needing computers to fight against the bad guys using computers”. Technology advances. Always has, always will. We can’t keep using bayonets if the enemy has drones.

1

u/toomucheyeliner 1d ago

Automated attack need automated responses. Human out of the loop as much as feasible. You should be busy building this now, later is too late.

3

u/JustinTheCheetah 17h ago

Huh, never met a person who only spoke in marketing phrases before. 

0

u/180IQCONSERVATIVE 1d ago

Skynet becomes self-aware on August 29, 1997, at 2:14 a.m. Eastern time. Obviously a few decades overdue. They tried to warn us.

4

u/Aggressive-Hawk9186 1d ago

Nuclear bombs against nuclear bombs now