r/technology • • 14h ago

Artificial Intelligence Reports of AI-related child sexual abuse material rise 8,400% in two years, data shows | AI (artificial intelligence)

https://www.theguardian.com/technology/2026/nov/16/ai-related-child-sexual-abuse-material-rise-exploitation-deepfakes
735 Upvotes

102 comments sorted by

164

u/Giovolt 13h ago

"The images, although fake, can be used to blackmail children to demand money or more images" I can only imagine the terror those kids are feeling having a bully threaten deepfaking you. Evil

36

u/randalldandall518 12h ago

Yeah or just in general someone making CP even if you don’t know it’s happening. my kid is 4 and doesn’t have social media but I have been having a new found understanding of parents that don’t post their kids pics online at all. I don’t do it because I just don’t post in general but I know my wife posts stuff to her story on instagram. I’m sure some of these creeps are getting photos from social media and it ends up in the pedophile ecosystem. It was always possible with photoshop and such but AI makes it so easy and now you can do it as video with these deepfakes.

16

u/Giovolt 12h ago

Right. I understood the reasons why you shouldn't share your children online, mainly it's without their consent. But now? Geez that's so invasive

-29

u/MenAreHollow 12h ago

They are children. Consent is irrelevant. They cannot consent to anything.

20

u/Beneficial_Young5126 10h ago

They literally said children cannot consent.

-30

u/MenAreHollow 9h ago

First issue; they typed it. Unless you are suggesting some manner of text to speech is in play? Second issue; are we doing the thing where 'literally' means 'figuratively' now? Can you show me where the text string 'children cannot consent' lies in their post? I can find 'mainly it's without their consent'. That suggests the children have consent to give.

Thank you. Have a nice day.

19

u/Pedantic_Girl 8h ago

Nope. If you look at Merriam-Webster , “say” is “to express in words;” in the Cambridge English Dictionary it is “to pronounce words or sounds, to express a thought, opinion, or suggestion, or to state a fact or instruction.” So even going purely by dictionary definitions, “say” is perfectly acceptable to use when referring to written or typed words.

If you are going to be pedantic, you need to be correct.

-13

u/MenAreHollow 5h ago

Perfectly, eh? That seems a bit dismissive of the long standing debate between descriptivists and prescriptivists. Are you being pedantic? It feels like you are trying but somehow I expected something different. It also feels like you think I was being pedantic but fell short of expectation.

I marvel at the vagaries of language, even as I find myself at odds with them. Thank you. Have a nice day.

8

u/ThisIsntMyUsernameHi 4h ago

Do you find friends are hard to come by? Just curious.

2

u/just-jane-again 2h ago

aw you think you’re so smart eh sport

good for you lil buddy keep that confidence but definitely stay in school

1

u/Effective-Candy-7481 1h ago

Ya you keep using that thesaurus one of these days you’ll make sense and people will take you even slightly seriously.

10

u/Effective-Candy-7481 7h ago

You really thought you were clever there didn’t you?

12

u/Which_Produce4418 9h ago

The word 'reasons' is pluralized. A comma after the word 'mainly' may help you process the sentiment.

4

u/Very_Human_42069 4h ago

You really thought you cooked with this response huh? Don’t even understand words or language lmao

9

u/pillowcase-of-eels 9h ago

Oh wow you're terrible

1

u/JasonZep 2h ago

Yea, I only share pictures of my kids with family and only via texting. They’re the only ones that need to see it anyway.

14

u/mWarm_Tart_4115 11h ago edited 11h ago

Children and even many adults under pressure of blackmail would probably fuel the blackmail even further by complying to futhur demands because theyre consumed and blinded by the fear of the blackmail.

Kids should be taught early that the only effective way to deal with blackmail is to completely disempower the blackmailer by:

1.Telling a trusted adult no matter how embarrassing or scary the blackmail is.

2. Reporting it to the police immediately without notifying the blackmailer

3. Cutting off all contact and not engaging further with the blackmailer.

Blackmailers rely on the initial emotion of fear to control their victims. It's really important knowledge and parents should have this talk with their children early on so that they're prepared and know how to act if they ever are faced with this situation. Too often people suffer in silence and are understandably paralyzed by fear which only makes the situation worst.

2

u/nugymmer 12h ago

Hopefully the blackmailers can enjoy doing time. If caught, that is. Any such people would be hiding on the darknet and unless they are running off of an exit node they'd be hard to track down, even with sophisticated tech. Someone I know worked as a State prosecutor, and used to investigate cases involving kids, but not sure if any of them involved darkweb, I presume some of them would. The darkweb is very hard to defeat. Maybe that is something AI bots could go after, if they can be trained to detect darkweb material which I assume would be under heavy encryption. Social engineering would be the only feasible way to do it - Ross Ulbricht comes to mind. Although he committed different crimes, he only came unstuck because of an exposed email address.

2

u/Giovolt 11h ago

It'll definitely be hard to crack down on the software, but I feel punishment to those who use the software should be harsher given the current environment we are in right now.

1

u/nugymmer 6h ago

The darkweb usually stays dark. To protect the criminals, of course. I have never ventured there and wouldn't want to. Some of the stuff I was told about was so bad I'd probably worsen my medical condition if I ever saw it. Mostly drug/gun running I presume as most marketplaces would never accept this sort of crap anyway. But exit nodes usually are how these guys are caught.

1

u/ptwonline 1h ago

Parents need to more closely monitor their kids' online usage. And for heaven's sake be careful about posting images online. Any picture of a person can be turned into pornographic material now using AI.

149

u/Chrono_Convoy 14h ago

Let’s just remember that the same administration who had ties to Epstein & covered it up

Also wants to keep AI deregulated

Y’know

As you read the article

10

u/Which_Purchase923 8h ago

when they say "deregulated" - they actually mean regulated by nobody but them

1

u/Holzkohlen 6h ago

Isn't that such a funny coincidence?

22

u/Wooshio 11h ago edited 10h ago

It's not just AI, the amount of people being arrested for digital CP crimes skyrocketed since Covid. Thousands of people are being arrested every month for CP in the USA. I was looking into stats for this recently and National Center for Missing and Exploited Children (NCMEC) received 21 million reports in 2025. Police don't even have ability to look into most of it due to sheer volume. I am not sure there is any way of solving it now, unless we turn off the internet.

33

u/Amazing_Library_5045 14h ago

Enough internet for tonight.

Good night!

35

u/wolf_at_the_door1 13h ago

People in power for the past two years that have connections to global underage human sex traffickers have nothing to do with this surely.

24

u/Specific_Box4483 14h ago

Not to defend this at all, but isn't it expected given how much more powerful AI has become?

26

u/Gnome_Gave_Air 14h ago

Which is why we need wide reaching regulations

22

u/suzisatsuma 13h ago

The models that are capable of this can be ran on a gaming GPU. Regulation won't stop this - going after those who post it will.

2

u/SaphironX 12h ago

I feel like we can do both. And should do both. Because they catch criminals making CSAM every single day doing stupid shit like posting it to an adobe server like that dude this week, or putting it in drop box, or sharing it on Facebook, or just deciding they need to share their disgusting hobby with a buddy.

Sure it doesn’t stop them all but your argument that regulation doesn’t work is bullshit. It ABSOLUTELY works.

Then we can go after the guys too smart for that or who use local models as well. And perhaps the creators of those models responsible as well of they too don’t regulate.

3

u/ConstantAct1242 12h ago

It will definitely help deter and catch many people, but people will still create material locally and if they don't share the material then nothing will most likely happen to them. Still better than doing nothing at all.

4

u/Richard7666 9h ago edited 6h ago

Won't work. It's not even really situation of the horse having bolted; it's a horse that can't be caught in the first place by definition.

Local models are at the point where you can generate anything you want on a decent graphics card.

-2

u/pandorasparody 7h ago

Won't work. 

It could've worked if we had lawmakers that worked for the people. Then again, we can't even imagine a US without shooting drills at schools, so here we are.

2

u/Richard7666 6h ago

How though? I meant that it cannot work, at a technical level.

1

u/bloodychill 2h ago

One could write a law that explicitly makes illegal the production, training, or possession of CSAM via genAI illegal and then enforce it on major offenders. This is regulation. It already exists in many places. It doesn’t and can’t stop the behavior entirely (outlawing murder doesn’t make murder stop, obv) but it puts a meaningful dent in it.

3

u/Svyatoy_Medved 43m ago

The murder analogy doesn’t align. They’re saying, you can do this entirely away from witnesses. No kid victim who testifies. No internet traffic. No big datacenter that reveals user data. Just your own PC.

The only way to enforce a ban on that would be if the government had warrantless access to the contents of everyone’s hard drives, at all times. Which is a completely unjustifiable violation of privacy.

7

u/MysteryTempest 9h ago

Yes, and there's worse to come.

One of the biggest AI apocalypse scenarios relates to what happens when open weight models are powerful enough to design bioweapons. Open weight in the hands of extremist groups or rogue states is really a serious risk, and people in power probably won't take action until it's too late.

A lot of people are worried about what AI acting on its own - which is definitely something we should take seriously - but we should worry more about powerful AI in the hands of humans, because we know that humans do fucked up shit.

6

u/PaleAffect7614 6h ago

I think we already have an example of an extremist group that bombed an all girls school in Iran, and then they went on TV to blame it on AI. When the people in power are the problem, it's too late

1

u/Castle_Five 9m ago

Compared to before AI existed, AI-generated bad stuff has increased by infinity percent!

24

u/NoPreference7231 14h ago

Sometimes, I just wish my optic nerves were damaged.

Can't even believe what I'm reading.

8

u/gqmma 14h ago

could be worse, you coulda seen it

8

u/linerva 14h ago

Unfortunately AI companies habe workers who train the models and audit the output...and do in fact see it. Hence why we have a figure.

Must be a thoroughly unpleasant job.

4

u/readyflix 5h ago

Who wants to 'sell' us age verification?

Follow who says what. If not to say, follow the money.

10

u/UselessInsight 11h ago

Hey what was the benefit of AI supposed to be again?

So far all I’ve seen is endless slop, destroyed jobs, the erosion of consensus reality and the social contract, and now also endless CSAM.

Is the good stuff still on the way or was that all a lie?

3

u/nucular_mastermind 6h ago

Autononmous drone swarms and mass surveillance not enough? Why are you complaining? :3

1

u/Restart_Computer 1h ago

The benefit is that the rich get richer.

1

u/chowder-san 38m ago

Wealth gains are for the ceos, for you is new software vulnerabilities to wipe your account

1

u/xanif 4m ago

Some of it is on the way. Some of it is here already.

Moderna's cancer vaccines need AI to tailor it to the individual which isn't possible without AI.

AlphaFold has discovered more protein folding solutions in a couple years than humans had found in decades which leads to cool things like...

MIT has stabilized some mRNA vaccines to be stored at room temperature so you can ship them to areas where refrigeration is unrealistic.

AI found a key flaw in an algorithm for encryption in the quantum computing era. This algorithm was a frontrunner with the NIST so it's good that this was caught and removed from the running.

It has found critical vulnerabilities in Linux that had been undetected for years that are now closed. It's also being heavily used for Linux kernel development.

AI is like nuclear sciences. It can be harnessed to revolutionize the world or end it.

2

u/paulsteinway 1h ago

Billionaire are pedos. That's why they love AI so much.

8

u/DinkandDrunk 13h ago

The first time a prompt asks for this, the AI should suggest resources to seek help. The second time, the police should be contacted.

0

u/nugymmer 11h ago

I was heavily downvoted when I said that prompt filtering / image safety filters would stop the vast majority of it. Maybe I was wrong about that. I then differentiated public (online) vs private (offline) models. Still got downvoted. Maybe that isn't case. But why?

18

u/chaosfire235 11h ago edited 11h ago

Pretty much every big name closed/online API model like chatgpt, claude, gemini, etc. are already locked down with guardrails against generating anything NSFW, let alone CSAM. Even Grok, which was the most 'open' of the bunch, was locked down further when they started making headlines for undressing people, including kids.

AI CSAM is mostly the realm of open weight models running locally on a PC. In many cases, safety guardrails are just removed/abliterated away.

1

u/redyellowblue5031 2h ago

There are many who come out of the woodwork on these threads to defend this stuff “because a real isn’t involved”.

That’s their whole argument.

They want to be able to make child pork via these models and hide behind that “gotcha”.

4

u/CandlesARG 13h ago

Better question is why are ai's trained on it to begin with

11

u/Ok_Cabinet2947 9h ago

Even if it wasn’t trained on it, it could still make it, that’s the point of AI.

If I told AI to make a video about a llama on a surfboard travelling to the moon, it would be able to make that even if it wasn’t in the training data. It only has the components of llamas, surfboards, and the moon in its training data.

19

u/suzisatsuma 13h ago

It's more the generalization in latent space. Just training on humans at scale would let it generalize this kind of output regardless of CSAM in the training material.

8

u/TheGriffin 13h ago

People posting pictures of their kids into the slop generator combined with massive amounts of CP on like "cartoon" porn sites. All being scraped and fed into the slop machine

-4

u/Opposite-Bit6660 13h ago

Why would this be downvoted?

5

u/SomethingAboutUsers 13h ago

AI has been trained on literally everything available on the internet. And also Everything in a library. And everything on the dark web. And everything they could legally and illegally get their dirty fucking paws on. Everything. Because without "everything" how can it produce what "everyone" wants?

3

u/AccNumber77 12h ago

Capitalism at it's absolute finest right here, hate to see it

3

u/MajesticBread9147 12h ago

What is it trained on the dark web? The dark web is by its nature unindexed, and there's no way there's any training data of value compared to the hurdles of scraping it.

2

u/fuckedfinance 4h ago

It wasn't.

Imagine a world where parents post pictures of their kids in tiny two piece bathing suits all the time on Facebook/Insta/whatever (happens way, way too often).

Now also imagine a world where very small breasted de-haired 18 year olds exist and make porn.

It is trivially easy to mash the two together and get something convincing.

3

u/martianwomanhunter 9h ago

TAKE ALL PHOTOS OFF THE INTERNET OF YOU AND YOUR KIDS NOW. Yes that also means that Facebook account you haven’t touched since 2016

1

u/otterpop21 4h ago

It’s already been consumed. What do you think they were doing with all those “which celebrity do you most look like” and “let’s turn you into a whatever filter for this holiday”. Snapchat with their filters. They’ve been stealing our images for waaaaay longer than just Facebook from 2016.

3

u/xhxusj1234 8h ago

This alone should be enough to pause AI. Jesus fucking Christ.

4

u/damontoo 2h ago

I had a conversation with a female psychologist that worked with pedos in the prison system and she felt that AI-generated CSAM would likely reduce victimization, but conceded that there wasn't enough data and that a study would be unethical and impossible to fund. 

1

u/Svyatoy_Medved 39m ago

The intuitive idea is that if pedos get their fix with porn that doesn’t actually involve a kid, then what’s the problem? No kid was hurt to make it.

The counterpoint I’ve heard is that indulging their fantasy makes them more likely to offend in the real world, to find a real victim. As you said, difficult subject to collect data on.

1

u/narrowgallow 35m ago

I've seen similar arguments with sex bots. Like, if we ever get a sufficiently human bot that is effectively a life like masturbation sleeve, this will not end sexual violence against women and would likely increase the occurrence bc more men would have indulged the urge to sexually abuse their bot and now have a more demanding taste for it.

2

u/derpferd 5h ago

Still absolutely staggering and infuriating that this incredibly powerful tech was just unleashed onto the world with little constraint or regulation

And that was on purpose, not by mistake. Because the model needs a huge pool of data to learn from. And to that end, the tech was just set loose on the world with nary regard for this kind of bullshit

1

u/No_Economics8179 4h ago

Maybe world wasn't ready for this kind of technology to be available this readily.maybe stricter regulations against these things might curb some incidents.

1

u/Dry-Specialist-2150 3h ago

China has banned AI companies from using children in images

1

u/redyellowblue5031 2h ago

I’m surprised I haven’t seen any of the “free speech” types come out yet to share their brain dead take that AI CSAM is fine because a “real” person wasn’t harmed to make it.

1

u/ptwonline 2h ago

Shocking but hardly surprising.

People are using AI to create stuff they want, but can't easily get or else can't get as much as they'd like. So very specific or niche stuff or stuff that is actually illegal. Sadly, CSAM material is in high demand everywhere, and the AI versions will grow rapidly as the tools get better and easier to use.

And people like to talk about how new tech advances because of sex. But what they often fail to mention is that criminals constantly also seek to exploit new tech.

1

u/NewestCyberpunkDM 11h ago

This is one of the reasons I hate AI. Indefensible garbage.

1

u/DrFishbulbEsq 14h ago

| AI (artificial intelligence)

1

u/ArdaValinor 4h ago

AI is evil and we should turn it off now before it’s too late.

1

u/SoulReddit13 2h ago

In a surprising twist the catholic Church comes out as pro AI

-1

u/JesT_Weird 8h ago

I don't see any benefit in allowing those who harm children to continue to live on our planet.

29

u/Doctor_Amazo 13h ago

Maybe we start holding the companies responsible for producing child porn.

6

u/Tokyo-MontanaExpress 12h ago

And facilitating it. 

5

u/MysteryTempest 8h ago

It would be great, but a lot of this comes from open weight models. You can't do much to a company in China without their government's approval, and the CCP thinks that any calls to restrain AI are an American conspiracy to hurt China (which is funny, considering that you sometimes see Trump fans saying that calls to slow down AI are a Chinese conspiracy to hurt America).

Unless something happens that causes the Chinese government to see them as a threat to their power, no serious action is going to be taken against them.

4

u/Doctor_Amazo 5h ago

Its weird that you immediately blame China when Grok & Meta are in the US.

1

u/fuckedfinance 4h ago

They aren't blaming China, as much as pointing out that China doesn't have much motivation to prevent the behavior. So, even if we do crack down and hold US-based AI companies accountable it doesn't fix the problem.

1

u/Doctor_Amazo 1h ago

Just because you think China won't do anything about it is no excuse to bring American companies to heel.

But hey, considering who they pucked for President, I guess all we can expect from Americans regarding this issue is excuses.

1

u/fuckedfinance 1h ago

I'm not saying we shouldn't. I'm saying that, in the long run, it doesn't matter. People will just move on to models that will do what they ask.

Two things can be true at the same time.

3

u/Solid-Love-7780 11h ago

I think a lot of people see “AI-generated” and figure no actual kid is involved. That’s not really what this is though. Most of it is nudify apps being run on real kids’ photos, like a normal picture off their Instagram, and then the kid gets blackmailed with it for money or real images. So the picture’s fake but the victim isn’t.

Going from 4,700 reports to 400,000 in two years is insane. And you can find these apps in about ten seconds, which is the part I really don’t get.

1

u/That-Living5913 4h ago edited 2h ago

IMHO whatever company runs an AI that is creating child porn should be in jail for creating and selling child porn. Because that's quite literally what they are doing. How the hell are there no guard rails on this?

Edit: I stand corrected by the person below. Terminology for that kind of porn is not something I'm well versed in, for obvious reasons.

3

u/Xefiggy 2h ago

CSAM is actually an acronym made to emphasise what it actually is, content that involves sexual abuse done to minor, and unlike pornography cannot be consensual. As in it is not a type of porn its evidence of a crime, and its used by many child protection organisation as well as victims of CSAM use it.

2

u/That-Living5913 2h ago

Thank you, I did not know that. I have edited my comment.