this post was submitted on 21 Apr 2024
280 points (97.9% liked)

World News

39032 readers
2275 users here now

A community for discussing events around the World

Rules:

Similarly, if you see posts along these lines, do not engage. Report them, block them, and live a happier life than they do. We see too many slapfights that boil down to "Mom! He's bugging me!" and "I'm not touching you!" Going forward, slapfights will result in removed comments and temp bans to cool off.

We ask that the users report any comment or post that violate the rules, to use critical thinking when reading, posting or commenting. Users that post off-topic spam, advocate violence, have multiple comments or posts removed, weaponize reports or violate the code of conduct will be banned.

All posts and comments will be reviewed on a case-by-case basis. This means that some content that violates the rules may be allowed, while other content that does not violate the rules may be removed. The moderators retain the right to remove any content and ban users.


Lemmy World Partners

News !news@lemmy.world

Politics !politics@lemmy.world

World Politics !globalpolitics@lemmy.world


Recommendations

For Firefox users, there is media bias / propaganda / fact check plugin.

https://addons.mozilla.org/en-US/firefox/addon/media-bias-fact-check/

founded 1 year ago
MODERATORS
 

A sex offender convicted of making more than 1,000 indecent images of children has been banned from using any “AI creating tools” for the next five years in the first known case of its kind.

Anthony Dover, 48, was ordered by a UK court “not to use, visit or access” artificial intelligence generation tools without the prior permission of police as a condition of a sexual harm prevention order imposed in February.

The ban prohibits him from using tools such as text-to-image generators, which can make lifelike pictures based on a written command, and “nudifying” websites used to make explicit “deepfakes”.

Dover, who was given a community order and £200 fine, has also been explicitly ordered not to use Stable Diffusion software, which has reportedly been exploited by paedophiles to create hyper-realistic child sexual abuse material, according to records from a sentencing hearing at Poole magistrates court.

you are viewing a single comment's thread
view the rest of the comments
[–] Tippon@lemmy.dbzer0.com 16 points 7 months ago (4 children)

Where does the training data come from to create indecent images of children?

[–] Dran_Arcana@lemmy.world 51 points 7 months ago* (last edited 7 months ago) (40 children)

It doesn't need csam data for training, it just needs to know what a boob looks like, and what a child looks like. I run some sdxl-based models at home and I've observed it can be difficult to avoid more often than you'd think. There are keywords in porn that blend the lines across datasets ("teen", "petite", "young", "small" etc). The word "girl" in particular I've found that if you add that to basically any porn prompt gives you a small chance of inadvertently creating the undesirable. You have to be really careful and use words like "woman", "adult", etc instead to convince your image model not to make things that look like children. If you've ever wondered why internet-based porn generators are on super heavy guardrails, this is why.

[–] Tippon@lemmy.dbzer0.com 3 points 6 months ago (1 children)

Thanks for the reply, it's given me a good idea of what's most likely happening :)

It's a shame that the rest of the thread went to shit, but unfortunately it's an emotional topic, and brings out emotional responses

[–] Dran_Arcana@lemmy.world 2 points 6 months ago

Always happy to try and productively add to someone's learning.

[–] PotatoKat@lemmy.world 3 points 6 months ago (1 children)
[–] Dran_Arcana@lemmy.world 2 points 6 months ago (1 children)

I'm not going to say that csam in training sets isn't a problem. However, even if you remove it, the model remains largely the same, and its capabilities remain functionally identical.

[–] PotatoKat@lemmy.world 0 points 6 months ago (1 children)

At that point it's still using photos of children to generate csam even if you could somehow assure the model is 100% free of csam

[–] Dran_Arcana@lemmy.world 1 points 6 months ago (1 children)

That would be true, it'd be pretty difficult to build a model without any pictures of children at all, and then try and describe to the model how to alter an adult to make a child. Is anyone asking for that though? To make it illegal to have regular pictures of children in these datasets?

[–] PotatoKat@lemmy.world 0 points 6 months ago (1 children)

No but it is a reason why generating csam should be illegal. You're using data trained on pictures of real kids

[–] Dran_Arcana@lemmy.world 0 points 6 months ago (1 children)

I'm not arguing whether or not it should be legal, I was just offering my first hand experience in regards to the capabilities of these local models since people seem to be confused as to how this actually works.

[–] PotatoKat@lemmy.world 0 points 6 months ago (1 children)

Is anyone asking for that though? To make it illegal to have regular pictures of children in these datasets?

I was responding to this part of your comment which directly refers to legality

[–] Dran_Arcana@lemmy.world 1 points 6 months ago

I guess I just misunderstood what you were arguing then. For posterity: I believe datasets containing children is fine, datasets containing csam is not, and the legality of generating csam should be left up to psychologists on whether or not it is a societal net benefit. Whichever way is better for children that exist is my vote.

load more comments (38 replies)
[–] Daxtron2@startrek.website 28 points 7 months ago (1 children)

The whole point of diffusion models is that you can generate new concepts using training data. Models trained on any nsfw images can combine those concepts with any of its non-nsfw concepts. Of course, that's not to say there isn't CSAM in any training data, because there objectively has been in the past, but there doesn't need to be any to generate it.

[–] Tippon@lemmy.dbzer0.com 3 points 6 months ago (1 children)

Thanks for the reply, that makes a lot of sense :)

[–] Daxtron2@startrek.website 3 points 6 months ago

Thanks for not being a dick! I aim to inform

[–] Turun@feddit.de 10 points 6 months ago (1 children)

Ai is able to fill in the last field in a table like "Old / young" vs "Clothed / naked" when given three of the four fields.

[–] randomaside@lemmy.dbzer0.com 5 points 7 months ago

Please reiterate your statement but instead using the "goose chase meme" format.