this post was submitted on 28 Aug 2023
1736 points (97.9% liked)

Lemmy.World Announcements

29163 readers
58 users here now

This Community is intended for posts about the Lemmy.world server by the admins.

Follow us for server news 🐘

Outages πŸ”₯

https://status.lemmy.world/

For support with issues at Lemmy.world, go to the Lemmy.world Support community.

Support e-mail

Any support requests are best sent to info@lemmy.world e-mail.

Report contact

Donations πŸ’—

If you would like to make a donation to support the cost of running this platform, please do so at the following donation URLs.

If you can, please use / switch to Ko-Fi, it has the lowest fees for us

Ko-Fi (Donate)

Bunq (Donate)

Open Collective backers and sponsors

Patreon

Join the team

founded 2 years ago
MODERATORS
 

Hello everyone,

We unfortunately have to close the !lemmyshitpost community for the time being. We have been fighting the CSAM (Child Sexual Assault Material) posts all day but there is nothing we can do because they will just post from another instance since we changed our registration policy.

We keep working on a solution, we have a few things in the works but that won't help us now.

Thank you for your understanding and apologies to our users, moderators and admins of other instances who had to deal with this.

Edit: @Striker@lemmy.world the moderator of the affected community made a post apologizing for what happened. But this could not be stopped even with 10 moderators. And if it wasn't his community it would have been another one. And it is clear this could happen on any instance.

But we will not give up. We are lucky to have a very dedicated team and we can hopefully make an announcement about what's next very soon.

Edit 2: removed that bit about the moderator tools. That came out a bit harsher than how we meant it. It's been a long day and having to deal with this kind of stuff got some of us a bit salty to say the least. Remember we also had to deal with people posting scat not too long ago so this isn't the first time we felt helpless. Anyway, I hope we can announce something more positive soon.

you are viewing a single comment's thread
view the rest of the comments
[–] dragontamer@lemmy.world 148 points 1 year ago (9 children)

Not that I'm familiar with Rust at all, but... perhaps we need to talk about this.

The only thing that could have prevented this is better moderation tools. And while a lot of the instance admins have been asking for this, it doesn’t seem to be on the developers roadmap for the time being. There are just two full-time developers on this project and they seem to have other priorities. No offense to them but it doesn’t inspire much faith for the future of Lemmy.

Lets be productive. What exactly are the moderation features needed, and what would be easiest to implement into the Lemmy source code? Are you talking about a mass-ban of users from specific instances? A ban of new accounts from instances? Like, what moderation tool exactly is needed here?

[–] Agamemnon@lemmy.world 116 points 1 year ago (3 children)

Speculating:

Restricting posting from accounts that don't meet some adjustable criteria. Like account age, comment count, prior moderation action, average comment length (upvote quota maybe not, because not all instances use it)

Automatic hash comparison of uploaded images with database of registered illegal content.

[–] dragontamer@lemmy.world 65 points 1 year ago

On various old-school forums, there's a simple (and automated) system of trust that progresses from new users (who might be spam)... where every new user might need a manual "approve post" before it shows up. (And this existed in Reddit in some communities too).

And then full powers granted to the user eventually (or in the case of StackOverlow, automated access to the moderator queue).

[–] MossyFeathers@pawb.social 11 points 1 year ago (3 children)

What are the chances of a hash collision in this instance? I know accidental hash collisions are usually super rare, but with enough people it'd probably still happen every now and then, especially if the system is designed to detect images similar to the original illegal image (to catch any minor edits).

Is there a way to use multiple hashes from different sources to help reduce collisions? For an example, checking both the MD5 and SHA256 hashes instead of just one or the other, and then it only gets flagged if both match within a certain degree.

[–] TsarVul@lemmy.world 26 points 1 year ago (1 children)

Traditional hash like MD5 and SHA256 are not locality-sensitive. Can't be used to detect match with certain degree. Otherwise, yes you are correct. Perceptual hashes can create false positive. Very unlikely, but yes it is possible. This is not a problem with perfect solution. Extraordinary edge cases must be resolved on a case by case basis.

And yes, simplest solution must be implemented first always. Tracking post reputation, captcha before post, wait for account to mature before can post, etc. The problem is that right now the only defense we have access to are mods. Mods are people, usually with eyeballs. Eyeballs which will be poisoned by CSAM so we can post memes and funnies without issues. This is not fair to them. We must do all we can, and if all we can includes perceptual hashing, we have moral obligation to do so.

[–] MossyFeathers@pawb.social 5 points 1 year ago* (last edited 1 year ago) (1 children)

Something I thought about that might be helpful is if mods had the ability to add a post delay on a community basis. Basically, the delay would be moderator adjustable, but only moderators and admins would be able to see the post for X number of minutes after being posted. It'd help for situations like ongoing attacks where you don't necessarily want to have to manually approve posts, but you want a chance to catch any garbage before the post goes public.

Edit: and yeah, one of the reasons I'm aware that perceptual hashes can have collisions is because a number of image viewers/cataloging tools like xnview mp or hydrus network use hash collisions to help identify duplicate images. However, I've seen collisions between unrelated images when lowering the sensitivity which is why I was wondering if there was a way to use multiple hashing algorithms to help reduce false positives without sacrificing the usefulness of it.

[–] Natanael@slrpnk.net 4 points 1 year ago

Or just making posts approval only with a mod queue

[–] fkn@lemmy.world 5 points 1 year ago

I'm surprised this isn't linked, there are services that does this for you.

And they are free.

https://blog.cloudflare.com/the-csam-scanning-tool/

[–] MsPenguinette@lemmy.world 3 points 1 year ago

I beleive there are several readily available databases of hashes of csam material for exactly this kind of scanning. Looks like there are some open source ones.

Some top results: https://github.com/topics/csam

This looks to be the top project: https://prostasia.org/project/csam-scanning-plugins/

[–] BuddyTheBeefalo@lemmy.ml 2 points 1 year ago

Could they not just change one pixel to get another hash?

[–] TsarVul@lemmy.world 40 points 1 year ago (2 children)

I guess it'd be a matter of incorporating something that hashes whatever it is that's being uploaded. One takes that hash and checks it against a database of known CSAM. If match, stop upload, ban user and complain to closest officer of the law. Reddit uses PhotoDNA and CSAI-Match. This is not a simple task.

[–] diffuselight@lemmy.world 27 points 1 year ago (1 children)

None of that really works anymore in the age of AI inpainting. Hashes / Perceptual worked well before but the people doing this are specifically interested in causing destruction and chaos with this content. they don’t need it to be authentic to do that.

It’s a problem that requires AI on the defensive side but even that is just going to be eternal arms race. This problem cannot be solved with technology, only mitigated.

The ability to exchange hashes on moderation actions against content may offer a way out, but it will change the decentralized nature of everything - basically bringing us back to the early days of the usenet, Usenet Death Penaty, etc.

[–] dragontamer@lemmy.world 47 points 1 year ago* (last edited 1 year ago) (2 children)

Not true.

A simple CAPTCHA got rid of a huge set of idiotic script-kiddies. CSAM being what it is, could (and should) result in an immediate IP ban. So if you're "dumb" enough to try to upload a well-known CSAM hash, then you absolutely deserve the harshest immediate ban automatically.


You're pretty much like the story of the economist who refuses to believe that $20 exists on a sidewalk. "Oh, but if that $20 really existed on the sidewalk there, then it would have been arbitraged away already". Well guess what? Human nature ain't economic theory. Human nature ain't cybersecurity.

Idiots will do dumb, easy attacks because they're dumb and easy. We need to defend against the dumb-and-easy attacks, before spending more time working on the harder, rarer attacks.

[–] AustralianSimon@lemmy.world 11 points 1 year ago* (last edited 1 year ago) (1 children)

You don't get their ip when they post from other instances. I'm surprised this hasn't resulted in defed.

[–] anlumo@lemmy.world 5 points 1 year ago (1 children)

Well, my home instance has defederated from lemmy.world due to this, that's why I had to create a local account here.

[–] AustralianSimon@lemmy.world 2 points 1 year ago* (last edited 1 year ago)

I mean defedding the instances the CSAM is coming from but also yes.

[–] rolaulten@startrek.website 3 points 1 year ago

I'm sorry but you don't want to use permanent IP bans. Most residential circuits are DHCP meaning banning via IP only has a short term positive effect.

That said automatic scanning of known hashes, and automatically reporting to relevant authorities with relevant details should be doable (provided there is a database somewhere - I honestly have never looked).

[–] Touching_Grass@lemmy.world 11 points 1 year ago (3 children)

Couldn't one small change in the picture change the whole hash?

[–] TsarVul@lemmy.world 20 points 1 year ago (2 children)

Good question. Yes. Also artefacts from compression can fuck it up. However hash comparison returns percentage of match. If match is good enough, it is CSAM. Davai ban. There is bigger issue however for developers of Lemmy, I assume. It is a philosophical pizdec. It is that if we elect to use PhotoDNA and CSAI Match, Lemmy is now at the whims of Microsoft and Google respectively.

[–] Lmaydev@programming.dev 10 points 1 year ago (1 children)

Honestly I'd rather that than see shit like this any day.

[–] Serinus@lemmy.world 11 points 1 year ago* (last edited 1 year ago) (1 children)

The bigger thing is that hash detection tools don't want to give access to just anyone, and just anyone can run a Lemmy instance. The concern is that you're effectively giving the CSAM people a way to know if they'll be detected.

Perhaps they can allow some of the biggest Lemmy instances to use the tech, but I wouldn't expect it to be available to everyone.

[–] shagie@programming.dev 5 points 1 year ago

Facebook and Reddit don't have local CSAM detection but rather use Google's APIs.

This isn't something that any average user can get access to. Even the largest Lemmy instances are small compared to Reddit and Facebook... and they don't have local testing either.

Part of this is also a "this isn't just detecting and blocking but also automated reporting".

Furthermore, Lemmy is AGPL, and providing a Lemmy instance with an implementation that would run the risk that it wouldn't be able to remain closed source (AGPL license violation).

[–] what_is_a_name@lemmy.world 8 points 1 year ago (1 children)

Mod tools are not Lemmy. Give admins and mods an option. Even a paid one. Hell. Admins of Lemmy.world could have us donate extra to cover costs of api services.

[–] TsarVul@lemmy.world 16 points 1 year ago

I agree. Perhaps what Lemmy developers can do is they can put slot for generic middleware before whatever the POST request is in Lemmy API for uploading content? This way, owner of instance can choose to put whatever middleware for CSAM they want. This way, we are not dependent on developers of Lemmy for solution to pedo problem.

[–] Nollij@sopuli.xyz 17 points 1 year ago (1 children)

If they hash the file binary data, like CRC32 or SHA, yes. But there are other hash types out there, which are more like "fingerprints" of an image. Think of how Shazam or Sound Hound can recognize a song playing, despite the extra wind, static, etc that's present. There are similar algorithms for images/videos.

No idea how difficult those are to implement, though.

[–] Railcar8095@lemm.ee 7 points 1 year ago

There are FOSS applications that can do that (czkawka for example). What I'm not sure it's if the specific algorithm used is available and, more importantly, if the csam hashes are available for general audiences. I would assume if they are any attacker could check first and get the right amount of changes.

[–] reverendsteveii@lemm.ee 4 points 1 year ago

One bit, in fact. Luckily there are other ways of comparing images without actually showing them to human eyes that allow you to calculate a percentage of similarity.

[–] AustralianSimon@lemmy.world 14 points 1 year ago (1 children)

Reddit had automod which was highly configurable.

[–] Serinus@lemmy.world 13 points 1 year ago* (last edited 1 year ago)

The best feature the current Lemmy devs could work on is making the process to onboard new devs smoother. We shouldn't expect anything more than that for the near future.

I haven't actually tried cloning and compiling, so if anyone has comments here they're more than welcome.

[–] gabe@literature.cafe 11 points 1 year ago

I think having a means of viewing uploaded images as an admin would be helpful, as well disabling external image caching. Like an "uploaded" gallery for admins to view that can potentially hook into Photodna/CSAI-Match or whatever.

[–] MrPoopyButthole@lemmy.world 10 points 1 year ago

I think it would be an AI autoscan that flags some posts for mod approval before they show up to the public and perhaps more fine-grained controls for how media is posted like for instance only allowing certain image hosting sites and no directly uploaded images.

[–] SubArcticTundra@lemmy.ml 6 points 1 year ago* (last edited 1 year ago)

I was just discussing this under another post and turns out that the Germans have already developed a rule-based auto moderator that they use on their instance:

https://github.com/Dakkaron/SquareModBot

This could be adopted by lemmy.world by simply modifying the config file

[–] snowe@programming.dev 4 points 1 year ago (1 children)

That statement is just outright wrong though. They could easily use CloudFlares CSAM monitoring and it never would have been a problem. A lot of people in these threads, including admins, have absolutely no idea what they’re talking about.

[–] sunaurus@lemm.ee 2 points 1 year ago (1 children)

Cloudflare CSAM protection is not available outside of the US, unfortunately.

[–] snowe@programming.dev 1 points 1 year ago

There are several other solutions including ones from Microsoft and Facebook.

[–] BURN@lemmy.world 3 points 1 year ago (1 children)

Probably hashing and scanning any uploaded media against some of the known DBs of CSAM hashes.

Iirc that’s how Reddit/FB/Insta/Etc. handle it

[–] shagie@programming.dev 16 points 1 year ago (1 children)

They're sent to a 3rd party that does the checks. For example https://developers.cloudflare.com/cache/reference/csam-scanning/

The actual DB of hashes isn't released to the public as it would enable those who traffic in such content to use it to find the material that doesn't match much more easily.

https://protectingchildren.google/#tools-to-fight-csam

That appears to be the one that Facebook and Reddit use.

[–] CoderKat@lemm.ee 1 points 1 year ago

The sad thing is that all we can usually do is make it harder for attackers. Which is absolutely still worth doing, to be clear. But if an attacker wants to cause trouble badly enough, there's always ways around everything. Eg, image detection can be foiled with enough transformation, account age limits can be gotten past by a patient attacker. Minimum karma can be botted (even easier than ever with AI) and Lemmy is especially easy to bot karma because you can just spin up an instance with all the bots your heart desires. If posts have to be approved, attackers can even just hotlink to innocent images and then change the image after it's approved.

Law enforcement can do a lot more than we can, by subpoenaing ISPs or VPNs. But law enforcement is slow and unreliable, so that's also imperfect.