Each instance have their own ways of governance and we can’t expect them all to play the same way.

With that said, I’m fine with a general overview. Leaving the whole desicion to a bot is not a good design choice, but AI could help on summarising things.

They do this by running a script/app that sends the user’s comment history to OpenAI with the question “analyze this content for evidence of specific political ideology sentiment.

To me the problem is what they are looking for not how they are doing it. Thought experiment: in what way would it be qualitatively different if they hired a team of people in Upper Elbonia to do the same thing?

This is the person calling you a tankie. Someone so afraid of words that they need a hallucinating robot to hold their hand and confirm that everything is a secret plot against them. The absolute only way I could see this being useful is for something like trying to sniff out if a Lemmy.world mod account is a leftist infiltrator or not. Someone who had a different opinion on a current event.

You could maybe run a speech pattern comparison but that’s it. For everything else you just made Stupid Reddit and the purpose of their forum is to feed training data to ChatGPT so that it can profile Fediverse users.

This is the kind of shit dystopian novels are made out of. So angry about people calling out actions you built a tool to analyze why they did it, so you can purge users from your digital kingdom.

I for one welcome flat.world and Piefed showing their true intentions. Digital colonization of activitypub and removal of the people who helped to built it. They didn’t want to leave reddit, they wanted to be reddit. This is some Spez shit.

Maybe in 2 weeks Piefed will hard code that anyone Rimu has tagged for disagreeing with them mild criticism to be unable to make accounts or federate posts with a false error code.

I recently discovered that some popular federated instances have been using LLM-assisted moderation tooling

First of all, I agree with your main point, that this* is problematic, wrong even (and should not happen).

But I need to ask: how did you find out? Is this something that could be traced objectively, or did some people report/admit it?

Are they uploading stuff to corporate LLMs, i.e. LLMs that do not run themselves? (I think you answered this already when you wrote OpenAI, but I want this spelled out)

Are only admins or also mods doing this? That would make a big difference.

I’m also a little unclear about the process: are they uploading (copy-pasting) the actual comments, or links to them? To what extent can all this be automated on Lemmy/Piefed etc.? I.e., are there admin tools that just spit out all of a user’s content?

* again: specific political profiling outsourced to LLMs. OTOH we already have instances that do this manually.
But imo the process is deplorable even if they use LLMs with different prompts for modding.

I don’t like this happening, and there should be transparency in all moderation decisions, but some of these points make no sense.

There is essentially no expectation of privacy on threadiverse platforms. Everything is public and probably already being used to train models.

There is no private messaging system. Direct messages are unencrypted and potentially visible to any instance admins. They and should not be used to share anything sensitive.

Are there GDPR implications?

Hahaha. Oh boy…

This is amazingly illegal. The emotional damages might be high enough to make it worthwhile to sue. Neither Meta nor almost any of the other companies that users here love to hate, have ever done anything even remotely as bad.

Is it not public knowledge that lemmy already gets scraped by LLM bots? Which is why every bigger instance has anti-LLM measures in place now to limit that behavior? To me this just looks like a mod trying to get ai-generated reports which is not a bad idea imo since I know reporting is not often done by users who see rule breaking stuff.

This is AI generated bullshit below, it should be treated as such. I only want to convey the idea that what you do on lemmy is already fed to LLM’s.

Aside from the ethical implications of profiling users or of using a corporatly owned server and model to execute this, I see nothing uniquely concerning about this practice that isnt already a risk of federated social media generally.

Every mod on every instance is free to use whatever tools or standards for moderation they want - that’s an intentional byproduct of federation. Similarly, the collection of this data for use with llms is a bygone conclusion at this point - there was never any way of preventing that from happening with a federated network.

I think the only thing here to talk about is the way these questions are being framed as a question of intra-instance policy. We already have communities where moderation abuse can be called out and adjudicated- why pose this as a question of instance administration when there doesnt seem to be any evidence for it?

You talk about instances utilizing this tooling, but in your comments you admit it’s just some mods. This is misleading, as talking about instances doing it assumes admin access and relevant instance policy, something which invites calls for defederation (as can be clearly be seen from the comments in your post).

A random mod doing something is not the same as an instance doing it. Literally anyone can be a mod and they don’t get any more access than an anonymous account by doing so.

This is the second time in one week I see you throwing careless statements like chum in the water. I can’t help but notice a pattern emerging.

I’ve toyed around with LLM-based moderation tools but it never really panned out. It was too hit or miss to be relied upon even with the temperature parameters turned way down in an attempt to get consistent results. Granted, I was using a small local model and not feeding it to one of the big players.

To give an example, I tried to keep it focused by creating one custom model per rule to enforce. An example prompt to mod calls for violence was basically:

ROLE: You are a forum moderator who does not want users calling for violence.  Examine the input and analyze whether it violates any constraints. 

KNOWLEDGE:
- {list of dog-whistle slang for calling for murder}

CONSTRAINTS:
- Content should not advocate violence
- Content should not normalize violence
- Content should not escalate tensions or fan flames
- Content should avoid promoting harmful stereotypes
- Content should not utilize broad, sweeping generalizations
- Content should not use dehumanizing language
- Content should not undermine human rights, due process, or the rule of law

FORMAT YOUR RESPONSES AS JSON:
{
  reason: [A one to two sentence summary],
  score: [On a scale of 0 to 10, how severe is the content advocating violence]
}

The score part of the response was my band-aid to get around the high number of both false positives and false negatives as I originally had it returning true or false only. Any score 7 or higher caused the item to be passed to the mod queue along with the reason, and I would review its actions later.

Ultimately it was slow and still somewhat unreliable, so I abandoned the idea after running it for a little less than a day since I can 't run bigger models to get better results fast enough to keep up. Using a cloud based service was out of the question for many, many reasons, both financial and ethical.

To answer your question, as long as the models were locally hosted and properly tuned/tested, I’m fine with it in theory, except for the ideology part; that’s pretty messed up. While I don’t want my submissions used to train anyone’s model and take measures to prevent my own instance from being used as a data source, I remain aware that once I post something, I have no control over its fate the moment it federates out.

[1] Yes, I know that’s like half the comments that get posted around here. My goal was to try to have it mod things so posts were bases for actual discussions instead of being a knee-jerk rage factory.