Hi all, I’m a Lemmy FOSS app contributor that’s made a couple of tools for people starting small instances including Lemmy Community Seeder (LCS) for building content on new server’s All Feeds and Lemmy Post Purger (LPP) for clearing old posts on smaller instances.
Today I’m releasing Lemmy Defederation Sync (LDS). When launching a new Lemmy instance, administrators may not understand the necessity of defederation with problem instances. Using LDS, you can sync your instance’s “blocked instance” list with that of another server(s) whose admins you trust.
Shared Fediblocks are a plague. Get added to one by somebody that personally dislikes you and every instance that uses the block will unknowingly block you.
Don’t worry, highly popular instances won’t get on the block list and small instances don’t matter enough for it to be a problem. Power always favours the powerful.
The issue is there’s no oversight here, or at the very least, no oversight that we can trust to be impartial.
When parties form a federation, it’s usually with a signed agreement, to maintain the integrity of the federation and keep it together. Shared standards, rules, ideals, regulations, etc. All I’m seeing here are parties trying to carve out a sub-federation, and with no neutral oversight, it’s just going to become a “people like me” circle.
And I guarantee at some point people are going to start thinking “Hey, if this instance isn’t subscribing to the shared blocklist, that must mean they want those other instances around, and therefore they deserve to be added to the list”
Then just like that, the fediverse has a defacto authority. Instead of one sepz, we have a council of them from the largest instances.
And I guarantee at some point people are going to start thinking “Hey, if this instance isn’t subscribing to the shared blocklist, that must mean they want those other instances around, and therefore they deserve to be added to the list”
Already is happening on the Mastodon/Pleroma side of the Fediverse. Instances getting de-federated because they dared allowed their users to see posts from “bad” instances
Lol, sometimes I feel like the ‘fediverse’ (hate that name BTW) is filled with the worst people.
Lemmy is built as a single instance with federation capabilities as an after thought. The biggest clue to this and the reason why it will always re-centralize is that communities are server bound. There is no fediverse wide community and they is by design. Read this closed issue
This is a terrible idea that steers lemmy into being an echo chamber. Let admins use their own judgement.
But isn’t that what this program does? It allows you to choose an instance with admins that you trust. And those who want to review every single one manually can still do that. I’d love this tool. The ones setting up these servers aren’t stupid. They can use their judgement and use this tool if they want!
And those who want to review every single one manually can still do that.
But will they? This tool promotes blindly trusting another instance block list without due diligence from the admin.
Why not set up your own instance?
Don’t know. That’s up to them. The problem is not the tool but the unreflected trust in blocklists. The Internet is huge, if Lemmy takes off so will the number of instances. The amount of decisions needed to get a legal instance working in many countries will be insurmountable. I’d rather piggyback on someone I do trust as a rough basis. It won’t be perfectly tuned with my informed decision but the alternative is me not setting the server up. The list of defederation can be reviewed. If you’re careful about the template its not blind trust. Much in the same way as using FOSS software without understanding all components isn’t blind trust if you’re careful about the source and verify downloads. It’s not perfect but the alternative is not using the software at all.
deleted by creator
This sounds a bit like how to bring the shared blocklists from Twitter to Lemmy. Those were a disaster on Twitter, and I don’t expect it’ll end any better here either…
Please don’t use tools like this. Manually curate instances you feel the need to defederate with. The Fediverse was built on a model not unlike that of email. You wouldn’t just randomly block whole email providers willy-nilly, so you shouldn’t do so here on the Fediverse either.
As a sysadmin that’s a terrible analogy:
- Shared email Blocklists are the norm, not the exception
- As a professional IT admin I would absolutely blackhole any vile hives of scum and villainy rather than dealing with their BS. If someone is going to do that for me I’ll use the tool.
Also a sysadmin.
> Shared email Blocklists are the norm, not the exception
Shared blocklists in IT are managed by industry professionals for the purpose of safety from malicious activity and there are vetted processes for being removed from days lists. False positives happen, but you aren’t hung out to dry if you get hit, you just go through the process and clear your name.
Most of this “Fediblock” nonsense is several orders of magnitude less reliable, and filled with toxic people pursing personal grudges. There’s no process to clear your name, and I’ve personally watched multiple admins and their entire communities be publicly mocked and told they “don’t owe you anything” for merely asking why they were blocked, let alone how to remedy the situation.
These are not remotely equivalent and anybody who trusts them is a fool. The Fediverse has a serious problem with vile, bitter people who would not be out of place running an HOA. If we are going to emulate the blocklists common in IT, we need professionals in charge of it, not nosy busybodies.
we need professionals in charge of it, not nosy busybodies.
Great, you form the not-for-profit company to manage this and get the buy-in from a critical mass of servers. Gold luck finding and vetting staff
The point is valid though The fact that they’re not going to set up the solution doesn’t invalidate their core message.
I can equally point out that you’re not providing a viable solution either.
Read my other reply, I’m not talking about email blocklists, my reference is to email providers doing that, which is extremely rare and done with explicit intent and good reason.
Secondly, while I won’t disagree there’s some vile content out there on the Fediverse, do you trust someone else to make that decision for you? Why would you let someone else decide what is and isn’t vile for you and those using your instance? Better yet, how would you feel if some popular instance decides you were the vile one, and because it was a common instance to use for blocking references, your instance is now cut off from a good chunk of the Fediverse?
This is exactly the sort of nonsense that swept Twitter with shared Blocklists, and the potential for negative impact on the Fediverse is even worse from it. Don’t let others decide make decisions for you just because it’s easy, as it doesn’t absolve you of responsibility when something goes wrong.
Now you compare EMails aka mostly private communication to a public forum, that’s even dummer…
The ActivityPub standard is modeled on email. Each instance is a server, and we all have inboxes. It’s a very apt comparison to use.
Also, unless the email is E2EE, then it’s not private, no more meaningfully so anyways.
🤦
Admins can and do use email server block lists, though, so maybe that’s a great example.
I suppose you’re right–for now. But at some point Lemmy etc will grow large enough to make manual blocking infeasible. Just how much effort does it take to start a new instance even today?
Wrong comparison. It’s not like a server admin using an email block lists (which are also often implemented badly, as anyone with a protonmail account knows…). It’s more like if Protonmail suddenly blocked Tutanota, or if Gmail blocked MS Exchange. The uproar and rage from that would be unceasing.
The Fediverse operates on a model like that of email, and in the email world blocking whole email servers from each other is very rare, and usually done with the most explicit of intent and for a very good reason. That’s how the Fediverse should operate as well.
Youre just flat wrong here. Basically every email admin uses RBLs to one degree or another.RBLs commonly block whole email servers that allow spam, not individual users. Thousands of email servers. You literally dont know what youre talking about.
The reasons “MS exchange” are not blocked is because they clean up their servers and ensure this bad behavior is not passively or actively allowed. Protonmail had some serious spam issues, which is why they were blocked. They were not doing the work it takes to be a good member of the “email” federation, so they were excluded from it.
RBLs are ruthless, because email has no enforcement mechanism for good behavior besides “you are not allowed to talk to me.” Thats the nature of federation. There is no central authority to appeal to, so each member of a federation instead sets their boundaries. You play nice, or other people dont have to deal with your shit, and RBLs are a proven and effective tool to help them accomplish that.
The only issue with reputation block lists for the fediverse right now is that it’s still very small. They will likely limit growth, but that’s up to the server admins to weigh agaisnt allowing hateful content. Youre always free to spin up your own server and federate with whoever is willing.
Thank you. I recently set up my own instance and realized the need for tools that can assist in managing.
For reasons an other commenter has said, I think things like fediseer are a better solution to this. The way they use for measuring trust is distributed, like Lemmy itself (just fewer instances, because it is not for use directly by thousands of users, but for admins who are fewer).
LCS
That sounds interesting!
and Lemmy Post Purger (LPP) for clearing old posts on smaller instances.
Does that permanently delete posts? Why would you do that?
Does that permanently delete posts? Why would you do that?
Reduce the footprint of the install. Text posts and comments are negligible but pictures chew through storage.
And does this only delete visual media by default? If not, this is worse than anything reddit has done ever. I frequently save posts and links to myself in the form of links, for later processing. This would mean that by the time I get to it (can easily be years, honestly), it will have disappeared forever.
Not everyone who participates with their own instance can afford storage. Some users might have bandwidth restrictions. It’s the Fediverse. Wild, unpredictable and anyone can participate.
- Always document things you want to keep, never rely on someone else to do it for you.
Just to give some context, I have a one user instance running on a very lightweight Debian container containing only lemmy. After the 2 weeks I’ve had it up it’s at 6gb storage used. No clue how it would scale with more users federating with more communities but I could see it getting pretty big pretty fast.
It doesn’t delete anything that anyone on the instance upvotes, downvotes, comments on, saves, etc. Its mainly a tool for personal instances.
regardless of the tools, you should never trust something online to stay for long, there are so many things that can cause a post to be deleted, poster deleting it, server going down, admin error, change of rules etc, lemmy has got the advantage of having an open api, use it to save your shit and don’t expect anything to stay online for long
Don’t know but it would be a good idea to ask your instance admin if you’re worried about it. They’re the ones that foot the bill for the server and it’s storage and the ones that would be doing the deleting whether using this tool or not.
Also, if anyone is interested about fediseer, there are useful resources in this post: https://lemmy.dbzer0.com/post/220288
Hi there! Looks like you linked to a Lemmy community using a URL instead of its name, which doesn’t work well for people on different instances. Try fixing it like this: [email protected]
I’m going to found a user community for it! I’m kind of cooky, so I’ll name it “the church of LDS”. Surely nothing litigious will happen.
Jokes aside, great work :)
Obviously opinions are divided on shared blocklists, but we’re at the beginning of the road and only time will tell how this impacts the fediverse. Email had to introduce blocklists as well, so it’s not surprising this also exists.
Email blocklists are based on spam and malware.
I’ve never heard of an email operator refusing to send or deliver SMTP messages to/from a certain provider because too many of its users support the wrong political party.
I haven’t heard of users or instances getting banned/blocked for supporting the wrong political party either unless racism, brigading, CP, and death threats are core tenets of said party, which leads me to believe you may be referring to the Republican party in the US?
As usual, the American can’t see anything beyond their little island, also, in case you didn’t know, tankie servs get routinely defederated, and it’s not for their racism/transphobia or whichever big word is so easy to brandish when angry
So now you also claim to know the private thoughts and motivations of Lemmy instance admins? What a special and unique ability you possess. I’m surprised you’re wasting your telepathic talent on this and not using it to get rich.
it’s called reading around, it’s like watching tiktoks but with text, though it requires some form of context guessing which will be a challenge since the conversations you can read online are not always directed at you
Well you ‘read around’ when you read my comment so you must be in agreement with me now, no? Isn’t that how it works or do you only believe the things that confirm your preconceived notions?
The fact that you use “TikToks with texts” as your analogy kinda confirms my suspicions that it’s the latter.
There is a need for something like this. Lots of folks here are die-hard free speech defenders, which I completely understand, but some are starting to get into ‘Freeze Peach’ territory side of things. You can’t expect every new Lemmy instance admin to manually research about this nazi instance or that pedo instance everytime, especially with self-hosted hobby projects. If there isn’t anything like this, then admins might as well as start copypasting blocked instances lists from big servers like Lemmy.world or Beehaw.org, which again makes it even less coordinated.
There are plenty of potential for abuse as well. Eventually it has to have some kind of a public wiki where the reason for block can be consulted and have an appeal process with a 3rd party ruling to handle these things.
Nice to see history repeating itself I guess.
Interesting experiment though. I’d like to see how this plays out.
And here all I can think is “latter-day saints”
Disgusting. Let’s defederate with everyone whose ideology my echo chamber disagrees with111!
I see where this can be useful, but is this list categorized or at least indicate why a certain instance was added?
It’s different blocking an instance for simply repeated toxic behavior vs political leanings. Sometimes an instance can have both.
Letting new admins know and select categories/tags to block or let through could be an improvement, if it isn’t there already:
- NSFW (sometimes people want a SFW site, like browsing at work)
- NSFL (porn might be okay but not everyone have fond memories of ogrish.com and likes)
- Illegal in X jurisdiction (e.g. CSAM)
- Far right or far left
- Repeated instances of troll / spam bot accounts
- Abandoned by admin
There is no way to specify reasonings for defederation in lemmy right now :(
This is the kind of project that hurts Lemmy as whole. Defederation leads to fragmentation making Lemmy less likely to be a replacement for Reddit.
I disagree that defederation hurts Lemmy. It doesn’t hurt it anymore than normal moderation hurt Lemmy. Users on defederated instances are more than able to create an account on a non problematic instance and follow the rules there.
Defederation is essentially Lemmy’s version of quarantining subs on Reddit. And nobody except maybe extremists thought quarantine was a bad idea
You forgot people who selfhost single user instances. So they would have to destroy the old instance and create and new one with a new domain, which is a lot of work and resources.
Edit: Please also notice the problem here is not defederation itself, but shared lists of defederation. Because most likely the list is super long and nobody would check if all instances are legitimately blocked.
If the single user on such an instance is so obnoxious as to be defederated from multiple larger instances, making them spend time and money to come back is a good thing.
If he actually is obnoxious, then yes. But what guaranties he is obnoxious? Or that he just statements were just misinterpreted. Happens a lot.
If you trust the judgment of the (admins of the) sites you are syncing with, this isn’t a problem. If you don’t trust that, don’t sync with them.
For instance, if I wanted to run my own server, I would absolutely sync with lemmy.world’s block list. I agree with their stated defederation policy, and have seen no evidence of the admins disregarding it.
Perhaps you are misunderstanding what this tool does?
I am making an analogy of current situation of mastodon landscape where similar a project is based on questionable sources. Currently beehaw is nice and is blocking Lemmy.world. This makes me believe beehaw then end up blocking Lemmy world.
What could a single user instance possibly do to be defederated on a massive scale? And at that point, why not simply join another instance?
A single admin on a larger instance could get a stick up his ass and defederate a smaller instance, which would then propagate to everyone else copying that ban list. Power tripping admins didn’t die with Reddit.
that’s exactly where shared blocklist are a problem, if for some reason or another someone’s instance get mistakingly defederated, which is far form unlikely in one the enormous instances that have to manage federation of hundreds of instances, then all of a sudden, that big instance everyone trust get their blocklist copied all over the verse and poor jane is blocked from everything and has no idea why
They could have done nothing, but because someone on a so called trusted source de-federated it because he did not like him, de-federation would accumulate.
Actually beehaw is a nice instance and is blocking lemmy.world, which too is fantastic. Thus sharing beehaw’s de-federation list would cut out lemmy.world from a huge audience. In this particulare case, I wonder what lemmy.world did wrong to be worth de-federating from.
So you see, sharing huge block lists would wrongfully cut out people. Since nobody would investigate the whole list because doing so would take weeks.
Joining another instance is out of the question for many because they are firm belivers of self-hosting and decentralization. Two principles that are pillars of the the fediverse.
This tool isn’t a shared master list like ad blockers use, it’s me spinning up my own instance and deciding I want to match my blocklist to lemmy.world or lemmy.ml or whoever, specifically. And who I sync from is a decision I make about who I trust etc. I can unmake it and switch my sync to another instance the moment I hear about something I disagree with.
You get a point. This changes a lot of things. I need to review again the project page. I first understood that this is a shared block list. If it is just a mirroring of a given instance, then it is not the same.
This tool is kind of for those single user instances tho and while something like this could hurt Lemmy that’s only the case if it’s used wrong, I doubt big instances will start to share block lists because they have the resources to gather them manually!
I will exagerate and ask you: are you discriminating between bigger and smaller instances? While smaller instances are definitely small, together they are big enough.
Tf? I just say that it’s probably mostly single user and small instances that will want something like that because they don’t have the moderation caipabilities and might trust the admins of a big instance, not sure what your sentence is supposed to mean tho so could you explain?
The problem is that Reddit has a massive user-base and profits from the FOMO on newcomers while Lemmy is a very small thing most people don’t know about and there are multiple cases of instances defederating others just because they feel like it… like the BS that beehaw.org pulled. Now we’ve lemmy.world and beehaw.org two of the largest instances that don’t talk to each other.
- Lemmy is not a single thing
- No one is being prevented from visiting defederated instances
- No one should be forced to view content they find objectionable
- Neither should they force their views onto others
- Defederation is the equivalent of walking away from someone you don’t want to engage with
- Defederation does not delete content on other instances
- Lemmy is not reddit
- Defederation is not censorship
This is when degeneration is used with sanity. However, people just use that feature to cancel people they don’t like. And this hurts smaller instances a lot.
Beehaw came out the gate saying this is how they’d operate. They were always aiming to be strictly moderated and only federate with strictly moderated and anti-hateful places.
It’s not a fault that they’re operating exactly as advertised and promoted.
Defederation does hurt Lemmy when overdone, but sometimes it simply needs to happen to keep unmoderated instances from harassing your instance. Shared blocklists when properly managed simplify this process and improves Lemmy’s capability to protect it’s users from spam and brigading.
this process and improves Lemmy’s capability to protect it’s users from spam and brigading
That assumes defederation only happens in those cases. You have an account from lemmy.world, so I guess you trust that instance. You know, the same instance that preemptively defederated from hexbear.net for political reasons. You see the problem?
You see the problem?
No. I just looked at that instance’s front page. If the lemmy.world admins didn’t take action, I don’t see how I could continue to trust their judgement in enforcing their code of conduct going forward.
Seeing the number of down votes I may say that most people do not take smaller or single instance users that are too common. They are the ones that get most hurt with shared defederation lists. This only encourages people to gather to well known instance or accept being cut out from the biggest part of the fediverse.
Disagree, this tool is meant for use by small instance servers, not large ones. Large ones have to manually handle their list since they are on the hook more for maintaining some kind of policy and announcement structure.) The only way it hurts them as a little guy is if they break the policies of a major instance enough to get blocked by the instance admins, which would take some work for a small instance. (The same or greater amount of effort that would get you banned if you lived on the major instance so no benefit to making it your home there.) And then they only get blocked by that instance and the other small instances using this tool who chose to sync from that major instance. They remain fine on other large instances and small instance that sync from those other large instances instead.
If a large instance does start squashing little people excessively then hopefully that hits the feed and people can pick a different instance to sync from.
It’s like pruning. Sometimes needed for infected parts, or suckers growing before their time. Too kills the plant.
deleted by creator
