Unclear - it didn’t have a useragent, but it was scraping all the posts: /api/v3/post/list?limit=50&sort=Old&page=56273 - it was on page 56273. The IP address is associated with a VPN, so I’m sure they’ll try through some other way - but that won’t help them :P

I don’t know what they plan to gain from this anyway - they’re better off scraping this content directly from Reddit themselves. I know a bot that shows how to do it ;)

All right, the server has been updated to the latest stable version.

I initially intended to leave it running for a few hours to make sure everything is stable, but somewhere along the server reboots, the bot got activated once more and errr, well… it’s been posting happily for some time now.

I guess that answers my question whether or not it’s still compatible with the new server version.

Dropping by for another update:

• Server has had Lemmy updated from 18.4 to 19.3
• pictrs has been updated from 0.4.x to 0.5 - that also took a nearly a day
• Just updated postgres from 15 to 16

There are still some updates and migrations left to do, but things are looking up, I think the worst is behind us.

Having said that, I’ll be taking the server offline now for the rest of the updates - hope this comment was properly propagated :D

Good news everyone, the bucket migration has finished! At first glance everything seems to be in working order.

To make the most out of the downtime, I’ll continue with an attempt at upgrading to a more recent Lemmy version this weekend. So there might be a bit more downtime, but then we should be good for a while.

Small update

So it’s been over 4 days of file transferring. There are about 13.6 million files in the source buckets, and yesterday we passed 14 million file checks.
So I’m guessing that means it needs to do multiple checks per file. I’m hoping it’s 2 checks per file, meaning the process is over half way.
The good news that most of the data has been transferred over (source shows 727GB used, destination shows 713 GB).
The bad news is that due to how the files are structured, it’s incredibly hard for the sync process to estimate which files it still has to move over, so it does need to check them all one-by-one.

Ironically, me posting this image above would mean there’s 1 file more that has to be synced over. But I’m willing to let it be a casualty of this war ;)

It does not, since it’s not a bridge, but a one-way syndication.

I’ll leave that to the reddit moderators and community to decide. There are thresholds in place to ensure they have time to filter out illegal material before it gets crossposted.

And again, so far no original poster has reported any posts on these bases. But I have no qualms with taking it down if they do.

It’s noble of you to want to protect those people, but the moment they post content to reddit, they have given reddit the right to duplicate and redistribute their content as it sees fit. So the way I see it, that ship has sailed, and I’m pretty sure most creators are aware of this.

Having said that, if someone ever reports they want to take their content offline, I’ll gladly do so. So far, that hasn’t happened yet though.

Edited to add: the lemmit service doesn’t even copy most content, only relatively small (animated) images. On account of storage (and processing) larger files costs money. So most content is hotlinked. That means that when the original is removed from Reddit, it’s removed from lemmy as well.

I might have been inclined to agree with you, if the posts themselves didn’t include all the attribution and other data that is on the reddit posts themselves.

How do you feel this robs from content creators, rather than giving them more exposure? It’s not redistributing any content that’s behind a paywall, for example.

I’ll consider your opinion.

Looks like it’s working. Time for a beer!

It’s done: !theyknew@lemmit.online.

Ah, I guess I must have overlooked that part. There are several reasons for not wanting to allow signups.

One is quite simple, cost. Right now this is running on a small, single core instance. It often stutters (especially when handling video updates), and that is not an issue, since that just means it’s going to take small while before updates are sent out. But you wouldn’t want to have that delay for actual users. Right now the costs are quite manageable, if I have to scale up in order to provide a fluent experience for its users, not so much.

Most of the other reasons come down to the responsibility of having to provide a home to any outside users that sign up. I don’t have the interest or time to maintain a community of people, nor to guarantee the uptime that such a server would require. It also wouldn’t work. The largest Lemmy instance in existence, lemmy.world, has defederated from this instance. So any users that sign up here, would be devoid from content on there. And as you said, any other instance can decide to do so at any time (in fact, I very much suggest they do so in the FAQ).

I could go on, but I think you get my drift.

Heya,

I still need to create some tools to make to easily add new subreddits to the bot. I’ll probably get around to that this weekend, and then I’ll add /r/theyknew and notify you. As far as I’m concerned, it’s a great contender for synchronisation/archiving.

Can’t blame you for that. Personally, I still think it excels at content where communication with OP is irrelevant, like !itookapicture@lemmit.online, !todayilearned@lemmit.online or !dataisbeautiful@lemmit.online. And by far best example of this, if you look at the subscriber count, is nsfw content.

Nope. That would be very hard to implement, and probably very confusing and disliked by other lemmy users.

I don’t know how the karma thresholds work behind the scenes, but might I suggest for the bot to do a “top for” sort instead? Like it will only repost top content for the past 6 hours only. This will also help get more quality content as well and avoid reposting low effort/quality posts.

This is effectively already kinda how it works. For each subreddit it periodically (anywhere between every 30 minutes to every 12 hours, based on subscriber count and posts per day) requests the “hot” content feed. It then checks each post if it has at least 20 upvotes, and a 80% upvote to downvote ratio. Those numbers are configurable, but that’s what they’re currently set to - I believe they’re a good mix between filtering out the complete garbage while still making sure it doesn’t miss good content is.

@criitz@reddthat.com @Halosheep@lemm.ee @Cagi@lemmy.ca @Hanabie@sh.itjust.works @SatanicNotMessianic@lemmy.ml @WallsToTheBalls@lemmynsfw.com @cypherpunks@lemmy.ml @DogMuffins@discuss.tchncs.de: First of all: those are some wonderful usernames. Secondly: I have taken your concerns to heart and made some changes. See my update here: https://lemmy.ml/post/6190779.

What.

You want to mirror a Lemmy community onto Lemmit? :s

Also, see sidebar.

Funnily enough, it initially was the intention to have the bot check up on everything it posted, to see if it got deleted. In that way, it would outsource moderation to Reddit. I never got around to that, and am not sure I ever will.

So for now, handing out moderation to others is a good workaround. In order for me to make you a mod, you’ll need to leave a comment in the community, and mention me @admin@lemmit.online.

Actually, checking out the subreddit in question, that’s exactly the kind of content I want to avoid on here. Most of the posts on there are to invoke discussion, either with the OP or other members. You’d be better off starting a new community on your own instance.