• zero_iq
    7•1 year ago

    Lots of traffic, lots of posts, lots of comments, … a torrent of incoming data. That’s going to need more storage, more bandwidth, more CPU power, higher running costs. The original instance hosting the community bears a higher load than the instances that duplicate it.

    Ideally, there would be a way to more evenly distribute this load across instances according to their resources, but from my (currently limited) knowledge, I don’t think Lemmy/ActivityPub is really geared for that kind of distributed computing, and currently I don’t believe that there’s a way to move subs between instances to offload them (although I believe some people may be working on that).

    Perhaps the Lemmy back-end could use a distributed architecture for serving requests and storage, such that anyone could run a backend server to donate resources without necessarily hosting an instance.

    For example, I currently have access to a fairly powerful spare server. I’m reluctant to host a Lemmy instance on it as I can’t guarantee its availability in the long term (so any communities/user accounts would be lost when it goes down), but while it’s available I’d happily donate CPU/storage/bandwidth to a Lemmy cloud, if such a thing existed.

    There are pros and cons to this approach, but it might be worth considering as Lemmy grows in popularity.

      • zero_iq
        1•1 year ago

        Funny how you say it’s not a problem, then go on to describe the problem that needs to be dealt with. Dealing with scaling is a problem, and it’s a problem that costs money.

        Posts like this: https://lemm.ee/post/58472 suggest it is a problem. The rise in traffic seen by Lemmy in the last few days is absolutely tiny compared to a site like reddit, and already instances are struggling to cope. The recent growth in user registrations represents only about 0.007% of reddit’s active user base. (~60K new Lemmy users vs 861,000,000 active monthly reddit users). A site like reddit costs millions to run.

        There are 190+ Lemmy instances last time I checked, yet almost all the brunt of this load has been borne by a handful of servers, which see an inordinate amount of traffic while 100+ other servers sit around idle. Why should a handful of “lucky” servers have to pay all the hosting costs? What if a volunteer-run instance explodes to reddit-like levels of popularity? It will simply fold, unless the volunteer has serious money to throw at the problem.

        • @SQL_InjectMe@partizle.com
          1•1 year ago

          Lemmy in the last few days is absolutely tiny compared to a site like reddit, and already instances are struggling to cope.

          While this is true, 5 days ago lemmy.ml, the biggest instance, was on a 67 EUR server which is very small. https://news.ycombinator.com/item?id=36270094

          Posts like this: https://lemm.ee/post/58472 suggest it is a problem

          This is a scaling problem (having more users means you need more mods) but I disagree with how they handled it and it isn’t a money related thing. My thoughts on this are in an older post when this was first announced https://partizle.com/comment/64178

          Why should a handful of “lucky” servers have to pay all the hosting costs?

          My initial idea is to use the something awful model of paying a one time fee to register an acount. The problem is that people would just sign up on another instance that doesn’t charge a fee but still add load to the lucky instance. Another approach could be to participate in communities on one of those lucky servers then you need to pay a one time fee to that server (comments would need to be removed by a bot if they’re not made by an approved user). I’m not saying that’s perfect, but it’s an idea. Adsense is another idea.