While I was asleep, apparently the site was hacked. Luckily, (big) part of the lemmy.world team is in US, and some early birds in EU also helped mitigate this.
As I am told, this was the issue:
- There is an vulnerability which was exploited
- Several people had their JWT cookies leaked, including at least one admin
- Attackers started changing site settings and posting fake announcements etc
Our mitigations:
- We removed the vulnerability
- Deleted all comments and private messages that contained the exploit
- Rotated JWT secret which invalidated all existing cookies
The vulnerability will be fixed by the Lemmy devs.
Details of the vulnerability are here
Many thanks for all that helped, and sorry for any inconvenience caused!
Update While we believe the admins accounts were what they were after, it could be that other users accounts were compromised. Your cookie could have been ‘stolen’ and the hacker could have had access to your account, creating posts and comments under your name, and accessing/changing your settings (which shows your e-mail).
For this, you would have had to be using lemmy.world at that time, and load a page that had the vulnerability in it.
Very impressed by how quickly action has been taken by this and other instances to patch the issue.
Very, seems like great work.
Hijacking the top comment to say I had problems with logging in to Lemmy.world today and liftoff was failing in odd ways.
I had to go into my web browser and clear my site cookies for lemmy.world to let me log in there.
In liftoff I had to go into the app settings in android to clear the cache and then remove and re-add my account for it to be able to log me in. (Press and hold on the account to remove it)
I’m on iOS with the Memmy app. It’s a work in progress that’s officially unfinished so I’m not surprised but it has also been a bit buggy. Doesn’t seem that I can log out without deleting and reinstalling the app so hopefully this doesn’t happen too often XD
So I was actually just struggling with that myself, also in the Memmy app in case that isn’t clear
What I did was add my account (again)
There was no warning or anything, and it populated the list with two of me.
At that point, a “delete account” option appeared under both of them. So I guess in normal circumstances, it wants you to keep one account around at all times?
I deleted one of them, and the app basically reinitialized. Both were gone and it showed me the welcome screen.
I logged back in, and now everything is back to normal
I just did edit account and then saved, it seemed to trick it into logging in again (secrets on my instance were also reset).
Ah interesting. I’ve had multiple accounts from the start so it was much easier for me. Just removed my main account and added it back.
Interesting. Definitely could be made clearer, I’ll make a post on the GitHub later about some of my suggestions.
Finally I found good instructions, was about to delete and reinstall until I followed this!
Interesting. Definitely could be made clearer, I’ll make a post on the GitHub later about some of my suggestions.
Whoops, glitched double response.
I did this, but I just didn’t delete either accounts and it worked fine. Idk if it’s detrimental to have two of the same but it worked for me.
I found I didn’t actually have to log out, just go into account settings and reconfirm everything without changing it
Go into account settings, clear your password, re-enter your password, save, go to feed and pull to refresh. That’s what worked for me.
No you can. You just remove the account from the accounts list. It’s labeled “delete this account” which is scary but it just removes it from Memmy. You can add it right back and that logs you back in. Not a great experience.
I sure hope this doesn’t happen a lot. This kind of barrier hurts site growth. I’ve managed a lot of large sites and seen a lot of bugs and when everyone gets logged out there is a measurable impact, and some folks never return. Just look at all the comments here saying “thank I didn’t know to do that.” For every one of those there are 100 people going “huh… Lemmy is down… oh well… on to something else…”
I was I able to upvote anything or subscribe. Seems like uninstalling and reinstalling fixed my issue
For Memmy, I went to the accounts page in the settings. Click d on my lemmy.world account then to the page where you can change the password then navigated away. That reactivated the account. Maybe we should add a ticket on Memmy’s GitHub about reactivating cookies when there’s an issue. Or at least place à poput to double check credentials or something.
thanks for posting this, I wouldn’t have figured that out lol
Good PSA. It took me a bit to figure it out, the app doesn’t make this obvious.
deleted by creator
Oh, I was wondering why it was showing me as logged in but wouldn’t let me upvote due to not being logged in. Your liftoff psa just cleared that right up for me, thanks!
How have I never thought of comment hijacking?!
deleted by creator
uh, why did you have negative one dislike?
Negative one upvotes would mean that enough people disliked me/another poster to bring my upvote total to zero. (Upvotes and likes are effectively the same thing, it’s just a naming convention). Reddit totals them up and seemingly Lemmy does as well.
huh that’s weird (yes I meant negative one downvote), I already know that the total can be either positive or negative, but shouldn’t the upvote number and downvote number be either positive or zero? (for now I’ll just accept it as a lemmy bug/ inconsistencies between instances)
Nope, just like Reddit it’s a value that ranges between negatives and positives. If I get two thousand upvotes, positive 2k. If I get two thousand downvotes, negative 1999 (because iirc you start with one by default).
Not exactly sure I understood what you meant by “either positive or zero”.
see your comment rn, it has 1 upvote (from yourself by default) and 0 dislike (so it’s not shown)
but in the screenshot I sent above you got 287 upvote and minus -1 downvote (making your total 288) which is mathematically correct but seems like an unintended behavior
for example this comment of mine normally have 9 upvote and 2 downvote (which is shown as a positive integer 2, not negative), making my total upvote 7
Just occurred to me that the app I use also shows separate counters. I fooled myself into thinking it was a single counter.
That’s interesting. Remember it’s a very new platform, minor bugs aren’t out of the ordinary.