Now AI blocking will get you more precise control over your website! Now AI crawlers will be allowed to grab your website! But don’t worry! They promised and pinky swore they wouldn’t train any AI on your data. So your data is protected!
Isn’t that great!!?
It’s annoying that Cloudflare is basically facilitating / enabling the proliferation of the “zero click” internet that a handful of the worlds richest people are so keen on… And while they are encouraging site owners to hold the door open for AI, Cloudflare continues to punish actual human website visitors with their “are you human” barriers to entry. It would be nice if some of the people running it actually valued the internet a little bit ffs.
Years of site load optimizations only to see it all gone by having a 20 second wait time before every other damn website. Thanks AI bros, I truly enjoy it
I hate cloudflare’s anti-bot, it’s so slow and buggy. Sometimes it won’t load the site I wanted to, return me back to the previous one and manipulate history so now I need to press back twice or more to get out
Fucking hell. Why have a website anymore? Can we get a separate Internet where AI doesn’t exist?
There’s gemini (terrible name unfortunately due to Google). But anything that becomes popular will just get flooded, too.
I would suggest Usenet but my guess is that there are bots there too.
I haven’t used Usenet for anything other than file sharing, but I think i will give it a try. I kind of imagine it is like the internet in IRC format. Does anyone have suggestions for interesting newsgroups?
Dude bots have been on Usenet before the WWW existed.
I meant modern LLM bots. But yeah.
needs something like treehouse from the novel otherland by tad williams. an anarchist online space, with changing entrypoints that are told only to members of said space… or to stay in reality, something like CACert, where you could get an SSL Certificate for your website for free by talking to people in person and stuff… but nowadays youd make really sure the person you’re inviting isnt some predatory submarine aibro looking for easy content…
I wonder if something like that could be achieved with Meshtastic or HAMNET.
The range is too small with Meshtastic to be useful for connecting a large group. I did some practical range testing with myself and a friend both running a node and me driving around to see what sort of range I could get. We’re 3.5 miles away from each other and it seemed like we’d need a decent amount of repeaters to be able to chat without an internet backbone doing the work.
Maybe in a smaller town with more active neighbors believing in this it would be easier. In a large (flat) metro area, it’s tough to get meaningful coverage without putting in a lot of work building human connections and convincing people to run nodes.
Then I wonder about using HAMNET as the backbone? HAM radio signals can go 100 miles/161 kilometers with the world record being like 4000 miles/6440 kilometers. Obviously you don’t actually need the Meshtastic for HAMNET, but my understanding is that Meshtastic and others like Meshcore are far cheaper and in places like the US you need a license for HAM.
Crawlers are a big fucking problem and we really didn’t build the internet to accomodate them. Maybe if we said in 1990 “hey what if datacenters with Tbps connections will mass query all websites” we’d have a solution by now, but obviously that wasn’t really an issue back then. Anubis seems great but proof of work to prove you are a human seems to have it’s issues.
The solution is smarter secure mirroring for any static public content, stuff like Jekyll + Git or Atproto based sites. For dynamic content there’s no universal solution against misbehaving data centers other than blocking them
Wait, can you expand more sbout Jekyll + git? Seems interessing but i can’t find anything, also, why AT protocol instead of ActivityPub?
Anubis isn’t really about keeping bots out, it’s about keeping your site from being DDOS’d by bots scraping it a million times per second.
Dear Tim Berners-Lee, Did you have a backup internet that the rest of us could use?
I mean, there’s the Yiggdrasil network that IMO deserves much more attention than it gets, but it’s not ai-proof, just obscure enough
Really rolls off the tongue.
The Internet is not the World Wide Web. The Internet was over 20 years old when Tim invented Hypertext.
Well, while we’re “well actually”-ing, Tim didn’t invent hypertext
Tim is given way too much credit
Yeah this is just a new version of robots.txt. There is no need for companies to respect it whatsoever.
Yeah, search engines used to enable us to look up stuff online.
You’re gonna hate when you find HOW they generated their search data for your searching.
not by doing this shit, that yellow line is mostly bot traffic that is ignoring robots.txt
I liked it. I regularly have lots of niche problems and back in the day I’d just put it into Google and find someone on Reddit or whatever who already tackled it 2 years ago. Now I don’t.
Also used to occasionally watch the web server logs and see how the search engines check for updates on the organization I’m volunteering at. Or index my employer’s website. Or the Fediverse apps I run. And tweak my visibility according to my liking. Now that changed quite dramatically as well.
Enlighten us.
awesome
That’s the spirit!
