I just got done reading the BBC article: Google just updated its algorithm. The Internet will never be the same. If you have not read it I suggest you do and then come back here.
Is it time for webmasters to block Google?
I’m not going to get into the technical aspects of blocking Google. I’m just proposing the idea that if Google isn’t sending traffic to our blogs and websites, either because it’s sending all the traffic to Reddit or because it’s using AI to summarize our writings on Google without credit, maybe it’s time we just block Googlebot and as many AI bots we can find via robots.txt.
This would freeze Google out and hand an exclusive to Bing, Duckduckgo, Yandex and Mojeek.
And why not? Speaking for myself, I do get some traffic from Google to the blog half of this website (I get no traffic from Google to the directory proper even though they spider it) and most of the Google traffic is very low quality – click and hit the back button. Do I really need that traffic? I have zero advertising, I’m not selling anything, I earn zero income from my link mongering and fitful blogging, so No, I don’t need whatever traffic crumbs Google drops from it’s table. I bet that is true for a lot of other webmasters too.
Anyway, that’s the short version of the thought: Is it time for the indieweb to block Google and AI bots via robots.txt? .
Liked this post? Follow this blog to get more. 
IMO, it was time to block Google 5 years ago.
Likewise Bing, Yandex, and every other corporate search engine.
If they want access to our websites they should be paying us.
I’d be less inclined to block corporate search engines IF we had 6 to 10 major search engines with their own indexes. But if all we have is a monopoly of Google then I’m thinking maybe Google is worth blocking.
The problem is, it’s not just the search engines, it’s all the other bots which are also commercial and send no traffic: SEO bots, AI bots, etc. I’m overrun with these and there are so many it would be a pain to list them all in robots.txt.
You don’t have to block bots individually. The following is valid:
User-agent: *
Disallow: /
You just might want to add specific directives above this one for Internet Archive or search engines that haven’t annoyed you.
That makes it easier. Thanks much.
I said that at the start of AdWords and AdSense. Obviously, nobody listened.