[ad_1]
On July 1, 2024, Reddit up to date its robots.txt file to ban most search engine and AI crawlers from accessing its content material – together with blocking Microsoft Bing. Reddit didn’t block Google Search, which we covered here a few weeks ago.
A Microsoft spokesperson informed me last night, “We respect the robots.txt customary. Bing stopped crawling Reddit after they carried out their up to date robots.txt file on July 1, which prohibits all crawling of their website.”
Reddit told The Verge, “We’ve been in discussions with a number of serps. We’ve been unable to achieve agreements with all of them, since some are unable or unwilling to make enforceable guarantees concerning their use of Reddit content material, together with their use for AI.”
Mark Williams-Prepare dinner confirmed me that Reddit’s new content was not developing in Bing Search, he posted about this on X after which a number of different media outlets covered it earlier than Microsoft despatched me this assertion.
Reddit additionally stated, “This isn’t in any respect associated to our current partnership with Google.” Sure, Google is paying Reddit $60 million annually to have the ability to entry content material on its website.
Remember that Google has hidden gems algorithm that helped the Reddit acquire quite a lot of visitors from Google. Reddit is seeing quite a lot of bad PR round Reddit dominating Google Search. I imply, Reddit is dominating Google’s search outcomes. Heck, Google even examined special treatment for Reddit not too way back.
In any occasion, Bing and lots of different search engine and AI bots are usually not permitted to crawl Reddit’s website anymore. Google can and does however different engines don’t.
Daring transfer Reddit – however with the Google licensing deal – I assume it is not too daring of a transfer.
Wild: Reddit now blocks all main serps *besides* Google from crawling content material
This may solely improve Google’s monopoly energy…
… and you’ll wager regulators shall be https://t.co/9uGDivBdYc
— Cyrus website positioning (@CyrusShepard) July 25, 2024
Timeline:
– Google boosts Reddit in search by over 400%
– Google publicizes a 60M take care of Reddit proper after
– Reddit publicizes it seeks to go public 48-72 hours after
– Reddit blocks all scraping for competitors serps shortly after👏Anti 👏Aggressive👏Habits https://t.co/Hzrw0zkjDo
— Ori Zilbershtein (@OriZilbershtein) July 24, 2024
That is a part of a disconcerting acceleration of monopolies increasing to additional block competitors and take from customers. Serps use to offer hyperlinks to related content material. Now they spam customers with adverts intermixed with scraped and content material laundered by AI with out attribution. https://t.co/xMvzir3DPA
— Tim Sweeney (@TimSweeneyEpic) July 24, 2024
Discussion board dialogue at X.
[ad_2]
Source link
