Do you know that typically these bot safety providers will serve Google and different search engines like google noindex directives? Google’s John Mueller mentioned this on X, saying, “Typically there’s bot-protection (or a login, interstitial, and so forth) triggering that has a noindex on it.”
He additionally added that it might be higher for them to serve 503 server standing codes. “Higher could be to make use of 503 for server-side blocks like bot-protection,” he wrote.
In the long run, the individual he replied to was having a special subject inflicting the noindex, it was not bot safety. He mentioned it was “some shenanigans with NextJS doing consumer aspect rendering as a substitute of server rendering.” “The HTML had a noindex tag, which was eliminated after rendering on the browser. The HTML proven on GSC is the one AFTER JavaScript rendering, which is why I couldn’t discover it and thought it was a GSC bug,” he added.
This was the error he was seeing:
Listed here are these posts:
wtf is occurring right here? why is Google not indexing this one?
They are saying there is a “noindex” tag, however there is not one. robots.txt seems good too.
Anybody? #seo assist pic.twitter.com/DYzJsLQ4Yk
— Guilherme (@goenning) December 26, 2023
Thanks John, finally discovered it due to @HamilcDev
Some shenanigans with NextJS doing consumer aspect rendering as a substitute of server rendering
The HTML had a noindex tag, which was eliminated after rendering on the browser. The HTML proven on GSC is the one AFTER JavaScript rendering,… https://t.co/NffU7suVLM
— Guilherme (@goenning) December 26, 2023
Discussion board dialogue at X.
