[ad_1]
Do you utilize a CDN for some or your entire web site and also you need to handle only one robots.txt file, as a substitute of each the CDN’s robots.txt file and your important website’s robots.txt file? Gary Illyes from Google mentioned that each one you want to do is redirect your important robots.txt to the CDN and management it on the CDN.
Gary wrote on LinkedIn:
You may have a CDN and you’ve got your important website. You may have two robots.txt information, one at https∶//cdn․instance․com/robots.txt and one at https∶//www․instance․com/robots.txt. You possibly can have only one central robots.txt with all the foundations, say in your CDN, which could aid you with protecting monitor of all guidelines you want to handle. All you need to do is to redirect https∶//www․instance․com/robots.txt to https∶//cdn․instance․com/robots.txt and crawlers that comply to RFC9309 will simply use the redirect goal because the robotstxt file of https∶//www․instance․com/. Bizarre or what.
Now, this isn’t an website positioning rating tip. It’s only a approach to maintain all the things beneath one robots.txt and never must handle a number of. This isn’t going that will help you rank higher or something like that.
I’m now ready to see adverts for website positioning companies to maneuver your robots.txt file to a CDN. Once more, this doesn’t offer you any website positioning profit or rating profit, that is simply to make it simpler for you if you need to preserve one file. In fact, for those who ever depart that CDN, then you want to cope with that, however it is not a lot completely different than transferring to a brand new host.
Discussion board dialogue at LinkedIn.
[ad_2]
Source link
