Again in mid-June, I seen that Google was not exhibiting a lot of my pictures in Google Search and Uncover and likewise some readers have been pointing it out to me. So I used the helpful Google Search Console URL Inspection device to seek out out these S3 URLs I used to be utilizing to host my pictures have been blocking Googlebot from crawling. Here’s a little bit of a case research from yours really of an indexing/crawling challenge I had for my picture URLs.
This AWS bug led to an 83% drop within the impressions my pictures have been getting from Google Search and Google Pictures. It led to a 76% drop in picture search associated clicks to this web site. I’m nonetheless down a number of weeks later by about 16% in impressions and 26% in clicks from picture search however it’s a big enchancment.
Right here is the Google Search Console Search Efficiency report exhibiting the impressions and clicks chart over time. You will notice the drop round June fifteenth, then it begin to decide again up round July eighth. Additionally, you will see that my picture site visitors has nonetheless not totally returned to its regular numbers pre-AWS bug, even after two months:
When Googlebot was attempting to entry my picture URLs on S3, Google was getting a 404 not discovered error. However after I visited the URLs with my laptop, they loaded simply superb. These are the identical picture URLs I’ve been utilizing on this web site for effectively over a decade and poof, sooner or later, AWS determined to dam Googlebot. I reached out to each Google and AWS in regards to the challenge and I believe it was a fairly large challenge. Tons of websites use S3 for picture and file storage, so Googlebot was doubtless getting tons of 404 errors. The bizarre half is that I noticed zero public complaints in regards to the challenge.
In any occasion, that is what Googlebot noticed once they tried to crawl these URLs:
AWS mounted it after a number of days:
That is what my pictures seemed like within the URL Inspection device in Google Search Console:
It ought to look one thing like this:
Since then, I made a decision to maneuver my pictures to AWS’s CloudFront – a service that was not out there after I first made this web site – which is why I used S3 again then for pictures. The S3 challenge with Googlebot continues to be mounted and dealing superb. However I’m not going again to S3 for pictures.
I ought to thank Glenn Gabe for additionally noticing the pictures going away early on in Google Uncover. Glenn additionally wrote up this image migration article which I reviewed earlier than making the change from AWS S3 to AWS CloudFront. I didn’t migrate my outdated pictures, I left them, as a result of AWS mounted the problem. However since late June, all my new pictures are utilizing CloudFront.
To be clear, this was not a Google bug, however an AWS change that led to AWS S3 blocking Googlebot. It’s now resolved nevertheless it looks like the harm has been completed… If the graphs change extra, I’ll replace this story under to doc the adjustments. However up to now, it has been flat for the previous 5 weeks or so, so I’m not anticipating massive adjustments sooner or later.
Discussion board dialogue at X.
