Google No Longer Automatically Indexes Websites – WTF?

Can you please take a few minutes today, go to Google, and search for your website using the site search filter? (EX: site:mysite.com) Do you have about the right number of pages showing up, or just a few pages, or worse yet – none?

The reason I am asking you to do this is that I’ve noticed recently that Google no longer automatically indexes websites, and thus is not sending traffic to those sites.

Here are a few examples:

• A couple months a new client came to me and asked me why her 4-month-old website wasn’t showing up in Google. It was in Bing and DuckDuckGo, but not Google. (I fixed this, but I neve did find the cause.)
• Yesterday I helped out someone on Reddit who could not figure out why his two-month-old website had not been indexed by Google. (I told him to submit a sitemap in Search Console, and see if that helped.)
• This morning I discovered that a site where I am the volunteer webmaster had not been indexed by Google. The site is a year old, perfectly functional, and I had already submitted a sitemap in Search Console, and yet even though Google has a sitemap showing 9 pages, Google had only indexed the home page. (DDG had indexed the entire site.)

Folks, I have been a web designer since 2016, and before that I was a blogger for 6 plus years. I have been deeply interested in Google SEO since 2010, and in all that time I’ve never heard of Google not indexing a site. Oh, sure, sometimes the website owner sets a noindex tag (accidents happen) but I have never heard of Google failing to index a site for no reason.

Google used to just index every site whether you wanted them to or not. That’s why the noindex and nofollow tags even exist, to give website owners more control over what Google did on their site.

This is so unheard of that I honestly didn’t use to bother to check whether a site shows up in Google unless I was asked to do so. I’ve literally never needed to do that when working on SEO (it would be like making sure gravity still works). in fact, Google’s own help pages say that you can pretty much assume that Google will index your site.

So if you have time today, you really should make sure that your site is showing up in Google. If Google neglected to index your site, you’re missing out on traffic.

So how do you fix this?

TBH, I am not sure. I would have thought that submitting a sitemap would fix this issue, but that didn’t help with the convention website. So what I am currently trying as a solution is using the “URL Inspection” feature in Search Console to force Google to index each page. It’s time and labor intensive, but I don’t have another option.

EDIT: The URL Inspection feature does work to get pages added to Google search results, and that is a relief. Alas, Google is only indexing the specific page I put into URL Inspection, which means I still have to solve this one page at a time. That is going to suck for anyone who has this problem with a huge site containing hundreds of pages.

EDIT: This post got a lot of attention on Hacker News and elsewhere, so much so that my site crashed several times on Monday morning. Several people listed the steps you are supposed to take to get indexed by Google. Those steps did not work for me, but they might help you, so:

How do you get your site listed in Google’s search results?

1. Set up a Google Analytics account for the site, and install the GA code so that Google can get data.
2. Set up a Google Search Console account, and install the GSC tag on the site.
3. Add a sitemap to that GSC account.
4. Email a link to the site from a Gmail account.
5. Tweet a link to the site, or share it on FB.
6. Link to the site from other sites.

The post Google No Longer Automatically Indexes Websites – WTF? appeared first on Nate Hoffelder.