Return to site

Getting indexed is tougher than ever

Google has raised the bar on what is needed to deserve indexing

As I write this blog post on 31st Dec 2018 it is tougher than ever to get Google to index pages.

Previously Google would index pages but then apply filters to determine if they should be rewarded or pushed down in the SERPS, now however it discovers links via sitemaps, crawling or via following links but then chooses not to index the pages it finds. This is a new level in its armoury and its fight against spam.

For the ordinary web site owner, trying to build links themselves or on a modest budget it represents yet another obstacle to get rankings, ever more the internet is skewed towards those with a large business and budget that is not solely dependant on the internet.

What criteria is it using then? From my own research the biggest single factor seems to be internal site links and the use of comments, pages that have enough of these get indexed, ones that don't do not. This appears to be aimed at the type of low quality, low cost link building technique that was once known as Angela and Paul's method - this used profile links from strong and reputable sites to build masses of links to a target, often with little or no content beyond a few words and a link back to the target site in question.

The action of a small number of bad people therefore impacts on all us good guys trying to make an honest living in this way.

Here are some of my own pages that are on high quality sites but nevertheless are having a tough time getting indexed:

IntenseDebate- Adrian Lawrence - This is a discussion script that appears on maybe 4,000 sites all of which then link back to the profile area. I've only been posting for a few days so it is kinda too early to conclude there is any issue here. Mostly my debates are about Conservative policies and politics.

Disqus - Adrian Lawrence - Another similar discussion media, I post on many varied topics (which may be part of the problem) but the profile stuggles to get indexed.

Surprising one here Wordpress.org ! My profile on Wordpress which is within the top 10 websites in the whole world struggles to get indexed, if that site has challenges then everyone does. My profile is not very rich with text, which I addresses only yesterday, so that could be the explanation there also.

Another great site is Vimeo which allows your to post videos in a very similar way to Youtube, I really like this site as it has a good vibe about it and some of the travel videos on there are absolutely ace. My little corner of vimeo can be found here. Only two vids live at the moment, but I plan for more soon.

The last one this blog post if Flickr again if there was ever a strong site which you won't expect to experience issues with it would be this one. See my pics on Fickr which is a site I really love by the way

Inclidently what I think is that you are always best now to focus on link building within the context of a long post between 2,000 and 4,000 words as life is only going to get tougher for profile type page and articles or blog posts that are shorter than this.

Hope you enjoyed reading this and that if you are experiencing a link indexing issues yourself you overcome it soon also.

As I write this blog post on 31st Dec 2018 it is tougher than ever to get Google to index pages.

Previously Google would index pages but then apply filters to determine if they should be rewarded or pushed down in the SERPS, now however it discovers links via sitemaps, crawling or via following links but then chooses not to index the pages it finds. This is a new level in its armoury and its fight against spam.

For the ordinary web site owner, trying to build links themselves or on a modest budget it represents yet another obstacle to get rankings, ever more the internet is skewed towards those with a large business and budget that is not solely dependant on the internet.

What criteria is it using then? From my own research the biggest single factor seems to be internal site links and the use of comments, pages that have enough of these get indexed, ones that don't do not. This appears to be aimed at the type of low quality, low cost link building technique that was once known as Angela and Paul's method - this used profile links from strong and reputable sites to build masses of links to a target, often with little or no content beyond a few words and a link back to the target site in question.

broken image

The action of a small number of bad people therefore impacts on all us good guys trying to make an honest living in this way.

Here are some of my own pages that are on high quality sites but nevertheless are having a tough time getting indexed:

IntenseDebate- Adrian Lawrence - This is a discussion script that appears on maybe 4,000 sites all of which then link back to the profile area. I've only been posting for a few days so it is kinda too early to conclude there is any issue here. Mostly my debates are about Conservative policies and politics.

Of course another factor in all this is the crawl budget, ever site has a limited number of pages that Google is prepared to index, the higher the Domain Authority of a site the more content and pages can be and will be indexed, Google then uses its own secret sauce to determine which pages it has crawled should then be included. The process is dynamic as pages are crawled then thing have to re-evaluated.

It will be interesting to see how links from quality sites such as Strinkingly impact on this equation, hopefully good quality inbound links will help to tip a page into indexing.

broken image

Hope you enjoyed reading this and that if you are experiencing a link indexing issues yourself you overcome it soon also.