We have had a few support calls asking us why our backlink counts are different to other providers and search engines. The purpose of this post is to explain the potential difference in link counts.
Firstly and most importantly, most of the major backlinks providers and search engine run their own crawler. Our crawler is independent of the others, and has its own set of rules ( algorithms ) that may result in it wandering around the Web in a different way to the others – hence it would be very unlikely that two backlink counts from two independent providers would be the same.
In the “External Backlinks” count on the search engine ( see https://www.majestic.com/reports/search?folder=&q=www.majesticseo.com ) we include almost all the external links we find – including “no follows” and “deleted” – for registered users though, we also include a breakdown of these counts for better comparison. The one type of link we exclude in this count is “mentions” – “mentions” refer to where a site or URL is mentioned on a page, but is not actually a link. We will return to mentions shortly.
The “External Backlinks” count is where the strength of our historical data comes in. The Graph on the search page shows the profile of links discovery at page level. For Majestic you can see a large spike in backlinks around June last year. This is where the free registration again yields benefit. When ‘logged in’, viewing the backlinks history provides hints to the answer – Majestic SEO has had a steady domain name growth, which indicates that the spike may have been due to an increase in the number of links, rather than the number of sites linking into Majestic SEO – an advanced report will let you find out more about this – but that may be for another article.
Returning to the theme of the post – “Why are the Majestic Link counts higher than search engine X”, For starters, Majestic SEO is a specialised backlinks product. Unlike the household name search engines, we do not need to process full text searches. The lack of this distraction means we can concentrate our efforts on the index of the links, which in turn means we can have a more comprehensive data set. You can read more about this in a piece of analysis we did a few years ago – Majestic Competitor Analysis.
Another question frequently asked is “Why include dead backlinks in the counts?”. Well, when we detect such links we do mark them as deleted, but Majestic is not a link checker – indeed, given the size of our index we have to stagger our re-crawls to ensure we do not put too much load on any one site. This means there may be a period between crawls when the link disappears and we have no way of knowing it. This is where advanced reports come to the rescue again – by specifying a recent date range in the “Source URL Options” section of the Advanced Report – Report Analysis Options screen. This can be used to limit the data in the advanced report to the last ( for example ) two months. The summary screen on the advanced Report gives a breakdown of all the numbers which enables a comparative analysis with indexes which focus only on recent data.
Returning to the theme of the post – “Why are the Majestic Link counts higher than search engine X”, For starters, Majestic SEO is a specialised backlinks product. Unlike the household name search engines, we do not need to process full text searches. The lack of this distraction means we can concentrate our efforts on the index of the links, which in turn means we can have a more comprehensive data set. You can read more about this in a piece of analysis we did a few years ago – Majestic Competitor Analysis.
Another question frequently asked is “Why include dead backlinks in the counts?”. Well, when we detect such links we do mark them as deleted, but Majestic is not a link checker – indeed, given the size of our index we have to stagger our re-crawls to ensure we do not put too much load on any one site. This means there may be a period between crawls when the link disappears and we have no way of knowing it. This is where advanced reports come to the rescue again – by specifying a recent date range in the “Source URL Options” section of the Advanced Report – Report Analysis Options screen. This can be used to limit the data in the advanced report to the last (for example) two months. The summary screen on the advanced Report gives a breakdown of all the numbers which enables a comparative analysis with indexes which focus only on recent data.
I promised earlier to return to mentions. In early 2009 it became apparent that the Majestic Data was appearing distorted by the inclusion of mentions in the dataset. We then took the decision to remove mentions from the totals on the website . Later we defaulted advanced reports to remove mentions, together with “no follows” and “deleted” links by default – though it is possible to include them in the options section of the advanced report.
I hope that explains why our backlink counts may be different to those seen on other sites, and demonstrated some of different ways to “dig deeper” into the headline counts.
Steve has been engaged with the Birmingham tech scene for years, co-organising the grassroots open-source conference YAPC:: Europe 2008, and founding the West Midlands Java User Group in 2014.
- Introducing Duplicate Link Detection - August 27, 2021
- Python – A practical introduction - February 25, 2020
- Get a list of pages on your site with links from other sites. - February 7, 2020
Always wondered why there are differences in backlink counts between Yahoo sitexplorer, SEOMOZ’s opensiteexplorer and your tools!
Now it explains.
Cheers
January 14, 2011 at 3:57 pmExcellent post – thanks for answering the question so thoroughly.
January 14, 2011 at 4:03 pmEnjoyed the read Majc
I must admit i like to select the option for see nofollow links – i find that if we de-select off some of the other options in the section of advanced reports you highlight above — i get to see some of the links that are often found from the social sphere and i see it as a bit of a twist on the “how are we doing in the social web” kind of link data… we like!!
With recent buzz around social media links counting for serp impact, indeed i must agree as these mass rss feeds – twatter et al – are simply popular websites – and there has always been focus in links on those.
Thanks for the read!
JLB
January 14, 2011 at 8:54 pmHa, yes, that explains quite a bit. Thanks!
January 31, 2011 at 7:56 pmAhhh, thank you for clarifying this! Very, very good to know.
(Oh — and as a P.S., did you mean “wandering” instead of “wondering”?)
February 5, 2011 at 10:41 pmThanks. Amended.
February 6, 2011 at 12:25 am