A Million here… A Million there…

Updated on 19th April ’13 – See footnote

IP counts and Subnet counts are no measure of link database superiority

As the market leader in Internet Mapping and link metrics for data analysts and on-line marketers, we are used to walking on a tightrope when it comes to comparing ourselves with our competitors. Many in the SEO industry will be very familiar with our ongoing friendly rivalry with SEOMoz, and their MozScape product – now just one component in the rich SEOMoz online marketing community and toolsuite. By now, many will be familiar with Ahrefs, a new comer onto the backlinks scene, who are making some strong claims as to the quality of their service. As market leader, we have mostly refrained from commenting on this new entrant to market, but it is fair to say that Ahrefs won plaudits over the quality of their interface, but have also faced the challenges every provider goes through when experiencing start-up issues. Like SeoMoz (and ourselves in our first years of operations around 2008), Ahrefs have had their fair share of teething problems. The service now seems to have stabilised into a solid offering, which has led to Warrior Forum / Black Hat World regular Matthew Woodward to perform an interesting product comparison, which focuses on comparing how Ahrefs and Majestic SEO IP Address counts compare.

In Matthew’s post, Ahrefs CEO Mr Dmitry Gerasimenko claimed that Ahrefs product is not only larger, but that Ahrefs would “Win any objective test”. We therefore believe it is time to compare the services on our blog.

Backlink counts are not the only aspect of the service that require objective comparison. For some time, SEOMoz had a superior product than Majestic due to MozRank and MozTrust. We like to feel we evened that score with Trust Flow and Citation Flow that were released about a year ago. This scope of broad analysis was used to good effect in the Majestic SEO / Open Site Explorer shoot-out. However, the claim that Ahrefs is bigger has been made, and we believe we owe it to our customers to respond to the claims made by our newest competitor.

Methodology & History

Majestic SEO offers two indexes – the Fresh Index and Historic Index, our Fresh Index is aimed to be frequently updated, and has 90 days worth of data, our Historic Index has 5 years of data and we aim to update it once a month. Comparisons to date have been against our smallest Index. We continue in this vein.

Earlier this year Matthew Woodward emerged as a data scientist in this space by looking at stats for 3 undisclosed domains in different seo tools, which suggested that the Ahrefs product was 2.27% more comprehensive than MajesticSEO’s Fresh Index. Following some lively online debate, in his more recent article, Matthew, supported by $10,000 worth of API access from Ahrefs, presented an analysis of the free Majestic Million (produced by our Fresh Index), which leads him the conclusion that his readers should “terminate your Majestic SEO account immediately & take out the Ahrefs trial* instead.” ( *This links to Matthew’s Ahrefs Affiliate Link).

The Majestic Million analysis used a similar strategy for his analysis as First Past the Post elections – each row presented a win on domain, IP Address and C-subnet to Majestic or a win to Ahrefs by comparing the relevant counts with our Fresh Index data and that from Ahrefs API, using $10000 of Ahrefs API calls in the process. Matthew found that Ahrefs had higher IP counts and Subnet Counts for most of the Domains tested than Majestic’s Fresh Index, and also, by summing up the wins on each line, and then using the difference in percentages of IPs, declared that “Ahrefs knows 57% more about Majestic SEO’s Majestic Million than they do”.

Matthew’s analysis provoked a number of discussions at Majestic HQ. We were aware of some discrepancies in Ahrefs counts – but this alone did not explain the difference, so we decided to do our own analysis on the data. We created a model which allowed us to predict what gain in IP address and Subnet counts Majestic required to have greater counts than those recorded from Ahrefs.

We found that just 10% swing on subnet counts would result in bigger totals being shown in the majority of the Majestic Million for Majestic Fresh Index, and a 17% swing on IP counts would result in the Majestic Fresh Index showing higher scores. Two engineers performed this experiment independently, and both came to the same conclusion.

Whilst this analysis was under way, we made a couple of observations about the Ahrefs data – this resulted in some interesting differences in how Majestic SEO and Ahrefs calculate totals, which we will look at now.

When is an “IP” not an “IP” and a “Domain” not a “Domain”?

One of the great aspects of Matthews experiment was that it gave a chance to make some information public on how Ahrefs get their numbers.

To Dmitry, and Ahrefs credit – they provided clarification on this issue and admitted to a bug in their system which resulted in exaggerated IP counts being shown:

Majestic SEO was one of the first to display IP numbers with backlinks information from our own index. Our information is presented on a row by row basis, so in an advanced report it is possible to see which IP we recorded against each domain. The theory is simple – if many domains are hosted on the same IP, then there is an increased chance of generated content or a single editor strategy that can undermine the value of “IP wide” backlinks. It’s easy for any server to host thousands of websites, so for us it has been a long standing simple measure of quality. Likewise, our subnet counts emphasise this, dampening the effect of potentially spammy domains which may not be a good a place for white hat relationship building over more authoritative blogs and sites.

Ahrefs have taken a different approach – one which we feel actually emphasises, rather than downplays, the effect of low quality links on a site’s link count. To quote Dmitry again:

The “round robin” justification of, in Dmitrys words “2 + 2 = 5” has two main effects:

1)      It renders Majestic’s IP counts and Ahrefs Domain counts incomparable as an indicator of index size
2)      A great deal of care must be taken when comparing Ahrefs IP counts against each other within the Ahrefs site – after all – a link from a blog hosted on a webserver with 10000 other domains could be hosted using 7 different IPs. We believe, and I hope you will agree, a link from a website which does not share it’s IP with many other sites is better than one that does.

It’s not just on IP counts that Ahrefs has opted for to generate an algorithm that presents the largest total. At Majestic – we consider sites like Blogspot and Tumblr to be free hosts rather than valid top level domains. Ahrefs Dmitry has a different view:

We believe that treating subdomains of webs.com and Tumblr with the same weight as any “.co.uk” or “.es” site greatly distorts referring domain level metrics. We would ask if link-building using Ahrefs counts as a guide rewards practices that could result in link manipulation penalties in the major search engines?

A Bug in Majestic?

Unfortunately we did not get to see the release of Matthew’s post until after the event. As we were in the middle of a publicised maintenance cycle, we could not respond as quickly as Ahrefs to bugs we found as a result of the analysis. We have still reacted within the week.

We discovered that some of our IP information had ceased to be updated – which unfortunately for us had the opposite effect on us that the Ahrefs bug has on them.  This bug was resolved over the weekend, and quickly fed into our fresh index, resulting in us repeating the experiment, based on our new numbers and Ahref’s revised spreadsheet – amounting to just over 1 million lookups after the cleansing of some bad lines in the Ahrefs data which did not affect the overall results.

The new results

We’ve updated test file on Tuesday 16 April 2013, you can download it here. Up to date version of Majestic Million can be found here.

A Majestic Victory?

We don’t want to gloat – because it’s a shallow victory at best. Indeed, on IP numbers Ahrefs still wins (possibly due to unusual IP counting approach that they have), but also only by a small margin (45% to 54% with some ties). Our offering has moved on substantially since IP address counts and Domain name counts were core metrics. We have found that many of our customers are fully aware of the challenges of dealing with conflicting ways of analysing backlinks, pages, domains etc, which is why we invested so much effort into our award winning Flow Metrics.

Conclusion.

We have seen that IP Address counts and domains counts are open to interpretation. Both Ahrefs and Majestic have opted for different measures of each statistic – Ahrefs choosing algorithms that will tend to show higher totals for the same data than Majestic. Both parties will no doubt claim that their method has advantages over the other, but it is clear that due to the significant differences in calculating them, IP Address count and Subnet count are not representative measures of index size between at least these two providers – which brings us full circle back to the beginning of this article.

Given what we have seen in terms of how these counts can fluctuate due to inevitable errors when dealing with the size of data processed by Ahrefs and Majestic, The scale of the Majestic Million test was impressive, but ultimately at a $10,000 price tag to purchase the required Ahrefs data, in terms of ongoing value to the end user, little more than a publicity stunt (assuming Ahrefs don’t make a commitment to release their top million for free). Our bulk link checking tool supports bulk file uploads which can get data on 1 mln custom domains for as little as $100.

A full product survey must take into account a number of factors – including ease of use, price (Majestic SEO costs $49 per month compared to $79), Flow Metrics (we believe we have the best in the industry), support, continuity of service (Majestic places great emphasis on maintaining service levels – powered by a philosophy of independent data gathering), and let’s not forget face to face presence at conferences – Dixon flies to a number of conferences world-wide, including several in the  USA, and our team of Ambassadors represent us in many territories.

Whilst we continue to monitor Ahrefs, we would respectfully suggest they have been premature in claiming the link intelligence crown – not that it matters, as regardless of who sits at top spot at any given moment in this market, without constant innovation, it won’t be for long. There is only one clear winner in the battle for market share between Ahrefs, MozScape and Majestic SEO, and that winner is those who use the services these companies provide.

Footnote

Some readers may have noticed that we have not provided a link to Matthews Majestic Million article above. Regrettably in addition to the data comparison (which we welcome, despite the conclusion), Matthew has also inappropriately used his post as a platform to insinuate bad conduct on the part of members of the Majestic SEO team and our Commercial partners. We do not concur with his interpretation or his approach. We will not comment on conspiracy theories, or promote articles that make derogatory comments regarding our team members or business partners, and hence have been unable to inline a link to his article without this warning. A non-clickable link to the article follows:

matthewwoodward.co.uk/experiments/ahrefs-majestic-seo-1-million-domain-showdown/

Should you have any concerns or observations regarding the conduct of Majestic SEO or our staff, please contact us via the support centre. We will moderate out comments that deviate from the maths or logic of the product offerings.

Update ( 19/04/2013 16:15 BST ) – A response from Matthew Woodward:

Matthew contacted us following this post with a polite request for a correction – pointing out that following the publication of his post he became aware that Ahrefs had closed their affiliate program, and also that we accidentally included his “non-affiliate affiliate link” to Ahrefs as a “follow“ link in our article ( We have now added the “rel=nofollow” tag – and apologise for the genuine mistake ). Matthew states that he understands why we have not linked to his article but asks if we would be willing to add a piece about how the data for his test can be downloaded and independently verified.

Matthew – we have been happy to amend the article as appropriate, and apologise for not making your release of the test and data for download and independent verification more prominent in our original piece. The data and code can be gained by following the non-clickable link above – please however be aware of the cost ( paid to Ahrefs for API usage ) if you choose to download his code, validate it, and repeat the analysis. Majestic SEO can take no responsibility for code or data offered on third party sites.

• Author
• Recent Posts

Alex

Founder and Managing Director of Majestic-12 Ltd (trading as Majestic-SEO)

Latest posts by Alex (see all)

• Historic Index Update - July 26, 2015
• July Historic Index Update Published - July 31, 2014
• April Historic Index Update - April 16, 2014

Back