Counting Backlinks, and our 2021 Link Index Strategy -

It’s the start of the year, so in keeping with tradition, we would like to share some of our plans with you for 2021.

The theme for this year will be developing our Historic and Fresh indexes as independent products. We will focus on making Historic a comprehensive store for link audit, and Fresh a clearer, deeper, contemporary view on the web.

We will share more details over the next weeks and months, but read on to discover…

Why raw backlink counts can suck as a measure of link building success
Why we will start to show FEWER backlinks in Fresh, and why that is good for you
How we plan to add wider pin-pointing of key link level crawl events in Fresh
The new differences between Fresh and Historic
Some actions you may wish to take to prepare for the upcoming changes

While we are excited about some of the changes we have planned for 2021, we are also a little nervous. At Majestic, we pride ourselves on the stability of our product and put a lot of effort into trying to avoid disruptive changes to data.

We want to move the Majestic Fresh Index more towards Link Intelligence than Link Counting, however that will come at the cost of increased backlink volatility. Backlink numbers will change. This blog post is to make sure that you know what is on our roadmap, and give you a chance to prepare.

If you are involved in data audit or have a dependency on backlink counts PLEASE read this article carefully and let us know if you have any concerns.

A refresher on the hazards of depending on backlink counts.

It’s probably fair to say that there is a lot of hype around backlink counts. Some seem to claim that big backlink counts are good one day, and bad the next. While quality links are so more important than raw link volume, industry fascination with raw backlink numbers doesn’t seem to want to go away.

We are sceptical about the use of backlink counts as a KPI, and established Trust Flow and Citation Flow to address clear flaws in using External Backlink or Referring Domain counts as false measures of page popularity.

Additionally, there is still no industry standard on what a backlink is.

If a page has two text links with the anchor text “Click Here”, a frame link AND an image link to another page, is this one backlink, or four? What happens to the count if another text link is added to the same page?

If two different affiliates link to the same piece of content with different affiliate IDs, is this two links or one (or three, including the one for the canonical form)?

Majestic supply backlink counts for two reasons:

A count of Backlinks is a measure that’s indicative of scale of data held about a URL and the crawlability of a site.
Backlink counts are sometimes used with referring domain counts. The ratio can be helpful to understand a site’s link profile. SEOs can use a signal like a high backlink to domain ratio as a prompt for further research to understand a client’s Link Profile.

To be clear, however, regardless of where you get them from, we believe that for the majority of use cases, raw backlink counts have limited benefit as a measure of page or domain quality.

Lower backlink counts in Fresh, why?

Over the last few years we have increased the breadth of data held in Fresh. With the introduction of Visibility Flow in 2020, we committed Majestic to a course of moving from page-level to link-level measures of relationship importance – the first of the major backlink providers to do so.

The focus on links, rather than pages is an important drive towards greater backlink fidelity. This builds on work announced in September 2020.

Our focus in 2021 will be to make the Fresh Index deliver a high-fidelity view of the most valuable, unique and notable links on the web – in a way that increases the relevancy of the underlying data and makes it more valuable to your research.

When we have more to share, we will go into far greater detail, but to summarise with an example…

Large swathes of the web are powered by Content Management Systems and blog platforms that have a tendency to inject loads of duplicate links all over sites. Other parts of the web are made of automated directory sites, repeating the same groups of links over and over again.

A huge part of our mission to enhance Backlink Fidelity, is to identify these links and deprecate the significance of them in our index. This drive for better link level reporting will eventually result in a variation of backlink counts in the Fresh Index.

Don’t worry – we will discuss this in greater detail as we roll out changes.

Better reporting of crawl events in Fresh

Crawling and reporting at entire-web-scale is a challenge. Doing that on the commodity hardware used by Majestic and others in the industry takes the difficulty up to the next level.

As a data savvy digital professional, you will appreciate that all backlink providers apply some degree of compromise in order to deliver a commercially-viable offering.

At Majestic, we have always tried to balance the need to squeeze as much crawl information into our backlink indexes as possible. An area of compromise we have been keen to address is the reporting of key crawl event dates across the various reports and data extracts we offer.

As part of our drive to deliver more comprehensive Fresh Index data, we are working on a release that will add more specific and useful information on crawl events. For example, you’ll be able to find out if we’ve seen (but not yet indexed) your new URL.

There may be some caveats and details around the Memory Effect and other subtle index related nuances, but all will be explored in a future post.

Fresh and Historic, different databases, different services

Way back in the early days of Majestic, the difference between our Fresh and Historic indexes was simple…. Fresh contained the last few months of crawl data, and Historic contained the past few years. The tools available to each index were almost identical, the date range you chose was one of the few differences.

What has been incredibly interesting for us is the change in use of our indexes as the products have matured.

Historic, an index that contains around 10 trillion URLs from the past five-and-a-half years, is valued for Link Audit and entrepreneurial data analysis.

Fresh, the more compact index with the latest 1 trillion URLs from the last four months, is being used more as a window on the web for Digital PR, link building and informing domain valuation.

Rather than shoe-horn every new feature into both indexes, we have been working on an improvement program to strengthen the best parts of each index. In 2020 we finished our primary wave of investments into our Fresh and Historic infrastructure improvement program.

With Historic, we put an eye-watering amount of investment into our infrastructure. This has delivered constant regular builds, twice monthly. Where compiling the massive index used to lag a few months (at worst) behind Fresh, it is now only a few weeks behind. This vastly improves the relevancy and recency of data for you link auditors and analysts.

With Fresh, we have been on a drive to increase the breadth of data that we hold on each link and domain…

In 2021, we will continue to build on the strengths of each of these indexes. Our upcoming developments will enrich the best Fresh data to provide some of the most comprehensive Link Intelligence data that you will find anywhere.

This means that Fresh and Historic toolsets will continue to separate as they become different, yet complimentary, products within our Link Index.

Fresh will deliver a high definition and wide view on the best and most recent links.

Historic will continue to be a raw data powerhouse with an emphasis on depth and volume.

If you use the API

We know that Customers who use our API value index stability and we encourage you to maintain an eye on our blog, twitter feed and newsletter for relevant updates. For our part, we aim to take an “API last” approach to ensure you have to time to prepare for any changes that may impact how you communicate information to your clients.

We pride ourselves on the stability of our API, and have a long track record of dog-fooding, that is, we test new features on our site rather than risk passing on disruption to existing clients.

What should you do now?

If you haven’t already done so, you can find out some more background to these changes in our backlink fidelity post.
Fresh backlink counts will almost certainly change, so if your customers rely on Majestic Backlink counts as a KPI, please use this opportunity to consider educating them in the value of the other metrics we offer such as Trust Flow and Citation Flow, or visual tools such as Link Context and Link Graph
Stay tuned to the blog or our Twitter stream for updates.
If you’re worried, or want to ask any questions, please reach out to our Customer Services team (support@majestic.com) if anything you read is cause for concern.