Roadmap: Backlink Fidelity vs Backlink Volatility.

TLDR; Backlink quality is subjective. Removing data helps some, and hinders others. We, therefore, introduce “Backlink Fidelity” – trying to achieve maximum benefit for as wide as possible audience. Backlink Fidelity is hard, but worthwhile endeavor, essential for creating a web map which provides maximum utility.

The purpose of this post is to describe some changes coming on our roadmap over the next twelve months as we seek to build a bigger, better, faster web map with more utility. All while enhancing our cost/benefit proposition. This follows similar previous roadmap posts like the announcement regarding hardware upgrades. We aren’t aiming for Moz style TAGFEE transparency – we like to hold a few surprise releases close to our chest. But when changes we are planning could impact you or your business, we believe openness is the best policy.

You have asked us to deliver a number of potentially contradictory goals:

• Bigger Index (hence pressure on storage)
• Faster Index (hence pressure on processing)
• Cost Effective solution for you (hence pressure across our resources)
• Accurate mapping derived from crawl (with a need for resilience, hence more pressure across all parts of our system and business)

This sets us a challenge. An approach is “here’s four options, choose three”. Or even “Here’s four options, choose your favorite”. We want to deliver on all fronts, giving you a bigger index covering longer periods, built faster, at a low price point, which contains data you value.

To date, Majestic has been able to ride the back of technical advancements in storage and processing. However, evidence, such as reports of the end of Moore’s law , and bleeding edge commentary on “Hot Chips” suggests that while amazing progress has been made in delivering more resources for your dollar, there is no replacement for innovation.

Fortunately for Majestic, we have maintained a lead in understanding internet cartography, having been established in 2004 and trading since 2008, initially as Majestic SEO and now as Majestic.

We believe that we have hidden resources – a backlinks equivalent to “dark matter” which takes space in our index with limited value to our customers. If we can reduce the amount of “dark matter” without impacting the quality of data you consume, we can free up resources to use on the goals listed above, and more besides. However, something has to give to achieve this. We aim to increase the clarity of how we communicate backlink counts. so we can seek to focus our efforts on the data that mean the most to you, and reduce the amount we spend providing data you don’t care about. The result should be amazing, but may result in backlink count volatility, and more counts related to backlinks in the short to medium term.

What is Backlink Fidelity?

Measuring backlinks is hard . There is no industry agreement on how to measure backlinks. Backlinks, like referring domains, is a term that means different things to different providers. To make matters more complex, different audiences have different needs. For some, losing data results in data purity, for others, that data loss represents a unacceptable level of service deterioration.

• The Toxic Link Analyst wants a list of low quality links.
• The Digital PR wants to know which well-regarded publications have featured good quality editorial links.
• The Affiliate analyst demands a wide variety of otherwise mostly duplicate URLs but with query parameters (affiliate IDs) being very valuable.
• For SEO audit, many suggest compiling lists from as many backlink checkers as possible (perhaps using a tool like Kerboo to stitch the list together later)

Those of you who wear more than one digital marketing hat may recognise multiple truths in the list above. If you know of a significant purpose we have missed, please mention in the comments below!

We refer to the challenge of getting a good list of backlinks that satisfy as many different audiences as possible with as little noise as possible as “Backlink Fidelity”. Backlink Fidelity is different to Backlink Quality as Backlink Quality is inherently subjective. What may be good for one purpose, may end up compromising another.

How do you achieve Backlink Fidelity?

Web indexing has challenges. Some of these challenges, like crawl budgets are well known to SEO’s. At the heart of web indexing is a chicken and egg conundrum. To create a great index, you need to crawl intelligently. However, to crawl intelligently, one needs to have intelligence – good data and metrics with which to rank the importance of crawling. The result is a feedback loop, whereby metrics are generated from the index produced by the web crawl, which in turn inform the crawl.

The drive to produce a better, richer web map is an ongoing mission. A consequence of this process is variance in backlink counts. As more data is gathered, backlink counts can appear to rise, even if the rise is just the result of more existing links being examined. Backlink variance is a normal part of building a web index, as has been discussed previously.

Majestic has invested heavily in gathering more data from its crawlers, most recently Link Context. We have been able to develop new services and metrics like Visibility Flow. We are now entering the phase where we can use these new metrics to inform crawling and indexing.

Does this mean Majestic has bad backlinks?

NO! It’s certainly true that Majestic has taken a cautious approach to data removal. Majestic was one of the first players and an early market leader in search engine independent backlinks analysis. With this position comes experience, and a broad market to serve.

Backlink Fidelity is not an issue unique to Majestic. We are aware of a number of approaches taken to measuring and managing the number of linking relationships within backlink checkers. Our aim has been to serve a broad audience, and to provide metrics and counts representative of the underlying data we make available. We talk to a range of customers from across a number of different sectors as to what is important to them. We believe our approach to date has been true to our version of delivering a versatile web map. We aim to deliver Link Intelligence data that transcends any one digital marketing discipline.

Being cautious on enhancing backlink fidelity has been costly. Storing web scale link data is expensive. We appreciate that some may seek to address this expense by simply storing less data.

We have been hesitant to rush to copy others efforts of culling links because:

• We fear throwing out the good with the bad. You have told us you value our data.
• We respect your use case of our data. We are extremely cautious about telling you what aspects of our data you value.
• Majestic powers a range of tools. A number of professionals rely upon us. We treat stability seriously, and have chosen to prioritise a diversity of utility of our data over storage costs.

Others may have different priorities.

We believe that enhancing Backlink Fidelity is a worthwhile aim in its own right and should not be confused with a cost reduction exercise. If backlink counts are reduced just because of storage, it stands to reason that data utility will be reduced.

What do I have to do next?

This post is the first part in an ongoing conversation on how the benefits and consequences of backlink volatility. We are still analysing data. We want to share our findings with you and embark on a period of change with you.

Our next step will be to provide additional metrics within the tool – we will describe these in an upcoming post, but wanted to provide you, our valued customers a message that we are listening to your needs, we are continuing to innovate and we value the relationship we have with you, and the relationships you have with clients and stakeholders.

We hope that these additional data points will provide clarity and reference to inform discussion around data utility and value.

We will be explaining more in subsequent posts, and adding in-tool help to explain these measures better.

We invite feedback on an ongoing basis, and may update this post based on your feedback. Comments are welcomed below, though we do recognise that many of our users prefer the discretion of our contact form and support system (https://majestic.com/help/contact-us).

Back