New Feature: Index Merger

We are excited to share a new Experimental tool with you – Index Merger. For a domain/subdomain/URL, this tool gives you a single, de-duplicated downloadable file containing both Fresh AND Historic data.

As an added bonus this downloadable report sources additional crawl data that is yet to make it into a full build of the Fresh Index. We call this additional data ”incremental data”.

The Index Merger is a new, different way for you to pull data from Majestic. Because we are using some innovative new technology, we want to understand more about system load before we integrate the tool further on our site.  You can find the Index Merger on the ‘experimental‘ section of the site.

Why Index Merging?

Combining and de-duplicating backlink data from the Fresh & Historic index can be a time and resource-intensive task. Our engineers believe they have come up with a way for Majestic to perform this complex challenge server-side, saving you time and effort.

What Data Gets Merged?

Index Merger does more than keeping one instance of a backlink from one of the indexes.

All data is merged across the indexes. Items in the Historic Index are updated with relevant values from the Fresh Index. For each backlink, we keep the earliest ‘First Seen’ date (usually from somepoint during the 15+ years the Historic index covers). The latest ‘Last Seen’ date is preserved, often from the Fresh Index, as are other data points including crawl results, page titles, Link Context data, and whether the link is live/deleted.

How much does this report cost?

Resource usage calculations are based on the combined size of the Fresh and Historic data needed to generate the report.

For example, if the Fresh index has 1,000 backlinks for a domain and the Historic index has 2,000 backlinks, generating the report for this domain will use 3,000 analysis units. The Incremental data is added for free 🙂

In order to produce the report, it is necessary to analyse ALL Fresh and Historic index backlinks. Without this analysis, it is not possible to de-duplicate the data. Charging is therefore based on the combined number of backlinks from each index processed before merging occurs. The resources used to generate the download is the same as if you downloaded reports from both indexes and merged the data yourself.

Most sites reported in Majestic are available to download, as long as there are sufficient analysis units available on the account. However, a small number of larger sites are not supported. IF you receieve an error message suggesting there are too many backlinks to generate a download – please contact us for other ways to obtain the data.

What is ‘Experimental’ all about?

The experimental category in Majestic provides an opportunity to release features and tools earlier than otherwise possible. Experimental tools help us gain valuable insights from customers. They tend to be built on the same data used elsewhere, but have aspects of novelty. Some of the savings on overhead come from simple things – like translations. The tools tend to have an English language interface. Because Experimental tools are subject to change, they are often less integrated than other features on the site.

The experimental category is designed for experienced power users and is therefore included in the PRO plan.

Following our UI and menu overhaul earlier this year, we plan to begin graduating some of the more popular tools from the Experimental section in 2023.

The future of Index Merger and your feedback

This is your opportunity to help us get this new feature right for you. We plan to incorporate the index merger tool into other parts of our website. Launching the tool as experimental means we have greater flexibility to change and tweak it’s function without disrupting your work flow. We plan to build this new download feature as an option in other tools and reports across the Majestic site.

Before we do that, we want your feedback!

We would LOVE to hear of any other uses you can think of that would be helpful for other SEOs so we can ensure users can get the most out of this new feature.

Back