Majestic’s Historic Link Index has just been released and it is the most comprehensive link index ever! Although Majestic does not report the overall volume of links known (and may not even know!), they do report on several some vital statistics. For example:

The Number of pages CRAWLED in Historic Index

This has rocketed 13.5 percent from 1.0 TRILLION to 1.14 TRILLION pages. For the purists, the new number is :

1,136,685,742,504

Rumours that Majestic might be falling behind are somewhat exaggerated. Now other Indexes are aging in a similar manner to ours, a cursory look at the other index numbers revealed intereting numbers

This number is 700% larger than the numbers quoted by our nearest rival!

700 percent equates to a staggering difference, lathough the performance enhancement is not entirely down to an increase in crawling capacity (which has not especially chaged. If anything, the actual crawling rates have reduced. However, Majestic’s crawling has increased efficiency over speed. More about this on Thursday, but in short we are able to better surface new URLs which – whilst seen – have not yet been crawled.

Next, the following increment is even more impressive:

The Number of URLs in the Index

The number of URLs Majestic has seen and recorded in the Historic Index has increased this month by a whopping 27 Percent to over 6 TRILLION URLs! Again for the purists, the new number is:

6,043,450,503,470

The Cartographer’s Paradox

Many customers of Majestic take the issue of size extremely seriously. This writer has had long conversations with Googler engineers, Bing Engineers and other big web crawlers over the years and eventually all crawlers understand the limitation of such a quest. Majestic’s head of development talks about the cartographer’s paradox. When trying to map the world, you could get more and more and more detailed… but at SOME point, to map absolutely everything, you would have to create a whole new world, identical to the one you are trying to map. On Thursday, we will talk more about this and how it applies to Majestic’s crawlers. In the meantime, please enjoy our new – biggest ever – Historic Index.

Majestic aims to document historic index numbers in this feed.

 


Follow me

Dixon Jones

Marketing Director at Majestic
Marketing Director at Majestic.
Follow me

Leave a Comment

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>
*

ALL IT TAKES IS 30 MINUTES...
Do you need some extra help using Majestic? Sign up for a free demo and let a Majestic Expert take you on a tour.
THANK YOU!
If you have any questions in the meantime, please contact help@majestic.com
You have successfully registered for a Majestic Demo. A Customer Advisor will contact you shortly to schedule a suitable time to connect.