Our new index is live and we have just set a few new records. We now have knowledge of over 3 trillion URLs and we have crawled over 300 billion of them. What’s more, we have dramatically improved our recrawl rate:

Index Statistics

307,174,759,713
Pages crawled

3,003,219,049,123
Unique URLs

The data actually includes a few links crawled in December, but because these few days might obscure December’s graph (which would have been extrapolated in December based on just a day or so of information) we have put set the charts to map to the end of November.

Faster Crawl Rate

Our last record is a new crawl rate. You might have noticed in November we announced a new intelligent crawl methodology. We can now report that of the half a billion URLs that we are crawling EVERY DAY, a whopping 300,000,000 are RE-crawls. This means that links on busy pages will now get updated and checked much more between updates.

Comments

Comments are closed.

ALL IT TAKES IS 30 MINUTES...
Do you need some extra help using Majestic? Sign up for a free demo and let a Majestic Expert take you on a tour.
THANK YOU!
If you have any questions in the meantime, please contact support@majestic.com
You have successfully registered for a Majestic Demo. A Customer Advisor will contact you shortly to schedule a suitable time to connect.
DARMOWE SESJE INSTRUKTAŻOWE PO POLSKU!
Potrzebujesz pomocy? Nie wiesz jak zacząć? Zapisz się na darmową sesję demo przez Internet za darmo!
DZIĘKUJEMY!
Jeśli miałbyś jeszcze jakiekolwiek pytania, napisz do nas na adres: support@majestic.com
Udało Ci się pomyślnie zarejestrować na darmową sesję demo z Majestic. Nasz doradca skontaktuje się z Tobą wkrótce by zarezerwować termin.