Our new index is live and we have just set a few new records. We now have knowledge of over 3 trillion URLs and we have crawled over 300 billion of them. What’s more, we have dramatically improved our recrawl rate:
Index Statistics
307,174,759,713
Pages crawled
3,003,219,049,123
Unique URLs
The data actually includes a few links crawled in December, but because these few days might obscure December’s graph (which would have been extrapolated in December based on just a day or so of information) we have put set the charts to map to the end of November.
Faster Crawl Rate
Our last record is a new crawl rate. You might have noticed in November we announced a new intelligent crawl methodology. We can now report that of the half a billion URLs that we are crawling EVERY DAY, a whopping 300,000,000 are RE-crawls. This means that links on busy pages will now get updated and checked much more between updates.
- How Important will Backlinks be in 2023? - February 20, 2023
- What is in a Link? - October 25, 2022
- An Interview with… Ash Nallawalla - August 23, 2022
Wow! Keep up the good work!
December 15, 2010 at 4:44 pmCome on, good work
December 17, 2010 at 7:50 amWhat a number! And I guess half of it isn’t worth a single click 😉 *lol*
December 17, 2010 at 11:06 pmKeep up the good work, SEO Experts need good quality information to assess our marketing strategies.
John
December 19, 2010 at 7:25 pmSEO Expert and Marketing Strategist
well nice to see this website going to explore its tools step by step . looking great tools . Regards
December 21, 2010 at 1:15 pmFraz
SEO consultant
Awesome, keep up the good work.
January 1, 2011 at 8:00 pmKeep up the good work,go………
January 12, 2011 at 3:58 am