We have now updated our Historic Index with data until end of October 2013. Here are the new stats:

Historic Index –

Unique Pages crawled: 601,375,778,666

Unique URLs: 2,329,017,065,332

Date range: 25 Mar 2008 to 26 Oct 2013

Comments

  • Dirk

    by “we have” you mean who? I am newbie here – you have your own crawling robots that are indexing webpages or what?

    December 2, 2013 at 2:34 pm
    • Dixon Jones

      Hi Dirk – welcome.
      Yes – we have our own crawlers. we are the largest Link Intelligence database on the planet. We crawl around 187 billion URLs in the last 90 days, and discovered nearly 700 billion URLs. “We” are a UK based company crawling through an international project. Our bot is called “MJ12 Bot” and it obeys robots.txt.

      December 2, 2013 at 4:07 pm

Comments are closed.