At Thinkvisibilty last weekend, I was demonstrating Majestic SEO and Richard Baxter (SEO Gadget) said that he had not realized that we were showing newly found links in between index updates. He suggested that I should tell people, because it is data that is not available elsewhere.
How to see “Fresh Crawl” information
You can start tracking fresh crawl data only after you have requested an advanced report for a root domain. (We support fresh data crawls for full root domain reports, not individual URL reports). After you have analysed the domain for the first time, you do not have to wait until a full update to be able to see links that we find from that day on. You can see them and drill down to them straight from the control panel, in the Advanced reports list:
From here you can see that although our last index was announced on the third of September, almost all of the sites in the control panel have new data in the fresh crawl section. You can then drill down on either of the fresh crawl columns. The “Last Fresh Crawl” column shows you the actual links that we found on the date in question, to reveal the link, anchor text and source flags (whether it is nofollowed, or in an image for example)
In addition, you can see all the newly found links, day by day, by selecting the “Fresh Crawl” drill down, rather than the date. This then lists the number of links that we find, day by day, and again you can drill down to see them in detail or export to CSV.
How powerful is that?
You need to bear in mind that the date we FIND a new link may not bear any relation to the day the link was actually created. However, the very fact that we can report as we go, so to speak, may well please dedicated link builders who would like to be able to generate independent reports on links found. A rapid increase here may either suggest something going viral or could suggest the presence of a paid link campaign starting. Alternitively, if our crawlers find a new site-wide link, then the numbers may suddenly be inflated on a day as we crawl the new pages. For this reason, we switch on a toggle called “domain clustering” so that on the detail, by default, you see the strongest link from a given domain, and we also tell you the number of other links from that site. You can switch domain clustering off if you choose.