Because Majestic is good at making maps of how the web links together, it’s a great tool for finding pages on your site that return a 404 error (are broken), but well linked to.
Knowing what pages are broken on your site and having an effective strategy to deal with them can help your site present a better user experience and even convert better than a site without a 404 Strategy.
But what is a 404 error, and how do they happen?
A “HTTP 404 Not Found” code is the technical code used by web developers and website admins to describe missing pages. These 404 pages are often encountered accidentally, and unfortunately can leave visitors frustrated and confused.
As part of SEO in 2023, we asked Rebecca Berbel from Oncrawl how 404s happen, and she shared that ending up with a page that gives a 404 Not Found error is often due to a link that’s directing to an incorrect page, which could be a page on your site or it could be a link to content that’s published elsewhere.
To find which of your pages return a 404 error, you can use the Top Pages report in Site Explorer, which is available on all paid plans. The pages tab has a set of optional filters you can easily apply to the results of your query. To find which of the top pages on your site may return 404’s, choose the “HTTP 404 Not Found” option from the crawl result filter as shown below.
This will return a list of those pages in the results which returned a 404 code last time we checked the site ( the visit time is shown in the “Last Seen Column” ). A 404 check is an important part of a site audit – potentially uncovering content opportunities. If you find a missing page on your site with good metrics, it can be worth investigating further using Archive.org’s Way “back machine” to unearth what the page looked like prior to deletion.
SEO’s often look further than a client’s own site when searching for 404’s, as they may want to find link opportunities by finding relevant broken links on either competitor sites or authority sites within their own niche. There are a number of ways of finding this data in Majestic, as we will see.
1. Finding external backlinks to 404 pages using a backlink export.
Site Explorer contains a backlinks export feature that contains the HTTP status code of the target page. To use this export on windows based PC’s you’ll need a tool like 7-zip (www.7-zip.org) to uncompress the “gz” data.
1.1 Downloading the data
The backlinks download can be found in the cog – “Download All Backlinks -> Fresh Index” will enable you to download all backlinks for the URL in question for the Fresh Index:
You should then see a screen similar to the one below, if you don’t, check out “1.2 Accessing the new Backlinks Export”, below:
Click “Build Download” to generate your download, this will start extracting all the data from our systems and packing it into a download:
At this point, the system has your request and you can close the “Full Export” pop-up and get on with work – the download will appear on your downloads page when done. Alternatively, you can leave the pop-up open, and when the download is done, it’ll tell you:
At this point, depending on browser settings, etc., Majestic will try to automatically send a copy of the data to you to save. Once the file is downloaded and uncompressed, import it into Excel:
A filter can be applied to the “Target Crawl Result” to limit the rows to backlinks to pages with 404s. More filters can be added, to for example, limit source pages to those in the English language and those with more than one outlink (“Source External OutLinks”):
1.2 Accessing the new Backlinks Export
If all of this is new to you, then you MAY be presented with the old “Download All Backlinks” screen, shown below. TO upgrade, just click the button labelled “Want to try the new Full Export?”
The download will remember your choice and present you with the new box next time you log on.
Another screen you may see is this one:
This screen is trying to save you resource units by offering a quick download. Unfortunately, while slightly more resource efficient, the columns in the file are a little different to the ones mentioned above. For the purpose of this guide, it’s probably worth clicking the “No, I want a Raw Export” link at the bottom of the screen, from which you can get to the first menu.
2. Using the Pages tab as a hub of exploration.
The technique we shared in Part One is an effective means of finding backlinks to 404 pages on a given site. However, this can result in big files that can sometimes be difficult to work with or share without resorting to specialised data analytics tools or custom code.
If you’d like to get a number of smaller files, then you may be able to use the “Top Pages” tab in Site Explorer as a hub for your “backlinks to 404 pages” discovery efforts. It may not list ALL the 404 pages we’ve discovered on a given site, but tends to find the ones in the top few thousand (depending on the level of your plan).
Finding 404s using the Pages tab is easy:
First, query Site Explorer for the site you want, and click the “Pages” tab. The Pages tab includes a number of filters to let you manage the data returned in this report.
To filter our all the pages apart from the ones that returned a “404”, open the Crawl result menu, and look at the “HTTP 404 Not Found” option. The page will then show just the 404 pages!
But how do we get the backlinks to these 404 pages?
Once again, we use the “COG” menu. This time, we use the COG next to the 404 pages of interest.
Now we can generate downloads for the URLs we are interested in by selecting “Download All Backlinks” and choosing the “Fresh Index” option. This presents a similar set of options to those presented in part one, BUT will just produce the backlinks for the page in question. This can be great when you want more manageable file sizes, or are sharing the work over a wider team, but potentially less efficient than the first method if you have the tools to manage the data!
3. HINT – The “up and coming audit” that’s especially valuable IF the niche is new to you!
Finding 404’s on sites you know about is valuable. But how do you go about finding 404’s on sites you aren’t aware of?
Majestic has a few tools that help you discover new players in any given market vertical. If you’ve got a Majestic subscription, it may be worth trying some as a part of a regular audit. Let’s have a quick rundown:
3.1 Related Sites
The “Related Sites” tab is built into Site Explorer and uses “advanced heuristics” ( also known as computerised guesswork ) to produce a list of up to 100 sites which may be similar to the site queried.
As of time of researching this post, a “Related Sites” query for Majestic suggests Ahrefs, Moz and Semrush as being the top three sites related to Majestic. Even if you are 100% sure you are totally familiar with all of the sites in your niche, occasionally checking the related sites tab can provide valuable intelligence on new entrants to your market.
3.2 Link Graph
Link Graph is the newest addition to Majestic. Like the Related Sites tab, it can be found in Majestic Site Explorer.
Link Graph provides a means of visual exploration of the sites that link to you, and the sites that link to them. It can be a helpful tool to find authority sites within a niche that is unfamiliar to you, or to discover collaborative opportunities that may be two or three links away from your site. Options are available to select groups of sites of interest or export data for offline processing.
3.3 Clique Hunter
Tools that use Link Intersection to find potential Link building opportunities have been around for years – dating back to Aaron Walls famous Hub Finder. Most backlink checkers provide a similar feature, and Majestic is no exception. The Majestic tool for finding link opportunities based on links that may link out to your competitors but not you is called “Clique Hunter”, and it can be found within the “Compare Domains” set of tools, available on the PRO plan and higher:
Enter a number of sites:
Clicking the orange search button provides both a visualization and a list of potential sites worth investigating, both in a visual form, and a downloadable set of data:
In conclusion
We’ve looked at ways you can use Majestic to find pages on a given website which produce 404 missing content errors. We’ve also seen how Majestic can be used as a tool to find other sites worth analyzing in the hunt for 404 link building opportunities.
Regular use of a backlink checker like Majestic can provide link repair and link building opportunities. A subscription to Majestic unlocks a range of tools to compliment link audit and link building, helping to form the basis of a digital marketing intelligence platform by offering Majestic Tracking Campaigns.
Link Tracking campaigns track the visibility of sites online, recording metrics like Trust Flow, Citation Flow and other key link metrics. This information can then be referred to in regular 404 checks, and act as a source of commercial intelligence going forwards. A regular check and add on players can turn Majestic from a link research tool into the base of a digital marketing research tool – highlighting competitor and authority link growth in your niche.
- October 25th, 2024 – Majestic turns 20! - October 25, 2024
- Key SEO Tactics for Entity Optimisation (Live Podcast) - October 24, 2024
- How AI is being used to power organic growth – Live Podcast - October 4, 2024