If you have been hit by a large drop in rankings in Google recently and you have built links – over the years – to try to help your SEO, then check your Google WebmasterTools messages. If you received a message about “artificial or unnatural links” then you probably have some quite serious work ahead of you before you can submit a re-inclusion request that will reverse the penalties. Fortunately, Majestic SEO has several investigative tools to help you get to the root of the problem… identifying your problem links. If you don’t follow a structured approach, you are in danger of throwing out the baby with the bathwater, as you cull your site of inbound links from websites that added them in good faith – creating potential relationship issues within your company which could have been avoided.
I hope this blog post helps you in your hour of need, to give you some methodologies for cleaning up your bad links, by giving you five unique investigations that you can start using right away.
Investigation 1: Finding those Site Wide Links.
(Required: Majestic Site Explorer, Silver subscription. Or use GWT for free)
Most of these investigations are free – but finding the websites that have site-wide links to you is best seen in Majestic Site Explorer. If you cannot bring yourself to part with a silver subscription, this is one report which has a similar download in Google’s Webmastertools.
If you are logged in to Majestic SEO, the data you need is just two clicks from the home page:
1: Type your site into the Site Explorer Bar on the home page. 2: Switch the option to Historic Index and 3: Click Explore. You probably want Historic Index for this investigation, because Google will have penalized your site also based on historic data. Whilst all your decent links are bound to be in the Fresh index (which includes old links as long as they are strong enough to continually get recrawled by us) this analysis is all about rooting out your rubbish! In addition, links only just created are unlikely to be part of Google’s current link network penalties as they have been targeting paid networks and link exchange networks and have been working on this for many months.
4: Click on the Referring Domains tab. If the sort order is different, I suggest you sort by “Matched Links”, descending. This will list sites linking to you based on the number of links they have pointing to you. This makes it very easy to see site wide links into your site, and because we also include the Alexa ranking for each domain, you can quickly assess whether the sitewide links need further investigation or whether they are benign:
If you are happy with the majority of those site-wide links – do NOT jump into getting rid of them all. But these sites should all really be showing up in your referring sites data in your analytics. Otherwise, they really might be dangerous as they seem to have little or no practical value outside Google manipulation. It might be that the referrer data gets lost in the link methodology though – so do check before you go overboard.
In our example, we do get traffic from the sites on this list – even though they might at first seem a little odd. That is because they are mostly site assessment tools that use our light API or show our back-link history tool. These can send us more traffic than Google – so we are unlikely to care about Google’s opinion of why we think the link might be there. But you may.
One of the good things about this report – apart from it being so quick to generate – is that the Site Explorer excludes nofollow links. So whilst DMoz.fr has half a million links into MajesticSEO in the footer, they are no-followed and therefore will not be problematic for Google and will not show in this report.
Investigation 2: Seek out over-hyped anchor text
(Required: Standard report. Free for self verified sites.)
Looking at the types of penalties that have been reported, Google is especially penalizing sites who over-egg anchor text links at this stage. So wouldn’t it be great to see all your links based on how frequently the anchor text is used?
Well no problem. The best way to do this is to generate a standard report. To do this, do steps 1 to 3 above and then click on the “Generate Report” icon or button:
Then get your standard report:
The report will now be accessible from your “reports” button. The link to this is always in the very top right hand corner of your screen.
When you navigate to the report, you can sort the links based on anchor text:
Strangely – we suddenly see that for some reason, we have over 50 domains linking to us with the anchor text “pandaranol stats”. By flipping the standard report to analyze by back-links instead of anchor text and filtering to JUST see links with the anchor text “pandaranol stats” we can see these all listed. Indeed, we can see that all these links originated from one article and are actually being picked up within a heavily used RSS feed:
When we go to the article, it seems to be a very well researched article – although it is not in English so I cannot really tell. However – what I can see is that if I decided that this was harming our business, at least I could simply talk to the owners of the RSS feed, rather than all 51 sites syndicating the content.
Investigation 3: Finding the ONE BAD 301
This one is harder – because you will need to be reviewing links one by one – but a common tactic by the SEOs over the years has been to buy up parked domains and redirect them to an SEO’d site. Another tactic has been to 301 WordPress articles already ranking for a phrase. This means that the parked domain or url might have 1,000 links going to it, but in standard back-link reports, they seem like just 1 link, which is the parked domain or URL.
Spotting an odd link in Majestic SEO, just click on the “Explorer” icon (A ship’s wheel) usually showing next to the link – or put the link you want to look at into the site explorer search bar. Majestic reports on what kind of response was found when we crawled that URL if we know:
The screenshot above shows what you are looking for. (I have for the screenshot used a page I KNOW redirects, which is the non-www version of our home page.) Where this is a third party URL, linking to your site from a page which is, itself, a 301 redirect, all those external backlinks from all those referring domains may also need to be looked at for signs of unnatural activity. In this case, that would be another 142,502 links from 127 domains if I was worried. But luckily – if these WERE the root of my backlink problem, I would only need to get the owner of the url that redirects to send his redirection elsewhere to get rid of all of these links as a Google link profile problem.
Investigation 4: Finding Link Networks that leave no on-page footprint
(Requires subscription. The bigger the better)
This is a little theoretical – but hear me out. Just as you can use Clique Hunter to source GOOD neighbourhoods, now you can use Clique Hunter to help source BAD neighbourhoods… or link networks.
If you have found a web page which is clearly selling links… including anchor text links to your site… then presumably there are plenty of other sites in a similar boat. You can check this, to a point, by seeing if those other sites are also comprehensively NOT ranking for the anchor text they have listed on the suspect site. Now obviously, if that is the case, then getting your links off the suspect site is a no-brainer. But before you do, you may be able to root out more of the network, by collecting the other websites that are being penalized as well as your own, and popping up to ten of them into Clique-Hunter:
This will not only return a list of websites that link to ALL (or most) of the sites in your polluted list, but will also tell you how many links each site has to each of the penalized sites. As with most of Majestic SEO’s reports, you can export Clique Hunter results to a CSV and then tackle the likely suspects one at a time if necessary – or look for feeds, which might indicate the source of the pollution.
The length of the CSV file depends on your options and your subscription level. A subscription will show you from 1,000 up to 25,000 referring domains (if we have them) depending on the level.
Investigation 5: finding your poorest back-links
(Free for your own sites)
Majestic Site Explorer and our Standard reports are mainly designed to show you the strongest links. Often, the weakest ones fall off the bottom of these reports. However, advanced reports now have a new option – letting you download ALL your links into a CSV file, ordered by ACRank.
If you have the energy, you can go and get all these links and start looking at them from the bottom up. This is a massive task, though. But the option is there if you really want it! Here’s what you do.
Follow steps 1-3 at the start of this article, but instead of getting a standard report, switch to an advanced report. Then navigate to the report from the reports and tools menu. In the Download options, there is the chance to download every link to your site.
Good Luck!
I hope these tips give you some ideas for structuring your own approach to handling a message from Google about unnatural links.
- How Important will Backlinks be in 2023? - February 20, 2023
- What is in a Link? - October 25, 2022
- An Interview with… Ash Nallawalla - August 23, 2022
Thanks for that useful tircks.
April 23, 2012 at 11:50 amVery good article that guys. Very good indeed, thanks.
April 23, 2012 at 1:46 pmFake links are also often used by Hackers to destroy the ranking of Websites in Google. Might be good to have such an tool to monitor whats going on.
April 23, 2012 at 2:20 pmYou are right – there does seem to be an increase in “Negative SEO” recently, but we have no tools to be the arbiter of a “real” or a “fake” link. A link is a matter of fact. “Fake” becomes a matter of opinion, which we would only want to tackle is we could do so at scale, for every link in our database.
April 23, 2012 at 3:11 pmWhat about using Google Alerts to track new links into the domain. Of course you would need to eyeball all these everyday to make sure they are legit and not Negative SEO.
May 16, 2012 at 3:34 amThank you for sharing these 5 useful processes.
Just a few questions:
1. Quite often, I find that the historic index contains a lot of dead links. These are links that no longer exist. Is there any way using majestic to weed out the dead ones and only keep the live ones only?
2. When working our the anchor text distribution during investigation 2, how are site-wide links treated? Are all site-wide links being counted as external backlinks?
3. Given that Google is removing thousands of websites from its index, how is Majestic SEO going to treat these sites? Will you stop reporting them or not? Will the historic and fresh indexes treat these sites equally?
April 23, 2012 at 2:28 pmHi Modi,
Great questions
1: The Historic index does include dead links… not least because we do not exclude a link even if we DO see it as dead. We simply flag it as “deleted”. You can filter out deleted links in the filters of advanced reports, however, if you have a very large site then using the Fresh index may be more practical as this will only include links that we have checked within 30 days. The other way to weed out the dead ones is to “ping” all of the links and get them verified. Tools that do that include Linkdex and CognitiveSEO and there may be others through our partners. We cannot check 3.7 trillion links every week – so the third party tools can start with all links to a site and can then use their own spiders/technology to verify the ones you specify.
2: site wide links are counted on every page that we see a link from – so if a site has 1000 pages that link to you, the numbers will say there are 1000 links from that domain. However – in standard reports we default to showing you one per domain… this is EASILY overridden in the standard report filters on the fly. In advanced reports – the “Download links” button will give you all of the sitewides.
3: Our index is independent of Google and will remain so. If Google has banned a site, it does not mean there are no longer links to that site. We map the link graph – not Google’s interpretation of it.
Hope that helps.
April 23, 2012 at 3:09 pmThanks for the useful responses.
Going back to your 1st answer, I guess that the fresh index would also include dead links but to a lesser extent. I’m a bit confused with the definitions of the two indexes as I’ve read that in the case of the fresh index found URLs will be dropped after 30 days so they get rediscovered later. Could you possibly clarify this a little bit more?
Also, with regards to the historic index, what is the length of time it covers and how often is it being recreated? So, say I pull out all historic index backlinks today, when will that get updated again? Is there any way to know when was the last time the historic index was updated? Does the fresh index update the historic one or are the two completely separate?
April 23, 2012 at 3:31 pm>>I’m a bit confused with the definitions of the two indexes as I’ve read that in the case of the fresh index found URLs will be dropped after 30 days so they get rediscovered later. << Yes - if we find a URL, we record the date first found and last seen in each database. If a link does not get reseen for 30 days it will drop out of Fresh, but if we THEN rediscover it, we will assume it is new in the Fresh Index. Given the size of the web, this happens much more than you might expect, but the trade off is fast indexing/updating vs database size.
April 23, 2012 at 5:48 pmGreat guide to recovering a site. I actually did the entire process last night for someone — nice to see that my methodology is validated here! I was able to find a bunch of problematic links, so hopefully removing them and having a reconsideration request from G will get our rankings back.
April 23, 2012 at 3:41 pmWell please come back and tell us all if your re-inclusion worked! 🙂
April 23, 2012 at 3:42 pmGreat methodology. Looks like the main offender behind many of these notifications are site-wide links i.e. blogrolls.
April 23, 2012 at 3:58 pmDepending on the implications of the over optimization penalty that’s on the way, these tips could be very, very helpful in diagnosing and fixing a penalty that’s link-based.
Thanks for sharing Dixon. Best regards.
April 23, 2012 at 11:25 pmHow can I get a free trial of majestic site explorer?
April 24, 2012 at 7:54 amYou would need to see us in person at a conference. Alterntively by Joining SEMPO I think there is an offer on. You can also get reports on your own website free of charge along with lots of other stuff. The site has updated since this webinar – but have a look at what you get for free. Maybe that should be the subject of my next post.
April 24, 2012 at 8:01 amIs this post a sales pitch for your tool or an honest opinion about what is going on with Google right now? How do you know that the SERP drop is due to a penalty? Have you got word from Google or is this just a speculation?
Google didn’t say that if you have unnatural links you will get a penalty…or anything like that. You are just speculating right?
I have read a lot about this from people like Erik Ward and the most logical conclusion is that to assume you have lost rank due to link value loss or complete link loss.
Majestic seo is a great tool but this post is not!
April 24, 2012 at 10:18 amHi David. Sites are being penalised all over the place right now. Matt Cutts also just posted on Inside Search too, so I wasn’t too far off the mark.
April 25, 2012 at 2:53 amDavid Pull your head in, did you not read the part at the start that said all suggestions can be done on the free version except one??
Even if it was a sales pitch, it’s a damn good tool and should be in the hands of as many decent people as possible, so by majestic not promoting they would be doing the community a disservice.
May 16, 2012 at 2:23 pmThis is a very interesting topic, and I liked your article a lot. We monitor ourselves a few websites with unnatural links, and now majesticseo makes it easy 🙂
April 24, 2012 at 3:28 pmLots of webmasters remove their “bad” links in the moment. It seems, it doesn´t help…
Yeah I agree with Yousaf. Have you noticed that alot of one page websites are causing the same issue! any links that you have gathered where the linking domain have been de indexed has caused problems that I have seen.
April 24, 2012 at 10:49 pmI’m still not convinced Google is applying penalties to sites with crap links. Clearly they’ve done a fantastic job in identifying these bogus link networks and discounting them, and sites that relied on them are taking a bath, but penalties? No. Too dangerous. If this were on the level all I need to do to get a good listing is spam my competitors with links from SEOLinkVine.
I think Google is using some scare tactics again.
Clearly there are changes afoot that have already really improved the listings, but going back and trying to get delisted from spammy sites is a waste of time. Just keep building GOOD links and leave your spammy ways behind and I’m pretty sure you’ll see Google is still treating you fairly.
April 25, 2012 at 4:37 amRyan,Yes, you are correct, but no foollw links do serve other purposes in that they help to promote your brand overall and can drive traffic to your site.Oh and in the future do not try to sneak in links in your comments as this may count as spam
December 25, 2012 at 4:47 pmThank you for this great article. There’s one thing I don’t understand: what do you mean by “over-egg” in the phrase “Google is especially penalizing sites who over-egg anchor text links”. From the context I’m guessing you mean “over-use”?
April 25, 2012 at 7:27 amSorry. That means “Using the same anchor text repeatedly in links”
April 25, 2012 at 3:05 pmYeah great post on how to find spammy links. Not 100% sure you need to get them all removed, depends on if you have a penalty or not. Most sites those links are just devalued and if that is the case it might be better to focus on getting quality links instead.
April 25, 2012 at 2:29 pmThis is crazy. Google is throwing out the baby with the bathwater.
I understand that blogrolls are potentially subject to abuse, but they are also a long-standing tradition in the blogosphere, and when used properly, should actually trigger a HIGHER transfer of authority than a single link.
Putting a site in your blogroll means “I like this site and read it on an ongoing basis. I’m willing to recommend it and stake my reputation on that recommendation.” That should convey far more authority than “I read this post on another site and here’s what I think about it.”
It’s also common practice for small groups/associations to make a blogroll of their members — networking groups, volunteer groups, etc. And it’s been that way for a decade.
And what about your social media links? Those social media buttons are standard to do sitewide. What if you don’t do buttons but text links?
Frankly, this is about the most ridiculous move on Google’s part I’ve seen to date.
April 26, 2012 at 5:48 pmI don’t think Google is objecting to Blogroll links… they are objecting to blatant manipulation. I am more than comforatbel leaving my personal Blogroll links… but as Matt Cutts is on there, I am sure he’d be sending me a cease and desist if it was wrong!
Seriously – I cannot speak for them, but Google isn’t objecting to blogroll links. That doesn’t mean Google would never consider blogroll links as against guidelines, but you would have to be quite blatant.
Dixon.
April 26, 2012 at 6:08 pmLink everything else, it depends on the link. If you have 50 blogroll links going to a bunch of unrelated sites then that is probably a bad signal. Also, if some of those sites have been identified as spammy then that is also probably not a good thing for your site either. >
April 28, 2012 at 2:26 pmgot it . but confused as already got warning message from google .
will it work for me ??
May 1, 2012 at 12:46 pmTime will tell. These are only pointers. We are not a consultancy – just a technology.
May 2, 2012 at 1:01 pmThanks for a great post, Dixon – it really opened our eyes and even confirmed us in some of the thougts we already had (especially the part about site wide links … a word I wasn’t familiar with before now).
To avoid any misunderstandings, let me point out that we are not hit by “Penguin” – I just find the update extremely interesting, as always when something “big” happens in google 🙂
Anyway, what is your opinion about site wide links from high quality sites with high relevance? Our main site, Texaspoker.dk, has 500.000 links from one of our close partners, Betxpert.com (as their name reveals, they are all about sports betting and we are all about poker) – should we be worried about those links?
Betxpert.com has placed a link in their footer beside some links included in the content when relevant on their site …. it might be a good idea to ask them to remove the footer link?
Thanks a lot – and if this question is too specific, please accept my apologies. I just got extremely encouraged by this “spot on” post about some of the things we have been talking about the last couple of weeks.
Good day all.
Nicolai K
May 4, 2012 at 9:17 amI can’t give individual help on this blog – but if you are worried about sitewides, check if others on the same site are getting penalized and look out for the email from Google. If you get the email, thenI would be worried. However… many of the unnatural links that I think are being penalized are MUCH more blatantly manipulating Google than partner footer links.
May 4, 2012 at 10:13 amThanks for the tips! I haven’t been hit by the pingiun update but will keep these investigative tools in mind for the future.
May 4, 2012 at 4:42 pmThanks, well written and usefull information.
May 5, 2012 at 8:04 amGreat tips on Penguin for how to investigate unnatural links and delve into Google Webmaster tools.
Thanks!
May 10, 2012 at 2:37 pmHi Dixon,
We just completed our own investigations using majesticseo as our source of backlink data which backs up a lot of what you say above.
We also found out some interesting things relating to stemmed versions of keywords and how the “strength” of exact match domains have altered.
You can read the report here http://www.sayulabs.co.uk/how-have-the-recent-google-panda-and-penguin-updates-affected-rankings/
May 15, 2012 at 7:19 pmExcellent article.
May 15, 2012 at 7:48 pmVideo walk thru please?
May 15, 2012 at 8:18 pmWe’ll get on it!
May 15, 2012 at 8:25 pmThanks a lot 🙂 I had got a penalty due to Panda, used Majestic and was finally able to get the penalty removed !!
May 16, 2012 at 6:17 amOh great! Hope you blogged about how you got out of Panda hell.
May 16, 2012 at 11:20 amThank you for your post on this subject. It is a really useful guide to help with this common problem.
Once client of mine has had a problem with the ‘unnatural links’ warning although this was before I took over as seo consultant.
I am a little worried about the anchor text issue – because there is only some much variance that can be used to create them. However, I am doubtful about it being Google’s biggest grudge here. Relevance, in my opinion, is more important. If a poster is referring to a brand for example, it is highly likely that the anchor text will be (or at least include) the brand. That consequently means the regular repetition of anchor text but they are still going to look natural.
I think that it is more likely to be sitewide links, (paid for) rererrer sites, and dodgy links pointing to old URLs and then being redirected that are the problem.
Would the best remedy be to accept the penalty and aim to create better links going forward, or to look to remove the offending links?
The reason I ask is because Google, I’m sure, has stated in the past that it will not penalise a site for bad links pointing to it (because a site owner cannot control which sites decide to link to it). I know that sounds insane – especially bearing in mind the number of sites that have been affected by the “unnatural links email”. However, is it not actually a case of these links having lost their value rather than “being bad”?
Based on this, I think that the former should be the best remedy: i.e make the effort to improve the link strategy going forward – rather than correcting mistakes made in the past.
May 16, 2012 at 9:20 amI think very bad links will – in the long run – remain a problem. So get rid if you can. Google is going to want to see you sweat before it forgives.
However – don’t through the baby out with the bath water. If links are bringing you traffic, don’t go killing them! That would be suicide and increase Google’s monopoly through the back door.
May 16, 2012 at 11:23 am> I agree with Will here. Google will come back and take the “bad links” penalties away (if they were penalties at all). The main reason is that it would be too easy to game the system that way. For instance if I were in position 9 and I wanted to get to 1, then all I would have to do is create some really bad spun content and post it to the blog networks that Google already knows about with links to the above 8 sites. Then I would just sit back and wait for my competitors to drop like a rock. Simple.
So – I think Google is better than that. If the Penquin update truly did penalize links and not just devalue them, then they will have to retract or refine the algo update as only crap sites would rise to the top and blackhat SEO’s would find it really easy to game Google, which is counter-intuitive to what Google is trying to accomplish.
May 16, 2012 at 3:02 pmThanks for the post guys. I think this areas is a really interesting one. We’ve seen one of our own websites hit, but when we did some analysis – using majestic – it did appear that we had too many keyword rich anchor texts, I think compounded by the fact that we have a keyword rich url, but its a legitimate ecommerce website.
May 16, 2012 at 12:47 pmGoogle is surely making it harder and harder to rank and stay there.
May 16, 2012 at 3:33 pmWhat if you dont have a Google webmaster account, some of us are just guys and girls who set up wordpress sites, following the advice of the day I did not use spammy links, instead I used the highly recommended BMR to increase my links with what were at the time regarded as quality. Now my sites have dropped but with no webmaster account no idea if I can get them back even using this as I have no idea what Google has done.
May 16, 2012 at 4:32 pmThen now is the time to go and get a Google Webmster tools account! Go on. It is free. Just start at google.com/webmasters.
May 16, 2012 at 5:11 pm> Your post is actually really helpful! I was curious about whether having a webmaster account was actually detrimental to a site’s listing i.e. if you have one – Google assumes that you are in the business of ‘manipulating’ rankings…I was even thinking about removing sites from mine! However, you have shown that the penalty has affected those without webmaster tools too! So to the dustbin with that myth – hurrah 🙂
May 16, 2012 at 7:03 pmI would not be quite so hasty. The latest SERPs are full of inconsistencies and my worry is that we all ‘pile in’ and change everything.
I found this link particularly enlightening !!!
http://www.webpronews.com/the-blurry-lines-of-googles-paid-links-policy-2012-05
May 16, 2012 at 5:16 pmAnother killer majestic seo resource. I have been using majesticseo for about a year now and it literally kicks butt.
That being said I agree with Dr Ian above it is probably not the best idea going and stripping away all your links just yet. My understanding is that poor links have been devalued and no longer count. So in theory you could be taking away links that are actually giving you rankings.
May 17, 2012 at 9:11 amOne of the ‘leading’ SEO agencies in the UK has a system totally reliant on a huge number of blogs, registered to the same person, on the same server and ALL with links to all its clients websites.
May 17, 2012 at 5:36 pmAND, Google seems to ignore the network.
What do you do when Google is the culprit? We lost all of our rankings after 15 years of total compliance with their guidelines. After some research in Google’s Webmaster tools I see in external links to us that they report 2 websites have OVER 40,000 back links to us.
This is bogus. Neither site even has 40,000 pages to link to us from! When I go to their site I see maybe 5 back links. That’s it. How do you fix this when Google obviously has a reporting issue that penalizes you for THEIR reporting mistake??
HELP!
October 13, 2012 at 12:36 amHi Rebecca,
I did look at your back-link profile. I can certainly see 40,000 links and more in our historic data, although many of these may no longer exist and our spiders may have operated like Google… finding the links, then not recrawling the data. However – there certainly are links to your site that I would say were created with no other purpose than to push up rankings and these are the one to focus on. If you honestly think there are none left (or very few) then put in a re-inclusion request – but be warned… tens of thousands of people are in the same boat as you and Google is not in a charitable state on mind. Unfortunately, I cannot provide further consulting advice via MajesticSEO and I took off the name of your site, because it would spark a specific discussion which – if it is going to be public – would be better had on the Google discussion forums or a place like the “review my site” forum in the members only area of webmasterworld.com (other forums are available.)
October 13, 2012 at 12:41 pmThanks for the info. I would like to know which one is the most damaging one on that list, any idea ?
October 29, 2012 at 9:02 pmThis methodology will help you identify candidates, but ultimately, you shoul look at the pages individually.
October 30, 2012 at 7:35 amAnyone who know provide the service to remove this badlinks? How much does it going to cost?
November 16, 2012 at 12:54 amThere are companies like www.removeem.com who use your Majestic account data to help with this process. You would need to contact them directly.
November 16, 2012 at 5:57 pmThanks a lot for this post. Its a lot of work to do, but should be worth it. 🙂
December 16, 2012 at 9:10 pmKind regards, Frank.