Contact us for a free quote we would love to work for you

How to fix crawl errors fast

I had been asked to review a website, that was doing well and things changed over time.  The site had never performed to where the customer thought it should have in regards to the topic and now it had dropped further.

I ran my SEO software on the site and found a few basic errors but the website was well established, used all the right social sites, and blogged as part of their site regularly.

I then asked for their Google Webmaster account information so I could check and see what Google had to say about the site. Most of the information was pretty normal I checked the Sitemap and found issues and that will be a topic of another blog. So I went to Diagnostics.

This is shown below by the red error for those that don’t get into their account much.

Google Webmaster Tools 1

Then I went to the section crawl errors, shown below by the red arrow.

Google Webmaster Tools 2

Here is the chart of the site showing the graph of the errors. I have removed the actual errors of this site due to my privacy statement to my client.

I have condensed the error graph to fit in this document but you can see there were a number of crawl errors and these went on for months. Using techniques and software I will show you how to fix these using one of our test websites http:crewwebsite.com. How we started fixing these and you can see the significant drop. The little bumps are because the more GoogleBot crawls and as the site changes daily a few errors have popped back in.

Crew Website is not in our Webmaster Tools so I will start with the first piece of software I tried Screaming Frog SEO Spider 1.9

This software you download on your computer, type in your url and it searches every inch of your website, it will show the good the, relocated and the bad. It is very good at finding out everything about your website. Below in a very condensed screen shot are the errors it found for CrewWebsite.com I have it sorted with the crawl errors first and not every page.

Screaming Frog 2

This screen shot is not a good representation of the software and past the shot to the right is more information to try to help you find it. There is a paid version and I have not tried it or researched this tool other than to find out it finds errors very well.

But I still could not find the errors in the website fast enough and fix them timely to get through the huge amount of errors this site had.

So with a little research I found an online tool that helped me find the errors quickly.

I found Online Broken Link Checker It is an online tool that checks for broken links, like the name implies.

broken link 1

There site is much nicer but for the limited space, I am showing you where you will put your url.

broken link 3

Set for report all occurrences after you have typed your domain name.

Put the security code in and hit the find broken links now button. If you have a very large site set this way you may need two things, lots to drink and maybe a snack. If your site is over 3000 pages, uhg time to clean house a little but a big commercial site is often that big and I have checked some active blogs trying this tool out and they reached the 3000 page limit. So for free you can do up to 3000 pages. If you want to do more contact Broken Link and they will tailor a plan for you which can include automatic schedule runs and Email and more.

But for my purpose the free version worked on this site.

Here is the report for Crewwebsite.com:

It found 5 broken links compared to Screaming Frogs 186 errors, but these are the ones normally seen in crawl errors in Webmaster tools and on the site I had something like 300 errors to get through. Looking at these errors on CrewWebsite.com I knew what they were, but if you’re not an expert (and I am not no one truly is today” and want to help yourself. Next you would hit the src blue letters and below is a very small section of the code of the page it returns. Where you see the <a href=”page_id=2″> this is where the problem is:


This is a WordPress Site and original the permalink setting was default as shown below.

crewweb1
Following many SEO concepts the Permalink was changed to Post Name:

crewweb2
In this theme there was a custom box with text and the page url so I had to change the <a href=”page_id=2″> to a href=http://crewwebsite.com/news the new url and that problem was fixed as shown below. The http://crewwebsitecom/?page_id=2> is no longer there, I could go ahead and fix the rest of the problems, but I am going to make a video of the process.

crewweb4

And put your website address so I can look at it and we will help fix your problems. You will be amazed at how cheaply we will do this and at the same time make any recommendations we see that can be fixed easily.

All items from Google, Screaming Frog, and Broken Links are property of each company exclusively.

Copyright © 2012 KillerBeeMarketing.com

Comments are closed.