What are broken links?
Links that no longer work are Broken Links.
This could be because the page has gone or because there was a typo in the URL.
Either way, the link is broken and this leads to poor user experience.
You should make sure users have the best experience possible. So, you need to fix broken links.
In this article, we are going to look at how you can find and fix these links. We will look at four tools to find broken links.
Let's start with the difference between dead and broken links.
What is the difference between dead and broken links?
The term dead link and broken link are often used interchangeably but they are different.
A dead link is a link that did work when first created but then the page linked to it disappears. This happens on e-commerce sites when a product is no longer for sale and the website owner deletes the page.
A user will visit your site, click the dead link and see that the page has gone.
This is sometimes referred to as a 404 page. A 404 page is the “not found” page and the number comes from the status code a computer sends when a page is not found.
So how is this different for a broken link?
What causes a broken link?
The term “broken link” is a catch-all term for any link that does not work. A dead link is a type of broken link. There are many other reasons why a link becomes broken:
- The URL is wrong
- The website owner has removed the page
- The web server is down or unavailable
- You do not have permission to the page
- Robots.txt is blocking access
- Redirects are stoping access
- and many more technical reasons
The site Dead Link City is a website built to show all the different types of broken links that exist.
There are 74 different types of broken links on that one page and it is a great site to test Broken Link tools.
To find the best tool we are going to use the Dead Link City site to check some tools and find the best.
The tools we are going to look at are:
- W3 Link Check - Online tool from the web standards body.
- Dead Link Checker - Online tool that you can use to check a single page for free.
- Broken Link Checker - A command-line tool that you can run as part of a website developer pipeline.
- Screaming Frog - A technical SEO tool that you can use to crawl a website and find broken links.
Let's look at each of these tools and see which one performs the best.
How do you find and fix a broken link?
We are going to run through each tool and find the one that can find the most links.
If the tool can find all 74 then that is the best tool.
If we can automate the tool so that it can run daily then it will get extra points.
Let's start with the W3 Link Check tool:
W3 Link Check
The site is easy to use and is great for checking a site one page at a time. Go to the link check tool and enter your URL into the box.
Leave the default settings and click the “check” button.
After the check is complete you will get a very detailed summary of the broken links. In my test, the w3 link checker was able to find 69 out of 74 of the broken links.
It also offered a lot of detail on each of the errors and suggested fixes to the problem. Here is the advice for the 404 pages:
The link is broken. Double-check that you have not made any typo, or mistake in copy-pasting. If the link points to a resource that no longer exists, you may want to remove or fix the link.
This is a good tool but it did miss 5 of the broken links.
Let's see if the Dead Link Checker can do any better.
Dead Link Checker
Dead Link Checker works in the same way as the W3 tool go ahead and add your URL into the text box and click “Check”:
Dead Link Checker will then give you a long list of all the broken links. In my test, it found all 74 of the broken links!
There are a few downsides. The tool does not give much information on how to fix the broken links. You are on your own to figure out what is wrong and fix the errors.
They do offer a paid service that will check your pages automatically for you every day. This costs from $10 a month to check 50,000 pages on your site.
This tool is the best at finding broken links. But, you must be happy to investigate the cause of the broken link yourself.
The last two tools are useful for web developers and Technical SEOs.
Broken Link Checker
We are going to install and use a command-line tool called broken-link-checker. If you are not comfortable with using the terminal app then this is not the tool for you. Yet, this tool is useful for web developers who can add it to their daily checks.
It uses node.js so you need to make sure that you have this installed on your computer.
The fastest way to do this on a mac is by using a tool called Brew. To install this open your terminal app.
brew install node
Once you have node installed you can install the broken link checker using this command:
npm install broken-link-checker -g
Now you have a new tool called blc installed. To run a broken link check on Dead Link City run this command:
blc http://www.deadlinkcity.com/ -ro
The -ro at the end of the line means that the scanner will crawl the website and keep the links in order.
Here are the results:
Getting links from: http://www.deadlinkcity.com/
├───OK─── http://www.deadlinkcity.com/style.css?r=2
├─BROKEN─ http://www.deadlinkcity.com/missing-head-link-href.txt (HTTP_404)
├───OK─── http://www.deadlinkcity.com/examplestyle.css
├───OK─── http://www.deadlinkcity.com/comparison.asp
├───OK─── http://www.deadlinkcity.com/errorlist.asp
├─BROKEN─ http://www.deadlinkcity.com/error-page.asp?e=400 (HTTP_400)
├─BROKEN─ http://www.deadlinkcity.com/error-page.asp?e=401 (HTTP_401)
├─BROKEN─ http://www.deadlinkcity.com/error-page.asp?e=402 (HTTP_402)
...
├─BROKEN─ http://www.deadlinkcity.com/missing-q-cite.html (HTTP_404)
├─BROKEN─ http://www.deadlinkcity.com/missing-script-src.js (HTTP_404)
├─BROKEN─ http://www.deadlinkcity.com/missing-video-src.mp4 (HTTP_404)
├─BROKEN─ http://www.deadlinkcity.com/images/missing-video-poster.jpg (HTTP_404)
└───OK─── http://www.deadlinkcity.com/ (CACHED)
Finished! 87 links found. 64 broken.This tool has only found 64 of the broken links. Which is not as good as the Dead Link Checker.
I would still recommend this tool for web developers as it fits as part of your website build process. This means that developers can catch broken links before they go live.
Let's look at the last tool Screaming Frog.
Screaming Frog
The last tool we are going to look at is popular with Technical SEOs to dive into the technical details of a site. This tool will crawl a web site and return loads of information. It is free for sites with less than 500 pages.
To do this first download screaming frog and install it on to your computer. Once installed you will see a screen like this:
Type the URL in the bar at the top and click “Start”.
Once the scan is complete you will see that there are lots of data in the tables. Click on the “response codes” tab to see the errors:
There is a dropdown so you can select the type of response codes:
Here are the results of the scan:
Screaming Frog was only able to find 66 of the 74 broken links.
If you already have Screaming Frog this is how you would use it to find broken links but remember it is not going to find them all and it won't tell you how to fix them.
Wrapping Up, What are broken links?
We have looked at what are broken links and what causes them.
Out of the four tools, there was only one that found all the links. The tool I recommend is Dead Link Checker which found all 74 broken links. If you can afford it then signup to the automatic checking.
For web developers, I would add the Broken Link Checker to your development workflow. This will catch any broken links before they go live.
If you are a Technical SEO and you already have Screaming Frog installed then you can give that a go. It only found 66 of the broken links. Yet, if you understand Screaming Frog you may be able to tweak the settings to discover all 74.