Crawl Errors in Search Console? What They Mean and Fixes

1. Introduction

Quick Answer: A crawl error is a failed fetch. Googlebot asked your server for a URL and got back an error, a timeout, or nothing at all — so that page couldn’t be read, and if Google can’t read a page, it can’t rank it.

Crawling is step one of getting found on Google. Before a page can be indexed or ranked, Googlebot has to fetch it successfully, and a crawl error breaks that first step. The good news: crawl errors are specific and diagnosable. Each type points at a clear cause, and most trace back to your own server or CMS.

Even a small business site can leak rankings when Google keeps hitting dead URLs. The rest of this guide is a plain-English tour of every common crawl error and the exact fix for each. If a page is crawled fine but still won’t rank, that’s a different track — see our guide on pages that won’t get indexed.

2. What Crawl Errors Actually Mean

Quick Answer: “Crawl error” is an umbrella term for any URL Googlebot tried but failed to fetch or read properly. It covers missing pages (404), server failures (5xx), blocked pages, broken redirects, and DNS problems — each with its own cause and its own fix.

People still say “crawl errors” out of habit, but Google split the concept up years ago. Today an error is one of two things: Google reached your server but got an error code back, or it couldn’t reach your server at all. Everything else is a variation on those two.

One important note: not every flagged URL needs fixing. A 404 on a page you deleted on purpose is working as intended. The skill is telling real errors from healthy noise, and this guide draws that line for each type. Pages Google chose not to index for other reasons show up separately as Excluded in the Coverage report.

3. Site-Level vs URL-Level Errors

Quick Answer: Site-level errors hit your whole domain at once — DNS failures, server outages, and robots.txt problems. URL-level errors hit single pages — 404s, soft 404s, redirect errors, and access blocks. Site-level issues are the emergency; fix those first.

This split decides your priority. If Google can’t resolve your domain or reach your server, nothing gets crawled and rankings can slide across the board within days. A single 404 on an old blog post is minor by comparison. Always scan a fresh error list for site-wide problems first.

4. Which Crawl Errors Show Up Most

Quick Answer: Across ZenWeb client audits, “Not found” (404) is by far the most common crawl error, followed by server errors and robots.txt blocks. Knowing the usual suspects tells you what to check first when a fresh error list appears.

Not every error is equally likely. When we audit a Malaysian SME site, the same ranking shows up over and over. Deleted pages that are still linked somewhere top the list — which is why a 404 is the error most owners meet first.

5. Where to Find Crawl Errors in Search Console

Quick Answer: Google retired the standalone “Crawl Errors” report in 2019. Crawl errors now live in three places: the Pages report (per-URL indexing reasons), the Crawl Stats report (host status and response codes), and the URL Inspection tool (a single live URL).

If you are hunting for the old report, stop — it is gone. Google folded its data into newer tools. Here is where each crawl error now surfaces:

• Pages report. Under “Why pages aren’t indexed”, this lists URL-level errors — Not found (404), Soft 404, Server error (5xx), redirect problems, and blocked pages — with a table you can click into per issue, as Google’s Page indexing report documentation explains.
• Crawl Stats report. Found under Settings, this shows host status, total crawl requests, and a breakdown by response code — your window into site-level DNS and server problems, per the Crawl Stats report documentation.
• URL Inspection. Paste any single URL to see the last crawl result, the returned status code, and whether Google could fetch and render it live.

6. Server Errors, 404s, and Access Blocks

Quick Answer: A 5xx server error means your server failed to respond and Google slows its crawl. A 404 means the page is gone. A 403 or 401 means Google was refused access. Each has a different fix, so read the returned status code before you act.

Most URL-level errors fall into a few families:

• Server error (5xx). Your server was overloaded, timed out, or crashed when Google called. Google reads a sustained 5xx as “back off” and crawls less until the server recovers.
• Not found (404). The URL points to nothing. Fine if you deleted the page on purpose; a problem if the page should exist or is still linked from elsewhere.
• Soft 404. The page returns a “200 OK” but looks empty or says “not found” in its content. Google treats it as missing anyway. Common on thin or out-of-stock pages.
• Blocked / 403 / 401. Google was actively refused — by robots.txt, a login wall, a firewall, or a security plugin blocking the Googlebot user-agent.

Google’s own reference on HTTP status codes, network and DNS errors confirms how each code is handled: 4xx codes exclude the URL, sustained 5xx codes slow crawling and can eventually drop pages, and network or DNS failures are treated much like a 5xx.

7. What Crawl Errors Do to Indexing

Quick Answer: An unfixed crawl error eventually costs you the page. If Google can’t fetch a URL, it stops indexing it, and already-ranked pages drop out over days to a few weeks depending on the error type.

Left alone, a crawl error removes pages from Google, and how fast depends on the type. A server or DNS failure signals your site may be down, so Google drops unreachable URLs quickly. A single 404 takes longer, because Google waits to be sure the page is really gone.

This is also how a healthy page can quietly vanish. If a URL that used to rank starts throwing a 404 or 5xx, it will slide out of the index on the timeline above. When that happens to a page you want back, follow our guide on getting a deindexed page back.

8. How to Fix Crawl Errors, Step by Step

Quick Answer: Sort errors by type in the Pages report, confirm the failure with URL Inspection, fix it at the source based on the status code, make the URL return the correct code, then use Validate Fix so Google re-checks in a batch.

The workflow is the same for every error — only the middle step changes. Work through it in order:

1. Sort by error type. Open the Pages report and group the flagged URLs by reason so you fix all 404s together, all 5xx together, and so on.
2. Confirm with URL Inspection. Run a live test on a sample URL to see the real status code — sometimes the error has already cleared, or a plugin is blocking only Googlebot.
3. Fix at the source. For a 404, restore the page or 301-redirect it to the best match. For a 5xx, fix server capacity or the crashing script. For a robots.txt or 403 block, unblock Googlebot. For a redirect error, point the redirect at a real, final URL.
4. Return the right code. A live page must return 200, a moved page a 301, a genuinely gone page a clean 404 or 410. Overloaded servers should serve a temporary 503, not a 200.
5. Validate the fix. Click “Validate Fix” on the issue in the Pages report so Google re-crawls the affected URLs as a batch and clears them once they pass.

9. How Long Re-Crawling Takes After a Fix

Quick Answer: After you fix and validate, most crawl errors clear within a few days to two weeks. Unblocking robots.txt and DNS fixes resolve fastest; restored 404s take longest because Google re-crawls low-priority URLs less often.

Fixing the error is not instant recovery — Google has to come back, re-crawl, and confirm. Clicking “Validate Fix” speeds this up by queuing the URLs together. The typical timeline across client sites:

10. Where Crawl Errors Cluster, by Site Size

Quick Answer: The bigger the site, the higher its crawl-error rate. Small brochure sites rarely break 3% of crawled URLs; large e-commerce catalogues often run past 15% because filters, out-of-stock pages, and old URLs pile up faster than they get cleaned.

Crawl errors are partly a numbers game. A five-page site has little to go wrong; a 20,000-URL store generates dead pages constantly — discontinued products, expired filters, session URLs. This is where crawl budget matters: every request Google wastes on a broken URL is one it didn’t spend on a page you want ranked.

If your catalogue is large, pair this with a clean sitemap so Google spends its crawl on live pages — our guide on fixing sitemap errors covers that. Slow pages waste crawl budget too, which ties into passing Core Web Vitals.

11. Fix It Yourself, or Get an SEO Team?

Quick Answer: A handful of 404s on a small site is a DIY job — restore or redirect, then validate. Recurring server errors, DNS failures, or a large site with hundreds of errors usually needs an SEO team, because those trace to hosting and site architecture, not single pages.

Be honest about the scale. If your Pages report shows a dozen 404s from an old menu change, you can clear them yourself in an afternoon with redirects — the tools are free and the steps are the five above.

Get help when errors keep returning, when Crawl Stats shows a red host status, or when a big catalogue is bleeding crawl budget. Those point to deeper causes — under-powered hosting, a misconfigured CDN, a plugin blocking Googlebot, or a URL structure that spawns duplicates. That is what our SEO team does daily, and it often overlaps with cleaning up duplicate content.

12. Conclusion

Quick Answer: Crawl errors are fixable signals, not penalties. Triage site-level failures first, read the status code, fix at the source, and validate — and the pages come back.

Crawl errors look scary in red, but they are among the most solvable SEO problems you will meet. Each one names its cause and has a clean fix. Sort by type, clear the site-wide emergencies before the one-off pages, return the correct status code, and let Google re-crawl. Most sites clear the backlog inside two weeks with rankings intact.

13. Frequently Asked Questions

1. What are crawl errors in Google Search Console?

Crawl errors are URLs Googlebot tried to fetch but couldn’t read properly. They cover missing pages (404), server failures (5xx), soft 404s, broken redirects, blocked pages, and DNS problems. Each type has a different cause and fix, and if Google can’t crawl a page, it can’t index or rank it.

2. Where did the old Crawl Errors report go?

Google retired the standalone Crawl Errors report in 2019. Its data moved into the Pages (Index) report for per-URL issues, the Crawl Stats report under Settings for site-level host and response-code data, and the URL Inspection tool for checking a single live URL.

3. Do crawl errors hurt my Google rankings?

Indirectly, yes. A crawl error doesn’t trigger a penalty, but if Google can’t fetch a page, it eventually drops that page from the index — so you lose the ranking. Site-wide server or DNS errors are the most damaging because they can affect every page at once.

4. How do I fix a server error (5xx) in Search Console?

Check whether your hosting was overloaded or a script crashed when Google called. Fix the capacity or code issue, and make overloaded servers return a temporary 503 rather than a 200. Then run URL Inspection to confirm a 200 response and click Validate Fix so Google re-crawls.

5. How long does Google take to clear a crawl error after I fix it?

Usually a few days to two weeks. Unblocking robots.txt, restoring DNS, and fixing servers clear fastest — often under a week. Restored or redirected 404s take longest because Google re-crawls low-priority URLs less often. Clicking Validate Fix in the Pages report speeds the batch up.