How to Fix Broken Pages (4xx & 5xx Errors)
Your audit found pages returning 4xx status codes (client errors like 404 Not Found or 403 Forbidden) or 5xx status codes (server errors like 500 or 503) instead of a healthy 200 OK. These are dead ends for search engines and AI crawlers: the content is unreachable, ranking signals are lost, and users hit error screens. This guide explains each error class, why broken pages hurt both classic search and AI answer engines, and the exact fix per platform.
What this means
A broken page is any URL that returns an HTTP status in the 4xx or 5xx range instead of 200 OK. The two ranges mean very different things.
4xx codes are client errors: the request itself was rejected. The most common is 404 Not Found (the URL doesn't exist), followed by 410 Gone (deliberately removed), 403 Forbidden (access denied, often a permissions or firewall issue), and 401 Unauthorized (authentication required). A 404 is normal for a genuinely deleted page. It becomes a problem when a URL that should work returns one, or when internal links and sitemaps still point to it.
5xx codes are server errors: your server received the request but failed to fulfill it. Common ones are 500 Internal Server Error (an unhandled crash, PHP fatal, or misconfiguration), 502 Bad Gateway and 504 Gateway Timeout (an upstream or origin server didn't respond in time, common behind a proxy or CDN), and 503 Service Unavailable (the server is overloaded or in maintenance mode). 5xx errors are more urgent than 404s because they signal your site is unstable, not just that one page is missing.
Your audit flagged this because the crawler either requested a URL directly or followed a link or sitemap entry, and got a non-200 response where a live page was expected.
Why it matters
Broken pages break the chain that search and AI systems depend on. When Googlebot follows a link to a page that returns 404 or 500, the destination gets no ranking value, and any link equity pointing at it is lost instead of flowing through your site. Persistent 4xx and 5xx responses also waste crawl budget: search engines spend requests hitting dead URLs instead of discovering and refreshing your real content, which slows indexing of the pages you care about.
5xx errors are especially damaging. A page that returns 500 or 503 during a crawl can eventually be dropped from the index if the error persists, because Google treats a server error as a signal the content is temporarily or permanently unavailable. Frequent 5xx responses across many URLs tell Google your site is unreliable, and it will crawl you more cautiously and less often.
For AI answer engines the effect is sharper. Crawlers like GPTBot, ClaudeBot, PerplexityBot, and Google's crawlers can only cite content they can actually fetch. A 4xx or 5xx page is invisible to them, so it can never appear as a source in ChatGPT, Perplexity, or a Google AI Overview. If a URL that was previously cited starts returning an error, you lose that citation and any referral traffic with it.
There's a user-trust cost too. A 404 or a raw 500 error page is a dead end that pushes visitors back to search results, and those bounces compound the ranking damage.
How to fix it
- 1
Identify every broken URL and its exact status code
You can't fix what you can't see. Pull the full list of failing URLs from your audit, then cross-check in Google Search Console under Indexing > Pages (look for 'Not found (404)', 'Server error (5xx)', and 'Soft 404'). Group them by status code, because the fix for a 404 (missing page) is nothing like the fix for a 500 (server crash). For each URL, note where it is linked from, since an internal link pointing at a dead page is a separate problem you fix in step four.
- 2
Fix 5xx server errors first
5xx errors are the most urgent because they can get healthy pages dropped from the index. Check your server error logs (in cPanel, open the error log; on a stack you control, check the web server and application logs) to find the actual cause: a PHP fatal error, a database connection failure, an exhausted memory limit, or a plugin conflict. A brief 503 during a real maintenance window is fine if it returns the correct header, but a persistent 500, 502, 503, or 504 needs a code or infrastructure fix. Behind Cloudflare or another CDN, a 502 or 504 usually means your origin server is timing out or down, not the CDN.
- 3
Redirect or restore removed pages instead of leaving 404s
For a page that moved or was replaced, add a 301 permanent redirect to the closest equivalent live URL. This passes ranking signals to the new page and keeps users out of a dead end. Do not blanket-redirect every 404 to your homepage; Google treats an irrelevant redirect as a soft 404 and ignores it. For content that is genuinely gone with no replacement, either restore it if it still has value, or serve a clean 404 (or 410 Gone if the removal is permanent) so search engines drop it cleanly. A custom 404 page with search and navigation links helps users recover.
- 4
Remove or update internal links and sitemap entries pointing at dead URLs
A 404 that nothing links to is low priority. A 404 that your navigation, footer, or sitemap points at is actively harmful, because you are telling crawlers to keep visiting it. Update every internal link to point at the working URL, or remove it if there is no replacement. Regenerate your XML sitemap so it contains only 200-status, indexable URLs, then resubmit it in Search Console. A sitemap should never list URLs that redirect or error.
- 5
Resolve 403 and 401 errors from permissions or firewall rules
A 403 Forbidden on a page that should be public is usually a file-permission problem, a misconfigured rule in .htaccess or Nginx, or an over-aggressive WAF or bot-blocking rule rejecting legitimate crawlers. Confirm the page is meant to be public, check file and folder permissions, and review any security-plugin or firewall rules that block by user agent or request rate. If your CDN or firewall is challenging or blocking crawlers like Googlebot or GPTBot, allowlist the verified bots you want to index and cite your content.
- 6
Recrawl, validate, and monitor for regressions
After fixing each group, use Search Console's URL Inspection tool to request indexing on key pages, and click 'Validate Fix' on the Pages report so Google recrawls the affected URLs. Re-run your site audit to confirm the 4xx/5xx count has dropped to zero, or that only intentional 404/410s remain. Broken pages tend to recur when content is deleted or restructured, so schedule a recurring crawl or watch the Search Console Pages report to catch new errors early.
Example
# Permanent redirect: old URL -> relevant live replacement
Redirect 301 /old-product-page /products/new-product
# Content permanently removed, no replacement: tell crawlers it's Gone
Redirect 410 /discontinued-item
# Point your custom, useful 404 page at real errors
ErrorDocument 404 /404.html
# --- Nginx equivalent ---
# location = /old-product-page { return 301 /products/new-product; }
# location = /discontinued-item { return 410; }
# error_page 404 /404.html;Redirect a removed page to its replacement with a 301, and serve a 410 for content that's permanently gone (Apache .htaccess).
Platform-specific steps
Both Rank Math (free) and Yoast Premium include a Redirections manager that logs 404s and lets you add 301s without editing .htaccess. Open the redirect module, review the 404 log, and map each dead URL to a live one. For 5xx errors, look for a plugin or theme conflict and raise the PHP memory limit. Both plugins regenerate the XML sitemap automatically, so it excludes errored URLs once they are fixed.
Shopify auto-creates a 301 redirect when you change a product or page handle, but deleting a product leaves a 404. Go to Online Store > Navigation > URL Redirects to add redirects from old paths to relevant live products or collections. You can't edit server config, so 5xx errors are rare and handled by Shopify; if you see one, it's usually an app or theme-code issue to investigate in the theme editor.
Use the URL Redirect Manager under Marketing & SEO to add 301 redirects from old URLs to live pages. Wix serves a default 404 that you can customize. Because Wix manages the server, 5xx errors are uncommon; if pages error, check for a publishing issue or contact Wix support rather than looking for a server misconfiguration.
Add 301 redirects under Settings > Advanced > URL Mappings using the format /old-url -> /new-url 301. Squarespace hosts and manages the server, so focus on redirecting removed pages and fixing internal links; you won't edit server config for 5xx issues, which are handled on Squarespace's side.
If you're behind Cloudflare and seeing 502 or 504, the problem is almost always your origin server timing out or being unreachable, not Cloudflare. Check origin health and the error analytics. Use Bulk Redirects or a Redirect Rule to serve 301s at the edge. Make sure your WAF or bot-management rules aren't returning 403 to legitimate crawlers like Googlebot or GPTBot.
For static or custom sites, handle redirects in your server config (Apache .htaccess or Nginx). In Next.js, use the redirects() function in next.config.js, which emits proper 301 or 308 responses. Make sure deleted routes return a real 404 (Next.js does this by default) rather than a 200 page with 'not found' text, which becomes a soft 404. Fix 5xx errors by checking your Node or server logs for unhandled exceptions.
Frequently asked
A 4xx is a client error: the request was rejected, usually because the URL doesn't exist (404), access is denied (403), or the resource was removed (410). A 5xx is a server error: the request was valid but your server failed to respond correctly, from a crash (500), an unresponsive upstream (502/504), or overload (503). 5xx errors are more urgent because they indicate your site is unstable and can cause healthy pages to be dropped from the index.
A 404 on a page that was genuinely deleted and isn't linked anywhere is normal and won't hurt you. The problem is 404s on URLs that should work, or 404s that internal links, external backlinks, or your sitemap still point to, because those waste crawl budget and lose ranking signals. Fix the ones that matter by restoring the page or 301-redirecting it to a relevant live URL, and leave harmless 404s alone.
No. Redirecting a missing page to your homepage is treated by Google as a soft 404, so the redirect is effectively ignored and you gain nothing, while confusing users. Redirect a removed page only to a genuinely relevant equivalent, such as a replacement article or the parent category. If there's no relevant destination, serve a proper 404 or 410 and let search engines drop the URL cleanly.
Yes. AI crawlers such as GPTBot, ClaudeBot, and PerplexityBot can only use content they successfully fetch. A page returning a 4xx or 5xx status is unreachable, so it can never appear as a source in an AI answer, and if a previously cited URL starts erroring you lose that citation. Keeping pages on a 200 OK response is a prerequisite for appearing in AI Overviews and chatbot answers.
A 500 error on WordPress is most often a plugin conflict, a theme error, a corrupted .htaccess file, or an exhausted PHP memory limit. Enable WP_DEBUG or check your server error log to find the failing file, then deactivate plugins one at a time (or switch to a default theme) to isolate the cause. Re-saving your permalinks regenerates .htaccess, and raising the PHP memory limit resolves many of the remaining cases.
Does your site have this issue?
Run a free, AI-powered audit and we’ll flag this and 150+ other checks in about a minute. No signup.