Colquhoun.AI
← All guides

Research Guides

A Page Can Load and Still Be Missing: A Soft-404 Audit Worksheet

· Colquhoun.shop editorial

A successful HTTP response answers a narrow question: did the server return a response labeled successful? It does not prove that the visitor received the page they requested.

An article address can display a polished navigation bar, a footer and a “nothing here” message while returning 200 OK. A product URL can load its usual template but omit every product detail after an API failure. Both deserve investigation even if an uptime monitor stays green.

This worksheet helps separate missing content, temporary failures and genuinely useful pages. It is a manual website-review method, not a claim that Colquhoun AI currently performs these checks automatically.

The distinction worth recording

A conventional 404 Not Found response tells the client that the requested resource cannot be found. MDN notes that the status does not say whether the absence is temporary or permanent.[1]

A soft 404 is different: the response suggests success, but the content appears to be an error or empty page. Google documents that a successful response can still produce a soft-404 classification when the content suggests an error, an empty page or an error message.[2] That classification is not a status code you can read from the server.

Do not label every short page a soft 404. A concise contact page can completely satisfy its purpose. Nor should you treat your own visual suspicion as proof of Google's classification. Record those as separate observations.

Start with a small, deliberate sample

Choose a few URLs from each important page type: articles, product details, categories and public account-help pages. Include a known working URL, a known removed URL and a URL reported by a visitor or Search Console.

If you test a deliberately nonexistent address, use a harmless unique path on a site you own. Do not generate thousands of random URLs, submit forms or probe private resources. A single control is often enough to reveal a catch-all route that serves the same empty template for everything.

Keep the test conditions visible: date, device, signed-in state and whether JavaScript ran. A restricted page viewed without authentication should not be confused with a missing public article.

Use a two-layer worksheet

Field What to capture
Requested URL The original address, not just the final destination
Expected purpose What a visitor reasonably expects here
Response chain Initial status, redirects and final status
Visible result Main heading, useful content and any error message
Rendered behavior Whether content arrives after scripts run or remains absent
Search evidence Exact Search Console classification, if available
Likely cause Deleted resource, routing problem, dependency failure or unknown
Proposed action Restore content, relevant redirect, correct error status or investigate
Retest evidence Status and visible result after the repair

The response chain and the page's usefulness belong in different columns. Combining them into one “passed” checkbox is how this problem gets overlooked.

Use browser developer tools to inspect the document request. A separate header request can be helpful, but do not assume a server handles HEAD identically to the GET used to load the page. Inspect the actual document response when results conflict.

Choose the repair from the intended outcome

The page should still exist. Restore the missing content or repair its loading path. Check both the initial response and the rendered result. Returning 200 is appropriate only when the successful representation is actually being served; changing the label alone does not restore an article.

The page moved to a real replacement. Redirect to that replacement, then verify that the destination serves the promised content. Google's documentation treats permanent redirects as strong signals for processing the target URL.[2] Relevance still requires human judgment. Sending every deleted article to the homepage is not a useful substitute for identifying a matching replacement.

The resource is gone and has no replacement. Return an appropriate missing-resource response while offering helpful navigation. A readable error page and a correct 404 can coexist. MDN's example uses a custom error page without converting the response into a success.[1]

The failure is temporary. Investigate the unavailable dependency or service rather than declaring the content permanently gone. Google treats server errors differently from missing pages and can reduce crawling during server failures; persistent server errors can eventually lead to removal from the index.[2] Choose status handling with the application's actual failure mode in mind, not solely to keep an SEO report green.

Fictional example: the empty guide template

Suppose /guides/window-insulation/ returns 200, displays the site header and says “Guide unavailable.” The editorial team confirms the guide was accidentally omitted during a content migration.

The initial audit record should say: “Missing expected guide content; suspected soft-404 pattern.” It should not say “Google has deindexed this URL” without search evidence.

The repair is to restore the guide, not redirect visitors to an unrelated buying page. Retesting checks that the original URL returns the guide, images and internal links work, and the error message is absent. A separate follow-up checks Search Console after Google has had an opportunity to revisit the page. A successful local retest does not guarantee an immediate search-report update.

Close the issue with evidence, not a green status alone

A useful completion note contains the original symptom, the intended behavior, the repair and a reproducible retest. Keep screenshots or short content excerpts alongside the status record so another reviewer can see what changed.

The practical rule is simple: verify the requested content as well as the response. Availability monitoring tells you whether a server answers. A content-aware review tells you whether that answer helps the visitor.

Sources

  1. MDN, 404 Not Found. Checked November 12, 2026.
  2. Google, How HTTP status codes affect Google's crawlers. Checked November 12, 2026. Covers successful responses, soft-404 outcomes, redirects and server errors.
This guide describes the built-in Web Reader and Keyword Density behavior verified for this publication. Available data depends on the page response; these tools do not measure visits or search rankings.