Skip to content
Methodology

Soft-404 hygiene

ia.soft-404 · 1.13

Read MUST Recommendation AR-READ-02

Why it matters

Agents and crawlers rely on a real 404 or 410 status to know a resource is missing. A 200 with the app shell (soft 404) makes them believe every probed path exists, pollutes indexes, and hides broken links from monitoring.

How to fix it

Answer unknown paths with HTTP 404 or 410, not with HTTP 200 or a redirect.

  1. Request a random path that does not exist and record the status.

  2. Configure the server or the single-page app router to return 404 for unknown routes.

    location / {
      try_files $uri $uri/ =404;
    }
  3. Do not redirect unknown paths to the home page.

Verify

curl -s -o /dev/null -w '%{http_code}\n' https://{domain}/isagentready-missing-check

Expected: 404 or 410

Run a new scan for this checkpoint:

isagentready results {domain} --checkpoint ia.soft-404
Details Mapping, lifecycle and references
AR mapping
Full match
Score role
Core
Introduced
2026-08-25
Methodology version
2026-10-22

Checkpoint data (JSON)