Research Guides
Count Crawl Requests Without Calling Them Indexed Pages
A chart shows 10,000 crawl requests. That does not mean Google discovered 10,000 articles, indexed 10,000 pages or sent 10,000 visitors. Before using a crawling chart in a website report, write down what the counter actually counts.
This worksheet helps an authorized site owner turn a Crawl Stats observation into a bounded investigation. It does not describe a Colquhoun.shop integration or claim that increasing crawl volume is an SEO goal.
1. Put the counting unit beside the number
Google's Crawl Stats documentation says repeated requests for one URL are counted separately, as are server-side redirect-chain requests. The totals can include page resources and unsuccessful attempts. The example URLs are not a complete inventory, and the report may differ from server logs because some requests are not counted. Google: Crawl Stats report
Label a copied figure “crawl requests,” not “pages indexed.” Keep the selected property, dates and grouping with it. If a colleague cannot tell which view produced the number, the figure is not ready for a slide or client summary.
2. Choose a question the evidence can answer
Start with a specific operational concern: “Did the proportion of unsuccessful requests change during the maintenance window?” Avoid beginning with a verdict such as “Google dislikes our new articles.” The second statement assumes a cause before the investigation has established what changed.
Google describes this as an advanced report that smaller sites with fewer than a thousand pages generally should not need to use. Do not manufacture a crawl-budget project for a small site merely because a chart is available.
3. Separate request frequency from URL coverage
Consider an invented local request sample containing 100 records. One URL appears 40 times; 60 other URLs each appear once. The sample therefore contains 100 requests across 61 distinct requested URLs. Neither count establishes how many pages entered a search index.
If the frequently requested URL accounts for most of a change, investigate that concentration rather than rewriting all 61 pages. Record what the URL serves and why its behavior matters to readers. Do not assume frequency alone means the URL is important, defective or wasteful.
This arithmetic applies to the invented sample. Do not apply it to the report's example URL list as though that list were a complete export of every request.
4. Use a request-investigation card
- Question: the operational concern, without an assumed cause.
- Scope: property, host view, dates and selected breakdown.
- Observation: the displayed metric and its unit.
- Examples: relevant requested URLs, marked as examples.
- Corroboration: authorized logs or deployment records inspected separately.
- Limit: what the available evidence cannot establish.
- Next action: one owner and one testable follow-up.
When comparing a chart with logs, document their time boundaries and included systems before expecting agreement. Keep the original records unchanged. A reconciliation worksheet should explain differences, not silently edit one source until its total matches another.
5. Report the investigation without turning it into a ranking claim
A useful conclusion might be: “The selected view shows a request increase during the release window. We will inspect the affected URL examples and deployment record before attributing the change.” That is more actionable than declaring a crawl spike an SEO success.
Keep separate evidence for crawling behavior, indexing status and search traffic. If only the crawling view was reviewed, say so. The purpose of this worksheet is to make the next technical check clearer, not to create a new number for a marketing headline.