How-to steps · Chapter 6, S for Structure
See how often Googlebot visits your pages
You are here for this line in chapter 6 of the book:
☐ Compare Search Console Crawl stats example URLs fetched over 90 days with target pages; list potential crawl gaps.
Chapter 6, S for Structure
- Go to Settings (bottom of the left menu), then Crawl stats, then Open report. It is available on top-level properties.
- Read total crawl requests, download size and response time, then the breakdowns by response, file type, purpose (discovery or refresh) and Googlebot type.
- Click into the breakdowns for example URLs. Compare them with your ten priority pages: a priority page Googlebot rarely requests is a crawl gap worth an internal link or a sitemap check.
Larger sites with server access can go further with a log file analyser, which records every bot visit rather than a sample.
To find pages that few or no other pages link to, Hawk Academy's free Internal Link Checker scans your sitemap and up to 100 pages and lists orphans and weakly linked pages.
You're done when you have a short list of priority pages Googlebot rarely requests, each with its fix: an internal link, a sitemap entry or a speed check.