XML sitemap audit: what to check before and after a release

A valid sitemap is only the first check

An XML sitemap audit should tell you whether the file is usable, whether it lists the pages you intend to publish, and whether those pages behave as expected. Keep those questions separate from whether a search engine has actually indexed them.

A sitemap is a discovery hint. Google does not guarantee that submitting one leads to crawling or indexing. Use it to describe your intended public URL set, then compare that set with page responses and Search Console evidence. Google’s sitemap overview ↗

A clean result at one level does not establish a clean result at the next. For example, a well-formed sitemap can list a URL that redirects to a missing page.

1. Fetch and validate the file

Start with the sitemap URL declared by your site, rather than assuming that every site uses /sitemap.xml. Save a copy with the audit date, response status and final URL. Check that the response contains the intended XML rather than an HTML error page.

• Identify the root: a URL sitemap lists page locations; a sitemap index lists child sitemap locations. Counting index entries does not give you the page count.
• Follow the index: fetch the child files and record failures. A healthy top-level response can hide a broken child sitemap.
• Validate syntax: use an XML parser or validator, including entity escaping. A file extension or a string match for a root tag is insufficient.
• Check limits and URLs: Google documents a maximum of 50,000 URLs and 50 MB uncompressed per sitemap. Use UTF-8 and fully qualified absolute URLs. See Google’s sitemap requirements.

The free sitemap checker can provide a quick first look at a supported file: response, detected root type, entry count, duplicates and a sample. It does not recursively audit all child files, validate the full XML schema or fetch every listed page. Use the result to choose the next check.

Record both raw entries and unique URLs. If the raw count is 1,020 and the unique count is 1,000, you have 20 repeated entries beyond the first occurrences. That does not necessarily mean 20 distinct URLs are duplicated: a single URL could account for several repeats.

2. Inspect what the listed pages return

Build a working table with the declared URL, final status, redirect target, canonical and indexing directives. For a small site, inspect the whole intended set. For a large site, begin with representative templates and all suspicious changes, clearly recording what remains unchecked.

Google recommends listing the canonical URLs you want shown in search. A sitemap declaration is one signal; it does not force Google’s canonical selection. Sitemap URL guidance ↗

If you find a page blocked in robots.txt and marked noindex, do not assume both instructions reinforce each other. Google must retrieve the page to read its noindex directive. Determine the intended outcome and review the relevant access rule. Google’s noindex documentation ↗

Use a crawler from the technical SEO audit shortlist for the page checks, or follow the open-source audit workflow to turn discrepancies into a fix queue.

3. Make lastmod reflect a real update

Check a sample of lastmod dates against actual page changes. Google says it uses this value when it is consistently accurate and tied to a significant update, such as changes to main content, structured data or links. Google’s lastmod guidance ↗

A practical audit compares three pages: an unchanged older page, a recently edited page and a newly published page. Then fetch the sitemap twice without changing content. If every timestamp advances with each fetch, investigate whether your generator is using the request time instead of a content update time.

Save the generator’s source of truth in the audit record. A content record’s update timestamp may be appropriate if it changes for meaningful edits. A global deployment timestamp needs closer review when most pages did not change. Do not invent a recent date simply to make a page appear fresh.

4. Compare URL sets before and after a release

Counts are a useful alarm, but they can conceal replacements. Compare exact URL sets and review additions and removals separately. Preserve the original files so another person can reproduce the result.

Classify removed URLs as intentional retirements, planned URL moves or unexpected losses. For a planned move, verify the old URL’s response, the destination and updated internal links. For an unexpected removal, check whether the page still exists but fell out of the sitemap query.

Sitemap Cohort Auditor offers a documented comparison workflow. After obtaining the project and meeting its Node.js requirements, its CLI can compare saved sitemap files and produce JSON:

node ./bin/sitemap-cohort-auditor.mjs ./after/sitemap.xml --compare ./before/sitemap.xml --json

The project documents local and remote inputs, sitemap indexes and URL-set comparisons. This command is an example from its documented interface, not a claim that we benchmarked the tool. It does not replace fetching pages, checking their canonicals or checking indexing. Project usage and limitations ↗

For release checks, write the expected change before deployment: “30 new guides, zero removals” is testable. An arbitrary rule such as “count must never fall” will flag legitimate retirements and can miss equal-count replacements.

Your sitemap audit checklist

1. Record the sitemap URL, date, final response and saved file.
2. Identify index versus URL sitemap; inspect child files where present.
3. Validate XML, encoding, limits and absolute URL formatting.
4. Count raw entries, unique URLs and duplicates separately.
5. Check listed pages against intended statuses, canonicals and directives.
6. Investigate important pages missing from the sitemap or link crawl.
7. Compare lastmod values with significant content updates.
8. Review added and removed URLs against the release plan.
9. Assign fixes, rerun affected checks and inspect Search Console processing separately.

Finish with an evidence-based conclusion: which files were inspected, how many pages were checked, what changed and which questions remain open. “Sitemap passes” without that scope is too broad to be useful.

Common questions

Does a sitemap need every URL on my site?

Use the intended canonical pages you want discovered for search. Account routes, internal search combinations and duplicate URL variants need a deliberate policy rather than automatic inclusion.

Why did Search Console accept the sitemap but not index every page?

Processing a sitemap and selecting individual pages for indexing are different steps. Inspect representative affected URLs and their reported reasons; a successful file submission does not settle page-level questions.

Can the free checker verify an entire sitemap index?

It reports on the fetched file. If that file is an index, its entries are child sitemaps, not a complete inventory of their pages. Inspect the children with a workflow that explicitly follows them.

Read next