Crawl and diagnose / 12 picks

Best technical SEO tools
for site audits.

Compare curated technical SEO crawlers and audit tools for finding indexability, metadata, performance and internal-link issues.

Compare products ↓
Editor’s note

A focused shortlist,
not a pay-to-rank table.

This collection centers on crawling and diagnostic workflows. It includes desktop, cloud and self-hosted options so teams can choose based on site size, deployment preference and reporting needs.

How to read this guideProducts are selected from the reviewed directory using the criteria below. Placement is editorial and does not claim hands-on testing of every product.
The directory view

Explore every pick.

Open a listing for features, compatibility, screenshots and direct links.

NewSkill

Toprank

★ 3.2k

SEO and paid acquisition skills for Claude Code

Open-source agent workflows connected to Search Console, Google Ads and Meta Ads with actionable technical SEO recommendations. Its documented capabilities include GSC analysis, Technical audits and Google Ads insights. It is most relevant to workflows involving Agent Skills and Search Console.

Active repoUpdated today
Agent SkillsSearch ConsoleTechnical SEO
Skill

SEO Skill

One skill and 50+ audit tools for AI agents

A developer-focused SEO audit toolkit that helps coding agents find, explain and fix organic and AI search visibility issues. Its documented capabilities include 50+ audit utilities, Typed crawler and Actionable reports. It is most relevant to workflows involving Agent Skills and SEO Audit.

VerifiedChecked today
Agent SkillsSEO AuditDeveloper Tools
Open Source

SEOnaut

★ 738

Simple, self-hosted technical SEO auditing

A mature open-source crawler that discovers broken links, redirect problems, metadata issues and structural SEO errors. Its documented capabilities include Full-site crawler, Issue severity scoring and Redirect analysis. It is most relevant to workflows involving Technical SEO and Crawler.

Active repoUpdated 1mo ago
Technical SEOCrawlerSelf-hosted
NewTool

Scrunch

Monitor and improve how AI platforms understand your brand

Scrunch tracks prompt performance, citations, competitors and AI crawler activity, then pairs those insights with an agent experience layer for serving structured content to AI systems. Its documented capabilities include Prompt-level monitoring, Citation analysis and AI bot observability.

VerifiedChecked today
AI VisibilityGEOTechnical SEO
NewTool

Sitebulb

Visual technical SEO audits on desktop and cloud

Sitebulb crawls websites and turns technical findings into prioritized hints, visual reports and audit explanations for consultants, agencies and in-house SEO teams. Its documented capabilities include Technical site crawling, Prioritized audit hints and Data visualization.

VerifiedChecked today
Technical SEOCrawlerSEO Audit
NewTool

Screaming Frog SEO Spider

The industry-standard desktop crawler for technical SEO

SEO Spider crawls websites to audit links, redirects, metadata, canonicals, structured data, rendering and many other technical signals with flexible exports and integrations. Its documented capabilities include Website crawling, JavaScript rendering and Structured data validation.

VerifiedChecked today
Technical SEOCrawlerSEO Audit
NewOpen Source

Unlighthouse

★ 4.7k

Run Lighthouse performance scans across an entire site

Unlighthouse combines site crawling with Lighthouse to scan many pages, visualize performance and accessibility results, and run locally or in continuous integration. Its documented capabilities include Site-wide Lighthouse scans, Performance dashboards and Route discovery.

Active repoRelease v0.18.0
Technical SEOPerformanceOpen Source
NewOpen Source

SiteOne Crawler

★ 805

Cross-platform audits for SEO, security and accessibility

SiteOne Crawler analyzes websites for technical SEO, security, accessibility and performance issues through a cross-platform CLI and desktop-friendly reporting workflow. Its documented capabilities include SEO and link audits, Security checks and Accessibility analysis.

Active repoRelease v2.5.1
Technical SEOCrawlerOpen Source
NewTool

Ahrefs Webmaster Tools

Free site audits and organic search insights for verified sites

Ahrefs Webmaster Tools gives site owners free access to technical auditing and search-performance data for verified properties, including health issues, backlinks and organic keywords. Its documented capabilities include Site Audit, Backlink reports and Organic keyword data.

VerifiedChecked today
Technical SEOSEO AuditFree Tools
NewSkill

Agentic SEO Skill

★ 783

LLM-first SEO analysis with specialist sub-skills and scripts

An open-source SEO capability pack for coding agents with specialized workflows, evidence-collection scripts and focused agents covering technical, content and AI-search audits. Its documented capabilities include Specialized SEO sub-skills, Evidence collection scripts and Technical audits.

Active repoRelease v3.0.1
Agent SkillsSEO AuditOpen Source
NewSkill

Codex SEO

★ 459

A Codex-first suite of repeatable SEO workflows

Codex SEO packages technical audits, GEO and AEO, schema, backlinks, local search and reporting into a Codex-native orchestrator with specialist workflows and integrations. Its documented capabilities include Codex-native orchestrator, Technical and content SEO and GEO and AEO workflows.

Active repoRelease v1.9.6-codex.5
Agent SkillsTechnical SEOGEO
NewSkill

Claude SEO

★ 12k

Universal SEO workflows and specialist agents for Claude Code

Claude SEO combines sub-skills and specialist agents for technical SEO, E-E-A-T, schema, GEO, local search, e-commerce and reporting with optional data integrations. Its documented capabilities include Specialist sub-agents, Technical SEO and Schema and E-E-A-T.

Active repoRelease v2.2.4
Agent SkillsTechnical SEOGEO
Selection criteria

What makes the cut.

  1. 01Site crawling or technical diagnostic capability
  2. 02Actionable issue reporting
  3. 03A maintained product or inspectable open-source project
FAQ

Before you choose.

Short answers to the questions that matter most for this workflow.

What should a technical SEO audit tool check?+

Common checks include crawlability, indexability, status codes, canonicals, metadata, internal links, structured data and performance signals.

Do I need both a crawler and Search Console?+

They answer different questions. A crawler inspects the site you expose to bots, while Search Console reports how Google discovered, indexed and served it.

When is a self-hosted crawler useful?+

Self-hosting can help when you need data control, repeatable automation or custom integrations, provided your team can maintain the infrastructure.