How to Run a Deep SEO and Answer Engine Optimization (AEO) Audit with Claude Code (2026 Guide)

A deep SEO and Answer Engine Optimization (AEO) audit with Claude Code works by feeding it your site's raw crawl export, XML sitemap, and Search Console data, then using it as an agent to cross-reference crawlability, structured data, and answer-first content structure — and to write the actual fixes itself, from missing JSON-LD to a robots.txt rule blocking an AI crawler you didn't mean to block. The difference from a manual audit isn't just speed: the same session that finds an issue can open the file and patch it, then re-check its own work.

SEO vs. AEO vs. GEO: What's Actually Different in 2026

Classic SEO optimizes for ranking in a list of blue links. AEO (Answer Engine Optimization) optimizes for being the thing an AI system selects, summarizes, or cites directly — in ChatGPT, Perplexity, Google AI Overviews, or Claude itself. You'll also see the term GEO (Generative Engine Optimization) used for the same idea. In practice they share almost the entire technical foundation: a page that's crawlable, fast, and marked up correctly for classic search is already most of the way to being retrievable by an AI system. AEO adds a second, content-shaped layer on top: can a machine lift a clean, self-contained answer out of your page without having to interpret buried context first.

What You Need Before You Start

The Claude Code CLI, run from your site's repo or a checkout of the rendered HTML.
A crawl export — Screaming Frog's "Internal All" report (source_url, destination_url, status_code, link_count) is the de facto standard for this.
Google Search Console exports: the Coverage report (all tabs, as CSV) and a 90-day Performance report (all queries/pages, as CSV).
Your XML sitemap index and its child sitemaps.
Optional but useful: a Puppeteer MCP server wired into Claude Code so it can actually load pages in a browser rather than just reading static HTML, and the open-source claude-seo skill, which packages a lot of the checks below into a single /seo audit command.

Step 1 — Check Whether AI Crawlers Can Even Reach Your Content

Before anything else, have Claude Code read your live robots.txt and classify every user-agent rule against the two categories AI crawlers actually fall into:

Category	Crawlers	Purpose
Training crawlers	GPTBot, Google-Extended, ClaudeBot, CCBot	Scrape content into model training data
Retrieval / search bots	OAI-SearchBot, PerplexityBot, Claude-SearchBot, Claude-User, ChatGPT-User	Fetch pages live to answer a specific user query — blocking these directly removes you from that answer

A common and defensible 2026 configuration blocks the training-only crawlers while explicitly allowing the retrieval bots, since blocking a retrieval bot has an immediate, visible cost (you stop being citable) while blocking a training-only bot mostly doesn't. The trap Claude Code is good at catching: a blanket Disallow: / rule, or a wildcard rule written to catch one bot, that accidentally nets a retrieval bot too. Cross-check robots.txt intent against real server logs — the major commercial AI bots publicly commit to honoring robots.txt and have been observed doing so, so if your rule says allow but your logs show zero hits, the problem is upstream (DNS, WAF, or a CDN-level block) rather than the file itself.

Step 2 — Feed Claude Code Your Technical Exports and Let It Cross-Reference Them

This is the part that's tedious by hand and fast as an agentic loop. A workable six-stage pipeline: parse the raw CSV/JSON/XML → validate status codes, canonicals, and schema → analyze for 4xx clusters, orphaned pages, and schema errors → prioritize by crawl-budget and citation impact → translate the technical finding into plain language → report. Concretely, ask Claude Code to:

Parse the GSC Coverage CSV, pull every URL flagged "Submitted URL not found" (404) or a server error, group them by directory path, and return an error_summary with totals by status code plus a top_affected_directories list.
Extract every <loc> from the sitemap, make a HEAD request against each, and return a status_breakdown plus a list of non-2xx URLs.
Cross-reference the sitemap against the crawl export to flag orphaned pages (present in the sitemap, linked from nowhere internally), mismatched canonicals (sitemap <loc> disagrees with the page's own canonical tag), and stale lastmod timestamps that don't match actual content changes.

Tell it explicitly to return JSON with no markdown wrapper for anything that feeds a script, and a markdown report for anything a human will read — mixing the two formats is the most common way this kind of output becomes unusable downstream.

Step 3 — Extract and Validate Every Schema Block on the Site

Have Claude Code walk the crawl export and pull every <script type="application/ld+json"> block it finds, then validate each against the required fields for its declared @type. The schema types that matter most for AEO specifically, in rough priority order: Organization (establishes the brand entity), Article (for blog/news content), FAQPage and HowTo (the two types most directly tied to being lifted into an AI-generated answer), and Person/author schema linking content to a named, credentialed writer. When you're having Claude Code generate new schema rather than just check it, use a builder-validator pattern: one pass generates the JSON-LD from your page's actual entity data, a second pass validates that output against the JSON Schema for that type before it ships, catching missing required fields or wrong types automatically instead of trusting a single hand-reviewed pass.

Step 4 — Audit for "Answer-First" Extractability

This is the pass that's specific to AEO rather than shared with classic SEO. Point Claude Code at your page templates and rendered content and have it flag, per page:

Whether the direct answer to the page's core question appears in the first paragraph, or is buried after several paragraphs of throat-clearing.
Whether headings (<h2>/<h3>) are phrased as the actual question a reader or an AI system would ask, with the content directly beneath answering that exact heading.
Paragraph length — two to four sentences is the target; anything longer is harder for a model to cleanly extract as a standalone answer.
Whether the page defines its core term explicitly ("X is …") rather than assuming the reader already knows it.

Because Claude Code can grep across the whole templates directory or a full HTML export at once, this check scales to hundreds of pages in the time it'd take to manually review a handful.

Step 5 — Check Authority (E-E-A-T) and Freshness Signals

AI systems weigh trust signals alongside extractability. Have the audit check for: a named author (not "Admin" or no byline) with a bio and a link to a credential source like a LinkedIn profile, consistent name/organization data across the site's About page and its schema, a visible publication date and a genuine "last updated" timestamp (not one that silently bumps on every deploy), and broken outbound citations to sources the content relies on for credibility.

Reference: Categories an Automated Audit Can Cover

The open-source claude-seo Claude Code skill is a useful reference point for scope — it runs a technical pass across nine categories (crawlability, indexability, security, URL structure, mobile optimization, Core Web Vitals/INP, structured data, JavaScript rendering, and IndexNow implementation) and rolls everything into a 0–100 health score weighted roughly: content quality 23%, technical SEO 22%, on-page optimization 20%, schema 10%, performance 10%, AI readiness 10%, images 5%. Treat a score like that as a triage tool for where to look next, not as a ranking prediction.

Implementing Fixes, Not Just Finding Them

The real advantage of running this through an agentic coding tool rather than a SaaS audit dashboard is that finding and fixing happen in the same loop. Once Claude Code has flagged a missing FAQPage schema block, an accidentally-blocked retrieval bot, or a buried answer paragraph, it can open the actual template or content file and make the edit directly, then re-run the same extraction check against the changed file to confirm the fix landed before you consider the item closed. That closing loop — find, fix, re-verify, in the same session — is the part a static checklist or a one-off report can't do for you.

Measuring Whether Any of This Worked

Technical correctness isn't the end goal — being cited is. Track it with: manual or tool-assisted prompts against ChatGPT, Perplexity, and Google AI Overviews to see if and how your brand gets described and cited; AI-referral traffic segmented out in GA4; and, if you're competing on a specific topic, a rough share-of-voice comparison against the competitors who do get cited. None of this is a one-time project — re-run the same Claude Code audit on a schedule (quarterly is a reasonable default for high-value pages) rather than treating the first pass as done.

FAQ

What's the actual difference between an SEO audit and an AEO audit?

An SEO audit checks whether a page can be crawled, indexed, and ranked by traditional search engines. An AEO audit checks all of that plus whether a specific answer can be cleanly extracted from the page and cited by an AI system — which adds checks like answer-first paragraph placement, FAQ/HowTo schema, and named-author trust signals on top of the standard technical checklist.

Can Claude Code actually crawl a live website by itself?

Not natively out of the box, but wiring in a Puppeteer MCP server gives it real browser control to navigate pages, take screenshots, and inspect rendered HTML. Most deep audits, though, work better by feeding it existing crawl data (a Screaming Frog export) and Search Console CSVs rather than having it crawl blind.

Will blocking GPTBot hurt my search rankings?

No — GPTBot is a training crawler used to gather data for future model training, not a retrieval bot that determines whether you show up in a live AI answer. Blocking it doesn't affect classic search rankings or your citability in ChatGPT's live search results, which is handled by the separate OAI-SearchBot user-agent.

What's the single fastest fix to make for AEO?

Rewriting your intro paragraphs to state the direct answer in the first two to three sentences, before any throat-clearing context. It's a content edit, not a technical change, so it needs no deploy pipeline, and it's the factor most directly tied to whether an AI system can lift a clean answer from the page at all.

Do I need Screaming Frog and GSC exports, or can Claude Code do this from scratch?

You'll get a far more accurate audit feeding it real crawl and Search Console data than asking it to infer site structure from a handful of fetched pages. The exports give it ground truth on status codes, canonicals, and actual search performance that it can't reliably reconstruct on its own.

What is llms.txt and do I need one?

It's a proposed plain-text file at your site root that gives AI crawlers a high-level, machine-readable map of your most important content and definitions, similar in spirit to a sitemap but written for language models rather than search indexers. If you've already invested seriously in schema markup, adding one is a small additional step with outsized upside; it's not a replacement for the structured-data and crawlability work above.

How often should I re-run the audit?

Quarterly for high-value pages is a reasonable default, with an ad hoc re-run any time you change templates, migrate CMS, or notice a drop in AI-referral traffic in GA4. AEO in particular isn't a one-off project — the answer engines' crawling and citation behavior keeps shifting.

Can Claude Code write the schema markup fixes itself, or just flag what's missing?

It can generate the actual JSON-LD from your page's entity data and insert it into the template. The safer pattern is to have it generate the markup in one pass and validate that output against the schema's required fields in a second pass, so a malformed or incomplete block doesn't ship silently.

Further reading: