Run a crawlability audit before chasing AI visibility

Crawlability is often discussed as though a page is either visible or invisible. Ecommerce sites fail in more complicated ways.

A URL can return successfully but block crawling. It can be crawled but render without product facts. It can render correctly but point its canonical elsewhere. It can be indexable but receive no internal links. Each failure interrupts a different part of discovery.

Before buying an AI-visibility project, operators should test the technical path on the pages that matter to customers and revenue.

Choose representative page types

Do not begin with a site-wide warning count.

Select samples from hero products, long-tail products, categories, editorial guides, FAQs, policies, variants, pagination, filtered pages, discontinued products, and out-of-stock states. Include mobile behavior and more than one locale if the site serves them.

Add URLs with known revenue, backlinks, feed traffic, or support importance. This sample shows whether a template-level issue can damage many pages and helps the team rank impact.

Test access and response behavior

Confirm the final status code, redirect chain, robots access, authentication state, and response stability. Check whether important scripts, images, or resources required for meaningful rendering are blocked.

Watch for soft failures: a page can return a successful code while showing an error, empty product, unavailable shell, or redirect-like message. Confirm that canonical host, protocol, and trailing-slash rules behave consistently.

Test from outside internal sessions. A page that works only with cookies or a cached browser state is not reliably public.

Inspect the rendered product truth

Compare raw response and rendered output where the stack uses client-side code.

The product name, description, price context, availability, variants, identifiers, important attributes, shipping or return references, and internal links should be discoverable in a stable experience. Do not create facts only for crawlers that users cannot see.

Test failure states and slower loading conditions. If essential content appears only after a fragile API request, the page may be understandable in one inspection and empty in another.

Review canonical and index signals

Check canonical URL, page-level index directives, HTTP headers, sitemap inclusion, alternate-language references, and structured-data URL values for agreement.

Variant and filtered-page strategy requires deliberate decisions. A canonical that collapses valuable distinct products can hide them. Indexing every filter can create near-duplicate noise. Document the intended state by template so engineers and SEO operators are not fixing toward different goals.

Retired pages need an explicit replacement, redirect, or removal policy based on customer value and available equivalents.

Audit internal discovery

A valid page can remain practically invisible if nothing links to it.

Trace navigation, category pages, related products, editorial links, breadcrumbs, and XML sitemaps. Important pages should have descriptive, stable paths from other useful pages. Avoid relying only on internal search forms that crawlers may not use.

Check whether pagination and faceted navigation expose the intended catalog without generating endless combinations. Internal linking should reflect product priority and shopper journeys, not only database structure.

Prioritize by customer and revenue impact

Classify findings as critical, high, medium, or low using affected page value and failure severity.

A noindex on a hero category, a canonical to the wrong SKU, or an empty rendered product page deserves immediate work. Duplicate low-value filters may be important but should not displace a broken purchase path.

Assign template owner, expected correction, test URL, deadline, and rollback plan. Group systemic issues so one engineering fix resolves many pages safely.

After deployment, test both the original sample and a second set of URLs from the same template. Confirm that the correction did not create a new canonical, pagination, or rendering problem elsewhere. Record release version and verification evidence.

Add monitors for status changes, accidental noindex directives, canonical drift, missing product content, and sitemap anomalies on high-value pages. A quarterly audit finds accumulated issues; lightweight alerts reduce the time a new regression can remain invisible.

The Operator Read

AI visibility begins with a page that systems can reliably access, render, interpret, and connect to the right URL.

Test representative pages across every discovery layer. Prioritize the failures that hide product truth or misdirect customers. Verify live behavior after the fix and monitor for regression.

Technical accessibility is not the whole growth strategy. It is the gate every content and product strategy must pass through.