AI Visibility Checklist

Nothing else on this list matters if the systems cannot fetch the page. Most AI visibility problems are ordinary access problems wearing a new name.

• Separate the training crawlers from the search crawlers

  Confirmed#

  These are different bots with different jobs, and the distinction decides whether you appear in answers. Training crawlers (GPTBot, ClaudeBot, Google-Extended) collect material for future model versions and produce no citation today. Search crawlers (OAI-SearchBot, Claude-SearchBot, PerplexityBot) build the retrieval indexes that answers are assembled from right now. Blocking the first costs you nothing this quarter. Blocking the second removes you from answers.

  How to do it

  1. 01Open your live robots.txt and write down every AI user-agent it names.
  2. 02Sort each one into training or search using the provider's own documentation, not a blog post.
  3. 03Confirm every search crawler is allowed. These are the ones that cost you citations if blocked.
  4. 04Treat training access as a separate licensing decision for the business to make, not an SEO default.

  User-agent: OAI-SearchBot
  Allow: /
  
  User-agent: Claude-SearchBot
  Allow: /
  
  User-agent: PerplexityBot
  Allow: /

  OpenAI bot documentation

• Allow OAI-SearchBot explicitly

  Confirmed#

  This is the crawler that surfaces sites in ChatGPT's search features. It is frequently missing from robots.txt files that carefully allow GPTBot, because GPTBot is the famous one. Allowing the training crawler while omitting the search crawler is the single most common configuration error in this area.

  How to do it

  1. 01Fetch yoursite.com/robots.txt in a browser and search the text for OAI-SearchBot.
  2. 02If it is absent and you have a blanket disallow for unnamed agents, add an explicit allow block.
  3. 03Check the live file rather than your source file. CDNs and platform-generated robots.txt often differ from what is in the repository.
  4. 04Over the following weeks, grep your access logs for OAI-SearchBot to confirm it is arriving and receiving 200s.

  User-agent: OAI-SearchBot
  Allow: /

  OpenAI bot documentation

• Allow Claude-SearchBot and Claude-User

  Confirmed#

  Anthropic runs three bots. ClaudeBot collects training data, Claude-SearchBot improves search result quality, and Claude-User fetches pages when a person asks Claude a question. The second and third are the ones tied to appearing in answers.

  How to do it

  1. 01Add explicit allow blocks for Claude-SearchBot and Claude-User.
  2. 02Keep the ClaudeBot decision separate. Blocking training while allowing search is a coherent position.
  3. 03If you use bot management, allowlist Anthropic's published IP ranges so the requests are not challenged before robots.txt is ever consulted.

  User-agent: Claude-SearchBot
  Allow: /
  
  User-agent: Claude-User
  Allow: /

  Anthropic crawler documentation

• Drop anthropic-ai and Claude-Web from your robots.txt

  Confirmed#

  Both appear in almost every AI crawler listicle. Neither appears in Anthropic's current crawler documentation. Lines addressing user-agents that no longer exist are not harmful, but they are a reliable sign that a robots.txt was assembled from blog posts rather than from primary documentation.

  How to do it

  1. 01Search your robots.txt for anthropic-ai and Claude-Web and delete those blocks.
  2. 02Check every other AI agent in the file against the provider's current documentation while you are in there.
  3. 03Put a recurring reminder in the calendar to re-check this list every quarter. Providers add and rename agents without announcement.

  Anthropic crawler documentation

• Understand that robots.txt does not govern user-triggered fetches

  Confirmed#

  ChatGPT-User, Claude-User and Perplexity-User fire when a person asks a question, not on a crawl schedule. OpenAI states plainly that robots.txt rules may not apply to ChatGPT-User. Allowing or disallowing these agents in robots.txt is close to decorative. If you genuinely need to stop them, that is a server-level decision, not a robots.txt one.

  How to do it

  1. 01Stop treating a robots.txt line for these agents as a control. Keep it if you like, but record it as a statement of intent.
  2. 02If a real requirement exists to block them, enforce it at the edge with a rule matching the user-agent and returning 403.
  3. 03Verify the request genuinely came from the provider before acting on the user-agent string, which anyone can spoof. Each provider publishes an IP list for this.
  4. 04Confirm any block you add does not also catch the search crawlers, which is easy to do with a loose pattern match.

  OpenAI bot documentation

• Use preview controls, not Google-Extended, to manage AI Overviews

  Confirmed#

  Google-Extended governs whether your content may be used to train future Gemini models. Google states directly that it does not affect inclusion in Google Search and is not a ranking signal. AI Overviews are assembled from the Search index, which Googlebot builds. The controls that do reach AI Overviews and AI Mode are the snippet preview controls.

  How to do it

  1. 01To stay eligible for AI Overviews with no length cap, set max-snippet:-1.
  2. 02To exclude a whole page from snippets and from being used as direct input to AI Overviews and AI Mode, use nosnippet.
  3. 03To exclude only part of a page, wrap that element in data-nosnippet. This is the right tool for pricing, boilerplate or anything you do not want quoted out of context.
  4. 04Decide Google-Extended separately, as a Gemini training question. It has no bearing on either of the above.

  <!-- Eligible, no snippet length cap -->
  <meta name="robots" content="max-snippet:-1">
  
  <!-- Excluded from snippets and AI Overviews input -->
  <meta name="robots" content="nosnippet">
  
  <!-- Exclude one block only -->
  <div data-nosnippet>Not for quoting</div>

  Google: AI features and your website

• Serve meaningful HTML without client-side rendering

  Observed#

  Assume the fetcher runs no JavaScript. Googlebot renders; most AI fetchers are far simpler, and a page whose content arrives only after hydration can be fetched successfully and still yield nothing. View the raw response, not the rendered DOM, and confirm the substance is in the source.

  How to do it

  1. 01Fetch the page the way a simple client would and read what comes back.
  2. 02Confirm the headline, the main body text and any facts you want quoted are all present in that raw response.
  3. 03If the response is mostly an empty shell, move the important content to server rendering or static generation.
  4. 04Repeat for your highest-value pages, not just the homepage. Product and article templates usually differ.

  curl -sL -A "OAI-SearchBot" https://example.com/page | \
    sed 's/<[^>]*>//g' | tr -s '[:space:]' ' ' | head -c 2000

• Check that access controls have not quietly blocked the bots

  Observed#

  WAF rules, bot management, rate limiting and geo-blocking all sit above robots.txt and enforce silently. Confirm in server logs that the search crawlers are receiving 200s. A crawler pattern that stops without explanation is usually infrastructure, not algorithm.

  How to do it

  1. 01Filter your access logs by each search crawler user-agent and look at the status codes, not just the hit count.
  2. 02Treat a wall of 403s or 429s as the finding. That is a block or a rate limit, not disinterest.
  3. 03Review any managed bot protection. Default rulesets frequently challenge AI crawlers as unknown automation.
  4. 04Allowlist the providers' published IP ranges rather than trusting the user-agent string alone.

  grep -Ei "OAI-SearchBot|Claude-SearchBot|PerplexityBot|GPTBot" access.log \
    | awk '{print $NF, $9}' | sort | uniq -c | sort -rn

Once a page is fetched, the question is whether a model can isolate a specific claim, attach it to your brand, and reproduce it without ambiguity.

• Mark up entities with schema.org

  Confirmed#

  Two platforms have said on the record that structured data helps their systems. Microsoft's Fabrice Canel stated at SMX Munich that schema markup helps Microsoft's LLMs understand content, and Google's Ryan Levering described structured data as material to grounding their generative systems at Search Central Live. This is one of the few technical GEO recommendations with direct confirmation from the platforms rather than from vendors.

  How to do it

  1. 01Pick the type that genuinely describes the page: Article, Product, FAQPage, HowTo, Organization, Person.
  2. 02Emit it as JSON-LD in the head. Do not mark up anything a visitor cannot see on the page.
  3. 03Validate every template with the Rich Results Test and the schema.org validator before shipping.
  4. 04Connect related entities with @id rather than repeating the same organisation block on every page.

  {
    "@context": "https://schema.org",
    "@graph": [
      {
        "@type": "Person",
        "@id": "https://sharaki.me/#person",
        "name": "Emad Sharaki",
        "url": "https://sharaki.me",
        "jobTitle": "Senior SEO & GEO Strategist",
        "sameAs": [
          "https://www.linkedin.com/in/emadsharaki/",
          "https://x.com/emadsharaki",
          "https://github.com/emadsharaki"
        ]
      },
      {
        "@type": "HowTo",
        "name": "AI Visibility Checklist",
        "author": { "@id": "https://sharaki.me/#person" }
      }
    ]
  }

  Google structured data guidance

• Make Organization and Person entities unambiguous

  Observed#

  Give the entity one canonical page, a stable @id, and sameAs links to the profiles that corroborate it. Models resolve entities by cross-referencing; a brand that appears under three spellings across four properties is three weak entities rather than one strong one.

  How to do it

  1. 01Choose one canonical URL to be the entity's home and give it a permanent @id.
  2. 02List every profile you control under sameAs: LinkedIn, GitHub, Crunchbase, Wikidata, the lot.
  3. 03Audit those profiles for a single consistent name, role and location. Fix the ones that disagree.
  4. 04Reference the same @id from every other page rather than restating the entity each time.

• Write self-contained answers under descriptive headings

  Observed#

  A retrieved passage arrives without its page around it. If a section only makes sense after reading the two above it, it cannot be quoted. Each heading should pose a real question and the passage beneath it should answer that question completely, in the first sentence or two.

  How to do it

  1. 01Rewrite headings as the question a person would actually type or say.
  2. 02Answer in the first sentence beneath the heading. Put the context after it, not before.
  3. 03Read each section on its own, with everything above it covered. If it no longer makes sense, it cannot be cited.
  4. 04Keep each answer to a length that can be lifted whole, roughly forty to eighty words.

• State the subject by name rather than by pronoun

  Observed#

  Repeat the brand or product name where a human writer would reach for it or this. Anaphora resolves fine inside a document and badly inside a retrieved fragment, and a fragment that never names its subject is hard to attribute to you.

  How to do it

  1. 01Search the draft for sentences opening with it, this, they or the product and name the subject instead.
  2. 02Make sure the brand name appears at least once in every section, not only in the intro.
  3. 03Accept the mild repetition. It reads slightly heavier to a human and far more clearly to a retriever.

• Put facts in tables and lists, not in prose

  Observed#

  Specifications, prices, comparisons and steps extract far more cleanly from a structured block than from a paragraph describing them. This is the least glamorous item here and one of the most reliable.

  How to do it

  1. 01Find every paragraph that describes several parallel things and convert it to a table.
  2. 02Use real table markup with headers, not a grid of divs styled to look like one.
  3. 03Give each table a caption or a heading immediately above it that says what it contains.
  4. 04Turn any sequence of actions into an ordered list.

• Date the content and keep the date honest

  Observed#

  Systems answering time-sensitive questions favour material they can date. A visible published and updated date, matching the dateModified in your markup, is worth more than an artificially refreshed timestamp, which is a signal readers and systems both learn to discount.

  How to do it

  1. 01Show the published date, and the updated date when there has been a real update.
  2. 02Make datePublished and dateModified in your markup match what the page displays.
  3. 03Never bump the date without changing the content. It is trivially detectable and it trains people to distrust the byline.
  4. 04For anything that goes stale, say what was verified and when, in the text itself.

Access and structure make a page quotable. Neither makes it worth quoting. This is the section most checklists skip, and it is the one that decides outcomes.

• Publish something that cannot be sourced elsewhere

  Observed#

  A model assembling an answer selects among many pages saying the same thing. Original data, a documented method, a first-hand account, a real number: these give a reason to choose your page specifically. Content that summarises what is already well covered competes only on authority signals you probably do not have.

  How to do it

  1. 01List what you can see that nobody outside your organisation can: your own data, your own tests, what you learned the hard way.
  2. 02Pick the one that answers a question people already ask and publish it with the method attached.
  3. 03Before writing anything else, ask what this page contains that the top five results do not. If the answer is nothing, do not write it.
  4. 04Repeat the study on a schedule. A dataset that updates becomes a reference; a one-off becomes an archive.

• Put the number in the sentence

  Observed#

  Significantly faster is unquotable. Thirty percent faster, measured across seven markets is quotable, attributable and checkable. Specificity is what converts a page from background reading into a citation.

  How to do it

  1. 01Find every vague quantifier in the draft: significant, dramatic, many, most, faster.
  2. 02Replace each with the actual figure, or cut the claim if you do not have one.
  3. 03Attach the scope to the number: over what period, across what sample, measured how.
  4. 04Put the strongest number in the opening sentence of its section, where a retriever will find it.

• Answer the question the way it is asked

  Observed#

  Conversational prompts are longer and more specific than keyword queries. Cover the real question, including the awkward parts, rather than the shortest version of it that ranks. Comparisons, limitations and when not to use this are frequently retrieved and rarely written.

  How to do it

  1. 01Collect the questions people actually ask you: sales calls, support tickets, conference hallways.
  2. 02Write the page around those sentences rather than around a keyword.
  3. 03Include the sections your competitors avoid: limitations, honest comparisons, when the answer is do not.
  4. 04Test by asking an assistant the same question and reading which pages it chose and why.

• Show the working

  Observed#

  Method, sample size, date, limitations. A claim that shows how it was reached survives scrutiny by both a human editor and a system weighing sources. It is also what makes a page worth linking to, which feeds the next section.

  How to do it

  1. 01State the method in plain language near the finding, not in a footnote.
  2. 02Give the sample size, the date range and the tools used.
  3. 03Name the limitations yourself. It costs less than being corrected in public.
  4. 04Publish the underlying data where you can. A downloadable dataset attracts links no summary will.

How a system decides you are worth trusting. This is where GEO diverges most sharply from classical SEO, and where the industry's numbers deserve the most scepticism.

• Treat unlinked brand mentions as first-class

  Observed#

  Several independent measurements report that the volume of brand mentions across the web tracks AI citation rates more closely than backlink counts do. The direction of this finding is consistent across sources; the exact coefficients vary by who is selling what, so use it to set priorities rather than to forecast results.

  How to do it

  1. 01Monitor mentions of the brand name, not just links to the domain.
  2. 02Chase the mention rather than the link when pitching. A named reference in a respected piece is worth pursuing on its own terms.
  3. 03Give people something concrete to mention: a named framework, a dataset, a number.
  4. 04Correct or claim mentions that name you wrongly. Entity consistency is the point.

• Be present where the systems already look

  Observed#

  Community platforms, established publications and reference sites are cited heavily and disproportionately. Being discussed in those places is a different activity from publishing on your own site, and for a new domain it usually moves faster.

  How to do it

  1. 01Ask an assistant your ten most important questions and record every domain it cites.
  2. 02That list is your target list. It is usually shorter and more boring than expected.
  3. 03Participate on those platforms as a person who is genuinely useful, not as a brand placing content.
  4. 04Re-run the same ten questions each quarter and watch whether the cited set moves.

• Optimise per platform, not in general

  Observed#

  Overlap between the domains cited by different assistants is low. A page that performs in one may be invisible in another, and a single AI visibility score averaged across platforms hides more than it reveals. Measure them separately.

  How to do it

  1. 01Report each platform on its own row. Never average them into one score.
  2. 02Identify which platform actually sends the traffic or the buyers you care about and weight the work accordingly.
  3. 03Compare the cited sources per platform. They favour visibly different kinds of source.

• Keep claims about yourself consistent everywhere

  Observed#

  Site, profiles, directories and press should agree on what you do, where you are and what you are called. Contradictory descriptions across properties weaken entity resolution, and they are noticed by human evaluators for exactly the same reason.

  How to do it

  1. 01List every property that describes you and read them side by side.
  2. 02Fix the disagreements: name, role, location, founding date, what you actually do.
  3. 03Write the canonical description once and reuse it rather than improvising each time.
  4. 04Re-check after any change of role, name or address, which is when drift creeps in.

Unmeasured AI visibility work is indistinguishable from superstition. Set the baseline before you change anything.

• Track a fixed prompt set over time

  Observed#

  Write the questions a real buyer would ask, keep the list stable, and run it on a schedule. Changing the prompts between runs makes the results incomparable, which is the most common flaw in AI visibility reporting.

  How to do it

  1. 01Write twenty to fifty questions covering the buying journey, in the words a person would use.
  2. 02Freeze the list. Version it if you must change it, and never compare across versions.
  3. 03Run it on the same schedule against every platform that matters.
  4. 04Record the full answer text, not a yes or no. You will want to re-read it later.

• Record share of mention, sentiment and which competitors displaced you

  Observed#

  Presence alone is a thin metric. Being mentioned third, hedged, and alongside two competitors is a different result from being the sole recommendation, and only the fuller picture tells you what to fix.

  How to do it

  1. 01For each answer log whether you appear, in what position, in what tone, and who else is named.
  2. 02Track the competitor set over time. A new name appearing repeatedly is an early warning.
  3. 03Report share of mention rather than a raw count so the number survives changes in prompt volume.

• Capture the cited URLs, not just the brand name

  Observed#

  Knowing which specific pages get pulled in tells you what to produce more of. It also reveals when a competitor's page, a forum thread or an outdated article is answering on your behalf.

  How to do it

  1. 01Log every cited URL alongside the answer.
  2. 02Group by domain to see who is answering for your category, and by page to see what format wins.
  3. 03Watch for your own outdated pages being cited. That is a fast fix with real impact.

• Read your server logs for the search crawlers

  Observed#

  Crawl activity from OAI-SearchBot, Claude-SearchBot and PerplexityBot is the earliest signal available, and it moves well before citations do. It is also the only measurement in this section that does not depend on a third-party tool.

  How to do it

  1. 01Report AI crawler hits weekly, split by user-agent and by URL.
  2. 02Watch which sections they favour. That is the closest thing to a free relevance signal you will get.
  3. 03Treat a sudden drop as an infrastructure incident and check status codes first.

  grep -Ei "OAI-SearchBot|Claude-SearchBot|PerplexityBot" access.log \
    | awk '{print $7}' | sort | uniq -c | sort -rn | head -25

• Expect variance and sample accordingly

  Observed#

  The same prompt returns different answers across runs, sessions and regions. A single check is an anecdote. Repeat each prompt several times and report the distribution, not the one result that suits the slide.

  How to do it

  1. 01Run every prompt at least three times per cycle, in fresh sessions.
  2. 02Report the share of runs you appeared in, not whether you appeared.
  3. 03Test from the markets you sell into. Answers differ by region.
  4. 04Set a threshold for what counts as a real change before you start reporting movements to anyone.

Recommendations that circulate widely and do not survive contact with primary sources. Each is here with the evidence against it, and with what to do instead.

• llms.txt as a route into Google

  Contested#

  Google updated its documentation in June 2026 to state that you do not need machine-readable files, AI text files or Markdown to appear in Google Search including its generative features, because Search does not use them. Independent log analyses also find the major AI crawlers overwhelmingly skip the file and fetch HTML directly. It is harmless to publish and reasonable as a convention bet, but it is not a Google lever and should not sit near the top of a plan.

  What to do instead

  1. 01Publish it if you want to, in ten minutes, and then forget about it.
  2. 02Spend the time you saved on the HTML the crawlers do fetch.
  3. 03Check your own logs for requests to /llms.txt before you believe anyone's claim about it, including this one.

  Google says llms.txt will not help or hurt rankings

• Blocking Google-Extended to stay out of AI Overviews

  Contested#

  Google-Extended controls Gemini model training only. Google states it does not affect Search inclusion and is not a ranking signal. AI Overviews draw on the Search index built by Googlebot, so this block does not achieve what it is usually deployed to achieve.

  What to do instead

  1. 01Use nosnippet or data-nosnippet instead. Those are the controls that reach AI Overviews and AI Mode.
  2. 02Keep the Google-Extended decision as what it is: whether Gemini may train on your content.
  3. 03Understand the trade before you act. Excluding yourself from snippets excludes you from the answer entirely.

  Google: AI features and your website

• Relying on robots.txt to control user-triggered agents

  Contested#

  OpenAI states that robots.txt rules may not apply to ChatGPT-User, and the equivalent user agents from other providers behave the same way by design. A robots.txt line addressed to them is a statement of intent, not a control.

  What to do instead

  1. 01Enforce at the edge if you have a real requirement, and verify the source IP before acting.
  2. 02Otherwise leave it alone and stop reporting it as a control that is in place.

  OpenAI bot documentation

• Precise multipliers from vendor case studies

  Contested#

  Figures of the form schema produces 2.5x more citations circulate constantly and originate almost entirely with companies selling AI visibility products. The underlying direction is often right and confirmed elsewhere; the decimal places are marketing. Cite the platform statements, run your own measurement, and leave the multipliers alone.

  What to do instead

  1. 01Before repeating a statistic, find who funded it and what they sell.
  2. 02Cite the platform's own words when a platform has spoken. That is what survives being challenged in a meeting.
  3. 03Produce your own number from your own prompt set. It is the only one that describes your site.