Skip the AI Visibility Checker: Audit Your Site by Hand

AI (artificial intelligence) visibility is whether LLMs (large language models) such as ChatGPT, Claude, and Perplexity can find, parse, and cite a page. You can audit it by hand with eight checks: llms.txt, robots.txt bot access, JSON-LD presence, author and dateModified fields, FAQPage markup, a markdown mirror, and question-phrased headings. No paid checker required.

TL;DR#

• AI visibility means a model can find, parse, and cite your page, not just rank it. None of them cost anything to run.
• Per a scan of 93 pages ranking for AEO (answer engine optimization) or GEO (generative engine optimization) queries on 2026-08-09, only 52.7% served a working llms.txt file. Just 8.6% published a markdown mirror of their own URLs.
• Robots.txt is the closest thing to a hard gate here: a named Disallow line tells that specific AI bot not to fetch the page. The exception is user-initiated fetches — OpenAI documents that robots.txt rules may not apply to ChatGPT-User, because a person asked for that page.
• Passing every check does not guarantee a citation. My own site blocks zero AI bots and runs a 10,750-byte llms.txt file, yet logged 33 impressions and 0 clicks in Google Search Console over a 90-day window.
• Every check here needs nothing more than a browser and, sometimes, a terminal.

What Does AI Visibility Mean, and How Is It Different From Rankings?#

Search rankings measure position on a results page built for humans to click. AI visibility measures something else: whether a model, while generating an answer, retrieves a page, parses it, and decides it is worth citing. A page can rank on page one and still never get pulled into a ChatGPT or Perplexity answer; retrieval works differently from classic ranking.

Most retrieval systems convert both a question and candidate pages into numeric vectors, then match on similarity before generating text. An embedding playground makes that comparison tangible by scoring how close several texts sit in meaning.

The eight checks below measure structural readiness: whether the door is open and the content is legible to a crawler. This guide replaces a paid AI visibility checker with those eight signals, run by hand. None of them measure the outcome: whether a model actually chooses to cite a page over a competitor's. That gap between readiness and outcome shows up directly in the case study near the end of this guide.

The Eight Checks You Can Run By Hand#

Every check below runs in a browser, and occasionally a terminal. None needs a login. Run them against your own homepage or best-performing post, in order, and keep a tally. Each maps to a row in the reference table further down, next to how many ranking pages already pass it.

1. Does /llms.txt Return a Real File?#

Open example.com/llms.txt in a browser, or run curl -I https://example.com/llms.txt from a terminal. A pass is an HTTP (Hypertext Transfer Protocol) 200 status with a content type other than text/html. A fail is a redirect to your homepage or a styled 404 page. The llms.txt convention was proposed by Jeremy Howard at llmstxt.org in September 2024. It gives language models a compact, curated map of a site's key pages, since context windows are too small to hold most websites whole. It is a voluntary convention, not a Google or OpenAI standard.

2. Does robots.txt Block Any Named AI Bot?#

Open yourdomain.com/robots.txt and look for a User-agent line matching any of these bots, each with its own rule below:

• GPTBot, ChatGPT-User, OAI-SearchBot (OpenAI)
• ClaudeBot, Claude-User (Anthropic)
• PerplexityBot, Perplexity-User (Perplexity)
• Google-Extended (Google's AI-training signal, separate from Googlebot)
• Applebot-Extended, Bytespider, CCBot, meta-externalagent (Apple, ByteDance, Common Crawl, Meta)

A Disallow: / under one of those names blocks only that bot; the same line under User-agent: * blocks everyone. Google's documentation states that robots.txt "tells search engine crawlers which URLs the crawler can access on your site," and warns that "it is not a mechanism for keeping a web page out of Google." OpenAI's documentation adds a narrower point: ChatGPT-User requests are triggered live by a person's question, so "robots.txt rules may not apply" to it. Per the market scan behind this guide, 8.6% of 93 pages blocked at least one AI bot outright. That included forbes.com, which disallowed eight named bots and still ranked #4 for "answer engine optimization" the same day.

3. Is There Any JSON-LD on the Page at All?#

Right-click the page, choose View Page Source, and search for application/ld+json. Each match opens a block of JSON-LD (JavaScript Object Notation for Linked Data), schema.org's usual markup format. Google's free Rich Results Test runs the same search without asking you to read raw HTML (HyperText Markup Language). A JSON-LD block exposes properties such as content type, author, and publish date directly, instead of leaving a model to infer them from paragraph text. Per the same scan, 81.7% of the 93 ranking pages carried at least one JSON-LD block, the most common of the eight signals measured here.

4. Does the JSON-LD Name an Author?#

Inside the JSON-LD block from the previous check, search for the string "author". A pass looks like an author object naming a person or organization. A fail means the block covers other properties but skips attribution. According to the same scan, 62.4% of the 93 pages included an author property inside their JSON-LD.

5. Does the JSON-LD Include a dateModified Field?#

Search the same JSON-LD block for "dateModified". This is separate from datePublished, and separate from any visible "Updated on" text elsewhere on the page. Per the same scan, 59.1% of the 93 pages carried a dateModified property. A page that gets updated but never touches this field loses the signal even though the content changed.

6. Does Any JSON-LD Block Carry FAQPage?#

Search the JSON-LD for "@type": "FAQPage". One caveat matters here. Google stopped showing its FAQ rich result in search on May 7, 2026, and removed the supporting documentation from Search Central in June 2026. It no longer earns the old expandable-question snippet in classic search. It may still mark which block of text answers which question, a hint some answer engines could use; this scan measures presence only, not whether that hint changes what gets cited. As of the same scan, 25.8% of the 93 pages carried FAQPage markup somewhere in their JSON-LD.

7. Does a Markdown Mirror of the URL Exist?#

Take any page's canonical URL (Uniform Resource Locator) and add .md to the end, for example example.com/blog/post-slug.md, then request it. A pass returns 200 with a markdown or plain-text content type. A fail returns the site's normal 404. Unlike llms.txt, a markdown mirror must be built per page, a stronger signal of deliberate effort than one root-level file. Per the same scan, only 8.6% of the 93 pages, 8 of them, served a working .md version of their own URL.

8. Are Any Headings Phrased as Questions?#

Open the page in reading mode, or inspect it and list every h2 and h3 element, then count how many end in a question mark. A heading like "What Does robots.txt Actually Block?" mirrors how a person phrases a prompt, which a plain label like "Robots.txt Overview" does not. As of the same scan, 74.2% of pages carried at least one question-phrased heading, so having one is close to standard practice. Across every heading on every page, though, only 18.7% (410 of 2,195) were phrased as questions.

Signal Reference Table#

Every baseline below comes from one scan I ran on 2026-08-09. I pulled the top 20 United States Google results for seven queries. They were: llms.txt, answer engine optimization, generative engine optimization, how to get cited by ai, geo vs seo, aeo vs seo, and ai visibility checker, all live through DataForSEO. After excluding platforms such as YouTube, Reddit, LinkedIn, Wikipedia, and X, 93 pages across 78 domains remained. Each signal used the exact test described above.

Which of These Eight Signals Actually Matters?#

Of the eight, robots.txt comes closest to a hard gate. If a named bot's group carries Disallow: /, that crawler is told not to fetch the page, and everything else stops mattering for it. The rule is not absolute: OpenAI states robots.txt may not apply to ChatGPT-User, since those requests come from a person rather than a scheduled crawl. Treat a block as decisive for training and search crawlers, unreliable for user-triggered ones. Fix any unintended block first, and check intended ones too, since a rule aimed at one crawler widens easily via a stray User-agent: * line.

The other seven are readiness signals, not gates. Only 8.6% of ranking pages serve a markdown mirror, so skipping it leaves you with the 91.4% majority. llms.txt is closer to even at 52.7%, so its absence puts a page in the 47.3% minority. JSON-LD missing an author or dateModified field is a smaller gap than no JSON-LD at all, since 81.7% of ranking pages clear that first bar. FAQPage markup and question-phrased headings are formatting choices applied to a page that has already been fetched, so they sit later in the chain than access and parsing. This scan measured only whether pages carry them, not what those signals do once present.

What These Checks Don't Show#

All eight checks are static. They confirm a crawler is technically permitted to reach a page and that the page hands over some machine-readable structure once it arrives. None of them measure whether the content is deep, accurate, or distinctive enough to be worth citing over a competitor's page. Neither this checklist nor any AI visibility checker tests that part. A page can pass every check here and still be invisible.

My Own Site Passed Almost Every Check. It Still Got Zero Clicks.#

My site, alexturik.com, blocks none of the twelve major AI crawlers covered in check 2. Its llms.txt file runs 10,750 bytes, above the 8,072-byte median for domains that have one. It serves 15 separate markdown mirrors, well past this scan's 8.6% adoption rate.

According to Google Search Console, over a 90-day window (May 8 to August 6, 2026) the entire site recorded 33 total impressions and 0 clicks. Those numbers span the only four pages with any visibility. The page built specifically to be found for this topic, /extensions/aeo-checker, logged 4 impressions and 0 clicks. It ranked at position 88.5 for "aeo checkers." Technical readiness opened the door. It did not make the page visible to a person, or to a model summarizing results.

This blog's median post runs 1,383 words against the market's 2,511-word median, about 1.8 times shorter. Question-phrased headings are 5.4% of this blog's headings (7 of 130) against the sample's 18.7% share. FAQPage markup sits on three product pages under /plugins, none in the blog. None of these gaps is confirmed as the cause of zero clicks. What they show: a clean technical scan and competitive, well-built content are different projects, and this guide only audits the first.

One failure mode sits outside these eight checks entirely: a stray noindex value on the meta robots tag, often left over from staging. Another is pages that render fine for humans but stay empty for crawlers that skip JavaScript. For the fuller list, see why sites don't show up in ChatGPT or Perplexity.

If You'd Rather Automate This#

Running all eight checks by hand takes about fifteen minutes. If you would rather automate the scan, full disclosure: I built a Chrome extension called AEO Checker that runs a superset of these checks, nine in total, on whatever page is open, and returns a 0-100 score with a fix instruction on every warning. That is a direct conflict of interest, and it is exactly why every check above works without installing anything.

FAQ#

Three FAQ (frequently asked questions) items, pulled from the same search results this guide targets:

How to answer engine optimization?#

Answer engine optimization means structuring a page so answer engines can lift a complete response directly from it. That means a direct-answer paragraph near the top, one clear definition, JSON-LD that names the author and date, and headings phrased as the questions people actually ask. Run the eight checks in this guide first; they show which pieces your page is missing.

What's the best answer engine optimization tool?#

There is no independently verified "best" AI visibility checker. This guide does not test or rank third-party AEO products, and every vendor scores its own product differently. What is verifiable is the underlying signal set: llms.txt, robots.txt bot access, JSON-LD completeness, FAQPage markup, and question-phrased headings. The eight checks above test exactly those signals, by hand, at no cost.

Is ChatGPT an answer engine?#

Yes. ChatGPT answers questions directly instead of returning a list of links, which is the working definition of an answer engine used throughout this guide. Three of the crawlers OpenAI documents matter for this audit: GPTBot for model training, OAI-SearchBot for search indexing, and ChatGPT-User for requests triggered live by someone asking a question inside the chat. OpenAI also documents OAI-AdsBot, which checks advertising links rather than gathering answers.

Part of the Answer Engine Optimization cluster — the full reading order in dependency sequence, plus a glossary of every term used across these articles.