How I Added llms.txt to My Next.js Blog in 15 Minutes

If ChatGPT and Perplexity can't find your best posts, you didn't give them a map. llms.txt is the map.

No magic. No black box. A markdown file at the root of your site that tells the language model: here are the ten resources I exist for — start here.

What it actually is#

llms.txt is not an alternative to robots.txt, and not a replacement for sitemap.xml. It's a separate genre. A curator's index for AI agents. Jeremy Howard (co-founder of Answer.AI) proposed it on September 3, 2024, and the formal spec now lives at llmstxt.org. In the time since, the standard has crossed from academic experiment to industry practice.

The base logic is simple. Model context windows are too small to ingest most sites whole, and converting HTML — with its navigation, ads, and JavaScript — into LLM-friendly text is expensive and unreliable. Parsing your HTML burns tokens for nothing. Instead, you hand the agent clean Markdown with a list of URLs and one-line descriptions. The agent sees the structure in two seconds. And — far more importantly — it sees that structure through your eyes, not through the eyes of a sitemap generator that exports everything underneath.

Who's already on board#

Anthropic. Mintlify. Vercel. Astro. Deno. FastHTML. Stripe shipped support recently. Cursor lets you add third-party documentation through @Docs > Add new doc and uses llms-full.txt as a context source. The list grows monthly — the directory.llmstxt.cloud catalog shows it visibly.

It's spreading the way robots.txt did in 1994.

How it differs from sitemap.xml#

Sitemap says "here is everything I have." llms.txt says "here is what I exist for."

Structure — three sections decide everything#

# Site name
 
> One-line summary
 
## Section name
 
- [Page title](url): One-line description
- [Another page](url): Another description
 
## Optional
 
- [Less critical resource](url): Why it's optional

H1 with the name. A blockquote with one sentence that explains what the site is about. Then H2 sections with bulleted lists. The Optional section explicitly marks less-critical resources the agent can skip if context budget is tight. This is the formal part of the spec — don't break it.

Two implementations for Next.js#

Two paths. Pick based on how often you publish.

Option A — static (5 minutes)#

Create public/llms.txt. Fill it by hand. Deploy.

# alexturik.com
 
> Practical notes on Next.js, AEO tooling, and shipping side projects fast.
 
## Featured Posts
 
- [How I Added llms.txt to My Next.js Blog in 15 Minutes](https://alexturik.com/blog/how-i-added-llms-txt-to-my-nextjs-blog-in-15-minutes): Curator's index for AI crawlers — implementation walkthrough.
- [The 3-Word Tag That Hides Your Best Pages from AI](https://alexturik.com/blog/the-3-word-tag-that-hides-your-best-pages-from-ai): How `noindex` silently kills AI traffic.
 
## About
 
- [Author bio](https://alexturik.com): Who I am and what I build.

Works. Simple. Hopelessly dead a week later, because you will guaranteed forget to update it.

Option B — dynamic, via a route handler (10 minutes)#

This is the one I run on this site. A route handler in the Next.js App Router reads frontmatter from every post and generates the markdown at build time. Write it once — keeps itself in sync forever.

import { getAllPosts } from '@/lib/posts';
import { siteConfig } from '@/lib/site-config';
 
export const dynamic = 'force-static';
 
const HEADER = (url: string) => `# Oleksii Turovskyi
 
> Full-stack developer specializing in Next.js, React, Node.js, and WordPress. Builds rapid MVPs and maintains high-load systems.
 
## About
 
- **Name:** Oleksii Turovskyi
- **Role:** Full-stack developer
- **Email:** alexturik@gmail.com
- **Website:** ${url}
 
## Pages
 
- [Homepage](${url})
- [Blog](${url}/blog)
- [Sitemap](${url}/sitemap.xml)
`;
 
const FOOTER = `## Optional
 
- [GitHub](https://github.com/turovskiy)
- [LinkedIn](https://linkedin.com/in/alexturik)
- [X / Twitter](https://x.com/alexturik)
`;
 
export async function GET() {
  const url = siteConfig.url;
  const posts = await getAllPosts();
 
  const writingSection = posts.length === 0 ? '' :
    '\n## Writing\n\n' +
    posts
      .map((p) => {
        const html = `${url}/blog/${p.slug}`;
        const md = `${url}/blog/${p.slug}.md`;
        return `- [${p.frontmatter.title}](${html}) ([Markdown](${md})) — ${p.frontmatter.description}`;
      })
      .join('\n') +
    '\n';
 
  const body = `${HEADER(url)}${writingSection}\n${FOOTER}`;
 
  return new Response(body, {
    status: 200,
    headers: {
      'Content-Type': 'text/plain; charset=utf-8',
      'Cache-Control': 'public, max-age=3600, s-maxage=86400',
    },
  });
}

The file lives at src/app/llms.txt/route.ts — Next.js automatically serves it at /llms.txt. No rewrites, no manual config. Each blog post gets two links: the HTML version and a Markdown version (/blog/<slug>.md), so an agent that prefers raw text can grab content without parsing HTML.

One nuance: if your post source is Contentlayer, Velite, Notion API, or Sanity, just swap out the getAllPosts() loader. The rest of the architecture doesn't change.

What goes in, what stays out#

llms.txt is not a sitemap. Duplicating it as a sitemap is a curation failure. The logic is the opposite: only what you want to see in AI citations goes here.

Worth including:

• Top 10 evergreen posts — the ones that age slowly and accumulate traffic over years.
• Tools/calculators you've built. Agents love linking to specific utilities.
• Author bio. Your personal positioning in the landscape — not a vanity entry.
• Pricing/services pages, if you run a commercial site.

Don't include:

• Legal pages (terms, privacy). Agents don't care, chatbot users don't either.
• Listing pages (/blog, /tags/foo). That's the classic sitemap's job.
• Login, account, dashboard — private space behind auth anyway.
• Category indexes without your own added value.

A short test: "Do I want this exact page cited as an authoritative source in a ChatGPT answer?" If the answer is anything other than yes — drop it.

Bonus tier — llms-full.txt#

If llms.txt is the table of contents, llms-full.txt is the entire book in a single file. Mintlify generates both formats automatically so documentation is readable by LLM models without extra fetch requests. The agent receives the whole context at once.

Should you do it? If you have fewer than 50 posts — yes, definitely. More than that — you start hitting tokens: a too-large file just gets truncated by the agent, and the value drops sharply. I wrote one route handler that maps the same posts and concatenates the raw content from each MDX file. Same pattern as Option B above, just with body text instead of summaries.

The MIME-type gotcha that breaks half the implementations#

Your server has to serve llms.txt with Content-Type: text/plain or text/markdown. If it serves text/html — agents can quietly skip the file.

In the route handler above this is wired into the headers. In the static public/ variant, most CDNs (including Vercel) auto-set text/plain for .txt extensions, but it's worth validating.

Check it with one terminal command:

curl -I https://yoursite.com/llms.txt

Look for Content-Type: text/plain or text/markdown in the response. If you see text/html — you have a config problem. Open issue.

This is the failure mode that's the easiest to ship and the hardest to notice. Maybe 30% of sites that drop a file in public/ end up with a CDN serving it as HTML, and the file is invisible to agents from day one.

Ready to check yours?

This very post is part of the llms.txt on alexturik.com — sufficient meta for you?

If you're on Next.js and haven't shipped this standard yet — the clock is running. Fifteen minutes of work. Zero downside. The reward is your best material being visible to the entire generation of AI traffic that's forming right now.

Follow me on LinkedIn. For an AEO audit or implementation help on your platform, get in touch.