How to Create an llms.txt File: Step-by-Step Tutorial (2026)

Implementing an llms.txt file is technically straightforward, but requires careful execution to satisfy standard validators and crawler prerequisites. This practical guide covers everything from inspecting competitor files and configuring server headers to copy-paste code snippets for modern web frameworks.

How to See the llms.txt File of Any Website

Before deploying a custom implementation, it is highly beneficial to audit how industry leaders structure their files. To see the file of any website, users simply append /llms.txt to the root domain (e.g., https://www.anthropic.com/llms.txt or https://www.cloudflare.com/llms.txt).

Because some browser configurations attempt to render plain text files improperly or download them as binaries, developers should inspect files utilizing command-line utilities:

# Check HTTP status code, Content-Type, and response headers
curl -I https://example.com/llms.txt

# Print raw, unformatted Markdown directly to your terminal
curl -s https://example.com/llms.txt

llms.txt File Encoding, MIME Types & HTTP Headers

As established in the technical specification, incorrect header configuration will render the file invisible to AI engines. Regardless of the deployment platform, the server must be configured to output specific HTTP parameters:

• Strict UTF-8 Encoding: Must be encoded in UTF-8 without corrupt byte sequences to ensure seamless LLM tokenization.
• HTTP 200 OK Status: A direct 200 OK status code is required by strict specification linters and automated agents. While closed CMS platforms (like Weebly, HubSpot, or BigCommerce) often rely on 301 redirects to CDN-hosted assets as a fallback, routing through a Cloudflare Worker edge proxy ensures a true root 200 OK response.
• Correct MIME Type: The payload must be served as Content-Type: text/plain; charset=utf-8 or Content-Type: text/markdown; charset=utf-8.
• HTTP Link Header: Announce the file globally on all web pages using: Link: <https://www.llms-text.com/llms.txt>; rel="describedby".

Copy-Paste Code Snippets for Modern Frameworks

For dynamic web applications, hardcoding a static text file is less efficient than generating it programmatically. The following implementations generate compliant llms.txt files across various stacks:

1. Next.js (App Router)

In Next.js, create a Route Handler at app/llms.txt/route.ts. Using force-static combined with Incremental Static Regeneration (ISR) ensures high performance and edge caching:

// app/llms.txt/route.ts
import { NextResponse } from 'next/server';

export const dynamic = 'force-static';
export const revalidate = 3600; // Hourly ISR

export async function GET() {
  const content = `# Project Documentation
> High-performance platform built for AI agent integration.

## Core Features
- [Getting Started](https://example.com/docs/start.md): Initial setup and configuration.
- [API Reference](https://example.com/docs/api.md): Complete RESTful API endpoints.

## Optional
- [Changelog](https://example.com/changelog.md): Historic version releases.
`;

  return new NextResponse(content, {
    headers: {
      'Content-Type': 'text/plain; charset=utf-8',
    },
  });
}

2. Astro

Astro handles root-level text files effortlessly using static file routing. Create src/pages/llms.txt.ts:

// src/pages/llms.txt.ts
export async function GET() {
  const markdownText = `# Astro Site Overview
> A blazing fast static site builder.

## Guides
- [Routing](https://example.com/routing.md): How routing works.`;

  return new Response(markdownText, {
    headers: { 'Content-Type': 'text/plain; charset=utf-8' },
  });
}

3. Static HTML & Webflow

For static websites, save your file as llms.txt in UTF-8 encoding and place it in your public web root directory (such as public/ or dist/). In Webflow, you do not need FTP or server access: navigate to Site settings → SEO → LLMs.txt, click Upload file, select your UTF-8 encoded llms.txt (under 100 KB), and publish your site to your custom domain.

4. WordPress

In WordPress, modern SEO plugins (Yoast SEO, Rank Math, AIOSEO, and SEOPress) feature built-in llms.txt toggles out of the box. For a custom, rich llms.txt and llms-full.txt file from this generator, upload the file directly to your root folder (where wp-config.php resides) via SFTP, cPanel File Manager, or temporarily with WP File Manager (deactivating it after upload). Alternatively, inject the route programmatically via functions.php (be sure to visit Settings → Permalinks → Save Changes to flush rewrite rules):

// Add to functions.php
add_action('init', function() {
    add_rewrite_rule('^llms\\.txt$', 'index.php?llms_txt=1', 'top');
});

add_filter('query_vars', function($vars) {
    $vars[] = 'llms_txt';
    return $vars;
});

add_action('template_redirect', function() {
    if (get_query_var('llms_txt')) {
        header('Content-Type: text/plain; charset=utf-8');
        echo "# Brand Site\n> Corporate blog and services.\n\n## Services\n- [Consulting](https://example.com/consulting): Strategy docs.";
        exit;
    }
});

5. Cloudflare Workers

For enterprise edge deployments, a Cloudflare Worker can intercept the route and serve the file globally with ultra-low latency:

export default {
  async fetch(request, env, ctx) {
    const url = new URL(request.url);
    if (url.pathname === '/llms.txt') {
      const content = `# Edge Platform\n> Global routing network.\n\n## Docs\n- [Setup](https://example.com/setup.md): Deployment steps.`;
      return new Response(content, {
        headers: { 'Content-Type': 'text/plain; charset=utf-8' }
      });
    }
    return fetch(request);
  }
};

Step-by-Step Implementation Methodology

1. Step 1: Content Curation and Auditing: Review your sitemap and curate the list down to 10 to 20 highest-value, evergreen URLs (product features, documentation, architecture, pricing).
2. Step 2: Structural Drafting: Write your single H1 tag, craft your blockquote summary, group links under 4 to 7 logical H2 categories, and ensure every link includes a 10-to-20 word description.
3. Step 3: Generate Clean Markdown Twins: Wherever possible, provide pure .md versions of your pages without HTML wrappers.
4. Step 4: Deploy and Configure Headers: Upload the file to your root domain and inject the HTTP Link header on your primary HTML templates.
5. Step 5: Validation and Testing: Verify the live file using the llms.txt Validator to ensure perfect standard compliance.

Create Your llms.txt File in Minutes

Don't want to build it manually? Use the free llms.txt Generator to automatically crawl your domain and export standard-compliant llms.txt and llms-full.txt files instantly.