ByteScope

llms.txt Generator

Turn your sitemap into an llms.txt you can publish today.

Sitemap URLs and page heads are fetched by our Worker and not stored; an uploaded XML file stays in your browser.

1. Sitemap

or upload the XML:

Where to put it and which Content-Type

Upload the file to your site root so it answers at /llms.txt, and serve it with Content-Type: text/plain; charset=utf-8. Without the charset, non-English titles can be garbled; a single-page app that answers every path with its HTML shell will serve that shell instead of the file.

Cloudflare Pages or Workers static assets: _headers in the output folder

/llms.txt
  Content-Type: text/plain; charset=utf-8

Netlify: _headers in the publish folder

/llms.txt
  Content-Type: text/plain; charset=utf-8

nginx: inside the server block

location = /llms.txt {
    types { text/plain txt; }
    charset utf-8;
}

Vercel: vercel.json at the project root

{
  "headers": [
    {
      "source": "/llms.txt",
      "headers": [
        {
          "key": "Content-Type",
          "value": "text/plain; charset=utf-8"
        }
      ]
    }
  ]
}

After deploying, run the AI search check on your homepage to confirm it is read as text.

About this tool

llms.txt is a Markdown file at your site root that gives AI tools a short, curated map of your pages: an H1 with the site name, a one-line summary in a blockquote, then H2 sections of - [title](url): description links (the llmstxt.org convention). This tool builds one from your sitemap, so you do not have to type hundreds of lines by hand.

How it works

Paste a sitemap URL or upload the XML. A sitemap index is followed automatically, up to 20 child sitemaps. Pages are grouped into sections by the first path segment, a group with only one page folds into the top section, and every section can be renamed. If your URLs start with a language prefix such as /en/ or /ja/, pick one language so the file is not three copies of the same site. Exclusion rules use robots.txt pattern syntax: * matches anything and a trailing $ anchors the end. The page cap defaults to 500 and goes up to 2,000.

Titles and descriptions

Fetching titles reads each page's <title> and <meta name="description"> through our Worker, 10 pages per request with at most 2 requests in flight. The Worker allows 60 requests a minute per IP address, so a 500-page site takes about a minute, and a rate-limit pause is waited out automatically. You can stop at any point and keep what has arrived, or skip the step and use each URL's path as its title. Nothing is written by an AI: the descriptions are the ones already on your pages.

Where to put it

Serve the file at /llms.txt on your site root with Content-Type: text/plain; charset=utf-8. Most hosts already send text/plain for .txt, but some leave out the charset, which garbles non-English titles, and a single-page app often answers unknown paths with its HTML shell, which an AI tool then reads as your llms.txt. The snippets under the preview set the header on Cloudflare, Netlify, nginx and Vercel.

Frequently asked questions

What is the llms.txt format?

A Markdown file proposed at llmstxt.org: one # heading with the site or project name, an optional > summary, then ## sections listing links as - [title](url): notes. A section named Optional marks links an AI tool can skip when its context is short.

How do I create llms.txt from a sitemap?

Paste the sitemap URL (usually /sitemap.xml, or the Sitemap: line in robots.txt), pick a language if the site has several, add exclusions, fetch titles, rename sections if you like, then download the file and upload it to your site root.

Which Content-Type should llms.txt use?

text/plain; charset=utf-8. The content is Markdown, but plain text is what every client accepts, and the charset keeps non-English titles readable. After deploying, run the AI search check on your homepage: its llms.txt card flags a file that comes back as HTML.

Does it use AI to write the descriptions?

No. Each link uses the page's own <title> and meta description, cleaned of line breaks and extra spaces. If a page has no description the link has no notes. Edit the file afterwards if you want a better summary.

Why are some titles just URL paths?

That page's head could not be read: it returned an error, redirected somewhere that failed, was not HTML, or has no <title>. It also happens for every page when you skip the fetch or stop it early. The path is a readable fallback, not an error in the file.

Is my sitemap or site stored?

No. An uploaded XML file is parsed in your browser. A sitemap URL and the page heads are fetched by our Worker, which returns them to your browser and keeps nothing, with 60 requests a minute per IP address.

Related tools