llms.txt for AI Crawlers — Without a SaaS
Search engines have sitemaps. AI assistants and crawlers are starting to look for llms.txt — a markdown index at your domain root that points to canonical pages with short descriptions. I did not buy a SaaS for that. I generate public/llms.txt from the same route list that feeds the sitemap and prerender scripts.
The SEO article covers titles and structured data for Google. This one covers the parallel index for tools that read llms.txt before they scrape your whole site.
scripts/seo-routes.cjs already lists static pages and every writing article from writing-articles.json. scripts/generate-llms-txt.cjs reads allRoutes, groups them into Main pages, Writing, and Engineering sections, and emits markdown links in the format defined at llmstxt.org.