Skip to content

feat(agent-ready): add Link headers and llms.txt for AI agent discoverability - #254

Open
Hale730 wants to merge 1 commit into
cloudflare:mainfrom
Hale730:hporter/agent-ready
Open

Hale730 wants to merge 1 commit into
cloudflare:mainfrom
Hale730:hporter/agent-ready

Conversation

@Hale730

@Hale730 Hale730 commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Two small additions to improve our score on isitagentready.com:

  • public/_headers — Add RFC 8288 Link: response headers to the homepage pointing to /sitemap-index.xml (already auto-generated by @astrojs/sitemap) and the new /llms.txt. Lets AI agents discover key resources from HTTP headers without parsing HTML.
  • public/llms.txt — LLM-friendly site index following the llmstxt.org convention. Summarises the site and links to the five focus areas plus About / Discovery sections, so AI agents can orient themselves without crawling the full HTML site.

What I deliberately skipped

The scanner flagged several other checks that don't apply to a static content site:

  • DNS-AID (DNS records — needs zone admin)
  • API catalog (we don't have public APIs)
  • OAuth discovery / Protected Resource / auth.md (no auth on this site)
  • MCP Server Card (no MCP server)
  • Agent Skills index (no published skills)
  • WebMCP (overkill for read-only content)

Markdown content negotiation would also help (returning text/markdown when Accept: text/markdown is set), but it's a much bigger lift (generate markdown variants of every page + Workers rewrite rule). Happy to do that in a follow-up if reviewers want it.

…rability

Two small additions to improve the site's agent-readiness score
(isitagentready.com):

- public/_headers: add RFC 8288 Link response headers on the homepage
  pointing to /sitemap-index.xml (already generated by @astrojs/sitemap)
  and the new /llms.txt.
- public/llms.txt: LLM-friendly site index following the llms.txt
  convention (llmstxt.org). Summarises the site and links to the five
  focus areas plus About / Discovery sections, so AI agents can orient
  themselves without crawling the full HTML.

Skipped checks from the scanner that don't apply to a static content
site (DNS-AID, API catalog, OAuth discovery, MCP server card, agent
skills index, WebMCP) — these are for sites with APIs, auth, or MCP
servers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant