Requirements

Large language models (LLMs) should be provided with concise, curated guidance about a site's content, so that LLM-based tools (chat assistants, coding agents, AI-powered search) can find and use the site's most relevant information more reliably than by crawling and parsing the full HTML.

This is supported by serving an llms.txt file as explained here: https://llmstxt.org/.

For example, the CoreMedia website is accessible as https://www.coremedia.com. LLM-based tools look for a file called llms.txt by performing an HTTP GET request to https://www.coremedia.com/llms.txt.

An llms.txt file may look like this:

        # CoreMedia

        > CoreMedia is a composable Digital Experience Platform (DXP)
        > designed specifically for enterprise scalability.

        ## Content Management System (CMS)

        CoreMedia Content Cloud is the foundation for creating and
        managing digital experiences.

        - [Content Management Overview](https://www.coremedia.com/content-management-system):
          The central hub for orchestrating omnichannel content.

        ## Developer & Technical Resources

        - [CoreMedia Documentation](https://documentation.coremedia.com/):
          The official technical hub.
      

Solution

Editors provide the content as a CMDownload document named llms.txt, placed as a direct child of the site's root folder, next to the site indicator document. Its data property holds the actual payload, either as text/plain or text/markdown. If no such document exists, or its data blob has a wrong mime type, no llms.txt is served for that site (the CAE responds with 404 Not Found) — providing an llms.txt is entirely optional.

Blueprint's cae-base-lib module provides a LlmsHandler which is responsible for serving an llms.txt file. A LlmsHandler instance is configured in blueprint-handlers.xml. It handles URLs like http://<cae-host>:<cae-port>/blueprint/servlet/service/llms/<site-segment>

This is the internal URL of the CAE's servlet container, not the public URL. In order to have the correct external URL for the LLMs file one needs to configure the reverse proxy in front of the CAE to forward incoming GET requests for https://<site-host>/llms.txt to http://<cae-host>:<cae-port>/blueprint/servlet/service/llms/<site-segment>. The Docker Compose based environments shipped with this repository use Traefik for this, see the dynamic configuration under global/deployment/docker/blueprint/traefik/dynamic-configuration.

The LlmsHandler is responsible for requests like this due to the path element /llms. The last path element of this URL (<site-segment>) is evaluated by LlmsHandler to resolve the site whose llms.txt has been requested.