Playbooks Introduction

Last updated 16 minutes ago

Create and customize playbooks for your projects.

Table of Contents

Getting Started
KIO Prompt Flow
Tips for Writing Good Playbooks
Adding or Modifying Playbooks
- Via GitHub Repository
- Via CoreMedia Content Repository
Using Tools
Debugging Playbooks
- Enabling Debug Output
Actuator Endpoint

This guide explains how to develop custom playbooks for KIO Co-Pilot. It is intended for prompt engineers who want to extend KIO’s default capabilities by writing custom playbooks or modifying prefabricated playbooks from CoreMedia.

CoreMedia provides a set of initial playbooks and general guidelines for prompt engineering. Our Customer Success team is available to guide customers and help them navigate prompt-related tasks. Additionally, we support the creation and tailoring of playbooks to fit their unique setups and needs. CoreMedia is not responsible for the entire lifecycle of playbooks — for example, updating them based on model changes or new use cases.

Getting Started

Playbooks are a way to teach KIO how to perform complex tasks. They are written in natural language and can be easily extended or modified. KIO loads playbooks at runtime, so you can add new playbooks, or modify existing ones, without having to restart and/or redeploy KIO. You can think of Playbooks as a no-code scripting tool that allows you to guide KIO through a complex workflow.

In general, if you find yourself writing more prompt text than you’d like, or if KIO often misunderstands you, unless you’re very specific, that’s not ideal. But it’s a good sign that you might want to create a Playbook for that task.

A playbook’s structure is rather simple: It has an id, a name, a description, and a steps section. The name is just for your information, the description is what KIO will look at when choosing one or more appropriate playbooks given the user’s input, and the steps section is the detailed explanation of what you want KIO to do.

Look at one of CoreMedias default playbooks (optimize-seo.md):

id: @coremedia/optimize-seo

name: Optimize Metadata for SEO

description: Use this when the user asks to generate and/or optimize SEO (metadata)

steps:
* Check if the content item you want to optimize inherits from `CMTeasable` (typical examples are `CMArticle`, `CMChannel`, and others).
  If it doesn't, then tell the user the type of the actual content item, and that it is not sensible to generate SEO metadata for it.
* Otherwise, do the following:
    * For property `htmlTitle`, create plain text that can be used as "HTML title" in a web page with a maximum length of 100 characters.
    * For property `htmlDescription`, create plain text that can be used as "HTML meta description" in a web page with a maximum length of 400 characters.
    * For property `keywords`, create a comma-separated list of lower-case keywords with maximum length of 900 characters.
    * At the end update the content.

This is an example of a simple playbook that extracts good metadata like htmlTitle and keywords by looking at a content item’s "payload", and applies them to the same content item.

When KIO has this playbook available, you can be rather short in your prompt:

> optimize SEO for this
< To optimize the SEO for the content, I'll first check if it inherits from `CMTeasable`. If it does, I'll proceed to generate the HTML title, meta description, and keywords. Let's start by examining the content to determine its type.

< The content item [Annual Report on fiscal 2020 with press conference and investor and analyst conference call - 7128](http://studio/content/7128) is of type `CMArticle`, which inherits from `CMTeasable`. I will proceed to optimize the SEO metadata for this content.

### SEO Metadata Optimization
- **HTML Title**: 'Annual Report 2019: Financial Highlights & Insights'
- **HTML Description**: 'Explore the Annual Report on the fiscal year 2019, featuring insights from our press conference and calls with investors and analysts.'
- **Keywords**: 'annual report, fiscal 2019, financial achievements, press conference, investor insights, analyst conference, company report, financial overview'

I will now update these properties in the system.

< The SEO optimization for the content [Annual Report on the Fiscal Year 2019 - 7128](http://studio/content/7128) has been successfully completed. Here are the updates:

- **HTML Title**: Annual Report 2019: Financial Highlights & Insights
- **HTML Description**: Explore the Annual Report on the fiscal year 2019, featuring insights from our press conference and calls with investors and analysts.
- **Keywords**: annual report, fiscal 2019, financial achievements, press conference, investor insights, analyst conference, company report, financial overview

If you need further adjustments or have any other requests, feel free to let me know!

As you can see, KIO is internally prompted to explain the steps that it plans, and its internal reasoning, in rather great detail to the user. This seems sensible to ensure that the user understands what KIO is doing, and why it is doing it.

KIO will examine the description of the playbook to pick the correct playbook(s). Therefore, it’s good practice to explain in reasonable detail what the playbook does, and how it relates to a certain user’s intent. It has been proven to work well to also give a few concrete examples of what the user might ask. This is like a prompting best practice called few shot prompting. Here’s an example of a good playbook description:

Use this when the user asks to assign taxonomy tags to a content item. Typical user prompts are "assign tags", "tag this content", or similar.

KIO Prompt Flow

KIO employs a technique called prompt chaining that enhances the prompt in multiple steps before it presents the final prompt to the agent. In particular, when KIO is asked a question, it will first try to understand the user’s intent. Then it checks if there’s at least one playbook available that has a description that matches the user’s intent, and then injects it into the prompt. This diagram depicts the simplified prompt flow:

Tips for Writing Good Playbooks

Be prepared for some trial and error

First and foremost: Don’t be afraid to experiment! Prompt engineering (and writing playbooks is exactly that) is a bit of a trial-and-error process. You might have to write a few playbooks before you get the hang of it. Don’t give up too early! Usually, you will get KIO to do what you want it to do, but it might take a few iterations.
Be specific, but not too specific

The more specific you are in your playbook, the less likely KIO is to misunderstand you. For example, if you want KIO to only ever update the htmlTitle property, say so explicitly. However, you can also rely on KIO to infer the correct things from the context and by making use of its toolkit: For example, you can also be a bit more vague and write "update all properties that hold SEO metadata. Look at the property names to infer what they are".
Give examples

One shot prompting or few shot prompting have proven to be good techniques to improve KIO’s accuracy in interacting with the content repository. Therefore, it’s a good idea to give one (or a few) example(s), especially for complex tasks. For example, when you want KIO to update a complex property such as Markup or Struct, it has proven to be efficient to just give KIO an example XML snippet that it can use as a template. For example, the playbook snippet below has shown to work well:

(...)
* Modify the property `variants` of this new content item, like this:
* Consider this example xml:

  <Struct xmlns='http://www.coremedia.com/2008/struct' xmlns:xlink='http://www.w3.org/1999/xlink'>
      <StructProperty Name='mappings'>
          <Struct>
              <StructProperty Name='variant-7c8a525e-3649-4fb9-8534-31cc3c5da5ba'>
                  <Struct>
                      <StringProperty Name='segmentId' Length='255'>cmec:///p13n/segment/78206;cmecType=segment</StringProperty>
                      <LinkProperty Name='target' LinkType='coremedia:///cap/contenttype/CMTeasable' xlink:href='coremedia:///cap/content/7302'/>
                  </Struct>
              </StructProperty>
              <StructProperty Name='variant-a3b7371c-6300-48a7-9c33-13933f6a407b'>
                  <Struct>
                      <StringProperty Name='segmentId' Length='255'>cmec:///p13n/segment/78204;cmecType=segment</StringProperty>
                      <LinkProperty Name='target' LinkType='coremedia:///cap/contenttype/CMTeasable' xlink:href='coremedia:///cap/content/6522'/>
                  </Struct>
              </StructProperty>
          </Struct>
      </StructProperty>
      <BooleanProperty Name='kioModified'>true</BooleanProperty>
  </Struct>

* Contruct an XML string exactly like this (including the whitespace), except:
  * In the LinkProperty attribute xlink:href, replace the value with the link to the content item you created in the previous step. Hint: Change the ID to the ID of the content items you copied
  * In the StringProperty body, replace the cmec: link with the segment ID you found in the previous steps for the respective segment. Always use the numerical segment ID, not the name.

Be very explicit when explaining paths and folders

Humans often have a hard time parsing complex folder structures, and surprisingly, it turns out that KIO sometimes gets confused with that, too. Therefore, it has shown to be good practice to be super-precise when explaining paths and folders. The example playbook snippet below has proven to work well:

Look at the repository path closely. It is always structured like this: `/Sites/<Site Name>/<Country Name>/<Language>/<Folder1>/<Folder2>/.../<Content Item Name>`. Now, create a copy of the source content. You must place the copy in the same folder as the original content.  For example, when the path of the original content item is `/Sites/MySite/United States/English/Editorial/Homepage/Blog/My Blog Post`, and you are asked to translate to German, the target path would be `/Sites/MySite/United States/English/Editorial/Homepage/Blog/My Blog Post - DE`.

Stay up to date on prompt engineering best practices

Prompt engineering is a relatively new field, and best practices are still evolving. It’s a good idea to stay up to date on the latest research and best practices. Good starting points are Prompting Guide, or, in the case of OpenAI, the OpenAI Prompt Engineering guidelines.

Adding or Modifying Playbooks

The KIO Backend loads playbooks from different sources:

GitHub Repository
CoreMedia Content Repository

Via GitHub Repository

CoreMedia provides a GitHub repository with default playbooks. You can use these playbooks as a starting point for your own playbooks. To get access to the repository, contact the CoreMedia support team. Please refer to the Installation documentation for more information on how to connect your KIO Backend to the GitHub repository. Content changes are updated automatically every 30 seconds. You do not have to restart the KIO Backend to load the new playbooks.

The configuration properties for the GitHub repository are:

# To configure this, you need to create a personal access token on GitHub (https://github.com/settings/tokens) and grant it access to the repository ("repo" scope).
kio.backend.playbooks.github.token=CONFIGURE_ME
kio.backend.playbooks.github.repository=coremedia-contributions/kio-playground
kio.backend.playbooks.github.branch=main
kio.backend.playbooks.github.path=copilot/playbooks
kio.backend.playbooks.github.refresh-interval=30s

Note: You also can configure your clone of coremedia-contributions/kio-playground or configure your own GitHub repository with your own playbooks.

Via CoreMedia Content Repository

You can also store playbooks in the CoreMedia Content Repository. Just create a Settings document below the path /Settings/Options/Settings/Playbooks via Studio and fill out all required fields. The KIO Backend will automatically load the playbooks from the repository. You do not have to restart the KIO Backend to load the new playbooks.

The configuration properties for the CoreMedia Content Repository are:

kio.backend.playbooks.cm.path=/Settings/Options/Settings/Playbooks

The playbooks stored in the CoreMedia Content Repository have precedence over the playbooks stored in the GitHub repository. This means you can refine or overwrite the default playbooks from CoreMedia by storing your own playbooks in the CoreMedia Content Repository. You need to make sure to use the same id as the default playbook you want to overwrite.

Using Tools

Playbooks use tools to perform specific tasks, such as KIO interacting with the CoreMedia Content Repository for actions like reading, updating, or searching for content items.

KIO automatically detects available tools and uses them as needed in playbook steps. As a playbook developer, understanding and effectively using these tools is essential.

New tools may be added in future KIO releases, so regularly check for updates.

To view available tools, you can simply ask KIO:

> List all tools that are available.

< I have the following tools available:
    1. Query Service Tool: Query content items using CoreMedia Query Language (CQL).
    2. Repository Metadata Tool: List all content types and their properties.
    3. Update Content Tool: Update properties of existing content items.
...

For details on a specific tool, you can also ask KIO:

> Show me examples on how to use the Update Content Tool.

< Here's an example of how you might use the Update Content Tool:

Task: Update the title and detail text of a content item.
Example Prompt:
Update the content item with ID 12345. Change the title to 'New Exciting Title' and update the detail text to 'This is the updated content detail text.'

KIO can combine multiple tools internally. Related to the example above, KIO might use the Query Service Tool to find the content item and resolve the content ID first, and then use the Update Content Tool to apply the changes. With that in mind, you can also update content by ID, path, or just by referencing the current article or similar.

For a better understanding of how KIO uses tools in playbooks, you can enable debug output as described in the next section.

Debugging Playbooks

When you write a playbook, you might want to test and debug them. For more insights which playbook and which tools are used by KIO you can enable the debug mode in KIO. In this mode, KIO will only show the debug output of the selected playbook and tool calls to you.

Enabling Debug Output

For developing playbooks and tools, enable debug output to see called tools with input and output parameters displayed as generic JSON.

Below is an example for UpdateContentTool:

Configure debug output by editing the EditorPreferences directly in the user’s home folder. Add the following Boolean to a top-level kio Struct. If not set, the default value is false.

Actuator Endpoint

The KIO Backend provides an actuator endpoint /actuator/kio to get information about the number of playbooks loaded and the number of updated playbooks.

Is this page useful?