Multi-Agent Architecture and Parallel Execution
Understand how KIO's orchestrator delegates work to specialized agents, and how parallel execution work.
KIO Co-Pilot is not a single monolithic AI model. Under the hood, the KIO backend uses a multi-agent architecture: an orchestrator that talks to you and a set of specialized agents that it delegates focused subtasks to. For requests that touch multiple content items (for example, "optimize SEO for the selected articles"), the orchestrator can also split the work into chunks and run several agents in parallel to speed things up. Or even for operations on the same content item, as long as they do not interfere with each other they can be parallelized (e.g. "read_only" operations).
This section explains how this works and which configuration options are available to administrators to tune response time and AI resource (token) consumption for the parallel execution of requests.
Orchestrator and Agents
Every conversation you have with KIO is handled by the orchestrator. The orchestrator is the only component that replies to you directly. It interprets your request and, whenever a part of the task requires specialized handling, it delegates that part to an agent.
An agent is a focused, specialized worker with its own instructions and its own set of Tools. Agents do their work and report their result back to the orchestrator — On the user interface, you see only the agents' status updates and, ultimately, the orchestrator’s summarized response.
|
This split keeps each agent’s task narrow and reliable, and keeps the conversation you see coherent, since only the orchestrator ever talks to you. |
Splitting a request across specialized agents like this means a single user prompt can result in several internal AI calls, each with its own orchestration overhead (its own instructions, its own back-and-forth with the AI model). This is a deliberate design trade-off: it improves reliability and maintainability of individual tasks, and keeps each agent’s own context window smaller and more focused, since it only needs to reason about its own subtask rather than the whole conversation. Depending on the request, this can either increase or reduce the total amount of AI resources (tokens) consumed compared to a single large prompt — the smaller, focused context of each agent can offset the added orchestration overhead, but does not always do so. Administrators should monitor actual AI resource consumption for their own workloads rather than assuming a fixed direction of the effect.
Bulk and Parallel Execution
Some requests naturally apply the same operation to multiple, independent content items — for example, "add keywords to the selected articles" or "optimize SEO for the currently selected items" (see Studio Context for how KIO recognizes a multi-item selection).
For these bulk requests, the orchestrator does not process every item one after another. Instead, it:
-
Splits the list of content items into balanced chunks.
-
Starts one agent per chunk, running these agents in parallel.
-
Waits for all agents to finish and collects their results.
-
Reports a combined summary of what changed back to you.
|
KIO always processes items independently. If items in your request depend on each other (for example, an operation on item B needs the result of item A first), KIO processes them one after another instead of in parallel. |
Response Time vs. AI Resource Consumption
Running more agents in parallel for a bulk request speeds up how quickly the whole batch completes. This reliably reduces response time. Its effect on total AI resource (token) consumption is less clear-cut, though: every additional parallel agent adds its own orchestration overhead (its own instructions, its own back-and-forth with the AI model), but each agent also works with a smaller, more focused context window (fewer items per chunk) than a single agent processing the whole batch would need. Depending on the request, the reduced context per agent may offset or even outweigh the additional orchestration overhead; however, the parallel use of multiple agents generally results in higher total token consumption.
This is a deliberate, configurable trade-off:
-
Fewer parallel agents → higher response time for large bulk/multi-item requests; AI resource consumption is probably lower, depending on the request. resource consumption is probably higher, depending on the request.
|
Parallelization often goes hand in hand with better cacheability (higher amount of cached tokens). Because parallel agents work with smaller, more uniform context windows, more of their input can be served from the AI provider’s prompt cache. Since cached tokens are significantly cheaper than uncached ones, this can lower the overall cost. |
|
Because the net effect on token consumption depends on the specific request and content, we recommend monitoring actual AI resource consumption for your workloads (for example, via your LLM provider’s usage/cost dashboard) rather than assuming that more parallel agents always cost more. |
|
To get more insights into the actual token consumption of your requests, enable Enable Debug Output in the KIO Settings menu. When enabled, KIO shows debug information below each response, including per-request token usage, which helps you compare the impact of different parallel agent configurations on your workloads. See KIO Settings for details on how to toggle this option. |
Configuring Parallel Agent Execution
Administrators can tune response time and AI resource consumption for bulk/multi-item requests via
the KIO backend configuration. The relevant properties (kio.backend.max-parallel-agents-per-request and
kio.backend.max-bulk-items) are documented in
Configuring Parallel Agent Execution
in the Configuring KIO Co-Pilot guide.