docs: add context engine documentation
Add dedicated docs page for the pluggable context engine system: - Full lifecycle explanation (ingest, assemble, compact, afterTurn) - Legacy engine behavior documentation - Plugin engine authoring guide with code examples - ContextEngine interface reference table - ownsCompaction semantics - Subagent lifecycle hooks (prepareSubagentSpawn, onSubagentEnded) - systemPromptAddition mechanism - Relationship to compaction, memory plugins, and session pruning - Configuration reference and tips Also: - Add context-engine to docs nav (Agents > Fundamentals, after Context) - Add /context-engine redirect - Cross-link from context.md and compaction.md
This commit is contained in:
parent
6f795fd60e
commit
228448e6b3
@ -97,6 +97,17 @@ compaction and can run alongside it.
|
||||
|
||||
See [OpenAI provider](/providers/openai) for model params and overrides.
|
||||
|
||||
## Custom context engines
|
||||
|
||||
Compaction behavior is owned by the active
|
||||
[context engine](/concepts/context-engine). The legacy engine uses the built-in
|
||||
summarization described above. Plugin engines (selected via
|
||||
`plugins.slots.contextEngine`) can implement any compaction strategy — DAG
|
||||
summaries, vector retrieval, incremental condensation, etc.
|
||||
|
||||
When a plugin engine sets `ownsCompaction: true`, OpenClaw delegates all
|
||||
compaction decisions to the engine and does not run built-in auto-compaction.
|
||||
|
||||
## Tips
|
||||
|
||||
- Use `/compact` when sessions feel stale or context is bloated.
|
||||
|
||||
215
docs/concepts/context-engine.md
Normal file
215
docs/concepts/context-engine.md
Normal file
@ -0,0 +1,215 @@
|
||||
---
|
||||
summary: "Context engine: pluggable context assembly, compaction, and subagent lifecycle"
|
||||
read_when:
|
||||
- You want to understand how OpenClaw assembles model context
|
||||
- You are switching between the legacy engine and a plugin engine
|
||||
- You are building a context engine plugin
|
||||
title: "Context Engine"
|
||||
---
|
||||
|
||||
# Context Engine
|
||||
|
||||
A **context engine** controls how OpenClaw builds model context for each run.
|
||||
It decides which messages to include, how to summarize older history, and how
|
||||
to manage context across subagent boundaries.
|
||||
|
||||
OpenClaw ships with a built-in `legacy` engine. Plugins can register
|
||||
alternative engines that replace the entire context pipeline.
|
||||
|
||||
## Quick start
|
||||
|
||||
Check which engine is active:
|
||||
|
||||
```bash
|
||||
openclaw doctor
|
||||
# or inspect config directly:
|
||||
cat ~/.openclaw/openclaw.json | jq '.plugins.slots.contextEngine'
|
||||
```
|
||||
|
||||
Switch engines:
|
||||
|
||||
```json5
|
||||
// openclaw.json
|
||||
{
|
||||
plugins: {
|
||||
slots: {
|
||||
contextEngine: "lossless-claw", // or "legacy" (default)
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
Restart the gateway after changing the slot.
|
||||
|
||||
## How it works
|
||||
|
||||
Every time OpenClaw runs a model prompt, the context engine participates at
|
||||
four lifecycle points:
|
||||
|
||||
1. **Ingest** — called when a new message is added to the session. The engine
|
||||
can store or index the message in its own data store.
|
||||
2. **Assemble** — called before each model run. The engine returns an ordered
|
||||
set of messages (and an optional `systemPromptAddition`) that fit within
|
||||
the token budget.
|
||||
3. **Compact** — called when the context window is full, or when the user runs
|
||||
`/compact`. The engine summarizes older history to free space.
|
||||
4. **After turn** — called after a run completes. The engine can persist state,
|
||||
trigger background compaction, or update indexes.
|
||||
|
||||
### Subagent lifecycle (optional)
|
||||
|
||||
Engines can also manage context across subagent boundaries:
|
||||
|
||||
- **prepareSubagentSpawn** — set up shared state before a child session starts.
|
||||
Returns a rollback handle in case the spawn fails.
|
||||
- **onSubagentEnded** — clean up when a subagent session completes or is swept.
|
||||
|
||||
### System prompt addition
|
||||
|
||||
The `assemble` method can return a `systemPromptAddition` string. OpenClaw
|
||||
prepends this to the system prompt for the run. This lets engines inject
|
||||
dynamic recall guidance, retrieval instructions, or context-aware hints
|
||||
without requiring static workspace files.
|
||||
|
||||
## The legacy engine
|
||||
|
||||
The built-in `legacy` engine preserves OpenClaw's original behavior:
|
||||
|
||||
- **Ingest**: no-op (the session manager handles message persistence directly).
|
||||
- **Assemble**: pass-through (the existing sanitize → validate → limit pipeline
|
||||
in the runtime handles context assembly).
|
||||
- **Compact**: delegates to the built-in summarization compaction, which creates
|
||||
a single summary of older messages and keeps recent messages intact.
|
||||
- **After turn**: no-op.
|
||||
|
||||
The legacy engine does not register tools or provide a `systemPromptAddition`.
|
||||
|
||||
When no `plugins.slots.contextEngine` is set (or it's set to `"legacy"`), this
|
||||
engine is used automatically.
|
||||
|
||||
## Plugin engines
|
||||
|
||||
A plugin can register a context engine using the plugin API:
|
||||
|
||||
```ts
|
||||
export default function register(api) {
|
||||
api.registerContextEngine("my-engine", () => ({
|
||||
info: {
|
||||
id: "my-engine",
|
||||
name: "My Context Engine",
|
||||
ownsCompaction: true,
|
||||
},
|
||||
|
||||
async ingest({ sessionId, message, isHeartbeat }) {
|
||||
// Store the message in your data store
|
||||
return { ingested: true };
|
||||
},
|
||||
|
||||
async assemble({ sessionId, messages, tokenBudget }) {
|
||||
// Return messages that fit the budget
|
||||
return {
|
||||
messages: buildContext(messages, tokenBudget),
|
||||
estimatedTokens: countTokens(messages),
|
||||
systemPromptAddition: "Use lcm_grep to search history...",
|
||||
};
|
||||
},
|
||||
|
||||
async compact({ sessionId, force }) {
|
||||
// Summarize older context
|
||||
return { ok: true, compacted: true };
|
||||
},
|
||||
}));
|
||||
}
|
||||
```
|
||||
|
||||
Then enable it in config:
|
||||
|
||||
```json5
|
||||
{
|
||||
plugins: {
|
||||
slots: {
|
||||
contextEngine: "my-engine",
|
||||
},
|
||||
entries: {
|
||||
"my-engine": {
|
||||
enabled: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
### The ContextEngine interface
|
||||
|
||||
Required methods:
|
||||
|
||||
| Method | Purpose |
|
||||
| ------------------ | -------------------------------------------------------- |
|
||||
| `info` | Engine id, name, version, and whether it owns compaction |
|
||||
| `ingest(params)` | Store a single message |
|
||||
| `assemble(params)` | Build context for a model run |
|
||||
| `compact(params)` | Summarize/reduce context |
|
||||
|
||||
Optional methods:
|
||||
|
||||
| Method | Purpose |
|
||||
| ------------------------------ | ----------------------------------------- |
|
||||
| `bootstrap(params)` | Initialize engine state for a new session |
|
||||
| `ingestBatch(params)` | Ingest a completed turn as a batch |
|
||||
| `afterTurn(params)` | Post-run lifecycle work |
|
||||
| `prepareSubagentSpawn(params)` | Set up shared state for a child session |
|
||||
| `onSubagentEnded(params)` | Clean up after a subagent ends |
|
||||
| `dispose()` | Release resources |
|
||||
|
||||
### ownsCompaction
|
||||
|
||||
When `info.ownsCompaction` is `true`, the engine manages its own compaction
|
||||
lifecycle. OpenClaw will not trigger the built-in auto-compaction; instead it
|
||||
delegates entirely to the engine's `compact()` method. The engine may also
|
||||
run compaction proactively in `afterTurn()`.
|
||||
|
||||
When `false` or unset, OpenClaw's built-in auto-compaction logic runs
|
||||
alongside the engine.
|
||||
|
||||
## Configuration reference
|
||||
|
||||
```json5
|
||||
{
|
||||
plugins: {
|
||||
slots: {
|
||||
// Select the active context engine. Default: "legacy".
|
||||
// Set to a plugin id to use a plugin engine.
|
||||
contextEngine: "legacy",
|
||||
},
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
The slot is exclusive — only one context engine can be active at a time. If
|
||||
multiple plugins declare `kind: "context-engine"`, only the one selected in
|
||||
`plugins.slots.contextEngine` loads. Others are disabled with diagnostics.
|
||||
|
||||
## Relationship to compaction and memory
|
||||
|
||||
- **Compaction** is one responsibility of the context engine. The legacy engine
|
||||
delegates to OpenClaw's built-in summarization. Plugin engines can implement
|
||||
any compaction strategy (DAG summaries, vector retrieval, etc.).
|
||||
- **Memory plugins** (`plugins.slots.memory`) are separate from context engines.
|
||||
Memory plugins provide search/retrieval; context engines control what the
|
||||
model sees. They can work together — a context engine might use memory
|
||||
plugin data during assembly.
|
||||
- **Session pruning** (trimming old tool results in-memory) still runs
|
||||
regardless of which context engine is active.
|
||||
|
||||
## Tips
|
||||
|
||||
- Use `openclaw doctor` to verify your engine is loading correctly.
|
||||
- If switching engines, existing sessions continue with their current history.
|
||||
The new engine takes over for future runs.
|
||||
- Engine errors are logged and surfaced in diagnostics. If a plugin engine
|
||||
fails to load, OpenClaw falls back to the legacy engine with a warning.
|
||||
- For development, use `openclaw plugins install -l ./my-engine` to link a
|
||||
local plugin directory without copying.
|
||||
|
||||
See also: [Compaction](/concepts/compaction), [Context](/concepts/context),
|
||||
[Plugins](/tools/plugin), [Plugin manifest](/plugins/manifest).
|
||||
@ -157,7 +157,8 @@ By default, OpenClaw uses the built-in `legacy` context engine for assembly and
|
||||
compaction. If you install a plugin that provides `kind: "context-engine"` and
|
||||
select it with `plugins.slots.contextEngine`, OpenClaw delegates context
|
||||
assembly, `/compact`, and related subagent context lifecycle hooks to that
|
||||
engine instead.
|
||||
engine instead. See [Context Engine](/concepts/context-engine) for the full
|
||||
pluggable interface, lifecycle hooks, and configuration.
|
||||
|
||||
## What `/context` actually reports
|
||||
|
||||
|
||||
@ -59,6 +59,10 @@
|
||||
"source": "/compaction",
|
||||
"destination": "/concepts/compaction"
|
||||
},
|
||||
{
|
||||
"source": "/context-engine",
|
||||
"destination": "/concepts/context-engine"
|
||||
},
|
||||
{
|
||||
"source": "/cron",
|
||||
"destination": "/cron-jobs"
|
||||
@ -952,6 +956,7 @@
|
||||
"concepts/agent-loop",
|
||||
"concepts/system-prompt",
|
||||
"concepts/context",
|
||||
"concepts/context-engine",
|
||||
"concepts/agent-workspace",
|
||||
"concepts/oauth"
|
||||
]
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user