diff --git a/docs/.internal/extension-host-migration/openclaw-capability-catalog-and-arbitration-spec.md b/docs/.internal/extension-host-migration/openclaw-capability-catalog-and-arbitration-spec.md index 14354afd80f..580bdecc686 100644 --- a/docs/.internal/extension-host-migration/openclaw-capability-catalog-and-arbitration-spec.md +++ b/docs/.internal/extension-host-migration/openclaw-capability-catalog-and-arbitration-spec.md @@ -64,6 +64,7 @@ What has been implemented: - context-engine registration and runtime resolution now route through `src/extension-host/context-engine-runtime.ts` ahead of broader catalog-backed ownership - exclusive-slot selection and default-slot resolution now route through `src/extension-host/slot-arbitration.ts` ahead of broader catalog-backed arbitration ownership - ACP backend registration and runtime resolution now route through `src/extension-host/acp-runtime-backend-registry.ts` ahead of broader catalog-backed backend ownership +- embedding-provider auto-selection, provider creation, local-setup guidance, and primary and fallback routing now route through `src/extension-host/embedding-runtime-registry.ts` while `src/memory/embeddings.ts` remains the compatibility facade - media-provider normalization, built-in registry construction, override merging, and runtime lookup now route through `src/extension-host/media-runtime-registry.ts` while `src/media-understanding/providers/index.ts` remains the compatibility facade - TTS provider metadata, provider ordering, API-key resolution, configuration checks, and telephony support now route through `src/extension-host/tts-runtime-registry.ts` while `src/tts/tts.ts` remains the synthesis execution owner - legacy internal-hook bridging and typed prompt-injection compatibility policy now route through `src/extension-host/hook-compat.ts` ahead of broader catalog-backed registry ownership @@ -617,7 +618,7 @@ Capability selection must emit structured events for: - channel capabilities from `extensions/discord/src/channel.ts:74`, `extensions/slack/src/channel.ts:107`, and `extensions/telegram/src/channel.ts:120` collapse into canonical messaging action families - diffs becomes an agent-visible tool family plus a host-managed route surface from `extensions/diffs/index.ts:27` - provider integration from `extensions/google-gemini-cli-auth/index.ts:24` becomes operator-visible setup and auth capabilities -- embedding provider overrides, plus broader media- and TTS-runtime execution and fallback ownership, should become runtime-internal subsystem registries rather than remaining part of a universal plugin-provider API +- broader embedding-, media-, and TTS-runtime execution and fallback ownership should become runtime-internal subsystem registries rather than remaining part of a universal plugin-provider API - extension-backed web search should become an agent-visible tool family unless it is only a runtime-internal backend feeding another host-owned surface - voice-call from `extensions/voice-call/index.ts:230` becomes a mix of agent-visible actions, runtime providers, and operator surfaces - ACP backend registration from `extensions/acpx/src/service.ts:55` becomes runtime-internal backend arbitration @@ -634,7 +635,7 @@ Capability selection must emit structured events for: 6. Migrate the existing provider auth and setup selection path onto host-owned setup catalogs and canonical provider metadata. 7. Add provider selection logic for the broader messaging action family before migrating all channels. 8. Add runtime-backend and context-engine arbitration using the same rank and slot model where appropriate. -9. Finish the media- and TTS-runtime migrations and add a host-owned embedding subsystem registry with explicit capability routing and built-in fallback policy. +9. Finish broader embedding-, media-, and TTS-runtime migrations with explicit capability routing and built-in fallback policy. 10. Decide whether extension-backed search needs only canonical tool publication or also a host-owned runtime registry for internal search backends, and keep those two cases distinct. 11. Ensure lightweight setup catalogs can be built from static descriptors alone. 12. Add a reviewed core registry for canonical action families and document how new ids are introduced. diff --git a/docs/.internal/extension-host-migration/openclaw-extension-contribution-schema-spec.md b/docs/.internal/extension-host-migration/openclaw-extension-contribution-schema-spec.md index 55ca7d2cbf0..9f8414eebac 100644 --- a/docs/.internal/extension-host-migration/openclaw-extension-contribution-schema-spec.md +++ b/docs/.internal/extension-host-migration/openclaw-extension-contribution-schema-spec.md @@ -43,6 +43,7 @@ What has been implemented: - context-engine registration and runtime resolution now route through `src/extension-host/context-engine-runtime.ts` while `src/context-engine/registry.ts` remains the compatibility facade - exclusive-slot selection and default-slot resolution now route through `src/extension-host/slot-arbitration.ts` while `src/plugins/slots.ts` remains the compatibility facade - ACP backend registration and runtime resolution now route through `src/extension-host/acp-runtime-backend-registry.ts` while `src/acp/runtime/registry.ts` remains the compatibility facade +- embedding-provider auto-selection, provider creation, local-setup guidance, and primary and fallback routing now route through `src/extension-host/embedding-runtime-registry.ts` while `src/memory/embeddings.ts` remains the compatibility facade - media-provider normalization, built-in registry construction, override merging, and runtime lookup now route through `src/extension-host/media-runtime-registry.ts` while `src/media-understanding/providers/index.ts` remains the compatibility facade - TTS provider metadata, provider ordering, API-key resolution, configuration checks, and telephony support now route through `src/extension-host/tts-runtime-registry.ts` while `src/tts/tts.ts` remains the synthesis execution owner - legacy internal-hook bridging and typed prompt-injection compatibility policy now route through `src/extension-host/hook-compat.ts` diff --git a/docs/.internal/extension-host-migration/openclaw-extension-host-implementation-guide.md b/docs/.internal/extension-host-migration/openclaw-extension-host-implementation-guide.md index a525087310e..f3231f87abe 100644 --- a/docs/.internal/extension-host-migration/openclaw-extension-host-implementation-guide.md +++ b/docs/.internal/extension-host-migration/openclaw-extension-host-implementation-guide.md @@ -102,6 +102,7 @@ What has been implemented so far: - plugin provider projection from registry entries into runtime provider objects now routes through `src/extension-host/provider-runtime.ts` - plugin provider discovery filtering, order grouping, and result normalization now route through `src/extension-host/provider-discovery.ts` - provider matching, auth-method selection, config-patch merging, and default-model application now route through `src/extension-host/provider-auth.ts` +- embedding-provider auto-selection, provider creation, local-setup guidance, and primary and fallback routing now route through `src/extension-host/embedding-runtime-registry.ts` while `src/memory/embeddings.ts` remains the compatibility facade - media-provider normalization, built-in registry construction, override merging, and runtime lookup now route through `src/extension-host/media-runtime-registry.ts` while `src/media-understanding/providers/index.ts` remains the compatibility facade - TTS provider metadata, provider ordering, API-key resolution, configuration checks, and telephony support now route through `src/extension-host/tts-runtime-registry.ts` while `src/tts/tts.ts` remains the synthesis execution owner - provider onboarding option building, model-picker entry building, and provider-method choice resolution now route through `src/extension-host/provider-wizard.ts` @@ -241,6 +242,7 @@ Committed implementation slices so far: - `17b7f6a3e4` `Context: extract extension host engine runtime` - `871086537b` `Plugins: extract slot arbitration` - `b7868d06ba` `ACP: extract runtime backend registry` +- `df0cb8193c` `Memory: extract embedding runtime registry` - `e592f60fa7` `Media: extract runtime provider registry` - `36711383f6` `TTS: extract runtime registry` - `89414ed857` `Docs: track extension host migration internally` @@ -252,7 +254,7 @@ What is still missing for these phases: - broader lifecycle ownership beyond the loader state machine, service-lifecycle boundary, CLI-lifecycle boundary, session-owned activation state, and explicit discovery-policy, activation-policy, and finalization-policy outcomes, remaining policy gate ownership, and broad host-owned registries described for Phase 2 - minimal SDK compatibility work beyond preserving current behavior indirectly through existing loading - host-owned conversation binding, interaction routing, ingress claim, and generic interactive control surfaces -- the embedding runtime registry, plus broader media- and TTS-runtime execution and fallback ownership +- broader embedding-, media-, and TTS-runtime execution and fallback ownership - explicit support for extension-backed search, with a generic split between agent-visible tool publication and optional runtime-internal search backends - any pilot migration, event pipeline, canonical catalog, or arbitration implementation @@ -262,7 +264,7 @@ Recent plan refinements: - it now explicitly treats interactive callback routing, namespace ownership, dedupe, and fallback behavior as first-class migration surfaces - it now explicitly treats inbound claim as a canonical ingress-stage concern rather than a permanent plugin-era hook shape - it now explicitly treats Telegram and Discord as the first validated rollout targets for interactive control surfaces while keeping the underlying contracts generic, host-owned, and kernel-agnostic -- it now explicitly treats embeddings as a remaining host-owned subsystem runtime target and media understanding plus TTS as in-progress host-owned subsystem runtimes, all with capability routing, typed request envelopes, provider-id normalization, and fallback policy +- it now explicitly treats embeddings, media understanding, and TTS as in-progress host-owned subsystem runtimes, with embedding selection and fallback now extracted and the remaining execution ownership still pending, all with capability routing, typed request envelopes, provider-id normalization, and fallback policy - it now explicitly rejects widening the legacy `registerProvider(...)` or `ProviderPlugin` surface into a universal runtime API while retaining capability routing, typed request envelopes, provider-id normalization, and fallback behavior where those are part of the target model - it now explicitly treats extension-backed search as either a canonical tool contribution or a host-owned runtime backend depending on whether the search surface is agent-visible diff --git a/docs/.internal/extension-host-migration/openclaw-extension-host-lifecycle-and-security-spec.md b/docs/.internal/extension-host-migration/openclaw-extension-host-lifecycle-and-security-spec.md index 00a99b2a4ca..f3495a9c332 100644 --- a/docs/.internal/extension-host-migration/openclaw-extension-host-lifecycle-and-security-spec.md +++ b/docs/.internal/extension-host-migration/openclaw-extension-host-lifecycle-and-security-spec.md @@ -42,6 +42,7 @@ What has been implemented: - context-engine registration and runtime resolution now delegate through `src/extension-host/context-engine-runtime.ts` while `src/context-engine/registry.ts` remains the compatibility facade - exclusive-slot selection and default-slot resolution now delegate through `src/extension-host/slot-arbitration.ts` while `src/plugins/slots.ts` remains the compatibility facade - ACP backend registration and runtime resolution now delegate through `src/extension-host/acp-runtime-backend-registry.ts` while `src/acp/runtime/registry.ts` remains the compatibility facade +- embedding-provider auto-selection, provider creation, local-setup guidance, and primary and fallback routing now delegate through `src/extension-host/embedding-runtime-registry.ts` while `src/memory/embeddings.ts` remains the compatibility facade - TTS provider metadata, provider ordering, API-key resolution, configuration checks, and telephony support now delegate through `src/extension-host/tts-runtime-registry.ts` while `src/tts/tts.ts` remains the synthesis execution owner - legacy internal-hook bridging and typed prompt-injection compatibility policy now delegate through `src/extension-host/hook-compat.ts` - compatibility `OpenClawPluginApi` composition and logger shaping now delegate through `src/extension-host/plugin-api.ts` @@ -145,7 +146,7 @@ What is still pending from this spec: - broader extension-host lifecycle ownership beyond the loader state machine, service-lifecycle boundary, CLI-lifecycle boundary, session-owned activation state, and explicit discovery-policy, activation-policy, and finalization-policy outcomes - activation pipeline ownership - host-owned registries for setup, CLI, routes, services, slots, and backends -- the embedding runtime registry, plus broader media- and TTS-runtime execution and fallback ownership, including explicit fallback and override policy instead of plugin-era capability reads +- broader embedding-, media-, and TTS-runtime execution and fallback ownership, including explicit fallback and override policy instead of plugin-era capability reads - a clear host-owned split for extension-backed search between agent-visible tool publication and any optional runtime-internal search backend registry - permission-mode enforcement - per-extension state ownership and migration @@ -744,7 +745,7 @@ The host must emit structured telemetry for: 4. Add a policy evaluator that understands advisory versus enforced permission modes. 5. Add host-owned credential and per-extension state boundaries for extension services. 6. Generalize backend registration into a host-managed `capability.runtime-backend` registry. -7. Finish media- and TTS-runtime execution and fallback ownership and add a host-owned embedding runtime registry instead of widening `registerProvider(...)`. +7. Finish broader embedding-, media-, and TTS-runtime execution and fallback ownership instead of widening `registerProvider(...)`. 8. Keep extension-backed search generic by publishing agent-visible search through tool contracts and using runtime-backend only for search backends consumed internally by the host or another subsystem. 9. Add slot-backed provider management for context engines and other exclusive runtime providers. 10. Preserve provenance, origin precedence, and current workspace and bundled enablement rules in host policy. diff --git a/docs/.internal/extension-host-migration/openclaw-kernel-event-pipeline-spec.md b/docs/.internal/extension-host-migration/openclaw-kernel-event-pipeline-spec.md index fb2fd0fb612..51146bea225 100644 --- a/docs/.internal/extension-host-migration/openclaw-kernel-event-pipeline-spec.md +++ b/docs/.internal/extension-host-migration/openclaw-kernel-event-pipeline-spec.md @@ -39,6 +39,7 @@ Relevant prerequisite work that has landed: - context-engine registration and runtime resolution now have a host-owned helper boundary in `src/extension-host/context-engine-runtime.ts` - exclusive-slot selection and default-slot resolution now have a host-owned helper boundary in `src/extension-host/slot-arbitration.ts` - ACP backend registration and runtime resolution now have a host-owned helper boundary in `src/extension-host/acp-runtime-backend-registry.ts` +- embedding-provider auto-selection, provider creation, local-setup guidance, and primary and fallback routing now have a host-owned helper boundary in `src/extension-host/embedding-runtime-registry.ts` - media-provider normalization, built-in registry construction, override merging, and runtime lookup now have a host-owned helper boundary in `src/extension-host/media-runtime-registry.ts` - TTS provider metadata, provider ordering, API-key resolution, configuration checks, and telephony support now have a host-owned helper boundary in `src/extension-host/tts-runtime-registry.ts` - loader cache key construction and registry cache control now have a host-owned helper boundary diff --git a/docs/.internal/extension-host-migration/openclaw-kernel-extension-host-transition-plan.md b/docs/.internal/extension-host-migration/openclaw-kernel-extension-host-transition-plan.md index d923cf1aef4..1fbd7cd007e 100644 --- a/docs/.internal/extension-host-migration/openclaw-kernel-extension-host-transition-plan.md +++ b/docs/.internal/extension-host-migration/openclaw-kernel-extension-host-transition-plan.md @@ -88,6 +88,7 @@ What has landed: - plugin provider projection from registry entries into runtime provider objects now routes through `src/extension-host/provider-runtime.ts` - plugin provider discovery filtering, order grouping, and result normalization now route through `src/extension-host/provider-discovery.ts` - provider matching, auth-method selection, config-patch merging, and default-model application now route through `src/extension-host/provider-auth.ts` +- embedding-provider auto-selection, provider creation, local-setup guidance, and primary and fallback routing now route through `src/extension-host/embedding-runtime-registry.ts` while `src/memory/embeddings.ts` remains the compatibility facade - media-provider normalization, built-in registry construction, override merging, and runtime lookup now route through `src/extension-host/media-runtime-registry.ts` while `src/media-understanding/providers/index.ts` remains the compatibility facade - TTS provider metadata, provider ordering, API-key resolution, configuration checks, and telephony support now route through `src/extension-host/tts-runtime-registry.ts` while `src/tts/tts.ts` remains the synthesis execution owner - provider onboarding option building, model-picker entry building, and provider-method choice resolution now route through `src/extension-host/provider-wizard.ts` @@ -227,6 +228,7 @@ Committed implementation slices so far: - `17b7f6a3e4` `Context: extract extension host engine runtime` - `871086537b` `Plugins: extract slot arbitration` - `b7868d06ba` `ACP: extract runtime backend registry` +- `df0cb8193c` `Memory: extract embedding runtime registry` - `e592f60fa7` `Media: extract runtime provider registry` - `36711383f6` `TTS: extract runtime registry` - `89414ed857` `Docs: track extension host migration internally` @@ -237,7 +239,7 @@ What has not landed: - keeping the cutover inventory current as more surfaces move - broader lifecycle ownership beyond the loader state machine, session-owned activation state, and explicit discovery-policy, activation-policy, and finalization-policy outcomes, plus remaining policy semantics - host-owned registration surfaces beyond the first normalization helpers and low-risk channel, provider, gateway-method, HTTP-route, tool, CLI, service, command, context-engine, and hook compatibility write slices -- the embedding runtime registry, plus broader media- and TTS-runtime execution and fallback ownership +- broader embedding-, media-, and TTS-runtime execution and fallback ownership - SDK compatibility translation work - canonical event stages - canonical capability catalogs diff --git a/src/extension-host/cutover-inventory.md b/src/extension-host/cutover-inventory.md index c7c34c0e1f9..523a97300af 100644 --- a/src/extension-host/cutover-inventory.md +++ b/src/extension-host/cutover-inventory.md @@ -99,7 +99,7 @@ This is an implementation checklist, not a future-design spec. | Interactive channel control verbs for bound agents | product-shaped runtime helpers added under `src/plugins/runtime/*` and direct channel-specific helpers in extension code | host-owned adapter runtime contracts and interaction capabilities | `not started` | The host needs a bounded first-cut set of control verbs for interactive agents, such as typing leases plus message or conversation actions. Those verbs should be expressed as generic host-owned adapter capabilities, even if the first validated rollout only exercises them through Telegram and Discord. | | Slot arbitration | `src/plugins/slots.ts` | `src/extension-host/slot-arbitration.ts` | `partial` | Exclusive-slot selection and default-slot resolution now route through a host-owned slot-arbitration helper while `src/plugins/slots.ts` remains the compatibility facade. Broader slot and catalog arbitration are still pending. | | ACP backend registry | `src/acp/runtime/registry.ts` | `src/extension-host/acp-runtime-backend-registry.ts` | `partial` | ACP backend registration and resolution now route through a host-owned ACP runtime-backend registry while `src/acp/runtime/registry.ts` remains the compatibility facade. Broader runtime-backend catalog ownership and ACPX pilot migration are still pending. | -| Embedding provider registry and fallback routing | `src/memory/embeddings.ts` plus plugin provider capability filtering through `src/plugins/runtime.ts` | host-owned embedding runtime registry for a typed backend family | `not started` | This is a real missing scope area. Embedding providers should be modeled as host-owned subsystem runtimes with explicit capability metadata, request envelopes, provider-id normalization, and fallback rules, not by widening legacy `registerProvider(...)` as the long-term architecture. | +| Embedding provider registry and fallback routing | `src/memory/embeddings.ts` plus plugin provider capability filtering through `src/plugins/runtime.ts` | `src/extension-host/embedding-runtime-registry.ts` | `partial` | Embedding-provider auto-selection, provider creation, local-setup guidance, and primary and fallback routing now route through a host-owned embedding runtime-registry helper while `src/memory/embeddings.ts` remains the compatibility facade. Broader embedding-runtime execution ownership and catalog-backed runtime-family registration are still pending. | | Media-understanding provider registry and fallback routing | `src/media-understanding/providers/index.ts`, `src/media-understanding/runner.ts`, `src/media-understanding/runner.entries.ts`, `src/media-understanding/resolve.ts`, plus plugin provider capability filtering through `src/plugins/runtime.ts` | `src/extension-host/media-runtime-registry.ts` | `partial` | Media-provider normalization, built-in registry construction, override merging, and runtime lookup now route through a host-owned media runtime-registry helper while `src/media-understanding/providers/index.ts` remains the compatibility facade. Broader media-runtime execution and fallback ownership, plus the remaining embedding runtime registry, are still pending. | | TTS provider registry and telephony override routing | `src/tts/tts.ts`, `src/gateway/server-methods/tts.ts`, and `src/auto-reply/reply/commands-tts.ts` | `src/extension-host/tts-runtime-registry.ts` | `partial` | Built-in TTS provider metadata, provider ordering, API-key resolution, configuration checks, and telephony support now route through a host-owned TTS runtime-registry helper while `src/tts/tts.ts` remains the synthesis execution owner. Broader TTS runtime execution and fallback ownership are still pending. | | Onboarding/install/setup surfaces | `src/plugins/install.ts`, package manifests, channel catalog, onboarding commands | host-owned static descriptors | `partial` | Static metadata normalization has started; full setup/install descriptor migration is not done. | @@ -133,7 +133,7 @@ That pattern has been used for: - gateway method-id aggregation, plugin diagnostic shaping, and extra-handler composition - host-owned runtime registry read accessors for provider, tool, service, CLI, gateway-method, and HTTP-route consumers, plus the broader CLI pre-load fast path those accessors enabled - explicit scoping of still-unimplemented migration targets: conversation binding ownership, interactive callback routing, ingress claim semantics, and bounded first-cut interactive channel controls -- explicit scoping of remaining subsystem-runtime targets: the embedding runtime registry, broader media-runtime execution and fallback ownership, and broader TTS-runtime execution and fallback ownership, all with capability routing and fallback +- explicit scoping of remaining subsystem-runtime targets: broader embedding-, media-, and TTS-runtime execution and fallback ownership, all with capability routing and fallback - explicit scoping of extension-backed search as either a canonical tool contribution or an optional host-owned runtime backend, rather than as another universal provider surface ## Immediate Next Targets @@ -160,7 +160,7 @@ The following remain legacy-owned today: - interaction namespace routing, dedupe, and callback fallback rules - canonical ingress claim semantics - generic host-owned interactive channel control contracts -- embedding runtime registry, plus broader media- and TTS-runtime execution and fallback ownership +- broader embedding-, media-, and TTS-runtime execution and fallback ownership - a clear host-owned split for extension-backed search between canonical tool publication and any optional runtime-internal search backend registry - channel runtime compatibility bridges - pilot parity tracking