feat: add Tavily as a bundled web search plugin with search and extract tools (#49200)

Merged via squash.

Prepared head SHA: ece9226e886004f1e0536dd5de3ddc2946fc118c
Co-authored-by: lakshyaag-tavily <266572148+lakshyaag-tavily@users.noreply.github.com>
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Reviewed-by: @gumadeiras
This commit is contained in:
Lakshya Agarwal 2026-03-20 01:06:26 -04:00 committed by GitHub
parent 914fc265c5
commit b36e456b09
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
37 changed files with 1378 additions and 124 deletions

4
.github/labeler.yml vendored
View File

@ -293,6 +293,10 @@
- changed-files:
- any-glob-to-any-file:
- "extensions/synthetic/**"
"extensions: tavily":
- changed-files:
- any-glob-to-any-file:
- "extensions/tavily/**"
"extensions: talk-voice":
- changed-files:
- any-glob-to-any-file:

View File

@ -47,6 +47,7 @@ Docs: https://docs.openclaw.ai
- Plugins/Xiaomi: switch the bundled Xiaomi provider to the `/v1` OpenAI-compatible endpoint and add MiMo V2 Pro plus MiMo V2 Omni to the built-in catalog. (#49214) thanks @DJjjjhao.
- Plugins/Matrix: add `allowBots` room policy so configured Matrix bot accounts can talk to each other, with optional mention-only gating. Thanks @gumadeiras.
- Plugins/Matrix: add per-account `allowPrivateNetwork` opt-in for private/internal homeservers, while keeping public cleartext homeservers blocked. Thanks @gumadeiras.
- Web tools/Tavily: add Tavily as a bundled web-search provider with dedicated `tavily_search` and `tavily_extract` tools, using canonical plugin-owned config under `plugins.entries.tavily.config.webSearch.*`. (#49200) thanks @lakshyaag-tavily.
### Fixes

View File

@ -1031,6 +1031,7 @@
"tools/exec",
"tools/exec-approvals",
"tools/firecrawl",
"tools/tavily",
"tools/llm-task",
"tools/lobster",
"tools/loop-detection",

View File

@ -38,6 +38,7 @@ Scope intent:
- `plugins.entries.moonshot.config.webSearch.apiKey`
- `plugins.entries.perplexity.config.webSearch.apiKey`
- `plugins.entries.firecrawl.config.webSearch.apiKey`
- `plugins.entries.tavily.config.webSearch.apiKey`
- `tools.web.search.apiKey`
- `tools.web.search.gemini.apiKey`
- `tools.web.search.grok.apiKey`

View File

@ -551,6 +551,13 @@
"path": "tools.web.search.perplexity.apiKey",
"secretShape": "secret_input",
"optIn": true
},
{
"id": "plugins.entries.tavily.config.webSearch.apiKey",
"configFile": "openclaw.json",
"path": "plugins.entries.tavily.config.webSearch.apiKey",
"secretShape": "secret_input",
"optIn": true
}
]
}

View File

@ -256,7 +256,7 @@ Enable with `tools.loopDetection.enabled: true` (default is `false`).
### `web_search`
Search the web using Brave, Firecrawl, Gemini, Grok, Kimi, or Perplexity.
Search the web using Brave, Firecrawl, Gemini, Grok, Kimi, Perplexity, or Tavily.
Core parameters:

125
docs/tools/tavily.md Normal file
View File

@ -0,0 +1,125 @@
---
summary: "Tavily search and extract tools"
read_when:
- You want Tavily-backed web search
- You need a Tavily API key
- You want Tavily as a web_search provider
- You want content extraction from URLs
title: "Tavily"
---
# Tavily
OpenClaw can use **Tavily** in two ways:
- as the `web_search` provider
- as explicit plugin tools: `tavily_search` and `tavily_extract`
Tavily is a search API designed for AI applications, returning structured results
optimized for LLM consumption. It supports configurable search depth, topic
filtering, domain filters, AI-generated answer summaries, and content extraction
from URLs (including JavaScript-rendered pages).
## Get an API key
1. Create a Tavily account at [tavily.com](https://tavily.com/).
2. Generate an API key in the dashboard.
3. Store it in config or set `TAVILY_API_KEY` in the gateway environment.
## Configure Tavily search
```json5
{
plugins: {
entries: {
tavily: {
enabled: true,
config: {
webSearch: {
apiKey: "tvly-...", // optional if TAVILY_API_KEY is set
baseUrl: "https://api.tavily.com",
},
},
},
},
},
tools: {
web: {
search: {
provider: "tavily",
},
},
},
}
```
Notes:
- Choosing Tavily in onboarding or `openclaw configure --section web` enables
the bundled Tavily plugin automatically.
- Store Tavily config under `plugins.entries.tavily.config.webSearch.*`.
- `web_search` with Tavily supports `query` and `count` (up to 20 results).
- For Tavily-specific controls like `search_depth`, `topic`, `include_answer`,
or domain filters, use `tavily_search`.
## Tavily plugin tools
### `tavily_search`
Use this when you want Tavily-specific search controls instead of generic
`web_search`.
| Parameter | Description |
| ----------------- | --------------------------------------------------------------------- |
| `query` | Search query string (keep under 400 characters) |
| `search_depth` | `basic` (default, balanced) or `advanced` (highest relevance, slower) |
| `topic` | `general` (default), `news` (real-time updates), or `finance` |
| `max_results` | Number of results, 1-20 (default: 5) |
| `include_answer` | Include an AI-generated answer summary (default: false) |
| `time_range` | Filter by recency: `day`, `week`, `month`, or `year` |
| `include_domains` | Array of domains to restrict results to |
| `exclude_domains` | Array of domains to exclude from results |
**Search depth:**
| Depth | Speed | Relevance | Best for |
| ---------- | ------ | --------- | ----------------------------------- |
| `basic` | Faster | High | General-purpose queries (default) |
| `advanced` | Slower | Highest | Precision, specific facts, research |
### `tavily_extract`
Use this to extract clean content from one or more URLs. Handles
JavaScript-rendered pages and supports query-focused chunking for targeted
extraction.
| Parameter | Description |
| ------------------- | ---------------------------------------------------------- |
| `urls` | Array of URLs to extract (1-20 per request) |
| `query` | Rerank extracted chunks by relevance to this query |
| `extract_depth` | `basic` (default, fast) or `advanced` (for JS-heavy pages) |
| `chunks_per_source` | Chunks per URL, 1-5 (requires `query`) |
| `include_images` | Include image URLs in results (default: false) |
**Extract depth:**
| Depth | When to use |
| ---------- | ----------------------------------------- |
| `basic` | Simple pages - try this first |
| `advanced` | JS-rendered SPAs, dynamic content, tables |
Tips:
- Max 20 URLs per request. Batch larger lists into multiple calls.
- Use `query` + `chunks_per_source` to get only relevant content instead of full pages.
- Try `basic` first; fall back to `advanced` if content is missing or incomplete.
## Choosing the right tool
| Need | Tool |
| ------------------------------------ | ---------------- |
| Quick web search, no special options | `web_search` |
| Search with depth, topic, AI answers | `tavily_search` |
| Extract content from specific URLs | `tavily_extract` |
See [Web tools](/tools/web) for the full web tool setup and provider comparison.

View File

@ -1,5 +1,5 @@
---
summary: "Web search + fetch tools (Brave, Firecrawl, Gemini, Grok, Kimi, and Perplexity providers)"
summary: "Web search + fetch tools (Brave, Firecrawl, Gemini, Grok, Kimi, Perplexity, and Tavily providers)"
read_when:
- You want to enable web_search or web_fetch
- You need provider API key setup
@ -11,7 +11,7 @@ title: "Web Tools"
OpenClaw ships two lightweight web tools:
- `web_search` — Search the web using Brave Search API, Firecrawl Search, Gemini with Google Search grounding, Grok, Kimi, or Perplexity Search API.
- `web_search` — Search the web using Brave Search API, Firecrawl Search, Gemini with Google Search grounding, Grok, Kimi, Perplexity Search API, or Tavily Search API.
- `web_fetch` — HTTP fetch + readable extraction (HTML → markdown/text).
These are **not** browser automation. For JS-heavy sites or logins, use the
@ -25,8 +25,9 @@ These are **not** browser automation. For JS-heavy sites or logins, use the
(HTML → markdown/text). It does **not** execute JavaScript.
- `web_fetch` is enabled by default (unless explicitly disabled).
- The bundled Firecrawl plugin also adds `firecrawl_search` and `firecrawl_scrape` when enabled.
- The bundled Tavily plugin also adds `tavily_search` and `tavily_extract` when enabled.
See [Brave Search setup](/tools/brave-search) and [Perplexity Search setup](/tools/perplexity-search) for provider-specific details.
See [Brave Search setup](/tools/brave-search), [Perplexity Search setup](/tools/perplexity-search), and [Tavily Search setup](/tools/tavily) for provider-specific details.
## Choosing a search provider
@ -38,6 +39,7 @@ See [Brave Search setup](/tools/brave-search) and [Perplexity Search setup](/too
| **Grok** | AI-synthesized answers + citations | — | Uses xAI web-grounded responses | `XAI_API_KEY` |
| **Kimi** | AI-synthesized answers + citations | — | Uses Moonshot web search | `KIMI_API_KEY` / `MOONSHOT_API_KEY` |
| **Perplexity Search API** | Structured results with snippets | `country`, `language`, time, `domain_filter` | Supports content extraction controls; OpenRouter uses Sonar compatibility path | `PERPLEXITY_API_KEY` / `OPENROUTER_API_KEY` |
| **Tavily Search API** | Structured results with snippets | Use `tavily_search` for Tavily-specific search options | Search depth, topic filtering, AI answers, URL extraction via `tavily_extract` | `TAVILY_API_KEY` |
### Auto-detection
@ -49,6 +51,7 @@ The table above is alphabetical. If no `provider` is explicitly set, runtime aut
4. **Kimi**`KIMI_API_KEY` / `MOONSHOT_API_KEY` env var or `plugins.entries.moonshot.config.webSearch.apiKey`
5. **Perplexity**`PERPLEXITY_API_KEY`, `OPENROUTER_API_KEY`, or `plugins.entries.perplexity.config.webSearch.apiKey`
6. **Firecrawl**`FIRECRAWL_API_KEY` env var or `plugins.entries.firecrawl.config.webSearch.apiKey`
7. **Tavily**`TAVILY_API_KEY` env var or `plugins.entries.tavily.config.webSearch.apiKey`
If no keys are found, it falls back to Brave (you'll get a missing-key error prompting you to configure one).
@ -97,6 +100,7 @@ See [Perplexity Search API Docs](https://docs.perplexity.ai/guides/search-quicks
- Grok: `plugins.entries.xai.config.webSearch.apiKey`
- Kimi: `plugins.entries.moonshot.config.webSearch.apiKey`
- Perplexity: `plugins.entries.perplexity.config.webSearch.apiKey`
- Tavily: `plugins.entries.tavily.config.webSearch.apiKey`
All of these fields also support SecretRef objects.
@ -108,6 +112,7 @@ All of these fields also support SecretRef objects.
- Grok: `XAI_API_KEY`
- Kimi: `KIMI_API_KEY` or `MOONSHOT_API_KEY`
- Perplexity: `PERPLEXITY_API_KEY` or `OPENROUTER_API_KEY`
- Tavily: `TAVILY_API_KEY`
For a gateway install, put these in `~/.openclaw/.env` (or your service environment). See [Env vars](/help/faq#how-does-openclaw-load-environment-variables).
@ -176,6 +181,36 @@ For a gateway install, put these in `~/.openclaw/.env` (or your service environm
When you choose Firecrawl in onboarding or `openclaw configure --section web`, OpenClaw enables the bundled Firecrawl plugin automatically so `web_search`, `firecrawl_search`, and `firecrawl_scrape` are all available.
**Tavily Search:**
```json5
{
plugins: {
entries: {
tavily: {
enabled: true,
config: {
webSearch: {
apiKey: "tvly-...", // optional if TAVILY_API_KEY is set
baseUrl: "https://api.tavily.com",
},
},
},
},
},
tools: {
web: {
search: {
enabled: true,
provider: "tavily",
},
},
},
}
```
When you choose Tavily in onboarding or `openclaw configure --section web`, OpenClaw enables the bundled Tavily plugin automatically so `web_search`, `tavily_search`, and `tavily_extract` are all available.
**Brave LLM Context mode:**
```json5
@ -326,6 +361,7 @@ Search the web using your configured provider.
- **Grok**: `XAI_API_KEY` or `plugins.entries.xai.config.webSearch.apiKey`
- **Kimi**: `KIMI_API_KEY`, `MOONSHOT_API_KEY`, or `plugins.entries.moonshot.config.webSearch.apiKey`
- **Perplexity**: `PERPLEXITY_API_KEY`, `OPENROUTER_API_KEY`, or `plugins.entries.perplexity.config.webSearch.apiKey`
- **Tavily**: `TAVILY_API_KEY` or `plugins.entries.tavily.config.webSearch.apiKey`
- All provider key fields above support SecretRef objects.
### Config
@ -369,6 +405,8 @@ If you set `plugins.entries.perplexity.config.webSearch.baseUrl` / `model`, use
Firecrawl `web_search` supports `query` and `count`. For Firecrawl-specific controls like `sources`, `categories`, result scraping, or scrape timeout, use `firecrawl_search` from the bundled Firecrawl plugin.
Tavily `web_search` supports `query` and `count` (up to 20 results). For Tavily-specific controls like `search_depth`, `topic`, `include_answer`, or domain filters, use `tavily_search` from the bundled Tavily plugin. For URL content extraction, use `tavily_extract`. See [Tavily](/tools/tavily) for details.
**Examples:**
```javascript

View File

@ -1,5 +1,8 @@
{
"id": "brave",
"providerAuthEnvVars": {
"brave": ["BRAVE_API_KEY"]
},
"uiHints": {
"webSearch.apiKey": {
"label": "Brave Search API Key",

View File

@ -1,5 +1,8 @@
{
"id": "firecrawl",
"providerAuthEnvVars": {
"firecrawl": ["FIRECRAWL_API_KEY"]
},
"uiHints": {
"webSearch.apiKey": {
"label": "Firecrawl Search API Key",

View File

@ -1,5 +1,8 @@
{
"id": "perplexity",
"providerAuthEnvVars": {
"perplexity": ["PERPLEXITY_API_KEY", "OPENROUTER_API_KEY"]
},
"uiHints": {
"webSearch.apiKey": {
"label": "Perplexity API Key",

View File

@ -0,0 +1,41 @@
import { describe, expect, it } from "vitest";
import plugin from "./index.js";
describe("tavily plugin", () => {
it("exports a valid plugin entry with correct id and name", () => {
expect(plugin.id).toBe("tavily");
expect(plugin.name).toBe("Tavily Plugin");
expect(typeof plugin.register).toBe("function");
});
it("registers web search provider and two tools", () => {
const registrations: {
webSearchProviders: unknown[];
tools: unknown[];
} = { webSearchProviders: [], tools: [] };
const mockApi = {
registerWebSearchProvider(provider: unknown) {
registrations.webSearchProviders.push(provider);
},
registerTool(tool: unknown) {
registrations.tools.push(tool);
},
config: {},
};
plugin.register(mockApi as never);
expect(registrations.webSearchProviders).toHaveLength(1);
expect(registrations.tools).toHaveLength(2);
const provider = registrations.webSearchProviders[0] as Record<string, unknown>;
expect(provider.id).toBe("tavily");
expect(provider.autoDetectOrder).toBe(70);
expect(provider.envVars).toEqual(["TAVILY_API_KEY"]);
const toolNames = registrations.tools.map((t) => (t as Record<string, unknown>).name);
expect(toolNames).toContain("tavily_search");
expect(toolNames).toContain("tavily_extract");
});
});

View File

@ -0,0 +1,15 @@
import { definePluginEntry, type AnyAgentTool } from "openclaw/plugin-sdk/core";
import { createTavilyExtractTool } from "./src/tavily-extract-tool.js";
import { createTavilyWebSearchProvider } from "./src/tavily-search-provider.js";
import { createTavilySearchTool } from "./src/tavily-search-tool.js";
export default definePluginEntry({
id: "tavily",
name: "Tavily Plugin",
description: "Bundled Tavily search and extract plugin",
register(api) {
api.registerWebSearchProvider(createTavilyWebSearchProvider());
api.registerTool(createTavilySearchTool(api) as AnyAgentTool);
api.registerTool(createTavilyExtractTool(api) as AnyAgentTool);
},
});

View File

@ -0,0 +1,37 @@
{
"id": "tavily",
"skills": ["./skills"],
"providerAuthEnvVars": {
"tavily": ["TAVILY_API_KEY"]
},
"uiHints": {
"webSearch.apiKey": {
"label": "Tavily API Key",
"help": "Tavily API key for web search and extraction (fallback: TAVILY_API_KEY env var).",
"sensitive": true,
"placeholder": "tvly-..."
},
"webSearch.baseUrl": {
"label": "Tavily Base URL",
"help": "Tavily API base URL override."
}
},
"configSchema": {
"type": "object",
"additionalProperties": false,
"properties": {
"webSearch": {
"type": "object",
"additionalProperties": false,
"properties": {
"apiKey": {
"type": ["string", "object"]
},
"baseUrl": {
"type": "string"
}
}
}
}
}
}

View File

@ -0,0 +1,12 @@
{
"name": "@openclaw/tavily-plugin",
"version": "2026.3.17",
"private": true,
"description": "OpenClaw Tavily plugin",
"type": "module",
"openclaw": {
"extensions": [
"./index.ts"
]
}
}

View File

@ -0,0 +1,94 @@
---
name: tavily
description: Tavily web search, content extraction, and research tools.
metadata:
{ "openclaw": { "emoji": "🔍", "requires": { "config": ["plugins.entries.tavily.enabled"] } } }
---
# Tavily Tools
## When to use which tool
| Need | Tool | When |
| ---------------------------- | ---------------- | ------------------------------------------------------------- |
| Quick web search | `web_search` | Basic queries, no special options needed |
| Search with advanced options | `tavily_search` | Need depth, topic, domain filters, time ranges, or AI answers |
| Extract content from URLs | `tavily_extract` | Have specific URLs, need their content |
## web_search
Tavily powers this automatically when selected as the search provider. Use for
straightforward queries where you don't need Tavily-specific options.
| Parameter | Description |
| --------- | ------------------------ |
| `query` | Search query string |
| `count` | Number of results (1-20) |
## tavily_search
Use when you need fine-grained control over search behavior.
| Parameter | Description |
| ----------------- | --------------------------------------------------------------------- |
| `query` | Search query string (keep under 400 characters) |
| `search_depth` | `basic` (default, balanced) or `advanced` (highest relevance, slower) |
| `topic` | `general` (default), `news` (real-time updates), or `finance` |
| `max_results` | Number of results, 1-20 (default: 5) |
| `include_answer` | Include an AI-generated answer summary (default: false) |
| `time_range` | Filter by recency: `day`, `week`, `month`, or `year` |
| `include_domains` | Array of domains to restrict results to |
| `exclude_domains` | Array of domains to exclude from results |
### Search depth
| Depth | Speed | Relevance | Best for |
| ---------- | ------ | --------- | -------------------------------------------- |
| `basic` | Faster | High | General-purpose queries (default) |
| `advanced` | Slower | Highest | Precision, specific facts, detailed research |
### Tips
- **Keep queries under 400 characters** — think search query, not prompt.
- **Break complex queries into sub-queries** for better results.
- **Use `include_domains`** to focus on trusted sources.
- **Use `time_range`** for recent information (news, current events).
- **Use `include_answer`** when you need a quick synthesized answer.
## tavily_extract
Use when you have specific URLs and need their content. Handles JavaScript-rendered
pages and returns clean markdown. Supports query-focused chunking for targeted
extraction.
| Parameter | Description |
| ------------------- | ------------------------------------------------------------------ |
| `urls` | Array of URLs to extract (1-20 per request) |
| `query` | Rerank extracted chunks by relevance to this query |
| `extract_depth` | `basic` (default, fast) or `advanced` (for JS-heavy pages, tables) |
| `chunks_per_source` | Chunks per URL, 1-5 (requires `query`) |
| `include_images` | Include image URLs in results (default: false) |
### Extract depth
| Depth | When to use |
| ---------- | ----------------------------------------------------------- |
| `basic` | Simple pages — try this first |
| `advanced` | JS-rendered SPAs, dynamic content, tables, embedded content |
### Tips
- **Max 20 URLs per request** — batch larger lists into multiple calls.
- **Use `query` + `chunks_per_source`** to get only relevant content instead of full pages.
- **Try `basic` first**, fall back to `advanced` if content is missing or incomplete.
- If `tavily_search` results already contain the snippets you need, skip the extract step.
## Choosing the right workflow
Follow this escalation pattern — start simple, escalate only when needed:
1. **`web_search`** — Quick lookup, no special options needed.
2. **`tavily_search`** — Need depth control, topic filtering, domain filters, time ranges, or AI answers.
3. **`tavily_extract`** — Have specific URLs, need their full content or targeted chunks.
Combine search + extract when you need to find pages first, then get their full content.

View File

@ -0,0 +1,71 @@
import type { OpenClawConfig } from "openclaw/plugin-sdk/config-runtime";
import { normalizeResolvedSecretInputString } from "openclaw/plugin-sdk/config-runtime";
import { normalizeSecretInput } from "openclaw/plugin-sdk/provider-auth";
export const DEFAULT_TAVILY_BASE_URL = "https://api.tavily.com";
export const DEFAULT_TAVILY_SEARCH_TIMEOUT_SECONDS = 30;
export const DEFAULT_TAVILY_EXTRACT_TIMEOUT_SECONDS = 60;
type TavilySearchConfig =
| {
apiKey?: unknown;
baseUrl?: string;
}
| undefined;
type PluginEntryConfig = {
webSearch?: {
apiKey?: unknown;
baseUrl?: string;
};
};
export function resolveTavilySearchConfig(cfg?: OpenClawConfig): TavilySearchConfig {
const pluginConfig = cfg?.plugins?.entries?.tavily?.config as PluginEntryConfig;
const pluginWebSearch = pluginConfig?.webSearch;
if (pluginWebSearch && typeof pluginWebSearch === "object" && !Array.isArray(pluginWebSearch)) {
return pluginWebSearch;
}
return undefined;
}
function normalizeConfiguredSecret(value: unknown, path: string): string | undefined {
return normalizeSecretInput(
normalizeResolvedSecretInputString({
value,
path,
}),
);
}
export function resolveTavilyApiKey(cfg?: OpenClawConfig): string | undefined {
const search = resolveTavilySearchConfig(cfg);
return (
normalizeConfiguredSecret(search?.apiKey, "plugins.entries.tavily.config.webSearch.apiKey") ||
normalizeSecretInput(process.env.TAVILY_API_KEY) ||
undefined
);
}
export function resolveTavilyBaseUrl(cfg?: OpenClawConfig): string {
const search = resolveTavilySearchConfig(cfg);
const configured =
(typeof search?.baseUrl === "string" ? search.baseUrl.trim() : "") ||
normalizeSecretInput(process.env.TAVILY_BASE_URL) ||
"";
return configured || DEFAULT_TAVILY_BASE_URL;
}
export function resolveTavilySearchTimeoutSeconds(override?: number): number {
if (typeof override === "number" && Number.isFinite(override) && override > 0) {
return Math.floor(override);
}
return DEFAULT_TAVILY_SEARCH_TIMEOUT_SECONDS;
}
export function resolveTavilyExtractTimeoutSeconds(override?: number): number {
if (typeof override === "number" && Number.isFinite(override) && override > 0) {
return Math.floor(override);
}
return DEFAULT_TAVILY_EXTRACT_TIMEOUT_SECONDS;
}

View File

@ -0,0 +1,286 @@
import type { OpenClawConfig } from "openclaw/plugin-sdk/config-runtime";
import { withTrustedWebToolsEndpoint } from "openclaw/plugin-sdk/provider-web-search";
import {
DEFAULT_CACHE_TTL_MINUTES,
normalizeCacheKey,
readCache,
readResponseText,
resolveCacheTtlMs,
writeCache,
} from "openclaw/plugin-sdk/provider-web-search";
import { wrapExternalContent, wrapWebContent } from "openclaw/plugin-sdk/security-runtime";
import {
DEFAULT_TAVILY_BASE_URL,
resolveTavilyApiKey,
resolveTavilyBaseUrl,
resolveTavilyExtractTimeoutSeconds,
resolveTavilySearchTimeoutSeconds,
} from "./config.js";
const SEARCH_CACHE = new Map<
string,
{ value: Record<string, unknown>; expiresAt: number; insertedAt: number }
>();
const EXTRACT_CACHE = new Map<
string,
{ value: Record<string, unknown>; expiresAt: number; insertedAt: number }
>();
const DEFAULT_SEARCH_COUNT = 5;
const DEFAULT_ERROR_MAX_BYTES = 64_000;
export type TavilySearchParams = {
cfg?: OpenClawConfig;
query: string;
searchDepth?: string;
topic?: string;
maxResults?: number;
includeAnswer?: boolean;
timeRange?: string;
includeDomains?: string[];
excludeDomains?: string[];
timeoutSeconds?: number;
};
export type TavilyExtractParams = {
cfg?: OpenClawConfig;
urls: string[];
query?: string;
extractDepth?: string;
chunksPerSource?: number;
includeImages?: boolean;
timeoutSeconds?: number;
};
function resolveEndpoint(baseUrl: string, pathname: string): string {
const trimmed = baseUrl.trim();
if (!trimmed) {
return `${DEFAULT_TAVILY_BASE_URL}${pathname}`;
}
try {
const url = new URL(trimmed);
// Always append the endpoint pathname to the base URL path,
// supporting both bare hosts and reverse-proxy path prefixes.
url.pathname = url.pathname.replace(/\/$/, "") + pathname;
return url.toString();
} catch {
return `${DEFAULT_TAVILY_BASE_URL}${pathname}`;
}
}
async function postTavilyJson(params: {
baseUrl: string;
pathname: string;
apiKey: string;
body: Record<string, unknown>;
timeoutSeconds: number;
errorLabel: string;
}): Promise<Record<string, unknown>> {
const endpoint = resolveEndpoint(params.baseUrl, params.pathname);
return await withTrustedWebToolsEndpoint(
{
url: endpoint,
timeoutSeconds: params.timeoutSeconds,
init: {
method: "POST",
headers: {
Accept: "application/json",
Authorization: `Bearer ${params.apiKey}`,
"Content-Type": "application/json",
},
body: JSON.stringify(params.body),
},
},
async ({ response }) => {
if (!response.ok) {
const detail = await readResponseText(response, { maxBytes: DEFAULT_ERROR_MAX_BYTES });
throw new Error(
`${params.errorLabel} API error (${response.status}): ${detail.text || response.statusText}`,
);
}
return (await response.json()) as Record<string, unknown>;
},
);
}
export async function runTavilySearch(
params: TavilySearchParams,
): Promise<Record<string, unknown>> {
const apiKey = resolveTavilyApiKey(params.cfg);
if (!apiKey) {
throw new Error(
"web_search (tavily) needs a Tavily API key. Set TAVILY_API_KEY in the Gateway environment, or configure plugins.entries.tavily.config.webSearch.apiKey.",
);
}
const count =
typeof params.maxResults === "number" && Number.isFinite(params.maxResults)
? Math.max(1, Math.min(20, Math.floor(params.maxResults)))
: DEFAULT_SEARCH_COUNT;
const timeoutSeconds = resolveTavilySearchTimeoutSeconds(params.timeoutSeconds);
const baseUrl = resolveTavilyBaseUrl(params.cfg);
const cacheKey = normalizeCacheKey(
JSON.stringify({
type: "tavily-search",
q: params.query,
count,
baseUrl,
searchDepth: params.searchDepth,
topic: params.topic,
includeAnswer: params.includeAnswer,
timeRange: params.timeRange,
includeDomains: params.includeDomains,
excludeDomains: params.excludeDomains,
}),
);
const cached = readCache(SEARCH_CACHE, cacheKey);
if (cached) {
return { ...cached.value, cached: true };
}
const body: Record<string, unknown> = {
query: params.query,
max_results: count,
};
if (params.searchDepth) body.search_depth = params.searchDepth;
if (params.topic) body.topic = params.topic;
if (params.includeAnswer) body.include_answer = true;
if (params.timeRange) body.time_range = params.timeRange;
if (params.includeDomains?.length) body.include_domains = params.includeDomains;
if (params.excludeDomains?.length) body.exclude_domains = params.excludeDomains;
const start = Date.now();
const payload = await postTavilyJson({
baseUrl,
pathname: "/search",
apiKey,
body,
timeoutSeconds,
errorLabel: "Tavily Search",
});
const rawResults = Array.isArray(payload.results) ? payload.results : [];
const results = rawResults.map((r: Record<string, unknown>) => ({
title: typeof r.title === "string" ? wrapWebContent(r.title, "web_search") : "",
url: typeof r.url === "string" ? r.url : "",
snippet: typeof r.content === "string" ? wrapWebContent(r.content, "web_search") : "",
score: typeof r.score === "number" ? r.score : undefined,
...(typeof r.published_date === "string" ? { published: r.published_date } : {}),
}));
const result: Record<string, unknown> = {
query: params.query,
provider: "tavily",
count: results.length,
tookMs: Date.now() - start,
externalContent: {
untrusted: true,
source: "web_search",
provider: "tavily",
wrapped: true,
},
results,
};
if (typeof payload.answer === "string" && payload.answer) {
result.answer = wrapWebContent(payload.answer, "web_search");
}
writeCache(
SEARCH_CACHE,
cacheKey,
result,
resolveCacheTtlMs(undefined, DEFAULT_CACHE_TTL_MINUTES),
);
return result;
}
export async function runTavilyExtract(
params: TavilyExtractParams,
): Promise<Record<string, unknown>> {
const apiKey = resolveTavilyApiKey(params.cfg);
if (!apiKey) {
throw new Error(
"tavily_extract needs a Tavily API key. Set TAVILY_API_KEY in the Gateway environment, or configure plugins.entries.tavily.config.webSearch.apiKey.",
);
}
const baseUrl = resolveTavilyBaseUrl(params.cfg);
const timeoutSeconds = resolveTavilyExtractTimeoutSeconds(params.timeoutSeconds);
const cacheKey = normalizeCacheKey(
JSON.stringify({
type: "tavily-extract",
urls: params.urls,
baseUrl,
query: params.query,
extractDepth: params.extractDepth,
chunksPerSource: params.chunksPerSource,
includeImages: params.includeImages,
}),
);
const cached = readCache(EXTRACT_CACHE, cacheKey);
if (cached) {
return { ...cached.value, cached: true };
}
const body: Record<string, unknown> = { urls: params.urls };
if (params.query) body.query = params.query;
if (params.extractDepth) body.extract_depth = params.extractDepth;
if (params.chunksPerSource) body.chunks_per_source = params.chunksPerSource;
if (params.includeImages) body.include_images = true;
const start = Date.now();
const payload = await postTavilyJson({
baseUrl,
pathname: "/extract",
apiKey,
body,
timeoutSeconds,
errorLabel: "Tavily Extract",
});
const rawResults = Array.isArray(payload.results) ? payload.results : [];
const results = rawResults.map((r: Record<string, unknown>) => ({
url: typeof r.url === "string" ? r.url : "",
rawContent:
typeof r.raw_content === "string"
? wrapExternalContent(r.raw_content, { source: "web_fetch", includeWarning: false })
: "",
...(typeof r.content === "string"
? { content: wrapExternalContent(r.content, { source: "web_fetch", includeWarning: false }) }
: {}),
...(Array.isArray(r.images)
? {
images: (r.images as string[]).map((img) =>
wrapExternalContent(String(img), { source: "web_fetch", includeWarning: false }),
),
}
: {}),
}));
const failedResults = Array.isArray(payload.failed_results) ? payload.failed_results : [];
const result: Record<string, unknown> = {
provider: "tavily",
count: results.length,
tookMs: Date.now() - start,
externalContent: {
untrusted: true,
source: "web_fetch",
provider: "tavily",
wrapped: true,
},
results,
...(failedResults.length > 0 ? { failedResults } : {}),
};
writeCache(
EXTRACT_CACHE,
cacheKey,
result,
resolveCacheTtlMs(undefined, DEFAULT_CACHE_TTL_MINUTES),
);
return result;
}
export const __testing = {
postTavilyJson,
};

View File

@ -0,0 +1,53 @@
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/plugin-runtime";
import { beforeEach, describe, expect, it, vi } from "vitest";
vi.mock("./tavily-client.js", () => ({
runTavilyExtract: vi.fn(async (params: unknown) => ({ ok: true, params })),
}));
import { runTavilyExtract } from "./tavily-client.js";
import { createTavilyExtractTool } from "./tavily-extract-tool.js";
function fakeApi(): OpenClawPluginApi {
return {
config: {},
} as OpenClawPluginApi;
}
describe("tavily_extract", () => {
beforeEach(() => {
vi.clearAllMocks();
});
it("rejects chunks_per_source without query", async () => {
const tool = createTavilyExtractTool(fakeApi());
await expect(
tool.execute("id", {
urls: ["https://example.com"],
chunks_per_source: 2,
}),
).rejects.toThrow("tavily_extract requires query when chunks_per_source is set.");
expect(runTavilyExtract).not.toHaveBeenCalled();
});
it("forwards query-scoped chunking when query is provided", async () => {
const tool = createTavilyExtractTool(fakeApi());
await tool.execute("id", {
urls: ["https://example.com"],
query: "pricing",
chunks_per_source: 2,
});
expect(runTavilyExtract).toHaveBeenCalledWith(
expect.objectContaining({
cfg: {},
urls: ["https://example.com"],
query: "pricing",
chunksPerSource: 2,
}),
);
});
});

View File

@ -0,0 +1,74 @@
import { Type } from "@sinclair/typebox";
import { optionalStringEnum } from "openclaw/plugin-sdk/agent-runtime";
import { jsonResult, readNumberParam, readStringParam } from "openclaw/plugin-sdk/agent-runtime";
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/plugin-runtime";
import { runTavilyExtract } from "./tavily-client.js";
const TavilyExtractToolSchema = Type.Object(
{
urls: Type.Array(Type.String(), {
description: "One or more URLs to extract content from (max 20).",
minItems: 1,
maxItems: 20,
}),
query: Type.Optional(
Type.String({
description: "Rerank extracted chunks by relevance to this query.",
}),
),
extract_depth: optionalStringEnum(["basic", "advanced"] as const, {
description: '"basic" (default) or "advanced" (for JS-heavy pages).',
}),
chunks_per_source: Type.Optional(
Type.Number({
description: "Chunks per URL (1-5, requires query).",
minimum: 1,
maximum: 5,
}),
),
include_images: Type.Optional(
Type.Boolean({
description: "Include image URLs in extraction results.",
}),
),
},
{ additionalProperties: false },
);
export function createTavilyExtractTool(api: OpenClawPluginApi) {
return {
name: "tavily_extract",
label: "Tavily Extract",
description:
"Extract clean content from one or more URLs using Tavily. Handles JS-rendered pages. Supports query-focused chunking.",
parameters: TavilyExtractToolSchema,
execute: async (_toolCallId: string, rawParams: Record<string, unknown>) => {
const urls = Array.isArray(rawParams.urls)
? (rawParams.urls as string[]).filter(Boolean)
: [];
if (urls.length === 0) {
throw new Error("tavily_extract requires at least one URL.");
}
const query = readStringParam(rawParams, "query") || undefined;
const extractDepth = readStringParam(rawParams, "extract_depth") || undefined;
const chunksPerSource = readNumberParam(rawParams, "chunks_per_source", {
integer: true,
});
if (chunksPerSource !== undefined && !query) {
throw new Error("tavily_extract requires query when chunks_per_source is set.");
}
const includeImages = rawParams.include_images === true;
return jsonResult(
await runTavilyExtract({
cfg: api.config,
urls,
query,
extractDepth,
chunksPerSource,
includeImages,
}),
);
},
};
}

View File

@ -0,0 +1,76 @@
import { Type } from "@sinclair/typebox";
import {
enablePluginInConfig,
resolveProviderWebSearchPluginConfig,
setProviderWebSearchPluginConfigValue,
type WebSearchProviderPlugin,
} from "openclaw/plugin-sdk/provider-web-search";
import { runTavilySearch } from "./tavily-client.js";
const GenericTavilySearchSchema = Type.Object(
{
query: Type.String({ description: "Search query string." }),
count: Type.Optional(
Type.Number({
description: "Number of results to return (1-20).",
minimum: 1,
maximum: 20,
}),
),
},
{ additionalProperties: false },
);
function getScopedCredentialValue(searchConfig?: Record<string, unknown>): unknown {
const scoped = searchConfig?.tavily;
if (!scoped || typeof scoped !== "object" || Array.isArray(scoped)) {
return undefined;
}
return (scoped as Record<string, unknown>).apiKey;
}
function setScopedCredentialValue(
searchConfigTarget: Record<string, unknown>,
value: unknown,
): void {
const scoped = searchConfigTarget.tavily;
if (!scoped || typeof scoped !== "object" || Array.isArray(scoped)) {
searchConfigTarget.tavily = { apiKey: value };
return;
}
(scoped as Record<string, unknown>).apiKey = value;
}
export function createTavilyWebSearchProvider(): WebSearchProviderPlugin {
return {
id: "tavily",
label: "Tavily Search",
hint: "Structured results with domain filters and AI answer summaries",
envVars: ["TAVILY_API_KEY"],
placeholder: "tvly-...",
signupUrl: "https://tavily.com/",
docsUrl: "https://docs.openclaw.ai/tools/tavily",
autoDetectOrder: 70,
credentialPath: "plugins.entries.tavily.config.webSearch.apiKey",
inactiveSecretPaths: ["plugins.entries.tavily.config.webSearch.apiKey"],
getCredentialValue: getScopedCredentialValue,
setCredentialValue: setScopedCredentialValue,
getConfiguredCredentialValue: (config) =>
resolveProviderWebSearchPluginConfig(config, "tavily")?.apiKey,
setConfiguredCredentialValue: (configTarget, value) => {
setProviderWebSearchPluginConfigValue(configTarget, "tavily", "apiKey", value);
},
applySelectionConfig: (config) => enablePluginInConfig(config, "tavily").config,
createTool: (ctx) => ({
description:
"Search the web using Tavily. Returns structured results with snippets. Use tavily_search for Tavily-specific options like search depth, topic filtering, or AI answers.",
parameters: GenericTavilySearchSchema,
execute: async (args) =>
await runTavilySearch({
cfg: ctx.config,
query: typeof args.query === "string" ? args.query : "",
maxResults: typeof args.count === "number" ? args.count : undefined,
}),
}),
};
}

View File

@ -0,0 +1,81 @@
import { Type } from "@sinclair/typebox";
import { optionalStringEnum } from "openclaw/plugin-sdk/agent-runtime";
import { jsonResult, readNumberParam, readStringParam } from "openclaw/plugin-sdk/agent-runtime";
import type { OpenClawPluginApi } from "openclaw/plugin-sdk/plugin-runtime";
import { runTavilySearch } from "./tavily-client.js";
const TavilySearchToolSchema = Type.Object(
{
query: Type.String({ description: "Search query string." }),
search_depth: optionalStringEnum(["basic", "advanced"] as const, {
description: 'Search depth: "basic" (default, faster) or "advanced" (more thorough).',
}),
topic: optionalStringEnum(["general", "news", "finance"] as const, {
description: 'Search topic: "general" (default), "news", or "finance".',
}),
max_results: Type.Optional(
Type.Number({
description: "Number of results to return (1-20).",
minimum: 1,
maximum: 20,
}),
),
include_answer: Type.Optional(
Type.Boolean({
description: "Include an AI-generated answer summary (default: false).",
}),
),
time_range: optionalStringEnum(["day", "week", "month", "year"] as const, {
description: "Filter results by recency: 'day', 'week', 'month', or 'year'.",
}),
include_domains: Type.Optional(
Type.Array(Type.String(), {
description: "Only include results from these domains.",
}),
),
exclude_domains: Type.Optional(
Type.Array(Type.String(), {
description: "Exclude results from these domains.",
}),
),
},
{ additionalProperties: false },
);
export function createTavilySearchTool(api: OpenClawPluginApi) {
return {
name: "tavily_search",
label: "Tavily Search",
description:
"Search the web using Tavily Search API. Supports search depth, topic filtering, domain filters, time ranges, and AI answer summaries.",
parameters: TavilySearchToolSchema,
execute: async (_toolCallId: string, rawParams: Record<string, unknown>) => {
const query = readStringParam(rawParams, "query", { required: true });
const searchDepth = readStringParam(rawParams, "search_depth") || undefined;
const topic = readStringParam(rawParams, "topic") || undefined;
const maxResults = readNumberParam(rawParams, "max_results", { integer: true });
const includeAnswer = rawParams.include_answer === true;
const timeRange = readStringParam(rawParams, "time_range") || undefined;
const includeDomains = Array.isArray(rawParams.include_domains)
? (rawParams.include_domains as string[]).filter(Boolean)
: undefined;
const excludeDomains = Array.isArray(rawParams.exclude_domains)
? (rawParams.exclude_domains as string[]).filter(Boolean)
: undefined;
return jsonResult(
await runTavilySearch({
cfg: api.config,
query,
searchDepth,
topic,
maxResults,
includeAnswer,
timeRange,
includeDomains: includeDomains?.length ? includeDomains : undefined,
excludeDomains: excludeDomains?.length ? excludeDomains : undefined,
}),
);
},
};
}

2
pnpm-lock.yaml generated
View File

@ -519,6 +519,8 @@ importers:
extensions/synthetic: {}
extensions/tavily: {}
extensions/telegram:
dependencies:
'@grammyjs/runner':

View File

@ -1,123 +1,35 @@
import type { OpenClawConfig } from "../../config/config.js";
import { normalizeResolvedSecretInputString } from "../../config/types.secrets.js";
import { logVerbose } from "../../globals.js";
import type { PluginWebSearchProviderEntry } from "../../plugins/types.js";
import { resolvePluginWebSearchProviders } from "../../plugins/web-search-providers.js";
import type { RuntimeWebSearchMetadata } from "../../secrets/runtime-web-tools.types.js";
import { normalizeSecretInput } from "../../utils/normalize-secret-input.js";
import {
resolveWebSearchDefinition,
resolveWebSearchProviderId,
} from "../../web-search/runtime.js";
import type { AnyAgentTool } from "./common.js";
import { jsonResult } from "./common.js";
import { SEARCH_CACHE } from "./web-search-provider-common.js";
import {
resolveSearchConfig,
resolveSearchEnabled,
type WebSearchConfig,
} from "./web-search-provider-config.js";
function readProviderEnvValue(envVars: string[]): string | undefined {
for (const envVar of envVars) {
const value = normalizeSecretInput(process.env[envVar]);
if (value) {
return value;
}
}
return undefined;
}
function hasProviderCredential(
provider: PluginWebSearchProviderEntry,
search: WebSearchConfig | undefined,
): boolean {
const rawValue = provider.getCredentialValue(search as Record<string, unknown> | undefined);
const fromConfig = normalizeSecretInput(
normalizeResolvedSecretInputString({
value: rawValue,
path: provider.credentialPath,
}),
);
return Boolean(fromConfig || readProviderEnvValue(provider.envVars));
}
function resolveSearchProvider(search?: WebSearchConfig): string {
const providers = resolvePluginWebSearchProviders({
bundledAllowlistCompat: true,
});
const raw =
search && "provider" in search && typeof search.provider === "string"
? search.provider.trim().toLowerCase()
: "";
if (raw) {
const explicit = providers.find((provider) => provider.id === raw);
if (explicit) {
return explicit.id;
}
}
if (!raw) {
for (const provider of providers) {
if (!hasProviderCredential(provider, search)) {
continue;
}
logVerbose(
`web_search: no provider configured, auto-detected "${provider.id}" from available API keys`,
);
return provider.id;
}
}
return providers[0]?.id ?? "";
}
export function createWebSearchTool(options?: {
config?: OpenClawConfig;
sandboxed?: boolean;
runtimeWebSearch?: RuntimeWebSearchMetadata;
}): AnyAgentTool | null {
const search = resolveSearchConfig(options?.config);
if (!resolveSearchEnabled({ search, sandboxed: options?.sandboxed })) {
return null;
}
const providers = resolvePluginWebSearchProviders({
config: options?.config,
bundledAllowlistCompat: true,
});
if (providers.length === 0) {
return null;
}
const providerId =
options?.runtimeWebSearch?.selectedProvider ??
options?.runtimeWebSearch?.providerConfigured ??
resolveSearchProvider(search);
const provider =
providers.find((entry) => entry.id === providerId) ??
providers.find((entry) => entry.id === resolveSearchProvider(search)) ??
providers[0];
if (!provider) {
return null;
}
const definition = provider.createTool({
config: options?.config,
searchConfig: search as Record<string, unknown> | undefined,
runtimeMetadata: options?.runtimeWebSearch,
});
if (!definition) {
const resolved = resolveWebSearchDefinition(options);
if (!resolved) {
return null;
}
return {
label: "Web Search",
name: "web_search",
description: definition.description,
parameters: definition.parameters,
execute: async (_toolCallId, args) => jsonResult(await definition.execute(args)),
description: resolved.definition.description,
parameters: resolved.definition.parameters,
execute: async (_toolCallId, args) => jsonResult(await resolved.definition.execute(args)),
};
}
export const __testing = {
SEARCH_CACHE,
resolveSearchProvider,
resolveSearchProvider: (
search?: NonNullable<NonNullable<OpenClawConfig["tools"]>["web"]>["search"],
) => resolveWebSearchProviderId({ search }),
};

View File

@ -48,6 +48,15 @@ function createPerplexityConfig(apiKey: string, enabled?: boolean): OpenClawConf
};
}
function pluginWebSearchApiKey(config: OpenClawConfig, pluginId: string): unknown {
const entry = (
config.plugins?.entries as
| Record<string, { config?: { webSearch?: { apiKey?: unknown } } }>
| undefined
)?.[pluginId];
return entry?.config?.webSearch?.apiKey;
}
async function runBlankPerplexityKeyEntry(
apiKey: string,
enabled?: boolean,
@ -88,8 +97,9 @@ describe("setupSearch", () => {
});
const result = await setupSearch(cfg, runtime, prompter);
expect(result.tools?.web?.search?.provider).toBe("perplexity");
expect(result.tools?.web?.search?.perplexity?.apiKey).toBe("pplx-test-key");
expect(pluginWebSearchApiKey(result, "perplexity")).toBe("pplx-test-key");
expect(result.tools?.web?.search?.enabled).toBe(true);
expect(result.plugins?.entries?.perplexity?.enabled).toBe(true);
});
it("sets provider and key for brave", async () => {
@ -101,7 +111,8 @@ describe("setupSearch", () => {
const result = await setupSearch(cfg, runtime, prompter);
expect(result.tools?.web?.search?.provider).toBe("brave");
expect(result.tools?.web?.search?.enabled).toBe(true);
expect(result.tools?.web?.search?.apiKey).toBe("BSA-test-key");
expect(pluginWebSearchApiKey(result, "brave")).toBe("BSA-test-key");
expect(result.plugins?.entries?.brave?.enabled).toBe(true);
});
it("sets provider and key for gemini", async () => {
@ -113,7 +124,8 @@ describe("setupSearch", () => {
const result = await setupSearch(cfg, runtime, prompter);
expect(result.tools?.web?.search?.provider).toBe("gemini");
expect(result.tools?.web?.search?.enabled).toBe(true);
expect(result.tools?.web?.search?.gemini?.apiKey).toBe("AIza-test");
expect(pluginWebSearchApiKey(result, "google")).toBe("AIza-test");
expect(result.plugins?.entries?.google?.enabled).toBe(true);
});
it("sets provider and key for firecrawl and enables the plugin", async () => {
@ -125,7 +137,7 @@ describe("setupSearch", () => {
const result = await setupSearch(cfg, runtime, prompter);
expect(result.tools?.web?.search?.provider).toBe("firecrawl");
expect(result.tools?.web?.search?.enabled).toBe(true);
expect(result.tools?.web?.search?.firecrawl?.apiKey).toBe("fc-test-key");
expect(pluginWebSearchApiKey(result, "firecrawl")).toBe("fc-test-key");
expect(result.plugins?.entries?.firecrawl?.enabled).toBe(true);
});
@ -150,7 +162,21 @@ describe("setupSearch", () => {
const result = await setupSearch(cfg, runtime, prompter);
expect(result.tools?.web?.search?.provider).toBe("kimi");
expect(result.tools?.web?.search?.enabled).toBe(true);
expect(result.tools?.web?.search?.kimi?.apiKey).toBe("sk-moonshot");
expect(pluginWebSearchApiKey(result, "moonshot")).toBe("sk-moonshot");
expect(result.plugins?.entries?.moonshot?.enabled).toBe(true);
});
it("sets provider and key for tavily and enables the plugin", async () => {
const cfg: OpenClawConfig = {};
const { prompter } = createPrompter({
selectValue: "tavily",
textValue: "tvly-test-key",
});
const result = await setupSearch(cfg, runtime, prompter);
expect(result.tools?.web?.search?.provider).toBe("tavily");
expect(result.tools?.web?.search?.enabled).toBe(true);
expect(pluginWebSearchApiKey(result, "tavily")).toBe("tvly-test-key");
expect(result.plugins?.entries?.tavily?.enabled).toBe(true);
});
it("shows missing-key note when no key is provided and no env var", async () => {
@ -198,7 +224,7 @@ describe("setupSearch", () => {
"stored-pplx-key", // pragma: allowlist secret
);
expect(result.tools?.web?.search?.provider).toBe("perplexity");
expect(result.tools?.web?.search?.perplexity?.apiKey).toBe("stored-pplx-key");
expect(pluginWebSearchApiKey(result, "perplexity")).toBe("stored-pplx-key");
expect(result.tools?.web?.search?.enabled).toBe(true);
expect(prompter.text).not.toHaveBeenCalled();
});
@ -209,11 +235,43 @@ describe("setupSearch", () => {
false,
);
expect(result.tools?.web?.search?.provider).toBe("perplexity");
expect(result.tools?.web?.search?.perplexity?.apiKey).toBe("stored-pplx-key");
expect(pluginWebSearchApiKey(result, "perplexity")).toBe("stored-pplx-key");
expect(result.tools?.web?.search?.enabled).toBe(false);
expect(prompter.text).not.toHaveBeenCalled();
});
it("quickstart skips key prompt when canonical plugin config key exists", async () => {
const cfg: OpenClawConfig = {
tools: {
web: {
search: {
provider: "tavily",
},
},
},
plugins: {
entries: {
tavily: {
enabled: true,
config: {
webSearch: {
apiKey: "tvly-existing-key",
},
},
},
},
},
};
const { prompter } = createPrompter({ selectValue: "tavily" });
const result = await setupSearch(cfg, runtime, prompter, {
quickstartDefaults: true,
});
expect(result.tools?.web?.search?.provider).toBe("tavily");
expect(pluginWebSearchApiKey(result, "tavily")).toBe("tvly-existing-key");
expect(result.tools?.web?.search?.enabled).toBe(true);
expect(prompter.text).not.toHaveBeenCalled();
});
it("quickstart falls through to key prompt when no key and no env var", async () => {
const original = process.env.XAI_API_KEY;
delete process.env.XAI_API_KEY;
@ -268,7 +326,7 @@ describe("setupSearch", () => {
secretInputMode: "ref", // pragma: allowlist secret
});
expect(result.tools?.web?.search?.provider).toBe("perplexity");
expect(result.tools?.web?.search?.perplexity?.apiKey).toEqual({
expect(pluginWebSearchApiKey(result, "perplexity")).toEqual({
source: "env",
provider: "default",
id: "PERPLEXITY_API_KEY", // pragma: allowlist secret
@ -299,7 +357,7 @@ describe("setupSearch", () => {
const result = await setupSearch(cfg, runtime, prompter, {
secretInputMode: "ref", // pragma: allowlist secret
});
expect(result.tools?.web?.search?.perplexity?.apiKey).toEqual({
expect(pluginWebSearchApiKey(result, "perplexity")).toEqual({
source: "env",
provider: "default",
id: "OPENROUTER_API_KEY", // pragma: allowlist secret
@ -326,14 +384,41 @@ describe("setupSearch", () => {
secretInputMode: "ref", // pragma: allowlist secret
});
expect(result.tools?.web?.search?.provider).toBe("brave");
expect(result.tools?.web?.search?.apiKey).toEqual({
expect(pluginWebSearchApiKey(result, "brave")).toEqual({
source: "env",
provider: "default",
id: "BRAVE_API_KEY",
});
expect(result.plugins?.entries?.brave?.enabled).toBe(true);
expect(prompter.text).not.toHaveBeenCalled();
});
it("stores env-backed SecretRef when secretInputMode=ref for tavily", async () => {
const original = process.env.TAVILY_API_KEY;
delete process.env.TAVILY_API_KEY;
const cfg: OpenClawConfig = {};
try {
const { prompter } = createPrompter({ selectValue: "tavily" });
const result = await setupSearch(cfg, runtime, prompter, {
secretInputMode: "ref", // pragma: allowlist secret
});
expect(result.tools?.web?.search?.provider).toBe("tavily");
expect(pluginWebSearchApiKey(result, "tavily")).toEqual({
source: "env",
provider: "default",
id: "TAVILY_API_KEY",
});
expect(result.plugins?.entries?.tavily?.enabled).toBe(true);
expect(prompter.text).not.toHaveBeenCalled();
} finally {
if (original === undefined) {
delete process.env.TAVILY_API_KEY;
} else {
process.env.TAVILY_API_KEY = original;
}
}
});
it("stores plaintext key when secretInputMode is unset", async () => {
const cfg: OpenClawConfig = {};
const { prompter } = createPrompter({
@ -341,12 +426,20 @@ describe("setupSearch", () => {
textValue: "BSA-plain",
});
const result = await setupSearch(cfg, runtime, prompter);
expect(result.tools?.web?.search?.apiKey).toBe("BSA-plain");
expect(pluginWebSearchApiKey(result, "brave")).toBe("BSA-plain");
});
it("exports all 6 providers in SEARCH_PROVIDER_OPTIONS", () => {
expect(SEARCH_PROVIDER_OPTIONS).toHaveLength(6);
it("exports all 7 providers in SEARCH_PROVIDER_OPTIONS", () => {
expect(SEARCH_PROVIDER_OPTIONS).toHaveLength(7);
const values = SEARCH_PROVIDER_OPTIONS.map((e) => e.value);
expect(values).toEqual(["brave", "gemini", "grok", "kimi", "perplexity", "firecrawl"]);
expect(values).toEqual([
"brave",
"gemini",
"grok",
"kimi",
"perplexity",
"firecrawl",
"tavily",
]);
});
});

View File

@ -53,7 +53,10 @@ function rawKeyValue(config: OpenClawConfig, provider: SearchProvider): unknown
config,
bundledAllowlistCompat: true,
}).find((candidate) => candidate.id === provider);
return entry?.getCredentialValue(search as Record<string, unknown> | undefined);
return (
entry?.getConfiguredCredentialValue?.(config) ??
entry?.getCredentialValue(search as Record<string, unknown> | undefined)
);
}
/** Returns the plaintext key string, or undefined for SecretRefs/missing. */
@ -104,7 +107,7 @@ export function applySearchKey(
bundledAllowlistCompat: true,
}).find((candidate) => candidate.id === provider);
const search: MutableSearchConfig = { ...config.tools?.web?.search, provider, enabled: true };
if (providerEntry) {
if (providerEntry && !providerEntry.setConfiguredCredentialValue) {
providerEntry.setCredentialValue(search, key);
}
const nextBase: OpenClawConfig = {
@ -114,7 +117,9 @@ export function applySearchKey(
web: { ...config.tools?.web, search },
},
};
return providerEntry?.applySelectionConfig?.(nextBase) ?? nextBase;
const next = providerEntry?.applySelectionConfig?.(nextBase) ?? nextBase;
providerEntry?.setConfiguredCredentialValue?.(next, key);
return next;
}
function applyProviderOnly(config: OpenClawConfig, provider: SearchProvider): OpenClawConfig {

View File

@ -59,6 +59,13 @@ vi.mock("../plugins/web-search-providers.js", () => {
getCredentialValue: getScoped("perplexity"),
getConfiguredCredentialValue: getConfigured("perplexity"),
},
{
id: "tavily",
envVars: ["TAVILY_API_KEY"],
credentialPath: "plugins.entries.tavily.config.webSearch.apiKey",
getCredentialValue: getScoped("tavily"),
getConfiguredCredentialValue: getConfigured("tavily"),
},
],
};
});
@ -66,6 +73,17 @@ vi.mock("../plugins/web-search-providers.js", () => {
const { __testing } = await import("../agents/tools/web-search.js");
const { resolveSearchProvider } = __testing;
function pluginWebSearchApiKey(
config: Record<string, unknown> | undefined,
pluginId: string,
): unknown {
return (
config?.plugins as
| { entries?: Record<string, { config?: { webSearch?: { apiKey?: unknown } } }> }
| undefined
)?.entries?.[pluginId]?.config?.webSearch?.apiKey;
}
describe("web search provider config", () => {
it("accepts perplexity provider and config", () => {
const res = validateConfigObjectWithPlugins(
@ -113,6 +131,50 @@ describe("web search provider config", () => {
expect(res.ok).toBe(true);
});
it("accepts tavily provider config on the plugin-owned path", () => {
const res = validateConfigObjectWithPlugins(
buildWebSearchProviderConfig({
enabled: true,
provider: "tavily",
providerConfig: {
apiKey: {
source: "env",
provider: "default",
id: "TAVILY_API_KEY",
},
baseUrl: "https://api.tavily.com",
},
}),
);
expect(res.ok).toBe(true);
});
it("does not migrate the nonexistent legacy Tavily scoped config", () => {
const res = validateConfigObjectWithPlugins({
tools: {
web: {
search: {
provider: "tavily",
tavily: {
apiKey: "tvly-test-key",
},
},
},
},
});
expect(res.ok).toBe(true);
if (!res.ok) {
return;
}
expect(res.config.tools?.web?.search?.provider).toBe("tavily");
expect((res.config.tools?.web?.search as Record<string, unknown> | undefined)?.tavily).toBe(
undefined,
);
expect(pluginWebSearchApiKey(res.config as Record<string, unknown>, "tavily")).toBe(undefined);
});
it("accepts gemini provider with no extra config", () => {
const res = validateConfigObjectWithPlugins(
buildWebSearchProviderConfig({
@ -161,6 +223,7 @@ describe("web search provider auto-detection", () => {
delete process.env.MOONSHOT_API_KEY;
delete process.env.PERPLEXITY_API_KEY;
delete process.env.OPENROUTER_API_KEY;
delete process.env.TAVILY_API_KEY;
delete process.env.XAI_API_KEY;
delete process.env.KIMI_API_KEY;
delete process.env.MOONSHOT_API_KEY;
@ -185,6 +248,11 @@ describe("web search provider auto-detection", () => {
expect(resolveSearchProvider({})).toBe("gemini");
});
it("auto-detects tavily when only TAVILY_API_KEY is set", () => {
process.env.TAVILY_API_KEY = "tvly-test-key"; // pragma: allowlist secret
expect(resolveSearchProvider({})).toBe("tavily");
});
it("auto-detects firecrawl when only FIRECRAWL_API_KEY is set", () => {
process.env.FIRECRAWL_API_KEY = "fc-test-key"; // pragma: allowlist secret
expect(resolveSearchProvider({})).toBe("firecrawl");

View File

@ -2,10 +2,12 @@
export const BUNDLED_PROVIDER_AUTH_ENV_VAR_CANDIDATES = {
anthropic: ["ANTHROPIC_OAUTH_TOKEN", "ANTHROPIC_API_KEY"],
brave: ["BRAVE_API_KEY"],
byteplus: ["BYTEPLUS_API_KEY"],
chutes: ["CHUTES_API_KEY", "CHUTES_OAUTH_TOKEN"],
"cloudflare-ai-gateway": ["CLOUDFLARE_AI_GATEWAY_API_KEY"],
fal: ["FAL_KEY"],
firecrawl: ["FIRECRAWL_API_KEY"],
"github-copilot": ["COPILOT_GITHUB_TOKEN", "GH_TOKEN", "GITHUB_TOKEN"],
google: ["GEMINI_API_KEY", "GOOGLE_API_KEY"],
huggingface: ["HUGGINGFACE_HUB_TOKEN", "HF_TOKEN"],
@ -23,10 +25,12 @@ export const BUNDLED_PROVIDER_AUTH_ENV_VAR_CANDIDATES = {
opencode: ["OPENCODE_API_KEY", "OPENCODE_ZEN_API_KEY"],
"opencode-go": ["OPENCODE_API_KEY", "OPENCODE_ZEN_API_KEY"],
openrouter: ["OPENROUTER_API_KEY"],
perplexity: ["PERPLEXITY_API_KEY", "OPENROUTER_API_KEY"],
qianfan: ["QIANFAN_API_KEY"],
"qwen-portal": ["QWEN_OAUTH_TOKEN", "QWEN_PORTAL_API_KEY"],
sglang: ["SGLANG_API_KEY"],
synthetic: ["SYNTHETIC_API_KEY"],
tavily: ["TAVILY_API_KEY"],
together: ["TOGETHER_API_KEY"],
venice: ["VENICE_API_KEY"],
"vercel-ai-gateway": ["AI_GATEWAY_API_KEY"],

View File

@ -31,15 +31,22 @@ describe("bundled provider auth env vars", () => {
});
it("reads bundled provider auth env vars from plugin manifests", () => {
expect(BUNDLED_PROVIDER_AUTH_ENV_VAR_CANDIDATES.brave).toEqual(["BRAVE_API_KEY"]);
expect(BUNDLED_PROVIDER_AUTH_ENV_VAR_CANDIDATES.firecrawl).toEqual(["FIRECRAWL_API_KEY"]);
expect(BUNDLED_PROVIDER_AUTH_ENV_VAR_CANDIDATES["github-copilot"]).toEqual([
"COPILOT_GITHUB_TOKEN",
"GH_TOKEN",
"GITHUB_TOKEN",
]);
expect(BUNDLED_PROVIDER_AUTH_ENV_VAR_CANDIDATES.perplexity).toEqual([
"PERPLEXITY_API_KEY",
"OPENROUTER_API_KEY",
]);
expect(BUNDLED_PROVIDER_AUTH_ENV_VAR_CANDIDATES["qwen-portal"]).toEqual([
"QWEN_OAUTH_TOKEN",
"QWEN_PORTAL_API_KEY",
]);
expect(BUNDLED_PROVIDER_AUTH_ENV_VAR_CANDIDATES.tavily).toEqual(["TAVILY_API_KEY"]);
expect(BUNDLED_PROVIDER_AUTH_ENV_VAR_CANDIDATES["minimax-portal"]).toEqual([
"MINIMAX_OAUTH_TOKEN",
"MINIMAX_API_KEY",

View File

@ -71,6 +71,7 @@ describe("bundled web search metadata", () => {
"google",
"moonshot",
"perplexity",
"tavily",
"xai",
]);
});

View File

@ -191,6 +191,21 @@ const BUNDLED_WEB_SEARCH_PROVIDER_DESCRIPTORS = [
credentialScope: { kind: "scoped", key: "firecrawl" },
applySelectionConfig: (config) => enablePluginInConfig(config, "firecrawl").config,
},
{
pluginId: "tavily",
id: "tavily",
label: "Tavily Search",
hint: "Structured results with domain filters and AI answer summaries",
envVars: ["TAVILY_API_KEY"],
placeholder: "tvly-...",
signupUrl: "https://tavily.com/",
docsUrl: "https://docs.openclaw.ai/tools/tavily",
autoDetectOrder: 70,
credentialPath: "plugins.entries.tavily.config.webSearch.apiKey",
inactiveSecretPaths: ["plugins.entries.tavily.config.webSearch.apiKey"],
credentialScope: { kind: "scoped", key: "tavily" },
applySelectionConfig: (config) => enablePluginInConfig(config, "tavily").config,
},
] as const satisfies ReadonlyArray<BundledWebSearchProviderDescriptor>;
export const BUNDLED_WEB_SEARCH_PLUGIN_IDS = [

View File

@ -146,6 +146,7 @@ describe("plugin contract registry", () => {
expect(findWebSearchIdsForPlugin("google")).toEqual(["gemini"]);
expect(findWebSearchIdsForPlugin("moonshot")).toEqual(["kimi"]);
expect(findWebSearchIdsForPlugin("perplexity")).toEqual(["perplexity"]);
expect(findWebSearchIdsForPlugin("tavily")).toEqual(["tavily"]);
expect(findWebSearchIdsForPlugin("xai")).toEqual(["grok"]);
});
@ -183,6 +184,14 @@ describe("plugin contract registry", () => {
webSearchProviderIds: ["firecrawl"],
toolNames: ["firecrawl_search", "firecrawl_scrape"],
});
expect(findRegistrationForPlugin("tavily")).toMatchObject({
providerIds: [],
speechProviderIds: [],
mediaUnderstandingProviderIds: [],
imageGenerationProviderIds: [],
webSearchProviderIds: ["tavily"],
toolNames: ["tavily_search", "tavily_extract"],
});
});
it("tracks speech registrations on bundled provider plugins", () => {

View File

@ -29,6 +29,7 @@ import qianfanPlugin from "../../../extensions/qianfan/index.js";
import qwenPortalAuthPlugin from "../../../extensions/qwen-portal-auth/index.js";
import sglangPlugin from "../../../extensions/sglang/index.js";
import syntheticPlugin from "../../../extensions/synthetic/index.js";
import tavilyPlugin from "../../../extensions/tavily/index.js";
import togetherPlugin from "../../../extensions/together/index.js";
import venicePlugin from "../../../extensions/venice/index.js";
import vercelAiGatewayPlugin from "../../../extensions/vercel-ai-gateway/index.js";
@ -84,9 +85,9 @@ const bundledWebSearchPlugins: Array<RegistrablePlugin & { credentialValue: unkn
{ ...googlePlugin, credentialValue: "AIza-test" },
{ ...moonshotPlugin, credentialValue: "sk-test" },
{ ...perplexityPlugin, credentialValue: "pplx-test" },
{ ...tavilyPlugin, credentialValue: "tvly-test" },
{ ...xaiPlugin, credentialValue: "xai-test" },
];
const bundledSpeechPlugins: RegistrablePlugin[] = [elevenLabsPlugin, microsoftPlugin, openAIPlugin];
const bundledMediaUnderstandingPlugins: RegistrablePlugin[] = [

View File

@ -15,6 +15,7 @@ const BUNDLED_WEB_SEARCH_PROVIDERS = [
{ pluginId: "moonshot", id: "kimi", order: 40 },
{ pluginId: "perplexity", id: "perplexity", order: 50 },
{ pluginId: "firecrawl", id: "firecrawl", order: 60 },
{ pluginId: "tavily", id: "tavily", order: 70 },
] as const;
const { loadOpenClawPluginsMock } = vi.hoisted(() => ({
@ -96,6 +97,7 @@ describe("resolvePluginWebSearchProviders", () => {
"moonshot:kimi",
"perplexity:perplexity",
"firecrawl:firecrawl",
"tavily:tavily",
]);
expect(providers.map((provider) => provider.credentialPath)).toEqual([
"plugins.entries.brave.config.webSearch.apiKey",
@ -104,6 +106,7 @@ describe("resolvePluginWebSearchProviders", () => {
"plugins.entries.moonshot.config.webSearch.apiKey",
"plugins.entries.perplexity.config.webSearch.apiKey",
"plugins.entries.firecrawl.config.webSearch.apiKey",
"plugins.entries.tavily.config.webSearch.apiKey",
]);
expect(providers.find((provider) => provider.id === "firecrawl")?.applySelectionConfig).toEqual(
expect.any(Function),
@ -130,6 +133,7 @@ describe("resolvePluginWebSearchProviders", () => {
"moonshot",
"perplexity",
"firecrawl",
"tavily",
]);
});
@ -183,6 +187,7 @@ describe("resolvePluginWebSearchProviders", () => {
"moonshot:kimi",
"perplexity:perplexity",
"firecrawl:firecrawl",
"tavily:tavily",
]);
expect(loadOpenClawPluginsMock).not.toHaveBeenCalled();
});

View File

@ -8,10 +8,28 @@ import {
describe("provider env vars", () => {
it("keeps the auth scrub list broader than the global secret env list", () => {
expect(listKnownProviderAuthEnvVarNames()).toEqual(
expect.arrayContaining(["GITHUB_TOKEN", "GH_TOKEN", "ANTHROPIC_OAUTH_TOKEN"]),
expect.arrayContaining([
"GITHUB_TOKEN",
"GH_TOKEN",
"ANTHROPIC_OAUTH_TOKEN",
"BRAVE_API_KEY",
"FIRECRAWL_API_KEY",
"PERPLEXITY_API_KEY",
"OPENROUTER_API_KEY",
"TAVILY_API_KEY",
]),
);
expect(listKnownSecretEnvVarNames()).toEqual(
expect.arrayContaining(["GITHUB_TOKEN", "GH_TOKEN", "ANTHROPIC_OAUTH_TOKEN"]),
expect.arrayContaining([
"GITHUB_TOKEN",
"GH_TOKEN",
"ANTHROPIC_OAUTH_TOKEN",
"BRAVE_API_KEY",
"FIRECRAWL_API_KEY",
"PERPLEXITY_API_KEY",
"OPENROUTER_API_KEY",
"TAVILY_API_KEY",
]),
);
expect(listKnownProviderAuthEnvVarNames()).toEqual(
expect.arrayContaining(["MINIMAX_CODE_PLAN_KEY"]),

View File

@ -843,6 +843,17 @@ const SECRET_TARGET_REGISTRY: SecretTargetRegistryEntry[] = [
includeInConfigure: true,
includeInAudit: true,
},
{
id: "plugins.entries.tavily.config.webSearch.apiKey",
targetType: "plugins.entries.tavily.config.webSearch.apiKey",
configFile: "openclaw.json",
pathPattern: "plugins.entries.tavily.config.webSearch.apiKey",
secretShape: SECRET_INPUT_SHAPE,
expectedResolvedValue: "string",
includeInPlan: true,
includeInConfigure: true,
includeInAudit: true,
},
];
export { SECRET_TARGET_REGISTRY };

View File

@ -1,8 +1,15 @@
import { afterEach, describe, expect, it } from "vitest";
import type { OpenClawConfig } from "../config/config.js";
import { createEmptyPluginRegistry } from "../plugins/registry.js";
import { setActivePluginRegistry } from "../plugins/runtime.js";
import { runWebSearch } from "./runtime.js";
type TestPluginWebSearchConfig = {
webSearch?: {
apiKey?: unknown;
};
};
describe("web search runtime", () => {
afterEach(() => {
setActivePluginRegistry(createEmptyPluginRegistry());
@ -44,4 +51,74 @@ describe("web search runtime", () => {
result: { query: "hello", ok: true },
});
});
it("auto-detects a provider from canonical plugin-owned credentials", async () => {
const registry = createEmptyPluginRegistry();
registry.webSearchProviders.push({
pluginId: "custom-search",
pluginName: "Custom Search",
provider: {
id: "custom",
label: "Custom Search",
hint: "Custom runtime provider",
envVars: ["CUSTOM_SEARCH_API_KEY"],
placeholder: "custom-...",
signupUrl: "https://example.com/signup",
credentialPath: "plugins.entries.custom-search.config.webSearch.apiKey",
autoDetectOrder: 1,
getCredentialValue: () => undefined,
setCredentialValue: () => {},
getConfiguredCredentialValue: (config) => {
const pluginConfig = config?.plugins?.entries?.["custom-search"]?.config as
| TestPluginWebSearchConfig
| undefined;
return pluginConfig?.webSearch?.apiKey;
},
setConfiguredCredentialValue: (configTarget, value) => {
configTarget.plugins = {
...configTarget.plugins,
entries: {
...configTarget.plugins?.entries,
"custom-search": {
enabled: true,
config: { webSearch: { apiKey: value } },
},
},
};
},
createTool: () => ({
description: "custom",
parameters: {},
execute: async (args) => ({ ...args, ok: true }),
}),
},
source: "test",
});
setActivePluginRegistry(registry);
const config: OpenClawConfig = {
plugins: {
entries: {
"custom-search": {
enabled: true,
config: {
webSearch: {
apiKey: "custom-config-key",
},
},
},
},
},
};
await expect(
runWebSearch({
config,
args: { query: "hello" },
}),
).resolves.toEqual({
provider: "custom",
result: { query: "hello", ok: true },
});
});
});