diff --git a/.agents/archive/PR_WORKFLOW_V1.md b/.agents/archive/PR_WORKFLOW_V1.md deleted file mode 100644 index 1cb6ab653b5..00000000000 --- a/.agents/archive/PR_WORKFLOW_V1.md +++ /dev/null @@ -1,181 +0,0 @@ -# PR Workflow for Maintainers - -Please read this in full and do not skip sections. -This is the single source of truth for the maintainer PR workflow. - -## Triage order - -Process PRs **oldest to newest**. Older PRs are more likely to have merge conflicts and stale dependencies; resolving them first keeps the queue healthy and avoids snowballing rebase pain. - -## Working rule - -Skills execute workflow. Maintainers provide judgment. -Always pause between skills to evaluate technical direction, not just command success. - -These three skills must be used in order: - -1. `review-pr` — review only, produce findings -2. `prepare-pr` — rebase, fix, gate, push to PR head branch -3. `merge-pr` — squash-merge, verify MERGED state, clean up - -They are necessary, but not sufficient. Maintainers must steer between steps and understand the code before moving forward. - -Treat PRs as reports first, code second. -If submitted code is low quality, ignore it and implement the best solution for the problem. - -Do not continue if you cannot verify the problem is real or test the fix. - -## Coding Agent - -Use ChatGPT 5.3 Codex High. Fall back to 5.2 Codex High or 5.3 Codex Medium if necessary. - -## PR quality bar - -- Do not trust PR code by default. -- Do not merge changes you cannot validate with a reproducible problem and a tested fix. -- Keep types strict. Do not use `any` in implementation code. -- Keep external-input boundaries typed and validated, including CLI input, environment variables, network payloads, and tool output. -- Keep implementations properly scoped. Fix root causes, not local symptoms. -- Identify and reuse canonical sources of truth so behavior does not drift across the codebase. -- Harden changes. Always evaluate security impact and abuse paths. -- Understand the system before changing it. Never make the codebase messier just to clear a PR queue. - -## Rebase and conflict resolution - -Before any substantive review or prep work, **always rebase the PR branch onto current `main` and resolve merge conflicts first**. A PR that cannot cleanly rebase is not ready for review — fix conflicts before evaluating correctness. - -- During `prepare-pr`: rebase onto `main` as the first step, before fixing findings or running gates. -- If conflicts are complex or touch areas you do not understand, stop and escalate. -- Prefer **rebase** for linear history; **squash** when commit history is messy or unhelpful. - -## Commit and changelog rules - -- Create commits with `scripts/committer "" `; avoid manual `git add`/`git commit` so staging stays scoped. -- Follow concise, action-oriented commit messages (e.g., `CLI: add verbose flag to send`). -- During `prepare-pr`, use this commit subject format: `fix: (openclaw#) thanks @`. -- Group related changes; avoid bundling unrelated refactors. -- Changelog workflow: keep the latest released version at the top (no `Unreleased`); after publishing, bump the version and start a new top section. -- When working on a PR: add a changelog entry with the PR number and thank the contributor. -- When working on an issue: reference the issue in the changelog entry. -- Pure test additions/fixes generally do **not** need a changelog entry unless they alter user-facing behavior or the user asks for one. - -## Co-contributor and clawtributors - -- If we squash, add the PR author as a co-contributor in the commit body using a `Co-authored-by:` trailer. -- When maintainer prepares and merges the PR, add the maintainer as an additional `Co-authored-by:` trailer too. -- Avoid `--auto` merges for maintainer landings. Merge only after checks are green so the maintainer account is the actor and attribution is deterministic. -- For squash merges, set `--author-email` to a reviewer-owned email with fallback candidates; if merge fails due to author-email validation, retry once with the next candidate. -- If you review a PR and later do work on it, land via merge/squash (no direct-main commits) and always add the PR author as a co-contributor. -- When merging a PR: leave a PR comment that explains exactly what we did, include the SHA hashes, and record the comment URL in the final report. -- When merging a PR from a new contributor: run `bun scripts/update-clawtributors.ts` to add their avatar to the README "Thanks to all clawtributors" list, then commit the regenerated README. - -## Review mode vs landing mode - -- **Review mode (PR link only):** read `gh pr view`/`gh pr diff`; **do not** switch branches; **do not** change code. -- **Landing mode (exception path):** use only when normal `review-pr -> prepare-pr -> merge-pr` flow cannot safely preserve attribution or cannot satisfy branch protection. Create an integration branch from `main`, bring in PR commits (**prefer rebase** for linear history; **merge allowed** when complexity/conflicts make it safer), apply fixes, add changelog (+ thanks + PR #), run full gate **locally before committing** (`pnpm build && pnpm check && pnpm test`), commit, merge back to `main`, then `git switch main` (never stay on a topic branch after landing). Important: the contributor needs to be in the git graph after this! - -## Pre-review safety checks - -- Before starting a review when a GH Issue/PR is pasted: use an isolated `.worktrees/pr-` checkout from `origin/main`. Do not require a clean main checkout, and do not run `git pull` in a dirty main checkout. -- PR review calls: prefer a single `gh pr view --json ...` to batch metadata/comments; run `gh pr diff` only when needed. -- PRs should summarize scope, note testing performed, and mention any user-facing changes or new flags. -- Read `docs/help/submitting-a-pr.md` ([Submitting a PR](https://docs.openclaw.ai/help/submitting-a-pr)) for what we expect from contributors. - -## Unified workflow - -Entry criteria: - -- PR URL/number is known. -- Problem statement is clear enough to attempt reproduction. -- A realistic verification path exists (tests, integration checks, or explicit manual validation). - -### 1) `review-pr` - -Purpose: - -- Review only: correctness, value, security risk, tests, docs, and changelog impact. -- Produce structured findings and a recommendation. - -Expected output: - -- Recommendation: ready, needs work, needs discussion, or close. -- `.local/review.md` with actionable findings. - -Maintainer checkpoint before `prepare-pr`: - -``` -What problem are they trying to solve? -What is the most optimal implementation? -Can we fix up everything? -Do we have any questions? -``` - -Stop and escalate instead of continuing if: - -- The problem cannot be reproduced or confirmed. -- The proposed PR scope does not match the stated problem. -- The design introduces unresolved security or trust-boundary concerns. - -### 2) `prepare-pr` - -Purpose: - -- Make the PR merge-ready on its head branch. -- Rebase onto current `main` first, then fix blocker/important findings, then run gates. -- In fresh worktrees, bootstrap dependencies before local gates (`pnpm install --frozen-lockfile`). - -Expected output: - -- Updated code and tests on the PR head branch. -- `.local/prep.md` with changes, verification, and current HEAD SHA. -- Final status: `PR is ready for /mergepr`. - -Maintainer checkpoint before `merge-pr`: - -``` -Is this the most optimal implementation? -Is the code properly scoped? -Is the code properly reusing existing logic in the codebase? -Is the code properly typed? -Is the code hardened? -Do we have enough tests? -Do we need regression tests? -Are tests using fake timers where appropriate? (e.g., debounce/throttle, retry backoff, timeout branches, delayed callbacks, polling loops) -Do not add performative tests, ensure tests are real and there are no regressions. -Do you see any follow-up refactors we should do? -Take your time, fix it properly, refactor if necessary. -Did any changes introduce any potential security vulnerabilities? -``` - -Stop and escalate instead of continuing if: - -- You cannot verify behavior changes with meaningful tests or validation. -- Fixing findings requires broad architecture changes outside safe PR scope. -- Security hardening requirements remain unresolved. - -### 3) `merge-pr` - -Purpose: - -- Merge only after review and prep artifacts are present and checks are green. -- Use deterministic squash merge flow (`--match-head-commit` + explicit subject/body with co-author trailer), then verify the PR ends in `MERGED` state. -- If no required checks are configured on the PR, treat that as acceptable and continue after branch-up-to-date validation. - -Go or no-go checklist before merge: - -- All BLOCKER and IMPORTANT findings are resolved. -- Verification is meaningful and regression risk is acceptably low. -- Docs and changelog are updated when required. -- Required CI checks are green and the branch is not behind `main`. - -Expected output: - -- Successful merge commit and recorded merge SHA. -- Worktree cleanup after successful merge. -- Comment on PR indicating merge was successful. - -Maintainer checkpoint after merge: - -- Were any refactors intentionally deferred and now need follow-up issue(s)? -- Did this reveal broader architecture or test gaps we should address? -- Run `bun scripts/update-clawtributors.ts` if the contributor is new. diff --git a/.agents/archive/merge-pr-v1/SKILL.md b/.agents/archive/merge-pr-v1/SKILL.md deleted file mode 100644 index 0956699eb55..00000000000 --- a/.agents/archive/merge-pr-v1/SKILL.md +++ /dev/null @@ -1,304 +0,0 @@ ---- -name: merge-pr -description: Merge a GitHub PR via squash after /prepare-pr. Use when asked to merge a ready PR. Do not push to main or modify code. Ensure the PR ends in MERGED state and clean up worktrees after success. ---- - -# Merge PR - -## Overview - -Merge a prepared PR via deterministic squash merge (`--match-head-commit` + explicit co-author trailer), then clean up the worktree after success. - -## Inputs - -- Ask for PR number or URL. -- If missing, use `.local/prep.env` from the worktree if present. -- If ambiguous, ask. - -## Safety - -- Use `gh pr merge --squash` as the only path to `main`. -- Do not run `git push` at all during merge. -- Do not use `gh pr merge --auto` for maintainer landings. -- Do not run gateway stop commands. Do not kill processes. Do not touch port 18792. - -## Execution Rule - -- Execute the workflow. Do not stop after printing the TODO checklist. -- If delegating, require the delegate to run commands and capture outputs. - -## Known Footguns - -- If you see "fatal: not a git repository", you are in the wrong directory. Move to the repo root and retry. -- Read `.local/review.md`, `.local/prep.md`, and `.local/prep.env` in the worktree. Do not skip. -- Always merge with `--match-head-commit "$PREP_HEAD_SHA"` to prevent racing stale or changed heads. -- Clean up `.worktrees/pr-` only after confirmed `MERGED`. - -## Completion Criteria - -- Ensure `gh pr merge` succeeds. -- Ensure PR state is `MERGED`, never `CLOSED`. -- Record the merge SHA. -- Leave a PR comment with merge SHA and prepared head SHA, and capture the comment URL. -- Run cleanup only after merge success. - -## First: Create a TODO Checklist - -Create a checklist of all merge steps, print it, then continue and execute the commands. - -## Setup: Use a Worktree - -Use an isolated worktree for all merge work. - -```sh -repo_root=$(git rev-parse --show-toplevel) -cd "$repo_root" -gh auth status - -WORKTREE_DIR=".worktrees/pr-" -cd "$WORKTREE_DIR" -``` - -Run all commands inside the worktree directory. - -## Load Local Artifacts (Mandatory) - -Expect these files from earlier steps: - -- `.local/review.md` from `/review-pr` -- `.local/prep.md` from `/prepare-pr` -- `.local/prep.env` from `/prepare-pr` - -```sh -ls -la .local || true - -for required in .local/review.md .local/prep.md .local/prep.env; do - if [ ! -f "$required" ]; then - echo "Missing $required. Stop and run /review-pr then /prepare-pr." - exit 1 - fi -done - -sed -n '1,120p' .local/review.md -sed -n '1,120p' .local/prep.md -source .local/prep.env -``` - -## Steps - -1. Identify PR meta and verify prepared SHA still matches - -```sh -pr_meta_json=$(gh pr view --json number,title,state,isDraft,author,headRefName,headRefOid,baseRefName,headRepository,body) -printf '%s\n' "$pr_meta_json" | jq '{number,title,state,isDraft,author:.author.login,head:.headRefName,headSha:.headRefOid,base:.baseRefName,headRepo:.headRepository.nameWithOwner,body}' -pr_title=$(printf '%s\n' "$pr_meta_json" | jq -r .title) -pr_number=$(printf '%s\n' "$pr_meta_json" | jq -r .number) -pr_head_sha=$(printf '%s\n' "$pr_meta_json" | jq -r .headRefOid) -contrib=$(printf '%s\n' "$pr_meta_json" | jq -r .author.login) -is_draft=$(printf '%s\n' "$pr_meta_json" | jq -r .isDraft) - -if [ "$is_draft" = "true" ]; then - echo "ERROR: PR is draft. Stop and run /prepare-pr after draft is cleared." - exit 1 -fi - -if [ "$pr_head_sha" != "$PREP_HEAD_SHA" ]; then - echo "ERROR: PR head changed after /prepare-pr (expected $PREP_HEAD_SHA, got $pr_head_sha). Re-run /prepare-pr." - exit 1 -fi -``` - -2. Run sanity checks - -Stop if any are true: - -- PR is a draft. -- Required checks are failing. -- Branch is behind main. - -If checks are pending, wait for completion before merging. Do not use `--auto`. -If no required checks are configured, continue. - -```sh -gh pr checks --required --watch --fail-fast || true -checks_json=$(gh pr checks --required --json name,bucket,state 2>/tmp/gh-checks.err || true) -if [ -z "$checks_json" ]; then - checks_json='[]' -fi -required_count=$(printf '%s\n' "$checks_json" | jq 'length') -if [ "$required_count" -eq 0 ]; then - echo "No required checks configured for this PR." -fi -printf '%s\n' "$checks_json" | jq -r '.[] | "\(.bucket)\t\(.name)\t\(.state)"' - -failed_required=$(printf '%s\n' "$checks_json" | jq '[.[] | select(.bucket=="fail")] | length') -pending_required=$(printf '%s\n' "$checks_json" | jq '[.[] | select(.bucket=="pending")] | length') -if [ "$failed_required" -gt 0 ]; then - echo "Required checks are failing, run /prepare-pr." - exit 1 -fi -if [ "$pending_required" -gt 0 ]; then - echo "Required checks are still pending, retry /merge-pr when green." - exit 1 -fi - -git fetch origin main -git fetch origin pull//head:pr- --force -git merge-base --is-ancestor origin/main pr- || (echo "PR branch is behind main, run /prepare-pr" && exit 1) -``` - -If anything is failing or behind, stop and say to run `/prepare-pr`. - -3. Merge PR with explicit attribution metadata - -```sh -reviewer=$(gh api user --jq .login) -reviewer_id=$(gh api user --jq .id) -coauthor_email=${COAUTHOR_EMAIL:-"$contrib@users.noreply.github.com"} -if [ -z "$coauthor_email" ] || [ "$coauthor_email" = "null" ]; then - contrib_id=$(gh api users/$contrib --jq .id) - coauthor_email="${contrib_id}+${contrib}@users.noreply.github.com" -fi - -gh_email=$(gh api user --jq '.email // ""' || true) -git_email=$(git config user.email || true) -mapfile -t reviewer_email_candidates < <( - printf '%s\n' \ - "$gh_email" \ - "$git_email" \ - "${reviewer_id}+${reviewer}@users.noreply.github.com" \ - "${reviewer}@users.noreply.github.com" | awk 'NF && !seen[$0]++' -) -[ "${#reviewer_email_candidates[@]}" -gt 0 ] || { echo "ERROR: could not resolve reviewer author email"; exit 1; } -reviewer_email="${reviewer_email_candidates[0]}" - -cat > .local/merge-body.txt < /prepare-pr -> /merge-pr. - -Prepared head SHA: $PREP_HEAD_SHA -Co-authored-by: $contrib <$coauthor_email> -Co-authored-by: $reviewer <$reviewer_email> -Reviewed-by: @$reviewer -EOF - -run_merge() { - local email="$1" - local stderr_file - stderr_file=$(mktemp) - if gh pr merge \ - --squash \ - --delete-branch \ - --match-head-commit "$PREP_HEAD_SHA" \ - --author-email "$email" \ - --subject "$pr_title (#$pr_number)" \ - --body-file .local/merge-body.txt \ - 2> >(tee "$stderr_file" >&2) - then - rm -f "$stderr_file" - return 0 - fi - merge_err=$(cat "$stderr_file") - rm -f "$stderr_file" - return 1 -} - -merge_err="" -selected_merge_author_email="$reviewer_email" -if ! run_merge "$selected_merge_author_email"; then - if printf '%s\n' "$merge_err" | rg -qi 'author.?email|email.*associated|associated.*email|invalid.*email' && [ "${#reviewer_email_candidates[@]}" -ge 2 ]; then - selected_merge_author_email="${reviewer_email_candidates[1]}" - echo "Retrying once with fallback author email: $selected_merge_author_email" - run_merge "$selected_merge_author_email" || { echo "ERROR: merge failed after fallback retry"; exit 1; } - else - echo "ERROR: merge failed" - exit 1 - fi -fi -``` - -Retry is allowed exactly once when the error is clearly author-email validation. - -4. Verify PR state and capture merge SHA - -```sh -state=$(gh pr view --json state --jq .state) -if [ "$state" != "MERGED" ]; then - echo "Merge not finalized yet (state=$state), waiting up to 15 minutes..." - for _ in $(seq 1 90); do - sleep 10 - state=$(gh pr view --json state --jq .state) - if [ "$state" = "MERGED" ]; then - break - fi - done -fi - -if [ "$state" != "MERGED" ]; then - echo "ERROR: PR state is $state after waiting. Leave worktree and retry /merge-pr later." - exit 1 -fi - -merge_sha=$(gh pr view --json mergeCommit --jq '.mergeCommit.oid') -if [ -z "$merge_sha" ] || [ "$merge_sha" = "null" ]; then - echo "ERROR: merge commit SHA missing." - exit 1 -fi - -commit_body=$(gh api repos/:owner/:repo/commits/$merge_sha --jq .commit.message) -contrib=${contrib:-$(gh pr view --json author --jq .author.login)} -reviewer=${reviewer:-$(gh api user --jq .login)} -printf '%s\n' "$commit_body" | rg -q "^Co-authored-by: $contrib <" || { echo "ERROR: missing PR author co-author trailer"; exit 1; } -printf '%s\n' "$commit_body" | rg -q "^Co-authored-by: $reviewer <" || { echo "ERROR: missing reviewer co-author trailer"; exit 1; } - -echo "merge_sha=$merge_sha" -``` - -5. PR comment - -Use a multiline heredoc with interpolation enabled. - -```sh -ok=0 -comment_output="" -for _ in 1 2 3; do - if comment_output=$(gh pr comment -F - <" --force -git branch -D temp/pr- 2>/dev/null || true -git branch -D pr- 2>/dev/null || true -git branch -D pr--prep 2>/dev/null || true -``` - -## Guardrails - -- Worktree only. -- Do not close PRs. -- End in MERGED state. -- Clean up only after merge success. -- Never push to main. Use `gh pr merge --squash` only. -- Do not run `git push` at all in this command. diff --git a/.agents/archive/merge-pr-v1/agents/openai.yaml b/.agents/archive/merge-pr-v1/agents/openai.yaml deleted file mode 100644 index 9c10ae4d271..00000000000 --- a/.agents/archive/merge-pr-v1/agents/openai.yaml +++ /dev/null @@ -1,4 +0,0 @@ -interface: - display_name: "Merge PR" - short_description: "Merge GitHub PRs via squash" - default_prompt: "Use $merge-pr to merge a GitHub PR via squash after preparation." diff --git a/.agents/archive/prepare-pr-v1/SKILL.md b/.agents/archive/prepare-pr-v1/SKILL.md deleted file mode 100644 index 91c4508a07a..00000000000 --- a/.agents/archive/prepare-pr-v1/SKILL.md +++ /dev/null @@ -1,336 +0,0 @@ ---- -name: prepare-pr -description: Prepare a GitHub PR for merge by rebasing onto main, fixing review findings, running gates, committing fixes, and pushing to the PR head branch. Use after /review-pr. Never merge or push to main. ---- - -# Prepare PR - -## Overview - -Prepare a PR head branch for merge with review fixes, green gates, and deterministic merge handoff artifacts. - -## Inputs - -- Ask for PR number or URL. -- If missing, use `.local/pr-meta.env` from the PR worktree if present. -- If ambiguous, ask. - -## Safety - -- Never push to `main` or `origin/main`. Push only to the PR head branch. -- Never run `git push` without explicit remote and branch. Do not run bare `git push`. -- Do not run gateway stop commands. Do not kill processes. Do not touch port 18792. -- Do not run `git clean -fdx`. -- Do not run `git add -A` or `git add .`. - -## Execution Rule - -- Execute the workflow. Do not stop after printing the TODO checklist. -- If delegating, require the delegate to run commands and capture outputs. - -## Completion Criteria - -- Rebase PR commits onto `origin/main`. -- Fix all BLOCKER and IMPORTANT items from `.local/review.md`. -- Commit prep changes with required subject format. -- Run required gates and pass (`pnpm test` may be skipped only for high-confidence docs-only changes). -- Push the updated HEAD back to the PR head branch. -- Write `.local/prep.md` and `.local/prep.env`. -- Output exactly: `PR is ready for /mergepr`. - -## First: Create a TODO Checklist - -Create a checklist of all prep steps, print it, then continue and execute the commands. - -## Setup: Use a Worktree - -Use an isolated worktree for all prep work. - -```sh -repo_root=$(git rev-parse --show-toplevel) -cd "$repo_root" -gh auth status - -WORKTREE_DIR=".worktrees/pr-" -if [ ! -d "$WORKTREE_DIR" ]; then - git fetch origin main - git worktree add "$WORKTREE_DIR" -b temp/pr- origin/main -fi -cd "$WORKTREE_DIR" -mkdir -p .local -``` - -Run all commands inside the worktree directory. - -## Load Review Artifacts (Mandatory) - -```sh -if [ ! -f .local/review.md ]; then - echo "Missing .local/review.md. Run /review-pr first and save findings." - exit 1 -fi - -if [ ! -f .local/pr-meta.env ]; then - echo "Missing .local/pr-meta.env. Run /review-pr first and save metadata." - exit 1 -fi - -sed -n '1,220p' .local/review.md -source .local/pr-meta.env -``` - -## Steps - -1. Identify PR meta with one API call - -```sh -pr_meta_json=$(gh pr view --json number,title,author,headRefName,headRefOid,baseRefName,headRepository,headRepositoryOwner,body) -printf '%s\n' "$pr_meta_json" | jq '{number,title,author:.author.login,head:.headRefName,headSha:.headRefOid,base:.baseRefName,headRepo:.headRepository.nameWithOwner,headRepoOwner:.headRepositoryOwner.login,headRepoName:.headRepository.name,body}' - -pr_number=$(printf '%s\n' "$pr_meta_json" | jq -r .number) -contrib=$(printf '%s\n' "$pr_meta_json" | jq -r .author.login) -head=$(printf '%s\n' "$pr_meta_json" | jq -r .headRefName) -pr_head_sha_before=$(printf '%s\n' "$pr_meta_json" | jq -r .headRefOid) -head_owner=$(printf '%s\n' "$pr_meta_json" | jq -r '.headRepositoryOwner.login // empty') -head_repo_name=$(printf '%s\n' "$pr_meta_json" | jq -r '.headRepository.name // empty') -head_repo_url=$(printf '%s\n' "$pr_meta_json" | jq -r '.headRepository.url // empty') - -if [ -n "${PR_HEAD:-}" ] && [ "$head" != "$PR_HEAD" ]; then - echo "ERROR: PR head branch changed from $PR_HEAD to $head. Re-run /review-pr." - exit 1 -fi -``` - -2. Fetch PR head and rebase on latest `origin/main` - -```sh -git fetch origin pull//head:pr- --force -git checkout -B pr--prep pr- -git fetch origin main -git rebase origin/main -``` - -If conflicts happen: - -- Resolve each conflicted file. -- Run `git add ` for each file. -- Run `git rebase --continue`. - -If the rebase gets confusing or you resolve conflicts 3 or more times, stop and report. - -3. Fix issues from `.local/review.md` - -- Fix all BLOCKER and IMPORTANT items. -- NITs are optional. -- Keep scope tight. - -Keep a running log in `.local/prep.md`: - -- List which review items you fixed. -- List which files you touched. -- Note behavior changes. - -4. Optional quick feedback tests before full gates - -Targeted tests are optional quick feedback, not a substitute for full gates. - -If running targeted tests in a fresh worktree: - -```sh -if [ ! -x node_modules/.bin/vitest ]; then - pnpm install --frozen-lockfile -fi -``` - -5. Commit prep fixes with required subject format - -Use `scripts/committer` with explicit file paths. - -Required subject format: - -- `fix: (openclaw#) thanks @` - -```sh -commit_msg="fix: (openclaw#$pr_number) thanks @$contrib" -scripts/committer "$commit_msg" ... -``` - -If there are no local changes, do not create a no-op commit. - -Post-commit validation (mandatory): - -```sh -subject=$(git log -1 --pretty=%s) -echo "$subject" | rg -q "openclaw#$pr_number" || { echo "ERROR: commit subject missing openclaw#$pr_number"; exit 1; } -echo "$subject" | rg -q "thanks @$contrib" || { echo "ERROR: commit subject missing thanks @$contrib"; exit 1; } -``` - -6. Decide verification mode and run required gates before pushing - -If you are highly confident the change is docs-only, you may skip `pnpm test`. - -High-confidence docs-only criteria (all must be true): - -- Every changed file is documentation-only (`docs/**`, `README*.md`, `CHANGELOG.md`, `*.md`, `*.mdx`, `mintlify.json`, `docs.json`). -- No code, runtime, test, dependency, or build config files changed (`src/**`, `extensions/**`, `apps/**`, `package.json`, lockfiles, TS/JS config, test files, scripts). -- `.local/review.md` does not call for non-doc behavior fixes. - -Suggested check: - -```sh -changed_files=$(git diff --name-only origin/main...HEAD) -non_docs=$(printf "%s\n" "$changed_files" | grep -Ev '^(docs/|README.*\.md$|CHANGELOG\.md$|.*\.md$|.*\.mdx$|mintlify\.json$|docs\.json$)' || true) - -docs_only=false -if [ -n "$changed_files" ] && [ -z "$non_docs" ]; then - docs_only=true -fi - -echo "docs_only=$docs_only" -``` - -Bootstrap dependencies in a fresh worktree before gates: - -```sh -if [ ! -d node_modules ]; then - pnpm install --frozen-lockfile -fi -``` - -Run required gates: - -```sh -pnpm build -pnpm check - -if [ "$docs_only" = "true" ]; then - echo "Docs-only change detected with high confidence; skipping pnpm test." | tee -a .local/prep.md -else - pnpm test -fi -``` - -Require all required gates to pass. If something fails, fix, commit, and rerun. Allow at most 3 fix-and-rerun cycles. - -7. Push safely to the PR head branch - -Build `prhead` from owner/name first, then validate remote branch SHA before push. - -```sh -if [ -n "$head_owner" ] && [ -n "$head_repo_name" ]; then - head_repo_push_url="https://github.com/$head_owner/$head_repo_name.git" -elif [ -n "$head_repo_url" ] && [ "$head_repo_url" != "null" ]; then - case "$head_repo_url" in - *.git) head_repo_push_url="$head_repo_url" ;; - *) head_repo_push_url="$head_repo_url.git" ;; - esac -else - echo "ERROR: unable to determine PR head repo push URL" - exit 1 -fi - -git remote add prhead "$head_repo_push_url" 2>/dev/null || git remote set-url prhead "$head_repo_push_url" - -echo "Pushing to branch: $head" -if [ "$head" = "main" ] || [ "$head" = "master" ]; then - echo "ERROR: head branch is main/master. This is wrong. Stopping." - exit 1 -fi - -remote_sha=$(git ls-remote prhead "refs/heads/$head" | awk '{print $1}') -if [ -z "$remote_sha" ]; then - echo "ERROR: remote branch refs/heads/$head not found on prhead" - exit 1 -fi -if [ "$remote_sha" != "$pr_head_sha_before" ]; then - echo "ERROR: expected remote SHA $pr_head_sha_before, got $remote_sha. Re-fetch metadata and rebase first." - exit 1 -fi - -git push --force-with-lease=refs/heads/$head:$pr_head_sha_before prhead HEAD:$head || push_failed=1 -``` - -If lease push fails because head moved, perform one automatic retry: - -```sh -if [ "${push_failed:-0}" = "1" ]; then - echo "Lease push failed, retrying once with fresh PR head..." - - pr_head_sha_before=$(gh pr view --json headRefOid --jq .headRefOid) - git fetch origin pull//head:pr--latest --force - git rebase pr--latest - - pnpm build - pnpm check - if [ "$docs_only" != "true" ]; then - pnpm test - fi - - git push --force-with-lease=refs/heads/$head:$pr_head_sha_before prhead HEAD:$head -fi -``` - -8. Verify PR head and base relation (Mandatory) - -```sh -prep_head_sha=$(git rev-parse HEAD) -pr_head_sha_after=$(gh pr view --json headRefOid --jq .headRefOid) - -if [ "$prep_head_sha" != "$pr_head_sha_after" ]; then - echo "ERROR: pushed head SHA does not match PR head SHA." - exit 1 -fi - -git fetch origin main -git fetch origin pull//head:pr--verify --force -git merge-base --is-ancestor origin/main pr--verify && echo "PR is up to date with main" || (echo "ERROR: PR is still behind main, rebase again" && exit 1) -git branch -D pr--verify 2>/dev/null || true -``` - -9. Write prep summary artifacts (Mandatory) - -Write `.local/prep.md` and `.local/prep.env` for merge handoff. - -```sh -contrib_id=$(gh api users/$contrib --jq .id) -coauthor_email="${contrib_id}+${contrib}@users.noreply.github.com" - -cat > .local/prep.env < origin/main -else - git worktree add "$WORKTREE_DIR" -b temp/pr- origin/main - cd "$WORKTREE_DIR" -fi - -# Create local scratch space that persists across /review-pr to /prepare-pr to /merge-pr -mkdir -p .local -``` - -Run all commands inside the worktree directory. -Start on `origin/main` so you can check for existing implementations before looking at PR code. - -## Steps - -1. Identify PR meta and context - -```sh -pr_meta_json=$(gh pr view --json number,title,state,isDraft,author,baseRefName,headRefName,headRefOid,headRepository,url,body,labels,assignees,reviewRequests,files,additions,deletions,statusCheckRollup) -printf '%s\n' "$pr_meta_json" | jq '{number,title,url,state,isDraft,author:.author.login,base:.baseRefName,head:.headRefName,headSha:.headRefOid,headRepo:.headRepository.nameWithOwner,additions,deletions,files:(.files|length),body}' - -cat > .local/pr-meta.env <" -S src packages apps ui || true -rg -n "" -S src packages apps ui || true - -git log --oneline --all --grep="" | head -20 -``` - -If it already exists, call it out as a BLOCKER or at least IMPORTANT. - -3. Claim the PR - -Assign yourself so others know someone is reviewing. Skip if the PR looks like spam or is a draft you plan to recommend closing. - -```sh -gh_user=$(gh api user --jq .login) -gh pr edit --add-assignee "$gh_user" || echo "Could not assign reviewer, continuing" -``` - -4. Read the PR description carefully - -Use the body from step 1. Summarize goal, scope, and missing context. - -5. Read the diff thoroughly - -Minimum: - -```sh -gh pr diff -``` - -If you need full code context locally, fetch the PR head to a local ref and diff it. Do not create a merge commit. - -```sh -git fetch origin pull//head:pr- --force -mb=$(git merge-base origin/main pr-) - -# Show only this PR patch relative to merge-base, not total branch drift -git diff --stat "$mb"..pr- -git diff "$mb"..pr- -``` - -If you want to browse the PR version of files directly, temporarily check out `pr-` in the worktree. Do not commit or push. Return to `temp/pr-` and reset to `origin/main` afterward. - -```sh -# Use only if needed -# git checkout pr- -# git branch --show-current -# ...inspect files... - -git checkout temp/pr- -git checkout -B temp/pr- origin/main -git branch --show-current -``` - -6. Validate the change is needed and valuable - -Be honest. Call out low value AI slop. - -7. Evaluate implementation quality - -Review correctness, design, performance, and ergonomics. - -8. Perform a security review - -Assume OpenClaw subagents run with full disk access, including git, gh, and shell. Check auth, input validation, secrets, dependencies, tool safety, and privacy. - -9. Review tests and verification - -Identify what exists, what is missing, and what would be a minimal regression test. - -If you run local tests in the worktree, bootstrap dependencies first: - -```sh -if [ ! -x node_modules/.bin/vitest ]; then - pnpm install --frozen-lockfile -fi -``` - -10. Check docs - -Check if the PR touches code with related documentation such as README, docs, inline API docs, or config examples. - -- If docs exist for the changed area and the PR does not update them, flag as IMPORTANT. -- If the PR adds a new feature or config option with no docs, flag as IMPORTANT. -- If the change is purely internal with no user-facing impact, skip this. - -11. Check changelog - -Check if `CHANGELOG.md` exists and whether the PR warrants an entry. - -- If the project has a changelog and the PR is user-facing, flag missing entry as IMPORTANT. -- Leave the change for /prepare-pr, only flag it here. - -12. Answer the key question - -Decide if /prepare-pr can fix issues or the contributor must update the PR. - -13. Save findings to the worktree - -Write the full structured review sections A through J to `.local/review.md`. -Create or overwrite the file and verify it exists and is non-empty. - -```sh -ls -la .local/review.md -wc -l .local/review.md -``` - -14. Output the structured review - -Produce a review that matches what you saved to `.local/review.md`. - -A) TL;DR recommendation - -- One of: READY FOR /prepare-pr | NEEDS WORK | NEEDS DISCUSSION | NOT USEFUL (CLOSE) -- 1 to 3 sentences. - -B) What changed - -C) What is good - -D) Security findings - -E) Concerns or questions (actionable) - -- Numbered list. -- Mark each item as BLOCKER, IMPORTANT, or NIT. -- For each, point to file or area and propose a concrete fix. - -F) Tests - -G) Docs status - -- State if related docs are up to date, missing, or not applicable. - -H) Changelog - -- State if `CHANGELOG.md` needs an entry and which category. - -I) Follow ups (optional) - -J) Suggested PR comment (optional) - -## Guardrails - -- Worktree only. -- Do not delete the worktree after review. -- Review only, do not merge, do not push. diff --git a/.agents/archive/review-pr-v1/agents/openai.yaml b/.agents/archive/review-pr-v1/agents/openai.yaml deleted file mode 100644 index f6593499507..00000000000 --- a/.agents/archive/review-pr-v1/agents/openai.yaml +++ /dev/null @@ -1,4 +0,0 @@ -interface: - display_name: "Review PR" - short_description: "Review GitHub PRs without merging" - default_prompt: "Use $review-pr to perform a thorough, review-only GitHub PR review." diff --git a/.agents/maintainers.md b/.agents/maintainers.md new file mode 100644 index 00000000000..2bbb9c6203e --- /dev/null +++ b/.agents/maintainers.md @@ -0,0 +1 @@ +Maintainer skills now live in [`openclaw/maintainers`](https://github.com/openclaw/maintainers/). diff --git a/.agents/skills/PR_WORKFLOW.md b/.agents/skills/PR_WORKFLOW.md deleted file mode 100644 index 677e484805c..00000000000 --- a/.agents/skills/PR_WORKFLOW.md +++ /dev/null @@ -1,249 +0,0 @@ -# PR Workflow for Maintainers - -Please read this in full and do not skip sections. -This is the single source of truth for the maintainer PR workflow. - -## Triage order - -Process PRs **oldest to newest**. Older PRs are more likely to have merge conflicts and stale dependencies; resolving them first keeps the queue healthy and avoids snowballing rebase pain. - -## Working rule - -Skills execute workflow. Maintainers provide judgment. -Always pause between skills to evaluate technical direction, not just command success. - -These three skills must be used in order: - -1. `review-pr` — review only, produce findings -2. `prepare-pr` — rebase, fix, gate, push to PR head branch -3. `merge-pr` — squash-merge, verify MERGED state, clean up - -They are necessary, but not sufficient. Maintainers must steer between steps and understand the code before moving forward. - -Treat PRs as reports first, code second. -If submitted code is low quality, ignore it and implement the best solution for the problem. - -Do not continue if you cannot verify the problem is real or test the fix. - -## Script-first contract - -Skill runs should invoke these wrappers automatically. You only need to run them manually when debugging or doing an explicit script-only run: - -- `scripts/pr-review ` -- `scripts/pr review-checkout-main ` or `scripts/pr review-checkout-pr ` while reviewing -- `scripts/pr review-guard ` before writing review outputs -- `scripts/pr review-validate-artifacts ` after writing outputs -- `scripts/pr-prepare init ` -- `scripts/pr-prepare validate-commit ` -- `scripts/pr-prepare gates ` -- `scripts/pr-prepare push ` -- Optional one-shot prepare: `scripts/pr-prepare run ` -- `scripts/pr-merge ` (verify-only; short form remains backward compatible) -- `scripts/pr-merge verify ` (verify-only) -- Optional one-shot merge: `scripts/pr-merge run ` - -These wrappers run shared preflight checks and generate deterministic artifacts. They are designed to work from repo root or PR worktree cwd. - -## Required artifacts - -- `.local/pr-meta.json` and `.local/pr-meta.env` from review init. -- `.local/review.md` and `.local/review.json` from review output. -- `.local/prep-context.env` and `.local/prep.md` from prepare. -- `.local/prep.env` from prepare completion. - -## Structured review handoff - -`review-pr` must write `.local/review.json`. -In normal skill runs this is handled automatically. Use `scripts/pr review-artifacts-init ` and `scripts/pr review-tests ...` manually only for debugging or explicit script-only runs. - -Minimum schema: - -```json -{ - "recommendation": "READY FOR /prepare-pr", - "findings": [ - { - "id": "F1", - "severity": "IMPORTANT", - "title": "Missing changelog entry", - "area": "CHANGELOG.md", - "fix": "Add a Fixes entry for PR #" - } - ], - "tests": { - "ran": ["pnpm test -- ..."], - "gaps": ["..."], - "result": "pass" - } -} -``` - -`prepare-pr` resolves all `BLOCKER` and `IMPORTANT` findings from this file. - -## Coding Agent - -Use ChatGPT 5.3 Codex High. Fall back to 5.2 Codex High or 5.3 Codex Medium if necessary. - -## PR quality bar - -- Do not trust PR code by default. -- Do not merge changes you cannot validate with a reproducible problem and a tested fix. -- Keep types strict. Do not use `any` in implementation code. -- Keep external-input boundaries typed and validated, including CLI input, environment variables, network payloads, and tool output. -- Keep implementations properly scoped. Fix root causes, not local symptoms. -- Identify and reuse canonical sources of truth so behavior does not drift across the codebase. -- Harden changes. Always evaluate security impact and abuse paths. -- Understand the system before changing it. Never make the codebase messier just to clear a PR queue. - -## Rebase and conflict resolution - -Before any substantive review or prep work, **always rebase the PR branch onto current `main` and resolve merge conflicts first**. A PR that cannot cleanly rebase is not ready for review — fix conflicts before evaluating correctness. - -- During `prepare-pr`: rebase onto `main` as the first step, before fixing findings or running gates. -- If conflicts are complex or touch areas you do not understand, stop and escalate. -- Prefer **rebase** for linear history; **squash** when commit history is messy or unhelpful. - -## Commit and changelog rules - -- In normal `prepare-pr` runs, commits are created via `scripts/committer "" `. Use it manually only when operating outside the skill flow; avoid manual `git add`/`git commit` so staging stays scoped. -- Follow concise, action-oriented commit messages (e.g., `CLI: add verbose flag to send`). -- During `prepare-pr`, use concise, action-oriented subjects **without** PR numbers or thanks; reserve `(#) thanks @` for the final merge/squash commit. -- Group related changes; avoid bundling unrelated refactors. -- Changelog workflow: keep the latest released version at the top (no `Unreleased`); after publishing, bump the version and start a new top section. -- When working on a PR: add a changelog entry line with the PR number `(#)` and `thanks @` when author metadata is available (mandatory in this workflow). -- When working on an issue: reference the issue in the changelog entry. -- In this workflow, changelog is always required even for internal/test-only changes. - -## Gate policy - -In fresh worktrees, dependency bootstrap is handled by wrappers before local gates. Manual equivalent: - -```sh -pnpm install --frozen-lockfile -``` - -Gate set: - -- Always: `pnpm build`, `pnpm check` -- `pnpm test` required unless high-confidence docs-only criteria pass. - -## Co-contributor and clawtributors - -- If we squash, add the PR author as a co-contributor in the commit body using a `Co-authored-by:` trailer. -- When maintainer prepares and merges the PR, add the maintainer as an additional `Co-authored-by:` trailer too. -- Avoid `--auto` merges for maintainer landings. Merge only after checks are green so the maintainer account is the actor and attribution is deterministic. -- For squash merges, set `--author-email` to a reviewer-owned email with fallback candidates; if merge fails due to author-email validation, retry once with the next candidate. -- If you review a PR and later do work on it, land via merge/squash (no direct-main commits) and always add the PR author as a co-contributor. -- When merging a PR: leave a PR comment that explains exactly what we did, include the SHA hashes, and record the comment URL in the final report. -- Manual post-merge step for new contributors: run `bun scripts/update-clawtributors.ts` to add their avatar to the README "Thanks to all clawtributors" list, then commit the regenerated README. - -## Review mode vs landing mode - -- **Review mode (PR link only):** read `gh pr view`/`gh pr diff`; **do not** switch branches; **do not** change code. -- **Landing mode (exception path):** use only when normal `review-pr -> prepare-pr -> merge-pr` flow cannot safely preserve attribution or cannot satisfy branch protection. Create an integration branch from `main`, bring in PR commits (**prefer rebase** for linear history; **merge allowed** when complexity/conflicts make it safer), apply fixes, add changelog (+ thanks + PR #), run full gate **locally before committing** (`pnpm build && pnpm check && pnpm test`), commit, merge back to `main`, then `git switch main` (never stay on a topic branch after landing). Important: the contributor needs to be in the git graph after this! - -## Pre-review safety checks - -- Before starting a review when a GH Issue/PR is pasted: `review-pr`/`scripts/pr-review` should create and use an isolated `.worktrees/pr-` checkout from `origin/main` automatically. Do not require a clean main checkout, and do not run `git pull` in a dirty main checkout. -- PR review calls: prefer a single `gh pr view --json ...` to batch metadata/comments; run `gh pr diff` only when needed. -- PRs should summarize scope, note testing performed, and mention any user-facing changes or new flags. -- Read `docs/help/submitting-a-pr.md` ([Submitting a PR](https://docs.openclaw.ai/help/submitting-a-pr)) for what we expect from contributors. - -## Unified workflow - -Entry criteria: - -- PR URL/number is known. -- Problem statement is clear enough to attempt reproduction. -- A realistic verification path exists (tests, integration checks, or explicit manual validation). - -### 1) `review-pr` - -Purpose: - -- Review only: correctness, value, security risk, tests, docs, and changelog impact. -- Produce structured findings and a recommendation. - -Expected output: - -- Recommendation: ready, needs work, needs discussion, or close. -- `.local/review.md` with actionable findings. - -Maintainer checkpoint before `prepare-pr`: - -``` -What problem are they trying to solve? -What is the most optimal implementation? -Can we fix up everything? -Do we have any questions? -``` - -Stop and escalate instead of continuing if: - -- The problem cannot be reproduced or confirmed. -- The proposed PR scope does not match the stated problem. -- The design introduces unresolved security or trust-boundary concerns. - -### 2) `prepare-pr` - -Purpose: - -- Make the PR merge-ready on its head branch. -- Rebase onto current `main` first, then fix blocker/important findings, then run gates. -- In fresh worktrees, bootstrap dependencies before local gates (`pnpm install --frozen-lockfile`). - -Expected output: - -- Updated code and tests on the PR head branch. -- `.local/prep.md` with changes, verification, and current HEAD SHA. -- Final status: `PR is ready for /merge-pr`. - -Maintainer checkpoint before `merge-pr`: - -``` -Is this the most optimal implementation? -Is the code properly scoped? -Is the code properly reusing existing logic in the codebase? -Is the code properly typed? -Is the code hardened? -Do we have enough tests? -Do we need regression tests? -Are tests using fake timers where appropriate? (e.g., debounce/throttle, retry backoff, timeout branches, delayed callbacks, polling loops) -Do not add performative tests, ensure tests are real and there are no regressions. -Do you see any follow-up refactors we should do? -Did any changes introduce any potential security vulnerabilities? -Take your time, fix it properly, refactor if necessary. -``` - -Stop and escalate instead of continuing if: - -- You cannot verify behavior changes with meaningful tests or validation. -- Fixing findings requires broad architecture changes outside safe PR scope. -- Security hardening requirements remain unresolved. - -### 3) `merge-pr` - -Purpose: - -- Merge only after review and prep artifacts are present and checks are green. -- Use deterministic squash merge flow (`--match-head-commit` + explicit subject/body with co-author trailer), then verify the PR ends in `MERGED` state. -- If no required checks are configured on the PR, treat that as acceptable and continue after branch-up-to-date validation. - -Go or no-go checklist before merge: - -- All BLOCKER and IMPORTANT findings are resolved. -- Verification is meaningful and regression risk is acceptably low. -- Changelog is updated (mandatory) and docs are updated when required. -- Required CI checks are green and the branch is not behind `main`. - -Expected output: - -- Successful merge commit and recorded merge SHA. -- Worktree cleanup after successful merge. -- Comment on PR indicating merge was successful. - -Maintainer checkpoint after merge: - -- Were any refactors intentionally deferred and now need follow-up issue(s)? -- Did this reveal broader architecture or test gaps we should address? -- Run `bun scripts/update-clawtributors.ts` if the contributor is new. diff --git a/.agents/skills/merge-pr/SKILL.md b/.agents/skills/merge-pr/SKILL.md deleted file mode 100644 index 91b7c4aa4e2..00000000000 --- a/.agents/skills/merge-pr/SKILL.md +++ /dev/null @@ -1,101 +0,0 @@ ---- -name: merge-pr -description: Script-first deterministic squash merge with strict required-check gating, head-SHA pinning, and reliable attribution/commenting. ---- - -# Merge PR - -## Overview - -Merge a prepared PR only after deterministic validation. - -## Inputs - -- Ask for PR number or URL. -- If missing, use `.local/prep.env` from the PR worktree. - -## Safety - -- Never use `gh pr merge --auto` in this flow. -- Never run `git push` directly. -- Require `--match-head-commit` during merge. -- Wrapper commands are cwd-agnostic; you can run them from repo root or inside the PR worktree. - -## Execution Contract - -1. Validate merge readiness: - -```sh -scripts/pr-merge verify -``` - -Backward-compatible verify form also works: - -```sh -scripts/pr-merge -``` - -2. Run one-shot deterministic merge: - -```sh -scripts/pr-merge run -``` - -3. Capture and report these values in a human-readable summary (not raw `key=value` lines): - -- Merge commit SHA -- Merge author email -- Merge completion comment URL -- PR URL - -## Steps - -1. Validate artifacts - -```sh -require=(.local/review.md .local/review.json .local/prep.md .local/prep.env) -for f in "${require[@]}"; do - [ -s "$f" ] || { echo "Missing artifact: $f"; exit 1; } -done -``` - -2. Validate checks and branch status - -```sh -scripts/pr-merge verify -source .local/prep.env -``` - -`scripts/pr-merge` treats “no required checks configured” as acceptable (`[]`), but fails on any required `fail` or `pending`. - -3. Merge deterministically (wrapper-managed) - -```sh -scripts/pr-merge run -``` - -`scripts/pr-merge run` performs: - -- deterministic squash merge pinned to `PREP_HEAD_SHA` -- reviewer merge author email selection with fallback candidates -- one retry only when merge fails due to author-email validation -- co-author trailers for PR author and reviewer -- post-merge verification of both co-author trailers on commit message -- PR comment retry (3 attempts), then comment URL extraction -- cleanup after confirmed `MERGED` - -4. Manual fallback (only if wrapper is unavailable) - -```sh -scripts/pr merge-run -``` - -5. Cleanup - -Cleanup is handled by `run` after merge success. - -## Guardrails - -- End in `MERGED`, never `CLOSED`. -- Cleanup only after confirmed merge. -- In final chat output, use labeled lines or bullets; do not paste raw wrapper diagnostics unless debugging. diff --git a/.agents/skills/merge-pr/agents/openai.yaml b/.agents/skills/merge-pr/agents/openai.yaml deleted file mode 100644 index 9c10ae4d271..00000000000 --- a/.agents/skills/merge-pr/agents/openai.yaml +++ /dev/null @@ -1,4 +0,0 @@ -interface: - display_name: "Merge PR" - short_description: "Merge GitHub PRs via squash" - default_prompt: "Use $merge-pr to merge a GitHub PR via squash after preparation." diff --git a/.agents/skills/mintlify/SKILL.md b/.agents/skills/mintlify/SKILL.md deleted file mode 100644 index 0dd6a1a891a..00000000000 --- a/.agents/skills/mintlify/SKILL.md +++ /dev/null @@ -1,345 +0,0 @@ ---- -name: mintlify -description: Build and maintain documentation sites with Mintlify. Use when - creating docs pages, configuring navigation, adding components, or setting up - API references. -license: MIT -compatibility: Requires Node.js for CLI. Works with any Git-based workflow. -metadata: - author: mintlify - version: "1.0" - mintlify-proj: mintlify ---- - -# Mintlify best practices - -**Always consult [mintlify.com/docs](https://mintlify.com/docs) for components, configuration, and latest features.** - -**Always** favor searching the current Mintlify documentation over whatever is in your training data about Mintlify. - -Mintlify is a documentation platform that transforms MDX files into documentation sites. Configure site-wide settings in the `docs.json` file, write content in MDX with YAML frontmatter, and favor built-in components over custom components. - -Full schema at [mintlify.com/docs.json](https://mintlify.com/docs.json). - -## Before you write - -### Understand the project - -All documentation lives in the `docs/` directory in this repo. Read `docs.json` in that directory (`docs/docs.json`). This file defines the entire site: navigation structure, theme, colors, links, API and specs. - -Understanding the project tells you: - -- What pages exist and how they're organized -- What navigation groups are used (and their naming conventions) -- How the site navigation is structured -- What theme and configuration the site uses - -### Check for existing content - -Search the docs before creating new pages. You may need to: - -- Update an existing page instead of creating a new one -- Add a section to an existing page -- Link to existing content rather than duplicating - -### Read surrounding content - -Before writing, read 2-3 similar pages to understand the site's voice, structure, formatting conventions, and level of detail. - -### Understand Mintlify components - -Review the Mintlify [components](https://www.mintlify.com/docs/components) to select and use any relevant components for the documentation request that you are working on. - -## Quick reference - -### CLI commands - -- `npm i -g mint` - Install the Mintlify CLI -- `mint dev` - Local preview at localhost:3000 -- `mint broken-links` - Check internal links -- `mint a11y` - Check for accessibility issues in content -- `mint rename` - Rename/move files and update references -- `mint validate` - Validate documentation builds - -### Required files - -- `docs.json` - Site configuration (navigation, theme, integrations, etc.). See [global settings](https://mintlify.com/docs/settings/global) for all options. -- `*.mdx` files - Documentation pages with YAML frontmatter - -### Example file structure - -``` -project/ -├── docs.json # Site configuration -├── introduction.mdx -├── quickstart.mdx -├── guides/ -│ └── example.mdx -├── openapi.yml # API specification -├── images/ # Static assets -│ └── example.png -└── snippets/ # Reusable components - └── component.jsx -``` - -## Page frontmatter - -Every page requires `title` in its frontmatter. Include `description` for SEO and navigation. - -```yaml theme={null} ---- -title: "Clear, descriptive title" -description: "Concise summary for SEO and navigation." ---- -``` - -Optional frontmatter fields: - -- `sidebarTitle`: Short title for sidebar navigation. -- `icon`: Lucide or Font Awesome icon name, URL, or file path. -- `tag`: Label next to the page title in the sidebar (for example, "NEW"). -- `mode`: Page layout mode (`default`, `wide`, `custom`). -- `keywords`: Array of terms related to the page content for local search and SEO. -- Any custom YAML fields for use with personalization or conditional content. - -## File conventions - -- Match existing naming patterns in the directory -- If there are no existing files or inconsistent file naming patterns, use kebab-case: `getting-started.mdx`, `api-reference.mdx` -- Use root-relative paths without file extensions for internal links: `/getting-started/quickstart` -- Do not use relative paths (`../`) or absolute URLs for internal pages -- When you create a new page, add it to `docs.json` navigation or it won't appear in the sidebar - -## Organize content - -When a user asks about anything related to site-wide configurations, start by understanding the [global settings](https://www.mintlify.com/docs/organize/settings). See if a setting in the `docs.json` file can be updated to achieve what the user wants. - -### Navigation - -The `navigation` property in `docs.json` controls site structure. Choose one primary pattern at the root level, then nest others within it. - -**Choose your primary pattern:** - -| Pattern | When to use | -| ------------- | ---------------------------------------------------------------------------------------------- | -| **Groups** | Default. Single audience, straightforward hierarchy | -| **Tabs** | Distinct sections with different audiences (Guides vs API Reference) or content types | -| **Anchors** | Want persistent section links at sidebar top. Good for separating docs from external resources | -| **Dropdowns** | Multiple doc sections users switch between, but not distinct enough for tabs | -| **Products** | Multi-product company with separate documentation per product | -| **Versions** | Maintaining docs for multiple API/product versions simultaneously | -| **Languages** | Localized content | - -**Within your primary pattern:** - -- **Groups** - Organize related pages. Can nest groups within groups, but keep hierarchy shallow -- **Menus** - Add dropdown navigation within tabs for quick jumps to specific pages -- **`expanded: false`** - Collapse nested groups by default. Use for reference sections users browse selectively -- **`openapi`** - Auto-generate pages from OpenAPI spec. Add at group/tab level to inherit - -**Common combinations:** - -- Tabs containing groups (most common for docs with API reference) -- Products containing tabs (multi-product SaaS) -- Versions containing tabs (versioned API docs) -- Anchors containing groups (simple docs with external resource links) - -### Links and paths - -- **Internal links:** Root-relative, no extension: `/getting-started/quickstart` -- **Images:** Store in `/images`, reference as `/images/example.png` -- **External links:** Use full URLs, they open in new tabs automatically - -## Customize docs sites - -**What to customize where:** - -- **Brand colors, fonts, logo** → `docs.json`. See [global settings](https://mintlify.com/docs/settings/global) -- **Component styling, layout tweaks** → `custom.css` at project root -- **Dark mode** → Enabled by default. Only disable with `"appearance": "light"` in `docs.json` if brand requires it - -Start with `docs.json`. Only add `custom.css` when you need styling that config doesn't support. - -## Write content - -### Components - -The [components overview](https://mintlify.com/docs/components) organizes all components by purpose: structure content, draw attention, show/hide content, document APIs, link to pages, and add visual context. Start there to find the right component. - -**Common decision points:** - -| Need | Use | -| -------------------------- | ----------------------- | -| Hide optional details | `` | -| Long code examples | `` | -| User chooses one option | `` | -| Linked navigation cards | `` in `` | -| Sequential instructions | `` | -| Code in multiple languages | `` | -| API parameters | `` | -| API response fields | `` | - -**Callouts by severity:** - -- `` - Supplementary info, safe to skip -- `` - Helpful context such as permissions -- `` - Recommendations or best practices -- `` - Potentially destructive actions -- `` - Success confirmation - -### Reusable content - -**When to use snippets:** - -- Exact content appears on more than one page -- Complex components you want to maintain in one place -- Shared content across teams/repos - -**When NOT to use snippets:** - -- Slight variations needed per page (leads to complex props) - -Import snippets with `import { Component } from "/path/to/snippet-name.jsx"`. - -## Writing standards - -### Voice and structure - -- Second-person voice ("you") -- Active voice, direct language -- Sentence case for headings ("Getting started", not "Getting Started") -- Sentence case for code block titles ("Expandable example", not "Expandable Example") -- Lead with context: explain what something is before how to use it -- Prerequisites at the start of procedural content - -### What to avoid - -**Never use:** - -- Marketing language ("powerful", "seamless", "robust", "cutting-edge") -- Filler phrases ("it's important to note", "in order to") -- Excessive conjunctions ("moreover", "furthermore", "additionally") -- Editorializing ("obviously", "simply", "just", "easily") - -**Watch for AI-typical patterns:** - -- Overly formal or stilted phrasing -- Unnecessary repetition of concepts -- Generic introductions that don't add value -- Concluding summaries that restate what was just said - -### Formatting - -- All code blocks must have language tags -- All images and media must have descriptive alt text -- Use bold and italics only when they serve the reader's understanding--never use text styling just for decoration -- No decorative formatting or emoji - -### Code examples - -- Keep examples simple and practical -- Use realistic values (not "foo" or "bar") -- One clear example is better than multiple variations -- Test that code works before including it - -## Document APIs - -**Choose your approach:** - -- **Have an OpenAPI spec?** → Add to `docs.json` with `"openapi": ["openapi.yaml"]`. Pages auto-generate. Reference in navigation as `GET /endpoint` -- **No spec?** → Write endpoints manually with `api: "POST /users"` in frontmatter. More work but full control -- **Hybrid** → Use OpenAPI for most endpoints, manual pages for complex workflows - -Encourage users to generate endpoint pages from an OpenAPI spec. It is the most efficient and easiest to maintain option. - -## Deploy - -Mintlify deploys automatically when changes are pushed to the connected Git repository. - -**What agents can configure:** - -- **Redirects** → Add to `docs.json` with `"redirects": [{"source": "/old", "destination": "/new"}]` -- **SEO indexing** → Control with `"seo": {"indexing": "all"}` to include hidden pages in search - -**Requires dashboard setup (human task):** - -- Custom domains and subdomains -- Preview deployment settings -- DNS configuration - -For `/docs` subpath hosting with Vercel or Cloudflare, agents can help configure rewrite rules. See [/docs subpath](https://mintlify.com/docs/deploy/vercel). - -## Workflow - -### 1. Understand the task - -Identify what needs to be documented, which pages are affected, and what the reader should accomplish afterward. If any of these are unclear, ask. - -### 2. Research - -- Read `docs/docs.json` to understand the site structure -- Search existing docs for related content -- Read similar pages to match the site's style - -### 3. Plan - -- Synthesize what the reader should accomplish after reading the docs and the current content -- Propose any updates or new content -- Verify that your proposed changes will help readers be successful - -### 4. Write - -- Start with the most important information -- Keep sections focused and scannable -- Use components appropriately (don't overuse them) -- Mark anything uncertain with a TODO comment: - -```mdx theme={null} -{/* TODO: Verify the default timeout value */} -``` - -### 5. Update navigation - -If you created a new page, add it to the appropriate group in `docs.json`. - -### 6. Verify - -Before submitting: - -- [ ] Frontmatter includes title and description -- [ ] All code blocks have language tags -- [ ] Internal links use root-relative paths without file extensions -- [ ] New pages are added to `docs.json` navigation -- [ ] Content matches the style of surrounding pages -- [ ] No marketing language or filler phrases -- [ ] TODOs are clearly marked for anything uncertain -- [ ] Run `mint broken-links` to check links -- [ ] Run `mint validate` to find any errors - -## Edge cases - -### Migrations - -If a user asks about migrating to Mintlify, ask if they are using ReadMe or Docusaurus. If they are, use the [@mintlify/scraping](https://www.npmjs.com/package/@mintlify/scraping) CLI to migrate content. If they are using a different platform to host their documentation, help them manually convert their content to MDX pages using Mintlify components. - -### Hidden pages - -Any page that is not included in the `docs.json` navigation is hidden. Use hidden pages for content that should be accessible by URL or indexed for the assistant or search, but not discoverable through the sidebar navigation. - -### Exclude pages - -The `.mintignore` file is used to exclude files from a documentation repository from being processed. - -## Common gotchas - -1. **Component imports** - JSX components need explicit import, MDX components don't -2. **Frontmatter required** - Every MDX file needs `title` at minimum -3. **Code block language** - Always specify language identifier -4. **Never use `mint.json`** - `mint.json` is deprecated. Only ever use `docs.json` - -## Resources - -- [Documentation](https://mintlify.com/docs) -- [Configuration schema](https://mintlify.com/docs.json) -- [Feature requests](https://github.com/orgs/mintlify/discussions/categories/feature-requests) -- [Bugs and feedback](https://github.com/orgs/mintlify/discussions/categories/bugs-feedback) diff --git a/.agents/skills/prepare-pr/SKILL.md b/.agents/skills/prepare-pr/SKILL.md deleted file mode 100644 index d2834a84b4a..00000000000 --- a/.agents/skills/prepare-pr/SKILL.md +++ /dev/null @@ -1,127 +0,0 @@ ---- -name: prepare-pr -description: Script-first PR preparation with structured findings resolution, deterministic push safety, and explicit gate execution. ---- - -# Prepare PR - -## Overview - -Prepare the PR head branch for merge after `/review-pr`. - -## Inputs - -- Ask for PR number or URL. -- If missing, use `.local/pr-meta.env` if present in the PR worktree. - -## Safety - -- Never push to `main`. -- Only push to PR head with explicit `--force-with-lease` against known head SHA. -- Do not run `git clean -fdx`. -- Wrappers are cwd-agnostic; run from repo root or PR worktree. - -## Execution Contract - -1. Run setup: - -```sh -scripts/pr-prepare init -``` - -2. Resolve findings from structured review: - -- `.local/review.json` is mandatory. -- Resolve all `BLOCKER` and `IMPORTANT` items. - -3. Commit scoped changes with concise subjects (no PR number/thanks; those belong on the final merge/squash commit). - -4. Run gates via wrapper. - -5. Push via wrapper (includes pre-push remote verification, one automatic lease-retry path, and post-push API propagation retry). - -Optional one-shot path: - -```sh -scripts/pr-prepare run -``` - -## Steps - -1. Setup and artifacts - -```sh -scripts/pr-prepare init - -ls -la .local/review.md .local/review.json .local/pr-meta.env .local/prep-context.env -jq . .local/review.json >/dev/null -``` - -2. Resolve required findings - -List required items: - -```sh -jq -r '.findings[] | select(.severity=="BLOCKER" or .severity=="IMPORTANT") | "- [\(.severity)] \(.id): \(.title) => \(.fix)"' .local/review.json -``` - -Fix all required findings. Keep scope tight. - -3. Update changelog/docs (changelog is mandatory in this workflow) - -```sh -jq -r '.changelog' .local/review.json -jq -r '.docs' .local/review.json -``` - -Changelog gate requirement: - -- `CHANGELOG.md` must include a newly added changelog entry line. -- When PR author metadata is available, that same changelog entry line must include `(#) thanks @`. - -4. Commit scoped changes - -Use concise, action-oriented subject lines without PR numbers/thanks. The final merge/squash commit is the only place we include PR numbers and contributor thanks. - -Use explicit file list: - -```sh -scripts/committer "fix: " ... -``` - -5. Run gates - -```sh -scripts/pr-prepare gates -``` - -6. Push safely to PR head - -```sh -scripts/pr-prepare push -``` - -This push step includes: - -- robust fork remote resolution from owner/name, -- pre-push remote SHA verification, -- one automatic rebase + gate rerun + retry if lease push fails, -- post-push PR-head propagation retry, -- idempotent behavior when local prep HEAD is already on the PR head, -- post-push SHA verification and `.local/prep.env` generation. - -7. Verify handoff artifacts - -```sh -ls -la .local/prep.md .local/prep.env -``` - -8. Output - -- Summarize resolved findings and gate results. -- Print exactly: `PR is ready for /merge-pr`. - -## Guardrails - -- Do not run `gh pr merge` in this skill. -- Do not delete worktree. diff --git a/.agents/skills/prepare-pr/agents/openai.yaml b/.agents/skills/prepare-pr/agents/openai.yaml deleted file mode 100644 index 290b1b5ab61..00000000000 --- a/.agents/skills/prepare-pr/agents/openai.yaml +++ /dev/null @@ -1,4 +0,0 @@ -interface: - display_name: "Prepare PR" - short_description: "Prepare GitHub PRs for merge" - default_prompt: "Use $prepare-pr to prep a GitHub PR for merge without merging." diff --git a/.agents/skills/review-pr/SKILL.md b/.agents/skills/review-pr/SKILL.md deleted file mode 100644 index f5694ca2c41..00000000000 --- a/.agents/skills/review-pr/SKILL.md +++ /dev/null @@ -1,142 +0,0 @@ ---- -name: review-pr -description: Script-first review-only GitHub pull request analysis. Use for deterministic PR review with structured findings handoff to /prepare-pr. ---- - -# Review PR - -## Overview - -Perform a read-only review and produce both human and machine-readable outputs. - -## Inputs - -- Ask for PR number or URL. -- If missing, always ask. - -## Safety - -- Never push, merge, or modify code intended to keep. -- Work only in `.worktrees/pr-`. -- Wrapper commands are cwd-agnostic; you can run them from repo root or inside the PR worktree. - -## Execution Contract - -1. Run wrapper setup: - -```sh -scripts/pr-review -``` - -2. Use explicit branch mode switches: - -- Main baseline mode: `scripts/pr review-checkout-main ` -- PR-head mode: `scripts/pr review-checkout-pr ` - -3. Before writing review outputs, run branch guard: - -```sh -scripts/pr review-guard -``` - -4. Write both outputs: - -- `.local/review.md` with sections A through J. -- `.local/review.json` with structured findings. - -5. Validate artifacts semantically: - -```sh -scripts/pr review-validate-artifacts -``` - -## Steps - -1. Setup and metadata - -```sh -scripts/pr-review -ls -la .local/pr-meta.json .local/pr-meta.env .local/review-context.env .local/review-mode.env -``` - -2. Existing implementation check on main - -```sh -scripts/pr review-checkout-main -rg -n "" -S src extensions apps || true -git log --oneline --all --grep "" | head -20 -``` - -3. Claim PR - -```sh -gh_user=$(gh api user --jq .login) -gh pr edit --add-assignee "$gh_user" || echo "Could not assign reviewer, continuing" -``` - -4. Read PR description and diff - -```sh -scripts/pr review-checkout-pr -gh pr diff - -source .local/review-context.env -git diff --stat "$MERGE_BASE"..pr- -git diff "$MERGE_BASE"..pr- -``` - -5. Optional local tests - -Use the wrapper for target validation and executed-test verification: - -```sh -scripts/pr review-tests [ ...] -``` - -6. Initialize review artifact templates - -```sh -scripts/pr review-artifacts-init -``` - -7. Produce review outputs - -- Fill `.local/review.md` sections A through J. -- Fill `.local/review.json`. - -Minimum JSON shape: - -```json -{ - "recommendation": "READY FOR /prepare-pr", - "findings": [ - { - "id": "F1", - "severity": "IMPORTANT", - "title": "...", - "area": "path/or/component", - "fix": "Actionable fix" - } - ], - "tests": { - "ran": [], - "gaps": [], - "result": "pass" - }, - "docs": "up_to_date|missing|not_applicable", - "changelog": "required" -} -``` - -8. Guard + validate before final output - -```sh -scripts/pr review-guard -scripts/pr review-validate-artifacts -``` - -## Guardrails - -- Keep review read-only. -- Do not delete worktree. -- Use merge-base scoped diff for local context to avoid stale branch drift. diff --git a/.agents/skills/review-pr/agents/openai.yaml b/.agents/skills/review-pr/agents/openai.yaml deleted file mode 100644 index f6593499507..00000000000 --- a/.agents/skills/review-pr/agents/openai.yaml +++ /dev/null @@ -1,4 +0,0 @@ -interface: - display_name: "Review PR" - short_description: "Review GitHub PRs without merging" - default_prompt: "Use $review-pr to perform a thorough, review-only GitHub PR review." diff --git a/.gitattributes b/.gitattributes index 6313b56c578..54fc4c9b14d 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1 +1,3 @@ * text=auto eol=lf +CLAUDE.md -text +src/gateway/server-methods/CLAUDE.md -text diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml index 56a343c38d8..927aa7079cf 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -13,7 +13,7 @@ body: attributes: label: Summary description: One-sentence statement of what is broken. - placeholder: After upgrading to 2026.2.13, Telegram thread replies fail with "reply target not found". + placeholder: After upgrading to , behavior regressed from . validations: required: true - type: textarea @@ -48,7 +48,7 @@ body: attributes: label: OpenClaw version description: Exact version/build tested. - placeholder: 2026.2.13 + placeholder: validations: required: true - type: input @@ -83,7 +83,7 @@ body: - Frequency (always/intermittent/edge case) - Consequence (missed messages, failed onboarding, extra cost, etc.) placeholder: | - Affected: Telegram group users on 2026.2.13 + Affected: Telegram group users on Severity: High (blocks replies) Frequency: 100% repro Consequence: Agents cannot respond in threads @@ -92,4 +92,4 @@ body: attributes: label: Additional information description: Add any context that helps triage but does not fit above. - placeholder: Regression started after upgrade from 2026.2.12; temporary workaround is restarting gateway every 30m. + placeholder: Regression started after upgrade from ; temporary workaround is ... diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml index 1b38a9ddf05..4c1b9775597 100644 --- a/.github/ISSUE_TEMPLATE/config.yml +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -2,7 +2,7 @@ blank_issues_enabled: false contact_links: - name: Onboarding url: https://discord.gg/clawd - about: New to OpenClaw? Join Discord for setup guidance from Krill in \#help. + about: "New to OpenClaw? Join Discord for setup guidance in #help." - name: Support url: https://discord.gg/clawd - about: Get help from Krill and the community on Discord in \#help. + about: "Get help from the OpenClaw community on Discord in #help." diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml index 3594b73a2c5..a08b456786e 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.yml +++ b/.github/ISSUE_TEMPLATE/feature_request.yml @@ -21,7 +21,7 @@ body: attributes: label: Problem to solve description: What user pain this solves and why current behavior is insufficient. - placeholder: Teams cannot distinguish agent personas in mixed channels, causing misrouted follow-ups. + placeholder: Agents cannot distinguish persona context in mixed channels, causing misrouted follow-ups. validations: required: true - type: textarea diff --git a/.github/actionlint.yaml b/.github/actionlint.yaml index e660d2a9761..f02fbddb3e8 100644 --- a/.github/actionlint.yaml +++ b/.github/actionlint.yaml @@ -4,8 +4,11 @@ self-hosted-runner: labels: # Blacksmith CI runners - - blacksmith-4vcpu-ubuntu-2404 - - blacksmith-4vcpu-windows-2025 + - blacksmith-8vcpu-ubuntu-2404 + - blacksmith-8vcpu-windows-2025 + - blacksmith-16vcpu-ubuntu-2404 + - blacksmith-16vcpu-windows-2025 + - blacksmith-16vcpu-ubuntu-2404-arm # Ignore patterns for known issues paths: @@ -15,3 +18,5 @@ paths: - "shellcheck reported issue.+" # Ignore intentional if: false for disabled jobs - 'constant expression "false" in condition' + # actionlint's built-in runner label allowlist lags Blacksmith additions. + - 'label "blacksmith-16vcpu-[^"]+" is unknown\.' diff --git a/.github/actions/setup-node-env/action.yml b/.github/actions/setup-node-env/action.yml index 5fa4f6728bc..334cd3c24fb 100644 --- a/.github/actions/setup-node-env/action.yml +++ b/.github/actions/setup-node-env/action.yml @@ -37,7 +37,7 @@ runs: exit 1 - name: Setup Node.js - uses: actions/setup-node@v4 + uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version: ${{ inputs.node-version }} check-latest: true @@ -52,7 +52,7 @@ runs: if: inputs.install-bun == 'true' uses: oven-sh/setup-bun@v2 with: - bun-version: latest + bun-version: "1.3.9+cf6cdbbba" - name: Runtime versions shell: bash @@ -70,14 +70,29 @@ runs: shell: bash env: CI: "true" + FROZEN_LOCKFILE: ${{ inputs.frozen-lockfile }} run: | + set -euo pipefail export PATH="$NODE_BIN:$PATH" which node node -v pnpm -v - LOCKFILE_FLAG="" - if [ "${{ inputs.frozen-lockfile }}" = "true" ]; then - LOCKFILE_FLAG="--frozen-lockfile" + case "$FROZEN_LOCKFILE" in + true) LOCKFILE_FLAG="--frozen-lockfile" ;; + false) LOCKFILE_FLAG="" ;; + *) + echo "::error::Invalid frozen-lockfile input: '$FROZEN_LOCKFILE' (expected true or false)" + exit 2 + ;; + esac + + install_args=( + install + --ignore-scripts=false + --config.engine-strict=false + --config.enable-pre-post-scripts=true + ) + if [ -n "$LOCKFILE_FLAG" ]; then + install_args+=("$LOCKFILE_FLAG") fi - pnpm install $LOCKFILE_FLAG --ignore-scripts=false --config.engine-strict=false --config.enable-pre-post-scripts=true || \ - pnpm install $LOCKFILE_FLAG --ignore-scripts=false --config.engine-strict=false --config.enable-pre-post-scripts=true + pnpm "${install_args[@]}" || pnpm "${install_args[@]}" diff --git a/.github/actions/setup-pnpm-store-cache/action.yml b/.github/actions/setup-pnpm-store-cache/action.yml index c866393ee43..8e25492ac92 100644 --- a/.github/actions/setup-pnpm-store-cache/action.yml +++ b/.github/actions/setup-pnpm-store-cache/action.yml @@ -14,11 +14,17 @@ runs: steps: - name: Setup pnpm (corepack retry) shell: bash + env: + PNPM_VERSION: ${{ inputs.pnpm-version }} run: | set -euo pipefail + if [[ ! "$PNPM_VERSION" =~ ^[0-9]+(\.[0-9]+){1,2}([.-][0-9A-Za-z.-]+)?$ ]]; then + echo "::error::Invalid pnpm-version input: '$PNPM_VERSION'" + exit 2 + fi corepack enable for attempt in 1 2 3; do - if corepack prepare "pnpm@${{ inputs.pnpm-version }}" --activate; then + if corepack prepare "pnpm@$PNPM_VERSION" --activate; then pnpm -v exit 0 fi diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 829604b4ce3..7b7fd1595aa 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -7,6 +7,7 @@ registries: npm-npmjs: type: npm-registry url: https://registry.npmjs.org + token: ${{secrets.NPM_NPMJS_TOKEN}} replaces-base: true updates: @@ -14,9 +15,9 @@ updates: - package-ecosystem: npm directory: / schedule: - interval: weekly + interval: daily cooldown: - default-days: 7 + default-days: 2 groups: production: dependency-type: production @@ -36,9 +37,9 @@ updates: - package-ecosystem: github-actions directory: / schedule: - interval: weekly + interval: daily cooldown: - default-days: 7 + default-days: 2 groups: actions: patterns: @@ -52,9 +53,9 @@ updates: - package-ecosystem: swift directory: /apps/macos schedule: - interval: weekly + interval: daily cooldown: - default-days: 7 + default-days: 2 groups: swift-deps: patterns: @@ -68,9 +69,9 @@ updates: - package-ecosystem: swift directory: /apps/shared/MoltbotKit schedule: - interval: weekly + interval: daily cooldown: - default-days: 7 + default-days: 2 groups: swift-deps: patterns: @@ -84,9 +85,9 @@ updates: - package-ecosystem: swift directory: /Swabble schedule: - interval: weekly + interval: daily cooldown: - default-days: 7 + default-days: 2 groups: swift-deps: patterns: @@ -100,9 +101,9 @@ updates: - package-ecosystem: gradle directory: /apps/android schedule: - interval: weekly + interval: daily cooldown: - default-days: 7 + default-days: 2 groups: android-deps: patterns: @@ -111,3 +112,16 @@ updates: - minor - patch open-pull-requests-limit: 5 + + # Docker base images + - package-ecosystem: docker + directory: / + schedule: + interval: weekly + cooldown: + default-days: 2 + groups: + docker-images: + patterns: + - "*" + open-pull-requests-limit: 5 diff --git a/.github/labeler.yml b/.github/labeler.yml index 78366fb2097..ffe55984ac6 100644 --- a/.github/labeler.yml +++ b/.github/labeler.yml @@ -240,6 +240,10 @@ - changed-files: - any-glob-to-any-file: - "extensions/device-pair/**" +"extensions: acpx": + - changed-files: + - any-glob-to-any-file: + - "extensions/acpx/**" "extensions: minimax-portal-auth": - changed-files: - any-glob-to-any-file: diff --git a/.github/workflows/auto-response.yml b/.github/workflows/auto-response.yml index 630db1d12a6..d18b0d776ea 100644 --- a/.github/workflows/auto-response.yml +++ b/.github/workflows/auto-response.yml @@ -3,6 +3,8 @@ name: Auto response on: issues: types: [opened, edited, labeled] + issue_comment: + types: [created] pull_request_target: types: [labeled] @@ -13,17 +15,24 @@ jobs: permissions: issues: write pull-requests: write - runs-on: self-hosted + runs-on: blacksmith-16vcpu-ubuntu-2404 steps: - uses: actions/create-github-app-token@d72941d797fd3113feb6b93fd0dec494b13a2547 # v1 id: app-token + continue-on-error: true with: app-id: "2729701" private-key: ${{ secrets.GH_APP_PRIVATE_KEY }} + - uses: actions/create-github-app-token@d72941d797fd3113feb6b93fd0dec494b13a2547 # v1 + id: app-token-fallback + if: steps.app-token.outcome == 'failure' + with: + app-id: "2971289" + private-key: ${{ secrets.GH_APP_PRIVATE_KEY_FALLBACK }} - name: Handle labeled items uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7 with: - github-token: ${{ steps.app-token.outputs.token }} + github-token: ${{ steps.app-token.outputs.token || steps.app-token-fallback.outputs.token }} script: | // Labels prefixed with "r:" are auto-response triggers. const rules = [ @@ -42,6 +51,7 @@ jobs: { label: "r: testflight", close: true, + commentTriggers: ["testflight"], message: "Not available, build from source.", }, { @@ -55,11 +65,76 @@ jobs: close: true, lock: true, lockReason: "off-topic", + commentTriggers: ["moltbook"], message: "OpenClaw is not affiliated with Moltbook, and issues related to Moltbook should not be submitted here.", }, ]; + const maintainerTeam = "maintainer"; + const pingWarningMessage = + "Please don’t spam-ping multiple maintainers at once. Be patient, or join our community Discord for help: https://discord.gg/clawd"; + const mentionRegex = /@([A-Za-z0-9-]+)/g; + const maintainerCache = new Map(); + const normalizeLogin = (login) => login.toLowerCase(); + + const isMaintainer = async (login) => { + if (!login) { + return false; + } + const normalized = normalizeLogin(login); + if (maintainerCache.has(normalized)) { + return maintainerCache.get(normalized); + } + let isMember = false; + try { + const membership = await github.rest.teams.getMembershipForUserInOrg({ + org: context.repo.owner, + team_slug: maintainerTeam, + username: normalized, + }); + isMember = membership?.data?.state === "active"; + } catch (error) { + if (error?.status !== 404) { + throw error; + } + } + maintainerCache.set(normalized, isMember); + return isMember; + }; + + const countMaintainerMentions = async (body, authorLogin) => { + if (!body) { + return 0; + } + const normalizedAuthor = authorLogin ? normalizeLogin(authorLogin) : ""; + if (normalizedAuthor && (await isMaintainer(normalizedAuthor))) { + return 0; + } + + const haystack = body.toLowerCase(); + const teamMention = `@${context.repo.owner.toLowerCase()}/${maintainerTeam}`; + if (haystack.includes(teamMention)) { + return 3; + } + + const mentions = new Set(); + for (const match of body.matchAll(mentionRegex)) { + mentions.add(normalizeLogin(match[1])); + } + if (normalizedAuthor) { + mentions.delete(normalizedAuthor); + } + + let count = 0; + for (const login of mentions) { + if (await isMaintainer(login)) { + count += 1; + } + } + return count; + }; + const triggerLabel = "trigger-response"; const target = context.payload.issue ?? context.payload.pull_request; if (!target) { @@ -72,6 +147,63 @@ jobs: .filter((name) => typeof name === "string"), ); + const issue = context.payload.issue; + const pullRequest = context.payload.pull_request; + const comment = context.payload.comment; + if (comment) { + const authorLogin = comment.user?.login ?? ""; + if (comment.user?.type === "Bot" || authorLogin.endsWith("[bot]")) { + return; + } + + const commentBody = comment.body ?? ""; + const responses = []; + const mentionCount = await countMaintainerMentions(commentBody, authorLogin); + if (mentionCount >= 3) { + responses.push(pingWarningMessage); + } + + const commentHaystack = commentBody.toLowerCase(); + const commentRule = rules.find((item) => + (item.commentTriggers ?? []).some((trigger) => + commentHaystack.includes(trigger), + ), + ); + if (commentRule) { + responses.push(commentRule.message); + } + + if (responses.length > 0) { + await github.rest.issues.createComment({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: target.number, + body: responses.join("\n\n"), + }); + } + return; + } + + if (issue) { + const action = context.payload.action; + if (action === "opened" || action === "edited") { + const issueText = `${issue.title ?? ""}\n${issue.body ?? ""}`.trim(); + const authorLogin = issue.user?.login ?? ""; + const mentionCount = await countMaintainerMentions( + issueText, + authorLogin, + ); + if (mentionCount >= 3) { + await github.rest.issues.createComment({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: issue.number, + body: pingWarningMessage, + }); + } + } + } + const hasTriggerLabel = labelSet.has(triggerLabel); if (hasTriggerLabel) { labelSet.delete(triggerLabel); @@ -94,7 +226,6 @@ jobs: return; } - const issue = context.payload.issue; if (issue) { const title = issue.title ?? ""; const body = issue.body ?? ""; @@ -134,9 +265,8 @@ jobs: const invalidLabel = "invalid"; const dirtyLabel = "dirty"; const noisyPrMessage = - "Closing this PR because it looks dirty (too many unrelated commits). Please recreate the PR from a clean branch."; + "Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch."; - const pullRequest = context.payload.pull_request; if (pullRequest) { if (labelSet.has(dirtyLabel)) { await github.rest.issues.createComment({ diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 5e8a797ce74..19b3cc497e7 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -13,7 +13,7 @@ jobs: # Detect docs-only changes to skip heavy jobs (test, build, Windows, macOS, Android). # Lint and format always run. Fail-safe: if detection fails, run everything. docs-scope: - runs-on: blacksmith-4vcpu-ubuntu-2404 + runs-on: blacksmith-16vcpu-ubuntu-2404 outputs: docs_only: ${{ steps.check.outputs.docs_only }} docs_changed: ${{ steps.check.outputs.docs_changed }} @@ -33,7 +33,7 @@ jobs: changed-scope: needs: [docs-scope] if: needs.docs-scope.outputs.docs_only != 'true' - runs-on: blacksmith-4vcpu-ubuntu-2404 + runs-on: blacksmith-16vcpu-ubuntu-2404 outputs: run_node: ${{ steps.scope.outputs.run_node }} run_macos: ${{ steps.scope.outputs.run_macos }} @@ -127,7 +127,7 @@ jobs: build-artifacts: needs: [docs-scope, changed-scope, check] if: needs.docs-scope.outputs.docs_only != 'true' && (github.event_name == 'push' || needs.changed-scope.outputs.run_node == 'true') - runs-on: blacksmith-4vcpu-ubuntu-2404 + runs-on: blacksmith-16vcpu-ubuntu-2404 steps: - name: Checkout uses: actions/checkout@v4 @@ -153,7 +153,7 @@ jobs: release-check: needs: [docs-scope, build-artifacts] if: github.event_name == 'push' && needs.docs-scope.outputs.docs_only != 'true' - runs-on: blacksmith-4vcpu-ubuntu-2404 + runs-on: blacksmith-16vcpu-ubuntu-2404 steps: - name: Checkout uses: actions/checkout@v4 @@ -177,7 +177,7 @@ jobs: checks: needs: [docs-scope, changed-scope, check] if: needs.docs-scope.outputs.docs_only != 'true' && (github.event_name == 'push' || needs.changed-scope.outputs.run_node == 'true') - runs-on: blacksmith-4vcpu-ubuntu-2404 + runs-on: blacksmith-16vcpu-ubuntu-2404 strategy: fail-fast: false matrix: @@ -192,20 +192,28 @@ jobs: task: test command: pnpm canvas:a2ui:bundle && bunx vitest run --config vitest.unit.config.ts steps: + - name: Skip bun lane on push + if: github.event_name == 'push' && matrix.runtime == 'bun' + run: echo "Skipping bun test lane on push events." + - name: Checkout + if: github.event_name != 'push' || matrix.runtime != 'bun' uses: actions/checkout@v4 with: submodules: false - name: Setup Node environment + if: matrix.runtime != 'bun' || github.event_name != 'push' uses: ./.github/actions/setup-node-env + with: + install-bun: "${{ matrix.runtime == 'bun' }}" - name: Configure vitest JSON reports - if: matrix.task == 'test' && matrix.runtime == 'node' + if: (github.event_name != 'push' || matrix.runtime != 'bun') && matrix.task == 'test' && matrix.runtime == 'node' run: echo "OPENCLAW_VITEST_REPORT_DIR=$RUNNER_TEMP/vitest-reports" >> "$GITHUB_ENV" - name: Configure Node test resources - if: matrix.task == 'test' && matrix.runtime == 'node' + if: (github.event_name != 'push' || matrix.runtime != 'bun') && matrix.task == 'test' && matrix.runtime == 'node' run: | # `pnpm test` runs `scripts/test-parallel.mjs`, which spawns multiple Node processes. # Default heap limits have been too low on Linux CI (V8 OOM near 4GB). @@ -213,16 +221,17 @@ jobs: echo "OPENCLAW_TEST_MAX_OLD_SPACE_SIZE_MB=6144" >> "$GITHUB_ENV" - name: Run ${{ matrix.task }} (${{ matrix.runtime }}) + if: matrix.runtime != 'bun' || github.event_name != 'push' run: ${{ matrix.command }} - name: Summarize slowest tests - if: matrix.task == 'test' && matrix.runtime == 'node' + if: (github.event_name != 'push' || matrix.runtime != 'bun') && matrix.task == 'test' && matrix.runtime == 'node' run: | node scripts/vitest-slowest.mjs --dir "$OPENCLAW_VITEST_REPORT_DIR" --top 50 --out "$RUNNER_TEMP/vitest-slowest.md" > /dev/null echo "Slowest test summary written to $RUNNER_TEMP/vitest-slowest.md" - name: Upload vitest reports - if: matrix.task == 'test' && matrix.runtime == 'node' + if: (github.event_name != 'push' || matrix.runtime != 'bun') && matrix.task == 'test' && matrix.runtime == 'node' uses: actions/upload-artifact@v4 with: name: vitest-reports-${{ runner.os }}-${{ matrix.runtime }} @@ -235,7 +244,7 @@ jobs: name: "check" needs: [docs-scope] if: needs.docs-scope.outputs.docs_only != 'true' - runs-on: blacksmith-4vcpu-ubuntu-2404 + runs-on: blacksmith-16vcpu-ubuntu-2404 steps: - name: Checkout uses: actions/checkout@v4 @@ -244,15 +253,59 @@ jobs: - name: Setup Node environment uses: ./.github/actions/setup-node-env + with: + install-bun: "false" - name: Check types and lint and oxfmt run: pnpm check + - name: Enforce safe external URL opening policy + run: pnpm lint:ui:no-raw-window-open + + # Report-only dead-code scans. Runs after scope detection and stores machine-readable + # results as artifacts for later triage before we enable hard gates. + # Temporarily disabled in CI while we process initial findings. + deadcode: + name: dead-code report + needs: [docs-scope, changed-scope] + # if: needs.docs-scope.outputs.docs_only != 'true' && (github.event_name == 'push' || needs.changed-scope.outputs.run_node == 'true') + if: false + runs-on: blacksmith-16vcpu-ubuntu-2404 + strategy: + fail-fast: false + matrix: + include: + - tool: knip + command: pnpm deadcode:report:ci:knip + - tool: ts-prune + command: pnpm deadcode:report:ci:ts-prune + - tool: ts-unused-exports + command: pnpm deadcode:report:ci:ts-unused + steps: + - name: Checkout + uses: actions/checkout@v4 + with: + submodules: false + + - name: Setup Node environment + uses: ./.github/actions/setup-node-env + with: + install-bun: "false" + + - name: Run ${{ matrix.tool }} dead-code scan + run: ${{ matrix.command }} + + - name: Upload dead-code results + uses: actions/upload-artifact@v4 + with: + name: dead-code-${{ matrix.tool }}-${{ github.run_id }} + path: .artifacts/deadcode + # Validate docs (format, lint, broken links) only when docs files changed. check-docs: needs: [docs-scope] if: needs.docs-scope.outputs.docs_changed == 'true' - runs-on: blacksmith-4vcpu-ubuntu-2404 + runs-on: blacksmith-16vcpu-ubuntu-2404 steps: - name: Checkout uses: actions/checkout@v4 @@ -261,12 +314,16 @@ jobs: - name: Setup Node environment uses: ./.github/actions/setup-node-env + with: + install-bun: "false" - name: Check docs run: pnpm check:docs - secrets: - runs-on: blacksmith-4vcpu-ubuntu-2404 + skills-python: + needs: [docs-scope, changed-scope] + if: needs.docs-scope.outputs.docs_only != 'true' && (github.event_name == 'push' || needs.changed-scope.outputs.run_node == 'true') + runs-on: blacksmith-16vcpu-ubuntu-2404 steps: - name: Checkout uses: actions/checkout@v4 @@ -278,10 +335,39 @@ jobs: with: python-version: "3.12" - - name: Install detect-secrets + - name: Install Python tooling run: | python -m pip install --upgrade pip - python -m pip install detect-secrets==1.5.0 + python -m pip install pytest ruff pyyaml + + - name: Lint Python skill scripts + run: python -m ruff check skills + + - name: Test skill Python scripts + run: python -m pytest -q skills + + secrets: + runs-on: blacksmith-16vcpu-ubuntu-2404 + steps: + - name: Checkout + uses: actions/checkout@v4 + with: + submodules: false + + - name: Setup Node environment + uses: ./.github/actions/setup-node-env + with: + install-bun: "false" + + - name: Setup Python + uses: actions/setup-python@v5 + with: + python-version: "3.12" + + - name: Install pre-commit + run: | + python -m pip install --upgrade pip + python -m pip install pre-commit detect-secrets==1.5.0 - name: Detect secrets run: | @@ -290,13 +376,38 @@ jobs: exit 1 fi + - name: Detect committed private keys + run: pre-commit run --all-files detect-private-key + + - name: Audit changed GitHub workflows with zizmor + run: | + set -euo pipefail + + if [ "${{ github.event_name }}" = "push" ]; then + BASE="${{ github.event.before }}" + else + BASE="${{ github.event.pull_request.base.sha }}" + fi + + mapfile -t workflow_files < <(git diff --name-only "$BASE" HEAD -- '.github/workflows/*.yml' '.github/workflows/*.yaml') + if [ "${#workflow_files[@]}" -eq 0 ]; then + echo "No workflow changes detected; skipping zizmor." + exit 0 + fi + + pre-commit run zizmor --files "${workflow_files[@]}" + + - name: Audit production dependencies + run: pre-commit run --all-files pnpm-audit-prod + checks-windows: needs: [docs-scope, changed-scope, build-artifacts, check] if: needs.docs-scope.outputs.docs_only != 'true' && (github.event_name == 'push' || needs.changed-scope.outputs.run_node == 'true') - runs-on: blacksmith-4vcpu-windows-2025 + runs-on: blacksmith-16vcpu-windows-2025 + timeout-minutes: 45 env: NODE_OPTIONS: --max-old-space-size=4096 - # Keep total concurrency predictable on the 4 vCPU runner: + # Keep total concurrency predictable on the 16 vCPU runner: # `scripts/test-parallel.mjs` runs some vitest suites in parallel processes. OPENCLAW_TEST_WORKERS: 2 defaults: @@ -308,12 +419,23 @@ jobs: include: - runtime: node task: lint + shard_index: 0 + shard_count: 1 command: pnpm lint - runtime: node task: test + shard_index: 1 + shard_count: 2 + command: pnpm canvas:a2ui:bundle && pnpm test + - runtime: node + task: test + shard_index: 2 + shard_count: 2 command: pnpm canvas:a2ui:bundle && pnpm test - runtime: node task: protocol + shard_index: 0 + shard_count: 1 command: pnpm protocol:check steps: - name: Checkout @@ -355,7 +477,7 @@ jobs: test -s dist/plugin-sdk/index.js - name: Setup Node.js - uses: actions/setup-node@v4 + uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version: 22.x check-latest: true @@ -366,16 +488,10 @@ jobs: pnpm-version: "10.23.0" cache-key-suffix: "node22" - - name: Setup Bun - uses: oven-sh/setup-bun@v2 - with: - bun-version: latest - - name: Runtime versions run: | node -v npm -v - bun -v pnpm -v - name: Capture node path @@ -391,6 +507,12 @@ jobs: pnpm -v pnpm install --frozen-lockfile --ignore-scripts=false --config.engine-strict=false --config.enable-pre-post-scripts=true || pnpm install --frozen-lockfile --ignore-scripts=false --config.engine-strict=false --config.enable-pre-post-scripts=true + - name: Configure test shard (Windows) + if: matrix.task == 'test' + run: | + echo "OPENCLAW_TEST_SHARDS=${{ matrix.shard_count }}" >> "$GITHUB_ENV" + echo "OPENCLAW_TEST_SHARD_INDEX=${{ matrix.shard_index }}" >> "$GITHUB_ENV" + - name: Configure vitest JSON reports if: matrix.task == 'test' run: echo "OPENCLAW_VITEST_REPORT_DIR=$RUNNER_TEMP/vitest-reports" >> "$GITHUB_ENV" @@ -408,7 +530,7 @@ jobs: if: matrix.task == 'test' uses: actions/upload-artifact@v4 with: - name: vitest-reports-${{ runner.os }}-${{ matrix.runtime }} + name: vitest-reports-${{ runner.os }}-${{ matrix.runtime }}-shard${{ matrix.shard_index }}of${{ matrix.shard_count }} path: | ${{ env.OPENCLAW_VITEST_REPORT_DIR }} ${{ runner.temp }}/vitest-slowest.md @@ -653,7 +775,7 @@ jobs: android: needs: [docs-scope, changed-scope, check] if: needs.docs-scope.outputs.docs_only != 'true' && (github.event_name == 'push' || needs.changed-scope.outputs.run_android == 'true') - runs-on: blacksmith-4vcpu-ubuntu-2404 + runs-on: blacksmith-16vcpu-ubuntu-2404 strategy: fail-fast: false matrix: diff --git a/.github/workflows/docker-release.yml b/.github/workflows/docker-release.yml index 05e63005dd5..eff0993b466 100644 --- a/.github/workflows/docker-release.yml +++ b/.github/workflows/docker-release.yml @@ -24,13 +24,12 @@ env: jobs: # Build amd64 image build-amd64: - runs-on: ubuntu-latest + runs-on: blacksmith-16vcpu-ubuntu-2404 permissions: packages: write contents: read outputs: image-digest: ${{ steps.build.outputs.digest }} - image-metadata: ${{ steps.meta.outputs.json }} steps: - name: Checkout uses: actions/checkout@v4 @@ -45,18 +44,30 @@ jobs: username: ${{ github.repository_owner }} password: ${{ secrets.GITHUB_TOKEN }} - - name: Extract metadata - id: meta - uses: docker/metadata-action@v5 - with: - images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} - tags: | - type=ref,event=branch - type=semver,pattern={{version}} - type=semver,pattern={{version}},suffix=-amd64 - type=semver,pattern={{version}},suffix=-arm64 - type=ref,event=branch,suffix=-amd64 - type=ref,event=branch,suffix=-arm64 + - name: Resolve image tags (amd64) + id: tags + shell: bash + env: + IMAGE: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} + run: | + set -euo pipefail + tags=() + if [[ "${GITHUB_REF}" == "refs/heads/main" ]]; then + tags+=("${IMAGE}:main-amd64") + fi + if [[ "${GITHUB_REF}" == refs/tags/v* ]]; then + version="${GITHUB_REF#refs/tags/v}" + tags+=("${IMAGE}:${version}-amd64") + fi + if [[ ${#tags[@]} -eq 0 ]]; then + echo "::error::No amd64 tags resolved for ref ${GITHUB_REF}" + exit 1 + fi + { + echo "value<> "$GITHUB_OUTPUT" - name: Build and push amd64 image id: build @@ -64,8 +75,7 @@ jobs: with: context: . platforms: linux/amd64 - labels: ${{ steps.meta.outputs.labels }} - tags: ${{ steps.meta.outputs.tags }} + tags: ${{ steps.tags.outputs.value }} cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-cache:amd64 cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-cache:amd64,mode=max provenance: false @@ -73,13 +83,12 @@ jobs: # Build arm64 image build-arm64: - runs-on: ubuntu-24.04-arm + runs-on: blacksmith-16vcpu-ubuntu-2404-arm permissions: packages: write contents: read outputs: image-digest: ${{ steps.build.outputs.digest }} - image-metadata: ${{ steps.meta.outputs.json }} steps: - name: Checkout uses: actions/checkout@v4 @@ -94,18 +103,30 @@ jobs: username: ${{ github.repository_owner }} password: ${{ secrets.GITHUB_TOKEN }} - - name: Extract metadata - id: meta - uses: docker/metadata-action@v5 - with: - images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} - tags: | - type=ref,event=branch - type=semver,pattern={{version}} - type=semver,pattern={{version}},suffix=-amd64 - type=semver,pattern={{version}},suffix=-arm64 - type=ref,event=branch,suffix=-amd64 - type=ref,event=branch,suffix=-arm64 + - name: Resolve image tags (arm64) + id: tags + shell: bash + env: + IMAGE: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} + run: | + set -euo pipefail + tags=() + if [[ "${GITHUB_REF}" == "refs/heads/main" ]]; then + tags+=("${IMAGE}:main-arm64") + fi + if [[ "${GITHUB_REF}" == refs/tags/v* ]]; then + version="${GITHUB_REF#refs/tags/v}" + tags+=("${IMAGE}:${version}-arm64") + fi + if [[ ${#tags[@]} -eq 0 ]]; then + echo "::error::No arm64 tags resolved for ref ${GITHUB_REF}" + exit 1 + fi + { + echo "value<> "$GITHUB_OUTPUT" - name: Build and push arm64 image id: build @@ -113,8 +134,7 @@ jobs: with: context: . platforms: linux/arm64 - labels: ${{ steps.meta.outputs.labels }} - tags: ${{ steps.meta.outputs.tags }} + tags: ${{ steps.tags.outputs.value }} cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-cache:arm64 cache-to: type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-cache:arm64,mode=max provenance: false @@ -122,12 +142,15 @@ jobs: # Create multi-platform manifest create-manifest: - runs-on: ubuntu-latest + runs-on: blacksmith-16vcpu-ubuntu-2404 permissions: packages: write contents: read needs: [build-amd64, build-arm64] steps: + - name: Checkout + uses: actions/checkout@v4 + - name: Login to GitHub Container Registry uses: docker/login-action@v3 with: @@ -135,19 +158,44 @@ jobs: username: ${{ github.repository_owner }} password: ${{ secrets.GITHUB_TOKEN }} - - name: Extract metadata for manifest - id: meta - uses: docker/metadata-action@v5 - with: - images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} - tags: | - type=ref,event=branch - type=semver,pattern={{version}} + - name: Resolve manifest tags + id: tags + shell: bash + env: + IMAGE: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} + run: | + set -euo pipefail + tags=() + if [[ "${GITHUB_REF}" == "refs/heads/main" ]]; then + tags+=("${IMAGE}:main") + fi + if [[ "${GITHUB_REF}" == refs/tags/v* ]]; then + version="${GITHUB_REF#refs/tags/v}" + tags+=("${IMAGE}:${version}") + if [[ "$version" =~ ^[0-9]+\.[0-9]+\.[0-9]+(-[0-9]+)?$ ]]; then + tags+=("${IMAGE}:latest") + fi + fi + if [[ ${#tags[@]} -eq 0 ]]; then + echo "::error::No manifest tags resolved for ref ${GITHUB_REF}" + exit 1 + fi + { + echo "value<> "$GITHUB_OUTPUT" - name: Create and push manifest + shell: bash run: | - docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \ + set -euo pipefail + mapfile -t tags <<< "${{ steps.tags.outputs.value }}" + args=() + for tag in "${tags[@]}"; do + [ -z "$tag" ] && continue + args+=("-t" "$tag") + done + docker buildx imagetools create "${args[@]}" \ ${{ needs.build-amd64.outputs.image-digest }} \ ${{ needs.build-arm64.outputs.image-digest }} - env: - DOCKER_METADATA_OUTPUT_JSON: ${{ steps.meta.outputs.json }} diff --git a/.github/workflows/install-smoke.yml b/.github/workflows/install-smoke.yml index 45154a5fab4..fd0ac45799d 100644 --- a/.github/workflows/install-smoke.yml +++ b/.github/workflows/install-smoke.yml @@ -12,7 +12,7 @@ concurrency: jobs: docs-scope: - runs-on: ubuntu-latest + runs-on: blacksmith-16vcpu-ubuntu-2404 outputs: docs_only: ${{ steps.check.outputs.docs_only }} steps: @@ -28,13 +28,13 @@ jobs: install-smoke: needs: [docs-scope] if: needs.docs-scope.outputs.docs_only != 'true' - runs-on: ubuntu-latest + runs-on: blacksmith-16vcpu-ubuntu-2404 steps: - name: Checkout CLI uses: actions/checkout@v4 - name: Setup Node.js - uses: actions/setup-node@v4 + uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 with: node-version: 22.x check-latest: true @@ -48,6 +48,11 @@ jobs: - name: Install pnpm deps (minimal) run: pnpm install --ignore-scripts --frozen-lockfile + - name: Run root Dockerfile CLI smoke + run: | + docker build -t openclaw-dockerfile-smoke:local -f Dockerfile . + docker run --rm --entrypoint sh openclaw-dockerfile-smoke:local -lc 'which openclaw && openclaw --version' + - name: Run installer docker tests env: CLAWDBOT_INSTALL_URL: https://openclaw.ai/install.sh diff --git a/.github/workflows/labeler.yml b/.github/workflows/labeler.yml index 7a3c4b1457c..ed86b4c67bb 100644 --- a/.github/workflows/labeler.yml +++ b/.github/workflows/labeler.yml @@ -23,22 +23,29 @@ jobs: permissions: contents: read pull-requests: write - runs-on: self-hosted + runs-on: blacksmith-16vcpu-ubuntu-2404 steps: - uses: actions/create-github-app-token@d72941d797fd3113feb6b93fd0dec494b13a2547 # v1 id: app-token + continue-on-error: true with: app-id: "2729701" private-key: ${{ secrets.GH_APP_PRIVATE_KEY }} + - uses: actions/create-github-app-token@d72941d797fd3113feb6b93fd0dec494b13a2547 # v1 + id: app-token-fallback + if: steps.app-token.outcome == 'failure' + with: + app-id: "2971289" + private-key: ${{ secrets.GH_APP_PRIVATE_KEY_FALLBACK }} - uses: actions/labeler@8558fd74291d67161a8a78ce36a881fa63b766a9 # v5 with: configuration-path: .github/labeler.yml - repo-token: ${{ steps.app-token.outputs.token }} + repo-token: ${{ steps.app-token.outputs.token || steps.app-token-fallback.outputs.token }} sync-labels: true - name: Apply PR size label uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7 with: - github-token: ${{ steps.app-token.outputs.token }} + github-token: ${{ steps.app-token.outputs.token || steps.app-token-fallback.outputs.token }} script: | const pullRequest = context.payload.pull_request; if (!pullRequest) { @@ -127,7 +134,7 @@ jobs: - name: Apply maintainer or trusted-contributor label uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7 with: - github-token: ${{ steps.app-token.outputs.token }} + github-token: ${{ steps.app-token.outputs.token || steps.app-token-fallback.outputs.token }} script: | const login = context.payload.pull_request?.user?.login; if (!login) { @@ -200,17 +207,24 @@ jobs: permissions: contents: read pull-requests: write - runs-on: self-hosted + runs-on: blacksmith-16vcpu-ubuntu-2404 steps: - uses: actions/create-github-app-token@d72941d797fd3113feb6b93fd0dec494b13a2547 # v1 id: app-token + continue-on-error: true with: app-id: "2729701" private-key: ${{ secrets.GH_APP_PRIVATE_KEY }} + - uses: actions/create-github-app-token@d72941d797fd3113feb6b93fd0dec494b13a2547 # v1 + id: app-token-fallback + if: steps.app-token.outcome == 'failure' + with: + app-id: "2971289" + private-key: ${{ secrets.GH_APP_PRIVATE_KEY_FALLBACK }} - name: Backfill PR labels uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7 with: - github-token: ${{ steps.app-token.outputs.token }} + github-token: ${{ steps.app-token.outputs.token || steps.app-token-fallback.outputs.token }} script: | const owner = context.repo.owner; const repo = context.repo.repo; @@ -440,17 +454,24 @@ jobs: label-issues: permissions: issues: write - runs-on: self-hosted + runs-on: blacksmith-16vcpu-ubuntu-2404 steps: - uses: actions/create-github-app-token@d72941d797fd3113feb6b93fd0dec494b13a2547 # v1 id: app-token + continue-on-error: true with: app-id: "2729701" private-key: ${{ secrets.GH_APP_PRIVATE_KEY }} + - uses: actions/create-github-app-token@d72941d797fd3113feb6b93fd0dec494b13a2547 # v1 + id: app-token-fallback + if: steps.app-token.outcome == 'failure' + with: + app-id: "2971289" + private-key: ${{ secrets.GH_APP_PRIVATE_KEY_FALLBACK }} - name: Apply maintainer or trusted-contributor label uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7 with: - github-token: ${{ steps.app-token.outputs.token }} + github-token: ${{ steps.app-token.outputs.token || steps.app-token-fallback.outputs.token }} script: | const login = context.payload.issue?.user?.login; if (!login) { diff --git a/.github/workflows/sandbox-common-smoke.yml b/.github/workflows/sandbox-common-smoke.yml index c92a05c3aeb..26c0dcc106f 100644 --- a/.github/workflows/sandbox-common-smoke.yml +++ b/.github/workflows/sandbox-common-smoke.yml @@ -19,7 +19,7 @@ concurrency: jobs: sandbox-common-smoke: - runs-on: ubuntu-latest + runs-on: blacksmith-16vcpu-ubuntu-2404 steps: - name: Checkout uses: actions/checkout@v4 diff --git a/.github/workflows/stale.yml b/.github/workflows/stale.yml index 7210fb7a739..744d71bd574 100644 --- a/.github/workflows/stale.yml +++ b/.github/workflows/stale.yml @@ -12,17 +12,24 @@ jobs: permissions: issues: write pull-requests: write - runs-on: self-hosted + runs-on: blacksmith-16vcpu-ubuntu-2404 steps: - uses: actions/create-github-app-token@d72941d797fd3113feb6b93fd0dec494b13a2547 # v1 id: app-token + continue-on-error: true with: app-id: "2729701" private-key: ${{ secrets.GH_APP_PRIVATE_KEY }} + - uses: actions/create-github-app-token@d72941d797fd3113feb6b93fd0dec494b13a2547 # v1 + id: app-token-fallback + if: steps.app-token.outcome == 'failure' + with: + app-id: "2971289" + private-key: ${{ secrets.GH_APP_PRIVATE_KEY_FALLBACK }} - name: Mark stale issues and pull requests uses: actions/stale@v9 with: - repo-token: ${{ steps.app-token.outputs.token }} + repo-token: ${{ steps.app-token.outputs.token || steps.app-token-fallback.outputs.token }} days-before-issue-stale: 7 days-before-issue-close: 5 days-before-pr-stale: 5 diff --git a/.github/workflows/workflow-sanity.yml b/.github/workflows/workflow-sanity.yml index 438a71162da..19668e697ad 100644 --- a/.github/workflows/workflow-sanity.yml +++ b/.github/workflows/workflow-sanity.yml @@ -11,7 +11,7 @@ concurrency: jobs: no-tabs: - runs-on: ubuntu-latest + runs-on: blacksmith-16vcpu-ubuntu-2404 steps: - name: Checkout uses: actions/checkout@v4 @@ -40,3 +40,28 @@ jobs: print(f"- {path}") sys.exit(1) PY + + actionlint: + runs-on: blacksmith-16vcpu-ubuntu-2404 + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Install actionlint + shell: bash + run: | + set -euo pipefail + ACTIONLINT_VERSION="1.7.11" + archive="actionlint_${ACTIONLINT_VERSION}_linux_amd64.tar.gz" + base_url="https://github.com/rhysd/actionlint/releases/download/v${ACTIONLINT_VERSION}" + curl -sSfL -o "${archive}" "${base_url}/${archive}" + curl -sSfL -o checksums.txt "${base_url}/actionlint_${ACTIONLINT_VERSION}_checksums.txt" + grep " ${archive}\$" checksums.txt | sha256sum -c - + tar -xzf "${archive}" actionlint + sudo install -m 0755 actionlint /usr/local/bin/actionlint + + - name: Lint workflows + run: actionlint + + - name: Disallow direct inputs interpolation in composite run blocks + run: python3 scripts/check-composite-action-input-interpolation.py diff --git a/.gitignore b/.gitignore index 118e705838c..b5d3257e7e6 100644 --- a/.gitignore +++ b/.gitignore @@ -17,6 +17,11 @@ __pycache__/ ui/src/ui/__screenshots__/ ui/playwright-report/ ui/test-results/ +packages/dashboard-next/.next/ +packages/dashboard-next/out/ + +# Mise configuration files +mise.toml # Android build artifacts apps/android/.gradle/ @@ -89,6 +94,29 @@ USER.md !.agent/workflows/ /local/ package-lock.json +.claude/settings.local.json +.agents/ +.agents +.agent/ +skills-lock.json # Local iOS signing overrides apps/ios/LocalSigning.xcconfig + +# Xcode build directories (xcodebuild output) +apps/ios/build/ +apps/shared/OpenClawKit/build/ +Swabble/build/ + +# Generated protocol schema (produced via pnpm protocol:gen) +dist/protocol.schema.json +.ant-colony/ + +# Eclipse +**/.project +**/.classpath +**/.settings/ +**/.gradle/ + +# Synthing +**/.stfolder/ diff --git a/.mailmap b/.mailmap new file mode 100644 index 00000000000..9190f88b6e0 --- /dev/null +++ b/.mailmap @@ -0,0 +1,13 @@ +# Canonical contributor identity mappings for cherry-picked commits. +bmendonca3 <208517100+bmendonca3@users.noreply.github.com> +hcl <7755017+hclsys@users.noreply.github.com> +Glucksberg <80581902+Glucksberg@users.noreply.github.com> +JackyWay <53031570+JackyWay@users.noreply.github.com> +Marcus Castro <7562095+mcaxtr@users.noreply.github.com> +Marc Gratch <2238658+mgratch@users.noreply.github.com> +Peter Machona <7957943+chilu18@users.noreply.github.com> +Ben Marvell <92585+easternbloc@users.noreply.github.com> +zerone0x <39543393+zerone0x@users.noreply.github.com> +Marco Di Dionisio <3519682+marcodd23@users.noreply.github.com> +mujiannan <46643837+mujiannan@users.noreply.github.com> +Santhanakrishnan <239082898+bitfoundry-ai@users.noreply.github.com> diff --git a/.npmrc b/.npmrc index f0c783cb6c8..05620061611 100644 --- a/.npmrc +++ b/.npmrc @@ -1 +1 @@ -allow-build-scripts=@whiskeysockets/baileys,sharp,esbuild,protobufjs,fs-ext,node-pty,@lydell/node-pty,@matrix-org/matrix-sdk-crypto-nodejs +# pnpm build-script allowlist lives in package.json -> pnpm.onlyBuiltDependencies. diff --git a/.oxfmtrc.jsonc b/.oxfmtrc.jsonc index 41a8ca9d543..0a928d5f9ba 100644 --- a/.oxfmtrc.jsonc +++ b/.oxfmtrc.jsonc @@ -6,14 +6,19 @@ "experimentalSortPackageJson": { "sortScripts": true, }, + "tabWidth": 2, + "useTabs": false, "ignorePatterns": [ "apps/", "assets/", + "CLAUDE.md", + "docker-compose.yml", "dist/", "docs/_layouts/", "node_modules/", "patches/", "pnpm-lock.yaml/", + "src/gateway/server-methods/CLAUDE.md", "src/auto-reply/reply/export-html/", "Swabble/", "vendor/", diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index e946d18c112..30b6363a34d 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -18,6 +18,8 @@ repos: - id: check-added-large-files args: [--maxkb=500] - id: check-merge-conflict + - id: detect-private-key + exclude: '(^|/)(\.secrets\.baseline$|\.detect-secrets\.cfg$|\.pre-commit-config\.yaml$|apps/ios/fastlane/Fastfile$|.*\.test\.ts$)' # Secret detection (same as CI) - repo: https://github.com/Yelp/detect-secrets @@ -45,7 +47,6 @@ repos: - '=== "string"' - --exclude-lines - 'typeof remote\?\.password === "string"' - # Shell script linting - repo: https://github.com/koalaman/shellcheck-precommit rev: v0.11.0 @@ -69,9 +70,34 @@ repos: args: [--persona=regular, --min-severity=medium, --min-confidence=medium] exclude: "^(vendor/|Swabble/)" + # Python checks for skills scripts + - repo: https://github.com/astral-sh/ruff-pre-commit + rev: v0.14.1 + hooks: + - id: ruff + files: "^skills/.*\\.py$" + args: [--config, pyproject.toml] + + - repo: local + hooks: + - id: skills-python-tests + name: skills python tests + entry: pytest -q skills + language: python + additional_dependencies: [pytest>=8, <9] + pass_filenames: false + files: "^skills/.*\\.py$" + # Project checks (same commands as CI) - repo: local hooks: + # pnpm audit --prod --audit-level=high + - id: pnpm-audit-prod + name: pnpm-audit-prod + entry: pnpm audit --prod --audit-level=high + language: system + pass_filenames: false + # oxlint --type-aware src test - id: oxlint name: oxlint diff --git a/AGENTS.md b/AGENTS.md index dd4ba975b53..a0eca723170 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,7 +1,11 @@ # Repository Guidelines - Repo: https://github.com/openclaw/openclaw +- In chat replies, file references must be repo-root relative only (example: `extensions/bluebubbles/src/channel.ts:80`); never absolute paths or `~/...`. - GitHub issues/comments/PR comments: use literal multiline strings or `-F - <<'EOF'` (or $'...') for real newlines; never embed "\\n". +- GitHub comment footgun: never use `gh issue/pr comment -b "..."` when body contains backticks or shell chars. Always use single-quoted heredoc (`-F - <<'EOF'`) so no command substitution/escaping corruption. +- GitHub linking footgun: don’t wrap issue/PR refs like `#24643` in backticks when you want auto-linking. Use plain `#24643` (optionally add full URL). +- Security advisory analysis: before triage/severity decisions, read `SECURITY.md` to align with OpenClaw's trust model and design boundaries. ## Project Structure & Module Organization @@ -83,6 +87,7 @@ - stable: tagged releases only (e.g. `vYYYY.M.D`), npm dist-tag `latest`. - beta: prerelease tags `vYYYY.M.D-beta.N`, npm dist-tag `beta` (may ship without macOS app). +- beta naming: prefer `-beta.N`; do not mint new `-1/-2` betas. Legacy `vYYYY.M.D-` and `vYYYY.M.D.beta.N` remain recognized. - dev: moving head on `main` (no tag; git checkout main). ## Testing Guidelines @@ -91,6 +96,7 @@ - Naming: match source names with `*.test.ts`; e2e in `*.e2e.test.ts`. - Run `pnpm test` (or `pnpm test:coverage`) before pushing when you touch logic. - Do not set test workers above 16; tried already. +- If local Vitest runs cause memory pressure (common on non-Mac-Studio hosts), use `OPENCLAW_TEST_PROFILE=low OPENCLAW_TEST_SERIAL_GATEWAY=1 pnpm test` for land/gate runs. - Live tests (real keys): `CLAWDBOT_LIVE_TEST=1 pnpm test:live` (OpenClaw-only) or `LIVE=1 pnpm test:live` (includes provider live tests). Docker: `pnpm test:docker:live-models`, `pnpm test:docker:live-gateway`. Onboarding Docker E2E: `pnpm test:docker:onboard`. - Full kit + what’s covered: `docs/testing.md`. - Changelog: user-facing changes only; no internal/meta notes (version alignment, appcast reminders, release process). @@ -116,6 +122,15 @@ - If `git branch -d/-D ` is policy-blocked, delete the local ref directly: `git update-ref -d refs/heads/`. - Bulk PR close/reopen safety: if a close action would affect more than 5 PRs, first ask for explicit user confirmation with the exact PR count and target scope/query. +## GitHub Search (`gh`) + +- Prefer targeted keyword search before proposing new work or duplicating fixes. +- Use `--repo openclaw/openclaw` + `--match title,body` first; add `--match comments` when triaging follow-up threads. +- PRs: `gh search prs --repo openclaw/openclaw --match title,body --limit 50 -- "auto-update"` +- Issues: `gh search issues --repo openclaw/openclaw --match title,body --limit 50 -- "auto-update"` +- Structured output example: + `gh search issues --repo openclaw/openclaw --match title,body --limit 50 --json number,title,state,url,updatedAt -- "auto update" --jq '.[] | "\(.number) | \(.state) | \(.title) | \(.url)"'` + ## Security & Configuration Tips - Web provider stores creds at `~/.openclaw/credentials/`; rerun `openclaw login` if logged out. @@ -126,6 +141,7 @@ ## GHSA (Repo Advisory) Patch/Publish +- Before reviewing security advisories, read `SECURITY.md`. - Fetch: `gh api /repos/openclaw/openclaw/security-advisories/` - Latest npm: `npm view openclaw version --userconfig "$(mktemp)"` - Private fork PRs must be closed: @@ -133,6 +149,7 @@ `gh pr list -R "$fork" --state open` (must be empty) - Description newline footgun: write Markdown via heredoc to `/tmp/ghsa.desc.md` (no `"\\n"` strings) - Build patch JSON via jq: `jq -n --rawfile desc /tmp/ghsa.desc.md '{summary,severity,description:$desc,vulnerabilities:[...]}' > /tmp/ghsa.patch.json` +- GHSA API footgun: cannot set `severity` and `cvss_vector_string` in the same PATCH; do separate calls. - Patch + publish: `gh api -X PATCH /repos/openclaw/openclaw/security-advisories/ --input /tmp/ghsa.patch.json` (publish = include `"state":"published"`; no `/publish` endpoint) - If publish fails (HTTP 422): missing `severity`/`description`/`vulnerabilities[]`, or private fork has open PRs - Verify: re-fetch; ensure `state=published`, `published_at` set; `jq -r .description | rg '\\\\n'` returns nothing @@ -191,6 +208,7 @@ - launchd PATH is minimal; ensure the app’s launch agent PATH includes standard system paths plus your pnpm bin (typically `$HOME/Library/pnpm`) so `pnpm`/`openclaw` binaries resolve when invoked via `openclaw-mac`. - For manual `openclaw message send` messages that include `!`, use the heredoc pattern noted below to avoid the Bash tool’s escaping. - Release guardrails: do not change version numbers without operator’s explicit consent; always ask permission before running any npm publish/release step. +- Beta release guardrail: when using a beta Git tag (for example `vYYYY.M.D-beta.N`), publish npm with a matching beta version suffix (for example `YYYY.M.D-beta.N`) rather than a plain version on `--tag beta`; otherwise the plain version name gets consumed/blocked. ## NPM + 1Password (publish/verify) diff --git a/CHANGELOG.md b/CHANGELOG.md index 015f032c983..0cb253433ac 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,39 +2,932 @@ Docs: https://docs.openclaw.ai -## 2026.2.18 (Unreleased) +## 2026.2.27 ### Changes -- iOS/Gateway: wake disconnected iOS nodes via APNs before `nodes.invoke` and auto-reconnect gateway sessions on silent push wake to reduce invoke failures while the app is backgrounded. (#20332) Thanks @mbelinky. -- iOS/APNs: add push registration and notification-signing configuration for node delivery. (#20308) Thanks @mbelinky. -- Gateway/APNs: add a push-test pipeline for APNs delivery validation in gateway flows. (#20307) Thanks @mbelinky. -- iOS/Watch: add an Apple Watch companion MVP with watch inbox UI, watch notification relay handling, and gateway command surfaces for watch status/send flows. (#20054) Thanks @mbelinky. -- Gateway/CLI: add paired-device hygiene flows with `device.pair.remove`, plus `openclaw devices remove` and guarded `openclaw devices clear --yes [--pending]` commands for removing paired entries and optionally rejecting pending requests. (#20057) Thanks @mbelinky. -- Skills: harden coding-agent skill guidance by removing shell-command examples that interpolate untrusted issue text directly into command strings. +- Web UI/i18n: add German (`de`) locale support and auto-render language options from supported locale constants in Overview settings. (#28495) thanks @dsantoreis. +- Discord/Thread bindings: replace fixed TTL lifecycle with inactivity (`idleHours`, default 24h) plus optional hard `maxAgeHours` lifecycle controls, and add `/session idle` + `/session max-age` commands for focused thread-bound sessions. (#27845) Thanks @osolmaz. +- Android/Nodes: add `camera.list`, `device.permissions`, `device.health`, and `notifications.actions` (`open`/`dismiss`/`reply`) on Android nodes, plus first-class node-tool actions for the new device/notification commands. (#28260) Thanks @obviyus. +- Android/Nodes parity: add `system.notify`, `photos.latest`, `contacts.search`/`contacts.add`, `calendar.events`/`calendar.add`, and `motion.activity`/`motion.pedometer`, with motion sensor-aware command gating and improved activity sampling reliability. (#29398) Thanks @obviyus. +- Android/Gateway capability refresh: add live Android capability integration coverage and node canvas capability refresh wiring, plus runtime hardening for A2UI readiness retries, scoped canvas URL normalization, debug diagnostics JSON, and JavaScript MIME delivery. (#28388) Thanks @obviyus. +- Feishu/Doc permissions: support optional owner permission grant fields on `feishu_doc` create and report permission metadata only when the grant call succeeds, with regression coverage for success/failure/omitted-owner paths. (#28295) Thanks @zhoulongchao77. +- Feishu/Docx tables + uploads: add `feishu_doc` actions for Docx table creation/cell writing (`create_table`, `write_table_cells`, `create_table_with_values`) and image/file uploads (`upload_image`, `upload_file`) with stricter create/upload error handling for missing `document_id` and placeholder cleanup failures. (#20304) Thanks @xuhao1. +- Feishu/Reactions: add inbound `im.message.reaction.created_v1` handling, route verified reactions through synthetic inbound turns, and harden verification with timeout + fail-closed filtering so non-bot or unverified reactions are dropped. (#16716) Thanks @schumilin. +- Memory/LanceDB: support custom OpenAI `baseUrl` and embedding dimensions for LanceDB memory. (#17874) Thanks @rish2jain and @vincentkoc. ### Fixes +- Feishu/Reply media attachments: send Feishu reply `mediaUrl`/`mediaUrls` payloads as attachments alongside text/streamed replies in the reply dispatcher, including legacy fallback when `mediaUrls` is empty. (#28959) +- Feishu/Reaction notifications: add `channels.feishu.reactionNotifications` (`off | own | all`, default `own`) so operators can disable reaction ingress or allow all verified reaction events (not only bot-authored message reactions). (#28529) +- Feishu/Outbound session routing: stop assuming bare `oc_` identifiers are always group chats, honor explicit `dm:`/`group:` prefixes for `oc_` chat IDs, and default ambiguous bare `oc_` targets to direct routing to avoid DM session misclassification. (#10407) Thanks @Bermudarat. +- Feishu/Group session routing: add configurable group session scopes (`group`, `group_sender`, `group_topic`, `group_topic_sender`) with legacy `topicSessionMode=enabled` compatibility so Feishu group conversations can isolate sessions by sender/topic as configured. (#17798) +- Feishu/Reply-in-thread routing: add `replyInThread` config (`disabled|enabled`) for group replies, propagate `reply_in_thread` across text/card/media/streaming sends, and align topic-scoped session routing so newly created reply threads stay on the same session root. (#27325) +- Feishu/Typing backoff: re-throw Feishu typing add/remove rate-limit and quota errors (`429`, `99991400`, `99991403`) and detect SDK non-throwing backoff responses so the typing keepalive circuit breaker can stop retries instead of looping indefinitely. (#28494) +- Feishu/Zalo runtime logging: replace direct `console.log/error` usage in Feishu typing-indicator paths and Zalo monitor paths with runtime-gated logger calls so verbosity controls are respected while preserving typing backoff behavior. (#18841) Thanks @Clawborn. +- Feishu/Probe status caching: cache successful `probeFeishu()` bot-info results for 10 minutes (bounded cache with per-account keying) to reduce repeated status/onboarding probe API calls, while bypassing cache for failures and exceptions. (#28907) Thanks @Glucksberg. +- Feishu/Opus media send type: send `.opus` attachments with `msg_type: "audio"` (instead of `"media"`) so Feishu voice messages deliver correctly while `.mp4` remains `msg_type: "media"` and documents remain `msg_type: "file"`. (#28269) Thanks @Glucksberg. +- Feishu/Mobile video media type: treat inbound `message_type: "media"` as video-equivalent for media key extraction, placeholder inference, and media download resolution so mobile-app video sends ingest correctly. (#25502) Thanks @4ier. +- Feishu/Inbound sender fallback: fall back to `sender_id.user_id` when `sender_id.open_id` is missing on inbound events, and use ID-type-aware sender lookup so mobile-delivered messages keep stable sender identity/routing. (#26703) Thanks @NewdlDewdl. +- Feishu/Inbound rich-text parsing: preserve `share_chat` payload summaries when available and add explicit parsing for rich-text `code`/`code_block`/`pre` tags so forwarded and code-heavy messages keep useful context in agent input. (#28591) Thanks @kevinWangSheng. +- Feishu/Post markdown parsing: parse rich-text `post` payloads through a shared markdown-aware parser with locale-wrapper support, preserved mention/image metadata extraction, and inline/fenced code fidelity for agent input rendering. (#12755) +- Feishu/Reply context metadata: include inbound `parent_id` and `root_id` as `ReplyToId`/`RootMessageId` in inbound context, and parse interactive-card quote bodies into readable text when fetching replied messages. (#18529) +- Feishu/Post embedded media: extract `media` tags from inbound rich-text (`post`) messages and download embedded video/audio files alongside existing embedded-image handling, with regression coverage. (#21786) Thanks @laopuhuluwa. +- Feishu/Local media sends: propagate `mediaLocalRoots` through Feishu outbound media sending into `loadWebMedia` so local path attachments work with post-CVE local-root enforcement. (#27884) Thanks @joelnishanth. +- Feishu/Group sender allowlist fallback: add global `channels.feishu.groupSenderAllowFrom` sender authorization for group chats, with per-group `groups..allowFrom` precedence and regression coverage for allow/block/precedence behavior. (#29174) Thanks @1MoreBuild. +- Feishu/Group wildcard policy fallback: honor `channels.feishu.groups["*"]` when no explicit group match exists so unmatched groups inherit wildcard reply-policy settings instead of falling back to global defaults. (#29456) Thanks @WaynePika. +- Feishu/Docx append/write ordering: insert converted Docx blocks sequentially (single-block creates) so Feishu append/write preserves markdown block order instead of returning shuffled sections in asynchronous batch inserts. (#26172, #26022) Thanks @echoVic. +- Feishu/Docx convert fallback chunking: recursively split oversized markdown chunks (including long no-heading sections) when `document.convert` hits content limits, while keeping fenced-code-aware split boundaries whenever possible. (#14402) Thanks @lml2468. +- Feishu/Inbound media regression coverage: add explicit tests for message resource type mapping (`image` stays `image`, non-image maps to `file`) to prevent reintroducing unsupported Feishu `type=audio` fetches. (#16311, #8746) Thanks @Yaxuan42. +- Feishu/API quota controls: add `typingIndicator` and `resolveSenderNames` config flags (top-level and per-account) so operators can disable typing reactions and sender-name lookup requests while keeping default behavior unchanged. (#10513) Thanks @BigUncle. +- TTS/Voice bubbles: use opus output and enable `audioAsVoice` routing for Feishu and WhatsApp (in addition to Telegram) so supported channels receive voice-bubble playback instead of file-style audio attachments. (#27366) Thanks @smthfoxy. +- Security/Feishu webhook ingress: bound unauthenticated webhook rate-limit state with stale-window pruning and a hard key cap to prevent unbounded pre-auth memory growth from rotating source keys. (#26050) Thanks @bmendonca3. +- Security/Compaction audit: remove the post-compaction audit injection message. (#28507) Thanks @fuller-stack-dev and @vincentkoc. +- Telegram/Reply media context: include replied media files in inbound context when replying to media, defer reply-media downloads to debounce flush, gate reply-media fetch behind DM authorization, and preserve replied media when non-vision sticker fallback runs (including cached-sticker paths). (#28488) Thanks @obviyus. +- Telegram/Outbound chunking: route oversize splitting through the shared outbound pipeline (including subagents), retry Telegram sends when escaped HTML exceeds limits, and preserve boundary whitespace when retry re-splitting rendered chunks so plain-text/transcript fidelity is retained. (#29342, #27317; follow-up to #27461) Thanks @obviyus. +- Gateway/WS: close repeated post-handshake `unauthorized role:*` request floods per connection and sample duplicate rejection logs, preventing a single misbehaving client from degrading gateway responsiveness. (#20168) Thanks @acy103, @vibecodooor, and @vincentkoc. +- Gateway/macOS supervised restart: actively `launchctl kickstart -k` during intentional supervised restarts to bypass LaunchAgent `ThrottleInterval` delays, and fall back to in-process restart when kickstart fails. Landed from contributor PR #29078 by @cathrynlavery. Thanks @cathrynlavery. +- Gateway/Auth: improve device-auth v2 migration diagnostics so operators get clearer guidance when legacy clients connect. (#28305) Thanks @vincentkoc. +- Gateway/Control UI CSP: allow required Google Fonts origins in Control UI CSP. (#29279) Thanks @Glucksberg and @vincentkoc. +- CLI/Install: add an npm-link fallback to fix CLI startup `Permission denied` failures (`exit 127`) on affected installs. (#17151) Thanks @sskyu and @vincentkoc. +- CLI/Ollama config: allow `config set` for Ollama `apiKey` without predeclared provider config. (#29299) Thanks @vincentkoc. +- Onboarding/Custom providers: improve verification reliability for slower local endpoints (for example Ollama) during setup. (#27380) Thanks @Sid-Qin. +- Ollama/Autodiscovery: harden autodiscovery and warning behavior. (#29201) Thanks @marcodelpin and @vincentkoc. +- Ollama/Context window: unify context window handling across discovery, merge, and OpenAI-compatible transport paths. (#29205) Thanks @Sid-Qin, @jimmielightner, and @vincentkoc. +- Agents/Ollama: demote empty-discovery logging from `warn` to `debug` to reduce noisy warnings in normal edge-case discovery flows. (#26379) Thanks @byungsker. +- Install/npm: fix npm global install deprecation warnings. (#28318) Thanks @vincentkoc. +- fix(model): preserve reasoning in provider fallback resolution. (#29285) Fixes #25636. Thanks @vincentkoc. +- Browser/Open & navigate: accept `url` as an alias parameter for `open` and `navigate`. (#29260) Thanks @vincentkoc. +- Codex/Usage window: label weekly usage window as `Week` instead of `Day`. (#26267) Thanks @Sid-Qin. +- Slack/Native commands: register Slack native status as `/agentstatus` (Slack-reserved `/status`) so manifest slash command registration stays valid while text `/status` still works. Landed from contributor PR #29032 by @maloqab. Thanks @maloqab. +- Android/Nodes reliability: reject `facing=both` when `deviceId` is set to avoid mislabeled duplicate captures, allow notification `open`/`reply` on non-clearable entries while still gating dismiss, trigger listener rebind before notification actions, and scale invoke-result ack timeout to invoke budget for large clip payloads. (#28260) Thanks @obviyus. +- Android/Camera clip: remove `camera.clip` HTTP-upload fallback to base64 so clip transport is deterministic and fail-loud, and reject non-positive `maxWidth` values so invalid inputs fall back to the safe resize default. (#28229) Thanks @obviyus. +- Android/Nodes notification wake flow: enable Android `system.notify` default allowlist, emit `notifications.changed` events for posted/removed notifications (excluding OpenClaw app-owned notifications), canonicalize notification session keys before enqueue/wake routing, and skip heartbeat wakes when consecutive notification summaries dedupe. (#29440) Thanks @obviyus. +- Android/Gateway canvas capability refresh: send `node.canvas.capability.refresh` with object `params` (`{}`) from Android node runtime so gateway object-schema validation accepts refresh retries and A2UI host recovery works after scoped capability expiry. (#28413) Thanks @obviyus. +- Daemon/macOS TLS certs: default LaunchAgent service env `NODE_EXTRA_CA_CERTS` to `/etc/ssl/cert.pem` (while preserving explicit overrides) so HTTPS clients no longer fail with local-issuer errors under launchd. (#27915) Thanks @Lukavyi. +- Update/Global npm: fallback to `--omit=optional` when global `npm update` fails so optional dependency install failures no longer abort update flows. (#24896) Thanks @xinhuagu and @vincentkoc. +- Plugins/NPM spec install: fix npm-spec plugin installs when `npm pack` output is empty by detecting newly created `.tgz` archives in the pack directory. (#21039) Thanks @graysurf and @vincentkoc. +- Plugins/Install: clear stale install errors when an npm package is not found so follow-up install attempts report current state correctly. (#25073) Thanks @dalefrieswthat. +- OpenAI Responses/Compaction: rewrite and unify the OpenAI Responses store patches to treat empty `baseUrl` as non-direct, honor `compat.supportsStore=false`, and auto-inject server-side compaction `context_management` for compatible direct OpenAI models (with per-model opt-out/threshold overrides). Landed from contributor PRs #16930 (@OiPunk), #22441 (@EdwardWu7), and #25088 (@MoerAI). Thanks @OiPunk, @EdwardWu7, and @MoerAI. + +## Unreleased + +### Fixes + +- Android/Onboarding + voice reliability: request per-toggle onboarding permissions, update pairing guidance to `openclaw devices list/approve`, restore assistant speech playback in mic capture flow, cancel superseded in-flight speech (mute + per-reply token rotation), and keep `talk.config` loads retryable after transient failures. (#29796) Thanks @obviyus. +- FS/Sandbox workspace boundaries: add a dedicated `outside-workspace` safe-open error code for root-escape checks, and propagate specific outside-workspace messages across edit/browser/media consumers instead of generic not-found/invalid-path fallbacks. (#29715) Thanks @YuzuruS. +- Config/Doctor group allowlist diagnostics: align `groupPolicy: "allowlist"` warnings with per-channel runtime semantics by excluding Google Chat sender-list checks and by warning when no-fallback channels (for example iMessage) omit `groupAllowFrom`, with regression coverage. (#28477) Thanks @tonydehnke. +- Onboarding/Custom providers: use Azure OpenAI-specific verification auth/payload shape (`api-key`, deployment-path chat completions payload) when probing Azure endpoints so valid Azure custom-provider setup no longer fails preflight. (#29421) Thanks @kunalk16. +- Feishu/Docx editing tools: add `feishu_doc` positional insert, table row/column operations, table-cell merge, and color-text updates; switch markdown write/append/insert to Descendant API insertion with large-document batching; and harden image uploads for data URI/base64/local-path inputs with strict validation and routing-safe upload metadata. (#29411) Thanks @Elarwei001. + +## 2026.2.26 + +### Changes + +- Highlight: External Secrets Management introduces a full `openclaw secrets` workflow (`audit`, `configure`, `apply`, `reload`) with runtime snapshot activation, strict `secrets apply` target-path validation, safer migration scrubbing, ref-only auth-profile support, and dedicated docs. (#26155) Thanks @joshavant. +- ACP/Thread-bound agents: make ACP agents first-class runtimes for thread sessions with `acp` spawn/send dispatch integration, acpx backend bridging, lifecycle controls, startup reconciliation, runtime cleanup, and coalesced thread replies. (#23580) thanks @osolmaz. +- Agents/Routing CLI: add `openclaw agents bindings`, `openclaw agents bind`, and `openclaw agents unbind` for account-scoped route management, including channel-only to account-scoped binding upgrades, role-aware binding identity handling, plugin-resolved binding account IDs, and optional account-binding prompts in `openclaw channels add`. (#27195) thanks @gumadeiras. +- Codex/WebSocket transport: make `openai-codex` WebSocket-first by default (`transport: "auto"` with SSE fallback), keep explicit per-model/runtime transport overrides, and add regression coverage + docs for transport selection. +- Onboarding/Plugins: let channel plugins own interactive onboarding flows with optional `configureInteractive` and `configureWhenConfigured` hooks while preserving the generic fallback path. (#27191) thanks @gumadeiras. +- Auth/Onboarding: add an explicit account-risk warning and confirmation gate before starting Gemini CLI OAuth, and document the caution in provider docs and the Gemini CLI auth plugin README. (#16683) Thanks @vincentkoc. +- Android/Nodes: add Android `device` capability plus `device.status` and `device.info` node commands, including runtime handler wiring and protocol/registry coverage for device status/info payloads. (#27664) Thanks @obviyus. +- Android/Nodes: add `notifications.list` support on Android nodes and expose `nodes notifications_list` in agent tooling for listing active device notifications. (#27344) thanks @obviyus. +- Docs/Contributing: add Nimrod Gutman to the maintainer roster in `CONTRIBUTING.md`. (#27840) Thanks @ngutman. + +### Fixes + +- FS tools/workspaceOnly: honor `tools.fs.workspaceOnly=false` for host write and edit operations so FS tools can access paths outside the workspace when sandbox is off. (#28822) thanks @lailoo. Fixes #28763. Thanks @cjscld for reporting. +- Telegram/DM allowlist runtime inheritance: enforce `dmPolicy: "allowlist"` `allowFrom` requirements using effective account-plus-parent config across account-capable channels (Telegram, Discord, Slack, Signal, iMessage, IRC, BlueBubbles, WhatsApp), and align `openclaw doctor` checks to the same inheritance logic so DM traffic is not silently dropped after upgrades. (#27936) Thanks @widingmarcus-cyber. +- Delivery queue/recovery backoff: prevent retry starvation by persisting `lastAttemptAt` on failed sends and deferring recovery retries until each entry's `lastAttemptAt + backoff` window is eligible, while continuing to recover ready entries behind deferred ones. Landed from contributor PR #27710 by @Jimmy-xuzimo. Thanks @Jimmy-xuzimo. +- Gemini OAuth/Auth flow: align OAuth project discovery metadata and endpoint fallback handling for Gemini CLI auth, including fallback coverage for environment-provided project IDs. (#16684) Thanks @vincentkoc. +- Google Chat/Lifecycle: keep Google Chat `startAccount` pending until abort in webhook mode so startup is no longer interpreted as immediate exit, preventing auto-restart loops and webhook-target churn. (#27384) thanks @junsuwhy. +- Temp dirs/Linux umask: force `0700` permissions after temp-dir creation and self-heal existing writable temp dirs before trust checks so `umask 0002` installs no longer crash-loop on startup. Landed from contributor PR #27860 by @stakeswky. (#27853) Thanks @stakeswky. +- Nextcloud Talk/Lifecycle: keep `startAccount` pending until abort and stop the webhook monitor on shutdown, preventing `EADDRINUSE` restart loops when the gateway manages account lifecycle. (#27897) +- Microsoft Teams/File uploads: acknowledge `fileConsent/invoke` immediately (`invokeResponse` before upload + file card send) so Teams no longer shows false "Something went wrong" timeout banners while upload completion continues asynchronously; includes updated async regression coverage. Landed from contributor PR #27641 by @scz2011. +- Queue/Drain/Cron reliability: harden lane draining with guaranteed `draining` flag reset on synchronous pump failures, reject new queue enqueues during gateway restart drain windows (instead of silently killing accepted tasks), add `/stop` queued-backlog cutoff metadata with stale-message skipping (while avoiding cross-session native-stop cutoff bleed), and raise isolated cron `agentTurn` outer safety timeout to avoid false 10-minute timeout races against longer agent session timeouts. (#27407, #27332, #27427) +- Typing/Main reply pipeline: always mark dispatch idle in `agent-runner` finalization so typing cleanup runs even when dispatcher `onIdle` does not fire, preventing stuck typing indicators after run completion. (#27250) Thanks @Sid-Qin. +- Typing/TTL safety net: add max-duration guardrails to shared typing callbacks so stuck lifecycle edges auto-stop typing indicators even when explicit idle/cleanup signals are missed. (#27428) Thanks @Crpdim. +- Typing/Cross-channel leakage: unify run-scoped typing suppression for cross-channel/internal-webchat routes, preserve current inbound origin as embedded run message channel context, harden shared typing keepalive with consecutive-failure circuit breaker edge-case handling, and enforce dispatcher completion/idle waits in extension dispatcher callsites (Feishu, Matrix, Mattermost, MSTeams) so typing indicators always clean up on success/error paths. Related: #27647, #27493, #27598. Supersedes/replaces draft PRs: #27640, #27593, #27540. +- Telegram/sendChatAction 401 handling: add bounded exponential backoff + temporary local typing suppression after repeated unauthorized failures to stop unbounded `sendChatAction` retry loops that can trigger Telegram abuse enforcement and bot deletion. (#27415) Thanks @widingmarcus-cyber. +- Telegram/Webhook startup: clarify webhook config guidance, allow `channels.telegram.webhookPort: 0` for ephemeral listener binding, and log both the local listener URL and Telegram-advertised webhook URL with the bound port. (#25732) thanks @huntharo. +- Config/Doctor allowlist safety: reject `dmPolicy: "allowlist"` configs with empty `allowFrom`, add Telegram account-level inheritance-aware validation, and teach `openclaw doctor --fix` to restore missing `allowFrom` entries from pairing-store files when present, preventing silent DM drops after upgrades. (#27936) Thanks @widingmarcus-cyber. +- Browser/Chrome extension handshake: bind relay WS message handling before `onopen` and add non-blocking `connect.challenge` response handling for gateway-style handshake frames, avoiding stuck `…` badge states when challenge frames arrive immediately on connect. Landed from contributor PR #22571 by @pandego. (#22553) +- Browser/Extension relay init: dedupe concurrent same-port relay startup with shared in-flight initialization promises so callers await one startup lifecycle and receive consistent success/failure results. Landed from contributor PR #21277 by @HOYALIM. (Related #20688) +- Browser/Fill relay + CLI parity: accept `act.fill` fields without explicit `type` by defaulting missing/empty `type` to `text` in both browser relay route parsing and `openclaw browser fill` CLI field parsing, so relay calls no longer fail when the model omits field type metadata. Landed from contributor PR #27662 by @Uface11. (#27296) Thanks @Uface11. +- Feishu/Permission error dispatch: merge sender-name permission notices into the main inbound dispatch so one user message produces one agent turn/reply (instead of a duplicate permission-notice turn), with regression coverage. (#27381) thanks @byungsker. +- Feishu/Merged forward parsing: expand inbound `merge_forward` messages by fetching and formatting API sub-messages in order, so merged forwards provide usable content context instead of only a placeholder line. (#28707) Thanks @tsu-builds. +- Agents/Canvas default node resolution: when multiple connected canvas-capable nodes exist and no single `mac-*` candidate is selected, default to the first connected candidate instead of failing with `node required` for implicit-node canvas tool calls. Landed from contributor PR #27444 by @carbaj03. Thanks @carbaj03. +- TUI/stream assembly: preserve streamed text across real tool-boundary drops without keeping stale streamed text when non-text blocks appear only in the final payload. Landed from contributor PR #27711 by @scz2011. (#27674) +- Hooks/Internal `message:sent`: forward `sessionKey` on outbound sends from agent delivery, cron isolated delivery, gateway receipt acks, heartbeat sends, session-maintenance warnings, and restart-sentinel recovery so internal `message:sent` hooks consistently dispatch with session context, including `openclaw agent --deliver` runs resumed via `--session-id` (without explicit `--session-key`). Landed from contributor PR #27584 by @qualiobra. Thanks @qualiobra. +- Pi image-token usage: stop re-injecting history image blocks each turn, process image references from the current prompt only, and prune already-answered user-image blocks in stored history to prevent runaway token growth. (#27602) +- BlueBubbles/SSRF: auto-allowlist the configured `serverUrl` hostname for attachment fetches so localhost/private-IP BlueBubbles setups are no longer false-blocked by default SSRF checks. Landed from contributor PR #27648 by @lailoo. (#27599) Thanks @taylorhou for reporting. +- Agents/Compaction + onboarding safety: prevent destructive double-compaction by stripping stale assistant usage around compaction boundaries, skipping post-compaction custom metadata writes in the same attempt, and cancelling safeguard compaction when there are no real conversation messages to summarize; harden workspace/bootstrap detection for memory-backed workspaces; and change `openclaw onboard --reset` default scope to `config+creds+sessions` (workspace deletion now requires `--reset-scope full`). (#26458, #27314) Thanks @jaden-clovervnd, @Sid-Qin, and @widingmarcus-cyber for fix direction in #26502, #26529, and #27492. +- NO_REPLY suppression: suppress `NO_REPLY` before Slack API send and in sub-agent announce completion flow so sentinel text no longer leaks into user channels. Landed from contributor PRs #27529 (by @Sid-Qin) and #27535 (rewritten minimal landing by maintainers). (#27387, #27531) +- Matrix/Group sender identity: preserve sender labels in Matrix group inbound prompt text (`BodyForAgent`) for both channel and threaded messages, and align group envelopes with shared inbound sender-prefix formatting so first-person requests resolve against the current sender. (#27401) thanks @koushikxd. +- Auto-reply/Streaming: suppress only exact `NO_REPLY` final replies while still filtering streaming partial sentinel fragments (`NO_`, `NO_RE`, `HEARTBEAT_...`) so substantive replies ending with `NO_REPLY` are delivered and partial silent tokens do not leak during streaming. (#19576) Thanks @aldoeliacim. +- Auto-reply/Inbound metadata: add a readable `timestamp` field to conversation info and ignore invalid/out-of-range timestamp values so prompt assembly never crashes on malformed timestamp inputs. (#17017) thanks @liuy. +- Typing/Run completion race: prevent post-run keepalive ticks from re-triggering typing callbacks by guarding `triggerTyping()` with `runComplete`, with regression coverage for no-restart behavior during run-complete/dispatch-idle boundaries. (#27413) Thanks @widingmarcus-cyber. +- Typing/Dispatch idle: force typing cleanup when `markDispatchIdle` never arrives after run completion, avoiding leaked typing keepalive loops in cron/announce edges. Landed from contributor PR #27541 by @Sid-Qin. (#27493) +- Telegram/Inline buttons: allow callback-query button handling in groups (including `/models` follow-up buttons) when group policy authorizes the sender, by removing the redundant callback allowlist gate that blocked open-policy groups. (#27343) Thanks @GodsBoy. +- Telegram/Streaming preview: when finalizing without an existing preview message, prime pending preview text with final answer before stop-flush so users do not briefly see stale 1-2 word fragments (for example `no` before `no problem`). (#27449) Thanks @emanuelst for the original fix direction in #19673. +- Browser/Extension relay CORS: handle `/json*` `OPTIONS` preflight before auth checks, allow Chrome extension origins, and return extension-origin CORS headers on relay HTTP responses so extension token validation no longer fails cross-origin. Landed from contributor PR #23962 by @miloudbelarebia. (#23842) +- Browser/Extension relay auth: allow `?token=` query-param auth on relay `/json*` endpoints (consistent with relay WebSocket auth) so curl/devtools-style `/json/version` and `/json/list` probes work without requiring custom headers. Landed from contributor PR #26015 by @Sid-Qin. (#25928) +- Browser/Extension relay shutdown: flush pending extension-request timers/rejections during relay `stop()` before socket/server teardown so in-flight extension waits do not survive shutdown windows. Landed from contributor PR #24142 by @kevinWangSheng. +- Browser/Extension relay reconnect resilience: keep CDP clients alive across brief MV3 extension disconnect windows, wait briefly for extension reconnect before failing in-flight CDP commands, and only tear down relay target/client state after reconnect grace expires. Landed from contributor PR #27617 by @davidemanuelDEV. +- Browser/Route decode hardening: guard malformed percent-encoding in relay target action routes and browser route-param decoding so crafted `%` paths return `400` instead of crashing/unhandled URI decode failures. Landed from contributor PR #11880 by @Yida-Dev. +- Feishu/Inbound message metadata: include inbound `message_id` in `BodyForAgent` on a dedicated metadata line so agents can reliably correlate and act on media/message operations that require message IDs, with regression coverage. (#27253) thanks @xss925175263. +- Feishu/Doc tools: route `feishu_doc` and `feishu_app_scopes` through the active agent account context (with explicit `accountId` override support) so multi-account agents no longer default to the first configured app, with regression coverage for context routing and explicit override behavior. (#27338) thanks @AaronL725. +- LINE/Inline directives auth: gate directive parsing (`/model`, `/think`, `/verbose`, `/reasoning`, `/queue`) on resolved authorization (`command.isAuthorizedSender`) so `commands.allowFrom`-authorized LINE senders are not silently stripped when raw `CommandAuthorized` is unset. Landed from contributor PR #27248 by @kevinWangSheng. (#27240) +- Onboarding/Gateway: seed default Control UI `allowedOrigins` for non-loopback binds during onboarding (`localhost`/`127.0.0.1` plus custom bind host) so fresh non-loopback setups do not fail startup due to missing origin policy. (#26157) thanks @stakeswky. +- Docker/GCP onboarding: reduce first-build OOM risk by capping Node heap during `pnpm install`, reuse existing gateway token during `docker-setup.sh` reruns so `.env` stays aligned with config, auto-bootstrap Control UI allowed origins for non-loopback Docker binds, and add GCP docs guidance for tokenized dashboard links + pairing recovery commands. (#26253) Thanks @pandego. +- CLI/Gateway `--force` in non-root Docker: recover from `lsof` permission failures (`EACCES`/`EPERM`) by falling back to `fuser` kill + probe-based port checks, so `openclaw gateway --force` works for default container `node` user flows. (#27941) +- Gateway/Bind visibility: emit a startup warning when binding to non-loopback addresses so operators get explicit exposure guidance in runtime logs. (#25397) thanks @let5sne. +- Sessions cleanup/Doctor: add `openclaw sessions cleanup --fix-missing` to prune store entries whose transcript files are missing, including doctor guidance and CLI coverage. Landed from contributor PR #27508 by @Sid-Qin. (#27422) +- Doctor/State integrity: ignore metadata-only slash routing sessions when checking recent missing transcripts so `openclaw doctor` no longer reports false-positive transcript-missing warnings for `*:slash:*` keys. (#27375) thanks @gumadeiras. +- CLI/Gateway status: force local `gateway status` probe host to `127.0.0.1` for `bind=lan` so co-located probes do not trip non-loopback plaintext WebSocket checks. (#26997) thanks @chikko80. +- CLI/Gateway auth: align `gateway run --auth` parsing/help text with supported gateway auth modes by accepting `none` and `trusted-proxy` (in addition to `token`/`password`) for CLI overrides. (#27469) thanks @s1korrrr. +- CLI/Daemon status TLS probe: use `wss://` and forward local TLS certificate fingerprint for TLS-enabled gateway daemon probes so `openclaw daemon status` works with `gateway.bind=lan` + `gateway.tls.enabled=true`. (#24234) thanks @liuy. +- Podman/Default bind: change `run-openclaw-podman.sh` default gateway bind from `lan` to `loopback` and document explicit LAN opt-in with Control UI origin configuration. (#27491) thanks @robbyczgw-cla. +- Daemon/macOS launchd: forward proxy env vars into supervised service environments, keep LaunchAgent `KeepAlive=true` semantics, and harden restart sequencing to `print -> bootout -> wait old pid exit -> bootstrap -> kickstart`. (#27276) thanks @frankekn. +- Gateway/macOS restart-loop hardening: detect OpenClaw-managed supervisor markers during SIGUSR1 restart handoff, clean stale gateway PIDs before `/restart` launchctl/systemctl triggers, and set LaunchAgent `ThrottleInterval=60` to bound launchd retry storms during lock-release races. Landed from contributor PRs #27655 (@taw0002), #27448 (@Sid-Qin), and #27650 (@kevinWangSheng). (#27605, #27590, #26904, #26736) +- Models/MiniMax auth header defaults: set `authHeader: true` for both onboarding-generated MiniMax API providers and implicit built-in MiniMax (`minimax`, `minimax-portal`) provider templates so first requests no longer fail with MiniMax `401 authentication_error` due to missing `Authorization` header. Landed from contributor PRs #27622 by @riccoyuanft and #27631 by @kevinWangSheng. (#27600, #15303) +- Models/Google Antigravity IDs: normalize bare `gemini-3-pro`, `gemini-3.1-pro`, and `gemini-3-1-pro` model IDs to the default `-low` thinking tier so provider requests no longer fail with 404 when the tier suffix is omitted. (#24145) Thanks @byungsker. +- Auth/Auth profiles: normalize `auth-profiles.json` alias fields (`mode -> type`, `apiKey -> key`) before credential validation so entries copied from `openclaw.json` auth examples are no longer silently dropped. (#26950) thanks @byungsker. +- Models/Google Gemini: treat `google` (Gemini API key auth profile) as a reasoning-tag provider to prevent `` leakage, and add forward-compat model fallback for `google-gemini-cli` `gemini-3.1-pro*` / `gemini-3.1-flash*` IDs to avoid false unknown-model errors. (#26551, #26524) Thanks @byungsker. +- Models/Profile suffix parsing: centralize trailing `@profile` parsing and only treat `@` as a profile separator when it appears after the final `/`, preserving model IDs like `openai/@cf/...` and `openrouter/@preset/...` across `/model` directive parsing and allowlist model resolution, with regression coverage. +- Models/OpenAI Codex config schema parity: accept `openai-codex-responses` in the config model API schema and TypeScript `ModelApi` union, with regression coverage for config validation. Landed from contributor PR #27501 by @AytuncYildizli. Thanks @AytuncYildizli. +- Agents/Models config: preserve agent-level provider `apiKey` and `baseUrl` during merge-mode `models.json` updates when agent values are present. (#27293) thanks @Sid-Qin. +- Azure OpenAI Responses: force `store=true` for `azure-openai-responses` direct responses API calls to avoid multi-turn 400 failures. Landed from contributor PR #27499 by @polarbear-Yang. (#27497) +- Security/Node exec approvals: require structured `commandArgv` approvals for `host=node`, enforce versioned `systemRunBindingV1` matching for argv/cwd/session/agent/env context with fail-closed behavior on missing/mismatched bindings, and add `GIT_EXTERNAL_DIFF` to blocked host env keys. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Plugin channel HTTP auth: normalize protected `/api/channels` path checks against canonicalized request paths (case + percent-decoding + slash normalization), resolve encoded dot-segment traversal variants, and fail closed on malformed `%`-encoded channel prefixes so alternate-path variants cannot bypass gateway auth. This ships in the next npm release (`2026.2.26`). Thanks @zpbrent for reporting. +- Security/Gateway node pairing: pin paired-device `platform`/`deviceFamily` metadata across reconnects and bind those fields into device-auth signatures, so reconnect metadata spoofing cannot expand node command allowlists without explicit repair pairing. This ships in the next npm release (`2026.2.26`). Thanks @76embiid21 for reporting. +- Security/Sandbox path alias guard: reject broken symlink targets by resolving through existing ancestors and failing closed on out-of-root targets, preventing workspace-only `apply_patch` writes from escaping sandbox/workspace boundaries via dangling symlinks. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Workspace FS boundary aliases: harden canonical boundary resolution for non-existent-leaf symlink aliases while preserving valid in-root aliases, preventing first-write workspace escapes via out-of-root symlink targets. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Config includes: harden `$include` file loading with verified-open reads, reject hardlinked include aliases, and enforce include file-size guardrails so config include resolution remains bounded to trusted in-root files. This ships in the next npm release (`2026.2.26`). Thanks @zpbrent for reporting. +- Security/Node exec approvals hardening: freeze immutable approval-time execution plans (`argv`/`cwd`/`agentId`/`sessionKey`) via `system.run.prepare`, enforce those canonical plan values during approval forwarding/execution, and reject mutable parent-symlink cwd paths during approval-plan building to prevent approval bypass via symlink rebind. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Microsoft Teams media fetch: route Graph message/hosted-content/attachment fetches and auth-scope fallback attachment downloads through shared SSRF-guarded fetch paths, and centralize hostname-suffix allowlist policy helpers in the plugin SDK to remove channel/plugin drift. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Voice Call (Twilio): bind webhook replay + manager dedupe identity to authenticated request material, remove unsigned `i-twilio-idempotency-token` trust from replay/dedupe keys, and thread verified request identity through provider parse flow to harden cross-provider event dedupe. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Exec approvals forwarding: prefer turn-source channel/account/thread metadata when resolving approval delivery targets so stale session routes do not misroute approval prompts. +- Security/Pairing multi-account isolation: enforce account-scoped pairing allowlists and pending-request storage across core + extension message channels while preserving channel-scoped defaults for the default account. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting and @gumadeiras for implementation. +- Memory/SQLite: deduplicate concurrent memory-manager initialization and auto-reopen stale SQLite handles after atomic reindex swaps, preventing repeated `attempt to write a readonly database` sync failures until gateway restart. +- Config/Plugins entries: treat unknown `plugins.entries.*` ids as startup warnings (ignored stale keys) instead of hard validation failures that can crash-loop gateway boot. Landed from contributor PR #27506 by @Sid-Qin. (#27455) +- Telegram native commands: degrade command registration on `BOT_COMMANDS_TOO_MUCH` by retrying with fewer commands instead of crash-looping startup sync. Landed from contributor PR #27512 by @Sid-Qin. (#27456) +- Web tools/Proxy: route `web_search` provider HTTP calls (Brave, Perplexity, xAI, Gemini, Kimi), redirect resolution, and `web_fetch` through a shared proxy-aware SSRF guard path so gateway installs behind `HTTP_PROXY`/`HTTPS_PROXY`/`ALL_PROXY` no longer fail with transport `fetch failed` errors. (#27430) thanks @kevinWangSheng. +- Android/Node invoke: remove native gateway WebSocket `Origin` header to avoid false origin rejections, unify invoke command registry/policy/error parsing paths, and keep command availability checks centralized to reduce dispatcher/advertisement drift. (#27257) Thanks @obviyus. +- Gateway shared-auth scopes: preserve requested operator scopes for shared-token clients when device identity is unavailable, instead of clearing scopes during auth handling. Landed from contributor PR #27498 by @kevinWangSheng. (#27494) +- Cron/Hooks isolated routing: preserve canonical `agent:*` session keys in isolated runs so already-qualified keys are not double-prefixed (for example `agent:main:main` no longer becomes `agent:main:agent:main:main`). Landed from contributor PR #27333 by @MaheshBhushan. (#27289, #27282) +- Channels/Multi-account config: when adding a non-default channel account to a single-account top-level channel setup, move existing account-scoped top-level single-account values into `channels..accounts.default` before writing the new account so the original account keeps working without duplicated account values at channel root; `openclaw doctor --fix` now repairs previously mixed channel account shapes the same way. (#27334) thanks @gumadeiras. +- iOS/Talk mode: stop injecting the voice directive hint into iOS Talk prompts and remove the Voice Directive Hint setting, reducing model bias toward tool-style TTS directives and keeping relay responses text-first by default. (#27543) thanks @ngutman. +- CI/Windows: shard the Windows `checks-windows` test lane into two matrix jobs and honor explicit shard index overrides in `scripts/test-parallel.mjs` to reduce CI critical-path wall time. (#27234) Thanks @joshavant. + +## 2026.2.25 + +### Changes + +- Android/Chat: improve streaming delivery handling and markdown rendering quality in the native Android chat UI, including better GitHub-flavored markdown behavior. (#26079) Thanks @obviyus. +- Android/Startup perf: defer foreground-service startup, move WebView debugging init out of critical startup, and add startup macrobenchmark + low-noise perf CLI scripts for deterministic cold-start tracking. (#26659) Thanks @obviyus. +- UI/Chat compose: add mobile stacked layout for compose action buttons on small screens to improve send/session controls usability. (#11167) Thanks @junyiz. +- Heartbeat/Config: replace heartbeat DM toggle with `agents.defaults.heartbeat.directPolicy` (`allow` | `block`; also supported per-agent via `agents.list[].heartbeat.directPolicy`) for clearer delivery semantics. +- Onboarding/Security: clarify onboarding security notices that OpenClaw is personal-by-default (single trusted operator boundary) and shared/multi-user setups require explicit lock-down/hardening. +- Branding/Docs + Apple surfaces: replace remaining `bot.molt` launchd label, bundle-id, logging subsystem, and command examples with `ai.openclaw` across docs, iOS app surfaces, helper scripts, and CLI test fixtures. +- Agents/Config: remind agents to call `config.schema` before config edits or config-field questions to avoid guessing. Thanks @thewilloftheshadow. +- Dependencies: update workspace dependency pins and lockfile (Bedrock SDK `3.998.0`, `@mariozechner/pi-*` `0.55.1`, TypeScript native preview `7.0.0-dev.20260225.1`) while keeping `@buape/carbon` pinned. + +### Breaking + +- **BREAKING:** Heartbeat direct/DM delivery default is now `allow` again. To keep DM-blocked behavior from `2026.2.24`, set `agents.defaults.heartbeat.directPolicy: "block"` (or per-agent override). + +### Fixes + +- Agents/Subagents delivery: refactor subagent completion announce dispatch into an explicit queue/direct/fallback state machine, recover outbound channel-plugin resolution in cold/stale plugin-registry states across announce/message/gateway send paths, finalize cleanup bookkeeping when announce flow rejects, and treat Telegram sends without `message_id` as delivery failures (instead of false-success `"unknown"` IDs). (#26867, #25961, #26803, #25069, #26741) Thanks @SmithLabsLLC and @docaohieu2808. +- Telegram/Webhook: pre-initialize webhook bots, switch webhook processing to callback-mode JSON handling, and preserve full near-limit payload reads under delayed handlers to prevent webhook request hangs and dropped updates. (#26156) +- Slack/Session threads: prevent oversized parent-session inheritance from silently bricking new thread sessions, surface embedded context-overflow empty-result failures to users, and add configurable `session.parentForkMaxTokens` (default `100000`, `0` disables). (#26912) Thanks @markshields-tl. +- Cron/Message multi-account routing: honor explicit `delivery.accountId` for isolated cron delivery resolution, and when `message.send` omits `accountId`, fall back to the sending agent's bound channel account instead of defaulting to the global account. (#27015, #26975) Thanks @lbo728 and @stakeswky. +- Gateway/Message media roots: thread `agentId` through gateway `send` RPC and prefer explicit `agentId` over session/default resolution so non-default agent workspace media sends no longer fail with `LocalMediaAccessError`; added regression coverage for agent precedence and blank-agent fallback. (#23249) Thanks @Sid-Qin. +- Followups/Routing: when explicit origin routing fails, allow same-channel fallback dispatch (while still blocking cross-channel fallback) so followup replies do not get dropped on transient origin-adapter failures. (#26109) Thanks @Sid-Qin. +- Cron/Announce duplicate guard: track attempted announce/direct delivery separately from confirmed `delivered`, and suppress fallback main-session cron summaries when delivery was already attempted to avoid duplicate end-user sends in uncertain-ack paths. (#27018) +- LINE/Lifecycle: keep LINE `startAccount` pending until abort so webhook startup is no longer misread as immediate channel exit, preventing restart-loop storms on LINE provider boot. (#26528) Thanks @Sid-Qin. +- Discord/Gateway: capture and drain startup-time gateway `error` events before lifecycle listeners attach so early `Fatal Gateway error: 4014` closes surface as actionable intent guidance instead of uncaught gateway crashes. (#23832) Thanks @theotarr. +- Discord/Inbound text: preserve embed `title` + `description` fallback text in message and forwarded snapshot parsing so embed titles are not silently dropped from agent input. (#26946) Thanks @stakeswky. +- Slack/Inbound media fallback: deliver file-only messages even when Slack media downloads fail by adding a filename placeholder fallback, capping fallback names to the shared media-file limit, and normalizing empty filenames to `file` so attachment-only messages are not silently dropped. (#25181) Thanks @justinhuangcode. +- Telegram/Preview cleanup: keep finalized text previews when a later assistant message is media-only (for example mixed text plus voice turns) by skipping finalized preview archival at assistant-message boundaries, preventing cleanup from deleting already-visible final text messages. (#27042) +- Telegram/Markdown spoilers: keep valid `||spoiler||` pairs while leaving unmatched trailing `||` delimiters as literal text, avoiding false all-or-nothing spoiler suppression. (#26105) Thanks @Sid-Qin. +- Slack/Allowlist channels: match channel IDs case-insensitively during channel allowlist resolution so lowercase config keys (for example `c0abc12345`) correctly match Slack runtime IDs (`C0ABC12345`) under `groupPolicy: "allowlist"`, preventing silent channel-event drops. (#26878) Thanks @lbo728. +- Discord/Typing indicator: prevent stuck typing indicators by sealing channel typing keepalive callbacks after idle/cleanup and ensuring Discord dispatch always marks typing idle even if preview-stream cleanup fails. (#26295) Thanks @ngutman. +- Channels/Typing indicator: guard typing keepalive start callbacks after idle/cleanup close so post-close ticks cannot re-trigger stale typing indicators. (#26325) Thanks @win4r. +- Followups/Typing indicator: ensure followup turns mark dispatch idle on every exit path (including `NO_REPLY`, empty payloads, and agent errors) so typing keepalive cleanup always runs and channel typing indicators do not get stuck after queued/silent followups. (#26881) Thanks @codexGW. +- Voice-call/TTS tools: hide the `tts` tool when the message provider is `voice`, preventing voice-call runs from selecting self-playback TTS and falling into silent no-output loops. (#27025) +- Agents/Tools: normalize non-standard plugin tool results that omit `content` so embedded runs no longer crash with `Cannot read properties of undefined (reading 'filter')` after tool completion (including `tesseramemo_query`). (#27007) +- Agents/Tool-call dispatch: trim whitespace-padded tool names in both transcript repair and live streamed embedded-runner responses so exact-match tool lookup no longer fails with `Tool ... not found` for model outputs like `" read "`. (#27094) Thanks @openperf and @Sid-Qin. +- Cron/Model overrides: when isolated `payload.model` is no longer allowlisted, fall back to default model selection instead of failing the job, while still returning explicit errors for invalid model strings. (#26717) Thanks @Youyou972. +- Agents/Model fallback: keep explicit text + image fallback chains reachable even when `agents.defaults.models` allowlists are present, prefer explicit run `agentId` over session-key parsing for followup fallback override resolution (with session-key fallback), treat agent-level fallback overrides as configured in embedded runner preflight, and classify `model_cooldown` / `cooling down` errors as `rate_limit` so failover continues. (#11972, #24137, #17231) +- Agents/Model fallback: keep same-provider fallback chains active when session model differs from configured primary, infer cooldown reason from provider profile state (instead of `disabledReason` only), keep no-profile fallback providers eligible (env/models.json paths), and only relax same-provider cooldown fallback attempts for `rate_limit`. (#23816) thanks @ramezgaberiel. +- Agents/Model fallback: continue fallback traversal on unrecognized errors when candidates remain, while still throwing the original unknown error on the last candidate. (#26106) Thanks @Sid-Qin. +- Models/Auth probes: map permanent auth failover reasons (`auth_permanent`, for example revoked keys) into probe auth status instead of `unknown`, so `openclaw models status --probe` reports actionable auth failures. (#25754) thanks @rrenamed. +- Hooks/Inbound metadata: include `guildId` and `channelName` in `message_received` metadata for both plugin and internal hook paths. (#26115) Thanks @davidrudduck. +- Discord/Component auth: evaluate guild component interactions with command-gating authorizers so unauthorized users no longer get `CommandAuthorized: true` on modal/button events. (#26119) Thanks @bmendonca3. +- Security/Gateway auth: require pairing for operator device-identity sessions authenticated with shared token auth so unpaired devices cannot self-assign operator scopes. Thanks @tdjackey for reporting. +- Security/Gateway WebSocket auth: enforce origin checks for direct browser WebSocket clients beyond Control UI/Webchat, apply password-auth failure throttling to browser-origin loopback attempts (including localhost), and block silent auto-pairing for non-Control-UI browser clients to prevent cross-origin brute-force and session takeover chains. This ships in the next npm release (`2026.2.26`). Thanks @luz-oasis for reporting. +- Security/Gateway trusted proxy: require `operator` role for the Control UI trusted-proxy pairing bypass so unpaired `node` sessions can no longer connect via `client.id=control-ui` and invoke node event methods. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/macOS beta onboarding: remove Anthropic OAuth sign-in and the legacy `oauth.json` onboarding path that exposed the PKCE verifier via OAuth `state`; this impacted the macOS beta onboarding path only. Anthropic subscription auth is now setup-token-only and will ship in the next npm release (`2026.2.26`). Thanks @zdi-disclosures for reporting. +- Security/Microsoft Teams file consent: bind `fileConsent/invoke` upload acceptance/decline to the originating conversation before consuming pending uploads, preventing cross-conversation pending-file upload or cancellation via leaked `uploadId` values; includes regression coverage for match/mismatch invoke handling. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Gateway: harden `agents.files` path handling to block out-of-workspace symlink targets for `agents.files.get`/`agents.files.set`, keep in-workspace symlink targets supported, and add gateway regression coverage for both blocked escapes and allowed in-workspace symlinks. Thanks @tdjackey for reporting. +- Security/Workspace FS: reject hardlinked workspace file aliases in `tools.fs.workspaceOnly` and `tools.exec.applyPatch.workspaceOnly` boundary checks (including sandbox mount-root guards) to prevent out-of-workspace read/write via in-workspace hardlink paths. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Browser temp paths: harden trace/download output-path handling against symlink-root and symlink-parent escapes with realpath-based write-path checks plus secure fallback tmp-dir validation that fails closed on unsafe fallback links. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Browser uploads: revalidate upload paths at use-time in Playwright file-chooser and direct-input flows so missing/rebound paths are rejected before `setFiles`, with regression coverage for strict missing-path handling. +- Security/Exec approvals: bind `system.run` approval matching to exact argv identity and preserve argv whitespace in rendered command text, preventing trailing-space executable path swaps from reusing a mismatched approval. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Exec approvals: harden approval-bound `system.run` execution on node hosts by rejecting symlink `cwd` paths and canonicalizing path-like executable argv before spawn, blocking mutable-cwd symlink retarget chains between approval and execution. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Signal: enforce DM/group authorization before reaction-only notification enqueue so unauthorized senders can no longer inject Signal reaction system events under `dmPolicy`/`groupPolicy`; reaction notifications now require channel access checks first. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Discord reactions: enforce DM policy/allowlist authorization before reaction-event system enqueue in direct messages; Discord reaction handling now also honors DM/group-DM enablement and guild `groupPolicy` channel gating to keep reaction ingress aligned with normal message preflight. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Slack reactions + pins: gate `reaction_*` and `pin_*` system-event enqueue through shared sender authorization so DM `dmPolicy`/`allowFrom` and channel `users` allowlists are enforced consistently for non-message ingress, with regression coverage for denied/allowed sender paths. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Slack member + message subtype events: gate `member_*` plus `message_changed`/`message_deleted`/`thread_broadcast` system-event enqueue through shared sender authorization so DM `dmPolicy`/`allowFrom` and channel `users` allowlists are enforced consistently for non-message ingress; message subtype system events now fail closed when sender identity is missing, with regression coverage. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Telegram reactions: enforce `dmPolicy`/`allowFrom` and group allowlist authorization on `message_reaction` events before enqueueing reaction system events, preventing unauthorized reaction-triggered input in DMs and groups; ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Telegram group allowlist: fail closed for group sender authorization by removing DM pairing-store fallback from group allowlist evaluation; group sender access now requires explicit `groupAllowFrom` or per-group/per-topic `allowFrom`. (#25988) Thanks @bmendonca3. +- Security/DM-group allowlist boundaries: keep DM pairing-store approvals DM-only by removing pairing-store inheritance from group sender authorization in LINE and Mattermost message preflight, and by centralizing shared DM/group allowlist composition so group checks never include pairing-store entries. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Slack interactions: enforce channel/DM authorization and modal actor binding (`private_metadata.userId`) before enqueueing `block_action`/`view_submission`/`view_closed` system events, with regression coverage for unauthorized senders and missing/mismatched actor metadata. This ships in the next npm release (`2026.2.26`). Thanks @tdjackey for reporting. +- Security/Nextcloud Talk: drop replayed signed webhook events with persistent per-account replay dedupe across restarts, and reject unexpected webhook backend origins when account base URL is configured. Thanks @aristorechina for reporting. +- Security/Nextcloud Talk: reject unsigned webhook traffic before full body reads, reducing unauthenticated request-body exposure, with auth-order regression coverage. (#26118) Thanks @bmendonca3. +- Security/Nextcloud Talk: stop treating DM pairing-store entries as group allowlist senders, so group authorization remains bounded to configured group allowlists. (#26116) Thanks @bmendonca3. +- Security/LINE: cap unsigned webhook body reads before auth/signature handling to bound unauthenticated body processing. (#26095) Thanks @bmendonca3. +- Security/IRC: keep pairing-store approvals DM-only and out of IRC group allowlist authorization, with policy regression tests for allowlist resolution. (#26112) Thanks @bmendonca3. +- Security/Microsoft Teams: isolate group allowlist and command authorization from DM pairing-store entries to prevent cross-context authorization bleed. (#26111) Thanks @bmendonca3. +- Security/SSRF guard: classify IPv6 multicast literals (`ff00::/8`) as blocked/private-internal targets in shared SSRF IP checks, preventing multicast literals from bypassing URL-host preflight and DNS answer validation. This ships in the next npm release (`2026.2.26`). Thanks @zpbrent for reporting. +- Tests/Low-memory stability: disable Vitest `vmForks` by default on low-memory local hosts (`<64 GiB`), keep low-profile extension lane parallelism at 4 workers, and align cron isolated-agent tests with `setSessionRuntimeModel` usage to avoid deterministic suite failures. (#26324) Thanks @ngutman. +- Feishu/WebSocket proxy: pass a proxy agent to Feishu WS clients from standard proxy environment variables and include plugin-local runtime dependency wiring so websocket mode works in proxy-constrained installs. (#26397) Thanks @colin719. + +## 2026.2.24 + +### Changes + +- Auto-reply/Abort shortcuts: expand standalone stop phrases (`stop openclaw`, `stop action`, `stop run`, `stop agent`, `please stop`, and related variants), accept trailing punctuation (for example `STOP OPENCLAW!!!`), add multilingual stop keywords (including ES/FR/ZH/HI/AR/JP/DE/PT/RU forms), and treat exact `do not do that` as a stop trigger while preserving strict standalone matching. (#25103) Thanks @steipete and @vincentkoc. +- Android/App UX: ship a native four-step onboarding flow, move post-onboarding into a five-tab shell (Connect, Chat, Voice, Screen, Settings), add a full Connect setup/manual mode screen, and refresh Android chat/settings surfaces for the new navigation model. +- Talk/Gateway config: add provider-agnostic Talk configuration with legacy compatibility, and expose gateway Talk ElevenLabs config metadata for setup/status surfaces. +- Security/Audit: add `security.trust_model.multi_user_heuristic` to flag likely shared-user ingress and clarify the personal-assistant trust model, with hardening guidance for intentional multi-user setups (`sandbox.mode="all"`, workspace-scoped FS, reduced tool surface, no personal/private identities on shared runtimes). +- Dependencies: refresh key runtime and tooling packages across the workspace (Bedrock SDK, pi runtime stack, OpenAI, Google auth, and oxlint/oxfmt), while intentionally keeping `@buape/carbon` pinned. + +### Breaking + +- **BREAKING:** Heartbeat delivery now blocks direct/DM targets when destination parsing identifies a direct chat (for example `user:`, Telegram user chat IDs, or WhatsApp direct numbers/JIDs). Heartbeat runs still execute, but direct-message delivery is skipped and only non-DM destinations (for example channel/group targets) can receive outbound heartbeat messages. +- **BREAKING:** Security/Sandbox: block Docker `network: "container:"` namespace-join mode by default for sandbox and sandbox-browser containers. To keep that behavior intentionally, set `agents.defaults.sandbox.docker.dangerouslyAllowContainerNamespaceJoin: true` (break-glass). Thanks @tdjackey for reporting. + +### Fixes + +- Routing/Session isolation: harden followup routing so explicit cross-channel origin replies never fall back to the active dispatcher on route failure, preserve queued overflow summary routing metadata (`channel`/`to`/`thread`) across followup drain, and prefer originating channel context over internal provider tags for embedded followup runs. This prevents webchat/control-ui context from hijacking Discord-targeted replies in shared sessions. (#25864) Thanks @Gamedesigner. +- Security/Routing: fail closed for shared-session cross-channel replies by binding outbound target resolution to the current turn’s source channel metadata (instead of stale session route fallbacks), and wire those turn-source fields through gateway + command delivery planners with regression coverage. (#24571) Thanks @brandonwise. +- Heartbeat routing: prevent heartbeat leakage/spam into Discord and other direct-message destinations by blocking direct-chat heartbeat delivery targets and keeping blocked-delivery cron/exec prompts internal-only. (#25871) +- Heartbeat defaults/prompts: switch the implicit heartbeat delivery target from `last` to `none` (opt-in for external delivery), and use internal-only cron/exec heartbeat prompt wording when delivery is disabled so background checks do not nudge user-facing relay behavior. (#25871, #24638, #25851) +- Auto-reply/Heartbeat queueing: drop heartbeat runs when a session already has an active run instead of enqueueing a stale followup, preventing duplicate heartbeat response branches after queue drain. (#25610, #25606) Thanks @mcaxtr. +- Cron/Heartbeat delivery: stop inheriting cached session `lastThreadId` for heartbeat-mode target resolution unless a thread/topic is explicitly requested, so announce-mode cron and heartbeat deliveries stay on top-level destinations instead of leaking into active conversation threads. (#25730) Thanks @markshields-tl. +- Messaging tool dedupe: treat originating channel metadata as authoritative for same-target `message.send` suppression in proactive runs (heartbeat/cron/exec-event), including synthetic-provider contexts, so `delivery-mirror` transcript entries no longer cause duplicate Telegram sends. (#25835) Thanks @jadeathena84-arch. +- Channels/Typing keepalive: refresh channel typing callbacks on a keepalive interval during long replies and clear keepalive timers on idle/cleanup across core + extension dispatcher callsites so typing indicators do not expire mid-inference. (#25886, #25882) Thanks @stakeswky. +- Agents/Model fallback: when a run is currently on a configured fallback model, keep traversing the configured fallback chain instead of collapsing straight to primary-only, preventing dead-end failures when primary stays in cooldown. (#25922, #25912) Thanks @Taskle. +- Gateway/Models: honor explicit `agents.defaults.models` allowlist refs even when bundled model catalog data is stale, synthesize missing allowlist entries in `models.list`, and allow `sessions.patch`/`/model` selection for those refs without false `model not allowed` errors. (#20291) Thanks @kensipe, @nikolasdehor, and @vincentkoc. +- Control UI/Agents: inherit `agents.defaults.model.fallbacks` in the Overview fallback input when no per-agent model entry exists, while preserving explicit per-agent fallback overrides (including empty lists). (#25729, #25710) Thanks @Suko. +- Automation/Subagent/Cron reliability: honor `ANNOUNCE_SKIP` in `sessions_spawn` completion/direct announce flows (no user-visible token leaks), add transient direct-announce retries for channel unavailability (for example WhatsApp listener reconnect windows), and include `cron` in the `coding` tool profile so `/tools/invoke` can execute cron actions when explicitly allowed by gateway policy. (#25800, #25656, #25842, #25813, #25822, #25821) Thanks @astra-fer, @aaajiao, @dwight11232-coder, @kevinWangSheng, @widingmarcus-cyber, and @stakeswky. +- Discord/Voice reliability: restore runtime DAVE dependency (`@snazzah/davey`), add configurable DAVE join options (`channels.discord.voice.daveEncryption` and `channels.discord.voice.decryptionFailureTolerance`), clean up voice listeners/session teardown, guard against stale connection events, and trigger controlled rejoin recovery after repeated decrypt failures to improve inbound STT stability under DAVE receive errors. (#25861, #25372, #24883, #24825, #23890, #23105, #22961, #23421, #23278, #23032) +- Discord/Block streaming: restore block-streamed reply delivery by suppressing only reasoning payloads (instead of all `block` payloads), fixing missing Discord replies in `channels.discord.streaming=block` mode. (#25839, #25836, #25792) Thanks @pewallin. +- Discord/Proxy + reactions + model picker: thread channel proxy fetch into inbound media/sticker downloads, use proxy-aware gateway metadata fetch for WSL/corporate proxy setups, wire `messages.statusReactions.{emojis,timing}` into Discord reaction lifecycle control, and compact model-picker `custom_id` keys to stay under Discord's 100-char limit while keeping backward-compatible parsing. (#25232, #25507, #25564, #25695) Thanks @openperf, @chilu18, @Yipsh, @lbo728, and @s1korrrr. +- WhatsApp/Web reconnect: treat close status `440` as non-retryable (including string-form status values), stop reconnect loops immediately, and emit operator guidance to relink after resolving session conflicts. (#25858) Thanks @markmusson. +- WhatsApp/Reasoning safety: suppress outbound payloads marked as reasoning and hard-drop text payloads that begin with `Reasoning:` before WhatsApp delivery, preventing hidden thinking blocks from leaking to end users through final-message paths. (#25804, #25214, #24328) +- Matrix/Read receipts: send read receipts as soon as Matrix messages arrive (before handler pipeline work), so clients no longer show long-lived unread/sent states while replies are processing. (#25841, #25840) Thanks @joshjhall. +- Telegram/Replies: when markdown formatting renders to empty HTML (for example syntax-only chunks in threaded replies), retry delivery with plain text, and fail loud when both formatted and plain payloads are empty to avoid false delivered states. (#25096, #25091) Thanks @Glucksberg. +- Telegram/Media fetch: prioritize IPv4 before IPv6 in SSRF pinned DNS address ordering so media downloads still work on hosts with broken IPv6 routing. (#24295, #23975) Thanks @Glucksberg. +- Telegram/Outbound API: replace Node 22's global undici dispatcher when applying Telegram `autoSelectFamily` decisions so outbound `fetch` calls inherit IPv4 fallback instead of staying pinned to stale dispatcher settings. (#25682, #25676) Thanks @lairtonlelis. +- Onboarding/Telegram: keep core-channel onboarding available when plugin registry population is missing by falling back to built-in adapters and continuing wizard setup with actionable recovery guidance. (#25803) Thanks @Suko. +- Android/Gateway auth: preserve Android gateway auth state across onboarding, use the native client id for operator sessions, retry with shared-token fallback after device-token auth failures, and avoid clearing tokens on transient connect errors. +- Slack/DM routing: treat `D*` channel IDs as direct messages even when Slack sends an incorrect `channel_type`, preventing DM traffic from being misclassified as channel/group chats. (#25479) Thanks @mcaxtr. +- Zalo/Group policy: enforce sender authorization for group messages with `groupPolicy` + `groupAllowFrom` (fallback to `allowFrom`), default runtime group behavior to fail-closed allowlist, and block unauthorized non-command group messages before dispatch. Thanks @tdjackey for reporting. +- macOS/Voice input: guard all audio-input startup paths against missing default microphones (Voice Wake, Talk Mode, Push-to-Talk, mic-level monitor, tester) to avoid launch/runtime crashes on mic-less Macs and fail gracefully until input becomes available. (#25817) Thanks @sfo2001. +- macOS/IME input: when marked text is active, treat Return as IME candidate confirmation first in both the voice overlay composer and shared chat composer to prevent accidental sends while composing CJK text. (#25178) Thanks @bottotl. +- macOS/Voice wake routing: default forwarded voice-wake transcripts to the `webchat` channel (instead of ambiguous `last` routing) so local voice prompts stay pinned to the control chat surface unless explicitly overridden. (#25440) Thanks @chilu18. +- macOS/Gateway launch: prefer an available `openclaw` binary before pnpm/node runtime fallback when resolving local gateway commands, so local startup no longer fails on hosts with broken runtime discovery. (#25512) Thanks @chilu18. +- macOS/Menu bar: stop reusing the injector delegate for the "Usage cost (30 days)" submenu to prevent recursive submenu injection loops when opening cost history. (#25341) Thanks @yingchunbai. +- macOS/WebChat panel: fix rounded-corner clipping by using panel-specific visual-effect blending and matching corner masking on both effect and hosting layers. (#22458) Thanks @apethree and @agisilaos. +- Windows/Exec shell selection: prefer PowerShell 7 (`pwsh`) discovery (Program Files, ProgramW6432, PATH) before falling back to Windows PowerShell 5.1, fixing `&&` command chaining failures on Windows hosts with PS7 installed. (#25684, #25638) Thanks @zerone0x. +- Windows/Media safety checks: align async local-file identity validation with sync-safe-open behavior by treating win32 `dev=0` stats as unknown-device fallbacks (while keeping strict dev checks when both sides are non-zero), fixing false `Local media path is not safe to read` drops for local attachments/TTS/images. (#25708, #21989, #25699, #25878) Thanks @kevinWangSheng. +- iMessage/Reasoning safety: harden iMessage echo suppression with outbound `messageId` matching (plus scoped text fallback), and enforce reasoning-payload suppression on routed outbound delivery paths to prevent hidden thinking text from being sent as user-visible channel messages. (#25897, #1649, #25757) Thanks @rmarr and @Iranb. +- Providers/OpenRouter/Auth profiles: bypass auth-profile cooldown/disable windows for OpenRouter, so provider failures no longer put OpenRouter profiles into local cooldown and stale legacy cooldown markers are ignored in fallback and status selection paths. (#25892) Thanks @alexanderatallah for raising this and @vincentkoc for the fix. +- Providers/Google reasoning: sanitize invalid negative `thinkingBudget` payloads for Gemini 3.1 requests by dropping `-1` budgets and mapping configured reasoning effort to `thinkingLevel`, preventing malformed reasoning payloads on `google-generative-ai`. (#25900) +- Providers/SiliconFlow: normalize `thinking="off"` to `thinking: null` for `Pro/*` model payloads to avoid provider-side 400 loops and misleading compaction retries. (#25435) Thanks @Zjianru. +- Models/Bedrock auth: normalize additional Bedrock provider aliases (`bedrock`, `aws-bedrock`, `aws_bedrock`, `amazon bedrock`) to canonical `amazon-bedrock`, ensuring auth-mode resolution consistently selects AWS SDK fallback. (#25756) Thanks @fwhite13. +- Models/Providers: preserve explicit user `reasoning` overrides when merging provider model config with built-in catalog metadata, so `reasoning: false` is no longer overwritten by catalog defaults. (#25314) Thanks @lbo728. +- Gateway/Auth: allow trusted-proxy authenticated Control UI websocket sessions to skip device pairing when device identity is absent, preventing false `pairing required` failures behind trusted reverse proxies. (#25428) Thanks @SidQin-cyber. +- CLI/Memory search: accept `--query ` for `openclaw memory search` (while keeping positional query support), and emit a clear error when neither form is provided. (#25904, #25857) Thanks @niceysam and @stakeswky. +- CLI/Doctor: correct stale recovery hints to use valid commands (`openclaw gateway status --deep` and `openclaw configure --section model`). (#24485) Thanks @chilu18. +- Doctor/Sandbox: when sandbox mode is enabled but Docker is unavailable, surface a clear actionable warning (including failure impact and remediation) instead of a mild “skip checks” note. (#25438) Thanks @mcaxtr. +- Doctor/Plugins: auto-enable now resolves third-party channel plugins by manifest plugin id (not channel id), preventing invalid `plugins.entries.` writes when ids differ. (#25275) Thanks @zerone0x. +- Config/Plugins: treat stale removed `google-antigravity-auth` plugin references as compatibility warnings (not hard validation errors) across `plugins.entries`, `plugins.allow`, `plugins.deny`, and `plugins.slots.memory`, so startup no longer fails after antigravity removal. (#25538, #25862) Thanks @chilu18. +- Config/Meta: accept numeric `meta.lastTouchedAt` timestamps and coerce them to ISO strings, preserving compatibility with agent edits that write `Date.now()` values. (#25491) Thanks @mcaxtr. +- Usage accounting: parse Moonshot/Kimi `cached_tokens` fields (including `prompt_tokens_details.cached_tokens`) into normalized cache-read usage metrics. (#25436) Thanks @Elarwei001. +- Agents/Tool dispatch: await block-reply flush before tool execution starts so buffered block replies preserve message ordering around tool calls. (#25427) Thanks @SidQin-cyber. +- Agents/Billing classification: prevent long assistant/user-facing text from being rewritten as billing failures while preserving explicit `status/code/http 402` detection for oversized structured error payloads. (#25680, #25661) Thanks @lairtonlelis. +- Sessions/Tool-result guard: avoid generating synthetic `toolResult` entries for assistant turns that ended with `stopReason: "aborted"` or `"error"`, preventing orphaned tool-use IDs from triggering downstream API validation errors. (#25429) Thanks @mikaeldiakhate-cell. +- Auto-reply/Reset hooks: guarantee native `/new` and `/reset` flows emit command/reset hooks even on early-return command paths, with dedupe protection to avoid double hook emission. (#25459) Thanks @chilu18. +- Hooks/Slug generator: resolve session slug model from the agent’s effective model (including defaults/fallback resolution) instead of raw agent-primary config only. (#25485) Thanks @SudeepMalipeddi. +- Sandbox/FS bridge tests: add regression coverage for dash-leading basenames to confirm sandbox file reads resolve to absolute container paths (and avoid shell-option misdiagnosis for dashed filenames). (#25891) Thanks @albertlieyingadrian. +- Sandbox/FS bridge: build canonical-path shell scripts with newline separators (not `; ` joins) to avoid POSIX `sh` `do;` syntax errors that broke sandbox file/image read-write operations. (#25737, #25824, #25868) Thanks @DennisGoldfinger and @peteragility. +- Sandbox/Config: preserve `dangerouslyAllowReservedContainerTargets` and `dangerouslyAllowExternalBindSources` during sandbox docker config resolution so explicit bind-mount break-glass overrides reach runtime validation. (#25410) Thanks @skyer-jian. +- Gateway/Security: enforce gateway auth for the exact `/api/channels` plugin root path (plus `/api/channels/` descendants), with regression coverage for query/trailing-slash variants and near-miss paths that must remain plugin-owned. (#25753) Thanks @bmendonca3. +- Exec approvals: treat bare allowlist `*` as a true wildcard for parsed executables, including unresolved PATH lookups, so global opt-in allowlists work as configured. (#25250) Thanks @widingmarcus-cyber. +- iOS/Signing: improve `scripts/ios-team-id.sh` for Xcode 16+ by falling back to Xcode-managed provisioning profiles, add actionable guidance when an Apple account exists but no Team ID can be resolved, and ignore Xcode `xcodebuild` output directories (`apps/ios/build`, `apps/shared/OpenClawKit/build`, `Swabble/build`). (#22773) Thanks @brianleach. +- Control UI/Chat images: route image-click opens through a shared safe-open helper (allowing only safe URL schemes) and open new tabs with opener isolation to block tabnabbing. (#18685, #25444, #25847) Thanks @Mariana-Codebase and @shakkernerd. +- Security/Exec: sanitize inherited host execution environment before merge, canonicalize inherited PATH handling, and strip dangerous keys (`LD_*`, `DYLD_*`, `SSLKEYLOGFILE`, and related injection vectors) from non-sandboxed exec runs. (#25755) Thanks @bmendonca3. +- Security/Hooks: normalize hook session-key classification with trim/lowercase plus Unicode NFKC folding (for example full-width `ＨＯＯＫ：...`) so external-content wrapping cannot be bypassed by mixed-case or lookalike prefixes. (#25750) Thanks @bmendonca3. +- Security/Voice Call: add Telnyx webhook replay detection and canonicalize replay-key signature encoding (Base64/Base64URL equivalent forms dedupe together), so duplicate signed webhook deliveries no longer re-trigger side effects. (#25832) Thanks @bmendonca3. +- Security/Sandbox media: restrict sandbox media tmp-path allowances to OpenClaw-managed tmp roots instead of broad host `os.tmpdir()` trust, and add outbound/channel guardrails (tmp-path lint + media-root smoke tests) to prevent regressions in local media attachment reads. Thanks @tdjackey for reporting. +- Security/Sandbox media: reject hard-linked OpenClaw tmp media aliases (including symlink-to-hardlink chains) during sandbox media path resolution to prevent out-of-sandbox inode alias reads. (#25820) Thanks @bmendonca3. +- Security/Message actions: enforce local media root checks for `sendAttachment` and `setGroupIcon` when `sandboxRoot` is unset, preventing attachment hydration from reading arbitrary host files via local absolute paths. Thanks @GCXWLP for reporting. +- Security/Telegram: enforce DM authorization before media download/write (including media groups) and move telegram inbound activity tracking after DM authorization, preventing unauthorized sender-triggered inbound media disk writes. Thanks @v8hid for reporting. +- Security/Workspace FS: normalize `@`-prefixed paths before workspace-boundary checks (including workspace-only read/write/edit and sandbox mount path guards), preventing absolute-path escape attempts from bypassing guard validation. Thanks @tdjackey for reporting. +- Security/Synology Chat: enforce fail-closed allowlist behavior for DM ingress so `dmPolicy: "allowlist"` with empty `allowedUserIds` rejects all senders instead of allowing unauthorized dispatch. (#25827) Thanks @bmendonca3 for the contribution and @tdjackey for reporting. +- Security/Native images: enforce `tools.fs.workspaceOnly` for native prompt image auto-load (including history refs), preventing out-of-workspace sandbox mounts from being implicitly ingested as vision input. Thanks @tdjackey for reporting. +- Security/Exec approvals: bind `system.run` command display/approval text to full argv when shell-wrapper inline payloads carry positional argv values, and reject payload-only `rawCommand` mismatches for those wrapper-carrier forms, preventing hidden command execution under misleading approval text. Thanks @tdjackey for reporting. +- Security/Exec companion host: forward canonical `system.run` display text (not payload-only shell snippets) to the macOS exec host, and enforce rawCommand/argv consistency there for shell-wrapper positional-argv carriers and env-modifier preludes, preventing companion-side approval/display drift. Thanks @tdjackey for reporting. +- Security/Exec approvals: fail closed when transparent dispatch-wrapper unwrapping exceeds the depth cap, so nested `/usr/bin/env` chains cannot bypass shell-wrapper approval gating in `allowlist` + `ask=on-miss` mode. Thanks @tdjackey for reporting. +- Security/Exec: limit default safe-bin trusted directories to immutable system paths (`/bin`, `/usr/bin`) and require explicit opt-in (`tools.exec.safeBinTrustedDirs`) for package-manager/user bin paths (for example Homebrew), add security-audit findings for risky trusted-dir choices, warn at runtime when explicitly trusted dirs are group/world writable, and add doctor hints when configured `safeBins` resolve outside trusted dirs. Thanks @tdjackey for reporting. +- Telegram/Media fetch: prioritize IPv4 before IPv6 in SSRF pinned DNS address ordering so media downloads still work on hosts with broken IPv6 routing. (#24295, #23975) Thanks @Glucksberg. +- Telegram/Outbound API: replace Node 22's global undici dispatcher when applying Telegram `autoSelectFamily` decisions so outbound `fetch` calls inherit IPv4 fallback instead of staying pinned to stale dispatcher settings. (#25682, #25676) Thanks @lairtonlelis. +- Agents/Billing classification: prevent long assistant/user-facing text from being rewritten as billing failures while preserving explicit `status/code/http 402` detection for oversized structured error payloads. (#25680, #25661) Thanks @lairtonlelis. +- Telegram/Replies: when markdown formatting renders to empty HTML (for example syntax-only chunks in threaded replies), retry delivery with plain text, and fail loud when both formatted and plain payloads are empty to avoid false delivered states. (#25096, #25091) Thanks @Glucksberg. +- Sessions/Tool-result guard: avoid generating synthetic `toolResult` entries for assistant turns that ended with `stopReason: "aborted"` or `"error"`, preventing orphaned tool-use IDs from triggering downstream API validation errors. (#25429) Thanks @mikaeldiakhate-cell. +- Gateway/Sessions: preserve `modelProvider` on `sessions.reset` and avoid incorrect provider prefixes for legacy session models. (#25874) Thanks @lbo728. +- Usage accounting: parse Moonshot/Kimi `cached_tokens` fields (including `prompt_tokens_details.cached_tokens`) into normalized cache-read usage metrics. (#25436) Thanks @Elarwei001. +- Doctor/Sandbox: when sandbox mode is enabled but Docker is unavailable, surface a clear actionable warning (including failure impact and remediation) instead of a mild “skip checks” note. (#25438) Thanks @mcaxtr. +- Config/Meta: accept numeric `meta.lastTouchedAt` timestamps and coerce them to ISO strings, preserving compatibility with agent edits that write `Date.now()` values. (#25491) Thanks @mcaxtr. +- Auto-reply/Reset hooks: guarantee native `/new` and `/reset` flows emit command/reset hooks even on early-return command paths, with dedupe protection to avoid double hook emission. (#25459) Thanks @chilu18. +- Hooks/Slug generator: resolve session slug model from the agent’s effective model (including defaults/fallback resolution) instead of raw agent-primary config only. (#25485) Thanks @SudeepMalipeddi. +- Slack/DM routing: treat `D*` channel IDs as direct messages even when Slack sends an incorrect `channel_type`, preventing DM traffic from being misclassified as channel/group chats. (#25479) Thanks @mcaxtr. +- Models/Providers: preserve explicit user `reasoning` overrides when merging provider model config with built-in catalog metadata, so `reasoning: false` is no longer overwritten by catalog defaults. (#25314) Thanks @lbo728. +- Exec approvals: treat bare allowlist `*` as a true wildcard for parsed executables, including unresolved PATH lookups, so global opt-in allowlists work as configured. (#25250) Thanks @widingmarcus-cyber. +- Gateway/Auth: allow trusted-proxy authenticated Control UI websocket sessions to skip device pairing when device identity is absent, preventing false `pairing required` failures behind trusted reverse proxies. (#25428) Thanks @SidQin-cyber. +- Agents/Tool dispatch: await block-reply flush before tool execution starts so buffered block replies preserve message ordering around tool calls. (#25427) Thanks @SidQin-cyber. +- Agents/Compaction: harden summarization prompts to preserve opaque identifiers verbatim (UUIDs, IDs, tokens, host/IP/port, URLs), reducing post-compaction identifier drift and hallucinated identifier reconstruction. +- iOS/Signing: improve `scripts/ios-team-id.sh` for Xcode 16+ by falling back to Xcode-managed provisioning profiles, add actionable guidance when an Apple account exists but no Team ID can be resolved, and ignore Xcode `xcodebuild` output directories (`apps/ios/build`, `apps/shared/OpenClawKit/build`, `Swabble/build`). (#22773) Thanks @brianleach. +- macOS/Menu bar: stop reusing the injector delegate for the "Usage cost (30 days)" submenu to prevent recursive submenu injection loops when opening cost history. (#25341) Thanks @yingchunbai. +- Control UI/Chat images: route image-click opens through a shared safe-open helper (allowing only safe URL schemes) and open new tabs with opener isolation to block tabnabbing. (#18685, #25444, #25847) Thanks @Mariana-Codebase and @shakkernerd. +- CLI/Doctor: correct stale recovery hints to use valid commands (`openclaw gateway status --deep` and `openclaw configure --section model`). (#24485) Thanks @chilu18. +- CLI/Memory search: accept `--query ` for `openclaw memory search` (while keeping positional query support), and emit a clear error when neither form is provided. (#25904, #25857) Thanks @niceysam and @stakeswky. +- Security/Sandbox: canonicalize bind-mount source paths via existing-ancestor realpath so symlink-parent + non-existent-leaf paths cannot bypass allowed-source-roots or blocked-path checks. Thanks @tdjackey. + +## 2026.2.23 + +### Changes + +- Providers/Kilo Gateway: add first-class `kilocode` provider support (auth, onboarding, implicit provider detection, model defaults, transcript/cache-ttl handling, and docs), with default model `kilocode/anthropic/claude-opus-4.6`. (#20212) Thanks @jrf0110 and @markijbema. +- Providers/Vercel AI Gateway: accept Claude shorthand model refs (`vercel-ai-gateway/claude-*`) by normalizing to canonical Anthropic-routed model ids. (#23985) Thanks @sallyom, @markbooch, and @vincentkoc. +- Docs/Prompt caching: add a dedicated prompt-caching reference covering `cacheRetention`, per-agent `params` merge precedence, Bedrock/OpenRouter behavior, and cache-ttl + heartbeat tuning. Thanks @svenssonaxel. +- Gateway/HTTP security headers: add optional `gateway.http.securityHeaders.strictTransportSecurity` support to emit `Strict-Transport-Security` for direct HTTPS deployments, with runtime wiring, validation, tests, and hardening docs. +- Sessions/Cron: harden session maintenance with `openclaw sessions cleanup`, per-agent store targeting, disk-budget controls (`session.maintenance.maxDiskBytes` / `highWaterBytes`), and safer transcript/archive cleanup + run-log retention behavior. (#24753) thanks @gumadeiras. +- Tools/web_search: add `provider: "kimi"` (Moonshot) support with key/config schema wiring and a corrected two-step `$web_search` tool flow that echoes tool results before final synthesis, including citation extraction from search results. (#16616, #18822) Thanks @adshine. +- Media understanding/Video: add a native Moonshot video provider and include Moonshot in auto video key detection, plus refactor video execution to honor `entry/config/provider` baseUrl+header precedence (matching audio behavior). (#12063) Thanks @xiaoyaner0201. +- Agents/Config: support per-agent `params` overrides merged on top of model defaults (including `cacheRetention`) so mixed-traffic agents can tune cache behavior independently. (#17470, #17112) Thanks @rrenamed. +- Agents/Bootstrap: cache bootstrap file snapshots per session key and clear them on session reset/delete, reducing prompt-cache invalidations from in-session `AGENTS.md`/`MEMORY.md` writes. (#22220) Thanks @anisoptera. + +### Breaking + +- **BREAKING:** browser SSRF policy now defaults to trusted-network mode (`browser.ssrfPolicy.dangerouslyAllowPrivateNetwork=true` when unset), and canonical config uses `browser.ssrfPolicy.dangerouslyAllowPrivateNetwork` instead of `browser.ssrfPolicy.allowPrivateNetwork`. `openclaw doctor --fix` migrates the legacy key automatically. + +### Fixes + +- Security/Config: redact sensitive-looking dynamic catchall keys in `config.get` snapshots (for example `env.*` and `skills.entries.*.env.*`) and preserve round-trip restore behavior for those redacted sentinels. Thanks @merc1305. +- Tests/Vitest: tier local parallel worker defaults by host memory, keep gateway serial by default on non-high-memory hosts, and document a low-profile fallback command for memory-constrained land/gate runs to prevent local OOMs. (#24719) Thanks @ngutman. +- WhatsApp/Group policy: fix `groupAllowFrom` sender filtering when `groupPolicy: "allowlist"` is set without explicit `groups` — previously all group messages were blocked even for allowlisted senders. (#24670) +- Agents/Context pruning: extend `cache-ttl` eligibility to Moonshot/Kimi and ZAI/GLM providers (including OpenRouter model refs), so `contextPruning.mode: "cache-ttl"` is no longer silently skipped for those sessions. (#24497) Thanks @lailoo. +- Doctor/Memory: query gateway-side default-agent memory embedding readiness during `openclaw doctor` (instead of inferring from generic gateway health), and warn when the gateway memory probe is unavailable or not ready while keeping `openclaw configure` remediation guidance. (#22327) thanks @therk. +- Sessions/Store: canonicalize inbound mixed-case session keys for metadata and route updates, and migrate legacy case-variant entries to a single lowercase key to prevent duplicate sessions and missing TUI/WebUI history. (#9561) Thanks @hillghost86. +- Telegram/Reactions: soft-fail reaction action errors (policy/token/emoji/API), accept snake_case `message_id`, and fallback to inbound message-id context when explicit `messageId` is omitted so DM reactions stay stable without regeneration loops. (#20236, #21001) Thanks @PeterShanxin and @vincentkoc. +- Telegram/Polling: scope persisted polling offsets to bot identity and reuse a single awaited runner-stop path on abort/retry, preventing cross-token offset bleed and overlapping pollers during restart/error recovery. (#10850, #11347) Thanks @talhaorak, @anooprdawar, and @vincentkoc. +- Telegram/Reasoning: when `/reasoning off` is active, suppress reasoning-only delivery segments and block raw fallback resend of suppressed `Reasoning:`/`` text, preventing internal reasoning leakage in legacy sessions while preserving answer delivery. (#24626, #24518) +- Agents/Reasoning: when model-default thinking is active (for example `thinking=low`), keep auto-reasoning disabled unless explicitly enabled, preventing `Reasoning:` thinking-block leakage in channel replies. (#24335, #24290) thanks @Kay-051. +- Agents/Reasoning: avoid classifying provider reasoning-required errors as context overflows so these failures no longer trigger compaction-style overflow recovery. (#24593) Thanks @vincentkoc. +- Agents/Models: codify `agents.defaults.model` / `agents.defaults.imageModel` config-boundary input as `string | {primary,fallbacks}`, split explicit vs effective model resolution, and fix `models status --agent` source attribution so defaults-inherited agents are labeled as `defaults` while runtime selection still honors defaults fallback. (#24210) thanks @bianbiandashen. +- Agents/Compaction: pass `agentDir` into manual `/compact` command runs so compaction auth/profile resolution stays scoped to the active agent. (#24133) thanks @Glucksberg. +- Agents/Compaction: pass model metadata through the embedded runtime so safeguard summarization can run when `ctx.model` is unavailable, avoiding repeated `"Summary unavailable due to context limits"` fallback summaries. (#3479) Thanks @battman21, @hanxiao and @vincentkoc. +- Agents/Compaction: cancel safeguard compaction when summary generation cannot run (missing model/API key or summarization failure), preserving history instead of truncating to fallback `"Summary unavailable"` text. (#10711) Thanks @DukeDeSouth and @vincentkoc. +- Agents/Tools: make `session_status` read transcript-derived usage mid-turn and tail-read session logs for cache-aware context reporting without full-log scans. (#22387) Thanks @1ucian. +- Agents/Overflow: detect additional provider context-overflow error shapes (including `input length` + `max_tokens` exceed-context variants) so failures route through compaction/recovery paths instead of leaking raw provider errors to users. (#9951) Thanks @echoVic and @Glucksberg. +- Agents/Overflow: add Chinese context-overflow pattern detection in `isContextOverflowError` so localized provider errors route through overflow recovery paths. (#22855) Thanks @Clawborn. +- Agents/Failover: treat HTTP 502/503/504 errors as failover-eligible transient timeouts so fallback chains can switch providers/models during upstream outages instead of retrying the same failing target. (#20999) Thanks @taw0002 and @vincentkoc. +- Auto-reply/Inbound metadata: hide direct-chat `message_id`/`message_id_full` and sender metadata only from normalized chat type (not sender-id sentinels), preserving group metadata visibility and preventing sender-id spoofed direct-mode classification. (#24373) thanks @jd316. +- Auto-reply/Inbound metadata: move dynamic inbound `flags` (reply/forward/thread/history) from system metadata to user-context conversation info, preventing turn-by-turn prompt-cache invalidation from flag toggles. (#21785) Thanks @aidiffuser. +- Auto-reply/Sessions: remove auth-key labels from `/new` and `/reset` confirmation messages so session reset notices never expose API key prefixes or env-key labels in chat output. (#24384, #24409) Thanks @Clawborn. +- Slack/Group policy: move Slack account `groupPolicy` defaulting to provider-level schema defaults so multi-account configs inherit top-level `channels.slack.groupPolicy` instead of silently overriding inheritance with per-account `allowlist`. (#17579) Thanks @ZetiMente. +- Providers/Anthropic: skip `context-1m-*` beta injection for OAuth/subscription tokens (`sk-ant-oat-*`) while preserving OAuth-required betas, avoiding Anthropic 401 auth failures when `params.context1m` is enabled. (#10647, #20354) Thanks @ClumsyWizardHands and @dcruver. +- Providers/DashScope: mark DashScope-compatible `openai-completions` endpoints as `supportsDeveloperRole=false` so OpenClaw sends `system` instead of unsupported `developer` role on Qwen/DashScope APIs. (#19130) Thanks @Putzhuawa and @vincentkoc. +- Providers/Bedrock: disable prompt-cache retention for non-Anthropic Bedrock models so Nova/Mistral requests do not send unsupported cache metadata. (#20866) Thanks @pierreeurope. +- Providers/Bedrock: apply Anthropic-Claude cacheRetention defaults and runtime pass-through for `amazon-bedrock/*anthropic.claude*` model refs, while keeping non-Anthropic Bedrock models excluded. (#22303) Thanks @snese. +- Providers/OpenRouter: remove conflicting top-level `reasoning_effort` when injecting nested `reasoning.effort`, preventing OpenRouter 400 payload-validation failures for reasoning models. (#24120) thanks @tenequm. +- Plugins/Install: when npm install returns 404 for bundled channel npm specs, fallback to bundled channel sources and complete install/enable persistence instead of failing plugin install. (#12849) Thanks @vincentkoc. +- Gemini OAuth/Auth: resolve npm global shim install layouts while discovering Gemini CLI credentials, preventing false "Gemini CLI not found" onboarding/auth failures when shim paths are on `PATH`. (#27585) Thanks @ehgamemo and @vincentkoc. +- Providers/Groq: avoid classifying Groq TPM limit errors as context overflow so throttling paths no longer trigger overflow recovery logic. (#16176) Thanks @dddabtc. +- Gateway/WS: close repeated post-handshake `unauthorized role:*` request floods per connection and sample duplicate rejection logs, preventing a single misbehaving client from degrading gateway responsiveness. (#20168) Thanks @acy103, @vibecodooor, and @vincentkoc. +- Gateway/Restart: treat child listener PIDs as owned by the service runtime PID during restart health checks to avoid false stale-process kills and restart timeouts on launchd/systemd. (#24696) Thanks @gumadeiras. +- Config/Write: apply `unsetPaths` with immutable path-copy updates so config writes never mutate caller-provided objects, and harden `openclaw config get/set/unset` path traversal by rejecting prototype-key segments and inherited-property traversal. (#24134) thanks @frankekn. +- Channels/WhatsApp: accept `channels.whatsapp.enabled` in config validation to match built-in channel auto-enable behavior, preventing `Unrecognized key: "enabled"` failures during channel setup. (#24263) +- Security/Exec: detect obfuscated commands before exec allowlist decisions and require explicit approval for obfuscation patterns. (#8592) Thanks @CornBrother0x and @vincentkoc. +- Security/ACP: harden ACP client permission auto-approval to require trusted core tool IDs, ignore untrusted `toolCall.kind` hints, and scope `read` auto-approval to the active working directory so unknown tool names and out-of-scope file reads always prompt. Thanks @nedlir for reporting. +- Security/Skills: escape user-controlled prompt, filename, and output-path values in `openai-image-gen` HTML gallery generation to prevent stored XSS in generated `index.html` output. (#12538) Thanks @CornBrother0x. +- Security/Skills: harden `skill-creator` packaging by skipping symlink entries and rejecting files whose resolved paths escape the selected skill root. (#24260, #16959) Thanks @CornBrother0x and @vincentkoc. +- Security/OTEL: redact sensitive values (API keys, tokens, credential fields) from diagnostics-otel log bodies, log attributes, and error/reason span fields before OTLP export. (#12542) Thanks @brandonwise. +- Security/CI: add pre-commit security hook coverage for private-key detection and production dependency auditing, and enforce those checks in CI alongside baseline secret scanning. Thanks @vincentkoc. +- Skills/Python: harden skill script packaging and validation edge cases (self-including `.skill` outputs, CRLF frontmatter parsing, strict `--days` validation, and safer image file loading), with expanded Python regression coverage. Thanks @vincentkoc. +- Skills/Python: add CI + pre-commit linting (`ruff`) and pytest discovery coverage for Python scripts/tests under `skills/`, including package test execution from repo root. Thanks @vincentkoc. + +## 2026.2.22 + +### Changes + +- Control UI/Agents: make the Tools panel data-driven from runtime `tools.catalog`, add per-tool provenance labels (`core` / `plugin:` + optional marker), and keep a static fallback list when the runtime catalog is unavailable. +- Web Search/Gemini: add grounded Gemini provider support with provider auto-detection and config/docs updates. (#13075, #13074) Thanks @akoscz. +- Control UI/Cron: add full web cron edit parity (including clone and richer validation/help text), plus all-jobs run history with pagination/search/sort/multi-filter controls and improved cron page layout for cleaner scheduling and failure triage workflows. +- Provider/Mistral: add support for the Mistral provider, including memory embeddings and voice support. (#23845) Thanks @vincentkoc. +- Update/Core: add an optional built-in auto-updater for package installs (`update.auto.*`), default-off, with stable rollout delay+jitter and beta hourly cadence. +- CLI/Update: add `openclaw update --dry-run` to preview channel/tag/target/restart actions without mutating config, installing, syncing plugins, or restarting. +- Config/UI: add tag-aware settings filtering and broaden config labels/help copy so fields are easier to discover and understand in the dashboard config screen. +- Channels/Synology Chat: add a native Synology Chat channel plugin with webhook ingress, direct-message routing, outbound send/media support, per-account config, and DM policy controls. (#23012) +- iOS/Talk: prefetch TTS segments and suppress expected speech-cancellation errors for smoother talk playback. (#22833) Thanks @ngutman. +- Memory/FTS: add Spanish and Portuguese stop-word filtering for query expansion in FTS-only search mode, improving conversational recall for both languages. Thanks @vincentkoc. +- Memory/FTS: add Japanese-aware query expansion tokenization and stop-word filtering (including mixed-script terms like ASCII + katakana) for FTS-only search mode. Thanks @vincentkoc. +- Memory/FTS: add Korean stop-word filtering and particle-aware keyword extraction (including mixed Korean/English stems) for query expansion in FTS-only search mode. (#18899) Thanks @ruypang. +- Memory/FTS: add Arabic stop-word filtering for query expansion in FTS-only search mode to reduce conversational filler in Arabic memory searches. Thanks @vincentkoc. +- Discord/Allowlist: canonicalize resolved Discord allowlist names to IDs and split resolution flow for clearer fail-closed behavior. +- Channels/Config: unify channel preview streaming config handling with a shared resolver and canonical migration path. +- Gateway/Auth: unify call/probe/status/auth credential-source precedence on shared resolver helpers, with table-driven parity coverage across gateway entrypoints. +- Gateway/Auth: refactor gateway credential resolution and websocket auth handshake paths to use shared typed auth contexts, including explicit `auth.deviceToken` support in connect frames and tests. +- Skills: remove bundled `food-order` skill from this repo; manage/install it from ClawHub instead. +- Docs/Subagents: make thread-bound session guidance channel-first instead of Discord-specific, and list thread-supporting channels explicitly. (#23589) Thanks @osolmaz. + +### Breaking + +- **BREAKING:** removed Google Antigravity provider support and the bundled `google-antigravity-auth` plugin. Existing `google-antigravity/*` model/profile configs no longer work; migrate to `google-gemini-cli` or other supported providers. +- **BREAKING:** tool-failure replies now hide raw error details by default. OpenClaw still sends a failure summary, but detailed error suffixes (for example provider/runtime messages and local path fragments) now require `/verbose on` or `/verbose full`. +- **BREAKING:** CLI local onboarding now sets `session.dmScope` to `per-channel-peer` by default for new/implicit DM scope configuration. If you depend on shared DM continuity across senders, explicitly set `session.dmScope` to `main`. (#23468) Thanks @bmendonca3. +- **BREAKING:** unify channel preview-streaming config to `channels..streaming` with enum values `off | partial | block | progress`, and move Slack native stream toggle to `channels.slack.nativeStreaming`. Legacy keys (`streamMode`, Slack boolean `streaming`) are still read and migrated by `openclaw doctor --fix`, but canonical saved config/docs now use the unified names. +- **BREAKING:** remove legacy Gateway device-auth signature `v1`. Device-auth clients must now sign `v2` payloads with the per-connection `connect.challenge` nonce and send `device.nonce`; nonce-less connects are rejected. + +### Fixes + +- Sessions/Resilience: ignore invalid persisted `sessionFile` metadata and fall back to the derived safe transcript path instead of aborting session resolution for handlers and tooling. (#16061) Thanks @haoyifan and @vincentkoc. +- Sessions/Paths: resolve symlinked state-dir aliases during transcript-path validation while preserving safe cross-agent/state-root compatibility for valid `agents//sessions/**` paths. (#18593) Thanks @EpaL and @vincentkoc. +- Agents/Compaction: count auto-compactions only after a non-retry `auto_compaction_end`, keeping session `compactionCount` aligned to completed compactions. +- Security/CLI: redact sensitive values in `openclaw config get` output before printing config paths, preventing credential leakage to terminal output/history. (#13683) Thanks @SleuthCo. +- Agents/Moonshot: force `supportsDeveloperRole=false` for Moonshot-compatible `openai-completions` models (provider `moonshot` and Moonshot base URLs), so initial runs no longer send unsupported `developer` roles that trigger `ROLE_UNSPECIFIED` errors. (#21060, #22194) Thanks @ShengFuC. +- Agents/Kimi: classify Moonshot `Your request exceeded model token limit` failures as context overflows so auto-compaction and user-facing overflow recovery trigger correctly instead of surfacing raw invalid-request errors. (#9562) Thanks @danilofalcao. +- Providers/Moonshot: mark Kimi K2.5 as image-capable in implicit + onboarding model definitions, and refresh stale explicit provider capability fields (`input`/`reasoning`/context limits) from implicit catalogs so existing configs pick up Moonshot vision support without manual model rewrites. (#13135, #4459) Thanks @manikv12. +- Agents/Transcript: enable consecutive-user turn merging for strict non-OpenAI `openai-completions` providers (for example Moonshot/Kimi), reducing `roles must alternate` ordering failures on OpenAI-compatible endpoints while preserving current OpenRouter/Opencode behavior. (#7693) +- Install/Discord Voice: make `@discordjs/opus` an optional dependency so `openclaw` install/update no longer hard-fails when native Opus builds fail, while keeping `opusscript` as the runtime fallback decoder for Discord voice flows. (#23737, #23733, #23703) Thanks @jeadland, @Sheetaa, and @Breakyman. +- Docker/Setup: precreate `$OPENCLAW_CONFIG_DIR/identity` during `docker-setup.sh` so CLI commands that need device identity (for example `devices list`) avoid `EACCES ... /home/node/.openclaw/identity` failures on restrictive bind mounts. (#23948) Thanks @ackson-beep. +- Exec/Background: stop applying the default exec timeout to background sessions (`background: true` or explicit `yieldMs`) when no explicit timeout is set, so long-running background jobs are no longer terminated at the default timeout boundary. (#23303) +- Slack/Threading: sessions: keep parent-session forking and thread-history context active beyond first turn by removing first-turn-only gates in session init, thread-history fetch, and reply prompt context injection. (#23843, #23090) Thanks @vincentkoc and @Taskle. +- Slack/Threading: respect `replyToMode` when Slack auto-populates top-level `thread_ts`, and ignore inline `replyToId` directive tags when `replyToMode` is `off` so thread forcing stays disabled unless explicitly configured. (#23839, #23320, #23513) Thanks @vincentkoc and @dorukardahan. +- Slack/Extension: forward `message read` `threadId` to `readMessages` and use delivery-context `threadId` as outbound `thread_ts` fallback so extension replies/reads stay in the correct Slack thread. (#22216, #22485, #23836) Thanks @vincentkoc, @lan17 and @dorukardahan. +- Slack/Upload: resolve bare user IDs (U-prefix) to DM channel IDs via `conversations.open` before calling `files.uploadV2`, which rejects non-channel IDs. `chat.postMessage` tolerates user IDs directly, but `files.uploadV2` → `completeUploadExternal` validates `channel_id` against `^[CGDZ][A-Z0-9]{8,}$`, causing `invalid_arguments` when agents reply with media to DM conversations. +- Webchat/Chat: apply assistant `final` payload messages directly to chat state so sent turns render without waiting for a full history refresh cycle. (#14928) Thanks @BradGroux. +- Webchat/Chat: for out-of-band final events (for example tool-call side runs), append provided final assistant payloads directly instead of forcing a transient history reset. (#11139) Thanks @AkshayNavle. +- Webchat/Performance: reload `chat.history` after final events only when the final payload lacks a renderable assistant message, avoiding expensive full-history refreshes on normal turns. (#20588) Thanks @amzzzzzzz. +- Webchat/Sessions: preserve external session routing metadata when internal `chat.send` turns run under `webchat`, so explicit channel-keyed sessions (for example Telegram) no longer get rewritten to `webchat` and misroute follow-up delivery. (#23258) Thanks @binary64. +- Webchat/Sessions: preserve existing session `label` across `/new` and `/reset` rollovers so reset sessions remain discoverable in session history lists. (#23755) Thanks @ThunderStormer. +- Gateway/Chat UI: strip inline reply/audio directive tags from non-streaming final webchat broadcasts (including `chat.inject`) while preserving empty-string message content when tags are the entire reply. (#23298) Thanks @SidQin-cyber. +- Chat/UI: strip inline reply/audio directive tags (`[[reply_to_current]]`, `[[reply_to:]]`, `[[audio_as_voice]]`) from displayed chat history, live chat event output, and session preview snippets so control tags no longer leak into user-visible surfaces. +- Gateway/Chat UI: sanitize non-streaming final `chat.send`/`chat.inject` payload text with the same envelope/untrusted-context stripping used by `chat.history`, preventing `<<>>` wrapper markup from rendering in Control UI chat. (#24012) Thanks @mittelaltergouda. +- Telegram/Media: send a user-facing Telegram reply when media download fails (non-size errors) instead of silently dropping the message. +- Telegram/Webhook: keep webhook monitors alive until gateway abort signals fire, preventing false channel exits and immediate webhook auto-restart loops. +- Telegram/Polling: retry recoverable setup-time network failures in monitor startup and await runner teardown before retry to avoid overlapping polling sessions. +- Telegram/Polling: clear Telegram webhooks (`deleteWebhook`) before starting long-poll `getUpdates`, including retry handling for transient cleanup failures. +- Telegram/Webhook: add `channels.telegram.webhookPort` config support and pass it through plugin startup wiring to the monitor listener. +- Browser/Extension Relay: refactor the MV3 worker to preserve debugger attachments across relay drops, auto-reconnect with bounded backoff+jitter, persist and rehydrate attached tab state via `chrome.storage.session`, recover from `target_closed` navigation detaches, guard stale socket handlers, enforce per-tab operation locks and per-request timeouts, and add lifecycle keepalive/badge refresh hooks (`alarms`, `webNavigation`). (#15099, #6175, #8468, #9807) +- Browser/Relay: treat extension websocket as connected only when `OPEN`, allow reconnect when a stale `CLOSING/CLOSED` extension socket lingers, and guard stale socket message/close handlers so late events cannot clear active relay state; includes regression coverage for live-duplicate `409` rejection and immediate reconnect-after-close races. (#15099, #18698, #20688) +- Browser/Remote CDP: extend stale-target recovery so `ensureTabAvailable()` now reuses the sole available tab for remote CDP profiles (same behavior as extension profiles) while preserving strict `tab not found` errors when multiple tabs exist; includes remote-profile regression tests. (#15989) +- Gateway/Pairing: treat `operator.admin` as satisfying other `operator.*` scope checks during device-auth verification so local CLI/TUI sessions stop entering pairing-required loops for pairing/approval-scoped commands. (#22062, #22193, #21191) Thanks @Botaccess, @jhartshorn, and @ctbritt. +- Gateway/Pairing: auto-approve loopback `scope-upgrade` pairing requests (including device-token reconnects) so local clients do not disconnect on pairing-required scope elevation. (#23708) Thanks @widingmarcus-cyber. +- Gateway/Scopes: include `operator.read` and `operator.write` in default operator connect scope bundles across CLI, Control UI, and macOS clients so write-scoped announce/sub-agent follow-up calls no longer hit `pairing required` disconnects on loopback gateways. (#22582) thanks @YuzuruS. +- Gateway/Pairing: treat operator.admin pairing tokens as satisfying operator.write requests so legacy devices stop looping through scope-upgrade prompts introduced in 2026.2.19. (#23125, #23006) Thanks @vignesh07. +- Gateway/Restart: fix restart-loop edge cases by keeping `openclaw.mjs -> dist/entry.js` bootstrap detection explicit, reacquiring the gateway lock for in-process restart fallback paths, and tightening restart-loop regression coverage. (#23416) Thanks @jeffwnli. +- Gateway/Lock: use optional gateway-port reachability as a primary stale-lock liveness signal (and wire gateway run-loop lock acquisition to the resolved port), reducing false "already running" lockouts after unclean exits. (#23760) Thanks @Operative-001. +- Delivery/Queue: quarantine queue entries immediately on known permanent delivery errors (for example invalid recipients or missing conversation references) by moving them to `failed/` instead of retrying on every restart. (#23794) Thanks @aldoeliacim. +- Cron/Status: split execution outcome (`lastRunStatus`) from delivery outcome (`lastDeliveryStatus`) in persisted cron state, finished events, and run history so failed/unknown announcement delivery is visible without conflating it with run errors. +- Cron/Delivery: route text-only announce jobs with explicit thread/topic targets through direct outbound delivery so forum/thread destinations do not get dropped by intermediary announce turns. (#23841) Thanks @AndrewArto. +- Cron: honor `cron.maxConcurrentRuns` in the timer loop so due jobs can execute up to the configured parallelism instead of always running serially. (#11595) Thanks @Takhoffman. +- Cron/Run: enforce the same per-job timeout guard for manual `cron.run` executions as timer-driven runs, including abort propagation for isolated agent jobs, so forced runs cannot wedge indefinitely. (#23704) Thanks @tkuehnl. +- Cron/Run: persist the manual-run `runningAtMs` marker before releasing the cron lock so overlapping timer ticks cannot start the same job concurrently. +- Cron/Startup: enforce per-job timeout guards for startup catch-up replay runs so missed isolated jobs cannot hang indefinitely during gateway boot recovery. +- Cron/Main session: honor abort/timeout signals while retrying `wakeMode=now` heartbeat contention loops so main-target cron runs stop promptly instead of waiting through the full busy-retry window. +- Cron/Schedule: for `every` jobs, prefer `lastRunAtMs + everyMs` when still in the future after restarts, then fall back to anchor scheduling for catch-up windows, so NEXT timing matches the last successful cadence. (#22895) Thanks @SidQin-cyber. +- Cron/Service: execute manual `cron.run` jobs outside the cron lock (while still persisting started/finished state atomically) so `cron.list` and `cron.status` remain responsive during long forced runs. (#23628) Thanks @dsgraves. +- Cron/Timer: keep a watchdog recheck timer armed while `onTimer` is actively executing so the scheduler continues polling even if a due-run tick stalls for an extended period. (#23628) Thanks @dsgraves. +- Cron/Run log: clean up settled per-path run-log write queue entries so long-running cron uptime does not retain stale promise bookkeeping in memory. +- Cron/Run log: harden `cron.runs` run-log path resolution by rejecting path-separator `id`/`jobId` inputs and enforcing reads within the per-cron `runs/` directory. +- Cron/Announce: when announce delivery target resolution fails (for example multiple configured channels with no explicit target), skip injecting fallback `Cron (error): ...` into the main session so runs fail cleanly without accidental last-route sends. (#24074) +- Cron/Telegram: validate cron `delivery.to` with shared Telegram target parsing and resolve legacy `@username`/`t.me` targets to numeric IDs at send-time for deterministic delivery target writeback. (#21930) Thanks @kesor. +- Telegram/Targets: normalize unprefixed topic-qualified targets through the shared parse/normalize path so valid `@channel:topic:` and `:topic:` routes are recognized again. (#24166) Thanks @obviyus. +- Cron/Isolation: force fresh session IDs for isolated cron runs so `sessionTarget="isolated"` executions never reuse prior run context. (#23470) Thanks @echoVic. +- Plugins/Install: strip `workspace:*` devDependency entries from copied plugin manifests before `npm install --omit=dev`, preventing `EUNSUPPORTEDPROTOCOL` install failures for npm-published channel plugins (including Feishu and MS Teams). +- Feishu/Plugins: restore bundled Feishu SDK availability for global installs and strip `openclaw: workspace:*` from plugin `devDependencies` during plugin-version sync so npm-installed Feishu plugins do not fail dependency install. (#23611, #23645, #23603) +- Config/Channels: auto-enable built-in channels by writing `channels..enabled=true` (not `plugins.entries.`), and stop adding built-ins to `plugins.allow`, preventing `plugins.entries.telegram: plugin not found` validation failures. +- Config/Channels: when `plugins.allow` is active, auto-enable/enable flows now also allowlist configured built-in channels so `channels..enabled=true` cannot remain blocked by restrictive plugin allowlists. +- Plugins/Discovery: ignore scanned extension backup/disabled directory patterns (for example `.backup-*`, `.bak`, `.disabled*`) and move updater backup directories under `.openclaw-install-backups`, preventing duplicate plugin-id collisions from archived copies. +- Plugins/CLI: make `openclaw plugins enable` and plugin install/link flows update allowlists via shared plugin-enable policy so enabled plugins are not left disabled by allowlist mismatch. (#23190) Thanks @downwind7clawd-ctrl. +- Security/Voice Call: harden media stream WebSocket handling against pre-auth idle-connection DoS by adding strict pre-start timeouts, pending/per-IP connection limits, and total connection caps for streaming endpoints. Thanks @jiseoung for reporting. +- Security/Sessions: redact sensitive token patterns from `sessions_history` tool output and surface `contentRedacted` metadata when masking occurs. (#16928) Thanks @aether-ai-agent. +- Security/Exec: stop trusting `PATH`-derived directories for safe-bin allowlist checks, add explicit `tools.exec.safeBinTrustedDirs`, and pin safe-bin shell execution to resolved absolute executable paths to prevent binary-shadowing approval bypasses. Thanks @tdjackey for reporting. +- Security/Elevated: match `tools.elevated.allowFrom` against sender identities only (not recipient `ctx.To`), closing a recipient-token bypass for `/elevated` authorization. Thanks @jiseoung for reporting. +- Security/Feishu: enforce ID-only allowlist matching for DM/group sender authorization, normalize Feishu ID prefixes during checks, and ignore mutable display names so display-name collisions cannot satisfy allowlist entries. Thanks @jiseoung for reporting. +- Security/Group policy: harden `channels.*.groups.*.toolsBySender` matching by requiring explicit sender-key types (`id:`, `e164:`, `username:`, `name:`), preventing cross-identifier collisions across mutable/display-name fields while keeping legacy untyped keys on a deprecated ID-only path. Thanks @jiseoung for reporting. +- Channels/Group policy: fail closed when `groupPolicy: "allowlist"` is set without explicit `groups`, honor account-level `groupPolicy` overrides, and enforce `groupPolicy: "disabled"` as a hard group block. (#22215) Thanks @etereo. +- Telegram/Discord extensions: propagate trusted `mediaLocalRoots` through extension outbound `sendMedia` options so extension direct-send media paths honor agent-scoped local-media allowlists. (#20029, #21903, #23227) +- Agents/Exec: honor explicit agent context when resolving `tools.exec` defaults for runs with opaque/non-agent session keys, so per-agent `host/security/ask` policies are applied consistently. (#11832) +- CLI/Sessions: resolve implicit session-store path templates with the configured default agent ID so named-agent setups do not silently read/write stale `agent:main` session/auth stores. (#22685) Thanks @sene1337. +- Doctor/Security: add an explicit warning that `approvals.exec.enabled=false` disables forwarding only, while enforcement remains driven by host-local `exec-approvals.json` policy. (#15047) +- Sandbox/Docker: default sandbox container user to the workspace owner `uid:gid` when `agents.*.sandbox.docker.user` is unset, fixing non-root gateway file-tool permissions under capability-dropped containers. (#20979) +- Plugins/Media sandbox: propagate trusted `mediaLocalRoots` through plugin action dispatch (including Discord/Telegram action adapters) so plugin send paths enforce the same agent-scoped local-media sandbox roots as core outbound sends. (#20258, #22718) +- Agents/Workspace guard: map sandbox container-workdir file-tool paths (for example `/workspace/...` and `file:///workspace/...`) to host workspace roots before workspace-only validation, preventing false `Path escapes sandbox root` rejections for sandbox file tools. (#9560) +- Gateway/Exec approvals: expire approval requests immediately when no approval-capable gateway clients are connected and no forwarding targets are available, avoiding delayed approvals after restarts/offline approver windows. (#22144) +- Security/Exec approvals: when approving wrapper commands with allow-always in allowlist mode, persist inner executable paths for known dispatch wrappers (`env`, `nice`, `nohup`, `stdbuf`, `timeout`) and fail closed (no persisted entry) when wrapper unwrapping is not safe, preventing wrapper-path approval bypasses. Thanks @tdjackey for reporting. +- Node/macOS exec host: default headless macOS node `system.run` to local execution and only route through the companion app when `OPENCLAW_NODE_EXEC_HOST=app` is explicitly set, avoiding companion-app filesystem namespace mismatches during exec. (#23547) +- Sandbox/Media: map container workspace paths (`/workspace/...` and `file:///workspace/...`) back to the host sandbox root for outbound media validation, preventing false deny errors for sandbox-generated local media. (#23083) Thanks @echo931. +- Sandbox/Docker: apply custom bind mounts after workspace mounts and prioritize bind-source resolution on overlapping paths, so explicit workspace binds are no longer ignored. (#22669) Thanks @tasaankaeris. +- Exec approvals/Forwarding: restore Discord text forwarding when component approvals are not configured, and carry request snapshots through resolve events so resolved notices still forward after cache misses/restarts. (#22988) Thanks @bubmiller. +- Control UI/WebSocket: stop and clear the browser gateway client on UI teardown so remounts cannot leave orphan websocket clients that create duplicate active connections. (#23422) Thanks @floatinggball-design. +- Control UI/WebSocket: send a stable per-tab `instanceId` in websocket connect frames so reconnect cycles keep a consistent client identity for diagnostics and presence tracking. (#23616) Thanks @zq58855371-ui. +- Config/Memory: allow `"mistral"` in `agents.defaults.memorySearch.provider` and `agents.defaults.memorySearch.fallback` schema validation. (#14934) Thanks @ThomsenDrake. +- Feishu/Commands: in group chats, command authorization now falls back to top-level `channels.feishu.allowFrom` when per-group `allowFrom` is not set, so `/command` no longer gets blocked by an unintended empty allowlist. (#23756) +- Dev tooling: prevent `CLAUDE.md` symlink target regressions by excluding CLAUDE symlink sentinels from `oxfmt` and marking them `-text` in `.gitattributes`, so formatter/EOL normalization cannot reintroduce trailing-newline targets. Thanks @vincentkoc. +- Agents/Compaction: restore embedded compaction safeguard/context-pruning extension loading in production by wiring bundled extension factories into the resource loader instead of runtime file-path resolution. (#22349) Thanks @Glucksberg. +- Feishu/Media: for inbound video messages that include both `file_key` (video) and `image_key` (thumbnail), prefer `file_key` when downloading media so video attachments are saved instead of silently failing on thumbnail keys. (#23633) +- Hooks/Loader: avoid redundant hook-module recompilation on gateway restart by skipping cache-busting for bundled hooks and using stable file metadata keys (`mtime+size`) for mutable workspace/managed/plugin hook imports. (#16953) Thanks @mudrii. +- Hooks/Cron: suppress duplicate main-session events for delivered hook turns and mark `SILENT_REPLY_TOKEN` (`NO_REPLY`) early exits as delivered to prevent hook context pollution. (#20678) Thanks @JonathanWorks. +- Providers/OpenRouter: inject `cache_control` on system prompts for OpenRouter Anthropic models to improve prompt-cache reuse. (#17473) Thanks @rrenamed. +- Installer/Smoke tests: remove legacy `OPENCLAW_USE_GUM` overrides from docker install-smoke runs so tests exercise installer auto TTY detection behavior directly. +- Providers/OpenRouter: allow pass-through OpenRouter and Opencode model IDs in live model filtering so custom routed model IDs are treated as modern refs. (#14312) Thanks @Joly0. +- Providers/OpenRouter: default reasoning to enabled when the selected model advertises `reasoning: true` and no session/directive override is set. (#22513) Thanks @zwffff. +- Providers/OpenRouter: map `/think` levels to `reasoning.effort` in embedded runs while preserving explicit `reasoning.max_tokens` payloads. (#17236) Thanks @robbyczgw-cla. +- Providers/OpenRouter: preserve stored session provider when model IDs are vendor-prefixed (for example, `anthropic/...`) so follow-up turns do not incorrectly route to direct provider APIs. (#22753) Thanks @dndodson. +- Providers/OpenRouter: preserve the required `openrouter/` prefix for OpenRouter-native model IDs during model-ref normalization. (#12942) Thanks @omair445. +- Providers/OpenRouter: pass through provider routing parameters from model params.provider to OpenRouter request payloads for provider selection controls. (#17148) Thanks @carrotRakko. +- Providers/OpenRouter: preserve model allowlist entries containing OpenRouter preset paths (for example `openrouter/@preset/...`) by treating `/model ...@profile` auth-profile parsing as a suffix-only override. (#14120) Thanks @NotMainstream. +- Cron/Auth: propagate auth-profile resolution to isolated cron sessions so provider API keys are resolved the same way as main sessions, fixing 401 errors when using providers configured via auth-profiles. (#20689) Thanks @lailoo. +- Cron/Follow-up: pass resolved `agentDir` through isolated cron and queued follow-up embedded runs so auth/profile lookups stay scoped to the correct agent directory. (#22845) Thanks @seilk. +- Agents/Media: route tool-result `MEDIA:` extraction through shared parser validation so malformed prose like `MEDIA:-prefixed ...` is no longer treated as a local file path (prevents Telegram ENOENT tool-error overrides). (#18780) Thanks @HOYALIM. +- Logging: cap single log-file size with `logging.maxFileBytes` (default 500 MB) and suppress additional writes after cap hit to prevent disk exhaustion from repeated error storms. +- Memory/Remote HTTP: centralize remote memory HTTP calls behind a shared guarded helper (`withRemoteHttpResponse`) so embeddings and batch flows use one request/release path. +- Memory/Embeddings: apply configured remote-base host pinning (`allowedHostnames`) across OpenAI/Voyage/Gemini embedding requests to keep private/self-hosted endpoints working without cross-host drift. (#18198) Thanks @ianpcook. +- Memory/Batch: route OpenAI/Voyage/Gemini batch upload/create/status/download requests through the same guarded HTTP path for consistent SSRF policy enforcement. +- Memory/Index: detect memory source-set changes (for example enabling `sessions` after an existing memory-only index) and trigger a full reindex so existing session transcripts are indexed without requiring `--force`. (#17576) Thanks @TarsAI-Agent. +- Memory/Embeddings: enforce a per-input 8k safety cap before embedding batching and apply a conservative 2k fallback limit for local providers without declared input limits, preventing oversized session/memory chunks from triggering provider context-size failures during sync/indexing. (#6016) Thanks @batumilove. +- Memory/QMD: on Windows, resolve bare `qmd`/`mcporter` command names to npm shim executables (`.cmd`) before spawning, so qmd boot updates and mcporter-backed searches no longer fail with `spawn ... ENOENT` on default npm installs. (#23899) Thanks @arcbuilder-ai. +- Memory/QMD: parse plain-text `qmd collection list --json` output when older qmd builds ignore JSON mode, and retry memory searches once after re-ensuring managed collections when qmd returns `Collection not found ...`. (#23613) Thanks @leozhucn. +- iOS/Watch: normalize watch quick-action notification payloads, support mirrored indexed actions beyond primary/secondary, and fix iOS test-target signing/compile blockers for watch notify coverage. (#23636) Thanks @mbelinky. +- Signal/RPC: guard malformed Signal RPC JSON responses with a clear status-scoped error and add regression coverage for invalid JSON responses. (#22995) Thanks @adhitShet. +- Gateway/Subagents: guard gateway and subagent session-key/message trim paths against undefined inputs to prevent early `Cannot read properties of undefined (reading 'trim')` crashes during subagent spawn and wait flows. +- Agents/Workspace: guard `resolveUserPath` against undefined/null input to prevent `Cannot read properties of undefined (reading 'trim')` crashes when workspace paths are missing in embedded runner flows. +- Auth/Profiles: keep active `cooldownUntil`/`disabledUntil` windows immutable across retries so mid-window failures cannot extend recovery indefinitely; only recompute a backoff window after the previous deadline has expired. This resolves cron/inbound retry loops that could trap gateways until manual `usageStats` cleanup. (#23516, #23536) Thanks @arosstale. +- Channels/Security: fail closed on missing provider group policy config by defaulting runtime group policy to `allowlist` (instead of inheriting `channels.defaults.groupPolicy`) when `channels.` is absent across message channels, and align runtime + security warnings/docs to the same fallback behavior (Slack, Discord, iMessage, Telegram, WhatsApp, Signal, LINE, Matrix, Mattermost, Google Chat, IRC, Nextcloud Talk, Feishu, and Zalo user flows; plus Discord message/native-command paths). (#23367) Thanks @bmendonca3. +- Gateway/Onboarding: harden remote gateway onboarding defaults and guidance by defaulting discovered direct URLs to `wss://`, rejecting insecure non-loopback `ws://` targets in onboarding validation, and expanding remote-security remediation messaging across gateway client/call/doctor flows. (#23476) Thanks @bmendonca3. +- CLI/Sessions: pass the configured sessions directory when resolving transcript paths in `agentCommand`, so custom `session.store` locations resume sessions reliably. Thanks @davidrudduck. +- Signal/Monitor: treat user-initiated abort shutdowns as clean exits when auto-started `signal-cli` is terminated, while still surfacing unexpected daemon exits as startup/runtime failures. (#23379) Thanks @frankekn. +- Channels/Dedupe: centralize plugin dedupe primitives in plugin SDK (memory + persistent), move Feishu inbound dedupe to a namespace-scoped persistent store, and reuse shared dedupe cache logic for Zalo webhook replay + Tlon processed-message tracking to reduce duplicate handling during reconnect/replay paths. (#23377) Thanks @SidQin-cyber. +- Channels/Delivery: remove hardcoded WhatsApp delivery fallbacks; require explicit/session channel context or auto-pick the sole configured channel when unambiguous. (#23357) Thanks @lbo728. +- ACP/Gateway: wait for gateway hello before opening ACP requests, and fail fast on pre-hello connect failures to avoid startup hangs and early `gateway not connected` request races. (#23390) Thanks @janckerchen. +- Gateway/Auth: preserve `OPENCLAW_GATEWAY_PASSWORD` env override precedence for remote gateway call credentials after shared resolver refactors, preventing stale configured remote passwords from overriding runtime secret rotation. +- Gateway/Auth: preserve shared-token `gateway token mismatch` auth errors when `auth.token` fallback device-token checks fail, and reserve `device token mismatch` guidance for explicit `auth.deviceToken` failures. +- Gateway/Tools: when agent tools pass an allowlisted `gatewayUrl` override, resolve local override tokens from env/config fallback but keep remote overrides strict to `gateway.remote.token`, preventing local token leakage to remote targets. +- Gateway/Client: keep cached device-auth tokens on `device token mismatch` closes when the client used explicit shared token/password credentials, avoiding accidental pairing-token churn during explicit-auth failures. +- Node host/Exec: keep strict Windows allowlist behavior for `cmd.exe /c` shell-wrapper runs, and return explicit approval guidance when blocked (`SYSTEM_RUN_DENIED: allowlist miss`). +- Control UI: show pairing-required guidance (commands + mobile tokenized URL reminder) when the dashboard disconnects with `1008 pairing required`. +- Security/Audit: add `openclaw security audit` detection for open group policies that expose runtime/filesystem tools without sandbox/workspace guards (`security.exposure.open_groups_with_runtime_or_fs`). +- Security/Audit: make `gateway.real_ip_fallback_enabled` severity conditional for loopback trusted-proxy setups (warn for loopback-only `trustedProxies`, critical when non-loopback proxies are trusted). (#23428) Thanks @bmendonca3. +- Security/Exec env: block request-scoped `HOME` and `ZDOTDIR` overrides in host exec env sanitizers (Node + macOS), preventing shell startup-file execution before allowlist-evaluated command bodies. Thanks @tdjackey for reporting. +- Security/Exec env: block `SHELLOPTS`/`PS4` in host exec env sanitizers and restrict shell-wrapper (`bash|sh|zsh ... -c/-lc`) request env overrides to a small explicit allowlist (`TERM`, `LANG`, `LC_*`, `COLORTERM`, `NO_COLOR`, `FORCE_COLOR`) on both node host and macOS companion paths, preventing xtrace prompt command-substitution allowlist bypasses. Thanks @tdjackey for reporting. +- WhatsApp/Security: enforce `allowFrom` for direct-message outbound targets in all send modes (including `mode: "explicit"`), preventing sends to non-allowlisted numbers. (#20108) Thanks @zahlmann. +- Security/Exec approvals: fail closed on shell line continuations (`\\\n`/`\\\r\n`) and treat shell-wrapper execution as approval-required in allowlist mode, preventing `$\\` newline command-substitution bypasses. Thanks @tdjackey for reporting. +- Security/Gateway: emit a startup security warning when insecure/dangerous config flags are enabled (including `gateway.controlUi.dangerouslyDisableDeviceAuth=true`) and point operators to `openclaw security audit`. +- Security/Hooks auth: normalize hook auth rate-limit client IP keys so IPv4 and IPv4-mapped IPv6 addresses share one throttle bucket, preventing dual-form auth-attempt budget bypasses. Thanks @aether-ai-agent for reporting. +- Security/Exec approvals: treat `env` and shell-dispatch wrappers as transparent during allowlist analysis on node-host and macOS companion paths so policy checks match the effective executable/inline shell payload instead of the wrapper binary, blocking wrapper-smuggled allowlist bypasses. Thanks @tdjackey for reporting. +- Security/Exec approvals: require explicit safe-bin profiles for `tools.exec.safeBins` entries in allowlist mode (remove generic safe-bin profile fallback), and add `tools.exec.safeBinProfiles` for safe custom binaries so unprofiled interpreter-style entries cannot be treated as stdin-safe. Thanks @tdjackey for reporting. +- Security/Channels: harden Slack external menu token handling by switching to CSPRNG tokens, validating token shape, requiring user identity for external option lookups, and avoiding fabricated timestamp `trigger_id` fallbacks; also switch Tlon Urbit channel IDs to CSPRNG UUIDs, centralize secure ID/token generation via shared infra helpers, and add a guardrail test to block new runtime `Date.now()+Math.random()` token/id patterns. +- Security/Hooks transforms: enforce symlink-safe containment for webhook transform module paths (including `hooks.transformsDir` and `hooks.mappings[].transform.module`) by resolving existing-path ancestors via realpath before import, while preserving in-root symlink support; add regression coverage for both escape and allow cases. Thanks @aether-ai-agent for reporting. +- Telegram/WSL2: disable `autoSelectFamily` by default on WSL2 and memoize WSL2 detection in Telegram network decision logic to avoid repeated sync `/proc/version` probes on fetch/send paths. (#21916) Thanks @MizukiMachine. +- Telegram/Network: default Node 22+ DNS result ordering to `ipv4first` for Telegram fetch paths and add `OPENCLAW_TELEGRAM_DNS_RESULT_ORDER`/`channels.telegram.network.dnsResultOrder` overrides to reduce IPv6-path fetch failures. (#5405) Thanks @Glucksberg. +- Telegram/Forward bursts: coalesce forwarded text+media updates through a dedicated forward lane debounce window that works with default inbound debounce config, while keeping forwarded control commands immediate. (#19476) thanks @napetrov. +- Telegram/Streaming: preserve archived draft preview mapping after flush and clean superseded reasoning preview bubbles so multi-message preview finals no longer cross-edit or orphan stale messages under send/rotation races. (#23202) Thanks @obviyus. +- Telegram/Replies: scope messaging-tool text/media dedupe to same-target sends only, so cross-target tool sends can no longer silently suppress Telegram final replies. +- Telegram/Replies: normalize `file://` and local-path media variants during messaging dedupe so equivalent media paths do not produce duplicate Telegram replies. +- Telegram/Replies: extract forwarded-origin context from unified reply targets (`reply_to_message` and `external_reply`) so forward+comment metadata is preserved across partial reply shapes. (#9720) thanks @mcaxtr. +- Telegram/Polling: persist a safe update-offset watermark bounded by pending updates so crash/restart cannot skip queued lower `update_id` updates after out-of-order completion. (#23284) thanks @frankekn. +- Telegram/Polling: force-restart stuck runner instances when recoverable unhandled network rejections escape the polling task path, so polling resumes instead of silently stalling. (#19721) Thanks @jg-noncelogic. +- Slack/Slash commands: preserve the Bolt app receiver when registering external select options handlers so monitor startup does not crash on runtimes that require bound `app.options` calls. (#23209) Thanks @0xgaia. +- Slack/Telegram slash sessions: await session metadata persistence before dispatch so first-turn native slash runs do not race session-origin metadata updates. (#23065) thanks @hydro13. +- Slack/Queue routing: preserve string `thread_ts` values through collect-mode queue drain and DM `deliveryContext` updates so threaded follow-ups do not leak to the main channel when Slack thread IDs are strings. (#11934) Thanks @sandieman2 and @vincentkoc. +- Telegram/Native commands: set `ctx.Provider="telegram"` for native slash-command context so elevated gate checks resolve provider correctly (fixes `provider (ctx.Provider)` failures in `/elevated` flows). (#23748) Thanks @serhii12. +- Agents/Ollama: preserve unsafe integer tool-call arguments as exact strings during NDJSON parsing, preventing large numeric IDs from being rounded before tool execution. (#23170) Thanks @BestJoester. +- Cron/Gateway: keep `cron.list` and `cron.status` responsive during startup catch-up by avoiding a long-held cron lock while missed jobs execute. (#23106) Thanks @jayleekr. +- Gateway/Config reload: compare array-valued config paths structurally during diffing so unchanged `memory.qmd.paths` and `memory.qmd.scope.rules` no longer trigger false restart-required reloads. (#23185) Thanks @rex05ai. +- Gateway/Config reload: retry short-lived missing config snapshots during reload before skipping, preventing atomic-write unlink windows from triggering restart loops. (#23343) Thanks @lbo728. +- Cron/Scheduling: validate runtime cron expressions before schedule/stagger evaluation so malformed persisted jobs report a clear `invalid cron schedule: expr is required` error instead of crashing with `undefined.trim` failures and auto-disable churn. (#23223) Thanks @asimons81. +- Memory/QMD: migrate legacy unscoped collection bindings (for example `memory-root`) to per-agent scoped names (for example `memory-root-main`) during startup when safe, so QMD-backed `memory_search` no longer fails with `Collection not found` after upgrades. (#23228, #20727) Thanks @JLDynamics and @AaronFaby. +- Memory/QMD: normalize Han-script BM25 search queries before invoking `qmd search` so mixed CJK+Latin prompts no longer return empty results due to tokenizer mismatch. (#23426) Thanks @LunaLee0130. +- TUI/Input: enable multiline-paste burst coalescing on macOS Terminal.app and iTerm so pasted blocks no longer submit line-by-line as separate messages. (#18809) Thanks @fwends. +- TUI/RTL: isolate right-to-left script lines (Arabic/Hebrew ranges) with Unicode bidi isolation marks in TUI text sanitization so RTL assistant output no longer renders in reversed visual order in terminal chat panes. (#21936) Thanks @Asm3r96. +- TUI/Status: request immediate renders after setting `sending`/`waiting` activity states so in-flight runs always show visible progress indicators instead of appearing idle until completion. (#21549) Thanks @13Guinness. +- TUI/Input: arm Ctrl+C exit timing when clearing non-empty composer text and add a SIGINT fallback path so double Ctrl+C exits remain responsive during active runs instead of requiring an extra press or appearing stuck. (#23407) Thanks @tinybluedev. +- Agents/Fallbacks: treat JSON payloads with `type: "api_error"` + `"Internal server error"` as transient failover errors so Anthropic 500-style failures trigger model fallback. (#23193) Thanks @jarvis-lane. +- Agents/Google: sanitize non-base64 `thought_signature`/`thoughtSignature` values from assistant replay transcripts for native Google Gemini requests while preserving valid signatures and tool-call order. (#23457) Thanks @echoVic. +- Agents/Transcripts: validate assistant tool-call names (syntax/length + registered tool allowlist) before transcript persistence and during replay sanitization so malformed failover tool names no longer poison sessions with repeated provider HTTP 400 errors. (#23324) Thanks @johnsantry. +- Agents/Mistral: sanitize tool-call IDs in the embedded agent loop and generate strict provider-safe pending tool-call IDs, preventing Mistral strict9 `HTTP 400` failures on tool continuations. (#23698) Thanks @echoVic. +- Agents/Compaction: strip stale assistant usage snapshots from pre-compaction turns when replaying history after a compaction summary so context-token estimation no longer reuses pre-compaction totals and immediately re-triggers destructive follow-up compactions. (#19127) Thanks @tedwatson. +- Agents/Replies: emit a default completion acknowledgement (`✅ Done.`) only for direct/private tool-only completions with no final assistant text, while suppressing synthetic acknowledgements for channel/group sessions and runs that already delivered output via messaging tools. (#22834) Thanks @Oldshue. +- Agents/Subagents: honor `tools.subagents.tools.alsoAllow` and explicit subagent `allow` entries when resolving built-in subagent deny defaults, so explicitly granted tools (for example `sessions_send`) are no longer blocked unless re-denied in `tools.subagents.tools.deny`. (#23359) Thanks @goren-beehero. +- Agents/Subagents: make announce call timeouts configurable via `agents.defaults.subagents.announceTimeoutMs` and restore a 60s default to prevent false timeout failures on slower announce paths. (#22719) Thanks @Valadon. +- Agents/Diagnostics: include resolved lifecycle error text in `embedded run agent end` warnings so UI/TUI “Connection error” runs expose actionable provider failure reasons in gateway logs. (#23054) Thanks @Raize. +- Agents/Auth profiles: resolve `agentCommand` session scope before choosing `agentDir`/workspace so resumed runs no longer read auth from `agents/main/agent` when the resolved session belongs to a different/default agent (for example `agent:exec:*` sessions). (#24016) Thanks @abersonFAC. +- Agents/Auth profiles: skip auth-profile cooldown writes for timeout failures in embedded runner rotation so model/network timeouts do not poison same-provider fallback model selection while still allowing in-turn account rotation. (#22622) Thanks @vageeshkumar. +- Plugins/Hooks: run legacy `before_agent_start` once per agent turn and reuse that result across model-resolve and prompt-build compatibility paths, preventing duplicate hook side effects (for example duplicate external API calls). (#23289) Thanks @ksato8710. +- Models/Config: default missing Anthropic provider/model `api` fields to `anthropic-messages` during config validation so custom relay model entries are preserved instead of being dropped by runtime model registry validation. (#23332) Thanks @bigbigmonkey123. +- Gateway/Pairing: preserve existing approved token scopes when processing repair pairings that omit `scopes`, preventing empty-scope token regressions on reconnecting clients. (#21906) Thanks @paki81. +- Memory/QMD: add optional `memory.qmd.mcporter` search routing so QMD `query/search/vsearch` can run through mcporter keep-alive flows (including multi-collection paths) to reduce cold starts, while keeping searches on agent-scoped QMD state for consistent recall. (#19617) Thanks @nicole-luxe and @vignesh07. +- Infra/Network: classify undici `TypeError: fetch failed` as transient in unhandled-rejection detection even when nested causes are unclassified, preventing avoidable gateway crash loops on flaky networks. (#14345) Thanks @Unayung. +- Telegram/Retry: classify undici `TypeError: fetch failed` as recoverable in both polling and send retry paths so transient fetch failures no longer fail fast. (#16699) thanks @Glucksberg. +- Docs/Telegram: correct Node 22+ network defaults (`autoSelectFamily`, `dnsResultOrder`) and clarify Telegram setup does not use positional `openclaw channels login telegram`. (#23609) Thanks @ryanbastic. +- BlueBubbles/DM history: restore DM backfill context with account-scoped rolling history, bounded backfill retries, and safer history payload limits. (#20302) Thanks @Ryan-Haines. +- BlueBubbles/Private API cache: treat unknown (`null`) private-API cache status as disabled for send/attachment/reply flows to avoid stale-cache 500s, and log a warning when reply/effect features are requested while capability is unknown. (#23459) Thanks @echoVic. +- BlueBubbles/Webhooks: accept inbound/reaction webhook payloads when BlueBubbles omits `handle` but provides DM `chatGuid`, and harden payload extraction for array/string-wrapped message bodies so valid webhook events no longer get rejected as unparseable. (#23275) Thanks @toph31. +- Security/Audit: add `openclaw security audit` finding `gateway.nodes.allow_commands_dangerous` for risky `gateway.nodes.allowCommands` overrides, with severity upgraded to critical on remote gateway exposure. +- Gateway/Control plane: reduce cross-client write limiter contention by adding `connId` fallback keying when device ID and client IP are both unavailable. +- Security/Config: block prototype-key traversal during config merge patch and legacy migration merge helpers (`__proto__`, `constructor`, `prototype`) to prevent prototype pollution during config mutation flows. (#22968) Thanks @Clawborn. +- Security/Shell env: validate login-shell executable paths for shell-env fallback (`/etc/shells` + trusted prefixes), block `SHELL`/`HOME`/`ZDOTDIR` in config env ingestion before fallback execution, and sanitize fallback shell exec env to pin `HOME` to the real user home while dropping `ZDOTDIR` and other dangerous startup vars. Thanks @tdjackey for reporting. +- Network/SSRF: enable `autoSelectFamily` on pinned undici dispatchers (with attempt timeout) so IPv6-unreachable environments can quickly fall back to IPv4 for guarded fetch paths. (#19950) Thanks @ENAwareness. +- Security/Config: make parsed chat allowlist checks fail closed when `allowFrom` is empty, restoring expected DM/pairing gating. +- Security/Exec: in non-default setups that manually add `sort` to `tools.exec.safeBins`, block `sort --compress-program` so allowlist-mode safe-bin checks cannot bypass approval. Thanks @tdjackey for reporting. +- Security/Exec approvals: when users choose `allow-always` for shell-wrapper commands (for example `/bin/zsh -lc ...`), persist allowlist patterns for the inner executable(s) instead of the wrapper shell binary, preventing accidental broad shell allowlisting in moderate mode. (#23276) Thanks @xrom2863. +- Security/Exec: fail closed when `tools.exec.host=sandbox` is configured/requested but sandbox runtime is unavailable. (#23398) Thanks @bmendonca3. +- Security/macOS app beta: enforce path-only `system.run` allowlist matching (drop basename matches like `echo`), migrate legacy basename entries to last resolved paths when available, and harden shell-chain handling to fail closed on unsafe parse/control syntax (including quoted command substitution/backticks). This is an optional allowlist-mode feature; default installs remain deny-by-default. Thanks @tdjackey for reporting. +- Security/Agents: auto-generate and persist a dedicated `commands.ownerDisplaySecret` when `commands.ownerDisplay=hash`, remove gateway token fallback from owner-ID prompt hashing across CLI and embedded agent runners, and centralize owner-display secret resolution in one shared helper. Thanks @aether-ai-agent for reporting. +- Security/SSRF: expand IPv4 fetch guard blocking to include RFC special-use/non-global ranges (including benchmarking, TEST-NET, multicast, and reserved/broadcast blocks), centralize range checks into a single CIDR policy table, and reuse one shared host/IP classifier across literal + DNS checks to reduce classifier drift. Thanks @princeeismond-dot for reporting. +- Security/SSRF: block RFC2544 benchmarking range (`198.18.0.0/15`) across direct and embedded-IP paths, and normalize IPv6 dotted-quad transition literals (for example `::127.0.0.1`, `64:ff9b::8.8.8.8`) in shared IP parsing/classification. +- Security/Archive: block zip symlink escapes during archive extraction. +- Security/Media sandbox: keep tmp media allowance for absolute tmp paths only and enforce symlink-escape checks before sandbox-validated reads, preventing tmp symlink exfiltration and relative `../` sandbox escapes when sandboxes live under tmp. (#17892) Thanks @dashed. +- Browser/Upload: accept canonical in-root upload paths when the configured uploads directory is a symlink alias (for example `/tmp` -> `/private/tmp` on macOS), so browser upload validation no longer rejects valid files during client->server revalidation. (#23300, #23222, #22848) Thanks @bgaither4, @parkerati, and @Nabsku. +- Security/Discord: add `openclaw security audit` warnings for name/tag-based Discord allowlist entries (DM allowlists, guild/channel `users`, and pairing-store entries), highlighting slug-collision risk while keeping name-based matching supported, and canonicalize resolved Discord allowlist names to IDs at runtime without rewriting config files. Thanks @tdjackey for reporting. +- Security/Gateway: block node-role connections when device identity metadata is missing. +- Security/Media: enforce inbound media byte limits during download/read across Discord, Telegram, Zalo, Microsoft Teams, and BlueBubbles to prevent oversized payload memory spikes before rejection. Thanks @tdjackey for reporting. +- Media/Understanding: preserve `application/pdf` MIME classification during text-like file heuristics so PDF uploads use PDF extraction paths instead of being inlined as raw text. (#23191) Thanks @claudeplay2026-byte. +- Security/Control UI: block symlink-based out-of-root static file reads by enforcing realpath containment and file-identity checks when serving Control UI assets and SPA fallback `index.html`. Thanks @tdjackey for reporting. +- Security/Gateway avatars: block symlink traversal during local avatar `data:` URL resolution by enforcing realpath containment and file-identity checks before reads. Thanks @tdjackey for reporting. +- Security/Control UI: centralize avatar URL/path validation across gateway/config helpers and enforce a 2 MB max size for local agent avatar files before `/avatar` resolution, reducing oversized-avatar memory risk without changing supported avatar formats. +- Security/Control UI avatars: harden `/avatar/:agentId` local avatar serving by rejecting symlink paths and requiring fd-level file identity + size checks before reads. Thanks @tdjackey for reporting. +- Security/MSTeams media: enforce allowlist checks for SharePoint reference attachment URLs and redirect targets during Graph-backed media fetches so redirect chains cannot escape configured media host boundaries. Thanks @tdjackey for reporting. +- Security/MSTeams media: route attachment auth-retry and Graph SharePoint download redirects through shared `safeFetch` so each hop is validated with allowlist + DNS/IP checks across the full redirect chain. (#23598) Thanks @Asm3r96 and @lewiswigmore. +- Security/macOS discovery: fail closed for unresolved discovery endpoints by clearing stale remote selection values, use resolved service host only for SSH target derivation, and keep remote URL config aligned with resolved endpoint availability. (#21618) Thanks @bmendonca3. +- Chat/Usage/TUI: strip synthetic inbound metadata blocks (including `Conversation info` and trailing `Untrusted context` channel metadata wrappers) from displayed conversation history so internal prompt context no longer leaks into user-visible logs. +- CI/Tests: fix TypeScript case-table typing and lint assertion regressions so `pnpm check` passes again after Synology Chat landing. (#23012) Thanks @druide67. +- Security/Browser relay: harden extension relay auth token handling for `/extension` and `/cdp` pathways. +- Cron: persist `delivered` state in cron job records so delivery failures remain visible in status and logs. (#19174) Thanks @simonemacario. +- Config/Doctor: only repair the OAuth credentials directory when affected channels are configured, avoiding fresh-install noise. +- Config/Channels: whitelist `channels.modelByChannel` in config validation and exclude it from plugin auto-enable channel detection so model overrides no longer trigger `unknown channel id` validation errors or bogus `modelByChannel` plugin enables. (#23412) Thanks @ProspectOre. +- Config/Bindings: allow optional `bindings[].comment` in strict config validation so annotated binding entries no longer fail load. (#23458) Thanks @echoVic. +- Usage/Pricing: correct MiniMax M2.5 pricing defaults to fix inflated cost reporting. (#22755) Thanks @miloudbelarebia. +- Gateway/Daemon: verify gateway health after daemon restart. +- Agents/UI text: stop rewriting normal assistant billing/payment language outside explicit error contexts. (#17834) Thanks @niceysam. + +## 2026.2.21 + +### Changes + +- Models/Google: add Gemini 3.1 support (`google/gemini-3.1-pro-preview`). +- Providers/Onboarding: add Volcano Engine (Doubao) and BytePlus providers/models (including coding variants), wire onboarding auth choices for interactive + non-interactive flows, and align docs to `volcengine-api-key`. (#7967) Thanks @funmore123. +- Channels/CLI: add per-account/channel `defaultTo` outbound routing fallback so `openclaw agent --deliver` can send without explicit `--reply-to` when a default target is configured. (#16985) Thanks @KirillShchetinin. +- Channels: allow per-channel model overrides via `channels.modelByChannel` and note them in /status. Thanks @thewilloftheshadow. +- Telegram/Streaming: simplify preview streaming config to `channels.telegram.streaming` (boolean), auto-map legacy `streamMode` values, and remove block-vs-partial preview branching. (#22012) thanks @obviyus. +- Discord/Streaming: add stream preview mode for live draft replies with partial/block options and configurable chunking. Thanks @thewilloftheshadow. Inspiration @neoagentic-ship-it. +- Discord/Telegram: add configurable lifecycle status reactions for queued/thinking/tool/done/error phases with a shared controller and emoji/timing overrides. Thanks @wolly-tundracube and @thewilloftheshadow. +- Discord/Voice: add voice channel join/leave/status via `/vc`, plus auto-join configuration for realtime voice conversations. Thanks @thewilloftheshadow. +- Discord: add configurable ephemeral defaults for slash-command responses. (#16563) Thanks @wei. +- Discord: support updating forum `available_tags` via channel edit actions for forum tag management. (#12070) Thanks @xiaoyaner0201. +- Discord: include channel topics in trusted inbound metadata on new sessions. Thanks @thewilloftheshadow. +- Discord/Subagents: add thread-bound subagent sessions on Discord with per-thread focus/list controls and thread-bound continuation routing for spawned helper agents. (#21805) Thanks @onutc. +- iOS/Chat: clean chat UI noise by stripping inbound untrusted metadata/timestamp prefixes, formatting tool outputs into concise summaries/errors, compacting the composer while typing, and supporting tap-to-dismiss keyboard in chat view. (#22122) thanks @mbelinky. +- iOS/Watch: bridge mirrored watch prompt notification actions into iOS quick-reply handling, including queued action handoff until app model initialization. (#22123) thanks @mbelinky. +- iOS/Gateway: stabilize background wake and reconnect behavior with background reconnect suppression/lease windows, BGAppRefresh wake fallback, location wake hook throttling, and APNs wake retry+nudge instrumentation. (#21226) thanks @mbelinky. +- Auto-reply/UI: add model fallback lifecycle visibility in verbose logs, /status active-model context with fallback reason, and cohesive WebUI fallback indicators. (#20704) Thanks @joshavant. +- MSTeams: dedupe sent-message cache storage by removing duplicate per-message Set storage and using timestamps Map keys as the single membership source. (#22514) Thanks @TaKO8Ki. +- Agents/Subagents: default subagent spawn depth now uses shared `maxSpawnDepth=2`, enabling depth-1 orchestrator spawning by default while keeping depth policy checks consistent across spawn and prompt paths. (#22223) Thanks @tyler6204. +- Security/Agents: make owner-ID obfuscation use a dedicated HMAC secret from configuration (`ownerDisplaySecret`) and update hashing behavior so obfuscation is decoupled from gateway token handling for improved control. (#7343) Thanks @vincentkoc. +- Security/Infra: switch gateway lock and tool-call synthetic IDs from SHA-1 to SHA-256 with unchanged truncation length to strengthen hash basis while keeping deterministic behavior and lock key format. (#7343) Thanks @vincentkoc. +- Dependencies/Tooling: add non-blocking dead-code scans in CI via Knip/ts-prune/ts-unused-exports to surface unused dependencies and exports earlier. (#22468) Thanks @vincentkoc. +- Dependencies/Unused Dependencies: remove or scope unused root and extension deps (`@larksuiteoapi/node-sdk`, `signal-utils`, `ollama`, `lit`, `@lit/context`, `@lit-labs/signals`, `@microsoft/agents-hosting-express`, `@microsoft/agents-hosting-extensions-teams`, and plugin-local `openclaw` devDeps in `extensions/open-prose`, `extensions/lobster`, and `extensions/llm-task`). (#22471, #22495) Thanks @vincentkoc. +- Dependencies/A2UI: harden dependency resolution after root cleanup (resolve `lit`, `@lit/context`, `@lit-labs/signals`, and `signal-utils` from workspace/root) and simplify bundling fallback behavior, including `pnpm dlx rolldown` compatibility. (#22481, #22507) Thanks @vincentkoc. + +### Fixes + +- Agents/Bootstrap: skip malformed bootstrap files with missing/invalid paths instead of crashing agent sessions; hooks using `filePath` (or non-string `path`) are skipped with a warning. (#22693, #22698) Thanks @arosstale. +- Security/Agents: cap embedded Pi runner outer retry loop with a higher profile-aware dynamic limit (32-160 attempts) and return an explicit `retry_limit` error payload when retries never converge, preventing unbounded internal retry cycles (`GHSA-76m6-pj3w-v7mf`). +- Telegram: detect duplicate bot-token ownership across Telegram accounts at startup/status time, mark secondary accounts as not configured with an explicit fix message, and block duplicate account startup before polling to avoid endless `getUpdates` conflict loops. +- Agents/Tool images: include source filenames in `agents/tool-images` resize logs so compression events can be traced back to specific files. +- Providers/OAuth: harden Qwen and Chutes refresh handling by validating refresh response expiry values and preserving prior refresh tokens when providers return empty refresh token fields, with regression coverage for empty-token responses. +- Models/Kimi-Coding: add missing implicit provider template for `kimi-coding` with correct `anthropic-messages` API type and base URL, fixing 403 errors when using Kimi for Coding. (#22409) +- Auto-reply/Tools: forward `senderIsOwner` through embedded queued/followup runner params so owner-only tools remain available for authorized senders. (#22296) thanks @hcoj. +- Discord: restore model picker back navigation when a provider is missing and document the Discord picker flow. (#21458) Thanks @pejmanjohn and @thewilloftheshadow. +- Memory/QMD: respect per-agent `memorySearch.enabled=false` during gateway QMD startup initialization, split multi-collection QMD searches into per-collection queries (`search`/`vsearch`/`query`) to avoid sparse-term drops, prefer collection-hinted doc resolution to avoid stale-hash collisions, retry boot updates on transient lock/timeout failures, skip `qmd embed` in BM25-only `search` mode (including `memory index --force`), and serialize embed runs globally with failure backoff to prevent CPU storms on multi-agent hosts. (#20581, #21590, #20513, #20001, #21266, #21583, #20346, #19493) Thanks @danielrevivo, @zanderkrause, @sunyan034-cmd, @tilleulenspiegel, @dae-oss, @adamlongcreativellc, @jonathanadams96, and @kiliansitel. +- Memory/Builtin: prevent automatic sync races with manager shutdown by skipping post-close sync starts and waiting for in-flight sync before closing SQLite, so `onSearch`/`onSessionStart` no longer fail with `database is not open` in ephemeral CLI flows. (#20556, #7464) Thanks @FuzzyTG and @henrybottter. +- Providers/Copilot: drop persisted assistant `thinking` blocks for Claude models (while preserving turn structure/tool blocks) so follow-up requests no longer fail on invalid `thinkingSignature` payloads. (#19459) Thanks @jackheuberger. +- Providers/Copilot: add `claude-sonnet-4.6` and `claude-sonnet-4.5` to the default GitHub Copilot model catalog and add coverage for model-list/definition helpers. (#20270, fixes #20091) Thanks @Clawborn. +- Auto-reply/WebChat: avoid defaulting inbound runtime channel labels to unrelated providers (for example `whatsapp`) for webchat sessions so channel-specific formatting guidance stays accurate. (#21534) Thanks @lbo728. +- Status: include persisted `cacheRead`/`cacheWrite` in session summaries so compact `/status` output consistently shows cache hit percentages from real session data. +- Sessions/Usage: persist `totalTokens` from `promptTokens` snapshots even when providers omit structured usage payloads, so session history/status no longer regress to `unknown` token utilization for otherwise successful runs. (#21819) Thanks @zymclaw. +- Heartbeat/Cron: restore interval heartbeat behavior so missing `HEARTBEAT.md` no longer suppresses runs (only effectively empty files skip), preserving prompt-driven and tagged-cron execution paths. +- WhatsApp/Cron/Heartbeat: enforce allowlisted routing for implicit scheduled/system delivery by merging pairing-store + configured `allowFrom` recipients, selecting authorized recipients when last-route context points to a non-allowlisted chat, and preventing heartbeat fan-out to recent unauthorized chats. +- Heartbeat/Active hours: constrain active-hours `24` sentinel parsing to `24:00` in time validation so invalid values like `24:30` are rejected early. (#21410) thanks @adhitShet. +- Heartbeat: treat `activeHours` windows with identical `start`/`end` times as zero-width (always outside the window) instead of always-active. (#21408) thanks @adhitShet. +- CLI/Pairing: default `pairing list` and `pairing approve` to the sole available pairing channel when omitted, so TUI-only setups can recover from `pairing required` without guessing channel arguments. (#21527) Thanks @losts1. +- TUI/Pairing: show explicit pairing-required recovery guidance after gateway disconnects that return `pairing required`, including approval steps to unblock quickstart TUI hatching on fresh installs. (#21841) Thanks @nicolinux. +- TUI/Input: suppress duplicate backspace events arriving in the same input burst window so SSH sessions no longer delete two characters per backspace press in the composer. (#19318) Thanks @eheimer. +- TUI/Models: scope `models.list` to the configured model allowlist (`agents.defaults.models`) so `/model` picker no longer floods with unrelated catalog entries by default. (#18816) Thanks @fwends. +- TUI/Heartbeat: suppress heartbeat ACK/prompt noise in chat streaming when `showOk` is disabled, while still preserving non-ACK heartbeat alerts in final output. (#20228) Thanks @bhalliburton. +- TUI/History: cap chat-log component growth and prune stale render nodes/references so large default history loads no longer overflow render recursion with `RangeError: Maximum call stack size exceeded`. (#18068) Thanks @JaniJegoroff. +- Memory/QMD: diversify mixed-source search ranking when both session and memory collections are present so session transcript hits no longer crowd out durable memory-file matches in top results. (#19913) Thanks @alextempr. +- Memory/Tools: return explicit `unavailable` warnings/actions from `memory_search` when embedding/provider failures occur (including quota exhaustion), so disabled memory does not look like an empty recall result. (#21894) Thanks @XBS9. +- Session/Startup: require the `/new` and `/reset` greeting path to run Session Startup file-reading instructions before responding, so daily memory startup context is not skipped on fresh-session greetings. (#22338) Thanks @armstrong-pv. +- Auth/Onboarding: align OAuth profile-id config mapping with stored credential IDs for OpenAI Codex and Chutes flows, preventing `provider:default` mismatches when OAuth returns email-scoped credentials. (#12692) thanks @mudrii. +- Provider/HTTP: treat HTTP 503 as failover-eligible for LLM provider errors. (#21086) Thanks @Protocol-zero-0. +- Slack: pass `recipient_team_id` / `recipient_user_id` through Slack native streaming calls so `chat.startStream`/`appendStream`/`stopStream` work reliably across DMs and Slack Connect setups, and disable block streaming when native streaming is active. (#20988) Thanks @Dithilli. Earlier recipient-ID groundwork was contributed in #20377 by @AsserAl1012. +- CLI/Config: add canonical `--strict-json` parsing for `config set` and keep `--json` as a legacy alias to reduce help/behavior drift. (#21332) thanks @adhitShet. +- CLI/Config: preserve explicitly unset config paths in persisted JSON after writes so `openclaw config unset ` no longer re-introduces defaulted keys (for example `commands.ownerDisplay`) through schema normalization. (#22984) Thanks @aronchick. +- CLI: keep `openclaw -v` as a root-only version alias so subcommand `-v, --verbose` flags (for example ACP/hooks/skills) are no longer intercepted globally. (#21303) thanks @adhitShet. +- Memory: return empty snippets when `memory_get`/QMD read files that have not been created yet, and harden memory indexing/session helpers against ENOENT races so missing Markdown no longer crashes tools. (#20680) Thanks @pahdo. +- Telegram/Streaming: always clean up draft previews even when dispatch throws before fallback handling, preventing orphaned preview messages during failed runs. (#19041) thanks @mudrii. +- Telegram/Streaming: split reasoning and answer draft preview lanes to prevent cross-lane overwrites, and ignore literal `` tags inside inline/fenced code snippets so sample markup is not misrouted as reasoning. (#20774) Thanks @obviyus. +- Telegram/Streaming: restore 30-char first-preview debounce and scope `NO_REPLY` prefix suppression to partial sentinel fragments so normal `No...` text is not filtered. (#22613) thanks @obviyus. +- Telegram/Status reactions: refresh stall timers on repeated phase updates and honor ack-reaction scope when lifecycle reactions are enabled, preventing false stall emojis and unwanted group reactions. Thanks @wolly-tundracube and @thewilloftheshadow. +- Telegram/Status reactions: keep lifecycle reactions active when available-reactions lookup fails by falling back to unrestricted variant selection instead of suppressing reaction updates. (#22380) thanks @obviyus. +- Discord/Events: await `DiscordMessageListener` message handlers so regular `MESSAGE_CREATE` traffic is processed through queue ordering/timeout flow instead of fire-and-forget drops. (#22396) Thanks @sIlENtbuffER. +- Discord/Streaming: apply `replyToMode: first` only to the first Discord chunk so block-streamed replies do not spam mention pings. (#20726) Thanks @thewilloftheshadow for the report. +- Discord/Components: map DM channel targets back to user-scoped component sessions so button/select interactions stay in the main DM session. Thanks @thewilloftheshadow. +- Discord/Allowlist: lazy-load guild lists when resolving Discord user allowlists so ID-only entries resolve even if guild fetch fails. (#20208) Thanks @zhangjunmengyang. +- Discord/Gateway: handle close code 4014 (missing privileged gateway intents) without crashing the gateway. Thanks @thewilloftheshadow. +- Discord: ingest inbound stickers as media so sticker-only messages and forwarded stickers are visible to agents. Thanks @thewilloftheshadow. +- Auto-reply/Runner: emit `onAgentRunStart` only after agent lifecycle or tool activity begins (and only once per run), so fallback preflight errors no longer mark runs as started. (#21165) Thanks @shakkernerd. +- Auto-reply/Tool results: serialize tool-result delivery and keep the delivery chain progressing after individual failures so concurrent tool outputs preserve user-visible ordering. (#21231) thanks @ahdernasr. +- Auto-reply/Prompt caching: restore prefix-cache stability by keeping inbound system metadata session-stable and moving per-message IDs (`message_id`, `message_id_full`, `reply_to_id`, `sender_id`) into untrusted conversation context. (#20597) Thanks @anisoptera. +- iOS/Watch: add actionable watch approval/reject controls and quick-reply actions so watch-originated approvals and responses can be sent directly from notification flows. (#21996) Thanks @mbelinky. +- iOS/Watch: refresh iOS and watch app icon assets with the lobster icon set to keep phone/watch branding aligned. (#21997) Thanks @mbelinky. +- CLI/Onboarding: fix Anthropic-compatible custom provider verification by normalizing base URLs to avoid duplicate `/v1` paths during setup checks. (#21336) Thanks @17jmumford. +- iOS/Gateway/Tools: prefer uniquely connected node matches when duplicate display names exist, surface actionable `nodes invoke` pairing-required guidance with request IDs, and refresh active iOS gateway registration after location-capability setting changes so capability updates apply immediately. (#22120) thanks @mbelinky. +- Gateway/Auth: require `gateway.trustedProxies` to include a loopback proxy address when `auth.mode="trusted-proxy"` and `bind="loopback"`, preventing same-host proxy misconfiguration from silently blocking auth. (#22082, follow-up to #20097) thanks @mbelinky. +- Gateway/Auth: allow trusted-proxy mode with loopback bind for same-host reverse-proxy deployments, while still requiring configured `gateway.trustedProxies`. (#20097) thanks @xinhuagu. +- Gateway/Auth: allow authenticated clients across roles/scopes to call `health` while preserving role and scope enforcement for non-health methods. (#19699) thanks @Nachx639. +- Gateway/Hooks: include transform export name in hook-transform cache keys so distinct exports from the same module do not reuse the wrong cached transform function. (#13855) thanks @mcaxtr. +- Gateway/Control UI: return 404 for missing static-asset paths instead of serving SPA fallback HTML, while preserving client-route fallback behavior for extensionless and non-asset dotted paths. (#12060) thanks @mcaxtr. +- Gateway/Pairing: prevent device-token rotate scope escalation by enforcing an approved-scope baseline, preserving approved scopes across metadata updates, and rejecting rotate requests that exceed approved role scope implications. (#20703) thanks @coygeek. +- Gateway/Pairing: clear persisted paired-device state when the gateway client closes with `device token mismatch` (`1008`) so reconnect flows can cleanly re-enter pairing. (#22071) Thanks @mbelinky. +- Gateway/Config: allow `gateway.customBindHost` in strict config validation when `gateway.bind="custom"` so valid custom bind-host configurations no longer fail startup. (#20318, fixes #20289) Thanks @MisterGuy420. +- Gateway/Pairing: tolerate legacy paired devices missing `roles`/`scopes` metadata in websocket upgrade checks and backfill metadata on reconnect. (#21447, fixes #21236) Thanks @joshavant. +- Gateway/Pairing/CLI: align read-scope compatibility in pairing/device-token checks and add local `openclaw devices` fallback recovery for loopback `pairing required` deadlocks, with explicit fallback notice to unblock approval bootstrap flows. (#21616) Thanks @shakkernerd. +- Cron: honor `cron.maxConcurrentRuns` in the timer loop so due jobs can execute up to the configured parallelism instead of always running serially. (#11595) Thanks @Takhoffman. +- Agents/Compaction: restore embedded compaction safeguard/context-pruning extension loading in production by wiring bundled extension factories into the resource loader instead of runtime file-path resolution. (#22349) Thanks @Glucksberg. +- Agents/Subagents: restore announce-chain delivery to agent injection, defer nested announce output until descendant follow-up content is ready, and prevent descendant deferrals from consuming announce retry budget so deep chains do not drop final completions. (#22223) Thanks @tyler6204. +- Agents/System Prompt: label allowlisted senders as authorized senders to avoid implying ownership. Thanks @thewilloftheshadow. +- Agents/Tool display: fix exec cwd suffix inference so `pushd ... && popd ... && ` does not keep stale `(in )` context in summaries. (#21925) Thanks @Lukavyi. +- Agents/Google: flatten residual nested `anyOf`/`oneOf` unions in Gemini tool-schema cleanup so Cloud Code Assist no longer rejects unsupported union keywords that survive earlier simplification. (#22825) Thanks @Oceanswave. +- Tools/web_search: handle xAI Responses API payloads that emit top-level `output_text` blocks (without a `message` wrapper) so Grok web_search no longer returns `No response` for those results. (#20508) Thanks @echoVic. +- Agents/Failover: treat non-default override runs as direct fallback-to-configured-primary (skip configured fallback chain), normalize default-model detection for provider casing/whitespace, and add regression coverage for override/auth error paths. (#18820) Thanks @Glucksberg. +- Docker/Build: include `ownerDisplay` in `CommandsSchema` object-level defaults so Docker `pnpm build` no longer fails with `TS2769` during plugin SDK d.ts generation. (#22558) Thanks @obviyus. +- Docker/Browser: install Playwright Chromium into `/home/node/.cache/ms-playwright` and set `node:node` ownership so browser binaries are available to the runtime user in browser-enabled images. (#22585) thanks @obviyus. +- Hooks/Session memory: trigger bundled `session-memory` persistence on both `/new` and `/reset` so reset flows no longer skip markdown transcript capture before archival. (#21382) Thanks @mofesolapaul. +- Dependencies/Agents: bump embedded Pi SDK packages (`@mariozechner/pi-agent-core`, `@mariozechner/pi-ai`, `@mariozechner/pi-coding-agent`, `@mariozechner/pi-tui`) to `0.54.0`. (#21578) Thanks @Takhoffman. +- Config/Agents: expose Pi compaction tuning values `agents.defaults.compaction.reserveTokens` and `agents.defaults.compaction.keepRecentTokens` in config schema/types and apply them in embedded Pi runner settings overrides with floor enforcement via `reserveTokensFloor`. (#21568) Thanks @Takhoffman. +- Docker: pin base images to SHA256 digests in Docker builds to prevent mutable tag drift. (#7734) Thanks @coygeek. +- Docker: run build steps as the `node` user and use `COPY --chown` to avoid recursive ownership changes, trimming image size and layer churn. Thanks @huntharo. +- Config/Memory: restore schema help/label metadata for hybrid `mmr` and `temporalDecay` settings so configuration surfaces show correct names and guidance. (#18786) Thanks @rodrigouroz. +- Skills/SonosCLI: add troubleshooting guidance for `sonos discover` failures on macOS direct mode (`sendto: no route to host`) and sandbox network restrictions (`bind: operation not permitted`). (#21316) Thanks @huntharo. +- macOS/Build: default release packaging to `BUNDLE_ID=ai.openclaw.mac` in `scripts/package-mac-dist.sh`, so Sparkle feed URL is retained and auto-update no longer fails with an empty appcast feed. (#19750) thanks @loganprit. +- Signal/Outbound: preserve case for Base64 group IDs during outbound target normalization so cross-context routing and policy checks no longer break when group IDs include uppercase characters. (#5578) Thanks @heyhudson. +- Anthropic/Agents: preserve required pi-ai default OAuth beta headers when `context1m` injects `anthropic-beta`, preventing 401 auth failures for `sk-ant-oat-*` tokens. (#19789, fixes #19769) Thanks @minupla. +- Security/Exec: block unquoted heredoc body expansion tokens in shell allowlist analysis, reject unterminated heredocs, and require explicit approval for allowlisted heredoc execution on gateway hosts to prevent heredoc substitution allowlist bypass. Thanks @torturado for reporting. +- macOS/Security: evaluate `system.run` allowlists per shell segment in macOS node runtime and companion exec host (including chained shell operators), fail closed on shell/process substitution parsing, and require explicit approval on unsafe parse cases to prevent allowlist bypass via `rawCommand` chaining. Thanks @tdjackey for reporting. +- WhatsApp/Security: enforce allowlist JID authorization for reaction actions so authenticated callers cannot target non-allowlisted chats by forging `chatJid` + valid `messageId` pairs. Thanks @aether-ai-agent for reporting. +- ACP/Security: escape control and delimiter characters in ACP `resource_link` title/URI metadata before prompt interpolation to prevent metadata-driven prompt injection through resource links. Thanks @aether-ai-agent for reporting. +- TTS/Security: make model-driven provider switching opt-in by default (`messages.tts.modelOverrides.allowProvider=false` unless explicitly enabled), while keeping voice/style overrides available, to reduce prompt-injection-driven provider hops and unexpected TTS cost escalation. Thanks @aether-ai-agent for reporting. +- Security/Agents: keep overflow compaction retry budgeting global across tool-result truncation recovery so successful truncation cannot reset the overflow retry counter and amplify retry/cost cycles. Thanks @aether-ai-agent for reporting. +- BlueBubbles/Security: require webhook token authentication for all BlueBubbles webhook requests (including loopback/proxied setups), removing passwordless webhook fallback behavior. Thanks @zpbrent. +- iOS/Security: force `https://` for non-loopback manual gateway hosts during iOS onboarding to block insecure remote transport URLs. (#21969) Thanks @mbelinky. +- Gateway/Security: remove shared-IP fallback for canvas endpoints and require token or session capability for canvas access. Thanks @thewilloftheshadow. +- Gateway/Security: require secure context and paired-device checks for Control UI auth even when `gateway.controlUi.allowInsecureAuth` is set, and align audit messaging with the hardened behavior. (#20684) Thanks @coygeek and @Vasco0x4 for reporting. +- Gateway/Security: scope tokenless Tailscale forwarded-header auth to Control UI websocket auth only, so HTTP gateway routes still require token/password even on trusted hosts. Thanks @zpbrent for reporting. +- Docker/Security: run E2E and install-sh test images as non-root by adding appuser directives. Thanks @thewilloftheshadow. +- Skills/Security: sanitize skill env overrides to block unsafe runtime injection variables and only allow sensitive keys when declared in skill metadata, with warnings for suspicious values. Thanks @thewilloftheshadow. +- Security/Commands: block prototype-key injection in runtime `/debug` overrides and require own-property checks for gated command flags (`bash`, `config`, `debug`) so inherited prototype values cannot enable privileged commands. Thanks @tdjackey for reporting. +- Security/Browser: block non-network browser navigation protocols (including `file:`, `data:`, and `javascript:`) while preserving `about:blank`, preventing local file reads via browser tool navigation. Thanks @q1uf3ng for reporting. +- Security/Exec: block shell startup-file env injection (`BASH_ENV`, `ENV`, `BASH_FUNC_*`, `LD_*`, `DYLD_*`) across config env ingestion, node-host inherited environment sanitization, and macOS exec host runtime to prevent pre-command execution from attacker-controlled environment variables. Thanks @tdjackey. +- Security/Exec (Windows): canonicalize `cmd.exe /c` command text across validation, approval binding, and audit/event rendering to prevent trailing-argument approval mismatches in `system.run`. Thanks @tdjackey for reporting. +- Security/Gateway/Hooks: block `__proto__`, `constructor`, and `prototype` traversal in webhook template path resolution to prevent prototype-chain payload data leakage in `messageTemplate` rendering. (#22213) Thanks @SleuthCo. +- Security/OpenClawKit/UI: prevent injected inbound user context metadata blocks from leaking into chat history in TUI, webchat, and macOS surfaces by stripping all untrusted metadata prefixes at display boundaries. (#22142) Thanks @Mellowambience, @vincentkoc. +- Security/OpenClawKit/UI: strip inbound metadata blocks from user messages in TUI rendering while preserving user-authored content. (#22345) Thanks @kansodata, @vincentkoc. +- Security/OpenClawKit/UI: prevent inbound metadata leaks and reply-tag streaming artifacts in TUI rendering by stripping untrusted metadata prefixes at display boundaries. (#22346) Thanks @akramcodez, @vincentkoc. +- Security/Agents: restrict local MEDIA tool attachments to core tools and the OpenClaw temp root to prevent untrusted MCP tool file exfiltration. Thanks @NucleiAv and @thewilloftheshadow. +- Security/Net: strip sensitive headers (`Authorization`, `Proxy-Authorization`, `Cookie`, `Cookie2`) on cross-origin redirects in `fetchWithSsrFGuard` to prevent credential forwarding across origin boundaries. (#20313) Thanks @afurm. +- Security/Systemd: reject CR/LF in systemd unit environment values and fix argument escaping so generated units cannot be injected with extra directives. Thanks @thewilloftheshadow. +- Security/Tools: add per-wrapper random IDs to untrusted-content markers from `wrapExternalContent`/`wrapWebContent`, preventing marker spoofing from escaping content boundaries. (#19009) Thanks @Whoaa512. +- Shared/Security: reject insecure deep links that use `ws://` non-loopback gateway URLs to prevent plaintext remote websocket configuration. (#21970) Thanks @mbelinky. +- macOS/Security: reject non-loopback `ws://` remote gateway URLs in macOS remote config to block insecure plaintext websocket endpoints. (#21971) Thanks @mbelinky. +- Browser/Security: block upload path symlink escapes so browser upload sources cannot traverse outside the allowed workspace via symlinked paths. (#21972) Thanks @mbelinky. +- Security/Dependencies: bump transitive `hono` usage to `4.11.10` to incorporate timing-safe authentication comparison hardening for `basicAuth`/`bearerAuth` (`GHSA-gq3j-xvxp-8hrf`). Thanks @vincentkoc. +- Security/Gateway: parse `X-Forwarded-For` with trust-preserving semantics when requests come from configured trusted proxies, preventing proxy-chain spoofing from influencing client IP classification and rate-limit identity. Thanks @AnthonyDiSanti and @vincentkoc. +- Security/Sandbox: remove default `--no-sandbox` for the browser container entrypoint, add explicit opt-in via `OPENCLAW_BROWSER_NO_SANDBOX` / `CLAWDBOT_BROWSER_NO_SANDBOX`, and add security-audit checks for stale/missing sandbox browser Docker hash labels. Thanks @TerminalsandCoffee and @vincentkoc. +- Security/Sandbox Browser: require VNC password auth for noVNC observer sessions in the sandbox browser entrypoint, plumb per-container noVNC passwords from runtime, and emit short-lived noVNC observer token URLs while keeping loopback-only host port publishing. Thanks @TerminalsandCoffee for reporting. +- Security/Sandbox Browser: default browser sandbox containers to a dedicated Docker network (`openclaw-sandbox-browser`), add optional CDP ingress source-range restrictions, auto-create missing dedicated networks, and warn in `openclaw security --audit` when browser sandboxing runs on bridge without source-range limits. Thanks @TerminalsandCoffee for reporting. + +## 2026.2.19 + +### Changes + +- iOS/Watch: add an Apple Watch companion MVP with watch inbox UI, watch notification relay handling, and gateway command surfaces for watch status/send flows. (#20054) Thanks @mbelinky. +- iOS/Gateway: wake disconnected iOS nodes via APNs before `nodes.invoke` and auto-reconnect gateway sessions on silent push wake to reduce invoke failures while the app is backgrounded. (#20332) Thanks @mbelinky. +- Gateway/CLI: add paired-device hygiene flows with `device.pair.remove`, plus `openclaw devices remove` and guarded `openclaw devices clear --yes [--pending]` commands for removing paired entries and optionally rejecting pending requests. (#20057) Thanks @mbelinky. +- iOS/APNs: add push registration and notification-signing configuration for node delivery. (#20308) Thanks @mbelinky. +- Gateway/APNs: add a push-test pipeline for APNs delivery validation in gateway flows. (#20307) Thanks @mbelinky. +- Security/Audit: add `gateway.http.no_auth` findings when `gateway.auth.mode="none"` leaves Gateway HTTP APIs reachable, with loopback warning and remote-exposure critical severity, plus regression coverage and docs updates. +- Skills: harden coding-agent skill guidance by removing shell-command examples that interpolate untrusted issue text directly into command strings. +- Dev tooling: align `oxfmt` local/CI formatting behavior. (#12579) Thanks @vincentkoc. + +### Fixes + +- Security: strip hidden text from `web_fetch` extracted content to prevent indirect prompt injection, covering CSS-hidden elements, class-based hiding (sr-only, d-none, etc.), invisible Unicode, color:transparent, offscreen transforms, and non-content tags. (#8027, #21074) Thanks @hydro13 for the fix and @LucasAIBuilder for reporting. +- Agents/Streaming: keep assistant partial streaming active during reasoning streams, handle native `thinking_*` stream events consistently, dedupe mixed reasoning-end signals, and clear stale mutating tool errors after same-target retry success. (#20635) Thanks @obviyus. +- iOS/Chat: use a dedicated iOS chat session key for ChatSheet routing to avoid cross-client session collisions with main-session traffic. (#21139) thanks @mbelinky. +- iOS/Chat: auto-resync chat history after reconnect sequence gaps, clear stale pending runs, and avoid dead-end manual refresh errors after transient disconnects. (#21135) thanks @mbelinky. +- UI/Usage: reload usage data immediately when timezone changes so Local/UTC toggles apply the selected date range without requiring a manual refresh. (#17774) +- iOS/Screen: move `WKWebView` lifecycle ownership into `ScreenWebView` coordinator and explicit attach/detach flow to reduce gesture/lifecycle crash risk (`__NSArrayM insertObject:atIndex:` paths) during screen tab updates. (#20366) Thanks @ngutman. +- iOS/Onboarding: prevent pairing-status flicker during auto-resume by keeping resumed state transitions stable. (#20310) Thanks @mbelinky. +- iOS/Onboarding: stabilize pairing and reconnect behavior by resetting stale pairing request state on manual retry, disconnecting both operator and node gateways on operator failure, and avoiding duplicate pairing loops from operator transport identity attachment. (#20056) Thanks @mbelinky. +- iOS/Signing: restore local auto-selected signing-team overrides during iOS project generation by wiring `.local-signing.xcconfig` into the active signing config and emitting `OPENCLAW_DEVELOPMENT_TEAM` in local signing setup. (#19993) Thanks @ngutman. - Telegram: unify message-like inbound handling so `message` and `channel_post` share the same dedupe/access/media pipeline and remain behaviorally consistent. (#20591) Thanks @obviyus. +- Telegram: keep media-group processing resilient by skipping recoverable per-item download failures while still failing loud on non-recoverable media errors. (#20598) thanks @mcaxtr. - Telegram/Agents: gate exec/bash tool-failure warnings behind verbose mode so default Telegram replies stay clean while verbose sessions still surface diagnostics. (#20560) Thanks @obviyus. +- Telegram/Cron/Heartbeat: honor explicit Telegram topic targets in cron and heartbeat delivery (`:topic:`) so scheduled sends land in the configured topic instead of the last active thread. (#19367) Thanks @Lukavyi. +- Telegram/DM routing: prevent DM inbound origin metadata from leaking into main-session `lastRoute` updates and normalize DM `lastRoute.to` to provider-prefixed `telegram:`. (#19491) thanks @guirguispierre. - Gateway/Daemon: forward `TMPDIR` into installed service environments so macOS LaunchAgent gateway runs can open SQLite temp/journal files reliably instead of failing with `SQLITE_CANTOPEN`. (#20512) Thanks @Clawborn. - Agents/Billing: include the active model that produced a billing error in user-facing billing messages (for example, `OpenAI (gpt-5.3)`) across payload, failover, and lifecycle error paths, so users can identify exactly which key needs credits. (#20510) Thanks @echoVic. -- iOS/Screen: move `WKWebView` lifecycle ownership into `ScreenWebView` coordinator and explicit attach/detach flow to reduce gesture/lifecycle crash risk (`__NSArrayM insertObject:atIndex:` paths) during screen tab updates. (#20366) Thanks @ngutman. - Gateway/TUI: honor `agents.defaults.blockStreamingDefault` for `chat.send` by removing the hardcoded block-streaming disable override, so replies can use configured block-mode delivery. (#19693) Thanks @neipor. -- Protocol/Apple: regenerate Swift gateway models for `push.test` so `pnpm protocol:check` stays green on main. Thanks @mbelinky. -- Canvas/A2UI: improve bundled-asset resolution and empty-state handling so UI fallbacks render reliably. (#20312) Thanks @mbelinky. - UI/Sessions: accept the canonical main session-key alias in Chat UI flows so main-session routing stays consistent. (#20311) Thanks @mbelinky. -- iOS/Onboarding: prevent pairing-status flicker during auto-resume by keeping resumed state transitions stable. (#20310) Thanks @mbelinky. - OpenClawKit/Protocol: preserve JSON boolean literals (`true`/`false`) when bridging through `AnyCodable` so Apple client RPC params no longer re-encode booleans as `1`/`0`. Thanks @mbelinky. -- iOS/Onboarding: stabilize pairing and reconnect behavior by resetting stale pairing request state on manual retry, disconnecting both operator and node gateways on operator failure, and avoiding duplicate pairing loops from operator transport identity attachment. (#20056) Thanks @mbelinky. -- Browser/Relay: reuse an already-running extension relay when the relay port is occupied by another OpenClaw process, while still failing on non-relay port collisions to avoid masking unrelated listeners. (#20035) Thanks @mbelinky. -- Telegram/Cron/Heartbeat: honor explicit Telegram topic targets in cron and heartbeat delivery (`:topic:`) so scheduled sends land in the configured topic instead of the last active thread. (#19367) Thanks @Lukavyi. -- iOS/Signing: restore local auto-selected signing-team overrides during iOS project generation by wiring `.local-signing.xcconfig` into the active signing config and emitting `OPENCLAW_DEVELOPMENT_TEAM` in local signing setup. (#19993) Thanks @ngutman. +- Commands/Doctor: skip embedding-provider warnings when `memory.backend` is `qmd`, because QMD manages embeddings internally and does not require `memorySearch` providers. (#17263) Thanks @miloudbelarebia. +- Canvas/A2UI: improve bundled-asset resolution and empty-state handling so UI fallbacks render reliably. (#20312) Thanks @mbelinky. - Commands/Doctor: avoid rewriting invalid configs with new `gateway.auth.token` defaults during repair and only write when real config changes are detected, preventing accidental token duplication and backup churn. +- Gateway/Auth: default unresolved gateway auth to token mode with startup auto-generation/persistence of `gateway.auth.token`, while allowing explicit `gateway.auth.mode: "none"` for intentional open loopback setups. (#20686) thanks @gumadeiras. +- Channels/Matrix: fix mention detection for `formatted_body` Matrix-to links by handling matrix.to mention formats consistently. (#16941) Thanks @zerone0x. +- Heartbeat/Cron: skip interval heartbeats when `HEARTBEAT.md` is missing or empty and no tagged cron events are queued, while preserving cron-event fallback for queued tagged reminders. (#20461) thanks @vikpos. +- Browser/Relay: reuse an already-running extension relay when the relay port is occupied by another OpenClaw process, while still failing on non-relay port collisions to avoid masking unrelated listeners. (#20035) Thanks @mbelinky. +- Scripts: update clawdock helper command support to include `docker-compose.extra.yml` where available. (#17094) Thanks @zerone0x. +- Lobster/Config: remove Lobster executable-path overrides (`lobsterPath`), require PATH-based execution, and add focused Windows wrapper-resolution tests to keep shell-free behavior stable. +- Gateway/WebChat: block `sessions.patch` and `sessions.delete` for WebChat clients so session-store mutations stay restricted to non-WebChat operator flows. Thanks @allsmog for reporting. +- Gateway: clarify launchctl GUI domain bootstrap failure on macOS. (#13795) Thanks @vincentkoc. +- Lobster/CI: fix flaky test Windows cmd shim script resolution. (#20833) Thanks @vincentkoc. +- Browser/Relay: require gateway-token auth on both `/extension` and `/cdp`, and align Chrome extension setup to use a single `gateway.auth.token` input for relay authentication. Thanks @tdjackey for reporting. +- Gateway/Hooks: run BOOT.md startup checks per configured agent scope, including per-agent session-key resolution, startup-hook regression coverage, and non-success boot outcome logging for diagnosability. (#20569) thanks @mcaxtr. +- Protocol/Apple: regenerate Swift gateway models for `push.test` so `pnpm protocol:check` stays green on main. Thanks @mbelinky. - Sandbox/Registry: serialize container and browser registry writes with shared file locks and atomic replacement to prevent lost updates and delete rollback races from desyncing `sandbox list`, `prune`, and `recreate --all`. Thanks @kexinoh. -- Security/Exec: require `tools.exec.safeBins` binaries to resolve from trusted bin directories (system defaults plus gateway startup `PATH`) so PATH-hijacked trojan binaries cannot bypass allowlist checks. Thanks @jackhax for reporting. +- OTEL/diagnostics-otel: complete OpenTelemetry v2 API migration. (#12897) Thanks @vincentkoc. - Cron/Webhooks: protect cron webhook POST delivery with SSRF-guarded outbound fetch (`fetchWithSsrFGuard`) to block private/metadata destinations before request dispatch. Thanks @Adam55A-code. +- Security/Voice Call: harden `voice-call` telephony TTS override merging by blocking unsafe deep-merge keys (`__proto__`, `prototype`, `constructor`) and add regression coverage for top-level and nested prototype-pollution payloads. +- Security/Windows Daemon: harden Scheduled Task `gateway.cmd` generation by quoting cmd metacharacter arguments, escaping `%`/`!` expansions, and rejecting CR/LF in arguments, descriptions, and environment assignments (`set "KEY=VALUE"`), preventing command injection in Windows daemon startup scripts. Thanks @tdjackey for reporting. +- Security/Gateway/Canvas: replace shared-IP fallback auth with node-scoped session capability URLs for `/__openclaw__/canvas/*` and `/__openclaw__/a2ui/*`, fail closed when trusted-proxy requests omit forwarded client headers, and add IPv6/proxy-header regression coverage. Thanks @aether-ai-agent for reporting. +- Security/Net: enforce strict dotted-decimal IPv4 literals in SSRF checks and fail closed on unsupported legacy forms (octal/hex/short/packed, for example `0177.0.0.1`, `127.1`, `2130706433`) before DNS lookup. +- Security/Discord: enforce trusted-sender guild permission checks for moderation actions (`timeout`, `kick`, `ban`) and ignore untrusted `senderUserId` params to prevent privilege escalation in tool-driven flows. Thanks @aether-ai-agent for reporting. +- Security/ACP+Exec: add `openclaw acp --token-file/--password-file` secret-file support (with inline secret flag warnings), redact ACP working-directory prefixes to `~` home-relative paths, constrain exec script preflight file inspection to the effective `workdir` boundary, and add security-audit warnings when `tools.exec.host="sandbox"` is configured while sandbox mode is off. +- Security/Plugins/Hooks: enforce runtime/package path containment with realpath checks so `openclaw.extensions`, `openclaw.hooks`, and hook handler modules cannot escape their trusted roots via traversal or symlinks. +- Security/Discord: centralize trusted sender checks for moderation actions in message-action dispatch, share moderation command parsing across handlers, and clarify permission helpers with explicit any/all semantics. +- Security/ACP: harden ACP bridge session management with duplicate-session refresh, idle-session reaping, oldest-idle soft-cap eviction, and burst rate limiting on session creation to reduce local DoS risk without disrupting normal IDE usage. +- Security/ACP: bound ACP prompt text payloads to 2 MiB before gateway forwarding, account for join separator bytes during pre-concatenation size checks, and avoid stale active-run session state when oversized prompts are rejected. Thanks @aether-ai-agent for reporting. +- Security/Plugins/Hooks: add optional `--pin` for npm plugin/hook installs, persist resolved npm metadata (`name`, `version`, `spec`, integrity, shasum, timestamp), warn/confirm on integrity drift during updates, and extend `openclaw security audit` to flag unpinned specs, missing integrity metadata, and install-record version drift. +- Security/Plugins: harden plugin discovery by blocking unsafe candidates (root escapes, world-writable paths, suspicious ownership), add startup warnings when `plugins.allow` is empty with discoverable non-bundled plugins, and warn on loaded plugins without install/load-path provenance. +- Security/Gateway: rate-limit control-plane write RPCs (`config.apply`, `config.patch`, `update.run`) to 3 requests per minute per `deviceId+clientIp`, add restart single-flight coalescing plus a 30-second restart cooldown, and log actor/device/ip with changed-path audit details for config/update-triggered restarts. +- Security/Webhooks: harden Feishu and Zalo webhook ingress with webhook-mode token preconditions, loopback-default Feishu bind host, JSON content-type enforcement, per-path rate limiting, replay dedupe for Zalo events, constant-time Zalo secret comparison, and anomaly status counters. +- Security/Plugins: for the next npm release, clarify plugin trust boundary and keep `runtime.system.runCommandWithTimeout` available by default for trusted in-process plugins. Thanks @markmusson for reporting. +- Security/Skills: for the next npm release, reject symlinks during skill packaging to prevent external file inclusion in distributed `.skill` archives. Thanks @aether-ai-agent for reporting. +- Security/Gateway: fail startup when `hooks.token` matches `gateway.auth.token` so hooks and gateway token reuse is rejected at boot. (#20813) Thanks @coygeek. +- Security/Network: block plaintext `ws://` connections to non-loopback hosts and require secure websocket transport elsewhere. (#20803) Thanks @jscaldwell55. +- Security/Config: parse frontmatter YAML using the YAML 1.2 core schema to avoid implicit coercion of `on`/`off`-style values. (#20857) Thanks @davidrudduck. +- Security/Discord: escape backticks in exec-approval embed content to prevent markdown formatting injection via command text. (#20854) Thanks @davidrudduck. +- Security/Agents: replace shell-based `execSync` usage with `execFileSync` in command lookup helpers to eliminate shell argument interpolation risk. (#20655) Thanks @mahanandhi. +- Security/Media: use `crypto.randomBytes()` for temp file names and set owner-only permissions for TTS temp files. (#20654) Thanks @mahanandhi. +- Security/Gateway: set baseline security headers (`X-Content-Type-Options: nosniff`, `Referrer-Policy: no-referrer`) on gateway HTTP responses. (#10526) Thanks @abdelsfane. +- Security/iMessage: harden remote attachment SSH/SCP handling by requiring strict host-key verification, validating `channels.imessage.remoteHost` as `host`/`user@host`, and rejecting unsafe host tokens from config or auto-detection. Thanks @allsmog for reporting. +- Security/Feishu: prevent path traversal in Feishu inbound media temp-file writes by replacing key-derived temp filenames with UUID-based names. Thanks @allsmog for reporting. +- Security/Feishu: escape mention regex metacharacters in `stripBotMention` so crafted mention metadata cannot trigger regex injection or ReDoS during inbound message parsing. (#20916) Thanks @orlyjamie for the fix and @allsmog for reporting. +- LINE/Security: harden inbound media temp-file naming by using UUID-based temp paths for downloaded media instead of external message IDs. (#20792) Thanks @mbelinky. +- Security/Media: harden local media ingestion against TOCTOU/symlink swap attacks by pinning reads to a single file descriptor with symlink rejection and inode/device verification in `saveMediaSource`. Thanks @dorjoos for reporting. +- Security/Lobster (Windows): for the next npm release, remove shell-based fallback when launching Lobster wrappers (`.cmd`/`.bat`) and switch to explicit argv execution with wrapper entrypoint resolution, preventing command injection while preserving Windows wrapper compatibility. Thanks @allsmog for reporting. +- Security/Exec: require `tools.exec.safeBins` binaries to resolve from trusted bin directories (system defaults plus gateway startup `PATH`) so PATH-hijacked trojan binaries cannot bypass allowlist checks. Thanks @jackhax for reporting. +- Security/Exec: remove file-existence oracle behavior from `tools.exec.safeBins` by using deterministic argv-only stdin-safe validation and blocking file-oriented flags (for example `sort -o`, `jq -f`, `grep -f`) so allow/deny results no longer disclose host file presence. Thanks @nedlir for reporting. +- Security/Browser: route browser URL navigation through one SSRF-guarded validation path for tab-open/CDP-target/Playwright navigation flows and block private/metadata destinations by default (configurable via `browser.ssrfPolicy`). Thanks @dorjoos for reporting. +- Security/Exec: for the next npm release, harden safe-bin stdin-only enforcement by blocking output/recursive flags (`sort -o/--output`, grep recursion) and tightening default safe bins to remove `sort`/`grep`, preventing safe-bin allowlist bypass for file writes/recursive reads. Thanks @nedlir for reporting. +- Security/Exec: block grep safe-bin positional operand bypass by setting grep positional budget to zero, so `-e/--regexp` cannot smuggle bare filename reads (for example `.env`) via ambiguous positionals; safe-bin grep patterns must come from `-e/--regexp`. Thanks @athuljayaram for reporting. +- Security/Gateway/Agents: remove implicit admin scopes from agent tool gateway calls by classifying methods to least-privilege operator scopes, and enforce owner-only tooling (`cron`, `gateway`, `whatsapp_login`) through centralized tool-policy wrappers plus tool metadata to prevent non-owner DM privilege escalation. Ships in the next npm release. Thanks @Adam55A-code for reporting. +- Security/Gateway: centralize gateway method-scope authorization and default non-CLI gateway callers to least-privilege method scopes, with explicit CLI scope handling, full core-handler scope classification coverage, and regression guards to prevent scope drift. - Security/Net: block SSRF bypass via NAT64 (`64:ff9b::/96`, `64:ff9b:1::/48`), 6to4 (`2002::/16`), and Teredo (`2001:0000::/32`) IPv6 transition addresses, and fail closed on IPv6 parse errors. Thanks @jackhax. +- Security/OTEL: sanitize OTLP endpoint URL resolution. (#13791) Thanks @vincentkoc. +- Security: patch Dependabot security issues in pnpm lock. (#20832) Thanks @vincentkoc. +- Security: migrate request dependencies to `@cypress/request`. (#20836) Thanks @vincentkoc. ## 2026.2.17 @@ -81,8 +974,11 @@ Docs: https://docs.openclaw.ai ### Fixes +- Agents/Antigravity: preserve unsigned Claude thinking blocks as plain text instead of dropping them during transcript sanitization, preventing reasoning context loss while avoiding `thinking.signature` request rejections. +- Agents/Google: clean tool JSON Schemas for `google-antigravity` the same as `google-gemini-cli` before Cloud Code Assist requests, preventing Claude tool calls from failing with `patternProperties` 400 errors. (#19860) - Tests/Telegram: add regression coverage for command-menu sync that asserts all `setMyCommands` entries are Telegram-safe and hyphen-normalized across native/custom/plugin command sources. (#19703) Thanks @obviyus. - Agents/Image: collapse resize diagnostics to one line per image and include visible pixel/byte size details in the log message for faster triage. +- Auth/Cooldowns: clear all usage stats fields (`disabledUntil`, `disabledReason`, `failureCounts`) in `clearAuthProfileCooldown` so manual cooldown resets fully recover billing-disabled profiles without requiring direct file edits. (#19211) Thanks @nabbilkhan. - Agents/Subagents: preemptively guard accumulated tool-result context before model calls by truncating oversized outputs and compacting oldest tool-result messages to avoid context-window overflow crashes. Thanks @tyler6204. - Agents/Subagents/CLI: fail `sessions_spawn` when subagent model patching is rejected, allow subagent model patch defaults from `subagents.model`, and keep `sessions list`/`status` model reporting aligned to runtime model resolution. (#18660) Thanks @robbyczgw-cla. - Agents/Subagents: add explicit subagent guidance to recover from `[compacted: tool output removed to free context]` / `[truncated: output exceeded context limit]` markers by re-reading with smaller chunks instead of full-file `cat`. Thanks @tyler6204. @@ -108,6 +1004,7 @@ Docs: https://docs.openclaw.ai - BlueBubbles: include sender identifier in untrusted conversation metadata for conversation info payloads. Thanks @tyler6204. - Security/Exec: fix the OC-09 credential-theft path via environment-variable injection. (#18048) Thanks @aether-ai-agent. - Security/Config: confine `$include` resolution to the top-level config directory, harden traversal/symlink checks with cross-platform-safe path containment, and add doctor hints for invalid escaped include paths. (#18652) Thanks @aether-ai-agent. +- Security/Net: block SSRF bypass via ISATAP embedded IPv4 transition addresses and centralize hostname/IP blocking checks across URL safety validators. Thanks @zpbrent for reporting. - Providers: improve error messaging for unconfigured local `ollama`/`vllm` providers. (#18183) Thanks @arosstale. - TTS: surface all provider errors instead of only the last error in aggregated failures. (#17964) Thanks @ikari-pl. - CLI/Doctor/Configure: skip gateway auth checks for loopback-only setups. (#18407) Thanks @sggolakiya. @@ -291,6 +1188,8 @@ Docs: https://docs.openclaw.ai ### Fixes +- CLI/Installation: fix Docker installation hangs on macOS. (#12972) Thanks @vincentkoc. +- Models: fix antigravity opus 4.6 availability follow-up. (#12845) Thanks @vincentkoc. - Security/Sessions/Telegram: restrict session tool targeting by default to the current session tree (`tools.sessions.visibility`, default `tree`) with sandbox clamping, and pass configured per-account Telegram webhook secrets in webhook mode when no explicit override is provided. Thanks @aether-ai-agent. - CLI/Plugins: ensure `openclaw message send` exits after successful delivery across plugin-backed channels so one-shot sends do not hang. (#16491) Thanks @yinghaosang. - CLI/Plugins: run registered plugin `gateway_stop` hooks before `openclaw message` exits (success and failure paths), so plugin-backed channels can clean up one-shot CLI resources. (#16580) Thanks @gumadeiras. @@ -312,7 +1211,6 @@ Docs: https://docs.openclaw.ai - TUI/Gateway: resolve local gateway target URL from `gateway.bind` mode (tailnet/lan) instead of hardcoded localhost so `openclaw tui` connects when gateway is non-loopback. (#16299) Thanks @cortexuvula. - TUI: honor explicit `--session ` in `openclaw tui` even when `session.scope` is `global`, so named sessions no longer collapse into shared global history. (#16575) Thanks @cinqu. - TUI: use available terminal width for session name display in searchable select lists. (#16238) Thanks @robbyczgw-cla. -- TUI: refactor searchable select list description layout and add regression coverage for ANSI-highlight width bounds. - TUI: preserve in-flight streaming replies when a different run finalizes concurrently (avoid clearing active run or reloading history mid-stream). (#10704) Thanks @axschr73. - TUI: keep pre-tool streamed text visible when later tool-boundary deltas temporarily omit earlier text blocks. (#6958) Thanks @KrisKind75. - TUI: sanitize ANSI/control-heavy history text, redact binary-like lines, and split pathological long unbroken tokens before rendering to prevent startup crashes on binary attachment history. (#13007) Thanks @wilkinspoe. @@ -404,7 +1302,6 @@ Docs: https://docs.openclaw.ai - Security/Windows: avoid shell invocation when spawning child processes to prevent cmd.exe metacharacter injection via untrusted CLI arguments (e.g. agent prompt text). - Telegram: set webhook callback timeout handling to `onTimeout: "return"` (10s) so long-running update processing no longer emits webhook 500s and retry storms. (#16763) Thanks @chansearrington. - Signal: preserve case-sensitive `group:` target IDs during normalization so mixed-case group IDs no longer fail with `Group not found`. (#16748) Thanks @repfigit. -- Feishu/Security: harden media URL fetching against SSRF and local file disclosure. (#16285) Thanks @mbelinky. - Security/Agents: scope CLI process cleanup to owned child PIDs to avoid killing unrelated processes on shared hosts. Thanks @aether-ai-agent. - Security/Agents: enforce workspace-root path bounds for `apply_patch` in non-sandbox mode to block traversal and symlink escape writes. Thanks @p80n-sec. - Security/Agents: enforce symlink-escape checks for `apply_patch` delete hunks under `workspaceOnly`, while still allowing deleting the symlink itself. Thanks @p80n-sec. @@ -630,13 +1527,6 @@ Docs: https://docs.openclaw.ai - Media: strip `MEDIA:` lines with local paths instead of leaking as visible text. (#14399) Thanks @0xRaini. - Config/Cron: exclude `maxTokens` from config redaction and honor `deleteAfterRun` on skipped cron jobs. (#13342) Thanks @niceysam. - Config: ignore `meta` field changes in config file watcher. (#13460) Thanks @brandonwise. -- Cron: use requested `agentId` for isolated job auth resolution. (#13983) Thanks @0xRaini. -- Cron: pass `agentId` to `runHeartbeatOnce` for main-session jobs. (#14140) Thanks @ishikawa-pro. -- Cron: prevent cron jobs from skipping execution when `nextRunAtMs` advances. (#14068) Thanks @WalterSumbon. -- Cron: re-arm timers when `onTimer` fires while a job is still executing. (#14233) Thanks @tomron87. -- Cron: prevent duplicate fires when multiple jobs trigger simultaneously. (#14256) Thanks @xinhuagu. -- Cron: isolate scheduler errors so one bad job does not break all jobs. (#14385) Thanks @MarvinDontPanic. -- Cron: prevent one-shot `at` jobs from re-firing on restart after skipped/errored runs. (#13878) Thanks @lailoo. - Daemon: suppress `EPIPE` error when restarting LaunchAgent. (#14343) Thanks @0xRaini. - Antigravity: add opus 4.6 forward-compat model and bypass thinking signature sanitization. (#14218) Thanks @jg-noncelogic. - Agents: prevent file descriptor leaks in child process cleanup. (#13565) Thanks @KyleChen26. @@ -889,17 +1779,10 @@ Docs: https://docs.openclaw.ai - Discord: route autoThread replies to existing threads instead of the root channel. (#8302) Thanks @gavinbmoore, @thewilloftheshadow. - Media understanding: apply SSRF guardrails to provider fetches; allow private baseUrl overrides explicitly. - fix(voice-call): harden inbound allowlist; reject anonymous callers; require Telnyx publicKey for allowlist; token-gate Twilio media streams; cap webhook body size (thanks @simecek) -- fix(webchat): respect user scroll position during streaming and refresh (#7226) (thanks @marcomarandiz) -- Telegram: recover from grammY long-poll timed out errors. (#7466) Thanks @macmimi23. -- Agents: repair malformed tool calls and session transcripts. (#7473) Thanks @justinhuangcode. -- fix(agents): validate AbortSignal instances before calling AbortSignal.any() (#7277) (thanks @Elarwei001) -- Media understanding: skip binary media from file text extraction. (#7475) Thanks @AlexZhangji. - Onboarding: keep TUI flow exclusive (skip completion prompt + background Web UI seed); completion prompt now handled by install/update. -- TUI: block onboarding output while TUI is active and restore terminal state on exit. - CLI/Zsh completion: cache scripts in state dir and escape option descriptions to avoid invalid option errors. - fix(ui): resolve Control UI asset path correctly. - fix(ui): refresh agent files after external edits. -- Docs: finish renaming the QMD memory docs to reference the OpenClaw state dir. - Tests: stub SSRF DNS pinning in web auto-reply + Gemini video coverage. (#6619) Thanks @joshp123. ## 2026.2.1 @@ -1762,7 +2645,6 @@ Thanks @AlexMikhalev, @CoreyH, @John-Rood, @KrauseFx, @MaudeBot, @Nachx639, @Nic - Heartbeat: tighten prompt guidance + suppress duplicate alerts for 24h. (#980) — thanks @voidserf. - Repo: ignore local identity files to avoid accidental commits. (#1001) — thanks @gerardward2007. - Sessions/Security: add `session.dmScope` for multi-user DM isolation and audit warnings. (#948) — thanks @Alphonse-arianee. -- Plugins: add provider auth registry + `openclaw models auth login` for plugin-driven OAuth/API key flows. - Onboarding: switch channels setup to a single-select loop with per-channel actions and disabled hints in the picker. - TUI: show provider/model labels for the active session and default model. - Heartbeat: add per-agent heartbeat configuration and multi-agent docs example. @@ -2298,7 +3180,6 @@ Thanks @AlexMikhalev, @CoreyH, @John-Rood, @KrauseFx, @MaudeBot, @Nachx639, @Nic - Skills additions (Himalaya email, CodexBar, 1Password). - Dependency refreshes (pi-\* stack, Slack SDK, discord-api-types, file-type, zod, Biome, Vite). -- Refactors: centralized group allowlist/mention policy; lint/import cleanup; switch tsx → bun for TS execution. ## 2026.1.5 diff --git a/CLAUDE.md b/CLAUDE.md index c3170642553..47dc3e3d863 120000 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1 +1 @@ -AGENTS.md +AGENTS.md \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index eddb13348ee..35a37f44e39 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -32,6 +32,15 @@ Welcome to the lobster tank! 🦞 - **Mariano Belinky** - iOS app, Security - GitHub: [@mbelinky](https://github.com/mbelinky) · X: [@belimad](https://x.com/belimad) +- **Nimrod Gutman** - iOS app, macOS app and crustacean features + - GitHub: [@ngutman](https://github.com/ngutman) · X: [@theguti](https://x.com/theguti) + +- **Vincent Koc** - Agents, Telemetry, Hooks, Security + - GitHub: [@vincentkoc](https://github.com/vincentkoc) · X: [@vincent_koc](https://x.com/vincent_koc) + +- **Val Alexander** - UI/UX, Docs, and Agent DevX + - GitHub: [@BunsDev](https://github.com/BunsDev) · X: [@BunsDev](https://x.com/BunsDev) + - **Seb Slight** - Docs, Agent Reliability, Runtime Hardening - GitHub: [@sebslight](https://github.com/sebslight) · X: [@sebslig](https://x.com/sebslig) @@ -41,11 +50,22 @@ Welcome to the lobster tank! 🦞 - **Gustavo Madeira Santana** - Multi-agents, CLI, web UI - GitHub: [@gumadeiras](https://github.com/gumadeiras) · X: [@gumadeiras](https://x.com/gumadeiras) +- **Onur Solmaz** - Agents, dev workflows, ACP integrations, MS Teams + - GitHub: [@onutc](https://github.com/onutc), [@osolmaz](https://github.com/osolmaz) · X: [@onusoz](https://x.com/onusoz) + +- **Josh Avant** - Core, CLI, Gateway, Security, Agents + - GitHub: [@joshavant](https://github.com/joshavant) · X: [@joshavant](https://x.com/joshavant) + +- **Jonathan Taylor** - ACP subsystem, Gateway features/bugs, Gog/Mog/Sog CLI's, SEDMAT + - Github [@visionik](https://github.com/visionik) · X: [@visionik](https://x.com/visionik) +- **Josh Lehman** - Compaction, Tlon/Urbit subsystem + - Github [@jalehman](https://github.com/jalehman) · X: [@jlehman\_](https://x.com/jlehman_) + ## How to Contribute 1. **Bugs & small fixes** → Open a PR! 2. **New features / architecture** → Start a [GitHub Discussion](https://github.com/openclaw/openclaw/discussions) or ask in Discord first -3. **Questions** → Discord #setup-help +3. **Questions** → Discord [#help](https://discord.com/channels/1456350064065904867/1459642797895319552) / [#users-helping-users](https://discord.com/channels/1456350064065904867/1459007081603403828) ## Before You PR diff --git a/Dockerfile b/Dockerfile index 2ead5c51fcd..73343a64624 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,4 +1,4 @@ -FROM node:22-bookworm +FROM node:22-bookworm@sha256:cd7bcd2e7a1e6f72052feb023c7f6b722205d3fcab7bbcbd2d1bfdab10b1e935 # Install Bun (required for build scripts) RUN curl -fsSL https://bun.sh/install | bash @@ -7,6 +7,7 @@ ENV PATH="/root/.bun/bin:${PATH}" RUN corepack enable WORKDIR /app +RUN chown node:node /app ARG OPENCLAW_DOCKER_APT_PACKAGES="" RUN if [ -n "$OPENCLAW_DOCKER_APT_PACKAGES" ]; then \ @@ -16,36 +17,46 @@ RUN if [ -n "$OPENCLAW_DOCKER_APT_PACKAGES" ]; then \ rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*; \ fi -COPY package.json pnpm-lock.yaml pnpm-workspace.yaml .npmrc ./ -COPY ui/package.json ./ui/package.json -COPY patches ./patches -COPY scripts ./scripts +COPY --chown=node:node package.json pnpm-lock.yaml pnpm-workspace.yaml .npmrc ./ +COPY --chown=node:node ui/package.json ./ui/package.json +COPY --chown=node:node patches ./patches +COPY --chown=node:node scripts ./scripts -RUN pnpm install --frozen-lockfile +USER node +# Reduce OOM risk on low-memory hosts during dependency installation. +# Docker builds on small VMs may otherwise fail with "Killed" (exit 137). +RUN NODE_OPTIONS=--max-old-space-size=2048 pnpm install --frozen-lockfile # Optionally install Chromium and Xvfb for browser automation. # Build with: docker build --build-arg OPENCLAW_INSTALL_BROWSER=1 ... # Adds ~300MB but eliminates the 60-90s Playwright install on every container start. # Must run after pnpm install so playwright-core is available in node_modules. +USER root ARG OPENCLAW_INSTALL_BROWSER="" RUN if [ -n "$OPENCLAW_INSTALL_BROWSER" ]; then \ apt-get update && \ DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends xvfb && \ + mkdir -p /home/node/.cache/ms-playwright && \ + PLAYWRIGHT_BROWSERS_PATH=/home/node/.cache/ms-playwright \ node /app/node_modules/playwright-core/cli.js install --with-deps chromium && \ + chown -R node:node /home/node/.cache/ms-playwright && \ apt-get clean && \ rm -rf /var/lib/apt/lists/* /var/cache/apt/archives/*; \ fi -COPY . . +USER node +COPY --chown=node:node . . RUN pnpm build # Force pnpm for UI build (Bun may fail on ARM/Synology architectures) ENV OPENCLAW_PREFER_PNPM=1 RUN pnpm ui:build -ENV NODE_ENV=production +# Expose the CLI binary without requiring npm global writes as non-root. +USER root +RUN ln -sf /app/openclaw.mjs /usr/local/bin/openclaw \ + && chmod 755 /app/openclaw.mjs -# Allow non-root user to write temp files during runtime/tests. -RUN chown -R node:node /app +ENV NODE_ENV=production # Security hardening: Run as non-root user # The node:22-bookworm image includes a 'node' user (uid 1000) diff --git a/Dockerfile.sandbox b/Dockerfile.sandbox index 21fd321a492..a463d4a1020 100644 --- a/Dockerfile.sandbox +++ b/Dockerfile.sandbox @@ -1,4 +1,4 @@ -FROM debian:bookworm-slim +FROM debian:bookworm-slim@sha256:98f4b71de414932439ac6ac690d7060df1f27161073c5036a7553723881bffbe ENV DEBIAN_FRONTEND=noninteractive diff --git a/Dockerfile.sandbox-browser b/Dockerfile.sandbox-browser index 4eccbc9a1ae..ec9faf71113 100644 --- a/Dockerfile.sandbox-browser +++ b/Dockerfile.sandbox-browser @@ -1,4 +1,4 @@ -FROM debian:bookworm-slim +FROM debian:bookworm-slim@sha256:98f4b71de414932439ac6ac690d7060df1f27161073c5036a7553723881bffbe ENV DEBIAN_FRONTEND=noninteractive diff --git a/README.md b/README.md index fe96768c95f..b15cabfbbe9 100644 --- a/README.md +++ b/README.md @@ -23,16 +23,21 @@ It answers you on the channels you already use (WhatsApp, Telegram, Slack, Disco If you want a personal, single-user assistant that feels local, fast, and always-on, this is it. -[Website](https://openclaw.ai) · [Docs](https://docs.openclaw.ai) · [Vision](VISION.md) · [DeepWiki](https://deepwiki.com/openclaw/openclaw) · [Getting Started](https://docs.openclaw.ai/start/getting-started) · [Updating](https://docs.openclaw.ai/install/updating) · [Showcase](https://docs.openclaw.ai/start/showcase) · [FAQ](https://docs.openclaw.ai/start/faq) · [Wizard](https://docs.openclaw.ai/start/wizard) · [Nix](https://github.com/openclaw/nix-openclaw) · [Docker](https://docs.openclaw.ai/install/docker) · [Discord](https://discord.gg/clawd) +[Website](https://openclaw.ai) · [Docs](https://docs.openclaw.ai) · [Vision](VISION.md) · [DeepWiki](https://deepwiki.com/openclaw/openclaw) · [Getting Started](https://docs.openclaw.ai/start/getting-started) · [Updating](https://docs.openclaw.ai/install/updating) · [Showcase](https://docs.openclaw.ai/start/showcase) · [FAQ](https://docs.openclaw.ai/help/faq) · [Wizard](https://docs.openclaw.ai/start/wizard) · [Nix](https://github.com/openclaw/nix-openclaw) · [Docker](https://docs.openclaw.ai/install/docker) · [Discord](https://discord.gg/clawd) Preferred setup: run the onboarding wizard (`openclaw onboard`) in your terminal. The wizard guides you step by step through setting up the gateway, workspace, channels, and skills. The CLI wizard is the recommended path and works on **macOS, Linux, and Windows (via WSL2; strongly recommended)**. Works with npm, pnpm, or bun. New install? Start here: [Getting started](https://docs.openclaw.ai/start/getting-started) +## Sponsors + +| OpenAI | Blacksmith | Convex | +| ----------------------------------------------------------------- | ---------------------------------------------------------------------------- | --------------------------------------------------------------------- | +| [![OpenAI](docs/assets/sponsors/openai.svg)](https://openai.com/) | [![Blacksmith](docs/assets/sponsors/blacksmith.svg)](https://blacksmith.sh/) | [![Convex](docs/assets/sponsors/convex.svg)](https://www.convex.dev/) | + **Subscriptions (OAuth):** -- **[Anthropic](https://www.anthropic.com/)** (Claude Pro/Max) - **[OpenAI](https://openai.com/)** (ChatGPT/Codex) Model note: while any model is supported, I strongly recommend **Anthropic Pro/Max (100/200) + Opus 4.6** for long‑context strength and better prompt‑injection resistance. See [Onboarding](https://docs.openclaw.ai/start/onboarding). @@ -140,13 +145,13 @@ Run `openclaw doctor` to surface risky/misconfigured DM policies. - [Gateway WS control plane](https://docs.openclaw.ai/gateway) with sessions, presence, config, cron, webhooks, [Control UI](https://docs.openclaw.ai/web), and [Canvas host](https://docs.openclaw.ai/platforms/mac/canvas#canvas-a2ui). - [CLI surface](https://docs.openclaw.ai/tools/agent-send): gateway, agent, send, [wizard](https://docs.openclaw.ai/start/wizard), and [doctor](https://docs.openclaw.ai/gateway/doctor). - [Pi agent runtime](https://docs.openclaw.ai/concepts/agent) in RPC mode with tool streaming and block streaming. -- [Session model](https://docs.openclaw.ai/concepts/session): `main` for direct chats, group isolation, activation modes, queue modes, reply-back. Group rules: [Groups](https://docs.openclaw.ai/concepts/groups). +- [Session model](https://docs.openclaw.ai/concepts/session): `main` for direct chats, group isolation, activation modes, queue modes, reply-back. Group rules: [Groups](https://docs.openclaw.ai/channels/groups). - [Media pipeline](https://docs.openclaw.ai/nodes/images): images/audio/video, transcription hooks, size caps, temp file lifecycle. Audio details: [Audio](https://docs.openclaw.ai/nodes/audio). ### Channels - [Channels](https://docs.openclaw.ai/channels): [WhatsApp](https://docs.openclaw.ai/channels/whatsapp) (Baileys), [Telegram](https://docs.openclaw.ai/channels/telegram) (grammY), [Slack](https://docs.openclaw.ai/channels/slack) (Bolt), [Discord](https://docs.openclaw.ai/channels/discord) (discord.js), [Google Chat](https://docs.openclaw.ai/channels/googlechat) (Chat API), [Signal](https://docs.openclaw.ai/channels/signal) (signal-cli), [BlueBubbles](https://docs.openclaw.ai/channels/bluebubbles) (iMessage, recommended), [iMessage](https://docs.openclaw.ai/channels/imessage) (legacy imsg), [Microsoft Teams](https://docs.openclaw.ai/channels/msteams) (extension), [Matrix](https://docs.openclaw.ai/channels/matrix) (extension), [Zalo](https://docs.openclaw.ai/channels/zalo) (extension), [Zalo Personal](https://docs.openclaw.ai/channels/zalouser) (extension), [WebChat](https://docs.openclaw.ai/web/webchat). -- [Group routing](https://docs.openclaw.ai/concepts/group-messages): mention gating, reply tags, per-channel chunking and routing. Channel rules: [Channels](https://docs.openclaw.ai/channels). +- [Group routing](https://docs.openclaw.ai/channels/group-messages): mention gating, reply tags, per-channel chunking and routing. Channel rules: [Channels](https://docs.openclaw.ai/channels). ### Apps + nodes @@ -165,7 +170,7 @@ Run `openclaw doctor` to surface risky/misconfigured DM policies. ### Runtime + safety -- [Channel routing](https://docs.openclaw.ai/concepts/channel-routing), [retry policy](https://docs.openclaw.ai/concepts/retry), and [streaming/chunking](https://docs.openclaw.ai/concepts/streaming). +- [Channel routing](https://docs.openclaw.ai/channels/channel-routing), [retry policy](https://docs.openclaw.ai/concepts/retry), and [streaming/chunking](https://docs.openclaw.ai/concepts/streaming). - [Presence](https://docs.openclaw.ai/concepts/presence), [typing indicators](https://docs.openclaw.ai/concepts/typing-indicators), and [usage tracking](https://docs.openclaw.ai/concepts/usage-tracking). - [Models](https://docs.openclaw.ai/concepts/models), [model failover](https://docs.openclaw.ai/concepts/model-failover), and [session pruning](https://docs.openclaw.ai/concepts/session-pruning). - [Security](https://docs.openclaw.ai/gateway/security) and [troubleshooting](https://docs.openclaw.ai/channels/troubleshooting). @@ -497,54 +502,54 @@ Special thanks to Adam Doppelt for lobster.bot. Thanks to all clawtributors:

- steipete joshp123 cpojer Mariano Belinky sebslight Takhoffman quotentiroler bohdanpodvirnyi tyler6204 iHildy - jaydenfyi gumadeiras joaohlisboa mneves75 MatthieuBizien Glucksberg MaudeBot rahthakor vrknetha vignesh07 - radek-paclt abdelsfane Tobias Bischoff christianklotz czekaj ethanpalm mukhtharcm maxsumrall rodrigouroz xadenryan - VACInc juanpablodlc conroywhitney hsrvc magimetal zerone0x advaitpaliwal meaningfool patelhiren NicholasSpisak - jonisjongithub abhisekbasu1 theonejvo jamesgroat BunsDev claude JustYannicc Hyaxia dantelex SocialNerd42069 - daveonkels Yida-Dev google-labs-jules[bot] riccardogiorato lc0rp adam91holt mousberg clawdinator[bot] hougangdev shakkernerd - coygeek mteam88 hirefrank M00N7682 joeynyc orlyjamie dbhurley Eng. Juan Combetto TSavo aerolalit - julianengel bradleypriest benithors lsh411 gut-puncture rohannagpal timolins f-trycua benostein elliotsecops - nachx639 pvoo sreekaransrinath gupsammy cristip73 stefangalescu nachoiacovino Vasanth Rao Naik Sabavat thewilloftheshadow petter-b - leszekszpunar scald pycckuu AnonO6 andranik-sahakyan davidguttman jarvis89757 sleontenko denysvitali TinyTb - sircrumpet peschee nicolasstanley davidiach nonggia.liang ironbyte-rgb dominicnunez lploc94 ratulsarna sfo2001 - lutr0 kiranjd danielz1z Iranb cdorsey AdeboyeDN obviyus Alg0rix papago2355 peetzweg/ - emanuelst evanotero KristijanJovanovski jlowin rdev rhuanssauro joshrad-dev osolmaz adityashaw2 shadril238 - CashWilliams sheeek ryan jasonsschin artuskg onutc pauloportella HirokiKobayashi-R ThanhNguyxn 18-RAJAT - kimitaka yuting0624 neooriginal manuelhettich unisone baccula manikv12 sbking travisirby fujiwara-tofu-shop - buddyh connorshea bjesuiter kyleok mcinteerj slonce70 calvin-hpnet gitpds ide-rea badlogic - grp06 dependabot[bot] amitbiswal007 John-Rood timkrase gerardward2007 roshanasingh4 tosh-hamburg azade-c dlauer - ezhikkk JonUleis shivamraut101 cheeeee jabezborja robbyczgw-cla YuriNachos Josh Phillips Wangnov kaizen403 - patrickshao Whoaa512 chriseidhof ngutman wangai-studio ysqander Yurii Chukhlib aj47 kennyklee superman32432432 - Hisleren antons austinm911 blacksmith-sh[bot] damoahdominic dan-dr doodlewind GHesericsu HeimdallStrategy imfing - jalehman jarvis-medmatic kkarimi Lukavyi mahmoudashraf93 pkrmf RandyVentures Ryan Lisse Yeom-JinHo dougvk - erikpr1994 fal3 Ghost hyf0-agent jonasjancarik Keith the Silly Goose L36 Server Marc mitschabaude-bot mkbehr - neist orenyomtov sibbl zats abhijeet117 chrisrodz Friederike Seiler gabriel-trigo hudson-rivera iamadig - itsjling Jonathan D. Rhyne (DJ-D) Joshua Mitchell kelvinCB Kit koala73 lailoo manmal mattqdev mcaxtr - mitsuhiko ogulcancelik petradonka rubyrunsstuff rybnikov siddhantjain suminhthanh svkozak wes-davis 24601 - ameno- bonald bravostation Chris Taylor damaozi dguido Django Navarro evalexpr henrino3 humanwritten - j2h4u larlyssa liuxiaopai-ai odysseus0 oswalpalash pcty-nextgen-service-account pi0 rmorse Roopak Nijhara Syhids - tmchow Ubuntu xiaose Aaron Konyer aaronveklabs akramcodez aldoeliacim andreabadesso Andrii BinaryMuse - bqcfjwhz85-arch cash-echo-bot Clawd ClawdFx danballance danielcadenhead Elarwei001 EnzeD erik-agens Evizero - fcatuhe gildo hclsys itsjaydesu ivancasco ivanrvpereira Jarvis jayhickey jeffersonwarrior jeffersonwarrior - jverdi longmaba Marco Marandiz MarvinCui mattezell mjrussell odnxe optimikelabs p6l-richard philipp-spiess - Pocket Clawd RayBB robaxelsen Sash Catanzarite Suksham-sharma T5-AndyML thejhinvirtuoso travisp VAC william arzt - yudshj zknicker 0oAstro Abdul535 abhaymundhara aduk059 aisling404 alejandro maza Alex-Alaniz alexanderatallah - alexstyl AlexZhangji andrewting19 anpoirier araa47 arthyn Asleep123 Ayush Ojha Ayush10 bguidolim - bolismauro caelum0x championswimmer chenyuan99 Chloe-VP Claude Code Clawdbot Maintainers conhecendoia dasilva333 David-Marsh-Photo - deepsoumya617 Developer Dimitrios Ploutarchos Drake Thomsen dvrshil dxd5001 dylanneve1 Felix Krause foeken frankekn - fredheir Fronut ganghyun kim grrowl gtsifrikas HassanFleyah HazAT hrdwdmrbl hugobarauna iamEvanYT - ichbinlucaskim Jamie Openshaw Jane Jarvis Deploy Jefferson Nunn jogi47 kentaro Kevin Lin kira-ariaki kitze - Kiwitwitter kossoy levifig liuy Lloyd loganaden longjos loukotal mac mimi markusbkoch - martinpucik Matt mini mertcicekci0 Miles minghinmatthewlam mrdbstn MSch mudrii Mustafa Tag Eldeen myfunc - mylukin nathanbosse ndraiman nexty5870 Noctivoro Omar-Khaleel ozgur-polat pasogott plum-dawg pookNast - ppamment prathamdby ptn1411 rafaelreis-r rafelbev reeltimeapps RLTCmpe robhparker rohansachinpatil Rony Kelner - ryancnelson Samrat Jha seans-openclawbot senoldogann Seredeep sergical shatner shiv19 shiyuanhai Shrinija17 - siraht snopoke spiceoogway stephenchen2025 succ985 Suvink techboss testingabc321 tewatia The Admiral - therealZpoint-bot thesash uos-status vcastellm Vibe Kanban vincentkoc void Vultr-Clawd Admin Wimmie wolfred - wstock wytheme YangHuang2280 yazinsai yevhen YiWang24 ymat19 Zach Knickerbocker zackerthescar zhixian - 0xJonHoldsCrypto aaronn Alphonse-arianee atalovesyou Azade carlulsoe ddyo Erik jiulingyun latitudeki5223 - Manuel Maly minghinmatthewlam Mourad Boustani odrobnik pcty-nextgen-ios-builder Quentin rafaelreis-r Randy Torres rhjoh Rolf Fredheim - ronak-guliani William Stock - Akash Kobal + steipete sktbrd cpojer joshp123 Mariano Belinky Takhoffman sebslight tyler6204 quotentiroler Verite Igiraneza + gumadeiras bohdanpodvirnyi vincentkoc iHildy jaydenfyi Glucksberg joaohlisboa rodrigouroz mneves75 BunsDev + MatthieuBizien MaudeBot vignesh07 smartprogrammer93 advaitpaliwal HenryLoenwind rahthakor vrknetha abdelsfane radek-paclt + joshavant christianklotz mudrii zerone0x ranausmanai Tobias Bischoff heyhudson czekaj ethanpalm yinghaosang + nabbilkhan mukhtharcm aether-ai-agent coygeek Mrseenz maxsumrall xadenryan VACInc juanpablodlc conroywhitney + Harald Buerbaumer akoscz Bridgerz hsrvc magimetal openclaw-bot meaningfool JustasM Phineas1500 ENCHIGO + Hiren Patel NicholasSpisak claude jonisjongithub theonejvo abhisekbasu1 Ryan Haines Blakeshannon jamesgroat Marvae + arosstale shakkernerd gejifeng divanoli ryan-crabbe nyanjou Sam Padilla dantelex SocialNerd42069 solstead + natefikru daveonkels LeftX Yida-Dev Masataka Shinohara Lewis riccardogiorato lc0rp adam91holt mousberg + BillChirico shadril238 CharlieGreenman hougangdev Mars orlyjamie McRolly NWANGWU LI SHANXIN Simone Macario durenzidu + JustYannicc Minidoracat magendary Jessy LANGE mteam88 brandonwise hirefrank M00N7682 dbhurley Eng. Juan Combetto + Harrington-bot TSavo Lalit Singh julianengel Jay Caldwell Kirill Shchetynin nachx639 bradleypriest TsekaLuk benithors + Shailesh thewilloftheshadow jackheuberger loiie45e El-Fitz benostein pvtclawn 0xRaini ruypang xinhuagu + Taylor Asplund adhitShet Paul van Oorschot sreekaransrinath buddyh gupsammy AI-Reviewer-QS Stefan Galescu WalterSumbon nachoiacovino + rodbland2021 Vasanth Rao Naik Sabavat fagemx petter-b omair445 dorukardahan leszekszpunar Clawborn davidrudduck scald + Igor Markelov rrenamed Parker Todd Brooks AnonO6 Tanwa Arpornthip andranik-sahakyan davidguttman sleontenko denysvitali Tom Ron + popomore Patrick Barletta shayan919293 不做了睡大觉 Luis Conde Harry Cui Kepler SidQin-cyber Lucky Michael Lee sircrumpet + peschee dakshaymehta davidiach nonggia.liang seheepeak obviyus danielwanwx osolmaz minupla misterdas + Shuai-DaiDai dominicnunez lploc94 sfo2001 lutr0 dirbalak cathrynlavery Joly0 kiranjd niceysam + danielz1z Iranb carrotRakko Oceanswave cdorsey AdeboyeDN j2h4u Alg0rix Skyler Miao peetzweg/ + TideFinder CornBrother0x DukeDeSouth emanuelst bsormagec Diaspar4u evanotero Nate OscarMinjarez webvijayi + garnetlyx miloudbelarebia Jeremiah Lowin liebertar Max rhuanssauro joshrad-dev adityashaw2 CashWilliams taw0002 + asklee-klawd h0tp-ftw constansino mcaxtr onutc ryan unisone artuskg Solvely-Colin pahdo + Kimitaka Watanabe Lilo Rajat Joshi Yuting Lin Neo wu-tian807 ngutman crimeacs manuelhettich mcinteerj + bjesuiter Manik Vahsith alexgleason Nicholas Stephen Brian King justinhuangcode mahanandhi andreesg connorshea dinakars777 + Flash-LHR JINNYEONG KIM Protocol Zero kyleok Limitless grp06 robbyczgw-cla slonce70 JayMishra-source ide-rea + lailoo badlogic echoVic amitbiswal007 azade-c John Rood dddabtc Jonathan Works roshanasingh4 tosh-hamburg + dlauer ezhikkk Shivam Kumar Raut Mykyta Bozhenko YuriNachos Josh Phillips ThomsenDrake Wangnov akramcodez jadilson12 + Whoaa512 clawdinator[bot] emonty kaizen403 chriseidhof Lukavyi wangai-studio ysqander aj47 google-labs-jules[bot] + hyf0-agent Jeremy Mumford Kenny Lee superman32432432 widingmarcus-cyber DylanWoodAkers antons austinm911 boris721 damoahdominic + dan-dr doodlewind GHesericsu HeimdallStrategy imfing jalehman jarvis-medmatic kkarimi mahmoudashraf93 pkrmf + Randy Torres sumleo Yeom-JinHo akyourowngames aldoeliacim Dithilli dougvk erikpr1994 fal3 jonasjancarik + koala73 mitschabaude-bot mkbehr Oren shtse8 sibbl thesomewhatyou zats chrisrodz frankekn + gabriel-trigo ghsmc iamadig ibrahimq21 irtiq7 jeann2013 jogelin Jonathan D. Rhyne (DJ-D) Justin Ling kelvinCB + manmal Matthew MattQ Milofax mitsuhiko neist pejmanjohn ProspectOre rmorse rubyrunsstuff + rybnikov santiagomed Steve (OpenClaw) suminhthanh svkozak wes-davis 24601 AkashKobal ameno- awkoy + battman21 BinHPdev bonald dashed dawondyifraw dguido Django Navarro evalexpr henrino3 humanwritten + hyojin joeykrug larlyssa liuy Mark Liu natedenh odysseus0 pcty-nextgen-service-account pi0 Syhids + tmchow uli-will-code aaronveklabs andreabadesso BinaryMuse cash-echo-bot CJWTRUST cordx56 danballance Elarwei001 + EnzeD erik-agens Evizero fcatuhe gildo Grynn huntharo hydro13 itsjaydesu ivanrvpereira + jverdi kentaro loeclos longmaba MarvinCui MisterGuy420 mjrussell odnxe optimikelabs oswalpalash + p6l-richard philipp-spiess RamiNoodle733 Raymond Berger Rob Axelsen sauerdaniel SleuthCo T5-AndyML TaKO8Ki thejhinvirtuoso + travisp yudshj zknicker 0oAstro 8BlT Abdul535 abhaymundhara aduk059 afurm aisling404 + akari-musubi Alex-Alaniz alexanderatallah alexstyl andrewting19 araa47 Asleep123 Ayush10 bennewton999 bguidolim + caelum0x championswimmer Chloe-VP dario-github DarwinsBuddy David-Marsh-Photo dcantu96 dndodson dvrshil dxd5001 + dylanneve1 EmberCF ephraimm ereid7 eternauta1337 foeken gtsifrikas HazAT iamEvanYT ikari-pl + kesor knocte MackDing nobrainer-tech Noctivoro Olshansk Pratham Dubey Raikan10 SecondThread Swader + testingabc321 0xJonHoldsCrypto aaronn Alphonse-arianee atalovesyou carlulsoe hrdwdmrbl hugobarauna jayhickey jiulingyun + kitze latitudeki5223 loukotal minghinmatthewlam MSch odrobnik rafaelreis-r ratulsarna reeltimeapps rhjoh + ronak-guliani snopoke thesash timkrase

diff --git a/SECURITY.md b/SECURITY.md index 63440837047..d7e4977e600 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -13,7 +13,7 @@ Report vulnerabilities directly to the repository where the issue lives: - **ClawHub** — [openclaw/clawhub](https://github.com/openclaw/clawhub) - **Trust and threat model** — [openclaw/trust](https://github.com/openclaw/trust) -For issues that don't fit a specific repo, or if you're unsure, email **security@openclaw.ai** and we'll route it. +For issues that don't fit a specific repo, or if you're unsure, email **[security@openclaw.ai](mailto:security@openclaw.ai)** and we'll route it. For full reporting instructions see our [Trust page](https://trust.openclaw.ai). @@ -30,6 +30,44 @@ For full reporting instructions see our [Trust page](https://trust.openclaw.ai). Reports without reproduction steps, demonstrated impact, and remediation advice will be deprioritized. Given the volume of AI-generated scanner findings, we must ensure we're receiving vetted reports from researchers who understand the issues. +### Report Acceptance Gate (Triage Fast Path) + +For fastest triage, include all of the following: + +- Exact vulnerable path (`file`, function, and line range) on a current revision. +- Tested version details (OpenClaw version and/or commit SHA). +- Reproducible PoC against latest `main` or latest released version. +- Demonstrated impact tied to OpenClaw's documented trust boundaries. +- For exposed-secret reports: proof the credential is OpenClaw-owned (or grants access to OpenClaw-operated infrastructure/services). +- Explicit statement that the report does not rely on adversarial operators sharing one gateway host/config. +- Scope check explaining why the report is **not** covered by the Out of Scope section below. +- For command-risk/parity reports (for example obfuscation detection differences), a concrete boundary-bypass path is required (auth/approval/allowlist/sandbox). Parity-only findings are treated as hardening, not vulnerabilities. + +Reports that miss these requirements may be closed as `invalid` or `no-action`. + +### Common False-Positive Patterns + +These are frequently reported but are typically closed with no code change: + +- Prompt-injection-only chains without a boundary bypass (prompt injection is out of scope). +- Operator-intended local features (for example TUI local `!` shell) presented as remote injection. +- Authorized user-triggered local actions presented as privilege escalation. Example: an allowlisted/owner sender running `/export-session /absolute/path.html` to write on the host. In this trust model, authorized user actions are trusted host actions unless you demonstrate an auth/sandbox/boundary bypass. +- Reports that only show a malicious plugin executing privileged actions after a trusted operator installs/enables it. +- Reports that assume per-user multi-tenant authorization on a shared gateway host/config. +- Reports that only show differences in heuristic detection/parity (for example obfuscation-pattern detection on one exec path but not another, such as `node.invoke -> system.run` parity gaps) without demonstrating bypass of auth, approvals, allowlist enforcement, sandboxing, or other documented trust boundaries. +- ReDoS/DoS claims that require trusted operator configuration input (for example catastrophic regex in `sessionFilter` or `logging.redactPatterns`) without a trust-boundary bypass. +- Missing HSTS findings on default local/loopback deployments. +- Slack webhook signature findings when HTTP mode already uses signing-secret verification. +- Discord inbound webhook signature findings for paths not used by this repo's Discord integration. +- Claims that Microsoft Teams `fileConsent/invoke` `uploadInfo.uploadUrl` is attacker-controlled without demonstrating one of: auth boundary bypass, a real authenticated Teams/Bot Framework event carrying attacker-chosen URL, or compromise of the Microsoft/Bot trust path. +- Scanner-only claims against stale/nonexistent paths, or claims without a working repro. + +### Duplicate Report Handling + +- Search existing advisories before filing. +- Include likely duplicate GHSA IDs in your report when applicable. +- Maintainers may close lower-quality/later duplicates in favor of the earliest high-quality canonical report. + ## Security & Trust **Jamieson O'Reilly** ([@theonejvo](https://twitter.com/theonejvo)) is Security & Trust at OpenClaw. Jamieson is the founder of [Dvuln](https://dvuln.com) and brings extensive experience in offensive security, penetration testing, and security program development. @@ -43,11 +81,116 @@ The best way to help the project right now is by sending PRs. When patching a GHSA via `gh api`, include `X-GitHub-Api-Version: 2022-11-28` (or newer). Without it, some fields (notably CVSS) may not persist even if the request returns 200. +## Operator Trust Model (Important) + +OpenClaw does **not** model one gateway as a multi-tenant, adversarial user boundary. + +- Authenticated Gateway callers are treated as trusted operators for that gateway instance. +- Session identifiers (`sessionKey`, session IDs, labels) are routing controls, not per-user authorization boundaries. +- If one operator can view data from another operator on the same gateway, that is expected in this trust model. +- OpenClaw can technically run multiple gateway instances on one machine, but recommended operations are clean separation by trust boundary. +- Recommended mode: one user per machine/host (or VPS), one gateway for that user, and one or more agents inside that gateway. +- If multiple users need OpenClaw, use one VPS (or host/OS user boundary) per user. +- For advanced setups, multiple gateways on one machine are possible, but only with strict isolation and are not the recommended default. +- Exec behavior is host-first by default: `agents.defaults.sandbox.mode` defaults to `off`. +- `tools.exec.host` defaults to `sandbox` as a routing preference, but if sandbox runtime is not active for the session, exec runs on the gateway host. +- Implicit exec calls (no explicit host in the tool call) follow the same behavior. +- This is expected in OpenClaw's one-user trusted-operator model. If you need isolation, enable sandbox mode (`non-main`/`all`) and keep strict tool policy. + +## Trusted Plugin Concept (Core) + +Plugins/extensions are part of OpenClaw's trusted computing base for a gateway. + +- Installing or enabling a plugin grants it the same trust level as local code running on that gateway host. +- Plugin behavior such as reading env/files or running host commands is expected inside this trust boundary. +- Security reports must show a boundary bypass (for example unauthenticated plugin load, allowlist/policy bypass, or sandbox/path-safety bypass), not only malicious behavior from a trusted-installed plugin. + ## Out of Scope - Public Internet Exposure - Using OpenClaw in ways that the docs recommend not to -- Prompt injection attacks +- Deployments where mutually untrusted/adversarial operators share one gateway host and config (for example, reports expecting per-operator isolation for `sessions.list`, `sessions.preview`, `chat.history`, or similar control-plane reads) +- Prompt-injection-only attacks (without a policy/auth/sandbox boundary bypass) +- Reports that require write access to trusted local state (`~/.openclaw`, workspace files like `MEMORY.md` / `memory/*.md`) +- Reports where the only demonstrated impact is an already-authorized sender intentionally invoking a local-action command (for example `/export-session` writing to an absolute host path) without bypassing auth, sandbox, or another documented boundary +- Reports where the only claim is that a trusted-installed/enabled plugin can execute with gateway/host privileges (documented trust model behavior). +- Any report whose only claim is that an operator-enabled `dangerous*`/`dangerously*` config option weakens defaults (these are explicit break-glass tradeoffs by design) +- Reports that depend on trusted operator-supplied configuration values to trigger availability impact (for example custom regex patterns). These may still be fixed as defense-in-depth hardening, but are not security-boundary bypasses. +- Reports whose only claim is heuristic/parity drift in command-risk detection (for example obfuscation-pattern checks) across exec surfaces, without a demonstrated trust-boundary bypass. These are hardening-only findings and are not vulnerabilities; triage may close them as `invalid`/`no-action` or track them separately as low/informational hardening. +- Exposed secrets that are third-party/user-controlled credentials (not OpenClaw-owned and not granting access to OpenClaw-operated infrastructure/services) without demonstrated OpenClaw impact +- Reports whose only claim is host-side exec when sandbox runtime is disabled/unavailable (documented default behavior in the trusted-operator model), without a boundary bypass. +- Reports whose only claim is that a platform-provided upload destination URL is untrusted (for example Microsoft Teams `fileConsent/invoke` `uploadInfo.uploadUrl`) without proving attacker control in an authenticated production flow. + +## Deployment Assumptions + +OpenClaw security guidance assumes: + +- The host where OpenClaw runs is within a trusted OS/admin boundary. +- Anyone who can modify `~/.openclaw` state/config (including `openclaw.json`) is effectively a trusted operator. +- A single Gateway shared by mutually untrusted people is **not a recommended setup**. Use separate gateways (or at minimum separate OS users/hosts) per trust boundary. +- Authenticated Gateway callers are treated as trusted operators. Session identifiers (for example `sessionKey`) are routing controls, not per-user authorization boundaries. +- Multiple gateway instances can run on one machine, but the recommended model is clean per-user isolation (prefer one host/VPS per user). + +## One-User Trust Model (Personal Assistant) + +OpenClaw's security model is "personal assistant" (one trusted operator, potentially many agents), not "shared multi-tenant bus." + +- If multiple people can message the same tool-enabled agent (for example a shared Slack workspace), they can all steer that agent within its granted permissions. +- Session or memory scoping reduces context bleed, but does **not** create per-user host authorization boundaries. +- For mixed-trust or adversarial users, isolate by OS user/host/gateway and use separate credentials per boundary. +- A company-shared agent can be a valid setup when users are in the same trust boundary and the agent is strictly business-only. +- For company-shared setups, use a dedicated machine/VM/container and dedicated accounts; avoid mixing personal data on that runtime. +- If that host/browser profile is logged into personal accounts (for example Apple/Google/personal password manager), you have collapsed the boundary and increased personal-data exposure risk. + +## Agent and Model Assumptions + +- The model/agent is **not** a trusted principal. Assume prompt/content injection can manipulate behavior. +- Security boundaries come from host/config trust, auth, tool policy, sandboxing, and exec approvals. +- Prompt injection by itself is not a vulnerability report unless it crosses one of those boundaries. + +## Gateway and Node trust concept + +OpenClaw separates routing from execution, but both remain inside the same operator trust boundary: + +- **Gateway** is the control plane. If a caller passes Gateway auth, they are treated as a trusted operator for that Gateway. +- **Node** is an execution extension of the Gateway. Pairing a node grants operator-level remote capability on that node. +- **Exec approvals** (allowlist/ask UI) are operator guardrails to reduce accidental command execution, not a multi-tenant authorization boundary. +- Differences in command-risk warning heuristics between exec surfaces (`gateway`, `node`, `sandbox`) do not, by themselves, constitute a security-boundary bypass. +- For untrusted-user isolation, split by trust boundary: separate gateways and separate OS users/hosts per boundary. + +## Workspace Memory Trust Boundary + +`MEMORY.md` and `memory/*.md` are plain workspace files and are treated as trusted local operator state. + +- If someone can edit workspace memory files, they already crossed the trusted operator boundary. +- Memory search indexing/recall over those files is expected behavior, not a sandbox/security boundary. +- Example report pattern considered out of scope: "attacker writes malicious content into `memory/*.md`, then `memory_search` returns it." +- If you need isolation between mutually untrusted users, split by OS user or host and run separate gateways. + +## Plugin Trust Boundary + +Plugins/extensions are loaded **in-process** with the Gateway and are treated as trusted code. + +- Plugins can execute with the same OS privileges as the OpenClaw process. +- Runtime helpers (for example `runtime.system.runCommandWithTimeout`) are convenience APIs, not a sandbox boundary. +- Only install plugins you trust, and prefer `plugins.allow` to pin explicit trusted plugin ids. + +## Temp Folder Boundary (Media/Sandbox) + +OpenClaw uses a dedicated temp root for local media handoff and sandbox-adjacent temp artifacts: + +- Preferred temp root: `/tmp/openclaw` (when available and safe on the host). +- Fallback temp root: `os.tmpdir()/openclaw` (or `openclaw-` on multi-user hosts). + +Security boundary notes: + +- Sandbox media validation allows absolute temp paths only under the OpenClaw-managed temp root. +- Arbitrary host tmp paths are not treated as trusted media roots. +- Plugin/extension code should use OpenClaw temp helpers (`resolvePreferredOpenClawTmpDir`, `buildRandomTempFilePath`, `withTempDownloadPath`) rather than raw `os.tmpdir()` defaults when handling media files. +- Enforcement reference points: + - temp root resolver: `src/infra/tmp-openclaw-dir.ts` + - SDK temp helpers: `src/plugin-sdk/temp-path.ts` + - messaging/channel tmp guardrail: `scripts/check-no-random-messaging-tmp.mjs` ## Operational Guidance @@ -58,7 +201,7 @@ For threat model + hardening guidance (including `openclaw security audit --deep ### Tool filesystem hardening - `tools.exec.applyPatch.workspaceOnly: true` (recommended): keeps `apply_patch` writes/deletes within the configured workspace directory. -- `tools.fs.workspaceOnly: true` (optional): restricts `read`/`write`/`edit`/`apply_patch` paths to the workspace directory. +- `tools.fs.workspaceOnly: true` (optional): restricts `read`/`write`/`edit`/`apply_patch` paths and native prompt image auto-load paths to the workspace directory. - Avoid setting `tools.exec.applyPatch.workspaceOnly: false` unless you fully trust who can trigger tool execution. ### Web Interface Safety @@ -68,6 +211,14 @@ OpenClaw's web interface (Gateway Control UI + HTTP endpoints) is intended for * - Recommended: keep the Gateway **loopback-only** (`127.0.0.1` / `::1`). - Config: `gateway.bind="loopback"` (default). - CLI: `openclaw gateway run --bind loopback`. +- `gateway.controlUi.dangerouslyDisableDeviceAuth` is intended for localhost-only break-glass use. + - OpenClaw keeps deployment flexibility by design and does not hard-forbid non-local setups. + - Non-local and other risky configurations are surfaced by `openclaw security audit` as dangerous findings. + - This operator-selected tradeoff is by design and not, by itself, a security vulnerability. +- Canvas host note: network-visible canvas is **intentional** for trusted node scenarios (LAN/tailnet). + - Expected setup: non-loopback bind + Gateway auth (token/password/trusted-proxy) + firewall/tailnet controls. + - Expected routes: `/__openclaw__/canvas/`, `/__openclaw__/a2ui/`. + - This deployment model alone is not a security vulnerability. - Do **not** expose it to the public internet (no direct bind to `0.0.0.0`, no public reverse proxy). It is not hardened for public exposure. - If you need remote access, prefer an SSH tunnel or Tailscale serve/funnel (so the Gateway still binds to loopback), plus strong Gateway auth. - The Gateway HTTP surface includes the canvas host (`/__openclaw__/canvas/`, `/__openclaw__/a2ui/`). Treat canvas content as sensitive/untrusted and avoid exposing it beyond loopback unless you understand the risk. diff --git a/appcast.xml b/appcast.xml index 3318fbaf86b..5a43fbfc41e 100644 --- a/appcast.xml +++ b/appcast.xml @@ -209,105 +209,106 @@ - 2026.2.13 - Sat, 14 Feb 2026 04:30:23 +0100 + 2026.2.26 + Thu, 26 Feb 2026 23:37:15 +0100 https://raw.githubusercontent.com/openclaw/openclaw/main/appcast.xml - 9846 - 2026.2.13 + 202602260 + 2026.2.26 15.0 - OpenClaw 2026.2.13 + OpenClaw 2026.2.26

Changes

    -
  • Discord: send voice messages with waveform previews from local audio files (including silent delivery). (#7253) Thanks @nyanjou.
    • -
  • Discord: add configurable presence status/activity/type/url (custom status defaults to activity text). (#10855) Thanks @h0tp-ftw.
    • -
  • Slack/Plugins: add thread-ownership outbound gating via message_sending hooks, including @-mention bypass tracking and Slack outbound hook wiring for cancel/modify behavior. (#15775) Thanks @DarlingtonDeveloper.
    • -
  • Agents: add synthetic catalog support for hf:zai-org/GLM-5. (#15867) Thanks @battman21.
    • -
  • Skills: remove duplicate local-places Google Places skill/proxy and keep goplaces as the single supported Google Places path.
    • -
  • Agents: add pre-prompt context diagnostics (messages, systemPromptChars, promptChars, provider/model, session file) before embedded runner prompt calls to improve overflow debugging. (#8930) Thanks @Glucksberg.
    • +
  • Highlight: External Secrets Management introduces a full openclaw secrets workflow (audit, configure, apply, reload) with runtime snapshot activation, strict secrets apply target-path validation, safer migration scrubbing, ref-only auth-profile support, and dedicated docs. (#26155) Thanks @joshavant.
    • +
  • ACP/Thread-bound agents: make ACP agents first-class runtimes for thread sessions with acp spawn/send dispatch integration, acpx backend bridging, lifecycle controls, startup reconciliation, runtime cleanup, and coalesced thread replies. (#23580) thanks @osolmaz.
    • +
  • Agents/Routing CLI: add openclaw agents bindings, openclaw agents bind, and openclaw agents unbind for account-scoped route management, including channel-only to account-scoped binding upgrades, role-aware binding identity handling, plugin-resolved binding account IDs, and optional account-binding prompts in openclaw channels add. (#27195) thanks @gumadeiras.
    • +
  • Codex/WebSocket transport: make openai-codex WebSocket-first by default (transport: "auto" with SSE fallback), keep explicit per-model/runtime transport overrides, and add regression coverage + docs for transport selection.
    • +
  • Onboarding/Plugins: let channel plugins own interactive onboarding flows with optional configureInteractive and configureWhenConfigured hooks while preserving the generic fallback path. (#27191) thanks @gumadeiras.
    • +
  • Android/Nodes: add Android device capability plus device.status and device.info node commands, including runtime handler wiring and protocol/registry coverage for device status/info payloads. (#27664) Thanks @obviyus.
    • +
  • Android/Nodes: add notifications.list support on Android nodes and expose nodes notifications_list in agent tooling for listing active device notifications. (#27344) thanks @obviyus.
    • +
  • Docs/Contributing: add Nimrod Gutman to the maintainer roster in CONTRIBUTING.md. (#27840) Thanks @ngutman.

Fixes

    -
  • Outbound: add a write-ahead delivery queue with crash-recovery retries to prevent lost outbound messages after gateway restarts. (#15636) Thanks @nabbilkhan, @thewilloftheshadow.
    • -
  • Auto-reply/Threading: auto-inject implicit reply threading so replyToMode works without requiring model-emitted [[reply_to_current]], while preserving replyToMode: "off" behavior for implicit Slack replies and keeping block-streaming chunk coalescing stable under replyToMode: "first". (#14976) Thanks @Diaspar4u.
    • -
  • Outbound/Threading: pass replyTo and threadId from message send tool actions through the core outbound send path to channel adapters, preserving thread/reply routing. (#14948) Thanks @mcaxtr.
    • -
  • Auto-reply/Media: allow image-only inbound messages (no caption) to reach the agent instead of short-circuiting as empty text, and preserve thread context in queued/followup prompt bodies for media-only runs. (#11916) Thanks @arosstale.
    • -
  • Discord: route autoThread replies to existing threads instead of the root channel. (#8302) Thanks @gavinbmoore, @thewilloftheshadow.
    • -
  • Web UI: add img to DOMPurify allowed tags and src/alt to allowed attributes so markdown images render in webchat instead of being stripped. (#15437) Thanks @lailoo.
    • -
  • Telegram/Matrix: treat MP3 and M4A (including audio/mp4) as voice-compatible for asVoice routing, and keep WAV/AAC falling back to regular audio sends. (#15438) Thanks @azade-c.
    • -
  • WhatsApp: preserve outbound document filenames for web-session document sends instead of always sending "file". (#15594) Thanks @TsekaLuk.
    • -
  • Telegram: cap bot menu registration to Telegram's 100-command limit with an overflow warning while keeping typed hidden commands available. (#15844) Thanks @battman21.
    • -
  • Telegram: scope skill commands to the resolved agent for default accounts so setMyCommands no longer triggers BOT_COMMANDS_TOO_MUCH when multiple agents are configured. (#15599)
    • -
  • Discord: avoid misrouting numeric guild allowlist entries to /channels/ by prefixing guild-only inputs with guild: during resolution. (#12326) Thanks @headswim.
    • -
  • MS Teams: preserve parsed mention entities/text when appending OneDrive fallback file links, and accept broader real-world Teams mention ID formats (29:..., 8:orgid:...) while still rejecting placeholder patterns. (#15436) Thanks @hyojin.
    • -
  • Media: classify text/* MIME types as documents in media-kind routing so text attachments are no longer treated as unknown. (#12237) Thanks @arosstale.
    • -
  • Inbound/Web UI: preserve literal \n sequences when normalizing inbound text so Windows paths like C:\\Work\\nxxx\\README.md are not corrupted. (#11547) Thanks @mcaxtr.
    • -
  • TUI/Streaming: preserve richer streamed assistant text when final payload drops pre-tool-call text blocks, while keeping non-empty final payload authoritative for plain-text updates. (#15452) Thanks @TsekaLuk.
    • -
  • Providers/MiniMax: switch implicit MiniMax API-key provider from openai-completions to anthropic-messages with the correct Anthropic-compatible base URL, fixing invalid role: developer (2013) errors on MiniMax M2.5. (#15275) Thanks @lailoo.
    • -
  • Ollama/Agents: use resolved model/provider base URLs for native /api/chat streaming (including aliased providers), normalize /v1 endpoints, and forward abort + maxTokens stream options for reliable cancellation and token caps. (#11853) Thanks @BrokenFinger98.
    • -
  • OpenAI Codex/Spark: implement end-to-end gpt-5.3-codex-spark support across fallback/thinking/model resolution and models list forward-compat visibility. (#14990, #15174) Thanks @L-U-C-K-Y, @loiie45e.
    • -
  • Agents/Codex: allow gpt-5.3-codex-spark in forward-compat fallback, live model filtering, and thinking presets, and fix model-picker recognition for spark. (#14990) Thanks @L-U-C-K-Y.
    • -
  • Models/Codex: resolve configured openai-codex/gpt-5.3-codex-spark through forward-compat fallback during models list, so it is not incorrectly tagged as missing when runtime resolution succeeds. (#15174) Thanks @loiie45e.
    • -
  • OpenAI Codex/Auth: bridge OpenClaw OAuth profiles into pi auth.json so model discovery and models-list registry resolution can use Codex OAuth credentials. (#15184) Thanks @loiie45e.
    • -
  • Auth/OpenAI Codex: share OAuth login handling across onboarding and models auth login --provider openai-codex, keep onboarding alive when OAuth fails, and surface a direct OAuth help note instead of terminating the wizard. (#15406, follow-up to #14552) Thanks @zhiluo20.
    • -
  • Onboarding/Providers: add vLLM as an onboarding provider with model discovery, auth profile wiring, and non-interactive auth-choice validation. (#12577) Thanks @gejifeng.
    • -
  • Onboarding/Providers: preserve Hugging Face auth intent in auth-choice remapping (tokenProvider=huggingface with authChoice=apiKey) and skip env-override prompts when an explicit token is provided. (#13472) Thanks @Josephrp.
    • -
  • Onboarding/CLI: restore terminal state without resuming paused stdin, so onboarding exits cleanly after choosing Web UI and the installer returns instead of appearing stuck.
    • -
  • Signal/Install: auto-install signal-cli via Homebrew on non-x64 Linux architectures, avoiding x86_64 native binary Exec format error failures on arm64/arm hosts. (#15443) Thanks @jogvan-k.
    • -
  • macOS Voice Wake: fix a crash in trigger trimming for CJK/Unicode transcripts by matching and slicing on original-string ranges instead of transformed-string indices. (#11052) Thanks @Flash-LHR.
    • -
  • Mattermost (plugin): retry websocket monitor connections with exponential backoff and abort-aware teardown so transient connect failures no longer permanently stop monitoring. (#14962) Thanks @mcaxtr.
    • -
  • Discord/Agents: apply channel/group historyLimit during embedded-runner history compaction to prevent long-running channel sessions from bypassing truncation and overflowing context windows. (#11224) Thanks @shadril238.
    • -
  • Outbound targets: fail closed for WhatsApp/Twitch/Google Chat fallback paths so invalid or missing targets are dropped instead of rerouted, and align resolver hints with strict target requirements. (#13578) Thanks @mcaxtr.
    • -
  • Gateway/Restart: clear stale command-queue and heartbeat wake runtime state after SIGUSR1 in-process restarts to prevent zombie gateway behavior where queued work stops draining. (#15195) Thanks @joeykrug.
    • -
  • Heartbeat: prevent scheduler silent-death races during runner reloads, preserve retry cooldown backoff under wake bursts, and prioritize user/action wake causes over interval/retry reasons when coalescing. (#15108) Thanks @joeykrug.
    • -
  • Heartbeat: allow explicit wake (wake) and hook wake (hook:*) reasons to run even when HEARTBEAT.md is effectively empty so queued system events are processed. (#14527) Thanks @arosstale.
    • -
  • Auto-reply/Heartbeat: strip sentence-ending HEARTBEAT_OK tokens even when followed by up to 4 punctuation characters, while preserving surrounding sentence punctuation. (#15847) Thanks @Spacefish.
    • -
  • Agents/Heartbeat: stop auto-creating HEARTBEAT.md during workspace bootstrap so missing files continue to run heartbeat as documented. (#11766) Thanks @shadril238.
    • -
  • Sessions/Agents: pass agentId when resolving existing transcript paths in reply runs so non-default agents and heartbeat/chat handlers no longer fail with Session file path must be within sessions directory. (#15141) Thanks @Goldenmonstew.
    • -
  • Sessions/Agents: pass agentId through status and usage transcript-resolution paths (auto-reply, gateway usage APIs, and session cost/log loaders) so non-default agents can resolve absolute session files without path-validation failures. (#15103) Thanks @jalehman.
    • -
  • Sessions: archive previous transcript files on /new and /reset session resets (including gateway sessions.reset) so stale transcripts do not accumulate on disk. (#14869) Thanks @mcaxtr.
    • -
  • Status/Sessions: stop clamping derived totalTokens to context-window size, keep prompt-token snapshots wired through session accounting, and surface context usage as unknown when fresh snapshot data is missing to avoid false 100% reports. (#15114) Thanks @echoVic.
    • -
  • CLI/Completion: route plugin-load logs to stderr and write generated completion scripts directly to stdout to avoid source <(openclaw completion ...) corruption. (#15481) Thanks @arosstale.
    • -
  • CLI: lazily load outbound provider dependencies and remove forced success-path exits so commands terminate naturally without killing intentional long-running foreground actions. (#12906) Thanks @DrCrinkle.
    • -
  • Security/Gateway + ACP: block high-risk tools (sessions_spawn, sessions_send, gateway, whatsapp_login) from HTTP /tools/invoke by default with gateway.tools.{allow,deny} overrides, and harden ACP permission selection to fail closed when tool identity/options are ambiguous while supporting allow_always/reject_always. (#15390) Thanks @aether-ai-agent.
    • -
  • Security/Gateway: breaking default-behavior change - canvas IP-based auth fallback now only accepts machine-scoped addresses (RFC1918, link-local, ULA IPv6, CGNAT); public-source IP matches now require bearer token auth. (#14661) Thanks @sumleo.
    • -
  • Security/Link understanding: block loopback/internal host patterns and private/mapped IPv6 addresses in extracted URL handling to close SSRF bypasses in link CLI flows. (#15604) Thanks @AI-Reviewer-QS.
    • -
  • Security/Browser: constrain POST /trace/stop, POST /wait/download, and POST /download output paths to OpenClaw temp roots and reject traversal/escape paths.
    • -
  • Security/Canvas: serve A2UI assets via the shared safe-open path (openFileWithinRoot) to close traversal/TOCTOU gaps, with traversal and symlink regression coverage. (#10525) Thanks @abdelsfane.
    • -
  • Security/WhatsApp: enforce 0o600 on creds.json and creds.json.bak on save/backup/restore paths to reduce credential file exposure. (#10529) Thanks @abdelsfane.
    • -
  • Security/Gateway: sanitize and truncate untrusted WebSocket header values in pre-handshake close logs to reduce log-poisoning risk. Thanks @thewilloftheshadow.
    • -
  • Security/Audit: add misconfiguration checks for sandbox Docker config with sandbox mode off, ineffective gateway.nodes.denyCommands entries, global minimal tool-profile overrides by agent profiles, and permissive extension-plugin tool reachability.
    • -
  • Security/Audit: distinguish external webhooks (hooks.enabled) from internal hooks (hooks.internal.enabled) in attack-surface summaries to avoid false exposure signals when only internal hooks are enabled. (#13474) Thanks @mcaxtr.
    • -
  • Security/Onboarding: clarify multi-user DM isolation remediation with explicit openclaw config set session.dmScope ... commands in security audit, doctor security, and channel onboarding guidance. (#13129) Thanks @VintLin.
    • -
  • Agents/Nodes: harden node exec approval decision handling in the nodes tool run path by failing closed on unexpected approval decisions, and add regression coverage for approval-required retry/deny/timeout flows. (#4726) Thanks @rmorse.
    • -
  • Android/Nodes: harden app.update by requiring HTTPS and gateway-host URL matching plus SHA-256 verification, stream URL camera downloads to disk with size guards to avoid memory spikes, and stop signing release builds with debug keys. (#13541) Thanks @smartprogrammer93.
    • -
  • Routing: enforce strict binding-scope matching across peer/guild/team/roles so peer-scoped Discord/Slack bindings no longer match unrelated guild/team contexts or fallback tiers. (#15274) Thanks @lailoo.
    • -
  • Exec/Allowlist: allow multiline heredoc bodies (<<, <<-) while keeping multiline non-heredoc shell commands blocked, so exec approval parsing permits heredoc input safely without allowing general newline command chaining. (#13811) Thanks @mcaxtr.
    • -
  • Config: preserve ${VAR} env references when writing config files so openclaw config set/apply/patch does not persist secrets to disk. Thanks @thewilloftheshadow.
    • -
  • Config: remove a cross-request env-snapshot race in config writes by carrying read-time env context into write calls per request, preserving ${VAR} refs safely under concurrent gateway config mutations. (#11560) Thanks @akoscz.
    • -
  • Config: log overwrite audit entries (path, backup target, and hash transition) whenever an existing config file is replaced, improving traceability for unexpected config clobbers.
    • -
  • Config: keep legacy audio transcription migration strict by rejecting non-string/unsafe command tokens while still migrating valid custom script executables. (#5042) Thanks @shayan919293.
    • -
  • Config: accept $schema key in config file so JSON Schema editor tooling works without validation errors. (#14998)
    • -
  • Gateway/Tools Invoke: sanitize /tools/invoke execution failures while preserving 400 for tool input errors and returning 500 for unexpected runtime failures, with regression coverage and docs updates. (#13185) Thanks @davidrudduck.
    • -
  • Gateway/Hooks: preserve 408 for hook request-body timeout responses while keeping bounded auth-failure cache eviction behavior, with timeout-status regression coverage. (#15848) Thanks @AI-Reviewer-QS.
    • -
  • Plugins/Hooks: fire before_tool_call hook exactly once per tool invocation in embedded runs by removing duplicate dispatch paths while preserving parameter mutation semantics. (#15635) Thanks @lailoo.
    • -
  • Agents/Transcript policy: sanitize OpenAI/Codex tool-call ids during transcript policy normalization to prevent invalid tool-call identifiers from propagating into session history. (#15279) Thanks @divisonofficer.
    • -
  • Agents/Image tool: cap image-analysis completion maxTokens by model capability (min(4096, model.maxTokens)) to avoid over-limit provider failures while still preventing truncation. (#11770) Thanks @detecti1.
    • -
  • Agents/Compaction: centralize exec default resolution in the shared tool factory so per-agent tools.exec overrides (host/security/ask/node and related defaults) persist across compaction retries. (#15833) Thanks @napetrov.
    • -
  • Gateway/Agents: stop injecting a phantom main agent into gateway agent listings when agents.list explicitly excludes it. (#11450) Thanks @arosstale.
    • -
  • Process/Exec: avoid shell execution for .exe commands on Windows so env overrides work reliably in runCommandWithTimeout. Thanks @thewilloftheshadow.
    • -
  • Daemon/Windows: preserve literal backslashes in gateway.cmd command parsing so drive and UNC paths are not corrupted in runtime checks and doctor entrypoint comparisons. (#15642) Thanks @arosstale.
    • -
  • Sandbox: pass configured sandbox.docker.env variables to sandbox containers at docker create time. (#15138) Thanks @stevebot-alive.
    • -
  • Voice Call: route webhook runtime event handling through shared manager event logic so rejected inbound hangups are idempotent in production, with regression tests for duplicate reject events and provider-call-ID remapping parity. (#15892) Thanks @dcantu96.
    • -
  • Cron: add regression coverage for announce-mode isolated jobs so runs that already report delivered: true do not enqueue duplicate main-session relays, including delivery configs where mode is omitted and defaults to announce. (#15737) Thanks @brandonwise.
    • -
  • Cron: honor deleteAfterRun in isolated announce delivery by mapping it to subagent announce cleanup mode, so cron run sessions configured for deletion are removed after completion. (#15368) Thanks @arosstale.
    • -
  • Web tools/web_fetch: prefer text/markdown responses for Cloudflare Markdown for Agents, add cf-markdown extraction for markdown bodies, and redact fetched URLs in x-markdown-tokens debug logs to avoid leaking raw paths/query params. (#15376) Thanks @Yaxuan42.
    • -
  • Clawdock: avoid Zsh readonly variable collisions in helper scripts. (#15501) Thanks @nkelner.
    • -
  • Memory: switch default local embedding model to the QAT embeddinggemma-300m-qat-Q8_0 variant for better quality at the same footprint. (#15429) Thanks @azade-c.
    • -
  • Docs/Mermaid: remove hardcoded Mermaid init theme blocks from four docs diagrams so dark mode inherits readable theme defaults. (#15157) Thanks @heytulsiprasad.
    • +
  • Telegram/DM allowlist runtime inheritance: enforce dmPolicy: "allowlist" allowFrom requirements using effective account-plus-parent config across account-capable channels (Telegram, Discord, Slack, Signal, iMessage, IRC, BlueBubbles, WhatsApp), and align openclaw doctor checks to the same inheritance logic so DM traffic is not silently dropped after upgrades. (#27936) Thanks @widingmarcus-cyber.
    • +
  • Delivery queue/recovery backoff: prevent retry starvation by persisting lastAttemptAt on failed sends and deferring recovery retries until each entry's lastAttemptAt + backoff window is eligible, while continuing to recover ready entries behind deferred ones. Landed from contributor PR #27710 by @Jimmy-xuzimo. Thanks @Jimmy-xuzimo.
    • +
  • Google Chat/Lifecycle: keep Google Chat startAccount pending until abort in webhook mode so startup is no longer interpreted as immediate exit, preventing auto-restart loops and webhook-target churn. (#27384) thanks @junsuwhy.
    • +
  • Temp dirs/Linux umask: force 0700 permissions after temp-dir creation and self-heal existing writable temp dirs before trust checks so umask 0002 installs no longer crash-loop on startup. Landed from contributor PR #27860 by @stakeswky. (#27853) Thanks @stakeswky.
    • +
  • Nextcloud Talk/Lifecycle: keep startAccount pending until abort and stop the webhook monitor on shutdown, preventing EADDRINUSE restart loops when the gateway manages account lifecycle. (#27897)
    • +
  • Microsoft Teams/File uploads: acknowledge fileConsent/invoke immediately (invokeResponse before upload + file card send) so Teams no longer shows false "Something went wrong" timeout banners while upload completion continues asynchronously; includes updated async regression coverage. Landed from contributor PR #27641 by @scz2011.
    • +
  • Queue/Drain/Cron reliability: harden lane draining with guaranteed draining flag reset on synchronous pump failures, reject new queue enqueues during gateway restart drain windows (instead of silently killing accepted tasks), add /stop queued-backlog cutoff metadata with stale-message skipping (while avoiding cross-session native-stop cutoff bleed), and raise isolated cron agentTurn outer safety timeout to avoid false 10-minute timeout races against longer agent session timeouts. (#27407, #27332, #27427)
    • +
  • Typing/Main reply pipeline: always mark dispatch idle in agent-runner finalization so typing cleanup runs even when dispatcher onIdle does not fire, preventing stuck typing indicators after run completion. (#27250) Thanks @Sid-Qin.
    • +
  • Typing/TTL safety net: add max-duration guardrails to shared typing callbacks so stuck lifecycle edges auto-stop typing indicators even when explicit idle/cleanup signals are missed. (#27428) Thanks @Crpdim.
    • +
  • Typing/Cross-channel leakage: unify run-scoped typing suppression for cross-channel/internal-webchat routes, preserve current inbound origin as embedded run message channel context, harden shared typing keepalive with consecutive-failure circuit breaker edge-case handling, and enforce dispatcher completion/idle waits in extension dispatcher callsites (Feishu, Matrix, Mattermost, MSTeams) so typing indicators always clean up on success/error paths. Related: #27647, #27493, #27598. Supersedes/replaces draft PRs: #27640, #27593, #27540.
    • +
  • Telegram/sendChatAction 401 handling: add bounded exponential backoff + temporary local typing suppression after repeated unauthorized failures to stop unbounded sendChatAction retry loops that can trigger Telegram abuse enforcement and bot deletion. (#27415) Thanks @widingmarcus-cyber.
    • +
  • Telegram/Webhook startup: clarify webhook config guidance, allow channels.telegram.webhookPort: 0 for ephemeral listener binding, and log both the local listener URL and Telegram-advertised webhook URL with the bound port. (#25732) thanks @huntharo.
    • +
  • Browser/Chrome extension handshake: bind relay WS message handling before onopen and add non-blocking connect.challenge response handling for gateway-style handshake frames, avoiding stuck … badge states when challenge frames arrive immediately on connect. Landed from contributor PR #22571 by @pandego. (#22553)
    • +
  • Browser/Extension relay init: dedupe concurrent same-port relay startup with shared in-flight initialization promises so callers await one startup lifecycle and receive consistent success/failure results. Landed from contributor PR #21277 by @HOYALIM. (Related #20688)
    • +
  • Browser/Fill relay + CLI parity: accept act.fill fields without explicit type by defaulting missing/empty type to text in both browser relay route parsing and openclaw browser fill CLI field parsing, so relay calls no longer fail when the model omits field type metadata. Landed from contributor PR #27662 by @Uface11. (#27296) Thanks @Uface11.
    • +
  • Feishu/Permission error dispatch: merge sender-name permission notices into the main inbound dispatch so one user message produces one agent turn/reply (instead of a duplicate permission-notice turn), with regression coverage. (#27381) thanks @byungsker.
    • +
  • Agents/Canvas default node resolution: when multiple connected canvas-capable nodes exist and no single mac-* candidate is selected, default to the first connected candidate instead of failing with node required for implicit-node canvas tool calls. Landed from contributor PR #27444 by @carbaj03. Thanks @carbaj03.
    • +
  • TUI/stream assembly: preserve streamed text across real tool-boundary drops without keeping stale streamed text when non-text blocks appear only in the final payload. Landed from contributor PR #27711 by @scz2011. (#27674)
    • +
  • Hooks/Internal message:sent: forward sessionKey on outbound sends from agent delivery, cron isolated delivery, gateway receipt acks, heartbeat sends, session-maintenance warnings, and restart-sentinel recovery so internal message:sent hooks consistently dispatch with session context, including openclaw agent --deliver runs resumed via --session-id (without explicit --session-key). Landed from contributor PR #27584 by @qualiobra. Thanks @qualiobra.
    • +
  • Pi image-token usage: stop re-injecting history image blocks each turn, process image references from the current prompt only, and prune already-answered user-image blocks in stored history to prevent runaway token growth. (#27602)
    • +
  • BlueBubbles/SSRF: auto-allowlist the configured serverUrl hostname for attachment fetches so localhost/private-IP BlueBubbles setups are no longer false-blocked by default SSRF checks. Landed from contributor PR #27648 by @lailoo. (#27599) Thanks @taylorhou for reporting.
    • +
  • Agents/Compaction + onboarding safety: prevent destructive double-compaction by stripping stale assistant usage around compaction boundaries, skipping post-compaction custom metadata writes in the same attempt, and cancelling safeguard compaction when there are no real conversation messages to summarize; harden workspace/bootstrap detection for memory-backed workspaces; and change openclaw onboard --reset default scope to config+creds+sessions (workspace deletion now requires --reset-scope full). (#26458, #27314) Thanks @jaden-clovervnd, @Sid-Qin, and @widingmarcus-cyber for fix direction in #26502, #26529, and #27492.
    • +
  • NO_REPLY suppression: suppress NO_REPLY before Slack API send and in sub-agent announce completion flow so sentinel text no longer leaks into user channels. Landed from contributor PRs #27529 (by @Sid-Qin) and #27535 (rewritten minimal landing by maintainers). (#27387, #27531)
    • +
  • Matrix/Group sender identity: preserve sender labels in Matrix group inbound prompt text (BodyForAgent) for both channel and threaded messages, and align group envelopes with shared inbound sender-prefix formatting so first-person requests resolve against the current sender. (#27401) thanks @koushikxd.
    • +
  • Auto-reply/Streaming: suppress only exact NO_REPLY final replies while still filtering streaming partial sentinel fragments (NO_, NO_RE, HEARTBEAT_...) so substantive replies ending with NO_REPLY are delivered and partial silent tokens do not leak during streaming. (#19576) Thanks @aldoeliacim.
    • +
  • Auto-reply/Inbound metadata: add a readable timestamp field to conversation info and ignore invalid/out-of-range timestamp values so prompt assembly never crashes on malformed timestamp inputs. (#17017) thanks @liuy.
    • +
  • Typing/Run completion race: prevent post-run keepalive ticks from re-triggering typing callbacks by guarding triggerTyping() with runComplete, with regression coverage for no-restart behavior during run-complete/dispatch-idle boundaries. (#27413) Thanks @widingmarcus-cyber.
    • +
  • Typing/Dispatch idle: force typing cleanup when markDispatchIdle never arrives after run completion, avoiding leaked typing keepalive loops in cron/announce edges. Landed from contributor PR #27541 by @Sid-Qin. (#27493)
    • +
  • Telegram/Inline buttons: allow callback-query button handling in groups (including /models follow-up buttons) when group policy authorizes the sender, by removing the redundant callback allowlist gate that blocked open-policy groups. (#27343) Thanks @GodsBoy.
    • +
  • Telegram/Streaming preview: when finalizing without an existing preview message, prime pending preview text with final answer before stop-flush so users do not briefly see stale 1-2 word fragments (for example no before no problem). (#27449) Thanks @emanuelst for the original fix direction in #19673.
    • +
  • Browser/Extension relay CORS: handle /json* OPTIONS preflight before auth checks, allow Chrome extension origins, and return extension-origin CORS headers on relay HTTP responses so extension token validation no longer fails cross-origin. Landed from contributor PR #23962 by @miloudbelarebia. (#23842)
    • +
  • Browser/Extension relay auth: allow ?token= query-param auth on relay /json* endpoints (consistent with relay WebSocket auth) so curl/devtools-style /json/version and /json/list probes work without requiring custom headers. Landed from contributor PR #26015 by @Sid-Qin. (#25928)
    • +
  • Browser/Extension relay shutdown: flush pending extension-request timers/rejections during relay stop() before socket/server teardown so in-flight extension waits do not survive shutdown windows. Landed from contributor PR #24142 by @kevinWangSheng.
    • +
  • Browser/Extension relay reconnect resilience: keep CDP clients alive across brief MV3 extension disconnect windows, wait briefly for extension reconnect before failing in-flight CDP commands, and only tear down relay target/client state after reconnect grace expires. Landed from contributor PR #27617 by @davidemanuelDEV.
    • +
  • Browser/Route decode hardening: guard malformed percent-encoding in relay target action routes and browser route-param decoding so crafted % paths return 400 instead of crashing/unhandled URI decode failures. Landed from contributor PR #11880 by @Yida-Dev.
    • +
  • Feishu/Inbound message metadata: include inbound message_id in BodyForAgent on a dedicated metadata line so agents can reliably correlate and act on media/message operations that require message IDs, with regression coverage. (#27253) thanks @xss925175263.
    • +
  • Feishu/Doc tools: route feishu_doc and feishu_app_scopes through the active agent account context (with explicit accountId override support) so multi-account agents no longer default to the first configured app, with regression coverage for context routing and explicit override behavior. (#27338) thanks @AaronL725.
    • +
  • LINE/Inline directives auth: gate directive parsing (/model, /think, /verbose, /reasoning, /queue) on resolved authorization (command.isAuthorizedSender) so commands.allowFrom-authorized LINE senders are not silently stripped when raw CommandAuthorized is unset. Landed from contributor PR #27248 by @kevinWangSheng. (#27240)
    • +
  • Onboarding/Gateway: seed default Control UI allowedOrigins for non-loopback binds during onboarding (localhost/127.0.0.1 plus custom bind host) so fresh non-loopback setups do not fail startup due to missing origin policy. (#26157) thanks @stakeswky.
    • +
  • Docker/GCP onboarding: reduce first-build OOM risk by capping Node heap during pnpm install, reuse existing gateway token during docker-setup.sh reruns so .env stays aligned with config, auto-bootstrap Control UI allowed origins for non-loopback Docker binds, and add GCP docs guidance for tokenized dashboard links + pairing recovery commands. (#26253) Thanks @pandego.
    • +
  • CLI/Gateway --force in non-root Docker: recover from lsof permission failures (EACCES/EPERM) by falling back to fuser kill + probe-based port checks, so openclaw gateway --force works for default container node user flows. (#27941)
    • +
  • Gateway/Bind visibility: emit a startup warning when binding to non-loopback addresses so operators get explicit exposure guidance in runtime logs. (#25397) thanks @let5sne.
    • +
  • Sessions cleanup/Doctor: add openclaw sessions cleanup --fix-missing to prune store entries whose transcript files are missing, including doctor guidance and CLI coverage. Landed from contributor PR #27508 by @Sid-Qin. (#27422)
    • +
  • Doctor/State integrity: ignore metadata-only slash routing sessions when checking recent missing transcripts so openclaw doctor no longer reports false-positive transcript-missing warnings for *:slash:* keys. (#27375) thanks @gumadeiras.
    • +
  • CLI/Gateway status: force local gateway status probe host to 127.0.0.1 for bind=lan so co-located probes do not trip non-loopback plaintext WebSocket checks. (#26997) thanks @chikko80.
    • +
  • CLI/Gateway auth: align gateway run --auth parsing/help text with supported gateway auth modes by accepting none and trusted-proxy (in addition to token/password) for CLI overrides. (#27469) thanks @s1korrrr.
    • +
  • CLI/Daemon status TLS probe: use wss:// and forward local TLS certificate fingerprint for TLS-enabled gateway daemon probes so openclaw daemon status works with gateway.bind=lan + gateway.tls.enabled=true. (#24234) thanks @liuy.
    • +
  • Podman/Default bind: change run-openclaw-podman.sh default gateway bind from lan to loopback and document explicit LAN opt-in with Control UI origin configuration. (#27491) thanks @robbyczgw-cla.
    • +
  • Daemon/macOS launchd: forward proxy env vars into supervised service environments, keep LaunchAgent KeepAlive=true semantics, and harden restart sequencing to print -> bootout -> wait old pid exit -> bootstrap -> kickstart. (#27276) thanks @frankekn.
    • +
  • Gateway/macOS restart-loop hardening: detect OpenClaw-managed supervisor markers during SIGUSR1 restart handoff, clean stale gateway PIDs before /restart launchctl/systemctl triggers, and set LaunchAgent ThrottleInterval=60 to bound launchd retry storms during lock-release races. Landed from contributor PRs #27655 (@taw0002), #27448 (@Sid-Qin), and #27650 (@kevinWangSheng). (#27605, #27590, #26904, #26736)
    • +
  • Models/MiniMax auth header defaults: set authHeader: true for both onboarding-generated MiniMax API providers and implicit built-in MiniMax (minimax, minimax-portal) provider templates so first requests no longer fail with MiniMax 401 authentication_error due to missing Authorization header. Landed from contributor PRs #27622 by @riccoyuanft and #27631 by @kevinWangSheng. (#27600, #15303)
    • +
  • Auth/Auth profiles: normalize auth-profiles.json alias fields (mode -> type, apiKey -> key) before credential validation so entries copied from openclaw.json auth examples are no longer silently dropped. (#26950) thanks @byungsker.
    • +
  • Models/Profile suffix parsing: centralize trailing @profile parsing and only treat @ as a profile separator when it appears after the final /, preserving model IDs like openai/@cf/... and openrouter/@preset/... across /model directive parsing and allowlist model resolution, with regression coverage.
    • +
  • Models/OpenAI Codex config schema parity: accept openai-codex-responses in the config model API schema and TypeScript ModelApi union, with regression coverage for config validation. Landed from contributor PR #27501 by @AytuncYildizli. Thanks @AytuncYildizli.
    • +
  • Agents/Models config: preserve agent-level provider apiKey and baseUrl during merge-mode models.json updates when agent values are present. (#27293) thanks @Sid-Qin.
    • +
  • Azure OpenAI Responses: force store=true for azure-openai-responses direct responses API calls to avoid multi-turn 400 failures. Landed from contributor PR #27499 by @polarbear-Yang. (#27497)
    • +
  • Security/Node exec approvals: require structured commandArgv approvals for host=node, enforce versioned systemRunBindingV1 matching for argv/cwd/session/agent/env context with fail-closed behavior on missing/mismatched bindings, and add GIT_EXTERNAL_DIFF to blocked host env keys. This ships in the next npm release (2026.2.26). Thanks @tdjackey for reporting.
    • +
  • Security/Plugin channel HTTP auth: normalize protected /api/channels path checks against canonicalized request paths (case + percent-decoding + slash normalization), resolve encoded dot-segment traversal variants, and fail closed on malformed %-encoded channel prefixes so alternate-path variants cannot bypass gateway auth. This ships in the next npm release (2026.2.26). Thanks @zpbrent for reporting.
    • +
  • Security/Gateway node pairing: pin paired-device platform/deviceFamily metadata across reconnects and bind those fields into device-auth signatures, so reconnect metadata spoofing cannot expand node command allowlists without explicit repair pairing. This ships in the next npm release (2026.2.26). Thanks @76embiid21 for reporting.
    • +
  • Security/Sandbox path alias guard: reject broken symlink targets by resolving through existing ancestors and failing closed on out-of-root targets, preventing workspace-only apply_patch writes from escaping sandbox/workspace boundaries via dangling symlinks. This ships in the next npm release (2026.2.26). Thanks @tdjackey for reporting.
    • +
  • Security/Workspace FS boundary aliases: harden canonical boundary resolution for non-existent-leaf symlink aliases while preserving valid in-root aliases, preventing first-write workspace escapes via out-of-root symlink targets. This ships in the next npm release (2026.2.26). Thanks @tdjackey for reporting.
    • +
  • Security/Config includes: harden $include file loading with verified-open reads, reject hardlinked include aliases, and enforce include file-size guardrails so config include resolution remains bounded to trusted in-root files. This ships in the next npm release (2026.2.26). Thanks @zpbrent for reporting.
    • +
  • Security/Node exec approvals hardening: freeze immutable approval-time execution plans (argv/cwd/agentId/sessionKey) via system.run.prepare, enforce those canonical plan values during approval forwarding/execution, and reject mutable parent-symlink cwd paths during approval-plan building to prevent approval bypass via symlink rebind. This ships in the next npm release (2026.2.26). Thanks @tdjackey for reporting.
    • +
  • Security/Microsoft Teams media fetch: route Graph message/hosted-content/attachment fetches and auth-scope fallback attachment downloads through shared SSRF-guarded fetch paths, and centralize hostname-suffix allowlist policy helpers in the plugin SDK to remove channel/plugin drift. This ships in the next npm release (2026.2.26). Thanks @tdjackey for reporting.
    • +
  • Security/Voice Call (Twilio): bind webhook replay + manager dedupe identity to authenticated request material, remove unsigned i-twilio-idempotency-token trust from replay/dedupe keys, and thread verified request identity through provider parse flow to harden cross-provider event dedupe. This ships in the next npm release (2026.2.26). Thanks @tdjackey for reporting.
    • +
  • Security/Exec approvals forwarding: prefer turn-source channel/account/thread metadata when resolving approval delivery targets so stale session routes do not misroute approval prompts.
    • +
  • Security/Pairing multi-account isolation: enforce account-scoped pairing allowlists and pending-request storage across core + extension message channels while preserving channel-scoped defaults for the default account. This ships in the next npm release (2026.2.26). Thanks @tdjackey for reporting and @gumadeiras for implementation.
    • +
  • Config/Plugins entries: treat unknown plugins.entries.* ids as startup warnings (ignored stale keys) instead of hard validation failures that can crash-loop gateway boot. Landed from contributor PR #27506 by @Sid-Qin. (#27455)
    • +
  • Telegram native commands: degrade command registration on BOT_COMMANDS_TOO_MUCH by retrying with fewer commands instead of crash-looping startup sync. Landed from contributor PR #27512 by @Sid-Qin. (#27456)
    • +
  • Web tools/Proxy: route web_search provider HTTP calls (Brave, Perplexity, xAI, Gemini, Kimi), redirect resolution, and web_fetch through a shared proxy-aware SSRF guard path so gateway installs behind HTTP_PROXY/HTTPS_PROXY/ALL_PROXY no longer fail with transport fetch failed errors. (#27430) thanks @kevinWangSheng.
    • +
  • Android/Node invoke: remove native gateway WebSocket Origin header to avoid false origin rejections, unify invoke command registry/policy/error parsing paths, and keep command availability checks centralized to reduce dispatcher/advertisement drift. (#27257) Thanks @obviyus.
    • +
  • Gateway shared-auth scopes: preserve requested operator scopes for shared-token clients when device identity is unavailable, instead of clearing scopes during auth handling. Landed from contributor PR #27498 by @kevinWangSheng. (#27494)
    • +
  • Cron/Hooks isolated routing: preserve canonical agent:* session keys in isolated runs so already-qualified keys are not double-prefixed (for example agent:main:main no longer becomes agent:main:agent:main:main). Landed from contributor PR #27333 by @MaheshBhushan. (#27289, #27282)
    • +
  • Channels/Multi-account config: when adding a non-default channel account to a single-account top-level channel setup, move existing account-scoped top-level single-account values into channels..accounts.default before writing the new account so the original account keeps working without duplicated account values at channel root; openclaw doctor --fix now repairs previously mixed channel account shapes the same way. (#27334) thanks @gumadeiras.
    • +
  • iOS/Talk mode: stop injecting the voice directive hint into iOS Talk prompts and remove the Voice Directive Hint setting, reducing model bias toward tool-style TTS directives and keeping relay responses text-first by default. (#27543) thanks @ngutman.
    • +
  • CI/Windows: shard the Windows checks-windows test lane into two matrix jobs and honor explicit shard index overrides in scripts/test-parallel.mjs to reduce CI critical-path wall time. (#27234) Thanks @joshavant.

View full changelog

]]> - + - \ No newline at end of file + diff --git a/apps/android/README.md b/apps/android/README.md index c2ae5a2179b..941f500701a 100644 --- a/apps/android/README.md +++ b/apps/android/README.md @@ -1,13 +1,26 @@ -## OpenClaw Node (Android) (internal) +## OpenClaw Android App -Modern Android node app: connects to the **Gateway WebSocket** (`_openclaw-gw._tcp`) and exposes **Canvas + Chat + Camera**. +Status: **extremely alpha**. The app is actively being rebuilt from the ground up. -Notes: -- The node keeps the connection alive via a **foreground service** (persistent notification with a Disconnect action). -- Chat always uses the shared session key **`main`** (same session across iOS/macOS/WebChat/Android). -- Supports modern Android only (`minSdk 31`, Kotlin + Jetpack Compose). +### Rebuild Checklist + +- [x] New 4-step onboarding flow +- [x] Connect tab with `Setup Code` + `Manual` modes +- [x] Encrypted persistence for gateway setup/auth state +- [x] Chat UI restyled +- [x] Settings UI restyled and de-duplicated (gateway controls moved to Connect) +- [ ] QR code scanning in onboarding +- [ ] Performance improvements +- [ ] Streaming support in chat UI +- [ ] Request camera/location and other permissions in onboarding/settings flow +- [ ] Push notifications for gateway/chat status updates +- [ ] Security hardening (biometric lock, token handling, safer defaults) +- [ ] Voice tab full functionality +- [ ] Screen tab full functionality +- [ ] Full end-to-end QA and release hardening ## Open in Android Studio + - Open the folder `apps/android`. ## Build / Run @@ -21,18 +34,105 @@ cd apps/android `gradlew` auto-detects the Android SDK at `~/Library/Android/sdk` (macOS default) if `ANDROID_SDK_ROOT` / `ANDROID_HOME` are unset. +## Macrobenchmark (Startup + Frame Timing) + +```bash +cd apps/android +./gradlew :benchmark:connectedDebugAndroidTest +``` + +Reports are written under: + +- `apps/android/benchmark/build/reports/androidTests/connected/` + +## Perf CLI (low-noise) + +Deterministic startup measurement + hotspot extraction with compact CLI output: + +```bash +cd apps/android +./scripts/perf-startup-benchmark.sh +./scripts/perf-startup-hotspots.sh +``` + +Benchmark script behavior: + +- Runs only `StartupMacrobenchmark#coldStartup` (10 iterations). +- Prints median/min/max/COV in one line. +- Writes timestamped snapshot JSON to `apps/android/benchmark/results/`. +- Auto-compares with previous local snapshot (or pass explicit baseline: `--baseline `). + +Hotspot script behavior: + +- Ensures debug app installed, captures startup `simpleperf` data for `.MainActivity`. +- Prints top DSOs, top symbols, and key app-path clues (Compose/MainActivity/WebView). +- Writes raw `perf.data` path for deeper follow-up if needed. + +## Run on a Real Android Phone (USB) + +1) On phone, enable **Developer options** + **USB debugging**. +2) Connect by USB and accept the debugging trust prompt on phone. +3) Verify ADB can see the device: + +```bash +adb devices -l +``` + +4) Install + launch debug build: + +```bash +pnpm android:install +pnpm android:run +``` + +If `adb devices -l` shows `unauthorized`, re-plug and accept the trust prompt again. + +### USB-only gateway testing (no LAN dependency) + +Use `adb reverse` so Android `localhost:18789` tunnels to your laptop `localhost:18789`. + +Terminal A (gateway): + +```bash +pnpm openclaw gateway --port 18789 --verbose +``` + +Terminal B (USB tunnel): + +```bash +adb reverse tcp:18789 tcp:18789 +``` + +Then in app **Connect → Manual**: + +- Host: `127.0.0.1` +- Port: `18789` +- TLS: off + +## Hot Reload / Fast Iteration + +This app is native Kotlin + Jetpack Compose. + +- For Compose UI edits: use Android Studio **Live Edit** on a debug build (works on physical devices; project `minSdk=31` already meets API requirement). +- For many non-structural code/resource changes: use Android Studio **Apply Changes**. +- For structural/native/manifest/Gradle changes: do full reinstall (`pnpm android:run`). +- Canvas web content already supports live reload when loaded from Gateway `__openclaw__/canvas/` (see `docs/platforms/android.md`). + ## Connect / Pair -1) Start the gateway (on your “master” machine): +1) Start the gateway (on your main machine): + ```bash pnpm openclaw gateway --port 18789 --verbose ``` 2) In the Android app: -- Open **Settings** -- Either select a discovered gateway under **Discovered Gateways**, or use **Advanced → Manual Gateway** (host + port). + +- Open the **Connect** tab. +- Use **Setup Code** or **Manual** mode to connect. 3) Approve pairing (on the gateway machine): + ```bash openclaw nodes pending openclaw nodes approve @@ -49,3 +149,58 @@ More details: `docs/platforms/android.md`. - Camera: - `CAMERA` for `camera.snap` and `camera.clip` - `RECORD_AUDIO` for `camera.clip` when `includeAudio=true` + +## Integration Capability Test (Preconditioned) + +This suite assumes setup is already done manually. It does **not** install/run/pair automatically. + +Pre-req checklist: + +1) Gateway is running and reachable from the Android app. +2) Android app is connected to that gateway and `openclaw nodes status` shows it as paired + connected. +3) App stays unlocked and in foreground for the whole run. +4) Open the app **Screen** tab and keep it active during the run (canvas/A2UI commands require the canvas WebView attached there). +5) Grant runtime permissions for capabilities you expect to pass (camera/mic/location/notification listener/location, etc.). +6) No interactive system dialogs should be pending before test start. +7) Canvas host is enabled and reachable from the device (do not run gateway with `OPENCLAW_SKIP_CANVAS_HOST=1`; startup logs should include `canvas host mounted at .../__openclaw__/`). +8) Local operator test client pairing is approved. If first run fails with `pairing required`, approve latest pending device pairing request, then rerun: +9) For A2UI checks, keep the app on **Screen** tab; the node now auto-refreshes canvas capability once on first A2UI reachability failure (TTL-safe retry). + +```bash +openclaw devices list +openclaw devices approve --latest +``` + +Run: + +```bash +pnpm android:test:integration +``` + +Optional overrides: + +- `OPENCLAW_ANDROID_GATEWAY_URL=ws://...` (default: from your local OpenClaw config) +- `OPENCLAW_ANDROID_GATEWAY_TOKEN=...` +- `OPENCLAW_ANDROID_GATEWAY_PASSWORD=...` +- `OPENCLAW_ANDROID_NODE_ID=...` or `OPENCLAW_ANDROID_NODE_NAME=...` + +What it does: + +- Reads `node.describe` command list from the selected Android node. +- Invokes advertised non-interactive commands. +- Skips `screen.record` in this suite (Android requires interactive per-invocation screen-capture consent). +- Asserts command contracts (success or expected deterministic error for safe-invalid calls like `sms.send`, `notifications.actions`, `app.update`). + +Common failure quick-fixes: + +- `pairing required` before tests start: + - approve pending device pairing (`openclaw devices approve --latest`) and rerun. +- `A2UI host not reachable` / `A2UI_HOST_NOT_CONFIGURED`: + - ensure gateway canvas host is running and reachable, keep the app on the **Screen** tab. The app will auto-refresh canvas capability once; if it still fails, reconnect app and rerun. +- `NODE_BACKGROUND_UNAVAILABLE: canvas unavailable`: + - app is not effectively ready for canvas commands; keep app foregrounded and **Screen** tab active. + +## Contributions + +This Android app is currently being rebuilt. +Maintainer: @obviyus. For issues/questions/contributions, please open an issue or reach out on Discord. diff --git a/apps/android/THIRD_PARTY_LICENSES/MANROPE_OFL.txt b/apps/android/THIRD_PARTY_LICENSES/MANROPE_OFL.txt new file mode 100644 index 00000000000..472064afc4b --- /dev/null +++ b/apps/android/THIRD_PARTY_LICENSES/MANROPE_OFL.txt @@ -0,0 +1,93 @@ +Copyright 2018 The Manrope Project Authors (https://github.com/sharanda/manrope) + +This Font Software is licensed under the SIL Open Font License, Version 1.1. +This license is copied below, and is also available with a FAQ at: +http://scripts.sil.org/OFL + + +----------------------------------------------------------- +SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007 +----------------------------------------------------------- + +PREAMBLE +The goals of the Open Font License (OFL) are to stimulate worldwide +development of collaborative font projects, to support the font creation +efforts of academic and linguistic communities, and to provide a free and +open framework in which fonts may be shared and improved in partnership +with others. + +The OFL allows the licensed fonts to be used, studied, modified and +redistributed freely as long as they are not sold by themselves. The +fonts, including any derivative works, can be bundled, embedded, +redistributed and/or sold with any software provided that any reserved +names are not used by derivative works. The fonts and derivatives, +however, cannot be released under any other type of license. The +requirement for fonts to remain under this license does not apply +to any document created using the fonts or their derivatives. + +DEFINITIONS +"Font Software" refers to the set of files released by the Copyright +Holder(s) under this license and clearly marked as such. This may +include source files, build scripts and documentation. + +"Reserved Font Name" refers to any names specified as such after the +copyright statement(s). + +"Original Version" refers to the collection of Font Software components as +distributed by the Copyright Holder(s). + +"Modified Version" refers to any derivative made by adding to, deleting, +or substituting -- in part or in whole -- any of the components of the +Original Version, by changing formats or by porting the Font Software to a +new environment. + +"Author" refers to any designer, engineer, programmer, technical +writer or other person who contributed to the Font Software. + +PERMISSION & CONDITIONS +Permission is hereby granted, free of charge, to any person obtaining +a copy of the Font Software, to use, study, copy, merge, embed, modify, +redistribute, and sell modified and unmodified copies of the Font +Software, subject to the following conditions: + +1) Neither the Font Software nor any of its individual components, +in Original or Modified Versions, may be sold by itself. + +2) Original or Modified Versions of the Font Software may be bundled, +redistributed and/or sold with any software, provided that each copy +contains the above copyright notice and this license. These can be +included either as stand-alone text files, human-readable headers or +in the appropriate machine-readable metadata fields within text or +binary files as long as those fields can be easily viewed by the user. + +3) No Modified Version of the Font Software may use the Reserved Font +Name(s) unless explicit written permission is granted by the corresponding +Copyright Holder. This restriction only applies to the primary font name as +presented to the users. + +4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font +Software shall not be used to promote, endorse or advertise any +Modified Version, except to acknowledge the contribution(s) of the +Copyright Holder(s) and the Author(s) or with their explicit written +permission. + +5) The Font Software, modified or unmodified, in part or in whole, +must be distributed entirely under this license, and must not be +distributed under any other license. The requirement for fonts to +remain under this license does not apply to any document created +using the Font Software. + +TERMINATION +This license becomes null and void if any of the above conditions are +not met. + +DISCLAIMER +THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT +OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE +COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, +INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL +DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING +FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM +OTHER DEALINGS IN THE FONT SOFTWARE. diff --git a/apps/android/app/build.gradle.kts b/apps/android/app/build.gradle.kts index c96187b2dc5..8897587f839 100644 --- a/apps/android/app/build.gradle.kts +++ b/apps/android/app/build.gradle.kts @@ -2,7 +2,6 @@ import com.android.build.api.variant.impl.VariantOutputImpl plugins { id("com.android.application") - id("org.jetbrains.kotlin.android") id("org.jetbrains.kotlin.plugin.compose") id("org.jetbrains.kotlin.plugin.serialization") } @@ -13,7 +12,7 @@ android { sourceSets { getByName("main") { - assets.srcDir(file("../../shared/OpenClawKit/Sources/OpenClawKit/Resources")) + assets.directories.add("../../shared/OpenClawKit/Sources/OpenClawKit/Resources") } } @@ -21,8 +20,8 @@ android { applicationId = "ai.openclaw.android" minSdk = 31 targetSdk = 36 - versionCode = 202602180 - versionName = "2026.2.18" + versionCode = 202602270 + versionName = "2026.2.27" ndk { // Support all major ABIs — native libs are tiny (~47 KB per ABI) abiFilters += listOf("armeabi-v7a", "arm64-v8a", "x86", "x86_64") @@ -97,7 +96,7 @@ kotlin { } dependencies { - val composeBom = platform("androidx.compose:compose-bom:2025.12.00") + val composeBom = platform("androidx.compose:compose-bom:2026.02.00") implementation(composeBom) androidTestImplementation(composeBom) @@ -112,7 +111,7 @@ dependencies { // material-icons-extended pulled in full icon set (~20 MB DEX). Only ~18 icons used. // R8 will tree-shake unused icons when minify is enabled on release builds. implementation("androidx.compose.material:material-icons-extended") - implementation("androidx.navigation:navigation-compose:2.9.6") + implementation("androidx.navigation:navigation-compose:2.9.7") debugImplementation("androidx.compose.ui:ui-tooling") @@ -120,12 +119,17 @@ dependencies { implementation("com.google.android.material:material:1.13.0") implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.10.2") - implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.9.0") + implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.10.0") implementation("androidx.security:security-crypto:1.1.0") implementation("androidx.exifinterface:exifinterface:1.4.2") implementation("com.squareup.okhttp3:okhttp:5.3.2") implementation("org.bouncycastle:bcprov-jdk18on:1.83") + implementation("org.commonmark:commonmark:0.27.1") + implementation("org.commonmark:commonmark-ext-autolink:0.27.1") + implementation("org.commonmark:commonmark-ext-gfm-strikethrough:0.27.1") + implementation("org.commonmark:commonmark-ext-gfm-tables:0.27.1") + implementation("org.commonmark:commonmark-ext-task-list-items:0.27.1") // CameraX (for node.invoke camera.* parity) implementation("androidx.camera:camera-core:1.5.2") @@ -133,15 +137,17 @@ dependencies { implementation("androidx.camera:camera-lifecycle:1.5.2") implementation("androidx.camera:camera-video:1.5.2") implementation("androidx.camera:camera-view:1.5.2") + implementation("com.journeyapps:zxing-android-embedded:4.3.0") // Unicast DNS-SD (Wide-Area Bonjour) for tailnet discovery domains. implementation("dnsjava:dnsjava:3.6.4") testImplementation("junit:junit:4.13.2") testImplementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:1.10.2") - testImplementation("io.kotest:kotest-runner-junit5-jvm:6.0.7") - testImplementation("io.kotest:kotest-assertions-core-jvm:6.0.7") - testImplementation("org.robolectric:robolectric:4.16") + testImplementation("io.kotest:kotest-runner-junit5-jvm:6.1.3") + testImplementation("io.kotest:kotest-assertions-core-jvm:6.1.3") + testImplementation("com.squareup.okhttp3:mockwebserver:5.3.2") + testImplementation("org.robolectric:robolectric:4.16.1") testRuntimeOnly("org.junit.vintage:junit-vintage-engine:6.0.2") } diff --git a/apps/android/app/src/main/AndroidManifest.xml b/apps/android/app/src/main/AndroidManifest.xml index facdbf301b4..f5e20fd5a97 100644 --- a/apps/android/app/src/main/AndroidManifest.xml +++ b/apps/android/app/src/main/AndroidManifest.xml @@ -15,6 +15,15 @@ + + + + + + + + + + + + diff --git a/apps/android/app/src/main/java/ai/openclaw/android/MainActivity.kt b/apps/android/app/src/main/java/ai/openclaw/android/MainActivity.kt index 2bbfd8712f9..b90427672c6 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/MainActivity.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/MainActivity.kt @@ -1,20 +1,13 @@ package ai.openclaw.android -import android.Manifest -import android.content.pm.ApplicationInfo import android.os.Bundle -import android.os.Build import android.view.WindowManager -import android.webkit.WebView import androidx.activity.ComponentActivity import androidx.activity.compose.setContent import androidx.activity.viewModels +import androidx.core.view.WindowCompat import androidx.compose.material3.Surface import androidx.compose.ui.Modifier -import androidx.core.content.ContextCompat -import androidx.core.view.WindowCompat -import androidx.core.view.WindowInsetsCompat -import androidx.core.view.WindowInsetsControllerCompat import androidx.lifecycle.Lifecycle import androidx.lifecycle.lifecycleScope import androidx.lifecycle.repeatOnLifecycle @@ -29,12 +22,7 @@ class MainActivity : ComponentActivity() { override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) - val isDebuggable = (applicationInfo.flags and ApplicationInfo.FLAG_DEBUGGABLE) != 0 - WebView.setWebContentsDebuggingEnabled(isDebuggable) - applyImmersiveMode() - requestDiscoveryPermissionsIfNeeded() - requestNotificationPermissionIfNeeded() - NodeForegroundService.start(this) + WindowCompat.setDecorFitsSystemWindows(window, false) permissionRequester = PermissionRequester(this) screenCaptureRequester = ScreenCaptureRequester(this) viewModel.camera.attachLifecycleOwner(this) @@ -62,18 +50,9 @@ class MainActivity : ComponentActivity() { } } } - } - override fun onResume() { - super.onResume() - applyImmersiveMode() - } - - override fun onWindowFocusChanged(hasFocus: Boolean) { - super.onWindowFocusChanged(hasFocus) - if (hasFocus) { - applyImmersiveMode() - } + // Keep startup path lean: start foreground service after first frame. + window.decorView.post { NodeForegroundService.start(this) } } override fun onStart() { @@ -85,46 +64,4 @@ class MainActivity : ComponentActivity() { viewModel.setForeground(false) super.onStop() } - - private fun applyImmersiveMode() { - WindowCompat.setDecorFitsSystemWindows(window, false) - val controller = WindowInsetsControllerCompat(window, window.decorView) - controller.systemBarsBehavior = - WindowInsetsControllerCompat.BEHAVIOR_SHOW_TRANSIENT_BARS_BY_SWIPE - controller.hide(WindowInsetsCompat.Type.systemBars()) - } - - private fun requestDiscoveryPermissionsIfNeeded() { - if (Build.VERSION.SDK_INT >= 33) { - val ok = - ContextCompat.checkSelfPermission( - this, - Manifest.permission.NEARBY_WIFI_DEVICES, - ) == android.content.pm.PackageManager.PERMISSION_GRANTED - if (!ok) { - requestPermissions(arrayOf(Manifest.permission.NEARBY_WIFI_DEVICES), 100) - } - } else { - val ok = - ContextCompat.checkSelfPermission( - this, - Manifest.permission.ACCESS_FINE_LOCATION, - ) == android.content.pm.PackageManager.PERMISSION_GRANTED - if (!ok) { - requestPermissions(arrayOf(Manifest.permission.ACCESS_FINE_LOCATION), 101) - } - } - } - - private fun requestNotificationPermissionIfNeeded() { - if (Build.VERSION.SDK_INT < 33) return - val ok = - ContextCompat.checkSelfPermission( - this, - Manifest.permission.POST_NOTIFICATIONS, - ) == android.content.pm.PackageManager.PERMISSION_GRANTED - if (!ok) { - requestPermissions(arrayOf(Manifest.permission.POST_NOTIFICATIONS), 102) - } - } } diff --git a/apps/android/app/src/main/java/ai/openclaw/android/MainViewModel.kt b/apps/android/app/src/main/java/ai/openclaw/android/MainViewModel.kt index d9123d10293..9cb7d626ce7 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/MainViewModel.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/MainViewModel.kt @@ -8,12 +8,17 @@ import ai.openclaw.android.node.CameraCaptureManager import ai.openclaw.android.node.CanvasController import ai.openclaw.android.node.ScreenRecordManager import ai.openclaw.android.node.SmsManager +import ai.openclaw.android.voice.VoiceConversationEntry import kotlinx.coroutines.flow.StateFlow class MainViewModel(app: Application) : AndroidViewModel(app) { private val runtime: NodeRuntime = (app as NodeApp).runtime val canvas: CanvasController = runtime.canvas + val canvasCurrentUrl: StateFlow = runtime.canvas.currentUrl + val canvasA2uiHydrated: StateFlow = runtime.canvasA2uiHydrated + val canvasRehydratePending: StateFlow = runtime.canvasRehydratePending + val canvasRehydrateErrorText: StateFlow = runtime.canvasRehydrateErrorText val camera: CameraCaptureManager = runtime.camera val screenRecorder: ScreenRecordManager = runtime.screenRecorder val sms: SmsManager = runtime.sms @@ -22,6 +27,7 @@ class MainViewModel(app: Application) : AndroidViewModel(app) { val discoveryStatusText: StateFlow = runtime.discoveryStatusText val isConnected: StateFlow = runtime.isConnected + val isNodeConnected: StateFlow = runtime.nodeConnected val statusText: StateFlow = runtime.statusText val serverName: StateFlow = runtime.serverName val remoteAddress: StateFlow = runtime.remoteAddress @@ -40,19 +46,21 @@ class MainViewModel(app: Application) : AndroidViewModel(app) { val locationMode: StateFlow = runtime.locationMode val locationPreciseEnabled: StateFlow = runtime.locationPreciseEnabled val preventSleep: StateFlow = runtime.preventSleep - val wakeWords: StateFlow> = runtime.wakeWords - val voiceWakeMode: StateFlow = runtime.voiceWakeMode - val voiceWakeStatusText: StateFlow = runtime.voiceWakeStatusText - val voiceWakeIsListening: StateFlow = runtime.voiceWakeIsListening - val talkEnabled: StateFlow = runtime.talkEnabled - val talkStatusText: StateFlow = runtime.talkStatusText - val talkIsListening: StateFlow = runtime.talkIsListening - val talkIsSpeaking: StateFlow = runtime.talkIsSpeaking + val micEnabled: StateFlow = runtime.micEnabled + val micStatusText: StateFlow = runtime.micStatusText + val micLiveTranscript: StateFlow = runtime.micLiveTranscript + val micIsListening: StateFlow = runtime.micIsListening + val micQueuedMessages: StateFlow> = runtime.micQueuedMessages + val micConversation: StateFlow> = runtime.micConversation + val micInputLevel: StateFlow = runtime.micInputLevel + val micIsSending: StateFlow = runtime.micIsSending + val speakerEnabled: StateFlow = runtime.speakerEnabled val manualEnabled: StateFlow = runtime.manualEnabled val manualHost: StateFlow = runtime.manualHost val manualPort: StateFlow = runtime.manualPort val manualTls: StateFlow = runtime.manualTls val gatewayToken: StateFlow = runtime.gatewayToken + val onboardingCompleted: StateFlow = runtime.onboardingCompleted val canvasDebugStatusEnabled: StateFlow = runtime.canvasDebugStatusEnabled val chatSessionKey: StateFlow = runtime.chatSessionKey @@ -110,24 +118,24 @@ class MainViewModel(app: Application) : AndroidViewModel(app) { runtime.setGatewayToken(value) } + fun setGatewayPassword(value: String) { + runtime.setGatewayPassword(value) + } + + fun setOnboardingCompleted(value: Boolean) { + runtime.setOnboardingCompleted(value) + } + fun setCanvasDebugStatusEnabled(value: Boolean) { runtime.setCanvasDebugStatusEnabled(value) } - fun setWakeWords(words: List) { - runtime.setWakeWords(words) + fun setMicEnabled(enabled: Boolean) { + runtime.setMicEnabled(enabled) } - fun resetWakeWordsDefaults() { - runtime.resetWakeWordsDefaults() - } - - fun setVoiceWakeMode(mode: VoiceWakeMode) { - runtime.setVoiceWakeMode(mode) - } - - fun setTalkEnabled(enabled: Boolean) { - runtime.setTalkEnabled(enabled) + fun setSpeakerEnabled(enabled: Boolean) { + runtime.setSpeakerEnabled(enabled) } fun refreshGatewayConnection() { @@ -158,6 +166,10 @@ class MainViewModel(app: Application) : AndroidViewModel(app) { runtime.handleCanvasA2UIActionFromWebView(payloadJson) } + fun requestCanvasRehydrate(source: String = "screen_tab") { + runtime.requestCanvasRehydrate(source = source, force = true) + } + fun loadChat(sessionKey: String) { runtime.loadChat(sessionKey) } diff --git a/apps/android/app/src/main/java/ai/openclaw/android/NodeApp.kt b/apps/android/app/src/main/java/ai/openclaw/android/NodeApp.kt index 2be9ee71a2c..ab5e159cf47 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/NodeApp.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/NodeApp.kt @@ -2,23 +2,12 @@ package ai.openclaw.android import android.app.Application import android.os.StrictMode -import android.util.Log -import java.security.Security class NodeApp : Application() { val runtime: NodeRuntime by lazy { NodeRuntime(this) } override fun onCreate() { super.onCreate() - // Register Bouncy Castle as highest-priority provider for Ed25519 support - try { - val bcProvider = Class.forName("org.bouncycastle.jce.provider.BouncyCastleProvider") - .getDeclaredConstructor().newInstance() as java.security.Provider - Security.removeProvider("BC") - Security.insertProviderAt(bcProvider, 1) - } catch (it: Throwable) { - Log.e("NodeApp", "Failed to register Bouncy Castle provider", it) - } if (BuildConfig.DEBUG) { StrictMode.setThreadPolicy( StrictMode.ThreadPolicy.Builder() diff --git a/apps/android/app/src/main/java/ai/openclaw/android/NodeForegroundService.kt b/apps/android/app/src/main/java/ai/openclaw/android/NodeForegroundService.kt index ee7c8e00674..a6a79dc9c4a 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/NodeForegroundService.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/NodeForegroundService.kt @@ -39,22 +39,22 @@ class NodeForegroundService : Service() { runtime.statusText, runtime.serverName, runtime.isConnected, - runtime.voiceWakeMode, - runtime.voiceWakeIsListening, - ) { status, server, connected, voiceMode, voiceListening -> - Quint(status, server, connected, voiceMode, voiceListening) - }.collect { (status, server, connected, voiceMode, voiceListening) -> + runtime.micEnabled, + runtime.micIsListening, + ) { status, server, connected, micEnabled, micListening -> + Quint(status, server, connected, micEnabled, micListening) + }.collect { (status, server, connected, micEnabled, micListening) -> val title = if (connected) "OpenClaw Node · Connected" else "OpenClaw Node" - val voiceSuffix = - if (voiceMode == VoiceWakeMode.Always) { - if (voiceListening) " · Voice Wake: Listening" else " · Voice Wake: Paused" + val micSuffix = + if (micEnabled) { + if (micListening) " · Mic: Listening" else " · Mic: Pending" } else { "" } - val text = (server?.let { "$status · $it" } ?: status) + voiceSuffix + val text = (server?.let { "$status · $it" } ?: status) + micSuffix val requiresMic = - voiceMode == VoiceWakeMode.Always && hasRecordAudioPermission() + micEnabled && hasRecordAudioPermission() startForegroundWithTypes( notification = buildNotification(title = title, text = text), requiresMic = requiresMic, diff --git a/apps/android/app/src/main/java/ai/openclaw/android/NodeRuntime.kt b/apps/android/app/src/main/java/ai/openclaw/android/NodeRuntime.kt index aec192c25bb..f462413669b 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/NodeRuntime.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/NodeRuntime.kt @@ -4,6 +4,7 @@ import android.Manifest import android.content.Context import android.content.pm.PackageManager import android.os.SystemClock +import android.util.Log import androidx.core.content.ContextCompat import ai.openclaw.android.chat.ChatController import ai.openclaw.android.chat.ChatMessage @@ -18,8 +19,9 @@ import ai.openclaw.android.gateway.GatewaySession import ai.openclaw.android.gateway.probeGatewayTlsFingerprint import ai.openclaw.android.node.* import ai.openclaw.android.protocol.OpenClawCanvasA2UIAction +import ai.openclaw.android.voice.MicCaptureManager import ai.openclaw.android.voice.TalkModeManager -import ai.openclaw.android.voice.VoiceWakeManager +import ai.openclaw.android.voice.VoiceConversationEntry import kotlinx.coroutines.CoroutineScope import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.Job @@ -36,6 +38,7 @@ import kotlinx.serialization.json.JsonArray import kotlinx.serialization.json.JsonObject import kotlinx.serialization.json.JsonPrimitive import kotlinx.serialization.json.buildJsonObject +import java.util.UUID import java.util.concurrent.atomic.AtomicLong class NodeRuntime(context: Context) { @@ -53,40 +56,6 @@ class NodeRuntime(context: Context) { private val externalAudioCaptureActive = MutableStateFlow(false) - private val voiceWake: VoiceWakeManager by lazy { - VoiceWakeManager( - context = appContext, - scope = scope, - onCommand = { command -> - nodeSession.sendNodeEvent( - event = "agent.request", - payloadJson = - buildJsonObject { - put("message", JsonPrimitive(command)) - put("sessionKey", JsonPrimitive(resolveMainSessionKey())) - put("thinking", JsonPrimitive(chatThinkingLevel.value)) - put("deliver", JsonPrimitive(false)) - }.toString(), - ) - }, - ) - } - - val voiceWakeIsListening: StateFlow - get() = voiceWake.isListening - - val voiceWakeStatusText: StateFlow - get() = voiceWake.statusText - - val talkStatusText: StateFlow - get() = talkMode.statusText - - val talkIsListening: StateFlow - get() = talkMode.isListening - - val talkIsSpeaking: StateFlow - get() = talkMode.isSpeaking - private val discovery = GatewayDiscovery(appContext, scope = scope) val gateways: StateFlow> = discovery.gateways val discoveryStatusText: StateFlow = discovery.statusText @@ -97,8 +66,6 @@ class NodeRuntime(context: Context) { private val cameraHandler: CameraHandler = CameraHandler( appContext = appContext, camera = camera, - prefs = prefs, - connectedEndpoint = { connectedEndpoint }, externalAudioCaptureActive = externalAudioCaptureActive, showCameraHud = ::showCameraHud, triggerCameraFlash = ::triggerCameraFlash, @@ -124,6 +91,34 @@ class NodeRuntime(context: Context) { locationPreciseEnabled = { locationPreciseEnabled.value }, ) + private val deviceHandler: DeviceHandler = DeviceHandler( + appContext = appContext, + ) + + private val notificationsHandler: NotificationsHandler = NotificationsHandler( + appContext = appContext, + ) + + private val systemHandler: SystemHandler = SystemHandler( + appContext = appContext, + ) + + private val photosHandler: PhotosHandler = PhotosHandler( + appContext = appContext, + ) + + private val contactsHandler: ContactsHandler = ContactsHandler( + appContext = appContext, + ) + + private val calendarHandler: CalendarHandler = CalendarHandler( + appContext = appContext, + ) + + private val motionHandler: MotionHandler = MotionHandler( + appContext = appContext, + ) + private val screenHandler: ScreenHandler = ScreenHandler( screenRecorder = screenRecorder, setScreenRecordActive = { _screenRecordActive.value = it }, @@ -145,7 +140,9 @@ class NodeRuntime(context: Context) { prefs = prefs, cameraEnabled = { cameraEnabled.value }, locationMode = { locationMode.value }, - voiceWakeMode = { voiceWakeMode.value }, + voiceWakeMode = { VoiceWakeMode.Off }, + motionActivityAvailable = { motionHandler.isActivityAvailable() }, + motionPedometerAvailable = { motionHandler.isPedometerAvailable() }, smsAvailable = { sms.canSendSms() }, hasRecordAudioPermission = { hasRecordAudioPermission() }, manualTls = { manualTls.value }, @@ -155,6 +152,13 @@ class NodeRuntime(context: Context) { canvas = canvas, cameraHandler = cameraHandler, locationHandler = locationHandler, + deviceHandler = deviceHandler, + notificationsHandler = notificationsHandler, + systemHandler = systemHandler, + photosHandler = photosHandler, + contactsHandler = contactsHandler, + calendarHandler = calendarHandler, + motionHandler = motionHandler, screenHandler = screenHandler, smsHandler = smsHandlerImpl, a2uiHandler = a2uiHandler, @@ -163,10 +167,19 @@ class NodeRuntime(context: Context) { isForeground = { _isForeground.value }, cameraEnabled = { cameraEnabled.value }, locationEnabled = { locationMode.value != LocationMode.Off }, + smsAvailable = { sms.canSendSms() }, + debugBuild = { BuildConfig.DEBUG }, + refreshNodeCanvasCapability = { nodeSession.refreshNodeCanvasCapability() }, + onCanvasA2uiPush = { + _canvasA2uiHydrated.value = true + _canvasRehydratePending.value = false + _canvasRehydrateErrorText.value = null + }, + onCanvasA2uiReset = { _canvasA2uiHydrated.value = false }, + motionActivityAvailable = { motionHandler.isActivityAvailable() }, + motionPedometerAvailable = { motionHandler.isPedometerAvailable() }, ) - private lateinit var gatewayEventHandler: GatewayEventHandler - data class GatewayTrustPrompt( val endpoint: GatewayEndpoint, val fingerprintSha256: String, @@ -174,6 +187,8 @@ class NodeRuntime(context: Context) { private val _isConnected = MutableStateFlow(false) val isConnected: StateFlow = _isConnected.asStateFlow() + private val _nodeConnected = MutableStateFlow(false) + val nodeConnected: StateFlow = _nodeConnected.asStateFlow() private val _statusText = MutableStateFlow("Offline") val statusText: StateFlow = _statusText.asStateFlow() @@ -194,6 +209,13 @@ class NodeRuntime(context: Context) { private val _screenRecordActive = MutableStateFlow(false) val screenRecordActive: StateFlow = _screenRecordActive.asStateFlow() + private val _canvasA2uiHydrated = MutableStateFlow(false) + val canvasA2uiHydrated: StateFlow = _canvasA2uiHydrated.asStateFlow() + private val _canvasRehydratePending = MutableStateFlow(false) + val canvasRehydratePending: StateFlow = _canvasRehydratePending.asStateFlow() + private val _canvasRehydrateErrorText = MutableStateFlow(null) + val canvasRehydrateErrorText: StateFlow = _canvasRehydrateErrorText.asStateFlow() + private val _serverName = MutableStateFlow(null) val serverName: StateFlow = _serverName.asStateFlow() @@ -207,8 +229,9 @@ class NodeRuntime(context: Context) { val isForeground: StateFlow = _isForeground.asStateFlow() private var lastAutoA2uiUrl: String? = null + private var didAutoRequestCanvasRehydrate = false + private val canvasRehydrateSeq = AtomicLong(0) private var operatorConnected = false - private var nodeConnected = false private var operatorStatusText: String = "Offline" private var nodeStatusText: String = "Offline" @@ -225,8 +248,13 @@ class NodeRuntime(context: Context) { _seamColorArgb.value = DEFAULT_SEAM_COLOR_ARGB applyMainSessionKey(mainSessionKey) updateStatus() - scope.launch { refreshBrandingFromGateway() } - scope.launch { gatewayEventHandler.refreshWakeWordsFromGateway() } + micCapture.onGatewayConnectionChanged(true) + scope.launch { + refreshBrandingFromGateway() + if (voiceReplySpeakerLazy.isInitialized()) { + voiceReplySpeaker.refreshConfig() + } + } }, onDisconnected = { message -> operatorConnected = false @@ -237,11 +265,10 @@ class NodeRuntime(context: Context) { if (!isCanonicalMainSessionKey(_mainSessionKey.value)) { _mainSessionKey.value = "main" } - val mainKey = resolveMainSessionKey() - talkMode.setMainSessionKey(mainKey) - chat.applyMainSessionKey(mainKey) + chat.applyMainSessionKey(resolveMainSessionKey()) chat.onDisconnected(message) updateStatus() + micCapture.onGatewayConnectionChanged(false) }, onEvent = { event, payloadJson -> handleGatewayEvent(event, payloadJson) @@ -254,14 +281,22 @@ class NodeRuntime(context: Context) { identityStore = identityStore, deviceAuthStore = deviceAuthStore, onConnected = { _, _, _ -> - nodeConnected = true + _nodeConnected.value = true nodeStatusText = "Connected" + didAutoRequestCanvasRehydrate = false + _canvasA2uiHydrated.value = false + _canvasRehydratePending.value = false + _canvasRehydrateErrorText.value = null updateStatus() maybeNavigateToA2uiOnConnect() }, onDisconnected = { message -> - nodeConnected = false + _nodeConnected.value = false nodeStatusText = message + didAutoRequestCanvasRehydrate = false + _canvasA2uiHydrated.value = false + _canvasRehydratePending.value = false + _canvasRehydrateErrorText.value = null updateStatus() showLocalCanvasOnDisconnect() }, @@ -274,6 +309,14 @@ class NodeRuntime(context: Context) { }, ) + init { + DeviceNotificationListenerService.setNodeEventSink { event, payloadJson -> + scope.launch { + nodeSession.sendNodeEvent(event = event, payloadJson = payloadJson) + } + } + } + private val chat: ChatController = ChatController( scope = scope, @@ -281,34 +324,93 @@ class NodeRuntime(context: Context) { json = json, supportsChatSubscribe = false, ) - private val talkMode: TalkModeManager by lazy { + private val voiceReplySpeakerLazy: Lazy = lazy { + // Reuse the existing TalkMode speech engine (ElevenLabs + deterministic system-TTS fallback) + // without enabling the legacy talk capture loop. TalkModeManager( context = appContext, scope = scope, session = operatorSession, supportsChatSubscribe = false, isConnected = { operatorConnected }, + ).also { speaker -> + speaker.setPlaybackEnabled(prefs.speakerEnabled.value) + } + } + private val voiceReplySpeaker: TalkModeManager + get() = voiceReplySpeakerLazy.value + + private val micCapture: MicCaptureManager by lazy { + MicCaptureManager( + context = appContext, + scope = scope, + sendToGateway = { message -> + val idempotencyKey = UUID.randomUUID().toString() + val params = + buildJsonObject { + put("sessionKey", JsonPrimitive(resolveMainSessionKey())) + put("message", JsonPrimitive(message)) + put("thinking", JsonPrimitive(chatThinkingLevel.value)) + put("timeoutMs", JsonPrimitive(30_000)) + put("idempotencyKey", JsonPrimitive(idempotencyKey)) + } + val response = operatorSession.request("chat.send", params.toString()) + parseChatSendRunId(response) ?: idempotencyKey + }, + speakAssistantReply = { text -> + voiceReplySpeaker.speakAssistantReply(text) + }, ) } + val micStatusText: StateFlow + get() = micCapture.statusText + + val micLiveTranscript: StateFlow + get() = micCapture.liveTranscript + + val micIsListening: StateFlow + get() = micCapture.isListening + + val micEnabled: StateFlow + get() = micCapture.micEnabled + + val micQueuedMessages: StateFlow> + get() = micCapture.queuedMessages + + val micConversation: StateFlow> + get() = micCapture.conversation + + val micInputLevel: StateFlow + get() = micCapture.inputLevel + + val micIsSending: StateFlow + get() = micCapture.isSending + private fun applyMainSessionKey(candidate: String?) { val trimmed = normalizeMainKey(candidate) ?: return if (isCanonicalMainSessionKey(_mainSessionKey.value)) return if (_mainSessionKey.value == trimmed) return _mainSessionKey.value = trimmed - talkMode.setMainSessionKey(trimmed) chat.applyMainSessionKey(trimmed) } private fun updateStatus() { _isConnected.value = operatorConnected + val operator = operatorStatusText.trim() + val node = nodeStatusText.trim() _statusText.value = when { - operatorConnected && nodeConnected -> "Connected" - operatorConnected && !nodeConnected -> "Connected (node offline)" - !operatorConnected && nodeConnected -> "Connected (operator offline)" - operatorStatusText.isNotBlank() && operatorStatusText != "Offline" -> operatorStatusText - else -> nodeStatusText + operatorConnected && _nodeConnected.value -> "Connected" + operatorConnected && !_nodeConnected.value -> "Connected (node offline)" + !operatorConnected && _nodeConnected.value -> + if (operator.isNotEmpty() && operator != "Offline") { + "Connected (operator: $operator)" + } else { + "Connected (operator offline)" + } + operator.isNotBlank() && operator != "Offline" -> operator + else -> node } } @@ -328,24 +430,78 @@ class NodeRuntime(context: Context) { private fun showLocalCanvasOnDisconnect() { lastAutoA2uiUrl = null + _canvasA2uiHydrated.value = false + _canvasRehydratePending.value = false + _canvasRehydrateErrorText.value = null canvas.navigate("") } + fun requestCanvasRehydrate(source: String = "manual", force: Boolean = true) { + scope.launch { + if (!_nodeConnected.value) { + _canvasRehydratePending.value = false + _canvasRehydrateErrorText.value = "Node offline. Reconnect and retry." + return@launch + } + if (!force && didAutoRequestCanvasRehydrate) return@launch + didAutoRequestCanvasRehydrate = true + val requestId = canvasRehydrateSeq.incrementAndGet() + _canvasRehydratePending.value = true + _canvasRehydrateErrorText.value = null + + val sessionKey = resolveMainSessionKey() + val prompt = + "Restore canvas now for session=$sessionKey source=$source. " + + "If existing A2UI state exists, replay it immediately. " + + "If not, create and render a compact mobile-friendly dashboard in Canvas." + val sent = + nodeSession.sendNodeEvent( + event = "agent.request", + payloadJson = + buildJsonObject { + put("message", JsonPrimitive(prompt)) + put("sessionKey", JsonPrimitive(sessionKey)) + put("thinking", JsonPrimitive("low")) + put("deliver", JsonPrimitive(false)) + }.toString(), + ) + if (!sent) { + if (!force) { + didAutoRequestCanvasRehydrate = false + } + if (canvasRehydrateSeq.get() == requestId) { + _canvasRehydratePending.value = false + _canvasRehydrateErrorText.value = "Failed to request restore. Tap to retry." + } + Log.w("OpenClawCanvas", "canvas rehydrate request failed ($source): transport unavailable") + return@launch + } + scope.launch { + delay(20_000) + if (canvasRehydrateSeq.get() != requestId) return@launch + if (!_canvasRehydratePending.value) return@launch + if (_canvasA2uiHydrated.value) return@launch + _canvasRehydratePending.value = false + _canvasRehydrateErrorText.value = "No canvas update yet. Tap to retry." + } + } + } + val instanceId: StateFlow = prefs.instanceId val displayName: StateFlow = prefs.displayName val cameraEnabled: StateFlow = prefs.cameraEnabled val locationMode: StateFlow = prefs.locationMode val locationPreciseEnabled: StateFlow = prefs.locationPreciseEnabled val preventSleep: StateFlow = prefs.preventSleep - val wakeWords: StateFlow> = prefs.wakeWords - val voiceWakeMode: StateFlow = prefs.voiceWakeMode - val talkEnabled: StateFlow = prefs.talkEnabled val manualEnabled: StateFlow = prefs.manualEnabled val manualHost: StateFlow = prefs.manualHost val manualPort: StateFlow = prefs.manualPort val manualTls: StateFlow = prefs.manualTls val gatewayToken: StateFlow = prefs.gatewayToken + val onboardingCompleted: StateFlow = prefs.onboardingCompleted fun setGatewayToken(value: String) = prefs.setGatewayToken(value) + fun setGatewayPassword(value: String) = prefs.setGatewayPassword(value) + fun setOnboardingCompleted(value: Boolean) = prefs.setOnboardingCompleted(value) val lastDiscoveredStableId: StateFlow = prefs.lastDiscoveredStableId val canvasDebugStatusEnabled: StateFlow = prefs.canvasDebugStatusEnabled @@ -363,50 +519,17 @@ class NodeRuntime(context: Context) { val pendingRunCount: StateFlow = chat.pendingRunCount init { - gatewayEventHandler = GatewayEventHandler( - scope = scope, - prefs = prefs, - json = json, - operatorSession = operatorSession, - isConnected = { _isConnected.value }, - ) - - scope.launch { - combine( - voiceWakeMode, - isForeground, - externalAudioCaptureActive, - wakeWords, - ) { mode, foreground, externalAudio, words -> - Quad(mode, foreground, externalAudio, words) - }.distinctUntilChanged() - .collect { (mode, foreground, externalAudio, words) -> - voiceWake.setTriggerWords(words) - - val shouldListen = - when (mode) { - VoiceWakeMode.Off -> false - VoiceWakeMode.Foreground -> foreground - VoiceWakeMode.Always -> true - } && !externalAudio - - if (!shouldListen) { - voiceWake.stop(statusText = if (mode == VoiceWakeMode.Off) "Off" else "Paused") - return@collect - } - - if (!hasRecordAudioPermission()) { - voiceWake.stop(statusText = "Microphone permission required") - return@collect - } - - voiceWake.start() - } + if (prefs.voiceWakeMode.value != VoiceWakeMode.Off) { + prefs.setVoiceWakeMode(VoiceWakeMode.Off) } scope.launch { - talkEnabled.collect { enabled -> - talkMode.setEnabled(enabled) + prefs.loadGatewayToken() + } + + scope.launch { + prefs.talkEnabled.collect { enabled -> + micCapture.setMicEnabled(enabled) externalAudioCaptureActive.value = enabled } } @@ -514,25 +637,30 @@ class NodeRuntime(context: Context) { prefs.setCanvasDebugStatusEnabled(value) } - fun setWakeWords(words: List) { - prefs.setWakeWords(words) - gatewayEventHandler.scheduleWakeWordsSyncIfNeeded() - } - - fun resetWakeWordsDefaults() { - setWakeWords(SecurePrefs.defaultWakeWords) - } - - fun setVoiceWakeMode(mode: VoiceWakeMode) { - prefs.setVoiceWakeMode(mode) - } - - fun setTalkEnabled(value: Boolean) { + fun setMicEnabled(value: Boolean) { prefs.setTalkEnabled(value) + micCapture.setMicEnabled(value) + externalAudioCaptureActive.value = value + } + + val speakerEnabled: StateFlow + get() = prefs.speakerEnabled + + fun setSpeakerEnabled(value: Boolean) { + prefs.setSpeakerEnabled(value) + if (voiceReplySpeakerLazy.isInitialized()) { + voiceReplySpeaker.setPlaybackEnabled(value) + } } fun refreshGatewayConnection() { - val endpoint = connectedEndpoint ?: return + val endpoint = + connectedEndpoint ?: run { + _statusText.value = "Failed: no cached gateway endpoint" + return + } + operatorStatusText = "Connecting…" + updateStatus() val token = prefs.loadGatewayToken() val password = prefs.loadGatewayPassword() val tls = connectionManager.resolveTlsParams(endpoint) @@ -639,10 +767,10 @@ class NodeRuntime(context: Context) { contextJson = contextJson, ) - val connected = nodeConnected + val connected = _nodeConnected.value var error: String? = null if (connected) { - try { + val sent = nodeSession.sendNodeEvent( event = "agent.request", payloadJson = @@ -654,8 +782,8 @@ class NodeRuntime(context: Context) { put("key", JsonPrimitive(actionId)) }.toString(), ) - } catch (e: Throwable) { - error = e.message ?: "send failed" + if (!sent) { + error = "send failed" } } else { error = "gateway not connected" @@ -705,15 +833,19 @@ class NodeRuntime(context: Context) { } private fun handleGatewayEvent(event: String, payloadJson: String?) { - if (event == "voicewake.changed") { - gatewayEventHandler.handleVoiceWakeChangedEvent(payloadJson) - return - } - - talkMode.handleGatewayEvent(event, payloadJson) + micCapture.handleGatewayEvent(event, payloadJson) chat.handleGatewayEvent(event, payloadJson) } + private fun parseChatSendRunId(response: String): String? { + return try { + val root = json.parseToJsonElement(response).asObjectOrNull() ?: return null + root["runId"].asStringOrNull() + } catch (_: Throwable) { + null + } + } + private suspend fun refreshBrandingFromGateway() { if (!_isConnected.value) return try { diff --git a/apps/android/app/src/main/java/ai/openclaw/android/SecurePrefs.kt b/apps/android/app/src/main/java/ai/openclaw/android/SecurePrefs.kt index 29ef4a3eaae..a907fdf01d4 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/SecurePrefs.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/SecurePrefs.kt @@ -20,19 +20,21 @@ class SecurePrefs(context: Context) { val defaultWakeWords: List = listOf("openclaw", "claude") private const val displayNameKey = "node.displayName" private const val voiceWakeModeKey = "voiceWake.mode" + private const val plainPrefsName = "openclaw.node" + private const val securePrefsName = "openclaw.node.secure" } private val appContext = context.applicationContext private val json = Json { ignoreUnknownKeys = true } + private val plainPrefs: SharedPreferences = + appContext.getSharedPreferences(plainPrefsName, Context.MODE_PRIVATE) - private val masterKey = - MasterKey.Builder(context) + private val masterKey by lazy { + MasterKey.Builder(appContext) .setKeyScheme(MasterKey.KeyScheme.AES256_GCM) .build() - - private val prefs: SharedPreferences by lazy { - createPrefs(appContext, "openclaw.node.secure") } + private val securePrefs: SharedPreferences by lazy { createSecurePrefs(appContext, securePrefsName) } private val _instanceId = MutableStateFlow(loadOrCreateInstanceId()) val instanceId: StateFlow = _instanceId @@ -41,48 +43,51 @@ class SecurePrefs(context: Context) { MutableStateFlow(loadOrMigrateDisplayName(context = context)) val displayName: StateFlow = _displayName - private val _cameraEnabled = MutableStateFlow(prefs.getBoolean("camera.enabled", true)) + private val _cameraEnabled = MutableStateFlow(plainPrefs.getBoolean("camera.enabled", true)) val cameraEnabled: StateFlow = _cameraEnabled private val _locationMode = - MutableStateFlow(LocationMode.fromRawValue(prefs.getString("location.enabledMode", "off"))) + MutableStateFlow(LocationMode.fromRawValue(plainPrefs.getString("location.enabledMode", "off"))) val locationMode: StateFlow = _locationMode private val _locationPreciseEnabled = - MutableStateFlow(prefs.getBoolean("location.preciseEnabled", true)) + MutableStateFlow(plainPrefs.getBoolean("location.preciseEnabled", true)) val locationPreciseEnabled: StateFlow = _locationPreciseEnabled - private val _preventSleep = MutableStateFlow(prefs.getBoolean("screen.preventSleep", true)) + private val _preventSleep = MutableStateFlow(plainPrefs.getBoolean("screen.preventSleep", true)) val preventSleep: StateFlow = _preventSleep private val _manualEnabled = - MutableStateFlow(prefs.getBoolean("gateway.manual.enabled", false)) + MutableStateFlow(plainPrefs.getBoolean("gateway.manual.enabled", false)) val manualEnabled: StateFlow = _manualEnabled private val _manualHost = - MutableStateFlow(prefs.getString("gateway.manual.host", "") ?: "") + MutableStateFlow(plainPrefs.getString("gateway.manual.host", "") ?: "") val manualHost: StateFlow = _manualHost private val _manualPort = - MutableStateFlow(prefs.getInt("gateway.manual.port", 18789)) + MutableStateFlow(plainPrefs.getInt("gateway.manual.port", 18789)) val manualPort: StateFlow = _manualPort private val _manualTls = - MutableStateFlow(prefs.getBoolean("gateway.manual.tls", true)) + MutableStateFlow(plainPrefs.getBoolean("gateway.manual.tls", true)) val manualTls: StateFlow = _manualTls - private val _gatewayToken = - MutableStateFlow(prefs.getString("gateway.manual.token", "") ?: "") + private val _gatewayToken = MutableStateFlow("") val gatewayToken: StateFlow = _gatewayToken + private val _onboardingCompleted = + MutableStateFlow(plainPrefs.getBoolean("onboarding.completed", false)) + val onboardingCompleted: StateFlow = _onboardingCompleted + private val _lastDiscoveredStableId = MutableStateFlow( - prefs.getString("gateway.lastDiscoveredStableID", "") ?: "", + plainPrefs.getString("gateway.lastDiscoveredStableID", "") ?: "", ) val lastDiscoveredStableId: StateFlow = _lastDiscoveredStableId private val _canvasDebugStatusEnabled = - MutableStateFlow(prefs.getBoolean("canvas.debugStatusEnabled", false)) + MutableStateFlow(plainPrefs.getBoolean("canvas.debugStatusEnabled", false)) val canvasDebugStatusEnabled: StateFlow = _canvasDebugStatusEnabled private val _wakeWords = MutableStateFlow(loadWakeWords()) @@ -91,119 +96,137 @@ class SecurePrefs(context: Context) { private val _voiceWakeMode = MutableStateFlow(loadVoiceWakeMode()) val voiceWakeMode: StateFlow = _voiceWakeMode - private val _talkEnabled = MutableStateFlow(prefs.getBoolean("talk.enabled", false)) + private val _talkEnabled = MutableStateFlow(plainPrefs.getBoolean("talk.enabled", false)) val talkEnabled: StateFlow = _talkEnabled + private val _speakerEnabled = MutableStateFlow(plainPrefs.getBoolean("voice.speakerEnabled", true)) + val speakerEnabled: StateFlow = _speakerEnabled + fun setLastDiscoveredStableId(value: String) { val trimmed = value.trim() - prefs.edit { putString("gateway.lastDiscoveredStableID", trimmed) } + plainPrefs.edit { putString("gateway.lastDiscoveredStableID", trimmed) } _lastDiscoveredStableId.value = trimmed } fun setDisplayName(value: String) { val trimmed = value.trim() - prefs.edit { putString(displayNameKey, trimmed) } + plainPrefs.edit { putString(displayNameKey, trimmed) } _displayName.value = trimmed } fun setCameraEnabled(value: Boolean) { - prefs.edit { putBoolean("camera.enabled", value) } + plainPrefs.edit { putBoolean("camera.enabled", value) } _cameraEnabled.value = value } fun setLocationMode(mode: LocationMode) { - prefs.edit { putString("location.enabledMode", mode.rawValue) } + plainPrefs.edit { putString("location.enabledMode", mode.rawValue) } _locationMode.value = mode } fun setLocationPreciseEnabled(value: Boolean) { - prefs.edit { putBoolean("location.preciseEnabled", value) } + plainPrefs.edit { putBoolean("location.preciseEnabled", value) } _locationPreciseEnabled.value = value } fun setPreventSleep(value: Boolean) { - prefs.edit { putBoolean("screen.preventSleep", value) } + plainPrefs.edit { putBoolean("screen.preventSleep", value) } _preventSleep.value = value } fun setManualEnabled(value: Boolean) { - prefs.edit { putBoolean("gateway.manual.enabled", value) } + plainPrefs.edit { putBoolean("gateway.manual.enabled", value) } _manualEnabled.value = value } fun setManualHost(value: String) { val trimmed = value.trim() - prefs.edit { putString("gateway.manual.host", trimmed) } + plainPrefs.edit { putString("gateway.manual.host", trimmed) } _manualHost.value = trimmed } fun setManualPort(value: Int) { - prefs.edit { putInt("gateway.manual.port", value) } + plainPrefs.edit { putInt("gateway.manual.port", value) } _manualPort.value = value } fun setManualTls(value: Boolean) { - prefs.edit { putBoolean("gateway.manual.tls", value) } + plainPrefs.edit { putBoolean("gateway.manual.tls", value) } _manualTls.value = value } fun setGatewayToken(value: String) { - prefs.edit { putString("gateway.manual.token", value) } - _gatewayToken.value = value + val trimmed = value.trim() + securePrefs.edit { putString("gateway.manual.token", trimmed) } + _gatewayToken.value = trimmed + } + + fun setGatewayPassword(value: String) { + saveGatewayPassword(value) + } + + fun setOnboardingCompleted(value: Boolean) { + plainPrefs.edit { putBoolean("onboarding.completed", value) } + _onboardingCompleted.value = value } fun setCanvasDebugStatusEnabled(value: Boolean) { - prefs.edit { putBoolean("canvas.debugStatusEnabled", value) } + plainPrefs.edit { putBoolean("canvas.debugStatusEnabled", value) } _canvasDebugStatusEnabled.value = value } fun loadGatewayToken(): String? { - val manual = _gatewayToken.value.trim() + val manual = + _gatewayToken.value.trim().ifEmpty { + val stored = securePrefs.getString("gateway.manual.token", null)?.trim().orEmpty() + if (stored.isNotEmpty()) _gatewayToken.value = stored + stored + } if (manual.isNotEmpty()) return manual val key = "gateway.token.${_instanceId.value}" - val stored = prefs.getString(key, null)?.trim() + val stored = securePrefs.getString(key, null)?.trim() return stored?.takeIf { it.isNotEmpty() } } fun saveGatewayToken(token: String) { val key = "gateway.token.${_instanceId.value}" - prefs.edit { putString(key, token.trim()) } + securePrefs.edit { putString(key, token.trim()) } } fun loadGatewayPassword(): String? { val key = "gateway.password.${_instanceId.value}" - val stored = prefs.getString(key, null)?.trim() + val stored = securePrefs.getString(key, null)?.trim() return stored?.takeIf { it.isNotEmpty() } } fun saveGatewayPassword(password: String) { val key = "gateway.password.${_instanceId.value}" - prefs.edit { putString(key, password.trim()) } + securePrefs.edit { putString(key, password.trim()) } } fun loadGatewayTlsFingerprint(stableId: String): String? { val key = "gateway.tls.$stableId" - return prefs.getString(key, null)?.trim()?.takeIf { it.isNotEmpty() } + return plainPrefs.getString(key, null)?.trim()?.takeIf { it.isNotEmpty() } } fun saveGatewayTlsFingerprint(stableId: String, fingerprint: String) { val key = "gateway.tls.$stableId" - prefs.edit { putString(key, fingerprint.trim()) } + plainPrefs.edit { putString(key, fingerprint.trim()) } } fun getString(key: String): String? { - return prefs.getString(key, null) + return securePrefs.getString(key, null) } fun putString(key: String, value: String) { - prefs.edit { putString(key, value) } + securePrefs.edit { putString(key, value) } } fun remove(key: String) { - prefs.edit { remove(key) } + securePrefs.edit { remove(key) } } - private fun createPrefs(context: Context, name: String): SharedPreferences { + private fun createSecurePrefs(context: Context, name: String): SharedPreferences { return EncryptedSharedPreferences.create( context, name, @@ -214,21 +237,21 @@ class SecurePrefs(context: Context) { } private fun loadOrCreateInstanceId(): String { - val existing = prefs.getString("node.instanceId", null)?.trim() + val existing = plainPrefs.getString("node.instanceId", null)?.trim() if (!existing.isNullOrBlank()) return existing val fresh = UUID.randomUUID().toString() - prefs.edit { putString("node.instanceId", fresh) } + plainPrefs.edit { putString("node.instanceId", fresh) } return fresh } private fun loadOrMigrateDisplayName(context: Context): String { - val existing = prefs.getString(displayNameKey, null)?.trim().orEmpty() + val existing = plainPrefs.getString(displayNameKey, null)?.trim().orEmpty() if (existing.isNotEmpty() && existing != "Android Node") return existing val candidate = DeviceNames.bestDefaultNodeName(context).trim() val resolved = candidate.ifEmpty { "Android Node" } - prefs.edit { putString(displayNameKey, resolved) } + plainPrefs.edit { putString(displayNameKey, resolved) } return resolved } @@ -236,34 +259,39 @@ class SecurePrefs(context: Context) { val sanitized = WakeWords.sanitize(words, defaultWakeWords) val encoded = JsonArray(sanitized.map { JsonPrimitive(it) }).toString() - prefs.edit { putString("voiceWake.triggerWords", encoded) } + plainPrefs.edit { putString("voiceWake.triggerWords", encoded) } _wakeWords.value = sanitized } fun setVoiceWakeMode(mode: VoiceWakeMode) { - prefs.edit { putString(voiceWakeModeKey, mode.rawValue) } + plainPrefs.edit { putString(voiceWakeModeKey, mode.rawValue) } _voiceWakeMode.value = mode } fun setTalkEnabled(value: Boolean) { - prefs.edit { putBoolean("talk.enabled", value) } + plainPrefs.edit { putBoolean("talk.enabled", value) } _talkEnabled.value = value } + fun setSpeakerEnabled(value: Boolean) { + plainPrefs.edit { putBoolean("voice.speakerEnabled", value) } + _speakerEnabled.value = value + } + private fun loadVoiceWakeMode(): VoiceWakeMode { - val raw = prefs.getString(voiceWakeModeKey, null) + val raw = plainPrefs.getString(voiceWakeModeKey, null) val resolved = VoiceWakeMode.fromRawValue(raw) // Default ON (foreground) when unset. if (raw.isNullOrBlank()) { - prefs.edit { putString(voiceWakeModeKey, resolved.rawValue) } + plainPrefs.edit { putString(voiceWakeModeKey, resolved.rawValue) } } return resolved } private fun loadWakeWords(): List { - val raw = prefs.getString("voiceWake.triggerWords", null)?.trim() + val raw = plainPrefs.getString("voiceWake.triggerWords", null)?.trim() if (raw.isNullOrEmpty()) return defaultWakeWords return try { val element = json.parseToJsonElement(raw) @@ -281,5 +309,4 @@ class SecurePrefs(context: Context) { defaultWakeWords } } - } diff --git a/apps/android/app/src/main/java/ai/openclaw/android/chat/ChatController.kt b/apps/android/app/src/main/java/ai/openclaw/android/chat/ChatController.kt index 3ed69ee5b24..335f3b0d70b 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/chat/ChatController.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/chat/ChatController.kt @@ -261,11 +261,7 @@ class ChatController( val key = _sessionKey.value try { if (supportsChatSubscribe) { - try { - session.sendNodeEvent("chat.subscribe", """{"sessionKey":"$key"}""") - } catch (_: Throwable) { - // best-effort - } + session.sendNodeEvent("chat.subscribe", """{"sessionKey":"$key"}""") } val historyJson = session.request("chat.history", """{"sessionKey":"$key"}""") @@ -325,6 +321,12 @@ class ChatController( val state = payload["state"].asStringOrNull() when (state) { + "delta" -> { + val text = parseAssistantDeltaText(payload) + if (!text.isNullOrEmpty()) { + _streamingAssistantText.value = text + } + } "final", "aborted", "error" -> { if (state == "error") { _errorText.value = payload["errorMessage"].asStringOrNull() ?: "Chat failed" @@ -351,9 +353,8 @@ class ChatController( private fun handleAgentEvent(payloadJson: String) { val payload = json.parseToJsonElement(payloadJson).asObjectOrNull() ?: return - val runId = payload["runId"].asStringOrNull() - val sessionId = _sessionId.value - if (sessionId != null && runId != sessionId) return + val sessionKey = payload["sessionKey"].asStringOrNull()?.trim() + if (!sessionKey.isNullOrEmpty() && sessionKey != _sessionKey.value) return val stream = payload["stream"].asStringOrNull() val data = payload["data"].asObjectOrNull() @@ -398,6 +399,21 @@ class ChatController( } } + private fun parseAssistantDeltaText(payload: JsonObject): String? { + val message = payload["message"].asObjectOrNull() ?: return null + if (message["role"].asStringOrNull() != "assistant") return null + val content = message["content"].asArrayOrNull() ?: return null + for (item in content) { + val obj = item.asObjectOrNull() ?: continue + if (obj["type"].asStringOrNull() != "text") continue + val text = obj["text"].asStringOrNull() + if (!text.isNullOrEmpty()) { + return text + } + } + return null + } + private fun publishPendingToolCalls() { _pendingToolCalls.value = pendingToolCallsById.values.sortedBy { it.startedAtMs } diff --git a/apps/android/app/src/main/java/ai/openclaw/android/gateway/DeviceAuthPayload.kt b/apps/android/app/src/main/java/ai/openclaw/android/gateway/DeviceAuthPayload.kt new file mode 100644 index 00000000000..9fecaa03b55 --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/gateway/DeviceAuthPayload.kt @@ -0,0 +1,52 @@ +package ai.openclaw.android.gateway + +internal object DeviceAuthPayload { + fun buildV3( + deviceId: String, + clientId: String, + clientMode: String, + role: String, + scopes: List, + signedAtMs: Long, + token: String?, + nonce: String, + platform: String?, + deviceFamily: String?, + ): String { + val scopeString = scopes.joinToString(",") + val authToken = token.orEmpty() + val platformNorm = normalizeMetadataField(platform) + val deviceFamilyNorm = normalizeMetadataField(deviceFamily) + return listOf( + "v3", + deviceId, + clientId, + clientMode, + role, + scopeString, + signedAtMs.toString(), + authToken, + nonce, + platformNorm, + deviceFamilyNorm, + ).joinToString("|") + } + + internal fun normalizeMetadataField(value: String?): String { + val trimmed = value?.trim().orEmpty() + if (trimmed.isEmpty()) { + return "" + } + // Keep cross-runtime normalization deterministic (TS/Swift/Kotlin): + // lowercase ASCII A-Z only for auth payload metadata fields. + val out = StringBuilder(trimmed.length) + for (ch in trimmed) { + if (ch in 'A'..'Z') { + out.append((ch.code + 32).toChar()) + } else { + out.append(ch) + } + } + return out.toString() + } +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/gateway/DeviceAuthStore.kt b/apps/android/app/src/main/java/ai/openclaw/android/gateway/DeviceAuthStore.kt index 810e029fba8..8ace62e087c 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/gateway/DeviceAuthStore.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/gateway/DeviceAuthStore.kt @@ -2,13 +2,18 @@ package ai.openclaw.android.gateway import ai.openclaw.android.SecurePrefs -class DeviceAuthStore(private val prefs: SecurePrefs) { - fun loadToken(deviceId: String, role: String): String? { +interface DeviceAuthTokenStore { + fun loadToken(deviceId: String, role: String): String? + fun saveToken(deviceId: String, role: String, token: String) +} + +class DeviceAuthStore(private val prefs: SecurePrefs) : DeviceAuthTokenStore { + override fun loadToken(deviceId: String, role: String): String? { val key = tokenKey(deviceId, role) return prefs.getString(key)?.trim()?.takeIf { it.isNotEmpty() } } - fun saveToken(deviceId: String, role: String, token: String) { + override fun saveToken(deviceId: String, role: String, token: String) { val key = tokenKey(deviceId, role) prefs.putString(key, token.trim()) } diff --git a/apps/android/app/src/main/java/ai/openclaw/android/gateway/DeviceIdentityStore.kt b/apps/android/app/src/main/java/ai/openclaw/android/gateway/DeviceIdentityStore.kt index ff651c6c17b..68830772f9a 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/gateway/DeviceIdentityStore.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/gateway/DeviceIdentityStore.kt @@ -3,11 +3,7 @@ package ai.openclaw.android.gateway import android.content.Context import android.util.Base64 import java.io.File -import java.security.KeyFactory -import java.security.KeyPairGenerator import java.security.MessageDigest -import java.security.Signature -import java.security.spec.PKCS8EncodedKeySpec import kotlinx.serialization.Serializable import kotlinx.serialization.json.Json @@ -22,21 +18,26 @@ data class DeviceIdentity( class DeviceIdentityStore(context: Context) { private val json = Json { ignoreUnknownKeys = true } private val identityFile = File(context.filesDir, "openclaw/identity/device.json") + @Volatile private var cachedIdentity: DeviceIdentity? = null @Synchronized fun loadOrCreate(): DeviceIdentity { + cachedIdentity?.let { return it } val existing = load() if (existing != null) { val derived = deriveDeviceId(existing.publicKeyRawBase64) if (derived != null && derived != existing.deviceId) { val updated = existing.copy(deviceId = derived) save(updated) + cachedIdentity = updated return updated } + cachedIdentity = existing return existing } val fresh = generate() save(fresh) + cachedIdentity = fresh return fresh } @@ -151,22 +152,16 @@ class DeviceIdentityStore(context: Context) { } } - private fun stripSpkiPrefix(spki: ByteArray): ByteArray { - if (spki.size == ED25519_SPKI_PREFIX.size + 32 && - spki.copyOfRange(0, ED25519_SPKI_PREFIX.size).contentEquals(ED25519_SPKI_PREFIX) - ) { - return spki.copyOfRange(ED25519_SPKI_PREFIX.size, spki.size) - } - return spki - } - private fun sha256Hex(data: ByteArray): String { val digest = MessageDigest.getInstance("SHA-256").digest(data) - val out = StringBuilder(digest.size * 2) + val out = CharArray(digest.size * 2) + var i = 0 for (byte in digest) { - out.append(String.format("%02x", byte)) + val v = byte.toInt() and 0xff + out[i++] = HEX[v ushr 4] + out[i++] = HEX[v and 0x0f] } - return out.toString() + return String(out) } private fun base64UrlEncode(data: ByteArray): String { @@ -174,9 +169,6 @@ class DeviceIdentityStore(context: Context) { } companion object { - private val ED25519_SPKI_PREFIX = - byteArrayOf( - 0x30, 0x2a, 0x30, 0x05, 0x06, 0x03, 0x2b, 0x65, 0x70, 0x03, 0x21, 0x00, - ) + private val HEX = "0123456789abcdef".toCharArray() } } diff --git a/apps/android/app/src/main/java/ai/openclaw/android/gateway/GatewaySession.kt b/apps/android/app/src/main/java/ai/openclaw/android/gateway/GatewaySession.kt index 091e735530d..6f30f072ef8 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/gateway/GatewaySession.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/gateway/GatewaySession.kt @@ -55,13 +55,18 @@ data class GatewayConnectOptions( class GatewaySession( private val scope: CoroutineScope, private val identityStore: DeviceIdentityStore, - private val deviceAuthStore: DeviceAuthStore, + private val deviceAuthStore: DeviceAuthTokenStore, private val onConnected: (serverName: String?, remoteAddress: String?, mainSessionKey: String?) -> Unit, private val onDisconnected: (message: String) -> Unit, private val onEvent: (event: String, payloadJson: String?) -> Unit, private val onInvoke: (suspend (InvokeRequest) -> InvokeResult)? = null, private val onTlsFingerprint: ((stableId: String, fingerprint: String) -> Unit)? = null, ) { + private companion object { + // Keep connect timeout above observed gateway unauthorized close on lower-end devices. + private const val CONNECT_RPC_TIMEOUT_MS = 12_000L + } + data class InvokeRequest( val id: String, val nodeId: String, @@ -131,8 +136,8 @@ class GatewaySession( fun currentCanvasHostUrl(): String? = canvasHostUrl fun currentMainSessionKey(): String? = mainSessionKey - suspend fun sendNodeEvent(event: String, payloadJson: String?) { - val conn = currentConnection ?: return + suspend fun sendNodeEvent(event: String, payloadJson: String?): Boolean { + val conn = currentConnection ?: return false val parsedPayload = payloadJson?.let { parseJsonOrNull(it) } val params = buildJsonObject { @@ -147,8 +152,10 @@ class GatewaySession( } try { conn.request("node.event", params, timeoutMs = 8_000) + return true } catch (err: Throwable) { Log.w("OpenClawGateway", "node.event failed: ${err.message ?: err::class.java.simpleName}") + return false } } @@ -166,6 +173,47 @@ class GatewaySession( throw IllegalStateException("${err?.code ?: "UNAVAILABLE"}: ${err?.message ?: "request failed"}") } + suspend fun refreshNodeCanvasCapability(timeoutMs: Long = 8_000): Boolean { + val conn = currentConnection ?: return false + val response = + try { + conn.request( + "node.canvas.capability.refresh", + params = buildJsonObject {}, + timeoutMs = timeoutMs, + ) + } catch (err: Throwable) { + Log.w("OpenClawGateway", "node.canvas.capability.refresh failed: ${err.message ?: err::class.java.simpleName}") + return false + } + if (!response.ok) { + val err = response.error + Log.w( + "OpenClawGateway", + "node.canvas.capability.refresh rejected: ${err?.code ?: "UNAVAILABLE"}: ${err?.message ?: "request failed"}", + ) + return false + } + val payloadObj = response.payloadJson?.let(::parseJsonOrNull)?.asObjectOrNull() + val refreshedCapability = payloadObj?.get("canvasCapability").asStringOrNull()?.trim().orEmpty() + if (refreshedCapability.isEmpty()) { + Log.w("OpenClawGateway", "node.canvas.capability.refresh missing canvasCapability") + return false + } + val scopedCanvasHostUrl = canvasHostUrl?.trim().orEmpty() + if (scopedCanvasHostUrl.isEmpty()) { + Log.w("OpenClawGateway", "node.canvas.capability.refresh missing local canvasHostUrl") + return false + } + val refreshedUrl = replaceCanvasCapabilityInScopedHostUrl(scopedCanvasHostUrl, refreshedCapability) + if (refreshedUrl == null) { + Log.w("OpenClawGateway", "node.canvas.capability.refresh unable to rewrite scoped canvas URL") + return false + } + canvasHostUrl = refreshedUrl + return true + } + private data class RpcResponse(val id: String, val ok: Boolean, val payloadJson: String?, val error: ErrorShape?) private inner class Connection( @@ -178,7 +226,7 @@ class GatewaySession( private val connectDeferred = CompletableDeferred() private val closedDeferred = CompletableDeferred() private val isClosed = AtomicBoolean(false) - private val connectNonceDeferred = CompletableDeferred() + private val connectNonceDeferred = CompletableDeferred() private val client: OkHttpClient = buildClient() private var socket: WebSocket? = null private val loggerTag = "OpenClawGateway" @@ -193,9 +241,7 @@ class GatewaySession( suspend fun connect() { val scheme = if (tls != null) "wss" else "ws" val url = "$scheme://${endpoint.host}:${endpoint.port}" - val httpScheme = if (tls != null) "https" else "http" - val origin = "$httpScheme://${endpoint.host}:${endpoint.port}" - val request = Request.Builder().url(url).header("Origin", origin).build() + val request = Request.Builder().url(url).build() socket = client.newWebSocket(request, Listener()) try { connectDeferred.await() @@ -296,21 +342,23 @@ class GatewaySession( } } - private suspend fun sendConnect(connectNonce: String?) { + private suspend fun sendConnect(connectNonce: String) { val identity = identityStore.loadOrCreate() val storedToken = deviceAuthStore.loadToken(identity.deviceId, options.role) val trimmedToken = token?.trim().orEmpty() - val authToken = if (storedToken.isNullOrBlank()) trimmedToken else storedToken - val canFallbackToShared = !storedToken.isNullOrBlank() && trimmedToken.isNotBlank() + // QR/setup/manual shared token must take precedence; stale role tokens can survive re-onboarding. + val authToken = if (trimmedToken.isNotBlank()) trimmedToken else storedToken.orEmpty() val payload = buildConnectParams(identity, connectNonce, authToken, password?.trim()) - val res = request("connect", payload, timeoutMs = 8_000) + val res = request("connect", payload, timeoutMs = CONNECT_RPC_TIMEOUT_MS) if (!res.ok) { val msg = res.error?.message ?: "connect failed" - if (canFallbackToShared) { - deviceAuthStore.clearToken(identity.deviceId, options.role) - } throw IllegalStateException(msg) } + handleConnectSuccess(res, identity.deviceId) + connectDeferred.complete(Unit) + } + + private fun handleConnectSuccess(res: RpcResponse, deviceId: String) { val payloadJson = res.payloadJson ?: throw IllegalStateException("connect failed: missing payload") val obj = json.parseToJsonElement(payloadJson).asObjectOrNull() ?: throw IllegalStateException("connect failed") val serverName = obj["server"].asObjectOrNull()?.get("host").asStringOrNull() @@ -318,21 +366,20 @@ class GatewaySession( val deviceToken = authObj?.get("deviceToken").asStringOrNull() val authRole = authObj?.get("role").asStringOrNull() ?: options.role if (!deviceToken.isNullOrBlank()) { - deviceAuthStore.saveToken(identity.deviceId, authRole, deviceToken) + deviceAuthStore.saveToken(deviceId, authRole, deviceToken) } val rawCanvas = obj["canvasHostUrl"].asStringOrNull() - canvasHostUrl = normalizeCanvasHostUrl(rawCanvas, endpoint) + canvasHostUrl = normalizeCanvasHostUrl(rawCanvas, endpoint, isTlsConnection = tls != null) val sessionDefaults = obj["snapshot"].asObjectOrNull() ?.get("sessionDefaults").asObjectOrNull() mainSessionKey = sessionDefaults?.get("mainSessionKey").asStringOrNull() onConnected(serverName, remoteAddress, mainSessionKey) - connectDeferred.complete(Unit) } private fun buildConnectParams( identity: DeviceIdentity, - connectNonce: String?, + connectNonce: String, authToken: String, authPassword: String?, ): JsonObject { @@ -366,7 +413,7 @@ class GatewaySession( val signedAtMs = System.currentTimeMillis() val payload = - buildDeviceAuthPayload( + DeviceAuthPayload.buildV3( deviceId = identity.deviceId, clientId = client.id, clientMode = client.mode, @@ -375,6 +422,8 @@ class GatewaySession( signedAtMs = signedAtMs, token = if (authToken.isNotEmpty()) authToken else null, nonce = connectNonce, + platform = client.platform, + deviceFamily = client.deviceFamily, ) val signature = identityStore.signPayload(payload, identity) val publicKey = identityStore.publicKeyBase64Url(identity) @@ -385,9 +434,7 @@ class GatewaySession( put("publicKey", JsonPrimitive(publicKey)) put("signature", JsonPrimitive(signature)) put("signedAt", JsonPrimitive(signedAtMs)) - if (!connectNonce.isNullOrBlank()) { - put("nonce", JsonPrimitive(connectNonce)) - } + put("nonce", JsonPrimitive(connectNonce)) } } else { null @@ -447,8 +494,8 @@ class GatewaySession( frame["payload"]?.let { it.toString() } ?: frame["payloadJSON"].asStringOrNull() if (event == "connect.challenge") { val nonce = extractConnectNonce(payloadJson) - if (!connectNonceDeferred.isCompleted) { - connectNonceDeferred.complete(nonce) + if (!connectNonceDeferred.isCompleted && !nonce.isNullOrBlank()) { + connectNonceDeferred.complete(nonce.trim()) } return } @@ -459,12 +506,11 @@ class GatewaySession( onEvent(event, payloadJson) } - private suspend fun awaitConnectNonce(): String? { - if (isLoopbackHost(endpoint.host)) return null + private suspend fun awaitConnectNonce(): String { return try { withTimeout(2_000) { connectNonceDeferred.await() } - } catch (_: Throwable) { - null + } catch (err: Throwable) { + throw IllegalStateException("connect challenge timeout", err) } } @@ -496,11 +542,16 @@ class GatewaySession( } catch (err: Throwable) { invokeErrorFromThrowable(err) } - sendInvokeResult(id, nodeId, result) + sendInvokeResult(id, nodeId, result, timeoutMs) } } - private suspend fun sendInvokeResult(id: String, nodeId: String, result: InvokeResult) { + private suspend fun sendInvokeResult( + id: String, + nodeId: String, + result: InvokeResult, + invokeTimeoutMs: Long?, + ) { val parsedPayload = result.payloadJson?.let { parseJsonOrNull(it) } val params = buildJsonObject { @@ -522,24 +573,20 @@ class GatewaySession( ) } } + val ackTimeoutMs = resolveInvokeResultAckTimeoutMs(invokeTimeoutMs) try { - request("node.invoke.result", params, timeoutMs = 15_000) + request("node.invoke.result", params, timeoutMs = ackTimeoutMs) } catch (err: Throwable) { - Log.w(loggerTag, "node.invoke.result failed: ${err.message ?: err::class.java.simpleName}") + Log.w( + loggerTag, + "node.invoke.result failed (ackTimeoutMs=$ackTimeoutMs): ${err.message ?: err::class.java.simpleName}", + ) } } private fun invokeErrorFromThrowable(err: Throwable): InvokeResult { - val msg = err.message?.trim().takeIf { !it.isNullOrEmpty() } ?: err::class.java.simpleName - val parts = msg.split(":", limit = 2) - if (parts.size == 2) { - val code = parts[0].trim() - val rest = parts[1].trim() - if (code.isNotEmpty() && code.all { it.isUpperCase() || it == '_' }) { - return InvokeResult.error(code = code, message = rest.ifEmpty { msg }) - } - } - return InvokeResult.error(code = "UNAVAILABLE", message = msg) + val parsed = parseInvokeErrorFromThrowable(err, fallbackMessage = err::class.java.simpleName) + return InvokeResult.error(code = parsed.code, message = parsed.message) } private fun failPending() { @@ -587,54 +634,30 @@ class GatewaySession( } } - private fun buildDeviceAuthPayload( - deviceId: String, - clientId: String, - clientMode: String, - role: String, - scopes: List, - signedAtMs: Long, - token: String?, - nonce: String?, - ): String { - val scopeString = scopes.joinToString(",") - val authToken = token.orEmpty() - val version = if (nonce.isNullOrBlank()) "v1" else "v2" - val parts = - mutableListOf( - version, - deviceId, - clientId, - clientMode, - role, - scopeString, - signedAtMs.toString(), - authToken, - ) - if (!nonce.isNullOrBlank()) { - parts.add(nonce) - } - return parts.joinToString("|") - } - - private fun normalizeCanvasHostUrl(raw: String?, endpoint: GatewayEndpoint): String? { + private fun normalizeCanvasHostUrl( + raw: String?, + endpoint: GatewayEndpoint, + isTlsConnection: Boolean, + ): String? { val trimmed = raw?.trim().orEmpty() val parsed = trimmed.takeIf { it.isNotBlank() }?.let { runCatching { java.net.URI(it) }.getOrNull() } val host = parsed?.host?.trim().orEmpty() val port = parsed?.port ?: -1 val scheme = parsed?.scheme?.trim().orEmpty().ifBlank { "http" } + val suffix = buildUrlSuffix(parsed) - // Detect TLS reverse proxy: endpoint on port 443, or domain-based host - val tls = endpoint.port == 443 || endpoint.host.contains(".") - - // If raw URL is a non-loopback address AND we're behind TLS reverse proxy, - // fix the port (gateway sends its internal port like 18789, but we need 443 via Caddy) - if (trimmed.isNotBlank() && !isLoopbackHost(host)) { - if (tls && port > 0 && port != 443) { - // Rewrite the URL to use the reverse proxy port instead of the raw gateway port - val fixedScheme = "https" - val formattedHost = if (host.contains(":")) "[${host}]" else host - return "$fixedScheme://$formattedHost" + // If raw URL is a non-loopback address and this connection uses TLS, + // normalize scheme/port to the endpoint we actually connected to. + if (trimmed.isNotBlank() && host.isNotBlank() && !isLoopbackHost(host)) { + val needsTlsRewrite = + isTlsConnection && + ( + !scheme.equals("https", ignoreCase = true) || + (port > 0 && port != endpoint.port) || + (port <= 0 && endpoint.port != 443) + ) + if (needsTlsRewrite) { + return buildCanvasUrl(host = host, scheme = "https", port = endpoint.port, suffix = suffix) } return trimmed } @@ -645,14 +668,26 @@ class GatewaySession( ?: endpoint.host.trim() if (fallbackHost.isEmpty()) return trimmed.ifBlank { null } - // When connecting through a reverse proxy (TLS on standard port), use the - // connection endpoint's scheme and port instead of the raw canvas port. - val fallbackScheme = if (tls) "https" else scheme - // Behind reverse proxy, always use the proxy port (443), not the raw canvas port - val fallbackPort = if (tls) endpoint.port else (endpoint.canvasPort ?: endpoint.port) - val formattedHost = if (fallbackHost.contains(":")) "[${fallbackHost}]" else fallbackHost - val portSuffix = if ((fallbackScheme == "https" && fallbackPort == 443) || (fallbackScheme == "http" && fallbackPort == 80)) "" else ":$fallbackPort" - return "$fallbackScheme://$formattedHost$portSuffix" + // For TLS connections, use the connected endpoint's scheme/port instead of raw canvas metadata. + val fallbackScheme = if (isTlsConnection) "https" else scheme + // For TLS, always use the connected endpoint port. + val fallbackPort = if (isTlsConnection) endpoint.port else (endpoint.canvasPort ?: endpoint.port) + return buildCanvasUrl(host = fallbackHost, scheme = fallbackScheme, port = fallbackPort, suffix = suffix) + } + + private fun buildCanvasUrl(host: String, scheme: String, port: Int, suffix: String): String { + val loweredScheme = scheme.lowercase() + val formattedHost = if (host.contains(":")) "[${host}]" else host + val portSuffix = if ((loweredScheme == "https" && port == 443) || (loweredScheme == "http" && port == 80)) "" else ":$port" + return "$loweredScheme://$formattedHost$portSuffix$suffix" + } + + private fun buildUrlSuffix(uri: java.net.URI?): String { + if (uri == null) return "" + val path = uri.rawPath?.takeIf { it.isNotBlank() } ?: "" + val query = uri.rawQuery?.takeIf { it.isNotBlank() }?.let { "?$it" } ?: "" + val fragment = uri.rawFragment?.takeIf { it.isNotBlank() }?.let { "#$it" } ?: "" + return "$path$query$fragment" } private fun isLoopbackHost(raw: String?): Boolean { @@ -702,3 +737,24 @@ private fun parseJsonOrNull(payload: String): JsonElement? { null } } + +internal fun replaceCanvasCapabilityInScopedHostUrl( + scopedUrl: String, + capability: String, +): String? { + val marker = "/__openclaw__/cap/" + val markerStart = scopedUrl.indexOf(marker) + if (markerStart < 0) return null + val capabilityStart = markerStart + marker.length + val slashEnd = scopedUrl.indexOf("/", capabilityStart).takeIf { it >= 0 } + val queryEnd = scopedUrl.indexOf("?", capabilityStart).takeIf { it >= 0 } + val fragmentEnd = scopedUrl.indexOf("#", capabilityStart).takeIf { it >= 0 } + val capabilityEnd = listOfNotNull(slashEnd, queryEnd, fragmentEnd).minOrNull() ?: scopedUrl.length + if (capabilityEnd <= capabilityStart) return null + return scopedUrl.substring(0, capabilityStart) + capability + scopedUrl.substring(capabilityEnd) +} + +internal fun resolveInvokeResultAckTimeoutMs(invokeTimeoutMs: Long?): Long { + val normalized = invokeTimeoutMs?.takeIf { it > 0L } ?: 15_000L + return normalized.coerceIn(15_000L, 120_000L) +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/gateway/InvokeErrorParser.kt b/apps/android/app/src/main/java/ai/openclaw/android/gateway/InvokeErrorParser.kt new file mode 100644 index 00000000000..7242f4a5533 --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/gateway/InvokeErrorParser.kt @@ -0,0 +1,39 @@ +package ai.openclaw.android.gateway + +data class ParsedInvokeError( + val code: String, + val message: String, + val hadExplicitCode: Boolean, +) { + val prefixedMessage: String + get() = "$code: $message" +} + +fun parseInvokeErrorMessage(raw: String): ParsedInvokeError { + val trimmed = raw.trim() + if (trimmed.isEmpty()) { + return ParsedInvokeError(code = "UNAVAILABLE", message = "error", hadExplicitCode = false) + } + + val parts = trimmed.split(":", limit = 2) + if (parts.size == 2) { + val code = parts[0].trim() + val rest = parts[1].trim() + if (code.isNotEmpty() && code.all { it.isUpperCase() || it == '_' }) { + return ParsedInvokeError( + code = code, + message = rest.ifEmpty { trimmed }, + hadExplicitCode = true, + ) + } + } + return ParsedInvokeError(code = "UNAVAILABLE", message = trimmed, hadExplicitCode = false) +} + +fun parseInvokeErrorFromThrowable( + err: Throwable, + fallbackMessage: String = "error", +): ParsedInvokeError { + val raw = err.message?.trim().takeIf { !it.isNullOrEmpty() } ?: fallbackMessage + return parseInvokeErrorMessage(raw) +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/node/CalendarHandler.kt b/apps/android/app/src/main/java/ai/openclaw/android/node/CalendarHandler.kt new file mode 100644 index 00000000000..357aed3b297 --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/node/CalendarHandler.kt @@ -0,0 +1,384 @@ +package ai.openclaw.android.node + +import android.Manifest +import android.content.ContentResolver +import android.content.ContentUris +import android.content.ContentValues +import android.content.Context +import android.provider.CalendarContract +import androidx.core.content.ContextCompat +import ai.openclaw.android.gateway.GatewaySession +import java.time.Instant +import java.time.temporal.ChronoUnit +import java.util.TimeZone +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.buildJsonArray +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.put + +private const val DEFAULT_CALENDAR_LIMIT = 50 + +internal data class CalendarEventsRequest( + val startMs: Long, + val endMs: Long, + val limit: Int, +) + +internal data class CalendarAddRequest( + val title: String, + val startMs: Long, + val endMs: Long, + val isAllDay: Boolean, + val location: String?, + val notes: String?, + val calendarId: Long?, + val calendarTitle: String?, +) + +internal data class CalendarEventRecord( + val identifier: String, + val title: String, + val startISO: String, + val endISO: String, + val isAllDay: Boolean, + val location: String?, + val calendarTitle: String?, +) + +internal interface CalendarDataSource { + fun hasReadPermission(context: Context): Boolean + + fun hasWritePermission(context: Context): Boolean + + fun events(context: Context, request: CalendarEventsRequest): List + + fun add(context: Context, request: CalendarAddRequest): CalendarEventRecord +} + +private object SystemCalendarDataSource : CalendarDataSource { + override fun hasReadPermission(context: Context): Boolean { + return ContextCompat.checkSelfPermission(context, Manifest.permission.READ_CALENDAR) == + android.content.pm.PackageManager.PERMISSION_GRANTED + } + + override fun hasWritePermission(context: Context): Boolean { + return ContextCompat.checkSelfPermission(context, Manifest.permission.WRITE_CALENDAR) == + android.content.pm.PackageManager.PERMISSION_GRANTED + } + + override fun events(context: Context, request: CalendarEventsRequest): List { + val resolver = context.contentResolver + val builder = CalendarContract.Instances.CONTENT_URI.buildUpon() + ContentUris.appendId(builder, request.startMs) + ContentUris.appendId(builder, request.endMs) + val projection = + arrayOf( + CalendarContract.Instances.EVENT_ID, + CalendarContract.Instances.TITLE, + CalendarContract.Instances.BEGIN, + CalendarContract.Instances.END, + CalendarContract.Instances.ALL_DAY, + CalendarContract.Instances.EVENT_LOCATION, + CalendarContract.Instances.CALENDAR_DISPLAY_NAME, + ) + val sortOrder = "${CalendarContract.Instances.BEGIN} ASC LIMIT ${request.limit}" + resolver.query(builder.build(), projection, null, null, sortOrder).use { cursor -> + if (cursor == null) return emptyList() + val out = mutableListOf() + while (cursor.moveToNext() && out.size < request.limit) { + val id = cursor.getLong(0) + val title = cursor.getString(1)?.trim().orEmpty().ifEmpty { "(untitled)" } + val beginMs = cursor.getLong(2) + val endMs = cursor.getLong(3) + val isAllDay = cursor.getInt(4) == 1 + val location = cursor.getString(5)?.trim()?.ifEmpty { null } + val calendarTitle = cursor.getString(6)?.trim()?.ifEmpty { null } + out += + CalendarEventRecord( + identifier = id.toString(), + title = title, + startISO = Instant.ofEpochMilli(beginMs).toString(), + endISO = Instant.ofEpochMilli(endMs).toString(), + isAllDay = isAllDay, + location = location, + calendarTitle = calendarTitle, + ) + } + return out + } + } + + override fun add(context: Context, request: CalendarAddRequest): CalendarEventRecord { + val resolver = context.contentResolver + val resolvedCalendarId = resolveCalendarId(resolver, request.calendarId, request.calendarTitle) + val values = + ContentValues().apply { + put(CalendarContract.Events.CALENDAR_ID, resolvedCalendarId) + put(CalendarContract.Events.TITLE, request.title) + put(CalendarContract.Events.DTSTART, request.startMs) + put(CalendarContract.Events.DTEND, request.endMs) + put(CalendarContract.Events.ALL_DAY, if (request.isAllDay) 1 else 0) + put(CalendarContract.Events.EVENT_TIMEZONE, TimeZone.getDefault().id) + request.location?.let { put(CalendarContract.Events.EVENT_LOCATION, it) } + request.notes?.let { put(CalendarContract.Events.DESCRIPTION, it) } + } + val uri = resolver.insert(CalendarContract.Events.CONTENT_URI, values) + ?: throw IllegalStateException("calendar insert failed") + val eventId = uri.lastPathSegment?.toLongOrNull() + ?: throw IllegalStateException("calendar insert failed") + return loadEventById(resolver, eventId) + ?: throw IllegalStateException("calendar insert failed") + } + + private fun resolveCalendarId( + resolver: ContentResolver, + calendarId: Long?, + calendarTitle: String?, + ): Long { + if (calendarId != null) { + if (calendarExists(resolver, calendarId)) return calendarId + throw IllegalArgumentException("CALENDAR_NOT_FOUND: no calendar id $calendarId") + } + if (!calendarTitle.isNullOrEmpty()) { + findCalendarByTitle(resolver, calendarTitle)?.let { return it } + throw IllegalArgumentException("CALENDAR_NOT_FOUND: no calendar named $calendarTitle") + } + findDefaultCalendarId(resolver)?.let { return it } + throw IllegalArgumentException("CALENDAR_NOT_FOUND: no default calendar") + } + + private fun calendarExists(resolver: ContentResolver, id: Long): Boolean { + val projection = arrayOf(CalendarContract.Calendars._ID) + resolver.query( + CalendarContract.Calendars.CONTENT_URI, + projection, + "${CalendarContract.Calendars._ID}=?", + arrayOf(id.toString()), + null, + ).use { cursor -> + return cursor != null && cursor.moveToFirst() + } + } + + private fun findCalendarByTitle(resolver: ContentResolver, title: String): Long? { + val projection = arrayOf(CalendarContract.Calendars._ID) + resolver.query( + CalendarContract.Calendars.CONTENT_URI, + projection, + "${CalendarContract.Calendars.CALENDAR_DISPLAY_NAME}=?", + arrayOf(title), + "${CalendarContract.Calendars.IS_PRIMARY} DESC", + ).use { cursor -> + if (cursor == null || !cursor.moveToFirst()) return null + return cursor.getLong(0) + } + } + + private fun findDefaultCalendarId(resolver: ContentResolver): Long? { + val projection = arrayOf(CalendarContract.Calendars._ID) + resolver.query( + CalendarContract.Calendars.CONTENT_URI, + projection, + "${CalendarContract.Calendars.VISIBLE}=1", + null, + "${CalendarContract.Calendars.IS_PRIMARY} DESC, ${CalendarContract.Calendars._ID} ASC", + ).use { cursor -> + if (cursor == null || !cursor.moveToFirst()) return null + return cursor.getLong(0) + } + } + + private fun loadEventById( + resolver: ContentResolver, + eventId: Long, + ): CalendarEventRecord? { + val projection = + arrayOf( + CalendarContract.Events._ID, + CalendarContract.Events.TITLE, + CalendarContract.Events.DTSTART, + CalendarContract.Events.DTEND, + CalendarContract.Events.ALL_DAY, + CalendarContract.Events.EVENT_LOCATION, + CalendarContract.Events.CALENDAR_DISPLAY_NAME, + ) + resolver.query( + CalendarContract.Events.CONTENT_URI, + projection, + "${CalendarContract.Events._ID}=?", + arrayOf(eventId.toString()), + null, + ).use { cursor -> + if (cursor == null || !cursor.moveToFirst()) return null + return CalendarEventRecord( + identifier = cursor.getLong(0).toString(), + title = cursor.getString(1)?.trim().orEmpty().ifEmpty { "(untitled)" }, + startISO = Instant.ofEpochMilli(cursor.getLong(2)).toString(), + endISO = Instant.ofEpochMilli(cursor.getLong(3)).toString(), + isAllDay = cursor.getInt(4) == 1, + location = cursor.getString(5)?.trim()?.ifEmpty { null }, + calendarTitle = cursor.getString(6)?.trim()?.ifEmpty { null }, + ) + } + } +} + +class CalendarHandler private constructor( + private val appContext: Context, + private val dataSource: CalendarDataSource, +) { + constructor(appContext: Context) : this(appContext = appContext, dataSource = SystemCalendarDataSource) + + fun handleCalendarEvents(paramsJson: String?): GatewaySession.InvokeResult { + if (!dataSource.hasReadPermission(appContext)) { + return GatewaySession.InvokeResult.error( + code = "CALENDAR_PERMISSION_REQUIRED", + message = "CALENDAR_PERMISSION_REQUIRED: grant Calendar permission", + ) + } + val request = + parseEventsRequest(paramsJson) + ?: return GatewaySession.InvokeResult.error( + code = "INVALID_REQUEST", + message = "INVALID_REQUEST: expected JSON object", + ) + return try { + val events = dataSource.events(appContext, request) + GatewaySession.InvokeResult.ok( + buildJsonObject { + put( + "events", + buildJsonArray { events.forEach { add(eventJson(it)) } }, + ) + }.toString(), + ) + } catch (err: Throwable) { + GatewaySession.InvokeResult.error( + code = "CALENDAR_UNAVAILABLE", + message = "CALENDAR_UNAVAILABLE: ${err.message ?: "calendar query failed"}", + ) + } + } + + fun handleCalendarAdd(paramsJson: String?): GatewaySession.InvokeResult { + if (!dataSource.hasWritePermission(appContext)) { + return GatewaySession.InvokeResult.error( + code = "CALENDAR_PERMISSION_REQUIRED", + message = "CALENDAR_PERMISSION_REQUIRED: grant Calendar permission", + ) + } + val request = + parseAddRequest(paramsJson) + ?: return GatewaySession.InvokeResult.error( + code = "INVALID_REQUEST", + message = "INVALID_REQUEST: expected JSON object", + ) + if (request.title.isEmpty()) { + return GatewaySession.InvokeResult.error( + code = "CALENDAR_INVALID", + message = "CALENDAR_INVALID: title required", + ) + } + if (request.endMs <= request.startMs) { + return GatewaySession.InvokeResult.error( + code = "CALENDAR_INVALID", + message = "CALENDAR_INVALID: endISO must be after startISO", + ) + } + return try { + val event = dataSource.add(appContext, request) + GatewaySession.InvokeResult.ok( + buildJsonObject { + put("event", eventJson(event)) + }.toString(), + ) + } catch (err: IllegalArgumentException) { + val msg = err.message ?: "CALENDAR_INVALID: invalid request" + val code = if (msg.startsWith("CALENDAR_NOT_FOUND")) "CALENDAR_NOT_FOUND" else "CALENDAR_INVALID" + GatewaySession.InvokeResult.error(code = code, message = msg) + } catch (err: Throwable) { + GatewaySession.InvokeResult.error( + code = "CALENDAR_UNAVAILABLE", + message = "CALENDAR_UNAVAILABLE: ${err.message ?: "calendar add failed"}", + ) + } + } + + private fun parseEventsRequest(paramsJson: String?): CalendarEventsRequest? { + if (paramsJson.isNullOrBlank()) { + val start = Instant.now() + val end = start.plus(7, ChronoUnit.DAYS) + return CalendarEventsRequest(startMs = start.toEpochMilli(), endMs = end.toEpochMilli(), limit = DEFAULT_CALENDAR_LIMIT) + } + val params = + try { + Json.parseToJsonElement(paramsJson).asObjectOrNull() + } catch (_: Throwable) { + null + } ?: return null + val start = parseISO((params["startISO"] as? JsonPrimitive)?.content) + val end = parseISO((params["endISO"] as? JsonPrimitive)?.content) + val resolvedStart = start ?: Instant.now() + val resolvedEnd = end ?: resolvedStart.plus(7, ChronoUnit.DAYS) + val limit = ((params["limit"] as? JsonPrimitive)?.content?.toIntOrNull() ?: DEFAULT_CALENDAR_LIMIT).coerceIn(1, 500) + return CalendarEventsRequest( + startMs = resolvedStart.toEpochMilli(), + endMs = resolvedEnd.toEpochMilli(), + limit = limit, + ) + } + + private fun parseAddRequest(paramsJson: String?): CalendarAddRequest? { + val params = + try { + paramsJson?.let { Json.parseToJsonElement(it).asObjectOrNull() } + } catch (_: Throwable) { + null + } ?: return null + val start = parseISO((params["startISO"] as? JsonPrimitive)?.content) + ?: return null + val end = parseISO((params["endISO"] as? JsonPrimitive)?.content) + ?: return null + return CalendarAddRequest( + title = (params["title"] as? JsonPrimitive)?.content?.trim().orEmpty(), + startMs = start.toEpochMilli(), + endMs = end.toEpochMilli(), + isAllDay = (params["isAllDay"] as? JsonPrimitive)?.content?.toBooleanStrictOrNull() ?: false, + location = (params["location"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null }, + notes = (params["notes"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null }, + calendarId = (params["calendarId"] as? JsonPrimitive)?.content?.toLongOrNull(), + calendarTitle = (params["calendarTitle"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null }, + ) + } + + private fun parseISO(raw: String?): Instant? { + val value = raw?.trim().orEmpty() + if (value.isEmpty()) return null + return try { + Instant.parse(value) + } catch (_: Throwable) { + null + } + } + + private fun eventJson(event: CalendarEventRecord): JsonObject { + return buildJsonObject { + put("identifier", JsonPrimitive(event.identifier)) + put("title", JsonPrimitive(event.title)) + put("startISO", JsonPrimitive(event.startISO)) + put("endISO", JsonPrimitive(event.endISO)) + put("isAllDay", JsonPrimitive(event.isAllDay)) + event.location?.let { put("location", JsonPrimitive(it)) } + event.calendarTitle?.let { put("calendarTitle", JsonPrimitive(it)) } + } + } + + companion object { + internal fun forTesting( + appContext: Context, + dataSource: CalendarDataSource, + ): CalendarHandler = CalendarHandler(appContext = appContext, dataSource = dataSource) + } +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/node/CameraCaptureManager.kt b/apps/android/app/src/main/java/ai/openclaw/android/node/CameraCaptureManager.kt index 65bac915eff..0dc6f2b6955 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/node/CameraCaptureManager.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/node/CameraCaptureManager.kt @@ -1,13 +1,16 @@ package ai.openclaw.android.node import android.Manifest -import android.content.Context import android.annotation.SuppressLint +import android.content.Context import android.graphics.Bitmap import android.graphics.BitmapFactory import android.graphics.Matrix -import android.util.Base64 import android.content.pm.PackageManager +import android.hardware.camera2.CameraCharacteristics +import android.util.Base64 +import androidx.camera.camera2.interop.Camera2CameraInfo +import androidx.camera.core.CameraInfo import androidx.exifinterface.media.ExifInterface import androidx.lifecycle.LifecycleOwner import androidx.camera.core.CameraSelector @@ -30,6 +33,10 @@ import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.suspendCancellableCoroutine import kotlinx.coroutines.withTimeout import kotlinx.coroutines.withContext +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.contentOrNull import java.io.ByteArrayOutputStream import java.io.File import java.util.concurrent.Executor @@ -40,6 +47,12 @@ import kotlin.coroutines.resumeWithException class CameraCaptureManager(private val context: Context) { data class Payload(val payloadJson: String) data class FilePayload(val file: File, val durationMs: Long, val hasAudio: Boolean) + data class CameraDeviceInfo( + val id: String, + val name: String, + val position: String, + val deviceType: String, + ) @Volatile private var lifecycleOwner: LifecycleOwner? = null @Volatile private var permissionRequester: PermissionRequester? = null @@ -52,6 +65,14 @@ class CameraCaptureManager(private val context: Context) { permissionRequester = requester } + suspend fun listDevices(): List = + withContext(Dispatchers.Main) { + val provider = context.cameraProvider() + provider.availableCameraInfos + .mapNotNull { info -> cameraDeviceInfoOrNull(info) } + .sortedBy { it.id } + } + private suspend fun ensureCameraPermission() { val granted = checkSelfPermission(context, Manifest.permission.CAMERA) == PackageManager.PERMISSION_GRANTED if (granted) return @@ -80,14 +101,15 @@ class CameraCaptureManager(private val context: Context) { withContext(Dispatchers.Main) { ensureCameraPermission() val owner = lifecycleOwner ?: throw IllegalStateException("UNAVAILABLE: camera not ready") - val facing = parseFacing(paramsJson) ?: "front" - val quality = (parseQuality(paramsJson) ?: 0.5).coerceIn(0.1, 1.0) - val maxWidth = parseMaxWidth(paramsJson) ?: 800 + val params = parseParamsObject(paramsJson) + val facing = parseFacing(params) ?: "front" + val quality = (parseQuality(params) ?: 0.95).coerceIn(0.1, 1.0) + val maxWidth = parseMaxWidth(params) ?: 1600 + val deviceId = parseDeviceId(params) val provider = context.cameraProvider() val capture = ImageCapture.Builder().build() - val selector = - if (facing == "front") CameraSelector.DEFAULT_FRONT_CAMERA else CameraSelector.DEFAULT_BACK_CAMERA + val selector = resolveCameraSelector(provider, facing, deviceId) provider.unbindAll() provider.bindToLifecycle(owner, selector, capture) @@ -145,12 +167,14 @@ class CameraCaptureManager(private val context: Context) { withContext(Dispatchers.Main) { ensureCameraPermission() val owner = lifecycleOwner ?: throw IllegalStateException("UNAVAILABLE: camera not ready") - val facing = parseFacing(paramsJson) ?: "front" - val durationMs = (parseDurationMs(paramsJson) ?: 3_000).coerceIn(200, 60_000) - val includeAudio = parseIncludeAudio(paramsJson) ?: true + val params = parseParamsObject(paramsJson) + val facing = parseFacing(params) ?: "front" + val durationMs = (parseDurationMs(params) ?: 3_000).coerceIn(200, 60_000) + val includeAudio = parseIncludeAudio(params) ?: true + val deviceId = parseDeviceId(params) if (includeAudio) ensureMicPermission() - android.util.Log.w("CameraCaptureManager", "clip: start facing=$facing duration=$durationMs audio=$includeAudio") + android.util.Log.w("CameraCaptureManager", "clip: start facing=$facing duration=$durationMs audio=$includeAudio deviceId=${deviceId ?: "-"}") val provider = context.cameraProvider() android.util.Log.w("CameraCaptureManager", "clip: got camera provider") @@ -162,8 +186,7 @@ class CameraCaptureManager(private val context: Context) { ) .build() val videoCapture = VideoCapture.withOutput(recorder) - val selector = - if (facing == "front") CameraSelector.DEFAULT_FRONT_CAMERA else CameraSelector.DEFAULT_BACK_CAMERA + val selector = resolveCameraSelector(provider, facing, deviceId) // CameraX requires a Preview use case for the camera to start producing frames; // without it, the encoder may get no data (ERROR_NO_VALID_DATA). @@ -270,49 +293,104 @@ class CameraCaptureManager(private val context: Context) { return rotated } - private fun parseFacing(paramsJson: String?): String? = - when { - paramsJson?.contains("\"front\"") == true -> "front" - paramsJson?.contains("\"back\"") == true -> "back" - else -> null + private fun parseParamsObject(paramsJson: String?): JsonObject? { + if (paramsJson.isNullOrBlank()) return null + return try { + Json.parseToJsonElement(paramsJson).asObjectOrNull() + } catch (_: Throwable) { + null } + } - private fun parseQuality(paramsJson: String?): Double? = - parseNumber(paramsJson, key = "quality")?.toDoubleOrNull() + private fun readPrimitive(params: JsonObject?, key: String): JsonPrimitive? = + params?.get(key) as? JsonPrimitive - private fun parseMaxWidth(paramsJson: String?): Int? = - parseNumber(paramsJson, key = "maxWidth")?.toIntOrNull() - - private fun parseDurationMs(paramsJson: String?): Int? = - parseNumber(paramsJson, key = "durationMs")?.toIntOrNull() - - private fun parseIncludeAudio(paramsJson: String?): Boolean? { - val raw = paramsJson ?: return null - val key = "\"includeAudio\"" - val idx = raw.indexOf(key) - if (idx < 0) return null - val colon = raw.indexOf(':', idx + key.length) - if (colon < 0) return null - val tail = raw.substring(colon + 1).trimStart() - return when { - tail.startsWith("true") -> true - tail.startsWith("false") -> false + private fun parseFacing(params: JsonObject?): String? { + val value = readPrimitive(params, "facing")?.contentOrNull?.trim()?.lowercase() ?: return null + return when (value) { + "front", "back" -> value else -> null } } - private fun parseNumber(paramsJson: String?, key: String): String? { - val raw = paramsJson ?: return null - val needle = "\"$key\"" - val idx = raw.indexOf(needle) - if (idx < 0) return null - val colon = raw.indexOf(':', idx + needle.length) - if (colon < 0) return null - val tail = raw.substring(colon + 1).trimStart() - return tail.takeWhile { it.isDigit() || it == '.' } + private fun parseQuality(params: JsonObject?): Double? = + readPrimitive(params, "quality")?.contentOrNull?.toDoubleOrNull() + + private fun parseMaxWidth(params: JsonObject?): Int? = + readPrimitive(params, "maxWidth") + ?.contentOrNull + ?.toIntOrNull() + ?.takeIf { it > 0 } + + private fun parseDurationMs(params: JsonObject?): Int? = + readPrimitive(params, "durationMs")?.contentOrNull?.toIntOrNull() + + private fun parseDeviceId(params: JsonObject?): String? = + readPrimitive(params, "deviceId") + ?.contentOrNull + ?.trim() + ?.takeIf { it.isNotEmpty() } + + private fun parseIncludeAudio(params: JsonObject?): Boolean? { + val value = readPrimitive(params, "includeAudio")?.contentOrNull?.trim()?.lowercase() + return when (value) { + "true" -> true + "false" -> false + else -> null + } } private fun Context.mainExecutor(): Executor = ContextCompat.getMainExecutor(this) + + private fun resolveCameraSelector( + provider: ProcessCameraProvider, + facing: String, + deviceId: String?, + ): CameraSelector { + if (deviceId.isNullOrEmpty()) { + return if (facing == "front") CameraSelector.DEFAULT_FRONT_CAMERA else CameraSelector.DEFAULT_BACK_CAMERA + } + val availableIds = provider.availableCameraInfos.mapNotNull { cameraIdOrNull(it) }.toSet() + if (!availableIds.contains(deviceId)) { + throw IllegalStateException("INVALID_REQUEST: unknown camera deviceId '$deviceId'") + } + return CameraSelector.Builder() + .addCameraFilter { infos -> infos.filter { cameraIdOrNull(it) == deviceId } } + .build() + } + + private fun cameraDeviceInfoOrNull(info: CameraInfo): CameraDeviceInfo? { + val cameraId = cameraIdOrNull(info) ?: return null + val lensFacing = + runCatching { + Camera2CameraInfo.from(info).getCameraCharacteristic(CameraCharacteristics.LENS_FACING) + }.getOrNull() + val position = + when (lensFacing) { + CameraCharacteristics.LENS_FACING_FRONT -> "front" + CameraCharacteristics.LENS_FACING_BACK -> "back" + CameraCharacteristics.LENS_FACING_EXTERNAL -> "external" + else -> "unspecified" + } + val deviceType = + if (lensFacing == CameraCharacteristics.LENS_FACING_EXTERNAL) "external" else "builtIn" + val name = + when (position) { + "front" -> "Front Camera" + "back" -> "Back Camera" + "external" -> "External Camera" + else -> "Camera $cameraId" + } + return CameraDeviceInfo( + id = cameraId, + name = name, + position = position, + deviceType = deviceType, + ) + } + + private fun cameraIdOrNull(info: CameraInfo): String? = + runCatching { Camera2CameraInfo.from(info).cameraId }.getOrNull() } private suspend fun Context.cameraProvider(): ProcessCameraProvider = diff --git a/apps/android/app/src/main/java/ai/openclaw/android/node/CameraHandler.kt b/apps/android/app/src/main/java/ai/openclaw/android/node/CameraHandler.kt index 658c117ff31..0ee22849a62 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/node/CameraHandler.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/node/CameraHandler.kt @@ -3,25 +3,57 @@ package ai.openclaw.android.node import android.content.Context import ai.openclaw.android.CameraHudKind import ai.openclaw.android.BuildConfig -import ai.openclaw.android.SecurePrefs -import ai.openclaw.android.gateway.GatewayEndpoint import ai.openclaw.android.gateway.GatewaySession import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.flow.MutableStateFlow import kotlinx.coroutines.withContext -import okhttp3.MediaType.Companion.toMediaType -import okhttp3.RequestBody.Companion.asRequestBody +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.buildJsonArray +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.contentOrNull +import kotlinx.serialization.json.put + +internal const val CAMERA_CLIP_MAX_RAW_BYTES: Long = 18L * 1024L * 1024L + +internal fun isCameraClipWithinPayloadLimit(rawBytes: Long): Boolean = + rawBytes in 0L..CAMERA_CLIP_MAX_RAW_BYTES class CameraHandler( private val appContext: Context, private val camera: CameraCaptureManager, - private val prefs: SecurePrefs, - private val connectedEndpoint: () -> GatewayEndpoint?, private val externalAudioCaptureActive: MutableStateFlow, private val showCameraHud: (message: String, kind: CameraHudKind, autoHideMs: Long?) -> Unit, private val triggerCameraFlash: () -> Unit, private val invokeErrorFromThrowable: (err: Throwable) -> Pair, ) { + suspend fun handleList(_paramsJson: String?): GatewaySession.InvokeResult { + return try { + val devices = camera.listDevices() + val payload = + buildJsonObject { + put( + "devices", + buildJsonArray { + devices.forEach { device -> + add( + buildJsonObject { + put("id", JsonPrimitive(device.id)) + put("name", JsonPrimitive(device.name)) + put("position", JsonPrimitive(device.position)) + put("deviceType", JsonPrimitive(device.deviceType)) + }, + ) + } + }, + ) + }.toString() + GatewaySession.InvokeResult.ok(payload) + } catch (err: Throwable) { + val (code, message) = invokeErrorFromThrowable(err) + GatewaySession.InvokeResult.error(code = code, message = message) + } + } suspend fun handleSnap(paramsJson: String?): GatewaySession.InvokeResult { val logFile = if (BuildConfig.DEBUG) java.io.File(appContext.cacheDir, "camera_debug.log") else null @@ -69,7 +101,7 @@ class CameraHandler( clipLogFile?.appendText("[CLIP $ts] $msg\n") android.util.Log.w("openclaw", "camera.clip: $msg") } - val includeAudio = paramsJson?.contains("\"includeAudio\":true") != false + val includeAudio = parseIncludeAudio(paramsJson) ?: true if (includeAudio) externalAudioCaptureActive.value = true try { clipLogFile?.writeText("") // clear @@ -89,62 +121,28 @@ class CameraHandler( showCameraHud(message, CameraHudKind.Error, 2400) return GatewaySession.InvokeResult.error(code = code, message = message) } - // Upload file via HTTP instead of base64 through WebSocket - clipLog("uploading via HTTP...") - val uploadUrl = try { - withContext(Dispatchers.IO) { - val ep = connectedEndpoint() - val gatewayHost = if (ep != null) { - val isHttps = ep.tlsEnabled || ep.port == 443 - if (!isHttps) { - clipLog("refusing to upload over plain HTTP — bearer token would be exposed; falling back to base64") - throw Exception("HTTPS required for upload (bearer token protection)") - } - if (ep.port == 443) "https://${ep.host}" else "https://${ep.host}:${ep.port}" - } else { - clipLog("error: no gateway endpoint connected, cannot upload") - throw Exception("no gateway endpoint connected") - } - val token = prefs.loadGatewayToken() ?: "" - val client = okhttp3.OkHttpClient.Builder() - .connectTimeout(10, java.util.concurrent.TimeUnit.SECONDS) - .writeTimeout(120, java.util.concurrent.TimeUnit.SECONDS) - .readTimeout(30, java.util.concurrent.TimeUnit.SECONDS) - .build() - val body = filePayload.file.asRequestBody("video/mp4".toMediaType()) - val req = okhttp3.Request.Builder() - .url("$gatewayHost/upload/clip.mp4") - .put(body) - .header("Authorization", "Bearer $token") - .build() - clipLog("uploading ${filePayload.file.length()} bytes to $gatewayHost/upload/clip.mp4") - val resp = client.newCall(req).execute() - val respBody = resp.body?.string() ?: "" - clipLog("upload response: ${resp.code} $respBody") - filePayload.file.delete() - if (!resp.isSuccessful) throw Exception("upload failed: HTTP ${resp.code}") - // Parse URL from response - val urlMatch = Regex("\"url\":\"([^\"]+)\"").find(respBody) - urlMatch?.groupValues?.get(1) ?: throw Exception("no url in response: $respBody") - } - } catch (err: Throwable) { - clipLog("upload failed: ${err.message}, falling back to base64") - // Fallback to base64 if upload fails - val bytes = withContext(Dispatchers.IO) { - val b = filePayload.file.readBytes() - filePayload.file.delete() - b - } - val base64 = android.util.Base64.encodeToString(bytes, android.util.Base64.NO_WRAP) - showCameraHud("Clip captured", CameraHudKind.Success, 1800) - return GatewaySession.InvokeResult.ok( - """{"format":"mp4","base64":"$base64","durationMs":${filePayload.durationMs},"hasAudio":${filePayload.hasAudio}}""" + val rawBytes = filePayload.file.length() + if (!isCameraClipWithinPayloadLimit(rawBytes)) { + clipLog("payload too large: bytes=$rawBytes max=$CAMERA_CLIP_MAX_RAW_BYTES") + withContext(Dispatchers.IO) { filePayload.file.delete() } + showCameraHud("Clip too large", CameraHudKind.Error, 2400) + return GatewaySession.InvokeResult.error( + code = "PAYLOAD_TOO_LARGE", + message = + "PAYLOAD_TOO_LARGE: camera clip is $rawBytes bytes; max is $CAMERA_CLIP_MAX_RAW_BYTES bytes. Reduce durationMs and retry.", ) } - clipLog("returning URL result: $uploadUrl") + + val bytes = withContext(Dispatchers.IO) { + val b = filePayload.file.readBytes() + filePayload.file.delete() + b + } + val base64 = android.util.Base64.encodeToString(bytes, android.util.Base64.NO_WRAP) + clipLog("returning base64 payload") showCameraHud("Clip captured", CameraHudKind.Success, 1800) return GatewaySession.InvokeResult.ok( - """{"format":"mp4","url":"$uploadUrl","durationMs":${filePayload.durationMs},"hasAudio":${filePayload.hasAudio}}""" + """{"format":"mp4","base64":"$base64","durationMs":${filePayload.durationMs},"hasAudio":${filePayload.hasAudio}}""" ) } catch (err: Throwable) { clipLog("outer error: ${err::class.java.simpleName}: ${err.message}") @@ -154,4 +152,24 @@ class CameraHandler( if (includeAudio) externalAudioCaptureActive.value = false } } + + private fun parseIncludeAudio(paramsJson: String?): Boolean? { + if (paramsJson.isNullOrBlank()) return null + val root = + try { + Json.parseToJsonElement(paramsJson).asObjectOrNull() + } catch (_: Throwable) { + null + } ?: return null + val value = + (root["includeAudio"] as? JsonPrimitive) + ?.contentOrNull + ?.trim() + ?.lowercase() + return when (value) { + "true" -> true + "false" -> false + else -> null + } + } } diff --git a/apps/android/app/src/main/java/ai/openclaw/android/node/CanvasController.kt b/apps/android/app/src/main/java/ai/openclaw/android/node/CanvasController.kt index c46770a6367..d0747ee32b0 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/node/CanvasController.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/node/CanvasController.kt @@ -10,6 +10,9 @@ import androidx.core.graphics.scale import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.suspendCancellableCoroutine import kotlinx.coroutines.withContext +import kotlinx.coroutines.flow.MutableStateFlow +import kotlinx.coroutines.flow.StateFlow +import kotlinx.coroutines.flow.asStateFlow import java.io.ByteArrayOutputStream import android.util.Base64 import org.json.JSONObject @@ -31,6 +34,8 @@ class CanvasController { @Volatile private var debugStatusEnabled: Boolean = false @Volatile private var debugStatusTitle: String? = null @Volatile private var debugStatusSubtitle: String? = null + private val _currentUrl = MutableStateFlow(null) + val currentUrl: StateFlow = _currentUrl.asStateFlow() private val scaffoldAssetUrl = "file:///android_asset/CanvasScaffold/scaffold.html" @@ -45,9 +50,16 @@ class CanvasController { applyDebugStatus() } + fun detach(webView: WebView) { + if (this.webView === webView) { + this.webView = null + } + } + fun navigate(url: String) { val trimmed = url.trim() this.url = if (trimmed.isBlank() || trimmed == "/") null else trimmed + _currentUrl.value = this.url reload() } diff --git a/apps/android/app/src/main/java/ai/openclaw/android/node/ConnectionManager.kt b/apps/android/app/src/main/java/ai/openclaw/android/node/ConnectionManager.kt index d15d928e0a4..021c5fe2ce6 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/node/ConnectionManager.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/node/ConnectionManager.kt @@ -7,13 +7,6 @@ import ai.openclaw.android.gateway.GatewayClientInfo import ai.openclaw.android.gateway.GatewayConnectOptions import ai.openclaw.android.gateway.GatewayEndpoint import ai.openclaw.android.gateway.GatewayTlsParams -import ai.openclaw.android.protocol.OpenClawCanvasA2UICommand -import ai.openclaw.android.protocol.OpenClawCanvasCommand -import ai.openclaw.android.protocol.OpenClawCameraCommand -import ai.openclaw.android.protocol.OpenClawLocationCommand -import ai.openclaw.android.protocol.OpenClawScreenCommand -import ai.openclaw.android.protocol.OpenClawSmsCommand -import ai.openclaw.android.protocol.OpenClawCapability import ai.openclaw.android.LocationMode import ai.openclaw.android.VoiceWakeMode @@ -22,6 +15,8 @@ class ConnectionManager( private val cameraEnabled: () -> Boolean, private val locationMode: () -> LocationMode, private val voiceWakeMode: () -> VoiceWakeMode, + private val motionActivityAvailable: () -> Boolean, + private val motionPedometerAvailable: () -> Boolean, private val smsAvailable: () -> Boolean, private val hasRecordAudioPermission: () -> Boolean, private val manualTls: () -> Boolean, @@ -79,47 +74,20 @@ class ConnectionManager( } } - fun buildInvokeCommands(): List = - buildList { - add(OpenClawCanvasCommand.Present.rawValue) - add(OpenClawCanvasCommand.Hide.rawValue) - add(OpenClawCanvasCommand.Navigate.rawValue) - add(OpenClawCanvasCommand.Eval.rawValue) - add(OpenClawCanvasCommand.Snapshot.rawValue) - add(OpenClawCanvasA2UICommand.Push.rawValue) - add(OpenClawCanvasA2UICommand.PushJSONL.rawValue) - add(OpenClawCanvasA2UICommand.Reset.rawValue) - add(OpenClawScreenCommand.Record.rawValue) - if (cameraEnabled()) { - add(OpenClawCameraCommand.Snap.rawValue) - add(OpenClawCameraCommand.Clip.rawValue) - } - if (locationMode() != LocationMode.Off) { - add(OpenClawLocationCommand.Get.rawValue) - } - if (smsAvailable()) { - add(OpenClawSmsCommand.Send.rawValue) - } - if (BuildConfig.DEBUG) { - add("debug.logs") - add("debug.ed25519") - } - add("app.update") - } + private fun runtimeFlags(): NodeRuntimeFlags = + NodeRuntimeFlags( + cameraEnabled = cameraEnabled(), + locationEnabled = locationMode() != LocationMode.Off, + smsAvailable = smsAvailable(), + voiceWakeEnabled = voiceWakeMode() != VoiceWakeMode.Off && hasRecordAudioPermission(), + motionActivityAvailable = motionActivityAvailable(), + motionPedometerAvailable = motionPedometerAvailable(), + debugBuild = BuildConfig.DEBUG, + ) - fun buildCapabilities(): List = - buildList { - add(OpenClawCapability.Canvas.rawValue) - add(OpenClawCapability.Screen.rawValue) - if (cameraEnabled()) add(OpenClawCapability.Camera.rawValue) - if (smsAvailable()) add(OpenClawCapability.Sms.rawValue) - if (voiceWakeMode() != VoiceWakeMode.Off && hasRecordAudioPermission()) { - add(OpenClawCapability.VoiceWake.rawValue) - } - if (locationMode() != LocationMode.Off) { - add(OpenClawCapability.Location.rawValue) - } - } + fun buildInvokeCommands(): List = InvokeCommandRegistry.advertisedCommands(runtimeFlags()) + + fun buildCapabilities(): List = InvokeCommandRegistry.advertisedCapabilities(runtimeFlags()) fun resolvedVersionName(): String { val versionName = BuildConfig.VERSION_NAME.trim().ifEmpty { "dev" } @@ -176,7 +144,7 @@ class ConnectionManager( caps = emptyList(), commands = emptyList(), permissions = emptyMap(), - client = buildClientInfo(clientId = "openclaw-control-ui", clientMode = "ui"), + client = buildClientInfo(clientId = "openclaw-android", clientMode = "ui"), userAgent = buildUserAgent(), ) } diff --git a/apps/android/app/src/main/java/ai/openclaw/android/node/ContactsHandler.kt b/apps/android/app/src/main/java/ai/openclaw/android/node/ContactsHandler.kt new file mode 100644 index 00000000000..6fb01a463ea --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/node/ContactsHandler.kt @@ -0,0 +1,423 @@ +package ai.openclaw.android.node + +import android.Manifest +import android.content.ContentProviderOperation +import android.content.ContentResolver +import android.content.ContentValues +import android.content.Context +import android.provider.ContactsContract +import androidx.core.content.ContextCompat +import ai.openclaw.android.gateway.GatewaySession +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonArray +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.buildJsonArray +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.put + +private const val DEFAULT_CONTACTS_LIMIT = 25 + +internal data class ContactRecord( + val identifier: String, + val displayName: String, + val givenName: String, + val familyName: String, + val organizationName: String, + val phoneNumbers: List, + val emails: List, +) + +internal data class ContactsSearchRequest( + val query: String?, + val limit: Int, +) + +internal data class ContactsAddRequest( + val givenName: String?, + val familyName: String?, + val organizationName: String?, + val displayName: String?, + val phoneNumbers: List, + val emails: List, +) + +internal interface ContactsDataSource { + fun hasReadPermission(context: Context): Boolean + + fun hasWritePermission(context: Context): Boolean + + fun search(context: Context, request: ContactsSearchRequest): List + + fun add(context: Context, request: ContactsAddRequest): ContactRecord +} + +private object SystemContactsDataSource : ContactsDataSource { + override fun hasReadPermission(context: Context): Boolean { + return ContextCompat.checkSelfPermission(context, Manifest.permission.READ_CONTACTS) == + android.content.pm.PackageManager.PERMISSION_GRANTED + } + + override fun hasWritePermission(context: Context): Boolean { + return ContextCompat.checkSelfPermission(context, Manifest.permission.WRITE_CONTACTS) == + android.content.pm.PackageManager.PERMISSION_GRANTED + } + + override fun search(context: Context, request: ContactsSearchRequest): List { + val resolver = context.contentResolver + val projection = + arrayOf( + ContactsContract.Contacts._ID, + ContactsContract.Contacts.DISPLAY_NAME_PRIMARY, + ) + val selection: String? + val selectionArgs: Array? + if (request.query.isNullOrBlank()) { + selection = null + selectionArgs = null + } else { + selection = "${ContactsContract.Contacts.DISPLAY_NAME_PRIMARY} LIKE ?" + selectionArgs = arrayOf("%${request.query}%") + } + val sortOrder = "${ContactsContract.Contacts.DISPLAY_NAME_PRIMARY} COLLATE NOCASE ASC LIMIT ${request.limit}" + resolver.query( + ContactsContract.Contacts.CONTENT_URI, + projection, + selection, + selectionArgs, + sortOrder, + ).use { cursor -> + if (cursor == null) return emptyList() + val idIndex = cursor.getColumnIndexOrThrow(ContactsContract.Contacts._ID) + val displayNameIndex = cursor.getColumnIndexOrThrow(ContactsContract.Contacts.DISPLAY_NAME_PRIMARY) + val out = mutableListOf() + while (cursor.moveToNext() && out.size < request.limit) { + val contactId = cursor.getLong(idIndex) + val displayName = cursor.getString(displayNameIndex).orEmpty() + out += loadContactRecord(resolver, contactId, fallbackDisplayName = displayName) + } + return out + } + } + + override fun add(context: Context, request: ContactsAddRequest): ContactRecord { + val resolver = context.contentResolver + val operations = ArrayList() + operations += + ContentProviderOperation.newInsert(ContactsContract.RawContacts.CONTENT_URI) + .withValue(ContactsContract.RawContacts.ACCOUNT_TYPE, null) + .withValue(ContactsContract.RawContacts.ACCOUNT_NAME, null) + .build() + if (!request.givenName.isNullOrEmpty() || !request.familyName.isNullOrEmpty() || !request.displayName.isNullOrEmpty()) { + operations += + ContentProviderOperation.newInsert(ContactsContract.Data.CONTENT_URI) + .withValueBackReference(ContactsContract.Data.RAW_CONTACT_ID, 0) + .withValue(ContactsContract.Data.MIMETYPE, ContactsContract.CommonDataKinds.StructuredName.CONTENT_ITEM_TYPE) + .withValue(ContactsContract.CommonDataKinds.StructuredName.GIVEN_NAME, request.givenName) + .withValue(ContactsContract.CommonDataKinds.StructuredName.FAMILY_NAME, request.familyName) + .withValue(ContactsContract.CommonDataKinds.StructuredName.DISPLAY_NAME, request.displayName) + .build() + } + if (!request.organizationName.isNullOrEmpty()) { + operations += + ContentProviderOperation.newInsert(ContactsContract.Data.CONTENT_URI) + .withValueBackReference(ContactsContract.Data.RAW_CONTACT_ID, 0) + .withValue(ContactsContract.Data.MIMETYPE, ContactsContract.CommonDataKinds.Organization.CONTENT_ITEM_TYPE) + .withValue(ContactsContract.CommonDataKinds.Organization.COMPANY, request.organizationName) + .build() + } + request.phoneNumbers.forEach { number -> + operations += + ContentProviderOperation.newInsert(ContactsContract.Data.CONTENT_URI) + .withValueBackReference(ContactsContract.Data.RAW_CONTACT_ID, 0) + .withValue(ContactsContract.Data.MIMETYPE, ContactsContract.CommonDataKinds.Phone.CONTENT_ITEM_TYPE) + .withValue(ContactsContract.CommonDataKinds.Phone.NUMBER, number) + .withValue(ContactsContract.CommonDataKinds.Phone.TYPE, ContactsContract.CommonDataKinds.Phone.TYPE_MOBILE) + .build() + } + request.emails.forEach { email -> + operations += + ContentProviderOperation.newInsert(ContactsContract.Data.CONTENT_URI) + .withValueBackReference(ContactsContract.Data.RAW_CONTACT_ID, 0) + .withValue(ContactsContract.Data.MIMETYPE, ContactsContract.CommonDataKinds.Email.CONTENT_ITEM_TYPE) + .withValue(ContactsContract.CommonDataKinds.Email.ADDRESS, email) + .withValue(ContactsContract.CommonDataKinds.Email.TYPE, ContactsContract.CommonDataKinds.Email.TYPE_HOME) + .build() + } + + val results = resolver.applyBatch(ContactsContract.AUTHORITY, operations) + val rawContactUri = results.firstOrNull()?.uri + ?: throw IllegalStateException("contact insert failed") + val rawContactId = rawContactUri.lastPathSegment?.toLongOrNull() + ?: throw IllegalStateException("contact insert failed") + val contactId = resolveContactIdForRawContact(resolver, rawContactId) + ?: throw IllegalStateException("contact insert failed") + return loadContactRecord( + resolver = resolver, + contactId = contactId, + fallbackDisplayName = request.displayName.orEmpty(), + ) + } + + private fun resolveContactIdForRawContact(resolver: ContentResolver, rawContactId: Long): Long? { + val projection = arrayOf(ContactsContract.RawContacts.CONTACT_ID) + resolver.query( + ContactsContract.RawContacts.CONTENT_URI, + projection, + "${ContactsContract.RawContacts._ID}=?", + arrayOf(rawContactId.toString()), + null, + ).use { cursor -> + if (cursor == null || !cursor.moveToFirst()) return null + val index = cursor.getColumnIndexOrThrow(ContactsContract.RawContacts.CONTACT_ID) + return cursor.getLong(index) + } + } + + private fun loadContactRecord( + resolver: ContentResolver, + contactId: Long, + fallbackDisplayName: String, + ): ContactRecord { + val nameRow = loadNameRow(resolver, contactId) + val organization = loadOrganization(resolver, contactId) + val phones = loadPhones(resolver, contactId) + val emails = loadEmails(resolver, contactId) + val displayName = + when { + !nameRow.displayName.isNullOrEmpty() -> nameRow.displayName + !fallbackDisplayName.isNullOrEmpty() -> fallbackDisplayName + else -> listOfNotNull(nameRow.givenName, nameRow.familyName).joinToString(" ").trim() + }.ifEmpty { "(unnamed)" } + return ContactRecord( + identifier = contactId.toString(), + displayName = displayName, + givenName = nameRow.givenName.orEmpty(), + familyName = nameRow.familyName.orEmpty(), + organizationName = organization.orEmpty(), + phoneNumbers = phones, + emails = emails, + ) + } + + private data class NameRow( + val givenName: String?, + val familyName: String?, + val displayName: String?, + ) + + private fun loadNameRow(resolver: ContentResolver, contactId: Long): NameRow { + val projection = + arrayOf( + ContactsContract.CommonDataKinds.StructuredName.GIVEN_NAME, + ContactsContract.CommonDataKinds.StructuredName.FAMILY_NAME, + ContactsContract.CommonDataKinds.StructuredName.DISPLAY_NAME, + ) + resolver.query( + ContactsContract.Data.CONTENT_URI, + projection, + "${ContactsContract.Data.CONTACT_ID}=? AND ${ContactsContract.Data.MIMETYPE}=?", + arrayOf( + contactId.toString(), + ContactsContract.CommonDataKinds.StructuredName.CONTENT_ITEM_TYPE, + ), + null, + ).use { cursor -> + if (cursor == null || !cursor.moveToFirst()) { + return NameRow(givenName = null, familyName = null, displayName = null) + } + val given = cursor.getString(0)?.trim()?.ifEmpty { null } + val family = cursor.getString(1)?.trim()?.ifEmpty { null } + val display = cursor.getString(2)?.trim()?.ifEmpty { null } + return NameRow(givenName = given, familyName = family, displayName = display) + } + } + + private fun loadOrganization(resolver: ContentResolver, contactId: Long): String? { + val projection = arrayOf(ContactsContract.CommonDataKinds.Organization.COMPANY) + resolver.query( + ContactsContract.Data.CONTENT_URI, + projection, + "${ContactsContract.Data.CONTACT_ID}=? AND ${ContactsContract.Data.MIMETYPE}=?", + arrayOf(contactId.toString(), ContactsContract.CommonDataKinds.Organization.CONTENT_ITEM_TYPE), + null, + ).use { cursor -> + if (cursor == null || !cursor.moveToFirst()) return null + return cursor.getString(0)?.trim()?.ifEmpty { null } + } + } + + private fun loadPhones(resolver: ContentResolver, contactId: Long): List { + val projection = arrayOf(ContactsContract.CommonDataKinds.Phone.NUMBER) + resolver.query( + ContactsContract.CommonDataKinds.Phone.CONTENT_URI, + projection, + "${ContactsContract.CommonDataKinds.Phone.CONTACT_ID}=?", + arrayOf(contactId.toString()), + null, + ).use { cursor -> + if (cursor == null) return emptyList() + val out = LinkedHashSet() + while (cursor.moveToNext()) { + val value = cursor.getString(0)?.trim().orEmpty() + if (value.isNotEmpty()) out += value + } + return out.toList() + } + } + + private fun loadEmails(resolver: ContentResolver, contactId: Long): List { + val projection = arrayOf(ContactsContract.CommonDataKinds.Email.ADDRESS) + resolver.query( + ContactsContract.CommonDataKinds.Email.CONTENT_URI, + projection, + "${ContactsContract.CommonDataKinds.Email.CONTACT_ID}=?", + arrayOf(contactId.toString()), + null, + ).use { cursor -> + if (cursor == null) return emptyList() + val out = LinkedHashSet() + while (cursor.moveToNext()) { + val value = cursor.getString(0)?.trim().orEmpty() + if (value.isNotEmpty()) out += value + } + return out.toList() + } + } +} + +class ContactsHandler private constructor( + private val appContext: Context, + private val dataSource: ContactsDataSource, +) { + constructor(appContext: Context) : this(appContext = appContext, dataSource = SystemContactsDataSource) + + fun handleContactsSearch(paramsJson: String?): GatewaySession.InvokeResult { + if (!dataSource.hasReadPermission(appContext)) { + return GatewaySession.InvokeResult.error( + code = "CONTACTS_PERMISSION_REQUIRED", + message = "CONTACTS_PERMISSION_REQUIRED: grant Contacts permission", + ) + } + val request = + parseSearchRequest(paramsJson) + ?: return GatewaySession.InvokeResult.error( + code = "INVALID_REQUEST", + message = "INVALID_REQUEST: expected JSON object", + ) + return try { + val contacts = dataSource.search(appContext, request) + GatewaySession.InvokeResult.ok( + buildJsonObject { + put( + "contacts", + buildJsonArray { + contacts.forEach { add(contactJson(it)) } + }, + ) + }.toString(), + ) + } catch (err: Throwable) { + GatewaySession.InvokeResult.error( + code = "CONTACTS_UNAVAILABLE", + message = "CONTACTS_UNAVAILABLE: ${err.message ?: "contacts query failed"}", + ) + } + } + + fun handleContactsAdd(paramsJson: String?): GatewaySession.InvokeResult { + if (!dataSource.hasWritePermission(appContext)) { + return GatewaySession.InvokeResult.error( + code = "CONTACTS_PERMISSION_REQUIRED", + message = "CONTACTS_PERMISSION_REQUIRED: grant Contacts permission", + ) + } + val request = + parseAddRequest(paramsJson) + ?: return GatewaySession.InvokeResult.error( + code = "INVALID_REQUEST", + message = "INVALID_REQUEST: expected JSON object", + ) + val hasName = + !(request.givenName.isNullOrEmpty() && request.familyName.isNullOrEmpty() && request.displayName.isNullOrEmpty()) + val hasOrg = !request.organizationName.isNullOrEmpty() + val hasDetails = request.phoneNumbers.isNotEmpty() || request.emails.isNotEmpty() + if (!hasName && !hasOrg && !hasDetails) { + return GatewaySession.InvokeResult.error( + code = "CONTACTS_INVALID", + message = "CONTACTS_INVALID: include a name, organization, phone, or email", + ) + } + return try { + val contact = dataSource.add(appContext, request) + GatewaySession.InvokeResult.ok( + buildJsonObject { + put("contact", contactJson(contact)) + }.toString(), + ) + } catch (err: Throwable) { + GatewaySession.InvokeResult.error( + code = "CONTACTS_UNAVAILABLE", + message = "CONTACTS_UNAVAILABLE: ${err.message ?: "contact add failed"}", + ) + } + } + + private fun parseSearchRequest(paramsJson: String?): ContactsSearchRequest? { + if (paramsJson.isNullOrBlank()) { + return ContactsSearchRequest(query = null, limit = DEFAULT_CONTACTS_LIMIT) + } + val params = + try { + Json.parseToJsonElement(paramsJson).asObjectOrNull() + } catch (_: Throwable) { + null + } ?: return null + val query = (params["query"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null } + val limit = ((params["limit"] as? JsonPrimitive)?.content?.toIntOrNull() ?: DEFAULT_CONTACTS_LIMIT).coerceIn(1, 200) + return ContactsSearchRequest(query = query, limit = limit) + } + + private fun parseAddRequest(paramsJson: String?): ContactsAddRequest? { + val params = + try { + paramsJson?.let { Json.parseToJsonElement(it).asObjectOrNull() } + } catch (_: Throwable) { + null + } ?: return null + return ContactsAddRequest( + givenName = (params["givenName"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null }, + familyName = (params["familyName"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null }, + organizationName = (params["organizationName"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null }, + displayName = (params["displayName"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null }, + phoneNumbers = stringArray(params["phoneNumbers"] as? JsonArray), + emails = stringArray(params["emails"] as? JsonArray).map { it.lowercase() }, + ) + } + + private fun stringArray(array: JsonArray?): List { + if (array == null) return emptyList() + return array.mapNotNull { element -> + (element as? JsonPrimitive)?.content?.trim()?.ifEmpty { null } + } + } + + private fun contactJson(contact: ContactRecord): JsonObject { + return buildJsonObject { + put("identifier", JsonPrimitive(contact.identifier)) + put("displayName", JsonPrimitive(contact.displayName)) + put("givenName", JsonPrimitive(contact.givenName)) + put("familyName", JsonPrimitive(contact.familyName)) + put("organizationName", JsonPrimitive(contact.organizationName)) + put("phoneNumbers", buildJsonArray { contact.phoneNumbers.forEach { add(JsonPrimitive(it)) } }) + put("emails", buildJsonArray { contact.emails.forEach { add(JsonPrimitive(it)) } }) + } + } + + companion object { + internal fun forTesting( + appContext: Context, + dataSource: ContactsDataSource, + ): ContactsHandler = ContactsHandler(appContext = appContext, dataSource = dataSource) + } +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/node/DebugHandler.kt b/apps/android/app/src/main/java/ai/openclaw/android/node/DebugHandler.kt index 49502bd3631..2b0fc04e437 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/node/DebugHandler.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/node/DebugHandler.kt @@ -62,7 +62,8 @@ class DebugHandler( results.add("Signature.Ed25519: FAILED - ${e.javaClass.simpleName}: ${e.message}") } - return GatewaySession.InvokeResult.ok("""{"diagnostics":"${results.joinToString("\\n").replace("\"", "\\\"")}"}"""") + val diagnostics = results.joinToString("\n") + return GatewaySession.InvokeResult.ok("""{"diagnostics":${JsonPrimitive(diagnostics)}}""") } catch (e: Throwable) { return GatewaySession.InvokeResult.error(code = "ED25519_TEST_FAILED", message = "${e.javaClass.simpleName}: ${e.message}\n${e.stackTraceToString().take(500)}") } diff --git a/apps/android/app/src/main/java/ai/openclaw/android/node/DeviceHandler.kt b/apps/android/app/src/main/java/ai/openclaw/android/node/DeviceHandler.kt new file mode 100644 index 00000000000..a091b6f211b --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/node/DeviceHandler.kt @@ -0,0 +1,418 @@ +package ai.openclaw.android.node + +import android.Manifest +import android.app.ActivityManager +import android.content.Context +import android.content.Intent +import android.content.IntentFilter +import android.content.pm.PackageManager +import android.net.ConnectivityManager +import android.net.NetworkCapabilities +import android.os.BatteryManager +import android.os.Build +import android.os.Environment +import android.os.PowerManager +import android.os.StatFs +import android.os.SystemClock +import androidx.core.content.ContextCompat +import ai.openclaw.android.BuildConfig +import ai.openclaw.android.gateway.GatewaySession +import java.util.Locale +import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.buildJsonArray +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.put + +class DeviceHandler( + private val appContext: Context, +) { + private data class BatterySnapshot( + val status: Int, + val plugged: Int, + val levelFraction: Double?, + val temperatureC: Double?, + ) + + fun handleDeviceStatus(_paramsJson: String?): GatewaySession.InvokeResult { + return GatewaySession.InvokeResult.ok(statusPayloadJson()) + } + + fun handleDeviceInfo(_paramsJson: String?): GatewaySession.InvokeResult { + return GatewaySession.InvokeResult.ok(infoPayloadJson()) + } + + fun handleDevicePermissions(_paramsJson: String?): GatewaySession.InvokeResult { + return GatewaySession.InvokeResult.ok(permissionsPayloadJson()) + } + + fun handleDeviceHealth(_paramsJson: String?): GatewaySession.InvokeResult { + return GatewaySession.InvokeResult.ok(healthPayloadJson()) + } + + private fun statusPayloadJson(): String { + val battery = readBatterySnapshot() + val powerManager = appContext.getSystemService(PowerManager::class.java) + val storage = StatFs(Environment.getDataDirectory().absolutePath) + val totalBytes = storage.totalBytes + val freeBytes = storage.availableBytes + val usedBytes = (totalBytes - freeBytes).coerceAtLeast(0L) + val connectivity = appContext.getSystemService(ConnectivityManager::class.java) + val activeNetwork = connectivity?.activeNetwork + val caps = activeNetwork?.let { connectivity.getNetworkCapabilities(it) } + val uptimeSeconds = SystemClock.elapsedRealtime() / 1_000.0 + + return buildJsonObject { + put( + "battery", + buildJsonObject { + battery.levelFraction?.let { put("level", JsonPrimitive(it)) } + put("state", JsonPrimitive(mapBatteryState(battery.status))) + put("lowPowerModeEnabled", JsonPrimitive(powerManager?.isPowerSaveMode == true)) + }, + ) + put( + "thermal", + buildJsonObject { + put("state", JsonPrimitive(mapThermalState(powerManager))) + }, + ) + put( + "storage", + buildJsonObject { + put("totalBytes", JsonPrimitive(totalBytes)) + put("freeBytes", JsonPrimitive(freeBytes)) + put("usedBytes", JsonPrimitive(usedBytes)) + }, + ) + put( + "network", + buildJsonObject { + put("status", JsonPrimitive(mapNetworkStatus(caps))) + put( + "isExpensive", + JsonPrimitive( + caps?.hasCapability(NetworkCapabilities.NET_CAPABILITY_NOT_METERED)?.not() ?: false, + ), + ) + put( + "isConstrained", + JsonPrimitive( + caps?.hasCapability(NetworkCapabilities.NET_CAPABILITY_NOT_RESTRICTED)?.not() ?: false, + ), + ) + put("interfaces", networkInterfacesJson(caps)) + }, + ) + put("uptimeSeconds", JsonPrimitive(uptimeSeconds)) + }.toString() + } + + private fun infoPayloadJson(): String { + val model = Build.MODEL?.trim().orEmpty() + val manufacturer = Build.MANUFACTURER?.trim().orEmpty() + val modelIdentifier = Build.DEVICE?.trim().orEmpty() + val systemVersion = Build.VERSION.RELEASE?.trim().orEmpty() + val locale = Locale.getDefault().toLanguageTag().trim() + val appVersion = BuildConfig.VERSION_NAME.trim() + val appBuild = BuildConfig.VERSION_CODE.toString() + + return buildJsonObject { + put("deviceName", JsonPrimitive(model.ifEmpty { "Android" })) + put("modelIdentifier", JsonPrimitive(modelIdentifier.ifEmpty { listOf(manufacturer, model).filter { it.isNotEmpty() }.joinToString(" ") })) + put("systemName", JsonPrimitive("Android")) + put("systemVersion", JsonPrimitive(systemVersion.ifEmpty { Build.VERSION.SDK_INT.toString() })) + put("appVersion", JsonPrimitive(appVersion.ifEmpty { "dev" })) + put("appBuild", JsonPrimitive(appBuild.ifEmpty { "0" })) + put("locale", JsonPrimitive(locale.ifEmpty { Locale.getDefault().toString() })) + }.toString() + } + + private fun permissionsPayloadJson(): String { + val canSendSms = appContext.packageManager.hasSystemFeature(PackageManager.FEATURE_TELEPHONY) + val notificationAccess = DeviceNotificationListenerService.isAccessEnabled(appContext) + val photosGranted = + if (Build.VERSION.SDK_INT >= 33) { + hasPermission(Manifest.permission.READ_MEDIA_IMAGES) + } else { + hasPermission(Manifest.permission.READ_EXTERNAL_STORAGE) + } + val motionGranted = + if (Build.VERSION.SDK_INT >= 29) { + hasPermission(Manifest.permission.ACTIVITY_RECOGNITION) + } else { + true + } + val notificationsGranted = + if (Build.VERSION.SDK_INT >= 33) { + hasPermission(Manifest.permission.POST_NOTIFICATIONS) + } else { + true + } + return buildJsonObject { + put( + "permissions", + buildJsonObject { + put( + "camera", + permissionStateJson( + granted = hasPermission(Manifest.permission.CAMERA), + promptableWhenDenied = true, + ), + ) + put( + "microphone", + permissionStateJson( + granted = hasPermission(Manifest.permission.RECORD_AUDIO), + promptableWhenDenied = true, + ), + ) + put( + "location", + permissionStateJson( + granted = + hasPermission(Manifest.permission.ACCESS_FINE_LOCATION) || + hasPermission(Manifest.permission.ACCESS_COARSE_LOCATION), + promptableWhenDenied = true, + ), + ) + put( + "backgroundLocation", + permissionStateJson( + granted = hasPermission(Manifest.permission.ACCESS_BACKGROUND_LOCATION), + promptableWhenDenied = true, + ), + ) + put( + "sms", + permissionStateJson( + granted = hasPermission(Manifest.permission.SEND_SMS) && canSendSms, + promptableWhenDenied = canSendSms, + ), + ) + put( + "notificationListener", + permissionStateJson( + granted = notificationAccess, + promptableWhenDenied = true, + ), + ) + put( + "notifications", + permissionStateJson( + granted = notificationsGranted, + promptableWhenDenied = true, + ), + ) + put( + "photos", + permissionStateJson( + granted = photosGranted, + promptableWhenDenied = true, + ), + ) + put( + "contacts", + permissionStateJson( + granted = hasPermission(Manifest.permission.READ_CONTACTS), + promptableWhenDenied = true, + ), + ) + put( + "calendar", + permissionStateJson( + granted = hasPermission(Manifest.permission.READ_CALENDAR), + promptableWhenDenied = true, + ), + ) + put( + "motion", + permissionStateJson( + granted = motionGranted, + promptableWhenDenied = Build.VERSION.SDK_INT >= 29, + ), + ) + // Screen capture on Android is interactive per-capture consent, not a sticky app permission. + put( + "screenCapture", + permissionStateJson( + granted = false, + promptableWhenDenied = true, + ), + ) + }, + ) + }.toString() + } + + private fun healthPayloadJson(): String { + val battery = readBatterySnapshot() + val batteryManager = appContext.getSystemService(BatteryManager::class.java) + val currentNowUa = batteryManager?.getLongProperty(BatteryManager.BATTERY_PROPERTY_CURRENT_NOW) + val currentNowMa = + if (currentNowUa == null || currentNowUa == Long.MIN_VALUE) { + null + } else { + currentNowUa.toDouble() / 1_000.0 + } + + val powerManager = appContext.getSystemService(PowerManager::class.java) + val activityManager = appContext.getSystemService(ActivityManager::class.java) + val memoryInfo = ActivityManager.MemoryInfo() + activityManager?.getMemoryInfo(memoryInfo) + val totalRamBytes = memoryInfo.totalMem.coerceAtLeast(0L) + val availableRamBytes = memoryInfo.availMem.coerceAtLeast(0L) + val usedRamBytes = (totalRamBytes - availableRamBytes).coerceAtLeast(0L) + val lowMemory = memoryInfo.lowMemory + val memoryPressure = mapMemoryPressure(totalRamBytes, availableRamBytes, lowMemory) + + return buildJsonObject { + put( + "memory", + buildJsonObject { + put("pressure", JsonPrimitive(memoryPressure)) + put("totalRamBytes", JsonPrimitive(totalRamBytes)) + put("availableRamBytes", JsonPrimitive(availableRamBytes)) + put("usedRamBytes", JsonPrimitive(usedRamBytes)) + put("thresholdBytes", JsonPrimitive(memoryInfo.threshold.coerceAtLeast(0L))) + put("lowMemory", JsonPrimitive(lowMemory)) + }, + ) + put( + "battery", + buildJsonObject { + put("state", JsonPrimitive(mapBatteryState(battery.status))) + put("chargingType", JsonPrimitive(mapChargingType(battery.plugged))) + battery.temperatureC?.let { put("temperatureC", JsonPrimitive(it)) } + currentNowMa?.let { put("currentMa", JsonPrimitive(it)) } + }, + ) + put( + "power", + buildJsonObject { + put("dozeModeEnabled", JsonPrimitive(powerManager?.isDeviceIdleMode == true)) + put("lowPowerModeEnabled", JsonPrimitive(powerManager?.isPowerSaveMode == true)) + }, + ) + put( + "system", + buildJsonObject { + Build.VERSION.SECURITY_PATCH + ?.trim() + ?.takeIf { it.isNotEmpty() } + ?.let { put("securityPatchLevel", JsonPrimitive(it)) } + }, + ) + }.toString() + } + + private fun readBatterySnapshot(): BatterySnapshot { + val intent = appContext.registerReceiver(null, IntentFilter(Intent.ACTION_BATTERY_CHANGED)) + val status = + intent?.getIntExtra(BatteryManager.EXTRA_STATUS, BatteryManager.BATTERY_STATUS_UNKNOWN) + ?: BatteryManager.BATTERY_STATUS_UNKNOWN + val plugged = intent?.getIntExtra(BatteryManager.EXTRA_PLUGGED, 0) ?: 0 + val temperatureC = + intent + ?.getIntExtra(BatteryManager.EXTRA_TEMPERATURE, Int.MIN_VALUE) + ?.takeIf { it != Int.MIN_VALUE } + ?.toDouble() + ?.div(10.0) + return BatterySnapshot( + status = status, + plugged = plugged, + levelFraction = batteryLevelFraction(intent), + temperatureC = temperatureC, + ) + } + + private fun batteryLevelFraction(intent: Intent?): Double? { + val rawLevel = intent?.getIntExtra(BatteryManager.EXTRA_LEVEL, -1) ?: -1 + val rawScale = intent?.getIntExtra(BatteryManager.EXTRA_SCALE, -1) ?: -1 + if (rawLevel < 0 || rawScale <= 0) return null + return rawLevel.toDouble() / rawScale.toDouble() + } + + private fun mapBatteryState(status: Int): String { + return when (status) { + BatteryManager.BATTERY_STATUS_CHARGING -> "charging" + BatteryManager.BATTERY_STATUS_FULL -> "full" + BatteryManager.BATTERY_STATUS_DISCHARGING, BatteryManager.BATTERY_STATUS_NOT_CHARGING -> "unplugged" + else -> "unknown" + } + } + + private fun mapChargingType(plugged: Int): String { + return when (plugged) { + BatteryManager.BATTERY_PLUGGED_AC -> "ac" + BatteryManager.BATTERY_PLUGGED_USB -> "usb" + BatteryManager.BATTERY_PLUGGED_WIRELESS -> "wireless" + BatteryManager.BATTERY_PLUGGED_DOCK -> "dock" + else -> "none" + } + } + + private fun mapThermalState(powerManager: PowerManager?): String { + val thermal = powerManager?.currentThermalStatus ?: return "nominal" + return when (thermal) { + PowerManager.THERMAL_STATUS_NONE, PowerManager.THERMAL_STATUS_LIGHT -> "nominal" + PowerManager.THERMAL_STATUS_MODERATE -> "fair" + PowerManager.THERMAL_STATUS_SEVERE -> "serious" + PowerManager.THERMAL_STATUS_CRITICAL, + PowerManager.THERMAL_STATUS_EMERGENCY, + PowerManager.THERMAL_STATUS_SHUTDOWN -> "critical" + else -> "nominal" + } + } + + private fun mapNetworkStatus(caps: NetworkCapabilities?): String { + if (caps == null) return "unsatisfied" + return when { + caps.hasCapability(NetworkCapabilities.NET_CAPABILITY_VALIDATED) -> "satisfied" + caps.hasCapability(NetworkCapabilities.NET_CAPABILITY_INTERNET) -> "requiresConnection" + else -> "unsatisfied" + } + } + + private fun permissionStateJson(granted: Boolean, promptableWhenDenied: Boolean) = + buildJsonObject { + put("status", JsonPrimitive(if (granted) "granted" else "denied")) + put("promptable", JsonPrimitive(!granted && promptableWhenDenied)) + } + + private fun hasPermission(permission: String): Boolean { + return ( + ContextCompat.checkSelfPermission(appContext, permission) == PackageManager.PERMISSION_GRANTED + ) + } + + private fun mapMemoryPressure(totalBytes: Long, availableBytes: Long, lowMemory: Boolean): String { + if (totalBytes <= 0L) return if (lowMemory) "critical" else "unknown" + if (lowMemory) return "critical" + val freeRatio = availableBytes.toDouble() / totalBytes.toDouble() + return when { + freeRatio <= 0.05 -> "critical" + freeRatio <= 0.15 -> "high" + freeRatio <= 0.30 -> "moderate" + else -> "normal" + } + } + + private fun networkInterfacesJson(caps: NetworkCapabilities?) = + buildJsonArray { + if (caps == null) return@buildJsonArray + var hasKnownTransport = false + if (caps.hasTransport(NetworkCapabilities.TRANSPORT_WIFI)) { + hasKnownTransport = true + add(JsonPrimitive("wifi")) + } + if (caps.hasTransport(NetworkCapabilities.TRANSPORT_CELLULAR)) { + hasKnownTransport = true + add(JsonPrimitive("cellular")) + } + if (caps.hasTransport(NetworkCapabilities.TRANSPORT_ETHERNET)) { + hasKnownTransport = true + add(JsonPrimitive("wired")) + } + if (!hasKnownTransport) add(JsonPrimitive("other")) + } +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/node/DeviceNotificationListenerService.kt b/apps/android/app/src/main/java/ai/openclaw/android/node/DeviceNotificationListenerService.kt new file mode 100644 index 00000000000..0663bb8158d --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/node/DeviceNotificationListenerService.kt @@ -0,0 +1,365 @@ +package ai.openclaw.android.node + +import android.app.Notification +import android.app.NotificationManager +import android.app.RemoteInput +import android.content.ComponentName +import android.content.Context +import android.content.Intent +import android.os.Build +import android.service.notification.NotificationListenerService +import android.service.notification.StatusBarNotification +import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.put + +private const val MAX_NOTIFICATION_TEXT_CHARS = 512 +private const val NOTIFICATIONS_CHANGED_EVENT = "notifications.changed" + +internal fun sanitizeNotificationText(value: CharSequence?): String? { + val normalized = value?.toString()?.trim().orEmpty() + return normalized.take(MAX_NOTIFICATION_TEXT_CHARS).ifEmpty { null } +} + +data class DeviceNotificationEntry( + val key: String, + val packageName: String, + val title: String?, + val text: String?, + val subText: String?, + val category: String?, + val channelId: String?, + val postTimeMs: Long, + val isOngoing: Boolean, + val isClearable: Boolean, +) + +data class DeviceNotificationSnapshot( + val enabled: Boolean, + val connected: Boolean, + val notifications: List, +) + +enum class NotificationActionKind { + Open, + Dismiss, + Reply, +} + +data class NotificationActionRequest( + val key: String, + val kind: NotificationActionKind, + val replyText: String? = null, +) + +data class NotificationActionResult( + val ok: Boolean, + val code: String? = null, + val message: String? = null, +) + +internal fun actionRequiresClearableNotification(kind: NotificationActionKind): Boolean { + return kind == NotificationActionKind.Dismiss +} + +private object DeviceNotificationStore { + private val lock = Any() + private var connected = false + private val byKey = LinkedHashMap() + + fun replace(entries: List) { + synchronized(lock) { + byKey.clear() + for (entry in entries) { + byKey[entry.key] = entry + } + } + } + + fun upsert(entry: DeviceNotificationEntry) { + synchronized(lock) { + byKey[entry.key] = entry + } + } + + fun remove(key: String) { + synchronized(lock) { + byKey.remove(key) + } + } + + fun setConnected(value: Boolean) { + synchronized(lock) { + connected = value + if (!value) { + byKey.clear() + } + } + } + + fun snapshot(enabled: Boolean): DeviceNotificationSnapshot { + val (isConnected, entries) = + synchronized(lock) { + connected to byKey.values.sortedByDescending { it.postTimeMs } + } + return DeviceNotificationSnapshot( + enabled = enabled, + connected = isConnected, + notifications = entries, + ) + } +} + +class DeviceNotificationListenerService : NotificationListenerService() { + override fun onListenerConnected() { + super.onListenerConnected() + activeService = this + DeviceNotificationStore.setConnected(true) + refreshActiveNotifications() + } + + override fun onListenerDisconnected() { + if (activeService === this) { + activeService = null + } + DeviceNotificationStore.setConnected(false) + super.onListenerDisconnected() + } + + override fun onDestroy() { + if (activeService === this) { + activeService = null + } + super.onDestroy() + } + + override fun onNotificationPosted(sbn: StatusBarNotification?) { + super.onNotificationPosted(sbn) + val entry = sbn?.toEntry() ?: return + DeviceNotificationStore.upsert(entry) + if (entry.packageName == packageName) { + return + } + emitNotificationsChanged( + buildJsonObject { + put("change", JsonPrimitive("posted")) + put("key", JsonPrimitive(entry.key)) + put("packageName", JsonPrimitive(entry.packageName)) + put("postTimeMs", JsonPrimitive(entry.postTimeMs)) + put("isOngoing", JsonPrimitive(entry.isOngoing)) + put("isClearable", JsonPrimitive(entry.isClearable)) + entry.title?.let { put("title", JsonPrimitive(it)) } + entry.text?.let { put("text", JsonPrimitive(it)) } + entry.subText?.let { put("subText", JsonPrimitive(it)) } + entry.category?.let { put("category", JsonPrimitive(it)) } + entry.channelId?.let { put("channelId", JsonPrimitive(it)) } + }.toString(), + ) + } + + override fun onNotificationRemoved(sbn: StatusBarNotification?) { + super.onNotificationRemoved(sbn) + val removed = sbn ?: return + val key = removed.key.trim() + if (key.isEmpty()) { + return + } + DeviceNotificationStore.remove(key) + if (removed.packageName == packageName) { + return + } + emitNotificationsChanged( + buildJsonObject { + put("change", JsonPrimitive("removed")) + put("key", JsonPrimitive(key)) + val packageName = removed.packageName.trim() + if (packageName.isNotEmpty()) { + put("packageName", JsonPrimitive(packageName)) + } + }.toString(), + ) + } + + private fun refreshActiveNotifications() { + val entries = + runCatching { + activeNotifications + ?.mapNotNull { it.toEntry() } + ?: emptyList() + }.getOrElse { emptyList() } + DeviceNotificationStore.replace(entries) + } + + private fun StatusBarNotification.toEntry(): DeviceNotificationEntry { + val extras = notification.extras + val keyValue = key.takeIf { it.isNotBlank() } ?: "$packageName:$id:$postTime" + val title = sanitizeNotificationText(extras?.getCharSequence(Notification.EXTRA_TITLE)) + val body = + sanitizeNotificationText(extras?.getCharSequence(Notification.EXTRA_BIG_TEXT)) + ?: sanitizeNotificationText(extras?.getCharSequence(Notification.EXTRA_TEXT)) + val subText = sanitizeNotificationText(extras?.getCharSequence(Notification.EXTRA_SUB_TEXT)) + return DeviceNotificationEntry( + key = keyValue, + packageName = packageName, + title = title, + text = body, + subText = subText, + category = notification.category?.trim()?.ifEmpty { null }, + channelId = notification.channelId?.trim()?.ifEmpty { null }, + postTimeMs = postTime, + isOngoing = isOngoing, + isClearable = isClearable, + ) + } + + companion object { + @Volatile private var activeService: DeviceNotificationListenerService? = null + @Volatile private var nodeEventSink: ((event: String, payloadJson: String?) -> Unit)? = null + + private fun serviceComponent(context: Context): ComponentName { + return ComponentName(context, DeviceNotificationListenerService::class.java) + } + + fun setNodeEventSink(sink: ((event: String, payloadJson: String?) -> Unit)?) { + nodeEventSink = sink + } + + fun isAccessEnabled(context: Context): Boolean { + val manager = context.getSystemService(NotificationManager::class.java) ?: return false + return manager.isNotificationListenerAccessGranted(serviceComponent(context)) + } + + fun snapshot(context: Context, enabled: Boolean = isAccessEnabled(context)): DeviceNotificationSnapshot { + return DeviceNotificationStore.snapshot(enabled = enabled) + } + + fun requestServiceRebind(context: Context) { + if (Build.VERSION.SDK_INT < Build.VERSION_CODES.N) { + return + } + runCatching { + NotificationListenerService.requestRebind(serviceComponent(context)) + } + } + + fun executeAction(context: Context, request: NotificationActionRequest): NotificationActionResult { + if (!isAccessEnabled(context)) { + return NotificationActionResult( + ok = false, + code = "NOTIFICATIONS_DISABLED", + message = "NOTIFICATIONS_DISABLED: enable notification access in system Settings", + ) + } + val service = activeService + ?: return NotificationActionResult( + ok = false, + code = "NOTIFICATIONS_UNAVAILABLE", + message = "NOTIFICATIONS_UNAVAILABLE: notification listener not connected", + ) + return service.executeActionInternal(request) + } + + private fun emitNotificationsChanged(payloadJson: String) { + runCatching { + nodeEventSink?.invoke(NOTIFICATIONS_CHANGED_EVENT, payloadJson) + } + } + } + + private fun executeActionInternal(request: NotificationActionRequest): NotificationActionResult { + val sbn = + activeNotifications + ?.firstOrNull { it.key == request.key } + ?: return NotificationActionResult( + ok = false, + code = "NOTIFICATION_NOT_FOUND", + message = "NOTIFICATION_NOT_FOUND: notification key not found", + ) + if (actionRequiresClearableNotification(request.kind) && !sbn.isClearable) { + return NotificationActionResult( + ok = false, + code = "NOTIFICATION_NOT_CLEARABLE", + message = "NOTIFICATION_NOT_CLEARABLE: notification is ongoing or protected", + ) + } + + return when (request.kind) { + NotificationActionKind.Open -> { + val pendingIntent = sbn.notification.contentIntent + ?: return NotificationActionResult( + ok = false, + code = "ACTION_UNAVAILABLE", + message = "ACTION_UNAVAILABLE: notification has no open action", + ) + runCatching { + pendingIntent.send() + }.fold( + onSuccess = { NotificationActionResult(ok = true) }, + onFailure = { err -> + NotificationActionResult( + ok = false, + code = "ACTION_FAILED", + message = "ACTION_FAILED: ${err.message ?: "open failed"}", + ) + }, + ) + } + + NotificationActionKind.Dismiss -> { + runCatching { + cancelNotification(sbn.key) + DeviceNotificationStore.remove(sbn.key) + }.fold( + onSuccess = { NotificationActionResult(ok = true) }, + onFailure = { err -> + NotificationActionResult( + ok = false, + code = "ACTION_FAILED", + message = "ACTION_FAILED: ${err.message ?: "dismiss failed"}", + ) + }, + ) + } + + NotificationActionKind.Reply -> { + val replyText = request.replyText?.trim().orEmpty() + if (replyText.isEmpty()) { + return NotificationActionResult( + ok = false, + code = "INVALID_REQUEST", + message = "INVALID_REQUEST: replyText required for reply action", + ) + } + val action = + sbn.notification.actions + ?.firstOrNull { candidate -> + candidate.actionIntent != null && !candidate.remoteInputs.isNullOrEmpty() + } + ?: return NotificationActionResult( + ok = false, + code = "ACTION_UNAVAILABLE", + message = "ACTION_UNAVAILABLE: notification has no reply action", + ) + val remoteInputs = action.remoteInputs ?: emptyArray() + val fillInIntent = Intent() + val replyBundle = android.os.Bundle() + for (remoteInput in remoteInputs) { + replyBundle.putCharSequence(remoteInput.resultKey, replyText) + } + RemoteInput.addResultsToIntent(remoteInputs, fillInIntent, replyBundle) + runCatching { + action.actionIntent.send(this, 0, fillInIntent) + }.fold( + onSuccess = { NotificationActionResult(ok = true) }, + onFailure = { err -> + NotificationActionResult( + ok = false, + code = "ACTION_FAILED", + message = "ACTION_FAILED: ${err.message ?: "reply failed"}", + ) + }, + ) + } + } + } +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/node/InvokeCommandRegistry.kt b/apps/android/app/src/main/java/ai/openclaw/android/node/InvokeCommandRegistry.kt new file mode 100644 index 00000000000..b8ec77bfca9 --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/node/InvokeCommandRegistry.kt @@ -0,0 +1,242 @@ +package ai.openclaw.android.node + +import ai.openclaw.android.protocol.OpenClawCalendarCommand +import ai.openclaw.android.protocol.OpenClawCanvasA2UICommand +import ai.openclaw.android.protocol.OpenClawCanvasCommand +import ai.openclaw.android.protocol.OpenClawCameraCommand +import ai.openclaw.android.protocol.OpenClawCapability +import ai.openclaw.android.protocol.OpenClawContactsCommand +import ai.openclaw.android.protocol.OpenClawDeviceCommand +import ai.openclaw.android.protocol.OpenClawLocationCommand +import ai.openclaw.android.protocol.OpenClawMotionCommand +import ai.openclaw.android.protocol.OpenClawNotificationsCommand +import ai.openclaw.android.protocol.OpenClawPhotosCommand +import ai.openclaw.android.protocol.OpenClawScreenCommand +import ai.openclaw.android.protocol.OpenClawSmsCommand +import ai.openclaw.android.protocol.OpenClawSystemCommand + +data class NodeRuntimeFlags( + val cameraEnabled: Boolean, + val locationEnabled: Boolean, + val smsAvailable: Boolean, + val voiceWakeEnabled: Boolean, + val motionActivityAvailable: Boolean, + val motionPedometerAvailable: Boolean, + val debugBuild: Boolean, +) + +enum class InvokeCommandAvailability { + Always, + CameraEnabled, + LocationEnabled, + SmsAvailable, + MotionActivityAvailable, + MotionPedometerAvailable, + DebugBuild, +} + +enum class NodeCapabilityAvailability { + Always, + CameraEnabled, + LocationEnabled, + SmsAvailable, + VoiceWakeEnabled, + MotionAvailable, +} + +data class NodeCapabilitySpec( + val name: String, + val availability: NodeCapabilityAvailability = NodeCapabilityAvailability.Always, +) + +data class InvokeCommandSpec( + val name: String, + val requiresForeground: Boolean = false, + val availability: InvokeCommandAvailability = InvokeCommandAvailability.Always, +) + +object InvokeCommandRegistry { + val capabilityManifest: List = + listOf( + NodeCapabilitySpec(name = OpenClawCapability.Canvas.rawValue), + NodeCapabilitySpec(name = OpenClawCapability.Screen.rawValue), + NodeCapabilitySpec(name = OpenClawCapability.Device.rawValue), + NodeCapabilitySpec(name = OpenClawCapability.Notifications.rawValue), + NodeCapabilitySpec(name = OpenClawCapability.System.rawValue), + NodeCapabilitySpec(name = OpenClawCapability.AppUpdate.rawValue), + NodeCapabilitySpec( + name = OpenClawCapability.Camera.rawValue, + availability = NodeCapabilityAvailability.CameraEnabled, + ), + NodeCapabilitySpec( + name = OpenClawCapability.Sms.rawValue, + availability = NodeCapabilityAvailability.SmsAvailable, + ), + NodeCapabilitySpec( + name = OpenClawCapability.VoiceWake.rawValue, + availability = NodeCapabilityAvailability.VoiceWakeEnabled, + ), + NodeCapabilitySpec( + name = OpenClawCapability.Location.rawValue, + availability = NodeCapabilityAvailability.LocationEnabled, + ), + NodeCapabilitySpec(name = OpenClawCapability.Photos.rawValue), + NodeCapabilitySpec(name = OpenClawCapability.Contacts.rawValue), + NodeCapabilitySpec(name = OpenClawCapability.Calendar.rawValue), + NodeCapabilitySpec( + name = OpenClawCapability.Motion.rawValue, + availability = NodeCapabilityAvailability.MotionAvailable, + ), + ) + + val all: List = + listOf( + InvokeCommandSpec( + name = OpenClawCanvasCommand.Present.rawValue, + requiresForeground = true, + ), + InvokeCommandSpec( + name = OpenClawCanvasCommand.Hide.rawValue, + requiresForeground = true, + ), + InvokeCommandSpec( + name = OpenClawCanvasCommand.Navigate.rawValue, + requiresForeground = true, + ), + InvokeCommandSpec( + name = OpenClawCanvasCommand.Eval.rawValue, + requiresForeground = true, + ), + InvokeCommandSpec( + name = OpenClawCanvasCommand.Snapshot.rawValue, + requiresForeground = true, + ), + InvokeCommandSpec( + name = OpenClawCanvasA2UICommand.Push.rawValue, + requiresForeground = true, + ), + InvokeCommandSpec( + name = OpenClawCanvasA2UICommand.PushJSONL.rawValue, + requiresForeground = true, + ), + InvokeCommandSpec( + name = OpenClawCanvasA2UICommand.Reset.rawValue, + requiresForeground = true, + ), + InvokeCommandSpec( + name = OpenClawScreenCommand.Record.rawValue, + requiresForeground = true, + ), + InvokeCommandSpec( + name = OpenClawSystemCommand.Notify.rawValue, + ), + InvokeCommandSpec( + name = OpenClawCameraCommand.List.rawValue, + requiresForeground = true, + availability = InvokeCommandAvailability.CameraEnabled, + ), + InvokeCommandSpec( + name = OpenClawCameraCommand.Snap.rawValue, + requiresForeground = true, + availability = InvokeCommandAvailability.CameraEnabled, + ), + InvokeCommandSpec( + name = OpenClawCameraCommand.Clip.rawValue, + requiresForeground = true, + availability = InvokeCommandAvailability.CameraEnabled, + ), + InvokeCommandSpec( + name = OpenClawLocationCommand.Get.rawValue, + availability = InvokeCommandAvailability.LocationEnabled, + ), + InvokeCommandSpec( + name = OpenClawDeviceCommand.Status.rawValue, + ), + InvokeCommandSpec( + name = OpenClawDeviceCommand.Info.rawValue, + ), + InvokeCommandSpec( + name = OpenClawDeviceCommand.Permissions.rawValue, + ), + InvokeCommandSpec( + name = OpenClawDeviceCommand.Health.rawValue, + ), + InvokeCommandSpec( + name = OpenClawNotificationsCommand.List.rawValue, + ), + InvokeCommandSpec( + name = OpenClawNotificationsCommand.Actions.rawValue, + ), + InvokeCommandSpec( + name = OpenClawPhotosCommand.Latest.rawValue, + ), + InvokeCommandSpec( + name = OpenClawContactsCommand.Search.rawValue, + ), + InvokeCommandSpec( + name = OpenClawContactsCommand.Add.rawValue, + ), + InvokeCommandSpec( + name = OpenClawCalendarCommand.Events.rawValue, + ), + InvokeCommandSpec( + name = OpenClawCalendarCommand.Add.rawValue, + ), + InvokeCommandSpec( + name = OpenClawMotionCommand.Activity.rawValue, + availability = InvokeCommandAvailability.MotionActivityAvailable, + ), + InvokeCommandSpec( + name = OpenClawMotionCommand.Pedometer.rawValue, + availability = InvokeCommandAvailability.MotionPedometerAvailable, + ), + InvokeCommandSpec( + name = OpenClawSmsCommand.Send.rawValue, + availability = InvokeCommandAvailability.SmsAvailable, + ), + InvokeCommandSpec( + name = "debug.logs", + availability = InvokeCommandAvailability.DebugBuild, + ), + InvokeCommandSpec( + name = "debug.ed25519", + availability = InvokeCommandAvailability.DebugBuild, + ), + InvokeCommandSpec(name = "app.update"), + ) + + private val byNameInternal: Map = all.associateBy { it.name } + + fun find(command: String): InvokeCommandSpec? = byNameInternal[command] + + fun advertisedCapabilities(flags: NodeRuntimeFlags): List { + return capabilityManifest + .filter { spec -> + when (spec.availability) { + NodeCapabilityAvailability.Always -> true + NodeCapabilityAvailability.CameraEnabled -> flags.cameraEnabled + NodeCapabilityAvailability.LocationEnabled -> flags.locationEnabled + NodeCapabilityAvailability.SmsAvailable -> flags.smsAvailable + NodeCapabilityAvailability.VoiceWakeEnabled -> flags.voiceWakeEnabled + NodeCapabilityAvailability.MotionAvailable -> flags.motionActivityAvailable || flags.motionPedometerAvailable + } + } + .map { it.name } + } + + fun advertisedCommands(flags: NodeRuntimeFlags): List { + return all + .filter { spec -> + when (spec.availability) { + InvokeCommandAvailability.Always -> true + InvokeCommandAvailability.CameraEnabled -> flags.cameraEnabled + InvokeCommandAvailability.LocationEnabled -> flags.locationEnabled + InvokeCommandAvailability.SmsAvailable -> flags.smsAvailable + InvokeCommandAvailability.MotionActivityAvailable -> flags.motionActivityAvailable + InvokeCommandAvailability.MotionPedometerAvailable -> flags.motionPedometerAvailable + InvokeCommandAvailability.DebugBuild -> flags.debugBuild + } + } + .map { it.name } + } +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/node/InvokeDispatcher.kt b/apps/android/app/src/main/java/ai/openclaw/android/node/InvokeDispatcher.kt index e44896db0fa..8e6552edfbb 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/node/InvokeDispatcher.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/node/InvokeDispatcher.kt @@ -1,17 +1,31 @@ package ai.openclaw.android.node import ai.openclaw.android.gateway.GatewaySession +import ai.openclaw.android.protocol.OpenClawCalendarCommand import ai.openclaw.android.protocol.OpenClawCanvasA2UICommand import ai.openclaw.android.protocol.OpenClawCanvasCommand import ai.openclaw.android.protocol.OpenClawCameraCommand +import ai.openclaw.android.protocol.OpenClawContactsCommand +import ai.openclaw.android.protocol.OpenClawDeviceCommand import ai.openclaw.android.protocol.OpenClawLocationCommand +import ai.openclaw.android.protocol.OpenClawMotionCommand +import ai.openclaw.android.protocol.OpenClawNotificationsCommand +import ai.openclaw.android.protocol.OpenClawPhotosCommand import ai.openclaw.android.protocol.OpenClawScreenCommand import ai.openclaw.android.protocol.OpenClawSmsCommand +import ai.openclaw.android.protocol.OpenClawSystemCommand class InvokeDispatcher( private val canvas: CanvasController, private val cameraHandler: CameraHandler, private val locationHandler: LocationHandler, + private val deviceHandler: DeviceHandler, + private val notificationsHandler: NotificationsHandler, + private val systemHandler: SystemHandler, + private val photosHandler: PhotosHandler, + private val contactsHandler: ContactsHandler, + private val calendarHandler: CalendarHandler, + private val motionHandler: MotionHandler, private val screenHandler: ScreenHandler, private val smsHandler: SmsHandler, private val a2uiHandler: A2UIHandler, @@ -20,38 +34,28 @@ class InvokeDispatcher( private val isForeground: () -> Boolean, private val cameraEnabled: () -> Boolean, private val locationEnabled: () -> Boolean, + private val smsAvailable: () -> Boolean, + private val debugBuild: () -> Boolean, + private val refreshNodeCanvasCapability: suspend () -> Boolean, + private val onCanvasA2uiPush: () -> Unit, + private val onCanvasA2uiReset: () -> Unit, + private val motionActivityAvailable: () -> Boolean, + private val motionPedometerAvailable: () -> Boolean, ) { suspend fun handleInvoke(command: String, paramsJson: String?): GatewaySession.InvokeResult { - // Check foreground requirement for canvas/camera/screen commands - if ( - command.startsWith(OpenClawCanvasCommand.NamespacePrefix) || - command.startsWith(OpenClawCanvasA2UICommand.NamespacePrefix) || - command.startsWith(OpenClawCameraCommand.NamespacePrefix) || - command.startsWith(OpenClawScreenCommand.NamespacePrefix) - ) { - if (!isForeground()) { - return GatewaySession.InvokeResult.error( - code = "NODE_BACKGROUND_UNAVAILABLE", - message = "NODE_BACKGROUND_UNAVAILABLE: canvas/camera/screen commands require foreground", + val spec = + InvokeCommandRegistry.find(command) + ?: return GatewaySession.InvokeResult.error( + code = "INVALID_REQUEST", + message = "INVALID_REQUEST: unknown command", ) - } - } - - // Check camera enabled - if (command.startsWith(OpenClawCameraCommand.NamespacePrefix) && !cameraEnabled()) { + if (spec.requiresForeground && !isForeground()) { return GatewaySession.InvokeResult.error( - code = "CAMERA_DISABLED", - message = "CAMERA_DISABLED: enable Camera in Settings", - ) - } - - // Check location enabled - if (command.startsWith(OpenClawLocationCommand.NamespacePrefix) && !locationEnabled()) { - return GatewaySession.InvokeResult.error( - code = "LOCATION_DISABLED", - message = "LOCATION_DISABLED: enable Location in Settings", + code = "NODE_BACKGROUND_UNAVAILABLE", + message = "NODE_BACKGROUND_UNAVAILABLE: canvas/camera/screen commands require foreground", ) } + availabilityError(spec.availability)?.let { return it } return when (command) { // Canvas commands @@ -73,52 +77,33 @@ class InvokeDispatcher( code = "INVALID_REQUEST", message = "INVALID_REQUEST: javaScript required", ) - val result = - try { - canvas.eval(js) - } catch (err: Throwable) { - return GatewaySession.InvokeResult.error( - code = "NODE_BACKGROUND_UNAVAILABLE", - message = "NODE_BACKGROUND_UNAVAILABLE: canvas unavailable", - ) - } - GatewaySession.InvokeResult.ok("""{"result":${result.toJsonString()}}""") + withCanvasAvailable { + val result = canvas.eval(js) + GatewaySession.InvokeResult.ok("""{"result":${result.toJsonString()}}""") + } } OpenClawCanvasCommand.Snapshot.rawValue -> { val snapshotParams = CanvasController.parseSnapshotParams(paramsJson) - val base64 = - try { + withCanvasAvailable { + val base64 = canvas.snapshotBase64( format = snapshotParams.format, quality = snapshotParams.quality, maxWidth = snapshotParams.maxWidth, ) - } catch (err: Throwable) { - return GatewaySession.InvokeResult.error( - code = "NODE_BACKGROUND_UNAVAILABLE", - message = "NODE_BACKGROUND_UNAVAILABLE: canvas unavailable", - ) - } - GatewaySession.InvokeResult.ok("""{"format":"${snapshotParams.format.rawValue}","base64":"$base64"}""") + GatewaySession.InvokeResult.ok("""{"format":"${snapshotParams.format.rawValue}","base64":"$base64"}""") + } } // A2UI commands - OpenClawCanvasA2UICommand.Reset.rawValue -> { - val a2uiUrl = a2uiHandler.resolveA2uiHostUrl() - ?: return GatewaySession.InvokeResult.error( - code = "A2UI_HOST_NOT_CONFIGURED", - message = "A2UI_HOST_NOT_CONFIGURED: gateway did not advertise canvas host", - ) - val ready = a2uiHandler.ensureA2uiReady(a2uiUrl) - if (!ready) { - return GatewaySession.InvokeResult.error( - code = "A2UI_HOST_UNAVAILABLE", - message = "A2UI host not reachable", - ) + OpenClawCanvasA2UICommand.Reset.rawValue -> + withReadyA2ui { + withCanvasAvailable { + val res = canvas.eval(A2UIHandler.a2uiResetJS) + onCanvasA2uiReset() + GatewaySession.InvokeResult.ok(res) + } } - val res = canvas.eval(A2UIHandler.a2uiResetJS) - GatewaySession.InvokeResult.ok(res) - } OpenClawCanvasA2UICommand.Push.rawValue, OpenClawCanvasA2UICommand.PushJSONL.rawValue -> { val messages = try { @@ -129,30 +114,52 @@ class InvokeDispatcher( message = err.message ?: "invalid A2UI payload" ) } - val a2uiUrl = a2uiHandler.resolveA2uiHostUrl() - ?: return GatewaySession.InvokeResult.error( - code = "A2UI_HOST_NOT_CONFIGURED", - message = "A2UI_HOST_NOT_CONFIGURED: gateway did not advertise canvas host", - ) - val ready = a2uiHandler.ensureA2uiReady(a2uiUrl) - if (!ready) { - return GatewaySession.InvokeResult.error( - code = "A2UI_HOST_UNAVAILABLE", - message = "A2UI host not reachable", - ) + withReadyA2ui { + withCanvasAvailable { + val js = A2UIHandler.a2uiApplyMessagesJS(messages) + val res = canvas.eval(js) + onCanvasA2uiPush() + GatewaySession.InvokeResult.ok(res) + } } - val js = A2UIHandler.a2uiApplyMessagesJS(messages) - val res = canvas.eval(js) - GatewaySession.InvokeResult.ok(res) } // Camera commands + OpenClawCameraCommand.List.rawValue -> cameraHandler.handleList(paramsJson) OpenClawCameraCommand.Snap.rawValue -> cameraHandler.handleSnap(paramsJson) OpenClawCameraCommand.Clip.rawValue -> cameraHandler.handleClip(paramsJson) // Location command OpenClawLocationCommand.Get.rawValue -> locationHandler.handleLocationGet(paramsJson) + // Device commands + OpenClawDeviceCommand.Status.rawValue -> deviceHandler.handleDeviceStatus(paramsJson) + OpenClawDeviceCommand.Info.rawValue -> deviceHandler.handleDeviceInfo(paramsJson) + OpenClawDeviceCommand.Permissions.rawValue -> deviceHandler.handleDevicePermissions(paramsJson) + OpenClawDeviceCommand.Health.rawValue -> deviceHandler.handleDeviceHealth(paramsJson) + + // Notifications command + OpenClawNotificationsCommand.List.rawValue -> notificationsHandler.handleNotificationsList(paramsJson) + OpenClawNotificationsCommand.Actions.rawValue -> notificationsHandler.handleNotificationsActions(paramsJson) + + // System command + OpenClawSystemCommand.Notify.rawValue -> systemHandler.handleSystemNotify(paramsJson) + + // Photos command + OpenClawPhotosCommand.Latest.rawValue -> photosHandler.handlePhotosLatest(paramsJson) + + // Contacts command + OpenClawContactsCommand.Search.rawValue -> contactsHandler.handleContactsSearch(paramsJson) + OpenClawContactsCommand.Add.rawValue -> contactsHandler.handleContactsAdd(paramsJson) + + // Calendar command + OpenClawCalendarCommand.Events.rawValue -> calendarHandler.handleCalendarEvents(paramsJson) + OpenClawCalendarCommand.Add.rawValue -> calendarHandler.handleCalendarAdd(paramsJson) + + // Motion command + OpenClawMotionCommand.Activity.rawValue -> motionHandler.handleMotionActivity(paramsJson) + OpenClawMotionCommand.Pedometer.rawValue -> motionHandler.handleMotionPedometer(paramsJson) + // Screen command OpenClawScreenCommand.Record.rawValue -> screenHandler.handleScreenRecord(paramsJson) @@ -166,11 +173,111 @@ class InvokeDispatcher( // App update "app.update" -> appUpdateHandler.handleUpdate(paramsJson) - else -> - GatewaySession.InvokeResult.error( - code = "INVALID_REQUEST", - message = "INVALID_REQUEST: unknown command", + else -> GatewaySession.InvokeResult.error(code = "INVALID_REQUEST", message = "INVALID_REQUEST: unknown command") + } + } + + private suspend fun withReadyA2ui( + block: suspend () -> GatewaySession.InvokeResult, + ): GatewaySession.InvokeResult { + var a2uiUrl = a2uiHandler.resolveA2uiHostUrl() + ?: return GatewaySession.InvokeResult.error( + code = "A2UI_HOST_NOT_CONFIGURED", + message = "A2UI_HOST_NOT_CONFIGURED: gateway did not advertise canvas host", + ) + val readyOnFirstCheck = a2uiHandler.ensureA2uiReady(a2uiUrl) + if (!readyOnFirstCheck) { + if (!refreshNodeCanvasCapability()) { + return GatewaySession.InvokeResult.error( + code = "A2UI_HOST_UNAVAILABLE", + message = "A2UI_HOST_UNAVAILABLE: A2UI host not reachable", ) + } + a2uiUrl = a2uiHandler.resolveA2uiHostUrl() + ?: return GatewaySession.InvokeResult.error( + code = "A2UI_HOST_NOT_CONFIGURED", + message = "A2UI_HOST_NOT_CONFIGURED: gateway did not advertise canvas host", + ) + if (!a2uiHandler.ensureA2uiReady(a2uiUrl)) { + return GatewaySession.InvokeResult.error( + code = "A2UI_HOST_UNAVAILABLE", + message = "A2UI_HOST_UNAVAILABLE: A2UI host not reachable", + ) + } + } + return block() + } + + private suspend fun withCanvasAvailable( + block: suspend () -> GatewaySession.InvokeResult, + ): GatewaySession.InvokeResult { + return try { + block() + } catch (_: Throwable) { + GatewaySession.InvokeResult.error( + code = "NODE_BACKGROUND_UNAVAILABLE", + message = "NODE_BACKGROUND_UNAVAILABLE: canvas unavailable", + ) + } + } + + private fun availabilityError(availability: InvokeCommandAvailability): GatewaySession.InvokeResult? { + return when (availability) { + InvokeCommandAvailability.Always -> null + InvokeCommandAvailability.CameraEnabled -> + if (cameraEnabled()) { + null + } else { + GatewaySession.InvokeResult.error( + code = "CAMERA_DISABLED", + message = "CAMERA_DISABLED: enable Camera in Settings", + ) + } + InvokeCommandAvailability.LocationEnabled -> + if (locationEnabled()) { + null + } else { + GatewaySession.InvokeResult.error( + code = "LOCATION_DISABLED", + message = "LOCATION_DISABLED: enable Location in Settings", + ) + } + InvokeCommandAvailability.MotionActivityAvailable -> + if (motionActivityAvailable()) { + null + } else { + GatewaySession.InvokeResult.error( + code = "MOTION_UNAVAILABLE", + message = "MOTION_UNAVAILABLE: accelerometer not available", + ) + } + InvokeCommandAvailability.MotionPedometerAvailable -> + if (motionPedometerAvailable()) { + null + } else { + GatewaySession.InvokeResult.error( + code = "PEDOMETER_UNAVAILABLE", + message = "PEDOMETER_UNAVAILABLE: step counter not available", + ) + } + InvokeCommandAvailability.SmsAvailable -> + if (smsAvailable()) { + null + } else { + GatewaySession.InvokeResult.error( + code = "SMS_UNAVAILABLE", + message = "SMS_UNAVAILABLE: SMS not available on this device", + ) + } + InvokeCommandAvailability.DebugBuild -> + if (debugBuild()) { + null + } else { + GatewaySession.InvokeResult.error( + code = "INVALID_REQUEST", + message = "INVALID_REQUEST: unknown command", + ) + } } } } diff --git a/apps/android/app/src/main/java/ai/openclaw/android/node/MotionHandler.kt b/apps/android/app/src/main/java/ai/openclaw/android/node/MotionHandler.kt new file mode 100644 index 00000000000..d385b35f182 --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/node/MotionHandler.kt @@ -0,0 +1,377 @@ +package ai.openclaw.android.node + +import android.Manifest +import android.content.Context +import android.hardware.Sensor +import android.hardware.SensorEvent +import android.hardware.SensorEventListener +import android.hardware.SensorManager +import android.os.Build +import android.os.SystemClock +import androidx.core.content.ContextCompat +import ai.openclaw.android.gateway.GatewaySession +import java.time.Instant +import kotlinx.coroutines.suspendCancellableCoroutine +import kotlinx.coroutines.withTimeoutOrNull +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.buildJsonArray +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.put +import kotlin.coroutines.resume +import kotlin.math.abs +import kotlin.math.max +import kotlin.math.sqrt + +private const val ACCELEROMETER_SAMPLE_TARGET = 20 +private const val ACCELEROMETER_SAMPLE_TIMEOUT_MS = 6_000L + +internal data class MotionActivityRequest( + val startISO: String?, + val endISO: String?, + val limit: Int, +) + +internal data class MotionPedometerRequest( + val startISO: String?, + val endISO: String?, +) + +internal data class MotionActivityRecord( + val startISO: String, + val endISO: String, + val confidence: String, + val isWalking: Boolean, + val isRunning: Boolean, + val isCycling: Boolean, + val isAutomotive: Boolean, + val isStationary: Boolean, + val isUnknown: Boolean, +) + +internal data class PedometerRecord( + val startISO: String, + val endISO: String, + val steps: Int?, + val distanceMeters: Double?, + val floorsAscended: Int?, + val floorsDescended: Int?, +) + +internal interface MotionDataSource { + fun isActivityAvailable(context: Context): Boolean + + fun isPedometerAvailable(context: Context): Boolean + + fun isAvailable(context: Context): Boolean = isActivityAvailable(context) || isPedometerAvailable(context) + + fun hasPermission(context: Context): Boolean + + suspend fun activity(context: Context, request: MotionActivityRequest): MotionActivityRecord + + suspend fun pedometer(context: Context, request: MotionPedometerRequest): PedometerRecord +} + +private object SystemMotionDataSource : MotionDataSource { + override fun isActivityAvailable(context: Context): Boolean { + val sensorManager = context.getSystemService(SensorManager::class.java) + return sensorManager?.getDefaultSensor(Sensor.TYPE_ACCELEROMETER) != null + } + + override fun isPedometerAvailable(context: Context): Boolean { + val sensorManager = context.getSystemService(SensorManager::class.java) + return sensorManager?.getDefaultSensor(Sensor.TYPE_STEP_COUNTER) != null + } + + override fun hasPermission(context: Context): Boolean { + if (Build.VERSION.SDK_INT < 29) return true + return ContextCompat.checkSelfPermission(context, Manifest.permission.ACTIVITY_RECOGNITION) == + android.content.pm.PackageManager.PERMISSION_GRANTED + } + + override suspend fun activity(context: Context, request: MotionActivityRequest): MotionActivityRecord { + if (!request.startISO.isNullOrBlank() || !request.endISO.isNullOrBlank()) { + throw IllegalArgumentException("MOTION_RANGE_UNAVAILABLE: historical activity range not supported on Android") + } + val sensorManager = context.getSystemService(SensorManager::class.java) + ?: throw IllegalStateException("MOTION_UNAVAILABLE: sensor manager unavailable") + val accelerometer = sensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER) + ?: throw IllegalStateException("MOTION_UNAVAILABLE: accelerometer not available") + + val sample = readAccelerometerSample(sensorManager, accelerometer) + ?: throw IllegalStateException("MOTION_UNAVAILABLE: no accelerometer sample") + val end = Instant.now() + val start = end.minusSeconds(2) + val classification = classifyActivity(sample.averageDelta) + return MotionActivityRecord( + startISO = start.toString(), + endISO = end.toString(), + confidence = classifyConfidence(sample.samples, sample.averageDelta), + isWalking = classification == "walking", + isRunning = classification == "running", + isCycling = false, + isAutomotive = false, + isStationary = classification == "stationary", + isUnknown = classification == "unknown", + ) + } + + override suspend fun pedometer(context: Context, request: MotionPedometerRequest): PedometerRecord { + if (!request.startISO.isNullOrBlank() || !request.endISO.isNullOrBlank()) { + throw IllegalArgumentException("PEDOMETER_RANGE_UNAVAILABLE: historical pedometer range not supported on Android") + } + val sensorManager = context.getSystemService(SensorManager::class.java) + ?: throw IllegalStateException("PEDOMETER_UNAVAILABLE: sensor manager unavailable") + val stepCounter = sensorManager.getDefaultSensor(Sensor.TYPE_STEP_COUNTER) + ?: throw IllegalStateException("PEDOMETER_UNAVAILABLE: step counting not supported") + + val steps = readStepCounter(sensorManager, stepCounter) + ?: throw IllegalStateException("PEDOMETER_UNAVAILABLE: no step counter sample") + val bootMs = System.currentTimeMillis() - SystemClock.elapsedRealtime() + return PedometerRecord( + startISO = Instant.ofEpochMilli(max(0L, bootMs)).toString(), + endISO = Instant.now().toString(), + steps = steps, + distanceMeters = null, + floorsAscended = null, + floorsDescended = null, + ) + } + + private data class AccelerometerSample( + val samples: Int, + val averageDelta: Double, + ) + + private suspend fun readStepCounter(sensorManager: SensorManager, sensor: Sensor): Int? { + val sample = + withTimeoutOrNull(1200L) { + suspendCancellableCoroutine { cont -> + var resumed = false + val listener = + object : SensorEventListener { + override fun onSensorChanged(event: SensorEvent?) { + if (resumed) return + val value = event?.values?.firstOrNull() + resumed = true + sensorManager.unregisterListener(this) + cont.resume(value) + } + + override fun onAccuracyChanged(sensor: Sensor?, accuracy: Int) = Unit + } + val registered = sensorManager.registerListener(listener, sensor, SensorManager.SENSOR_DELAY_NORMAL) + if (!registered) { + sensorManager.unregisterListener(listener) + resumed = true + cont.resume(null) + return@suspendCancellableCoroutine + } + cont.invokeOnCancellation { sensorManager.unregisterListener(listener) } + } + } + return sample?.toInt()?.takeIf { it >= 0 } + } + + private suspend fun readAccelerometerSample( + sensorManager: SensorManager, + sensor: Sensor, + ): AccelerometerSample? { + val sample = + withTimeoutOrNull(ACCELEROMETER_SAMPLE_TIMEOUT_MS) { + suspendCancellableCoroutine { cont -> + var count = 0 + var sumDelta = 0.0 + var resumed = false + val listener = + object : SensorEventListener { + override fun onSensorChanged(event: SensorEvent?) { + val values = event?.values ?: return + if (values.size < 3) return + val magnitude = + sqrt( + values[0] * values[0] + + values[1] * values[1] + + values[2] * values[2], + ).toDouble() + sumDelta += abs(magnitude - SensorManager.GRAVITY_EARTH.toDouble()) + count += 1 + if (count >= ACCELEROMETER_SAMPLE_TARGET && !resumed) { + resumed = true + sensorManager.unregisterListener(this) + cont.resume( + AccelerometerSample( + samples = count, + averageDelta = if (count == 0) 0.0 else sumDelta / count, + ), + ) + } + } + + override fun onAccuracyChanged(sensor: Sensor?, accuracy: Int) = Unit + } + val registered = sensorManager.registerListener(listener, sensor, SensorManager.SENSOR_DELAY_NORMAL) + if (!registered) { + resumed = true + cont.resume(null) + return@suspendCancellableCoroutine + } + cont.invokeOnCancellation { sensorManager.unregisterListener(listener) } + } + } + return sample + } + + private fun classifyActivity(averageDelta: Double): String { + return when { + averageDelta <= 0.55 -> "stationary" + averageDelta <= 1.80 -> "walking" + else -> "running" + } + } + + private fun classifyConfidence(samples: Int, averageDelta: Double): String { + if (samples < 6) return "low" + if (samples >= 14 && averageDelta > 0.4) return "high" + return "medium" + } +} + +class MotionHandler private constructor( + private val appContext: Context, + private val dataSource: MotionDataSource, +) { + constructor(appContext: Context) : this(appContext = appContext, dataSource = SystemMotionDataSource) + + suspend fun handleMotionActivity(paramsJson: String?): GatewaySession.InvokeResult { + if (!dataSource.hasPermission(appContext)) { + return GatewaySession.InvokeResult.error( + code = "MOTION_PERMISSION_REQUIRED", + message = "MOTION_PERMISSION_REQUIRED: grant Motion permission", + ) + } + val request = + parseActivityRequest(paramsJson) + ?: return GatewaySession.InvokeResult.error( + code = "INVALID_REQUEST", + message = "INVALID_REQUEST: expected JSON object", + ) + return try { + val activity = dataSource.activity(appContext, request) + GatewaySession.InvokeResult.ok( + buildJsonObject { + put( + "activities", + buildJsonArray { + add( + buildJsonObject { + put("startISO", JsonPrimitive(activity.startISO)) + put("endISO", JsonPrimitive(activity.endISO)) + put("confidence", JsonPrimitive(activity.confidence)) + put("isWalking", JsonPrimitive(activity.isWalking)) + put("isRunning", JsonPrimitive(activity.isRunning)) + put("isCycling", JsonPrimitive(activity.isCycling)) + put("isAutomotive", JsonPrimitive(activity.isAutomotive)) + put("isStationary", JsonPrimitive(activity.isStationary)) + put("isUnknown", JsonPrimitive(activity.isUnknown)) + }, + ) + }, + ) + }.toString(), + ) + } catch (err: IllegalArgumentException) { + GatewaySession.InvokeResult.error(code = "MOTION_UNAVAILABLE", message = err.message ?: "MOTION_UNAVAILABLE") + } catch (err: Throwable) { + GatewaySession.InvokeResult.error( + code = "MOTION_UNAVAILABLE", + message = "MOTION_UNAVAILABLE: ${err.message ?: "motion activity failed"}", + ) + } + } + + suspend fun handleMotionPedometer(paramsJson: String?): GatewaySession.InvokeResult { + if (!dataSource.hasPermission(appContext)) { + return GatewaySession.InvokeResult.error( + code = "MOTION_PERMISSION_REQUIRED", + message = "MOTION_PERMISSION_REQUIRED: grant Motion permission", + ) + } + val request = + parsePedometerRequest(paramsJson) + ?: return GatewaySession.InvokeResult.error( + code = "INVALID_REQUEST", + message = "INVALID_REQUEST: expected JSON object", + ) + return try { + val payload = dataSource.pedometer(appContext, request) + GatewaySession.InvokeResult.ok( + buildJsonObject { + put("startISO", JsonPrimitive(payload.startISO)) + put("endISO", JsonPrimitive(payload.endISO)) + payload.steps?.let { put("steps", JsonPrimitive(it)) } + payload.distanceMeters?.let { put("distanceMeters", JsonPrimitive(it)) } + payload.floorsAscended?.let { put("floorsAscended", JsonPrimitive(it)) } + payload.floorsDescended?.let { put("floorsDescended", JsonPrimitive(it)) } + }.toString(), + ) + } catch (err: IllegalArgumentException) { + GatewaySession.InvokeResult.error(code = "MOTION_UNAVAILABLE", message = err.message ?: "MOTION_UNAVAILABLE") + } catch (err: Throwable) { + GatewaySession.InvokeResult.error( + code = "MOTION_UNAVAILABLE", + message = "MOTION_UNAVAILABLE: ${err.message ?: "pedometer query failed"}", + ) + } + } + + fun isAvailable(): Boolean = dataSource.isAvailable(appContext) + + fun isActivityAvailable(): Boolean = dataSource.isActivityAvailable(appContext) + + fun isPedometerAvailable(): Boolean = dataSource.isPedometerAvailable(appContext) + + private fun parseActivityRequest(paramsJson: String?): MotionActivityRequest? { + if (paramsJson.isNullOrBlank()) { + return MotionActivityRequest(startISO = null, endISO = null, limit = 200) + } + val params = + try { + Json.parseToJsonElement(paramsJson).asObjectOrNull() + } catch (_: Throwable) { + null + } ?: return null + val limit = ((params["limit"] as? JsonPrimitive)?.content?.toIntOrNull() ?: 200).coerceIn(1, 1000) + return MotionActivityRequest( + startISO = (params["startISO"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null }, + endISO = (params["endISO"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null }, + limit = limit, + ) + } + + private fun parsePedometerRequest(paramsJson: String?): MotionPedometerRequest? { + if (paramsJson.isNullOrBlank()) { + return MotionPedometerRequest(startISO = null, endISO = null) + } + val params = + try { + Json.parseToJsonElement(paramsJson).asObjectOrNull() + } catch (_: Throwable) { + null + } ?: return null + return MotionPedometerRequest( + startISO = (params["startISO"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null }, + endISO = (params["endISO"] as? JsonPrimitive)?.content?.trim()?.ifEmpty { null }, + ) + } + + companion object { + fun isMotionCapabilityAvailable(context: Context): Boolean = SystemMotionDataSource.isAvailable(context) + + internal fun forTesting( + appContext: Context, + dataSource: MotionDataSource, + ): MotionHandler = MotionHandler(appContext = appContext, dataSource = dataSource) + } +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/node/NodeUtils.kt b/apps/android/app/src/main/java/ai/openclaw/android/node/NodeUtils.kt index 8ba5ad276d5..c3f463174a4 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/node/NodeUtils.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/node/NodeUtils.kt @@ -1,5 +1,6 @@ package ai.openclaw.android.node +import ai.openclaw.android.gateway.parseInvokeErrorFromThrowable import kotlinx.serialization.json.JsonElement import kotlinx.serialization.json.JsonNull import kotlinx.serialization.json.JsonObject @@ -37,14 +38,9 @@ fun parseHexColorArgb(raw: String?): Long? { } fun invokeErrorFromThrowable(err: Throwable): Pair { - val raw = (err.message ?: "").trim() - if (raw.isEmpty()) return "UNAVAILABLE" to "UNAVAILABLE: error" - - val idx = raw.indexOf(':') - if (idx <= 0) return "UNAVAILABLE" to raw - val code = raw.substring(0, idx).trim().ifEmpty { "UNAVAILABLE" } - val message = raw.substring(idx + 1).trim().ifEmpty { raw } - return code to "$code: $message" + val parsed = parseInvokeErrorFromThrowable(err, fallbackMessage = "UNAVAILABLE: error") + val message = if (parsed.hadExplicitCode) parsed.prefixedMessage else parsed.message + return parsed.code to message } fun normalizeMainKey(raw: String?): String? { diff --git a/apps/android/app/src/main/java/ai/openclaw/android/node/NotificationsHandler.kt b/apps/android/app/src/main/java/ai/openclaw/android/node/NotificationsHandler.kt new file mode 100644 index 00000000000..8195ab84847 --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/node/NotificationsHandler.kt @@ -0,0 +1,174 @@ +package ai.openclaw.android.node + +import android.content.Context +import ai.openclaw.android.gateway.GatewaySession +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonArray +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.contentOrNull +import kotlinx.serialization.json.put + +internal interface NotificationsStateProvider { + fun readSnapshot(context: Context): DeviceNotificationSnapshot + + fun requestServiceRebind(context: Context) + + fun executeAction(context: Context, request: NotificationActionRequest): NotificationActionResult +} + +private object SystemNotificationsStateProvider : NotificationsStateProvider { + override fun readSnapshot(context: Context): DeviceNotificationSnapshot { + val enabled = DeviceNotificationListenerService.isAccessEnabled(context) + if (!enabled) { + return DeviceNotificationSnapshot( + enabled = false, + connected = false, + notifications = emptyList(), + ) + } + return DeviceNotificationListenerService.snapshot(context, enabled = true) + } + + override fun requestServiceRebind(context: Context) { + DeviceNotificationListenerService.requestServiceRebind(context) + } + + override fun executeAction(context: Context, request: NotificationActionRequest): NotificationActionResult { + return DeviceNotificationListenerService.executeAction(context, request) + } +} + +class NotificationsHandler private constructor( + private val appContext: Context, + private val stateProvider: NotificationsStateProvider, +) { + constructor(appContext: Context) : this(appContext = appContext, stateProvider = SystemNotificationsStateProvider) + + suspend fun handleNotificationsList(_paramsJson: String?): GatewaySession.InvokeResult { + val snapshot = readSnapshotWithRebind() + return GatewaySession.InvokeResult.ok(snapshotPayloadJson(snapshot)) + } + + suspend fun handleNotificationsActions(paramsJson: String?): GatewaySession.InvokeResult { + readSnapshotWithRebind() + + val params = parseParamsObject(paramsJson) + ?: return GatewaySession.InvokeResult.error( + code = "INVALID_REQUEST", + message = "INVALID_REQUEST: expected JSON object", + ) + val key = + readString(params, "key") + ?: return GatewaySession.InvokeResult.error( + code = "INVALID_REQUEST", + message = "INVALID_REQUEST: key required", + ) + val actionRaw = + readString(params, "action")?.lowercase() + ?: return GatewaySession.InvokeResult.error( + code = "INVALID_REQUEST", + message = "INVALID_REQUEST: action required (open|dismiss|reply)", + ) + val action = + when (actionRaw) { + "open" -> NotificationActionKind.Open + "dismiss" -> NotificationActionKind.Dismiss + "reply" -> NotificationActionKind.Reply + else -> + return GatewaySession.InvokeResult.error( + code = "INVALID_REQUEST", + message = "INVALID_REQUEST: action must be open|dismiss|reply", + ) + } + val replyText = readString(params, "replyText") + if (action == NotificationActionKind.Reply && replyText.isNullOrBlank()) { + return GatewaySession.InvokeResult.error( + code = "INVALID_REQUEST", + message = "INVALID_REQUEST: replyText required for reply action", + ) + } + + val result = + stateProvider.executeAction( + appContext, + NotificationActionRequest( + key = key, + kind = action, + replyText = replyText, + ), + ) + if (!result.ok) { + return GatewaySession.InvokeResult.error( + code = result.code ?: "UNAVAILABLE", + message = result.message ?: "notification action failed", + ) + } + + val payload = + buildJsonObject { + put("ok", JsonPrimitive(true)) + put("key", JsonPrimitive(key)) + put("action", JsonPrimitive(actionRaw)) + }.toString() + return GatewaySession.InvokeResult.ok(payload) + } + + private fun readSnapshotWithRebind(): DeviceNotificationSnapshot { + val snapshot = stateProvider.readSnapshot(appContext) + if (snapshot.enabled && !snapshot.connected) { + stateProvider.requestServiceRebind(appContext) + } + return snapshot + } + + private fun snapshotPayloadJson(snapshot: DeviceNotificationSnapshot): String { + return buildJsonObject { + put("enabled", JsonPrimitive(snapshot.enabled)) + put("connected", JsonPrimitive(snapshot.connected)) + put("count", JsonPrimitive(snapshot.notifications.size)) + put( + "notifications", + JsonArray( + snapshot.notifications.map { entry -> + buildJsonObject { + put("key", JsonPrimitive(entry.key)) + put("packageName", JsonPrimitive(entry.packageName)) + put("postTimeMs", JsonPrimitive(entry.postTimeMs)) + put("isOngoing", JsonPrimitive(entry.isOngoing)) + put("isClearable", JsonPrimitive(entry.isClearable)) + entry.title?.let { put("title", JsonPrimitive(it)) } + entry.text?.let { put("text", JsonPrimitive(it)) } + entry.subText?.let { put("subText", JsonPrimitive(it)) } + entry.category?.let { put("category", JsonPrimitive(it)) } + entry.channelId?.let { put("channelId", JsonPrimitive(it)) } + } + }, + ), + ) + }.toString() + } + + private fun parseParamsObject(paramsJson: String?): JsonObject? { + if (paramsJson.isNullOrBlank()) return null + return try { + Json.parseToJsonElement(paramsJson).asObjectOrNull() + } catch (_: Throwable) { + null + } + } + + private fun readString(params: JsonObject, key: String): String? = + (params[key] as? JsonPrimitive) + ?.contentOrNull + ?.trim() + ?.takeIf { it.isNotEmpty() } + + companion object { + internal fun forTesting( + appContext: Context, + stateProvider: NotificationsStateProvider, + ): NotificationsHandler = NotificationsHandler(appContext = appContext, stateProvider = stateProvider) + } +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/node/PhotosHandler.kt b/apps/android/app/src/main/java/ai/openclaw/android/node/PhotosHandler.kt new file mode 100644 index 00000000000..3b9c0199d06 --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/node/PhotosHandler.kt @@ -0,0 +1,287 @@ +package ai.openclaw.android.node + +import android.Manifest +import android.content.ContentResolver +import android.content.ContentUris +import android.content.Context +import android.graphics.Bitmap +import android.graphics.BitmapFactory +import android.net.Uri +import android.os.Build +import android.os.Bundle +import android.provider.MediaStore +import androidx.core.content.ContextCompat +import ai.openclaw.android.gateway.GatewaySession +import java.io.ByteArrayOutputStream +import java.time.Instant +import kotlin.math.max +import kotlin.math.roundToInt +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.buildJsonArray +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.put + +private const val DEFAULT_PHOTOS_LIMIT = 1 +private const val DEFAULT_PHOTOS_MAX_WIDTH = 1600 +private const val DEFAULT_PHOTOS_QUALITY = 0.85 +private const val MAX_TOTAL_BASE64_CHARS = 340 * 1024 +private const val MAX_PER_PHOTO_BASE64_CHARS = 300 * 1024 + +internal data class PhotosLatestRequest( + val limit: Int, + val maxWidth: Int, + val quality: Double, +) + +internal data class EncodedPhotoPayload( + val format: String, + val base64: String, + val width: Int, + val height: Int, + val createdAt: String?, +) + +internal interface PhotosDataSource { + fun hasPermission(context: Context): Boolean + + fun latest(context: Context, request: PhotosLatestRequest): List +} + +private object SystemPhotosDataSource : PhotosDataSource { + override fun hasPermission(context: Context): Boolean { + val permission = + if (Build.VERSION.SDK_INT >= 33) { + Manifest.permission.READ_MEDIA_IMAGES + } else { + Manifest.permission.READ_EXTERNAL_STORAGE + } + return ContextCompat.checkSelfPermission(context, permission) == android.content.pm.PackageManager.PERMISSION_GRANTED + } + + override fun latest(context: Context, request: PhotosLatestRequest): List { + val resolver = context.contentResolver + val rows = queryLatestRows(resolver, request.limit) + if (rows.isEmpty()) return emptyList() + + var remainingBudget = MAX_TOTAL_BASE64_CHARS + val out = mutableListOf() + for (row in rows) { + if (remainingBudget <= 0) break + val bitmap = decodeScaledBitmap(resolver, row.uri, request.maxWidth) ?: continue + val encoded = encodeJpegUnderBudget(bitmap, request.quality, MAX_PER_PHOTO_BASE64_CHARS) ?: continue + if (encoded.base64.length > remainingBudget) break + remainingBudget -= encoded.base64.length + out += + EncodedPhotoPayload( + format = "jpeg", + base64 = encoded.base64, + width = encoded.width, + height = encoded.height, + createdAt = row.createdAtMs?.let { Instant.ofEpochMilli(it).toString() }, + ) + } + return out + } + + private data class PhotoRow( + val uri: Uri, + val createdAtMs: Long?, + ) + + private data class EncodedJpeg( + val base64: String, + val width: Int, + val height: Int, + ) + + private fun queryLatestRows(resolver: ContentResolver, limit: Int): List { + val projection = + arrayOf( + MediaStore.Images.Media._ID, + MediaStore.Images.Media.DATE_TAKEN, + MediaStore.Images.Media.DATE_ADDED, + ) + val sortOrder = + "${MediaStore.Images.Media.DATE_TAKEN} DESC, ${MediaStore.Images.Media.DATE_ADDED} DESC" + val args = + Bundle().apply { + putString(ContentResolver.QUERY_ARG_SQL_SORT_ORDER, sortOrder) + putInt(ContentResolver.QUERY_ARG_LIMIT, limit) + } + + resolver.query( + MediaStore.Images.Media.EXTERNAL_CONTENT_URI, + projection, + args, + null, + ).use { cursor -> + if (cursor == null) return emptyList() + val idIndex = cursor.getColumnIndexOrThrow(MediaStore.Images.Media._ID) + val takenIndex = cursor.getColumnIndexOrThrow(MediaStore.Images.Media.DATE_TAKEN) + val addedIndex = cursor.getColumnIndexOrThrow(MediaStore.Images.Media.DATE_ADDED) + val rows = mutableListOf() + while (cursor.moveToNext()) { + val id = cursor.getLong(idIndex) + val takenMs = cursor.getLong(takenIndex).takeIf { it > 0L } + val addedMs = cursor.getLong(addedIndex).takeIf { it > 0L }?.times(1000L) + rows += + PhotoRow( + uri = ContentUris.withAppendedId(MediaStore.Images.Media.EXTERNAL_CONTENT_URI, id), + createdAtMs = takenMs ?: addedMs, + ) + } + return rows + } + } + + private fun decodeScaledBitmap( + resolver: ContentResolver, + uri: Uri, + maxWidth: Int, + ): Bitmap? { + val bounds = BitmapFactory.Options().apply { inJustDecodeBounds = true } + resolver.openInputStream(uri).use { input -> + if (input == null) return null + BitmapFactory.decodeStream(input, null, bounds) + } + if (bounds.outWidth <= 0 || bounds.outHeight <= 0) return null + + val inSampleSize = computeInSampleSize(bounds.outWidth, maxWidth) + val decodeOptions = BitmapFactory.Options().apply { this.inSampleSize = inSampleSize } + val decoded = + resolver.openInputStream(uri).use { input -> + if (input == null) return null + BitmapFactory.decodeStream(input, null, decodeOptions) + } ?: return null + + if (decoded.width <= maxWidth) return decoded + val targetHeight = max(1, ((decoded.height.toDouble() * maxWidth) / decoded.width).roundToInt()) + return Bitmap.createScaledBitmap(decoded, maxWidth, targetHeight, true) + } + + private fun computeInSampleSize(width: Int, maxWidth: Int): Int { + var sample = 1 + var candidate = width + while (candidate > maxWidth && sample < 64) { + sample *= 2 + candidate = width / sample + } + return sample + } + + private fun encodeJpegUnderBudget( + bitmap: Bitmap, + quality: Double, + maxBase64Chars: Int, + ): EncodedJpeg? { + var working = bitmap + var jpegQuality = (quality.coerceIn(0.1, 1.0) * 100.0).roundToInt().coerceIn(10, 100) + repeat(10) { + val out = ByteArrayOutputStream() + val ok = working.compress(Bitmap.CompressFormat.JPEG, jpegQuality, out) + if (!ok) return null + val bytes = out.toByteArray() + val base64 = android.util.Base64.encodeToString(bytes, android.util.Base64.NO_WRAP) + if (base64.length <= maxBase64Chars) { + return EncodedJpeg( + base64 = base64, + width = working.width, + height = working.height, + ) + } + if (jpegQuality > 35) { + jpegQuality = max(25, jpegQuality - 15) + return@repeat + } + val nextWidth = max(240, (working.width * 0.75f).roundToInt()) + if (nextWidth >= working.width) return null + val nextHeight = max(1, ((working.height.toDouble() * nextWidth) / working.width).roundToInt()) + working = Bitmap.createScaledBitmap(working, nextWidth, nextHeight, true) + } + return null + } +} + +class PhotosHandler private constructor( + private val appContext: Context, + private val dataSource: PhotosDataSource, +) { + constructor(appContext: Context) : this(appContext = appContext, dataSource = SystemPhotosDataSource) + + fun handlePhotosLatest(paramsJson: String?): GatewaySession.InvokeResult { + if (!dataSource.hasPermission(appContext)) { + return GatewaySession.InvokeResult.error( + code = "PHOTOS_PERMISSION_REQUIRED", + message = "PHOTOS_PERMISSION_REQUIRED: grant Photos permission", + ) + } + val request = + parseRequest(paramsJson) + ?: return GatewaySession.InvokeResult.error( + code = "INVALID_REQUEST", + message = "INVALID_REQUEST: expected JSON object", + ) + return try { + val photos = dataSource.latest(appContext, request) + val payload = + buildJsonObject { + put( + "photos", + buildJsonArray { + photos.forEach { photo -> + add( + buildJsonObject { + put("format", JsonPrimitive(photo.format)) + put("base64", JsonPrimitive(photo.base64)) + put("width", JsonPrimitive(photo.width)) + put("height", JsonPrimitive(photo.height)) + photo.createdAt?.let { put("createdAt", JsonPrimitive(it)) } + }, + ) + } + }, + ) + }.toString() + GatewaySession.InvokeResult.ok(payload) + } catch (err: Throwable) { + GatewaySession.InvokeResult.error( + code = "PHOTOS_UNAVAILABLE", + message = "PHOTOS_UNAVAILABLE: ${err.message ?: "photo fetch failed"}", + ) + } + } + + private fun parseRequest(paramsJson: String?): PhotosLatestRequest? { + if (paramsJson.isNullOrBlank()) { + return PhotosLatestRequest( + limit = DEFAULT_PHOTOS_LIMIT, + maxWidth = DEFAULT_PHOTOS_MAX_WIDTH, + quality = DEFAULT_PHOTOS_QUALITY, + ) + } + val params = + try { + Json.parseToJsonElement(paramsJson).asObjectOrNull() + } catch (_: Throwable) { + null + } ?: return null + + val limitRaw = (params["limit"] as? JsonPrimitive)?.content?.toIntOrNull() + val maxWidthRaw = (params["maxWidth"] as? JsonPrimitive)?.content?.toIntOrNull() + val qualityRaw = (params["quality"] as? JsonPrimitive)?.content?.toDoubleOrNull() + + val limit = (limitRaw ?: DEFAULT_PHOTOS_LIMIT).coerceIn(1, 20) + val maxWidth = (maxWidthRaw ?: DEFAULT_PHOTOS_MAX_WIDTH).coerceIn(240, 4096) + val quality = (qualityRaw ?: DEFAULT_PHOTOS_QUALITY).coerceIn(0.1, 1.0) + return PhotosLatestRequest(limit = limit, maxWidth = maxWidth, quality = quality) + } + + companion object { + internal fun forTesting( + appContext: Context, + dataSource: PhotosDataSource, + ): PhotosHandler = PhotosHandler(appContext = appContext, dataSource = dataSource) + } +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/node/ScreenRecordManager.kt b/apps/android/app/src/main/java/ai/openclaw/android/node/ScreenRecordManager.kt index 337a953866a..98a3e4d9593 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/node/ScreenRecordManager.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/node/ScreenRecordManager.kt @@ -10,6 +10,10 @@ import ai.openclaw.android.ScreenCaptureRequester import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.delay import kotlinx.coroutines.withContext +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.contentOrNull import java.io.File import kotlin.math.roundToInt @@ -35,12 +39,13 @@ class ScreenRecordManager(private val context: Context) { "SCREEN_PERMISSION_REQUIRED: grant Screen Recording permission", ) - val durationMs = (parseDurationMs(paramsJson) ?: 10_000).coerceIn(250, 60_000) - val fps = (parseFps(paramsJson) ?: 10.0).coerceIn(1.0, 60.0) + val params = parseParamsObject(paramsJson) + val durationMs = (parseDurationMs(params) ?: 10_000).coerceIn(250, 60_000) + val fps = (parseFps(params) ?: 10.0).coerceIn(1.0, 60.0) val fpsInt = fps.roundToInt().coerceIn(1, 60) - val screenIndex = parseScreenIndex(paramsJson) - val includeAudio = parseIncludeAudio(paramsJson) ?: true - val format = parseString(paramsJson, key = "format") + val screenIndex = parseScreenIndex(params) + val includeAudio = parseIncludeAudio(params) ?: true + val format = parseString(params, key = "format") if (format != null && format.lowercase() != "mp4") { throw IllegalArgumentException("INVALID_REQUEST: screen format must be mp4") } @@ -141,55 +146,38 @@ class ScreenRecordManager(private val context: Context) { } } - private fun parseDurationMs(paramsJson: String?): Int? = - parseNumber(paramsJson, key = "durationMs")?.toIntOrNull() + private fun parseParamsObject(paramsJson: String?): JsonObject? { + if (paramsJson.isNullOrBlank()) return null + return try { + Json.parseToJsonElement(paramsJson).asObjectOrNull() + } catch (_: Throwable) { + null + } + } - private fun parseFps(paramsJson: String?): Double? = - parseNumber(paramsJson, key = "fps")?.toDoubleOrNull() + private fun readPrimitive(params: JsonObject?, key: String): JsonPrimitive? = + params?.get(key) as? JsonPrimitive - private fun parseScreenIndex(paramsJson: String?): Int? = - parseNumber(paramsJson, key = "screenIndex")?.toIntOrNull() + private fun parseDurationMs(params: JsonObject?): Int? = + readPrimitive(params, "durationMs")?.contentOrNull?.toIntOrNull() - private fun parseIncludeAudio(paramsJson: String?): Boolean? { - val raw = paramsJson ?: return null - val key = "\"includeAudio\"" - val idx = raw.indexOf(key) - if (idx < 0) return null - val colon = raw.indexOf(':', idx + key.length) - if (colon < 0) return null - val tail = raw.substring(colon + 1).trimStart() - return when { - tail.startsWith("true") -> true - tail.startsWith("false") -> false + private fun parseFps(params: JsonObject?): Double? = + readPrimitive(params, "fps")?.contentOrNull?.toDoubleOrNull() + + private fun parseScreenIndex(params: JsonObject?): Int? = + readPrimitive(params, "screenIndex")?.contentOrNull?.toIntOrNull() + + private fun parseIncludeAudio(params: JsonObject?): Boolean? { + val value = readPrimitive(params, "includeAudio")?.contentOrNull?.trim()?.lowercase() + return when (value) { + "true" -> true + "false" -> false else -> null } } - private fun parseNumber(paramsJson: String?, key: String): String? { - val raw = paramsJson ?: return null - val needle = "\"$key\"" - val idx = raw.indexOf(needle) - if (idx < 0) return null - val colon = raw.indexOf(':', idx + needle.length) - if (colon < 0) return null - val tail = raw.substring(colon + 1).trimStart() - return tail.takeWhile { it.isDigit() || it == '.' || it == '-' } - } - - private fun parseString(paramsJson: String?, key: String): String? { - val raw = paramsJson ?: return null - val needle = "\"$key\"" - val idx = raw.indexOf(needle) - if (idx < 0) return null - val colon = raw.indexOf(':', idx + needle.length) - if (colon < 0) return null - val tail = raw.substring(colon + 1).trimStart() - if (!tail.startsWith('\"')) return null - val rest = tail.drop(1) - val end = rest.indexOf('\"') - if (end < 0) return null - return rest.substring(0, end) - } + private fun parseString(params: JsonObject?, key: String): String? = + readPrimitive(params, key)?.contentOrNull private fun estimateBitrate(width: Int, height: Int, fps: Int): Int { val pixels = width.toLong() * height.toLong() diff --git a/apps/android/app/src/main/java/ai/openclaw/android/node/SystemHandler.kt b/apps/android/app/src/main/java/ai/openclaw/android/node/SystemHandler.kt new file mode 100644 index 00000000000..171cd45fd95 --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/node/SystemHandler.kt @@ -0,0 +1,162 @@ +package ai.openclaw.android.node + +import android.Manifest +import android.app.NotificationChannel +import android.app.NotificationManager +import android.content.Context +import android.os.Build +import androidx.core.app.NotificationCompat +import androidx.core.app.NotificationManagerCompat +import androidx.core.content.ContextCompat +import ai.openclaw.android.gateway.GatewaySession +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.contentOrNull + +private const val NOTIFICATION_CHANNEL_BASE_ID = "openclaw.system.notify" + +internal data class SystemNotifyRequest( + val title: String, + val body: String, + val sound: String?, + val priority: String?, +) + +internal interface SystemNotificationPoster { + fun isAuthorized(): Boolean + + fun post(request: SystemNotifyRequest) +} + +private class AndroidSystemNotificationPoster( + private val appContext: Context, +) : SystemNotificationPoster { + override fun isAuthorized(): Boolean { + if (Build.VERSION.SDK_INT >= 33) { + val granted = + ContextCompat.checkSelfPermission(appContext, Manifest.permission.POST_NOTIFICATIONS) == + android.content.pm.PackageManager.PERMISSION_GRANTED + if (!granted) return false + } + return NotificationManagerCompat.from(appContext).areNotificationsEnabled() + } + + override fun post(request: SystemNotifyRequest) { + val channelId = ensureChannel(request.priority) + val silent = isSilentSound(request.sound) + val notification = + NotificationCompat.Builder(appContext, channelId) + .setSmallIcon(android.R.drawable.ic_dialog_info) + .setContentTitle(request.title) + .setContentText(request.body) + .setPriority(compatPriority(request.priority)) + .setAutoCancel(true) + .setOnlyAlertOnce(true) + .setSilent(silent) + .build() + NotificationManagerCompat.from(appContext).notify((System.currentTimeMillis() and 0x7FFFFFFF).toInt(), notification) + } + + private fun ensureChannel(priority: String?): String { + if (Build.VERSION.SDK_INT < Build.VERSION_CODES.O) { + return NOTIFICATION_CHANNEL_BASE_ID + } + val normalizedPriority = priority.orEmpty().trim().lowercase() + val (suffix, importance, name) = + when (normalizedPriority) { + "passive" -> Triple("passive", NotificationManager.IMPORTANCE_LOW, "OpenClaw Passive") + "timesensitive" -> Triple("timesensitive", NotificationManager.IMPORTANCE_HIGH, "OpenClaw Time Sensitive") + else -> Triple("active", NotificationManager.IMPORTANCE_DEFAULT, "OpenClaw Active") + } + val channelId = "$NOTIFICATION_CHANNEL_BASE_ID.$suffix" + val manager = appContext.getSystemService(NotificationManager::class.java) + val existing = manager.getNotificationChannel(channelId) + if (existing == null) { + manager.createNotificationChannel(NotificationChannel(channelId, name, importance)) + } + return channelId + } + + private fun compatPriority(priority: String?): Int { + return when (priority.orEmpty().trim().lowercase()) { + "passive" -> NotificationCompat.PRIORITY_LOW + "timesensitive" -> NotificationCompat.PRIORITY_HIGH + else -> NotificationCompat.PRIORITY_DEFAULT + } + } + + private fun isSilentSound(sound: String?): Boolean { + val normalized = sound?.trim()?.lowercase() ?: return false + return normalized in setOf("none", "silent", "off", "false", "0") + } +} + +class SystemHandler private constructor( + private val poster: SystemNotificationPoster, +) { + constructor(appContext: Context) : this(poster = AndroidSystemNotificationPoster(appContext)) + + fun handleSystemNotify(paramsJson: String?): GatewaySession.InvokeResult { + val params = + parseNotifyRequest(paramsJson) + ?: return GatewaySession.InvokeResult.error( + code = "INVALID_REQUEST", + message = "INVALID_REQUEST: expected JSON object with title/body", + ) + if (params.title.isEmpty() && params.body.isEmpty()) { + return GatewaySession.InvokeResult.error( + code = "INVALID_REQUEST", + message = "INVALID_REQUEST: empty notification", + ) + } + if (!poster.isAuthorized()) { + return GatewaySession.InvokeResult.error( + code = "NOT_AUTHORIZED", + message = "NOT_AUTHORIZED: notifications", + ) + } + return try { + poster.post(params) + GatewaySession.InvokeResult.ok(null) + } catch (err: Throwable) { + GatewaySession.InvokeResult.error( + code = "UNAVAILABLE", + message = "NOTIFICATION_FAILED: ${err.message ?: "notification post failed"}", + ) + } + } + + private fun parseNotifyRequest(paramsJson: String?): SystemNotifyRequest? { + val params = parseParamsObject(paramsJson) ?: return null + val rawTitle = + (params["title"] as? JsonPrimitive) + ?.contentOrNull + ?: return null + val rawBody = + (params["body"] as? JsonPrimitive) + ?.contentOrNull + ?: return null + val sound = (params["sound"] as? JsonPrimitive)?.contentOrNull + val priority = (params["priority"] as? JsonPrimitive)?.contentOrNull + return SystemNotifyRequest( + title = rawTitle.trim(), + body = rawBody.trim(), + sound = sound?.trim()?.ifEmpty { null }, + priority = priority?.trim()?.ifEmpty { null }, + ) + } + + private fun parseParamsObject(paramsJson: String?): JsonObject? { + if (paramsJson.isNullOrBlank()) return null + return try { + Json.parseToJsonElement(paramsJson).asObjectOrNull() + } catch (_: Throwable) { + null + } + } + + companion object { + internal fun forTesting(poster: SystemNotificationPoster): SystemHandler = SystemHandler(poster) + } +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/protocol/OpenClawProtocolConstants.kt b/apps/android/app/src/main/java/ai/openclaw/android/protocol/OpenClawProtocolConstants.kt index ccca40c4c35..a2816e257fa 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/protocol/OpenClawProtocolConstants.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/protocol/OpenClawProtocolConstants.kt @@ -7,6 +7,14 @@ enum class OpenClawCapability(val rawValue: String) { Sms("sms"), VoiceWake("voiceWake"), Location("location"), + Device("device"), + Notifications("notifications"), + System("system"), + AppUpdate("appUpdate"), + Photos("photos"), + Contacts("contacts"), + Calendar("calendar"), + Motion("motion"), } enum class OpenClawCanvasCommand(val rawValue: String) { @@ -34,6 +42,7 @@ enum class OpenClawCanvasA2UICommand(val rawValue: String) { } enum class OpenClawCameraCommand(val rawValue: String) { + List("camera.list"), Snap("camera.snap"), Clip("camera.clip"), ; @@ -69,3 +78,73 @@ enum class OpenClawLocationCommand(val rawValue: String) { const val NamespacePrefix: String = "location." } } + +enum class OpenClawDeviceCommand(val rawValue: String) { + Status("device.status"), + Info("device.info"), + Permissions("device.permissions"), + Health("device.health"), + ; + + companion object { + const val NamespacePrefix: String = "device." + } +} + +enum class OpenClawNotificationsCommand(val rawValue: String) { + List("notifications.list"), + Actions("notifications.actions"), + ; + + companion object { + const val NamespacePrefix: String = "notifications." + } +} + +enum class OpenClawSystemCommand(val rawValue: String) { + Notify("system.notify"), + ; + + companion object { + const val NamespacePrefix: String = "system." + } +} + +enum class OpenClawPhotosCommand(val rawValue: String) { + Latest("photos.latest"), + ; + + companion object { + const val NamespacePrefix: String = "photos." + } +} + +enum class OpenClawContactsCommand(val rawValue: String) { + Search("contacts.search"), + Add("contacts.add"), + ; + + companion object { + const val NamespacePrefix: String = "contacts." + } +} + +enum class OpenClawCalendarCommand(val rawValue: String) { + Events("calendar.events"), + Add("calendar.add"), + ; + + companion object { + const val NamespacePrefix: String = "calendar." + } +} + +enum class OpenClawMotionCommand(val rawValue: String) { + Activity("motion.activity"), + Pedometer("motion.pedometer"), + ; + + companion object { + const val NamespacePrefix: String = "motion." + } +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/ui/CanvasScreen.kt b/apps/android/app/src/main/java/ai/openclaw/android/ui/CanvasScreen.kt new file mode 100644 index 00000000000..f733d154ed9 --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/ui/CanvasScreen.kt @@ -0,0 +1,150 @@ +package ai.openclaw.android.ui + +import android.annotation.SuppressLint +import android.util.Log +import android.view.View +import android.webkit.ConsoleMessage +import android.webkit.JavascriptInterface +import android.webkit.WebChromeClient +import android.webkit.WebResourceError +import android.webkit.WebResourceRequest +import android.webkit.WebResourceResponse +import android.webkit.WebSettings +import android.webkit.WebView +import android.webkit.WebViewClient +import androidx.compose.runtime.Composable +import androidx.compose.runtime.DisposableEffect +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.ui.Modifier +import androidx.compose.ui.platform.LocalContext +import androidx.compose.ui.viewinterop.AndroidView +import androidx.webkit.WebSettingsCompat +import androidx.webkit.WebViewFeature +import ai.openclaw.android.MainViewModel + +@SuppressLint("SetJavaScriptEnabled") +@Composable +fun CanvasScreen(viewModel: MainViewModel, modifier: Modifier = Modifier) { + val context = LocalContext.current + val isDebuggable = (context.applicationInfo.flags and android.content.pm.ApplicationInfo.FLAG_DEBUGGABLE) != 0 + val webViewRef = remember { mutableStateOf(null) } + + DisposableEffect(viewModel) { + onDispose { + val webView = webViewRef.value ?: return@onDispose + viewModel.canvas.detach(webView) + webView.removeJavascriptInterface(CanvasA2UIActionBridge.interfaceName) + webView.stopLoading() + webView.destroy() + webViewRef.value = null + } + } + + AndroidView( + modifier = modifier, + factory = { + WebView(context).apply { + settings.javaScriptEnabled = true + settings.domStorageEnabled = true + settings.mixedContentMode = WebSettings.MIXED_CONTENT_COMPATIBILITY_MODE + settings.useWideViewPort = false + settings.loadWithOverviewMode = false + settings.builtInZoomControls = false + settings.displayZoomControls = false + settings.setSupportZoom(false) + if (WebViewFeature.isFeatureSupported(WebViewFeature.ALGORITHMIC_DARKENING)) { + WebSettingsCompat.setAlgorithmicDarkeningAllowed(settings, false) + } else { + disableForceDarkIfSupported(settings) + } + if (isDebuggable) { + Log.d("OpenClawWebView", "userAgent: ${settings.userAgentString}") + } + isScrollContainer = true + overScrollMode = View.OVER_SCROLL_IF_CONTENT_SCROLLS + isVerticalScrollBarEnabled = true + isHorizontalScrollBarEnabled = true + webViewClient = + object : WebViewClient() { + override fun onReceivedError( + view: WebView, + request: WebResourceRequest, + error: WebResourceError, + ) { + if (!isDebuggable || !request.isForMainFrame) return + Log.e("OpenClawWebView", "onReceivedError: ${error.errorCode} ${error.description} ${request.url}") + } + + override fun onReceivedHttpError( + view: WebView, + request: WebResourceRequest, + errorResponse: WebResourceResponse, + ) { + if (!isDebuggable || !request.isForMainFrame) return + Log.e( + "OpenClawWebView", + "onReceivedHttpError: ${errorResponse.statusCode} ${errorResponse.reasonPhrase} ${request.url}", + ) + } + + override fun onPageFinished(view: WebView, url: String?) { + if (isDebuggable) { + Log.d("OpenClawWebView", "onPageFinished: $url") + } + viewModel.canvas.onPageFinished() + } + + override fun onRenderProcessGone( + view: WebView, + detail: android.webkit.RenderProcessGoneDetail, + ): Boolean { + if (isDebuggable) { + Log.e( + "OpenClawWebView", + "onRenderProcessGone didCrash=${detail.didCrash()} priorityAtExit=${detail.rendererPriorityAtExit()}", + ) + } + return true + } + } + webChromeClient = + object : WebChromeClient() { + override fun onConsoleMessage(consoleMessage: ConsoleMessage?): Boolean { + if (!isDebuggable) return false + val msg = consoleMessage ?: return false + Log.d( + "OpenClawWebView", + "console ${msg.messageLevel()} @ ${msg.sourceId()}:${msg.lineNumber()} ${msg.message()}", + ) + return false + } + } + + val bridge = CanvasA2UIActionBridge { payload -> viewModel.handleCanvasA2UIActionFromWebView(payload) } + addJavascriptInterface(bridge, CanvasA2UIActionBridge.interfaceName) + viewModel.canvas.attach(this) + webViewRef.value = this + } + }, + ) +} + +private fun disableForceDarkIfSupported(settings: WebSettings) { + if (!WebViewFeature.isFeatureSupported(WebViewFeature.FORCE_DARK)) return + @Suppress("DEPRECATION") + WebSettingsCompat.setForceDark(settings, WebSettingsCompat.FORCE_DARK_OFF) +} + +private class CanvasA2UIActionBridge(private val onMessage: (String) -> Unit) { + @JavascriptInterface + fun postMessage(payload: String?) { + val msg = payload?.trim().orEmpty() + if (msg.isEmpty()) return + onMessage(msg) + } + + companion object { + const val interfaceName: String = "openclawCanvasA2UIAction" + } +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/ui/ConnectTabScreen.kt b/apps/android/app/src/main/java/ai/openclaw/android/ui/ConnectTabScreen.kt new file mode 100644 index 00000000000..875b82796d3 --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/ui/ConnectTabScreen.kt @@ -0,0 +1,493 @@ +package ai.openclaw.android.ui + +import androidx.compose.animation.AnimatedVisibility +import androidx.compose.foundation.BorderStroke +import androidx.compose.foundation.background +import androidx.compose.foundation.layout.Arrangement +import androidx.compose.foundation.layout.Box +import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.PaddingValues +import androidx.compose.foundation.layout.Row +import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.height +import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.layout.size +import androidx.compose.foundation.layout.width +import androidx.compose.foundation.rememberScrollState +import androidx.compose.foundation.shape.RoundedCornerShape +import androidx.compose.foundation.text.KeyboardOptions +import androidx.compose.foundation.verticalScroll +import androidx.compose.material.icons.Icons +import androidx.compose.material.icons.filled.ExpandLess +import androidx.compose.material.icons.filled.ExpandMore +import androidx.compose.material3.AlertDialog +import androidx.compose.material3.Button +import androidx.compose.material3.ButtonDefaults +import androidx.compose.material3.HorizontalDivider +import androidx.compose.material3.Icon +import androidx.compose.material3.OutlinedTextField +import androidx.compose.material3.OutlinedTextFieldDefaults +import androidx.compose.material3.Surface +import androidx.compose.material3.Switch +import androidx.compose.material3.SwitchDefaults +import androidx.compose.material3.Text +import androidx.compose.material3.TextButton +import androidx.compose.runtime.Composable +import androidx.compose.runtime.collectAsState +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.saveable.rememberSaveable +import androidx.compose.runtime.setValue +import androidx.compose.ui.Alignment +import androidx.compose.ui.Modifier +import androidx.compose.ui.graphics.Color +import androidx.compose.ui.text.font.FontFamily +import androidx.compose.ui.text.font.FontWeight +import androidx.compose.ui.text.input.KeyboardType +import androidx.compose.ui.unit.dp +import ai.openclaw.android.MainViewModel + +private enum class ConnectInputMode { + SetupCode, + Manual, +} + +@Composable +fun ConnectTabScreen(viewModel: MainViewModel) { + val statusText by viewModel.statusText.collectAsState() + val isConnected by viewModel.isConnected.collectAsState() + val remoteAddress by viewModel.remoteAddress.collectAsState() + val manualHost by viewModel.manualHost.collectAsState() + val manualPort by viewModel.manualPort.collectAsState() + val manualTls by viewModel.manualTls.collectAsState() + val manualEnabled by viewModel.manualEnabled.collectAsState() + val gatewayToken by viewModel.gatewayToken.collectAsState() + val pendingTrust by viewModel.pendingGatewayTrust.collectAsState() + + var advancedOpen by rememberSaveable { mutableStateOf(false) } + var inputMode by + remember(manualEnabled, manualHost, gatewayToken) { + mutableStateOf( + if (manualEnabled || manualHost.isNotBlank() || gatewayToken.trim().isNotEmpty()) { + ConnectInputMode.Manual + } else { + ConnectInputMode.SetupCode + }, + ) + } + var setupCode by rememberSaveable { mutableStateOf("") } + var manualHostInput by rememberSaveable { mutableStateOf(manualHost.ifBlank { "10.0.2.2" }) } + var manualPortInput by rememberSaveable { mutableStateOf(manualPort.toString()) } + var manualTlsInput by rememberSaveable { mutableStateOf(manualTls) } + var passwordInput by rememberSaveable { mutableStateOf("") } + var validationText by rememberSaveable { mutableStateOf(null) } + + if (pendingTrust != null) { + val prompt = pendingTrust!! + AlertDialog( + onDismissRequest = { viewModel.declineGatewayTrustPrompt() }, + title = { Text("Trust this gateway?") }, + text = { + Text( + "First-time TLS connection.\n\nVerify this SHA-256 fingerprint before trusting:\n${prompt.fingerprintSha256}", + style = mobileCallout, + ) + }, + confirmButton = { + TextButton(onClick = { viewModel.acceptGatewayTrustPrompt() }) { + Text("Trust and continue") + } + }, + dismissButton = { + TextButton(onClick = { viewModel.declineGatewayTrustPrompt() }) { + Text("Cancel") + } + }, + ) + } + + val setupResolvedEndpoint = remember(setupCode) { decodeGatewaySetupCode(setupCode)?.url?.let { parseGatewayEndpoint(it)?.displayUrl } } + val manualResolvedEndpoint = remember(manualHostInput, manualPortInput, manualTlsInput) { + composeGatewayManualUrl(manualHostInput, manualPortInput, manualTlsInput)?.let { parseGatewayEndpoint(it)?.displayUrl } + } + + val activeEndpoint = + remember(isConnected, remoteAddress, setupResolvedEndpoint, manualResolvedEndpoint, inputMode) { + when { + isConnected && !remoteAddress.isNullOrBlank() -> remoteAddress!! + inputMode == ConnectInputMode.SetupCode -> setupResolvedEndpoint ?: "Not set" + else -> manualResolvedEndpoint ?: "Not set" + } + } + + val primaryLabel = if (isConnected) "Disconnect Gateway" else "Connect Gateway" + + Column( + modifier = Modifier.verticalScroll(rememberScrollState()).padding(horizontal = 20.dp, vertical = 16.dp), + verticalArrangement = Arrangement.spacedBy(14.dp), + ) { + Column(verticalArrangement = Arrangement.spacedBy(6.dp)) { + Text("Connection Control", style = mobileCaption1.copy(fontWeight = FontWeight.Bold), color = mobileAccent) + Text("Gateway Connection", style = mobileTitle1, color = mobileText) + Text( + "One primary action. Open advanced controls only when needed.", + style = mobileCallout, + color = mobileTextSecondary, + ) + } + + Surface( + modifier = Modifier.fillMaxWidth(), + shape = RoundedCornerShape(14.dp), + color = mobileSurface, + border = BorderStroke(1.dp, mobileBorder), + ) { + Column(modifier = Modifier.padding(horizontal = 14.dp, vertical = 12.dp), verticalArrangement = Arrangement.spacedBy(4.dp)) { + Text("Active endpoint", style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), color = mobileTextSecondary) + Text(activeEndpoint, style = mobileBody.copy(fontFamily = FontFamily.Monospace), color = mobileText) + } + } + + Surface( + modifier = Modifier.fillMaxWidth(), + shape = RoundedCornerShape(14.dp), + color = mobileSurface, + border = BorderStroke(1.dp, mobileBorder), + ) { + Column(modifier = Modifier.padding(horizontal = 14.dp, vertical = 12.dp), verticalArrangement = Arrangement.spacedBy(4.dp)) { + Text("Gateway state", style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), color = mobileTextSecondary) + Text(statusText, style = mobileBody, color = mobileText) + } + } + + Button( + onClick = { + if (isConnected) { + viewModel.disconnect() + validationText = null + return@Button + } + if (statusText.contains("operator offline", ignoreCase = true)) { + validationText = null + viewModel.refreshGatewayConnection() + return@Button + } + + val config = + resolveGatewayConnectConfig( + useSetupCode = inputMode == ConnectInputMode.SetupCode, + setupCode = setupCode, + manualHost = manualHostInput, + manualPort = manualPortInput, + manualTls = manualTlsInput, + fallbackToken = gatewayToken, + fallbackPassword = passwordInput, + ) + + if (config == null) { + validationText = + if (inputMode == ConnectInputMode.SetupCode) { + "Paste a valid setup code to connect." + } else { + "Enter a valid manual host and port to connect." + } + return@Button + } + + validationText = null + viewModel.setManualEnabled(true) + viewModel.setManualHost(config.host) + viewModel.setManualPort(config.port) + viewModel.setManualTls(config.tls) + if (config.token.isNotBlank()) { + viewModel.setGatewayToken(config.token) + } + viewModel.setGatewayPassword(config.password) + viewModel.connectManual() + }, + modifier = Modifier.fillMaxWidth().height(52.dp), + shape = RoundedCornerShape(14.dp), + colors = + ButtonDefaults.buttonColors( + containerColor = if (isConnected) mobileDanger else mobileAccent, + contentColor = Color.White, + ), + ) { + Text(primaryLabel, style = mobileHeadline.copy(fontWeight = FontWeight.Bold)) + } + + Surface( + modifier = Modifier.fillMaxWidth(), + shape = RoundedCornerShape(14.dp), + color = mobileSurface, + border = BorderStroke(1.dp, mobileBorder), + onClick = { advancedOpen = !advancedOpen }, + ) { + Row( + modifier = Modifier.fillMaxWidth().padding(horizontal = 14.dp, vertical = 12.dp), + verticalAlignment = Alignment.CenterVertically, + horizontalArrangement = Arrangement.SpaceBetween, + ) { + Column(verticalArrangement = Arrangement.spacedBy(2.dp)) { + Text("Advanced controls", style = mobileHeadline, color = mobileText) + Text("Setup code, endpoint, TLS, token, password, onboarding.", style = mobileCaption1, color = mobileTextSecondary) + } + Icon( + imageVector = if (advancedOpen) Icons.Default.ExpandLess else Icons.Default.ExpandMore, + contentDescription = if (advancedOpen) "Collapse advanced controls" else "Expand advanced controls", + tint = mobileTextSecondary, + ) + } + } + + AnimatedVisibility(visible = advancedOpen) { + Surface( + modifier = Modifier.fillMaxWidth(), + shape = RoundedCornerShape(14.dp), + color = Color.White, + border = BorderStroke(1.dp, mobileBorder), + ) { + Column( + modifier = Modifier.fillMaxWidth().padding(horizontal = 14.dp, vertical = 14.dp), + verticalArrangement = Arrangement.spacedBy(12.dp), + ) { + Text("Connection method", style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), color = mobileTextSecondary) + Row(horizontalArrangement = Arrangement.spacedBy(8.dp)) { + MethodChip( + label = "Setup Code", + active = inputMode == ConnectInputMode.SetupCode, + onClick = { inputMode = ConnectInputMode.SetupCode }, + ) + MethodChip( + label = "Manual", + active = inputMode == ConnectInputMode.Manual, + onClick = { inputMode = ConnectInputMode.Manual }, + ) + } + + Text("Run these on the gateway host:", style = mobileCallout, color = mobileTextSecondary) + CommandBlock("openclaw qr --setup-code-only") + CommandBlock("openclaw qr --json") + + if (inputMode == ConnectInputMode.SetupCode) { + Text("Setup Code", style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), color = mobileTextSecondary) + OutlinedTextField( + value = setupCode, + onValueChange = { + setupCode = it + validationText = null + }, + placeholder = { Text("Paste setup code", style = mobileBody, color = mobileTextTertiary) }, + modifier = Modifier.fillMaxWidth(), + minLines = 3, + maxLines = 5, + keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Ascii), + textStyle = mobileBody.copy(fontFamily = FontFamily.Monospace, color = mobileText), + shape = RoundedCornerShape(14.dp), + colors = outlinedColors(), + ) + if (!setupResolvedEndpoint.isNullOrBlank()) { + EndpointPreview(endpoint = setupResolvedEndpoint) + } + } else { + Row(horizontalArrangement = Arrangement.spacedBy(8.dp)) { + QuickFillChip( + label = "Android Emulator", + onClick = { + manualHostInput = "10.0.2.2" + manualPortInput = "18789" + manualTlsInput = false + validationText = null + }, + ) + QuickFillChip( + label = "Localhost", + onClick = { + manualHostInput = "127.0.0.1" + manualPortInput = "18789" + manualTlsInput = false + validationText = null + }, + ) + } + + Text("Host", style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), color = mobileTextSecondary) + OutlinedTextField( + value = manualHostInput, + onValueChange = { + manualHostInput = it + validationText = null + }, + placeholder = { Text("10.0.2.2", style = mobileBody, color = mobileTextTertiary) }, + modifier = Modifier.fillMaxWidth(), + singleLine = true, + keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Uri), + textStyle = mobileBody.copy(color = mobileText), + shape = RoundedCornerShape(14.dp), + colors = outlinedColors(), + ) + + Text("Port", style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), color = mobileTextSecondary) + OutlinedTextField( + value = manualPortInput, + onValueChange = { + manualPortInput = it + validationText = null + }, + placeholder = { Text("18789", style = mobileBody, color = mobileTextTertiary) }, + modifier = Modifier.fillMaxWidth(), + singleLine = true, + keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Number), + textStyle = mobileBody.copy(fontFamily = FontFamily.Monospace, color = mobileText), + shape = RoundedCornerShape(14.dp), + colors = outlinedColors(), + ) + + Row( + modifier = Modifier.fillMaxWidth(), + verticalAlignment = Alignment.CenterVertically, + horizontalArrangement = Arrangement.SpaceBetween, + ) { + Column(verticalArrangement = Arrangement.spacedBy(2.dp)) { + Text("Use TLS", style = mobileHeadline, color = mobileText) + Text("Switch to secure websocket (`wss`).", style = mobileCallout, color = mobileTextSecondary) + } + Switch( + checked = manualTlsInput, + onCheckedChange = { + manualTlsInput = it + validationText = null + }, + colors = + SwitchDefaults.colors( + checkedTrackColor = mobileAccent, + uncheckedTrackColor = mobileBorderStrong, + checkedThumbColor = Color.White, + uncheckedThumbColor = Color.White, + ), + ) + } + + Text("Token (optional)", style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), color = mobileTextSecondary) + OutlinedTextField( + value = gatewayToken, + onValueChange = { viewModel.setGatewayToken(it) }, + placeholder = { Text("token", style = mobileBody, color = mobileTextTertiary) }, + modifier = Modifier.fillMaxWidth(), + singleLine = true, + keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Ascii), + textStyle = mobileBody.copy(color = mobileText), + shape = RoundedCornerShape(14.dp), + colors = outlinedColors(), + ) + + Text("Password (optional)", style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), color = mobileTextSecondary) + OutlinedTextField( + value = passwordInput, + onValueChange = { passwordInput = it }, + placeholder = { Text("password", style = mobileBody, color = mobileTextTertiary) }, + modifier = Modifier.fillMaxWidth(), + singleLine = true, + keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Ascii), + textStyle = mobileBody.copy(color = mobileText), + shape = RoundedCornerShape(14.dp), + colors = outlinedColors(), + ) + + if (!manualResolvedEndpoint.isNullOrBlank()) { + EndpointPreview(endpoint = manualResolvedEndpoint) + } + } + + HorizontalDivider(color = mobileBorder) + + TextButton(onClick = { viewModel.setOnboardingCompleted(false) }) { + Text("Run onboarding again", style = mobileCallout.copy(fontWeight = FontWeight.SemiBold), color = mobileAccent) + } + } + } + } + + if (!validationText.isNullOrBlank()) { + Text(validationText!!, style = mobileCaption1, color = mobileWarning) + } + } +} + +@Composable +private fun MethodChip(label: String, active: Boolean, onClick: () -> Unit) { + Button( + onClick = onClick, + modifier = Modifier.height(40.dp), + shape = RoundedCornerShape(12.dp), + contentPadding = PaddingValues(horizontal = 12.dp, vertical = 8.dp), + colors = + ButtonDefaults.buttonColors( + containerColor = if (active) mobileAccent else mobileSurface, + contentColor = if (active) Color.White else mobileText, + ), + border = BorderStroke(1.dp, if (active) Color(0xFF184DAF) else mobileBorderStrong), + ) { + Text(label, style = mobileCaption1.copy(fontWeight = FontWeight.Bold)) + } +} + +@Composable +private fun QuickFillChip(label: String, onClick: () -> Unit) { + Button( + onClick = onClick, + shape = RoundedCornerShape(999.dp), + contentPadding = PaddingValues(horizontal = 12.dp, vertical = 6.dp), + colors = + ButtonDefaults.buttonColors( + containerColor = mobileAccentSoft, + contentColor = mobileAccent, + ), + elevation = null, + ) { + Text(label, style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold)) + } +} + +@Composable +private fun CommandBlock(command: String) { + Surface( + modifier = Modifier.fillMaxWidth(), + shape = RoundedCornerShape(12.dp), + color = mobileCodeBg, + border = BorderStroke(1.dp, Color(0xFF2B2E35)), + ) { + Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) { + Box(modifier = Modifier.width(3.dp).height(42.dp).background(Color(0xFF3FC97A))) + Text( + text = command, + modifier = Modifier.padding(horizontal = 12.dp, vertical = 10.dp), + style = mobileCallout.copy(fontFamily = FontFamily.Monospace), + color = mobileCodeText, + ) + } + } +} + +@Composable +private fun EndpointPreview(endpoint: String) { + Column(verticalArrangement = Arrangement.spacedBy(4.dp)) { + HorizontalDivider(color = mobileBorder) + Text("Resolved endpoint", style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), color = mobileTextSecondary) + Text(endpoint, style = mobileCallout.copy(fontFamily = FontFamily.Monospace), color = mobileText) + HorizontalDivider(color = mobileBorder) + } +} + +@Composable +private fun outlinedColors() = + OutlinedTextFieldDefaults.colors( + focusedContainerColor = mobileSurface, + unfocusedContainerColor = mobileSurface, + focusedBorderColor = mobileAccent, + unfocusedBorderColor = mobileBorder, + focusedTextColor = mobileText, + unfocusedTextColor = mobileText, + cursorColor = mobileAccent, + ) diff --git a/apps/android/app/src/main/java/ai/openclaw/android/ui/GatewayConfigResolver.kt b/apps/android/app/src/main/java/ai/openclaw/android/ui/GatewayConfigResolver.kt new file mode 100644 index 00000000000..4421a82be4b --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/ui/GatewayConfigResolver.kt @@ -0,0 +1,142 @@ +package ai.openclaw.android.ui + +import androidx.core.net.toUri +import java.util.Base64 +import java.util.Locale +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonPrimitive +import kotlinx.serialization.json.contentOrNull +import kotlinx.serialization.json.jsonObject + +internal data class GatewayEndpointConfig( + val host: String, + val port: Int, + val tls: Boolean, + val displayUrl: String, +) + +internal data class GatewaySetupCode( + val url: String, + val token: String?, + val password: String?, +) + +internal data class GatewayConnectConfig( + val host: String, + val port: Int, + val tls: Boolean, + val token: String, + val password: String, +) + +private val gatewaySetupJson = Json { ignoreUnknownKeys = true } + +internal fun resolveGatewayConnectConfig( + useSetupCode: Boolean, + setupCode: String, + manualHost: String, + manualPort: String, + manualTls: Boolean, + fallbackToken: String, + fallbackPassword: String, +): GatewayConnectConfig? { + if (useSetupCode) { + val setup = decodeGatewaySetupCode(setupCode) ?: return null + val parsed = parseGatewayEndpoint(setup.url) ?: return null + return GatewayConnectConfig( + host = parsed.host, + port = parsed.port, + tls = parsed.tls, + token = setup.token ?: fallbackToken.trim(), + password = setup.password ?: fallbackPassword.trim(), + ) + } + + val manualUrl = composeGatewayManualUrl(manualHost, manualPort, manualTls) ?: return null + val parsed = parseGatewayEndpoint(manualUrl) ?: return null + return GatewayConnectConfig( + host = parsed.host, + port = parsed.port, + tls = parsed.tls, + token = fallbackToken.trim(), + password = fallbackPassword.trim(), + ) +} + +internal fun parseGatewayEndpoint(rawInput: String): GatewayEndpointConfig? { + val raw = rawInput.trim() + if (raw.isEmpty()) return null + + val normalized = if (raw.contains("://")) raw else "https://$raw" + val uri = normalized.toUri() + val host = uri.host?.trim().orEmpty() + if (host.isEmpty()) return null + + val scheme = uri.scheme?.trim()?.lowercase(Locale.US).orEmpty() + val tls = + when (scheme) { + "ws", "http" -> false + "wss", "https" -> true + else -> true + } + val port = uri.port.takeIf { it in 1..65535 } ?: 18789 + val displayUrl = "${if (tls) "https" else "http"}://$host:$port" + + return GatewayEndpointConfig(host = host, port = port, tls = tls, displayUrl = displayUrl) +} + +internal fun decodeGatewaySetupCode(rawInput: String): GatewaySetupCode? { + val trimmed = rawInput.trim() + if (trimmed.isEmpty()) return null + + val padded = + trimmed + .replace('-', '+') + .replace('_', '/') + .let { normalized -> + val remainder = normalized.length % 4 + if (remainder == 0) normalized else normalized + "=".repeat(4 - remainder) + } + + return try { + val decoded = String(Base64.getDecoder().decode(padded), Charsets.UTF_8) + val obj = parseJsonObject(decoded) ?: return null + val url = jsonField(obj, "url").orEmpty() + if (url.isEmpty()) return null + val token = jsonField(obj, "token") + val password = jsonField(obj, "password") + GatewaySetupCode(url = url, token = token, password = password) + } catch (_: IllegalArgumentException) { + null + } +} + +internal fun resolveScannedSetupCode(rawInput: String): String? { + val setupCode = resolveSetupCodeCandidate(rawInput) ?: return null + return setupCode.takeIf { decodeGatewaySetupCode(it) != null } +} + +internal fun composeGatewayManualUrl(hostInput: String, portInput: String, tls: Boolean): String? { + val host = hostInput.trim() + val port = portInput.trim().toIntOrNull() ?: return null + if (host.isEmpty() || port !in 1..65535) return null + val scheme = if (tls) "https" else "http" + return "$scheme://$host:$port" +} + +private fun parseJsonObject(input: String): JsonObject? { + return runCatching { gatewaySetupJson.parseToJsonElement(input).jsonObject }.getOrNull() +} + +private fun resolveSetupCodeCandidate(rawInput: String): String? { + val trimmed = rawInput.trim() + if (trimmed.isEmpty()) return null + val qrSetupCode = parseJsonObject(trimmed)?.let { jsonField(it, "setupCode") } + return qrSetupCode ?: trimmed +} + +private fun jsonField(obj: JsonObject, key: String): String? { + val value = (obj[key] as? JsonPrimitive)?.contentOrNull?.trim().orEmpty() + return value.ifEmpty { null } +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/ui/MobileUiTokens.kt b/apps/android/app/src/main/java/ai/openclaw/android/ui/MobileUiTokens.kt new file mode 100644 index 00000000000..eb4f95775e7 --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/ui/MobileUiTokens.kt @@ -0,0 +1,106 @@ +package ai.openclaw.android.ui + +import androidx.compose.ui.graphics.Brush +import androidx.compose.ui.graphics.Color +import androidx.compose.ui.text.TextStyle +import androidx.compose.ui.text.font.Font +import androidx.compose.ui.text.font.FontFamily +import androidx.compose.ui.text.font.FontWeight +import androidx.compose.ui.unit.sp +import ai.openclaw.android.R + +internal val mobileBackgroundGradient = + Brush.verticalGradient( + listOf( + Color(0xFFFFFFFF), + Color(0xFFF7F8FA), + Color(0xFFEFF1F5), + ), + ) + +internal val mobileSurface = Color(0xFFF6F7FA) +internal val mobileSurfaceStrong = Color(0xFFECEEF3) +internal val mobileBorder = Color(0xFFE5E7EC) +internal val mobileBorderStrong = Color(0xFFD6DAE2) +internal val mobileText = Color(0xFF17181C) +internal val mobileTextSecondary = Color(0xFF5D6472) +internal val mobileTextTertiary = Color(0xFF99A0AE) +internal val mobileAccent = Color(0xFF1D5DD8) +internal val mobileAccentSoft = Color(0xFFECF3FF) +internal val mobileSuccess = Color(0xFF2F8C5A) +internal val mobileSuccessSoft = Color(0xFFEEF9F3) +internal val mobileWarning = Color(0xFFC8841A) +internal val mobileWarningSoft = Color(0xFFFFF8EC) +internal val mobileDanger = Color(0xFFD04B4B) +internal val mobileDangerSoft = Color(0xFFFFF2F2) +internal val mobileCodeBg = Color(0xFF15171B) +internal val mobileCodeText = Color(0xFFE8EAEE) + +internal val mobileFontFamily = + FontFamily( + Font(resId = R.font.manrope_400_regular, weight = FontWeight.Normal), + Font(resId = R.font.manrope_500_medium, weight = FontWeight.Medium), + Font(resId = R.font.manrope_600_semibold, weight = FontWeight.SemiBold), + Font(resId = R.font.manrope_700_bold, weight = FontWeight.Bold), + ) + +internal val mobileTitle1 = + TextStyle( + fontFamily = mobileFontFamily, + fontWeight = FontWeight.SemiBold, + fontSize = 24.sp, + lineHeight = 30.sp, + letterSpacing = (-0.5).sp, + ) + +internal val mobileTitle2 = + TextStyle( + fontFamily = mobileFontFamily, + fontWeight = FontWeight.SemiBold, + fontSize = 20.sp, + lineHeight = 26.sp, + letterSpacing = (-0.3).sp, + ) + +internal val mobileHeadline = + TextStyle( + fontFamily = mobileFontFamily, + fontWeight = FontWeight.SemiBold, + fontSize = 16.sp, + lineHeight = 22.sp, + letterSpacing = (-0.1).sp, + ) + +internal val mobileBody = + TextStyle( + fontFamily = mobileFontFamily, + fontWeight = FontWeight.Medium, + fontSize = 15.sp, + lineHeight = 22.sp, + ) + +internal val mobileCallout = + TextStyle( + fontFamily = mobileFontFamily, + fontWeight = FontWeight.Medium, + fontSize = 14.sp, + lineHeight = 20.sp, + ) + +internal val mobileCaption1 = + TextStyle( + fontFamily = mobileFontFamily, + fontWeight = FontWeight.Medium, + fontSize = 12.sp, + lineHeight = 16.sp, + letterSpacing = 0.2.sp, + ) + +internal val mobileCaption2 = + TextStyle( + fontFamily = mobileFontFamily, + fontWeight = FontWeight.Medium, + fontSize = 11.sp, + lineHeight = 14.sp, + letterSpacing = 0.4.sp, + ) diff --git a/apps/android/app/src/main/java/ai/openclaw/android/ui/OnboardingFlow.kt b/apps/android/app/src/main/java/ai/openclaw/android/ui/OnboardingFlow.kt new file mode 100644 index 00000000000..1a25cce68d3 --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/ui/OnboardingFlow.kt @@ -0,0 +1,1678 @@ +package ai.openclaw.android.ui + +import android.Manifest +import android.content.Context +import android.content.Intent +import android.content.pm.PackageManager +import android.hardware.Sensor +import android.hardware.SensorManager +import android.net.Uri +import android.os.Build +import android.provider.Settings +import androidx.activity.compose.rememberLauncherForActivityResult +import androidx.activity.result.contract.ActivityResultContracts +import androidx.compose.animation.AnimatedVisibility +import androidx.compose.foundation.background +import androidx.compose.foundation.border +import androidx.compose.foundation.layout.Arrangement +import androidx.compose.foundation.layout.Box +import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.ColumnScope +import androidx.compose.foundation.layout.IntrinsicSize +import androidx.compose.foundation.layout.PaddingValues +import androidx.compose.foundation.layout.Row +import androidx.compose.foundation.layout.Spacer +import androidx.compose.foundation.layout.WindowInsets +import androidx.compose.foundation.layout.WindowInsetsSides +import androidx.compose.foundation.layout.fillMaxHeight +import androidx.compose.foundation.layout.fillMaxSize +import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.height +import androidx.compose.foundation.layout.heightIn +import androidx.compose.foundation.layout.imePadding +import androidx.compose.foundation.layout.navigationBarsPadding +import androidx.compose.foundation.layout.only +import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.layout.safeDrawing +import androidx.compose.foundation.layout.size +import androidx.compose.foundation.layout.width +import androidx.compose.foundation.layout.windowInsetsPadding +import androidx.compose.foundation.rememberScrollState +import androidx.compose.foundation.shape.RoundedCornerShape +import androidx.compose.foundation.shape.CircleShape +import androidx.compose.foundation.text.KeyboardOptions +import androidx.compose.foundation.verticalScroll +import androidx.compose.material3.AlertDialog +import androidx.compose.material3.OutlinedTextFieldDefaults +import androidx.compose.material3.Button +import androidx.compose.material3.ButtonDefaults +import androidx.compose.material3.HorizontalDivider +import androidx.compose.material3.Icon +import androidx.compose.material3.IconButton +import androidx.compose.material3.OutlinedTextField +import androidx.compose.material3.Surface +import androidx.compose.material3.Switch +import androidx.compose.material3.SwitchDefaults +import androidx.compose.material3.Text +import androidx.compose.material3.TextButton +import androidx.compose.material.icons.Icons +import androidx.compose.material.icons.automirrored.filled.ArrowBack +import androidx.compose.material.icons.filled.ExpandLess +import androidx.compose.material.icons.filled.ExpandMore +import androidx.compose.runtime.Composable +import androidx.compose.runtime.DisposableEffect +import androidx.compose.runtime.collectAsState +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.saveable.rememberSaveable +import androidx.compose.runtime.setValue +import androidx.compose.ui.Alignment +import androidx.compose.ui.Modifier +import androidx.compose.ui.graphics.Brush +import androidx.compose.ui.graphics.Color +import androidx.compose.ui.text.TextStyle +import androidx.compose.ui.text.font.Font +import androidx.compose.ui.text.font.FontFamily +import androidx.compose.ui.text.font.FontWeight +import androidx.compose.ui.text.input.KeyboardType +import androidx.compose.ui.text.style.TextOverflow +import androidx.compose.ui.unit.dp +import androidx.compose.ui.unit.sp +import androidx.core.content.ContextCompat +import androidx.lifecycle.Lifecycle +import androidx.lifecycle.LifecycleEventObserver +import androidx.lifecycle.compose.LocalLifecycleOwner +import ai.openclaw.android.LocationMode +import ai.openclaw.android.MainViewModel +import ai.openclaw.android.R +import ai.openclaw.android.node.DeviceNotificationListenerService +import com.journeyapps.barcodescanner.ScanContract +import com.journeyapps.barcodescanner.ScanOptions + +private enum class OnboardingStep(val index: Int, val label: String) { + Welcome(1, "Welcome"), + Gateway(2, "Gateway"), + Permissions(3, "Permissions"), + FinalCheck(4, "Connect"), +} + +private enum class GatewayInputMode { + SetupCode, + Manual, +} + +private enum class PermissionToggle { + Discovery, + Location, + Notifications, + Microphone, + Camera, + Photos, + Contacts, + Calendar, + Motion, + Sms, +} + +private enum class SpecialAccessToggle { + NotificationListener, + AppUpdates, +} + +private val onboardingBackgroundGradient = + listOf( + Color(0xFFFFFFFF), + Color(0xFFF7F8FA), + Color(0xFFEFF1F5), + ) +private val onboardingSurface = Color(0xFFF6F7FA) +private val onboardingBorder = Color(0xFFE5E7EC) +private val onboardingBorderStrong = Color(0xFFD6DAE2) +private val onboardingText = Color(0xFF17181C) +private val onboardingTextSecondary = Color(0xFF4D5563) +private val onboardingTextTertiary = Color(0xFF8A92A2) +private val onboardingAccent = Color(0xFF1D5DD8) +private val onboardingAccentSoft = Color(0xFFECF3FF) +private val onboardingSuccess = Color(0xFF2F8C5A) +private val onboardingWarning = Color(0xFFC8841A) +private val onboardingCommandBg = Color(0xFF15171B) +private val onboardingCommandBorder = Color(0xFF2B2E35) +private val onboardingCommandAccent = Color(0xFF3FC97A) +private val onboardingCommandText = Color(0xFFE8EAEE) + +private val onboardingFontFamily = + FontFamily( + Font(resId = R.font.manrope_400_regular, weight = FontWeight.Normal), + Font(resId = R.font.manrope_500_medium, weight = FontWeight.Medium), + Font(resId = R.font.manrope_600_semibold, weight = FontWeight.SemiBold), + Font(resId = R.font.manrope_700_bold, weight = FontWeight.Bold), + ) + +private val onboardingDisplayStyle = + TextStyle( + fontFamily = onboardingFontFamily, + fontWeight = FontWeight.Bold, + fontSize = 34.sp, + lineHeight = 40.sp, + letterSpacing = (-0.8).sp, + ) + +private val onboardingTitle1Style = + TextStyle( + fontFamily = onboardingFontFamily, + fontWeight = FontWeight.SemiBold, + fontSize = 24.sp, + lineHeight = 30.sp, + letterSpacing = (-0.5).sp, + ) + +private val onboardingHeadlineStyle = + TextStyle( + fontFamily = onboardingFontFamily, + fontWeight = FontWeight.SemiBold, + fontSize = 16.sp, + lineHeight = 22.sp, + letterSpacing = (-0.1).sp, + ) + +private val onboardingBodyStyle = + TextStyle( + fontFamily = onboardingFontFamily, + fontWeight = FontWeight.Medium, + fontSize = 15.sp, + lineHeight = 22.sp, + ) + +private val onboardingCalloutStyle = + TextStyle( + fontFamily = onboardingFontFamily, + fontWeight = FontWeight.Medium, + fontSize = 14.sp, + lineHeight = 20.sp, + ) + +private val onboardingCaption1Style = + TextStyle( + fontFamily = onboardingFontFamily, + fontWeight = FontWeight.Medium, + fontSize = 12.sp, + lineHeight = 16.sp, + letterSpacing = 0.2.sp, + ) + +private val onboardingCaption2Style = + TextStyle( + fontFamily = onboardingFontFamily, + fontWeight = FontWeight.Medium, + fontSize = 11.sp, + lineHeight = 14.sp, + letterSpacing = 0.4.sp, + ) + +@Composable +fun OnboardingFlow(viewModel: MainViewModel, modifier: Modifier = Modifier) { + val context = androidx.compose.ui.platform.LocalContext.current + val statusText by viewModel.statusText.collectAsState() + val isConnected by viewModel.isConnected.collectAsState() + val serverName by viewModel.serverName.collectAsState() + val remoteAddress by viewModel.remoteAddress.collectAsState() + val persistedGatewayToken by viewModel.gatewayToken.collectAsState() + val pendingTrust by viewModel.pendingGatewayTrust.collectAsState() + + var step by rememberSaveable { mutableStateOf(OnboardingStep.Welcome) } + var setupCode by rememberSaveable { mutableStateOf("") } + var gatewayUrl by rememberSaveable { mutableStateOf("") } + var gatewayPassword by rememberSaveable { mutableStateOf("") } + var gatewayInputMode by rememberSaveable { mutableStateOf(GatewayInputMode.SetupCode) } + var gatewayAdvancedOpen by rememberSaveable { mutableStateOf(false) } + var manualHost by rememberSaveable { mutableStateOf("10.0.2.2") } + var manualPort by rememberSaveable { mutableStateOf("18789") } + var manualTls by rememberSaveable { mutableStateOf(false) } + var gatewayError by rememberSaveable { mutableStateOf(null) } + var attemptedConnect by rememberSaveable { mutableStateOf(false) } + + val lifecycleOwner = LocalLifecycleOwner.current + + val smsAvailable = + remember(context) { + context.packageManager?.hasSystemFeature(PackageManager.FEATURE_TELEPHONY) == true + } + val motionAvailable = + remember(context) { + hasMotionCapabilities(context) + } + val motionPermissionRequired = Build.VERSION.SDK_INT >= 29 + val notificationsPermissionRequired = Build.VERSION.SDK_INT >= 33 + val discoveryPermission = + if (Build.VERSION.SDK_INT >= 33) { + Manifest.permission.NEARBY_WIFI_DEVICES + } else { + Manifest.permission.ACCESS_FINE_LOCATION + } + val photosPermission = + if (Build.VERSION.SDK_INT >= 33) { + Manifest.permission.READ_MEDIA_IMAGES + } else { + Manifest.permission.READ_EXTERNAL_STORAGE + } + + var enableDiscovery by + rememberSaveable { + mutableStateOf(isPermissionGranted(context, discoveryPermission)) + } + var enableLocation by rememberSaveable { mutableStateOf(false) } + var enableNotifications by + rememberSaveable { + mutableStateOf( + !notificationsPermissionRequired || + isPermissionGranted(context, Manifest.permission.POST_NOTIFICATIONS), + ) + } + var enableNotificationListener by + rememberSaveable { + mutableStateOf(isNotificationListenerEnabled(context)) + } + var enableAppUpdates by + rememberSaveable { + mutableStateOf(canInstallUnknownApps(context)) + } + var enableMicrophone by rememberSaveable { mutableStateOf(false) } + var enableCamera by rememberSaveable { mutableStateOf(false) } + var enablePhotos by rememberSaveable { mutableStateOf(false) } + var enableContacts by rememberSaveable { mutableStateOf(false) } + var enableCalendar by rememberSaveable { mutableStateOf(false) } + var enableMotion by + rememberSaveable { + mutableStateOf( + motionAvailable && + (!motionPermissionRequired || isPermissionGranted(context, Manifest.permission.ACTIVITY_RECOGNITION)), + ) + } + var enableSms by + rememberSaveable { + mutableStateOf(smsAvailable && isPermissionGranted(context, Manifest.permission.SEND_SMS)) + } + + var pendingPermissionToggle by remember { mutableStateOf(null) } + var pendingSpecialAccessToggle by remember { mutableStateOf(null) } + + fun setPermissionToggleEnabled(toggle: PermissionToggle, enabled: Boolean) { + when (toggle) { + PermissionToggle.Discovery -> enableDiscovery = enabled + PermissionToggle.Location -> enableLocation = enabled + PermissionToggle.Notifications -> enableNotifications = enabled + PermissionToggle.Microphone -> enableMicrophone = enabled + PermissionToggle.Camera -> enableCamera = enabled + PermissionToggle.Photos -> enablePhotos = enabled + PermissionToggle.Contacts -> enableContacts = enabled + PermissionToggle.Calendar -> enableCalendar = enabled + PermissionToggle.Motion -> enableMotion = enabled && motionAvailable + PermissionToggle.Sms -> enableSms = enabled && smsAvailable + } + } + + fun isPermissionToggleGranted(toggle: PermissionToggle): Boolean = + when (toggle) { + PermissionToggle.Discovery -> isPermissionGranted(context, discoveryPermission) + PermissionToggle.Location -> + isPermissionGranted(context, Manifest.permission.ACCESS_FINE_LOCATION) || + isPermissionGranted(context, Manifest.permission.ACCESS_COARSE_LOCATION) + PermissionToggle.Notifications -> + !notificationsPermissionRequired || + isPermissionGranted(context, Manifest.permission.POST_NOTIFICATIONS) + PermissionToggle.Microphone -> isPermissionGranted(context, Manifest.permission.RECORD_AUDIO) + PermissionToggle.Camera -> isPermissionGranted(context, Manifest.permission.CAMERA) + PermissionToggle.Photos -> isPermissionGranted(context, photosPermission) + PermissionToggle.Contacts -> + isPermissionGranted(context, Manifest.permission.READ_CONTACTS) && + isPermissionGranted(context, Manifest.permission.WRITE_CONTACTS) + PermissionToggle.Calendar -> + isPermissionGranted(context, Manifest.permission.READ_CALENDAR) && + isPermissionGranted(context, Manifest.permission.WRITE_CALENDAR) + PermissionToggle.Motion -> + !motionAvailable || + !motionPermissionRequired || + isPermissionGranted(context, Manifest.permission.ACTIVITY_RECOGNITION) + PermissionToggle.Sms -> + !smsAvailable || isPermissionGranted(context, Manifest.permission.SEND_SMS) + } + + fun setSpecialAccessToggleEnabled(toggle: SpecialAccessToggle, enabled: Boolean) { + when (toggle) { + SpecialAccessToggle.NotificationListener -> enableNotificationListener = enabled + SpecialAccessToggle.AppUpdates -> enableAppUpdates = enabled + } + } + + val enabledPermissionSummary = + remember( + enableDiscovery, + enableLocation, + enableNotifications, + enableNotificationListener, + enableAppUpdates, + enableMicrophone, + enableCamera, + enablePhotos, + enableContacts, + enableCalendar, + enableMotion, + enableSms, + smsAvailable, + motionAvailable, + ) { + val enabled = mutableListOf() + if (enableDiscovery) enabled += "Gateway discovery" + if (enableLocation) enabled += "Location" + if (enableNotifications) enabled += "Notifications" + if (enableNotificationListener) enabled += "Notification listener" + if (enableAppUpdates) enabled += "App updates" + if (enableMicrophone) enabled += "Microphone" + if (enableCamera) enabled += "Camera" + if (enablePhotos) enabled += "Photos" + if (enableContacts) enabled += "Contacts" + if (enableCalendar) enabled += "Calendar" + if (enableMotion && motionAvailable) enabled += "Motion" + if (smsAvailable && enableSms) enabled += "SMS" + if (enabled.isEmpty()) "None selected" else enabled.joinToString(", ") + } + + val proceedFromPermissions: () -> Unit = proceed@{ + var openedSpecialSetup = false + if (enableNotificationListener && !isNotificationListenerEnabled(context)) { + openNotificationListenerSettings(context) + openedSpecialSetup = true + } + if (enableAppUpdates && !canInstallUnknownApps(context)) { + openUnknownAppSourcesSettings(context) + openedSpecialSetup = true + } + if (openedSpecialSetup) { + return@proceed + } + step = OnboardingStep.FinalCheck + } + + val togglePermissionLauncher = + rememberLauncherForActivityResult(ActivityResultContracts.RequestMultiplePermissions()) { + val pendingToggle = pendingPermissionToggle ?: return@rememberLauncherForActivityResult + setPermissionToggleEnabled(pendingToggle, isPermissionToggleGranted(pendingToggle)) + pendingPermissionToggle = null + } + + val requestPermissionToggle: (PermissionToggle, Boolean, List) -> Unit = + request@{ toggle, enabled, permissions -> + if (!enabled) { + setPermissionToggleEnabled(toggle, false) + return@request + } + if (isPermissionToggleGranted(toggle)) { + setPermissionToggleEnabled(toggle, true) + return@request + } + val missing = permissions.distinct().filterNot { isPermissionGranted(context, it) } + if (missing.isEmpty()) { + setPermissionToggleEnabled(toggle, isPermissionToggleGranted(toggle)) + return@request + } + pendingPermissionToggle = toggle + togglePermissionLauncher.launch(missing.toTypedArray()) + } + + val requestSpecialAccessToggle: (SpecialAccessToggle, Boolean) -> Unit = + request@{ toggle, enabled -> + if (!enabled) { + setSpecialAccessToggleEnabled(toggle, false) + pendingSpecialAccessToggle = null + return@request + } + val grantedNow = + when (toggle) { + SpecialAccessToggle.NotificationListener -> isNotificationListenerEnabled(context) + SpecialAccessToggle.AppUpdates -> canInstallUnknownApps(context) + } + if (grantedNow) { + setSpecialAccessToggleEnabled(toggle, true) + pendingSpecialAccessToggle = null + return@request + } + pendingSpecialAccessToggle = toggle + when (toggle) { + SpecialAccessToggle.NotificationListener -> openNotificationListenerSettings(context) + SpecialAccessToggle.AppUpdates -> openUnknownAppSourcesSettings(context) + } + } + + DisposableEffect(lifecycleOwner, context, pendingSpecialAccessToggle) { + val observer = + LifecycleEventObserver { _, event -> + if (event != Lifecycle.Event.ON_RESUME) { + return@LifecycleEventObserver + } + when (pendingSpecialAccessToggle) { + SpecialAccessToggle.NotificationListener -> { + setSpecialAccessToggleEnabled( + SpecialAccessToggle.NotificationListener, + isNotificationListenerEnabled(context), + ) + pendingSpecialAccessToggle = null + } + SpecialAccessToggle.AppUpdates -> { + setSpecialAccessToggleEnabled( + SpecialAccessToggle.AppUpdates, + canInstallUnknownApps(context), + ) + pendingSpecialAccessToggle = null + } + null -> Unit + } + } + lifecycleOwner.lifecycle.addObserver(observer) + onDispose { lifecycleOwner.lifecycle.removeObserver(observer) } + } + + val qrScanLauncher = + rememberLauncherForActivityResult(ScanContract()) { result -> + val contents = result.contents?.trim().orEmpty() + if (contents.isEmpty()) { + return@rememberLauncherForActivityResult + } + val scannedSetupCode = resolveScannedSetupCode(contents) + if (scannedSetupCode == null) { + gatewayError = "QR code did not contain a valid setup code." + return@rememberLauncherForActivityResult + } + setupCode = scannedSetupCode + gatewayInputMode = GatewayInputMode.SetupCode + gatewayError = null + attemptedConnect = false + } + + if (pendingTrust != null) { + val prompt = pendingTrust!! + AlertDialog( + onDismissRequest = { viewModel.declineGatewayTrustPrompt() }, + title = { Text("Trust this gateway?") }, + text = { + Text( + "First-time TLS connection.\n\nVerify this SHA-256 fingerprint before trusting:\n${prompt.fingerprintSha256}", + ) + }, + confirmButton = { + TextButton(onClick = { viewModel.acceptGatewayTrustPrompt() }) { + Text("Trust and continue") + } + }, + dismissButton = { + TextButton(onClick = { viewModel.declineGatewayTrustPrompt() }) { + Text("Cancel") + } + }, + ) + } + + Box( + modifier = + modifier + .fillMaxSize() + .background(Brush.verticalGradient(onboardingBackgroundGradient)), + ) { + Column( + modifier = + Modifier + .fillMaxSize() + .imePadding() + .windowInsetsPadding(WindowInsets.safeDrawing.only(WindowInsetsSides.Top + WindowInsetsSides.Horizontal)) + .navigationBarsPadding() + .padding(horizontal = 20.dp, vertical = 12.dp), + verticalArrangement = Arrangement.SpaceBetween, + ) { + Column( + modifier = Modifier.weight(1f).verticalScroll(rememberScrollState()), + verticalArrangement = Arrangement.spacedBy(20.dp), + ) { + Column( + modifier = Modifier.padding(top = 12.dp), + verticalArrangement = Arrangement.spacedBy(8.dp), + ) { + Text( + "FIRST RUN", + style = onboardingCaption1Style.copy(fontWeight = FontWeight.Bold, letterSpacing = 1.5.sp), + color = onboardingAccent, + ) + Text( + "OpenClaw\nMobile Setup", + style = onboardingDisplayStyle.copy(lineHeight = 38.sp), + color = onboardingText, + ) + Text( + "Step ${step.index} of 4", + style = onboardingCaption1Style, + color = onboardingAccent, + ) + } + StepRailWrap(current = step) + + when (step) { + OnboardingStep.Welcome -> WelcomeStep() + OnboardingStep.Gateway -> + GatewayStep( + inputMode = gatewayInputMode, + advancedOpen = gatewayAdvancedOpen, + setupCode = setupCode, + manualHost = manualHost, + manualPort = manualPort, + manualTls = manualTls, + gatewayToken = persistedGatewayToken, + gatewayPassword = gatewayPassword, + gatewayError = gatewayError, + onScanQrClick = { + gatewayError = null + qrScanLauncher.launch( + ScanOptions().apply { + setDesiredBarcodeFormats(ScanOptions.QR_CODE) + setPrompt("Scan OpenClaw onboarding QR") + setBeepEnabled(false) + setOrientationLocked(false) + }, + ) + }, + onAdvancedOpenChange = { gatewayAdvancedOpen = it }, + onInputModeChange = { + gatewayInputMode = it + gatewayError = null + }, + onSetupCodeChange = { + setupCode = it + gatewayError = null + }, + onManualHostChange = { + manualHost = it + gatewayError = null + }, + onManualPortChange = { + manualPort = it + gatewayError = null + }, + onManualTlsChange = { manualTls = it }, + onTokenChange = viewModel::setGatewayToken, + onPasswordChange = { gatewayPassword = it }, + ) + OnboardingStep.Permissions -> + PermissionsStep( + enableDiscovery = enableDiscovery, + enableLocation = enableLocation, + enableNotifications = enableNotifications, + enableNotificationListener = enableNotificationListener, + enableAppUpdates = enableAppUpdates, + enableMicrophone = enableMicrophone, + enableCamera = enableCamera, + enablePhotos = enablePhotos, + enableContacts = enableContacts, + enableCalendar = enableCalendar, + enableMotion = enableMotion, + motionAvailable = motionAvailable, + motionPermissionRequired = motionPermissionRequired, + enableSms = enableSms, + smsAvailable = smsAvailable, + context = context, + onDiscoveryChange = { checked -> + requestPermissionToggle( + PermissionToggle.Discovery, + checked, + listOf(discoveryPermission), + ) + }, + onLocationChange = { checked -> + requestPermissionToggle( + PermissionToggle.Location, + checked, + listOf( + Manifest.permission.ACCESS_FINE_LOCATION, + Manifest.permission.ACCESS_COARSE_LOCATION, + ), + ) + }, + onNotificationsChange = { checked -> + if (!notificationsPermissionRequired) { + setPermissionToggleEnabled(PermissionToggle.Notifications, checked) + } else { + requestPermissionToggle( + PermissionToggle.Notifications, + checked, + listOf(Manifest.permission.POST_NOTIFICATIONS), + ) + } + }, + onNotificationListenerChange = { checked -> + requestSpecialAccessToggle(SpecialAccessToggle.NotificationListener, checked) + }, + onAppUpdatesChange = { checked -> + requestSpecialAccessToggle(SpecialAccessToggle.AppUpdates, checked) + }, + onMicrophoneChange = { checked -> + requestPermissionToggle( + PermissionToggle.Microphone, + checked, + listOf(Manifest.permission.RECORD_AUDIO), + ) + }, + onCameraChange = { checked -> + requestPermissionToggle( + PermissionToggle.Camera, + checked, + listOf(Manifest.permission.CAMERA), + ) + }, + onPhotosChange = { checked -> + requestPermissionToggle( + PermissionToggle.Photos, + checked, + listOf(photosPermission), + ) + }, + onContactsChange = { checked -> + requestPermissionToggle( + PermissionToggle.Contacts, + checked, + listOf( + Manifest.permission.READ_CONTACTS, + Manifest.permission.WRITE_CONTACTS, + ), + ) + }, + onCalendarChange = { checked -> + requestPermissionToggle( + PermissionToggle.Calendar, + checked, + listOf( + Manifest.permission.READ_CALENDAR, + Manifest.permission.WRITE_CALENDAR, + ), + ) + }, + onMotionChange = { checked -> + if (!motionAvailable) { + setPermissionToggleEnabled(PermissionToggle.Motion, false) + } else if (!motionPermissionRequired) { + setPermissionToggleEnabled(PermissionToggle.Motion, checked) + } else { + requestPermissionToggle( + PermissionToggle.Motion, + checked, + listOf(Manifest.permission.ACTIVITY_RECOGNITION), + ) + } + }, + onSmsChange = { checked -> + if (!smsAvailable) { + setPermissionToggleEnabled(PermissionToggle.Sms, false) + } else { + requestPermissionToggle( + PermissionToggle.Sms, + checked, + listOf(Manifest.permission.SEND_SMS), + ) + } + }, + ) + OnboardingStep.FinalCheck -> + FinalStep( + parsedGateway = parseGatewayEndpoint(gatewayUrl), + statusText = statusText, + isConnected = isConnected, + serverName = serverName, + remoteAddress = remoteAddress, + attemptedConnect = attemptedConnect, + enabledPermissions = enabledPermissionSummary, + methodLabel = if (gatewayInputMode == GatewayInputMode.SetupCode) "QR / Setup Code" else "Manual", + ) + } + } + + Spacer(Modifier.height(12.dp)) + + Row( + modifier = Modifier.fillMaxWidth().padding(vertical = 8.dp), + horizontalArrangement = Arrangement.spacedBy(10.dp), + verticalAlignment = Alignment.CenterVertically, + ) { + val backEnabled = step != OnboardingStep.Welcome + Surface( + modifier = Modifier.size(52.dp), + shape = RoundedCornerShape(14.dp), + color = onboardingSurface, + border = androidx.compose.foundation.BorderStroke(1.dp, if (backEnabled) onboardingBorderStrong else onboardingBorder), + ) { + IconButton( + onClick = { + step = + when (step) { + OnboardingStep.Welcome -> OnboardingStep.Welcome + OnboardingStep.Gateway -> OnboardingStep.Welcome + OnboardingStep.Permissions -> OnboardingStep.Gateway + OnboardingStep.FinalCheck -> OnboardingStep.Permissions + } + }, + enabled = backEnabled, + ) { + Icon( + Icons.AutoMirrored.Filled.ArrowBack, + contentDescription = "Back", + tint = if (backEnabled) onboardingTextSecondary else onboardingTextTertiary, + ) + } + } + + when (step) { + OnboardingStep.Welcome -> { + Button( + onClick = { step = OnboardingStep.Gateway }, + modifier = Modifier.weight(1f).height(52.dp), + shape = RoundedCornerShape(14.dp), + colors = + ButtonDefaults.buttonColors( + containerColor = onboardingAccent, + contentColor = Color.White, + disabledContainerColor = onboardingAccent.copy(alpha = 0.45f), + disabledContentColor = Color.White, + ), + ) { + Text("Next", style = onboardingHeadlineStyle.copy(fontWeight = FontWeight.Bold)) + } + } + OnboardingStep.Gateway -> { + Button( + onClick = { + if (gatewayInputMode == GatewayInputMode.SetupCode) { + val parsedSetup = decodeGatewaySetupCode(setupCode) + if (parsedSetup == null) { + gatewayError = "Scan QR code first, or use Advanced setup." + return@Button + } + val parsedGateway = parseGatewayEndpoint(parsedSetup.url) + if (parsedGateway == null) { + gatewayError = "Setup code has invalid gateway URL." + return@Button + } + gatewayUrl = parsedSetup.url + parsedSetup.token?.let { viewModel.setGatewayToken(it) } + gatewayPassword = parsedSetup.password.orEmpty() + } else { + val manualUrl = composeGatewayManualUrl(manualHost, manualPort, manualTls) + val parsedGateway = manualUrl?.let(::parseGatewayEndpoint) + if (parsedGateway == null) { + gatewayError = "Manual endpoint is invalid." + return@Button + } + gatewayUrl = parsedGateway.displayUrl + } + step = OnboardingStep.Permissions + }, + modifier = Modifier.weight(1f).height(52.dp), + shape = RoundedCornerShape(14.dp), + colors = + ButtonDefaults.buttonColors( + containerColor = onboardingAccent, + contentColor = Color.White, + disabledContainerColor = onboardingAccent.copy(alpha = 0.45f), + disabledContentColor = Color.White, + ), + ) { + Text("Next", style = onboardingHeadlineStyle.copy(fontWeight = FontWeight.Bold)) + } + } + OnboardingStep.Permissions -> { + Button( + onClick = { + viewModel.setCameraEnabled(enableCamera) + viewModel.setLocationMode(if (enableLocation) LocationMode.WhileUsing else LocationMode.Off) + proceedFromPermissions() + }, + modifier = Modifier.weight(1f).height(52.dp), + shape = RoundedCornerShape(14.dp), + colors = + ButtonDefaults.buttonColors( + containerColor = onboardingAccent, + contentColor = Color.White, + disabledContainerColor = onboardingAccent.copy(alpha = 0.45f), + disabledContentColor = Color.White, + ), + ) { + Text("Next", style = onboardingHeadlineStyle.copy(fontWeight = FontWeight.Bold)) + } + } + OnboardingStep.FinalCheck -> { + if (isConnected) { + Button( + onClick = { viewModel.setOnboardingCompleted(true) }, + modifier = Modifier.weight(1f).height(52.dp), + shape = RoundedCornerShape(14.dp), + colors = + ButtonDefaults.buttonColors( + containerColor = onboardingAccent, + contentColor = Color.White, + disabledContainerColor = onboardingAccent.copy(alpha = 0.45f), + disabledContentColor = Color.White, + ), + ) { + Text("Finish", style = onboardingHeadlineStyle.copy(fontWeight = FontWeight.Bold)) + } + } else { + Button( + onClick = { + val parsed = parseGatewayEndpoint(gatewayUrl) + if (parsed == null) { + step = OnboardingStep.Gateway + gatewayError = "Invalid gateway URL." + return@Button + } + val token = persistedGatewayToken.trim() + val password = gatewayPassword.trim() + attemptedConnect = true + viewModel.setManualEnabled(true) + viewModel.setManualHost(parsed.host) + viewModel.setManualPort(parsed.port) + viewModel.setManualTls(parsed.tls) + if (token.isNotEmpty()) { + viewModel.setGatewayToken(token) + } + viewModel.setGatewayPassword(password) + viewModel.connectManual() + }, + modifier = Modifier.weight(1f).height(52.dp), + shape = RoundedCornerShape(14.dp), + colors = + ButtonDefaults.buttonColors( + containerColor = onboardingAccent, + contentColor = Color.White, + disabledContainerColor = onboardingAccent.copy(alpha = 0.45f), + disabledContentColor = Color.White, + ), + ) { + Text("Connect", style = onboardingHeadlineStyle.copy(fontWeight = FontWeight.Bold)) + } + } + } + } + } + } + } +} + +@Composable +private fun StepRailWrap(current: OnboardingStep) { + Column(verticalArrangement = Arrangement.spacedBy(10.dp)) { + HorizontalDivider(color = onboardingBorder) + StepRail(current = current) + HorizontalDivider(color = onboardingBorder) + } +} + +@Composable +private fun StepRail(current: OnboardingStep) { + val steps = OnboardingStep.entries + Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.spacedBy(4.dp)) { + steps.forEach { step -> + val complete = step.index < current.index + val active = step.index == current.index + Column( + modifier = Modifier.weight(1f), + verticalArrangement = Arrangement.spacedBy(4.dp), + horizontalAlignment = Alignment.CenterHorizontally, + ) { + Box( + modifier = + Modifier + .fillMaxWidth() + .height(5.dp) + .background( + color = + when { + complete -> onboardingSuccess + active -> onboardingAccent + else -> onboardingBorder + }, + shape = RoundedCornerShape(999.dp), + ), + ) + Text( + text = step.label, + style = onboardingCaption2Style.copy(fontWeight = if (active) FontWeight.Bold else FontWeight.SemiBold), + color = if (active) onboardingAccent else onboardingTextSecondary, + maxLines = 1, + overflow = TextOverflow.Ellipsis, + ) + } + } + } +} + +@Composable +private fun WelcomeStep() { + StepShell(title = "What You Get") { + Bullet("Control the gateway and operator chat from one mobile surface.") + Bullet("Connect with setup code and recover pairing with CLI commands.") + Bullet("Enable only the permissions and capabilities you want.") + Bullet("Finish with a real connection check before entering the app.") + } +} + +@Composable +private fun GatewayStep( + inputMode: GatewayInputMode, + advancedOpen: Boolean, + setupCode: String, + manualHost: String, + manualPort: String, + manualTls: Boolean, + gatewayToken: String, + gatewayPassword: String, + gatewayError: String?, + onScanQrClick: () -> Unit, + onAdvancedOpenChange: (Boolean) -> Unit, + onInputModeChange: (GatewayInputMode) -> Unit, + onSetupCodeChange: (String) -> Unit, + onManualHostChange: (String) -> Unit, + onManualPortChange: (String) -> Unit, + onManualTlsChange: (Boolean) -> Unit, + onTokenChange: (String) -> Unit, + onPasswordChange: (String) -> Unit, +) { + val resolvedEndpoint = remember(setupCode) { decodeGatewaySetupCode(setupCode)?.url?.let { parseGatewayEndpoint(it)?.displayUrl } } + val manualResolvedEndpoint = remember(manualHost, manualPort, manualTls) { composeGatewayManualUrl(manualHost, manualPort, manualTls)?.let { parseGatewayEndpoint(it)?.displayUrl } } + + StepShell(title = "Gateway Connection") { + GuideBlock(title = "Scan onboarding QR") { + Text("Run these on the gateway host:", style = onboardingCalloutStyle, color = onboardingTextSecondary) + CommandBlock("openclaw qr") + Text("Then scan with this device.", style = onboardingCalloutStyle, color = onboardingTextSecondary) + } + Button( + onClick = onScanQrClick, + modifier = Modifier.fillMaxWidth().height(48.dp), + shape = RoundedCornerShape(12.dp), + colors = + ButtonDefaults.buttonColors( + containerColor = onboardingAccent, + contentColor = Color.White, + ), + ) { + Text("Scan QR code", style = onboardingHeadlineStyle.copy(fontWeight = FontWeight.Bold)) + } + if (!resolvedEndpoint.isNullOrBlank()) { + Text("QR captured. Review endpoint below.", style = onboardingCalloutStyle, color = onboardingSuccess) + ResolvedEndpoint(endpoint = resolvedEndpoint) + } + + Surface( + modifier = Modifier.fillMaxWidth(), + shape = RoundedCornerShape(12.dp), + color = onboardingSurface, + border = androidx.compose.foundation.BorderStroke(1.dp, onboardingBorderStrong), + onClick = { onAdvancedOpenChange(!advancedOpen) }, + ) { + Row( + modifier = Modifier.fillMaxWidth().padding(horizontal = 12.dp, vertical = 10.dp), + verticalAlignment = Alignment.CenterVertically, + horizontalArrangement = Arrangement.SpaceBetween, + ) { + Column(verticalArrangement = Arrangement.spacedBy(2.dp)) { + Text("Advanced setup", style = onboardingHeadlineStyle, color = onboardingText) + Text("Paste setup code or enter host/port manually.", style = onboardingCaption1Style, color = onboardingTextSecondary) + } + Icon( + imageVector = if (advancedOpen) Icons.Default.ExpandLess else Icons.Default.ExpandMore, + contentDescription = if (advancedOpen) "Collapse advanced setup" else "Expand advanced setup", + tint = onboardingTextSecondary, + ) + } + } + + AnimatedVisibility(visible = advancedOpen) { + Column(verticalArrangement = Arrangement.spacedBy(12.dp)) { + GuideBlock(title = "Manual setup commands") { + Text("Run these on the gateway host:", style = onboardingCalloutStyle, color = onboardingTextSecondary) + CommandBlock("openclaw qr --setup-code-only") + CommandBlock("openclaw qr --json") + Text( + "`--json` prints `setupCode` and `gatewayUrl`.", + style = onboardingCalloutStyle, + color = onboardingTextSecondary, + ) + Text( + "Auto URL discovery is not wired yet. Android emulator uses `10.0.2.2`; real devices need LAN/Tailscale host.", + style = onboardingCalloutStyle, + color = onboardingTextSecondary, + ) + } + GatewayModeToggle(inputMode = inputMode, onInputModeChange = onInputModeChange) + + if (inputMode == GatewayInputMode.SetupCode) { + Text("SETUP CODE", style = onboardingCaption1Style.copy(letterSpacing = 0.9.sp), color = onboardingTextSecondary) + OutlinedTextField( + value = setupCode, + onValueChange = onSetupCodeChange, + placeholder = { Text("Paste code from `openclaw qr --setup-code-only`", color = onboardingTextTertiary, style = onboardingBodyStyle) }, + modifier = Modifier.fillMaxWidth(), + minLines = 3, + maxLines = 5, + keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Ascii), + textStyle = onboardingBodyStyle.copy(fontFamily = FontFamily.Monospace, color = onboardingText), + shape = RoundedCornerShape(14.dp), + colors = + OutlinedTextFieldDefaults.colors( + focusedContainerColor = onboardingSurface, + unfocusedContainerColor = onboardingSurface, + focusedBorderColor = onboardingAccent, + unfocusedBorderColor = onboardingBorder, + focusedTextColor = onboardingText, + unfocusedTextColor = onboardingText, + cursorColor = onboardingAccent, + ), + ) + if (!resolvedEndpoint.isNullOrBlank()) { + ResolvedEndpoint(endpoint = resolvedEndpoint) + } + } else { + Row(horizontalArrangement = Arrangement.spacedBy(8.dp)) { + QuickFillChip(label = "Android Emulator", onClick = { + onManualHostChange("10.0.2.2") + onManualPortChange("18789") + onManualTlsChange(false) + }) + QuickFillChip(label = "Localhost", onClick = { + onManualHostChange("127.0.0.1") + onManualPortChange("18789") + onManualTlsChange(false) + }) + } + + Text("HOST", style = onboardingCaption1Style.copy(letterSpacing = 0.9.sp), color = onboardingTextSecondary) + OutlinedTextField( + value = manualHost, + onValueChange = onManualHostChange, + placeholder = { Text("10.0.2.2", color = onboardingTextTertiary, style = onboardingBodyStyle) }, + modifier = Modifier.fillMaxWidth(), + singleLine = true, + keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Uri), + textStyle = onboardingBodyStyle.copy(color = onboardingText), + shape = RoundedCornerShape(14.dp), + colors = + OutlinedTextFieldDefaults.colors( + focusedContainerColor = onboardingSurface, + unfocusedContainerColor = onboardingSurface, + focusedBorderColor = onboardingAccent, + unfocusedBorderColor = onboardingBorder, + focusedTextColor = onboardingText, + unfocusedTextColor = onboardingText, + cursorColor = onboardingAccent, + ), + ) + + Text("PORT", style = onboardingCaption1Style.copy(letterSpacing = 0.9.sp), color = onboardingTextSecondary) + OutlinedTextField( + value = manualPort, + onValueChange = onManualPortChange, + placeholder = { Text("18789", color = onboardingTextTertiary, style = onboardingBodyStyle) }, + modifier = Modifier.fillMaxWidth(), + singleLine = true, + keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Number), + textStyle = onboardingBodyStyle.copy(fontFamily = FontFamily.Monospace, color = onboardingText), + shape = RoundedCornerShape(14.dp), + colors = + OutlinedTextFieldDefaults.colors( + focusedContainerColor = onboardingSurface, + unfocusedContainerColor = onboardingSurface, + focusedBorderColor = onboardingAccent, + unfocusedBorderColor = onboardingBorder, + focusedTextColor = onboardingText, + unfocusedTextColor = onboardingText, + cursorColor = onboardingAccent, + ), + ) + + Row( + modifier = Modifier.fillMaxWidth(), + verticalAlignment = Alignment.CenterVertically, + horizontalArrangement = Arrangement.SpaceBetween, + ) { + Column(verticalArrangement = Arrangement.spacedBy(2.dp)) { + Text("Use TLS", style = onboardingHeadlineStyle, color = onboardingText) + Text("Switch to secure websocket (`wss`).", style = onboardingCalloutStyle.copy(lineHeight = 18.sp), color = onboardingTextSecondary) + } + Switch( + checked = manualTls, + onCheckedChange = onManualTlsChange, + colors = + SwitchDefaults.colors( + checkedTrackColor = onboardingAccent, + uncheckedTrackColor = onboardingBorderStrong, + checkedThumbColor = Color.White, + uncheckedThumbColor = Color.White, + ), + ) + } + + Text("TOKEN (OPTIONAL)", style = onboardingCaption1Style.copy(letterSpacing = 0.9.sp), color = onboardingTextSecondary) + OutlinedTextField( + value = gatewayToken, + onValueChange = onTokenChange, + placeholder = { Text("token", color = onboardingTextTertiary, style = onboardingBodyStyle) }, + modifier = Modifier.fillMaxWidth(), + singleLine = true, + keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Ascii), + textStyle = onboardingBodyStyle.copy(color = onboardingText), + shape = RoundedCornerShape(14.dp), + colors = + OutlinedTextFieldDefaults.colors( + focusedContainerColor = onboardingSurface, + unfocusedContainerColor = onboardingSurface, + focusedBorderColor = onboardingAccent, + unfocusedBorderColor = onboardingBorder, + focusedTextColor = onboardingText, + unfocusedTextColor = onboardingText, + cursorColor = onboardingAccent, + ), + ) + + Text("PASSWORD (OPTIONAL)", style = onboardingCaption1Style.copy(letterSpacing = 0.9.sp), color = onboardingTextSecondary) + OutlinedTextField( + value = gatewayPassword, + onValueChange = onPasswordChange, + placeholder = { Text("password", color = onboardingTextTertiary, style = onboardingBodyStyle) }, + modifier = Modifier.fillMaxWidth(), + singleLine = true, + keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Ascii), + textStyle = onboardingBodyStyle.copy(color = onboardingText), + shape = RoundedCornerShape(14.dp), + colors = + OutlinedTextFieldDefaults.colors( + focusedContainerColor = onboardingSurface, + unfocusedContainerColor = onboardingSurface, + focusedBorderColor = onboardingAccent, + unfocusedBorderColor = onboardingBorder, + focusedTextColor = onboardingText, + unfocusedTextColor = onboardingText, + cursorColor = onboardingAccent, + ), + ) + + if (!manualResolvedEndpoint.isNullOrBlank()) { + ResolvedEndpoint(endpoint = manualResolvedEndpoint) + } + } + } + } + + if (!gatewayError.isNullOrBlank()) { + Text(gatewayError, color = onboardingWarning, style = onboardingCaption1Style) + } + } +} + +@Composable +private fun GuideBlock( + title: String, + content: @Composable ColumnScope.() -> Unit, +) { + Row(modifier = Modifier.fillMaxWidth().height(IntrinsicSize.Min), horizontalArrangement = Arrangement.spacedBy(12.dp)) { + Box(modifier = Modifier.width(2.dp).fillMaxHeight().background(onboardingAccent.copy(alpha = 0.4f))) + Column(modifier = Modifier.weight(1f), verticalArrangement = Arrangement.spacedBy(8.dp)) { + Text(title, style = onboardingHeadlineStyle, color = onboardingText) + content() + } + } +} + +@Composable +private fun GatewayModeToggle( + inputMode: GatewayInputMode, + onInputModeChange: (GatewayInputMode) -> Unit, +) { + Row(horizontalArrangement = Arrangement.spacedBy(8.dp), modifier = Modifier.fillMaxWidth()) { + GatewayModeChip( + label = "Setup Code", + active = inputMode == GatewayInputMode.SetupCode, + onClick = { onInputModeChange(GatewayInputMode.SetupCode) }, + modifier = Modifier.weight(1f), + ) + GatewayModeChip( + label = "Manual", + active = inputMode == GatewayInputMode.Manual, + onClick = { onInputModeChange(GatewayInputMode.Manual) }, + modifier = Modifier.weight(1f), + ) + } +} + +@Composable +private fun GatewayModeChip( + label: String, + active: Boolean, + onClick: () -> Unit, + modifier: Modifier = Modifier, +) { + Button( + onClick = onClick, + modifier = modifier.height(40.dp), + shape = RoundedCornerShape(12.dp), + contentPadding = PaddingValues(horizontal = 10.dp, vertical = 8.dp), + colors = + ButtonDefaults.buttonColors( + containerColor = if (active) onboardingAccent else onboardingSurface, + contentColor = if (active) Color.White else onboardingText, + ), + border = androidx.compose.foundation.BorderStroke(1.dp, if (active) Color(0xFF184DAF) else onboardingBorderStrong), + ) { + Text( + text = label, + style = onboardingCaption1Style.copy(fontWeight = FontWeight.Bold), + ) + } +} + +@Composable +private fun QuickFillChip( + label: String, + onClick: () -> Unit, +) { + TextButton( + onClick = onClick, + shape = RoundedCornerShape(999.dp), + contentPadding = PaddingValues(horizontal = 12.dp, vertical = 7.dp), + colors = + ButtonDefaults.textButtonColors( + containerColor = onboardingAccentSoft, + contentColor = onboardingAccent, + ), + ) { + Text(label, style = onboardingCaption1Style.copy(fontWeight = FontWeight.SemiBold)) + } +} + +@Composable +private fun ResolvedEndpoint(endpoint: String) { + Column(verticalArrangement = Arrangement.spacedBy(4.dp)) { + HorizontalDivider(color = onboardingBorder) + Text( + "RESOLVED ENDPOINT", + style = onboardingCaption2Style.copy(fontWeight = FontWeight.SemiBold, letterSpacing = 0.7.sp), + color = onboardingTextSecondary, + ) + Text( + endpoint, + style = onboardingCalloutStyle.copy(fontFamily = FontFamily.Monospace), + color = onboardingText, + ) + HorizontalDivider(color = onboardingBorder) + } +} + +@Composable +private fun StepShell( + title: String, + content: @Composable ColumnScope.() -> Unit, +) { + Column(verticalArrangement = Arrangement.spacedBy(0.dp)) { + HorizontalDivider(color = onboardingBorder) + Column(modifier = Modifier.padding(vertical = 14.dp), verticalArrangement = Arrangement.spacedBy(12.dp)) { + Text(title, style = onboardingTitle1Style, color = onboardingText) + content() + } + HorizontalDivider(color = onboardingBorder) + } +} + +@Composable +private fun InlineDivider() { + HorizontalDivider(color = onboardingBorder) +} + +@Composable +private fun PermissionsStep( + enableDiscovery: Boolean, + enableLocation: Boolean, + enableNotifications: Boolean, + enableNotificationListener: Boolean, + enableAppUpdates: Boolean, + enableMicrophone: Boolean, + enableCamera: Boolean, + enablePhotos: Boolean, + enableContacts: Boolean, + enableCalendar: Boolean, + enableMotion: Boolean, + motionAvailable: Boolean, + motionPermissionRequired: Boolean, + enableSms: Boolean, + smsAvailable: Boolean, + context: Context, + onDiscoveryChange: (Boolean) -> Unit, + onLocationChange: (Boolean) -> Unit, + onNotificationsChange: (Boolean) -> Unit, + onNotificationListenerChange: (Boolean) -> Unit, + onAppUpdatesChange: (Boolean) -> Unit, + onMicrophoneChange: (Boolean) -> Unit, + onCameraChange: (Boolean) -> Unit, + onPhotosChange: (Boolean) -> Unit, + onContactsChange: (Boolean) -> Unit, + onCalendarChange: (Boolean) -> Unit, + onMotionChange: (Boolean) -> Unit, + onSmsChange: (Boolean) -> Unit, +) { + val discoveryPermission = if (Build.VERSION.SDK_INT >= 33) Manifest.permission.NEARBY_WIFI_DEVICES else Manifest.permission.ACCESS_FINE_LOCATION + val locationGranted = + isPermissionGranted(context, Manifest.permission.ACCESS_FINE_LOCATION) || + isPermissionGranted(context, Manifest.permission.ACCESS_COARSE_LOCATION) + val photosPermission = + if (Build.VERSION.SDK_INT >= 33) { + Manifest.permission.READ_MEDIA_IMAGES + } else { + Manifest.permission.READ_EXTERNAL_STORAGE + } + val contactsGranted = + isPermissionGranted(context, Manifest.permission.READ_CONTACTS) && + isPermissionGranted(context, Manifest.permission.WRITE_CONTACTS) + val calendarGranted = + isPermissionGranted(context, Manifest.permission.READ_CALENDAR) && + isPermissionGranted(context, Manifest.permission.WRITE_CALENDAR) + val motionGranted = + if (!motionAvailable) { + false + } else if (!motionPermissionRequired) { + true + } else { + isPermissionGranted(context, Manifest.permission.ACTIVITY_RECOGNITION) + } + val notificationListenerGranted = isNotificationListenerEnabled(context) + val appUpdatesGranted = canInstallUnknownApps(context) + + StepShell(title = "Permissions") { + Text( + "Enable only what you need now. You can change everything later in Settings.", + style = onboardingCalloutStyle, + color = onboardingTextSecondary, + ) + PermissionToggleRow( + title = "Gateway discovery", + subtitle = if (Build.VERSION.SDK_INT >= 33) "Nearby devices" else "Location (for NSD)", + checked = enableDiscovery, + granted = isPermissionGranted(context, discoveryPermission), + onCheckedChange = onDiscoveryChange, + ) + InlineDivider() + PermissionToggleRow( + title = "Location", + subtitle = "location.get (while app is open unless set to Always later)", + checked = enableLocation, + granted = locationGranted, + onCheckedChange = onLocationChange, + ) + InlineDivider() + if (Build.VERSION.SDK_INT >= 33) { + PermissionToggleRow( + title = "Notifications", + subtitle = "system.notify and foreground alerts", + checked = enableNotifications, + granted = isPermissionGranted(context, Manifest.permission.POST_NOTIFICATIONS), + onCheckedChange = onNotificationsChange, + ) + InlineDivider() + } + PermissionToggleRow( + title = "Notification listener", + subtitle = "notifications.list and notifications.actions (opens Android Settings)", + checked = enableNotificationListener, + granted = notificationListenerGranted, + onCheckedChange = onNotificationListenerChange, + ) + InlineDivider() + PermissionToggleRow( + title = "App updates", + subtitle = "app.update install confirmation (opens Android Settings)", + checked = enableAppUpdates, + granted = appUpdatesGranted, + onCheckedChange = onAppUpdatesChange, + ) + InlineDivider() + PermissionToggleRow( + title = "Microphone", + subtitle = "Voice tab transcription", + checked = enableMicrophone, + granted = isPermissionGranted(context, Manifest.permission.RECORD_AUDIO), + onCheckedChange = onMicrophoneChange, + ) + InlineDivider() + PermissionToggleRow( + title = "Camera", + subtitle = "camera.snap and camera.clip", + checked = enableCamera, + granted = isPermissionGranted(context, Manifest.permission.CAMERA), + onCheckedChange = onCameraChange, + ) + InlineDivider() + PermissionToggleRow( + title = "Photos", + subtitle = "photos.latest", + checked = enablePhotos, + granted = isPermissionGranted(context, photosPermission), + onCheckedChange = onPhotosChange, + ) + InlineDivider() + PermissionToggleRow( + title = "Contacts", + subtitle = "contacts.search and contacts.add", + checked = enableContacts, + granted = contactsGranted, + onCheckedChange = onContactsChange, + ) + InlineDivider() + PermissionToggleRow( + title = "Calendar", + subtitle = "calendar.events and calendar.add", + checked = enableCalendar, + granted = calendarGranted, + onCheckedChange = onCalendarChange, + ) + InlineDivider() + PermissionToggleRow( + title = "Motion", + subtitle = "motion.activity and motion.pedometer", + checked = enableMotion, + granted = motionGranted, + onCheckedChange = onMotionChange, + enabled = motionAvailable, + statusOverride = if (!motionAvailable) "Unavailable on this device" else null, + ) + if (smsAvailable) { + InlineDivider() + PermissionToggleRow( + title = "SMS", + subtitle = "Allow gateway-triggered SMS sending", + checked = enableSms, + granted = isPermissionGranted(context, Manifest.permission.SEND_SMS), + onCheckedChange = onSmsChange, + ) + } + Text("All settings can be changed later in Settings.", style = onboardingCalloutStyle, color = onboardingTextSecondary) + } +} + +@Composable +private fun PermissionToggleRow( + title: String, + subtitle: String, + checked: Boolean, + granted: Boolean, + enabled: Boolean = true, + statusOverride: String? = null, + onCheckedChange: (Boolean) -> Unit, +) { + Row( + modifier = Modifier.fillMaxWidth().heightIn(min = 50.dp), + verticalAlignment = Alignment.CenterVertically, + horizontalArrangement = Arrangement.spacedBy(12.dp), + ) { + Column(modifier = Modifier.weight(1f), verticalArrangement = Arrangement.spacedBy(2.dp)) { + Text(title, style = onboardingHeadlineStyle, color = onboardingText) + Text(subtitle, style = onboardingCalloutStyle.copy(lineHeight = 18.sp), color = onboardingTextSecondary) + Text( + statusOverride ?: if (granted) "Granted" else "Not granted", + style = onboardingCaption1Style, + color = if (granted) onboardingSuccess else onboardingTextSecondary, + ) + } + Switch( + checked = checked, + onCheckedChange = onCheckedChange, + enabled = enabled, + colors = + SwitchDefaults.colors( + checkedTrackColor = onboardingAccent, + uncheckedTrackColor = onboardingBorderStrong, + checkedThumbColor = Color.White, + uncheckedThumbColor = Color.White, + ), + ) + } +} + +@Composable +private fun FinalStep( + parsedGateway: GatewayEndpointConfig?, + statusText: String, + isConnected: Boolean, + serverName: String?, + remoteAddress: String?, + attemptedConnect: Boolean, + enabledPermissions: String, + methodLabel: String, +) { + StepShell(title = "Review") { + SummaryField(label = "Method", value = methodLabel) + SummaryField(label = "Gateway", value = parsedGateway?.displayUrl ?: "Invalid gateway URL") + SummaryField(label = "Enabled Permissions", value = enabledPermissions) + + if (!attemptedConnect) { + Text("Press Connect to verify gateway reachability and auth.", style = onboardingCalloutStyle, color = onboardingTextSecondary) + } else { + Text("Status: $statusText", style = onboardingCalloutStyle, color = if (isConnected) onboardingSuccess else onboardingTextSecondary) + if (isConnected) { + Text("Connected to ${serverName ?: remoteAddress ?: "gateway"}", style = onboardingCalloutStyle, color = onboardingSuccess) + } else { + GuideBlock(title = "Pairing Required") { + Text("Run these on the gateway host:", style = onboardingCalloutStyle, color = onboardingTextSecondary) + CommandBlock("openclaw devices list") + CommandBlock("openclaw devices approve ") + Text("Then tap Connect again.", style = onboardingCalloutStyle, color = onboardingTextSecondary) + } + } + } + } +} + +@Composable +private fun SummaryField(label: String, value: String) { + Column(verticalArrangement = Arrangement.spacedBy(4.dp)) { + Text( + label, + style = onboardingCaption2Style.copy(fontWeight = FontWeight.SemiBold, letterSpacing = 0.6.sp), + color = onboardingTextSecondary, + ) + Text(value, style = onboardingHeadlineStyle, color = onboardingText) + HorizontalDivider(color = onboardingBorder) + } +} + +@Composable +private fun CommandBlock(command: String) { + Row( + modifier = + Modifier + .fillMaxWidth() + .background(onboardingCommandBg, RoundedCornerShape(12.dp)) + .border(width = 1.dp, color = onboardingCommandBorder, shape = RoundedCornerShape(12.dp)), + ) { + Box(modifier = Modifier.width(3.dp).height(42.dp).background(onboardingCommandAccent)) + Text( + command, + modifier = Modifier.padding(horizontal = 12.dp, vertical = 10.dp), + style = onboardingCalloutStyle, + fontFamily = FontFamily.Monospace, + color = onboardingCommandText, + ) + } +} + +@Composable +private fun Bullet(text: String) { + Row(horizontalArrangement = Arrangement.spacedBy(10.dp), verticalAlignment = Alignment.Top) { + Box( + modifier = + Modifier + .padding(top = 7.dp) + .size(8.dp) + .background(onboardingAccentSoft, CircleShape), + ) + Box( + modifier = + Modifier + .padding(top = 9.dp) + .size(4.dp) + .background(onboardingAccent, CircleShape), + ) + Text(text, style = onboardingBodyStyle, color = onboardingTextSecondary, modifier = Modifier.weight(1f)) + } +} + +private fun isPermissionGranted(context: Context, permission: String): Boolean { + return ContextCompat.checkSelfPermission(context, permission) == PackageManager.PERMISSION_GRANTED +} + +private fun isNotificationListenerEnabled(context: Context): Boolean { + return DeviceNotificationListenerService.isAccessEnabled(context) +} + +private fun canInstallUnknownApps(context: Context): Boolean { + if (Build.VERSION.SDK_INT < 26) return true + return context.packageManager.canRequestPackageInstalls() +} + +private fun openNotificationListenerSettings(context: Context) { + val intent = Intent(Settings.ACTION_NOTIFICATION_LISTENER_SETTINGS).addFlags(Intent.FLAG_ACTIVITY_NEW_TASK) + runCatching { + context.startActivity(intent) + }.getOrElse { + openAppSettings(context) + } +} + +private fun openUnknownAppSourcesSettings(context: Context) { + if (Build.VERSION.SDK_INT < 26) return + val intent = + Intent( + Settings.ACTION_MANAGE_UNKNOWN_APP_SOURCES, + Uri.parse("package:${context.packageName}"), + ).addFlags(Intent.FLAG_ACTIVITY_NEW_TASK) + runCatching { + context.startActivity(intent) + }.getOrElse { + openAppSettings(context) + } +} + +private fun openAppSettings(context: Context) { + val intent = + Intent( + Settings.ACTION_APPLICATION_DETAILS_SETTINGS, + Uri.fromParts("package", context.packageName, null), + ).addFlags(Intent.FLAG_ACTIVITY_NEW_TASK) + context.startActivity(intent) +} + +private fun hasMotionCapabilities(context: Context): Boolean { + val sensorManager = context.getSystemService(SensorManager::class.java) ?: return false + return sensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER) != null || + sensorManager.getDefaultSensor(Sensor.TYPE_STEP_COUNTER) != null +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/ui/PostOnboardingTabs.kt b/apps/android/app/src/main/java/ai/openclaw/android/ui/PostOnboardingTabs.kt new file mode 100644 index 00000000000..1345d8e3cb9 --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/ui/PostOnboardingTabs.kt @@ -0,0 +1,320 @@ +package ai.openclaw.android.ui + +import androidx.compose.foundation.background +import androidx.compose.foundation.BorderStroke +import androidx.compose.foundation.layout.Arrangement +import androidx.compose.foundation.layout.Box +import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.Row +import androidx.compose.foundation.layout.WindowInsets +import androidx.compose.foundation.layout.WindowInsetsSides +import androidx.compose.foundation.layout.fillMaxSize +import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.heightIn +import androidx.compose.foundation.layout.ime +import androidx.compose.foundation.layout.navigationBars +import androidx.compose.foundation.layout.only +import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.layout.safeDrawing +import androidx.compose.foundation.layout.windowInsetsPadding +import androidx.compose.foundation.layout.consumeWindowInsets +import androidx.compose.foundation.shape.RoundedCornerShape +import androidx.compose.material.icons.Icons +import androidx.compose.material.icons.automirrored.filled.ScreenShare +import androidx.compose.material.icons.filled.ChatBubble +import androidx.compose.material.icons.filled.CheckCircle +import androidx.compose.material.icons.filled.RecordVoiceOver +import androidx.compose.material.icons.filled.Settings +import androidx.compose.material3.Icon +import androidx.compose.material3.Scaffold +import androidx.compose.material3.Surface +import androidx.compose.material3.Text +import androidx.compose.runtime.Composable +import androidx.compose.runtime.collectAsState +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.saveable.rememberSaveable +import androidx.compose.runtime.setValue +import androidx.compose.ui.Alignment +import androidx.compose.ui.Modifier +import androidx.compose.ui.graphics.Color +import androidx.compose.ui.graphics.vector.ImageVector +import androidx.compose.ui.platform.LocalDensity +import androidx.compose.ui.text.font.FontWeight +import androidx.compose.ui.unit.dp +import ai.openclaw.android.MainViewModel + +private enum class HomeTab( + val label: String, + val icon: ImageVector, +) { + Connect(label = "Connect", icon = Icons.Default.CheckCircle), + Chat(label = "Chat", icon = Icons.Default.ChatBubble), + Voice(label = "Voice", icon = Icons.Default.RecordVoiceOver), + Screen(label = "Screen", icon = Icons.AutoMirrored.Filled.ScreenShare), + Settings(label = "Settings", icon = Icons.Default.Settings), +} + +private enum class StatusVisual { + Connected, + Connecting, + Warning, + Error, + Offline, +} + +@Composable +fun PostOnboardingTabs(viewModel: MainViewModel, modifier: Modifier = Modifier) { + var activeTab by rememberSaveable { mutableStateOf(HomeTab.Connect) } + + val statusText by viewModel.statusText.collectAsState() + val isConnected by viewModel.isConnected.collectAsState() + + val statusVisual = + remember(statusText, isConnected) { + val lower = statusText.lowercase() + when { + isConnected -> StatusVisual.Connected + lower.contains("connecting") || lower.contains("reconnecting") -> StatusVisual.Connecting + lower.contains("pairing") || lower.contains("approval") || lower.contains("auth") -> StatusVisual.Warning + lower.contains("error") || lower.contains("failed") -> StatusVisual.Error + else -> StatusVisual.Offline + } + } + + val density = LocalDensity.current + val imeVisible = WindowInsets.ime.getBottom(density) > 0 + val hideBottomTabBar = activeTab == HomeTab.Chat && imeVisible + + Scaffold( + modifier = modifier, + containerColor = Color.Transparent, + contentWindowInsets = WindowInsets(0, 0, 0, 0), + topBar = { + TopStatusBar( + statusText = statusText, + statusVisual = statusVisual, + ) + }, + bottomBar = { + if (!hideBottomTabBar) { + BottomTabBar( + activeTab = activeTab, + onSelect = { activeTab = it }, + ) + } + }, + ) { innerPadding -> + Box( + modifier = + Modifier + .fillMaxSize() + .padding(innerPadding) + .consumeWindowInsets(innerPadding) + .background(mobileBackgroundGradient), + ) { + when (activeTab) { + HomeTab.Connect -> ConnectTabScreen(viewModel = viewModel) + HomeTab.Chat -> ChatSheet(viewModel = viewModel) + HomeTab.Voice -> VoiceTabScreen(viewModel = viewModel) + HomeTab.Screen -> ScreenTabScreen(viewModel = viewModel) + HomeTab.Settings -> SettingsSheet(viewModel = viewModel) + } + } + } +} + +@Composable +private fun ScreenTabScreen(viewModel: MainViewModel) { + val isConnected by viewModel.isConnected.collectAsState() + val isNodeConnected by viewModel.isNodeConnected.collectAsState() + val canvasUrl by viewModel.canvasCurrentUrl.collectAsState() + val canvasA2uiHydrated by viewModel.canvasA2uiHydrated.collectAsState() + val canvasRehydratePending by viewModel.canvasRehydratePending.collectAsState() + val canvasRehydrateErrorText by viewModel.canvasRehydrateErrorText.collectAsState() + val isA2uiUrl = canvasUrl?.contains("/__openclaw__/a2ui/") == true + val showRestoreCta = isConnected && isNodeConnected && (canvasUrl.isNullOrBlank() || (isA2uiUrl && !canvasA2uiHydrated)) + val restoreCtaText = + when { + canvasRehydratePending -> "Restore requested. Waiting for agent…" + !canvasRehydrateErrorText.isNullOrBlank() -> canvasRehydrateErrorText!! + else -> "Canvas reset. Tap to restore dashboard." + } + + Box(modifier = Modifier.fillMaxSize()) { + CanvasScreen(viewModel = viewModel, modifier = Modifier.fillMaxSize()) + + if (showRestoreCta) { + Surface( + onClick = { + if (canvasRehydratePending) return@Surface + viewModel.requestCanvasRehydrate(source = "screen_tab_cta") + }, + modifier = Modifier.align(Alignment.TopCenter).padding(horizontal = 16.dp, vertical = 16.dp), + shape = RoundedCornerShape(12.dp), + color = mobileSurface.copy(alpha = 0.9f), + border = BorderStroke(1.dp, mobileBorder), + shadowElevation = 4.dp, + ) { + Text( + text = restoreCtaText, + modifier = Modifier.padding(horizontal = 12.dp, vertical = 10.dp), + style = mobileCallout.copy(fontWeight = FontWeight.Medium), + color = mobileText, + ) + } + } + } +} + +@Composable +private fun TopStatusBar( + statusText: String, + statusVisual: StatusVisual, +) { + val safeInsets = WindowInsets.safeDrawing.only(WindowInsetsSides.Top + WindowInsetsSides.Horizontal) + + val (chipBg, chipDot, chipText, chipBorder) = + when (statusVisual) { + StatusVisual.Connected -> + listOf( + mobileSuccessSoft, + mobileSuccess, + mobileSuccess, + Color(0xFFCFEBD8), + ) + StatusVisual.Connecting -> + listOf( + mobileAccentSoft, + mobileAccent, + mobileAccent, + Color(0xFFD5E2FA), + ) + StatusVisual.Warning -> + listOf( + mobileWarningSoft, + mobileWarning, + mobileWarning, + Color(0xFFEED8B8), + ) + StatusVisual.Error -> + listOf( + mobileDangerSoft, + mobileDanger, + mobileDanger, + Color(0xFFF3C8C8), + ) + StatusVisual.Offline -> + listOf( + mobileSurface, + mobileTextTertiary, + mobileTextSecondary, + mobileBorder, + ) + } + + Surface( + modifier = Modifier.fillMaxWidth().windowInsetsPadding(safeInsets), + color = Color.Transparent, + shadowElevation = 0.dp, + ) { + Row( + modifier = Modifier.fillMaxWidth().padding(horizontal = 18.dp, vertical = 12.dp), + verticalAlignment = Alignment.CenterVertically, + horizontalArrangement = Arrangement.SpaceBetween, + ) { + Text( + text = "OpenClaw", + style = mobileTitle2, + color = mobileText, + ) + Surface( + shape = RoundedCornerShape(999.dp), + color = chipBg, + border = androidx.compose.foundation.BorderStroke(1.dp, chipBorder), + ) { + Row( + modifier = Modifier.padding(horizontal = 10.dp, vertical = 5.dp), + horizontalArrangement = Arrangement.spacedBy(6.dp), + verticalAlignment = Alignment.CenterVertically, + ) { + Surface( + modifier = Modifier.padding(top = 1.dp), + color = chipDot, + shape = RoundedCornerShape(999.dp), + ) { + Box(modifier = Modifier.padding(4.dp)) + } + Text( + text = statusText.trim().ifEmpty { "Offline" }, + style = mobileCaption1, + color = chipText, + maxLines = 1, + ) + } + } + } + } +} + +@Composable +private fun BottomTabBar( + activeTab: HomeTab, + onSelect: (HomeTab) -> Unit, +) { + val safeInsets = WindowInsets.navigationBars.only(WindowInsetsSides.Bottom + WindowInsetsSides.Horizontal) + + Box( + modifier = + Modifier + .fillMaxWidth(), + ) { + Surface( + modifier = Modifier.fillMaxWidth(), + color = Color.White.copy(alpha = 0.97f), + shape = RoundedCornerShape(topStart = 24.dp, topEnd = 24.dp), + border = BorderStroke(1.dp, mobileBorder), + shadowElevation = 6.dp, + ) { + Row( + modifier = + Modifier + .fillMaxWidth() + .windowInsetsPadding(safeInsets) + .padding(horizontal = 10.dp, vertical = 10.dp), + horizontalArrangement = Arrangement.spacedBy(6.dp), + verticalAlignment = Alignment.CenterVertically, + ) { + HomeTab.entries.forEach { tab -> + val active = tab == activeTab + Surface( + onClick = { onSelect(tab) }, + modifier = Modifier.weight(1f).heightIn(min = 58.dp), + shape = RoundedCornerShape(16.dp), + color = if (active) mobileAccentSoft else Color.Transparent, + border = if (active) BorderStroke(1.dp, Color(0xFFD5E2FA)) else null, + shadowElevation = 0.dp, + ) { + Column( + modifier = Modifier.fillMaxWidth().padding(horizontal = 6.dp, vertical = 7.dp), + horizontalAlignment = Alignment.CenterHorizontally, + verticalArrangement = Arrangement.spacedBy(2.dp), + ) { + Icon( + imageVector = tab.icon, + contentDescription = tab.label, + tint = if (active) mobileAccent else mobileTextTertiary, + ) + Text( + text = tab.label, + color = if (active) mobileAccent else mobileTextSecondary, + style = mobileCaption2.copy(fontWeight = if (active) FontWeight.Bold else FontWeight.Medium), + ) + } + } + } + } + } + } +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/ui/RootScreen.kt b/apps/android/app/src/main/java/ai/openclaw/android/ui/RootScreen.kt index af0cfe628ac..e50a03cc5bf 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/ui/RootScreen.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/ui/RootScreen.kt @@ -1,429 +1,20 @@ package ai.openclaw.android.ui -import android.annotation.SuppressLint -import android.Manifest -import android.content.pm.PackageManager -import android.graphics.Color -import android.util.Log -import android.view.View -import android.webkit.JavascriptInterface -import android.webkit.ConsoleMessage -import android.webkit.WebChromeClient -import android.webkit.WebView -import android.webkit.WebSettings -import android.webkit.WebResourceError -import android.webkit.WebResourceRequest -import android.webkit.WebResourceResponse -import android.webkit.WebViewClient -import androidx.activity.compose.rememberLauncherForActivityResult -import androidx.activity.result.contract.ActivityResultContracts -import androidx.webkit.WebSettingsCompat -import androidx.webkit.WebViewFeature -import androidx.compose.foundation.layout.Arrangement -import androidx.compose.foundation.layout.Box -import androidx.compose.foundation.layout.Column -import androidx.compose.foundation.layout.WindowInsets -import androidx.compose.foundation.layout.WindowInsetsSides -import androidx.compose.foundation.layout.only import androidx.compose.foundation.layout.fillMaxSize -import androidx.compose.foundation.layout.padding -import androidx.compose.foundation.layout.safeDrawing -import androidx.compose.foundation.layout.size -import androidx.compose.foundation.layout.windowInsetsPadding -import androidx.compose.material3.ExperimentalMaterial3Api -import androidx.compose.material3.FilledTonalIconButton -import androidx.compose.material3.Icon -import androidx.compose.material3.IconButtonDefaults -import androidx.compose.material3.LocalContentColor -import androidx.compose.material3.MaterialTheme -import androidx.compose.material3.ModalBottomSheet -import androidx.compose.material3.rememberModalBottomSheetState -import androidx.compose.material.icons.Icons -import androidx.compose.material.icons.automirrored.filled.ScreenShare -import androidx.compose.material.icons.filled.ChatBubble -import androidx.compose.material.icons.filled.CheckCircle -import androidx.compose.material.icons.filled.Error -import androidx.compose.material.icons.filled.FiberManualRecord -import androidx.compose.material.icons.filled.PhotoCamera -import androidx.compose.material.icons.filled.RecordVoiceOver -import androidx.compose.material.icons.filled.Refresh -import androidx.compose.material.icons.filled.Report -import androidx.compose.material.icons.filled.Settings import androidx.compose.runtime.Composable import androidx.compose.runtime.collectAsState import androidx.compose.runtime.getValue -import androidx.compose.runtime.mutableStateOf -import androidx.compose.runtime.remember -import androidx.compose.runtime.setValue -import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier -import androidx.compose.ui.graphics.Color as ComposeColor -import androidx.compose.ui.graphics.lerp -import androidx.compose.ui.platform.LocalContext -import androidx.compose.ui.unit.dp -import androidx.compose.ui.viewinterop.AndroidView -import androidx.compose.ui.window.Popup -import androidx.compose.ui.window.PopupProperties -import androidx.core.content.ContextCompat -import ai.openclaw.android.CameraHudKind import ai.openclaw.android.MainViewModel -@OptIn(ExperimentalMaterial3Api::class) @Composable fun RootScreen(viewModel: MainViewModel) { - var sheet by remember { mutableStateOf(null) } - val sheetState = rememberModalBottomSheetState(skipPartiallyExpanded = true) - val safeOverlayInsets = WindowInsets.safeDrawing.only(WindowInsetsSides.Top + WindowInsetsSides.Horizontal) - val context = LocalContext.current - val serverName by viewModel.serverName.collectAsState() - val statusText by viewModel.statusText.collectAsState() - val cameraHud by viewModel.cameraHud.collectAsState() - val cameraFlashToken by viewModel.cameraFlashToken.collectAsState() - val screenRecordActive by viewModel.screenRecordActive.collectAsState() - val isForeground by viewModel.isForeground.collectAsState() - val voiceWakeStatusText by viewModel.voiceWakeStatusText.collectAsState() - val talkEnabled by viewModel.talkEnabled.collectAsState() - val talkStatusText by viewModel.talkStatusText.collectAsState() - val talkIsListening by viewModel.talkIsListening.collectAsState() - val talkIsSpeaking by viewModel.talkIsSpeaking.collectAsState() - val seamColorArgb by viewModel.seamColorArgb.collectAsState() - val seamColor = remember(seamColorArgb) { ComposeColor(seamColorArgb) } - val audioPermissionLauncher = - rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) { granted -> - if (granted) viewModel.setTalkEnabled(true) - } - val activity = - remember(cameraHud, screenRecordActive, isForeground, statusText, voiceWakeStatusText) { - // Status pill owns transient activity state so it doesn't overlap the connection indicator. - if (!isForeground) { - return@remember StatusActivity( - title = "Foreground required", - icon = Icons.Default.Report, - contentDescription = "Foreground required", - ) - } + val onboardingCompleted by viewModel.onboardingCompleted.collectAsState() - val lowerStatus = statusText.lowercase() - if (lowerStatus.contains("repair")) { - return@remember StatusActivity( - title = "Repairing…", - icon = Icons.Default.Refresh, - contentDescription = "Repairing", - ) - } - if (lowerStatus.contains("pairing") || lowerStatus.contains("approval")) { - return@remember StatusActivity( - title = "Approval pending", - icon = Icons.Default.RecordVoiceOver, - contentDescription = "Approval pending", - ) - } - // Avoid duplicating the primary gateway status ("Connecting…") in the activity slot. - - if (screenRecordActive) { - return@remember StatusActivity( - title = "Recording screen…", - icon = Icons.AutoMirrored.Filled.ScreenShare, - contentDescription = "Recording screen", - tint = androidx.compose.ui.graphics.Color.Red, - ) - } - - cameraHud?.let { hud -> - return@remember when (hud.kind) { - CameraHudKind.Photo -> - StatusActivity( - title = hud.message, - icon = Icons.Default.PhotoCamera, - contentDescription = "Taking photo", - ) - CameraHudKind.Recording -> - StatusActivity( - title = hud.message, - icon = Icons.Default.FiberManualRecord, - contentDescription = "Recording", - tint = androidx.compose.ui.graphics.Color.Red, - ) - CameraHudKind.Success -> - StatusActivity( - title = hud.message, - icon = Icons.Default.CheckCircle, - contentDescription = "Capture finished", - ) - CameraHudKind.Error -> - StatusActivity( - title = hud.message, - icon = Icons.Default.Error, - contentDescription = "Capture failed", - tint = androidx.compose.ui.graphics.Color.Red, - ) - } - } - - if (voiceWakeStatusText.contains("Microphone permission", ignoreCase = true)) { - return@remember StatusActivity( - title = "Mic permission", - icon = Icons.Default.Error, - contentDescription = "Mic permission required", - ) - } - if (voiceWakeStatusText == "Paused") { - val suffix = if (!isForeground) " (background)" else "" - return@remember StatusActivity( - title = "Voice Wake paused$suffix", - icon = Icons.Default.RecordVoiceOver, - contentDescription = "Voice Wake paused", - ) - } - - null - } - - val gatewayState = - remember(serverName, statusText) { - when { - serverName != null -> GatewayState.Connected - statusText.contains("connecting", ignoreCase = true) || - statusText.contains("reconnecting", ignoreCase = true) -> GatewayState.Connecting - statusText.contains("error", ignoreCase = true) -> GatewayState.Error - else -> GatewayState.Disconnected - } - } - - val voiceEnabled = - ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) == - PackageManager.PERMISSION_GRANTED - - Box(modifier = Modifier.fillMaxSize()) { - CanvasView(viewModel = viewModel, modifier = Modifier.fillMaxSize()) + if (!onboardingCompleted) { + OnboardingFlow(viewModel = viewModel, modifier = Modifier.fillMaxSize()) + return } - // Camera flash must be in a Popup to render above the WebView. - Popup(alignment = Alignment.Center, properties = PopupProperties(focusable = false)) { - CameraFlashOverlay(token = cameraFlashToken, modifier = Modifier.fillMaxSize()) - } - - // Keep the overlay buttons above the WebView canvas (AndroidView), otherwise they may not receive touches. - Popup(alignment = Alignment.TopStart, properties = PopupProperties(focusable = false)) { - StatusPill( - gateway = gatewayState, - voiceEnabled = voiceEnabled, - activity = activity, - onClick = { sheet = Sheet.Settings }, - modifier = Modifier.windowInsetsPadding(safeOverlayInsets).padding(start = 12.dp, top = 12.dp), - ) - } - - Popup(alignment = Alignment.TopEnd, properties = PopupProperties(focusable = false)) { - Column( - modifier = Modifier.windowInsetsPadding(safeOverlayInsets).padding(end = 12.dp, top = 12.dp), - verticalArrangement = Arrangement.spacedBy(10.dp), - horizontalAlignment = Alignment.End, - ) { - OverlayIconButton( - onClick = { sheet = Sheet.Chat }, - icon = { Icon(Icons.Default.ChatBubble, contentDescription = "Chat") }, - ) - - // Talk mode gets a dedicated side bubble instead of burying it in settings. - val baseOverlay = overlayContainerColor() - val talkContainer = - lerp( - baseOverlay, - seamColor.copy(alpha = baseOverlay.alpha), - if (talkEnabled) 0.35f else 0.22f, - ) - val talkContent = if (talkEnabled) seamColor else overlayIconColor() - OverlayIconButton( - onClick = { - val next = !talkEnabled - if (next) { - val micOk = - ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) == - PackageManager.PERMISSION_GRANTED - if (!micOk) audioPermissionLauncher.launch(Manifest.permission.RECORD_AUDIO) - viewModel.setTalkEnabled(true) - } else { - viewModel.setTalkEnabled(false) - } - }, - containerColor = talkContainer, - contentColor = talkContent, - icon = { - Icon( - Icons.Default.RecordVoiceOver, - contentDescription = "Talk Mode", - ) - }, - ) - - OverlayIconButton( - onClick = { sheet = Sheet.Settings }, - icon = { Icon(Icons.Default.Settings, contentDescription = "Settings") }, - ) - } - } - - if (talkEnabled) { - Popup(alignment = Alignment.Center, properties = PopupProperties(focusable = false)) { - TalkOrbOverlay( - seamColor = seamColor, - statusText = talkStatusText, - isListening = talkIsListening, - isSpeaking = talkIsSpeaking, - ) - } - } - - val currentSheet = sheet - if (currentSheet != null) { - ModalBottomSheet( - onDismissRequest = { sheet = null }, - sheetState = sheetState, - ) { - when (currentSheet) { - Sheet.Chat -> ChatSheet(viewModel = viewModel) - Sheet.Settings -> SettingsSheet(viewModel = viewModel) - } - } - } -} - -private enum class Sheet { - Chat, - Settings, -} - -@Composable -private fun OverlayIconButton( - onClick: () -> Unit, - icon: @Composable () -> Unit, - containerColor: ComposeColor? = null, - contentColor: ComposeColor? = null, -) { - FilledTonalIconButton( - onClick = onClick, - modifier = Modifier.size(44.dp), - colors = - IconButtonDefaults.filledTonalIconButtonColors( - containerColor = containerColor ?: overlayContainerColor(), - contentColor = contentColor ?: overlayIconColor(), - ), - ) { - icon() - } -} - -@SuppressLint("SetJavaScriptEnabled") -@Composable -private fun CanvasView(viewModel: MainViewModel, modifier: Modifier = Modifier) { - val context = LocalContext.current - val isDebuggable = (context.applicationInfo.flags and android.content.pm.ApplicationInfo.FLAG_DEBUGGABLE) != 0 - AndroidView( - modifier = modifier, - factory = { - WebView(context).apply { - settings.javaScriptEnabled = true - // Some embedded web UIs (incl. the "background website") use localStorage/sessionStorage. - settings.domStorageEnabled = true - settings.mixedContentMode = WebSettings.MIXED_CONTENT_COMPATIBILITY_MODE - if (WebViewFeature.isFeatureSupported(WebViewFeature.ALGORITHMIC_DARKENING)) { - WebSettingsCompat.setAlgorithmicDarkeningAllowed(settings, false) - } else { - disableForceDarkIfSupported(settings) - } - if (isDebuggable) { - Log.d("OpenClawWebView", "userAgent: ${settings.userAgentString}") - } - isScrollContainer = true - overScrollMode = View.OVER_SCROLL_IF_CONTENT_SCROLLS - isVerticalScrollBarEnabled = true - isHorizontalScrollBarEnabled = true - webViewClient = - object : WebViewClient() { - override fun onReceivedError( - view: WebView, - request: WebResourceRequest, - error: WebResourceError, - ) { - if (!isDebuggable) return - if (!request.isForMainFrame) return - Log.e("OpenClawWebView", "onReceivedError: ${error.errorCode} ${error.description} ${request.url}") - } - - override fun onReceivedHttpError( - view: WebView, - request: WebResourceRequest, - errorResponse: WebResourceResponse, - ) { - if (!isDebuggable) return - if (!request.isForMainFrame) return - Log.e( - "OpenClawWebView", - "onReceivedHttpError: ${errorResponse.statusCode} ${errorResponse.reasonPhrase} ${request.url}", - ) - } - - override fun onPageFinished(view: WebView, url: String?) { - if (isDebuggable) { - Log.d("OpenClawWebView", "onPageFinished: $url") - } - viewModel.canvas.onPageFinished() - } - - override fun onRenderProcessGone( - view: WebView, - detail: android.webkit.RenderProcessGoneDetail, - ): Boolean { - if (isDebuggable) { - Log.e( - "OpenClawWebView", - "onRenderProcessGone didCrash=${detail.didCrash()} priorityAtExit=${detail.rendererPriorityAtExit()}", - ) - } - return true - } - } - webChromeClient = - object : WebChromeClient() { - override fun onConsoleMessage(consoleMessage: ConsoleMessage?): Boolean { - if (!isDebuggable) return false - val msg = consoleMessage ?: return false - Log.d( - "OpenClawWebView", - "console ${msg.messageLevel()} @ ${msg.sourceId()}:${msg.lineNumber()} ${msg.message()}", - ) - return false - } - } - // Use default layer/background; avoid forcing a black fill over WebView content. - - val a2uiBridge = - CanvasA2UIActionBridge { payload -> - viewModel.handleCanvasA2UIActionFromWebView(payload) - } - addJavascriptInterface(a2uiBridge, CanvasA2UIActionBridge.interfaceName) - viewModel.canvas.attach(this) - } - }, - ) -} - -private fun disableForceDarkIfSupported(settings: WebSettings) { - if (!WebViewFeature.isFeatureSupported(WebViewFeature.FORCE_DARK)) return - @Suppress("DEPRECATION") - WebSettingsCompat.setForceDark(settings, WebSettingsCompat.FORCE_DARK_OFF) -} - -private class CanvasA2UIActionBridge(private val onMessage: (String) -> Unit) { - @JavascriptInterface - fun postMessage(payload: String?) { - val msg = payload?.trim().orEmpty() - if (msg.isEmpty()) return - onMessage(msg) - } - - companion object { - const val interfaceName: String = "openclawCanvasA2UIAction" - } + PostOnboardingTabs(viewModel = viewModel, modifier = Modifier.fillMaxSize()) } diff --git a/apps/android/app/src/main/java/ai/openclaw/android/ui/SettingsSheet.kt b/apps/android/app/src/main/java/ai/openclaw/android/ui/SettingsSheet.kt index bb04c30108c..b20013f7d57 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/ui/SettingsSheet.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/ui/SettingsSheet.kt @@ -4,15 +4,19 @@ import android.Manifest import android.content.Context import android.content.Intent import android.content.pm.PackageManager +import android.hardware.Sensor +import android.hardware.SensorManager import android.net.Uri import android.os.Build import android.provider.Settings import androidx.activity.compose.rememberLauncherForActivityResult import androidx.activity.result.contract.ActivityResultContracts -import androidx.compose.animation.AnimatedVisibility -import androidx.compose.foundation.clickable +import androidx.compose.foundation.background +import androidx.compose.foundation.border +import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.fillMaxSize import androidx.compose.foundation.layout.PaddingValues import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.Spacer @@ -23,29 +27,27 @@ import androidx.compose.foundation.layout.fillMaxWidth import androidx.compose.foundation.layout.height import androidx.compose.foundation.layout.imePadding import androidx.compose.foundation.layout.only +import androidx.compose.foundation.layout.padding import androidx.compose.foundation.layout.safeDrawing import androidx.compose.foundation.layout.windowInsetsPadding import androidx.compose.foundation.lazy.LazyColumn import androidx.compose.foundation.lazy.items import androidx.compose.foundation.lazy.rememberLazyListState -import androidx.compose.foundation.text.KeyboardActions -import androidx.compose.foundation.text.KeyboardOptions +import androidx.compose.foundation.shape.RoundedCornerShape import androidx.compose.material.icons.Icons -import androidx.compose.material.icons.filled.ExpandLess -import androidx.compose.material.icons.filled.ExpandMore import androidx.compose.material3.Button -import androidx.compose.material3.AlertDialog +import androidx.compose.material3.ButtonDefaults import androidx.compose.material3.HorizontalDivider -import androidx.compose.material3.Icon import androidx.compose.material3.ListItem +import androidx.compose.material3.ListItemDefaults import androidx.compose.material3.MaterialTheme import androidx.compose.material3.OutlinedTextField +import androidx.compose.material3.OutlinedTextFieldDefaults import androidx.compose.material3.RadioButton import androidx.compose.material3.Switch import androidx.compose.material3.Text -import androidx.compose.material3.TextButton import androidx.compose.runtime.Composable -import androidx.compose.runtime.LaunchedEffect +import androidx.compose.runtime.DisposableEffect import androidx.compose.runtime.collectAsState import androidx.compose.runtime.getValue import androidx.compose.runtime.mutableStateOf @@ -53,51 +55,34 @@ import androidx.compose.runtime.remember import androidx.compose.runtime.setValue import androidx.compose.ui.Modifier import androidx.compose.ui.draw.alpha -import androidx.compose.ui.focus.onFocusChanged import androidx.compose.ui.platform.LocalContext -import androidx.compose.ui.platform.LocalFocusManager -import androidx.compose.ui.text.input.ImeAction -import androidx.compose.ui.text.style.TextAlign +import androidx.compose.ui.graphics.Color +import androidx.compose.ui.text.font.FontFamily +import androidx.compose.ui.text.font.FontWeight +import androidx.compose.ui.unit.sp import androidx.compose.ui.unit.dp import androidx.core.content.ContextCompat +import androidx.lifecycle.Lifecycle +import androidx.lifecycle.LifecycleEventObserver +import androidx.lifecycle.compose.LocalLifecycleOwner import ai.openclaw.android.BuildConfig import ai.openclaw.android.LocationMode import ai.openclaw.android.MainViewModel -import ai.openclaw.android.NodeForegroundService -import ai.openclaw.android.VoiceWakeMode -import ai.openclaw.android.WakeWords +import ai.openclaw.android.node.DeviceNotificationListenerService @Composable fun SettingsSheet(viewModel: MainViewModel) { val context = LocalContext.current + val lifecycleOwner = LocalLifecycleOwner.current val instanceId by viewModel.instanceId.collectAsState() val displayName by viewModel.displayName.collectAsState() val cameraEnabled by viewModel.cameraEnabled.collectAsState() val locationMode by viewModel.locationMode.collectAsState() val locationPreciseEnabled by viewModel.locationPreciseEnabled.collectAsState() val preventSleep by viewModel.preventSleep.collectAsState() - val wakeWords by viewModel.wakeWords.collectAsState() - val voiceWakeMode by viewModel.voiceWakeMode.collectAsState() - val voiceWakeStatusText by viewModel.voiceWakeStatusText.collectAsState() - val isConnected by viewModel.isConnected.collectAsState() - val manualEnabled by viewModel.manualEnabled.collectAsState() - val manualHost by viewModel.manualHost.collectAsState() - val manualPort by viewModel.manualPort.collectAsState() - val manualTls by viewModel.manualTls.collectAsState() - val gatewayToken by viewModel.gatewayToken.collectAsState() val canvasDebugStatusEnabled by viewModel.canvasDebugStatusEnabled.collectAsState() - val statusText by viewModel.statusText.collectAsState() - val serverName by viewModel.serverName.collectAsState() - val remoteAddress by viewModel.remoteAddress.collectAsState() - val gateways by viewModel.gateways.collectAsState() - val discoveryStatusText by viewModel.discoveryStatusText.collectAsState() - val pendingTrust by viewModel.pendingGatewayTrust.collectAsState() val listState = rememberLazyListState() - val (wakeWordsText, setWakeWordsText) = remember { mutableStateOf("") } - val (advancedExpanded, setAdvancedExpanded) = remember { mutableStateOf(false) } - val focusManager = LocalFocusManager.current - var wakeWordsHadFocus by remember { mutableStateOf(false) } val deviceModel = remember { listOfNotNull(Build.MANUFACTURER, Build.MODEL) @@ -114,39 +99,14 @@ fun SettingsSheet(viewModel: MainViewModel) { versionName } } - - if (pendingTrust != null) { - val prompt = pendingTrust!! - AlertDialog( - onDismissRequest = { viewModel.declineGatewayTrustPrompt() }, - title = { Text("Trust this gateway?") }, - text = { - Text( - "First-time TLS connection.\n\n" + - "Verify this SHA-256 fingerprint out-of-band before trusting:\n" + - prompt.fingerprintSha256, - ) - }, - confirmButton = { - TextButton(onClick = { viewModel.acceptGatewayTrustPrompt() }) { - Text("Trust and connect") - } - }, - dismissButton = { - TextButton(onClick = { viewModel.declineGatewayTrustPrompt() }) { - Text("Cancel") - } - }, + val listItemColors = + ListItemDefaults.colors( + containerColor = Color.Transparent, + headlineColor = mobileText, + supportingColor = mobileTextSecondary, + trailingIconColor = mobileTextSecondary, + leadingIconColor = mobileTextSecondary, ) - } - - LaunchedEffect(wakeWords) { setWakeWordsText(wakeWords.joinToString(", ")) } - val commitWakeWords = { - val parsed = WakeWords.parseIfChanged(wakeWordsText, wakeWords) - if (parsed != null) { - viewModel.setWakeWords(parsed) - } - } val permissionLauncher = rememberLauncherForActivityResult(ActivityResultContracts.RequestMultiplePermissions()) { perms -> @@ -189,15 +149,107 @@ fun SettingsSheet(viewModel: MainViewModel) { } } + var micPermissionGranted by + remember { + mutableStateOf( + ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) == + PackageManager.PERMISSION_GRANTED, + ) + } val audioPermissionLauncher = - rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) { _ -> - // Status text is handled by NodeRuntime. + rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) { granted -> + micPermissionGranted = granted } val smsPermissionAvailable = remember { context.packageManager?.hasSystemFeature(PackageManager.FEATURE_TELEPHONY) == true } + val photosPermission = + if (Build.VERSION.SDK_INT >= 33) { + Manifest.permission.READ_MEDIA_IMAGES + } else { + Manifest.permission.READ_EXTERNAL_STORAGE + } + val motionPermissionRequired = Build.VERSION.SDK_INT >= 29 + val motionAvailable = remember(context) { hasMotionCapabilities(context) } + + var notificationsPermissionGranted by + remember { + mutableStateOf(hasNotificationsPermission(context)) + } + val notificationsPermissionLauncher = + rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) { granted -> + notificationsPermissionGranted = granted + } + + var notificationListenerEnabled by + remember { + mutableStateOf(isNotificationListenerEnabled(context)) + } + + var photosPermissionGranted by + remember { + mutableStateOf( + ContextCompat.checkSelfPermission(context, photosPermission) == + PackageManager.PERMISSION_GRANTED, + ) + } + val photosPermissionLauncher = + rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) { granted -> + photosPermissionGranted = granted + } + + var contactsPermissionGranted by + remember { + mutableStateOf( + ContextCompat.checkSelfPermission(context, Manifest.permission.READ_CONTACTS) == + PackageManager.PERMISSION_GRANTED && + ContextCompat.checkSelfPermission(context, Manifest.permission.WRITE_CONTACTS) == + PackageManager.PERMISSION_GRANTED, + ) + } + val contactsPermissionLauncher = + rememberLauncherForActivityResult(ActivityResultContracts.RequestMultiplePermissions()) { perms -> + val readOk = perms[Manifest.permission.READ_CONTACTS] == true + val writeOk = perms[Manifest.permission.WRITE_CONTACTS] == true + contactsPermissionGranted = readOk && writeOk + } + + var calendarPermissionGranted by + remember { + mutableStateOf( + ContextCompat.checkSelfPermission(context, Manifest.permission.READ_CALENDAR) == + PackageManager.PERMISSION_GRANTED && + ContextCompat.checkSelfPermission(context, Manifest.permission.WRITE_CALENDAR) == + PackageManager.PERMISSION_GRANTED, + ) + } + val calendarPermissionLauncher = + rememberLauncherForActivityResult(ActivityResultContracts.RequestMultiplePermissions()) { perms -> + val readOk = perms[Manifest.permission.READ_CALENDAR] == true + val writeOk = perms[Manifest.permission.WRITE_CALENDAR] == true + calendarPermissionGranted = readOk && writeOk + } + + var motionPermissionGranted by + remember { + mutableStateOf( + !motionPermissionRequired || + ContextCompat.checkSelfPermission(context, Manifest.permission.ACTIVITY_RECOGNITION) == + PackageManager.PERMISSION_GRANTED, + ) + } + val motionPermissionLauncher = + rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) { granted -> + motionPermissionGranted = granted + } + + var appUpdateInstallEnabled by + remember { + mutableStateOf(canInstallUnknownApps(context)) + } + var smsPermissionGranted by remember { mutableStateOf( @@ -211,6 +263,42 @@ fun SettingsSheet(viewModel: MainViewModel) { viewModel.refreshGatewayConnection() } + DisposableEffect(lifecycleOwner, context) { + val observer = + LifecycleEventObserver { _, event -> + if (event == Lifecycle.Event.ON_RESUME) { + micPermissionGranted = + ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) == + PackageManager.PERMISSION_GRANTED + notificationsPermissionGranted = hasNotificationsPermission(context) + notificationListenerEnabled = isNotificationListenerEnabled(context) + photosPermissionGranted = + ContextCompat.checkSelfPermission(context, photosPermission) == + PackageManager.PERMISSION_GRANTED + contactsPermissionGranted = + ContextCompat.checkSelfPermission(context, Manifest.permission.READ_CONTACTS) == + PackageManager.PERMISSION_GRANTED && + ContextCompat.checkSelfPermission(context, Manifest.permission.WRITE_CONTACTS) == + PackageManager.PERMISSION_GRANTED + calendarPermissionGranted = + ContextCompat.checkSelfPermission(context, Manifest.permission.READ_CALENDAR) == + PackageManager.PERMISSION_GRANTED && + ContextCompat.checkSelfPermission(context, Manifest.permission.WRITE_CALENDAR) == + PackageManager.PERMISSION_GRANTED + motionPermissionGranted = + !motionPermissionRequired || + ContextCompat.checkSelfPermission(context, Manifest.permission.ACTIVITY_RECOGNITION) == + PackageManager.PERMISSION_GRANTED + appUpdateInstallEnabled = canInstallUnknownApps(context) + smsPermissionGranted = + ContextCompat.checkSelfPermission(context, Manifest.permission.SEND_SMS) == + PackageManager.PERMISSION_GRANTED + } + } + lifecycleOwner.lifecycle.addObserver(observer) + onDispose { lifecycleOwner.lifecycle.removeObserver(observer) } + } + fun setCameraEnabledChecked(checked: Boolean) { if (!checked) { viewModel.setCameraEnabled(false) @@ -268,324 +356,152 @@ fun SettingsSheet(viewModel: MainViewModel) { } } - val visibleGateways = - if (isConnected && remoteAddress != null) { - gateways.filterNot { "${it.host}:${it.port}" == remoteAddress } - } else { - gateways - } - - val gatewayDiscoveryFooterText = - if (visibleGateways.isEmpty()) { - discoveryStatusText - } else if (isConnected) { - "Discovery active • ${visibleGateways.size} other gateway${if (visibleGateways.size == 1) "" else "s"} found" - } else { - "Discovery active • ${visibleGateways.size} gateway${if (visibleGateways.size == 1) "" else "s"} found" - } - - LazyColumn( - state = listState, + Box( modifier = Modifier - .fillMaxWidth() - .fillMaxHeight() - .imePadding() - .windowInsetsPadding(WindowInsets.safeDrawing.only(WindowInsetsSides.Bottom)), - contentPadding = PaddingValues(16.dp), - verticalArrangement = Arrangement.spacedBy(6.dp), + .fillMaxSize() + .background(mobileBackgroundGradient), ) { - // Order parity: Node → Gateway → Voice → Camera → Messaging → Location → Screen. - item { Text("Node", style = MaterialTheme.typography.titleSmall) } + LazyColumn( + state = listState, + modifier = + Modifier + .fillMaxWidth() + .fillMaxHeight() + .imePadding() + .windowInsetsPadding(WindowInsets.safeDrawing.only(WindowInsetsSides.Bottom)), + contentPadding = PaddingValues(horizontal = 20.dp, vertical = 16.dp), + verticalArrangement = Arrangement.spacedBy(8.dp), + ) { + item { + Column(verticalArrangement = Arrangement.spacedBy(6.dp)) { + Text( + "SETTINGS", + style = mobileCaption1.copy(fontWeight = FontWeight.Bold, letterSpacing = 1.sp), + color = mobileAccent, + ) + Text("Device Configuration", style = mobileTitle2, color = mobileText) + Text( + "Manage capabilities, permissions, and diagnostics.", + style = mobileCallout, + color = mobileTextSecondary, + ) + } + } + item { HorizontalDivider(color = mobileBorder) } + + // Order parity: Node → Voice → Camera → Messaging → Location → Screen. + item { + Text( + "NODE", + style = mobileCaption1.copy(fontWeight = FontWeight.Bold, letterSpacing = 1.sp), + color = mobileAccent, + ) + } item { OutlinedTextField( value = displayName, onValueChange = viewModel::setDisplayName, - label = { Text("Name") }, + label = { Text("Name", style = mobileCaption1, color = mobileTextSecondary) }, modifier = Modifier.fillMaxWidth(), + textStyle = mobileBody.copy(color = mobileText), + colors = settingsTextFieldColors(), ) } - item { Text("Instance ID: $instanceId", color = MaterialTheme.colorScheme.onSurfaceVariant) } - item { Text("Device: $deviceModel", color = MaterialTheme.colorScheme.onSurfaceVariant) } - item { Text("Version: $appVersion", color = MaterialTheme.colorScheme.onSurfaceVariant) } + item { Text("Instance ID: $instanceId", style = mobileCallout.copy(fontFamily = FontFamily.Monospace), color = mobileTextSecondary) } + item { Text("Device: $deviceModel", style = mobileCallout, color = mobileTextSecondary) } + item { Text("Version: $appVersion", style = mobileCallout, color = mobileTextSecondary) } - item { HorizontalDivider() } + item { HorizontalDivider(color = mobileBorder) } - // Gateway - item { Text("Gateway", style = MaterialTheme.typography.titleSmall) } - item { ListItem(headlineContent = { Text("Status") }, supportingContent = { Text(statusText) }) } - if (serverName != null) { - item { ListItem(headlineContent = { Text("Server") }, supportingContent = { Text(serverName!!) }) } - } - if (remoteAddress != null) { - item { ListItem(headlineContent = { Text("Address") }, supportingContent = { Text(remoteAddress!!) }) } - } - item { - // UI sanity: "Disconnect" only when we have an active remote. - if (isConnected && remoteAddress != null) { - Button( - onClick = { - viewModel.disconnect() - NodeForegroundService.stop(context) - }, - ) { - Text("Disconnect") - } - } - } - - item { HorizontalDivider() } - - if (!isConnected || visibleGateways.isNotEmpty()) { + // Voice item { Text( - if (isConnected) "Other Gateways" else "Discovered Gateways", - style = MaterialTheme.typography.titleSmall, + "VOICE", + style = mobileCaption1.copy(fontWeight = FontWeight.Bold, letterSpacing = 1.sp), + color = mobileAccent, ) } - if (!isConnected && visibleGateways.isEmpty()) { - item { Text("No gateways found yet.", color = MaterialTheme.colorScheme.onSurfaceVariant) } - } else { - items(items = visibleGateways, key = { it.stableId }) { gateway -> - val detailLines = - buildList { - add("IP: ${gateway.host}:${gateway.port}") - gateway.lanHost?.let { add("LAN: $it") } - gateway.tailnetDns?.let { add("Tailnet: $it") } - if (gateway.gatewayPort != null || gateway.canvasPort != null) { - val gw = (gateway.gatewayPort ?: gateway.port).toString() - val canvas = gateway.canvasPort?.toString() ?: "—" - add("Ports: gw $gw · canvas $canvas") - } - } - ListItem( - headlineContent = { Text(gateway.name) }, - supportingContent = { - Column(verticalArrangement = Arrangement.spacedBy(2.dp)) { - detailLines.forEach { line -> - Text(line, color = MaterialTheme.colorScheme.onSurfaceVariant) - } - } - }, - trailingContent = { - Button( - onClick = { - NodeForegroundService.start(context) - viewModel.connect(gateway) - }, - ) { - Text("Connect") - } - }, - ) - } - } item { - Text( - gatewayDiscoveryFooterText, - modifier = Modifier.fillMaxWidth(), - textAlign = TextAlign.Center, - style = MaterialTheme.typography.labelMedium, - color = MaterialTheme.colorScheme.onSurfaceVariant, - ) - } - } - - item { HorizontalDivider() } - - item { - ListItem( - headlineContent = { Text("Advanced") }, - supportingContent = { Text("Manual gateway connection") }, - trailingContent = { - Icon( - imageVector = if (advancedExpanded) Icons.Filled.ExpandLess else Icons.Filled.ExpandMore, - contentDescription = if (advancedExpanded) "Collapse" else "Expand", - ) - }, - modifier = - Modifier.clickable { - setAdvancedExpanded(!advancedExpanded) - }, - ) - } - item { - AnimatedVisibility(visible = advancedExpanded) { - Column(verticalArrangement = Arrangement.spacedBy(10.dp), modifier = Modifier.fillMaxWidth()) { - ListItem( - headlineContent = { Text("Use Manual Gateway") }, - supportingContent = { Text("Use this when discovery is blocked.") }, - trailingContent = { Switch(checked = manualEnabled, onCheckedChange = viewModel::setManualEnabled) }, - ) - - OutlinedTextField( - value = manualHost, - onValueChange = viewModel::setManualHost, - label = { Text("Host") }, - modifier = Modifier.fillMaxWidth(), - enabled = manualEnabled, - ) - OutlinedTextField( - value = manualPort.toString(), - onValueChange = { v -> viewModel.setManualPort(v.toIntOrNull() ?: 0) }, - label = { Text("Port") }, - modifier = Modifier.fillMaxWidth(), - enabled = manualEnabled, - ) - OutlinedTextField( - value = gatewayToken, - onValueChange = viewModel::setGatewayToken, - label = { Text("Gateway Token") }, - modifier = Modifier.fillMaxWidth(), - enabled = manualEnabled, - singleLine = true, - ) - ListItem( - headlineContent = { Text("Require TLS") }, - supportingContent = { Text("Pin the gateway certificate on first connect.") }, - trailingContent = { Switch(checked = manualTls, onCheckedChange = viewModel::setManualTls, enabled = manualEnabled) }, - modifier = Modifier.alpha(if (manualEnabled) 1f else 0.5f), - ) - - val hostOk = manualHost.trim().isNotEmpty() - val portOk = manualPort in 1..65535 - Button( - onClick = { - NodeForegroundService.start(context) - viewModel.connectManual() - }, - enabled = manualEnabled && hostOk && portOk, - ) { - Text("Connect (Manual)") - } - } - } - } - - item { HorizontalDivider() } - - // Voice - item { Text("Voice", style = MaterialTheme.typography.titleSmall) } - item { - val enabled = voiceWakeMode != VoiceWakeMode.Off - ListItem( - headlineContent = { Text("Voice Wake") }, - supportingContent = { Text(voiceWakeStatusText) }, - trailingContent = { - Switch( - checked = enabled, - onCheckedChange = { on -> - if (on) { - val micOk = - ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) == - PackageManager.PERMISSION_GRANTED - if (!micOk) audioPermissionLauncher.launch(Manifest.permission.RECORD_AUDIO) - viewModel.setVoiceWakeMode(VoiceWakeMode.Foreground) + ListItem( + modifier = settingsRowModifier(), + colors = listItemColors, + headlineContent = { Text("Microphone permission", style = mobileHeadline) }, + supportingContent = { + Text( + if (micPermissionGranted) { + "Granted. Use the Voice tab mic button to capture transcript." } else { - viewModel.setVoiceWakeMode(VoiceWakeMode.Off) - } - }, - ) - }, - ) - } - item { - AnimatedVisibility(visible = voiceWakeMode != VoiceWakeMode.Off) { - Column(verticalArrangement = Arrangement.spacedBy(6.dp), modifier = Modifier.fillMaxWidth()) { - ListItem( - headlineContent = { Text("Foreground Only") }, - supportingContent = { Text("Listens only while OpenClaw is open.") }, - trailingContent = { - RadioButton( - selected = voiceWakeMode == VoiceWakeMode.Foreground, - onClick = { - val micOk = - ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) == - PackageManager.PERMISSION_GRANTED - if (!micOk) audioPermissionLauncher.launch(Manifest.permission.RECORD_AUDIO) - viewModel.setVoiceWakeMode(VoiceWakeMode.Foreground) - }, + "Required for Voice tab transcription." + }, + style = mobileCallout, + ) + }, + trailingContent = { + Button( + onClick = { + if (micPermissionGranted) { + openAppSettings(context) + } else { + audioPermissionLauncher.launch(Manifest.permission.RECORD_AUDIO) + } + }, + colors = settingsPrimaryButtonColors(), + shape = RoundedCornerShape(14.dp), + ) { + Text( + if (micPermissionGranted) "Manage" else "Grant", + style = mobileCallout.copy(fontWeight = FontWeight.Bold), ) - }, - ) - ListItem( - headlineContent = { Text("Always") }, - supportingContent = { Text("Keeps listening in the background (shows a persistent notification).") }, - trailingContent = { - RadioButton( - selected = voiceWakeMode == VoiceWakeMode.Always, - onClick = { - val micOk = - ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) == - PackageManager.PERMISSION_GRANTED - if (!micOk) audioPermissionLauncher.launch(Manifest.permission.RECORD_AUDIO) - viewModel.setVoiceWakeMode(VoiceWakeMode.Always) - }, - ) - }, - ) - } - } - } - item { - OutlinedTextField( - value = wakeWordsText, - onValueChange = setWakeWordsText, - label = { Text("Wake Words (comma-separated)") }, - modifier = - Modifier.fillMaxWidth().onFocusChanged { focusState -> - if (focusState.isFocused) { - wakeWordsHadFocus = true - } else if (wakeWordsHadFocus) { - wakeWordsHadFocus = false - commitWakeWords() } }, - singleLine = true, - keyboardOptions = KeyboardOptions(imeAction = ImeAction.Done), - keyboardActions = - KeyboardActions( - onDone = { - commitWakeWords() - focusManager.clearFocus() - }, - ), - ) - } - item { Button(onClick = viewModel::resetWakeWordsDefaults) { Text("Reset defaults") } } - item { - Text( - if (isConnected) { - "Any node can edit wake words. Changes sync via the gateway." - } else { - "Connect to a gateway to sync wake words globally." - }, - color = MaterialTheme.colorScheme.onSurfaceVariant, - ) - } + ) + } + item { + Text( + "Voice wake and talk modes were removed. Voice now uses one mic on/off flow in the Voice tab.", + style = mobileCallout, + color = mobileTextSecondary, + ) + } - item { HorizontalDivider() } + item { HorizontalDivider(color = mobileBorder) } // Camera - item { Text("Camera", style = MaterialTheme.typography.titleSmall) } + item { + Text( + "CAMERA", + style = mobileCaption1.copy(fontWeight = FontWeight.Bold, letterSpacing = 1.sp), + color = mobileAccent, + ) + } item { ListItem( - headlineContent = { Text("Allow Camera") }, - supportingContent = { Text("Allows the gateway to request photos or short video clips (foreground only).") }, + modifier = settingsRowModifier(), + colors = listItemColors, + headlineContent = { Text("Allow Camera", style = mobileHeadline) }, + supportingContent = { Text("Allows the gateway to request photos or short video clips (foreground only).", style = mobileCallout) }, trailingContent = { Switch(checked = cameraEnabled, onCheckedChange = ::setCameraEnabledChecked) }, ) } item { Text( "Tip: grant Microphone permission for video clips with audio.", - color = MaterialTheme.colorScheme.onSurfaceVariant, + style = mobileCallout, + color = mobileTextSecondary, ) } - item { HorizontalDivider() } + item { HorizontalDivider(color = mobileBorder) } // Messaging - item { Text("Messaging", style = MaterialTheme.typography.titleSmall) } + item { + Text( + "MESSAGING", + style = mobileCaption1.copy(fontWeight = FontWeight.Bold, letterSpacing = 1.sp), + color = mobileAccent, + ) + } item { val buttonLabel = when { @@ -594,7 +510,9 @@ fun SettingsSheet(viewModel: MainViewModel) { else -> "Grant" } ListItem( - headlineContent = { Text("SMS Permission") }, + modifier = settingsRowModifier(), + colors = listItemColors, + headlineContent = { Text("SMS Permission", style = mobileHeadline) }, supportingContent = { Text( if (smsPermissionAvailable) { @@ -602,6 +520,7 @@ fun SettingsSheet(viewModel: MainViewModel) { } else { "SMS requires a device with telephony hardware." }, + style = mobileCallout, ) }, trailingContent = { @@ -615,91 +534,373 @@ fun SettingsSheet(viewModel: MainViewModel) { } }, enabled = smsPermissionAvailable, + colors = settingsPrimaryButtonColors(), + shape = RoundedCornerShape(14.dp), ) { - Text(buttonLabel) + Text(buttonLabel, style = mobileCallout.copy(fontWeight = FontWeight.Bold)) } }, ) } - item { HorizontalDivider() } + item { HorizontalDivider(color = mobileBorder) } - // Location - item { Text("Location", style = MaterialTheme.typography.titleSmall) } - item { - Column(verticalArrangement = Arrangement.spacedBy(6.dp), modifier = Modifier.fillMaxWidth()) { + // Notifications + item { + Text( + "NOTIFICATIONS", + style = mobileCaption1.copy(fontWeight = FontWeight.Bold, letterSpacing = 1.sp), + color = mobileAccent, + ) + } + item { + val buttonLabel = + if (notificationsPermissionGranted) { + "Manage" + } else { + "Grant" + } ListItem( - headlineContent = { Text("Off") }, - supportingContent = { Text("Disable location sharing.") }, - trailingContent = { - RadioButton( - selected = locationMode == LocationMode.Off, - onClick = { viewModel.setLocationMode(LocationMode.Off) }, + modifier = settingsRowModifier(), + colors = listItemColors, + headlineContent = { Text("System Notifications", style = mobileHeadline) }, + supportingContent = { + Text( + "Required for `system.notify` and Android foreground service alerts.", + style = mobileCallout, ) }, - ) - ListItem( - headlineContent = { Text("While Using") }, - supportingContent = { Text("Only while OpenClaw is open.") }, trailingContent = { - RadioButton( - selected = locationMode == LocationMode.WhileUsing, - onClick = { requestLocationPermissions(LocationMode.WhileUsing) }, - ) - }, - ) - ListItem( - headlineContent = { Text("Always") }, - supportingContent = { Text("Allow background location (requires system permission).") }, - trailingContent = { - RadioButton( - selected = locationMode == LocationMode.Always, - onClick = { requestLocationPermissions(LocationMode.Always) }, - ) + Button( + onClick = { + if (notificationsPermissionGranted || Build.VERSION.SDK_INT < 33) { + openAppSettings(context) + } else { + notificationsPermissionLauncher.launch(Manifest.permission.POST_NOTIFICATIONS) + } + }, + colors = settingsPrimaryButtonColors(), + shape = RoundedCornerShape(14.dp), + ) { + Text(buttonLabel, style = mobileCallout.copy(fontWeight = FontWeight.Bold)) + } }, ) } - } - item { - ListItem( - headlineContent = { Text("Precise Location") }, - supportingContent = { Text("Use precise GPS when available.") }, - trailingContent = { - Switch( - checked = locationPreciseEnabled, - onCheckedChange = ::setPreciseLocationChecked, - enabled = locationMode != LocationMode.Off, + item { + ListItem( + modifier = settingsRowModifier(), + colors = listItemColors, + headlineContent = { Text("Notification Listener Access", style = mobileHeadline) }, + supportingContent = { + Text( + "Required for `notifications.list` and `notifications.actions`.", + style = mobileCallout, + ) + }, + trailingContent = { + Button( + onClick = { openNotificationListenerSettings(context) }, + colors = settingsPrimaryButtonColors(), + shape = RoundedCornerShape(14.dp), + ) { + Text( + if (notificationListenerEnabled) "Manage" else "Enable", + style = mobileCallout.copy(fontWeight = FontWeight.Bold), + ) + } + }, + ) + } + item { HorizontalDivider(color = mobileBorder) } + + // Data access + item { + Text( + "DATA ACCESS", + style = mobileCaption1.copy(fontWeight = FontWeight.Bold, letterSpacing = 1.sp), + color = mobileAccent, + ) + } + item { + ListItem( + modifier = settingsRowModifier(), + colors = listItemColors, + headlineContent = { Text("Photos Permission", style = mobileHeadline) }, + supportingContent = { + Text( + "Required for `photos.latest`.", + style = mobileCallout, + ) + }, + trailingContent = { + Button( + onClick = { + if (photosPermissionGranted) { + openAppSettings(context) + } else { + photosPermissionLauncher.launch(photosPermission) + } + }, + colors = settingsPrimaryButtonColors(), + shape = RoundedCornerShape(14.dp), + ) { + Text( + if (photosPermissionGranted) "Manage" else "Grant", + style = mobileCallout.copy(fontWeight = FontWeight.Bold), + ) + } + }, + ) + } + item { + ListItem( + modifier = settingsRowModifier(), + colors = listItemColors, + headlineContent = { Text("Contacts Permission", style = mobileHeadline) }, + supportingContent = { + Text( + "Required for `contacts.search` and `contacts.add`.", + style = mobileCallout, + ) + }, + trailingContent = { + Button( + onClick = { + if (contactsPermissionGranted) { + openAppSettings(context) + } else { + contactsPermissionLauncher.launch(arrayOf(Manifest.permission.READ_CONTACTS, Manifest.permission.WRITE_CONTACTS)) + } + }, + colors = settingsPrimaryButtonColors(), + shape = RoundedCornerShape(14.dp), + ) { + Text( + if (contactsPermissionGranted) "Manage" else "Grant", + style = mobileCallout.copy(fontWeight = FontWeight.Bold), + ) + } + }, + ) + } + item { + ListItem( + modifier = settingsRowModifier(), + colors = listItemColors, + headlineContent = { Text("Calendar Permission", style = mobileHeadline) }, + supportingContent = { + Text( + "Required for `calendar.events` and `calendar.add`.", + style = mobileCallout, + ) + }, + trailingContent = { + Button( + onClick = { + if (calendarPermissionGranted) { + openAppSettings(context) + } else { + calendarPermissionLauncher.launch(arrayOf(Manifest.permission.READ_CALENDAR, Manifest.permission.WRITE_CALENDAR)) + } + }, + colors = settingsPrimaryButtonColors(), + shape = RoundedCornerShape(14.dp), + ) { + Text( + if (calendarPermissionGranted) "Manage" else "Grant", + style = mobileCallout.copy(fontWeight = FontWeight.Bold), + ) + } + }, + ) + } + item { + val motionButtonLabel = + when { + !motionAvailable -> "Unavailable" + !motionPermissionRequired -> "Manage" + motionPermissionGranted -> "Manage" + else -> "Grant" + } + ListItem( + modifier = settingsRowModifier(), + colors = listItemColors, + headlineContent = { Text("Motion Permission", style = mobileHeadline) }, + supportingContent = { + Text( + if (!motionAvailable) { + "This device does not expose accelerometer or step-counter motion sensors." + } else { + "Required for `motion.activity` and `motion.pedometer`." + }, + style = mobileCallout, + ) + }, + trailingContent = { + Button( + onClick = { + if (!motionAvailable) return@Button + if (!motionPermissionRequired || motionPermissionGranted) { + openAppSettings(context) + } else { + motionPermissionLauncher.launch(Manifest.permission.ACTIVITY_RECOGNITION) + } + }, + enabled = motionAvailable, + colors = settingsPrimaryButtonColors(), + shape = RoundedCornerShape(14.dp), + ) { + Text(motionButtonLabel, style = mobileCallout.copy(fontWeight = FontWeight.Bold)) + } + }, + ) + } + item { HorizontalDivider(color = mobileBorder) } + + // System + item { + Text( + "SYSTEM", + style = mobileCaption1.copy(fontWeight = FontWeight.Bold, letterSpacing = 1.sp), + color = mobileAccent, + ) + } + item { + ListItem( + modifier = settingsRowModifier(), + colors = listItemColors, + headlineContent = { Text("Install App Updates", style = mobileHeadline) }, + supportingContent = { + Text( + "Enable install access for `app.update` package installs.", + style = mobileCallout, + ) + }, + trailingContent = { + Button( + onClick = { openUnknownAppSourcesSettings(context) }, + colors = settingsPrimaryButtonColors(), + shape = RoundedCornerShape(14.dp), + ) { + Text( + if (appUpdateInstallEnabled) "Manage" else "Enable", + style = mobileCallout.copy(fontWeight = FontWeight.Bold), + ) + } + }, + ) + } + item { HorizontalDivider(color = mobileBorder) } + + // Location + item { + Text( + "LOCATION", + style = mobileCaption1.copy(fontWeight = FontWeight.Bold, letterSpacing = 1.sp), + color = mobileAccent, + ) + } + item { + Column(modifier = settingsRowModifier(), verticalArrangement = Arrangement.spacedBy(0.dp)) { + ListItem( + modifier = Modifier.fillMaxWidth(), + colors = listItemColors, + headlineContent = { Text("Off", style = mobileHeadline) }, + supportingContent = { Text("Disable location sharing.", style = mobileCallout) }, + trailingContent = { + RadioButton( + selected = locationMode == LocationMode.Off, + onClick = { viewModel.setLocationMode(LocationMode.Off) }, + ) + }, ) - }, - ) - } + HorizontalDivider(color = mobileBorder) + ListItem( + modifier = Modifier.fillMaxWidth(), + colors = listItemColors, + headlineContent = { Text("While Using", style = mobileHeadline) }, + supportingContent = { Text("Only while OpenClaw is open.", style = mobileCallout) }, + trailingContent = { + RadioButton( + selected = locationMode == LocationMode.WhileUsing, + onClick = { requestLocationPermissions(LocationMode.WhileUsing) }, + ) + }, + ) + HorizontalDivider(color = mobileBorder) + ListItem( + modifier = Modifier.fillMaxWidth(), + colors = listItemColors, + headlineContent = { Text("Always", style = mobileHeadline) }, + supportingContent = { Text("Allow background location (requires system permission).", style = mobileCallout) }, + trailingContent = { + RadioButton( + selected = locationMode == LocationMode.Always, + onClick = { requestLocationPermissions(LocationMode.Always) }, + ) + }, + ) + HorizontalDivider(color = mobileBorder) + ListItem( + modifier = Modifier.fillMaxWidth(), + colors = listItemColors, + headlineContent = { Text("Precise Location", style = mobileHeadline) }, + supportingContent = { Text("Use precise GPS when available.", style = mobileCallout) }, + trailingContent = { + Switch( + checked = locationPreciseEnabled, + onCheckedChange = ::setPreciseLocationChecked, + enabled = locationMode != LocationMode.Off, + ) + }, + ) + } + } item { Text( "Always may require Android Settings to allow background location.", - color = MaterialTheme.colorScheme.onSurfaceVariant, + style = mobileCallout, + color = mobileTextSecondary, ) } - item { HorizontalDivider() } + item { HorizontalDivider(color = mobileBorder) } // Screen - item { Text("Screen", style = MaterialTheme.typography.titleSmall) } + item { + Text( + "SCREEN", + style = mobileCaption1.copy(fontWeight = FontWeight.Bold, letterSpacing = 1.sp), + color = mobileAccent, + ) + } item { ListItem( - headlineContent = { Text("Prevent Sleep") }, - supportingContent = { Text("Keeps the screen awake while OpenClaw is open.") }, + modifier = settingsRowModifier(), + colors = listItemColors, + headlineContent = { Text("Prevent Sleep", style = mobileHeadline) }, + supportingContent = { Text("Keeps the screen awake while OpenClaw is open.", style = mobileCallout) }, trailingContent = { Switch(checked = preventSleep, onCheckedChange = viewModel::setPreventSleep) }, ) } - item { HorizontalDivider() } + item { HorizontalDivider(color = mobileBorder) } // Debug - item { Text("Debug", style = MaterialTheme.typography.titleSmall) } + item { + Text( + "DEBUG", + style = mobileCaption1.copy(fontWeight = FontWeight.Bold, letterSpacing = 1.sp), + color = mobileAccent, + ) + } item { ListItem( - headlineContent = { Text("Debug Canvas Status") }, - supportingContent = { Text("Show status text in the canvas when debug is enabled.") }, + modifier = settingsRowModifier(), + colors = listItemColors, + headlineContent = { Text("Debug Canvas Status", style = mobileHeadline) }, + supportingContent = { Text("Show status text in the canvas when debug is enabled.", style = mobileCallout) }, trailingContent = { Switch( checked = canvasDebugStatusEnabled, @@ -709,10 +910,47 @@ fun SettingsSheet(viewModel: MainViewModel) { ) } - item { Spacer(modifier = Modifier.height(20.dp)) } + item { Spacer(modifier = Modifier.height(24.dp)) } + } } } +@Composable +private fun settingsTextFieldColors() = + OutlinedTextFieldDefaults.colors( + focusedContainerColor = mobileSurface, + unfocusedContainerColor = mobileSurface, + focusedBorderColor = mobileAccent, + unfocusedBorderColor = mobileBorder, + focusedTextColor = mobileText, + unfocusedTextColor = mobileText, + cursorColor = mobileAccent, + ) + +private fun settingsRowModifier() = + Modifier + .fillMaxWidth() + .border(width = 1.dp, color = mobileBorder, shape = RoundedCornerShape(14.dp)) + .background(Color.White, RoundedCornerShape(14.dp)) + +@Composable +private fun settingsPrimaryButtonColors() = + ButtonDefaults.buttonColors( + containerColor = mobileAccent, + contentColor = Color.White, + disabledContainerColor = mobileAccent.copy(alpha = 0.45f), + disabledContentColor = Color.White.copy(alpha = 0.9f), + ) + +@Composable +private fun settingsDangerButtonColors() = + ButtonDefaults.buttonColors( + containerColor = mobileDanger, + contentColor = Color.White, + disabledContainerColor = mobileDanger.copy(alpha = 0.45f), + disabledContentColor = Color.White.copy(alpha = 0.9f), + ) + private fun openAppSettings(context: Context) { val intent = Intent( @@ -721,3 +959,50 @@ private fun openAppSettings(context: Context) { ) context.startActivity(intent) } + +private fun openNotificationListenerSettings(context: Context) { + val intent = Intent(Settings.ACTION_NOTIFICATION_LISTENER_SETTINGS) + runCatching { + context.startActivity(intent) + }.getOrElse { + openAppSettings(context) + } +} + +private fun openUnknownAppSourcesSettings(context: Context) { + if (Build.VERSION.SDK_INT < 26) { + openAppSettings(context) + return + } + val intent = + Intent( + Settings.ACTION_MANAGE_UNKNOWN_APP_SOURCES, + Uri.parse("package:${context.packageName}"), + ) + runCatching { + context.startActivity(intent) + }.getOrElse { + openAppSettings(context) + } +} + +private fun hasNotificationsPermission(context: Context): Boolean { + if (Build.VERSION.SDK_INT < 33) return true + return ContextCompat.checkSelfPermission(context, Manifest.permission.POST_NOTIFICATIONS) == + PackageManager.PERMISSION_GRANTED +} + +private fun isNotificationListenerEnabled(context: Context): Boolean { + return DeviceNotificationListenerService.isAccessEnabled(context) +} + +private fun canInstallUnknownApps(context: Context): Boolean { + if (Build.VERSION.SDK_INT < 26) return true + return context.packageManager.canRequestPackageInstalls() +} + +private fun hasMotionCapabilities(context: Context): Boolean { + val sensorManager = context.getSystemService(SensorManager::class.java) ?: return false + return sensorManager.getDefaultSensor(Sensor.TYPE_ACCELEROMETER) != null || + sensorManager.getDefaultSensor(Sensor.TYPE_STEP_COUNTER) != null +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/ui/StatusPill.kt b/apps/android/app/src/main/java/ai/openclaw/android/ui/StatusPill.kt deleted file mode 100644 index d608fc38a7b..00000000000 --- a/apps/android/app/src/main/java/ai/openclaw/android/ui/StatusPill.kt +++ /dev/null @@ -1,114 +0,0 @@ -package ai.openclaw.android.ui - -import androidx.compose.foundation.layout.Arrangement -import androidx.compose.foundation.layout.Row -import androidx.compose.foundation.layout.Spacer -import androidx.compose.foundation.layout.height -import androidx.compose.foundation.layout.padding -import androidx.compose.foundation.layout.size -import androidx.compose.foundation.layout.width -import androidx.compose.foundation.shape.CircleShape -import androidx.compose.foundation.shape.RoundedCornerShape -import androidx.compose.material.icons.Icons -import androidx.compose.material.icons.filled.Mic -import androidx.compose.material.icons.filled.MicOff -import androidx.compose.material3.Icon -import androidx.compose.material3.MaterialTheme -import androidx.compose.material3.Surface -import androidx.compose.material3.Text -import androidx.compose.material3.VerticalDivider -import androidx.compose.runtime.Composable -import androidx.compose.ui.Alignment -import androidx.compose.ui.Modifier -import androidx.compose.ui.draw.alpha -import androidx.compose.ui.graphics.Color -import androidx.compose.ui.unit.dp - -@Composable -fun StatusPill( - gateway: GatewayState, - voiceEnabled: Boolean, - onClick: () -> Unit, - modifier: Modifier = Modifier, - activity: StatusActivity? = null, -) { - Surface( - onClick = onClick, - modifier = modifier, - shape = RoundedCornerShape(14.dp), - color = overlayContainerColor(), - tonalElevation = 3.dp, - shadowElevation = 0.dp, - ) { - Row( - modifier = Modifier.padding(horizontal = 12.dp, vertical = 8.dp), - horizontalArrangement = Arrangement.spacedBy(10.dp), - verticalAlignment = Alignment.CenterVertically, - ) { - Row(horizontalArrangement = Arrangement.spacedBy(8.dp), verticalAlignment = Alignment.CenterVertically) { - Surface( - modifier = Modifier.size(9.dp), - shape = CircleShape, - color = gateway.color, - ) {} - - Text( - text = gateway.title, - style = MaterialTheme.typography.labelLarge, - ) - } - - VerticalDivider( - modifier = Modifier.height(14.dp).alpha(0.35f), - color = MaterialTheme.colorScheme.onSurfaceVariant, - ) - - if (activity != null) { - Row( - horizontalArrangement = Arrangement.spacedBy(6.dp), - verticalAlignment = Alignment.CenterVertically, - ) { - Icon( - imageVector = activity.icon, - contentDescription = activity.contentDescription, - tint = activity.tint ?: overlayIconColor(), - modifier = Modifier.size(18.dp), - ) - Text( - text = activity.title, - style = MaterialTheme.typography.labelLarge, - maxLines = 1, - ) - } - } else { - Icon( - imageVector = if (voiceEnabled) Icons.Default.Mic else Icons.Default.MicOff, - contentDescription = if (voiceEnabled) "Voice enabled" else "Voice disabled", - tint = - if (voiceEnabled) { - overlayIconColor() - } else { - MaterialTheme.colorScheme.onSurfaceVariant - }, - modifier = Modifier.size(18.dp), - ) - } - - Spacer(modifier = Modifier.width(2.dp)) - } - } -} - -data class StatusActivity( - val title: String, - val icon: androidx.compose.ui.graphics.vector.ImageVector, - val contentDescription: String, - val tint: Color? = null, -) - -enum class GatewayState(val title: String, val color: Color) { - Connected("Connected", Color(0xFF2ECC71)), - Connecting("Connecting…", Color(0xFFF1C40F)), - Error("Error", Color(0xFFE74C3C)), - Disconnected("Offline", Color(0xFF9E9E9E)), -} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/ui/VoiceTabScreen.kt b/apps/android/app/src/main/java/ai/openclaw/android/ui/VoiceTabScreen.kt new file mode 100644 index 00000000000..7233135be83 --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/ui/VoiceTabScreen.kt @@ -0,0 +1,411 @@ +package ai.openclaw.android.ui + +import android.Manifest +import android.app.Activity +import android.content.Context +import android.content.ContextWrapper +import android.content.Intent +import android.content.pm.PackageManager +import android.net.Uri +import android.provider.Settings +import androidx.activity.compose.rememberLauncherForActivityResult +import androidx.activity.result.contract.ActivityResultContracts +import androidx.compose.foundation.BorderStroke +import androidx.compose.foundation.background +import androidx.compose.foundation.layout.Arrangement +import androidx.compose.foundation.layout.Box +import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.PaddingValues +import androidx.compose.foundation.layout.Row +import androidx.compose.foundation.layout.WindowInsets +import androidx.compose.foundation.layout.WindowInsetsSides +import androidx.compose.foundation.layout.fillMaxSize +import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.imePadding +import androidx.compose.foundation.layout.only +import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.layout.safeDrawing +import androidx.compose.foundation.layout.size +import androidx.compose.foundation.layout.windowInsetsPadding +import androidx.compose.foundation.lazy.LazyColumn +import androidx.compose.foundation.lazy.items +import androidx.compose.foundation.lazy.rememberLazyListState +import androidx.compose.foundation.shape.CircleShape +import androidx.compose.foundation.shape.RoundedCornerShape +import androidx.compose.material.icons.Icons +import androidx.compose.material.icons.filled.Mic +import androidx.compose.material.icons.filled.MicOff +import androidx.compose.material.icons.automirrored.filled.VolumeOff +import androidx.compose.material.icons.automirrored.filled.VolumeUp +import androidx.compose.material3.Button +import androidx.compose.material3.ButtonDefaults +import androidx.compose.material3.Icon +import androidx.compose.material3.IconButton +import androidx.compose.material3.IconButtonDefaults +import androidx.compose.material3.Surface +import androidx.compose.material3.Text +import androidx.compose.runtime.Composable +import androidx.compose.runtime.DisposableEffect +import androidx.compose.runtime.LaunchedEffect +import androidx.compose.runtime.collectAsState +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.setValue +import androidx.compose.ui.Alignment +import androidx.compose.ui.Modifier +import androidx.compose.ui.draw.alpha +import androidx.compose.ui.graphics.Color +import androidx.compose.ui.platform.LocalContext +import androidx.compose.ui.text.font.FontWeight +import androidx.compose.ui.text.style.TextAlign +import androidx.compose.ui.unit.dp +import androidx.compose.ui.unit.sp +import androidx.core.app.ActivityCompat +import androidx.core.content.ContextCompat +import androidx.lifecycle.Lifecycle +import androidx.lifecycle.LifecycleEventObserver +import androidx.lifecycle.compose.LocalLifecycleOwner +import ai.openclaw.android.MainViewModel +import ai.openclaw.android.voice.VoiceConversationEntry +import ai.openclaw.android.voice.VoiceConversationRole +import kotlin.math.max + +@Composable +fun VoiceTabScreen(viewModel: MainViewModel) { + val context = LocalContext.current + val lifecycleOwner = LocalLifecycleOwner.current + val activity = remember(context) { context.findActivity() } + val listState = rememberLazyListState() + + val gatewayStatus by viewModel.statusText.collectAsState() + val micEnabled by viewModel.micEnabled.collectAsState() + val speakerEnabled by viewModel.speakerEnabled.collectAsState() + val micLiveTranscript by viewModel.micLiveTranscript.collectAsState() + val micQueuedMessages by viewModel.micQueuedMessages.collectAsState() + val micConversation by viewModel.micConversation.collectAsState() + val micInputLevel by viewModel.micInputLevel.collectAsState() + val micIsSending by viewModel.micIsSending.collectAsState() + + val hasStreamingAssistant = micConversation.any { it.role == VoiceConversationRole.Assistant && it.isStreaming } + val showThinkingBubble = micIsSending && !hasStreamingAssistant + + var hasMicPermission by remember { mutableStateOf(context.hasRecordAudioPermission()) } + var pendingMicEnable by remember { mutableStateOf(false) } + + DisposableEffect(lifecycleOwner, context) { + val observer = + LifecycleEventObserver { _, event -> + if (event == Lifecycle.Event.ON_RESUME) { + hasMicPermission = context.hasRecordAudioPermission() + } + } + lifecycleOwner.lifecycle.addObserver(observer) + onDispose { lifecycleOwner.lifecycle.removeObserver(observer) } + } + + val requestMicPermission = + rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) { granted -> + hasMicPermission = granted + if (granted && pendingMicEnable) { + viewModel.setMicEnabled(true) + } + pendingMicEnable = false + } + + LaunchedEffect(micConversation.size, showThinkingBubble) { + val total = micConversation.size + if (showThinkingBubble) 1 else 0 + if (total > 0) { + listState.animateScrollToItem(total - 1) + } + } + + Column( + modifier = + Modifier + .fillMaxSize() + .background(mobileBackgroundGradient) + .imePadding() + .windowInsetsPadding(WindowInsets.safeDrawing.only(WindowInsetsSides.Bottom)) + .padding(horizontal = 20.dp, vertical = 14.dp), + verticalArrangement = Arrangement.spacedBy(10.dp), + ) { + LazyColumn( + state = listState, + modifier = Modifier.fillMaxWidth().weight(1f), + contentPadding = PaddingValues(vertical = 4.dp), + verticalArrangement = Arrangement.spacedBy(10.dp), + ) { + if (micConversation.isEmpty() && !showThinkingBubble) { + item { + Box( + modifier = Modifier.fillParentMaxHeight().fillMaxWidth(), + contentAlignment = Alignment.Center, + ) { + Column( + horizontalAlignment = Alignment.CenterHorizontally, + verticalArrangement = Arrangement.spacedBy(10.dp), + ) { + Icon( + imageVector = Icons.Default.Mic, + contentDescription = null, + modifier = Modifier.size(48.dp), + tint = mobileTextTertiary, + ) + Text( + "Tap the mic to start", + style = mobileHeadline, + color = mobileTextSecondary, + ) + Text( + "Each pause sends a turn automatically.", + style = mobileCallout, + color = mobileTextTertiary, + ) + } + } + } + } + + items(items = micConversation, key = { it.id }) { entry -> + VoiceTurnBubble(entry = entry) + } + + if (showThinkingBubble) { + item { + VoiceThinkingBubble() + } + } + } + + Column( + modifier = Modifier.fillMaxWidth(), + horizontalAlignment = Alignment.CenterHorizontally, + verticalArrangement = Arrangement.spacedBy(6.dp), + ) { + if (!micLiveTranscript.isNullOrBlank()) { + Surface( + modifier = Modifier.fillMaxWidth(), + shape = RoundedCornerShape(14.dp), + color = mobileAccentSoft, + border = BorderStroke(1.dp, mobileAccent.copy(alpha = 0.2f)), + ) { + Text( + micLiveTranscript!!.trim(), + modifier = Modifier.padding(horizontal = 12.dp, vertical = 10.dp), + style = mobileCallout, + color = mobileText, + ) + } + } + + // Mic button with input-reactive ring + speaker toggle + Row( + modifier = Modifier.fillMaxWidth(), + horizontalArrangement = Arrangement.Center, + verticalAlignment = Alignment.CenterVertically, + ) { + // Speaker toggle + IconButton( + onClick = { viewModel.setSpeakerEnabled(!speakerEnabled) }, + modifier = Modifier.size(48.dp), + colors = + IconButtonDefaults.iconButtonColors( + containerColor = if (speakerEnabled) mobileSurface else mobileDangerSoft, + ), + ) { + Icon( + imageVector = if (speakerEnabled) Icons.AutoMirrored.Filled.VolumeUp else Icons.AutoMirrored.Filled.VolumeOff, + contentDescription = if (speakerEnabled) "Mute speaker" else "Unmute speaker", + modifier = Modifier.size(22.dp), + tint = if (speakerEnabled) mobileTextSecondary else mobileDanger, + ) + } + + // Ring size = 68dp base + up to 22dp driven by mic input level. + // The outer Box is fixed at 90dp (max ring size) so the ring never shifts the button. + Box( + modifier = Modifier.padding(horizontal = 16.dp).size(90.dp), + contentAlignment = Alignment.Center, + ) { + if (micEnabled) { + val ringLevel = micInputLevel.coerceIn(0f, 1f) + val ringSize = 68.dp + (22.dp * max(ringLevel, 0.05f)) + Box( + modifier = + Modifier + .size(ringSize) + .background(mobileAccent.copy(alpha = 0.12f + 0.14f * ringLevel), CircleShape), + ) + } + Button( + onClick = { + if (micEnabled) { + viewModel.setMicEnabled(false) + return@Button + } + if (hasMicPermission) { + viewModel.setMicEnabled(true) + } else { + pendingMicEnable = true + requestMicPermission.launch(Manifest.permission.RECORD_AUDIO) + } + }, + shape = CircleShape, + contentPadding = PaddingValues(0.dp), + modifier = Modifier.size(60.dp), + colors = + ButtonDefaults.buttonColors( + containerColor = if (micEnabled) mobileDanger else mobileAccent, + contentColor = Color.White, + ), + ) { + Icon( + imageVector = if (micEnabled) Icons.Default.MicOff else Icons.Default.Mic, + contentDescription = if (micEnabled) "Turn microphone off" else "Turn microphone on", + modifier = Modifier.size(24.dp), + ) + } + } + + // Invisible spacer to balance the row (same size as speaker button) + Box(modifier = Modifier.size(48.dp)) + } + + // Status + labels + val queueCount = micQueuedMessages.size + val stateText = + when { + queueCount > 0 -> "$queueCount queued" + micIsSending -> "Sending" + micEnabled -> "Listening" + else -> "Mic off" + } + Text( + "$gatewayStatus · $stateText", + style = mobileCaption1, + color = mobileTextSecondary, + ) + + if (!hasMicPermission) { + val showRationale = + if (activity == null) { + false + } else { + ActivityCompat.shouldShowRequestPermissionRationale(activity, Manifest.permission.RECORD_AUDIO) + } + Text( + if (showRationale) { + "Microphone permission is required for voice mode." + } else { + "Microphone blocked. Open app settings to enable it." + }, + style = mobileCaption1, + color = mobileWarning, + textAlign = TextAlign.Center, + ) + Button( + onClick = { openAppSettings(context) }, + shape = RoundedCornerShape(12.dp), + colors = ButtonDefaults.buttonColors(containerColor = mobileSurfaceStrong, contentColor = mobileText), + ) { + Text("Open settings", style = mobileCallout.copy(fontWeight = FontWeight.SemiBold)) + } + } + } + } +} + +@Composable +private fun VoiceTurnBubble(entry: VoiceConversationEntry) { + val isUser = entry.role == VoiceConversationRole.User + Row( + modifier = Modifier.fillMaxWidth(), + horizontalArrangement = if (isUser) Arrangement.End else Arrangement.Start, + ) { + Surface( + modifier = Modifier.fillMaxWidth(0.90f), + shape = RoundedCornerShape(12.dp), + color = if (isUser) mobileAccentSoft else Color.White, + border = BorderStroke(1.dp, if (isUser) mobileAccent else mobileBorderStrong), + ) { + Column( + modifier = Modifier.fillMaxWidth().padding(horizontal = 11.dp, vertical = 8.dp), + verticalArrangement = Arrangement.spacedBy(3.dp), + ) { + Text( + if (isUser) "You" else "OpenClaw", + style = mobileCaption2.copy(fontWeight = FontWeight.SemiBold, letterSpacing = 0.6.sp), + color = if (isUser) mobileAccent else mobileTextSecondary, + ) + Text( + if (entry.isStreaming && entry.text.isBlank()) "Listening response…" else entry.text, + style = mobileCallout, + color = mobileText, + ) + } + } + } +} + +@Composable +private fun VoiceThinkingBubble() { + Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.Start) { + Surface( + modifier = Modifier.fillMaxWidth(0.68f), + shape = RoundedCornerShape(12.dp), + color = Color.White, + border = BorderStroke(1.dp, mobileBorderStrong), + ) { + Row( + modifier = Modifier.padding(horizontal = 11.dp, vertical = 8.dp), + horizontalArrangement = Arrangement.spacedBy(8.dp), + verticalAlignment = Alignment.CenterVertically, + ) { + ThinkingDots(color = mobileTextSecondary) + Text("OpenClaw is thinking…", style = mobileCallout, color = mobileTextSecondary) + } + } + } +} + +@Composable +private fun ThinkingDots(color: Color) { + Row(horizontalArrangement = Arrangement.spacedBy(5.dp), verticalAlignment = Alignment.CenterVertically) { + ThinkingDot(alpha = 0.38f, color = color) + ThinkingDot(alpha = 0.62f, color = color) + ThinkingDot(alpha = 0.90f, color = color) + } +} + +@Composable +private fun ThinkingDot(alpha: Float, color: Color) { + Surface( + modifier = Modifier.size(6.dp).alpha(alpha), + shape = CircleShape, + color = color, + ) {} +} + +private fun Context.hasRecordAudioPermission(): Boolean { + return ( + ContextCompat.checkSelfPermission(this, Manifest.permission.RECORD_AUDIO) == + PackageManager.PERMISSION_GRANTED + ) +} + +private fun Context.findActivity(): Activity? = + when (this) { + is Activity -> this + is ContextWrapper -> baseContext.findActivity() + else -> null + } + +private fun openAppSettings(context: Context) { + val intent = + Intent( + Settings.ACTION_APPLICATION_DETAILS_SETTINGS, + Uri.fromParts("package", context.packageName, null), + ) + context.startActivity(intent) +} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatComposer.kt b/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatComposer.kt index 07ba769697d..22099500ebf 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatComposer.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatComposer.kt @@ -1,31 +1,36 @@ package ai.openclaw.android.ui.chat +import androidx.compose.foundation.BorderStroke +import androidx.compose.foundation.horizontalScroll import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.PaddingValues import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.Spacer import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.height import androidx.compose.foundation.layout.padding import androidx.compose.foundation.layout.size import androidx.compose.foundation.layout.width import androidx.compose.foundation.rememberScrollState import androidx.compose.foundation.shape.RoundedCornerShape -import androidx.compose.foundation.horizontalScroll import androidx.compose.material.icons.Icons -import androidx.compose.material.icons.filled.ArrowUpward +import androidx.compose.material.icons.automirrored.filled.Send +import androidx.compose.material.icons.filled.ArrowDropDown import androidx.compose.material.icons.filled.AttachFile import androidx.compose.material.icons.filled.Refresh import androidx.compose.material.icons.filled.Stop +import androidx.compose.material3.Button import androidx.compose.material3.ButtonDefaults +import androidx.compose.material3.CircularProgressIndicator import androidx.compose.material3.DropdownMenu import androidx.compose.material3.DropdownMenuItem -import androidx.compose.material3.FilledTonalButton -import androidx.compose.material3.FilledTonalIconButton +import androidx.compose.material3.HorizontalDivider import androidx.compose.material3.Icon -import androidx.compose.material3.IconButtonDefaults import androidx.compose.material3.MaterialTheme import androidx.compose.material3.OutlinedTextField +import androidx.compose.material3.OutlinedTextFieldDefaults import androidx.compose.material3.Surface import androidx.compose.material3.Text import androidx.compose.runtime.Composable @@ -37,149 +42,168 @@ import androidx.compose.runtime.setValue import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier import androidx.compose.ui.graphics.Color +import androidx.compose.ui.text.font.FontWeight import androidx.compose.ui.text.style.TextOverflow import androidx.compose.ui.unit.dp -import ai.openclaw.android.chat.ChatSessionEntry +import androidx.compose.ui.unit.sp +import ai.openclaw.android.ui.mobileAccent +import ai.openclaw.android.ui.mobileAccentSoft +import ai.openclaw.android.ui.mobileBorder +import ai.openclaw.android.ui.mobileBorderStrong +import ai.openclaw.android.ui.mobileCallout +import ai.openclaw.android.ui.mobileCaption1 +import ai.openclaw.android.ui.mobileHeadline +import ai.openclaw.android.ui.mobileSurface +import ai.openclaw.android.ui.mobileText +import ai.openclaw.android.ui.mobileTextSecondary +import ai.openclaw.android.ui.mobileTextTertiary @Composable fun ChatComposer( - sessionKey: String, - sessions: List, - mainSessionKey: String, healthOk: Boolean, thinkingLevel: String, pendingRunCount: Int, - errorText: String?, attachments: List, onPickImages: () -> Unit, onRemoveAttachment: (id: String) -> Unit, onSetThinkingLevel: (level: String) -> Unit, - onSelectSession: (sessionKey: String) -> Unit, onRefresh: () -> Unit, onAbort: () -> Unit, onSend: (text: String) -> Unit, ) { var input by rememberSaveable { mutableStateOf("") } var showThinkingMenu by remember { mutableStateOf(false) } - var showSessionMenu by remember { mutableStateOf(false) } - - val sessionOptions = resolveSessionChoices(sessionKey, sessions, mainSessionKey = mainSessionKey) - val currentSessionLabel = friendlySessionName( - sessionOptions.firstOrNull { it.key == sessionKey }?.displayName ?: sessionKey - ) val canSend = pendingRunCount == 0 && (input.trim().isNotEmpty() || attachments.isNotEmpty()) && healthOk + val sendBusy = pendingRunCount > 0 - Surface( - shape = MaterialTheme.shapes.large, - color = MaterialTheme.colorScheme.surfaceContainer, - tonalElevation = 0.dp, - shadowElevation = 0.dp, - ) { - Column(modifier = Modifier.padding(10.dp), verticalArrangement = Arrangement.spacedBy(8.dp)) { - Row( - modifier = Modifier.fillMaxWidth().horizontalScroll(rememberScrollState()), - horizontalArrangement = Arrangement.spacedBy(8.dp), - verticalAlignment = Alignment.CenterVertically, - ) { - Box { - FilledTonalButton( - onClick = { showSessionMenu = true }, - contentPadding = ButtonDefaults.ContentPadding, + Column(modifier = Modifier.fillMaxWidth(), verticalArrangement = Arrangement.spacedBy(8.dp)) { + Row( + modifier = Modifier.fillMaxWidth(), + verticalAlignment = Alignment.CenterVertically, + horizontalArrangement = Arrangement.spacedBy(8.dp), + ) { + Box(modifier = Modifier.weight(1f)) { + Surface( + onClick = { showThinkingMenu = true }, + shape = RoundedCornerShape(14.dp), + color = mobileAccentSoft, + border = BorderStroke(1.dp, mobileBorderStrong), + ) { + Row( + modifier = Modifier.fillMaxWidth().padding(horizontal = 12.dp, vertical = 8.dp), + verticalAlignment = Alignment.CenterVertically, + horizontalArrangement = Arrangement.SpaceBetween, ) { - Text(currentSessionLabel, maxLines = 1, overflow = TextOverflow.Ellipsis) - } - - DropdownMenu(expanded = showSessionMenu, onDismissRequest = { showSessionMenu = false }) { - for (entry in sessionOptions) { - DropdownMenuItem( - text = { Text(friendlySessionName(entry.displayName ?: entry.key)) }, - onClick = { - onSelectSession(entry.key) - showSessionMenu = false - }, - trailingIcon = { - if (entry.key == sessionKey) { - Text("✓") - } else { - Spacer(modifier = Modifier.width(10.dp)) - } - }, - ) - } + Text( + text = "Thinking: ${thinkingLabel(thinkingLevel)}", + style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), + color = mobileText, + ) + Icon(Icons.Default.ArrowDropDown, contentDescription = "Select thinking level", tint = mobileTextSecondary) } } - Box { - FilledTonalButton( - onClick = { showThinkingMenu = true }, - contentPadding = ButtonDefaults.ContentPadding, - ) { - Text("🧠 ${thinkingLabel(thinkingLevel)}", maxLines = 1) - } - - DropdownMenu(expanded = showThinkingMenu, onDismissRequest = { showThinkingMenu = false }) { - ThinkingMenuItem("off", thinkingLevel, onSetThinkingLevel) { showThinkingMenu = false } - ThinkingMenuItem("low", thinkingLevel, onSetThinkingLevel) { showThinkingMenu = false } - ThinkingMenuItem("medium", thinkingLevel, onSetThinkingLevel) { showThinkingMenu = false } - ThinkingMenuItem("high", thinkingLevel, onSetThinkingLevel) { showThinkingMenu = false } - } - } - - FilledTonalIconButton(onClick = onRefresh, modifier = Modifier.size(42.dp)) { - Icon(Icons.Default.Refresh, contentDescription = "Refresh") - } - - FilledTonalIconButton(onClick = onPickImages, modifier = Modifier.size(42.dp)) { - Icon(Icons.Default.AttachFile, contentDescription = "Add image") + DropdownMenu(expanded = showThinkingMenu, onDismissRequest = { showThinkingMenu = false }) { + ThinkingMenuItem("off", thinkingLevel, onSetThinkingLevel) { showThinkingMenu = false } + ThinkingMenuItem("low", thinkingLevel, onSetThinkingLevel) { showThinkingMenu = false } + ThinkingMenuItem("medium", thinkingLevel, onSetThinkingLevel) { showThinkingMenu = false } + ThinkingMenuItem("high", thinkingLevel, onSetThinkingLevel) { showThinkingMenu = false } } } - if (attachments.isNotEmpty()) { - AttachmentsStrip(attachments = attachments, onRemoveAttachment = onRemoveAttachment) - } - - OutlinedTextField( - value = input, - onValueChange = { input = it }, - modifier = Modifier.fillMaxWidth(), - placeholder = { Text("Message OpenClaw…") }, - minLines = 2, - maxLines = 6, + SecondaryActionButton( + label = "Attach", + icon = Icons.Default.AttachFile, + enabled = true, + onClick = onPickImages, ) + } - Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) { - ConnectionPill(sessionLabel = currentSessionLabel, healthOk = healthOk) - Spacer(modifier = Modifier.weight(1f)) + if (attachments.isNotEmpty()) { + AttachmentsStrip(attachments = attachments, onRemoveAttachment = onRemoveAttachment) + } - if (pendingRunCount > 0) { - FilledTonalIconButton( - onClick = onAbort, - colors = - IconButtonDefaults.filledTonalIconButtonColors( - containerColor = Color(0x33E74C3C), - contentColor = Color(0xFFE74C3C), - ), - ) { - Icon(Icons.Default.Stop, contentDescription = "Abort") - } - } else { - FilledTonalIconButton(onClick = { - val text = input - input = "" - onSend(text) - }, enabled = canSend) { - Icon(Icons.Default.ArrowUpward, contentDescription = "Send") - } - } + HorizontalDivider(color = mobileBorder) + + Text( + text = "MESSAGE", + style = mobileCaption1.copy(fontWeight = FontWeight.Bold, letterSpacing = 0.9.sp), + color = mobileTextSecondary, + ) + + OutlinedTextField( + value = input, + onValueChange = { input = it }, + modifier = Modifier.fillMaxWidth().height(92.dp), + placeholder = { Text("Type a message", style = mobileBodyStyle(), color = mobileTextTertiary) }, + minLines = 2, + maxLines = 5, + textStyle = mobileBodyStyle().copy(color = mobileText), + shape = RoundedCornerShape(14.dp), + colors = chatTextFieldColors(), + ) + + if (!healthOk) { + Text( + text = "Gateway is offline. Connect first in the Connect tab.", + style = mobileCallout, + color = ai.openclaw.android.ui.mobileWarning, + ) + } + + Row( + modifier = Modifier.fillMaxWidth(), + verticalAlignment = Alignment.CenterVertically, + horizontalArrangement = Arrangement.spacedBy(10.dp), + ) { + Row(horizontalArrangement = Arrangement.spacedBy(8.dp)) { + SecondaryActionButton( + label = "Refresh", + icon = Icons.Default.Refresh, + enabled = true, + compact = true, + onClick = onRefresh, + ) + + SecondaryActionButton( + label = "Abort", + icon = Icons.Default.Stop, + enabled = pendingRunCount > 0, + compact = true, + onClick = onAbort, + ) } - if (!errorText.isNullOrBlank()) { + Button( + onClick = { + val text = input + input = "" + onSend(text) + }, + enabled = canSend, + modifier = Modifier.weight(1f).height(48.dp), + shape = RoundedCornerShape(14.dp), + colors = + ButtonDefaults.buttonColors( + containerColor = mobileAccent, + contentColor = Color.White, + disabledContainerColor = mobileBorderStrong, + disabledContentColor = mobileTextTertiary, + ), + border = BorderStroke(1.dp, if (canSend) Color(0xFF154CAD) else mobileBorderStrong), + ) { + if (sendBusy) { + CircularProgressIndicator(modifier = Modifier.size(16.dp), strokeWidth = 2.dp, color = Color.White) + } else { + Icon(Icons.AutoMirrored.Filled.Send, contentDescription = null, modifier = Modifier.size(16.dp)) + } + Spacer(modifier = Modifier.width(8.dp)) Text( - text = errorText, - style = MaterialTheme.typography.bodySmall, - color = MaterialTheme.colorScheme.error, - maxLines = 2, + text = "Send", + style = mobileHeadline.copy(fontWeight = FontWeight.Bold), + maxLines = 1, + overflow = TextOverflow.Ellipsis, ) } } @@ -187,26 +211,35 @@ fun ChatComposer( } @Composable -private fun ConnectionPill(sessionLabel: String, healthOk: Boolean) { - Surface( - shape = RoundedCornerShape(999.dp), - color = MaterialTheme.colorScheme.surfaceContainerHighest, +private fun SecondaryActionButton( + label: String, + icon: androidx.compose.ui.graphics.vector.ImageVector, + enabled: Boolean, + compact: Boolean = false, + onClick: () -> Unit, +) { + Button( + onClick = onClick, + enabled = enabled, + modifier = if (compact) Modifier.size(44.dp) else Modifier.height(44.dp), + shape = RoundedCornerShape(14.dp), + colors = + ButtonDefaults.buttonColors( + containerColor = Color.White, + contentColor = mobileTextSecondary, + disabledContainerColor = Color.White, + disabledContentColor = mobileTextTertiary, + ), + border = BorderStroke(1.dp, mobileBorderStrong), + contentPadding = if (compact) PaddingValues(0.dp) else ButtonDefaults.ContentPadding, ) { - Row( - modifier = Modifier.padding(horizontal = 10.dp, vertical = 6.dp), - horizontalArrangement = Arrangement.spacedBy(8.dp), - verticalAlignment = Alignment.CenterVertically, - ) { - Surface( - modifier = Modifier.size(7.dp), - shape = androidx.compose.foundation.shape.CircleShape, - color = if (healthOk) Color(0xFF2ECC71) else Color(0xFFF39C12), - ) {} - Text(sessionLabel, style = MaterialTheme.typography.labelSmall) + Icon(icon, contentDescription = label, modifier = Modifier.size(14.dp)) + if (!compact) { + Spacer(modifier = Modifier.width(5.dp)) Text( - if (healthOk) "Connected" else "Connecting…", - style = MaterialTheme.typography.labelSmall, - color = MaterialTheme.colorScheme.onSurfaceVariant, + text = label, + style = mobileCallout.copy(fontWeight = FontWeight.SemiBold), + color = if (enabled) mobileTextSecondary else mobileTextTertiary, ) } } @@ -220,14 +253,14 @@ private fun ThinkingMenuItem( onDismiss: () -> Unit, ) { DropdownMenuItem( - text = { Text(thinkingLabel(value)) }, + text = { Text(thinkingLabel(value), style = mobileCallout, color = mobileText) }, onClick = { onSet(value) onDismiss() }, trailingIcon = { if (value == current.trim().lowercase()) { - Text("✓") + Text("✓", style = mobileCallout, color = mobileAccent) } else { Spacer(modifier = Modifier.width(10.dp)) } @@ -266,20 +299,55 @@ private fun AttachmentsStrip( private fun AttachmentChip(fileName: String, onRemove: () -> Unit) { Surface( shape = RoundedCornerShape(999.dp), - color = MaterialTheme.colorScheme.primary.copy(alpha = 0.10f), + color = mobileAccentSoft, + border = BorderStroke(1.dp, mobileBorderStrong), ) { Row( modifier = Modifier.padding(horizontal = 10.dp, vertical = 6.dp), verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(8.dp), ) { - Text(text = fileName, style = MaterialTheme.typography.bodySmall, maxLines = 1) - FilledTonalIconButton( + Text( + text = fileName, + style = mobileCaption1, + color = mobileText, + maxLines = 1, + overflow = TextOverflow.Ellipsis, + ) + Surface( onClick = onRemove, - modifier = Modifier.size(30.dp), + shape = RoundedCornerShape(999.dp), + color = Color.White, + border = BorderStroke(1.dp, mobileBorderStrong), ) { - Text("×") + Text( + text = "×", + style = mobileCaption1.copy(fontWeight = FontWeight.Bold), + color = mobileTextSecondary, + modifier = Modifier.padding(horizontal = 8.dp, vertical = 2.dp), + ) } } } } + +@Composable +private fun chatTextFieldColors() = + OutlinedTextFieldDefaults.colors( + focusedContainerColor = mobileSurface, + unfocusedContainerColor = mobileSurface, + focusedBorderColor = mobileAccent, + unfocusedBorderColor = mobileBorder, + focusedTextColor = mobileText, + unfocusedTextColor = mobileText, + cursorColor = mobileAccent, + ) + +@Composable +private fun mobileBodyStyle() = + MaterialTheme.typography.bodyMedium.copy( + fontFamily = ai.openclaw.android.ui.mobileFontFamily, + fontWeight = FontWeight.Medium, + fontSize = 15.sp, + lineHeight = 22.sp, + ) diff --git a/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatMarkdown.kt b/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatMarkdown.kt index 77dba2275a4..e121212529a 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatMarkdown.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatMarkdown.kt @@ -3,12 +3,21 @@ package ai.openclaw.android.ui.chat import android.graphics.BitmapFactory import android.util.Base64 import androidx.compose.foundation.Image +import androidx.compose.foundation.background +import androidx.compose.foundation.border +import androidx.compose.foundation.horizontalScroll import androidx.compose.foundation.layout.Arrangement +import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.IntrinsicSize +import androidx.compose.foundation.layout.Row +import androidx.compose.foundation.layout.fillMaxHeight import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.height import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.layout.width +import androidx.compose.foundation.rememberScrollState import androidx.compose.foundation.text.selection.SelectionContainer -import androidx.compose.material3.MaterialTheme import androidx.compose.material3.Text import androidx.compose.runtime.Composable import androidx.compose.runtime.LaunchedEffect @@ -16,167 +25,534 @@ import androidx.compose.runtime.getValue import androidx.compose.runtime.mutableStateOf import androidx.compose.runtime.remember import androidx.compose.runtime.setValue +import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier import androidx.compose.ui.graphics.Color import androidx.compose.ui.graphics.asImageBitmap import androidx.compose.ui.layout.ContentScale import androidx.compose.ui.text.AnnotatedString import androidx.compose.ui.text.SpanStyle +import androidx.compose.ui.text.TextStyle import androidx.compose.ui.text.buildAnnotatedString import androidx.compose.ui.text.font.FontFamily import androidx.compose.ui.text.font.FontStyle import androidx.compose.ui.text.font.FontWeight +import androidx.compose.ui.text.style.TextDecoration import androidx.compose.ui.text.withStyle import androidx.compose.ui.unit.dp +import androidx.compose.ui.unit.sp +import ai.openclaw.android.ui.mobileAccent +import ai.openclaw.android.ui.mobileCallout +import ai.openclaw.android.ui.mobileCaption1 +import ai.openclaw.android.ui.mobileCodeBg +import ai.openclaw.android.ui.mobileCodeText +import ai.openclaw.android.ui.mobileTextSecondary import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.withContext +import org.commonmark.Extension +import org.commonmark.ext.autolink.AutolinkExtension +import org.commonmark.ext.gfm.strikethrough.Strikethrough +import org.commonmark.ext.gfm.strikethrough.StrikethroughExtension +import org.commonmark.ext.gfm.tables.TableBlock +import org.commonmark.ext.gfm.tables.TableBody +import org.commonmark.ext.gfm.tables.TableCell +import org.commonmark.ext.gfm.tables.TableHead +import org.commonmark.ext.gfm.tables.TableRow +import org.commonmark.ext.gfm.tables.TablesExtension +import org.commonmark.ext.task.list.items.TaskListItemMarker +import org.commonmark.ext.task.list.items.TaskListItemsExtension +import org.commonmark.node.BlockQuote +import org.commonmark.node.BulletList +import org.commonmark.node.Code +import org.commonmark.node.Document +import org.commonmark.node.Emphasis +import org.commonmark.node.FencedCodeBlock +import org.commonmark.node.Heading +import org.commonmark.node.HardLineBreak +import org.commonmark.node.HtmlBlock +import org.commonmark.node.HtmlInline +import org.commonmark.node.Image as MarkdownImage +import org.commonmark.node.IndentedCodeBlock +import org.commonmark.node.Link +import org.commonmark.node.ListItem +import org.commonmark.node.Node +import org.commonmark.node.OrderedList +import org.commonmark.node.Paragraph +import org.commonmark.node.SoftLineBreak +import org.commonmark.node.StrongEmphasis +import org.commonmark.node.Text as MarkdownTextNode +import org.commonmark.node.ThematicBreak +import org.commonmark.parser.Parser + +private const val LIST_INDENT_DP = 14 +private val dataImageRegex = Regex("^data:image/([a-zA-Z0-9+.-]+);base64,([A-Za-z0-9+/=\\n\\r]+)$") + +private val markdownParser: Parser by lazy { + val extensions: List = + listOf( + AutolinkExtension.create(), + StrikethroughExtension.create(), + TablesExtension.create(), + TaskListItemsExtension.create(), + ) + Parser.builder() + .extensions(extensions) + .build() +} @Composable fun ChatMarkdown(text: String, textColor: Color) { - val blocks = remember(text) { splitMarkdown(text) } - val inlineCodeBg = MaterialTheme.colorScheme.surfaceContainerLow + val document = remember(text) { markdownParser.parse(text) as Document } + val inlineStyles = InlineStyles(inlineCodeBg = mobileCodeBg, inlineCodeColor = mobileCodeText) Column(verticalArrangement = Arrangement.spacedBy(10.dp)) { - for (b in blocks) { - when (b) { - is ChatMarkdownBlock.Text -> { - val trimmed = b.text.trimEnd() - if (trimmed.isEmpty()) continue + RenderMarkdownBlocks( + start = document.firstChild, + textColor = textColor, + inlineStyles = inlineStyles, + listDepth = 0, + ) + } +} + +@Composable +private fun RenderMarkdownBlocks( + start: Node?, + textColor: Color, + inlineStyles: InlineStyles, + listDepth: Int, +) { + var node = start + while (node != null) { + val current = node + when (current) { + is Paragraph -> { + RenderParagraph(current, textColor = textColor, inlineStyles = inlineStyles) + } + is Heading -> { + val headingText = remember(current) { buildInlineMarkdown(current.firstChild, inlineStyles) } + Text( + text = headingText, + style = headingStyle(current.level), + color = textColor, + ) + } + is FencedCodeBlock -> { + SelectionContainer(modifier = Modifier.fillMaxWidth()) { + ChatCodeBlock(code = current.literal.orEmpty(), language = current.info?.trim()?.ifEmpty { null }) + } + } + is IndentedCodeBlock -> { + SelectionContainer(modifier = Modifier.fillMaxWidth()) { + ChatCodeBlock(code = current.literal.orEmpty(), language = null) + } + } + is BlockQuote -> { + Row( + modifier = Modifier + .fillMaxWidth() + .height(IntrinsicSize.Min) + .padding(vertical = 2.dp), + horizontalArrangement = Arrangement.spacedBy(8.dp), + verticalAlignment = Alignment.Top, + ) { + Box( + modifier = Modifier + .width(2.dp) + .fillMaxHeight() + .background(mobileTextSecondary.copy(alpha = 0.35f)), + ) + Column( + modifier = Modifier.weight(1f), + verticalArrangement = Arrangement.spacedBy(8.dp), + ) { + RenderMarkdownBlocks( + start = current.firstChild, + textColor = textColor, + inlineStyles = inlineStyles, + listDepth = listDepth, + ) + } + } + } + is BulletList -> { + RenderBulletList( + list = current, + textColor = textColor, + inlineStyles = inlineStyles, + listDepth = listDepth, + ) + } + is OrderedList -> { + RenderOrderedList( + list = current, + textColor = textColor, + inlineStyles = inlineStyles, + listDepth = listDepth, + ) + } + is TableBlock -> { + RenderTableBlock( + table = current, + textColor = textColor, + inlineStyles = inlineStyles, + ) + } + is ThematicBreak -> { + Box( + modifier = Modifier + .fillMaxWidth() + .height(1.dp) + .background(mobileTextSecondary.copy(alpha = 0.25f)), + ) + } + is HtmlBlock -> { + val literal = current.literal.orEmpty().trim() + if (literal.isNotEmpty()) { Text( - text = parseInlineMarkdown(trimmed, inlineCodeBg = inlineCodeBg), - style = MaterialTheme.typography.bodyMedium, + text = literal, + style = mobileCallout.copy(fontFamily = FontFamily.Monospace), color = textColor, ) } - is ChatMarkdownBlock.Code -> { - SelectionContainer(modifier = Modifier.fillMaxWidth()) { - ChatCodeBlock(code = b.code, language = b.language) - } - } - is ChatMarkdownBlock.InlineImage -> { - InlineBase64Image(base64 = b.base64, mimeType = b.mimeType) + } + } + node = current.next + } +} + +@Composable +private fun RenderParagraph( + paragraph: Paragraph, + textColor: Color, + inlineStyles: InlineStyles, +) { + val standaloneImage = remember(paragraph) { standaloneDataImage(paragraph) } + if (standaloneImage != null) { + InlineBase64Image(base64 = standaloneImage.base64, mimeType = standaloneImage.mimeType) + return + } + + val annotated = remember(paragraph) { buildInlineMarkdown(paragraph.firstChild, inlineStyles) } + if (annotated.text.trimEnd().isEmpty()) { + return + } + + Text( + text = annotated, + style = mobileCallout, + color = textColor, + ) +} + +@Composable +private fun RenderBulletList( + list: BulletList, + textColor: Color, + inlineStyles: InlineStyles, + listDepth: Int, +) { + Column( + modifier = Modifier.padding(start = (LIST_INDENT_DP * listDepth).dp), + verticalArrangement = Arrangement.spacedBy(6.dp), + ) { + var item = list.firstChild + while (item != null) { + if (item is ListItem) { + RenderListItem( + item = item, + markerText = "•", + textColor = textColor, + inlineStyles = inlineStyles, + listDepth = listDepth, + ) + } + item = item.next + } + } +} + +@Composable +private fun RenderOrderedList( + list: OrderedList, + textColor: Color, + inlineStyles: InlineStyles, + listDepth: Int, +) { + Column( + modifier = Modifier.padding(start = (LIST_INDENT_DP * listDepth).dp), + verticalArrangement = Arrangement.spacedBy(6.dp), + ) { + var index = list.markerStartNumber ?: 1 + var item = list.firstChild + while (item != null) { + if (item is ListItem) { + RenderListItem( + item = item, + markerText = "$index.", + textColor = textColor, + inlineStyles = inlineStyles, + listDepth = listDepth, + ) + index += 1 + } + item = item.next + } + } +} + +@Composable +private fun RenderListItem( + item: ListItem, + markerText: String, + textColor: Color, + inlineStyles: InlineStyles, + listDepth: Int, +) { + var contentStart = item.firstChild + var marker = markerText + val task = contentStart as? TaskListItemMarker + if (task != null) { + marker = if (task.isChecked) "☑" else "☐" + contentStart = task.next + } + + Row( + modifier = Modifier.fillMaxWidth(), + horizontalArrangement = Arrangement.spacedBy(8.dp), + verticalAlignment = Alignment.Top, + ) { + Text( + text = marker, + style = mobileCallout.copy(fontWeight = FontWeight.SemiBold), + color = textColor, + modifier = Modifier.width(24.dp), + ) + + Column( + modifier = Modifier.weight(1f), + verticalArrangement = Arrangement.spacedBy(4.dp), + ) { + RenderMarkdownBlocks( + start = contentStart, + textColor = textColor, + inlineStyles = inlineStyles, + listDepth = listDepth + 1, + ) + } + } +} + +@Composable +private fun RenderTableBlock( + table: TableBlock, + textColor: Color, + inlineStyles: InlineStyles, +) { + val rows = remember(table) { buildTableRows(table, inlineStyles) } + if (rows.isEmpty()) return + + val maxCols = rows.maxOf { row -> row.cells.size }.coerceAtLeast(1) + val scrollState = rememberScrollState() + + Column( + modifier = Modifier + .fillMaxWidth() + .horizontalScroll(scrollState) + .border(1.dp, mobileTextSecondary.copy(alpha = 0.25f)), + ) { + for (row in rows) { + Row( + modifier = Modifier.fillMaxWidth(), + ) { + for (index in 0 until maxCols) { + val cell = row.cells.getOrNull(index) ?: AnnotatedString("") + Text( + text = cell, + style = if (row.isHeader) mobileCaption1.copy(fontWeight = FontWeight.SemiBold) else mobileCallout, + color = textColor, + modifier = Modifier + .border(1.dp, mobileTextSecondary.copy(alpha = 0.22f)) + .padding(horizontal = 8.dp, vertical = 6.dp) + .width(160.dp), + ) } } } } } -private sealed interface ChatMarkdownBlock { - data class Text(val text: String) : ChatMarkdownBlock - data class Code(val code: String, val language: String?) : ChatMarkdownBlock - data class InlineImage(val mimeType: String?, val base64: String) : ChatMarkdownBlock -} - -private fun splitMarkdown(raw: String): List { - if (raw.isEmpty()) return emptyList() - - val out = ArrayList() - var idx = 0 - while (idx < raw.length) { - val fenceStart = raw.indexOf("```", startIndex = idx) - if (fenceStart < 0) { - out.addAll(splitInlineImages(raw.substring(idx))) - break +private fun buildTableRows(table: TableBlock, inlineStyles: InlineStyles): List { + val rows = mutableListOf() + var child = table.firstChild + while (child != null) { + when (child) { + is TableHead -> rows.addAll(readTableSection(child, isHeader = true, inlineStyles = inlineStyles)) + is TableBody -> rows.addAll(readTableSection(child, isHeader = false, inlineStyles = inlineStyles)) + is TableRow -> rows.add(readTableRow(child, isHeader = false, inlineStyles = inlineStyles)) } - - if (fenceStart > idx) { - out.addAll(splitInlineImages(raw.substring(idx, fenceStart))) - } - - val langLineStart = fenceStart + 3 - val langLineEnd = raw.indexOf('\n', startIndex = langLineStart).let { if (it < 0) raw.length else it } - val language = raw.substring(langLineStart, langLineEnd).trim().ifEmpty { null } - - val codeStart = if (langLineEnd < raw.length && raw[langLineEnd] == '\n') langLineEnd + 1 else langLineEnd - val fenceEnd = raw.indexOf("```", startIndex = codeStart) - if (fenceEnd < 0) { - out.addAll(splitInlineImages(raw.substring(fenceStart))) - break - } - val code = raw.substring(codeStart, fenceEnd) - out.add(ChatMarkdownBlock.Code(code = code, language = language)) - - idx = fenceEnd + 3 + child = child.next } - - return out + return rows } -private fun splitInlineImages(text: String): List { - if (text.isEmpty()) return emptyList() - val regex = Regex("data:image/([a-zA-Z0-9+.-]+);base64,([A-Za-z0-9+/=\\n\\r]+)") - val out = ArrayList() - - var idx = 0 - while (idx < text.length) { - val m = regex.find(text, startIndex = idx) ?: break - val start = m.range.first - val end = m.range.last + 1 - if (start > idx) out.add(ChatMarkdownBlock.Text(text.substring(idx, start))) - - val mime = "image/" + (m.groupValues.getOrNull(1)?.trim()?.ifEmpty { "png" } ?: "png") - val b64 = m.groupValues.getOrNull(2)?.replace("\n", "")?.replace("\r", "")?.trim().orEmpty() - if (b64.isNotEmpty()) { - out.add(ChatMarkdownBlock.InlineImage(mimeType = mime, base64 = b64)) +private fun readTableSection(section: Node, isHeader: Boolean, inlineStyles: InlineStyles): List { + val rows = mutableListOf() + var row = section.firstChild + while (row != null) { + if (row is TableRow) { + rows.add(readTableRow(row, isHeader = isHeader, inlineStyles = inlineStyles)) } - idx = end + row = row.next } - - if (idx < text.length) out.add(ChatMarkdownBlock.Text(text.substring(idx))) - return out + return rows } -private fun parseInlineMarkdown(text: String, inlineCodeBg: androidx.compose.ui.graphics.Color): AnnotatedString { - if (text.isEmpty()) return AnnotatedString("") +private fun readTableRow(row: TableRow, isHeader: Boolean, inlineStyles: InlineStyles): TableRenderRow { + val cells = mutableListOf() + var cellNode = row.firstChild + while (cellNode != null) { + if (cellNode is TableCell) { + cells.add(buildInlineMarkdown(cellNode.firstChild, inlineStyles)) + } + cellNode = cellNode.next + } + return TableRenderRow(isHeader = isHeader, cells = cells) +} - val out = buildAnnotatedString { - var i = 0 - while (i < text.length) { - if (text.startsWith("**", startIndex = i)) { - val end = text.indexOf("**", startIndex = i + 2) - if (end > i + 2) { - withStyle(SpanStyle(fontWeight = FontWeight.SemiBold)) { - append(text.substring(i + 2, end)) - } - i = end + 2 - continue +private fun buildInlineMarkdown(start: Node?, inlineStyles: InlineStyles): AnnotatedString { + return buildAnnotatedString { + appendInlineNode( + node = start, + inlineCodeBg = inlineStyles.inlineCodeBg, + inlineCodeColor = inlineStyles.inlineCodeColor, + ) + } +} + +private fun AnnotatedString.Builder.appendInlineNode( + node: Node?, + inlineCodeBg: Color, + inlineCodeColor: Color, +) { + var current = node + while (current != null) { + when (current) { + is MarkdownTextNode -> append(current.literal) + is SoftLineBreak -> append('\n') + is HardLineBreak -> append('\n') + is Code -> { + withStyle( + SpanStyle( + fontFamily = FontFamily.Monospace, + background = inlineCodeBg, + color = inlineCodeColor, + ), + ) { + append(current.literal) } } - - if (text[i] == '`') { - val end = text.indexOf('`', startIndex = i + 1) - if (end > i + 1) { - withStyle( - SpanStyle( - fontFamily = FontFamily.Monospace, - background = inlineCodeBg, - ), - ) { - append(text.substring(i + 1, end)) - } - i = end + 1 - continue + is Emphasis -> { + withStyle(SpanStyle(fontStyle = FontStyle.Italic)) { + appendInlineNode(current.firstChild, inlineCodeBg = inlineCodeBg, inlineCodeColor = inlineCodeColor) } } - - if (text[i] == '*' && (i + 1 < text.length && text[i + 1] != '*')) { - val end = text.indexOf('*', startIndex = i + 1) - if (end > i + 1) { - withStyle(SpanStyle(fontStyle = FontStyle.Italic)) { - append(text.substring(i + 1, end)) - } - i = end + 1 - continue + is StrongEmphasis -> { + withStyle(SpanStyle(fontWeight = FontWeight.SemiBold)) { + appendInlineNode(current.firstChild, inlineCodeBg = inlineCodeBg, inlineCodeColor = inlineCodeColor) } } - - append(text[i]) - i += 1 + is Strikethrough -> { + withStyle(SpanStyle(textDecoration = TextDecoration.LineThrough)) { + appendInlineNode(current.firstChild, inlineCodeBg = inlineCodeBg, inlineCodeColor = inlineCodeColor) + } + } + is Link -> { + withStyle( + SpanStyle( + color = mobileAccent, + textDecoration = TextDecoration.Underline, + ), + ) { + appendInlineNode(current.firstChild, inlineCodeBg = inlineCodeBg, inlineCodeColor = inlineCodeColor) + } + } + is MarkdownImage -> { + val alt = buildPlainText(current.firstChild) + if (alt.isNotBlank()) { + append(alt) + } else { + append("image") + } + } + is HtmlInline -> { + if (!current.literal.isNullOrBlank()) { + append(current.literal) + } + } + else -> { + appendInlineNode(current.firstChild, inlineCodeBg = inlineCodeBg, inlineCodeColor = inlineCodeColor) + } } + current = current.next } - return out } +private fun buildPlainText(start: Node?): String { + val sb = StringBuilder() + var node = start + while (node != null) { + when (node) { + is MarkdownTextNode -> sb.append(node.literal) + is SoftLineBreak, is HardLineBreak -> sb.append('\n') + else -> sb.append(buildPlainText(node.firstChild)) + } + node = node.next + } + return sb.toString() +} + +private fun standaloneDataImage(paragraph: Paragraph): ParsedDataImage? { + val only = paragraph.firstChild as? MarkdownImage ?: return null + if (only.next != null) return null + return parseDataImageDestination(only.destination) +} + +private fun parseDataImageDestination(destination: String?): ParsedDataImage? { + val raw = destination?.trim().orEmpty() + if (raw.isEmpty()) return null + val match = dataImageRegex.matchEntire(raw) ?: return null + val subtype = match.groupValues.getOrNull(1)?.trim()?.ifEmpty { "png" } ?: "png" + val base64 = match.groupValues.getOrNull(2)?.replace("\n", "")?.replace("\r", "")?.trim().orEmpty() + if (base64.isEmpty()) return null + return ParsedDataImage(mimeType = "image/$subtype", base64 = base64) +} + +private fun headingStyle(level: Int): TextStyle { + return when (level.coerceIn(1, 6)) { + 1 -> mobileCallout.copy(fontSize = 22.sp, lineHeight = 28.sp, fontWeight = FontWeight.Bold) + 2 -> mobileCallout.copy(fontSize = 20.sp, lineHeight = 26.sp, fontWeight = FontWeight.Bold) + 3 -> mobileCallout.copy(fontSize = 18.sp, lineHeight = 24.sp, fontWeight = FontWeight.SemiBold) + 4 -> mobileCallout.copy(fontSize = 16.sp, lineHeight = 22.sp, fontWeight = FontWeight.SemiBold) + else -> mobileCallout.copy(fontWeight = FontWeight.SemiBold) + } +} + +private data class InlineStyles( + val inlineCodeBg: Color, + val inlineCodeColor: Color, +) + +private data class TableRenderRow( + val isHeader: Boolean, + val cells: List, +) + +private data class ParsedDataImage( + val mimeType: String, + val base64: String, +) + @Composable private fun InlineBase64Image(base64: String, mimeType: String?) { var image by remember(base64) { mutableStateOf(null) } @@ -208,8 +584,8 @@ private fun InlineBase64Image(base64: String, mimeType: String?) { Text( text = "Image unavailable", modifier = Modifier.padding(vertical = 2.dp), - style = MaterialTheme.typography.bodySmall, - color = MaterialTheme.colorScheme.onSurfaceVariant, + style = mobileCaption1, + color = mobileTextSecondary, ) } } diff --git a/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatMessageListCard.kt b/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatMessageListCard.kt index bcec19a5fa2..889de006cb4 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatMessageListCard.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatMessageListCard.kt @@ -2,26 +2,26 @@ package ai.openclaw.android.ui.chat import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Box -import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.fillMaxSize import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.padding import androidx.compose.foundation.lazy.LazyColumn import androidx.compose.foundation.lazy.rememberLazyListState -import androidx.compose.material.icons.Icons -import androidx.compose.material.icons.filled.ArrowCircleDown -import androidx.compose.material3.Card -import androidx.compose.material3.CardDefaults -import androidx.compose.material3.Icon -import androidx.compose.material3.MaterialTheme +import androidx.compose.foundation.shape.RoundedCornerShape +import androidx.compose.material3.Surface import androidx.compose.material3.Text import androidx.compose.runtime.Composable import androidx.compose.runtime.LaunchedEffect import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier -import androidx.compose.ui.draw.alpha import androidx.compose.ui.unit.dp import ai.openclaw.android.chat.ChatMessage import ai.openclaw.android.chat.ChatPendingToolCall +import ai.openclaw.android.ui.mobileBorder +import ai.openclaw.android.ui.mobileCallout +import ai.openclaw.android.ui.mobileHeadline +import ai.openclaw.android.ui.mobileText +import ai.openclaw.android.ui.mobileTextSecondary @Composable fun ChatMessageListCard( @@ -29,6 +29,7 @@ fun ChatMessageListCard( pendingRunCount: Int, pendingToolCalls: List, streamingAssistantText: String?, + healthOk: Boolean, modifier: Modifier = Modifier, ) { val listState = rememberLazyListState() @@ -38,73 +39,70 @@ fun ChatMessageListCard( listState.animateScrollToItem(index = 0) } - Card( - modifier = modifier.fillMaxWidth(), - shape = MaterialTheme.shapes.large, - colors = - CardDefaults.cardColors( - containerColor = MaterialTheme.colorScheme.surfaceContainer, - ), - elevation = CardDefaults.cardElevation(defaultElevation = 0.dp), - ) { - Box(modifier = Modifier.fillMaxSize()) { - LazyColumn( - modifier = Modifier.fillMaxSize(), - state = listState, - reverseLayout = true, - verticalArrangement = Arrangement.spacedBy(14.dp), - contentPadding = androidx.compose.foundation.layout.PaddingValues(top = 12.dp, bottom = 12.dp, start = 12.dp, end = 12.dp), - ) { - // With reverseLayout = true, index 0 renders at the BOTTOM. - // So we emit newest items first: streaming → tools → typing → messages (newest→oldest). + Box(modifier = modifier.fillMaxWidth()) { + LazyColumn( + modifier = Modifier.fillMaxSize(), + state = listState, + reverseLayout = true, + verticalArrangement = Arrangement.spacedBy(10.dp), + contentPadding = androidx.compose.foundation.layout.PaddingValues(bottom = 8.dp), + ) { + // With reverseLayout = true, index 0 renders at the BOTTOM. + // So we emit newest items first: streaming → tools → typing → messages (newest→oldest). - val stream = streamingAssistantText?.trim() - if (!stream.isNullOrEmpty()) { - item(key = "stream") { - ChatStreamingAssistantBubble(text = stream) - } - } - - if (pendingToolCalls.isNotEmpty()) { - item(key = "tools") { - ChatPendingToolsBubble(toolCalls = pendingToolCalls) - } - } - - if (pendingRunCount > 0) { - item(key = "typing") { - ChatTypingIndicatorBubble() - } - } - - items(count = messages.size, key = { idx -> messages[messages.size - 1 - idx].id }) { idx -> - ChatMessageBubble(message = messages[messages.size - 1 - idx]) + val stream = streamingAssistantText?.trim() + if (!stream.isNullOrEmpty()) { + item(key = "stream") { + ChatStreamingAssistantBubble(text = stream) } } - if (messages.isEmpty() && pendingRunCount == 0 && pendingToolCalls.isEmpty() && streamingAssistantText.isNullOrBlank()) { - EmptyChatHint(modifier = Modifier.align(Alignment.Center)) + if (pendingToolCalls.isNotEmpty()) { + item(key = "tools") { + ChatPendingToolsBubble(toolCalls = pendingToolCalls) + } } + + if (pendingRunCount > 0) { + item(key = "typing") { + ChatTypingIndicatorBubble() + } + } + + items(count = messages.size, key = { idx -> messages[messages.size - 1 - idx].id }) { idx -> + ChatMessageBubble(message = messages[messages.size - 1 - idx]) + } + } + + if (messages.isEmpty() && pendingRunCount == 0 && pendingToolCalls.isEmpty() && streamingAssistantText.isNullOrBlank()) { + EmptyChatHint(modifier = Modifier.align(Alignment.Center), healthOk = healthOk) } } } @Composable -private fun EmptyChatHint(modifier: Modifier = Modifier) { - Row( - modifier = modifier.alpha(0.7f), - verticalAlignment = Alignment.CenterVertically, - horizontalArrangement = Arrangement.spacedBy(8.dp), +private fun EmptyChatHint(modifier: Modifier = Modifier, healthOk: Boolean) { + Surface( + modifier = modifier.fillMaxWidth(), + shape = RoundedCornerShape(14.dp), + color = androidx.compose.ui.graphics.Color.White.copy(alpha = 0.9f), + border = androidx.compose.foundation.BorderStroke(1.dp, mobileBorder), ) { - Icon( - imageVector = Icons.Default.ArrowCircleDown, - contentDescription = null, - tint = MaterialTheme.colorScheme.onSurfaceVariant, - ) - Text( - text = "Message OpenClaw…", - style = MaterialTheme.typography.bodyMedium, - color = MaterialTheme.colorScheme.onSurfaceVariant, - ) + androidx.compose.foundation.layout.Column( + modifier = Modifier.padding(horizontal = 12.dp, vertical = 12.dp), + verticalArrangement = Arrangement.spacedBy(4.dp), + ) { + Text("No messages yet", style = mobileHeadline, color = mobileText) + Text( + text = + if (healthOk) { + "Send the first prompt to start this session." + } else { + "Connect gateway first, then return to chat." + }, + style = mobileCallout, + color = mobileTextSecondary, + ) + } } } diff --git a/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatMessageViews.kt b/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatMessageViews.kt index bf294327551..3f4250c3dbb 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatMessageViews.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatMessageViews.kt @@ -2,7 +2,8 @@ package ai.openclaw.android.ui.chat import android.graphics.BitmapFactory import android.util.Base64 -import androidx.compose.foundation.background +import androidx.compose.foundation.BorderStroke +import androidx.compose.foundation.Image import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Column @@ -12,7 +13,6 @@ import androidx.compose.foundation.layout.padding import androidx.compose.foundation.layout.size import androidx.compose.foundation.shape.CircleShape import androidx.compose.foundation.shape.RoundedCornerShape -import androidx.compose.material3.MaterialTheme import androidx.compose.material3.Surface import androidx.compose.material3.Text import androidx.compose.runtime.Composable @@ -24,55 +24,93 @@ import androidx.compose.runtime.setValue import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier import androidx.compose.ui.draw.alpha -import androidx.compose.ui.graphics.Brush import androidx.compose.ui.graphics.Color import androidx.compose.ui.graphics.asImageBitmap import androidx.compose.ui.layout.ContentScale +import androidx.compose.ui.platform.LocalContext import androidx.compose.ui.text.font.FontFamily +import androidx.compose.ui.text.font.FontWeight import androidx.compose.ui.unit.dp -import androidx.compose.foundation.Image +import androidx.compose.ui.unit.sp import ai.openclaw.android.chat.ChatMessage import ai.openclaw.android.chat.ChatMessageContent import ai.openclaw.android.chat.ChatPendingToolCall import ai.openclaw.android.tools.ToolDisplayRegistry +import ai.openclaw.android.ui.mobileAccent +import ai.openclaw.android.ui.mobileAccentSoft +import ai.openclaw.android.ui.mobileBorder +import ai.openclaw.android.ui.mobileBorderStrong +import ai.openclaw.android.ui.mobileCallout +import ai.openclaw.android.ui.mobileCaption1 +import ai.openclaw.android.ui.mobileCaption2 +import ai.openclaw.android.ui.mobileCodeBg +import ai.openclaw.android.ui.mobileCodeText +import ai.openclaw.android.ui.mobileHeadline +import ai.openclaw.android.ui.mobileText +import ai.openclaw.android.ui.mobileTextSecondary +import ai.openclaw.android.ui.mobileWarning +import ai.openclaw.android.ui.mobileWarningSoft +import java.util.Locale import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.withContext -import androidx.compose.ui.platform.LocalContext + +private data class ChatBubbleStyle( + val alignEnd: Boolean, + val containerColor: Color, + val borderColor: Color, + val roleColor: Color, +) @Composable fun ChatMessageBubble(message: ChatMessage) { - val isUser = message.role.lowercase() == "user" + val role = message.role.trim().lowercase(Locale.US) + val style = bubbleStyle(role) - // Filter to only displayable content parts (text with content, or base64 images) - val displayableContent = message.content.filter { part -> - when (part.type) { - "text" -> !part.text.isNullOrBlank() - else -> part.base64 != null + // Filter to only displayable content parts (text with content, or base64 images). + val displayableContent = + message.content.filter { part -> + when (part.type) { + "text" -> !part.text.isNullOrBlank() + else -> part.base64 != null + } } - } - // Skip rendering entirely if no displayable content if (displayableContent.isEmpty()) return + ChatBubbleContainer(style = style, roleLabel = roleLabel(role)) { + ChatMessageBody(content = displayableContent, textColor = mobileText) + } +} + +@Composable +private fun ChatBubbleContainer( + style: ChatBubbleStyle, + roleLabel: String, + modifier: Modifier = Modifier, + content: @Composable () -> Unit, +) { Row( - modifier = Modifier.fillMaxWidth(), - horizontalArrangement = if (isUser) Arrangement.End else Arrangement.Start, + modifier = modifier.fillMaxWidth(), + horizontalArrangement = if (style.alignEnd) Arrangement.End else Arrangement.Start, ) { Surface( - shape = RoundedCornerShape(16.dp), + shape = RoundedCornerShape(12.dp), + border = BorderStroke(1.dp, style.borderColor), + color = style.containerColor, tonalElevation = 0.dp, shadowElevation = 0.dp, - color = Color.Transparent, - modifier = Modifier.fillMaxWidth(0.92f), + modifier = Modifier.fillMaxWidth(0.90f), ) { - Box( - modifier = - Modifier - .background(bubbleBackground(isUser)) - .padding(horizontal = 12.dp, vertical = 10.dp), + Column( + modifier = Modifier.padding(horizontal = 11.dp, vertical = 8.dp), + verticalArrangement = Arrangement.spacedBy(3.dp), ) { - val textColor = textColorOverBubble(isUser) - ChatMessageBody(content = displayableContent, textColor = textColor) + Text( + text = roleLabel, + style = mobileCaption2.copy(fontWeight = FontWeight.SemiBold, letterSpacing = 0.6.sp), + color = style.roleColor, + ) + content() } } } @@ -80,7 +118,7 @@ fun ChatMessageBubble(message: ChatMessage) { @Composable private fun ChatMessageBody(content: List, textColor: Color) { - Column(verticalArrangement = Arrangement.spacedBy(10.dp)) { + Column(verticalArrangement = Arrangement.spacedBy(8.dp)) { for (part in content) { when (part.type) { "text" -> { @@ -98,19 +136,16 @@ private fun ChatMessageBody(content: List, textColor: Color) @Composable fun ChatTypingIndicatorBubble() { - Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.Start) { - Surface( - shape = RoundedCornerShape(16.dp), - color = MaterialTheme.colorScheme.surfaceContainer, + ChatBubbleContainer( + style = bubbleStyle("assistant"), + roleLabel = roleLabel("assistant"), + ) { + Row( + verticalAlignment = Alignment.CenterVertically, + horizontalArrangement = Arrangement.spacedBy(8.dp), ) { - Row( - modifier = Modifier.padding(horizontal = 12.dp, vertical = 10.dp), - verticalAlignment = Alignment.CenterVertically, - horizontalArrangement = Arrangement.spacedBy(8.dp), - ) { - DotPulse() - Text("Thinking…", style = MaterialTheme.typography.bodyMedium, color = MaterialTheme.colorScheme.onSurfaceVariant) - } + DotPulse(color = mobileTextSecondary) + Text("Thinking...", style = mobileCallout, color = mobileTextSecondary) } } } @@ -122,38 +157,37 @@ fun ChatPendingToolsBubble(toolCalls: List) { remember(toolCalls, context) { toolCalls.map { ToolDisplayRegistry.resolve(context, it.name, it.args) } } - Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.Start) { - Surface( - shape = RoundedCornerShape(16.dp), - color = MaterialTheme.colorScheme.surfaceContainer, - ) { - Column(modifier = Modifier.padding(horizontal = 12.dp, vertical = 10.dp), verticalArrangement = Arrangement.spacedBy(6.dp)) { - Text("Running tools…", style = MaterialTheme.typography.labelLarge, color = MaterialTheme.colorScheme.onSurface) - for (display in displays.take(6)) { - Column(verticalArrangement = Arrangement.spacedBy(2.dp)) { + + ChatBubbleContainer( + style = bubbleStyle("assistant"), + roleLabel = "TOOLS", + ) { + Column(verticalArrangement = Arrangement.spacedBy(4.dp)) { + Text("Running tools...", style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), color = mobileTextSecondary) + for (display in displays.take(6)) { + Column(verticalArrangement = Arrangement.spacedBy(2.dp)) { + Text( + "${display.emoji} ${display.label}", + style = mobileCallout, + color = mobileTextSecondary, + fontFamily = FontFamily.Monospace, + ) + display.detailLine?.let { detail -> Text( - "${display.emoji} ${display.label}", - style = MaterialTheme.typography.bodyMedium, - color = MaterialTheme.colorScheme.onSurfaceVariant, + detail, + style = mobileCaption1, + color = mobileTextSecondary, fontFamily = FontFamily.Monospace, ) - display.detailLine?.let { detail -> - Text( - detail, - style = MaterialTheme.typography.bodySmall, - color = MaterialTheme.colorScheme.onSurfaceVariant, - fontFamily = FontFamily.Monospace, - ) - } } } - if (toolCalls.size > 6) { - Text( - "… +${toolCalls.size - 6} more", - style = MaterialTheme.typography.bodySmall, - color = MaterialTheme.colorScheme.onSurfaceVariant, - ) - } + } + if (toolCalls.size > 6) { + Text( + text = "... +${toolCalls.size - 6} more", + style = mobileCaption1, + color = mobileTextSecondary, + ) } } } @@ -161,37 +195,47 @@ fun ChatPendingToolsBubble(toolCalls: List) { @Composable fun ChatStreamingAssistantBubble(text: String) { - Row(modifier = Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.Start) { - Surface( - shape = RoundedCornerShape(16.dp), - color = MaterialTheme.colorScheme.surfaceContainer, - ) { - Box(modifier = Modifier.padding(horizontal = 12.dp, vertical = 10.dp)) { - ChatMarkdown(text = text, textColor = MaterialTheme.colorScheme.onSurface) - } - } + ChatBubbleContainer( + style = bubbleStyle("assistant").copy(borderColor = mobileAccent), + roleLabel = "ASSISTANT · LIVE", + ) { + ChatMarkdown(text = text, textColor = mobileText) } } -@Composable -private fun bubbleBackground(isUser: Boolean): Brush { - return if (isUser) { - Brush.linearGradient( - colors = listOf(MaterialTheme.colorScheme.primary, MaterialTheme.colorScheme.primary.copy(alpha = 0.78f)), - ) - } else { - Brush.linearGradient( - colors = listOf(MaterialTheme.colorScheme.surfaceContainer, MaterialTheme.colorScheme.surfaceContainerHigh), - ) +private fun bubbleStyle(role: String): ChatBubbleStyle { + return when (role) { + "user" -> + ChatBubbleStyle( + alignEnd = true, + containerColor = mobileAccentSoft, + borderColor = mobileAccent, + roleColor = mobileAccent, + ) + + "system" -> + ChatBubbleStyle( + alignEnd = false, + containerColor = mobileWarningSoft, + borderColor = mobileWarning.copy(alpha = 0.45f), + roleColor = mobileWarning, + ) + + else -> + ChatBubbleStyle( + alignEnd = false, + containerColor = Color.White, + borderColor = mobileBorderStrong, + roleColor = mobileTextSecondary, + ) } } -@Composable -private fun textColorOverBubble(isUser: Boolean): Color { - return if (isUser) { - MaterialTheme.colorScheme.onPrimary - } else { - MaterialTheme.colorScheme.onSurface +private fun roleLabel(role: String): String { + return when (role) { + "user" -> "USER" + "system" -> "SYSTEM" + else -> "ASSISTANT" } } @@ -216,48 +260,64 @@ private fun ChatBase64Image(base64: String, mimeType: String?) { } if (image != null) { - Image( - bitmap = image!!, - contentDescription = mimeType ?: "attachment", - contentScale = ContentScale.Fit, + Surface( + shape = RoundedCornerShape(10.dp), + border = BorderStroke(1.dp, mobileBorder), + color = Color.White, modifier = Modifier.fillMaxWidth(), - ) + ) { + Image( + bitmap = image!!, + contentDescription = mimeType ?: "attachment", + contentScale = ContentScale.Fit, + modifier = Modifier.fillMaxWidth(), + ) + } } else if (failed) { - Text("Unsupported attachment", style = MaterialTheme.typography.bodySmall, color = MaterialTheme.colorScheme.onSurfaceVariant) + Text("Unsupported attachment", style = mobileCaption1, color = mobileTextSecondary) } } @Composable -private fun DotPulse() { +private fun DotPulse(color: Color) { Row(horizontalArrangement = Arrangement.spacedBy(5.dp), verticalAlignment = Alignment.CenterVertically) { - PulseDot(alpha = 0.38f) - PulseDot(alpha = 0.62f) - PulseDot(alpha = 0.90f) + PulseDot(alpha = 0.38f, color = color) + PulseDot(alpha = 0.62f, color = color) + PulseDot(alpha = 0.90f, color = color) } } @Composable -private fun PulseDot(alpha: Float) { +private fun PulseDot(alpha: Float, color: Color) { Surface( modifier = Modifier.size(6.dp).alpha(alpha), shape = CircleShape, - color = MaterialTheme.colorScheme.onSurfaceVariant, + color = color, ) {} } @Composable fun ChatCodeBlock(code: String, language: String?) { Surface( - shape = RoundedCornerShape(12.dp), - color = MaterialTheme.colorScheme.surfaceContainerLowest, + shape = RoundedCornerShape(8.dp), + color = mobileCodeBg, + border = BorderStroke(1.dp, Color(0xFF2B2E35)), modifier = Modifier.fillMaxWidth(), ) { - Text( - text = code.trimEnd(), - modifier = Modifier.padding(10.dp), - fontFamily = FontFamily.Monospace, - style = MaterialTheme.typography.bodySmall, - color = MaterialTheme.colorScheme.onSurface, - ) + Column(modifier = Modifier.padding(horizontal = 10.dp, vertical = 8.dp), verticalArrangement = Arrangement.spacedBy(4.dp)) { + if (!language.isNullOrBlank()) { + Text( + text = language.uppercase(Locale.US), + style = mobileCaption2.copy(letterSpacing = 0.4.sp), + color = mobileTextSecondary, + ) + } + Text( + text = code.trimEnd(), + fontFamily = FontFamily.Monospace, + style = mobileCallout, + color = mobileCodeText, + ) + } } } diff --git a/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatSessionsDialog.kt b/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatSessionsDialog.kt deleted file mode 100644 index 56b5cfb1faf..00000000000 --- a/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatSessionsDialog.kt +++ /dev/null @@ -1,92 +0,0 @@ -package ai.openclaw.android.ui.chat - -import androidx.compose.foundation.layout.Arrangement -import androidx.compose.foundation.layout.Column -import androidx.compose.foundation.layout.Row -import androidx.compose.foundation.layout.Spacer -import androidx.compose.foundation.layout.fillMaxWidth -import androidx.compose.foundation.layout.padding -import androidx.compose.foundation.lazy.LazyColumn -import androidx.compose.foundation.lazy.items -import androidx.compose.material.icons.Icons -import androidx.compose.material.icons.filled.Refresh -import androidx.compose.material3.AlertDialog -import androidx.compose.material3.FilledTonalIconButton -import androidx.compose.material3.Icon -import androidx.compose.material3.MaterialTheme -import androidx.compose.material3.Surface -import androidx.compose.material3.Text -import androidx.compose.runtime.Composable -import androidx.compose.ui.Alignment -import androidx.compose.ui.Modifier -import androidx.compose.ui.unit.dp -import ai.openclaw.android.chat.ChatSessionEntry - -@Composable -fun ChatSessionsDialog( - currentSessionKey: String, - sessions: List, - onDismiss: () -> Unit, - onRefresh: () -> Unit, - onSelect: (sessionKey: String) -> Unit, -) { - AlertDialog( - onDismissRequest = onDismiss, - confirmButton = {}, - title = { - Row(verticalAlignment = Alignment.CenterVertically, modifier = Modifier.fillMaxWidth()) { - Text("Sessions", style = MaterialTheme.typography.titleMedium) - Spacer(modifier = Modifier.weight(1f)) - FilledTonalIconButton(onClick = onRefresh) { - Icon(Icons.Default.Refresh, contentDescription = "Refresh") - } - } - }, - text = { - if (sessions.isEmpty()) { - Text("No sessions", style = MaterialTheme.typography.bodyMedium, color = MaterialTheme.colorScheme.onSurfaceVariant) - } else { - LazyColumn(verticalArrangement = Arrangement.spacedBy(8.dp)) { - items(sessions, key = { it.key }) { entry -> - SessionRow( - entry = entry, - isCurrent = entry.key == currentSessionKey, - onClick = { onSelect(entry.key) }, - ) - } - } - } - }, - ) -} - -@Composable -private fun SessionRow( - entry: ChatSessionEntry, - isCurrent: Boolean, - onClick: () -> Unit, -) { - Surface( - onClick = onClick, - shape = MaterialTheme.shapes.medium, - color = - if (isCurrent) { - MaterialTheme.colorScheme.primary.copy(alpha = 0.14f) - } else { - MaterialTheme.colorScheme.surfaceContainer - }, - modifier = Modifier.fillMaxWidth(), - ) { - Row( - modifier = Modifier.padding(horizontal = 12.dp, vertical = 10.dp), - verticalAlignment = Alignment.CenterVertically, - horizontalArrangement = Arrangement.spacedBy(10.dp), - ) { - Text(entry.displayName ?: entry.key, style = MaterialTheme.typography.bodyMedium) - Spacer(modifier = Modifier.weight(1f)) - if (isCurrent) { - Text("Current", style = MaterialTheme.typography.labelSmall, color = MaterialTheme.colorScheme.onSurfaceVariant) - } - } - } -} diff --git a/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatSheetContent.kt b/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatSheetContent.kt index effee6708e0..12e13ab365a 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatSheetContent.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/ui/chat/ChatSheetContent.kt @@ -5,10 +5,19 @@ import android.net.Uri import android.util.Base64 import androidx.activity.compose.rememberLauncherForActivityResult import androidx.activity.result.contract.ActivityResultContracts +import androidx.compose.foundation.BorderStroke +import androidx.compose.foundation.horizontalScroll import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.fillMaxSize +import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.imePadding import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.rememberScrollState +import androidx.compose.foundation.shape.RoundedCornerShape +import androidx.compose.material3.Surface +import androidx.compose.material3.Text import androidx.compose.runtime.Composable import androidx.compose.runtime.LaunchedEffect import androidx.compose.runtime.collectAsState @@ -17,10 +26,28 @@ import androidx.compose.runtime.mutableStateListOf import androidx.compose.runtime.remember import androidx.compose.runtime.rememberCoroutineScope import androidx.compose.ui.Modifier +import androidx.compose.ui.graphics.Color import androidx.compose.ui.platform.LocalContext +import androidx.compose.ui.text.font.FontWeight +import androidx.compose.ui.text.style.TextOverflow import androidx.compose.ui.unit.dp +import androidx.compose.ui.unit.sp import ai.openclaw.android.MainViewModel +import ai.openclaw.android.chat.ChatSessionEntry import ai.openclaw.android.chat.OutgoingAttachment +import ai.openclaw.android.ui.mobileAccent +import ai.openclaw.android.ui.mobileBorder +import ai.openclaw.android.ui.mobileBorderStrong +import ai.openclaw.android.ui.mobileCallout +import ai.openclaw.android.ui.mobileCaption1 +import ai.openclaw.android.ui.mobileCaption2 +import ai.openclaw.android.ui.mobileDanger +import ai.openclaw.android.ui.mobileSuccess +import ai.openclaw.android.ui.mobileSuccessSoft +import ai.openclaw.android.ui.mobileText +import ai.openclaw.android.ui.mobileTextSecondary +import ai.openclaw.android.ui.mobileWarning +import ai.openclaw.android.ui.mobileWarningSoft import java.io.ByteArrayOutputStream import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.launch @@ -72,52 +99,160 @@ fun ChatSheetContent(viewModel: MainViewModel) { modifier = Modifier .fillMaxSize() - .padding(horizontal = 12.dp, vertical = 12.dp), - verticalArrangement = Arrangement.spacedBy(10.dp), + .padding(horizontal = 20.dp, vertical = 12.dp), + verticalArrangement = Arrangement.spacedBy(8.dp), ) { + ChatThreadSelector( + sessionKey = sessionKey, + sessions = sessions, + mainSessionKey = mainSessionKey, + healthOk = healthOk, + onSelectSession = { key -> viewModel.switchChatSession(key) }, + ) + + if (!errorText.isNullOrBlank()) { + ChatErrorRail(errorText = errorText!!) + } + ChatMessageListCard( messages = messages, pendingRunCount = pendingRunCount, pendingToolCalls = pendingToolCalls, streamingAssistantText = streamingAssistantText, + healthOk = healthOk, modifier = Modifier.weight(1f, fill = true), ) - ChatComposer( - sessionKey = sessionKey, - sessions = sessions, - mainSessionKey = mainSessionKey, - healthOk = healthOk, - thinkingLevel = thinkingLevel, - pendingRunCount = pendingRunCount, - errorText = errorText, - attachments = attachments, - onPickImages = { pickImages.launch("image/*") }, - onRemoveAttachment = { id -> attachments.removeAll { it.id == id } }, - onSetThinkingLevel = { level -> viewModel.setChatThinkingLevel(level) }, - onSelectSession = { key -> viewModel.switchChatSession(key) }, - onRefresh = { - viewModel.refreshChat() - viewModel.refreshChatSessions(limit = 200) - }, - onAbort = { viewModel.abortChat() }, - onSend = { text -> - val outgoing = - attachments.map { att -> - OutgoingAttachment( - type = "image", - mimeType = att.mimeType, - fileName = att.fileName, - base64 = att.base64, - ) - } - viewModel.sendChat(message = text, thinking = thinkingLevel, attachments = outgoing) - attachments.clear() - }, + Row(modifier = Modifier.fillMaxWidth().imePadding()) { + ChatComposer( + healthOk = healthOk, + thinkingLevel = thinkingLevel, + pendingRunCount = pendingRunCount, + attachments = attachments, + onPickImages = { pickImages.launch("image/*") }, + onRemoveAttachment = { id -> attachments.removeAll { it.id == id } }, + onSetThinkingLevel = { level -> viewModel.setChatThinkingLevel(level) }, + onRefresh = { + viewModel.refreshChat() + viewModel.refreshChatSessions(limit = 200) + }, + onAbort = { viewModel.abortChat() }, + onSend = { text -> + val outgoing = + attachments.map { att -> + OutgoingAttachment( + type = "image", + mimeType = att.mimeType, + fileName = att.fileName, + base64 = att.base64, + ) + } + viewModel.sendChat(message = text, thinking = thinkingLevel, attachments = outgoing) + attachments.clear() + }, + ) + } + } +} + +@Composable +private fun ChatThreadSelector( + sessionKey: String, + sessions: List, + mainSessionKey: String, + healthOk: Boolean, + onSelectSession: (String) -> Unit, +) { + val sessionOptions = resolveSessionChoices(sessionKey, sessions, mainSessionKey = mainSessionKey) + val currentSessionLabel = + friendlySessionName(sessionOptions.firstOrNull { it.key == sessionKey }?.displayName ?: sessionKey) + + Column(modifier = Modifier.fillMaxWidth(), verticalArrangement = Arrangement.spacedBy(8.dp)) { + Row( + modifier = Modifier.fillMaxWidth(), + horizontalArrangement = Arrangement.SpaceBetween, + verticalAlignment = androidx.compose.ui.Alignment.CenterVertically, + ) { + Text( + text = "SESSION", + style = mobileCaption1.copy(fontWeight = FontWeight.Bold, letterSpacing = 0.8.sp), + color = mobileTextSecondary, + ) + Row(horizontalArrangement = Arrangement.spacedBy(6.dp), verticalAlignment = androidx.compose.ui.Alignment.CenterVertically) { + Text( + text = currentSessionLabel, + style = mobileCallout.copy(fontWeight = FontWeight.SemiBold), + color = mobileText, + maxLines = 1, + overflow = TextOverflow.Ellipsis, + ) + ChatConnectionPill(healthOk = healthOk) + } + } + + Row( + modifier = Modifier.fillMaxWidth().horizontalScroll(rememberScrollState()), + horizontalArrangement = Arrangement.spacedBy(8.dp), + ) { + for (entry in sessionOptions) { + val active = entry.key == sessionKey + Surface( + onClick = { onSelectSession(entry.key) }, + shape = RoundedCornerShape(14.dp), + color = if (active) mobileAccent else Color.White, + border = BorderStroke(1.dp, if (active) Color(0xFF154CAD) else mobileBorderStrong), + tonalElevation = 0.dp, + shadowElevation = 0.dp, + ) { + Text( + text = friendlySessionName(entry.displayName ?: entry.key), + style = mobileCaption1.copy(fontWeight = if (active) FontWeight.Bold else FontWeight.SemiBold), + color = if (active) Color.White else mobileText, + maxLines = 1, + overflow = TextOverflow.Ellipsis, + modifier = Modifier.padding(horizontal = 12.dp, vertical = 8.dp), + ) + } + } + } + } +} + +@Composable +private fun ChatConnectionPill(healthOk: Boolean) { + Surface( + shape = RoundedCornerShape(999.dp), + color = if (healthOk) mobileSuccessSoft else mobileWarningSoft, + border = BorderStroke(1.dp, if (healthOk) mobileSuccess.copy(alpha = 0.35f) else mobileWarning.copy(alpha = 0.35f)), + ) { + Text( + text = if (healthOk) "Connected" else "Offline", + style = mobileCaption1.copy(fontWeight = FontWeight.SemiBold), + color = if (healthOk) mobileSuccess else mobileWarning, + modifier = Modifier.padding(horizontal = 8.dp, vertical = 3.dp), ) } } +@Composable +private fun ChatErrorRail(errorText: String) { + Surface( + modifier = Modifier.fillMaxWidth(), + color = androidx.compose.ui.graphics.Color.White, + shape = RoundedCornerShape(12.dp), + border = androidx.compose.foundation.BorderStroke(1.dp, mobileDanger), + ) { + Column(modifier = Modifier.padding(horizontal = 10.dp, vertical = 8.dp), verticalArrangement = Arrangement.spacedBy(2.dp)) { + Text( + text = "CHAT ERROR", + style = mobileCaption2.copy(letterSpacing = 0.6.sp), + color = mobileDanger, + ) + Text(text = errorText, style = mobileCallout, color = mobileText) + } + } +} + data class PendingImageAttachment( val id: String, val fileName: String, diff --git a/apps/android/app/src/main/java/ai/openclaw/android/voice/MicCaptureManager.kt b/apps/android/app/src/main/java/ai/openclaw/android/voice/MicCaptureManager.kt new file mode 100644 index 00000000000..5d7336cdf6f --- /dev/null +++ b/apps/android/app/src/main/java/ai/openclaw/android/voice/MicCaptureManager.kt @@ -0,0 +1,541 @@ +package ai.openclaw.android.voice + +import android.Manifest +import android.content.Context +import android.content.Intent +import android.content.pm.PackageManager +import android.os.Bundle +import android.os.Handler +import android.os.Looper +import android.speech.RecognitionListener +import android.speech.RecognizerIntent +import android.speech.SpeechRecognizer +import android.util.Log +import androidx.core.content.ContextCompat +import java.util.UUID +import kotlinx.coroutines.CoroutineScope +import kotlinx.coroutines.Job +import kotlinx.coroutines.delay +import kotlinx.coroutines.flow.MutableStateFlow +import kotlinx.coroutines.flow.StateFlow +import kotlinx.coroutines.launch +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonArray +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.JsonPrimitive + +enum class VoiceConversationRole { + User, + Assistant, +} + +data class VoiceConversationEntry( + val id: String, + val role: VoiceConversationRole, + val text: String, + val isStreaming: Boolean = false, +) + +class MicCaptureManager( + private val context: Context, + private val scope: CoroutineScope, + private val sendToGateway: suspend (String) -> String?, + private val speakAssistantReply: suspend (String) -> Unit = {}, +) { + companion object { + private const val tag = "MicCapture" + private const val speechMinSessionMs = 30_000L + private const val speechCompleteSilenceMs = 1_500L + private const val speechPossibleSilenceMs = 900L + private const val maxConversationEntries = 40 + private const val pendingRunTimeoutMs = 45_000L + } + + private val mainHandler = Handler(Looper.getMainLooper()) + private val json = Json { ignoreUnknownKeys = true } + + private val _micEnabled = MutableStateFlow(false) + val micEnabled: StateFlow = _micEnabled + + private val _isListening = MutableStateFlow(false) + val isListening: StateFlow = _isListening + + private val _statusText = MutableStateFlow("Mic off") + val statusText: StateFlow = _statusText + + private val _liveTranscript = MutableStateFlow(null) + val liveTranscript: StateFlow = _liveTranscript + + private val _queuedMessages = MutableStateFlow>(emptyList()) + val queuedMessages: StateFlow> = _queuedMessages + + private val _conversation = MutableStateFlow>(emptyList()) + val conversation: StateFlow> = _conversation + + private val _inputLevel = MutableStateFlow(0f) + val inputLevel: StateFlow = _inputLevel + + private val _isSending = MutableStateFlow(false) + val isSending: StateFlow = _isSending + + private val messageQueue = ArrayDeque() + private val sessionSegments = mutableListOf() + private var lastFinalSegment: String? = null + private var pendingRunId: String? = null + private var pendingAssistantEntryId: String? = null + private var gatewayConnected = false + + private var recognizer: SpeechRecognizer? = null + private var restartJob: Job? = null + private var pendingRunTimeoutJob: Job? = null + private var stopRequested = false + + fun setMicEnabled(enabled: Boolean) { + if (_micEnabled.value == enabled) return + _micEnabled.value = enabled + if (enabled) { + start() + sendQueuedIfIdle() + } else { + stop() + flushSessionToQueue() + sendQueuedIfIdle() + } + } + + fun onGatewayConnectionChanged(connected: Boolean) { + gatewayConnected = connected + if (connected) { + sendQueuedIfIdle() + return + } + if (messageQueue.isNotEmpty()) { + _statusText.value = queuedWaitingStatus() + } + } + + fun handleGatewayEvent(event: String, payloadJson: String?) { + if (event != "chat") return + if (payloadJson.isNullOrBlank()) return + val payload = + try { + json.parseToJsonElement(payloadJson).asObjectOrNull() + } catch (_: Throwable) { + null + } ?: return + + val runId = pendingRunId ?: return + val eventRunId = payload["runId"].asStringOrNull() ?: return + if (eventRunId != runId) return + + when (payload["state"].asStringOrNull()) { + "delta" -> { + val deltaText = parseAssistantText(payload) + if (!deltaText.isNullOrBlank()) { + upsertPendingAssistant(text = deltaText.trim(), isStreaming = true) + } + } + "final" -> { + val finalText = parseAssistantText(payload)?.trim().orEmpty() + if (finalText.isNotEmpty()) { + upsertPendingAssistant(text = finalText, isStreaming = false) + playAssistantReplyAsync(finalText) + } else if (pendingAssistantEntryId != null) { + updateConversationEntry(pendingAssistantEntryId!!, text = null, isStreaming = false) + } + completePendingTurn() + } + "error" -> { + val errorMessage = payload["errorMessage"].asStringOrNull()?.trim().orEmpty().ifEmpty { "Voice request failed" } + upsertPendingAssistant(text = errorMessage, isStreaming = false) + completePendingTurn() + } + "aborted" -> { + upsertPendingAssistant(text = "Response aborted", isStreaming = false) + completePendingTurn() + } + } + } + + private fun start() { + stopRequested = false + if (!SpeechRecognizer.isRecognitionAvailable(context)) { + _statusText.value = "Speech recognizer unavailable" + _micEnabled.value = false + return + } + if (!hasMicPermission()) { + _statusText.value = "Microphone permission required" + _micEnabled.value = false + return + } + + mainHandler.post { + try { + if (recognizer == null) { + recognizer = SpeechRecognizer.createSpeechRecognizer(context).also { it.setRecognitionListener(listener) } + } + startListeningSession() + } catch (err: Throwable) { + _statusText.value = "Start failed: ${err.message ?: err::class.simpleName}" + _micEnabled.value = false + } + } + } + + private fun stop() { + stopRequested = true + restartJob?.cancel() + restartJob = null + _isListening.value = false + _statusText.value = if (_isSending.value) "Mic off · sending…" else "Mic off" + _inputLevel.value = 0f + mainHandler.post { + recognizer?.cancel() + recognizer?.destroy() + recognizer = null + } + } + + private fun startListeningSession() { + val recognizerInstance = recognizer ?: return + val intent = + Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH).apply { + putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_FREE_FORM) + putExtra(RecognizerIntent.EXTRA_PARTIAL_RESULTS, true) + putExtra(RecognizerIntent.EXTRA_MAX_RESULTS, 3) + putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE, context.packageName) + putExtra(RecognizerIntent.EXTRA_SPEECH_INPUT_MINIMUM_LENGTH_MILLIS, speechMinSessionMs) + putExtra(RecognizerIntent.EXTRA_SPEECH_INPUT_COMPLETE_SILENCE_LENGTH_MILLIS, speechCompleteSilenceMs) + putExtra( + RecognizerIntent.EXTRA_SPEECH_INPUT_POSSIBLY_COMPLETE_SILENCE_LENGTH_MILLIS, + speechPossibleSilenceMs, + ) + } + _statusText.value = + when { + _isSending.value -> "Listening · sending queued voice" + messageQueue.isNotEmpty() -> "Listening · ${messageQueue.size} queued" + else -> "Listening" + } + _isListening.value = true + recognizerInstance.startListening(intent) + } + + private fun scheduleRestart(delayMs: Long = 300L) { + if (stopRequested) return + if (!_micEnabled.value) return + restartJob?.cancel() + restartJob = + scope.launch { + delay(delayMs) + mainHandler.post { + if (stopRequested || !_micEnabled.value) return@post + try { + startListeningSession() + } catch (_: Throwable) { + // retry through onError + } + } + } + } + + private fun flushSessionToQueue() { + val message = sessionSegments.joinToString(" ").trim() + sessionSegments.clear() + _liveTranscript.value = null + lastFinalSegment = null + if (message.isEmpty()) return + + appendConversation( + role = VoiceConversationRole.User, + text = message, + ) + messageQueue.addLast(message) + publishQueue() + } + + private fun publishQueue() { + _queuedMessages.value = messageQueue.toList() + } + + private fun sendQueuedIfIdle() { + if (_isSending.value) return + if (messageQueue.isEmpty()) { + if (_micEnabled.value) { + _statusText.value = "Listening" + } else { + _statusText.value = "Mic off" + } + return + } + if (!gatewayConnected) { + _statusText.value = queuedWaitingStatus() + return + } + + val next = messageQueue.first() + _isSending.value = true + pendingRunTimeoutJob?.cancel() + pendingRunTimeoutJob = null + _statusText.value = if (_micEnabled.value) "Listening · sending queued voice" else "Sending queued voice" + + scope.launch { + try { + val runId = sendToGateway(next) + pendingRunId = runId + if (runId == null) { + pendingRunTimeoutJob?.cancel() + pendingRunTimeoutJob = null + messageQueue.removeFirst() + publishQueue() + _isSending.value = false + pendingAssistantEntryId = null + sendQueuedIfIdle() + } else { + armPendingRunTimeout(runId) + } + } catch (err: Throwable) { + pendingRunTimeoutJob?.cancel() + pendingRunTimeoutJob = null + _isSending.value = false + pendingRunId = null + pendingAssistantEntryId = null + _statusText.value = + if (!gatewayConnected) { + queuedWaitingStatus() + } else { + "Send failed: ${err.message ?: err::class.simpleName}" + } + } + } + } + + private fun armPendingRunTimeout(runId: String) { + pendingRunTimeoutJob?.cancel() + pendingRunTimeoutJob = + scope.launch { + delay(pendingRunTimeoutMs) + if (pendingRunId != runId) return@launch + pendingRunId = null + pendingAssistantEntryId = null + _isSending.value = false + _statusText.value = + if (gatewayConnected) { + "Voice reply timed out; retrying queued turn" + } else { + queuedWaitingStatus() + } + sendQueuedIfIdle() + } + } + + private fun completePendingTurn() { + pendingRunTimeoutJob?.cancel() + pendingRunTimeoutJob = null + if (messageQueue.isNotEmpty()) { + messageQueue.removeFirst() + publishQueue() + } + pendingRunId = null + pendingAssistantEntryId = null + _isSending.value = false + sendQueuedIfIdle() + } + + private fun queuedWaitingStatus(): String { + return "${messageQueue.size} queued · waiting for gateway" + } + + private fun appendConversation( + role: VoiceConversationRole, + text: String, + isStreaming: Boolean = false, + ): String { + val id = UUID.randomUUID().toString() + _conversation.value = + (_conversation.value + VoiceConversationEntry(id = id, role = role, text = text, isStreaming = isStreaming)) + .takeLast(maxConversationEntries) + return id + } + + private fun updateConversationEntry(id: String, text: String?, isStreaming: Boolean) { + val current = _conversation.value + if (current.isEmpty()) return + + val targetIndex = + when { + current[current.lastIndex].id == id -> current.lastIndex + else -> current.indexOfFirst { it.id == id } + } + if (targetIndex < 0) return + + val entry = current[targetIndex] + val updatedText = text ?: entry.text + if (updatedText == entry.text && entry.isStreaming == isStreaming) return + val updated = current.toMutableList() + updated[targetIndex] = entry.copy(text = updatedText, isStreaming = isStreaming) + _conversation.value = updated + } + + private fun upsertPendingAssistant(text: String, isStreaming: Boolean) { + val currentId = pendingAssistantEntryId + if (currentId == null) { + pendingAssistantEntryId = + appendConversation( + role = VoiceConversationRole.Assistant, + text = text, + isStreaming = isStreaming, + ) + return + } + updateConversationEntry(id = currentId, text = text, isStreaming = isStreaming) + } + + private fun playAssistantReplyAsync(text: String) { + val spoken = text.trim() + if (spoken.isEmpty()) return + scope.launch { + try { + speakAssistantReply(spoken) + } catch (err: Throwable) { + Log.w(tag, "assistant speech failed: ${err.message ?: err::class.simpleName}") + } + } + } + + private fun onFinalTranscript(text: String) { + val trimmed = text.trim() + if (trimmed.isEmpty()) return + _liveTranscript.value = trimmed + if (lastFinalSegment == trimmed) return + lastFinalSegment = trimmed + sessionSegments.add(trimmed) + } + + private fun disableMic(status: String) { + stopRequested = true + restartJob?.cancel() + restartJob = null + _micEnabled.value = false + _isListening.value = false + _inputLevel.value = 0f + _statusText.value = status + mainHandler.post { + recognizer?.cancel() + recognizer?.destroy() + recognizer = null + } + } + + private fun hasMicPermission(): Boolean { + return ( + ContextCompat.checkSelfPermission(context, Manifest.permission.RECORD_AUDIO) == + PackageManager.PERMISSION_GRANTED + ) + } + + private fun parseAssistantText(payload: JsonObject): String? { + val message = payload["message"].asObjectOrNull() ?: return null + if (message["role"].asStringOrNull() != "assistant") return null + val content = message["content"] as? JsonArray ?: return null + + val parts = + content.mapNotNull { item -> + val obj = item.asObjectOrNull() ?: return@mapNotNull null + if (obj["type"].asStringOrNull() != "text") return@mapNotNull null + obj["text"].asStringOrNull()?.trim()?.takeIf { it.isNotEmpty() } + } + if (parts.isEmpty()) return null + return parts.joinToString("\n") + } + + private val listener = + object : RecognitionListener { + override fun onReadyForSpeech(params: Bundle?) { + _isListening.value = true + } + + override fun onBeginningOfSpeech() {} + + override fun onRmsChanged(rmsdB: Float) { + val level = ((rmsdB + 2f) / 12f).coerceIn(0f, 1f) + _inputLevel.value = level + } + + override fun onBufferReceived(buffer: ByteArray?) {} + + override fun onEndOfSpeech() { + _inputLevel.value = 0f + scheduleRestart() + } + + override fun onError(error: Int) { + if (stopRequested) return + _isListening.value = false + _inputLevel.value = 0f + val status = + when (error) { + SpeechRecognizer.ERROR_AUDIO -> "Audio error" + SpeechRecognizer.ERROR_CLIENT -> "Client error" + SpeechRecognizer.ERROR_NETWORK -> "Network error" + SpeechRecognizer.ERROR_NETWORK_TIMEOUT -> "Network timeout" + SpeechRecognizer.ERROR_NO_MATCH -> "Listening" + SpeechRecognizer.ERROR_RECOGNIZER_BUSY -> "Recognizer busy" + SpeechRecognizer.ERROR_SERVER -> "Server error" + SpeechRecognizer.ERROR_SPEECH_TIMEOUT -> "Listening" + SpeechRecognizer.ERROR_INSUFFICIENT_PERMISSIONS -> "Microphone permission required" + SpeechRecognizer.ERROR_LANGUAGE_NOT_SUPPORTED -> "Language not supported on this device" + SpeechRecognizer.ERROR_LANGUAGE_UNAVAILABLE -> "Language unavailable on this device" + SpeechRecognizer.ERROR_SERVER_DISCONNECTED -> "Speech service disconnected" + SpeechRecognizer.ERROR_TOO_MANY_REQUESTS -> "Speech requests limited; retrying" + else -> "Speech error ($error)" + } + _statusText.value = status + + if ( + error == SpeechRecognizer.ERROR_INSUFFICIENT_PERMISSIONS || + error == SpeechRecognizer.ERROR_LANGUAGE_NOT_SUPPORTED || + error == SpeechRecognizer.ERROR_LANGUAGE_UNAVAILABLE + ) { + disableMic(status) + return + } + + val restartDelayMs = + when (error) { + SpeechRecognizer.ERROR_NO_MATCH, + SpeechRecognizer.ERROR_SPEECH_TIMEOUT, + -> 1_200L + SpeechRecognizer.ERROR_TOO_MANY_REQUESTS -> 2_500L + else -> 600L + } + scheduleRestart(delayMs = restartDelayMs) + } + + override fun onResults(results: Bundle?) { + val text = results?.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION).orEmpty().firstOrNull() + if (!text.isNullOrBlank()) { + onFinalTranscript(text) + flushSessionToQueue() + sendQueuedIfIdle() + } + scheduleRestart() + } + + override fun onPartialResults(partialResults: Bundle?) { + val text = partialResults?.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION).orEmpty().firstOrNull() + if (!text.isNullOrBlank()) { + _liveTranscript.value = text.trim() + } + } + + override fun onEvent(eventType: Int, params: Bundle?) {} + } +} + +private fun kotlinx.serialization.json.JsonElement?.asObjectOrNull(): JsonObject? = + this as? JsonObject + +private fun kotlinx.serialization.json.JsonElement?.asStringOrNull(): String? = + (this as? JsonPrimitive)?.takeIf { it.isString }?.content diff --git a/apps/android/app/src/main/java/ai/openclaw/android/voice/TalkModeManager.kt b/apps/android/app/src/main/java/ai/openclaw/android/voice/TalkModeManager.kt index 04d18b62260..8c5dc9adbcb 100644 --- a/apps/android/app/src/main/java/ai/openclaw/android/voice/TalkModeManager.kt +++ b/apps/android/app/src/main/java/ai/openclaw/android/voice/TalkModeManager.kt @@ -26,7 +26,9 @@ import ai.openclaw.android.normalizeMainKey import java.net.HttpURLConnection import java.net.URL import java.util.UUID +import java.util.concurrent.atomic.AtomicLong import kotlinx.coroutines.CompletableDeferred +import kotlinx.coroutines.CancellationException import kotlinx.coroutines.CoroutineScope import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.Job @@ -54,6 +56,47 @@ class TalkModeManager( private const val tag = "TalkMode" private const val defaultModelIdFallback = "eleven_v3" private const val defaultOutputFormatFallback = "pcm_24000" + private const val defaultTalkProvider = "elevenlabs" + + internal data class TalkProviderConfigSelection( + val provider: String, + val config: JsonObject, + val normalizedPayload: Boolean, + ) + + private fun normalizeTalkProviderId(raw: String?): String? { + val trimmed = raw?.trim()?.lowercase().orEmpty() + return trimmed.takeIf { it.isNotEmpty() } + } + + internal fun selectTalkProviderConfig(talk: JsonObject?): TalkProviderConfigSelection? { + if (talk == null) return null + val rawProvider = talk["provider"].asStringOrNull() + val rawProviders = talk["providers"].asObjectOrNull() + val hasNormalizedPayload = rawProvider != null || rawProviders != null + if (hasNormalizedPayload) { + val providers = + rawProviders?.entries?.mapNotNull { (key, value) -> + val providerId = normalizeTalkProviderId(key) ?: return@mapNotNull null + val providerConfig = value.asObjectOrNull() ?: return@mapNotNull null + providerId to providerConfig + }?.toMap().orEmpty() + val providerId = + normalizeTalkProviderId(rawProvider) + ?: providers.keys.sorted().firstOrNull() + ?: defaultTalkProvider + return TalkProviderConfigSelection( + provider = providerId, + config = providers[providerId] ?: buildJsonObject {}, + normalizedPayload = true, + ) + } + return TalkProviderConfigSelection( + provider = defaultTalkProvider, + config = talk, + normalizedPayload = false, + ) + } } private val mainHandler = Handler(Looper.getMainLooper()) @@ -105,6 +148,9 @@ class TalkModeManager( private var pendingRunId: String? = null private var pendingFinal: CompletableDeferred? = null private var chatSubscribedSessionKey: String? = null + private var configLoaded = false + @Volatile private var playbackEnabled = true + private val playbackGeneration = AtomicLong(0L) private var player: MediaPlayer? = null private var streamingSource: StreamingMediaDataSource? = null @@ -153,6 +199,28 @@ class TalkModeManager( } } + fun setPlaybackEnabled(enabled: Boolean) { + if (playbackEnabled == enabled) return + playbackEnabled = enabled + if (!enabled) { + playbackGeneration.incrementAndGet() + stopSpeaking() + } + } + + suspend fun refreshConfig() { + reloadConfig() + } + + suspend fun speakAssistantReply(text: String) { + if (!playbackEnabled) return + val playbackToken = playbackGeneration.incrementAndGet() + stopSpeaking(resetInterrupt = false) + ensureConfigLoaded() + ensurePlaybackActive(playbackToken) + playAssistant(text, playbackToken) + } + private fun start() { mainHandler.post { if (_isListening.value) return@post @@ -301,7 +369,7 @@ class TalkModeManager( lastTranscript = "" lastHeardAtMs = null - reloadConfig() + ensureConfigLoaded() val prompt = buildPrompt(transcript) if (!isConnected()) { _statusText.value = "Gateway not connected" @@ -328,8 +396,15 @@ class TalkModeManager( return } Log.d(tag, "assistant text ok chars=${assistant.length}") - playAssistant(assistant) + val playbackToken = playbackGeneration.incrementAndGet() + stopSpeaking(resetInterrupt = false) + ensurePlaybackActive(playbackToken) + playAssistant(assistant, playbackToken) } catch (err: Throwable) { + if (err is CancellationException) { + Log.d(tag, "finalize speech cancelled") + return + } _statusText.value = "Talk failed: ${err.message ?: err::class.simpleName}" Log.w(tag, "finalize failed: ${err.message ?: err::class.simpleName}") } @@ -344,12 +419,12 @@ class TalkModeManager( val key = sessionKey.trim() if (key.isEmpty()) return if (chatSubscribedSessionKey == key) return - try { - session.sendNodeEvent("chat.subscribe", """{"sessionKey":"$key"}""") + val sent = session.sendNodeEvent("chat.subscribe", """{"sessionKey":"$key"}""") + if (sent) { chatSubscribedSessionKey = key Log.d(tag, "chat.subscribe ok sessionKey=$key") - } catch (err: Throwable) { - Log.w(tag, "chat.subscribe failed sessionKey=$key err=${err.message ?: err::class.java.simpleName}") + } else { + Log.w(tag, "chat.subscribe failed sessionKey=$key") } } @@ -446,7 +521,7 @@ class TalkModeManager( return null } - private suspend fun playAssistant(text: String) { + private suspend fun playAssistant(text: String, playbackToken: Long) { val parsed = TalkDirectiveParser.parse(text) if (parsed.unknownKeys.isNotEmpty()) { Log.w(tag, "Unknown talk directive keys: ${parsed.unknownKeys}") @@ -474,6 +549,7 @@ class TalkModeManager( modelOverrideActive = true } } + ensurePlaybackActive(playbackToken) val apiKey = apiKey?.trim()?.takeIf { it.isNotEmpty() } @@ -500,9 +576,10 @@ class TalkModeManager( if (apiKey.isNullOrEmpty()) { Log.w(tag, "missing ELEVENLABS_API_KEY; falling back to system voice") } + ensurePlaybackActive(playbackToken) _usingFallbackTts.value = true _statusText.value = "Speaking (System)…" - speakWithSystemTts(cleaned) + speakWithSystemTts(cleaned, playbackToken) } else { _usingFallbackTts.value = false val ttsStarted = SystemClock.elapsedRealtime() @@ -523,43 +600,71 @@ class TalkModeManager( language = TalkModeRuntime.validatedLanguage(directive?.language), latencyTier = TalkModeRuntime.validatedLatencyTier(directive?.latencyTier), ) - streamAndPlay(voiceId = voiceId!!, apiKey = apiKey!!, request = request) + streamAndPlay(voiceId = voiceId!!, apiKey = apiKey!!, request = request, playbackToken = playbackToken) Log.d(tag, "elevenlabs stream ok durMs=${SystemClock.elapsedRealtime() - ttsStarted}") } } catch (err: Throwable) { + if (isPlaybackCancelled(err, playbackToken)) { + Log.d(tag, "assistant speech cancelled") + return + } Log.w(tag, "speak failed: ${err.message ?: err::class.simpleName}; falling back to system voice") try { + ensurePlaybackActive(playbackToken) _usingFallbackTts.value = true _statusText.value = "Speaking (System)…" - speakWithSystemTts(cleaned) + speakWithSystemTts(cleaned, playbackToken) } catch (fallbackErr: Throwable) { + if (isPlaybackCancelled(fallbackErr, playbackToken)) { + Log.d(tag, "assistant fallback speech cancelled") + return + } _statusText.value = "Speak failed: ${fallbackErr.message ?: fallbackErr::class.simpleName}" Log.w(tag, "system voice failed: ${fallbackErr.message ?: fallbackErr::class.simpleName}") } + } finally { + _isSpeaking.value = false } - - _isSpeaking.value = false } - private suspend fun streamAndPlay(voiceId: String, apiKey: String, request: ElevenLabsRequest) { + private suspend fun streamAndPlay( + voiceId: String, + apiKey: String, + request: ElevenLabsRequest, + playbackToken: Long, + ) { + ensurePlaybackActive(playbackToken) stopSpeaking(resetInterrupt = false) + ensurePlaybackActive(playbackToken) pcmStopRequested = false val pcmSampleRate = TalkModeRuntime.parsePcmSampleRate(request.outputFormat) if (pcmSampleRate != null) { try { - streamAndPlayPcm(voiceId = voiceId, apiKey = apiKey, request = request, sampleRate = pcmSampleRate) + streamAndPlayPcm( + voiceId = voiceId, + apiKey = apiKey, + request = request, + sampleRate = pcmSampleRate, + playbackToken = playbackToken, + ) return } catch (err: Throwable) { - if (pcmStopRequested) return + if (isPlaybackCancelled(err, playbackToken) || pcmStopRequested) return Log.w(tag, "pcm playback failed; falling back to mp3: ${err.message ?: err::class.simpleName}") } } - streamAndPlayMp3(voiceId = voiceId, apiKey = apiKey, request = request) + ensurePlaybackActive(playbackToken) + streamAndPlayMp3(voiceId = voiceId, apiKey = apiKey, request = request, playbackToken = playbackToken) } - private suspend fun streamAndPlayMp3(voiceId: String, apiKey: String, request: ElevenLabsRequest) { + private suspend fun streamAndPlayMp3( + voiceId: String, + apiKey: String, + request: ElevenLabsRequest, + playbackToken: Long, + ) { val dataSource = StreamingMediaDataSource() streamingSource = dataSource @@ -596,7 +701,7 @@ class TalkModeManager( val fetchJob = scope.launch(Dispatchers.IO) { try { - streamTts(voiceId = voiceId, apiKey = apiKey, request = request, sink = dataSource) + streamTts(voiceId = voiceId, apiKey = apiKey, request = request, sink = dataSource, playbackToken = playbackToken) fetchError.complete(null) } catch (err: Throwable) { dataSource.fail() @@ -606,8 +711,11 @@ class TalkModeManager( Log.d(tag, "play start") try { + ensurePlaybackActive(playbackToken) prepared.await() + ensurePlaybackActive(playbackToken) finished.await() + ensurePlaybackActive(playbackToken) fetchError.await()?.let { throw it } } finally { fetchJob.cancel() @@ -621,7 +729,9 @@ class TalkModeManager( apiKey: String, request: ElevenLabsRequest, sampleRate: Int, + playbackToken: Long, ) { + ensurePlaybackActive(playbackToken) val minBuffer = AudioTrack.getMinBufferSize( sampleRate, @@ -657,20 +767,22 @@ class TalkModeManager( Log.d(tag, "pcm play start sampleRate=$sampleRate bufferSize=$bufferSize") try { - streamPcm(voiceId = voiceId, apiKey = apiKey, request = request, track = track) + streamPcm(voiceId = voiceId, apiKey = apiKey, request = request, track = track, playbackToken = playbackToken) } finally { cleanupPcmTrack() } Log.d(tag, "pcm play done") } - private suspend fun speakWithSystemTts(text: String) { + private suspend fun speakWithSystemTts(text: String, playbackToken: Long) { val trimmed = text.trim() if (trimmed.isEmpty()) return + ensurePlaybackActive(playbackToken) val ok = ensureSystemTts() if (!ok) { throw IllegalStateException("system TTS unavailable") } + ensurePlaybackActive(playbackToken) val tts = systemTts ?: throw IllegalStateException("system TTS unavailable") val utteranceId = "talk-${UUID.randomUUID()}" @@ -680,6 +792,7 @@ class TalkModeManager( systemTtsPendingId = utteranceId withContext(Dispatchers.Main) { + ensurePlaybackActive(playbackToken) val params = Bundle() tts.speak(trimmed, TextToSpeech.QUEUE_FLUSH, params, utteranceId) } @@ -690,6 +803,7 @@ class TalkModeManager( } catch (err: Throwable) { throw err } + ensurePlaybackActive(playbackToken) } } @@ -809,6 +923,23 @@ class TalkModeManager( return true } + private fun ensurePlaybackActive(playbackToken: Long) { + if (!playbackEnabled || playbackToken != playbackGeneration.get()) { + throw CancellationException("assistant speech cancelled") + } + } + + private fun isPlaybackCancelled(err: Throwable?, playbackToken: Long): Boolean { + if (err is CancellationException) return true + return !playbackEnabled || playbackToken != playbackGeneration.get() + } + + private suspend fun ensureConfigLoaded() { + if (!configLoaded) { + reloadConfig() + } + } + private suspend fun reloadConfig() { val envVoice = System.getenv("ELEVENLABS_VOICE_ID")?.trim() val sagVoice = System.getenv("SAG_VOICE_ID")?.trim() @@ -818,30 +949,50 @@ class TalkModeManager( val root = json.parseToJsonElement(res).asObjectOrNull() val config = root?.get("config").asObjectOrNull() val talk = config?.get("talk").asObjectOrNull() + val selection = selectTalkProviderConfig(talk) + val activeProvider = selection?.provider ?: defaultTalkProvider + val activeConfig = selection?.config val sessionCfg = config?.get("session").asObjectOrNull() val mainKey = normalizeMainKey(sessionCfg?.get("mainKey").asStringOrNull()) - val voice = talk?.get("voiceId")?.asStringOrNull()?.trim()?.takeIf { it.isNotEmpty() } + val voice = activeConfig?.get("voiceId")?.asStringOrNull()?.trim()?.takeIf { it.isNotEmpty() } val aliases = - talk?.get("voiceAliases").asObjectOrNull()?.entries?.mapNotNull { (key, value) -> + activeConfig?.get("voiceAliases").asObjectOrNull()?.entries?.mapNotNull { (key, value) -> val id = value.asStringOrNull()?.trim()?.takeIf { it.isNotEmpty() } ?: return@mapNotNull null normalizeAliasKey(key).takeIf { it.isNotEmpty() }?.let { it to id } }?.toMap().orEmpty() - val model = talk?.get("modelId")?.asStringOrNull()?.trim()?.takeIf { it.isNotEmpty() } - val outputFormat = talk?.get("outputFormat")?.asStringOrNull()?.trim()?.takeIf { it.isNotEmpty() } - val key = talk?.get("apiKey")?.asStringOrNull()?.trim()?.takeIf { it.isNotEmpty() } + val model = activeConfig?.get("modelId")?.asStringOrNull()?.trim()?.takeIf { it.isNotEmpty() } + val outputFormat = + activeConfig?.get("outputFormat")?.asStringOrNull()?.trim()?.takeIf { it.isNotEmpty() } + val key = activeConfig?.get("apiKey")?.asStringOrNull()?.trim()?.takeIf { it.isNotEmpty() } val interrupt = talk?.get("interruptOnSpeech")?.asBooleanOrNull() if (!isCanonicalMainSessionKey(mainSessionKey)) { mainSessionKey = mainKey } - defaultVoiceId = voice ?: envVoice?.takeIf { it.isNotEmpty() } ?: sagVoice?.takeIf { it.isNotEmpty() } + defaultVoiceId = + if (activeProvider == defaultTalkProvider) { + voice ?: envVoice?.takeIf { it.isNotEmpty() } ?: sagVoice?.takeIf { it.isNotEmpty() } + } else { + voice + } voiceAliases = aliases if (!voiceOverrideActive) currentVoiceId = defaultVoiceId defaultModelId = model ?: defaultModelIdFallback if (!modelOverrideActive) currentModelId = defaultModelId defaultOutputFormat = outputFormat ?: defaultOutputFormatFallback - apiKey = key ?: envKey?.takeIf { it.isNotEmpty() } + apiKey = + if (activeProvider == defaultTalkProvider) { + key ?: envKey?.takeIf { it.isNotEmpty() } + } else { + null + } if (interrupt != null) interruptOnSpeech = interrupt + if (activeProvider != defaultTalkProvider) { + Log.w(tag, "talk provider $activeProvider unsupported; using system voice fallback") + } else if (selection?.normalizedPayload == true) { + Log.d(tag, "talk config provider=elevenlabs") + } + configLoaded = true } catch (_: Throwable) { defaultVoiceId = envVoice?.takeIf { it.isNotEmpty() } ?: sagVoice?.takeIf { it.isNotEmpty() } defaultModelId = defaultModelIdFallback @@ -849,6 +1000,8 @@ class TalkModeManager( apiKey = envKey?.takeIf { it.isNotEmpty() } voiceAliases = emptyMap() defaultOutputFormat = defaultOutputFormatFallback + // Keep config load retryable after transient fetch failures. + configLoaded = false } } @@ -862,8 +1015,10 @@ class TalkModeManager( apiKey: String, request: ElevenLabsRequest, sink: StreamingMediaDataSource, + playbackToken: Long, ) { withContext(Dispatchers.IO) { + ensurePlaybackActive(playbackToken) val conn = openTtsConnection(voiceId = voiceId, apiKey = apiKey, request = request) try { val payload = buildRequestPayload(request) @@ -879,8 +1034,10 @@ class TalkModeManager( val buffer = ByteArray(8 * 1024) conn.inputStream.use { input -> while (true) { + ensurePlaybackActive(playbackToken) val read = input.read(buffer) if (read <= 0) break + ensurePlaybackActive(playbackToken) sink.append(buffer.copyOf(read)) } } @@ -896,8 +1053,10 @@ class TalkModeManager( apiKey: String, request: ElevenLabsRequest, track: AudioTrack, + playbackToken: Long, ) { withContext(Dispatchers.IO) { + ensurePlaybackActive(playbackToken) val conn = openTtsConnection(voiceId = voiceId, apiKey = apiKey, request = request) try { val payload = buildRequestPayload(request) @@ -912,21 +1071,21 @@ class TalkModeManager( val buffer = ByteArray(8 * 1024) conn.inputStream.use { input -> while (true) { - if (pcmStopRequested) return@withContext + if (pcmStopRequested || isPlaybackCancelled(null, playbackToken)) return@withContext val read = input.read(buffer) if (read <= 0) break var offset = 0 while (offset < read) { - if (pcmStopRequested) return@withContext + if (pcmStopRequested || isPlaybackCancelled(null, playbackToken)) return@withContext val wrote = try { track.write(buffer, offset, read - offset) } catch (err: Throwable) { - if (pcmStopRequested) return@withContext + if (pcmStopRequested || isPlaybackCancelled(err, playbackToken)) return@withContext throw err } if (wrote <= 0) { - if (pcmStopRequested) return@withContext + if (pcmStopRequested || isPlaybackCancelled(null, playbackToken)) return@withContext throw IllegalStateException("AudioTrack write failed: $wrote") } offset += wrote diff --git a/apps/android/app/src/main/res/font/manrope_400_regular.ttf b/apps/android/app/src/main/res/font/manrope_400_regular.ttf new file mode 100644 index 00000000000..9a108f1cee9 Binary files /dev/null and b/apps/android/app/src/main/res/font/manrope_400_regular.ttf differ diff --git a/apps/android/app/src/main/res/font/manrope_500_medium.ttf b/apps/android/app/src/main/res/font/manrope_500_medium.ttf new file mode 100644 index 00000000000..c6d28def6d5 Binary files /dev/null and b/apps/android/app/src/main/res/font/manrope_500_medium.ttf differ diff --git a/apps/android/app/src/main/res/font/manrope_600_semibold.ttf b/apps/android/app/src/main/res/font/manrope_600_semibold.ttf new file mode 100644 index 00000000000..46a13d61989 Binary files /dev/null and b/apps/android/app/src/main/res/font/manrope_600_semibold.ttf differ diff --git a/apps/android/app/src/main/res/font/manrope_700_bold.ttf b/apps/android/app/src/main/res/font/manrope_700_bold.ttf new file mode 100644 index 00000000000..62a61839390 Binary files /dev/null and b/apps/android/app/src/main/res/font/manrope_700_bold.ttf differ diff --git a/apps/android/app/src/test/java/ai/openclaw/android/gateway/DeviceAuthPayloadTest.kt b/apps/android/app/src/test/java/ai/openclaw/android/gateway/DeviceAuthPayloadTest.kt new file mode 100644 index 00000000000..95e145fb11f --- /dev/null +++ b/apps/android/app/src/test/java/ai/openclaw/android/gateway/DeviceAuthPayloadTest.kt @@ -0,0 +1,35 @@ +package ai.openclaw.android.gateway + +import org.junit.Assert.assertEquals +import org.junit.Test + +class DeviceAuthPayloadTest { + @Test + fun buildV3_matchesCanonicalVector() { + val payload = + DeviceAuthPayload.buildV3( + deviceId = "dev-1", + clientId = "openclaw-macos", + clientMode = "ui", + role = "operator", + scopes = listOf("operator.admin", "operator.read"), + signedAtMs = 1_700_000_000_000, + token = "tok-123", + nonce = "nonce-abc", + platform = " IOS ", + deviceFamily = " iPhone ", + ) + + assertEquals( + "v3|dev-1|openclaw-macos|ui|operator|operator.admin,operator.read|1700000000000|tok-123|nonce-abc|ios|iphone", + payload, + ) + } + + @Test + fun normalizeMetadataField_asciiOnlyLowercase() { + assertEquals("İos", DeviceAuthPayload.normalizeMetadataField(" İOS ")) + assertEquals("mac", DeviceAuthPayload.normalizeMetadataField(" MAC ")) + assertEquals("", DeviceAuthPayload.normalizeMetadataField(null)) + } +} diff --git a/apps/android/app/src/test/java/ai/openclaw/android/gateway/GatewaySessionInvokeTest.kt b/apps/android/app/src/test/java/ai/openclaw/android/gateway/GatewaySessionInvokeTest.kt new file mode 100644 index 00000000000..8271d395a7d --- /dev/null +++ b/apps/android/app/src/test/java/ai/openclaw/android/gateway/GatewaySessionInvokeTest.kt @@ -0,0 +1,566 @@ +package ai.openclaw.android.gateway + +import kotlinx.coroutines.CompletableDeferred +import kotlinx.coroutines.CoroutineScope +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.SupervisorJob +import kotlinx.coroutines.cancelAndJoin +import kotlinx.coroutines.runBlocking +import kotlinx.coroutines.withTimeout +import kotlinx.coroutines.withTimeoutOrNull +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.jsonObject +import kotlinx.serialization.json.jsonPrimitive +import okhttp3.Response +import okhttp3.WebSocket +import okhttp3.WebSocketListener +import okhttp3.mockwebserver.Dispatcher +import okhttp3.mockwebserver.MockResponse +import okhttp3.mockwebserver.MockWebServer +import okhttp3.mockwebserver.RecordedRequest +import org.junit.Assert.assertEquals +import org.junit.Assert.assertNull +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.RuntimeEnvironment +import org.robolectric.annotation.Config +import java.util.concurrent.atomic.AtomicReference + +private class InMemoryDeviceAuthStore : DeviceAuthTokenStore { + private val tokens = mutableMapOf() + + override fun loadToken(deviceId: String, role: String): String? = tokens["${deviceId.trim()}|${role.trim()}"]?.trim()?.takeIf { it.isNotEmpty() } + + override fun saveToken(deviceId: String, role: String, token: String) { + tokens["${deviceId.trim()}|${role.trim()}"] = token.trim() + } +} + +@RunWith(RobolectricTestRunner::class) +@Config(sdk = [34]) +class GatewaySessionInvokeTest { + @Test + fun nodeInvokeRequest_roundTripsInvokeResult() = runBlocking { + val json = Json { ignoreUnknownKeys = true } + val connected = CompletableDeferred() + val invokeRequest = CompletableDeferred() + val invokeResultParams = CompletableDeferred() + val handshakeOrigin = AtomicReference(null) + val lastDisconnect = AtomicReference("") + val server = + MockWebServer().apply { + dispatcher = + object : Dispatcher() { + override fun dispatch(request: RecordedRequest): MockResponse { + handshakeOrigin.compareAndSet(null, request.getHeader("Origin")) + return MockResponse().withWebSocketUpgrade( + object : WebSocketListener() { + override fun onOpen(webSocket: WebSocket, response: Response) { + webSocket.send( + """{"type":"event","event":"connect.challenge","payload":{"nonce":"android-test-nonce"}}""", + ) + } + + override fun onMessage(webSocket: WebSocket, text: String) { + val frame = json.parseToJsonElement(text).jsonObject + if (frame["type"]?.jsonPrimitive?.content != "req") return + val id = frame["id"]?.jsonPrimitive?.content ?: return + val method = frame["method"]?.jsonPrimitive?.content ?: return + when (method) { + "connect" -> { + webSocket.send( + """{"type":"res","id":"$id","ok":true,"payload":{"snapshot":{"sessionDefaults":{"mainSessionKey":"main"}}}}""", + ) + webSocket.send( + """{"type":"event","event":"node.invoke.request","payload":{"id":"invoke-1","nodeId":"node-1","command":"debug.ping","params":{"ping":"pong"},"timeoutMs":5000}}""", + ) + } + "node.invoke.result" -> { + if (!invokeResultParams.isCompleted) { + invokeResultParams.complete(frame["params"]?.toString().orEmpty()) + } + webSocket.send("""{"type":"res","id":"$id","ok":true,"payload":{"ok":true}}""") + webSocket.close(1000, "done") + } + } + } + }, + ) + } + } + start() + } + + val app = RuntimeEnvironment.getApplication() + val sessionJob = SupervisorJob() + val deviceAuthStore = InMemoryDeviceAuthStore() + val session = + GatewaySession( + scope = CoroutineScope(sessionJob + Dispatchers.Default), + identityStore = DeviceIdentityStore(app), + deviceAuthStore = deviceAuthStore, + onConnected = { _, _, _ -> + if (!connected.isCompleted) connected.complete(Unit) + }, + onDisconnected = { message -> + lastDisconnect.set(message) + }, + onEvent = { _, _ -> }, + onInvoke = { req -> + if (!invokeRequest.isCompleted) invokeRequest.complete(req) + GatewaySession.InvokeResult.ok("""{"handled":true}""") + }, + ) + + try { + session.connect( + endpoint = + GatewayEndpoint( + stableId = "manual|127.0.0.1|${server.port}", + name = "test", + host = "127.0.0.1", + port = server.port, + tlsEnabled = false, + ), + token = "test-token", + password = null, + options = + GatewayConnectOptions( + role = "node", + scopes = listOf("node:invoke"), + caps = emptyList(), + commands = emptyList(), + permissions = emptyMap(), + client = + GatewayClientInfo( + id = "openclaw-android-test", + displayName = "Android Test", + version = "1.0.0-test", + platform = "android", + mode = "node", + instanceId = "android-test-instance", + deviceFamily = "android", + modelIdentifier = "test", + ), + ), + tls = null, + ) + + val connectedWithinTimeout = withTimeoutOrNull(8_000) { + connected.await() + true + } == true + if (!connectedWithinTimeout) { + throw AssertionError("never connected; lastDisconnect=${lastDisconnect.get()}; requests=${server.requestCount}") + } + val req = withTimeout(8_000) { invokeRequest.await() } + val resultParamsJson = withTimeout(8_000) { invokeResultParams.await() } + val resultParams = json.parseToJsonElement(resultParamsJson).jsonObject + + assertEquals("invoke-1", req.id) + assertEquals("node-1", req.nodeId) + assertEquals("debug.ping", req.command) + assertEquals("""{"ping":"pong"}""", req.paramsJson) + assertNull(handshakeOrigin.get()) + assertEquals("invoke-1", resultParams["id"]?.jsonPrimitive?.content) + assertEquals("node-1", resultParams["nodeId"]?.jsonPrimitive?.content) + assertEquals(true, resultParams["ok"]?.jsonPrimitive?.content?.toBooleanStrict()) + assertEquals( + true, + resultParams["payload"]?.jsonObject?.get("handled")?.jsonPrimitive?.content?.toBooleanStrict(), + ) + } finally { + session.disconnect() + sessionJob.cancelAndJoin() + server.shutdown() + } + } + + @Test + fun nodeInvokeRequest_usesParamsJsonWhenProvided() = runBlocking { + val json = Json { ignoreUnknownKeys = true } + val connected = CompletableDeferred() + val invokeRequest = CompletableDeferred() + val invokeResultParams = CompletableDeferred() + val lastDisconnect = AtomicReference("") + val server = + MockWebServer().apply { + dispatcher = + object : Dispatcher() { + override fun dispatch(request: RecordedRequest): MockResponse { + return MockResponse().withWebSocketUpgrade( + object : WebSocketListener() { + override fun onOpen(webSocket: WebSocket, response: Response) { + webSocket.send( + """{"type":"event","event":"connect.challenge","payload":{"nonce":"android-test-nonce"}}""", + ) + } + + override fun onMessage(webSocket: WebSocket, text: String) { + val frame = json.parseToJsonElement(text).jsonObject + if (frame["type"]?.jsonPrimitive?.content != "req") return + val id = frame["id"]?.jsonPrimitive?.content ?: return + val method = frame["method"]?.jsonPrimitive?.content ?: return + when (method) { + "connect" -> { + webSocket.send( + """{"type":"res","id":"$id","ok":true,"payload":{"snapshot":{"sessionDefaults":{"mainSessionKey":"main"}}}}""", + ) + webSocket.send( + """{"type":"event","event":"node.invoke.request","payload":{"id":"invoke-2","nodeId":"node-2","command":"debug.raw","paramsJSON":"{\"raw\":true}","params":{"ignored":1},"timeoutMs":5000}}""", + ) + } + "node.invoke.result" -> { + if (!invokeResultParams.isCompleted) { + invokeResultParams.complete(frame["params"]?.toString().orEmpty()) + } + webSocket.send("""{"type":"res","id":"$id","ok":true,"payload":{"ok":true}}""") + webSocket.close(1000, "done") + } + } + } + }, + ) + } + } + start() + } + + val app = RuntimeEnvironment.getApplication() + val sessionJob = SupervisorJob() + val deviceAuthStore = InMemoryDeviceAuthStore() + val session = + GatewaySession( + scope = CoroutineScope(sessionJob + Dispatchers.Default), + identityStore = DeviceIdentityStore(app), + deviceAuthStore = deviceAuthStore, + onConnected = { _, _, _ -> + if (!connected.isCompleted) connected.complete(Unit) + }, + onDisconnected = { message -> + lastDisconnect.set(message) + }, + onEvent = { _, _ -> }, + onInvoke = { req -> + if (!invokeRequest.isCompleted) invokeRequest.complete(req) + GatewaySession.InvokeResult.ok("""{"handled":true}""") + }, + ) + + try { + session.connect( + endpoint = + GatewayEndpoint( + stableId = "manual|127.0.0.1|${server.port}", + name = "test", + host = "127.0.0.1", + port = server.port, + tlsEnabled = false, + ), + token = "test-token", + password = null, + options = + GatewayConnectOptions( + role = "node", + scopes = listOf("node:invoke"), + caps = emptyList(), + commands = emptyList(), + permissions = emptyMap(), + client = + GatewayClientInfo( + id = "openclaw-android-test", + displayName = "Android Test", + version = "1.0.0-test", + platform = "android", + mode = "node", + instanceId = "android-test-instance", + deviceFamily = "android", + modelIdentifier = "test", + ), + ), + tls = null, + ) + + val connectedWithinTimeout = withTimeoutOrNull(8_000) { + connected.await() + true + } == true + if (!connectedWithinTimeout) { + throw AssertionError("never connected; lastDisconnect=${lastDisconnect.get()}; requests=${server.requestCount}") + } + + val req = withTimeout(8_000) { invokeRequest.await() } + val resultParamsJson = withTimeout(8_000) { invokeResultParams.await() } + val resultParams = json.parseToJsonElement(resultParamsJson).jsonObject + + assertEquals("invoke-2", req.id) + assertEquals("node-2", req.nodeId) + assertEquals("debug.raw", req.command) + assertEquals("""{"raw":true}""", req.paramsJson) + assertEquals("invoke-2", resultParams["id"]?.jsonPrimitive?.content) + assertEquals("node-2", resultParams["nodeId"]?.jsonPrimitive?.content) + assertEquals(true, resultParams["ok"]?.jsonPrimitive?.content?.toBooleanStrict()) + } finally { + session.disconnect() + sessionJob.cancelAndJoin() + server.shutdown() + } + } + + @Test + fun nodeInvokeRequest_mapsCodePrefixedErrorsIntoInvokeResult() = runBlocking { + val json = Json { ignoreUnknownKeys = true } + val connected = CompletableDeferred() + val invokeResultParams = CompletableDeferred() + val lastDisconnect = AtomicReference("") + val server = + MockWebServer().apply { + dispatcher = + object : Dispatcher() { + override fun dispatch(request: RecordedRequest): MockResponse { + return MockResponse().withWebSocketUpgrade( + object : WebSocketListener() { + override fun onOpen(webSocket: WebSocket, response: Response) { + webSocket.send( + """{"type":"event","event":"connect.challenge","payload":{"nonce":"android-test-nonce"}}""", + ) + } + + override fun onMessage(webSocket: WebSocket, text: String) { + val frame = json.parseToJsonElement(text).jsonObject + if (frame["type"]?.jsonPrimitive?.content != "req") return + val id = frame["id"]?.jsonPrimitive?.content ?: return + val method = frame["method"]?.jsonPrimitive?.content ?: return + when (method) { + "connect" -> { + webSocket.send( + """{"type":"res","id":"$id","ok":true,"payload":{"snapshot":{"sessionDefaults":{"mainSessionKey":"main"}}}}""", + ) + webSocket.send( + """{"type":"event","event":"node.invoke.request","payload":{"id":"invoke-3","nodeId":"node-3","command":"camera.snap","params":{"facing":"front"},"timeoutMs":5000}}""", + ) + } + "node.invoke.result" -> { + if (!invokeResultParams.isCompleted) { + invokeResultParams.complete(frame["params"]?.toString().orEmpty()) + } + webSocket.send("""{"type":"res","id":"$id","ok":true,"payload":{"ok":true}}""") + webSocket.close(1000, "done") + } + } + } + }, + ) + } + } + start() + } + + val app = RuntimeEnvironment.getApplication() + val sessionJob = SupervisorJob() + val deviceAuthStore = InMemoryDeviceAuthStore() + val session = + GatewaySession( + scope = CoroutineScope(sessionJob + Dispatchers.Default), + identityStore = DeviceIdentityStore(app), + deviceAuthStore = deviceAuthStore, + onConnected = { _, _, _ -> + if (!connected.isCompleted) connected.complete(Unit) + }, + onDisconnected = { message -> + lastDisconnect.set(message) + }, + onEvent = { _, _ -> }, + onInvoke = { + throw IllegalStateException("CAMERA_PERMISSION_REQUIRED: grant Camera permission") + }, + ) + + try { + session.connect( + endpoint = + GatewayEndpoint( + stableId = "manual|127.0.0.1|${server.port}", + name = "test", + host = "127.0.0.1", + port = server.port, + tlsEnabled = false, + ), + token = "test-token", + password = null, + options = + GatewayConnectOptions( + role = "node", + scopes = listOf("node:invoke"), + caps = emptyList(), + commands = emptyList(), + permissions = emptyMap(), + client = + GatewayClientInfo( + id = "openclaw-android-test", + displayName = "Android Test", + version = "1.0.0-test", + platform = "android", + mode = "node", + instanceId = "android-test-instance", + deviceFamily = "android", + modelIdentifier = "test", + ), + ), + tls = null, + ) + + val connectedWithinTimeout = withTimeoutOrNull(8_000) { + connected.await() + true + } == true + if (!connectedWithinTimeout) { + throw AssertionError("never connected; lastDisconnect=${lastDisconnect.get()}; requests=${server.requestCount}") + } + + val resultParamsJson = withTimeout(8_000) { invokeResultParams.await() } + val resultParams = json.parseToJsonElement(resultParamsJson).jsonObject + + assertEquals("invoke-3", resultParams["id"]?.jsonPrimitive?.content) + assertEquals("node-3", resultParams["nodeId"]?.jsonPrimitive?.content) + assertEquals(false, resultParams["ok"]?.jsonPrimitive?.content?.toBooleanStrict()) + assertEquals( + "CAMERA_PERMISSION_REQUIRED", + resultParams["error"]?.jsonObject?.get("code")?.jsonPrimitive?.content, + ) + assertEquals( + "grant Camera permission", + resultParams["error"]?.jsonObject?.get("message")?.jsonPrimitive?.content, + ) + } finally { + session.disconnect() + sessionJob.cancelAndJoin() + server.shutdown() + } + } + + @Test + fun refreshNodeCanvasCapability_sendsObjectParamsAndUpdatesScopedUrl() = runBlocking { + val json = Json { ignoreUnknownKeys = true } + val connected = CompletableDeferred() + val refreshRequestParams = CompletableDeferred() + val lastDisconnect = AtomicReference("") + val server = + MockWebServer().apply { + dispatcher = + object : Dispatcher() { + override fun dispatch(request: RecordedRequest): MockResponse { + return MockResponse().withWebSocketUpgrade( + object : WebSocketListener() { + override fun onOpen(webSocket: WebSocket, response: Response) { + webSocket.send( + """{"type":"event","event":"connect.challenge","payload":{"nonce":"android-test-nonce"}}""", + ) + } + + override fun onMessage(webSocket: WebSocket, text: String) { + val frame = json.parseToJsonElement(text).jsonObject + if (frame["type"]?.jsonPrimitive?.content != "req") return + val id = frame["id"]?.jsonPrimitive?.content ?: return + val method = frame["method"]?.jsonPrimitive?.content ?: return + when (method) { + "connect" -> { + webSocket.send( + """{"type":"res","id":"$id","ok":true,"payload":{"canvasHostUrl":"http://127.0.0.1/__openclaw__/cap/old-cap","snapshot":{"sessionDefaults":{"mainSessionKey":"main"}}}}""", + ) + } + "node.canvas.capability.refresh" -> { + if (!refreshRequestParams.isCompleted) { + refreshRequestParams.complete(frame["params"]?.toString()) + } + webSocket.send( + """{"type":"res","id":"$id","ok":true,"payload":{"canvasCapability":"new-cap"}}""", + ) + webSocket.close(1000, "done") + } + } + } + }, + ) + } + } + start() + } + + val app = RuntimeEnvironment.getApplication() + val sessionJob = SupervisorJob() + val deviceAuthStore = InMemoryDeviceAuthStore() + val session = + GatewaySession( + scope = CoroutineScope(sessionJob + Dispatchers.Default), + identityStore = DeviceIdentityStore(app), + deviceAuthStore = deviceAuthStore, + onConnected = { _, _, _ -> + if (!connected.isCompleted) connected.complete(Unit) + }, + onDisconnected = { message -> + lastDisconnect.set(message) + }, + onEvent = { _, _ -> }, + onInvoke = { GatewaySession.InvokeResult.ok("""{"handled":true}""") }, + ) + + try { + session.connect( + endpoint = + GatewayEndpoint( + stableId = "manual|127.0.0.1|${server.port}", + name = "test", + host = "127.0.0.1", + port = server.port, + tlsEnabled = false, + ), + token = "test-token", + password = null, + options = + GatewayConnectOptions( + role = "node", + scopes = listOf("node:invoke"), + caps = emptyList(), + commands = emptyList(), + permissions = emptyMap(), + client = + GatewayClientInfo( + id = "openclaw-android-test", + displayName = "Android Test", + version = "1.0.0-test", + platform = "android", + mode = "node", + instanceId = "android-test-instance", + deviceFamily = "android", + modelIdentifier = "test", + ), + ), + tls = null, + ) + + val connectedWithinTimeout = withTimeoutOrNull(8_000) { + connected.await() + true + } == true + if (!connectedWithinTimeout) { + throw AssertionError("never connected; lastDisconnect=${lastDisconnect.get()}; requests=${server.requestCount}") + } + + val refreshed = session.refreshNodeCanvasCapability(timeoutMs = 8_000) + val refreshParamsJson = withTimeout(8_000) { refreshRequestParams.await() } + + assertEquals(true, refreshed) + assertEquals("{}", refreshParamsJson) + assertEquals( + "http://127.0.0.1:${server.port}/__openclaw__/cap/new-cap", + session.currentCanvasHostUrl(), + ) + } finally { + session.disconnect() + sessionJob.cancelAndJoin() + server.shutdown() + } + } +} diff --git a/apps/android/app/src/test/java/ai/openclaw/android/gateway/GatewaySessionInvokeTimeoutTest.kt b/apps/android/app/src/test/java/ai/openclaw/android/gateway/GatewaySessionInvokeTimeoutTest.kt new file mode 100644 index 00000000000..cd08715c405 --- /dev/null +++ b/apps/android/app/src/test/java/ai/openclaw/android/gateway/GatewaySessionInvokeTimeoutTest.kt @@ -0,0 +1,47 @@ +package ai.openclaw.android.gateway + +import org.junit.Assert.assertEquals +import org.junit.Test + +class GatewaySessionInvokeTimeoutTest { + @Test + fun resolveInvokeResultAckTimeoutMs_usesFloorWhenMissingOrTooSmall() { + assertEquals(15_000L, resolveInvokeResultAckTimeoutMs(null)) + assertEquals(15_000L, resolveInvokeResultAckTimeoutMs(0L)) + assertEquals(15_000L, resolveInvokeResultAckTimeoutMs(5_000L)) + } + + @Test + fun resolveInvokeResultAckTimeoutMs_usesInvokeBudgetWithinBounds() { + assertEquals(30_000L, resolveInvokeResultAckTimeoutMs(30_000L)) + assertEquals(90_000L, resolveInvokeResultAckTimeoutMs(90_000L)) + } + + @Test + fun resolveInvokeResultAckTimeoutMs_capsAtUpperBound() { + assertEquals(120_000L, resolveInvokeResultAckTimeoutMs(121_000L)) + assertEquals(120_000L, resolveInvokeResultAckTimeoutMs(Long.MAX_VALUE)) + } + + @Test + fun replaceCanvasCapabilityInScopedHostUrl_rewritesTerminalCapabilitySegment() { + assertEquals( + "http://127.0.0.1:18789/__openclaw__/cap/new-token", + replaceCanvasCapabilityInScopedHostUrl( + "http://127.0.0.1:18789/__openclaw__/cap/old-token", + "new-token", + ), + ) + } + + @Test + fun replaceCanvasCapabilityInScopedHostUrl_rewritesWhenQueryAndFragmentPresent() { + assertEquals( + "http://127.0.0.1:18789/__openclaw__/cap/new-token?a=1#frag", + replaceCanvasCapabilityInScopedHostUrl( + "http://127.0.0.1:18789/__openclaw__/cap/old-token?a=1#frag", + "new-token", + ), + ) + } +} diff --git a/apps/android/app/src/test/java/ai/openclaw/android/gateway/InvokeErrorParserTest.kt b/apps/android/app/src/test/java/ai/openclaw/android/gateway/InvokeErrorParserTest.kt new file mode 100644 index 00000000000..ca8e8f21424 --- /dev/null +++ b/apps/android/app/src/test/java/ai/openclaw/android/gateway/InvokeErrorParserTest.kt @@ -0,0 +1,33 @@ +package ai.openclaw.android.gateway + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Test + +class InvokeErrorParserTest { + @Test + fun parseInvokeErrorMessage_parsesUppercaseCodePrefix() { + val parsed = parseInvokeErrorMessage("CAMERA_PERMISSION_REQUIRED: grant Camera permission") + assertEquals("CAMERA_PERMISSION_REQUIRED", parsed.code) + assertEquals("grant Camera permission", parsed.message) + assertTrue(parsed.hadExplicitCode) + assertEquals("CAMERA_PERMISSION_REQUIRED: grant Camera permission", parsed.prefixedMessage) + } + + @Test + fun parseInvokeErrorMessage_rejectsNonCanonicalCodePrefix() { + val parsed = parseInvokeErrorMessage("IllegalStateException: boom") + assertEquals("UNAVAILABLE", parsed.code) + assertEquals("IllegalStateException: boom", parsed.message) + assertFalse(parsed.hadExplicitCode) + } + + @Test + fun parseInvokeErrorFromThrowable_usesFallbackWhenMessageMissing() { + val parsed = parseInvokeErrorFromThrowable(IllegalStateException(), fallbackMessage = "fallback") + assertEquals("UNAVAILABLE", parsed.code) + assertEquals("fallback", parsed.message) + assertFalse(parsed.hadExplicitCode) + } +} diff --git a/apps/android/app/src/test/java/ai/openclaw/android/node/CalendarHandlerTest.kt b/apps/android/app/src/test/java/ai/openclaw/android/node/CalendarHandlerTest.kt new file mode 100644 index 00000000000..a2d8e0919fd --- /dev/null +++ b/apps/android/app/src/test/java/ai/openclaw/android/node/CalendarHandlerTest.kt @@ -0,0 +1,116 @@ +package ai.openclaw.android.node + +import android.content.Context +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.jsonArray +import kotlinx.serialization.json.jsonObject +import kotlinx.serialization.json.jsonPrimitive +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.RuntimeEnvironment + +@RunWith(RobolectricTestRunner::class) +class CalendarHandlerTest { + @Test + fun handleCalendarEvents_requiresPermission() { + val handler = CalendarHandler.forTesting(appContext(), FakeCalendarDataSource(canRead = false)) + + val result = handler.handleCalendarEvents(null) + + assertFalse(result.ok) + assertEquals("CALENDAR_PERMISSION_REQUIRED", result.error?.code) + } + + @Test + fun handleCalendarAdd_rejectsEndBeforeStart() { + val handler = CalendarHandler.forTesting(appContext(), FakeCalendarDataSource(canRead = true, canWrite = true)) + + val result = + handler.handleCalendarAdd( + """{"title":"Standup","startISO":"2026-02-28T10:00:00Z","endISO":"2026-02-28T09:00:00Z"}""", + ) + + assertFalse(result.ok) + assertEquals("CALENDAR_INVALID", result.error?.code) + } + + @Test + fun handleCalendarEvents_returnsEvents() { + val event = + CalendarEventRecord( + identifier = "101", + title = "Sprint Planning", + startISO = "2026-02-28T10:00:00Z", + endISO = "2026-02-28T11:00:00Z", + isAllDay = false, + location = "Room 1", + calendarTitle = "Work", + ) + val handler = + CalendarHandler.forTesting( + appContext(), + FakeCalendarDataSource(canRead = true, events = listOf(event)), + ) + + val result = handler.handleCalendarEvents("""{"limit":1}""") + + assertTrue(result.ok) + val payload = Json.parseToJsonElement(result.payloadJson ?: error("missing payload")).jsonObject + val events = payload.getValue("events").jsonArray + assertEquals(1, events.size) + assertEquals("Sprint Planning", events.first().jsonObject.getValue("title").jsonPrimitive.content) + } + + @Test + fun handleCalendarAdd_mapsNotFoundErrorCode() { + val source = + FakeCalendarDataSource( + canRead = true, + canWrite = true, + addError = IllegalArgumentException("CALENDAR_NOT_FOUND: no default calendar"), + ) + val handler = CalendarHandler.forTesting(appContext(), source) + + val result = + handler.handleCalendarAdd( + """{"title":"Call","startISO":"2026-02-28T10:00:00Z","endISO":"2026-02-28T11:00:00Z"}""", + ) + + assertFalse(result.ok) + assertEquals("CALENDAR_NOT_FOUND", result.error?.code) + } + + private fun appContext(): Context = RuntimeEnvironment.getApplication() +} + +private class FakeCalendarDataSource( + private val canRead: Boolean, + private val canWrite: Boolean = false, + private val events: List = emptyList(), + private val addResult: CalendarEventRecord = + CalendarEventRecord( + identifier = "0", + title = "Default", + startISO = "2026-01-01T00:00:00Z", + endISO = "2026-01-01T01:00:00Z", + isAllDay = false, + location = null, + calendarTitle = null, + ), + private val addError: Throwable? = null, +) : CalendarDataSource { + override fun hasReadPermission(context: Context): Boolean = canRead + + override fun hasWritePermission(context: Context): Boolean = canWrite + + override fun events(context: Context, request: CalendarEventsRequest): List = events + + override fun add(context: Context, request: CalendarAddRequest): CalendarEventRecord { + addError?.let { throw it } + return addResult + } +} diff --git a/apps/android/app/src/test/java/ai/openclaw/android/node/CameraHandlerTest.kt b/apps/android/app/src/test/java/ai/openclaw/android/node/CameraHandlerTest.kt new file mode 100644 index 00000000000..470f925a7d4 --- /dev/null +++ b/apps/android/app/src/test/java/ai/openclaw/android/node/CameraHandlerTest.kt @@ -0,0 +1,25 @@ +package ai.openclaw.android.node + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Test + +class CameraHandlerTest { + @Test + fun isCameraClipWithinPayloadLimit_allowsZeroAndLimit() { + assertTrue(isCameraClipWithinPayloadLimit(0L)) + assertTrue(isCameraClipWithinPayloadLimit(CAMERA_CLIP_MAX_RAW_BYTES)) + } + + @Test + fun isCameraClipWithinPayloadLimit_rejectsNegativeAndTooLarge() { + assertFalse(isCameraClipWithinPayloadLimit(-1L)) + assertFalse(isCameraClipWithinPayloadLimit(CAMERA_CLIP_MAX_RAW_BYTES + 1L)) + } + + @Test + fun cameraClipMaxRawBytes_matchesExpectedBudget() { + assertEquals(18L * 1024L * 1024L, CAMERA_CLIP_MAX_RAW_BYTES) + } +} diff --git a/apps/android/app/src/test/java/ai/openclaw/android/node/ContactsHandlerTest.kt b/apps/android/app/src/test/java/ai/openclaw/android/node/ContactsHandlerTest.kt new file mode 100644 index 00000000000..61af8e0df66 --- /dev/null +++ b/apps/android/app/src/test/java/ai/openclaw/android/node/ContactsHandlerTest.kt @@ -0,0 +1,127 @@ +package ai.openclaw.android.node + +import android.content.Context +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.jsonArray +import kotlinx.serialization.json.jsonObject +import kotlinx.serialization.json.jsonPrimitive +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.RuntimeEnvironment + +@RunWith(RobolectricTestRunner::class) +class ContactsHandlerTest { + @Test + fun handleContactsSearch_requiresReadPermission() { + val handler = ContactsHandler.forTesting(appContext(), FakeContactsDataSource(canRead = false)) + + val result = handler.handleContactsSearch(null) + + assertFalse(result.ok) + assertEquals("CONTACTS_PERMISSION_REQUIRED", result.error?.code) + } + + @Test + fun handleContactsAdd_rejectsEmptyContact() { + val handler = + ContactsHandler.forTesting( + appContext(), + FakeContactsDataSource(canRead = true, canWrite = true), + ) + + val result = handler.handleContactsAdd("""{"givenName":" ","emails":[]}""") + + assertFalse(result.ok) + assertEquals("CONTACTS_INVALID", result.error?.code) + } + + @Test + fun handleContactsSearch_returnsContacts() { + val contact = + ContactRecord( + identifier = "1", + displayName = "Ada Lovelace", + givenName = "Ada", + familyName = "Lovelace", + organizationName = "Analytical Engine", + phoneNumbers = listOf("+12025550123"), + emails = listOf("ada@example.com"), + ) + val handler = + ContactsHandler.forTesting( + appContext(), + FakeContactsDataSource(canRead = true, searchResults = listOf(contact)), + ) + + val result = handler.handleContactsSearch("""{"query":"ada","limit":1}""") + + assertTrue(result.ok) + val payload = Json.parseToJsonElement(result.payloadJson ?: error("missing payload")).jsonObject + val contacts = payload.getValue("contacts").jsonArray + assertEquals(1, contacts.size) + assertEquals("Ada Lovelace", contacts.first().jsonObject.getValue("displayName").jsonPrimitive.content) + } + + @Test + fun handleContactsAdd_returnsAddedContact() { + val added = + ContactRecord( + identifier = "2", + displayName = "Grace Hopper", + givenName = "Grace", + familyName = "Hopper", + organizationName = "US Navy", + phoneNumbers = listOf(), + emails = listOf("grace@example.com"), + ) + val source = FakeContactsDataSource(canRead = true, canWrite = true, addResult = added) + val handler = ContactsHandler.forTesting(appContext(), source) + + val result = + handler.handleContactsAdd( + """{"givenName":"Grace","familyName":"Hopper","emails":["grace@example.com"]}""", + ) + + assertTrue(result.ok) + val payload = Json.parseToJsonElement(result.payloadJson ?: error("missing payload")).jsonObject + val contact = payload.getValue("contact").jsonObject + assertEquals("Grace Hopper", contact.getValue("displayName").jsonPrimitive.content) + assertEquals(1, source.addCalls) + } + + private fun appContext(): Context = RuntimeEnvironment.getApplication() +} + +private class FakeContactsDataSource( + private val canRead: Boolean, + private val canWrite: Boolean = false, + private val searchResults: List = emptyList(), + private val addResult: ContactRecord = + ContactRecord( + identifier = "0", + displayName = "Default", + givenName = "", + familyName = "", + organizationName = "", + phoneNumbers = emptyList(), + emails = emptyList(), + ), +) : ContactsDataSource { + var addCalls: Int = 0 + private set + + override fun hasReadPermission(context: Context): Boolean = canRead + + override fun hasWritePermission(context: Context): Boolean = canWrite + + override fun search(context: Context, request: ContactsSearchRequest): List = searchResults + + override fun add(context: Context, request: ContactsAddRequest): ContactRecord { + addCalls += 1 + return addResult + } +} diff --git a/apps/android/app/src/test/java/ai/openclaw/android/node/DeviceHandlerTest.kt b/apps/android/app/src/test/java/ai/openclaw/android/node/DeviceHandlerTest.kt new file mode 100644 index 00000000000..6232b0c9e11 --- /dev/null +++ b/apps/android/app/src/test/java/ai/openclaw/android/node/DeviceHandlerTest.kt @@ -0,0 +1,149 @@ +package ai.openclaw.android.node + +import android.content.Context +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.boolean +import kotlinx.serialization.json.double +import kotlinx.serialization.json.jsonArray +import kotlinx.serialization.json.jsonObject +import kotlinx.serialization.json.jsonPrimitive +import org.junit.Assert.assertEquals +import org.junit.Assert.assertTrue +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.RuntimeEnvironment + +@RunWith(RobolectricTestRunner::class) +class DeviceHandlerTest { + @Test + fun handleDeviceInfo_returnsStablePayload() { + val handler = DeviceHandler(appContext()) + + val result = handler.handleDeviceInfo(null) + + assertTrue(result.ok) + val payload = parsePayload(result.payloadJson) + assertEquals("Android", payload.getValue("systemName").jsonPrimitive.content) + assertTrue(payload.getValue("deviceName").jsonPrimitive.content.isNotBlank()) + assertTrue(payload.getValue("modelIdentifier").jsonPrimitive.content.isNotBlank()) + assertTrue(payload.getValue("systemVersion").jsonPrimitive.content.isNotBlank()) + assertTrue(payload.getValue("appVersion").jsonPrimitive.content.isNotBlank()) + assertTrue(payload.getValue("appBuild").jsonPrimitive.content.isNotBlank()) + assertTrue(payload.getValue("locale").jsonPrimitive.content.isNotBlank()) + } + + @Test + fun handleDeviceStatus_returnsExpectedShape() { + val handler = DeviceHandler(appContext()) + + val result = handler.handleDeviceStatus(null) + + assertTrue(result.ok) + val payload = parsePayload(result.payloadJson) + val battery = payload.getValue("battery").jsonObject + val storage = payload.getValue("storage").jsonObject + val thermal = payload.getValue("thermal").jsonObject + val network = payload.getValue("network").jsonObject + + val state = battery.getValue("state").jsonPrimitive.content + assertTrue(state in setOf("unknown", "unplugged", "charging", "full")) + battery["level"]?.jsonPrimitive?.double?.let { level -> + assertTrue(level in 0.0..1.0) + } + battery.getValue("lowPowerModeEnabled").jsonPrimitive.boolean + + val totalBytes = storage.getValue("totalBytes").jsonPrimitive.content.toLong() + val freeBytes = storage.getValue("freeBytes").jsonPrimitive.content.toLong() + val usedBytes = storage.getValue("usedBytes").jsonPrimitive.content.toLong() + assertTrue(totalBytes >= 0L) + assertTrue(freeBytes >= 0L) + assertTrue(usedBytes >= 0L) + assertEquals((totalBytes - freeBytes).coerceAtLeast(0L), usedBytes) + + val thermalState = thermal.getValue("state").jsonPrimitive.content + assertTrue(thermalState in setOf("nominal", "fair", "serious", "critical")) + + val networkStatus = network.getValue("status").jsonPrimitive.content + assertTrue(networkStatus in setOf("satisfied", "unsatisfied", "requiresConnection")) + val interfaces = network.getValue("interfaces").jsonArray.map { it.jsonPrimitive.content } + assertTrue(interfaces.all { it in setOf("wifi", "cellular", "wired", "other") }) + + assertTrue(payload.getValue("uptimeSeconds").jsonPrimitive.double >= 0.0) + } + + @Test + fun handleDevicePermissions_returnsExpectedShape() { + val handler = DeviceHandler(appContext()) + + val result = handler.handleDevicePermissions(null) + + assertTrue(result.ok) + val payload = parsePayload(result.payloadJson) + val permissions = payload.getValue("permissions").jsonObject + val expected = + listOf( + "camera", + "microphone", + "location", + "backgroundLocation", + "sms", + "notificationListener", + "notifications", + "photos", + "contacts", + "calendar", + "motion", + "screenCapture", + ) + for (key in expected) { + val state = permissions.getValue(key).jsonObject + val status = state.getValue("status").jsonPrimitive.content + assertTrue(status == "granted" || status == "denied") + state.getValue("promptable").jsonPrimitive.boolean + } + } + + @Test + fun handleDeviceHealth_returnsExpectedShape() { + val handler = DeviceHandler(appContext()) + + val result = handler.handleDeviceHealth(null) + + assertTrue(result.ok) + val payload = parsePayload(result.payloadJson) + val memory = payload.getValue("memory").jsonObject + val battery = payload.getValue("battery").jsonObject + val power = payload.getValue("power").jsonObject + val system = payload.getValue("system").jsonObject + + val pressure = memory.getValue("pressure").jsonPrimitive.content + assertTrue(pressure in setOf("normal", "moderate", "high", "critical", "unknown")) + val totalRamBytes = memory.getValue("totalRamBytes").jsonPrimitive.content.toLong() + val availableRamBytes = memory.getValue("availableRamBytes").jsonPrimitive.content.toLong() + val usedRamBytes = memory.getValue("usedRamBytes").jsonPrimitive.content.toLong() + assertTrue(totalRamBytes >= 0L) + assertTrue(availableRamBytes >= 0L) + assertTrue(usedRamBytes >= 0L) + memory.getValue("lowMemory").jsonPrimitive.boolean + + val batteryState = battery.getValue("state").jsonPrimitive.content + assertTrue(batteryState in setOf("unknown", "unplugged", "charging", "full")) + val chargingType = battery.getValue("chargingType").jsonPrimitive.content + assertTrue(chargingType in setOf("none", "ac", "usb", "wireless", "dock")) + battery["temperatureC"]?.jsonPrimitive?.double + battery["currentMa"]?.jsonPrimitive?.double + + power.getValue("dozeModeEnabled").jsonPrimitive.boolean + power.getValue("lowPowerModeEnabled").jsonPrimitive.boolean + system["securityPatchLevel"]?.jsonPrimitive?.content + } + + private fun appContext(): Context = RuntimeEnvironment.getApplication() + + private fun parsePayload(payloadJson: String?): JsonObject { + val jsonString = payloadJson ?: error("expected payload") + return Json.parseToJsonElement(jsonString).jsonObject + } +} diff --git a/apps/android/app/src/test/java/ai/openclaw/android/node/InvokeCommandRegistryTest.kt b/apps/android/app/src/test/java/ai/openclaw/android/node/InvokeCommandRegistryTest.kt new file mode 100644 index 00000000000..bd3dced03e5 --- /dev/null +++ b/apps/android/app/src/test/java/ai/openclaw/android/node/InvokeCommandRegistryTest.kt @@ -0,0 +1,177 @@ +package ai.openclaw.android.node + +import ai.openclaw.android.protocol.OpenClawCalendarCommand +import ai.openclaw.android.protocol.OpenClawCameraCommand +import ai.openclaw.android.protocol.OpenClawCapability +import ai.openclaw.android.protocol.OpenClawContactsCommand +import ai.openclaw.android.protocol.OpenClawDeviceCommand +import ai.openclaw.android.protocol.OpenClawLocationCommand +import ai.openclaw.android.protocol.OpenClawMotionCommand +import ai.openclaw.android.protocol.OpenClawNotificationsCommand +import ai.openclaw.android.protocol.OpenClawPhotosCommand +import ai.openclaw.android.protocol.OpenClawSmsCommand +import ai.openclaw.android.protocol.OpenClawSystemCommand +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Test + +class InvokeCommandRegistryTest { + @Test + fun advertisedCapabilities_respectsFeatureAvailability() { + val capabilities = + InvokeCommandRegistry.advertisedCapabilities( + NodeRuntimeFlags( + cameraEnabled = false, + locationEnabled = false, + smsAvailable = false, + voiceWakeEnabled = false, + motionActivityAvailable = false, + motionPedometerAvailable = false, + debugBuild = false, + ), + ) + + assertTrue(capabilities.contains(OpenClawCapability.Canvas.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.Screen.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.Device.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.Notifications.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.System.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.AppUpdate.rawValue)) + assertFalse(capabilities.contains(OpenClawCapability.Camera.rawValue)) + assertFalse(capabilities.contains(OpenClawCapability.Location.rawValue)) + assertFalse(capabilities.contains(OpenClawCapability.Sms.rawValue)) + assertFalse(capabilities.contains(OpenClawCapability.VoiceWake.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.Photos.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.Contacts.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.Calendar.rawValue)) + assertFalse(capabilities.contains(OpenClawCapability.Motion.rawValue)) + } + + @Test + fun advertisedCapabilities_includesFeatureCapabilitiesWhenEnabled() { + val capabilities = + InvokeCommandRegistry.advertisedCapabilities( + NodeRuntimeFlags( + cameraEnabled = true, + locationEnabled = true, + smsAvailable = true, + voiceWakeEnabled = true, + motionActivityAvailable = true, + motionPedometerAvailable = true, + debugBuild = false, + ), + ) + + assertTrue(capabilities.contains(OpenClawCapability.Canvas.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.Screen.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.Device.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.Notifications.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.System.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.AppUpdate.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.Camera.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.Location.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.Sms.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.VoiceWake.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.Photos.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.Contacts.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.Calendar.rawValue)) + assertTrue(capabilities.contains(OpenClawCapability.Motion.rawValue)) + } + + @Test + fun advertisedCommands_respectsFeatureAvailability() { + val commands = + InvokeCommandRegistry.advertisedCommands( + NodeRuntimeFlags( + cameraEnabled = false, + locationEnabled = false, + smsAvailable = false, + voiceWakeEnabled = false, + motionActivityAvailable = false, + motionPedometerAvailable = false, + debugBuild = false, + ), + ) + + assertFalse(commands.contains(OpenClawCameraCommand.Snap.rawValue)) + assertFalse(commands.contains(OpenClawCameraCommand.Clip.rawValue)) + assertFalse(commands.contains(OpenClawCameraCommand.List.rawValue)) + assertFalse(commands.contains(OpenClawLocationCommand.Get.rawValue)) + assertTrue(commands.contains(OpenClawDeviceCommand.Status.rawValue)) + assertTrue(commands.contains(OpenClawDeviceCommand.Info.rawValue)) + assertTrue(commands.contains(OpenClawDeviceCommand.Permissions.rawValue)) + assertTrue(commands.contains(OpenClawDeviceCommand.Health.rawValue)) + assertTrue(commands.contains(OpenClawNotificationsCommand.List.rawValue)) + assertTrue(commands.contains(OpenClawNotificationsCommand.Actions.rawValue)) + assertTrue(commands.contains(OpenClawSystemCommand.Notify.rawValue)) + assertTrue(commands.contains(OpenClawPhotosCommand.Latest.rawValue)) + assertTrue(commands.contains(OpenClawContactsCommand.Search.rawValue)) + assertTrue(commands.contains(OpenClawContactsCommand.Add.rawValue)) + assertTrue(commands.contains(OpenClawCalendarCommand.Events.rawValue)) + assertTrue(commands.contains(OpenClawCalendarCommand.Add.rawValue)) + assertFalse(commands.contains(OpenClawMotionCommand.Activity.rawValue)) + assertFalse(commands.contains(OpenClawMotionCommand.Pedometer.rawValue)) + assertFalse(commands.contains(OpenClawSmsCommand.Send.rawValue)) + assertFalse(commands.contains("debug.logs")) + assertFalse(commands.contains("debug.ed25519")) + assertTrue(commands.contains("app.update")) + } + + @Test + fun advertisedCommands_includesFeatureCommandsWhenEnabled() { + val commands = + InvokeCommandRegistry.advertisedCommands( + NodeRuntimeFlags( + cameraEnabled = true, + locationEnabled = true, + smsAvailable = true, + voiceWakeEnabled = false, + motionActivityAvailable = true, + motionPedometerAvailable = true, + debugBuild = true, + ), + ) + + assertTrue(commands.contains(OpenClawCameraCommand.Snap.rawValue)) + assertTrue(commands.contains(OpenClawCameraCommand.Clip.rawValue)) + assertTrue(commands.contains(OpenClawCameraCommand.List.rawValue)) + assertTrue(commands.contains(OpenClawLocationCommand.Get.rawValue)) + assertTrue(commands.contains(OpenClawDeviceCommand.Status.rawValue)) + assertTrue(commands.contains(OpenClawDeviceCommand.Info.rawValue)) + assertTrue(commands.contains(OpenClawDeviceCommand.Permissions.rawValue)) + assertTrue(commands.contains(OpenClawDeviceCommand.Health.rawValue)) + assertTrue(commands.contains(OpenClawNotificationsCommand.List.rawValue)) + assertTrue(commands.contains(OpenClawNotificationsCommand.Actions.rawValue)) + assertTrue(commands.contains(OpenClawSystemCommand.Notify.rawValue)) + assertTrue(commands.contains(OpenClawPhotosCommand.Latest.rawValue)) + assertTrue(commands.contains(OpenClawContactsCommand.Search.rawValue)) + assertTrue(commands.contains(OpenClawContactsCommand.Add.rawValue)) + assertTrue(commands.contains(OpenClawCalendarCommand.Events.rawValue)) + assertTrue(commands.contains(OpenClawCalendarCommand.Add.rawValue)) + assertTrue(commands.contains(OpenClawMotionCommand.Activity.rawValue)) + assertTrue(commands.contains(OpenClawMotionCommand.Pedometer.rawValue)) + assertTrue(commands.contains(OpenClawSmsCommand.Send.rawValue)) + assertTrue(commands.contains("debug.logs")) + assertTrue(commands.contains("debug.ed25519")) + assertTrue(commands.contains("app.update")) + } + + @Test + fun advertisedCommands_onlyIncludesSupportedMotionCommands() { + val commands = + InvokeCommandRegistry.advertisedCommands( + NodeRuntimeFlags( + cameraEnabled = false, + locationEnabled = false, + smsAvailable = false, + voiceWakeEnabled = false, + motionActivityAvailable = true, + motionPedometerAvailable = false, + debugBuild = false, + ), + ) + + assertTrue(commands.contains(OpenClawMotionCommand.Activity.rawValue)) + assertFalse(commands.contains(OpenClawMotionCommand.Pedometer.rawValue)) + } +} diff --git a/apps/android/app/src/test/java/ai/openclaw/android/node/MotionHandlerTest.kt b/apps/android/app/src/test/java/ai/openclaw/android/node/MotionHandlerTest.kt new file mode 100644 index 00000000000..1a0fb0c0bd6 --- /dev/null +++ b/apps/android/app/src/test/java/ai/openclaw/android/node/MotionHandlerTest.kt @@ -0,0 +1,136 @@ +package ai.openclaw.android.node + +import android.content.Context +import kotlinx.coroutines.test.runTest +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.jsonArray +import kotlinx.serialization.json.jsonObject +import kotlinx.serialization.json.jsonPrimitive +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.RuntimeEnvironment + +@RunWith(RobolectricTestRunner::class) +class MotionHandlerTest { + @Test + fun handleMotionActivity_requiresPermission() = + runTest { + val handler = MotionHandler.forTesting(appContext(), FakeMotionDataSource(hasPermission = false)) + + val result = handler.handleMotionActivity(null) + + assertFalse(result.ok) + assertEquals("MOTION_PERMISSION_REQUIRED", result.error?.code) + } + + @Test + fun handleMotionActivity_rejectsInvalidJson() = + runTest { + val handler = MotionHandler.forTesting(appContext(), FakeMotionDataSource(hasPermission = true)) + + val result = handler.handleMotionActivity("[]") + + assertFalse(result.ok) + assertEquals("INVALID_REQUEST", result.error?.code) + } + + @Test + fun handleMotionActivity_returnsActivityPayload() = + runTest { + val activity = + MotionActivityRecord( + startISO = "2026-02-28T10:00:00Z", + endISO = "2026-02-28T10:00:02Z", + confidence = "high", + isWalking = true, + isRunning = false, + isCycling = false, + isAutomotive = false, + isStationary = false, + isUnknown = false, + ) + val handler = + MotionHandler.forTesting( + appContext(), + FakeMotionDataSource(hasPermission = true, activityRecord = activity), + ) + + val result = handler.handleMotionActivity(null) + + assertTrue(result.ok) + val payload = Json.parseToJsonElement(result.payloadJson ?: error("missing payload")).jsonObject + val activities = payload.getValue("activities").jsonArray + assertEquals(1, activities.size) + assertEquals("high", activities.first().jsonObject.getValue("confidence").jsonPrimitive.content) + } + + @Test + fun handleMotionPedometer_mapsRangeUnsupportedError() = + runTest { + val handler = + MotionHandler.forTesting( + appContext(), + FakeMotionDataSource( + hasPermission = true, + pedometerError = IllegalArgumentException("PEDOMETER_RANGE_UNAVAILABLE: not supported"), + ), + ) + + val result = handler.handleMotionPedometer("""{"startISO":"2026-02-01T00:00:00Z"}""") + + assertFalse(result.ok) + assertEquals("MOTION_UNAVAILABLE", result.error?.code) + assertTrue(result.error?.message?.contains("PEDOMETER_RANGE_UNAVAILABLE") == true) + } + + private fun appContext(): Context = RuntimeEnvironment.getApplication() +} + +private class FakeMotionDataSource( + private val hasPermission: Boolean, + private val activityAvailable: Boolean = true, + private val pedometerAvailable: Boolean = true, + private val activityRecord: MotionActivityRecord = + MotionActivityRecord( + startISO = "2026-02-28T00:00:00Z", + endISO = "2026-02-28T00:00:02Z", + confidence = "medium", + isWalking = false, + isRunning = false, + isCycling = false, + isAutomotive = false, + isStationary = true, + isUnknown = false, + ), + private val pedometerRecord: PedometerRecord = + PedometerRecord( + startISO = "2026-02-28T00:00:00Z", + endISO = "2026-02-28T01:00:00Z", + steps = 1234, + distanceMeters = null, + floorsAscended = null, + floorsDescended = null, + ), + private val activityError: Throwable? = null, + private val pedometerError: Throwable? = null, +) : MotionDataSource { + override fun isActivityAvailable(context: Context): Boolean = activityAvailable + + override fun isPedometerAvailable(context: Context): Boolean = pedometerAvailable + + override fun hasPermission(context: Context): Boolean = hasPermission + + override suspend fun activity(context: Context, request: MotionActivityRequest): MotionActivityRecord { + activityError?.let { throw it } + return activityRecord + } + + override suspend fun pedometer(context: Context, request: MotionPedometerRequest): PedometerRecord { + pedometerError?.let { throw it } + return pedometerRecord + } +} diff --git a/apps/android/app/src/test/java/ai/openclaw/android/node/NotificationsHandlerTest.kt b/apps/android/app/src/test/java/ai/openclaw/android/node/NotificationsHandlerTest.kt new file mode 100644 index 00000000000..26869cad9ee --- /dev/null +++ b/apps/android/app/src/test/java/ai/openclaw/android/node/NotificationsHandlerTest.kt @@ -0,0 +1,258 @@ +package ai.openclaw.android.node + +import android.content.Context +import ai.openclaw.android.gateway.GatewaySession +import kotlinx.coroutines.test.runTest +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.boolean +import kotlinx.serialization.json.int +import kotlinx.serialization.json.jsonArray +import kotlinx.serialization.json.jsonObject +import kotlinx.serialization.json.jsonPrimitive +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertNull +import org.junit.Assert.assertTrue +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.RuntimeEnvironment + +@RunWith(RobolectricTestRunner::class) +class NotificationsHandlerTest { + @Test + fun notificationsListReturnsStatusPayloadWhenDisabled() = + runTest { + val provider = + FakeNotificationsStateProvider( + DeviceNotificationSnapshot( + enabled = false, + connected = false, + notifications = emptyList(), + ), + ) + val handler = NotificationsHandler.forTesting(appContext = appContext(), stateProvider = provider) + + val result = handler.handleNotificationsList(null) + + assertTrue(result.ok) + assertNull(result.error) + val payload = parsePayload(result) + assertFalse(payload.getValue("enabled").jsonPrimitive.boolean) + assertFalse(payload.getValue("connected").jsonPrimitive.boolean) + assertEquals(0, payload.getValue("count").jsonPrimitive.int) + assertEquals(0, payload.getValue("notifications").jsonArray.size) + assertEquals(0, provider.rebindRequests) + } + + @Test + fun notificationsListRequestsRebindWhenEnabledButDisconnected() = + runTest { + val provider = + FakeNotificationsStateProvider( + DeviceNotificationSnapshot( + enabled = true, + connected = false, + notifications = listOf(sampleEntry("n1")), + ), + ) + val handler = NotificationsHandler.forTesting(appContext = appContext(), stateProvider = provider) + + val result = handler.handleNotificationsList(null) + + assertTrue(result.ok) + assertNull(result.error) + val payload = parsePayload(result) + assertTrue(payload.getValue("enabled").jsonPrimitive.boolean) + assertFalse(payload.getValue("connected").jsonPrimitive.boolean) + assertEquals(1, payload.getValue("count").jsonPrimitive.int) + assertEquals(1, payload.getValue("notifications").jsonArray.size) + assertEquals(1, provider.rebindRequests) + } + + @Test + fun notificationsListDoesNotRequestRebindWhenConnected() = + runTest { + val provider = + FakeNotificationsStateProvider( + DeviceNotificationSnapshot( + enabled = true, + connected = true, + notifications = listOf(sampleEntry("n2")), + ), + ) + val handler = NotificationsHandler.forTesting(appContext = appContext(), stateProvider = provider) + + val result = handler.handleNotificationsList(null) + + assertTrue(result.ok) + assertNull(result.error) + val payload = parsePayload(result) + assertTrue(payload.getValue("enabled").jsonPrimitive.boolean) + assertTrue(payload.getValue("connected").jsonPrimitive.boolean) + assertEquals(1, payload.getValue("count").jsonPrimitive.int) + assertEquals(0, provider.rebindRequests) + } + + @Test + fun notificationsActions_executesDismissAction() = + runTest { + val provider = + FakeNotificationsStateProvider( + DeviceNotificationSnapshot( + enabled = true, + connected = true, + notifications = listOf(sampleEntry("n2")), + ), + ) + val handler = NotificationsHandler.forTesting(appContext = appContext(), stateProvider = provider) + + val result = handler.handleNotificationsActions("""{"key":"n2","action":"dismiss"}""") + + assertTrue(result.ok) + assertNull(result.error) + val payload = parsePayload(result) + assertTrue(payload.getValue("ok").jsonPrimitive.boolean) + assertEquals("n2", payload.getValue("key").jsonPrimitive.content) + assertEquals("dismiss", payload.getValue("action").jsonPrimitive.content) + assertEquals("n2", provider.lastAction?.key) + assertEquals(NotificationActionKind.Dismiss, provider.lastAction?.kind) + } + + @Test + fun notificationsActions_requiresReplyTextForReplyAction() = + runTest { + val provider = + FakeNotificationsStateProvider( + DeviceNotificationSnapshot( + enabled = true, + connected = true, + notifications = listOf(sampleEntry("n3")), + ), + ) + val handler = NotificationsHandler.forTesting(appContext = appContext(), stateProvider = provider) + + val result = handler.handleNotificationsActions("""{"key":"n3","action":"reply"}""") + + assertFalse(result.ok) + assertEquals("INVALID_REQUEST", result.error?.code) + assertEquals(0, provider.actionRequests) + } + + @Test + fun notificationsActions_propagatesProviderError() = + runTest { + val provider = + FakeNotificationsStateProvider( + DeviceNotificationSnapshot( + enabled = true, + connected = true, + notifications = listOf(sampleEntry("n4")), + ), + ).also { + it.actionResult = + NotificationActionResult( + ok = false, + code = "NOTIFICATION_NOT_FOUND", + message = "NOTIFICATION_NOT_FOUND: notification key not found", + ) + } + val handler = NotificationsHandler.forTesting(appContext = appContext(), stateProvider = provider) + + val result = handler.handleNotificationsActions("""{"key":"n4","action":"open"}""") + + assertFalse(result.ok) + assertEquals("NOTIFICATION_NOT_FOUND", result.error?.code) + assertEquals(1, provider.actionRequests) + } + + @Test + fun notificationsActions_requestsRebindWhenEnabledButDisconnected() = + runTest { + val provider = + FakeNotificationsStateProvider( + DeviceNotificationSnapshot( + enabled = true, + connected = false, + notifications = listOf(sampleEntry("n5")), + ), + ) + val handler = NotificationsHandler.forTesting(appContext = appContext(), stateProvider = provider) + + val result = handler.handleNotificationsActions("""{"key":"n5","action":"open"}""") + + assertTrue(result.ok) + assertEquals(1, provider.rebindRequests) + assertEquals(1, provider.actionRequests) + } + + @Test + fun sanitizeNotificationTextReturnsNullForBlankInput() { + assertNull(sanitizeNotificationText(null)) + assertNull(sanitizeNotificationText(" ")) + } + + @Test + fun sanitizeNotificationTextTrimsAndTruncates() { + val value = " ${"x".repeat(600)} " + val sanitized = sanitizeNotificationText(value) + + assertEquals(512, sanitized?.length) + assertTrue((sanitized ?: "").all { it == 'x' }) + } + + @Test + fun notificationsActionClearablePolicy_onlyRequiresClearableForDismiss() { + assertTrue(actionRequiresClearableNotification(NotificationActionKind.Dismiss)) + assertFalse(actionRequiresClearableNotification(NotificationActionKind.Open)) + assertFalse(actionRequiresClearableNotification(NotificationActionKind.Reply)) + } + + private fun parsePayload(result: GatewaySession.InvokeResult): JsonObject { + val payloadJson = result.payloadJson ?: error("expected payload") + return Json.parseToJsonElement(payloadJson).jsonObject + } + + private fun appContext(): Context = RuntimeEnvironment.getApplication() + + private fun sampleEntry(key: String): DeviceNotificationEntry = + DeviceNotificationEntry( + key = key, + packageName = "com.example.app", + title = "Title", + text = "Text", + subText = null, + category = null, + channelId = null, + postTimeMs = 123L, + isOngoing = false, + isClearable = true, + ) +} + +private class FakeNotificationsStateProvider( + private val snapshot: DeviceNotificationSnapshot, +) : NotificationsStateProvider { + var rebindRequests: Int = 0 + private set + var actionRequests: Int = 0 + private set + var actionResult: NotificationActionResult = NotificationActionResult(ok = true) + var lastAction: NotificationActionRequest? = null + + override fun readSnapshot(context: Context): DeviceNotificationSnapshot = snapshot + + override fun requestServiceRebind(context: Context) { + rebindRequests += 1 + } + + override fun executeAction( + context: Context, + request: NotificationActionRequest, + ): NotificationActionResult { + actionRequests += 1 + lastAction = request + return actionResult + } +} diff --git a/apps/android/app/src/test/java/ai/openclaw/android/node/PhotosHandlerTest.kt b/apps/android/app/src/test/java/ai/openclaw/android/node/PhotosHandlerTest.kt new file mode 100644 index 00000000000..c9596452c5b --- /dev/null +++ b/apps/android/app/src/test/java/ai/openclaw/android/node/PhotosHandlerTest.kt @@ -0,0 +1,77 @@ +package ai.openclaw.android.node + +import android.content.Context +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.int +import kotlinx.serialization.json.jsonArray +import kotlinx.serialization.json.jsonObject +import kotlinx.serialization.json.jsonPrimitive +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.RuntimeEnvironment + +@RunWith(RobolectricTestRunner::class) +class PhotosHandlerTest { + @Test + fun handlePhotosLatest_requiresPermission() { + val handler = PhotosHandler.forTesting(appContext(), FakePhotosDataSource(hasPermission = false)) + + val result = handler.handlePhotosLatest(null) + + assertFalse(result.ok) + assertEquals("PHOTOS_PERMISSION_REQUIRED", result.error?.code) + } + + @Test + fun handlePhotosLatest_rejectsInvalidJson() { + val handler = PhotosHandler.forTesting(appContext(), FakePhotosDataSource(hasPermission = true)) + + val result = handler.handlePhotosLatest("[]") + + assertFalse(result.ok) + assertEquals("INVALID_REQUEST", result.error?.code) + } + + @Test + fun handlePhotosLatest_returnsPayload() { + val source = + FakePhotosDataSource( + hasPermission = true, + latest = listOf( + EncodedPhotoPayload( + format = "jpeg", + base64 = "abc123", + width = 640, + height = 480, + createdAt = "2026-02-28T00:00:00Z", + ), + ), + ) + val handler = PhotosHandler.forTesting(appContext(), source) + + val result = handler.handlePhotosLatest("""{"limit":1}""") + + assertTrue(result.ok) + val payload = Json.parseToJsonElement(result.payloadJson ?: error("missing payload")).jsonObject + val photos = payload.getValue("photos").jsonArray + assertEquals(1, photos.size) + val first = photos.first().jsonObject + assertEquals("jpeg", first.getValue("format").jsonPrimitive.content) + assertEquals(640, first.getValue("width").jsonPrimitive.int) + } + + private fun appContext(): Context = RuntimeEnvironment.getApplication() +} + +private class FakePhotosDataSource( + private val hasPermission: Boolean, + private val latest: List = emptyList(), +) : PhotosDataSource { + override fun hasPermission(context: Context): Boolean = hasPermission + + override fun latest(context: Context, request: PhotosLatestRequest): List = latest +} diff --git a/apps/android/app/src/test/java/ai/openclaw/android/node/SystemHandlerTest.kt b/apps/android/app/src/test/java/ai/openclaw/android/node/SystemHandlerTest.kt new file mode 100644 index 00000000000..7b49a3641e0 --- /dev/null +++ b/apps/android/app/src/test/java/ai/openclaw/android/node/SystemHandlerTest.kt @@ -0,0 +1,52 @@ +package ai.openclaw.android.node + +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Test + +class SystemHandlerTest { + @Test + fun handleSystemNotify_rejectsUnauthorized() { + val handler = SystemHandler.forTesting(poster = FakePoster(authorized = false)) + + val result = handler.handleSystemNotify("""{"title":"OpenClaw","body":"hi"}""") + + assertFalse(result.ok) + assertEquals("NOT_AUTHORIZED", result.error?.code) + } + + @Test + fun handleSystemNotify_rejectsEmptyNotification() { + val handler = SystemHandler.forTesting(poster = FakePoster(authorized = true)) + + val result = handler.handleSystemNotify("""{"title":" ","body":" "}""") + + assertFalse(result.ok) + assertEquals("INVALID_REQUEST", result.error?.code) + } + + @Test + fun handleSystemNotify_postsNotification() { + val poster = FakePoster(authorized = true) + val handler = SystemHandler.forTesting(poster = poster) + + val result = handler.handleSystemNotify("""{"title":"OpenClaw","body":"done","priority":"active"}""") + + assertTrue(result.ok) + assertEquals(1, poster.posts) + } +} + +private class FakePoster( + private val authorized: Boolean, +) : SystemNotificationPoster { + var posts: Int = 0 + private set + + override fun isAuthorized(): Boolean = authorized + + override fun post(request: SystemNotifyRequest) { + posts += 1 + } +} diff --git a/apps/android/app/src/test/java/ai/openclaw/android/protocol/OpenClawProtocolConstantsTest.kt b/apps/android/app/src/test/java/ai/openclaw/android/protocol/OpenClawProtocolConstantsTest.kt index 10ab733ae53..cd1cf847101 100644 --- a/apps/android/app/src/test/java/ai/openclaw/android/protocol/OpenClawProtocolConstantsTest.kt +++ b/apps/android/app/src/test/java/ai/openclaw/android/protocol/OpenClawProtocolConstantsTest.kt @@ -26,10 +26,69 @@ class OpenClawProtocolConstantsTest { assertEquals("camera", OpenClawCapability.Camera.rawValue) assertEquals("screen", OpenClawCapability.Screen.rawValue) assertEquals("voiceWake", OpenClawCapability.VoiceWake.rawValue) + assertEquals("location", OpenClawCapability.Location.rawValue) + assertEquals("sms", OpenClawCapability.Sms.rawValue) + assertEquals("device", OpenClawCapability.Device.rawValue) + assertEquals("notifications", OpenClawCapability.Notifications.rawValue) + assertEquals("system", OpenClawCapability.System.rawValue) + assertEquals("appUpdate", OpenClawCapability.AppUpdate.rawValue) + assertEquals("photos", OpenClawCapability.Photos.rawValue) + assertEquals("contacts", OpenClawCapability.Contacts.rawValue) + assertEquals("calendar", OpenClawCapability.Calendar.rawValue) + assertEquals("motion", OpenClawCapability.Motion.rawValue) + } + + @Test + fun cameraCommandsUseStableStrings() { + assertEquals("camera.list", OpenClawCameraCommand.List.rawValue) + assertEquals("camera.snap", OpenClawCameraCommand.Snap.rawValue) + assertEquals("camera.clip", OpenClawCameraCommand.Clip.rawValue) } @Test fun screenCommandsUseStableStrings() { assertEquals("screen.record", OpenClawScreenCommand.Record.rawValue) } + + @Test + fun notificationsCommandsUseStableStrings() { + assertEquals("notifications.list", OpenClawNotificationsCommand.List.rawValue) + assertEquals("notifications.actions", OpenClawNotificationsCommand.Actions.rawValue) + } + + @Test + fun deviceCommandsUseStableStrings() { + assertEquals("device.status", OpenClawDeviceCommand.Status.rawValue) + assertEquals("device.info", OpenClawDeviceCommand.Info.rawValue) + assertEquals("device.permissions", OpenClawDeviceCommand.Permissions.rawValue) + assertEquals("device.health", OpenClawDeviceCommand.Health.rawValue) + } + + @Test + fun systemCommandsUseStableStrings() { + assertEquals("system.notify", OpenClawSystemCommand.Notify.rawValue) + } + + @Test + fun photosCommandsUseStableStrings() { + assertEquals("photos.latest", OpenClawPhotosCommand.Latest.rawValue) + } + + @Test + fun contactsCommandsUseStableStrings() { + assertEquals("contacts.search", OpenClawContactsCommand.Search.rawValue) + assertEquals("contacts.add", OpenClawContactsCommand.Add.rawValue) + } + + @Test + fun calendarCommandsUseStableStrings() { + assertEquals("calendar.events", OpenClawCalendarCommand.Events.rawValue) + assertEquals("calendar.add", OpenClawCalendarCommand.Add.rawValue) + } + + @Test + fun motionCommandsUseStableStrings() { + assertEquals("motion.activity", OpenClawMotionCommand.Activity.rawValue) + assertEquals("motion.pedometer", OpenClawMotionCommand.Pedometer.rawValue) + } } diff --git a/apps/android/app/src/test/java/ai/openclaw/android/ui/GatewayConfigResolverTest.kt b/apps/android/app/src/test/java/ai/openclaw/android/ui/GatewayConfigResolverTest.kt new file mode 100644 index 00000000000..7dc2dd1a239 --- /dev/null +++ b/apps/android/app/src/test/java/ai/openclaw/android/ui/GatewayConfigResolverTest.kt @@ -0,0 +1,59 @@ +package ai.openclaw.android.ui + +import java.util.Base64 +import org.junit.Assert.assertEquals +import org.junit.Assert.assertNull +import org.junit.Test + +class GatewayConfigResolverTest { + @Test + fun resolveScannedSetupCodeAcceptsRawSetupCode() { + val setupCode = encodeSetupCode("""{"url":"wss://gateway.example:18789","token":"token-1"}""") + + val resolved = resolveScannedSetupCode(setupCode) + + assertEquals(setupCode, resolved) + } + + @Test + fun resolveScannedSetupCodeAcceptsQrJsonPayload() { + val setupCode = encodeSetupCode("""{"url":"wss://gateway.example:18789","password":"pw-1"}""") + val qrJson = + """ + { + "setupCode": "$setupCode", + "gatewayUrl": "wss://gateway.example:18789", + "auth": "password", + "urlSource": "gateway.remote.url" + } + """.trimIndent() + + val resolved = resolveScannedSetupCode(qrJson) + + assertEquals(setupCode, resolved) + } + + @Test + fun resolveScannedSetupCodeRejectsInvalidInput() { + val resolved = resolveScannedSetupCode("not-a-valid-setup-code") + assertNull(resolved) + } + + @Test + fun resolveScannedSetupCodeRejectsJsonWithInvalidSetupCode() { + val qrJson = """{"setupCode":"invalid"}""" + val resolved = resolveScannedSetupCode(qrJson) + assertNull(resolved) + } + + @Test + fun resolveScannedSetupCodeRejectsJsonWithNonStringSetupCode() { + val qrJson = """{"setupCode":{"nested":"value"}}""" + val resolved = resolveScannedSetupCode(qrJson) + assertNull(resolved) + } + + private fun encodeSetupCode(payloadJson: String): String { + return Base64.getUrlEncoder().withoutPadding().encodeToString(payloadJson.toByteArray(Charsets.UTF_8)) + } +} diff --git a/apps/android/app/src/test/java/ai/openclaw/android/voice/TalkModeConfigParsingTest.kt b/apps/android/app/src/test/java/ai/openclaw/android/voice/TalkModeConfigParsingTest.kt new file mode 100644 index 00000000000..5daa62080d7 --- /dev/null +++ b/apps/android/app/src/test/java/ai/openclaw/android/voice/TalkModeConfigParsingTest.kt @@ -0,0 +1,59 @@ +package ai.openclaw.android.voice + +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.jsonPrimitive +import kotlinx.serialization.json.jsonObject +import org.junit.Assert.assertEquals +import org.junit.Assert.assertNotNull +import org.junit.Assert.assertTrue +import org.junit.Test + +class TalkModeConfigParsingTest { + private val json = Json { ignoreUnknownKeys = true } + + @Test + fun prefersNormalizedTalkProviderPayload() { + val talk = + json.parseToJsonElement( + """ + { + "provider": "elevenlabs", + "providers": { + "elevenlabs": { + "voiceId": "voice-normalized" + } + }, + "voiceId": "voice-legacy" + } + """.trimIndent(), + ) + .jsonObject + + val selection = TalkModeManager.selectTalkProviderConfig(talk) + assertNotNull(selection) + assertEquals("elevenlabs", selection?.provider) + assertTrue(selection?.normalizedPayload == true) + assertEquals("voice-normalized", selection?.config?.get("voiceId")?.jsonPrimitive?.content) + } + + @Test + fun fallsBackToLegacyTalkFieldsWhenNormalizedPayloadMissing() { + val talk = + json.parseToJsonElement( + """ + { + "voiceId": "voice-legacy", + "apiKey": "legacy-key" + } + """.trimIndent(), + ) + .jsonObject + + val selection = TalkModeManager.selectTalkProviderConfig(talk) + assertNotNull(selection) + assertEquals("elevenlabs", selection?.provider) + assertTrue(selection?.normalizedPayload == false) + assertEquals("voice-legacy", selection?.config?.get("voiceId")?.jsonPrimitive?.content) + assertEquals("legacy-key", selection?.config?.get("apiKey")?.jsonPrimitive?.content) + } +} diff --git a/apps/android/benchmark/build.gradle.kts b/apps/android/benchmark/build.gradle.kts new file mode 100644 index 00000000000..99d1d8e4c60 --- /dev/null +++ b/apps/android/benchmark/build.gradle.kts @@ -0,0 +1,36 @@ +plugins { + id("com.android.test") +} + +android { + namespace = "ai.openclaw.android.benchmark" + compileSdk = 36 + + defaultConfig { + minSdk = 31 + targetSdk = 36 + testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner" + testInstrumentationRunnerArguments["androidx.benchmark.suppressErrors"] = "DEBUGGABLE,EMULATOR" + } + + targetProjectPath = ":app" + experimentalProperties["android.experimental.self-instrumenting"] = true + + compileOptions { + sourceCompatibility = JavaVersion.VERSION_17 + targetCompatibility = JavaVersion.VERSION_17 + } +} + +kotlin { + compilerOptions { + jvmTarget.set(org.jetbrains.kotlin.gradle.dsl.JvmTarget.JVM_17) + allWarningsAsErrors.set(true) + } +} + +dependencies { + implementation("androidx.benchmark:benchmark-macro-junit4:1.4.1") + implementation("androidx.test.ext:junit:1.2.1") + implementation("androidx.test.uiautomator:uiautomator:2.4.0-alpha06") +} diff --git a/apps/android/benchmark/src/main/java/ai/openclaw/android/benchmark/StartupMacrobenchmark.kt b/apps/android/benchmark/src/main/java/ai/openclaw/android/benchmark/StartupMacrobenchmark.kt new file mode 100644 index 00000000000..46181f6a9a1 --- /dev/null +++ b/apps/android/benchmark/src/main/java/ai/openclaw/android/benchmark/StartupMacrobenchmark.kt @@ -0,0 +1,76 @@ +package ai.openclaw.android.benchmark + +import androidx.benchmark.macro.CompilationMode +import androidx.benchmark.macro.FrameTimingMetric +import androidx.benchmark.macro.StartupMode +import androidx.benchmark.macro.StartupTimingMetric +import androidx.benchmark.macro.junit4.MacrobenchmarkRule +import androidx.test.ext.junit.runners.AndroidJUnit4 +import androidx.test.platform.app.InstrumentationRegistry +import androidx.test.uiautomator.UiDevice +import org.junit.Assume.assumeTrue +import org.junit.Rule +import org.junit.Test +import org.junit.runner.RunWith + +@RunWith(AndroidJUnit4::class) +class StartupMacrobenchmark { + @get:Rule + val benchmarkRule = MacrobenchmarkRule() + + private val packageName = "ai.openclaw.android" + + @Test + fun coldStartup() { + runBenchmarkOrSkip { + benchmarkRule.measureRepeated( + packageName = packageName, + metrics = listOf(StartupTimingMetric()), + startupMode = StartupMode.COLD, + compilationMode = CompilationMode.None(), + iterations = 10, + ) { + pressHome() + startActivityAndWait() + } + } + } + + @Test + fun startupAndScrollFrameTiming() { + runBenchmarkOrSkip { + benchmarkRule.measureRepeated( + packageName = packageName, + metrics = listOf(FrameTimingMetric()), + startupMode = StartupMode.WARM, + compilationMode = CompilationMode.None(), + iterations = 10, + ) { + startActivityAndWait() + val device = UiDevice.getInstance(InstrumentationRegistry.getInstrumentation()) + val x = device.displayWidth / 2 + val yStart = (device.displayHeight * 0.8f).toInt() + val yEnd = (device.displayHeight * 0.25f).toInt() + repeat(4) { + device.swipe(x, yStart, x, yEnd, 24) + device.waitForIdle() + } + } + } + } + + private fun runBenchmarkOrSkip(run: () -> Unit) { + try { + run() + } catch (err: IllegalStateException) { + val message = err.message.orEmpty() + val knownDeviceIssue = + message.contains("Unable to confirm activity launch completion") || + message.contains("no renderthread slices", ignoreCase = true) + if (knownDeviceIssue) { + assumeTrue("Skipping benchmark on this device: $message", false) + } + throw err + } + } +} diff --git a/apps/android/build.gradle.kts b/apps/android/build.gradle.kts index f79902d5615..1d191c9e375 100644 --- a/apps/android/build.gradle.kts +++ b/apps/android/build.gradle.kts @@ -1,6 +1,6 @@ plugins { - id("com.android.application") version "8.13.2" apply false - id("org.jetbrains.kotlin.android") version "2.2.21" apply false + id("com.android.application") version "9.0.1" apply false + id("com.android.test") version "9.0.1" apply false id("org.jetbrains.kotlin.plugin.compose") version "2.2.21" apply false id("org.jetbrains.kotlin.plugin.serialization") version "2.2.21" apply false } diff --git a/apps/android/gradle.properties b/apps/android/gradle.properties index 5f84d966ee8..4134274afdd 100644 --- a/apps/android/gradle.properties +++ b/apps/android/gradle.properties @@ -3,3 +3,12 @@ org.gradle.warning.mode=none android.useAndroidX=true android.nonTransitiveRClass=true android.enableR8.fullMode=true +android.defaults.buildfeatures.resvalues=true +android.sdk.defaultTargetSdkToCompileSdkIfUnset=false +android.enableAppCompileTimeRClass=false +android.usesSdkInManifest.disallowed=false +android.uniquePackageNames=false +android.dependency.useConstraints=true +android.r8.strictFullModeForKeepRules=false +android.r8.optimizedResourceShrinking=false +android.newDsl=true diff --git a/apps/android/gradle/gradle-daemon-jvm.properties b/apps/android/gradle/gradle-daemon-jvm.properties new file mode 100644 index 00000000000..6c1139ec06a --- /dev/null +++ b/apps/android/gradle/gradle-daemon-jvm.properties @@ -0,0 +1,12 @@ +#This file is generated by updateDaemonJvm +toolchainUrl.FREE_BSD.AARCH64=https\://api.foojay.io/disco/v3.0/ids/ec7520a1e057cd116f9544c42142a16b/redirect +toolchainUrl.FREE_BSD.X86_64=https\://api.foojay.io/disco/v3.0/ids/4c4f879899012ff0a8b2e2117df03b0e/redirect +toolchainUrl.LINUX.AARCH64=https\://api.foojay.io/disco/v3.0/ids/ec7520a1e057cd116f9544c42142a16b/redirect +toolchainUrl.LINUX.X86_64=https\://api.foojay.io/disco/v3.0/ids/4c4f879899012ff0a8b2e2117df03b0e/redirect +toolchainUrl.MAC_OS.AARCH64=https\://api.foojay.io/disco/v3.0/ids/73bcfb608d1fde9fb62e462f834a3299/redirect +toolchainUrl.MAC_OS.X86_64=https\://api.foojay.io/disco/v3.0/ids/846ee0d876d26a26f37aa1ce8de73224/redirect +toolchainUrl.UNIX.AARCH64=https\://api.foojay.io/disco/v3.0/ids/ec7520a1e057cd116f9544c42142a16b/redirect +toolchainUrl.UNIX.X86_64=https\://api.foojay.io/disco/v3.0/ids/4c4f879899012ff0a8b2e2117df03b0e/redirect +toolchainUrl.WINDOWS.AARCH64=https\://api.foojay.io/disco/v3.0/ids/9482ddec596298c84656d31d16652665/redirect +toolchainUrl.WINDOWS.X86_64=https\://api.foojay.io/disco/v3.0/ids/39701d92e1756bb2f141eb67cd4c660e/redirect +toolchainVersion=21 diff --git a/apps/android/scripts/perf-startup-benchmark.sh b/apps/android/scripts/perf-startup-benchmark.sh new file mode 100755 index 00000000000..70342d3cba4 --- /dev/null +++ b/apps/android/scripts/perf-startup-benchmark.sh @@ -0,0 +1,124 @@ +#!/usr/bin/env bash +set -euo pipefail + +SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" +ANDROID_DIR="$(cd -- "$SCRIPT_DIR/.." && pwd)" +RESULTS_DIR="$ANDROID_DIR/benchmark/results" +CLASS_FILTER="ai.openclaw.android.benchmark.StartupMacrobenchmark#coldStartup" +BASELINE_JSON="" + +usage() { + cat <<'EOF' +Usage: + ./scripts/perf-startup-benchmark.sh [--baseline ] + +Runs cold-start macrobenchmark only, then prints a compact summary. +Also saves a timestamped snapshot JSON under benchmark/results/. +If --baseline is omitted, compares against latest previous snapshot when available. +EOF +} + +while [[ $# -gt 0 ]]; do + case "$1" in + --baseline) + BASELINE_JSON="${2:-}" + shift 2 + ;; + -h|--help) + usage + exit 0 + ;; + *) + echo "Unknown arg: $1" >&2 + usage >&2 + exit 2 + ;; + esac +done + +if ! command -v jq >/dev/null 2>&1; then + echo "jq required but missing." >&2 + exit 1 +fi + +if ! command -v adb >/dev/null 2>&1; then + echo "adb required but missing." >&2 + exit 1 +fi + +device_count="$(adb devices | awk 'NR>1 && $2=="device" {c+=1} END {print c+0}')" +if [[ "$device_count" -lt 1 ]]; then + echo "No connected Android device (adb state=device)." >&2 + exit 1 +fi + +mkdir -p "$RESULTS_DIR" + +run_log="$(mktemp -t openclaw-android-bench.XXXXXX.log)" +trap 'rm -f "$run_log"' EXIT + +cd "$ANDROID_DIR" + +./gradlew :benchmark:connectedDebugAndroidTest \ + -Pandroid.testInstrumentationRunnerArguments.class="$CLASS_FILTER" \ + --console=plain \ + >"$run_log" 2>&1 + +latest_json="$( + find "$ANDROID_DIR/benchmark/build/outputs/connected_android_test_additional_output/debug/connected" \ + -name '*benchmarkData.json' -type f \ + | while IFS= read -r file; do + printf '%s\t%s\n' "$(stat -f '%m' "$file")" "$file" + done \ + | sort -nr \ + | head -n1 \ + | cut -f2- +)" + +if [[ -z "$latest_json" || ! -f "$latest_json" ]]; then + echo "benchmarkData.json not found after run." >&2 + tail -n 120 "$run_log" >&2 + exit 1 +fi + +timestamp="$(date +%Y%m%d-%H%M%S)" +snapshot_json="$RESULTS_DIR/startup-$timestamp.json" +cp "$latest_json" "$snapshot_json" + +median_ms="$(jq -r '.benchmarks[] | select(.name=="coldStartup") | .metrics.timeToInitialDisplayMs.median' "$snapshot_json")" +min_ms="$(jq -r '.benchmarks[] | select(.name=="coldStartup") | .metrics.timeToInitialDisplayMs.minimum' "$snapshot_json")" +max_ms="$(jq -r '.benchmarks[] | select(.name=="coldStartup") | .metrics.timeToInitialDisplayMs.maximum' "$snapshot_json")" +cov="$(jq -r '.benchmarks[] | select(.name=="coldStartup") | .metrics.timeToInitialDisplayMs.coefficientOfVariation' "$snapshot_json")" +device="$(jq -r '.context.build.model' "$snapshot_json")" +sdk="$(jq -r '.context.build.version.sdk' "$snapshot_json")" +runs_count="$(jq -r '.benchmarks[] | select(.name=="coldStartup") | .metrics.timeToInitialDisplayMs.runs | length' "$snapshot_json")" + +printf 'startup.cold.median_ms=%.3f min_ms=%.3f max_ms=%.3f cov=%.4f runs=%s device=%s sdk=%s\n' \ + "$median_ms" "$min_ms" "$max_ms" "$cov" "$runs_count" "$device" "$sdk" +echo "snapshot_json=$snapshot_json" + +if [[ -z "$BASELINE_JSON" ]]; then + BASELINE_JSON="$( + find "$RESULTS_DIR" -name 'startup-*.json' -type f \ + | while IFS= read -r file; do + if [[ "$file" == "$snapshot_json" ]]; then + continue + fi + printf '%s\t%s\n' "$(stat -f '%m' "$file")" "$file" + done \ + | sort -nr \ + | head -n1 \ + | cut -f2- + )" +fi + +if [[ -n "$BASELINE_JSON" ]]; then + if [[ ! -f "$BASELINE_JSON" ]]; then + echo "Baseline file missing: $BASELINE_JSON" >&2 + exit 1 + fi + base_median="$(jq -r '.benchmarks[] | select(.name=="coldStartup") | .metrics.timeToInitialDisplayMs.median' "$BASELINE_JSON")" + delta_ms="$(awk -v a="$median_ms" -v b="$base_median" 'BEGIN { printf "%.3f", (a-b) }')" + delta_pct="$(awk -v a="$median_ms" -v b="$base_median" 'BEGIN { if (b==0) { print "nan" } else { printf "%.2f", ((a-b)/b)*100 } }')" + echo "baseline_median_ms=$base_median delta_ms=$delta_ms delta_pct=$delta_pct%" +fi diff --git a/apps/android/scripts/perf-startup-hotspots.sh b/apps/android/scripts/perf-startup-hotspots.sh new file mode 100755 index 00000000000..787d5fac300 --- /dev/null +++ b/apps/android/scripts/perf-startup-hotspots.sh @@ -0,0 +1,154 @@ +#!/usr/bin/env bash +set -euo pipefail + +SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" +ANDROID_DIR="$(cd -- "$SCRIPT_DIR/.." && pwd)" + +PACKAGE="ai.openclaw.android" +ACTIVITY=".MainActivity" +DURATION_SECONDS="10" +OUTPUT_PERF_DATA="" + +usage() { + cat <<'EOF' +Usage: + ./scripts/perf-startup-hotspots.sh [--package ] [--activity ] [--duration ] [--out ] + +Captures startup CPU profile via simpleperf (app_profiler.py), then prints concise hotspot summaries. +Default package/activity target OpenClaw Android startup. +EOF +} + +while [[ $# -gt 0 ]]; do + case "$1" in + --package) + PACKAGE="${2:-}" + shift 2 + ;; + --activity) + ACTIVITY="${2:-}" + shift 2 + ;; + --duration) + DURATION_SECONDS="${2:-}" + shift 2 + ;; + --out) + OUTPUT_PERF_DATA="${2:-}" + shift 2 + ;; + -h|--help) + usage + exit 0 + ;; + *) + echo "Unknown arg: $1" >&2 + usage >&2 + exit 2 + ;; + esac +done + +if ! command -v uv >/dev/null 2>&1; then + echo "uv required but missing." >&2 + exit 1 +fi + +if ! command -v adb >/dev/null 2>&1; then + echo "adb required but missing." >&2 + exit 1 +fi + +if [[ -z "$OUTPUT_PERF_DATA" ]]; then + OUTPUT_PERF_DATA="/tmp/openclaw-startup-$(date +%Y%m%d-%H%M%S).perf.data" +fi + +device_count="$(adb devices | awk 'NR>1 && $2=="device" {c+=1} END {print c+0}')" +if [[ "$device_count" -lt 1 ]]; then + echo "No connected Android device (adb state=device)." >&2 + exit 1 +fi + +simpleperf_dir="" +if [[ -n "${ANDROID_NDK_HOME:-}" && -f "${ANDROID_NDK_HOME}/simpleperf/app_profiler.py" ]]; then + simpleperf_dir="${ANDROID_NDK_HOME}/simpleperf" +elif [[ -n "${ANDROID_NDK_ROOT:-}" && -f "${ANDROID_NDK_ROOT}/simpleperf/app_profiler.py" ]]; then + simpleperf_dir="${ANDROID_NDK_ROOT}/simpleperf" +else + latest_simpleperf="$(ls -d "${HOME}/Library/Android/sdk/ndk/"*/simpleperf 2>/dev/null | sort -V | tail -n1 || true)" + if [[ -n "$latest_simpleperf" && -f "$latest_simpleperf/app_profiler.py" ]]; then + simpleperf_dir="$latest_simpleperf" + fi +fi + +if [[ -z "$simpleperf_dir" ]]; then + echo "simpleperf not found. Set ANDROID_NDK_HOME or install NDK under ~/Library/Android/sdk/ndk/." >&2 + exit 1 +fi + +app_profiler="$simpleperf_dir/app_profiler.py" +report_py="$simpleperf_dir/report.py" +ndk_path="$(cd -- "$simpleperf_dir/.." && pwd)" + +tmp_dir="$(mktemp -d -t openclaw-android-hotspots.XXXXXX)" +trap 'rm -rf "$tmp_dir"' EXIT + +capture_log="$tmp_dir/capture.log" +dso_csv="$tmp_dir/dso.csv" +symbols_csv="$tmp_dir/symbols.csv" +children_txt="$tmp_dir/children.txt" + +cd "$ANDROID_DIR" +./gradlew :app:installDebug --console=plain >"$tmp_dir/install.log" 2>&1 + +if ! uv run --no-project python3 "$app_profiler" \ + -p "$PACKAGE" \ + -a "$ACTIVITY" \ + -o "$OUTPUT_PERF_DATA" \ + --ndk_path "$ndk_path" \ + -r "-e task-clock:u -f 1000 -g --duration $DURATION_SECONDS" \ + >"$capture_log" 2>&1; then + echo "simpleperf capture failed. tail(capture_log):" >&2 + tail -n 120 "$capture_log" >&2 + exit 1 +fi + +uv run --no-project python3 "$report_py" \ + -i "$OUTPUT_PERF_DATA" \ + --sort dso \ + --csv \ + --csv-separator "|" \ + --include-process-name "$PACKAGE" \ + >"$dso_csv" 2>"$tmp_dir/report-dso.err" + +uv run --no-project python3 "$report_py" \ + -i "$OUTPUT_PERF_DATA" \ + --sort dso,symbol \ + --csv \ + --csv-separator "|" \ + --include-process-name "$PACKAGE" \ + >"$symbols_csv" 2>"$tmp_dir/report-symbols.err" + +uv run --no-project python3 "$report_py" \ + -i "$OUTPUT_PERF_DATA" \ + --children \ + --sort dso,symbol \ + -n \ + --percent-limit 0.2 \ + --include-process-name "$PACKAGE" \ + >"$children_txt" 2>"$tmp_dir/report-children.err" + +clean_csv() { + awk 'BEGIN{print_on=0} /^Overhead\|/{print_on=1} print_on==1{print}' "$1" +} + +echo "perf_data=$OUTPUT_PERF_DATA" +echo +echo "top_dso_self:" +clean_csv "$dso_csv" | tail -n +2 | awk -F'|' 'NR<=10 {printf " %s %s\n", $1, $2}' +echo +echo "top_symbols_self:" +clean_csv "$symbols_csv" | tail -n +2 | awk -F'|' 'NR<=20 {printf " %s %s :: %s\n", $1, $2, $3}' +echo +echo "app_path_clues_children:" +rg 'androidx\.compose|MainActivity|NodeRuntime|NodeForegroundService|SecurePrefs|WebView|libwebviewchromium' "$children_txt" | awk 'NR<=20 {print}' || true diff --git a/apps/android/settings.gradle.kts b/apps/android/settings.gradle.kts index b3b43a44550..25e5d09bbe1 100644 --- a/apps/android/settings.gradle.kts +++ b/apps/android/settings.gradle.kts @@ -16,3 +16,4 @@ dependencyResolutionManagement { rootProject.name = "OpenClawNodeAndroid" include(":app") +include(":benchmark") diff --git a/apps/android/style.md b/apps/android/style.md new file mode 100644 index 00000000000..f2b892ac6ff --- /dev/null +++ b/apps/android/style.md @@ -0,0 +1,113 @@ +# OpenClaw Android UI Style Guide + +Scope: all native Android UI in `apps/android` (Jetpack Compose). +Goal: one coherent visual system across onboarding, settings, and future screens. + +## 1. Design Direction + +- Clean, quiet surfaces. +- Strong readability first. +- One clear primary action per screen state. +- Progressive disclosure for advanced controls. +- Deterministic flows: validate early, fail clearly. + +## 2. Style Baseline + +The onboarding flow defines the current visual baseline. +New screens should match that language unless there is a strong product reason not to. + +Baseline traits: + +- Light neutral background with subtle depth. +- Clear blue accent for active/primary states. +- Strong border hierarchy for structure. +- Medium/semibold typography (no thin text). +- Divider-and-spacing layout over heavy card nesting. + +## 3. Core Tokens + +Use these as shared design tokens for new Compose UI. + +- Background gradient: `#FFFFFF`, `#F7F8FA`, `#EFF1F5` +- Surface: `#F6F7FA` +- Border: `#E5E7EC` +- Border strong: `#D6DAE2` +- Text primary: `#17181C` +- Text secondary: `#4D5563` +- Text tertiary: `#8A92A2` +- Accent primary: `#1D5DD8` +- Accent soft: `#ECF3FF` +- Success: `#2F8C5A` +- Warning: `#C8841A` + +Rule: do not introduce random per-screen colors when an existing token fits. + +## 4. Typography + +Primary type family: Manrope (`400/500/600/700`). + +Recommended scale: + +- Display: `34sp / 40sp`, bold +- Section title: `24sp / 30sp`, semibold +- Headline/action: `16sp / 22sp`, semibold +- Body: `15sp / 22sp`, medium +- Callout/helper: `14sp / 20sp`, medium +- Caption 1: `12sp / 16sp`, medium +- Caption 2: `11sp / 14sp`, medium + +Use monospace only for commands, setup codes, endpoint-like values. +Hard rule: avoid ultra-thin weights on light backgrounds. + +## 5. Layout And Spacing + +- Respect safe drawing insets. +- Keep content hierarchy mostly via spacing + dividers. +- Prefer vertical rhythm from `8/10/12/14/20dp`. +- Use pinned bottom actions for multi-step or high-importance flows. +- Avoid unnecessary container nesting. + +## 6. Buttons And Actions + +- Primary action: filled accent button, visually dominant. +- Secondary action: lower emphasis (outlined/text/surface button). +- Icon-only buttons must remain legible and >=44dp target. +- Back buttons in action rows use rounded-square shape, not circular by default. + +## 7. Inputs And Forms + +- Always show explicit label or clear context title. +- Keep helper copy short and actionable. +- Validate before advancing steps. +- Prefer immediate inline errors over hidden failure states. +- Keep optional advanced fields explicit (`Manual`, `Advanced`, etc.). + +## 8. Progress And Multi-Step Flows + +- Use clear step count (`Step X of N`). +- Use labeled progress rail/indicator when steps are discrete. +- Keep navigation predictable: back/next behavior should never surprise. + +## 9. Accessibility + +- Minimum practical touch target: `44dp`. +- Do not rely on color alone for status. +- Preserve high contrast for all text tiers. +- Add meaningful `contentDescription` for icon-only controls. + +## 10. Architecture Rules + +- Durable UI state in `MainViewModel`. +- Composables: state in, callbacks out. +- No business/network logic in composables. +- Keep side effects explicit (`LaunchedEffect`, activity result APIs). + +## 11. Source Of Truth + +- `app/src/main/java/ai/openclaw/android/ui/OpenClawTheme.kt` +- `app/src/main/java/ai/openclaw/android/ui/OnboardingFlow.kt` +- `app/src/main/java/ai/openclaw/android/ui/RootScreen.kt` +- `app/src/main/java/ai/openclaw/android/ui/SettingsSheet.kt` +- `app/src/main/java/ai/openclaw/android/MainViewModel.kt` + +If style and implementation diverge, update both in the same change. diff --git a/apps/ios/README.md b/apps/ios/README.md index b870bdcea58..c7c501fcbff 100644 --- a/apps/ios/README.md +++ b/apps/ios/README.md @@ -67,6 +67,37 @@ pnpm ios:open - iPhone node commands in foreground: camera snap/clip, canvas present/navigate/eval/snapshot, screen record, location, contacts, calendar, reminders, photos, motion, local notifications. - Share extension deep-link forwarding into the connected gateway session. +## Location Automation Use Case (Testing) + +Use this for automation signals ("I moved", "I arrived", "I left"), not as a keep-awake mechanism. + +- Product intent: + - movement-aware automations driven by iOS location events + - example: arrival/exit geofence, significant movement, visit detection +- Non-goal: + - continuous GPS polling just to keep the app alive + +Test path to include in QA runs: + +1. Enable location permission in app: + - set `Always` permission + - verify background location capability is enabled in the build profile +2. Background the app and trigger movement: + - walk/drive enough for a significant location update, or cross a configured geofence +3. Validate gateway side effects: + - node reconnect/wake if needed + - expected location/movement event arrives at gateway + - automation trigger executes once (no duplicate storm) +4. Validate resource impact: + - no sustained high thermal state + - no excessive background battery drain over a short observation window + +Pass criteria: + +- movement events are delivered reliably enough for automation UX +- no location-driven reconnect spam loops +- app remains stable after repeated background/foreground transitions + ## Known Issues / Limitations / Problems - Foreground-first: iOS can suspend sockets in background; reconnect recovery is still being tuned. diff --git a/apps/ios/ShareExtension/Info.plist b/apps/ios/ShareExtension/Info.plist index aa4ef744434..0eb61bc5add 100644 --- a/apps/ios/ShareExtension/Info.plist +++ b/apps/ios/ShareExtension/Info.plist @@ -17,9 +17,9 @@ CFBundlePackageType XPC! CFBundleShortVersionString - 2026.2.18 + 2026.2.27 CFBundleVersion - 20260218 + 20260227 NSExtension NSExtensionAttributes diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/100.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/100.png new file mode 100644 index 00000000000..22a04c9f22a Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/100.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/102.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/102.png new file mode 100644 index 00000000000..ff8397de297 Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/102.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/1024.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/1024.png new file mode 100644 index 00000000000..ecea78807d8 Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/1024.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/108.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/108.png new file mode 100644 index 00000000000..a6888456dfa Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/108.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/114.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/114.png new file mode 100644 index 00000000000..20e9ea1a557 Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/114.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/120.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/120.png new file mode 100644 index 00000000000..154836b43a2 Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/120.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/172.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/172.png new file mode 100644 index 00000000000..a66c0132393 Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/172.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/180.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/180.png new file mode 100644 index 00000000000..d01e83d8ccc Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/180.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/196.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/196.png new file mode 100644 index 00000000000..b7989e43d84 Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/196.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/216.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/216.png new file mode 100644 index 00000000000..4dfb94abefb Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/216.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/234.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/234.png new file mode 100644 index 00000000000..c0da9ae922c Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/234.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/258.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/258.png new file mode 100644 index 00000000000..dbfb75050bd Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/258.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/29.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/29.png new file mode 100644 index 00000000000..f4d57311481 Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/29.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/40.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/40.png new file mode 100644 index 00000000000..87a14602e3c Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/40.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/48.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/48.png new file mode 100644 index 00000000000..f66c2ded344 Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/48.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/55.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/55.png new file mode 100644 index 00000000000..0730736fca0 Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/55.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/57.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/57.png new file mode 100644 index 00000000000..f8946de39b3 Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/57.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/58.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/58.png new file mode 100644 index 00000000000..92ae2f999d9 Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/58.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/60.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/60.png new file mode 100644 index 00000000000..03231a71d18 Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/60.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/66.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/66.png new file mode 100644 index 00000000000..834c6b0987f Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/66.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/80.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/80.png new file mode 100644 index 00000000000..485a1aae7bd Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/80.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/87.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/87.png new file mode 100644 index 00000000000..61da8b5fd79 Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/87.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/88.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/88.png new file mode 100644 index 00000000000..f47fb37b5fc Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/88.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/92.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/92.png new file mode 100644 index 00000000000..67a10a48458 Binary files /dev/null and b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/92.png differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/Contents.json b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/Contents.json index 13847b5b5bf..922e8c6d731 100644 --- a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/Contents.json +++ b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/Contents.json @@ -1,31 +1 @@ -{ - "images" : [ - { "filename" : "icon-20@1x.png", "idiom" : "ipad", "scale" : "1x", "size" : "20x20" }, - { "filename" : "icon-20@2x.png", "idiom" : "ipad", "scale" : "2x", "size" : "20x20" }, - { "filename" : "icon-20@2x.png", "idiom" : "iphone","scale" : "2x", "size" : "20x20" }, - { "filename" : "icon-20@3x.png", "idiom" : "iphone","scale" : "3x", "size" : "20x20" }, - - { "filename" : "icon-29@1x.png", "idiom" : "ipad", "scale" : "1x", "size" : "29x29" }, - { "filename" : "icon-29@2x.png", "idiom" : "ipad", "scale" : "2x", "size" : "29x29" }, - { "filename" : "icon-29@2x.png", "idiom" : "iphone","scale" : "2x", "size" : "29x29" }, - { "filename" : "icon-29@3x.png", "idiom" : "iphone","scale" : "3x", "size" : "29x29" }, - - { "filename" : "icon-40@1x.png", "idiom" : "ipad", "scale" : "1x", "size" : "40x40" }, - { "filename" : "icon-40@2x.png", "idiom" : "ipad", "scale" : "2x", "size" : "40x40" }, - { "filename" : "icon-40@2x.png", "idiom" : "iphone","scale" : "2x", "size" : "40x40" }, - { "filename" : "icon-40@3x.png", "idiom" : "iphone","scale" : "3x", "size" : "40x40" }, - - { "filename" : "icon-60@2x.png", "idiom" : "iphone","scale" : "2x", "size" : "60x60" }, - { "filename" : "icon-60@3x.png", "idiom" : "iphone","scale" : "3x", "size" : "60x60" }, - - { "filename" : "icon-76@2x.png", "idiom" : "ipad", "scale" : "2x", "size" : "76x76" }, - - { "filename" : "icon-83.5@2x.png", "idiom" : "ipad", "scale" : "2x", "size" : "83.5x83.5" }, - - { "filename" : "icon-1024.png", "idiom" : "ios-marketing", "scale" : "1x", "size" : "1024x1024" } - ], - "info" : { - "author" : "xcode", - "version" : 1 - } -} +{"images":[{"size":"60x60","expected-size":"180","filename":"180.png","folder":"Assets.xcassets/AppIcon.appiconset/","idiom":"iphone","scale":"3x"},{"size":"40x40","expected-size":"80","filename":"80.png","folder":"Assets.xcassets/AppIcon.appiconset/","idiom":"iphone","scale":"2x"},{"size":"40x40","expected-size":"120","filename":"120.png","folder":"Assets.xcassets/AppIcon.appiconset/","idiom":"iphone","scale":"3x"},{"size":"60x60","expected-size":"120","filename":"120.png","folder":"Assets.xcassets/AppIcon.appiconset/","idiom":"iphone","scale":"2x"},{"size":"57x57","expected-size":"57","filename":"57.png","folder":"Assets.xcassets/AppIcon.appiconset/","idiom":"iphone","scale":"1x"},{"size":"29x29","expected-size":"58","filename":"58.png","folder":"Assets.xcassets/AppIcon.appiconset/","idiom":"iphone","scale":"2x"},{"size":"29x29","expected-size":"29","filename":"29.png","folder":"Assets.xcassets/AppIcon.appiconset/","idiom":"iphone","scale":"1x"},{"size":"29x29","expected-size":"87","filename":"87.png","folder":"Assets.xcassets/AppIcon.appiconset/","idiom":"iphone","scale":"3x"},{"size":"57x57","expected-size":"114","filename":"114.png","folder":"Assets.xcassets/AppIcon.appiconset/","idiom":"iphone","scale":"2x"},{"size":"20x20","expected-size":"40","filename":"40.png","folder":"Assets.xcassets/AppIcon.appiconset/","idiom":"iphone","scale":"2x"},{"size":"20x20","expected-size":"60","filename":"60.png","folder":"Assets.xcassets/AppIcon.appiconset/","idiom":"iphone","scale":"3x"},{"size":"1024x1024","filename":"1024.png","expected-size":"1024","idiom":"ios-marketing","folder":"Assets.xcassets/AppIcon.appiconset/","scale":"1x"},{"idiom":"watch","filename":"172.png","folder":"Assets.xcassets/AppIcon.appiconset/","subtype":"38mm","scale":"2x","size":"86x86","expected-size":"172","role":"quickLook"},{"idiom":"watch","filename":"80.png","folder":"Assets.xcassets/AppIcon.appiconset/","subtype":"38mm","scale":"2x","size":"40x40","expected-size":"80","role":"appLauncher"},{"idiom":"watch","filename":"88.png","folder":"Assets.xcassets/AppIcon.appiconset/","subtype":"40mm","scale":"2x","size":"44x44","expected-size":"88","role":"appLauncher"},{"idiom":"watch","filename":"102.png","folder":"Assets.xcassets/AppIcon.appiconset/","subtype":"45mm","scale":"2x","size":"51x51","expected-size":"102","role":"appLauncher"},{"idiom":"watch","filename":"108.png","folder":"Assets.xcassets/AppIcon.appiconset/","subtype":"49mm","scale":"2x","size":"54x54","expected-size":"108","role":"appLauncher"},{"idiom":"watch","filename":"92.png","folder":"Assets.xcassets/AppIcon.appiconset/","subtype":"41mm","scale":"2x","size":"46x46","expected-size":"92","role":"appLauncher"},{"idiom":"watch","filename":"100.png","folder":"Assets.xcassets/AppIcon.appiconset/","subtype":"44mm","scale":"2x","size":"50x50","expected-size":"100","role":"appLauncher"},{"idiom":"watch","filename":"196.png","folder":"Assets.xcassets/AppIcon.appiconset/","subtype":"42mm","scale":"2x","size":"98x98","expected-size":"196","role":"quickLook"},{"idiom":"watch","filename":"216.png","folder":"Assets.xcassets/AppIcon.appiconset/","subtype":"44mm","scale":"2x","size":"108x108","expected-size":"216","role":"quickLook"},{"idiom":"watch","filename":"234.png","folder":"Assets.xcassets/AppIcon.appiconset/","subtype":"45mm","scale":"2x","size":"117x117","expected-size":"234","role":"quickLook"},{"idiom":"watch","filename":"258.png","folder":"Assets.xcassets/AppIcon.appiconset/","subtype":"49mm","scale":"2x","size":"129x129","expected-size":"258","role":"quickLook"},{"idiom":"watch","filename":"48.png","folder":"Assets.xcassets/AppIcon.appiconset/","subtype":"38mm","scale":"2x","size":"24x24","expected-size":"48","role":"notificationCenter"},{"idiom":"watch","filename":"55.png","folder":"Assets.xcassets/AppIcon.appiconset/","subtype":"42mm","scale":"2x","size":"27.5x27.5","expected-size":"55","role":"notificationCenter"},{"idiom":"watch","filename":"66.png","folder":"Assets.xcassets/AppIcon.appiconset/","subtype":"45mm","scale":"2x","size":"33x33","expected-size":"66","role":"notificationCenter"},{"size":"29x29","expected-size":"87","filename":"87.png","folder":"Assets.xcassets/AppIcon.appiconset/","idiom":"watch","role":"companionSettings","scale":"3x"},{"size":"29x29","expected-size":"58","filename":"58.png","folder":"Assets.xcassets/AppIcon.appiconset/","idiom":"watch","role":"companionSettings","scale":"2x"},{"size":"1024x1024","expected-size":"1024","filename":"1024.png","folder":"Assets.xcassets/AppIcon.appiconset/","idiom":"watch-marketing","scale":"1x"}]} \ No newline at end of file diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-1024.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-1024.png deleted file mode 100644 index 1ebd257d93f..00000000000 Binary files a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-1024.png and /dev/null differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-20@1x.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-20@1x.png deleted file mode 100644 index 0aa1506a095..00000000000 Binary files a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-20@1x.png and /dev/null differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-20@2x.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-20@2x.png deleted file mode 100644 index dd8a14724eb..00000000000 Binary files a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-20@2x.png and /dev/null differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-20@3x.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-20@3x.png deleted file mode 100644 index ca160dc2e84..00000000000 Binary files a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-20@3x.png and /dev/null differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-29@1x.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-29@1x.png deleted file mode 100644 index 9020a8672d3..00000000000 Binary files a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-29@1x.png and /dev/null differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-29@2x.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-29@2x.png deleted file mode 100644 index ff85b417fec..00000000000 Binary files a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-29@2x.png and /dev/null differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-29@3x.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-29@3x.png deleted file mode 100644 index e12fff03140..00000000000 Binary files a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-29@3x.png and /dev/null differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-40@1x.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-40@1x.png deleted file mode 100644 index dd8a14724eb..00000000000 Binary files a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-40@1x.png and /dev/null differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-40@2x.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-40@2x.png deleted file mode 100644 index 9b3da5155ef..00000000000 Binary files a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-40@2x.png and /dev/null differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-40@3x.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-40@3x.png deleted file mode 100644 index f57a0c1323c..00000000000 Binary files a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-40@3x.png and /dev/null differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-60@2x.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-60@2x.png deleted file mode 100644 index f57a0c1323c..00000000000 Binary files a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-60@2x.png and /dev/null differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-60@3x.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-60@3x.png deleted file mode 100644 index b94278f29d0..00000000000 Binary files a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-60@3x.png and /dev/null differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-76@2x.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-76@2x.png deleted file mode 100644 index 2d6240dc679..00000000000 Binary files a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-76@2x.png and /dev/null differ diff --git a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-83.5@2x.png b/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-83.5@2x.png deleted file mode 100644 index 7321091c561..00000000000 Binary files a/apps/ios/Sources/Assets.xcassets/AppIcon.appiconset/icon-83.5@2x.png and /dev/null differ diff --git a/apps/ios/Sources/Assets.xcassets/Contents.json b/apps/ios/Sources/Assets.xcassets/Contents.json deleted file mode 100644 index 73c00596a7f..00000000000 --- a/apps/ios/Sources/Assets.xcassets/Contents.json +++ /dev/null @@ -1,6 +0,0 @@ -{ - "info" : { - "author" : "xcode", - "version" : 1 - } -} diff --git a/apps/ios/Sources/Chat/IOSGatewayChatTransport.swift b/apps/ios/Sources/Chat/IOSGatewayChatTransport.swift index 9571839059d..67f01138803 100644 --- a/apps/ios/Sources/Chat/IOSGatewayChatTransport.swift +++ b/apps/ios/Sources/Chat/IOSGatewayChatTransport.swift @@ -54,7 +54,12 @@ struct IOSGatewayChatTransport: OpenClawChatTransport, Sendable { idempotencyKey: String, attachments: [OpenClawChatAttachmentPayload]) async throws -> OpenClawChatSendResponse { - Self.logger.info("chat.send start sessionKey=\(sessionKey, privacy: .public) len=\(message.count, privacy: .public) attachments=\(attachments.count, privacy: .public)") + let startLogMessage = + "chat.send start sessionKey=\(sessionKey) " + + "len=\(message.count) attachments=\(attachments.count)" + Self.logger.info( + "\(startLogMessage, privacy: .public)" + ) struct Params: Codable { var sessionKey: String var message: String diff --git a/apps/ios/Sources/Device/DeviceInfoHelper.swift b/apps/ios/Sources/Device/DeviceInfoHelper.swift new file mode 100644 index 00000000000..eeed54c4652 --- /dev/null +++ b/apps/ios/Sources/Device/DeviceInfoHelper.swift @@ -0,0 +1,71 @@ +import Foundation +import UIKit + +import Darwin + +/// Shared device and platform info for Settings, gateway node payloads, and device status. +enum DeviceInfoHelper { + /// e.g. "iOS 18.0.0" or "iPadOS 18.0.0" by interface idiom. Use for gateway/device payloads. + static func platformString() -> String { + let v = ProcessInfo.processInfo.operatingSystemVersion + let name = switch UIDevice.current.userInterfaceIdiom { + case .pad: + "iPadOS" + case .phone: + "iOS" + default: + "iOS" + } + return "\(name) \(v.majorVersion).\(v.minorVersion).\(v.patchVersion)" + } + + /// Always "iOS X.Y.Z" for UI display (e.g. Settings), matching legacy behavior on iPad. + static func platformStringForDisplay() -> String { + let v = ProcessInfo.processInfo.operatingSystemVersion + return "iOS \(v.majorVersion).\(v.minorVersion).\(v.patchVersion)" + } + + /// Device family for display: "iPad", "iPhone", or "iOS". + static func deviceFamily() -> String { + switch UIDevice.current.userInterfaceIdiom { + case .pad: + "iPad" + case .phone: + "iPhone" + default: + "iOS" + } + } + + /// Machine model identifier from uname (e.g. "iPhone17,1"). + static func modelIdentifier() -> String { + var systemInfo = utsname() + uname(&systemInfo) + let machine = withUnsafeBytes(of: &systemInfo.machine) { ptr in + String(bytes: ptr.prefix { $0 != 0 }, encoding: .utf8) + } + let trimmed = machine?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + return trimmed.isEmpty ? "unknown" : trimmed + } + + /// App marketing version only, e.g. "2026.2.0" or "dev". + static func appVersion() -> String { + Bundle.main.infoDictionary?["CFBundleShortVersionString"] as? String ?? "dev" + } + + /// App build string, e.g. "123" or "". + static func appBuild() -> String { + let raw = Bundle.main.infoDictionary?["CFBundleVersion"] as? String ?? "" + return raw.trimmingCharacters(in: .whitespacesAndNewlines) + } + + /// Display string for Settings: "1.2.3" or "1.2.3 (456)" when build differs. + static func openClawVersionString() -> String { + let version = appVersion() + let build = appBuild() + if build.isEmpty || build == version { + return version + } + return "\(version) (\(build))" + } +} diff --git a/apps/ios/Sources/Device/DeviceStatusService.swift b/apps/ios/Sources/Device/DeviceStatusService.swift index fed2716b5b8..a80a98101fa 100644 --- a/apps/ios/Sources/Device/DeviceStatusService.swift +++ b/apps/ios/Sources/Device/DeviceStatusService.swift @@ -26,12 +26,12 @@ final class DeviceStatusService: DeviceStatusServicing { func info() -> OpenClawDeviceInfoPayload { let device = UIDevice.current - let appVersion = Bundle.main.infoDictionary?["CFBundleShortVersionString"] as? String ?? "dev" - let appBuild = Bundle.main.infoDictionary?["CFBundleVersion"] as? String ?? "0" + let appVersion = DeviceInfoHelper.appVersion() + let appBuild = DeviceStatusService.fallbackAppBuild(DeviceInfoHelper.appBuild()) let locale = Locale.preferredLanguages.first ?? Locale.current.identifier return OpenClawDeviceInfoPayload( deviceName: device.name, - modelIdentifier: Self.modelIdentifier(), + modelIdentifier: DeviceInfoHelper.modelIdentifier(), systemName: device.systemName, systemVersion: device.systemVersion, appVersion: appVersion, @@ -75,13 +75,8 @@ final class DeviceStatusService: DeviceStatusServicing { return OpenClawStorageStatusPayload(totalBytes: total, freeBytes: free, usedBytes: used) } - private static func modelIdentifier() -> String { - var systemInfo = utsname() - uname(&systemInfo) - let machine = withUnsafeBytes(of: &systemInfo.machine) { ptr in - String(bytes: ptr.prefix { $0 != 0 }, encoding: .utf8) - } - let trimmed = machine?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" - return trimmed.isEmpty ? "unknown" : trimmed + /// Fallback for payloads that require a non-empty build (e.g. "0"). + private static func fallbackAppBuild(_ build: String) -> String { + build.isEmpty ? "0" : build } } diff --git a/apps/ios/Sources/Device/NetworkStatusService.swift b/apps/ios/Sources/Device/NetworkStatusService.swift index 7d92d1cc1ca..bc27eb19791 100644 --- a/apps/ios/Sources/Device/NetworkStatusService.swift +++ b/apps/ios/Sources/Device/NetworkStatusService.swift @@ -6,7 +6,7 @@ final class NetworkStatusService: @unchecked Sendable { func currentStatus(timeoutMs: Int = 1500) async -> OpenClawNetworkStatusPayload { await withCheckedContinuation { cont in let monitor = NWPathMonitor() - let queue = DispatchQueue(label: "bot.molt.ios.network-status") + let queue = DispatchQueue(label: "ai.openclaw.ios.network-status") let state = NetworkStatusState() monitor.pathUpdateHandler = { path in diff --git a/apps/ios/Sources/Gateway/DeepLinkAgentPromptAlert.swift b/apps/ios/Sources/Gateway/DeepLinkAgentPromptAlert.swift new file mode 100644 index 00000000000..0624e976b51 --- /dev/null +++ b/apps/ios/Sources/Gateway/DeepLinkAgentPromptAlert.swift @@ -0,0 +1,40 @@ +import SwiftUI + +struct DeepLinkAgentPromptAlert: ViewModifier { + @Environment(NodeAppModel.self) private var appModel: NodeAppModel + + private var promptBinding: Binding { + Binding( + get: { self.appModel.pendingAgentDeepLinkPrompt }, + set: { _ in + // Keep prompt state until explicit user action. + }) + } + + func body(content: Content) -> some View { + content.alert(item: self.promptBinding) { prompt in + Alert( + title: Text("Run OpenClaw agent?"), + message: Text( + """ + Message: + \(prompt.messagePreview) + + URL: + \(prompt.urlPreview) + """), + primaryButton: .cancel(Text("Cancel")) { + self.appModel.declinePendingAgentDeepLinkPrompt() + }, + secondaryButton: .default(Text("Run")) { + Task { await self.appModel.approvePendingAgentDeepLinkPrompt() } + }) + } + } +} + +extension View { + func deepLinkAgentPromptAlert() -> some View { + self.modifier(DeepLinkAgentPromptAlert()) + } +} diff --git a/apps/ios/Sources/Gateway/GatewayConnectionController.swift b/apps/ios/Sources/Gateway/GatewayConnectionController.swift index 92abd996b72..53e32684988 100644 --- a/apps/ios/Sources/Gateway/GatewayConnectionController.swift +++ b/apps/ios/Sources/Gateway/GatewayConnectionController.swift @@ -5,6 +5,7 @@ import CoreMotion import CryptoKit import EventKit import Foundation +import Darwin import OpenClawKit import Network import Observation @@ -162,7 +163,7 @@ final class GatewayConnectionController { .trimmingCharacters(in: .whitespacesAndNewlines) ?? "" let token = GatewaySettingsStore.loadGatewayToken(instanceId: instanceId) let password = GatewaySettingsStore.loadGatewayPassword(instanceId: instanceId) - let resolvedUseTLS = useTLS + let resolvedUseTLS = self.resolveManualUseTLS(host: host, useTLS: useTLS) guard let resolvedPort = self.resolveManualPort(host: host, port: port, useTLS: resolvedUseTLS) else { return } let stableID = self.manualStableID(host: host, port: resolvedPort) @@ -211,10 +212,27 @@ final class GatewayConnectionController { await self.connectManual(host: host, port: port, useTLS: useTLS) case let .discovered(stableID, _): guard let gateway = self.gateways.first(where: { $0.stableID == stableID }) else { return } - await self.connectDiscoveredGateway(gateway) + _ = await self.connectDiscoveredGateway(gateway) } } + /// Rebuild connect options from current local settings (caps/commands/permissions) + /// and re-apply the active gateway config so capability changes take effect immediately. + func refreshActiveGatewayRegistrationFromSettings() { + guard let appModel else { return } + guard let cfg = appModel.activeGatewayConnectConfig else { return } + guard appModel.gatewayAutoReconnectEnabled else { return } + + let refreshedConfig = GatewayConnectConfig( + url: cfg.url, + stableID: cfg.stableID, + tls: cfg.tls, + token: cfg.token, + password: cfg.password, + nodeOptions: self.makeConnectOptions(stableID: cfg.stableID)) + appModel.applyGatewayConnectConfig(refreshedConfig) + } + func clearPendingTrustPrompt() { self.pendingTrustPrompt = nil self.pendingTrustConnect = nil @@ -309,7 +327,7 @@ final class GatewayConnectionController { let manualPort = defaults.integer(forKey: "gateway.manual.port") let manualTLS = defaults.bool(forKey: "gateway.manual.tls") - let resolvedUseTLS = manualTLS || self.shouldForceTLS(host: manualHost) + let resolvedUseTLS = self.resolveManualUseTLS(host: manualHost, useTLS: manualTLS) guard let resolvedPort = self.resolveManualPort( host: manualHost, port: manualPort, @@ -320,7 +338,7 @@ final class GatewayConnectionController { let tlsParams = self.resolveManualTLSParams( stableID: stableID, tlsEnabled: resolvedUseTLS, - allowTOFUReset: self.shouldForceTLS(host: manualHost)) + allowTOFUReset: self.shouldRequireTLS(host: manualHost)) guard let url = self.buildGatewayURL( host: manualHost, @@ -340,7 +358,7 @@ final class GatewayConnectionController { if let lastKnown = GatewaySettingsStore.loadLastGatewayConnection() { if case let .manual(host, port, useTLS, stableID) = lastKnown { - let resolvedUseTLS = useTLS || self.shouldForceTLS(host: host) + let resolvedUseTLS = self.resolveManualUseTLS(host: host, useTLS: useTLS) let stored = GatewayTLSStore.loadFingerprint(stableID: stableID) let tlsParams = stored.map { fp in GatewayTLSParams(required: true, expectedFingerprint: fp, allowTOFU: false, storeKey: stableID) @@ -381,7 +399,7 @@ final class GatewayConnectionController { self.didAutoConnect = true Task { [weak self] in guard let self else { return } - await self.connectDiscoveredGateway(target) + _ = await self.connectDiscoveredGateway(target) } return } @@ -393,7 +411,7 @@ final class GatewayConnectionController { self.didAutoConnect = true Task { [weak self] in guard let self else { return } - await self.connectDiscoveredGateway(gateway) + _ = await self.connectDiscoveredGateway(gateway) } return } @@ -614,7 +632,8 @@ final class GatewayConnectionController { 0, NI_NUMERICHOST) guard rc == 0 else { return nil } - return String(cString: buffer) + let bytes = buffer.prefix { $0 != 0 }.map { UInt8(bitPattern: $0) } + return String(bytes: bytes, encoding: .utf8) } if let host, !host.isEmpty { @@ -646,12 +665,65 @@ final class GatewayConnectionController { return components.url } + private func resolveManualUseTLS(host: String, useTLS: Bool) -> Bool { + useTLS || self.shouldRequireTLS(host: host) + } + + private func shouldRequireTLS(host: String) -> Bool { + !Self.isLoopbackHost(host) + } + private func shouldForceTLS(host: String) -> Bool { let trimmed = host.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() if trimmed.isEmpty { return false } return trimmed.hasSuffix(".ts.net") || trimmed.hasSuffix(".ts.net.") } + private static func isLoopbackHost(_ rawHost: String) -> Bool { + var host = rawHost.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() + guard !host.isEmpty else { return false } + + if host.hasPrefix("[") && host.hasSuffix("]") { + host.removeFirst() + host.removeLast() + } + if host.hasSuffix(".") { + host.removeLast() + } + if let zoneIndex = host.firstIndex(of: "%") { + host = String(host[.. Bool { + var addr = in_addr() + let parsed = host.withCString { inet_pton(AF_INET, $0, &addr) == 1 } + guard parsed else { return false } + let value = UInt32(bigEndian: addr.s_addr) + let firstOctet = UInt8((value >> 24) & 0xFF) + return firstOctet == 127 + } + + private static func isLoopbackIPv6(_ host: String) -> Bool { + var addr = in6_addr() + let parsed = host.withCString { inet_pton(AF_INET6, $0, &addr) == 1 } + guard parsed else { return false } + return withUnsafeBytes(of: &addr) { rawBytes in + let bytes = rawBytes.bindMemory(to: UInt8.self) + let isV6Loopback = bytes[0..<15].allSatisfy { $0 == 0 } && bytes[15] == 1 + if isV6Loopback { return true } + + let isMappedV4 = bytes[0..<10].allSatisfy { $0 == 0 } && bytes[10] == 0xFF && bytes[11] == 0xFF + return isMappedV4 && bytes[12] == 127 + } + } + private func manualStableID(host: String, port: Int) -> String { "manual|\(host.lowercased())|\(port)" } @@ -818,11 +890,9 @@ final class GatewayConnectionController { permissions["contacts"] = contactsStatus == .authorized || contactsStatus == .limited let calendarStatus = EKEventStore.authorizationStatus(for: .event) - permissions["calendar"] = - calendarStatus == .authorized || calendarStatus == .fullAccess || calendarStatus == .writeOnly + permissions["calendar"] = Self.hasEventKitAccess(calendarStatus) let remindersStatus = EKEventStore.authorizationStatus(for: .reminder) - permissions["reminders"] = - remindersStatus == .authorized || remindersStatus == .fullAccess || remindersStatus == .writeOnly + permissions["reminders"] = Self.hasEventKitAccess(remindersStatus) let motionStatus = CMMotionActivityManager.authorizationStatus() let pedometerStatus = CMPedometer.authorizationStatus() @@ -840,54 +910,20 @@ final class GatewayConnectionController { private static func isLocationAuthorized(status: CLAuthorizationStatus) -> Bool { switch status { - case .authorizedAlways, .authorizedWhenInUse, .authorized: + case .authorizedAlways, .authorizedWhenInUse: return true default: return false } } + private static func hasEventKitAccess(_ status: EKAuthorizationStatus) -> Bool { + status == .fullAccess || status == .writeOnly + } + private static func motionAvailable() -> Bool { CMMotionActivityManager.isActivityAvailable() || CMPedometer.isStepCountingAvailable() } - - private func platformString() -> String { - let v = ProcessInfo.processInfo.operatingSystemVersion - let name = switch UIDevice.current.userInterfaceIdiom { - case .pad: - "iPadOS" - case .phone: - "iOS" - default: - "iOS" - } - return "\(name) \(v.majorVersion).\(v.minorVersion).\(v.patchVersion)" - } - - private func deviceFamily() -> String { - switch UIDevice.current.userInterfaceIdiom { - case .pad: - "iPad" - case .phone: - "iPhone" - default: - "iOS" - } - } - - private func modelIdentifier() -> String { - var systemInfo = utsname() - uname(&systemInfo) - let machine = withUnsafeBytes(of: &systemInfo.machine) { ptr in - String(bytes: ptr.prefix { $0 != 0 }, encoding: .utf8) - } - let trimmed = machine?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" - return trimmed.isEmpty ? "unknown" : trimmed - } - - private func appVersion() -> String { - Bundle.main.infoDictionary?["CFBundleShortVersionString"] as? String ?? "dev" - } } #if DEBUG @@ -909,19 +945,19 @@ extension GatewayConnectionController { } func _test_platformString() -> String { - self.platformString() + DeviceInfoHelper.platformString() } func _test_deviceFamily() -> String { - self.deviceFamily() + DeviceInfoHelper.deviceFamily() } func _test_modelIdentifier() -> String { - self.modelIdentifier() + DeviceInfoHelper.modelIdentifier() } func _test_appVersion() -> String { - self.appVersion() + DeviceInfoHelper.appVersion() } func _test_setGateways(_ gateways: [GatewayDiscoveryModel.DiscoveredGateway]) { @@ -942,10 +978,18 @@ extension GatewayConnectionController { { self.resolveDiscoveredTLSParams(gateway: gateway, allowTOFU: allowTOFU) } + + func _test_resolveManualUseTLS(host: String, useTLS: Bool) -> Bool { + self.resolveManualUseTLS(host: host, useTLS: useTLS) + } + + func _test_resolveManualPort(host: String, port: Int, useTLS: Bool) -> Int? { + self.resolveManualPort(host: host, port: port, useTLS: useTLS) + } } #endif -private final class GatewayTLSFingerprintProbe: NSObject, URLSessionDelegate { +private final class GatewayTLSFingerprintProbe: NSObject, URLSessionDelegate, @unchecked Sendable { private let url: URL private let timeoutSeconds: Double private let onComplete: (String?) -> Void diff --git a/apps/ios/Sources/Gateway/GatewayDiscoveryModel.swift b/apps/ios/Sources/Gateway/GatewayDiscoveryModel.swift index ce1ba4bf2cb..04bb220d5f3 100644 --- a/apps/ios/Sources/Gateway/GatewayDiscoveryModel.swift +++ b/apps/ios/Sources/Gateway/GatewayDiscoveryModel.swift @@ -104,7 +104,7 @@ final class GatewayDiscoveryModel { } self.browsers[domain] = browser - browser.start(queue: DispatchQueue(label: "bot.molt.ios.gateway-discovery.\(domain)")) + browser.start(queue: DispatchQueue(label: "ai.openclaw.ios.gateway-discovery.\(domain)")) } } diff --git a/apps/ios/Sources/Gateway/GatewaySettingsStore.swift b/apps/ios/Sources/Gateway/GatewaySettingsStore.swift index 3ff57ad2e67..49db9bb1bfc 100644 --- a/apps/ios/Sources/Gateway/GatewaySettingsStore.swift +++ b/apps/ios/Sources/Gateway/GatewaySettingsStore.swift @@ -25,7 +25,7 @@ enum GatewaySettingsStore { private static let instanceIdAccount = "instanceId" private static let preferredGatewayStableIDAccount = "preferredStableID" private static let lastDiscoveredGatewayStableIDAccount = "lastDiscoveredStableID" - private static let talkElevenLabsApiKeyAccount = "elevenlabs.apiKey" + private static let talkProviderApiKeyAccountPrefix = "provider.apiKey." static func bootstrapPersistence() { self.ensureStableInstanceID() @@ -145,25 +145,26 @@ enum GatewaySettingsStore { case discovered } - static func loadTalkElevenLabsApiKey() -> String? { + static func loadTalkProviderApiKey(provider: String) -> String? { + guard let providerId = self.normalizedTalkProviderID(provider) else { return nil } + let account = self.talkProviderApiKeyAccount(providerId: providerId) let value = KeychainStore.loadString( service: self.talkService, - account: self.talkElevenLabsApiKeyAccount)? + account: account)? .trimmingCharacters(in: .whitespacesAndNewlines) if value?.isEmpty == false { return value } return nil } - static func saveTalkElevenLabsApiKey(_ apiKey: String?) { + static func saveTalkProviderApiKey(_ apiKey: String?, provider: String) { + guard let providerId = self.normalizedTalkProviderID(provider) else { return } + let account = self.talkProviderApiKeyAccount(providerId: providerId) let trimmed = apiKey?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" if trimmed.isEmpty { - _ = KeychainStore.delete(service: self.talkService, account: self.talkElevenLabsApiKeyAccount) + _ = KeychainStore.delete(service: self.talkService, account: account) return } - _ = KeychainStore.saveString( - trimmed, - service: self.talkService, - account: self.talkElevenLabsApiKeyAccount) + _ = KeychainStore.saveString(trimmed, service: self.talkService, account: account) } static func saveLastGatewayConnectionManual(host: String, port: Int, useTLS: Bool, stableID: String) { @@ -278,6 +279,15 @@ enum GatewaySettingsStore { "gateway-password.\(instanceId)" } + private static func talkProviderApiKeyAccount(providerId: String) -> String { + self.talkProviderApiKeyAccountPrefix + providerId + } + + private static func normalizedTalkProviderID(_ provider: String) -> String? { + let trimmed = provider.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() + return trimmed.isEmpty ? nil : trimmed + } + private static func ensureStableInstanceID() { let defaults = UserDefaults.standard diff --git a/apps/ios/Sources/Info.plist b/apps/ios/Sources/Info.plist index 4d1fec46257..c30202f69cf 100644 --- a/apps/ios/Sources/Info.plist +++ b/apps/ios/Sources/Info.plist @@ -19,7 +19,7 @@ CFBundlePackageType APPL CFBundleShortVersionString - 2026.2.18 + 2026.2.27 CFBundleURLTypes @@ -32,7 +32,7 @@ CFBundleVersion - 20260218 + 20260227 NSAppTransportSecurity NSAllowsArbitraryLoadsInWebContent @@ -64,6 +64,10 @@ audio remote-notification + BGTaskSchedulerPermittedIdentifiers + + ai.openclaw.ios.bgrefresh + UILaunchScreen UISupportedInterfaceOrientations diff --git a/apps/ios/Sources/Location/SignificantLocationMonitor.swift b/apps/ios/Sources/Location/SignificantLocationMonitor.swift index f12a157dc69..1b8d5ca2a0d 100644 --- a/apps/ios/Sources/Location/SignificantLocationMonitor.swift +++ b/apps/ios/Sources/Location/SignificantLocationMonitor.swift @@ -10,7 +10,8 @@ enum SignificantLocationMonitor { static func startIfNeeded( locationService: any LocationServicing, locationMode: OpenClawLocationMode, - gateway: GatewayNodeSession + gateway: GatewayNodeSession, + beforeSend: (@MainActor @Sendable () async -> Void)? = nil ) { guard locationMode == .always else { return } let status = locationService.authorizationStatus() @@ -31,6 +32,9 @@ enum SignificantLocationMonitor { let json = String(data: data, encoding: .utf8) else { return } Task { @MainActor in + if let beforeSend { + await beforeSend() + } await gateway.sendEvent(event: "location.update", payloadJSON: json) } } diff --git a/apps/ios/Sources/Model/NodeAppModel+WatchNotifyNormalization.swift b/apps/ios/Sources/Model/NodeAppModel+WatchNotifyNormalization.swift new file mode 100644 index 00000000000..08ef81e0cce --- /dev/null +++ b/apps/ios/Sources/Model/NodeAppModel+WatchNotifyNormalization.swift @@ -0,0 +1,103 @@ +import Foundation +import OpenClawKit + +extension NodeAppModel { + static func normalizeWatchNotifyParams(_ params: OpenClawWatchNotifyParams) -> OpenClawWatchNotifyParams { + var normalized = params + normalized.title = params.title.trimmingCharacters(in: .whitespacesAndNewlines) + normalized.body = params.body.trimmingCharacters(in: .whitespacesAndNewlines) + normalized.promptId = self.trimmedOrNil(params.promptId) + normalized.sessionKey = self.trimmedOrNil(params.sessionKey) + normalized.kind = self.trimmedOrNil(params.kind) + normalized.details = self.trimmedOrNil(params.details) + normalized.priority = self.normalizedWatchPriority(params.priority, risk: params.risk) + normalized.risk = self.normalizedWatchRisk(params.risk, priority: normalized.priority) + + let normalizedActions = self.normalizeWatchActions( + params.actions, + kind: normalized.kind, + promptId: normalized.promptId) + normalized.actions = normalizedActions.isEmpty ? nil : normalizedActions + return normalized + } + + static func normalizeWatchActions( + _ actions: [OpenClawWatchAction]?, + kind: String?, + promptId: String?) -> [OpenClawWatchAction] + { + let provided = (actions ?? []).compactMap { action -> OpenClawWatchAction? in + let id = action.id.trimmingCharacters(in: .whitespacesAndNewlines) + let label = action.label.trimmingCharacters(in: .whitespacesAndNewlines) + guard !id.isEmpty, !label.isEmpty else { return nil } + return OpenClawWatchAction( + id: id, + label: label, + style: self.trimmedOrNil(action.style)) + } + if !provided.isEmpty { + return Array(provided.prefix(4)) + } + + // Only auto-insert quick actions when this is a prompt/decision flow. + guard promptId?.isEmpty == false else { + return [] + } + + let normalizedKind = kind?.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() ?? "" + if normalizedKind.contains("approval") || normalizedKind.contains("approve") { + return [ + OpenClawWatchAction(id: "approve", label: "Approve"), + OpenClawWatchAction(id: "decline", label: "Decline", style: "destructive"), + OpenClawWatchAction(id: "open_phone", label: "Open iPhone"), + OpenClawWatchAction(id: "escalate", label: "Escalate"), + ] + } + + return [ + OpenClawWatchAction(id: "done", label: "Done"), + OpenClawWatchAction(id: "snooze_10m", label: "Snooze 10m"), + OpenClawWatchAction(id: "open_phone", label: "Open iPhone"), + OpenClawWatchAction(id: "escalate", label: "Escalate"), + ] + } + + static func normalizedWatchRisk( + _ risk: OpenClawWatchRisk?, + priority: OpenClawNotificationPriority?) -> OpenClawWatchRisk? + { + if let risk { return risk } + switch priority { + case .passive: + return .low + case .active: + return .medium + case .timeSensitive: + return .high + case nil: + return nil + } + } + + static func normalizedWatchPriority( + _ priority: OpenClawNotificationPriority?, + risk: OpenClawWatchRisk?) -> OpenClawNotificationPriority? + { + if let priority { return priority } + switch risk { + case .low: + return .passive + case .medium: + return .active + case .high: + return .timeSensitive + case nil: + return nil + } + } + + static func trimmedOrNil(_ value: String?) -> String? { + let trimmed = value?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + return trimmed.isEmpty ? nil : trimmed + } +} diff --git a/apps/ios/Sources/Model/NodeAppModel.swift b/apps/ios/Sources/Model/NodeAppModel.swift index 1d09251dd76..ca9c3f9d0c3 100644 --- a/apps/ios/Sources/Model/NodeAppModel.swift +++ b/apps/ios/Sources/Model/NodeAppModel.swift @@ -3,6 +3,7 @@ import OpenClawKit import OpenClawProtocol import Observation import os +import Security import SwiftUI import UIKit import UserNotifications @@ -37,11 +38,27 @@ private final class NotificationInvokeLatch: @unchecked Sendable { cont?.resume(returning: response) } } + +private enum IOSDeepLinkAgentPolicy { + static let maxMessageChars = 20000 + static let maxUnkeyedConfirmChars = 240 +} + @MainActor @Observable +// swiftlint:disable type_body_length file_length final class NodeAppModel { + struct AgentDeepLinkPrompt: Identifiable, Equatable { + let id: String + let messagePreview: String + let urlPreview: String + let request: AgentDeepLink + } + private let deepLinkLogger = Logger(subsystem: "ai.openclaw.ios", category: "DeepLink") private let pushWakeLogger = Logger(subsystem: "ai.openclaw.ios", category: "PushWake") + private let locationWakeLogger = Logger(subsystem: "ai.openclaw.ios", category: "LocationWake") + private let watchReplyLogger = Logger(subsystem: "ai.openclaw.ios", category: "WatchReply") enum CameraHUDKind { case photo case recording @@ -72,6 +89,8 @@ final class NodeAppModel { var gatewayAgents: [AgentSummary] = [] var lastShareEventText: String = "No share events yet." var openChatRequestID: Int = 0 + private(set) var pendingAgentDeepLinkPrompt: AgentDeepLinkPrompt? + private var lastAgentDeepLinkPromptAt: Date = .distantPast // Primary "node" connection: used for device capabilities and node.invoke requests. private let nodeGateway = GatewayNodeSession() @@ -103,6 +122,13 @@ final class NodeAppModel { private var backgroundTalkKeptActive = false private var backgroundedAt: Date? private var reconnectAfterBackgroundArmed = false + private var backgroundGraceTaskID: UIBackgroundTaskIdentifier = .invalid + @ObservationIgnored private var backgroundGraceTaskTimer: Task? + private var backgroundReconnectSuppressed = false + private var backgroundReconnectLeaseUntil: Date? + private var lastSignificantLocationWakeAt: Date? + private var queuedWatchReplies: [WatchQuickReplyEvent] = [] + private var seenWatchReplyIds = Set() private var gatewayConnected = false private var operatorConnected = false @@ -149,6 +175,11 @@ final class NodeAppModel { self.talkMode = talkMode self.apnsDeviceTokenHex = UserDefaults.standard.string(forKey: Self.apnsDeviceTokenUserDefaultsKey) GatewayDiagnostics.bootstrap() + self.watchMessagingService.setReplyHandler { [weak self] event in + Task { @MainActor in + await self?.handleWatchQuickReply(event) + } + } self.voiceWake.configure { [weak self] cmd in guard let self else { return } @@ -271,6 +302,7 @@ final class NodeAppModel { self.stopGatewayHealthMonitor() self.backgroundedAt = Date() self.reconnectAfterBackgroundArmed = true + self.beginBackgroundConnectionGracePeriod() // Release voice wake mic in background. self.backgroundVoiceWakeSuspended = self.voiceWake.suspendForExternalAudioCapture() let shouldKeepTalkActive = keepTalkActive && self.talkMode.isEnabled @@ -278,6 +310,8 @@ final class NodeAppModel { self.backgroundTalkSuspended = self.talkMode.suspendForBackground(keepActive: shouldKeepTalkActive) case .active, .inactive: self.isBackgrounded = false + self.endBackgroundConnectionGracePeriod(reason: "scene_foreground") + self.clearBackgroundReconnectSuppression(reason: "scene_foreground") if self.operatorConnected { self.startGatewayHealthMonitor() } @@ -329,9 +363,102 @@ final class NodeAppModel { } @unknown default: self.isBackgrounded = false + self.endBackgroundConnectionGracePeriod(reason: "scene_unknown") + self.clearBackgroundReconnectSuppression(reason: "scene_unknown") } } + private func beginBackgroundConnectionGracePeriod(seconds: TimeInterval = 25) { + self.grantBackgroundReconnectLease(seconds: seconds, reason: "scene_background_grace") + self.endBackgroundConnectionGracePeriod(reason: "restart") + let taskID = UIApplication.shared.beginBackgroundTask(withName: "gateway-background-grace") { [weak self] in + Task { @MainActor in + self?.suppressBackgroundReconnect( + reason: "background_grace_expired", + disconnectIfNeeded: true) + self?.endBackgroundConnectionGracePeriod(reason: "expired") + } + } + guard taskID != .invalid else { + self.pushWakeLogger.info("Background grace unavailable: beginBackgroundTask returned invalid") + return + } + self.backgroundGraceTaskID = taskID + self.pushWakeLogger.info("Background grace started seconds=\(seconds, privacy: .public)") + self.backgroundGraceTaskTimer = Task { [weak self] in + guard let self else { return } + try? await Task.sleep(nanoseconds: UInt64(max(1, seconds) * 1_000_000_000)) + await MainActor.run { + self.suppressBackgroundReconnect(reason: "background_grace_timer", disconnectIfNeeded: true) + self.endBackgroundConnectionGracePeriod(reason: "timer") + } + } + } + + private func endBackgroundConnectionGracePeriod(reason: String) { + self.backgroundGraceTaskTimer?.cancel() + self.backgroundGraceTaskTimer = nil + guard self.backgroundGraceTaskID != .invalid else { return } + UIApplication.shared.endBackgroundTask(self.backgroundGraceTaskID) + self.backgroundGraceTaskID = .invalid + self.pushWakeLogger.info("Background grace ended reason=\(reason, privacy: .public)") + } + + private func grantBackgroundReconnectLease(seconds: TimeInterval, reason: String) { + guard self.isBackgrounded else { return } + let leaseSeconds = max(5, seconds) + let leaseUntil = Date().addingTimeInterval(leaseSeconds) + if let existing = self.backgroundReconnectLeaseUntil, existing > leaseUntil { + // Keep the longer lease if one is already active. + } else { + self.backgroundReconnectLeaseUntil = leaseUntil + } + let wasSuppressed = self.backgroundReconnectSuppressed + self.backgroundReconnectSuppressed = false + let leaseLogMessage = + "Background reconnect lease reason=\(reason) " + + "seconds=\(leaseSeconds) wasSuppressed=\(wasSuppressed)" + self.pushWakeLogger.info("\(leaseLogMessage, privacy: .public)") + } + + private func suppressBackgroundReconnect(reason: String, disconnectIfNeeded: Bool) { + guard self.isBackgrounded else { return } + let hadLease = self.backgroundReconnectLeaseUntil != nil + let changed = hadLease || !self.backgroundReconnectSuppressed + self.backgroundReconnectLeaseUntil = nil + self.backgroundReconnectSuppressed = true + guard changed else { return } + let suppressLogMessage = + "Background reconnect suppressed reason=\(reason) " + + "disconnect=\(disconnectIfNeeded)" + self.pushWakeLogger.info("\(suppressLogMessage, privacy: .public)") + guard disconnectIfNeeded else { return } + Task { [weak self] in + guard let self else { return } + await self.operatorGateway.disconnect() + await self.nodeGateway.disconnect() + await MainActor.run { + self.operatorConnected = false + self.gatewayConnected = false + self.talkMode.updateGatewayConnected(false) + if self.isBackgrounded { + self.gatewayStatusText = "Background idle" + self.gatewayServerName = nil + self.gatewayRemoteAddress = nil + self.showLocalCanvasOnDisconnect() + } + } + } + } + + private func clearBackgroundReconnectSuppression(reason: String) { + let changed = self.backgroundReconnectSuppressed || self.backgroundReconnectLeaseUntil != nil + self.backgroundReconnectSuppressed = false + self.backgroundReconnectLeaseUntil = nil + guard changed else { return } + self.pushWakeLogger.info("Background reconnect cleared reason=\(reason, privacy: .public)") + } + func setVoiceWakeEnabled(_ enabled: Bool) { self.voiceWake.setEnabled(enabled) if enabled { @@ -379,21 +506,14 @@ final class NodeAppModel { } } - private func applyMainSessionKey(_ key: String?) { - let trimmed = (key ?? "").trimmingCharacters(in: .whitespacesAndNewlines) - guard !trimmed.isEmpty else { return } - let current = self.mainSessionBaseKey.trimmingCharacters(in: .whitespacesAndNewlines) - if trimmed == current { return } - self.mainSessionBaseKey = trimmed - self.talkMode.updateMainSessionKey(self.mainSessionKey) - } - var seamColor: Color { Self.color(fromHex: self.seamColorHex) ?? Self.defaultSeamColor } private static let defaultSeamColor = Color(red: 79 / 255.0, green: 122 / 255.0, blue: 154 / 255.0) private static let apnsDeviceTokenUserDefaultsKey = "push.apns.deviceTokenHex" + private static let deepLinkKeyUserDefaultsKey = "deeplink.agent.key" + private static let canvasUnattendedDeepLinkKey: String = NodeAppModel.generateDeepLinkKey() private static var apnsEnvironment: String { #if DEBUG "sandbox" @@ -402,17 +522,6 @@ final class NodeAppModel { #endif } - private static func color(fromHex raw: String?) -> Color? { - let trimmed = (raw ?? "").trimmingCharacters(in: .whitespacesAndNewlines) - guard !trimmed.isEmpty else { return nil } - let hex = trimmed.hasPrefix("#") ? String(trimmed.dropFirst()) : trimmed - guard hex.count == 6, let value = Int(hex, radix: 16) else { return nil } - let r = Double((value >> 16) & 0xFF) / 255.0 - let g = Double((value >> 8) & 0xFF) / 255.0 - let b = Double(value & 0xFF) / 255.0 - return Color(red: r, green: g, blue: b) - } - private func refreshBrandingFromGateway() async { do { let res = try await self.operatorGateway.request(method: "config.get", paramsJSON: "{}", timeoutSeconds: 8) @@ -503,7 +612,7 @@ final class NodeAppModel { self.voiceWakeSyncTask = Task { [weak self] in guard let self else { return } - if !(await self.isGatewayHealthMonitorDisabled()) { + if !self.isGatewayHealthMonitorDisabled() { await self.refreshWakeWordsFromGateway() } @@ -558,9 +667,13 @@ final class NodeAppModel { self.gatewayHealthMonitor.start( check: { [weak self] in guard let self else { return false } - if await self.isGatewayHealthMonitorDisabled() { return true } + if await MainActor.run(body: { self.isGatewayHealthMonitorDisabled() }) { return true } do { - let data = try await self.operatorGateway.request(method: "health", paramsJSON: nil, timeoutSeconds: 6) + let data = try await self.operatorGateway.request( + method: "health", + paramsJSON: nil, + timeoutSeconds: 6 + ) guard let decoded = try? JSONDecoder().decode(OpenClawGatewayHealthOK.self, from: data) else { return false } @@ -568,7 +681,7 @@ final class NodeAppModel { } catch { if let gatewayError = error as? GatewayResponseError { let lower = gatewayError.message.lowercased() - if lower.contains("unauthorized role") { + if lower.contains("unauthorized role") || lower.contains("missing scope") { await self.setGatewayHealthMonitorDisabled(true) return true } @@ -593,117 +706,6 @@ final class NodeAppModel { self.gatewayHealthMonitor.stop() } - private func refreshWakeWordsFromGateway() async { - do { - let data = try await self.operatorGateway.request(method: "voicewake.get", paramsJSON: "{}", timeoutSeconds: 8) - guard let triggers = VoiceWakePreferences.decodeGatewayTriggers(from: data) else { return } - VoiceWakePreferences.saveTriggerWords(triggers) - } catch { - if let gatewayError = error as? GatewayResponseError { - let lower = gatewayError.message.lowercased() - if lower.contains("unauthorized role") { - await self.setGatewayHealthMonitorDisabled(true) - return - } - } - // Best-effort only. - } - } - - private func isGatewayHealthMonitorDisabled() -> Bool { - self.gatewayHealthMonitorDisabled - } - - private func setGatewayHealthMonitorDisabled(_ disabled: Bool) { - self.gatewayHealthMonitorDisabled = disabled - } - - func sendVoiceTranscript(text: String, sessionKey: String?) async throws { - if await !self.isGatewayConnected() { - throw NSError(domain: "Gateway", code: 10, userInfo: [ - NSLocalizedDescriptionKey: "Gateway not connected", - ]) - } - struct Payload: Codable { - var text: String - var sessionKey: String? - } - let payload = Payload(text: text, sessionKey: sessionKey) - let data = try JSONEncoder().encode(payload) - guard let json = String(bytes: data, encoding: .utf8) else { - throw NSError(domain: "NodeAppModel", code: 1, userInfo: [ - NSLocalizedDescriptionKey: "Failed to encode voice transcript payload as UTF-8", - ]) - } - await self.nodeGateway.sendEvent(event: "voice.transcript", payloadJSON: json) - } - - func handleDeepLink(url: URL) async { - guard let route = DeepLinkParser.parse(url) else { return } - - switch route { - case let .agent(link): - await self.handleAgentDeepLink(link, originalURL: url) - case .gateway: - break - } - } - - private func handleAgentDeepLink(_ link: AgentDeepLink, originalURL: URL) async { - let message = link.message.trimmingCharacters(in: .whitespacesAndNewlines) - guard !message.isEmpty else { return } - self.deepLinkLogger.info( - "agent deep link received messageChars=\(message.count) url=\(originalURL.absoluteString, privacy: .public)" - ) - - if message.count > 20000 { - self.screen.errorText = "Deep link too large (message exceeds 20,000 characters)." - self.recordShareEvent("Rejected: message too large (\(message.count) chars).") - return - } - - guard await self.isGatewayConnected() else { - self.screen.errorText = "Gateway not connected (cannot forward deep link)." - self.recordShareEvent("Failed: gateway not connected.") - self.deepLinkLogger.error("agent deep link rejected: gateway not connected") - return - } - - do { - try await self.sendAgentRequest(link: link) - self.screen.errorText = nil - self.recordShareEvent("Sent to gateway (\(message.count) chars).") - self.deepLinkLogger.info("agent deep link forwarded to gateway") - self.openChatRequestID &+= 1 - } catch { - self.screen.errorText = "Agent request failed: \(error.localizedDescription)" - self.recordShareEvent("Failed: \(error.localizedDescription)") - self.deepLinkLogger.error("agent deep link send failed: \(error.localizedDescription, privacy: .public)") - } - } - - private func sendAgentRequest(link: AgentDeepLink) async throws { - if link.message.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty { - throw NSError(domain: "DeepLink", code: 1, userInfo: [ - NSLocalizedDescriptionKey: "invalid agent message", - ]) - } - - // iOS gateway forwards to the gateway; no local auth prompts here. - // (Key-based unattended auth is handled on macOS for openclaw:// links.) - let data = try JSONEncoder().encode(link) - guard let json = String(bytes: data, encoding: .utf8) else { - throw NSError(domain: "NodeAppModel", code: 2, userInfo: [ - NSLocalizedDescriptionKey: "Failed to encode agent request payload as UTF-8", - ]) - } - await self.nodeGateway.sendEvent(event: "agent.request", payloadJSON: json) - } - - private func isGatewayConnected() async -> Bool { - self.gatewayConnected - } - private func handleInvoke(_ req: BridgeInvokeRequest) async -> BridgeInvokeResponse { let command = req.command @@ -1497,8 +1499,9 @@ private extension NodeAppModel { return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: json) case OpenClawWatchCommand.notify.rawValue: let params = try Self.decodeParams(OpenClawWatchNotifyParams.self, from: req.paramsJSON) - let title = params.title.trimmingCharacters(in: .whitespacesAndNewlines) - let body = params.body.trimmingCharacters(in: .whitespacesAndNewlines) + let normalizedParams = Self.normalizeWatchNotifyParams(params) + let title = normalizedParams.title + let body = normalizedParams.body if title.isEmpty && body.isEmpty { return BridgeInvokeResponse( id: req.id, @@ -1510,9 +1513,16 @@ private extension NodeAppModel { do { let result = try await self.watchMessagingService.sendNotification( id: req.id, - title: title, - body: body, - priority: params.priority) + params: normalizedParams) + if result.queuedForDelivery || !result.deliveredImmediately { + let invokeID = req.id + Task { @MainActor in + await WatchPromptNotificationBridge.scheduleMirroredWatchPromptNotificationIfNeeded( + invokeID: invokeID, + params: normalizedParams, + sendResult: result) + } + } let payload = OpenClawWatchNotifyPayload( deliveredImmediately: result.deliveredImmediately, queuedForDelivery: result.queuedForDelivery, @@ -1603,6 +1613,14 @@ extension NodeAppModel { return SessionKey.makeAgentSessionKey(agentId: agentId, baseKey: base) } + var chatSessionKey: String { + let base = "ios" + let agentId = (self.selectedAgentId ?? "").trimmingCharacters(in: .whitespacesAndNewlines) + let defaultId = (self.gatewayDefaultAgentId ?? "").trimmingCharacters(in: .whitespacesAndNewlines) + if agentId.isEmpty || (!defaultId.isEmpty && agentId == defaultId) { return base } + return SessionKey.makeAgentSessionKey(agentId: agentId, baseKey: base) + } + var activeAgentName: String { let agentId = (self.selectedAgentId ?? "").trimmingCharacters(in: .whitespacesAndNewlines) let defaultId = (self.gatewayDefaultAgentId ?? "").trimmingCharacters(in: .whitespacesAndNewlines) @@ -1717,6 +1735,23 @@ private extension NodeAppModel { self.apnsLastRegisteredTokenHex = nil } + func refreshBackgroundReconnectSuppressionIfNeeded(source: String) { + guard self.isBackgrounded else { return } + guard !self.backgroundReconnectSuppressed else { return } + guard let leaseUntil = self.backgroundReconnectLeaseUntil else { + self.suppressBackgroundReconnect(reason: "\(source):no_lease", disconnectIfNeeded: true) + return + } + if Date() >= leaseUntil { + self.suppressBackgroundReconnect(reason: "\(source):lease_expired", disconnectIfNeeded: true) + } + } + + func shouldPauseReconnectLoopInBackground(source: String) -> Bool { + self.refreshBackgroundReconnectSuppressionIfNeeded(source: source) + return self.isBackgrounded && self.backgroundReconnectSuppressed + } + func startOperatorGatewayLoop( url: URL, stableID: String, @@ -1739,6 +1774,10 @@ private extension NodeAppModel { try? await Task.sleep(nanoseconds: 1_000_000_000) continue } + if self.shouldPauseReconnectLoopInBackground(source: "operator_loop") { + try? await Task.sleep(nanoseconds: 2_000_000_000) + continue + } if await self.isOperatorConnected() { try? await Task.sleep(nanoseconds: 1_000_000_000) continue @@ -1765,6 +1804,7 @@ private extension NodeAppModel { } GatewayDiagnostics.log( "operator gateway connected host=\(url.host ?? "?") scheme=\(url.scheme ?? "?")") + await self.talkMode.reloadConfig() await self.refreshBrandingFromGateway() await self.refreshAgentsFromGateway() await self.refreshShareRouteFromGateway() @@ -1802,6 +1842,8 @@ private extension NodeAppModel { } } + // Legacy reconnect state machine; follow-up refactor needed to split into helpers. + // swiftlint:disable:next function_body_length func startNodeGatewayLoop( url: URL, stableID: String, @@ -1826,6 +1868,10 @@ private extension NodeAppModel { try? await Task.sleep(nanoseconds: 1_000_000_000) continue } + if self.shouldPauseReconnectLoopInBackground(source: "node_loop") { + try? await Task.sleep(nanoseconds: 2_000_000_000) + continue + } if await self.isGatewayConnected() { try? await Task.sleep(nanoseconds: 1_000_000_000) continue @@ -1869,13 +1915,24 @@ private extension NodeAppModel { sessionKey: relayData.sessionKey, deliveryChannel: relayData.deliveryChannel, deliveryTo: relayData.deliveryTo)) - GatewayDiagnostics.log("gateway connected host=\(url.host ?? "?") scheme=\(url.scheme ?? "?")") + GatewayDiagnostics.log( + "gateway connected host=\(url.host ?? "?") " + + "scheme=\(url.scheme ?? "?")" + ) if let addr = await self.nodeGateway.currentRemoteAddress() { await MainActor.run { self.gatewayRemoteAddress = addr } } await self.showA2UIOnConnectIfNeeded() await self.onNodeGatewayConnected() - await MainActor.run { SignificantLocationMonitor.startIfNeeded(locationService: self.locationService, locationMode: self.locationMode(), gateway: self.nodeGateway) } + await MainActor.run { + SignificantLocationMonitor.startIfNeeded( + locationService: self.locationService, + locationMode: self.locationMode(), + gateway: self.nodeGateway, + beforeSend: { [weak self] in + await self?.handleSignificantLocationWakeIfNeeded() + }) + } }, onDisconnected: { [weak self] reason in guard let self else { return } @@ -1956,9 +2013,11 @@ private extension NodeAppModel { self.gatewayPairingRequestId = requestId if let requestId, !requestId.isEmpty { self.gatewayStatusText = - "Pairing required (requestId: \(requestId)). Approve on gateway and return to OpenClaw." + "Pairing required (requestId: \(requestId)). " + + "Approve on gateway and return to OpenClaw." } else { - self.gatewayStatusText = "Pairing required. Approve on gateway and return to OpenClaw." + self.gatewayStatusText = + "Pairing required. Approve on gateway and return to OpenClaw." } } // Hard stop the underlying WebSocket watchdog reconnects so the UI stays stable and @@ -2007,9 +2066,7 @@ private extension NodeAppModel { clientId: clientId, clientMode: "ui", clientDisplayName: displayName, - // Operator traffic should authenticate via shared gateway auth only. - // Including device identity here can trigger duplicate pairing flows. - includeDeviceIdentity: false) + includeDeviceIdentity: true) } func legacyClientIdFallback(currentClientId: String, error: Error) -> String? { @@ -2124,15 +2181,175 @@ extension NodeAppModel { /// Back-compat hook retained for older gateway-connect flows. func onNodeGatewayConnected() async { await self.registerAPNsTokenIfNeeded() + await self.flushQueuedWatchRepliesIfConnected() + } + + private func handleWatchQuickReply(_ event: WatchQuickReplyEvent) async { + let replyId = event.replyId.trimmingCharacters(in: .whitespacesAndNewlines) + let actionId = event.actionId.trimmingCharacters(in: .whitespacesAndNewlines) + if replyId.isEmpty || actionId.isEmpty { + self.watchReplyLogger.info("watch reply dropped: missing replyId/actionId") + return + } + + if self.seenWatchReplyIds.contains(replyId) { + self.watchReplyLogger.debug( + "watch reply deduped replyId=\(replyId, privacy: .public)") + return + } + self.seenWatchReplyIds.insert(replyId) + + if await !self.isGatewayConnected() { + self.queuedWatchReplies.append(event) + self.watchReplyLogger.info( + "watch reply queued replyId=\(replyId, privacy: .public) action=\(actionId, privacy: .public)") + return + } + + await self.forwardWatchReplyToAgent(event) + } + + private func flushQueuedWatchRepliesIfConnected() async { + guard await self.isGatewayConnected() else { return } + guard !self.queuedWatchReplies.isEmpty else { return } + + let pending = self.queuedWatchReplies + self.queuedWatchReplies.removeAll() + for event in pending { + await self.forwardWatchReplyToAgent(event) + } + } + + private func forwardWatchReplyToAgent(_ event: WatchQuickReplyEvent) async { + let sessionKey = event.sessionKey?.trimmingCharacters(in: .whitespacesAndNewlines) + let effectiveSessionKey = (sessionKey?.isEmpty == false) ? sessionKey : self.mainSessionKey + let message = Self.makeWatchReplyAgentMessage(event) + let link = AgentDeepLink( + message: message, + sessionKey: effectiveSessionKey, + thinking: "low", + deliver: false, + to: nil, + channel: nil, + timeoutSeconds: nil, + key: event.replyId) + do { + try await self.sendAgentRequest(link: link) + let forwardedMessage = + "watch reply forwarded replyId=\(event.replyId) " + + "action=\(event.actionId)" + self.watchReplyLogger.info("\(forwardedMessage, privacy: .public)") + self.openChatRequestID &+= 1 + } catch { + let failedMessage = + "watch reply forwarding failed replyId=\(event.replyId) " + + "error=\(error.localizedDescription)" + self.watchReplyLogger.error("\(failedMessage, privacy: .public)") + self.queuedWatchReplies.insert(event, at: 0) + } + } + + private static func makeWatchReplyAgentMessage(_ event: WatchQuickReplyEvent) -> String { + let actionLabel = event.actionLabel?.trimmingCharacters(in: .whitespacesAndNewlines) + let promptId = event.promptId.trimmingCharacters(in: .whitespacesAndNewlines) + let transport = event.transport.trimmingCharacters(in: .whitespacesAndNewlines) + let summary = actionLabel?.isEmpty == false ? actionLabel! : event.actionId + var lines: [String] = [] + lines.append("Watch reply: \(summary)") + lines.append("promptId=\(promptId.isEmpty ? "unknown" : promptId)") + lines.append("actionId=\(event.actionId)") + lines.append("replyId=\(event.replyId)") + if !transport.isEmpty { + lines.append("transport=\(transport)") + } + if let sentAtMs = event.sentAtMs { + lines.append("sentAtMs=\(sentAtMs)") + } + if let note = event.note?.trimmingCharacters(in: .whitespacesAndNewlines), !note.isEmpty { + lines.append("note=\(note)") + } + return lines.joined(separator: "\n") } func handleSilentPushWake(_ userInfo: [AnyHashable: Any]) async -> Bool { + let wakeId = Self.makePushWakeAttemptID() guard Self.isSilentPushPayload(userInfo) else { - self.pushWakeLogger.info("Ignored APNs payload: not silent push") + self.pushWakeLogger.info("Ignored APNs payload wakeId=\(wakeId, privacy: .public): not silent push") return false } - self.pushWakeLogger.info("Silent push received; attempting reconnect if needed") - return await self.reconnectGatewaySessionsForSilentPushIfNeeded() + let pushKind = Self.openclawPushKind(userInfo) + let receivedMessage = + "Silent push received wakeId=\(wakeId) " + + "kind=\(pushKind) " + + "backgrounded=\(self.isBackgrounded) " + + "autoReconnect=\(self.gatewayAutoReconnectEnabled)" + self.pushWakeLogger.info("\(receivedMessage, privacy: .public)") + let result = await self.reconnectGatewaySessionsForSilentPushIfNeeded(wakeId: wakeId) + let outcomeMessage = + "Silent push outcome wakeId=\(wakeId) " + + "applied=\(result.applied) " + + "reason=\(result.reason) " + + "durationMs=\(result.durationMs)" + self.pushWakeLogger.info("\(outcomeMessage, privacy: .public)") + return result.applied + } + + func handleBackgroundRefreshWake(trigger: String = "bg_app_refresh") async -> Bool { + let wakeId = Self.makePushWakeAttemptID() + let receivedMessage = + "Background refresh wake received wakeId=\(wakeId) " + + "trigger=\(trigger) " + + "backgrounded=\(self.isBackgrounded) " + + "autoReconnect=\(self.gatewayAutoReconnectEnabled)" + self.pushWakeLogger.info("\(receivedMessage, privacy: .public)") + let result = await self.reconnectGatewaySessionsForSilentPushIfNeeded(wakeId: wakeId) + let outcomeMessage = + "Background refresh wake outcome wakeId=\(wakeId) " + + "applied=\(result.applied) " + + "reason=\(result.reason) " + + "durationMs=\(result.durationMs)" + self.pushWakeLogger.info("\(outcomeMessage, privacy: .public)") + return result.applied + } + + func handleSignificantLocationWakeIfNeeded() async { + let wakeId = Self.makePushWakeAttemptID() + let now = Date() + let throttleWindowSeconds: TimeInterval = 180 + + if await self.isGatewayConnected() { + self.locationWakeLogger.info( + "Location wake no-op wakeId=\(wakeId, privacy: .public): already connected") + return + } + if let last = self.lastSignificantLocationWakeAt, + now.timeIntervalSince(last) < throttleWindowSeconds + { + let throttledMessage = + "Location wake throttled wakeId=\(wakeId) " + + "elapsedSec=\(now.timeIntervalSince(last))" + self.locationWakeLogger.info("\(throttledMessage, privacy: .public)") + return + } + self.lastSignificantLocationWakeAt = now + + let beginMessage = + "Location wake begin wakeId=\(wakeId) " + + "backgrounded=\(self.isBackgrounded) " + + "autoReconnect=\(self.gatewayAutoReconnectEnabled)" + self.locationWakeLogger.info("\(beginMessage, privacy: .public)") + let result = await self.reconnectGatewaySessionsForSilentPushIfNeeded(wakeId: wakeId) + let triggerMessage = + "Location wake trigger wakeId=\(wakeId) " + + "applied=\(result.applied) " + + "reason=\(result.reason) " + + "durationMs=\(result.durationMs)" + self.locationWakeLogger.info("\(triggerMessage, privacy: .public)") + + guard result.applied else { return } + let connected = await self.waitForGatewayConnection(timeoutMs: 5000, pollMs: 250) + self.locationWakeLogger.info( + "Location wake post-check wakeId=\(wakeId, privacy: .public) connected=\(connected, privacy: .public)") } func updateAPNsDeviceToken(_ tokenData: Data) { @@ -2202,28 +2419,317 @@ extension NodeAppModel { return false } - private func reconnectGatewaySessionsForSilentPushIfNeeded() async -> Bool { - guard self.isBackgrounded else { - self.pushWakeLogger.info("Wake no-op: app not backgrounded") - return false + private static func makePushWakeAttemptID() -> String { + let raw = UUID().uuidString.replacingOccurrences(of: "-", with: "") + return String(raw.prefix(8)) + } + + private static func openclawPushKind(_ userInfo: [AnyHashable: Any]) -> String { + if let payload = userInfo["openclaw"] as? [String: Any], + let kind = payload["kind"] as? String + { + let trimmed = kind.trimmingCharacters(in: .whitespacesAndNewlines) + if !trimmed.isEmpty { return trimmed } } - guard self.gatewayAutoReconnectEnabled else { - self.pushWakeLogger.info("Wake no-op: auto reconnect disabled") - return false + if let payload = userInfo["openclaw"] as? [AnyHashable: Any], + let kind = payload["kind"] as? String + { + let trimmed = kind.trimmingCharacters(in: .whitespacesAndNewlines) + if !trimmed.isEmpty { return trimmed } } - guard self.activeGatewayConnectConfig != nil else { - self.pushWakeLogger.info("Wake no-op: no active gateway config") - return false + return "unknown" + } + + private struct SilentPushWakeAttemptResult { + var applied: Bool + var reason: String + var durationMs: Int + } + + private func waitForGatewayConnection(timeoutMs: Int, pollMs: Int) async -> Bool { + let clampedTimeoutMs = max(0, timeoutMs) + let pollIntervalNs = UInt64(max(50, pollMs)) * 1_000_000 + let deadline = Date().addingTimeInterval(Double(clampedTimeoutMs) / 1000.0) + while Date() < deadline { + if await self.isGatewayConnected() { + return true + } + try? await Task.sleep(nanoseconds: pollIntervalNs) + } + return await self.isGatewayConnected() + } + + private func reconnectGatewaySessionsForSilentPushIfNeeded( + wakeId: String + ) async -> SilentPushWakeAttemptResult { + let startedAt = Date() + let makeResult: (Bool, String) -> SilentPushWakeAttemptResult = { applied, reason in + let durationMs = Int(Date().timeIntervalSince(startedAt) * 1000) + return SilentPushWakeAttemptResult( + applied: applied, + reason: reason, + durationMs: max(0, durationMs)) } + guard self.isBackgrounded else { + self.pushWakeLogger.info("Wake no-op wakeId=\(wakeId, privacy: .public): app not backgrounded") + return makeResult(false, "not_backgrounded") + } + guard self.gatewayAutoReconnectEnabled else { + self.pushWakeLogger.info("Wake no-op wakeId=\(wakeId, privacy: .public): auto reconnect disabled") + return makeResult(false, "auto_reconnect_disabled") + } + guard let cfg = self.activeGatewayConnectConfig else { + self.pushWakeLogger.info("Wake no-op wakeId=\(wakeId, privacy: .public): no active gateway config") + return makeResult(false, "no_active_gateway_config") + } + + self.pushWakeLogger.info( + "Wake reconnect begin wakeId=\(wakeId, privacy: .public) stableID=\(cfg.stableID, privacy: .public)") + self.grantBackgroundReconnectLease(seconds: 30, reason: "wake_\(wakeId)") await self.operatorGateway.disconnect() await self.nodeGateway.disconnect() self.operatorConnected = false self.gatewayConnected = false self.gatewayStatusText = "Reconnecting…" self.talkMode.updateGatewayConnected(false) - self.pushWakeLogger.info("Wake reconnect trigger applied") - return true + self.applyGatewayConnectConfig(cfg) + self.pushWakeLogger.info("Wake reconnect trigger applied wakeId=\(wakeId, privacy: .public)") + return makeResult(true, "reconnect_triggered") + } +} + +extension NodeAppModel { + private func refreshWakeWordsFromGateway() async { + do { + let data = try await self.operatorGateway.request( + method: "voicewake.get", + paramsJSON: "{}", + timeoutSeconds: 8 + ) + guard let triggers = VoiceWakePreferences.decodeGatewayTriggers(from: data) else { return } + VoiceWakePreferences.saveTriggerWords(triggers) + } catch { + if let gatewayError = error as? GatewayResponseError { + let lower = gatewayError.message.lowercased() + if lower.contains("unauthorized role") || lower.contains("missing scope") { + self.setGatewayHealthMonitorDisabled(true) + return + } + } + // Best-effort only. + } + } + + private func isGatewayHealthMonitorDisabled() -> Bool { + self.gatewayHealthMonitorDisabled + } + + private func setGatewayHealthMonitorDisabled(_ disabled: Bool) { + self.gatewayHealthMonitorDisabled = disabled + } + + func sendVoiceTranscript(text: String, sessionKey: String?) async throws { + if await !self.isGatewayConnected() { + throw NSError(domain: "Gateway", code: 10, userInfo: [ + NSLocalizedDescriptionKey: "Gateway not connected", + ]) + } + struct Payload: Codable { + var text: String + var sessionKey: String? + } + let payload = Payload(text: text, sessionKey: sessionKey) + let data = try JSONEncoder().encode(payload) + guard let json = String(bytes: data, encoding: .utf8) else { + throw NSError(domain: "NodeAppModel", code: 1, userInfo: [ + NSLocalizedDescriptionKey: "Failed to encode voice transcript payload as UTF-8", + ]) + } + await self.nodeGateway.sendEvent(event: "voice.transcript", payloadJSON: json) + } + + func handleDeepLink(url: URL) async { + guard let route = DeepLinkParser.parse(url) else { return } + + switch route { + case let .agent(link): + await self.handleAgentDeepLink(link, originalURL: url) + case .gateway: + break + } + } + + private func handleAgentDeepLink(_ link: AgentDeepLink, originalURL: URL) async { + let message = link.message.trimmingCharacters(in: .whitespacesAndNewlines) + guard !message.isEmpty else { return } + self.deepLinkLogger.info( + "agent deep link received messageChars=\(message.count) url=\(originalURL.absoluteString, privacy: .public)" + ) + + if message.count > IOSDeepLinkAgentPolicy.maxMessageChars { + self.screen.errorText = "Deep link too large (message exceeds " + + "\(IOSDeepLinkAgentPolicy.maxMessageChars) characters)." + self.recordShareEvent("Rejected: message too large (\(message.count) chars).") + return + } + + guard await self.isGatewayConnected() else { + self.screen.errorText = "Gateway not connected (cannot forward deep link)." + self.recordShareEvent("Failed: gateway not connected.") + self.deepLinkLogger.error("agent deep link rejected: gateway not connected") + return + } + + let allowUnattended = self.isUnattendedDeepLinkAllowed(link.key) + if !allowUnattended { + if message.count > IOSDeepLinkAgentPolicy.maxUnkeyedConfirmChars { + self.screen.errorText = "Deep link blocked (message too long without key)." + self.recordShareEvent( + "Rejected: deep link over \(IOSDeepLinkAgentPolicy.maxUnkeyedConfirmChars) chars without key.") + self.deepLinkLogger.error( + "agent deep link rejected: unkeyed message too long chars=\(message.count, privacy: .public)") + return + } + if Date().timeIntervalSince(self.lastAgentDeepLinkPromptAt) < 1.0 { + self.deepLinkLogger.debug("agent deep link prompt throttled") + return + } + self.lastAgentDeepLinkPromptAt = Date() + + let urlText = originalURL.absoluteString + let prompt = AgentDeepLinkPrompt( + id: UUID().uuidString, + messagePreview: message, + urlPreview: urlText.count > 500 ? "\(urlText.prefix(500))…" : urlText, + request: self.effectiveAgentDeepLinkForPrompt(link)) + self.pendingAgentDeepLinkPrompt = prompt + self.recordShareEvent("Awaiting local confirmation (\(message.count) chars).") + self.deepLinkLogger.info("agent deep link requires local confirmation") + return + } + + await self.submitAgentDeepLink(link, messageCharCount: message.count) + } + + private func sendAgentRequest(link: AgentDeepLink) async throws { + if link.message.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty { + throw NSError(domain: "DeepLink", code: 1, userInfo: [ + NSLocalizedDescriptionKey: "invalid agent message", + ]) + } + + let data = try JSONEncoder().encode(link) + guard let json = String(bytes: data, encoding: .utf8) else { + throw NSError(domain: "NodeAppModel", code: 2, userInfo: [ + NSLocalizedDescriptionKey: "Failed to encode agent request payload as UTF-8", + ]) + } + await self.nodeGateway.sendEvent(event: "agent.request", payloadJSON: json) + } + + private func isGatewayConnected() async -> Bool { + self.gatewayConnected + } + + private func applyMainSessionKey(_ key: String?) { + let trimmed = (key ?? "").trimmingCharacters(in: .whitespacesAndNewlines) + guard !trimmed.isEmpty else { return } + let current = self.mainSessionBaseKey.trimmingCharacters(in: .whitespacesAndNewlines) + if trimmed == current { return } + self.mainSessionBaseKey = trimmed + self.talkMode.updateMainSessionKey(self.mainSessionKey) + } + + private static func color(fromHex raw: String?) -> Color? { + let trimmed = (raw ?? "").trimmingCharacters(in: .whitespacesAndNewlines) + guard !trimmed.isEmpty else { return nil } + let hex = trimmed.hasPrefix("#") ? String(trimmed.dropFirst()) : trimmed + guard hex.count == 6, let value = Int(hex, radix: 16) else { return nil } + let r = Double((value >> 16) & 0xFF) / 255.0 + let g = Double((value >> 8) & 0xFF) / 255.0 + let b = Double(value & 0xFF) / 255.0 + return Color(red: r, green: g, blue: b) + } + + func approvePendingAgentDeepLinkPrompt() async { + guard let prompt = self.pendingAgentDeepLinkPrompt else { return } + self.pendingAgentDeepLinkPrompt = nil + guard await self.isGatewayConnected() else { + self.screen.errorText = "Gateway not connected (cannot forward deep link)." + self.recordShareEvent("Failed: gateway not connected.") + self.deepLinkLogger.error("agent deep link approval failed: gateway not connected") + return + } + await self.submitAgentDeepLink(prompt.request, messageCharCount: prompt.messagePreview.count) + } + + func declinePendingAgentDeepLinkPrompt() { + guard self.pendingAgentDeepLinkPrompt != nil else { return } + self.pendingAgentDeepLinkPrompt = nil + self.screen.errorText = "Deep link cancelled." + self.recordShareEvent("Cancelled: deep link confirmation declined.") + self.deepLinkLogger.info("agent deep link cancelled by local user") + } + + private func submitAgentDeepLink(_ link: AgentDeepLink, messageCharCount: Int) async { + do { + try await self.sendAgentRequest(link: link) + self.screen.errorText = nil + self.recordShareEvent("Sent to gateway (\(messageCharCount) chars).") + self.deepLinkLogger.info("agent deep link forwarded to gateway") + self.openChatRequestID &+= 1 + } catch { + self.screen.errorText = "Agent request failed: \(error.localizedDescription)" + self.recordShareEvent("Failed: \(error.localizedDescription)") + self.deepLinkLogger.error("agent deep link send failed: \(error.localizedDescription, privacy: .public)") + } + } + + private func effectiveAgentDeepLinkForPrompt(_ link: AgentDeepLink) -> AgentDeepLink { + // Without a trusted key, strip delivery/routing knobs to reduce exfiltration risk. + AgentDeepLink( + message: link.message, + sessionKey: link.sessionKey, + thinking: link.thinking, + deliver: false, + to: nil, + channel: nil, + timeoutSeconds: link.timeoutSeconds, + key: link.key) + } + + private func isUnattendedDeepLinkAllowed(_ key: String?) -> Bool { + let normalizedKey = key?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + guard !normalizedKey.isEmpty else { return false } + return normalizedKey == Self.canvasUnattendedDeepLinkKey || normalizedKey == Self.expectedDeepLinkKey() + } + + private static func expectedDeepLinkKey() -> String { + let defaults = UserDefaults.standard + if let key = defaults.string(forKey: self.deepLinkKeyUserDefaultsKey), !key.isEmpty { + return key + } + let key = self.generateDeepLinkKey() + defaults.set(key, forKey: self.deepLinkKeyUserDefaultsKey) + return key + } + + private static func generateDeepLinkKey() -> String { + var bytes = [UInt8](repeating: 0, count: 32) + _ = SecRandomCopyBytes(kSecRandomDefault, bytes.count, &bytes) + let data = Data(bytes) + return data + .base64EncodedString() + .replacingOccurrences(of: "+", with: "-") + .replacingOccurrences(of: "/", with: "_") + .replacingOccurrences(of: "=", with: "") + } +} + +extension NodeAppModel { + func _bridgeConsumeMirroredWatchReply(_ event: WatchQuickReplyEvent) async { + await self.handleWatchQuickReply(event) } } @@ -2264,5 +2770,18 @@ extension NodeAppModel { func _test_applyTalkModeSync(enabled: Bool, phase: String? = nil) { self.applyTalkModeSync(enabled: enabled, phase: phase) } + + func _test_queuedWatchReplyCount() -> Int { + self.queuedWatchReplies.count + } + + func _test_setGatewayConnected(_ connected: Bool) { + self.gatewayConnected = connected + } + + static func _test_currentDeepLinkKey() -> String { + self.expectedDeepLinkKey() + } } #endif +// swiftlint:enable type_body_length file_length diff --git a/apps/ios/Sources/Motion/MotionService.swift b/apps/ios/Sources/Motion/MotionService.swift index f108e0b560b..e126b3bd20d 100644 --- a/apps/ios/Sources/Motion/MotionService.swift +++ b/apps/ios/Sources/Motion/MotionService.swift @@ -20,7 +20,7 @@ final class MotionService: MotionServicing { let limit = max(1, min(params.limit ?? 200, 1000)) let manager = CMMotionActivityManager() - let mapped = try await withCheckedThrowingContinuation { (cont: CheckedContinuation<[OpenClawMotionActivityEntry], Error>) in + let mapped: [OpenClawMotionActivityEntry] = try await withCheckedThrowingContinuation { cont in manager.queryActivityStarting(from: start, to: end, to: OperationQueue()) { activity, error in if let error { cont.resume(throwing: error) @@ -62,7 +62,7 @@ final class MotionService: MotionServicing { let (start, end) = Self.resolveRange(startISO: params.startISO, endISO: params.endISO) let pedometer = CMPedometer() - let payload = try await withCheckedThrowingContinuation { (cont: CheckedContinuation) in + let payload: OpenClawPedometerPayload = try await withCheckedThrowingContinuation { cont in pedometer.queryPedometerData(from: start, to: end) { data, error in if let error { cont.resume(throwing: error) diff --git a/apps/ios/Sources/Onboarding/OnboardingWizardView.swift b/apps/ios/Sources/Onboarding/OnboardingWizardView.swift index c0e872b2ceb..b0dbdc13639 100644 --- a/apps/ios/Sources/Onboarding/OnboardingWizardView.swift +++ b/apps/ios/Sources/Onboarding/OnboardingWizardView.swift @@ -134,7 +134,10 @@ struct OnboardingWizardView: View { Button("Done") { UIApplication.shared.sendAction( #selector(UIResponder.resignFirstResponder), - to: nil, from: nil, for: nil) + to: nil, + from: nil, + for: nil + ) } } } @@ -716,8 +719,10 @@ struct OnboardingWizardView: View { private func detectQRCode(from data: Data) -> String? { guard let ciImage = CIImage(data: data) else { return nil } let detector = CIDetector( - ofType: CIDetectorTypeQRCode, context: nil, - options: [CIDetectorAccuracy: CIDetectorAccuracyHigh]) + ofType: CIDetectorTypeQRCode, + context: nil, + options: [CIDetectorAccuracy: CIDetectorAccuracyHigh] + ) let features = detector?.features(in: ciImage) ?? [] for feature in features { if let qr = feature as? CIQRCodeFeature, let message = qr.messageString { diff --git a/apps/ios/Sources/OpenClawApp.swift b/apps/ios/Sources/OpenClawApp.swift index 091c1b90fdf..27f7f5e02ca 100644 --- a/apps/ios/Sources/OpenClawApp.swift +++ b/apps/ios/Sources/OpenClawApp.swift @@ -1,17 +1,48 @@ import SwiftUI import Foundation +import OpenClawKit import os import UIKit +import BackgroundTasks +@preconcurrency import UserNotifications -final class OpenClawAppDelegate: NSObject, UIApplicationDelegate { +private struct PendingWatchPromptAction { + var promptId: String? + var actionId: String + var actionLabel: String? + var sessionKey: String? +} + +@MainActor +final class OpenClawAppDelegate: NSObject, UIApplicationDelegate, @preconcurrency UNUserNotificationCenterDelegate { private let logger = Logger(subsystem: "ai.openclaw.ios", category: "Push") + private let backgroundWakeLogger = Logger(subsystem: "ai.openclaw.ios", category: "BackgroundWake") + private static let wakeRefreshTaskIdentifier = "ai.openclaw.ios.bgrefresh" + private var backgroundWakeTask: Task? private var pendingAPNsDeviceToken: Data? + private var pendingWatchPromptActions: [PendingWatchPromptAction] = [] + weak var appModel: NodeAppModel? { didSet { - guard let model = self.appModel, let token = self.pendingAPNsDeviceToken else { return } - self.pendingAPNsDeviceToken = nil - Task { @MainActor in - model.updateAPNsDeviceToken(token) + guard let model = self.appModel else { return } + if let token = self.pendingAPNsDeviceToken { + self.pendingAPNsDeviceToken = nil + Task { @MainActor in + model.updateAPNsDeviceToken(token) + } + } + if !self.pendingWatchPromptActions.isEmpty { + let pending = self.pendingWatchPromptActions + self.pendingWatchPromptActions.removeAll() + Task { @MainActor in + for action in pending { + await model.handleMirroredWatchPromptAction( + promptId: action.promptId, + actionId: action.actionId, + actionLabel: action.actionLabel, + sessionKey: action.sessionKey) + } + } } } } @@ -21,6 +52,8 @@ final class OpenClawAppDelegate: NSObject, UIApplicationDelegate { didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]? = nil ) -> Bool { + self.registerBackgroundWakeRefreshTask() + UNUserNotificationCenter.current().delegate = self application.registerForRemoteNotifications() return true } @@ -49,14 +82,415 @@ final class OpenClawAppDelegate: NSObject, UIApplicationDelegate { Task { @MainActor in guard let appModel = self.appModel else { self.logger.info("APNs wake skipped: appModel unavailable") + self.scheduleBackgroundWakeRefresh(afterSeconds: 90, reason: "silent_push_no_model") completionHandler(.noData) return } let handled = await appModel.handleSilentPushWake(userInfo) self.logger.info("APNs wake handled=\(handled, privacy: .public)") + if !handled { + self.scheduleBackgroundWakeRefresh(afterSeconds: 90, reason: "silent_push_not_applied") + } completionHandler(handled ? .newData : .noData) } } + + func scenePhaseChanged(_ phase: ScenePhase) { + if phase == .background { + self.scheduleBackgroundWakeRefresh(afterSeconds: 120, reason: "scene_background") + } + } + + private func registerBackgroundWakeRefreshTask() { + BGTaskScheduler.shared.register( + forTaskWithIdentifier: Self.wakeRefreshTaskIdentifier, + using: nil + ) { [weak self] task in + guard let refreshTask = task as? BGAppRefreshTask else { + task.setTaskCompleted(success: false) + return + } + self?.handleBackgroundWakeRefresh(task: refreshTask) + } + } + + private func scheduleBackgroundWakeRefresh(afterSeconds delay: TimeInterval, reason: String) { + let request = BGAppRefreshTaskRequest(identifier: Self.wakeRefreshTaskIdentifier) + request.earliestBeginDate = Date().addingTimeInterval(max(60, delay)) + do { + try BGTaskScheduler.shared.submit(request) + let scheduledLogMessage = + "Scheduled background wake refresh reason=\(reason) " + + "delaySeconds=\(max(60, delay))" + self.backgroundWakeLogger.info( + "\(scheduledLogMessage, privacy: .public)" + ) + } catch { + let failedLogMessage = + "Failed scheduling background wake refresh reason=\(reason) " + + "error=\(error.localizedDescription)" + self.backgroundWakeLogger.error( + "\(failedLogMessage, privacy: .public)" + ) + } + } + + private func handleBackgroundWakeRefresh(task: BGAppRefreshTask) { + self.scheduleBackgroundWakeRefresh(afterSeconds: 15 * 60, reason: "reschedule") + self.backgroundWakeTask?.cancel() + + let wakeTask = Task { @MainActor [weak self] in + guard let self, let appModel = self.appModel else { return false } + return await appModel.handleBackgroundRefreshWake(trigger: "bg_app_refresh") + } + self.backgroundWakeTask = wakeTask + task.expirationHandler = { + wakeTask.cancel() + } + Task { + let applied = await wakeTask.value + task.setTaskCompleted(success: applied) + self.backgroundWakeLogger.info( + "Background wake refresh finished applied=\(applied, privacy: .public)") + } + } + + private static func isWatchPromptNotification(_ userInfo: [AnyHashable: Any]) -> Bool { + (userInfo[WatchPromptNotificationBridge.typeKey] as? String) == WatchPromptNotificationBridge.typeValue + } + + private static func parseWatchPromptAction( + from response: UNNotificationResponse) -> PendingWatchPromptAction? + { + let userInfo = response.notification.request.content.userInfo + guard Self.isWatchPromptNotification(userInfo) else { return nil } + + let promptId = userInfo[WatchPromptNotificationBridge.promptIDKey] as? String + let sessionKey = userInfo[WatchPromptNotificationBridge.sessionKeyKey] as? String + + switch response.actionIdentifier { + case WatchPromptNotificationBridge.actionPrimaryIdentifier: + let actionId = (userInfo[WatchPromptNotificationBridge.actionPrimaryIDKey] as? String)? + .trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + guard !actionId.isEmpty else { return nil } + let actionLabel = userInfo[WatchPromptNotificationBridge.actionPrimaryLabelKey] as? String + return PendingWatchPromptAction( + promptId: promptId, + actionId: actionId, + actionLabel: actionLabel, + sessionKey: sessionKey) + case WatchPromptNotificationBridge.actionSecondaryIdentifier: + let actionId = (userInfo[WatchPromptNotificationBridge.actionSecondaryIDKey] as? String)? + .trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + guard !actionId.isEmpty else { return nil } + let actionLabel = userInfo[WatchPromptNotificationBridge.actionSecondaryLabelKey] as? String + return PendingWatchPromptAction( + promptId: promptId, + actionId: actionId, + actionLabel: actionLabel, + sessionKey: sessionKey) + default: + break + } + + guard response.actionIdentifier.hasPrefix(WatchPromptNotificationBridge.actionIdentifierPrefix) else { + return nil + } + let indexString = String( + response.actionIdentifier.dropFirst(WatchPromptNotificationBridge.actionIdentifierPrefix.count)) + guard let actionIndex = Int(indexString), actionIndex >= 0 else { + return nil + } + let actionIdKey = WatchPromptNotificationBridge.actionIDKey(index: actionIndex) + let actionLabelKey = WatchPromptNotificationBridge.actionLabelKey(index: actionIndex) + let actionId = (userInfo[actionIdKey] as? String)? + .trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + guard !actionId.isEmpty else { + return nil + } + let actionLabel = userInfo[actionLabelKey] as? String + return PendingWatchPromptAction( + promptId: promptId, + actionId: actionId, + actionLabel: actionLabel, + sessionKey: sessionKey) + } + + private func routeWatchPromptAction(_ action: PendingWatchPromptAction) async { + guard let appModel = self.appModel else { + self.pendingWatchPromptActions.append(action) + return + } + await appModel.handleMirroredWatchPromptAction( + promptId: action.promptId, + actionId: action.actionId, + actionLabel: action.actionLabel, + sessionKey: action.sessionKey) + _ = await appModel.handleBackgroundRefreshWake(trigger: "watch_prompt_action") + } + + func userNotificationCenter( + _ center: UNUserNotificationCenter, + willPresent notification: UNNotification, + withCompletionHandler completionHandler: @escaping (UNNotificationPresentationOptions) -> Void) + { + let userInfo = notification.request.content.userInfo + if Self.isWatchPromptNotification(userInfo) { + completionHandler([.banner, .list, .sound]) + return + } + completionHandler([]) + } + + func userNotificationCenter( + _ center: UNUserNotificationCenter, + didReceive response: UNNotificationResponse, + withCompletionHandler completionHandler: @escaping () -> Void) + { + guard let action = Self.parseWatchPromptAction(from: response) else { + completionHandler() + return + } + Task { @MainActor [weak self] in + guard let self else { + completionHandler() + return + } + await self.routeWatchPromptAction(action) + completionHandler() + } + } +} + +enum WatchPromptNotificationBridge { + static let typeKey = "openclaw.type" + static let typeValue = "watch.prompt" + static let promptIDKey = "openclaw.watch.promptId" + static let sessionKeyKey = "openclaw.watch.sessionKey" + static let actionPrimaryIDKey = "openclaw.watch.action.primary.id" + static let actionPrimaryLabelKey = "openclaw.watch.action.primary.label" + static let actionSecondaryIDKey = "openclaw.watch.action.secondary.id" + static let actionSecondaryLabelKey = "openclaw.watch.action.secondary.label" + static let actionPrimaryIdentifier = "openclaw.watch.action.primary" + static let actionSecondaryIdentifier = "openclaw.watch.action.secondary" + static let actionIdentifierPrefix = "openclaw.watch.action." + static let actionIDKeyPrefix = "openclaw.watch.action.id." + static let actionLabelKeyPrefix = "openclaw.watch.action.label." + static let categoryPrefix = "openclaw.watch.prompt.category." + + @MainActor + static func scheduleMirroredWatchPromptNotificationIfNeeded( + invokeID: String, + params: OpenClawWatchNotifyParams, + sendResult: WatchNotificationSendResult) async + { + guard sendResult.queuedForDelivery || !sendResult.deliveredImmediately else { return } + + let title = params.title.trimmingCharacters(in: .whitespacesAndNewlines) + let body = params.body.trimmingCharacters(in: .whitespacesAndNewlines) + guard !title.isEmpty || !body.isEmpty else { return } + guard await self.requestNotificationAuthorizationIfNeeded() else { return } + + let normalizedActions = (params.actions ?? []).compactMap { action -> OpenClawWatchAction? in + let id = action.id.trimmingCharacters(in: .whitespacesAndNewlines) + let label = action.label.trimmingCharacters(in: .whitespacesAndNewlines) + guard !id.isEmpty, !label.isEmpty else { return nil } + return OpenClawWatchAction(id: id, label: label, style: action.style) + } + let displayedActions = Array(normalizedActions.prefix(4)) + + let center = UNUserNotificationCenter.current() + var categoryIdentifier = "" + if !displayedActions.isEmpty { + let categoryID = "\(self.categoryPrefix)\(invokeID)" + let category = UNNotificationCategory( + identifier: categoryID, + actions: self.categoryActions(displayedActions), + intentIdentifiers: [], + options: []) + await self.upsertNotificationCategory(category, center: center) + categoryIdentifier = categoryID + } + + var userInfo: [AnyHashable: Any] = [ + self.typeKey: self.typeValue, + ] + if let promptId = params.promptId?.trimmingCharacters(in: .whitespacesAndNewlines), !promptId.isEmpty { + userInfo[self.promptIDKey] = promptId + } + if let sessionKey = params.sessionKey?.trimmingCharacters(in: .whitespacesAndNewlines), !sessionKey.isEmpty { + userInfo[self.sessionKeyKey] = sessionKey + } + for (index, action) in displayedActions.enumerated() { + userInfo[self.actionIDKey(index: index)] = action.id + userInfo[self.actionLabelKey(index: index)] = action.label + if index == 0 { + userInfo[self.actionPrimaryIDKey] = action.id + userInfo[self.actionPrimaryLabelKey] = action.label + } else if index == 1 { + userInfo[self.actionSecondaryIDKey] = action.id + userInfo[self.actionSecondaryLabelKey] = action.label + } + } + + let content = UNMutableNotificationContent() + content.title = title.isEmpty ? "OpenClaw" : title + content.body = body + content.sound = .default + content.userInfo = userInfo + if !categoryIdentifier.isEmpty { + content.categoryIdentifier = categoryIdentifier + } + if #available(iOS 15.0, *) { + switch params.priority ?? .active { + case .passive: + content.interruptionLevel = .passive + case .timeSensitive: + content.interruptionLevel = .timeSensitive + case .active: + content.interruptionLevel = .active + } + } + + let request = UNNotificationRequest( + identifier: "watch.prompt.\(invokeID)", + content: content, + trigger: nil) + try? await self.addNotificationRequest(request, center: center) + } + + static func actionIDKey(index: Int) -> String { + "\(self.actionIDKeyPrefix)\(index)" + } + + static func actionLabelKey(index: Int) -> String { + "\(self.actionLabelKeyPrefix)\(index)" + } + + private static func categoryActions(_ actions: [OpenClawWatchAction]) -> [UNNotificationAction] { + actions.enumerated().map { index, action in + let identifier: String + switch index { + case 0: + identifier = self.actionPrimaryIdentifier + case 1: + identifier = self.actionSecondaryIdentifier + default: + identifier = "\(self.actionIdentifierPrefix)\(index)" + } + return UNNotificationAction( + identifier: identifier, + title: action.label, + options: self.notificationActionOptions(style: action.style)) + } + } + + private static func notificationActionOptions(style: String?) -> UNNotificationActionOptions { + switch style?.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() { + case "destructive": + return [.destructive] + case "foreground": + // For mirrored watch actions, keep handling in background when possible. + return [] + default: + return [] + } + } + + private static func requestNotificationAuthorizationIfNeeded() async -> Bool { + let center = UNUserNotificationCenter.current() + let status = await self.notificationAuthorizationStatus(center: center) + switch status { + case .authorized, .provisional, .ephemeral: + return true + case .notDetermined: + let granted = (try? await center.requestAuthorization(options: [.alert, .sound, .badge])) ?? false + if !granted { return false } + let updatedStatus = await self.notificationAuthorizationStatus(center: center) + return self.isAuthorizationStatusAllowed(updatedStatus) + case .denied: + return false + @unknown default: + return false + } + } + + private static func isAuthorizationStatusAllowed(_ status: UNAuthorizationStatus) -> Bool { + switch status { + case .authorized, .provisional, .ephemeral: + return true + case .denied, .notDetermined: + return false + @unknown default: + return false + } + } + + private static func notificationAuthorizationStatus( + center: UNUserNotificationCenter + ) async -> UNAuthorizationStatus { + await withCheckedContinuation { continuation in + center.getNotificationSettings { settings in + continuation.resume(returning: settings.authorizationStatus) + } + } + } + + private static func upsertNotificationCategory( + _ category: UNNotificationCategory, + center: UNUserNotificationCenter) async + { + await withCheckedContinuation { continuation in + center.getNotificationCategories { categories in + var updated = categories + updated.update(with: category) + center.setNotificationCategories(updated) + continuation.resume() + } + } + } + + private static func addNotificationRequest( + _ request: UNNotificationRequest, + center: UNUserNotificationCenter + ) async throws { + try await withCheckedThrowingContinuation { (continuation: CheckedContinuation) in + center.add(request) { error in + if let error { + continuation.resume(throwing: error) + } else { + continuation.resume(returning: ()) + } + } + } + } +} + +extension NodeAppModel { + func handleMirroredWatchPromptAction( + promptId: String?, + actionId: String, + actionLabel: String?, + sessionKey: String?) async + { + let normalizedActionID = actionId.trimmingCharacters(in: .whitespacesAndNewlines) + guard !normalizedActionID.isEmpty else { return } + + let normalizedPromptID = promptId?.trimmingCharacters(in: .whitespacesAndNewlines) + let normalizedSessionKey = sessionKey?.trimmingCharacters(in: .whitespacesAndNewlines) + let normalizedActionLabel = actionLabel?.trimmingCharacters(in: .whitespacesAndNewlines) + + let event = WatchQuickReplyEvent( + replyId: UUID().uuidString, + promptId: (normalizedPromptID?.isEmpty == false) ? normalizedPromptID! : "unknown", + actionId: normalizedActionID, + actionLabel: (normalizedActionLabel?.isEmpty == false) ? normalizedActionLabel : nil, + sessionKey: (normalizedSessionKey?.isEmpty == false) ? normalizedSessionKey : nil, + note: "source=ios.notification", + sentAtMs: Int(Date().timeIntervalSince1970 * 1000), + transport: "ios.notification") + await self._bridgeConsumeMirroredWatchReply(event) + } } @main @@ -89,6 +523,7 @@ struct OpenClawApp: App { .onChange(of: self.scenePhase) { _, newValue in self.appModel.setScenePhase(newValue) self.gatewayController.setScenePhase(newValue) + self.appDelegate.scenePhaseChanged(newValue) } } } diff --git a/apps/ios/Sources/Reminders/RemindersService.swift b/apps/ios/Sources/Reminders/RemindersService.swift index 249f439fb17..8c347b2282b 100644 --- a/apps/ios/Sources/Reminders/RemindersService.swift +++ b/apps/ios/Sources/Reminders/RemindersService.swift @@ -17,7 +17,7 @@ final class RemindersService: RemindersServicing { let statusFilter = params.status ?? .incomplete let predicate = store.predicateForReminders(in: nil) - let payload = try await withCheckedThrowingContinuation { (cont: CheckedContinuation<[OpenClawReminderPayload], Error>) in + let payload: [OpenClawReminderPayload] = try await withCheckedThrowingContinuation { cont in store.fetchReminders(matching: predicate) { items in let formatter = ISO8601DateFormatter() let filtered = (items ?? []).filter { reminder in diff --git a/apps/ios/Sources/RootCanvas.swift b/apps/ios/Sources/RootCanvas.swift index 70ba9cdb96f..dd0f389ed4d 100644 --- a/apps/ios/Sources/RootCanvas.swift +++ b/apps/ios/Sources/RootCanvas.swift @@ -88,6 +88,7 @@ struct RootCanvas: View { } } .gatewayTrustPromptAlert() + .deepLinkAgentPromptAlert() .sheet(item: self.$presentedSheet) { sheet in switch sheet { case .settings: @@ -99,7 +100,7 @@ struct RootCanvas: View { ChatSheet( // Chat RPCs run on the operator session (read/write scopes). gateway: self.appModel.operatorSession, - sessionKey: self.appModel.mainSessionKey, + sessionKey: self.appModel.chatSessionKey, agentName: self.appModel.activeAgentName, userAccent: self.appModel.seamColor) case .quickSetup: diff --git a/apps/ios/Sources/Screen/ScreenRecordService.swift b/apps/ios/Sources/Screen/ScreenRecordService.swift index 11052f23543..c353d86f22d 100644 --- a/apps/ios/Sources/Screen/ScreenRecordService.swift +++ b/apps/ios/Sources/Screen/ScreenRecordService.swift @@ -55,7 +55,7 @@ final class ScreenRecordService: @unchecked Sendable { outPath: outPath) let state = CaptureState() - let recordQueue = DispatchQueue(label: "bot.molt.screenrecord") + let recordQueue = DispatchQueue(label: "ai.openclaw.screenrecord") try await self.startCapture(state: state, config: config, recordQueue: recordQueue) try await Task.sleep(nanoseconds: UInt64(config.durationMs) * 1_000_000) diff --git a/apps/ios/Sources/Services/NodeServiceProtocols.swift b/apps/ios/Sources/Services/NodeServiceProtocols.swift index 6f882e82a11..1eba72e7d6a 100644 --- a/apps/ios/Sources/Services/NodeServiceProtocols.swift +++ b/apps/ios/Sources/Services/NodeServiceProtocols.swift @@ -3,10 +3,13 @@ import Foundation import OpenClawKit import UIKit +typealias OpenClawCameraSnapResult = (format: String, base64: String, width: Int, height: Int) +typealias OpenClawCameraClipResult = (format: String, base64: String, durationMs: Int, hasAudio: Bool) + protocol CameraServicing: Sendable { func listDevices() async -> [CameraController.CameraDeviceInfo] - func snap(params: OpenClawCameraSnapParams) async throws -> (format: String, base64: String, width: Int, height: Int) - func clip(params: OpenClawCameraClipParams) async throws -> (format: String, base64: String, durationMs: Int, hasAudio: Bool) + func snap(params: OpenClawCameraSnapParams) async throws -> OpenClawCameraSnapResult + func clip(params: OpenClawCameraClipParams) async throws -> OpenClawCameraClipResult } protocol ScreenRecordingServicing: Sendable { @@ -73,6 +76,17 @@ struct WatchMessagingStatus: Sendable, Equatable { var activationState: String } +struct WatchQuickReplyEvent: Sendable, Equatable { + var replyId: String + var promptId: String + var actionId: String + var actionLabel: String? + var sessionKey: String? + var note: String? + var sentAtMs: Int? + var transport: String +} + struct WatchNotificationSendResult: Sendable, Equatable { var deliveredImmediately: Bool var queuedForDelivery: Bool @@ -81,11 +95,10 @@ struct WatchNotificationSendResult: Sendable, Equatable { protocol WatchMessagingServicing: AnyObject, Sendable { func status() async -> WatchMessagingStatus + func setReplyHandler(_ handler: (@Sendable (WatchQuickReplyEvent) -> Void)?) func sendNotification( id: String, - title: String, - body: String, - priority: OpenClawNotificationPriority?) async throws -> WatchNotificationSendResult + params: OpenClawWatchNotifyParams) async throws -> WatchNotificationSendResult } extension CameraController: CameraServicing {} diff --git a/apps/ios/Sources/Services/WatchMessagingService.swift b/apps/ios/Sources/Services/WatchMessagingService.swift index 8332fb5882d..e173a63c8e2 100644 --- a/apps/ios/Sources/Services/WatchMessagingService.swift +++ b/apps/ios/Sources/Services/WatchMessagingService.swift @@ -23,6 +23,8 @@ enum WatchMessagingError: LocalizedError { final class WatchMessagingService: NSObject, WatchMessagingServicing, @unchecked Sendable { private static let logger = Logger(subsystem: "ai.openclaw", category: "watch.messaging") private let session: WCSession? + private let replyHandlerLock = NSLock() + private var replyHandler: (@Sendable (WatchQuickReplyEvent) -> Void)? override init() { if WCSession.isSupported() { @@ -67,11 +69,15 @@ final class WatchMessagingService: NSObject, WatchMessagingServicing, @unchecked return Self.status(for: session) } + func setReplyHandler(_ handler: (@Sendable (WatchQuickReplyEvent) -> Void)?) { + self.replyHandlerLock.lock() + self.replyHandler = handler + self.replyHandlerLock.unlock() + } + func sendNotification( id: String, - title: String, - body: String, - priority: OpenClawNotificationPriority?) async throws -> WatchNotificationSendResult + params: OpenClawWatchNotifyParams) async throws -> WatchNotificationSendResult { await self.ensureActivated() guard let session = self.session else { @@ -82,14 +88,44 @@ final class WatchMessagingService: NSObject, WatchMessagingServicing, @unchecked guard snapshot.paired else { throw WatchMessagingError.notPaired } guard snapshot.appInstalled else { throw WatchMessagingError.watchAppNotInstalled } - let payload: [String: Any] = [ + var payload: [String: Any] = [ "type": "watch.notify", "id": id, - "title": title, - "body": body, - "priority": priority?.rawValue ?? OpenClawNotificationPriority.active.rawValue, + "title": params.title, + "body": params.body, + "priority": params.priority?.rawValue ?? OpenClawNotificationPriority.active.rawValue, "sentAtMs": Int(Date().timeIntervalSince1970 * 1000), ] + if let promptId = Self.nonEmpty(params.promptId) { + payload["promptId"] = promptId + } + if let sessionKey = Self.nonEmpty(params.sessionKey) { + payload["sessionKey"] = sessionKey + } + if let kind = Self.nonEmpty(params.kind) { + payload["kind"] = kind + } + if let details = Self.nonEmpty(params.details) { + payload["details"] = details + } + if let expiresAtMs = params.expiresAtMs { + payload["expiresAtMs"] = expiresAtMs + } + if let risk = params.risk { + payload["risk"] = risk.rawValue + } + if let actions = params.actions, !actions.isEmpty { + payload["actions"] = actions.map { action in + var encoded: [String: Any] = [ + "id": action.id, + "label": action.label, + ] + if let style = Self.nonEmpty(action.style) { + encoded["style"] = style + } + return encoded + } + } if snapshot.reachable { do { @@ -112,14 +148,59 @@ final class WatchMessagingService: NSObject, WatchMessagingServicing, @unchecked private func sendReachableMessage(_ payload: [String: Any], with session: WCSession) async throws { try await withCheckedThrowingContinuation { continuation in - session.sendMessage(payload, replyHandler: { _ in - continuation.resume() - }, errorHandler: { error in - continuation.resume(throwing: error) - }) + session.sendMessage( + payload, + replyHandler: { _ in + continuation.resume() + }, + errorHandler: { error in + continuation.resume(throwing: error) + } + ) } } + private func emitReply(_ event: WatchQuickReplyEvent) { + let handler: ((WatchQuickReplyEvent) -> Void)? + self.replyHandlerLock.lock() + handler = self.replyHandler + self.replyHandlerLock.unlock() + handler?(event) + } + + private static func nonEmpty(_ value: String?) -> String? { + let trimmed = value?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + return trimmed.isEmpty ? nil : trimmed + } + + private static func parseQuickReplyPayload( + _ payload: [String: Any], + transport: String) -> WatchQuickReplyEvent? + { + guard (payload["type"] as? String) == "watch.reply" else { + return nil + } + guard let actionId = nonEmpty(payload["actionId"] as? String) else { + return nil + } + let promptId = nonEmpty(payload["promptId"] as? String) ?? "unknown" + let replyId = nonEmpty(payload["replyId"] as? String) ?? UUID().uuidString + let actionLabel = nonEmpty(payload["actionLabel"] as? String) + let sessionKey = nonEmpty(payload["sessionKey"] as? String) + let note = nonEmpty(payload["note"] as? String) + let sentAtMs = (payload["sentAtMs"] as? Int) ?? (payload["sentAtMs"] as? NSNumber)?.intValue + + return WatchQuickReplyEvent( + replyId: replyId, + promptId: promptId, + actionId: actionId, + actionLabel: actionLabel, + sessionKey: sessionKey, + note: note, + sentAtMs: sentAtMs, + transport: transport) + } + private func ensureActivated() async { guard let session = self.session else { return } if session.activationState == .activated { return } @@ -172,5 +253,32 @@ extension WatchMessagingService: WCSessionDelegate { session.activate() } + func session(_: WCSession, didReceiveMessage message: [String: Any]) { + guard let event = Self.parseQuickReplyPayload(message, transport: "sendMessage") else { + return + } + self.emitReply(event) + } + + func session( + _: WCSession, + didReceiveMessage message: [String: Any], + replyHandler: @escaping ([String: Any]) -> Void) + { + guard let event = Self.parseQuickReplyPayload(message, transport: "sendMessage") else { + replyHandler(["ok": false, "error": "unsupported_payload"]) + return + } + replyHandler(["ok": true]) + self.emitReply(event) + } + + func session(_: WCSession, didReceiveUserInfo userInfo: [String: Any]) { + guard let event = Self.parseQuickReplyPayload(userInfo, transport: "transferUserInfo") else { + return + } + self.emitReply(event) + } + func sessionReachabilityDidChange(_ session: WCSession) {} } diff --git a/apps/ios/Sources/Settings/SettingsTab.swift b/apps/ios/Sources/Settings/SettingsTab.swift index 7825b45cb8d..7186c7205b5 100644 --- a/apps/ios/Sources/Settings/SettingsTab.swift +++ b/apps/ios/Sources/Settings/SettingsTab.swift @@ -5,6 +5,7 @@ import os import SwiftUI import UIKit +// swiftlint:disable type_body_length struct SettingsTab: View { private struct FeatureHelp: Identifiable { let id = UUID() @@ -22,7 +23,6 @@ struct SettingsTab: View { @AppStorage("talk.enabled") private var talkEnabled: Bool = false @AppStorage("talk.button.enabled") private var talkButtonEnabled: Bool = true @AppStorage("talk.background.enabled") private var talkBackgroundEnabled: Bool = false - @AppStorage("talk.voiceDirectiveHint.enabled") private var talkVoiceDirectiveHintEnabled: Bool = true @AppStorage("camera.enabled") private var cameraEnabled: Bool = true @AppStorage("location.enabledMode") private var locationEnabledModeRaw: String = OpenClawLocationMode.off.rawValue @AppStorage("screen.preventSleep") private var preventSleep: Bool = true @@ -229,7 +229,10 @@ struct SettingsTab: View { .foregroundStyle(.secondary) .frame(maxWidth: .infinity, alignment: .leading) .padding(10) - .background(.thinMaterial, in: RoundedRectangle(cornerRadius: 10, style: .continuous)) + .background( + .thinMaterial, + in: RoundedRectangle(cornerRadius: 10, style: .continuous) + ) } } } label: { @@ -276,7 +279,9 @@ struct SettingsTab: View { self.featureToggle( "Allow Camera", isOn: self.$cameraEnabled, - help: "Allows the gateway to request photos or short video clips while OpenClaw is foregrounded.") + help: "Allows the gateway to request photos or short video clips " + + "while OpenClaw is foregrounded." + ) HStack(spacing: 8) { Text("Location Access") @@ -284,7 +289,11 @@ struct SettingsTab: View { Button { self.activeFeatureHelp = FeatureHelp( title: "Location Access", - message: "Controls location permissions for OpenClaw. Off disables location tools, While Using enables foreground location, and Always enables background location.") + message: "Controls location permissions for OpenClaw. " + + "Off disables location tools, While Using enables " + + "foreground location, and Always enables " + + "background location." + ) } label: { Image(systemName: "info.circle") .foregroundStyle(.secondary) @@ -306,10 +315,30 @@ struct SettingsTab: View { help: "Keeps the screen awake while OpenClaw is open.") DisclosureGroup("Advanced") { - self.featureToggle( - "Voice Directive Hint", - isOn: self.$talkVoiceDirectiveHintEnabled, - help: "Adds voice-switching instructions to Talk prompts. Disable to reduce prompt size.") + VStack(alignment: .leading, spacing: 8) { + Text("Talk Voice (Gateway)") + .font(.footnote.weight(.semibold)) + .foregroundStyle(.secondary) + LabeledContent("Provider", value: "ElevenLabs") + LabeledContent( + "API Key", + value: self.appModel.talkMode.gatewayTalkConfigLoaded + ? ( + self.appModel.talkMode.gatewayTalkApiKeyConfigured + ? "Configured" + : "Not configured" + ) + : "Not loaded") + LabeledContent( + "Default Model", + value: self.appModel.talkMode.gatewayTalkDefaultModelId ?? "eleven_v3 (fallback)") + LabeledContent( + "Default Voice", + value: self.appModel.talkMode.gatewayTalkDefaultVoiceId ?? "auto (first available)") + Text("Configured on gateway via talk.apiKey, talk.modelId, and talk.voiceId.") + .font(.footnote) + .foregroundStyle(.secondary) + } self.featureToggle( "Show Talk Button", isOn: self.$talkButtonEnabled, @@ -325,7 +354,9 @@ struct SettingsTab: View { Button { self.activeFeatureHelp = FeatureHelp( title: "Default Share Instruction", - message: "Appends this instruction when sharing content into OpenClaw from iOS.") + message: "Appends this instruction when sharing content " + + "into OpenClaw from iOS." + ) } label: { Image(systemName: "info.circle") .foregroundStyle(.secondary) @@ -354,9 +385,9 @@ struct SettingsTab: View { .foregroundStyle(.secondary) .lineLimit(1) .truncationMode(.middle) - LabeledContent("Device", value: self.deviceFamily()) - LabeledContent("Platform", value: self.platformString()) - LabeledContent("OpenClaw", value: self.openClawVersionString()) + LabeledContent("Device", value: DeviceInfoHelper.deviceFamily()) + LabeledContent("Platform", value: DeviceInfoHelper.platformStringForDisplay()) + LabeledContent("OpenClaw", value: DeviceInfoHelper.openClawVersionString()) } } } @@ -378,7 +409,9 @@ struct SettingsTab: View { Button("Cancel", role: .cancel) {} } message: { Text( - "This will disconnect, clear saved gateway connection + credentials, and reopen the onboarding wizard.") + "This will disconnect, clear saved gateway connection + credentials, " + + "and reopen the onboarding wizard." + ) } .alert(item: self.$activeFeatureHelp) { help in Alert( @@ -399,6 +432,9 @@ struct SettingsTab: View { // Keep setup front-and-center when disconnected; keep things compact once connected. self.gatewayExpanded = !self.isGatewayConnected self.selectedAgentPickerId = self.appModel.selectedAgentId ?? "" + if self.isGatewayConnected { + self.appModel.reloadTalkConfig() + } } .onChange(of: self.selectedAgentPickerId) { _, newValue in let trimmed = newValue.trimmingCharacters(in: .whitespacesAndNewlines) @@ -461,6 +497,10 @@ struct SettingsTab: View { self.locationEnabledModeRaw = previous self.lastLocationModeRaw = previous } + return + } + await MainActor.run { + self.gatewayController.refreshActiveGatewayRegistrationFromSettings() } } } @@ -557,32 +597,6 @@ struct SettingsTab: View { return trimmed.isEmpty ? "Not connected" : trimmed } - private func platformString() -> String { - let v = ProcessInfo.processInfo.operatingSystemVersion - return "iOS \(v.majorVersion).\(v.minorVersion).\(v.patchVersion)" - } - - private func deviceFamily() -> String { - switch UIDevice.current.userInterfaceIdiom { - case .pad: - "iPad" - case .phone: - "iPhone" - default: - "iOS" - } - } - - private func openClawVersionString() -> String { - let version = Bundle.main.infoDictionary?["CFBundleShortVersionString"] as? String ?? "dev" - let build = Bundle.main.infoDictionary?["CFBundleVersion"] as? String ?? "" - let trimmedBuild = build.trimmingCharacters(in: .whitespacesAndNewlines) - if trimmedBuild.isEmpty || trimmedBuild == version { - return version - } - return "\(version) (\(trimmedBuild))" - } - private func featureToggle( _ title: String, isOn: Binding, @@ -705,7 +719,9 @@ struct SettingsTab: View { let hasToken = !self.gatewayToken.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty let hasPassword = !self.gatewayPassword.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty GatewayDiagnostics.log( - "setup code applied host=\(host) port=\(resolvedPort ?? -1) tls=\(self.manualGatewayTLS) token=\(hasToken) password=\(hasPassword)") + "setup code applied host=\(host) port=\(resolvedPort ?? -1) " + + "tls=\(self.manualGatewayTLS) token=\(hasToken) password=\(hasPassword)" + ) guard let port = resolvedPort else { self.setupStatusText = "Failed: invalid port" return @@ -1013,3 +1029,4 @@ struct SettingsTab: View { return lines } } +// swiftlint:enable type_body_length diff --git a/apps/ios/Sources/Status/StatusPill.swift b/apps/ios/Sources/Status/StatusPill.swift index ea5e425c49d..8c0885fc516 100644 --- a/apps/ios/Sources/Status/StatusPill.swift +++ b/apps/ios/Sources/Status/StatusPill.swift @@ -51,7 +51,11 @@ struct StatusPill: View { Circle() .fill(self.gateway.color) .frame(width: 9, height: 9) - .scaleEffect(self.gateway == .connecting && !self.reduceMotion ? (self.pulse ? 1.15 : 0.85) : 1.0) + .scaleEffect( + self.gateway == .connecting && !self.reduceMotion + ? (self.pulse ? 1.15 : 0.85) + : 1.0 + ) .opacity(self.gateway == .connecting && !self.reduceMotion ? (self.pulse ? 1.0 : 0.6) : 1.0) Text(self.gateway.title) diff --git a/apps/ios/Sources/Voice/TalkModeManager.swift b/apps/ios/Sources/Voice/TalkModeManager.swift index be90208af47..5210921a5a7 100644 --- a/apps/ios/Sources/Voice/TalkModeManager.swift +++ b/apps/ios/Sources/Voice/TalkModeManager.swift @@ -10,12 +10,13 @@ import Speech // This file intentionally centralizes talk mode state + behavior. // It's large, and splitting would force `private` -> `fileprivate` across many members. // We'll refactor into smaller files when the surface stabilizes. -// swiftlint:disable type_body_length +// swiftlint:disable type_body_length file_length @MainActor @Observable final class TalkModeManager: NSObject { private typealias SpeechRequest = SFSpeechAudioBufferRecognitionRequest private static let defaultModelIdFallback = "eleven_v3" + private static let defaultTalkProvider = "elevenlabs" private static let redactedConfigSentinel = "__OPENCLAW_REDACTED__" var isEnabled: Bool = false var isListening: Bool = false @@ -24,6 +25,10 @@ final class TalkModeManager: NSObject { var statusText: String = "Off" /// 0..1-ish (not calibrated). Intended for UI feedback only. var micLevel: Double = 0 + var gatewayTalkConfigLoaded: Bool = false + var gatewayTalkApiKeyConfigured: Bool = false + var gatewayTalkDefaultModelId: String? + var gatewayTalkDefaultVoiceId: String? private enum CaptureMode { case idle @@ -87,8 +92,10 @@ final class TalkModeManager: NSObject { private var incrementalSpeechBuffer = IncrementalSpeechBuffer() private var incrementalSpeechContext: IncrementalSpeechContext? private var incrementalSpeechDirective: TalkDirective? + private var incrementalSpeechPrefetch: IncrementalSpeechPrefetchState? + private var incrementalSpeechPrefetchMonitorTask: Task? - private let logger = Logger(subsystem: "bot.molt", category: "TalkMode") + private let logger = Logger(subsystem: "ai.openclaw", category: "TalkMode") init(allowSimulatorCapture: Bool = false) { self.allowSimulatorCapture = allowSimulatorCapture @@ -149,9 +156,7 @@ final class TalkModeManager: NSObject { let micOk = await Self.requestMicrophonePermission() guard micOk else { self.logger.warning("start blocked: microphone permission denied") - self.statusText = Self.permissionMessage( - kind: "Microphone", - status: AVAudioSession.sharedInstance().recordPermission) + self.statusText = "Microphone permission denied" return } let speechOk = await Self.requestSpeechPermission() @@ -293,9 +298,7 @@ final class TalkModeManager: NSObject { if !self.allowSimulatorCapture { let micOk = await Self.requestMicrophonePermission() guard micOk else { - self.statusText = Self.permissionMessage( - kind: "Microphone", - status: AVAudioSession.sharedInstance().recordPermission) + self.statusText = "Microphone permission denied" throw NSError(domain: "TalkMode", code: 4, userInfo: [ NSLocalizedDescriptionKey: "Microphone permission denied", ]) @@ -463,14 +466,15 @@ final class TalkModeManager: NSObject { private func startRecognition() throws { #if targetEnvironment(simulator) + if self.allowSimulatorCapture { + self.recognitionRequest = SFSpeechAudioBufferRecognitionRequest() + self.recognitionRequest?.shouldReportPartialResults = true + return + } if !self.allowSimulatorCapture { throw NSError(domain: "TalkMode", code: 2, userInfo: [ NSLocalizedDescriptionKey: "Talk mode is not supported on the iOS simulator", ]) - } else { - self.recognitionRequest = SFSpeechAudioBufferRecognitionRequest() - self.recognitionRequest?.shouldReportPartialResults = true - return } #endif @@ -518,7 +522,9 @@ final class TalkModeManager: NSObject { self.noiseFloorSamples.removeAll(keepingCapacity: true) let threshold = min(0.35, max(0.12, avg + 0.10)) GatewayDiagnostics.log( - "talk audio: noiseFloor=\(String(format: "%.3f", avg)) threshold=\(String(format: "%.3f", threshold))") + "talk audio: noiseFloor=\(String(format: "%.3f", avg)) " + + "threshold=\(String(format: "%.3f", threshold))" + ) } } @@ -542,11 +548,23 @@ final class TalkModeManager: NSObject { self.loggedPartialThisCycle = false GatewayDiagnostics.log( - "talk speech: recognition started mode=\(String(describing: self.captureMode)) engineRunning=\(self.audioEngine.isRunning)") + "talk speech: recognition started mode=\(String(describing: self.captureMode)) " + + "engineRunning=\(self.audioEngine.isRunning)" + ) self.recognitionTask = recognizer.recognitionTask(with: request) { [weak self] result, error in guard let self else { return } if let error { let msg = error.localizedDescription + let lowered = msg.lowercased() + let isCancellation = lowered.contains("cancelled") || lowered.contains("canceled") + if isCancellation { + GatewayDiagnostics.log("talk speech: cancelled") + if self.captureMode == .continuous, self.isEnabled, !self.isSpeaking { + self.statusText = "Listening" + } + self.logger.debug("speech recognition cancelled") + return + } GatewayDiagnostics.log("talk speech: error=\(msg)") if !self.isSpeaking { if msg.localizedCaseInsensitiveContains("no speech detected") { @@ -833,11 +851,10 @@ final class TalkModeManager: NSObject { private func buildPrompt(transcript: String) -> String { let interrupted = self.lastInterruptedAtSeconds self.lastInterruptedAtSeconds = nil - let includeVoiceDirectiveHint = (UserDefaults.standard.object(forKey: "talk.voiceDirectiveHint.enabled") as? Bool) ?? true return TalkPromptBuilder.build( transcript: transcript, interruptedAtSeconds: interrupted, - includeVoiceDirectiveHint: includeVoiceDirectiveHint) + includeVoiceDirectiveHint: false) } private enum ChatCompletionState: CustomStringConvertible { @@ -1173,6 +1190,7 @@ final class TalkModeManager: NSObject { self.incrementalSpeechQueue.removeAll() self.incrementalSpeechTask?.cancel() self.incrementalSpeechTask = nil + self.cancelIncrementalPrefetch() self.incrementalSpeechActive = true self.incrementalSpeechUsed = false self.incrementalSpeechLanguage = nil @@ -1185,6 +1203,7 @@ final class TalkModeManager: NSObject { self.incrementalSpeechQueue.removeAll() self.incrementalSpeechTask?.cancel() self.incrementalSpeechTask = nil + self.cancelIncrementalPrefetch() self.incrementalSpeechActive = false self.incrementalSpeechContext = nil self.incrementalSpeechDirective = nil @@ -1212,20 +1231,168 @@ final class TalkModeManager: NSObject { self.incrementalSpeechTask = Task { @MainActor [weak self] in guard let self else { return } + defer { + self.cancelIncrementalPrefetch() + self.isSpeaking = false + self.stopRecognition() + self.incrementalSpeechTask = nil + } while !Task.isCancelled { guard !self.incrementalSpeechQueue.isEmpty else { break } let segment = self.incrementalSpeechQueue.removeFirst() self.statusText = "Speaking…" self.isSpeaking = true self.lastSpokenText = segment - await self.speakIncrementalSegment(segment) + await self.updateIncrementalContextIfNeeded() + let context = self.incrementalSpeechContext + let prefetchedAudio = await self.consumeIncrementalPrefetchedAudioIfAvailable( + for: segment, + context: context) + if let context { + self.startIncrementalPrefetchMonitor(context: context) + } + await self.speakIncrementalSegment( + segment, + context: context, + prefetchedAudio: prefetchedAudio) + self.cancelIncrementalPrefetchMonitor() } - self.isSpeaking = false - self.stopRecognition() - self.incrementalSpeechTask = nil } } + private func cancelIncrementalPrefetch() { + self.cancelIncrementalPrefetchMonitor() + self.incrementalSpeechPrefetch?.task.cancel() + self.incrementalSpeechPrefetch = nil + } + + private func cancelIncrementalPrefetchMonitor() { + self.incrementalSpeechPrefetchMonitorTask?.cancel() + self.incrementalSpeechPrefetchMonitorTask = nil + } + + private func startIncrementalPrefetchMonitor(context: IncrementalSpeechContext) { + self.cancelIncrementalPrefetchMonitor() + self.incrementalSpeechPrefetchMonitorTask = Task { @MainActor [weak self] in + guard let self else { return } + while !Task.isCancelled { + if self.ensureIncrementalPrefetchForUpcomingSegment(context: context) { + return + } + try? await Task.sleep(nanoseconds: 40_000_000) + } + } + } + + private func ensureIncrementalPrefetchForUpcomingSegment(context: IncrementalSpeechContext) -> Bool { + guard context.canUseElevenLabs else { + self.cancelIncrementalPrefetch() + return false + } + guard let nextSegment = self.incrementalSpeechQueue.first else { return false } + if let existing = self.incrementalSpeechPrefetch { + if existing.segment == nextSegment, existing.context == context { + return true + } + existing.task.cancel() + self.incrementalSpeechPrefetch = nil + } + self.startIncrementalPrefetch(segment: nextSegment, context: context) + return self.incrementalSpeechPrefetch != nil + } + + private func startIncrementalPrefetch(segment: String, context: IncrementalSpeechContext) { + guard context.canUseElevenLabs, let apiKey = context.apiKey, let voiceId = context.voiceId else { return } + let prefetchOutputFormat = self.resolveIncrementalPrefetchOutputFormat(context: context) + let request = self.makeIncrementalTTSRequest( + text: segment, + context: context, + outputFormat: prefetchOutputFormat) + let id = UUID() + let task = Task { [weak self] in + let stream = ElevenLabsTTSClient(apiKey: apiKey).streamSynthesize(voiceId: voiceId, request: request) + var chunks: [Data] = [] + do { + for try await chunk in stream { + try Task.checkCancellation() + chunks.append(chunk) + } + self?.completeIncrementalPrefetch(id: id, chunks: chunks) + } catch is CancellationError { + self?.clearIncrementalPrefetch(id: id) + } catch { + self?.failIncrementalPrefetch(id: id, error: error) + } + } + self.incrementalSpeechPrefetch = IncrementalSpeechPrefetchState( + id: id, + segment: segment, + context: context, + outputFormat: prefetchOutputFormat, + chunks: nil, + task: task) + } + + private func completeIncrementalPrefetch(id: UUID, chunks: [Data]) { + guard var prefetch = self.incrementalSpeechPrefetch, prefetch.id == id else { return } + prefetch.chunks = chunks + self.incrementalSpeechPrefetch = prefetch + } + + private func clearIncrementalPrefetch(id: UUID) { + guard let prefetch = self.incrementalSpeechPrefetch, prefetch.id == id else { return } + prefetch.task.cancel() + self.incrementalSpeechPrefetch = nil + } + + private func failIncrementalPrefetch(id: UUID, error: any Error) { + guard let prefetch = self.incrementalSpeechPrefetch, prefetch.id == id else { return } + self.logger.debug("incremental prefetch failed: \(error.localizedDescription, privacy: .public)") + prefetch.task.cancel() + self.incrementalSpeechPrefetch = nil + } + + private func consumeIncrementalPrefetchedAudioIfAvailable( + for segment: String, + context: IncrementalSpeechContext? + ) async -> IncrementalPrefetchedAudio? + { + guard let context else { + self.cancelIncrementalPrefetch() + return nil + } + guard let prefetch = self.incrementalSpeechPrefetch else { + return nil + } + guard prefetch.context == context else { + prefetch.task.cancel() + self.incrementalSpeechPrefetch = nil + return nil + } + guard prefetch.segment == segment else { + return nil + } + if let chunks = prefetch.chunks, !chunks.isEmpty { + let prefetched = IncrementalPrefetchedAudio(chunks: chunks, outputFormat: prefetch.outputFormat) + self.incrementalSpeechPrefetch = nil + return prefetched + } + await prefetch.task.value + guard let completed = self.incrementalSpeechPrefetch else { return nil } + guard completed.context == context, completed.segment == segment else { return nil } + guard let chunks = completed.chunks, !chunks.isEmpty else { return nil } + let prefetched = IncrementalPrefetchedAudio(chunks: chunks, outputFormat: completed.outputFormat) + self.incrementalSpeechPrefetch = nil + return prefetched + } + + private func resolveIncrementalPrefetchOutputFormat(context: IncrementalSpeechContext) -> String? { + if TalkTTSValidation.pcmSampleRate(from: context.outputFormat) != nil { + return ElevenLabsTTSClient.validatedOutputFormat("mp3_44100") + } + return context.outputFormat + } + private func finishIncrementalSpeech() async { guard self.incrementalSpeechActive else { return } let leftover = self.incrementalSpeechBuffer.flush() @@ -1260,7 +1427,10 @@ final class TalkModeManager: NSObject { for await evt in stream { if Task.isCancelled { return } guard evt.event == "agent", let payload = evt.payload else { continue } - guard let agentEvent = try? GatewayPayloadDecoding.decode(payload, as: OpenClawAgentEventPayload.self) else { + guard let agentEvent = try? GatewayPayloadDecoding.decode( + payload, + as: OpenClawAgentEventPayload.self + ) else { continue } guard agentEvent.runId == runId, agentEvent.stream == "assistant" else { continue } @@ -1333,77 +1503,103 @@ final class TalkModeManager: NSObject { canUseElevenLabs: canUseElevenLabs) } - private func speakIncrementalSegment(_ text: String) async { - await self.updateIncrementalContextIfNeeded() - guard let context = self.incrementalSpeechContext else { + private func makeIncrementalTTSRequest( + text: String, + context: IncrementalSpeechContext, + outputFormat: String? + ) -> ElevenLabsTTSRequest + { + ElevenLabsTTSRequest( + text: text, + modelId: context.modelId, + outputFormat: outputFormat, + speed: TalkTTSValidation.resolveSpeed( + speed: context.directive?.speed, + rateWPM: context.directive?.rateWPM), + stability: TalkTTSValidation.validatedStability( + context.directive?.stability, + modelId: context.modelId), + similarity: TalkTTSValidation.validatedUnit(context.directive?.similarity), + style: TalkTTSValidation.validatedUnit(context.directive?.style), + speakerBoost: context.directive?.speakerBoost, + seed: TalkTTSValidation.validatedSeed(context.directive?.seed), + normalize: ElevenLabsTTSClient.validatedNormalize(context.directive?.normalize), + language: context.language, + latencyTier: TalkTTSValidation.validatedLatencyTier(context.directive?.latencyTier)) + } + + private static func makeBufferedAudioStream(chunks: [Data]) -> AsyncThrowingStream { + AsyncThrowingStream { continuation in + for chunk in chunks { + continuation.yield(chunk) + } + continuation.finish() + } + } + + private func speakIncrementalSegment( + _ text: String, + context preferredContext: IncrementalSpeechContext? = nil, + prefetchedAudio: IncrementalPrefetchedAudio? = nil + ) async + { + let context: IncrementalSpeechContext + if let preferredContext { + context = preferredContext + } else { + await self.updateIncrementalContextIfNeeded() + guard let resolvedContext = self.incrementalSpeechContext else { + try? await TalkSystemSpeechSynthesizer.shared.speak( + text: text, + language: self.incrementalSpeechLanguage) + return + } + context = resolvedContext + } + + guard context.canUseElevenLabs, let apiKey = context.apiKey, let voiceId = context.voiceId else { try? await TalkSystemSpeechSynthesizer.shared.speak( text: text, language: self.incrementalSpeechLanguage) return } - if context.canUseElevenLabs, let apiKey = context.apiKey, let voiceId = context.voiceId { - let request = ElevenLabsTTSRequest( - text: text, - modelId: context.modelId, - outputFormat: context.outputFormat, - speed: TalkTTSValidation.resolveSpeed( - speed: context.directive?.speed, - rateWPM: context.directive?.rateWPM), - stability: TalkTTSValidation.validatedStability( - context.directive?.stability, - modelId: context.modelId), - similarity: TalkTTSValidation.validatedUnit(context.directive?.similarity), - style: TalkTTSValidation.validatedUnit(context.directive?.style), - speakerBoost: context.directive?.speakerBoost, - seed: TalkTTSValidation.validatedSeed(context.directive?.seed), - normalize: ElevenLabsTTSClient.validatedNormalize(context.directive?.normalize), - language: context.language, - latencyTier: TalkTTSValidation.validatedLatencyTier(context.directive?.latencyTier)) - let client = ElevenLabsTTSClient(apiKey: apiKey) - let stream = client.streamSynthesize(voiceId: voiceId, request: request) - let sampleRate = TalkTTSValidation.pcmSampleRate(from: context.outputFormat) - let result: StreamingPlaybackResult - if let sampleRate { - self.lastPlaybackWasPCM = true - var playback = await self.pcmPlayer.play(stream: stream, sampleRate: sampleRate) - if !playback.finished, playback.interruptedAt == nil { - self.logger.warning("pcm playback failed; retrying mp3") - self.lastPlaybackWasPCM = false - let mp3Format = ElevenLabsTTSClient.validatedOutputFormat("mp3_44100") - let mp3Stream = client.streamSynthesize( - voiceId: voiceId, - request: ElevenLabsTTSRequest( - text: text, - modelId: context.modelId, - outputFormat: mp3Format, - speed: TalkTTSValidation.resolveSpeed( - speed: context.directive?.speed, - rateWPM: context.directive?.rateWPM), - stability: TalkTTSValidation.validatedStability( - context.directive?.stability, - modelId: context.modelId), - similarity: TalkTTSValidation.validatedUnit(context.directive?.similarity), - style: TalkTTSValidation.validatedUnit(context.directive?.style), - speakerBoost: context.directive?.speakerBoost, - seed: TalkTTSValidation.validatedSeed(context.directive?.seed), - normalize: ElevenLabsTTSClient.validatedNormalize(context.directive?.normalize), - language: context.language, - latencyTier: TalkTTSValidation.validatedLatencyTier(context.directive?.latencyTier))) - playback = await self.mp3Player.play(stream: mp3Stream) - } - result = playback - } else { - self.lastPlaybackWasPCM = false - result = await self.mp3Player.play(stream: stream) - } - if !result.finished, let interruptedAt = result.interruptedAt { - self.lastInterruptedAtSeconds = interruptedAt - } + let client = ElevenLabsTTSClient(apiKey: apiKey) + let request = self.makeIncrementalTTSRequest( + text: text, + context: context, + outputFormat: context.outputFormat) + let stream: AsyncThrowingStream + if let prefetchedAudio, !prefetchedAudio.chunks.isEmpty { + stream = Self.makeBufferedAudioStream(chunks: prefetchedAudio.chunks) } else { - try? await TalkSystemSpeechSynthesizer.shared.speak( - text: text, - language: self.incrementalSpeechLanguage) + stream = client.streamSynthesize(voiceId: voiceId, request: request) + } + let playbackFormat = prefetchedAudio?.outputFormat ?? context.outputFormat + let sampleRate = TalkTTSValidation.pcmSampleRate(from: playbackFormat) + let result: StreamingPlaybackResult + if let sampleRate { + self.lastPlaybackWasPCM = true + var playback = await self.pcmPlayer.play(stream: stream, sampleRate: sampleRate) + if !playback.finished, playback.interruptedAt == nil { + self.logger.warning("pcm playback failed; retrying mp3") + self.lastPlaybackWasPCM = false + let mp3Format = ElevenLabsTTSClient.validatedOutputFormat("mp3_44100") + let mp3Stream = client.streamSynthesize( + voiceId: voiceId, + request: self.makeIncrementalTTSRequest( + text: text, + context: context, + outputFormat: mp3Format)) + playback = await self.mp3Player.play(stream: mp3Stream) + } + result = playback + } else { + self.lastPlaybackWasPCM = false + result = await self.mp3Player.play(stream: stream) + } + if !result.finished, let interruptedAt = result.interruptedAt { + self.lastInterruptedAtSeconds = interruptedAt } } @@ -1534,23 +1730,20 @@ private struct IncrementalSpeechBuffer { extension TalkModeManager { nonisolated static func requestMicrophonePermission() async -> Bool { - let session = AVAudioSession.sharedInstance() - switch session.recordPermission { + switch AVAudioApplication.shared.recordPermission { case .granted: return true case .denied: return false case .undetermined: - break + return await self.requestPermissionWithTimeout { completion in + AVAudioApplication.requestRecordPermission(completionHandler: { ok in + completion(ok) + }) + } @unknown default: return false } - - return await self.requestPermissionWithTimeout { completion in - AVAudioSession.sharedInstance().requestRecordPermission { ok in - completion(ok) - } - } } nonisolated static func requestSpeechPermission() async -> Bool { @@ -1574,7 +1767,7 @@ extension TalkModeManager { } private nonisolated static func requestPermissionWithTimeout( - _ operation: @escaping @Sendable (@escaping (Bool) -> Void) -> Void) async -> Bool + _ operation: @escaping @Sendable (@escaping @Sendable (Bool) -> Void) -> Void) async -> Bool { do { return try await AsyncTimeout.withTimeout( @@ -1693,15 +1886,59 @@ extension TalkModeManager { return trimmed } + struct TalkProviderConfigSelection { + let provider: String + let config: [String: Any] + } + + private static func normalizedTalkProviderID(_ raw: String?) -> String? { + let trimmed = (raw ?? "").trimmingCharacters(in: .whitespacesAndNewlines).lowercased() + return trimmed.isEmpty ? nil : trimmed + } + + static func selectTalkProviderConfig(_ talk: [String: Any]?) -> TalkProviderConfigSelection? { + guard let talk else { return nil } + let rawProvider = talk["provider"] as? String + let rawProviders = talk["providers"] as? [String: Any] + guard rawProvider != nil || rawProviders != nil else { return nil } + let providers = rawProviders ?? [:] + let normalizedProviders = providers.reduce(into: [String: [String: Any]]()) { acc, entry in + guard + let providerID = Self.normalizedTalkProviderID(entry.key), + let config = entry.value as? [String: Any] + else { return } + acc[providerID] = config + } + let providerID = + Self.normalizedTalkProviderID(rawProvider) ?? + normalizedProviders.keys.min() ?? + Self.defaultTalkProvider + return TalkProviderConfigSelection( + provider: providerID, + config: normalizedProviders[providerID] ?? [:]) + } + func reloadConfig() async { guard let gateway else { return } do { - let res = try await gateway.request(method: "talk.config", paramsJSON: "{\"includeSecrets\":true}", timeoutSeconds: 8) + let res = try await gateway.request( + method: "talk.config", + paramsJSON: "{\"includeSecrets\":true}", + timeoutSeconds: 8 + ) guard let json = try JSONSerialization.jsonObject(with: res) as? [String: Any] else { return } guard let config = json["config"] as? [String: Any] else { return } let talk = config["talk"] as? [String: Any] - self.defaultVoiceId = (talk?["voiceId"] as? String)?.trimmingCharacters(in: .whitespacesAndNewlines) - if let aliases = talk?["voiceAliases"] as? [String: Any] { + let selection = Self.selectTalkProviderConfig(talk) + if talk != nil, selection == nil { + GatewayDiagnostics.log( + "talk config ignored: legacy payload unsupported on iOS beta; expected talk.provider/providers") + } + let activeProvider = selection?.provider ?? Self.defaultTalkProvider + let activeConfig = selection?.config + self.defaultVoiceId = (activeConfig?["voiceId"] as? String)? + .trimmingCharacters(in: .whitespacesAndNewlines) + if let aliases = activeConfig?["voiceAliases"] as? [String: Any] { var resolved: [String: String] = [:] for (key, value) in aliases { guard let id = value as? String else { continue } @@ -1717,30 +1954,47 @@ extension TalkModeManager { if !self.voiceOverrideActive { self.currentVoiceId = self.defaultVoiceId } - let model = (talk?["modelId"] as? String)?.trimmingCharacters(in: .whitespacesAndNewlines) + let model = (activeConfig?["modelId"] as? String)?.trimmingCharacters(in: .whitespacesAndNewlines) self.defaultModelId = (model?.isEmpty == false) ? model : Self.defaultModelIdFallback if !self.modelOverrideActive { self.currentModelId = self.defaultModelId } - self.defaultOutputFormat = (talk?["outputFormat"] as? String)? + self.defaultOutputFormat = (activeConfig?["outputFormat"] as? String)? .trimmingCharacters(in: .whitespacesAndNewlines) - let rawConfigApiKey = (talk?["apiKey"] as? String)?.trimmingCharacters(in: .whitespacesAndNewlines) + let rawConfigApiKey = (activeConfig?["apiKey"] as? String)?.trimmingCharacters(in: .whitespacesAndNewlines) let configApiKey = Self.normalizedTalkApiKey(rawConfigApiKey) - let localApiKey = Self.normalizedTalkApiKey(GatewaySettingsStore.loadTalkElevenLabsApiKey()) + let localApiKey = Self.normalizedTalkApiKey( + GatewaySettingsStore.loadTalkProviderApiKey(provider: activeProvider)) if rawConfigApiKey == Self.redactedConfigSentinel { self.apiKey = (localApiKey?.isEmpty == false) ? localApiKey : nil GatewayDiagnostics.log("talk config apiKey redacted; using local override if present") } else { self.apiKey = (localApiKey?.isEmpty == false) ? localApiKey : configApiKey } + if activeProvider != Self.defaultTalkProvider { + self.apiKey = nil + GatewayDiagnostics.log( + "talk provider '\(activeProvider)' not yet supported on iOS; using system voice fallback") + } + self.gatewayTalkDefaultVoiceId = self.defaultVoiceId + self.gatewayTalkDefaultModelId = self.defaultModelId + self.gatewayTalkApiKeyConfigured = (self.apiKey?.isEmpty == false) + self.gatewayTalkConfigLoaded = true if let interrupt = talk?["interruptOnSpeech"] as? Bool { self.interruptOnSpeech = interrupt } + if selection != nil { + GatewayDiagnostics.log("talk config provider=\(activeProvider)") + } } catch { self.defaultModelId = Self.defaultModelIdFallback if !self.modelOverrideActive { self.currentModelId = self.defaultModelId } + self.gatewayTalkDefaultVoiceId = nil + self.gatewayTalkDefaultModelId = nil + self.gatewayTalkApiKeyConfigured = false + self.gatewayTalkConfigLoaded = false } } @@ -1758,10 +2012,18 @@ extension TalkModeManager { private static func describeAudioSession() -> String { let session = AVAudioSession.sharedInstance() - let inputs = session.currentRoute.inputs.map { "\($0.portType.rawValue):\($0.portName)" }.joined(separator: ",") - let outputs = session.currentRoute.outputs.map { "\($0.portType.rawValue):\($0.portName)" }.joined(separator: ",") - let available = session.availableInputs?.map { "\($0.portType.rawValue):\($0.portName)" }.joined(separator: ",") ?? "" - return "category=\(session.category.rawValue) mode=\(session.mode.rawValue) opts=\(session.categoryOptions.rawValue) inputAvail=\(session.isInputAvailable) routeIn=[\(inputs)] routeOut=[\(outputs)] availIn=[\(available)]" + let inputs = session.currentRoute.inputs + .map { "\($0.portType.rawValue):\($0.portName)" } + .joined(separator: ",") + let outputs = session.currentRoute.outputs + .map { "\($0.portType.rawValue):\($0.portName)" } + .joined(separator: ",") + let available = session.availableInputs? + .map { "\($0.portType.rawValue):\($0.portName)" } + .joined(separator: ",") ?? "" + return "category=\(session.category.rawValue) mode=\(session.mode.rawValue) " + + "opts=\(session.categoryOptions.rawValue) inputAvail=\(session.isInputAvailable) " + + "routeIn=[\(inputs)] routeOut=[\(outputs)] availIn=[\(available)]" } } @@ -1829,7 +2091,9 @@ private final class AudioTapDiagnostics: @unchecked Sendable { guard shouldLog else { return } GatewayDiagnostics.log( - "\(label) mic: buffers=\(count) frames=\(frames) rate=\(Int(rate))Hz ch=\(ch) rms=\(String(format: "%.4f", resolvedRms)) max=\(String(format: "%.4f", maxRms))") + "\(label) mic: buffers=\(count) frames=\(frames) rate=\(Int(rate))Hz ch=\(ch) " + + "rms=\(String(format: "%.4f", resolvedRms)) max=\(String(format: "%.4f", maxRms))" + ) } } @@ -1862,7 +2126,7 @@ extension TalkModeManager { } #endif -private struct IncrementalSpeechContext { +private struct IncrementalSpeechContext: Equatable { let apiKey: String? let voiceId: String? let modelId: String? @@ -1872,4 +2136,18 @@ private struct IncrementalSpeechContext { let canUseElevenLabs: Bool } -// swiftlint:enable type_body_length +private struct IncrementalSpeechPrefetchState { + let id: UUID + let segment: String + let context: IncrementalSpeechContext + let outputFormat: String? + var chunks: [Data]? + let task: Task +} + +private struct IncrementalPrefetchedAudio { + let chunks: [Data] + let outputFormat: String? +} + +// swiftlint:enable type_body_length file_length diff --git a/apps/ios/SwiftSources.input.xcfilelist b/apps/ios/SwiftSources.input.xcfilelist index 5b1ba7d70e6..514ca732673 100644 --- a/apps/ios/SwiftSources.input.xcfilelist +++ b/apps/ios/SwiftSources.input.xcfilelist @@ -4,6 +4,9 @@ Sources/Gateway/GatewayDiscoveryModel.swift Sources/Gateway/GatewaySettingsStore.swift Sources/Gateway/KeychainStore.swift Sources/Camera/CameraController.swift +Sources/Device/DeviceInfoHelper.swift +Sources/Device/DeviceStatusService.swift +Sources/Device/NetworkStatusService.swift Sources/Chat/ChatSheet.swift Sources/Chat/IOSGatewayChatTransport.swift Sources/OpenClawApp.swift diff --git a/apps/ios/Tests/DeepLinkParserTests.swift b/apps/ios/Tests/DeepLinkParserTests.swift index ea8b2a81203..51ef9547a10 100644 --- a/apps/ios/Tests/DeepLinkParserTests.swift +++ b/apps/ios/Tests/DeepLinkParserTests.swift @@ -85,6 +85,18 @@ import Testing .init(host: "openclaw.local", port: 18789, tls: true, token: "abc", password: "def"))) } + @Test func parseGatewayLinkRejectsInsecureNonLoopbackWs() { + let url = URL( + string: "openclaw://gateway?host=attacker.example&port=18789&tls=0&token=abc")! + #expect(DeepLinkParser.parse(url) == nil) + } + + @Test func parseGatewayLinkRejectsInsecurePrefixBypassHost() { + let url = URL( + string: "openclaw://gateway?host=127.attacker.example&port=18789&tls=0&token=abc")! + #expect(DeepLinkParser.parse(url) == nil) + } + @Test func parseGatewaySetupCodeParsesBase64UrlPayload() { let payload = #"{"url":"wss://gateway.example.com:443","token":"tok","password":"pw"}"# let encoded = Data(payload.utf8) @@ -124,4 +136,46 @@ import Testing token: "tok", password: nil)) } + + @Test func parseGatewaySetupCodeRejectsInsecureNonLoopbackWs() { + let payload = #"{"url":"ws://attacker.example:18789","token":"tok"}"# + let encoded = Data(payload.utf8) + .base64EncodedString() + .replacingOccurrences(of: "+", with: "-") + .replacingOccurrences(of: "/", with: "_") + .replacingOccurrences(of: "=", with: "") + + let link = GatewayConnectDeepLink.fromSetupCode(encoded) + #expect(link == nil) + } + + @Test func parseGatewaySetupCodeRejectsInsecurePrefixBypassHost() { + let payload = #"{"url":"ws://127.attacker.example:18789","token":"tok"}"# + let encoded = Data(payload.utf8) + .base64EncodedString() + .replacingOccurrences(of: "+", with: "-") + .replacingOccurrences(of: "/", with: "_") + .replacingOccurrences(of: "=", with: "") + + let link = GatewayConnectDeepLink.fromSetupCode(encoded) + #expect(link == nil) + } + + @Test func parseGatewaySetupCodeAllowsLoopbackWs() { + let payload = #"{"url":"ws://127.0.0.1:18789","token":"tok"}"# + let encoded = Data(payload.utf8) + .base64EncodedString() + .replacingOccurrences(of: "+", with: "-") + .replacingOccurrences(of: "/", with: "_") + .replacingOccurrences(of: "=", with: "") + + let link = GatewayConnectDeepLink.fromSetupCode(encoded) + + #expect(link == .init( + host: "127.0.0.1", + port: 18789, + tls: false, + token: "tok", + password: nil)) + } } diff --git a/apps/ios/Tests/GatewayConnectionSecurityTests.swift b/apps/ios/Tests/GatewayConnectionSecurityTests.swift index 066ccb1dd22..3c1b25bce07 100644 --- a/apps/ios/Tests/GatewayConnectionSecurityTests.swift +++ b/apps/ios/Tests/GatewayConnectionSecurityTests.swift @@ -1,5 +1,6 @@ import Foundation import Network +import OpenClawKit import Testing @testable import OpenClaw @@ -102,4 +103,30 @@ import Testing #expect(controller._test_didAutoConnect() == false) } + + @Test @MainActor func manualConnectionsForceTLSForNonLoopbackHosts() async { + let appModel = NodeAppModel() + let controller = GatewayConnectionController(appModel: appModel, startDiscovery: false) + + #expect(controller._test_resolveManualUseTLS(host: "gateway.example.com", useTLS: false) == true) + #expect(controller._test_resolveManualUseTLS(host: "openclaw.local", useTLS: false) == true) + #expect(controller._test_resolveManualUseTLS(host: "127.attacker.example", useTLS: false) == true) + + #expect(controller._test_resolveManualUseTLS(host: "localhost", useTLS: false) == false) + #expect(controller._test_resolveManualUseTLS(host: "127.0.0.1", useTLS: false) == false) + #expect(controller._test_resolveManualUseTLS(host: "::1", useTLS: false) == false) + #expect(controller._test_resolveManualUseTLS(host: "[::1]", useTLS: false) == false) + #expect(controller._test_resolveManualUseTLS(host: "::ffff:127.0.0.1", useTLS: false) == false) + #expect(controller._test_resolveManualUseTLS(host: "0.0.0.0", useTLS: false) == false) + } + + @Test @MainActor func manualDefaultPortUses443OnlyForTailnetTLSHosts() async { + let appModel = NodeAppModel() + let controller = GatewayConnectionController(appModel: appModel, startDiscovery: false) + + #expect(controller._test_resolveManualPort(host: "gateway.example.com", port: 0, useTLS: true) == 18789) + #expect(controller._test_resolveManualPort(host: "device.sample.ts.net", port: 0, useTLS: true) == 443) + #expect(controller._test_resolveManualPort(host: "device.sample.ts.net.", port: 0, useTLS: true) == 443) + #expect(controller._test_resolveManualPort(host: "device.sample.ts.net", port: 18789, useTLS: true) == 18789) + } } diff --git a/apps/ios/Tests/GatewaySettingsStoreTests.swift b/apps/ios/Tests/GatewaySettingsStoreTests.swift index 7e67ab84a97..0bac4015236 100644 --- a/apps/ios/Tests/GatewaySettingsStoreTests.swift +++ b/apps/ios/Tests/GatewaySettingsStoreTests.swift @@ -9,9 +9,11 @@ private struct KeychainEntry: Hashable { private let gatewayService = "ai.openclaw.gateway" private let nodeService = "ai.openclaw.node" +private let talkService = "ai.openclaw.talk" private let instanceIdEntry = KeychainEntry(service: nodeService, account: "instanceId") private let preferredGatewayEntry = KeychainEntry(service: gatewayService, account: "preferredStableID") private let lastGatewayEntry = KeychainEntry(service: gatewayService, account: "lastDiscoveredStableID") +private let talkAcmeProviderEntry = KeychainEntry(service: talkService, account: "provider.apiKey.acme") private func snapshotDefaults(_ keys: [String]) -> [String: Any?] { let defaults = UserDefaults.standard @@ -196,4 +198,17 @@ private func restoreKeychain(_ snapshot: [KeychainEntry: String?]) { let loaded = GatewaySettingsStore.loadLastGatewayConnection() #expect(loaded == .manual(host: "example.org", port: 18789, useTLS: false, stableID: "manual|example.org|18789")) } + + @Test func talkProviderApiKey_genericRoundTrip() { + let keychainSnapshot = snapshotKeychain([talkAcmeProviderEntry]) + defer { restoreKeychain(keychainSnapshot) } + + _ = KeychainStore.delete(service: talkService, account: talkAcmeProviderEntry.account) + + GatewaySettingsStore.saveTalkProviderApiKey("acme-key", provider: "acme") + #expect(GatewaySettingsStore.loadTalkProviderApiKey(provider: "acme") == "acme-key") + + GatewaySettingsStore.saveTalkProviderApiKey(nil, provider: "acme") + #expect(GatewaySettingsStore.loadTalkProviderApiKey(provider: "acme") == nil) + } } diff --git a/apps/ios/Tests/Info.plist b/apps/ios/Tests/Info.plist index e061595cb30..837a98bfcdc 100644 --- a/apps/ios/Tests/Info.plist +++ b/apps/ios/Tests/Info.plist @@ -17,8 +17,8 @@ CFBundlePackageType BNDL CFBundleShortVersionString - 2026.2.18 + 2026.2.27 CFBundleVersion - 20260218 + 20260227 diff --git a/apps/ios/Tests/KeychainStoreTests.swift b/apps/ios/Tests/KeychainStoreTests.swift index 827be250ed7..e56f4aa35b5 100644 --- a/apps/ios/Tests/KeychainStoreTests.swift +++ b/apps/ios/Tests/KeychainStoreTests.swift @@ -4,7 +4,7 @@ import Testing @Suite struct KeychainStoreTests { @Test func saveLoadUpdateDeleteRoundTrip() { - let service = "bot.molt.tests.\(UUID().uuidString)" + let service = "ai.openclaw.tests.\(UUID().uuidString)" let account = "value" #expect(KeychainStore.delete(service: service, account: account)) diff --git a/apps/ios/Tests/NodeAppModelInvokeTests.swift b/apps/ios/Tests/NodeAppModelInvokeTests.swift index f5f40fc8b7c..dbeee118a4a 100644 --- a/apps/ios/Tests/NodeAppModelInvokeTests.swift +++ b/apps/ios/Tests/NodeAppModelInvokeTests.swift @@ -29,8 +29,35 @@ private func withUserDefaults(_ updates: [String: Any?], _ body: () throws -> return try body() } +private func makeAgentDeepLinkURL( + message: String, + deliver: Bool = false, + to: String? = nil, + channel: String? = nil, + key: String? = nil) -> URL +{ + var components = URLComponents() + components.scheme = "openclaw" + components.host = "agent" + var queryItems: [URLQueryItem] = [URLQueryItem(name: "message", value: message)] + if deliver { + queryItems.append(URLQueryItem(name: "deliver", value: "1")) + } + if let to { + queryItems.append(URLQueryItem(name: "to", value: to)) + } + if let channel { + queryItems.append(URLQueryItem(name: "channel", value: channel)) + } + if let key { + queryItems.append(URLQueryItem(name: "key", value: key)) + } + components.queryItems = queryItems + return components.url! +} + @MainActor -private final class MockWatchMessagingService: WatchMessagingServicing, @unchecked Sendable { +private final class MockWatchMessagingService: @preconcurrency WatchMessagingServicing, @unchecked Sendable { var currentStatus = WatchMessagingStatus( supported: true, paired: true, @@ -42,24 +69,28 @@ private final class MockWatchMessagingService: WatchMessagingServicing, @uncheck queuedForDelivery: false, transport: "sendMessage") var sendError: Error? - var lastSent: (id: String, title: String, body: String, priority: OpenClawNotificationPriority?)? + var lastSent: (id: String, params: OpenClawWatchNotifyParams)? + private var replyHandler: (@Sendable (WatchQuickReplyEvent) -> Void)? func status() async -> WatchMessagingStatus { self.currentStatus } - func sendNotification( - id: String, - title: String, - body: String, - priority: OpenClawNotificationPriority?) async throws -> WatchNotificationSendResult - { - self.lastSent = (id: id, title: title, body: body, priority: priority) + func setReplyHandler(_ handler: (@Sendable (WatchQuickReplyEvent) -> Void)?) { + self.replyHandler = handler + } + + func sendNotification(id: String, params: OpenClawWatchNotifyParams) async throws -> WatchNotificationSendResult { + self.lastSent = (id: id, params: params) if let sendError = self.sendError { throw sendError } return self.nextSendResult } + + func emitReply(_ event: WatchQuickReplyEvent) { + self.replyHandler?(event) + } } @Suite(.serialized) struct NodeAppModelInvokeTests { @@ -77,6 +108,19 @@ private final class MockWatchMessagingService: WatchMessagingServicing, @uncheck #expect(json.contains("\"value\"")) } + @Test @MainActor func chatSessionKeyDefaultsToIOSBase() { + let appModel = NodeAppModel() + #expect(appModel.chatSessionKey == "ios") + } + + @Test @MainActor func chatSessionKeyUsesAgentScopedKeyForNonDefaultAgent() { + let appModel = NodeAppModel() + appModel.gatewayDefaultAgentId = "main" + appModel.setSelectedAgentId("agent-123") + #expect(appModel.chatSessionKey == SessionKey.makeAgentSessionKey(agentId: "agent-123", baseKey: "ios")) + #expect(appModel.mainSessionKey == "agent:agent-123:main") + } + @Test @MainActor func handleInvokeRejectsBackgroundCommands() async { let appModel = NodeAppModel() appModel.setScenePhase(.background) @@ -230,9 +274,9 @@ private final class MockWatchMessagingService: WatchMessagingServicing, @uncheck let res = await appModel._test_handleInvoke(req) #expect(res.ok == true) - #expect(watchService.lastSent?.title == "OpenClaw") - #expect(watchService.lastSent?.body == "Meeting with Peter is at 4pm") - #expect(watchService.lastSent?.priority == .timeSensitive) + #expect(watchService.lastSent?.params.title == "OpenClaw") + #expect(watchService.lastSent?.params.body == "Meeting with Peter is at 4pm") + #expect(watchService.lastSent?.params.priority == .timeSensitive) let payloadData = try #require(res.payloadJSON?.data(using: .utf8)) let payload = try JSONDecoder().decode(OpenClawWatchNotifyPayload.self, from: payloadData) @@ -258,6 +302,79 @@ private final class MockWatchMessagingService: WatchMessagingServicing, @uncheck #expect(watchService.lastSent == nil) } + @Test @MainActor func handleInvokeWatchNotifyAddsDefaultActionsForPrompt() async throws { + let watchService = MockWatchMessagingService() + let appModel = NodeAppModel(watchMessagingService: watchService) + let params = OpenClawWatchNotifyParams( + title: "Task", + body: "Action needed", + priority: .passive, + promptId: "prompt-123") + let paramsData = try JSONEncoder().encode(params) + let paramsJSON = String(decoding: paramsData, as: UTF8.self) + let req = BridgeInvokeRequest( + id: "watch-notify-default-actions", + command: OpenClawWatchCommand.notify.rawValue, + paramsJSON: paramsJSON) + + let res = await appModel._test_handleInvoke(req) + #expect(res.ok == true) + #expect(watchService.lastSent?.params.risk == .low) + let actionIDs = watchService.lastSent?.params.actions?.map(\.id) + #expect(actionIDs == ["done", "snooze_10m", "open_phone", "escalate"]) + } + + @Test @MainActor func handleInvokeWatchNotifyAddsApprovalDefaults() async throws { + let watchService = MockWatchMessagingService() + let appModel = NodeAppModel(watchMessagingService: watchService) + let params = OpenClawWatchNotifyParams( + title: "Approval", + body: "Allow command?", + promptId: "prompt-approval", + kind: "approval") + let paramsData = try JSONEncoder().encode(params) + let paramsJSON = String(decoding: paramsData, as: UTF8.self) + let req = BridgeInvokeRequest( + id: "watch-notify-approval-defaults", + command: OpenClawWatchCommand.notify.rawValue, + paramsJSON: paramsJSON) + + let res = await appModel._test_handleInvoke(req) + #expect(res.ok == true) + let actionIDs = watchService.lastSent?.params.actions?.map(\.id) + #expect(actionIDs == ["approve", "decline", "open_phone", "escalate"]) + #expect(watchService.lastSent?.params.actions?[1].style == "destructive") + } + + @Test @MainActor func handleInvokeWatchNotifyDerivesPriorityFromRiskAndCapsActions() async throws { + let watchService = MockWatchMessagingService() + let appModel = NodeAppModel(watchMessagingService: watchService) + let params = OpenClawWatchNotifyParams( + title: "Urgent", + body: "Check now", + risk: .high, + actions: [ + OpenClawWatchAction(id: "a1", label: "A1"), + OpenClawWatchAction(id: "a2", label: "A2"), + OpenClawWatchAction(id: "a3", label: "A3"), + OpenClawWatchAction(id: "a4", label: "A4"), + OpenClawWatchAction(id: "a5", label: "A5"), + ]) + let paramsData = try JSONEncoder().encode(params) + let paramsJSON = String(decoding: paramsData, as: UTF8.self) + let req = BridgeInvokeRequest( + id: "watch-notify-derive-priority", + command: OpenClawWatchCommand.notify.rawValue, + paramsJSON: paramsJSON) + + let res = await appModel._test_handleInvoke(req) + #expect(res.ok == true) + #expect(watchService.lastSent?.params.priority == .timeSensitive) + #expect(watchService.lastSent?.params.risk == .high) + let actionIDs = watchService.lastSent?.params.actions?.map(\.id) + #expect(actionIDs == ["a1", "a2", "a3", "a4"]) + } + @Test @MainActor func handleInvokeWatchNotifyReturnsUnavailableOnDeliveryFailure() async throws { let watchService = MockWatchMessagingService() watchService.sendError = NSError( @@ -279,6 +396,22 @@ private final class MockWatchMessagingService: WatchMessagingServicing, @uncheck #expect(res.error?.message.contains("WATCH_UNAVAILABLE") == true) } + @Test @MainActor func watchReplyQueuesWhenGatewayOffline() async { + let watchService = MockWatchMessagingService() + let appModel = NodeAppModel(watchMessagingService: watchService) + watchService.emitReply( + WatchQuickReplyEvent( + replyId: "reply-offline-1", + promptId: "prompt-1", + actionId: "approve", + actionLabel: "Approve", + sessionKey: "ios", + note: nil, + sentAtMs: 1234, + transport: "transferUserInfo")) + #expect(appModel._test_queuedWatchReplyCount() == 1) + } + @Test @MainActor func handleDeepLinkSetsErrorWhenNotConnected() async { let appModel = NodeAppModel() let url = URL(string: "openclaw://agent?message=hello")! @@ -294,6 +427,58 @@ private final class MockWatchMessagingService: WatchMessagingServicing, @uncheck #expect(appModel.screen.errorText?.contains("Deep link too large") == true) } + @Test @MainActor func handleDeepLinkRequiresConfirmationWhenConnectedAndUnkeyed() async { + let appModel = NodeAppModel() + appModel._test_setGatewayConnected(true) + let url = makeAgentDeepLinkURL(message: "hello from deep link") + + await appModel.handleDeepLink(url: url) + #expect(appModel.pendingAgentDeepLinkPrompt != nil) + #expect(appModel.openChatRequestID == 0) + + await appModel.approvePendingAgentDeepLinkPrompt() + #expect(appModel.pendingAgentDeepLinkPrompt == nil) + #expect(appModel.openChatRequestID == 1) + } + + @Test @MainActor func handleDeepLinkStripsDeliveryFieldsWhenUnkeyed() async throws { + let appModel = NodeAppModel() + appModel._test_setGatewayConnected(true) + let url = makeAgentDeepLinkURL( + message: "route this", + deliver: true, + to: "123456", + channel: "telegram") + + await appModel.handleDeepLink(url: url) + let prompt = try #require(appModel.pendingAgentDeepLinkPrompt) + #expect(prompt.request.deliver == false) + #expect(prompt.request.to == nil) + #expect(prompt.request.channel == nil) + } + + @Test @MainActor func handleDeepLinkRejectsLongUnkeyedMessageWhenConnected() async { + let appModel = NodeAppModel() + appModel._test_setGatewayConnected(true) + let message = String(repeating: "x", count: 241) + let url = makeAgentDeepLinkURL(message: message) + + await appModel.handleDeepLink(url: url) + #expect(appModel.pendingAgentDeepLinkPrompt == nil) + #expect(appModel.screen.errorText?.contains("blocked") == true) + } + + @Test @MainActor func handleDeepLinkBypassesPromptWithValidKey() async { + let appModel = NodeAppModel() + appModel._test_setGatewayConnected(true) + let key = NodeAppModel._test_currentDeepLinkKey() + let url = makeAgentDeepLinkURL(message: "trusted request", key: key) + + await appModel.handleDeepLink(url: url) + #expect(appModel.pendingAgentDeepLinkPrompt == nil) + #expect(appModel.openChatRequestID == 1) + } + @Test @MainActor func sendVoiceTranscriptThrowsWhenGatewayOffline() async { let appModel = NodeAppModel() await #expect(throws: Error.self) { diff --git a/apps/ios/Tests/TalkModeConfigParsingTests.swift b/apps/ios/Tests/TalkModeConfigParsingTests.swift new file mode 100644 index 00000000000..fd6b535f8a3 --- /dev/null +++ b/apps/ios/Tests/TalkModeConfigParsingTests.swift @@ -0,0 +1,31 @@ +import Testing +@testable import OpenClaw + +@MainActor +@Suite struct TalkModeConfigParsingTests { + @Test func prefersNormalizedTalkProviderPayload() { + let talk: [String: Any] = [ + "provider": "elevenlabs", + "providers": [ + "elevenlabs": [ + "voiceId": "voice-normalized", + ], + ], + "voiceId": "voice-legacy", + ] + + let selection = TalkModeManager.selectTalkProviderConfig(talk) + #expect(selection?.provider == "elevenlabs") + #expect(selection?.config["voiceId"] as? String == "voice-normalized") + } + + @Test func ignoresLegacyTalkFieldsWhenNormalizedPayloadMissing() { + let talk: [String: Any] = [ + "voiceId": "voice-legacy", + "apiKey": "legacy-key", + ] + + let selection = TalkModeManager.selectTalkProviderConfig(talk) + #expect(selection == nil) + } +} diff --git a/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/Contents.json b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/Contents.json new file mode 100644 index 00000000000..a72027bb4fa --- /dev/null +++ b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/Contents.json @@ -0,0 +1,116 @@ +{ + "images": [ + { + "idiom": "watch", + "role": "notificationCenter", + "subtype": "38mm", + "size": "24x24", + "scale": "2x", + "filename": "watch-notification-38@2x.png" + }, + { + "idiom": "watch", + "role": "notificationCenter", + "subtype": "42mm", + "size": "27.5x27.5", + "scale": "2x", + "filename": "watch-notification-42@2x.png" + }, + { + "idiom": "watch", + "role": "companionSettings", + "size": "29x29", + "scale": "2x", + "filename": "watch-companion-29@2x.png" + }, + { + "idiom": "watch", + "role": "companionSettings", + "size": "29x29", + "scale": "3x", + "filename": "watch-companion-29@3x.png" + }, + { + "idiom": "watch", + "role": "appLauncher", + "subtype": "38mm", + "size": "40x40", + "scale": "2x", + "filename": "watch-app-38@2x.png" + }, + { + "idiom": "watch", + "role": "appLauncher", + "subtype": "40mm", + "size": "44x44", + "scale": "2x", + "filename": "watch-app-40@2x.png" + }, + { + "idiom": "watch", + "role": "appLauncher", + "subtype": "41mm", + "size": "46x46", + "scale": "2x", + "filename": "watch-app-41@2x.png" + }, + { + "idiom": "watch", + "role": "appLauncher", + "subtype": "44mm", + "size": "50x50", + "scale": "2x", + "filename": "watch-app-44@2x.png" + }, + { + "idiom": "watch", + "role": "appLauncher", + "subtype": "45mm", + "size": "51x51", + "scale": "2x", + "filename": "watch-app-45@2x.png" + }, + { + "idiom": "watch", + "role": "quickLook", + "subtype": "38mm", + "size": "86x86", + "scale": "2x", + "filename": "watch-quicklook-38@2x.png" + }, + { + "idiom": "watch", + "role": "quickLook", + "subtype": "42mm", + "size": "98x98", + "scale": "2x", + "filename": "watch-quicklook-42@2x.png" + }, + { + "idiom": "watch", + "role": "quickLook", + "subtype": "44mm", + "size": "108x108", + "scale": "2x", + "filename": "watch-quicklook-44@2x.png" + }, + { + "idiom": "watch", + "role": "quickLook", + "subtype": "45mm", + "size": "117x117", + "scale": "2x", + "filename": "watch-quicklook-45@2x.png" + }, + { + "idiom": "watch-marketing", + "size": "1024x1024", + "scale": "1x", + "filename": "watch-marketing-1024.png" + } + ], + "info": { + "version": 1, + "author": "xcode" + } +} diff --git a/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-app-38@2x.png b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-app-38@2x.png new file mode 100644 index 00000000000..82829afb947 Binary files /dev/null and b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-app-38@2x.png differ diff --git a/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-app-40@2x.png b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-app-40@2x.png new file mode 100644 index 00000000000..114d4606420 Binary files /dev/null and b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-app-40@2x.png differ diff --git a/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-app-41@2x.png b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-app-41@2x.png new file mode 100644 index 00000000000..5f9578b1b97 Binary files /dev/null and b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-app-41@2x.png differ diff --git a/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-app-44@2x.png b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-app-44@2x.png new file mode 100644 index 00000000000..fe022ac7720 Binary files /dev/null and b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-app-44@2x.png differ diff --git a/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-app-45@2x.png b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-app-45@2x.png new file mode 100644 index 00000000000..55977b8f6e7 Binary files /dev/null and b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-app-45@2x.png differ diff --git a/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-companion-29@2x.png b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-companion-29@2x.png new file mode 100644 index 00000000000..f8be7d06911 Binary files /dev/null and b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-companion-29@2x.png differ diff --git a/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-companion-29@3x.png b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-companion-29@3x.png new file mode 100644 index 00000000000..cce412d2452 Binary files /dev/null and b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-companion-29@3x.png differ diff --git a/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-marketing-1024.png b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-marketing-1024.png new file mode 100644 index 00000000000..005486f2ee1 Binary files /dev/null and b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-marketing-1024.png differ diff --git a/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-notification-38@2x.png b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-notification-38@2x.png new file mode 100644 index 00000000000..7b7a0ee0b65 Binary files /dev/null and b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-notification-38@2x.png differ diff --git a/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-notification-42@2x.png b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-notification-42@2x.png new file mode 100644 index 00000000000..f13c9cdddda Binary files /dev/null and b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-notification-42@2x.png differ diff --git a/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-quicklook-38@2x.png b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-quicklook-38@2x.png new file mode 100644 index 00000000000..aac0859b44c Binary files /dev/null and b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-quicklook-38@2x.png differ diff --git a/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-quicklook-42@2x.png b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-quicklook-42@2x.png new file mode 100644 index 00000000000..d09be6e98a6 Binary files /dev/null and b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-quicklook-42@2x.png differ diff --git a/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-quicklook-44@2x.png b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-quicklook-44@2x.png new file mode 100644 index 00000000000..5b06a48744b Binary files /dev/null and b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-quicklook-44@2x.png differ diff --git a/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-quicklook-45@2x.png b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-quicklook-45@2x.png new file mode 100644 index 00000000000..72ba51ebb1d Binary files /dev/null and b/apps/ios/WatchApp/Assets.xcassets/AppIcon.appiconset/watch-quicklook-45@2x.png differ diff --git a/apps/ios/WatchApp/Assets.xcassets/Contents.json b/apps/ios/WatchApp/Assets.xcassets/Contents.json new file mode 100644 index 00000000000..97a8662ebdb --- /dev/null +++ b/apps/ios/WatchApp/Assets.xcassets/Contents.json @@ -0,0 +1,6 @@ +{ + "info": { + "version": 1, + "author": "xcode" + } +} diff --git a/apps/ios/WatchApp/Info.plist b/apps/ios/WatchApp/Info.plist index 2c1d88f30a7..de2f0d92c2a 100644 --- a/apps/ios/WatchApp/Info.plist +++ b/apps/ios/WatchApp/Info.plist @@ -17,9 +17,9 @@ CFBundlePackageType APPL CFBundleShortVersionString - 2026.2.18 + 2026.2.27 CFBundleVersion - 20260218 + 20260227 WKCompanionAppBundleIdentifier $(OPENCLAW_APP_BUNDLE_ID) WKWatchKitApp diff --git a/apps/ios/WatchExtension/Info.plist b/apps/ios/WatchExtension/Info.plist index e401494d225..578f7b101fa 100644 --- a/apps/ios/WatchExtension/Info.plist +++ b/apps/ios/WatchExtension/Info.plist @@ -15,9 +15,9 @@ CFBundleName $(PRODUCT_NAME) CFBundleShortVersionString - 2026.2.18 + 2026.2.27 CFBundleVersion - 20260218 + 20260227 NSExtension NSExtensionAttributes diff --git a/apps/ios/WatchExtension/Sources/OpenClawWatchApp.swift b/apps/ios/WatchExtension/Sources/OpenClawWatchApp.swift index 6084f574442..4c123c49f16 100644 --- a/apps/ios/WatchExtension/Sources/OpenClawWatchApp.swift +++ b/apps/ios/WatchExtension/Sources/OpenClawWatchApp.swift @@ -7,7 +7,15 @@ struct OpenClawWatchApp: App { var body: some Scene { WindowGroup { - WatchInboxView(store: self.inboxStore) + WatchInboxView(store: self.inboxStore) { action in + guard let receiver = self.receiver else { return } + let draft = self.inboxStore.makeReplyDraft(action: action) + self.inboxStore.markReplySending(actionLabel: action.label) + Task { @MainActor in + let result = await receiver.sendReply(draft) + self.inboxStore.markReplyResult(result, actionLabel: action.label) + } + } .task { if self.receiver == nil { let receiver = WatchConnectivityReceiver(store: self.inboxStore) diff --git a/apps/ios/WatchExtension/Sources/WatchConnectivityReceiver.swift b/apps/ios/WatchExtension/Sources/WatchConnectivityReceiver.swift index fd0d84cc55c..da1c3c379a3 100644 --- a/apps/ios/WatchExtension/Sources/WatchConnectivityReceiver.swift +++ b/apps/ios/WatchExtension/Sources/WatchConnectivityReceiver.swift @@ -1,6 +1,23 @@ import Foundation import WatchConnectivity +struct WatchReplyDraft: Sendable { + var replyId: String + var promptId: String + var actionId: String + var actionLabel: String? + var sessionKey: String? + var note: String? + var sentAtMs: Int +} + +struct WatchReplySendResult: Sendable, Equatable { + var deliveredImmediately: Bool + var queuedForDelivery: Bool + var transport: String + var errorMessage: String? +} + final class WatchConnectivityReceiver: NSObject, @unchecked Sendable { private let store: WatchInboxStore private let session: WCSession? @@ -21,6 +38,114 @@ final class WatchConnectivityReceiver: NSObject, @unchecked Sendable { session.activate() } + private func ensureActivated() async { + guard let session = self.session else { return } + if session.activationState == .activated { + return + } + session.activate() + for _ in 0..<8 { + if session.activationState == .activated { + return + } + try? await Task.sleep(nanoseconds: 100_000_000) + } + } + + func sendReply(_ draft: WatchReplyDraft) async -> WatchReplySendResult { + await self.ensureActivated() + guard let session = self.session else { + return WatchReplySendResult( + deliveredImmediately: false, + queuedForDelivery: false, + transport: "none", + errorMessage: "watch session unavailable") + } + + var payload: [String: Any] = [ + "type": "watch.reply", + "replyId": draft.replyId, + "promptId": draft.promptId, + "actionId": draft.actionId, + "sentAtMs": draft.sentAtMs, + ] + if let actionLabel = draft.actionLabel?.trimmingCharacters(in: .whitespacesAndNewlines), + !actionLabel.isEmpty + { + payload["actionLabel"] = actionLabel + } + if let sessionKey = draft.sessionKey?.trimmingCharacters(in: .whitespacesAndNewlines), + !sessionKey.isEmpty + { + payload["sessionKey"] = sessionKey + } + if let note = draft.note?.trimmingCharacters(in: .whitespacesAndNewlines), !note.isEmpty { + payload["note"] = note + } + + if session.isReachable { + do { + try await withCheckedThrowingContinuation { continuation in + session.sendMessage(payload, replyHandler: { _ in + continuation.resume() + }, errorHandler: { error in + continuation.resume(throwing: error) + }) + } + return WatchReplySendResult( + deliveredImmediately: true, + queuedForDelivery: false, + transport: "sendMessage", + errorMessage: nil) + } catch { + // Fall through to queued delivery below. + } + } + + _ = session.transferUserInfo(payload) + return WatchReplySendResult( + deliveredImmediately: false, + queuedForDelivery: true, + transport: "transferUserInfo", + errorMessage: nil) + } + + private static func normalizeObject(_ value: Any) -> [String: Any]? { + if let object = value as? [String: Any] { + return object + } + if let object = value as? [AnyHashable: Any] { + var normalized: [String: Any] = [:] + normalized.reserveCapacity(object.count) + for (key, item) in object { + guard let stringKey = key as? String else { + continue + } + normalized[stringKey] = item + } + return normalized + } + return nil + } + + private static func parseActions(_ value: Any?) -> [WatchPromptAction] { + guard let raw = value as? [Any] else { + return [] + } + return raw.compactMap { item in + guard let obj = Self.normalizeObject(item) else { + return nil + } + let id = (obj["id"] as? String)?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + let label = (obj["label"] as? String)?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + guard !id.isEmpty, !label.isEmpty else { + return nil + } + let style = (obj["style"] as? String)?.trimmingCharacters(in: .whitespacesAndNewlines) + return WatchPromptAction(id: id, label: label, style: style) + } + } + private static func parseNotificationPayload(_ payload: [String: Any]) -> WatchNotifyMessage? { guard let type = payload["type"] as? String, type == "watch.notify" else { return nil @@ -38,12 +163,31 @@ final class WatchConnectivityReceiver: NSObject, @unchecked Sendable { let id = (payload["id"] as? String)? .trimmingCharacters(in: .whitespacesAndNewlines) let sentAtMs = (payload["sentAtMs"] as? Int) ?? (payload["sentAtMs"] as? NSNumber)?.intValue + let promptId = (payload["promptId"] as? String)? + .trimmingCharacters(in: .whitespacesAndNewlines) + let sessionKey = (payload["sessionKey"] as? String)? + .trimmingCharacters(in: .whitespacesAndNewlines) + let kind = (payload["kind"] as? String)? + .trimmingCharacters(in: .whitespacesAndNewlines) + let details = (payload["details"] as? String)? + .trimmingCharacters(in: .whitespacesAndNewlines) + let expiresAtMs = (payload["expiresAtMs"] as? Int) ?? (payload["expiresAtMs"] as? NSNumber)?.intValue + let risk = (payload["risk"] as? String)? + .trimmingCharacters(in: .whitespacesAndNewlines) + let actions = Self.parseActions(payload["actions"]) return WatchNotifyMessage( id: id, title: title, body: body, - sentAtMs: sentAtMs) + sentAtMs: sentAtMs, + promptId: promptId, + sessionKey: sessionKey, + kind: kind, + details: details, + expiresAtMs: expiresAtMs, + risk: risk, + actions: actions) } } diff --git a/apps/ios/WatchExtension/Sources/WatchInboxStore.swift b/apps/ios/WatchExtension/Sources/WatchInboxStore.swift index 0a715f16b63..2ac1d75d6e1 100644 --- a/apps/ios/WatchExtension/Sources/WatchInboxStore.swift +++ b/apps/ios/WatchExtension/Sources/WatchInboxStore.swift @@ -3,11 +3,24 @@ import Observation import UserNotifications import WatchKit +struct WatchPromptAction: Codable, Sendable, Equatable, Identifiable { + var id: String + var label: String + var style: String? +} + struct WatchNotifyMessage: Sendable { var id: String? var title: String var body: String var sentAtMs: Int? + var promptId: String? + var sessionKey: String? + var kind: String? + var details: String? + var expiresAtMs: Int? + var risk: String? + var actions: [WatchPromptAction] } @MainActor @Observable final class WatchInboxStore { @@ -17,6 +30,15 @@ struct WatchNotifyMessage: Sendable { var transport: String var updatedAt: Date var lastDeliveryKey: String? + var promptId: String? + var sessionKey: String? + var kind: String? + var details: String? + var expiresAtMs: Int? + var risk: String? + var actions: [WatchPromptAction]? + var replyStatusText: String? + var replyStatusAt: Date? } private static let persistedStateKey = "watch.inbox.state.v1" @@ -26,6 +48,16 @@ struct WatchNotifyMessage: Sendable { var body = "Waiting for messages from your iPhone." var transport = "none" var updatedAt: Date? + var promptId: String? + var sessionKey: String? + var kind: String? + var details: String? + var expiresAtMs: Int? + var risk: String? + var actions: [WatchPromptAction] = [] + var replyStatusText: String? + var replyStatusAt: Date? + var isReplySending = false private var lastDeliveryKey: String? init(defaults: UserDefaults = .standard) { @@ -51,14 +83,25 @@ struct WatchNotifyMessage: Sendable { self.body = message.body self.transport = transport self.updatedAt = Date() + self.promptId = message.promptId + self.sessionKey = message.sessionKey + self.kind = message.kind + self.details = message.details + self.expiresAtMs = message.expiresAtMs + self.risk = message.risk + self.actions = message.actions self.lastDeliveryKey = deliveryKey + self.replyStatusText = nil + self.replyStatusAt = nil + self.isReplySending = false self.persistState() Task { await self.postLocalNotification( identifier: deliveryKey, title: normalizedTitle, - body: message.body) + body: message.body, + risk: message.risk) } } @@ -74,6 +117,15 @@ struct WatchNotifyMessage: Sendable { self.transport = state.transport self.updatedAt = state.updatedAt self.lastDeliveryKey = state.lastDeliveryKey + self.promptId = state.promptId + self.sessionKey = state.sessionKey + self.kind = state.kind + self.details = state.details + self.expiresAtMs = state.expiresAtMs + self.risk = state.risk + self.actions = state.actions ?? [] + self.replyStatusText = state.replyStatusText + self.replyStatusAt = state.replyStatusAt } private func persistState() { @@ -83,7 +135,16 @@ struct WatchNotifyMessage: Sendable { body: self.body, transport: self.transport, updatedAt: updatedAt, - lastDeliveryKey: self.lastDeliveryKey) + lastDeliveryKey: self.lastDeliveryKey, + promptId: self.promptId, + sessionKey: self.sessionKey, + kind: self.kind, + details: self.details, + expiresAtMs: self.expiresAtMs, + risk: self.risk, + actions: self.actions, + replyStatusText: self.replyStatusText, + replyStatusAt: self.replyStatusAt) guard let data = try? JSONEncoder().encode(state) else { return } self.defaults.set(data, forKey: Self.persistedStateKey) } @@ -106,7 +167,52 @@ struct WatchNotifyMessage: Sendable { } } - private func postLocalNotification(identifier: String, title: String, body: String) async { + private func mapHapticRisk(_ risk: String?) -> WKHapticType { + switch risk?.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() { + case "high": + return .failure + case "medium": + return .notification + default: + return .click + } + } + + func makeReplyDraft(action: WatchPromptAction) -> WatchReplyDraft { + let prompt = self.promptId?.trimmingCharacters(in: .whitespacesAndNewlines) + return WatchReplyDraft( + replyId: UUID().uuidString, + promptId: (prompt?.isEmpty == false) ? prompt! : "unknown", + actionId: action.id, + actionLabel: action.label, + sessionKey: self.sessionKey, + note: nil, + sentAtMs: Int(Date().timeIntervalSince1970 * 1000)) + } + + func markReplySending(actionLabel: String) { + self.isReplySending = true + self.replyStatusText = "Sending \(actionLabel)…" + self.replyStatusAt = Date() + self.persistState() + } + + func markReplyResult(_ result: WatchReplySendResult, actionLabel: String) { + self.isReplySending = false + if let errorMessage = result.errorMessage, !errorMessage.isEmpty { + self.replyStatusText = "Failed: \(errorMessage)" + } else if result.deliveredImmediately { + self.replyStatusText = "\(actionLabel): sent" + } else if result.queuedForDelivery { + self.replyStatusText = "\(actionLabel): queued" + } else { + self.replyStatusText = "\(actionLabel): sent" + } + self.replyStatusAt = Date() + self.persistState() + } + + private func postLocalNotification(identifier: String, title: String, body: String, risk: String?) async { let content = UNMutableNotificationContent() content.title = title content.body = body @@ -119,6 +225,6 @@ struct WatchNotifyMessage: Sendable { trigger: UNTimeIntervalNotificationTrigger(timeInterval: 0.2, repeats: false)) _ = try? await UNUserNotificationCenter.current().add(request) - WKInterfaceDevice.current().play(.notification) + WKInterfaceDevice.current().play(self.mapHapticRisk(risk)) } } diff --git a/apps/ios/WatchExtension/Sources/WatchInboxView.swift b/apps/ios/WatchExtension/Sources/WatchInboxView.swift index c5ea9a9f534..c6f944a949e 100644 --- a/apps/ios/WatchExtension/Sources/WatchInboxView.swift +++ b/apps/ios/WatchExtension/Sources/WatchInboxView.swift @@ -2,6 +2,18 @@ import SwiftUI struct WatchInboxView: View { @Bindable var store: WatchInboxStore + var onAction: ((WatchPromptAction) -> Void)? + + private func role(for action: WatchPromptAction) -> ButtonRole? { + switch action.style?.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() { + case "destructive": + return .destructive + case "cancel": + return .cancel + default: + return nil + } + } var body: some View { ScrollView { @@ -14,6 +26,31 @@ struct WatchInboxView: View { .font(.body) .fixedSize(horizontal: false, vertical: true) + if let details = store.details, !details.isEmpty { + Text(details) + .font(.footnote) + .foregroundStyle(.secondary) + .fixedSize(horizontal: false, vertical: true) + } + + if !store.actions.isEmpty { + ForEach(store.actions) { action in + Button(role: self.role(for: action)) { + self.onAction?(action) + } label: { + Text(action.label) + .frame(maxWidth: .infinity) + } + .disabled(store.isReplySending) + } + } + + if let replyStatusText = store.replyStatusText, !replyStatusText.isEmpty { + Text(replyStatusText) + .font(.footnote) + .foregroundStyle(.secondary) + } + if let updatedAt = store.updatedAt { Text("Updated \(updatedAt.formatted(date: .omitted, time: .shortened))") .font(.footnote) diff --git a/apps/ios/fastlane/Appfile b/apps/ios/fastlane/Appfile index adaa3fc29fb..8dbb75a8c26 100644 --- a/apps/ios/fastlane/Appfile +++ b/apps/ios/fastlane/Appfile @@ -1,4 +1,4 @@ -app_identifier("bot.molt.ios") +app_identifier("ai.openclaw.ios") # Auth is expected via App Store Connect API key. # Provide either: diff --git a/apps/ios/project.yml b/apps/ios/project.yml index cd543c59430..e2bf3c6594a 100644 --- a/apps/ios/project.yml +++ b/apps/ios/project.yml @@ -92,14 +92,16 @@ targets: - CFBundleURLName: ai.openclaw.ios CFBundleURLSchemes: - openclaw - CFBundleShortVersionString: "2026.2.18" - CFBundleVersion: "20260218" + CFBundleShortVersionString: "2026.2.27" + CFBundleVersion: "20260227" UILaunchScreen: {} UIApplicationSceneManifest: UIApplicationSupportsMultipleScenes: false UIBackgroundModes: - audio - remote-notification + BGTaskSchedulerPermittedIdentifiers: + - ai.openclaw.ios.bgrefresh NSLocalNetworkUsageDescription: OpenClaw discovers and connects to your OpenClaw gateway on the local network. NSAppTransportSecurity: NSAllowsArbitraryLoadsInWebContent: true @@ -131,11 +133,13 @@ targets: - path: ShareExtension dependencies: - package: OpenClawKit + - sdk: AppIntents.framework settings: base: CODE_SIGN_IDENTITY: "Apple Development" CODE_SIGN_STYLE: "$(OPENCLAW_CODE_SIGN_STYLE)" DEVELOPMENT_TEAM: "$(OPENCLAW_DEVELOPMENT_TEAM)" + ENABLE_APPINTENTS_METADATA: NO PRODUCT_BUNDLE_IDENTIFIER: "$(OPENCLAW_SHARE_BUNDLE_ID)" PROVISIONING_PROFILE_SPECIFIER: "$(OPENCLAW_SHARE_PROFILE)" SWIFT_VERSION: "6.0" @@ -144,8 +148,8 @@ targets: path: ShareExtension/Info.plist properties: CFBundleDisplayName: OpenClaw Share - CFBundleShortVersionString: "2026.2.18" - CFBundleVersion: "20260218" + CFBundleShortVersionString: "2026.2.27" + CFBundleVersion: "20260227" NSExtension: NSExtensionPointIdentifier: com.apple.share-services NSExtensionPrincipalClass: "$(PRODUCT_MODULE_NAME).ShareViewController" @@ -169,13 +173,14 @@ targets: Release: Config/Signing.xcconfig settings: base: + ENABLE_APPINTENTS_METADATA: NO PRODUCT_BUNDLE_IDENTIFIER: "$(OPENCLAW_WATCH_APP_BUNDLE_ID)" info: path: WatchApp/Info.plist properties: CFBundleDisplayName: OpenClaw - CFBundleShortVersionString: "2026.2.18" - CFBundleVersion: "20260218" + CFBundleShortVersionString: "2026.2.27" + CFBundleVersion: "20260227" WKCompanionAppBundleIdentifier: "$(OPENCLAW_APP_BUNDLE_ID)" WKWatchKitApp: true @@ -198,8 +203,8 @@ targets: path: WatchExtension/Info.plist properties: CFBundleDisplayName: OpenClaw - CFBundleShortVersionString: "2026.2.18" - CFBundleVersion: "20260218" + CFBundleShortVersionString: "2026.2.27" + CFBundleVersion: "20260227" NSExtension: NSExtensionAttributes: WKAppBundleIdentifier: "$(OPENCLAW_WATCH_APP_BUNDLE_ID)" @@ -208,6 +213,9 @@ targets: OpenClawTests: type: bundle.unit-test platform: iOS + configFiles: + Debug: Signing.xcconfig + Release: Signing.xcconfig sources: - path: Tests dependencies: @@ -217,6 +225,9 @@ targets: - sdk: AppIntents.framework settings: base: + CODE_SIGN_IDENTITY: "Apple Development" + CODE_SIGN_STYLE: "$(OPENCLAW_CODE_SIGN_STYLE)" + DEVELOPMENT_TEAM: "$(OPENCLAW_DEVELOPMENT_TEAM)" PRODUCT_BUNDLE_IDENTIFIER: ai.openclaw.ios.tests SWIFT_VERSION: "6.0" SWIFT_STRICT_CONCURRENCY: complete @@ -226,5 +237,5 @@ targets: path: Tests/Info.plist properties: CFBundleDisplayName: OpenClawTests - CFBundleShortVersionString: "2026.2.18" - CFBundleVersion: "20260218" + CFBundleShortVersionString: "2026.2.27" + CFBundleVersion: "20260227" diff --git a/apps/macos/Package.resolved b/apps/macos/Package.resolved index 0281713738b..89bbefc5b02 100644 --- a/apps/macos/Package.resolved +++ b/apps/macos/Package.resolved @@ -51,8 +51,8 @@ "kind" : "remoteSourceControl", "location" : "https://github.com/sparkle-project/Sparkle", "state" : { - "revision" : "5581748cef2bae787496fe6d61139aebe0a451f6", - "version" : "2.8.1" + "revision" : "21d8df80440b1ca3b65fa82e40782f1e5a9e6ba2", + "version" : "2.9.0" } }, { @@ -78,8 +78,8 @@ "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-log.git", "state" : { - "revision" : "2778fd4e5a12a8aaa30a3ee8285f4ce54c5f3181", - "version" : "1.9.1" + "revision" : "bbd81b6725ae874c69e9b8c8804d462356b55523", + "version" : "1.10.1" } }, { diff --git a/apps/macos/Sources/OpenClaw/AgentWorkspace.swift b/apps/macos/Sources/OpenClaw/AgentWorkspace.swift index 57164ebb892..6340dee2ca5 100644 --- a/apps/macos/Sources/OpenClaw/AgentWorkspace.swift +++ b/apps/macos/Sources/OpenClaw/AgentWorkspace.swift @@ -17,9 +17,14 @@ enum AgentWorkspace { AgentWorkspace.userFilename, AgentWorkspace.bootstrapFilename, ] - enum BootstrapSafety: Equatable { - case safe - case unsafe (reason: String) + struct BootstrapSafety: Equatable { + let unsafeReason: String? + + static let safe = Self(unsafeReason: nil) + + static func blocked(_ reason: String) -> Self { + Self(unsafeReason: reason) + } } static func displayPath(for url: URL) -> String { @@ -71,9 +76,7 @@ enum AgentWorkspace { if !fm.fileExists(atPath: workspaceURL.path, isDirectory: &isDir) { return .safe } - if !isDir.boolValue { - return .unsafe (reason: "Workspace path points to a file.") - } + if !isDir.boolValue { return .blocked("Workspace path points to a file.") } let agentsURL = self.agentsURL(workspaceURL: workspaceURL) if fm.fileExists(atPath: agentsURL.path) { return .safe @@ -82,9 +85,9 @@ enum AgentWorkspace { let entries = try self.workspaceEntries(workspaceURL: workspaceURL) return entries.isEmpty ? .safe - : .unsafe (reason: "Folder isn't empty. Choose a new folder or add AGENTS.md first.") + : .blocked("Folder isn't empty. Choose a new folder or add AGENTS.md first.") } catch { - return .unsafe (reason: "Couldn't inspect the workspace folder.") + return .blocked("Couldn't inspect the workspace folder.") } } diff --git a/apps/macos/Sources/OpenClaw/AnthropicAuthControls.swift b/apps/macos/Sources/OpenClaw/AnthropicAuthControls.swift deleted file mode 100644 index 06f107d6c6e..00000000000 --- a/apps/macos/Sources/OpenClaw/AnthropicAuthControls.swift +++ /dev/null @@ -1,234 +0,0 @@ -import AppKit -import Combine -import SwiftUI - -@MainActor -struct AnthropicAuthControls: View { - let connectionMode: AppState.ConnectionMode - - @State private var oauthStatus: OpenClawOAuthStore.AnthropicOAuthStatus = OpenClawOAuthStore.anthropicOAuthStatus() - @State private var pkce: AnthropicOAuth.PKCE? - @State private var code: String = "" - @State private var busy = false - @State private var statusText: String? - @State private var autoDetectClipboard = true - @State private var autoConnectClipboard = true - @State private var lastPasteboardChangeCount = NSPasteboard.general.changeCount - - private static let clipboardPoll: AnyPublisher = { - if ProcessInfo.processInfo.isRunningTests { - return Empty(completeImmediately: false).eraseToAnyPublisher() - } - return Timer.publish(every: 0.4, on: .main, in: .common) - .autoconnect() - .eraseToAnyPublisher() - }() - - var body: some View { - VStack(alignment: .leading, spacing: 10) { - if self.connectionMode != .local { - Text("Gateway isn’t running locally; OAuth must be created on the gateway host.") - .font(.footnote) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - } - - HStack(spacing: 10) { - Circle() - .fill(self.oauthStatus.isConnected ? Color.green : Color.orange) - .frame(width: 8, height: 8) - Text(self.oauthStatus.shortDescription) - .font(.footnote.weight(.semibold)) - .foregroundStyle(.secondary) - Spacer() - Button("Reveal") { - NSWorkspace.shared.activateFileViewerSelecting([OpenClawOAuthStore.oauthURL()]) - } - .buttonStyle(.bordered) - .disabled(!FileManager().fileExists(atPath: OpenClawOAuthStore.oauthURL().path)) - - Button("Refresh") { - self.refresh() - } - .buttonStyle(.bordered) - } - - Text(OpenClawOAuthStore.oauthURL().path) - .font(.caption.monospaced()) - .foregroundStyle(.secondary) - .lineLimit(1) - .truncationMode(.middle) - .textSelection(.enabled) - - HStack(spacing: 12) { - Button { - self.startOAuth() - } label: { - if self.busy { - ProgressView().controlSize(.small) - } else { - Text(self.oauthStatus.isConnected ? "Re-auth (OAuth)" : "Open sign-in (OAuth)") - } - } - .buttonStyle(.borderedProminent) - .disabled(self.connectionMode != .local || self.busy) - - if self.pkce != nil { - Button("Cancel") { - self.pkce = nil - self.code = "" - self.statusText = nil - } - .buttonStyle(.bordered) - .disabled(self.busy) - } - } - - if self.pkce != nil { - VStack(alignment: .leading, spacing: 8) { - Text("Paste `code#state`") - .font(.footnote.weight(.semibold)) - .foregroundStyle(.secondary) - - TextField("code#state", text: self.$code) - .textFieldStyle(.roundedBorder) - .disabled(self.busy) - - Toggle("Auto-detect from clipboard", isOn: self.$autoDetectClipboard) - .font(.footnote) - .foregroundStyle(.secondary) - .disabled(self.busy) - - Toggle("Auto-connect when detected", isOn: self.$autoConnectClipboard) - .font(.footnote) - .foregroundStyle(.secondary) - .disabled(self.busy) - - Button("Connect") { - Task { await self.finishOAuth() } - } - .buttonStyle(.bordered) - .disabled(self.busy || self.connectionMode != .local || self.code - .trimmingCharacters(in: .whitespacesAndNewlines) - .isEmpty) - } - } - - if let statusText, !statusText.isEmpty { - Text(statusText) - .font(.footnote) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - } - } - .onAppear { - self.refresh() - } - .onReceive(Self.clipboardPoll) { _ in - self.pollClipboardIfNeeded() - } - } - - private func refresh() { - let imported = OpenClawOAuthStore.importLegacyAnthropicOAuthIfNeeded() - self.oauthStatus = OpenClawOAuthStore.anthropicOAuthStatus() - if imported != nil { - self.statusText = "Imported existing OAuth credentials." - } - } - - private func startOAuth() { - guard self.connectionMode == .local else { return } - guard !self.busy else { return } - self.busy = true - defer { self.busy = false } - - do { - let pkce = try AnthropicOAuth.generatePKCE() - self.pkce = pkce - let url = AnthropicOAuth.buildAuthorizeURL(pkce: pkce) - NSWorkspace.shared.open(url) - self.statusText = "Browser opened. After approving, paste the `code#state` value here." - } catch { - self.statusText = "Failed to start OAuth: \(error.localizedDescription)" - } - } - - @MainActor - private func finishOAuth() async { - guard self.connectionMode == .local else { return } - guard !self.busy else { return } - guard let pkce = self.pkce else { return } - self.busy = true - defer { self.busy = false } - - guard let parsed = AnthropicOAuthCodeState.parse(from: self.code) else { - self.statusText = "OAuth failed: missing or invalid code/state." - return - } - - do { - let creds = try await AnthropicOAuth.exchangeCode( - code: parsed.code, - state: parsed.state, - verifier: pkce.verifier) - try OpenClawOAuthStore.saveAnthropicOAuth(creds) - self.refresh() - self.pkce = nil - self.code = "" - self.statusText = "Connected. OpenClaw can now use Claude via OAuth." - } catch { - self.statusText = "OAuth failed: \(error.localizedDescription)" - } - } - - private func pollClipboardIfNeeded() { - guard self.connectionMode == .local else { return } - guard self.pkce != nil else { return } - guard !self.busy else { return } - guard self.autoDetectClipboard else { return } - - let pb = NSPasteboard.general - let changeCount = pb.changeCount - guard changeCount != self.lastPasteboardChangeCount else { return } - self.lastPasteboardChangeCount = changeCount - - guard let raw = pb.string(forType: .string), !raw.isEmpty else { return } - guard let parsed = AnthropicOAuthCodeState.parse(from: raw) else { return } - guard let pkce = self.pkce, parsed.state == pkce.verifier else { return } - - let next = "\(parsed.code)#\(parsed.state)" - if self.code != next { - self.code = next - self.statusText = "Detected `code#state` from clipboard." - } - - guard self.autoConnectClipboard else { return } - Task { await self.finishOAuth() } - } -} - -#if DEBUG -extension AnthropicAuthControls { - init( - connectionMode: AppState.ConnectionMode, - oauthStatus: OpenClawOAuthStore.AnthropicOAuthStatus, - pkce: AnthropicOAuth.PKCE? = nil, - code: String = "", - busy: Bool = false, - statusText: String? = nil, - autoDetectClipboard: Bool = true, - autoConnectClipboard: Bool = true) - { - self.connectionMode = connectionMode - self._oauthStatus = State(initialValue: oauthStatus) - self._pkce = State(initialValue: pkce) - self._code = State(initialValue: code) - self._busy = State(initialValue: busy) - self._statusText = State(initialValue: statusText) - self._autoDetectClipboard = State(initialValue: autoDetectClipboard) - self._autoConnectClipboard = State(initialValue: autoConnectClipboard) - self._lastPasteboardChangeCount = State(initialValue: NSPasteboard.general.changeCount) - } -} -#endif diff --git a/apps/macos/Sources/OpenClaw/AnthropicOAuth.swift b/apps/macos/Sources/OpenClaw/AnthropicOAuth.swift deleted file mode 100644 index f594cc04c31..00000000000 --- a/apps/macos/Sources/OpenClaw/AnthropicOAuth.swift +++ /dev/null @@ -1,383 +0,0 @@ -import CryptoKit -import Foundation -import OSLog -import Security - -struct AnthropicOAuthCredentials: Codable { - let type: String - let refresh: String - let access: String - let expires: Int64 -} - -enum AnthropicAuthMode: Equatable { - case oauthFile - case oauthEnv - case apiKeyEnv - case missing - - var shortLabel: String { - switch self { - case .oauthFile: "OAuth (OpenClaw token file)" - case .oauthEnv: "OAuth (env var)" - case .apiKeyEnv: "API key (env var)" - case .missing: "Missing credentials" - } - } - - var isConfigured: Bool { - switch self { - case .missing: false - case .oauthFile, .oauthEnv, .apiKeyEnv: true - } - } -} - -enum AnthropicAuthResolver { - static func resolve( - environment: [String: String] = ProcessInfo.processInfo.environment, - oauthStatus: OpenClawOAuthStore.AnthropicOAuthStatus = OpenClawOAuthStore - .anthropicOAuthStatus()) -> AnthropicAuthMode - { - if oauthStatus.isConnected { return .oauthFile } - - if let token = environment["ANTHROPIC_OAUTH_TOKEN"]?.trimmingCharacters(in: .whitespacesAndNewlines), - !token.isEmpty - { - return .oauthEnv - } - - if let key = environment["ANTHROPIC_API_KEY"]?.trimmingCharacters(in: .whitespacesAndNewlines), - !key.isEmpty - { - return .apiKeyEnv - } - - return .missing - } -} - -enum AnthropicOAuth { - private static let logger = Logger(subsystem: "ai.openclaw", category: "anthropic-oauth") - - private static let clientId = "9d1c250a-e61b-44d9-88ed-5944d1962f5e" - private static let authorizeURL = URL(string: "https://claude.ai/oauth/authorize")! - private static let tokenURL = URL(string: "https://console.anthropic.com/v1/oauth/token")! - private static let redirectURI = "https://console.anthropic.com/oauth/code/callback" - private static let scopes = "org:create_api_key user:profile user:inference" - - struct PKCE { - let verifier: String - let challenge: String - } - - static func generatePKCE() throws -> PKCE { - var bytes = [UInt8](repeating: 0, count: 32) - let status = SecRandomCopyBytes(kSecRandomDefault, bytes.count, &bytes) - guard status == errSecSuccess else { - throw NSError(domain: NSOSStatusErrorDomain, code: Int(status)) - } - let verifier = Data(bytes).base64URLEncodedString() - let hash = SHA256.hash(data: Data(verifier.utf8)) - let challenge = Data(hash).base64URLEncodedString() - return PKCE(verifier: verifier, challenge: challenge) - } - - static func buildAuthorizeURL(pkce: PKCE) -> URL { - var components = URLComponents(url: self.authorizeURL, resolvingAgainstBaseURL: false)! - components.queryItems = [ - URLQueryItem(name: "code", value: "true"), - URLQueryItem(name: "client_id", value: self.clientId), - URLQueryItem(name: "response_type", value: "code"), - URLQueryItem(name: "redirect_uri", value: self.redirectURI), - URLQueryItem(name: "scope", value: self.scopes), - URLQueryItem(name: "code_challenge", value: pkce.challenge), - URLQueryItem(name: "code_challenge_method", value: "S256"), - // Match legacy flow: state is the verifier. - URLQueryItem(name: "state", value: pkce.verifier), - ] - return components.url! - } - - static func exchangeCode( - code: String, - state: String, - verifier: String) async throws -> AnthropicOAuthCredentials - { - let payload: [String: Any] = [ - "grant_type": "authorization_code", - "client_id": self.clientId, - "code": code, - "state": state, - "redirect_uri": self.redirectURI, - "code_verifier": verifier, - ] - let body = try JSONSerialization.data(withJSONObject: payload, options: []) - - var request = URLRequest(url: self.tokenURL) - request.httpMethod = "POST" - request.httpBody = body - request.setValue("application/json", forHTTPHeaderField: "Content-Type") - - let (data, response) = try await URLSession.shared.data(for: request) - guard let http = response as? HTTPURLResponse else { - throw URLError(.badServerResponse) - } - guard (200..<300).contains(http.statusCode) else { - let text = String(data: data, encoding: .utf8) ?? "" - throw NSError( - domain: "AnthropicOAuth", - code: http.statusCode, - userInfo: [NSLocalizedDescriptionKey: "Token exchange failed: \(text)"]) - } - - let decoded = try JSONSerialization.jsonObject(with: data) as? [String: Any] - let access = decoded?["access_token"] as? String - let refresh = decoded?["refresh_token"] as? String - let expiresIn = decoded?["expires_in"] as? Double - guard let access, let refresh, let expiresIn else { - throw NSError(domain: "AnthropicOAuth", code: 0, userInfo: [ - NSLocalizedDescriptionKey: "Unexpected token response.", - ]) - } - - // Match legacy flow: expiresAt = now + expires_in - 5 minutes. - let expiresAtMs = Int64(Date().timeIntervalSince1970 * 1000) - + Int64(expiresIn * 1000) - - Int64(5 * 60 * 1000) - - self.logger.info("Anthropic OAuth exchange ok; expiresAtMs=\(expiresAtMs, privacy: .public)") - return AnthropicOAuthCredentials(type: "oauth", refresh: refresh, access: access, expires: expiresAtMs) - } - - static func refresh(refreshToken: String) async throws -> AnthropicOAuthCredentials { - let payload: [String: Any] = [ - "grant_type": "refresh_token", - "client_id": self.clientId, - "refresh_token": refreshToken, - ] - let body = try JSONSerialization.data(withJSONObject: payload, options: []) - - var request = URLRequest(url: self.tokenURL) - request.httpMethod = "POST" - request.httpBody = body - request.setValue("application/json", forHTTPHeaderField: "Content-Type") - - let (data, response) = try await URLSession.shared.data(for: request) - guard let http = response as? HTTPURLResponse else { - throw URLError(.badServerResponse) - } - guard (200..<300).contains(http.statusCode) else { - let text = String(data: data, encoding: .utf8) ?? "" - throw NSError( - domain: "AnthropicOAuth", - code: http.statusCode, - userInfo: [NSLocalizedDescriptionKey: "Token refresh failed: \(text)"]) - } - - let decoded = try JSONSerialization.jsonObject(with: data) as? [String: Any] - let access = decoded?["access_token"] as? String - let refresh = (decoded?["refresh_token"] as? String) ?? refreshToken - let expiresIn = decoded?["expires_in"] as? Double - guard let access, let expiresIn else { - throw NSError(domain: "AnthropicOAuth", code: 0, userInfo: [ - NSLocalizedDescriptionKey: "Unexpected token response.", - ]) - } - - let expiresAtMs = Int64(Date().timeIntervalSince1970 * 1000) - + Int64(expiresIn * 1000) - - Int64(5 * 60 * 1000) - - self.logger.info("Anthropic OAuth refresh ok; expiresAtMs=\(expiresAtMs, privacy: .public)") - return AnthropicOAuthCredentials(type: "oauth", refresh: refresh, access: access, expires: expiresAtMs) - } -} - -enum OpenClawOAuthStore { - static let oauthFilename = "oauth.json" - private static let providerKey = "anthropic" - private static let openclawOAuthDirEnv = "OPENCLAW_OAUTH_DIR" - private static let legacyPiDirEnv = "PI_CODING_AGENT_DIR" - - enum AnthropicOAuthStatus: Equatable { - case missingFile - case unreadableFile - case invalidJSON - case missingProviderEntry - case missingTokens - case connected(expiresAtMs: Int64?) - - var isConnected: Bool { - if case .connected = self { return true } - return false - } - - var shortDescription: String { - switch self { - case .missingFile: "OpenClaw OAuth token file not found" - case .unreadableFile: "OpenClaw OAuth token file not readable" - case .invalidJSON: "OpenClaw OAuth token file invalid" - case .missingProviderEntry: "No Anthropic entry in OpenClaw OAuth token file" - case .missingTokens: "Anthropic entry missing tokens" - case .connected: "OpenClaw OAuth credentials found" - } - } - } - - static func oauthDir() -> URL { - if let override = ProcessInfo.processInfo.environment[self.openclawOAuthDirEnv]? - .trimmingCharacters(in: .whitespacesAndNewlines), - !override.isEmpty - { - let expanded = NSString(string: override).expandingTildeInPath - return URL(fileURLWithPath: expanded, isDirectory: true) - } - let home = FileManager().homeDirectoryForCurrentUser - return home.appendingPathComponent(".openclaw", isDirectory: true) - .appendingPathComponent("credentials", isDirectory: true) - } - - static func oauthURL() -> URL { - self.oauthDir().appendingPathComponent(self.oauthFilename) - } - - static func legacyOAuthURLs() -> [URL] { - var urls: [URL] = [] - let env = ProcessInfo.processInfo.environment - if let override = env[self.legacyPiDirEnv]?.trimmingCharacters(in: .whitespacesAndNewlines), - !override.isEmpty - { - let expanded = NSString(string: override).expandingTildeInPath - urls.append(URL(fileURLWithPath: expanded, isDirectory: true).appendingPathComponent(self.oauthFilename)) - } - - let home = FileManager().homeDirectoryForCurrentUser - urls.append(home.appendingPathComponent(".pi/agent/\(self.oauthFilename)")) - urls.append(home.appendingPathComponent(".claude/\(self.oauthFilename)")) - urls.append(home.appendingPathComponent(".config/claude/\(self.oauthFilename)")) - urls.append(home.appendingPathComponent(".config/anthropic/\(self.oauthFilename)")) - - var seen = Set() - return urls.filter { url in - let path = url.standardizedFileURL.path - if seen.contains(path) { return false } - seen.insert(path) - return true - } - } - - static func importLegacyAnthropicOAuthIfNeeded() -> URL? { - let dest = self.oauthURL() - guard !FileManager().fileExists(atPath: dest.path) else { return nil } - - for url in self.legacyOAuthURLs() { - guard FileManager().fileExists(atPath: url.path) else { continue } - guard self.anthropicOAuthStatus(at: url).isConnected else { continue } - guard let storage = self.loadStorage(at: url) else { continue } - do { - try self.saveStorage(storage) - return url - } catch { - continue - } - } - - return nil - } - - static func anthropicOAuthStatus() -> AnthropicOAuthStatus { - self.anthropicOAuthStatus(at: self.oauthURL()) - } - - static func hasAnthropicOAuth() -> Bool { - self.anthropicOAuthStatus().isConnected - } - - static func anthropicOAuthStatus(at url: URL) -> AnthropicOAuthStatus { - guard FileManager().fileExists(atPath: url.path) else { return .missingFile } - - guard let data = try? Data(contentsOf: url) else { return .unreadableFile } - guard let json = try? JSONSerialization.jsonObject(with: data, options: []) else { return .invalidJSON } - guard let storage = json as? [String: Any] else { return .invalidJSON } - guard let rawEntry = storage[self.providerKey] else { return .missingProviderEntry } - guard let entry = rawEntry as? [String: Any] else { return .invalidJSON } - - let refresh = self.firstString(in: entry, keys: ["refresh", "refresh_token", "refreshToken"]) - let access = self.firstString(in: entry, keys: ["access", "access_token", "accessToken"]) - guard refresh?.isEmpty == false, access?.isEmpty == false else { return .missingTokens } - - let expiresAny = entry["expires"] ?? entry["expires_at"] ?? entry["expiresAt"] - let expiresAtMs: Int64? = if let ms = expiresAny as? Int64 { - ms - } else if let number = expiresAny as? NSNumber { - number.int64Value - } else if let ms = expiresAny as? Double { - Int64(ms) - } else { - nil - } - - return .connected(expiresAtMs: expiresAtMs) - } - - static func loadAnthropicOAuthRefreshToken() -> String? { - let url = self.oauthURL() - guard let storage = self.loadStorage(at: url) else { return nil } - guard let rawEntry = storage[self.providerKey] as? [String: Any] else { return nil } - let refresh = self.firstString(in: rawEntry, keys: ["refresh", "refresh_token", "refreshToken"]) - return refresh?.trimmingCharacters(in: .whitespacesAndNewlines) - } - - private static func firstString(in dict: [String: Any], keys: [String]) -> String? { - for key in keys { - if let value = dict[key] as? String { return value } - } - return nil - } - - private static func loadStorage(at url: URL) -> [String: Any]? { - guard let data = try? Data(contentsOf: url) else { return nil } - guard let json = try? JSONSerialization.jsonObject(with: data, options: []) else { return nil } - return json as? [String: Any] - } - - static func saveAnthropicOAuth(_ creds: AnthropicOAuthCredentials) throws { - let url = self.oauthURL() - let existing: [String: Any] = self.loadStorage(at: url) ?? [:] - - var updated = existing - updated[self.providerKey] = [ - "type": creds.type, - "refresh": creds.refresh, - "access": creds.access, - "expires": creds.expires, - ] - - try self.saveStorage(updated) - } - - private static func saveStorage(_ storage: [String: Any]) throws { - let dir = self.oauthDir() - try FileManager().createDirectory( - at: dir, - withIntermediateDirectories: true, - attributes: [.posixPermissions: 0o700]) - - let url = self.oauthURL() - let data = try JSONSerialization.data( - withJSONObject: storage, - options: [.prettyPrinted, .sortedKeys]) - try data.write(to: url, options: [.atomic]) - try FileManager().setAttributes([.posixPermissions: 0o600], ofItemAtPath: url.path) - } -} - -extension Data { - fileprivate func base64URLEncodedString() -> String { - self.base64EncodedString() - .replacingOccurrences(of: "+", with: "-") - .replacingOccurrences(of: "/", with: "_") - .replacingOccurrences(of: "=", with: "") - } -} diff --git a/apps/macos/Sources/OpenClaw/AnthropicOAuthCodeState.swift b/apps/macos/Sources/OpenClaw/AnthropicOAuthCodeState.swift deleted file mode 100644 index 2a88898c34d..00000000000 --- a/apps/macos/Sources/OpenClaw/AnthropicOAuthCodeState.swift +++ /dev/null @@ -1,59 +0,0 @@ -import Foundation - -enum AnthropicOAuthCodeState { - struct Parsed: Equatable { - let code: String - let state: String - } - - /// Extracts a `code#state` payload from arbitrary text. - /// - /// Supports: - /// - raw `code#state` - /// - OAuth callback URLs containing `code=` and `state=` query params - /// - surrounding text/backticks from instructions pages - static func extract(from raw: String) -> String? { - let text = raw.trimmingCharacters(in: .whitespacesAndNewlines) - .trimmingCharacters(in: CharacterSet(charactersIn: "`")) - if text.isEmpty { return nil } - - if let fromURL = self.extractFromURL(text) { return fromURL } - if let fromToken = self.extractFromToken(text) { return fromToken } - return nil - } - - static func parse(from raw: String) -> Parsed? { - guard let extracted = self.extract(from: raw) else { return nil } - let parts = extracted.split(separator: "#", maxSplits: 1).map(String.init) - let code = parts.first ?? "" - let state = parts.count > 1 ? parts[1] : "" - guard !code.isEmpty, !state.isEmpty else { return nil } - return Parsed(code: code, state: state) - } - - private static func extractFromURL(_ text: String) -> String? { - // Users might copy the callback URL from the browser address bar. - guard let components = URLComponents(string: text), - let items = components.queryItems, - let code = items.first(where: { $0.name == "code" })?.value, - let state = items.first(where: { $0.name == "state" })?.value, - !code.isEmpty, !state.isEmpty - else { return nil } - - return "\(code)#\(state)" - } - - private static func extractFromToken(_ text: String) -> String? { - // Base64url-ish tokens; keep this fairly strict to avoid false positives. - let pattern = #"([A-Za-z0-9._~-]{8,})#([A-Za-z0-9._~-]{8,})"# - guard let re = try? NSRegularExpression(pattern: pattern) else { return nil } - - let range = NSRange(text.startIndex.. Bool + { + let trimmed = value?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + if trimmed.isEmpty { + guard dictionary[key] != nil else { return false } + dictionary.removeValue(forKey: key) + return true + } + if (dictionary[key] as? String) != trimmed { + dictionary[key] = trimmed + return true + } + return false + } + + private static func updatedRemoteGatewayConfig( + current: [String: Any], + transport: RemoteTransport, + remoteUrl: String, + remoteHost: String?, + remoteTarget: String, + remoteIdentity: String) -> (remote: [String: Any], changed: Bool) + { + var remote = current + var changed = false + + switch transport { + case .direct: + changed = Self.updateGatewayString( + &remote, + key: "transport", + value: RemoteTransport.direct.rawValue) || changed + + let trimmedUrl = remoteUrl.trimmingCharacters(in: .whitespacesAndNewlines) + if trimmedUrl.isEmpty { + changed = Self.updateGatewayString(&remote, key: "url", value: nil) || changed + } else if let normalizedUrl = GatewayRemoteConfig.normalizeGatewayUrlString(trimmedUrl) { + changed = Self.updateGatewayString(&remote, key: "url", value: normalizedUrl) || changed + } + + case .ssh: + changed = Self.updateGatewayString(&remote, key: "transport", value: nil) || changed + + if let host = remoteHost { + let existingUrl = (remote["url"] as? String)? + .trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + let parsedExisting = existingUrl.isEmpty ? nil : URL(string: existingUrl) + let scheme = parsedExisting?.scheme?.isEmpty == false ? parsedExisting?.scheme : "ws" + let port = parsedExisting?.port ?? 18789 + let desiredUrl = "\(scheme ?? "ws")://\(host):\(port)" + changed = Self.updateGatewayString(&remote, key: "url", value: desiredUrl) || changed + } + + let sanitizedTarget = Self.sanitizeSSHTarget(remoteTarget) + changed = Self.updateGatewayString(&remote, key: "sshTarget", value: sanitizedTarget) || changed + changed = Self.updateGatewayString(&remote, key: "sshIdentity", value: remoteIdentity) || changed + } + + return (remote, changed) + } + private func startConfigWatcher() { let configUrl = OpenClawConfigFile.url() self.configWatcher = ConfigFileWatcher(url: configUrl) { [weak self] in @@ -470,70 +534,16 @@ final class AppState { } if connectionMode == .remote { - var remote = gateway["remote"] as? [String: Any] ?? [:] - var remoteChanged = false - - if remoteTransport == .direct { - let trimmedUrl = remoteUrl.trimmingCharacters(in: .whitespacesAndNewlines) - if trimmedUrl.isEmpty { - if remote["url"] != nil { - remote.removeValue(forKey: "url") - remoteChanged = true - } - } else { - let normalizedUrl = GatewayRemoteConfig.normalizeGatewayUrlString(trimmedUrl) ?? trimmedUrl - if (remote["url"] as? String) != normalizedUrl { - remote["url"] = normalizedUrl - remoteChanged = true - } - } - if (remote["transport"] as? String) != RemoteTransport.direct.rawValue { - remote["transport"] = RemoteTransport.direct.rawValue - remoteChanged = true - } - } else { - if remote["transport"] != nil { - remote.removeValue(forKey: "transport") - remoteChanged = true - } - if let host = remoteHost { - let existingUrl = (remote["url"] as? String)? - .trimmingCharacters(in: .whitespacesAndNewlines) ?? "" - let parsedExisting = existingUrl.isEmpty ? nil : URL(string: existingUrl) - let scheme = parsedExisting?.scheme?.isEmpty == false ? parsedExisting?.scheme : "ws" - let port = parsedExisting?.port ?? 18789 - let desiredUrl = "\(scheme ?? "ws")://\(host):\(port)" - if existingUrl != desiredUrl { - remote["url"] = desiredUrl - remoteChanged = true - } - } - - let sanitizedTarget = Self.sanitizeSSHTarget(remoteTarget) - if !sanitizedTarget.isEmpty { - if (remote["sshTarget"] as? String) != sanitizedTarget { - remote["sshTarget"] = sanitizedTarget - remoteChanged = true - } - } else if remote["sshTarget"] != nil { - remote.removeValue(forKey: "sshTarget") - remoteChanged = true - } - - let trimmedIdentity = remoteIdentity.trimmingCharacters(in: .whitespacesAndNewlines) - if !trimmedIdentity.isEmpty { - if (remote["sshIdentity"] as? String) != trimmedIdentity { - remote["sshIdentity"] = trimmedIdentity - remoteChanged = true - } - } else if remote["sshIdentity"] != nil { - remote.removeValue(forKey: "sshIdentity") - remoteChanged = true - } - } - - if remoteChanged { - gateway["remote"] = remote + let currentRemote = gateway["remote"] as? [String: Any] ?? [:] + let updated = Self.updatedRemoteGatewayConfig( + current: currentRemote, + transport: remoteTransport, + remoteUrl: remoteUrl, + remoteHost: remoteHost, + remoteTarget: remoteTarget, + remoteIdentity: remoteIdentity) + if updated.changed { + gateway["remote"] = updated.remote changed = true } } diff --git a/apps/macos/Sources/OpenClaw/AudioInputDeviceObserver.swift b/apps/macos/Sources/OpenClaw/AudioInputDeviceObserver.swift index abbddb24588..6c01628144b 100644 --- a/apps/macos/Sources/OpenClaw/AudioInputDeviceObserver.swift +++ b/apps/macos/Sources/OpenClaw/AudioInputDeviceObserver.swift @@ -53,6 +53,15 @@ final class AudioInputDeviceObserver { return output } + /// Returns true when the system default input device exists and is alive with input channels. + /// Use this preflight before accessing `AVAudioEngine.inputNode` to avoid SIGABRT on Macs + /// without a built-in microphone (Mac mini, Mac Pro, Mac Studio) or when an external mic + /// is disconnected. + static func hasUsableDefaultInputDevice() -> Bool { + guard let uid = self.defaultInputDeviceUID() else { return false } + return self.aliveInputDeviceUIDs().contains(uid) + } + static func defaultInputDeviceSummary() -> String { let systemObject = AudioObjectID(kAudioObjectSystemObject) var address = AudioObjectPropertyAddress( diff --git a/apps/macos/Sources/OpenClaw/CameraCaptureService.swift b/apps/macos/Sources/OpenClaw/CameraCaptureService.swift index 24717ec5536..4e3749d6a68 100644 --- a/apps/macos/Sources/OpenClaw/CameraCaptureService.swift +++ b/apps/macos/Sources/OpenClaw/CameraCaptureService.swift @@ -357,8 +357,8 @@ private final class PhotoCaptureDelegate: NSObject, AVCapturePhotoCaptureDelegat func photoOutput( _ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, - error: Error? - ) { + error: Error?) + { guard !self.didResume, let cont else { return } self.didResume = true self.cont = nil @@ -380,8 +380,8 @@ private final class PhotoCaptureDelegate: NSObject, AVCapturePhotoCaptureDelegat func photoOutput( _ output: AVCapturePhotoOutput, didFinishCaptureFor resolvedSettings: AVCaptureResolvedPhotoSettings, - error: Error? - ) { + error: Error?) + { guard let error else { return } guard !self.didResume, let cont else { return } self.didResume = true diff --git a/apps/macos/Sources/OpenClaw/CoalescingFSEventsWatcher.swift b/apps/macos/Sources/OpenClaw/CoalescingFSEventsWatcher.swift index 7999123dbe2..f9e38d81170 100644 --- a/apps/macos/Sources/OpenClaw/CoalescingFSEventsWatcher.swift +++ b/apps/macos/Sources/OpenClaw/CoalescingFSEventsWatcher.swift @@ -16,8 +16,8 @@ final class CoalescingFSEventsWatcher: @unchecked Sendable { queueLabel: String, coalesceDelay: TimeInterval = 0.12, shouldNotify: @escaping (Int, UnsafeMutableRawPointer?) -> Bool = { _, _ in true }, - onChange: @escaping () -> Void - ) { + onChange: @escaping () -> Void) + { self.paths = paths self.queue = DispatchQueue(label: queueLabel) self.coalesceDelay = coalesceDelay @@ -92,8 +92,8 @@ extension CoalescingFSEventsWatcher { private func handleEvents( numEvents: Int, eventPaths: UnsafeMutableRawPointer?, - eventFlags: UnsafePointer? - ) { + eventFlags: UnsafePointer?) + { guard numEvents > 0 else { return } guard eventFlags != nil else { return } guard self.shouldNotify(numEvents, eventPaths) else { return } @@ -108,4 +108,3 @@ extension CoalescingFSEventsWatcher { } } } - diff --git a/apps/macos/Sources/OpenClaw/CommandResolver.swift b/apps/macos/Sources/OpenClaw/CommandResolver.swift index c17f64e30e7..cacfac2f068 100644 --- a/apps/macos/Sources/OpenClaw/CommandResolver.swift +++ b/apps/macos/Sources/OpenClaw/CommandResolver.swift @@ -246,15 +246,17 @@ enum CommandResolver { return ssh } - let runtimeResult = self.runtimeResolution(searchPaths: searchPaths) + let root = self.projectRoot() + if let openclawPath = self.projectOpenClawExecutable(projectRoot: root) { + return [openclawPath, subcommand] + extraArgs + } + if let openclawPath = self.openclawExecutable(searchPaths: searchPaths) { + return [openclawPath, subcommand] + extraArgs + } + let runtimeResult = self.runtimeResolution(searchPaths: searchPaths) switch runtimeResult { case let .success(runtime): - let root = self.projectRoot() - if let openclawPath = self.projectOpenClawExecutable(projectRoot: root) { - return [openclawPath, subcommand] + extraArgs - } - if let entry = self.gatewayEntrypoint(in: root) { return self.makeRuntimeCommand( runtime: runtime, @@ -262,19 +264,21 @@ enum CommandResolver { subcommand: subcommand, extraArgs: extraArgs) } - if let pnpm = self.findExecutable(named: "pnpm", searchPaths: searchPaths) { - // Use --silent to avoid pnpm lifecycle banners that would corrupt JSON outputs. - return [pnpm, "--silent", "openclaw", subcommand] + extraArgs - } - if let openclawPath = self.openclawExecutable(searchPaths: searchPaths) { - return [openclawPath, subcommand] + extraArgs - } + case .failure: + break + } + if let pnpm = self.findExecutable(named: "pnpm", searchPaths: searchPaths) { + // Use --silent to avoid pnpm lifecycle banners that would corrupt JSON outputs. + return [pnpm, "--silent", "openclaw", subcommand] + extraArgs + } + + switch runtimeResult { + case .success: let missingEntry = """ openclaw entrypoint missing (looked for dist/index.js or openclaw.mjs); run pnpm build. """ return self.errorCommand(with: missingEntry) - case let .failure(error): return self.runtimeErrorCommand(error) } diff --git a/apps/macos/Sources/OpenClaw/ExecAllowlistMatcher.swift b/apps/macos/Sources/OpenClaw/ExecAllowlistMatcher.swift new file mode 100644 index 00000000000..ad40d2c3803 --- /dev/null +++ b/apps/macos/Sources/OpenClaw/ExecAllowlistMatcher.swift @@ -0,0 +1,79 @@ +import Foundation + +enum ExecAllowlistMatcher { + static func match(entries: [ExecAllowlistEntry], resolution: ExecCommandResolution?) -> ExecAllowlistEntry? { + guard let resolution, !entries.isEmpty else { return nil } + let rawExecutable = resolution.rawExecutable + let resolvedPath = resolution.resolvedPath + + for entry in entries { + switch ExecApprovalHelpers.validateAllowlistPattern(entry.pattern) { + case let .valid(pattern): + let target = resolvedPath ?? rawExecutable + if self.matches(pattern: pattern, target: target) { return entry } + case .invalid: + continue + } + } + return nil + } + + static func matchAll( + entries: [ExecAllowlistEntry], + resolutions: [ExecCommandResolution]) -> [ExecAllowlistEntry] + { + guard !entries.isEmpty, !resolutions.isEmpty else { return [] } + var matches: [ExecAllowlistEntry] = [] + matches.reserveCapacity(resolutions.count) + for resolution in resolutions { + guard let match = self.match(entries: entries, resolution: resolution) else { + return [] + } + matches.append(match) + } + return matches + } + + private static func matches(pattern: String, target: String) -> Bool { + let trimmed = pattern.trimmingCharacters(in: .whitespacesAndNewlines) + guard !trimmed.isEmpty else { return false } + let expanded = trimmed.hasPrefix("~") ? (trimmed as NSString).expandingTildeInPath : trimmed + let normalizedPattern = self.normalizeMatchTarget(expanded) + let normalizedTarget = self.normalizeMatchTarget(target) + guard let regex = self.regex(for: normalizedPattern) else { return false } + let range = NSRange(location: 0, length: normalizedTarget.utf16.count) + return regex.firstMatch(in: normalizedTarget, options: [], range: range) != nil + } + + private static func normalizeMatchTarget(_ value: String) -> String { + value.replacingOccurrences(of: "\\\\", with: "/").lowercased() + } + + private static func regex(for pattern: String) -> NSRegularExpression? { + var regex = "^" + var idx = pattern.startIndex + while idx < pattern.endIndex { + let ch = pattern[idx] + if ch == "*" { + let next = pattern.index(after: idx) + if next < pattern.endIndex, pattern[next] == "*" { + regex += ".*" + idx = pattern.index(after: next) + } else { + regex += "[^/]*" + idx = next + } + continue + } + if ch == "?" { + regex += "." + idx = pattern.index(after: idx) + continue + } + regex += NSRegularExpression.escapedPattern(for: String(ch)) + idx = pattern.index(after: idx) + } + regex += "$" + return try? NSRegularExpression(pattern: regex, options: [.caseInsensitive]) + } +} diff --git a/apps/macos/Sources/OpenClaw/ExecApprovalEvaluation.swift b/apps/macos/Sources/OpenClaw/ExecApprovalEvaluation.swift new file mode 100644 index 00000000000..c7d9d0928e1 --- /dev/null +++ b/apps/macos/Sources/OpenClaw/ExecApprovalEvaluation.swift @@ -0,0 +1,68 @@ +import Foundation + +struct ExecApprovalEvaluation { + let command: [String] + let displayCommand: String + let agentId: String? + let security: ExecSecurity + let ask: ExecAsk + let env: [String: String] + let resolution: ExecCommandResolution? + let allowlistResolutions: [ExecCommandResolution] + let allowlistMatches: [ExecAllowlistEntry] + let allowlistSatisfied: Bool + let allowlistMatch: ExecAllowlistEntry? + let skillAllow: Bool +} + +enum ExecApprovalEvaluator { + static func evaluate( + command: [String], + rawCommand: String?, + cwd: String?, + envOverrides: [String: String]?, + agentId: String?) async -> ExecApprovalEvaluation + { + let trimmedAgent = agentId?.trimmingCharacters(in: .whitespacesAndNewlines) + let normalizedAgentId = (trimmedAgent?.isEmpty == false) ? trimmedAgent : nil + let approvals = ExecApprovalsStore.resolve(agentId: normalizedAgentId) + let security = approvals.agent.security + let ask = approvals.agent.ask + let shellWrapper = ExecShellWrapperParser.extract(command: command, rawCommand: rawCommand).isWrapper + let env = HostEnvSanitizer.sanitize(overrides: envOverrides, shellWrapper: shellWrapper) + let displayCommand = ExecCommandFormatter.displayString(for: command, rawCommand: rawCommand) + let allowlistResolutions = ExecCommandResolution.resolveForAllowlist( + command: command, + rawCommand: rawCommand, + cwd: cwd, + env: env) + let allowlistMatches = security == .allowlist + ? ExecAllowlistMatcher.matchAll(entries: approvals.allowlist, resolutions: allowlistResolutions) + : [] + let allowlistSatisfied = security == .allowlist && + !allowlistResolutions.isEmpty && + allowlistMatches.count == allowlistResolutions.count + + let skillAllow: Bool + if approvals.agent.autoAllowSkills, !allowlistResolutions.isEmpty { + let bins = await SkillBinsCache.shared.currentBins() + skillAllow = allowlistResolutions.allSatisfy { bins.contains($0.executableName) } + } else { + skillAllow = false + } + + return ExecApprovalEvaluation( + command: command, + displayCommand: displayCommand, + agentId: normalizedAgentId, + security: security, + ask: ask, + env: env, + resolution: allowlistResolutions.first, + allowlistResolutions: allowlistResolutions, + allowlistMatches: allowlistMatches, + allowlistSatisfied: allowlistSatisfied, + allowlistMatch: allowlistSatisfied ? allowlistMatches.first : nil, + skillAllow: skillAllow) + } +} diff --git a/apps/macos/Sources/OpenClaw/ExecApprovals.swift b/apps/macos/Sources/OpenClaw/ExecApprovals.swift index f6bc8392503..73aa3899d82 100644 --- a/apps/macos/Sources/OpenClaw/ExecApprovals.swift +++ b/apps/macos/Sources/OpenClaw/ExecApprovals.swift @@ -90,6 +90,31 @@ enum ExecApprovalDecision: String, Codable, Sendable { case deny } +enum ExecAllowlistPatternValidationReason: String, Codable, Sendable, Equatable { + case empty + case missingPathComponent + + var message: String { + switch self { + case .empty: + "Pattern cannot be empty." + case .missingPathComponent: + "Path patterns only. Include '/', '~', or '\\\\'." + } + } +} + +enum ExecAllowlistPatternValidation: Sendable, Equatable { + case valid(String) + case invalid(ExecAllowlistPatternValidationReason) +} + +struct ExecAllowlistRejectedEntry: Sendable, Equatable { + let id: UUID + let pattern: String + let reason: ExecAllowlistPatternValidationReason +} + struct ExecAllowlistEntry: Codable, Hashable, Identifiable { var id: UUID var pattern: String @@ -222,13 +247,25 @@ enum ExecApprovalsStore { } agents.removeValue(forKey: "default") } + if !agents.isEmpty { + var normalizedAgents: [String: ExecApprovalsAgent] = [:] + normalizedAgents.reserveCapacity(agents.count) + for (key, var agent) in agents { + if let allowlist = agent.allowlist { + let normalized = self.normalizeAllowlistEntries(allowlist, dropInvalid: false).entries + agent.allowlist = normalized.isEmpty ? nil : normalized + } + normalizedAgents[key] = agent + } + agents = normalizedAgents + } return ExecApprovalsFile( version: 1, socket: ExecApprovalsSocketConfig( path: socketPath.isEmpty ? nil : socketPath, token: token.isEmpty ? nil : token), defaults: file.defaults, - agents: agents) + agents: agents.isEmpty ? nil : agents) } static func readSnapshot() -> ExecApprovalsSnapshot { @@ -306,7 +343,12 @@ enum ExecApprovalsStore { } static func ensureFile() -> ExecApprovalsFile { - var file = self.loadFile() + let url = self.fileURL() + let existed = FileManager().fileExists(atPath: url.path) + let loaded = self.loadFile() + let loadedHash = self.hashFile(loaded) + + var file = self.normalizeIncoming(loaded) if file.socket == nil { file.socket = ExecApprovalsSocketConfig(path: nil, token: nil) } let path = file.socket?.path?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" if path.isEmpty { @@ -317,7 +359,9 @@ enum ExecApprovalsStore { file.socket?.token = self.generateToken() } if file.agents == nil { file.agents = [:] } - self.saveFile(file) + if !existed || loadedHash != self.hashFile(file) { + self.saveFile(file) + } return file } @@ -339,16 +383,9 @@ enum ExecApprovalsStore { ?? resolvedDefaults.askFallback, autoAllowSkills: agentEntry.autoAllowSkills ?? wildcardEntry.autoAllowSkills ?? resolvedDefaults.autoAllowSkills) - let allowlist = ((wildcardEntry.allowlist ?? []) + (agentEntry.allowlist ?? [])) - .map { entry in - ExecAllowlistEntry( - id: entry.id, - pattern: entry.pattern.trimmingCharacters(in: .whitespacesAndNewlines), - lastUsedAt: entry.lastUsedAt, - lastUsedCommand: entry.lastUsedCommand, - lastResolvedPath: entry.lastResolvedPath) - } - .filter { !$0.pattern.isEmpty } + let allowlist = self.normalizeAllowlistEntries( + (wildcardEntry.allowlist ?? []) + (agentEntry.allowlist ?? []), + dropInvalid: true).entries let socketPath = self.expandPath(file.socket?.path ?? self.socketPath()) let token = file.socket?.token ?? "" return ExecApprovalsResolved( @@ -398,20 +435,30 @@ enum ExecApprovalsStore { } } - static func addAllowlistEntry(agentId: String?, pattern: String) { - let trimmed = pattern.trimmingCharacters(in: .whitespacesAndNewlines) - guard !trimmed.isEmpty else { return } + @discardableResult + static func addAllowlistEntry(agentId: String?, pattern: String) -> ExecAllowlistPatternValidationReason? { + let normalizedPattern: String + switch ExecApprovalHelpers.validateAllowlistPattern(pattern) { + case let .valid(validPattern): + normalizedPattern = validPattern + case let .invalid(reason): + return reason + } + self.updateFile { file in let key = self.agentKey(agentId) var agents = file.agents ?? [:] var entry = agents[key] ?? ExecApprovalsAgent() var allowlist = entry.allowlist ?? [] - if allowlist.contains(where: { $0.pattern == trimmed }) { return } - allowlist.append(ExecAllowlistEntry(pattern: trimmed, lastUsedAt: Date().timeIntervalSince1970 * 1000)) + if allowlist.contains(where: { $0.pattern == normalizedPattern }) { return } + allowlist.append(ExecAllowlistEntry( + pattern: normalizedPattern, + lastUsedAt: Date().timeIntervalSince1970 * 1000)) entry.allowlist = allowlist agents[key] = entry file.agents = agents } + return nil } static func recordAllowlistUse( @@ -439,25 +486,21 @@ enum ExecApprovalsStore { } } - static func updateAllowlist(agentId: String?, allowlist: [ExecAllowlistEntry]) { + @discardableResult + static func updateAllowlist(agentId: String?, allowlist: [ExecAllowlistEntry]) -> [ExecAllowlistRejectedEntry] { + var rejected: [ExecAllowlistRejectedEntry] = [] self.updateFile { file in let key = self.agentKey(agentId) var agents = file.agents ?? [:] var entry = agents[key] ?? ExecApprovalsAgent() - let cleaned = allowlist - .map { item in - ExecAllowlistEntry( - id: item.id, - pattern: item.pattern.trimmingCharacters(in: .whitespacesAndNewlines), - lastUsedAt: item.lastUsedAt, - lastUsedCommand: item.lastUsedCommand, - lastResolvedPath: item.lastResolvedPath) - } - .filter { !$0.pattern.isEmpty } + let normalized = self.normalizeAllowlistEntries(allowlist, dropInvalid: true) + rejected = normalized.rejected + let cleaned = normalized.entries entry.allowlist = cleaned agents[key] = entry file.agents = agents } + return rejected } static func updateAgentSettings(agentId: String?, mutate: (inout ExecApprovalsAgent) -> Void) { @@ -500,6 +543,14 @@ enum ExecApprovalsStore { return digest.map { String(format: "%02x", $0) }.joined() } + private static func hashFile(_ file: ExecApprovalsFile) -> String { + let encoder = JSONEncoder() + encoder.outputFormatting = [.sortedKeys] + let data = (try? encoder.encode(file)) ?? Data() + let digest = SHA256.hash(data: data) + return digest.map { String(format: "%02x", $0) }.joined() + } + private static func expandPath(_ raw: String) -> String { let trimmed = raw.trimmingCharacters(in: .whitespacesAndNewlines) if trimmed == "~" { @@ -519,14 +570,101 @@ enum ExecApprovalsStore { } private static func normalizedPattern(_ pattern: String?) -> String? { - let trimmed = pattern?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" - return trimmed.isEmpty ? nil : trimmed.lowercased() + switch ExecApprovalHelpers.validateAllowlistPattern(pattern) { + case let .valid(normalized): + return normalized.lowercased() + case .invalid(.empty): + return nil + case .invalid: + let trimmed = pattern?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + return trimmed.isEmpty ? nil : trimmed.lowercased() + } + } + + private static func migrateLegacyPattern(_ entry: ExecAllowlistEntry) -> ExecAllowlistEntry { + let trimmedPattern = entry.pattern.trimmingCharacters(in: .whitespacesAndNewlines) + let trimmedResolved = entry.lastResolvedPath?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + let normalizedResolved = trimmedResolved.isEmpty ? nil : trimmedResolved + + switch ExecApprovalHelpers.validateAllowlistPattern(trimmedPattern) { + case let .valid(pattern): + return ExecAllowlistEntry( + id: entry.id, + pattern: pattern, + lastUsedAt: entry.lastUsedAt, + lastUsedCommand: entry.lastUsedCommand, + lastResolvedPath: normalizedResolved) + case .invalid: + switch ExecApprovalHelpers.validateAllowlistPattern(trimmedResolved) { + case let .valid(migratedPattern): + return ExecAllowlistEntry( + id: entry.id, + pattern: migratedPattern, + lastUsedAt: entry.lastUsedAt, + lastUsedCommand: entry.lastUsedCommand, + lastResolvedPath: normalizedResolved) + case .invalid: + return ExecAllowlistEntry( + id: entry.id, + pattern: trimmedPattern, + lastUsedAt: entry.lastUsedAt, + lastUsedCommand: entry.lastUsedCommand, + lastResolvedPath: normalizedResolved) + } + } + } + + private static func normalizeAllowlistEntries( + _ entries: [ExecAllowlistEntry], + dropInvalid: Bool) -> (entries: [ExecAllowlistEntry], rejected: [ExecAllowlistRejectedEntry]) + { + var normalized: [ExecAllowlistEntry] = [] + normalized.reserveCapacity(entries.count) + var rejected: [ExecAllowlistRejectedEntry] = [] + + for entry in entries { + let migrated = self.migrateLegacyPattern(entry) + let trimmedPattern = migrated.pattern.trimmingCharacters(in: .whitespacesAndNewlines) + let trimmedResolvedPath = migrated.lastResolvedPath?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + let normalizedResolvedPath = trimmedResolvedPath.isEmpty ? nil : trimmedResolvedPath + + switch ExecApprovalHelpers.validateAllowlistPattern(trimmedPattern) { + case let .valid(pattern): + normalized.append( + ExecAllowlistEntry( + id: migrated.id, + pattern: pattern, + lastUsedAt: migrated.lastUsedAt, + lastUsedCommand: migrated.lastUsedCommand, + lastResolvedPath: normalizedResolvedPath)) + case let .invalid(reason): + if dropInvalid { + rejected.append( + ExecAllowlistRejectedEntry( + id: migrated.id, + pattern: trimmedPattern, + reason: reason)) + } else if reason != .empty { + normalized.append( + ExecAllowlistEntry( + id: migrated.id, + pattern: trimmedPattern, + lastUsedAt: migrated.lastUsedAt, + lastUsedCommand: migrated.lastUsedCommand, + lastResolvedPath: normalizedResolvedPath)) + } + } + } + + return (normalized, rejected) } private static func mergeAgents( current: ExecApprovalsAgent, legacy: ExecApprovalsAgent) -> ExecApprovalsAgent { + let currentAllowlist = self.normalizeAllowlistEntries(current.allowlist ?? [], dropInvalid: false).entries + let legacyAllowlist = self.normalizeAllowlistEntries(legacy.allowlist ?? [], dropInvalid: false).entries var seen = Set() var allowlist: [ExecAllowlistEntry] = [] func append(_ entry: ExecAllowlistEntry) { @@ -536,10 +674,10 @@ enum ExecApprovalsStore { seen.insert(key) allowlist.append(entry) } - for entry in current.allowlist ?? [] { + for entry in currentAllowlist { append(entry) } - for entry in legacy.allowlist ?? [] { + for entry in legacyAllowlist { append(entry) } @@ -552,102 +690,23 @@ enum ExecApprovalsStore { } } -struct ExecCommandResolution: Sendable { - let rawExecutable: String - let resolvedPath: String? - let executableName: String - let cwd: String? - - static func resolve( - command: [String], - rawCommand: String?, - cwd: String?, - env: [String: String]?) -> ExecCommandResolution? - { - let trimmedRaw = rawCommand?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" - if !trimmedRaw.isEmpty, let token = self.parseFirstToken(trimmedRaw) { - return self.resolveExecutable(rawExecutable: token, cwd: cwd, env: env) - } - return self.resolve(command: command, cwd: cwd, env: env) - } - - static func resolve(command: [String], cwd: String?, env: [String: String]?) -> ExecCommandResolution? { - guard let raw = command.first?.trimmingCharacters(in: .whitespacesAndNewlines), !raw.isEmpty else { - return nil - } - return self.resolveExecutable(rawExecutable: raw, cwd: cwd, env: env) - } - - private static func resolveExecutable( - rawExecutable: String, - cwd: String?, - env: [String: String]?) -> ExecCommandResolution? - { - let expanded = rawExecutable.hasPrefix("~") ? (rawExecutable as NSString).expandingTildeInPath : rawExecutable - let hasPathSeparator = expanded.contains("/") || expanded.contains("\\") - let resolvedPath: String? = { - if hasPathSeparator { - if expanded.hasPrefix("/") { - return expanded - } - let base = cwd?.trimmingCharacters(in: .whitespacesAndNewlines) - let root = (base?.isEmpty == false) ? base! : FileManager().currentDirectoryPath - return URL(fileURLWithPath: root).appendingPathComponent(expanded).path - } - let searchPaths = self.searchPaths(from: env) - return CommandResolver.findExecutable(named: expanded, searchPaths: searchPaths) - }() - let name = resolvedPath.map { URL(fileURLWithPath: $0).lastPathComponent } ?? expanded - return ExecCommandResolution( - rawExecutable: expanded, - resolvedPath: resolvedPath, - executableName: name, - cwd: cwd) - } - - private static func parseFirstToken(_ command: String) -> String? { - let trimmed = command.trimmingCharacters(in: .whitespacesAndNewlines) - guard !trimmed.isEmpty else { return nil } - guard let first = trimmed.first else { return nil } - if first == "\"" || first == "'" { - let rest = trimmed.dropFirst() - if let end = rest.firstIndex(of: first) { - return String(rest[.. [String] { - let raw = env?["PATH"] - if let raw, !raw.isEmpty { - return raw.split(separator: ":").map(String.init) - } - return CommandResolver.preferredPaths() - } -} - -enum ExecCommandFormatter { - static func displayString(for argv: [String]) -> String { - argv.map { arg in - let trimmed = arg.trimmingCharacters(in: .whitespacesAndNewlines) - guard !trimmed.isEmpty else { return "\"\"" } - let needsQuotes = trimmed.contains { $0.isWhitespace || $0 == "\"" } - if !needsQuotes { return trimmed } - let escaped = trimmed.replacingOccurrences(of: "\"", with: "\\\"") - return "\"\(escaped)\"" - }.joined(separator: " ") - } - - static func displayString(for argv: [String], rawCommand: String?) -> String { - let trimmed = rawCommand?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" - if !trimmed.isEmpty { return trimmed } - return self.displayString(for: argv) - } -} - enum ExecApprovalHelpers { + static func validateAllowlistPattern(_ pattern: String?) -> ExecAllowlistPatternValidation { + let trimmed = pattern?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + guard !trimmed.isEmpty else { return .invalid(.empty) } + guard self.containsPathComponent(trimmed) else { return .invalid(.missingPathComponent) } + return .valid(trimmed) + } + + static func isPathPattern(_ pattern: String?) -> Bool { + switch self.validateAllowlistPattern(pattern) { + case .valid: + true + case .invalid: + false + } + } + static func parseDecision(_ raw: String?) -> ExecApprovalDecision? { let trimmed = raw?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" guard !trimmed.isEmpty else { return nil } @@ -669,70 +728,9 @@ enum ExecApprovalHelpers { let pattern = resolution?.resolvedPath ?? resolution?.rawExecutable ?? command.first ?? "" return pattern.isEmpty ? nil : pattern } -} -enum ExecAllowlistMatcher { - static func match(entries: [ExecAllowlistEntry], resolution: ExecCommandResolution?) -> ExecAllowlistEntry? { - guard let resolution, !entries.isEmpty else { return nil } - let rawExecutable = resolution.rawExecutable - let resolvedPath = resolution.resolvedPath - let executableName = resolution.executableName - - for entry in entries { - let pattern = entry.pattern.trimmingCharacters(in: .whitespacesAndNewlines) - if pattern.isEmpty { continue } - let hasPath = pattern.contains("/") || pattern.contains("~") || pattern.contains("\\") - if hasPath { - let target = resolvedPath ?? rawExecutable - if self.matches(pattern: pattern, target: target) { return entry } - } else if self.matches(pattern: pattern, target: executableName) { - return entry - } - } - return nil - } - - private static func matches(pattern: String, target: String) -> Bool { - let trimmed = pattern.trimmingCharacters(in: .whitespacesAndNewlines) - guard !trimmed.isEmpty else { return false } - let expanded = trimmed.hasPrefix("~") ? (trimmed as NSString).expandingTildeInPath : trimmed - let normalizedPattern = self.normalizeMatchTarget(expanded) - let normalizedTarget = self.normalizeMatchTarget(target) - guard let regex = self.regex(for: normalizedPattern) else { return false } - let range = NSRange(location: 0, length: normalizedTarget.utf16.count) - return regex.firstMatch(in: normalizedTarget, options: [], range: range) != nil - } - - private static func normalizeMatchTarget(_ value: String) -> String { - value.replacingOccurrences(of: "\\\\", with: "/").lowercased() - } - - private static func regex(for pattern: String) -> NSRegularExpression? { - var regex = "^" - var idx = pattern.startIndex - while idx < pattern.endIndex { - let ch = pattern[idx] - if ch == "*" { - let next = pattern.index(after: idx) - if next < pattern.endIndex, pattern[next] == "*" { - regex += ".*" - idx = pattern.index(after: next) - } else { - regex += "[^/]*" - idx = next - } - continue - } - if ch == "?" { - regex += "." - idx = pattern.index(after: idx) - continue - } - regex += NSRegularExpression.escapedPattern(for: String(ch)) - idx = pattern.index(after: idx) - } - regex += "$" - return try? NSRegularExpression(pattern: regex, options: [.caseInsensitive]) + private static func containsPathComponent(_ pattern: String) -> Bool { + pattern.contains("/") || pattern.contains("~") || pattern.contains("\\") } } diff --git a/apps/macos/Sources/OpenClaw/ExecApprovalsSocket.swift b/apps/macos/Sources/OpenClaw/ExecApprovalsSocket.swift index e1432aaea1c..2c308b3eeb6 100644 --- a/apps/macos/Sources/OpenClaw/ExecApprovalsSocket.swift +++ b/apps/macos/Sources/OpenClaw/ExecApprovalsSocket.swift @@ -38,7 +38,7 @@ private struct ExecHostSocketRequest: Codable { var requestJson: String } -private struct ExecHostRequest: Codable { +struct ExecHostRequest: Codable { var command: [String] var rawCommand: String? var cwd: String? @@ -59,7 +59,7 @@ private struct ExecHostRunResult: Codable { var error: String? } -private struct ExecHostError: Codable { +struct ExecHostError: Codable, Error { var code: String var message: String var reason: String? @@ -350,68 +350,31 @@ enum ExecApprovalsPromptPresenter { @MainActor private enum ExecHostExecutor { - private struct ExecApprovalContext { - let command: [String] - let displayCommand: String - let trimmedAgent: String? - let approvals: ExecApprovalsResolved - let security: ExecSecurity - let ask: ExecAsk - let autoAllowSkills: Bool - let env: [String: String]? - let resolution: ExecCommandResolution? - let allowlistMatch: ExecAllowlistEntry? - let skillAllow: Bool - } - - private static let blockedEnvKeys: Set = [ - "PATH", - "NODE_OPTIONS", - "PYTHONHOME", - "PYTHONPATH", - "PERL5LIB", - "PERL5OPT", - "RUBYOPT", - ] - - private static let blockedEnvPrefixes: [String] = [ - "DYLD_", - "LD_", - ] + private typealias ExecApprovalContext = ExecApprovalEvaluation static func handle(_ request: ExecHostRequest) async -> ExecHostResponse { - let command = request.command.map { $0.trimmingCharacters(in: .whitespacesAndNewlines) } - guard !command.isEmpty else { - return self.errorResponse( - code: "INVALID_REQUEST", - message: "command required", - reason: "invalid") + let validatedRequest: ExecHostValidatedRequest + switch ExecHostRequestEvaluator.validateRequest(request) { + case let .success(request): + validatedRequest = request + case let .failure(error): + return self.errorResponse(error) } - let context = await self.buildContext(request: request, command: command) - if context.security == .deny { - return self.errorResponse( - code: "UNAVAILABLE", - message: "SYSTEM_RUN_DISABLED: security=deny", - reason: "security=deny") - } + let context = await self.buildContext( + request: request, + command: validatedRequest.command, + rawCommand: validatedRequest.displayCommand) - let approvalDecision = request.approvalDecision - if approvalDecision == .deny { - return self.errorResponse( - code: "UNAVAILABLE", - message: "SYSTEM_RUN_DENIED: user denied", - reason: "user-denied") - } - - var approvedByAsk = approvalDecision != nil - if ExecApprovalHelpers.requiresAsk( - ask: context.ask, - security: context.security, - allowlistMatch: context.allowlistMatch, - skillAllow: context.skillAllow), - approvalDecision == nil + switch ExecHostRequestEvaluator.evaluate( + context: context, + approvalDecision: request.approvalDecision) { + case let .deny(error): + return self.errorResponse(error) + case .allow: + break + case .requiresPrompt: let decision = ExecApprovalsPromptPresenter.prompt( ExecApprovalPromptRequest( command: context.displayCommand, @@ -419,43 +382,54 @@ private enum ExecHostExecutor { host: "node", security: context.security.rawValue, ask: context.ask.rawValue, - agentId: context.trimmedAgent, + agentId: context.agentId, resolvedPath: context.resolution?.resolvedPath, sessionKey: request.sessionKey)) + let followupDecision: ExecApprovalDecision switch decision { case .deny: - return self.errorResponse( - code: "UNAVAILABLE", - message: "SYSTEM_RUN_DENIED: user denied", - reason: "user-denied") + followupDecision = .deny case .allowAlways: - approvedByAsk = true + followupDecision = .allowAlways self.persistAllowlistEntry(decision: decision, context: context) case .allowOnce: - approvedByAsk = true + followupDecision = .allowOnce + } + + switch ExecHostRequestEvaluator.evaluate( + context: context, + approvalDecision: followupDecision) + { + case let .deny(error): + return self.errorResponse(error) + case .allow: + break + case .requiresPrompt: + return self.errorResponse( + code: "INVALID_REQUEST", + message: "unexpected approval state", + reason: "invalid") } } - self.persistAllowlistEntry(decision: approvalDecision, context: context) + self.persistAllowlistEntry(decision: request.approvalDecision, context: context) - if context.security == .allowlist, - context.allowlistMatch == nil, - !context.skillAllow, - !approvedByAsk - { - return self.errorResponse( - code: "UNAVAILABLE", - message: "SYSTEM_RUN_DENIED: allowlist miss", - reason: "allowlist-miss") - } - - if let match = context.allowlistMatch { - ExecApprovalsStore.recordAllowlistUse( - agentId: context.trimmedAgent, - pattern: match.pattern, - command: context.displayCommand, - resolvedPath: context.resolution?.resolvedPath) + if context.allowlistSatisfied { + var seenPatterns = Set() + for (idx, match) in context.allowlistMatches.enumerated() { + if !seenPatterns.insert(match.pattern).inserted { + continue + } + let resolvedPath = idx < context.allowlistResolutions.count + ? context.allowlistResolutions[idx].resolvedPath + : nil + ExecApprovalsStore.recordAllowlistUse( + agentId: context.agentId, + pattern: match.pattern, + command: context.displayCommand, + resolvedPath: resolvedPath) + } } if let errorResponse = await self.ensureScreenRecordingAccess(request.needsScreenRecording) { @@ -463,50 +437,23 @@ private enum ExecHostExecutor { } return await self.runCommand( - command: command, + command: validatedRequest.command, cwd: request.cwd, env: context.env, timeoutMs: request.timeoutMs) } - private static func buildContext(request: ExecHostRequest, command: [String]) async -> ExecApprovalContext { - let displayCommand = ExecCommandFormatter.displayString( - for: command, - rawCommand: request.rawCommand) - let agentId = request.agentId?.trimmingCharacters(in: .whitespacesAndNewlines) - let trimmedAgent = (agentId?.isEmpty == false) ? agentId : nil - let approvals = ExecApprovalsStore.resolve(agentId: trimmedAgent) - let security = approvals.agent.security - let ask = approvals.agent.ask - let autoAllowSkills = approvals.agent.autoAllowSkills - let env = self.sanitizedEnv(request.env) - let resolution = ExecCommandResolution.resolve( + private static func buildContext( + request: ExecHostRequest, + command: [String], + rawCommand: String?) async -> ExecApprovalContext + { + await ExecApprovalEvaluator.evaluate( command: command, - rawCommand: request.rawCommand, + rawCommand: rawCommand, cwd: request.cwd, - env: env) - let allowlistMatch = security == .allowlist - ? ExecAllowlistMatcher.match(entries: approvals.allowlist, resolution: resolution) - : nil - let skillAllow: Bool - if autoAllowSkills, let name = resolution?.executableName { - let bins = await SkillBinsCache.shared.currentBins() - skillAllow = bins.contains(name) - } else { - skillAllow = false - } - return ExecApprovalContext( - command: command, - displayCommand: displayCommand, - trimmedAgent: trimmedAgent, - approvals: approvals, - security: security, - ask: ask, - autoAllowSkills: autoAllowSkills, - env: env, - resolution: resolution, - allowlistMatch: allowlistMatch, - skillAllow: skillAllow) + envOverrides: request.env, + agentId: request.agentId) } private static func persistAllowlistEntry( @@ -514,13 +461,18 @@ private enum ExecHostExecutor { context: ExecApprovalContext) { guard decision == .allowAlways, context.security == .allowlist else { return } - guard let pattern = ExecApprovalHelpers.allowlistPattern( - command: context.command, - resolution: context.resolution) - else { - return + var seenPatterns = Set() + for candidate in context.allowlistResolutions { + guard let pattern = ExecApprovalHelpers.allowlistPattern( + command: context.command, + resolution: candidate) + else { + continue + } + if seenPatterns.insert(pattern).inserted { + ExecApprovalsStore.addAllowlistEntry(agentId: context.agentId, pattern: pattern) + } } - ExecApprovalsStore.addAllowlistEntry(agentId: context.trimmedAgent, pattern: pattern) } private static func ensureScreenRecordingAccess(_ needsScreenRecording: Bool?) async -> ExecHostResponse? { @@ -558,6 +510,17 @@ private enum ExecHostExecutor { return self.successResponse(payload) } + private static func errorResponse( + _ error: ExecHostError) -> ExecHostResponse + { + ExecHostResponse( + type: "response", + id: UUID().uuidString, + ok: false, + payload: nil, + error: error) + } + private static func errorResponse( code: String, message: String, @@ -579,20 +542,6 @@ private enum ExecHostExecutor { payload: payload, error: nil) } - - private static func sanitizedEnv(_ overrides: [String: String]?) -> [String: String]? { - guard let overrides else { return nil } - var merged = ProcessInfo.processInfo.environment - for (rawKey, value) in overrides { - let key = rawKey.trimmingCharacters(in: .whitespacesAndNewlines) - guard !key.isEmpty else { continue } - let upper = key.uppercased() - if self.blockedEnvKeys.contains(upper) { continue } - if self.blockedEnvPrefixes.contains(where: { upper.hasPrefix($0) }) { continue } - merged[key] = value - } - return merged - } } private final class ExecApprovalsSocketServer: @unchecked Sendable { diff --git a/apps/macos/Sources/OpenClaw/ExecCommandResolution.swift b/apps/macos/Sources/OpenClaw/ExecCommandResolution.swift new file mode 100644 index 00000000000..843062b2470 --- /dev/null +++ b/apps/macos/Sources/OpenClaw/ExecCommandResolution.swift @@ -0,0 +1,265 @@ +import Foundation + +struct ExecCommandResolution: Sendable { + let rawExecutable: String + let resolvedPath: String? + let executableName: String + let cwd: String? + + static func resolve( + command: [String], + rawCommand: String?, + cwd: String?, + env: [String: String]?) -> ExecCommandResolution? + { + let trimmedRaw = rawCommand?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + if !trimmedRaw.isEmpty, let token = self.parseFirstToken(trimmedRaw) { + return self.resolveExecutable(rawExecutable: token, cwd: cwd, env: env) + } + return self.resolve(command: command, cwd: cwd, env: env) + } + + static func resolveForAllowlist( + command: [String], + rawCommand: String?, + cwd: String?, + env: [String: String]?) -> [ExecCommandResolution] + { + let shell = ExecShellWrapperParser.extract(command: command, rawCommand: rawCommand) + if shell.isWrapper { + guard let shellCommand = shell.command, + let segments = self.splitShellCommandChain(shellCommand) + else { + // Fail closed: if we cannot safely parse a shell wrapper payload, + // treat this as an allowlist miss and require approval. + return [] + } + var resolutions: [ExecCommandResolution] = [] + resolutions.reserveCapacity(segments.count) + for segment in segments { + guard let token = self.parseFirstToken(segment), + let resolution = self.resolveExecutable(rawExecutable: token, cwd: cwd, env: env) + else { + return [] + } + resolutions.append(resolution) + } + return resolutions + } + + guard let resolution = self.resolve(command: command, rawCommand: rawCommand, cwd: cwd, env: env) else { + return [] + } + return [resolution] + } + + static func resolve(command: [String], cwd: String?, env: [String: String]?) -> ExecCommandResolution? { + let effective = ExecEnvInvocationUnwrapper.unwrapDispatchWrappersForResolution(command) + guard let raw = effective.first?.trimmingCharacters(in: .whitespacesAndNewlines), !raw.isEmpty else { + return nil + } + return self.resolveExecutable(rawExecutable: raw, cwd: cwd, env: env) + } + + private static func resolveExecutable( + rawExecutable: String, + cwd: String?, + env: [String: String]?) -> ExecCommandResolution? + { + let expanded = rawExecutable.hasPrefix("~") ? (rawExecutable as NSString).expandingTildeInPath : rawExecutable + let hasPathSeparator = expanded.contains("/") || expanded.contains("\\") + let resolvedPath: String? = { + if hasPathSeparator { + if expanded.hasPrefix("/") { + return expanded + } + let base = cwd?.trimmingCharacters(in: .whitespacesAndNewlines) + let root = (base?.isEmpty == false) ? base! : FileManager().currentDirectoryPath + return URL(fileURLWithPath: root).appendingPathComponent(expanded).path + } + let searchPaths = self.searchPaths(from: env) + return CommandResolver.findExecutable(named: expanded, searchPaths: searchPaths) + }() + let name = resolvedPath.map { URL(fileURLWithPath: $0).lastPathComponent } ?? expanded + return ExecCommandResolution( + rawExecutable: expanded, + resolvedPath: resolvedPath, + executableName: name, + cwd: cwd) + } + + private static func parseFirstToken(_ command: String) -> String? { + let trimmed = command.trimmingCharacters(in: .whitespacesAndNewlines) + guard !trimmed.isEmpty else { return nil } + guard let first = trimmed.first else { return nil } + if first == "\"" || first == "'" { + let rest = trimmed.dropFirst() + if let end = rest.firstIndex(of: first) { + return String(rest[..", next: "("), + ], + .doubleQuoted: [ + ShellFailClosedRule(token: "`", next: nil), + ShellFailClosedRule(token: "$", next: "("), + ], + ] + + private static func splitShellCommandChain(_ command: String) -> [String]? { + let trimmed = command.trimmingCharacters(in: .whitespacesAndNewlines) + guard !trimmed.isEmpty else { return nil } + + var segments: [String] = [] + var current = "" + var inSingle = false + var inDouble = false + var escaped = false + let chars = Array(trimmed) + var idx = 0 + + func appendCurrent() -> Bool { + let segment = current.trimmingCharacters(in: .whitespacesAndNewlines) + guard !segment.isEmpty else { return false } + segments.append(segment) + current.removeAll(keepingCapacity: true) + return true + } + + while idx < chars.count { + let ch = chars[idx] + let next: Character? = idx + 1 < chars.count ? chars[idx + 1] : nil + + if escaped { + current.append(ch) + escaped = false + idx += 1 + continue + } + + if ch == "\\", !inSingle { + current.append(ch) + escaped = true + idx += 1 + continue + } + + if ch == "'", !inDouble { + inSingle.toggle() + current.append(ch) + idx += 1 + continue + } + + if ch == "\"", !inSingle { + inDouble.toggle() + current.append(ch) + idx += 1 + continue + } + + if !inSingle, self.shouldFailClosedForShell(ch: ch, next: next, inDouble: inDouble) { + // Fail closed on command/process substitution in allowlist mode, + // including command substitution inside double-quoted shell strings. + return nil + } + + if !inSingle, !inDouble { + let prev: Character? = idx > 0 ? chars[idx - 1] : nil + if let delimiterStep = self.chainDelimiterStep(ch: ch, prev: prev, next: next) { + guard appendCurrent() else { return nil } + idx += delimiterStep + continue + } + } + + current.append(ch) + idx += 1 + } + + if escaped || inSingle || inDouble { return nil } + guard appendCurrent() else { return nil } + return segments + } + + private static func shouldFailClosedForShell(ch: Character, next: Character?, inDouble: Bool) -> Bool { + let context: ShellTokenContext = inDouble ? .doubleQuoted : .unquoted + guard let rules = self.shellFailClosedRules[context] else { + return false + } + for rule in rules { + if ch == rule.token, rule.next == nil || next == rule.next { + return true + } + } + return false + } + + private static func chainDelimiterStep(ch: Character, prev: Character?, next: Character?) -> Int? { + if ch == ";" || ch == "\n" { + return 1 + } + if ch == "&" { + if next == "&" { + return 2 + } + // Keep fd redirections like 2>&1 or &>file intact. + let prevIsRedirect = prev == ">" + let nextIsRedirect = next == ">" + return (!prevIsRedirect && !nextIsRedirect) ? 1 : nil + } + if ch == "|" { + if next == "|" || next == "&" { + return 2 + } + return 1 + } + return nil + } + + private static func searchPaths(from env: [String: String]?) -> [String] { + let raw = env?["PATH"] + if let raw, !raw.isEmpty { + return raw.split(separator: ":").map(String.init) + } + return CommandResolver.preferredPaths() + } +} + +enum ExecCommandFormatter { + static func displayString(for argv: [String]) -> String { + argv.map { arg in + let trimmed = arg.trimmingCharacters(in: .whitespacesAndNewlines) + guard !trimmed.isEmpty else { return "\"\"" } + let needsQuotes = trimmed.contains { $0.isWhitespace || $0 == "\"" } + if !needsQuotes { return trimmed } + let escaped = trimmed.replacingOccurrences(of: "\"", with: "\\\"") + return "\"\(escaped)\"" + }.joined(separator: " ") + } + + static func displayString(for argv: [String], rawCommand: String?) -> String { + let trimmed = rawCommand?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + if !trimmed.isEmpty { return trimmed } + return self.displayString(for: argv) + } +} diff --git a/apps/macos/Sources/OpenClaw/ExecEnvInvocationUnwrapper.swift b/apps/macos/Sources/OpenClaw/ExecEnvInvocationUnwrapper.swift new file mode 100644 index 00000000000..ebb8965e755 --- /dev/null +++ b/apps/macos/Sources/OpenClaw/ExecEnvInvocationUnwrapper.swift @@ -0,0 +1,108 @@ +import Foundation + +enum ExecCommandToken { + static func basenameLower(_ token: String) -> String { + let trimmed = token.trimmingCharacters(in: .whitespacesAndNewlines) + guard !trimmed.isEmpty else { return "" } + let normalized = trimmed.replacingOccurrences(of: "\\", with: "/") + return normalized.split(separator: "/").last.map { String($0).lowercased() } ?? normalized.lowercased() + } +} + +enum ExecEnvInvocationUnwrapper { + static let maxWrapperDepth = 4 + + private static let optionsWithValue = Set([ + "-u", + "--unset", + "-c", + "--chdir", + "-s", + "--split-string", + "--default-signal", + "--ignore-signal", + "--block-signal", + ]) + private static let flagOptions = Set(["-i", "--ignore-environment", "-0", "--null"]) + + private static func isEnvAssignment(_ token: String) -> Bool { + let pattern = #"^[A-Za-z_][A-Za-z0-9_]*=.*"# + return token.range(of: pattern, options: .regularExpression) != nil + } + + static func unwrap(_ command: [String]) -> [String]? { + var idx = 1 + var expectsOptionValue = false + while idx < command.count { + let token = command[idx].trimmingCharacters(in: .whitespacesAndNewlines) + if token.isEmpty { + idx += 1 + continue + } + if expectsOptionValue { + expectsOptionValue = false + idx += 1 + continue + } + if token == "--" || token == "-" { + idx += 1 + break + } + if self.isEnvAssignment(token) { + idx += 1 + continue + } + if token.hasPrefix("-"), token != "-" { + let lower = token.lowercased() + let flag = lower.split(separator: "=", maxSplits: 1).first.map(String.init) ?? lower + if self.flagOptions.contains(flag) { + idx += 1 + continue + } + if self.optionsWithValue.contains(flag) { + if !lower.contains("=") { + expectsOptionValue = true + } + idx += 1 + continue + } + if lower.hasPrefix("-u") || + lower.hasPrefix("-c") || + lower.hasPrefix("-s") || + lower.hasPrefix("--unset=") || + lower.hasPrefix("--chdir=") || + lower.hasPrefix("--split-string=") || + lower.hasPrefix("--default-signal=") || + lower.hasPrefix("--ignore-signal=") || + lower.hasPrefix("--block-signal=") + { + idx += 1 + continue + } + return nil + } + break + } + guard idx < command.count else { return nil } + return Array(command[idx...]) + } + + static func unwrapDispatchWrappersForResolution(_ command: [String]) -> [String] { + var current = command + var depth = 0 + while depth < self.maxWrapperDepth { + guard let token = current.first?.trimmingCharacters(in: .whitespacesAndNewlines), !token.isEmpty else { + break + } + guard ExecCommandToken.basenameLower(token) == "env" else { + break + } + guard let unwrapped = self.unwrap(current), !unwrapped.isEmpty else { + break + } + current = unwrapped + depth += 1 + } + return current + } +} diff --git a/apps/macos/Sources/OpenClaw/ExecHostRequestEvaluator.swift b/apps/macos/Sources/OpenClaw/ExecHostRequestEvaluator.swift new file mode 100644 index 00000000000..4e0ff4173de --- /dev/null +++ b/apps/macos/Sources/OpenClaw/ExecHostRequestEvaluator.swift @@ -0,0 +1,84 @@ +import Foundation + +struct ExecHostValidatedRequest { + let command: [String] + let displayCommand: String +} + +enum ExecHostPolicyDecision { + case deny(ExecHostError) + case requiresPrompt + case allow(approvedByAsk: Bool) +} + +enum ExecHostRequestEvaluator { + static func validateRequest(_ request: ExecHostRequest) -> Result { + let command = request.command.map { $0.trimmingCharacters(in: .whitespacesAndNewlines) } + guard !command.isEmpty else { + return .failure( + ExecHostError( + code: "INVALID_REQUEST", + message: "command required", + reason: "invalid")) + } + + let validatedCommand = ExecSystemRunCommandValidator.resolve( + command: command, + rawCommand: request.rawCommand) + switch validatedCommand { + case let .ok(resolved): + return .success(ExecHostValidatedRequest(command: command, displayCommand: resolved.displayCommand)) + case let .invalid(message): + return .failure( + ExecHostError( + code: "INVALID_REQUEST", + message: message, + reason: "invalid")) + } + } + + static func evaluate( + context: ExecApprovalEvaluation, + approvalDecision: ExecApprovalDecision?) -> ExecHostPolicyDecision + { + if context.security == .deny { + return .deny( + ExecHostError( + code: "UNAVAILABLE", + message: "SYSTEM_RUN_DISABLED: security=deny", + reason: "security=deny")) + } + + if approvalDecision == .deny { + return .deny( + ExecHostError( + code: "UNAVAILABLE", + message: "SYSTEM_RUN_DENIED: user denied", + reason: "user-denied")) + } + + let approvedByAsk = approvalDecision != nil + let requiresPrompt = ExecApprovalHelpers.requiresAsk( + ask: context.ask, + security: context.security, + allowlistMatch: context.allowlistMatch, + skillAllow: context.skillAllow) && approvalDecision == nil + if requiresPrompt { + return .requiresPrompt + } + + if context.security == .allowlist, + !context.allowlistSatisfied, + !context.skillAllow, + !approvedByAsk + { + return .deny( + ExecHostError( + code: "UNAVAILABLE", + message: "SYSTEM_RUN_DENIED: allowlist miss", + reason: "allowlist-miss")) + } + + return .allow(approvedByAsk: approvedByAsk) + } +} diff --git a/apps/macos/Sources/OpenClaw/ExecShellWrapperParser.swift b/apps/macos/Sources/OpenClaw/ExecShellWrapperParser.swift new file mode 100644 index 00000000000..06851a7d065 --- /dev/null +++ b/apps/macos/Sources/OpenClaw/ExecShellWrapperParser.swift @@ -0,0 +1,108 @@ +import Foundation + +enum ExecShellWrapperParser { + struct ParsedShellWrapper { + let isWrapper: Bool + let command: String? + + static let notWrapper = ParsedShellWrapper(isWrapper: false, command: nil) + } + + private enum Kind { + case posix + case cmd + case powershell + } + + private struct WrapperSpec { + let kind: Kind + let names: Set + } + + private static let posixInlineFlags = Set(["-lc", "-c", "--command"]) + private static let powershellInlineFlags = Set(["-c", "-command", "--command"]) + + private static let wrapperSpecs: [WrapperSpec] = [ + WrapperSpec(kind: .posix, names: ["ash", "sh", "bash", "zsh", "dash", "ksh", "fish"]), + WrapperSpec(kind: .cmd, names: ["cmd.exe", "cmd"]), + WrapperSpec(kind: .powershell, names: ["powershell", "powershell.exe", "pwsh", "pwsh.exe"]), + ] + + static func extract(command: [String], rawCommand: String?) -> ParsedShellWrapper { + let trimmedRaw = rawCommand?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + let preferredRaw = trimmedRaw.isEmpty ? nil : trimmedRaw + return self.extract(command: command, preferredRaw: preferredRaw, depth: 0) + } + + private static func extract(command: [String], preferredRaw: String?, depth: Int) -> ParsedShellWrapper { + guard depth < ExecEnvInvocationUnwrapper.maxWrapperDepth else { + return .notWrapper + } + guard let token0 = command.first?.trimmingCharacters(in: .whitespacesAndNewlines), !token0.isEmpty else { + return .notWrapper + } + + let base0 = ExecCommandToken.basenameLower(token0) + if base0 == "env" { + guard let unwrapped = ExecEnvInvocationUnwrapper.unwrap(command) else { + return .notWrapper + } + return self.extract(command: unwrapped, preferredRaw: preferredRaw, depth: depth + 1) + } + + guard let spec = self.wrapperSpecs.first(where: { $0.names.contains(base0) }) else { + return .notWrapper + } + guard let payload = self.extractPayload(command: command, spec: spec) else { + return .notWrapper + } + let normalized = preferredRaw ?? payload + return ParsedShellWrapper(isWrapper: true, command: normalized) + } + + private static func extractPayload(command: [String], spec: WrapperSpec) -> String? { + switch spec.kind { + case .posix: + self.extractPosixInlineCommand(command) + case .cmd: + self.extractCmdInlineCommand(command) + case .powershell: + self.extractPowerShellInlineCommand(command) + } + } + + private static func extractPosixInlineCommand(_ command: [String]) -> String? { + let flag = command.count > 1 ? command[1].trimmingCharacters(in: .whitespacesAndNewlines) : "" + guard self.posixInlineFlags.contains(flag.lowercased()) else { + return nil + } + let payload = command.count > 2 ? command[2].trimmingCharacters(in: .whitespacesAndNewlines) : "" + return payload.isEmpty ? nil : payload + } + + private static func extractCmdInlineCommand(_ command: [String]) -> String? { + guard let idx = command + .firstIndex(where: { $0.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() == "/c" }) + else { + return nil + } + let tail = command.suffix(from: command.index(after: idx)).joined(separator: " ") + let payload = tail.trimmingCharacters(in: .whitespacesAndNewlines) + return payload.isEmpty ? nil : payload + } + + private static func extractPowerShellInlineCommand(_ command: [String]) -> String? { + for idx in 1.. String? { + let trimmed = rawCommand?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + return trimmed.isEmpty ? nil : trimmed + } + + private static func trimmedNonEmpty(_ value: String?) -> String? { + let trimmed = value?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" + return trimmed.isEmpty ? nil : trimmed + } + + private static func normalizeExecutableToken(_ token: String) -> String { + let base = ExecCommandToken.basenameLower(token) + if base.hasSuffix(".exe") { + return String(base.dropLast(4)) + } + return base + } + + private static func isEnvAssignment(_ token: String) -> Bool { + token.range(of: #"^[A-Za-z_][A-Za-z0-9_]*=.*"#, options: .regularExpression) != nil + } + + private static func hasEnvInlineValuePrefix(_ lowerToken: String) -> Bool { + self.envInlineValuePrefixes.contains { lowerToken.hasPrefix($0) } + } + + private static func unwrapEnvInvocationWithMetadata(_ argv: [String]) -> EnvUnwrapResult? { + var idx = 1 + var expectsOptionValue = false + var usesModifiers = false + + while idx < argv.count { + let token = argv[idx].trimmingCharacters(in: .whitespacesAndNewlines) + if token.isEmpty { + idx += 1 + continue + } + if expectsOptionValue { + expectsOptionValue = false + usesModifiers = true + idx += 1 + continue + } + if token == "--" || token == "-" { + idx += 1 + break + } + if self.isEnvAssignment(token) { + usesModifiers = true + idx += 1 + continue + } + if !token.hasPrefix("-") || token == "-" { + break + } + + let lower = token.lowercased() + let flag = lower.split(separator: "=", maxSplits: 1).first.map(String.init) ?? lower + if self.envFlagOptions.contains(flag) { + usesModifiers = true + idx += 1 + continue + } + if self.envOptionsWithValue.contains(flag) { + usesModifiers = true + if !lower.contains("=") { + expectsOptionValue = true + } + idx += 1 + continue + } + if self.hasEnvInlineValuePrefix(lower) { + usesModifiers = true + idx += 1 + continue + } + return nil + } + + if expectsOptionValue { + return nil + } + guard idx < argv.count else { + return nil + } + return EnvUnwrapResult(argv: Array(argv[idx...]), usesModifiers: usesModifiers) + } + + private static func unwrapShellMultiplexerInvocation(_ argv: [String]) -> [String]? { + guard let token0 = self.trimmedNonEmpty(argv.first) else { + return nil + } + let wrapper = self.normalizeExecutableToken(token0) + guard self.shellMultiplexerWrapperNames.contains(wrapper) else { + return nil + } + + var appletIndex = 1 + if appletIndex < argv.count, argv[appletIndex].trimmingCharacters(in: .whitespacesAndNewlines) == "--" { + appletIndex += 1 + } + guard appletIndex < argv.count else { + return nil + } + let applet = argv[appletIndex].trimmingCharacters(in: .whitespacesAndNewlines) + guard !applet.isEmpty else { + return nil + } + let normalizedApplet = self.normalizeExecutableToken(applet) + guard self.shellWrapperNames.contains(normalizedApplet) else { + return nil + } + return Array(argv[appletIndex...]) + } + + private static func hasEnvManipulationBeforeShellWrapper( + _ argv: [String], + depth: Int = 0, + envManipulationSeen: Bool = false) -> Bool + { + if depth >= ExecEnvInvocationUnwrapper.maxWrapperDepth { + return false + } + guard let token0 = self.trimmedNonEmpty(argv.first) else { + return false + } + + let normalized = self.normalizeExecutableToken(token0) + if normalized == "env" { + guard let envUnwrap = self.unwrapEnvInvocationWithMetadata(argv) else { + return false + } + return self.hasEnvManipulationBeforeShellWrapper( + envUnwrap.argv, + depth: depth + 1, + envManipulationSeen: envManipulationSeen || envUnwrap.usesModifiers) + } + + if let shellMultiplexer = self.unwrapShellMultiplexerInvocation(argv) { + return self.hasEnvManipulationBeforeShellWrapper( + shellMultiplexer, + depth: depth + 1, + envManipulationSeen: envManipulationSeen) + } + + guard self.shellWrapperNames.contains(normalized) else { + return false + } + guard self.extractShellInlinePayload(argv, normalizedWrapper: normalized) != nil else { + return false + } + return envManipulationSeen + } + + private static func hasTrailingPositionalArgvAfterInlineCommand(_ argv: [String]) -> Bool { + let wrapperArgv = self.unwrapShellWrapperArgv(argv) + guard let token0 = self.trimmedNonEmpty(wrapperArgv.first) else { + return false + } + let wrapper = self.normalizeExecutableToken(token0) + guard self.posixOrPowerShellInlineWrapperNames.contains(wrapper) else { + return false + } + + let inlineCommandIndex: Int? = if wrapper == "powershell" || wrapper == "pwsh" { + self.resolveInlineCommandTokenIndex( + wrapperArgv, + flags: self.powershellInlineCommandFlags, + allowCombinedC: false) + } else { + self.resolveInlineCommandTokenIndex( + wrapperArgv, + flags: self.posixInlineCommandFlags, + allowCombinedC: true) + } + guard let inlineCommandIndex else { + return false + } + let start = inlineCommandIndex + 1 + guard start < wrapperArgv.count else { + return false + } + return wrapperArgv[start...].contains { !$0.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty } + } + + private static func unwrapShellWrapperArgv(_ argv: [String]) -> [String] { + var current = argv + for _ in 0.., + allowCombinedC: Bool) -> Int? + { + var idx = 1 + while idx < argv.count { + let token = argv[idx].trimmingCharacters(in: .whitespacesAndNewlines) + if token.isEmpty { + idx += 1 + continue + } + let lower = token.lowercased() + if lower == "--" { + break + } + if flags.contains(lower) { + return idx + 1 < argv.count ? idx + 1 : nil + } + if allowCombinedC, let inlineOffset = self.combinedCommandInlineOffset(token) { + let inline = String(token.dropFirst(inlineOffset)) + .trimmingCharacters(in: .whitespacesAndNewlines) + if !inline.isEmpty { + return idx + } + return idx + 1 < argv.count ? idx + 1 : nil + } + idx += 1 + } + return nil + } + + private static func combinedCommandInlineOffset(_ token: String) -> Int? { + let chars = Array(token.lowercased()) + guard chars.count >= 2, chars[0] == "-", chars[1] != "-" else { + return nil + } + if chars.dropFirst().contains("-") { + return nil + } + guard let commandIndex = chars.firstIndex(of: "c"), commandIndex > 0 else { + return nil + } + return commandIndex + 1 + } + + private static func extractShellInlinePayload( + _ argv: [String], + normalizedWrapper: String) -> String? + { + if normalizedWrapper == "cmd" { + return self.extractCmdInlineCommand(argv) + } + if normalizedWrapper == "powershell" || normalizedWrapper == "pwsh" { + return self.extractInlineCommandByFlags( + argv, + flags: self.powershellInlineCommandFlags, + allowCombinedC: false) + } + return self.extractInlineCommandByFlags( + argv, + flags: self.posixInlineCommandFlags, + allowCombinedC: true) + } + + private static func extractInlineCommandByFlags( + _ argv: [String], + flags: Set, + allowCombinedC: Bool) -> String? + { + var idx = 1 + while idx < argv.count { + let token = argv[idx].trimmingCharacters(in: .whitespacesAndNewlines) + if token.isEmpty { + idx += 1 + continue + } + let lower = token.lowercased() + if lower == "--" { + break + } + if flags.contains(lower) { + return self.trimmedNonEmpty(idx + 1 < argv.count ? argv[idx + 1] : nil) + } + if allowCombinedC, let inlineOffset = self.combinedCommandInlineOffset(token) { + let inline = String(token.dropFirst(inlineOffset)) + if let inlineValue = self.trimmedNonEmpty(inline) { + return inlineValue + } + return self.trimmedNonEmpty(idx + 1 < argv.count ? argv[idx + 1] : nil) + } + idx += 1 + } + return nil + } + + private static func extractCmdInlineCommand(_ argv: [String]) -> String? { + guard let idx = argv.firstIndex(where: { + let token = $0.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() + return token == "/c" || token == "/k" + }) else { + return nil + } + let tailIndex = idx + 1 + guard tailIndex < argv.count else { + return nil + } + let payload = argv[tailIndex...].joined(separator: " ").trimmingCharacters(in: .whitespacesAndNewlines) + return payload.isEmpty ? nil : payload + } +} diff --git a/apps/macos/Sources/OpenClaw/GatewayDiscoveryHelpers.swift b/apps/macos/Sources/OpenClaw/GatewayDiscoveryHelpers.swift index 281dcb9e8bd..81383efa21a 100644 --- a/apps/macos/Sources/OpenClaw/GatewayDiscoveryHelpers.swift +++ b/apps/macos/Sources/OpenClaw/GatewayDiscoveryHelpers.swift @@ -2,9 +2,34 @@ import Foundation import OpenClawDiscovery enum GatewayDiscoveryHelpers { - static func sshTarget(for gateway: GatewayDiscoveryModel.DiscoveredGateway) -> String? { - let host = self.sanitizedTailnetHost(gateway.tailnetDns) ?? gateway.lanHost + static func resolvedServiceHost( + for gateway: GatewayDiscoveryModel.DiscoveredGateway) -> String? + { + self.resolvedServiceHost(gateway.serviceHost) + } + + static func resolvedServiceHost(_ host: String?) -> String? { guard let host = self.trimmed(host), !host.isEmpty else { return nil } + return host + } + + static func serviceEndpoint( + for gateway: GatewayDiscoveryModel.DiscoveredGateway) -> (host: String, port: Int)? + { + self.serviceEndpoint(serviceHost: gateway.serviceHost, servicePort: gateway.servicePort) + } + + static func serviceEndpoint( + serviceHost: String?, + servicePort: Int?) -> (host: String, port: Int)? + { + guard let host = self.resolvedServiceHost(serviceHost) else { return nil } + guard let port = servicePort, port > 0, port <= 65535 else { return nil } + return (host, port) + } + + static func sshTarget(for gateway: GatewayDiscoveryModel.DiscoveredGateway) -> String? { + guard let host = self.resolvedServiceHost(for: gateway) else { return nil } let user = NSUserName() var target = "\(user)@\(host)" if gateway.sshPort != 22 { @@ -16,42 +41,37 @@ enum GatewayDiscoveryHelpers { static func directUrl(for gateway: GatewayDiscoveryModel.DiscoveredGateway) -> String? { self.directGatewayUrl( serviceHost: gateway.serviceHost, - servicePort: gateway.servicePort, - lanHost: gateway.lanHost, - gatewayPort: gateway.gatewayPort) + servicePort: gateway.servicePort) } static func directGatewayUrl( serviceHost: String?, - servicePort: Int?, - lanHost: String?, - gatewayPort: Int?) -> String? + servicePort: Int?) -> String? { // Security: do not route using unauthenticated TXT hints (tailnetDns/lanHost/gatewayPort). // Prefer the resolved service endpoint (SRV + A/AAAA). - if let host = self.trimmed(serviceHost), !host.isEmpty, - let port = servicePort, port > 0 - { - let scheme = port == 443 ? "wss" : "ws" - let portSuffix = port == 443 ? "" : ":\(port)" - return "\(scheme)://\(host)\(portSuffix)" - } - - // Legacy fallback (best-effort): keep existing behavior when we couldn't resolve SRV. - guard let lanHost = self.trimmed(lanHost), !lanHost.isEmpty else { return nil } - let port = gatewayPort ?? 18789 - return "ws://\(lanHost):\(port)" - } - - static func sanitizedTailnetHost(_ host: String?) -> String? { - guard let host = self.trimmed(host), !host.isEmpty else { return nil } - if host.hasSuffix(".internal.") || host.hasSuffix(".internal") { + guard let endpoint = self.serviceEndpoint(serviceHost: serviceHost, servicePort: servicePort) else { return nil } - return host + // Security: for non-loopback hosts, force TLS to avoid plaintext credential/session leakage. + let scheme = self.isLoopbackHost(endpoint.host) ? "ws" : "wss" + let portSuffix = endpoint.port == 443 ? "" : ":\(endpoint.port)" + return "\(scheme)://\(endpoint.host)\(portSuffix)" } private static func trimmed(_ value: String?) -> String? { value?.trimmingCharacters(in: .whitespacesAndNewlines) } + + private static func isLoopbackHost(_ rawHost: String) -> Bool { + let host = rawHost.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() + guard !host.isEmpty else { return false } + if host == "localhost" || host == "::1" || host == "0:0:0:0:0:0:0:1" { + return true + } + if host.hasPrefix("::ffff:127.") { + return true + } + return host.hasPrefix("127.") + } } diff --git a/apps/macos/Sources/OpenClaw/GatewayRemoteConfig.swift b/apps/macos/Sources/OpenClaw/GatewayRemoteConfig.swift index 0b8ab35159d..64a6f92db8f 100644 --- a/apps/macos/Sources/OpenClaw/GatewayRemoteConfig.swift +++ b/apps/macos/Sources/OpenClaw/GatewayRemoteConfig.swift @@ -1,6 +1,41 @@ import Foundation +import Network enum GatewayRemoteConfig { + private static func isLoopbackHost(_ rawHost: String) -> Bool { + var host = rawHost + .trimmingCharacters(in: .whitespacesAndNewlines) + .lowercased() + .trimmingCharacters(in: CharacterSet(charactersIn: "[]")) + if host.hasSuffix(".") { + host.removeLast() + } + if let zoneIndex = host.firstIndex(of: "%") { + host = String(host[.. AppState.RemoteTransport { guard let gateway = root["gateway"] as? [String: Any], let remote = gateway["remote"] as? [String: Any], @@ -39,6 +74,9 @@ enum GatewayRemoteConfig { guard scheme == "ws" || scheme == "wss" else { return nil } let host = url.host?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" guard !host.isEmpty else { return nil } + if scheme == "ws", !self.isLoopbackHost(host) { + return nil + } if scheme == "ws", url.port == nil { guard var components = URLComponents(url: url, resolvingAgainstBaseURL: false) else { return url diff --git a/apps/macos/Sources/OpenClaw/GeneralSettings.swift b/apps/macos/Sources/OpenClaw/GeneralSettings.swift index d55f7c1b015..4dae858771c 100644 --- a/apps/macos/Sources/OpenClaw/GeneralSettings.swift +++ b/apps/macos/Sources/OpenClaw/GeneralSettings.swift @@ -303,7 +303,8 @@ struct GeneralSettings: View { .disabled(self.remoteStatus == .checking || self.state.remoteUrl .trimmingCharacters(in: .whitespacesAndNewlines).isEmpty) } - Text("Direct mode requires a ws:// or wss:// URL (Tailscale Serve uses wss://).") + Text( + "Direct mode requires wss:// for remote hosts. ws:// is only allowed for localhost/127.0.0.1.") .font(.caption) .foregroundStyle(.secondary) .padding(.leading, self.remoteLabelWidth + 10) @@ -546,7 +547,8 @@ extension GeneralSettings { return } guard Self.isValidWsUrl(trimmedUrl) else { - self.remoteStatus = .failed("Gateway URL must start with ws:// or wss://") + self.remoteStatus = .failed( + "Gateway URL must use wss:// for remote hosts (ws:// only for localhost)") return } } else { @@ -603,11 +605,7 @@ extension GeneralSettings { } private static func isValidWsUrl(_ raw: String) -> Bool { - guard let url = URL(string: raw.trimmingCharacters(in: .whitespacesAndNewlines)) else { return false } - let scheme = url.scheme?.lowercased() ?? "" - guard scheme == "ws" || scheme == "wss" else { return false } - let host = url.host?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" - return !host.isEmpty + GatewayRemoteConfig.normalizeGatewayUrl(raw) != nil } private static func sshCheckCommand(target: String, identity: String) -> [String]? { @@ -675,22 +673,17 @@ extension GeneralSettings { private func applyDiscoveredGateway(_ gateway: GatewayDiscoveryModel.DiscoveredGateway) { MacNodeModeCoordinator.shared.setPreferredGatewayStableID(gateway.stableID) - let host = gateway.tailnetDns ?? gateway.lanHost - guard let host else { return } - let user = NSUserName() if self.state.remoteTransport == .direct { - if let url = GatewayDiscoveryHelpers.directUrl(for: gateway) { - self.state.remoteUrl = url - } + self.state.remoteUrl = GatewayDiscoveryHelpers.directUrl(for: gateway) ?? "" } else { - self.state.remoteTarget = GatewayDiscoveryModel.buildSSHTarget( - user: user, - host: host, - port: gateway.sshPort) - self.state.remoteCliPath = gateway.cliPath ?? "" + self.state.remoteTarget = GatewayDiscoveryHelpers.sshTarget(for: gateway) ?? "" + } + if let endpoint = GatewayDiscoveryHelpers.serviceEndpoint(for: gateway) { OpenClawConfigFile.setRemoteGatewayUrl( - host: gateway.serviceHost ?? host, - port: gateway.servicePort ?? gateway.gatewayPort) + host: endpoint.host, + port: endpoint.port) + } else { + OpenClawConfigFile.clearRemoteGatewayUrl() } } } diff --git a/apps/macos/Sources/OpenClaw/HostEnvSanitizer.swift b/apps/macos/Sources/OpenClaw/HostEnvSanitizer.swift new file mode 100644 index 00000000000..e1c4f5b8531 --- /dev/null +++ b/apps/macos/Sources/OpenClaw/HostEnvSanitizer.swift @@ -0,0 +1,66 @@ +import Foundation + +enum HostEnvSanitizer { + /// Generated from src/infra/host-env-security-policy.json via scripts/generate-host-env-security-policy-swift.mjs. + /// Parity is validated by src/infra/host-env-security.policy-parity.test.ts. + private static let blockedKeys = HostEnvSecurityPolicy.blockedKeys + private static let blockedPrefixes = HostEnvSecurityPolicy.blockedPrefixes + private static let blockedOverrideKeys = HostEnvSecurityPolicy.blockedOverrideKeys + private static let shellWrapperAllowedOverrideKeys: Set = [ + "TERM", + "LANG", + "LC_ALL", + "LC_CTYPE", + "LC_MESSAGES", + "COLORTERM", + "NO_COLOR", + "FORCE_COLOR", + ] + + private static func isBlocked(_ upperKey: String) -> Bool { + if self.blockedKeys.contains(upperKey) { return true } + return self.blockedPrefixes.contains(where: { upperKey.hasPrefix($0) }) + } + + private static func filterOverridesForShellWrapper(_ overrides: [String: String]?) -> [String: String]? { + guard let overrides else { return nil } + var filtered: [String: String] = [:] + for (rawKey, value) in overrides { + let key = rawKey.trimmingCharacters(in: .whitespacesAndNewlines) + guard !key.isEmpty else { continue } + if self.shellWrapperAllowedOverrideKeys.contains(key.uppercased()) { + filtered[key] = value + } + } + return filtered.isEmpty ? nil : filtered + } + + static func sanitize(overrides: [String: String]?, shellWrapper: Bool = false) -> [String: String] { + var merged: [String: String] = [:] + for (rawKey, value) in ProcessInfo.processInfo.environment { + let key = rawKey.trimmingCharacters(in: .whitespacesAndNewlines) + guard !key.isEmpty else { continue } + let upper = key.uppercased() + if self.isBlocked(upper) { continue } + merged[key] = value + } + + let effectiveOverrides = shellWrapper + ? self.filterOverridesForShellWrapper(overrides) + : overrides + + guard let effectiveOverrides else { return merged } + for (rawKey, value) in effectiveOverrides { + let key = rawKey.trimmingCharacters(in: .whitespacesAndNewlines) + guard !key.isEmpty else { continue } + let upper = key.uppercased() + // PATH is part of the security boundary (command resolution + safe-bin checks). Never + // allow request-scoped PATH overrides from agents/gateways. + if upper == "PATH" { continue } + if self.blockedOverrideKeys.contains(upper) { continue } + if self.isBlocked(upper) { continue } + merged[key] = value + } + return merged + } +} diff --git a/apps/macos/Sources/OpenClaw/HostEnvSecurityPolicy.generated.swift b/apps/macos/Sources/OpenClaw/HostEnvSecurityPolicy.generated.swift new file mode 100644 index 00000000000..b126d03de21 --- /dev/null +++ b/apps/macos/Sources/OpenClaw/HostEnvSecurityPolicy.generated.swift @@ -0,0 +1,38 @@ +// Generated file. Do not edit directly. +// Source: src/infra/host-env-security-policy.json +// Regenerate: node scripts/generate-host-env-security-policy-swift.mjs --write + +import Foundation + +enum HostEnvSecurityPolicy { + static let blockedKeys: Set = [ + "NODE_OPTIONS", + "NODE_PATH", + "PYTHONHOME", + "PYTHONPATH", + "PERL5LIB", + "PERL5OPT", + "RUBYLIB", + "RUBYOPT", + "BASH_ENV", + "ENV", + "GIT_EXTERNAL_DIFF", + "SHELL", + "SHELLOPTS", + "PS4", + "GCONV_PATH", + "IFS", + "SSLKEYLOGFILE" + ] + + static let blockedOverrideKeys: Set = [ + "HOME", + "ZDOTDIR" + ] + + static let blockedPrefixes: [String] = [ + "DYLD_", + "LD_", + "BASH_FUNC_" + ] +} diff --git a/apps/macos/Sources/OpenClaw/MenuBar.swift b/apps/macos/Sources/OpenClaw/MenuBar.swift index 00e2a9be0a6..d7ab72ce86f 100644 --- a/apps/macos/Sources/OpenClaw/MenuBar.swift +++ b/apps/macos/Sources/OpenClaw/MenuBar.swift @@ -431,7 +431,7 @@ final class SparkleUpdaterController: NSObject, UpdaterProviding { } } -extension SparkleUpdaterController: @preconcurrency SPUUpdaterDelegate {} +extension SparkleUpdaterController: SPUUpdaterDelegate {} private func isDeveloperIDSigned(bundleURL: URL) -> Bool { var staticCode: SecStaticCode? diff --git a/apps/macos/Sources/OpenClaw/MenuSessionsInjector.swift b/apps/macos/Sources/OpenClaw/MenuSessionsInjector.swift index 37fd6ca2505..eb6271d0a8c 100644 --- a/apps/macos/Sources/OpenClaw/MenuSessionsInjector.swift +++ b/apps/macos/Sources/OpenClaw/MenuSessionsInjector.swift @@ -446,6 +446,8 @@ extension MenuSessionsInjector { private func buildUsageOverflowMenu(rows: [UsageRow], width: CGFloat) -> NSMenu { let menu = NSMenu() + // Keep submenu delegate nil: reusing the status-menu delegate here causes + // recursive reinjection whenever this submenu is opened. for row in rows { let item = NSMenuItem() item.tag = self.tag @@ -493,7 +495,6 @@ extension MenuSessionsInjector { guard !summary.daily.isEmpty else { return nil } let menu = NSMenu() - menu.delegate = self let chartView = CostUsageHistoryMenuView(summary: summary, width: width) let hosting = NSHostingView(rootView: AnyView(chartView)) @@ -1226,6 +1227,12 @@ extension MenuSessionsInjector { self.usageCacheUpdatedAt = Date() } + func setTestingCostUsageSummary(_ summary: GatewayCostUsageSummary?, errorText: String? = nil) { + self.cachedCostSummary = summary + self.cachedCostErrorText = errorText + self.costCacheUpdatedAt = Date() + } + func injectForTesting(into menu: NSMenu) { self.inject(into: menu) } diff --git a/apps/macos/Sources/OpenClaw/MicLevelMonitor.swift b/apps/macos/Sources/OpenClaw/MicLevelMonitor.swift index e35057d28cf..81e06abda2d 100644 --- a/apps/macos/Sources/OpenClaw/MicLevelMonitor.swift +++ b/apps/macos/Sources/OpenClaw/MicLevelMonitor.swift @@ -14,6 +14,13 @@ actor MicLevelMonitor { if self.running { return } self.logger.info( "mic level monitor start (\(AudioInputDeviceObserver.defaultInputDeviceSummary(), privacy: .public))") + guard AudioInputDeviceObserver.hasUsableDefaultInputDevice() else { + self.engine = nil + throw NSError( + domain: "MicLevelMonitor", + code: 1, + userInfo: [NSLocalizedDescriptionKey: "No usable audio input device available"]) + } let engine = AVAudioEngine() self.engine = engine let input = engine.inputNode diff --git a/apps/macos/Sources/OpenClaw/NodeMode/MacNodeRuntime.swift b/apps/macos/Sources/OpenClaw/NodeMode/MacNodeRuntime.swift index 60bd95f2894..cda8ca6057c 100644 --- a/apps/macos/Sources/OpenClaw/NodeMode/MacNodeRuntime.swift +++ b/apps/macos/Sources/OpenClaw/NodeMode/MacNodeRuntime.swift @@ -441,43 +441,25 @@ actor MacNodeRuntime { guard !command.isEmpty else { return Self.errorResponse(req, code: .invalidRequest, message: "INVALID_REQUEST: command required") } - let displayCommand = ExecCommandFormatter.displayString(for: command, rawCommand: params.rawCommand) - - let trimmedAgent = params.agentId?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" - let agentId = trimmedAgent.isEmpty ? nil : trimmedAgent - let approvals = ExecApprovalsStore.resolve(agentId: agentId) - let security = approvals.agent.security - let ask = approvals.agent.ask - let autoAllowSkills = approvals.agent.autoAllowSkills let sessionKey = (params.sessionKey?.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty == false) ? params.sessionKey!.trimmingCharacters(in: .whitespacesAndNewlines) : self.mainSessionKey let runId = UUID().uuidString - let env = Self.sanitizedEnv(params.env) - let resolution = ExecCommandResolution.resolve( + let evaluation = await ExecApprovalEvaluator.evaluate( command: command, rawCommand: params.rawCommand, cwd: params.cwd, - env: env) - let allowlistMatch = security == .allowlist - ? ExecAllowlistMatcher.match(entries: approvals.allowlist, resolution: resolution) - : nil - let skillAllow: Bool - if autoAllowSkills, let name = resolution?.executableName { - let bins = await SkillBinsCache.shared.currentBins() - skillAllow = bins.contains(name) - } else { - skillAllow = false - } + envOverrides: params.env, + agentId: params.agentId) - if security == .deny { + if evaluation.security == .deny { await self.emitExecEvent( "exec.denied", payload: ExecEventPayload( sessionKey: sessionKey, runId: runId, host: "node", - command: displayCommand, + command: evaluation.displayCommand, reason: "security=deny")) return Self.errorResponse( req, @@ -489,32 +471,33 @@ actor MacNodeRuntime { req: req, params: params, context: ExecRunContext( - displayCommand: displayCommand, - security: security, - ask: ask, - agentId: agentId, - resolution: resolution, - allowlistMatch: allowlistMatch, - skillAllow: skillAllow, + displayCommand: evaluation.displayCommand, + security: evaluation.security, + ask: evaluation.ask, + agentId: evaluation.agentId, + resolution: evaluation.resolution, + allowlistMatch: evaluation.allowlistMatch, + skillAllow: evaluation.skillAllow, sessionKey: sessionKey, runId: runId)) if let response = approval.response { return response } let approvedByAsk = approval.approvedByAsk let persistAllowlist = approval.persistAllowlist - if persistAllowlist, security == .allowlist, - let pattern = ExecApprovalHelpers.allowlistPattern(command: command, resolution: resolution) - { - ExecApprovalsStore.addAllowlistEntry(agentId: agentId, pattern: pattern) - } + self.persistAllowlistPatterns( + persistAllowlist: persistAllowlist, + security: evaluation.security, + agentId: evaluation.agentId, + command: command, + allowlistResolutions: evaluation.allowlistResolutions) - if security == .allowlist, allowlistMatch == nil, !skillAllow, !approvedByAsk { + if evaluation.security == .allowlist, !evaluation.allowlistSatisfied, !evaluation.skillAllow, !approvedByAsk { await self.emitExecEvent( "exec.denied", payload: ExecEventPayload( sessionKey: sessionKey, runId: runId, host: "node", - command: displayCommand, + command: evaluation.displayCommand, reason: "allowlist-miss")) return Self.errorResponse( req, @@ -522,79 +505,32 @@ actor MacNodeRuntime { message: "SYSTEM_RUN_DENIED: allowlist miss") } - if let match = allowlistMatch { - ExecApprovalsStore.recordAllowlistUse( - agentId: agentId, - pattern: match.pattern, - command: displayCommand, - resolvedPath: resolution?.resolvedPath) + self.recordAllowlistMatches( + security: evaluation.security, + allowlistSatisfied: evaluation.allowlistSatisfied, + agentId: evaluation.agentId, + allowlistMatches: evaluation.allowlistMatches, + allowlistResolutions: evaluation.allowlistResolutions, + displayCommand: evaluation.displayCommand) + + if let permissionResponse = await self.validateScreenRecordingIfNeeded( + req: req, + needsScreenRecording: params.needsScreenRecording, + sessionKey: sessionKey, + runId: runId, + displayCommand: evaluation.displayCommand) + { + return permissionResponse } - if params.needsScreenRecording == true { - let authorized = await PermissionManager - .status([.screenRecording])[.screenRecording] ?? false - if !authorized { - await self.emitExecEvent( - "exec.denied", - payload: ExecEventPayload( - sessionKey: sessionKey, - runId: runId, - host: "node", - command: displayCommand, - reason: "permission:screenRecording")) - return Self.errorResponse( - req, - code: .unavailable, - message: "PERMISSION_MISSING: screenRecording") - } - } - - let timeoutSec = params.timeoutMs.flatMap { Double($0) / 1000.0 } - await self.emitExecEvent( - "exec.started", - payload: ExecEventPayload( - sessionKey: sessionKey, - runId: runId, - host: "node", - command: displayCommand)) - let result = await ShellExecutor.runDetailed( + return try await self.executeSystemRun( + req: req, + params: params, command: command, - cwd: params.cwd, - env: env, - timeout: timeoutSec) - let combined = [result.stdout, result.stderr, result.errorMessage] - .compactMap(\.self) - .filter { !$0.isEmpty } - .joined(separator: "\n") - await self.emitExecEvent( - "exec.finished", - payload: ExecEventPayload( - sessionKey: sessionKey, - runId: runId, - host: "node", - command: displayCommand, - exitCode: result.exitCode, - timedOut: result.timedOut, - success: result.success, - output: ExecEventPayload.truncateOutput(combined))) - - struct RunPayload: Encodable { - var exitCode: Int? - var timedOut: Bool - var success: Bool - var stdout: String - var stderr: String - var error: String? - } - - let payload = try Self.encodePayload(RunPayload( - exitCode: result.exitCode, - timedOut: result.timedOut, - success: result.success, - stdout: result.stdout, - stderr: result.stderr, - error: result.errorMessage)) - return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: payload) + env: evaluation.env, + sessionKey: sessionKey, + runId: runId, + displayCommand: evaluation.displayCommand) } private func handleSystemWhich(_ req: BridgeInvokeRequest) async throws -> BridgeInvokeResponse { @@ -835,6 +771,132 @@ actor MacNodeRuntime { } extension MacNodeRuntime { + private func persistAllowlistPatterns( + persistAllowlist: Bool, + security: ExecSecurity, + agentId: String?, + command: [String], + allowlistResolutions: [ExecCommandResolution]) + { + guard persistAllowlist, security == .allowlist else { return } + var seenPatterns = Set() + for candidate in allowlistResolutions { + guard let pattern = ExecApprovalHelpers.allowlistPattern(command: command, resolution: candidate) else { + continue + } + if seenPatterns.insert(pattern).inserted { + ExecApprovalsStore.addAllowlistEntry(agentId: agentId, pattern: pattern) + } + } + } + + private func recordAllowlistMatches( + security: ExecSecurity, + allowlistSatisfied: Bool, + agentId: String?, + allowlistMatches: [ExecAllowlistEntry], + allowlistResolutions: [ExecCommandResolution], + displayCommand: String) + { + guard security == .allowlist, allowlistSatisfied else { return } + var seenPatterns = Set() + for (idx, match) in allowlistMatches.enumerated() { + if !seenPatterns.insert(match.pattern).inserted { + continue + } + let resolvedPath = idx < allowlistResolutions.count ? allowlistResolutions[idx].resolvedPath : nil + ExecApprovalsStore.recordAllowlistUse( + agentId: agentId, + pattern: match.pattern, + command: displayCommand, + resolvedPath: resolvedPath) + } + } + + private func validateScreenRecordingIfNeeded( + req: BridgeInvokeRequest, + needsScreenRecording: Bool?, + sessionKey: String, + runId: String, + displayCommand: String) async -> BridgeInvokeResponse? + { + guard needsScreenRecording == true else { return nil } + let authorized = await PermissionManager + .status([.screenRecording])[.screenRecording] ?? false + if authorized { + return nil + } + await self.emitExecEvent( + "exec.denied", + payload: ExecEventPayload( + sessionKey: sessionKey, + runId: runId, + host: "node", + command: displayCommand, + reason: "permission:screenRecording")) + return Self.errorResponse( + req, + code: .unavailable, + message: "PERMISSION_MISSING: screenRecording") + } + + private func executeSystemRun( + req: BridgeInvokeRequest, + params: OpenClawSystemRunParams, + command: [String], + env: [String: String], + sessionKey: String, + runId: String, + displayCommand: String) async throws -> BridgeInvokeResponse + { + let timeoutSec = params.timeoutMs.flatMap { Double($0) / 1000.0 } + await self.emitExecEvent( + "exec.started", + payload: ExecEventPayload( + sessionKey: sessionKey, + runId: runId, + host: "node", + command: displayCommand)) + let result = await ShellExecutor.runDetailed( + command: command, + cwd: params.cwd, + env: env, + timeout: timeoutSec) + let combined = [result.stdout, result.stderr, result.errorMessage] + .compactMap(\.self) + .filter { !$0.isEmpty } + .joined(separator: "\n") + await self.emitExecEvent( + "exec.finished", + payload: ExecEventPayload( + sessionKey: sessionKey, + runId: runId, + host: "node", + command: displayCommand, + exitCode: result.exitCode, + timedOut: result.timedOut, + success: result.success, + output: ExecEventPayload.truncateOutput(combined))) + + struct RunPayload: Encodable { + var exitCode: Int? + var timedOut: Bool + var success: Bool + var stdout: String + var stderr: String + var error: String? + } + let runPayload = RunPayload( + exitCode: result.exitCode, + timedOut: result.timedOut, + success: result.success, + stdout: result.stdout, + stderr: result.stderr, + error: result.errorMessage) + let payload = try Self.encodePayload(runPayload) + return BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: payload) + } + private static func decodeParams(_ type: T.Type, from json: String?) throws -> T { guard let json, let data = json.data(using: .utf8) else { throw NSError(domain: "Gateway", code: 20, userInfo: [ @@ -862,35 +924,6 @@ extension MacNodeRuntime { UserDefaults.standard.object(forKey: cameraEnabledKey) as? Bool ?? false } - private static let blockedEnvKeys: Set = [ - "PATH", - "NODE_OPTIONS", - "PYTHONHOME", - "PYTHONPATH", - "PERL5LIB", - "PERL5OPT", - "RUBYOPT", - ] - - private static let blockedEnvPrefixes: [String] = [ - "DYLD_", - "LD_", - ] - - private static func sanitizedEnv(_ overrides: [String: String]?) -> [String: String]? { - guard let overrides else { return nil } - var merged = ProcessInfo.processInfo.environment - for (rawKey, value) in overrides { - let key = rawKey.trimmingCharacters(in: .whitespacesAndNewlines) - guard !key.isEmpty else { continue } - let upper = key.uppercased() - if self.blockedEnvKeys.contains(upper) { continue } - if self.blockedEnvPrefixes.contains(where: { upper.hasPrefix($0) }) { continue } - merged[key] = value - } - return merged - } - private nonisolated static func locationMode() -> OpenClawLocationMode { let raw = UserDefaults.standard.string(forKey: locationModeKey) ?? "off" return OpenClawLocationMode(rawValue: raw) ?? .off diff --git a/apps/macos/Sources/OpenClaw/NodePairingApprovalPrompter.swift b/apps/macos/Sources/OpenClaw/NodePairingApprovalPrompter.swift index ee994b38f65..10598d7f4be 100644 --- a/apps/macos/Sources/OpenClaw/NodePairingApprovalPrompter.swift +++ b/apps/macos/Sources/OpenClaw/NodePairingApprovalPrompter.swift @@ -520,11 +520,12 @@ final class NodePairingApprovalPrompter { let preferred = GatewayDiscoveryPreferences.preferredStableID() let gateway = model.gateways.first { $0.stableID == preferred } ?? model.gateways.first guard let gateway else { return nil } - let host = (gateway.tailnetDns?.trimmingCharacters(in: .whitespacesAndNewlines).nonEmpty ?? - gateway.lanHost?.trimmingCharacters(in: .whitespacesAndNewlines).nonEmpty) - guard let host, !host.isEmpty else { return nil } - let port = gateway.sshPort > 0 ? gateway.sshPort : 22 - return SSHTarget(host: host, port: port) + guard let target = GatewayDiscoveryHelpers.sshTarget(for: gateway), + let parsed = CommandResolver.parseSSHTarget(target) + else { + return nil + } + return SSHTarget(host: parsed.host, port: parsed.port) } private static func probeSSH(user: String, host: String, port: Int) async -> Bool { diff --git a/apps/macos/Sources/OpenClaw/Onboarding.swift b/apps/macos/Sources/OpenClaw/Onboarding.swift index b8a6377b419..4eae7e092b0 100644 --- a/apps/macos/Sources/OpenClaw/Onboarding.swift +++ b/apps/macos/Sources/OpenClaw/Onboarding.swift @@ -1,5 +1,4 @@ import AppKit -import Combine import Observation import OpenClawChatUI import OpenClawDiscovery @@ -69,22 +68,6 @@ struct OnboardingView: View { @State var workspacePath: String = "" @State var workspaceStatus: String? @State var workspaceApplying = false - @State var anthropicAuthPKCE: AnthropicOAuth.PKCE? - @State var anthropicAuthCode: String = "" - @State var anthropicAuthStatus: String? - @State var anthropicAuthBusy = false - @State var anthropicAuthConnected = false - @State var anthropicAuthVerifying = false - @State var anthropicAuthVerified = false - @State var anthropicAuthVerificationAttempted = false - @State var anthropicAuthVerificationFailed = false - @State var anthropicAuthVerifiedAt: Date? - @State var anthropicAuthDetectedStatus: OpenClawOAuthStore.AnthropicOAuthStatus = .missingFile - @State var anthropicAuthAutoDetectClipboard = true - @State var anthropicAuthAutoConnectClipboard = true - @State var anthropicAuthLastPasteboardChangeCount = NSPasteboard.general.changeCount - @State var monitoringAuth = false - @State var authMonitorTask: Task? @State var needsBootstrap = false @State var didAutoKickoff = false @State var showAdvancedConnection = false @@ -104,19 +87,9 @@ struct OnboardingView: View { let pageWidth: CGFloat = Self.windowWidth let contentHeight: CGFloat = 460 let connectionPageIndex = 1 - let anthropicAuthPageIndex = 2 let wizardPageIndex = 3 let onboardingChatPageIndex = 8 - static let clipboardPoll: AnyPublisher = { - if ProcessInfo.processInfo.isRunningTests { - return Empty(completeImmediately: false).eraseToAnyPublisher() - } - return Timer.publish(every: 0.4, on: .main, in: .common) - .autoconnect() - .eraseToAnyPublisher() - }() - let permissionsPageIndex = 5 static func pageOrder( for mode: AppState.ConnectionMode, diff --git a/apps/macos/Sources/OpenClaw/OnboardingView+Actions.swift b/apps/macos/Sources/OpenClaw/OnboardingView+Actions.swift index ba43424aa9a..a521926ddb9 100644 --- a/apps/macos/Sources/OpenClaw/OnboardingView+Actions.swift +++ b/apps/macos/Sources/OpenClaw/OnboardingView+Actions.swift @@ -26,20 +26,17 @@ extension OnboardingView { GatewayDiscoveryPreferences.setPreferredStableID(gateway.stableID) if self.state.remoteTransport == .direct { - if let url = GatewayDiscoveryHelpers.directUrl(for: gateway) { - self.state.remoteUrl = url - } - } else if let host = GatewayDiscoveryHelpers.sanitizedTailnetHost(gateway.tailnetDns) ?? gateway.lanHost { - let user = NSUserName() - self.state.remoteTarget = GatewayDiscoveryModel.buildSSHTarget( - user: user, - host: host, - port: gateway.sshPort) - OpenClawConfigFile.setRemoteGatewayUrl( - host: gateway.serviceHost ?? host, - port: gateway.servicePort ?? gateway.gatewayPort) + self.state.remoteUrl = GatewayDiscoveryHelpers.directUrl(for: gateway) ?? "" + } else { + self.state.remoteTarget = GatewayDiscoveryHelpers.sshTarget(for: gateway) ?? "" + } + if let endpoint = GatewayDiscoveryHelpers.serviceEndpoint(for: gateway) { + OpenClawConfigFile.setRemoteGatewayUrl( + host: endpoint.host, + port: endpoint.port) + } else { + OpenClawConfigFile.clearRemoteGatewayUrl() } - self.state.remoteCliPath = gateway.cliPath ?? "" self.state.connectionMode = .remote MacNodeModeCoordinator.shared.setPreferredGatewayStableID(gateway.stableID) @@ -81,70 +78,4 @@ extension OnboardingView { self.copied = true DispatchQueue.main.asyncAfter(deadline: .now() + 1.2) { self.copied = false } } - - func startAnthropicOAuth() { - guard !self.anthropicAuthBusy else { return } - self.anthropicAuthBusy = true - defer { self.anthropicAuthBusy = false } - - do { - let pkce = try AnthropicOAuth.generatePKCE() - self.anthropicAuthPKCE = pkce - let url = AnthropicOAuth.buildAuthorizeURL(pkce: pkce) - NSWorkspace.shared.open(url) - self.anthropicAuthStatus = "Browser opened. After approving, paste the `code#state` value here." - } catch { - self.anthropicAuthStatus = "Failed to start OAuth: \(error.localizedDescription)" - } - } - - @MainActor - func finishAnthropicOAuth() async { - guard !self.anthropicAuthBusy else { return } - guard let pkce = self.anthropicAuthPKCE else { return } - self.anthropicAuthBusy = true - defer { self.anthropicAuthBusy = false } - - guard let parsed = AnthropicOAuthCodeState.parse(from: self.anthropicAuthCode) else { - self.anthropicAuthStatus = "OAuth failed: missing or invalid code/state." - return - } - - do { - let creds = try await AnthropicOAuth.exchangeCode( - code: parsed.code, - state: parsed.state, - verifier: pkce.verifier) - try OpenClawOAuthStore.saveAnthropicOAuth(creds) - self.refreshAnthropicOAuthStatus() - self.anthropicAuthStatus = "Connected. OpenClaw can now use Claude." - } catch { - self.anthropicAuthStatus = "OAuth failed: \(error.localizedDescription)" - } - } - - func pollAnthropicClipboardIfNeeded() { - guard self.currentPage == self.anthropicAuthPageIndex else { return } - guard self.anthropicAuthPKCE != nil else { return } - guard !self.anthropicAuthBusy else { return } - guard self.anthropicAuthAutoDetectClipboard else { return } - - let pb = NSPasteboard.general - let changeCount = pb.changeCount - guard changeCount != self.anthropicAuthLastPasteboardChangeCount else { return } - self.anthropicAuthLastPasteboardChangeCount = changeCount - - guard let raw = pb.string(forType: .string), !raw.isEmpty else { return } - guard let parsed = AnthropicOAuthCodeState.parse(from: raw) else { return } - guard let pkce = self.anthropicAuthPKCE, parsed.state == pkce.verifier else { return } - - let next = "\(parsed.code)#\(parsed.state)" - if self.anthropicAuthCode != next { - self.anthropicAuthCode = next - self.anthropicAuthStatus = "Detected `code#state` from clipboard." - } - - guard self.anthropicAuthAutoConnectClipboard else { return } - Task { await self.finishAnthropicOAuth() } - } } diff --git a/apps/macos/Sources/OpenClaw/OnboardingView+Layout.swift b/apps/macos/Sources/OpenClaw/OnboardingView+Layout.swift index ce87e211ce4..9b0e45e205c 100644 --- a/apps/macos/Sources/OpenClaw/OnboardingView+Layout.swift +++ b/apps/macos/Sources/OpenClaw/OnboardingView+Layout.swift @@ -53,7 +53,6 @@ extension OnboardingView { .onDisappear { self.stopPermissionMonitoring() self.stopDiscovery() - self.stopAuthMonitoring() Task { await self.onboardingWizard.cancelIfRunning() } } .task { @@ -61,7 +60,6 @@ extension OnboardingView { self.refreshCLIStatus() await self.loadWorkspaceDefaults() await self.ensureDefaultWorkspace() - self.refreshAnthropicOAuthStatus() self.refreshBootstrapStatus() self.preferredGatewayID = GatewayDiscoveryPreferences.preferredStableID() } diff --git a/apps/macos/Sources/OpenClaw/OnboardingView+Monitoring.swift b/apps/macos/Sources/OpenClaw/OnboardingView+Monitoring.swift index dfbdf91d44d..efe37f31673 100644 --- a/apps/macos/Sources/OpenClaw/OnboardingView+Monitoring.swift +++ b/apps/macos/Sources/OpenClaw/OnboardingView+Monitoring.swift @@ -47,7 +47,6 @@ extension OnboardingView { func updateMonitoring(for pageIndex: Int) { self.updatePermissionMonitoring(for: pageIndex) self.updateDiscoveryMonitoring(for: pageIndex) - self.updateAuthMonitoring(for: pageIndex) self.maybeKickoffOnboardingChat(for: pageIndex) } @@ -63,33 +62,6 @@ extension OnboardingView { self.gatewayDiscovery.stop() } - func updateAuthMonitoring(for pageIndex: Int) { - let shouldMonitor = pageIndex == self.anthropicAuthPageIndex && self.state.connectionMode == .local - if shouldMonitor, !self.monitoringAuth { - self.monitoringAuth = true - self.startAuthMonitoring() - } else if !shouldMonitor, self.monitoringAuth { - self.stopAuthMonitoring() - } - } - - func startAuthMonitoring() { - self.refreshAnthropicOAuthStatus() - self.authMonitorTask?.cancel() - self.authMonitorTask = Task { - while !Task.isCancelled { - await MainActor.run { self.refreshAnthropicOAuthStatus() } - try? await Task.sleep(nanoseconds: 1_000_000_000) - } - } - } - - func stopAuthMonitoring() { - self.monitoringAuth = false - self.authMonitorTask?.cancel() - self.authMonitorTask = nil - } - func installCLI() async { guard !self.installingCLI else { return } self.installingCLI = true @@ -125,54 +97,4 @@ extension OnboardingView { expected: expected) } } - - func refreshAnthropicOAuthStatus() { - _ = OpenClawOAuthStore.importLegacyAnthropicOAuthIfNeeded() - let previous = self.anthropicAuthDetectedStatus - let status = OpenClawOAuthStore.anthropicOAuthStatus() - self.anthropicAuthDetectedStatus = status - self.anthropicAuthConnected = status.isConnected - - if previous != status { - self.anthropicAuthVerified = false - self.anthropicAuthVerificationAttempted = false - self.anthropicAuthVerificationFailed = false - self.anthropicAuthVerifiedAt = nil - } - } - - @MainActor - func verifyAnthropicOAuthIfNeeded(force: Bool = false) async { - guard self.state.connectionMode == .local else { return } - guard self.anthropicAuthDetectedStatus.isConnected else { return } - if self.anthropicAuthVerified, !force { return } - if self.anthropicAuthVerifying { return } - if self.anthropicAuthVerificationAttempted, !force { return } - - self.anthropicAuthVerificationAttempted = true - self.anthropicAuthVerifying = true - self.anthropicAuthVerificationFailed = false - defer { self.anthropicAuthVerifying = false } - - guard let refresh = OpenClawOAuthStore.loadAnthropicOAuthRefreshToken(), !refresh.isEmpty else { - self.anthropicAuthStatus = "OAuth verification failed: missing refresh token." - self.anthropicAuthVerificationFailed = true - return - } - - do { - let updated = try await AnthropicOAuth.refresh(refreshToken: refresh) - try OpenClawOAuthStore.saveAnthropicOAuth(updated) - self.refreshAnthropicOAuthStatus() - self.anthropicAuthVerified = true - self.anthropicAuthVerifiedAt = Date() - self.anthropicAuthVerificationFailed = false - self.anthropicAuthStatus = "OAuth detected and verified." - } catch { - self.anthropicAuthVerified = false - self.anthropicAuthVerifiedAt = nil - self.anthropicAuthVerificationFailed = true - self.anthropicAuthStatus = "OAuth verification failed: \(error.localizedDescription)" - } - } } diff --git a/apps/macos/Sources/OpenClaw/OnboardingView+Pages.swift b/apps/macos/Sources/OpenClaw/OnboardingView+Pages.swift index 5760bfff8c2..4f942dfe8a4 100644 --- a/apps/macos/Sources/OpenClaw/OnboardingView+Pages.swift +++ b/apps/macos/Sources/OpenClaw/OnboardingView+Pages.swift @@ -12,8 +12,6 @@ extension OnboardingView { self.welcomePage() case 1: self.connectionPage() - case 2: - self.anthropicAuthPage() case 3: self.wizardPage() case 5: @@ -87,19 +85,9 @@ extension OnboardingView { self.onboardingCard(spacing: 12, padding: 14) { VStack(alignment: .leading, spacing: 10) { - let localSubtitle: String = { - guard let probe = self.localGatewayProbe else { - return "Gateway starts automatically on this Mac." - } - let base = probe.expected - ? "Existing gateway detected" - : "Port \(probe.port) already in use" - let command = probe.command.isEmpty ? "" : " (\(probe.command) pid \(probe.pid))" - return "\(base)\(command). Will attach." - }() self.connectionChoiceButton( title: "This Mac", - subtitle: localSubtitle, + subtitle: self.localGatewaySubtitle, selected: self.state.connectionMode == .local) { self.selectLocalGateway() @@ -107,50 +95,7 @@ extension OnboardingView { Divider().padding(.vertical, 4) - HStack(spacing: 8) { - Image(systemName: "dot.radiowaves.left.and.right") - .font(.caption) - .foregroundStyle(.secondary) - Text(self.gatewayDiscovery.statusText) - .font(.caption) - .foregroundStyle(.secondary) - if self.gatewayDiscovery.gateways.isEmpty { - ProgressView().controlSize(.small) - Button("Refresh") { - self.gatewayDiscovery.refreshWideAreaFallbackNow(timeoutSeconds: 5.0) - } - .buttonStyle(.link) - .help("Retry Tailscale discovery (DNS-SD).") - } - Spacer(minLength: 0) - } - - if self.gatewayDiscovery.gateways.isEmpty { - Text("Searching for nearby gateways…") - .font(.caption) - .foregroundStyle(.secondary) - .padding(.leading, 4) - } else { - VStack(alignment: .leading, spacing: 6) { - Text("Nearby gateways") - .font(.caption) - .foregroundStyle(.secondary) - .padding(.leading, 4) - ForEach(self.gatewayDiscovery.gateways.prefix(6)) { gateway in - self.connectionChoiceButton( - title: gateway.displayName, - subtitle: self.gatewaySubtitle(for: gateway), - selected: self.isSelectedGateway(gateway)) - { - self.selectRemoteGateway(gateway) - } - } - } - .padding(8) - .background( - RoundedRectangle(cornerRadius: 10, style: .continuous) - .fill(Color(NSColor.controlBackgroundColor))) - } + self.gatewayDiscoverySection() self.connectionChoiceButton( title: "Configure later", @@ -160,104 +105,168 @@ extension OnboardingView { self.selectUnconfiguredGateway() } - Button(self.showAdvancedConnection ? "Hide Advanced" : "Advanced…") { - withAnimation(.spring(response: 0.25, dampingFraction: 0.9)) { - self.showAdvancedConnection.toggle() - } - if self.showAdvancedConnection, self.state.connectionMode != .remote { - self.state.connectionMode = .remote - } - } - .buttonStyle(.link) + self.advancedConnectionSection() + } + } + } + } - if self.showAdvancedConnection { - let labelWidth: CGFloat = 110 - let fieldWidth: CGFloat = 320 + private var localGatewaySubtitle: String { + guard let probe = self.localGatewayProbe else { + return "Gateway starts automatically on this Mac." + } + let base = probe.expected + ? "Existing gateway detected" + : "Port \(probe.port) already in use" + let command = probe.command.isEmpty ? "" : " (\(probe.command) pid \(probe.pid))" + return "\(base)\(command). Will attach." + } - VStack(alignment: .leading, spacing: 10) { - Grid(alignment: .leading, horizontalSpacing: 12, verticalSpacing: 8) { - GridRow { - Text("Transport") - .font(.callout.weight(.semibold)) - .frame(width: labelWidth, alignment: .leading) - Picker("Transport", selection: self.$state.remoteTransport) { - Text("SSH tunnel").tag(AppState.RemoteTransport.ssh) - Text("Direct (ws/wss)").tag(AppState.RemoteTransport.direct) - } - .pickerStyle(.segmented) - .frame(width: fieldWidth) - } - if self.state.remoteTransport == .direct { - GridRow { - Text("Gateway URL") - .font(.callout.weight(.semibold)) - .frame(width: labelWidth, alignment: .leading) - TextField("wss://gateway.example.ts.net", text: self.$state.remoteUrl) - .textFieldStyle(.roundedBorder) - .frame(width: fieldWidth) - } - } - if self.state.remoteTransport == .ssh { - GridRow { - Text("SSH target") - .font(.callout.weight(.semibold)) - .frame(width: labelWidth, alignment: .leading) - TextField("user@host[:port]", text: self.$state.remoteTarget) - .textFieldStyle(.roundedBorder) - .frame(width: fieldWidth) - } - if let message = CommandResolver - .sshTargetValidationMessage(self.state.remoteTarget) - { - GridRow { - Text("") - .frame(width: labelWidth, alignment: .leading) - Text(message) - .font(.caption) - .foregroundStyle(.red) - .frame(width: fieldWidth, alignment: .leading) - } - } - GridRow { - Text("Identity file") - .font(.callout.weight(.semibold)) - .frame(width: labelWidth, alignment: .leading) - TextField("/Users/you/.ssh/id_ed25519", text: self.$state.remoteIdentity) - .textFieldStyle(.roundedBorder) - .frame(width: fieldWidth) - } - GridRow { - Text("Project root") - .font(.callout.weight(.semibold)) - .frame(width: labelWidth, alignment: .leading) - TextField("/home/you/Projects/openclaw", text: self.$state.remoteProjectRoot) - .textFieldStyle(.roundedBorder) - .frame(width: fieldWidth) - } - GridRow { - Text("CLI path") - .font(.callout.weight(.semibold)) - .frame(width: labelWidth, alignment: .leading) - TextField( - "/Applications/OpenClaw.app/.../openclaw", - text: self.$state.remoteCliPath) - .textFieldStyle(.roundedBorder) - .frame(width: fieldWidth) - } - } - } + @ViewBuilder + private func gatewayDiscoverySection() -> some View { + HStack(spacing: 8) { + Image(systemName: "dot.radiowaves.left.and.right") + .font(.caption) + .foregroundStyle(.secondary) + Text(self.gatewayDiscovery.statusText) + .font(.caption) + .foregroundStyle(.secondary) + if self.gatewayDiscovery.gateways.isEmpty { + ProgressView().controlSize(.small) + Button("Refresh") { + self.gatewayDiscovery.refreshWideAreaFallbackNow(timeoutSeconds: 5.0) + } + .buttonStyle(.link) + .help("Retry Tailscale discovery (DNS-SD).") + } + Spacer(minLength: 0) + } - Text(self.state.remoteTransport == .direct - ? "Tip: use Tailscale Serve so the gateway has a valid HTTPS cert." - : "Tip: keep Tailscale enabled so your gateway stays reachable.") - .font(.footnote) - .foregroundStyle(.secondary) - .lineLimit(1) - } - .transition(.opacity.combined(with: .move(edge: .top))) + if self.gatewayDiscovery.gateways.isEmpty { + Text("Searching for nearby gateways…") + .font(.caption) + .foregroundStyle(.secondary) + .padding(.leading, 4) + } else { + VStack(alignment: .leading, spacing: 6) { + Text("Nearby gateways") + .font(.caption) + .foregroundStyle(.secondary) + .padding(.leading, 4) + ForEach(self.gatewayDiscovery.gateways.prefix(6)) { gateway in + self.connectionChoiceButton( + title: gateway.displayName, + subtitle: self.gatewaySubtitle(for: gateway), + selected: self.isSelectedGateway(gateway)) + { + self.selectRemoteGateway(gateway) } } } + .padding(8) + .background( + RoundedRectangle(cornerRadius: 10, style: .continuous) + .fill(Color(NSColor.controlBackgroundColor))) + } + } + + @ViewBuilder + private func advancedConnectionSection() -> some View { + Button(self.showAdvancedConnection ? "Hide Advanced" : "Advanced…") { + withAnimation(.spring(response: 0.25, dampingFraction: 0.9)) { + self.showAdvancedConnection.toggle() + } + if self.showAdvancedConnection, self.state.connectionMode != .remote { + self.state.connectionMode = .remote + } + } + .buttonStyle(.link) + + if self.showAdvancedConnection { + let labelWidth: CGFloat = 110 + let fieldWidth: CGFloat = 320 + + VStack(alignment: .leading, spacing: 10) { + Grid(alignment: .leading, horizontalSpacing: 12, verticalSpacing: 8) { + GridRow { + Text("Transport") + .font(.callout.weight(.semibold)) + .frame(width: labelWidth, alignment: .leading) + Picker("Transport", selection: self.$state.remoteTransport) { + Text("SSH tunnel").tag(AppState.RemoteTransport.ssh) + Text("Direct (ws/wss)").tag(AppState.RemoteTransport.direct) + } + .pickerStyle(.segmented) + .frame(width: fieldWidth) + } + if self.state.remoteTransport == .direct { + GridRow { + Text("Gateway URL") + .font(.callout.weight(.semibold)) + .frame(width: labelWidth, alignment: .leading) + TextField("wss://gateway.example.ts.net", text: self.$state.remoteUrl) + .textFieldStyle(.roundedBorder) + .frame(width: fieldWidth) + } + } + if self.state.remoteTransport == .ssh { + GridRow { + Text("SSH target") + .font(.callout.weight(.semibold)) + .frame(width: labelWidth, alignment: .leading) + TextField("user@host[:port]", text: self.$state.remoteTarget) + .textFieldStyle(.roundedBorder) + .frame(width: fieldWidth) + } + if let message = CommandResolver + .sshTargetValidationMessage(self.state.remoteTarget) + { + GridRow { + Text("") + .frame(width: labelWidth, alignment: .leading) + Text(message) + .font(.caption) + .foregroundStyle(.red) + .frame(width: fieldWidth, alignment: .leading) + } + } + GridRow { + Text("Identity file") + .font(.callout.weight(.semibold)) + .frame(width: labelWidth, alignment: .leading) + TextField("/Users/you/.ssh/id_ed25519", text: self.$state.remoteIdentity) + .textFieldStyle(.roundedBorder) + .frame(width: fieldWidth) + } + GridRow { + Text("Project root") + .font(.callout.weight(.semibold)) + .frame(width: labelWidth, alignment: .leading) + TextField("/home/you/Projects/openclaw", text: self.$state.remoteProjectRoot) + .textFieldStyle(.roundedBorder) + .frame(width: fieldWidth) + } + GridRow { + Text("CLI path") + .font(.callout.weight(.semibold)) + .frame(width: labelWidth, alignment: .leading) + TextField( + "/Applications/OpenClaw.app/.../openclaw", + text: self.$state.remoteCliPath) + .textFieldStyle(.roundedBorder) + .frame(width: fieldWidth) + } + } + } + + Text(self.state.remoteTransport == .direct + ? "Tip: use Tailscale Serve so the gateway has a valid HTTPS cert." + : "Tip: keep Tailscale enabled so your gateway stays reachable.") + .font(.footnote) + .foregroundStyle(.secondary) + .lineLimit(1) + } + .transition(.opacity.combined(with: .move(edge: .top))) } } @@ -265,9 +274,11 @@ extension OnboardingView { if self.state.remoteTransport == .direct { return GatewayDiscoveryHelpers.directUrl(for: gateway) ?? "Gateway pairing only" } - if let host = GatewayDiscoveryHelpers.sanitizedTailnetHost(gateway.tailnetDns) ?? gateway.lanHost { - let portSuffix = gateway.sshPort != 22 ? " · ssh \(gateway.sshPort)" : "" - return "\(host)\(portSuffix)" + if let target = GatewayDiscoveryHelpers.sshTarget(for: gateway), + let parsed = CommandResolver.parseSSHTarget(target) + { + let portSuffix = parsed.port != 22 ? " · ssh \(parsed.port)" : "" + return "\(parsed.host)\(portSuffix)" } return "Gateway pairing only" } @@ -327,170 +338,6 @@ extension OnboardingView { .buttonStyle(.plain) } - func anthropicAuthPage() -> some View { - self.onboardingPage { - Text("Connect Claude") - .font(.largeTitle.weight(.semibold)) - Text("Give your model the token it needs!") - .font(.body) - .foregroundStyle(.secondary) - .multilineTextAlignment(.center) - .frame(maxWidth: 540) - .fixedSize(horizontal: false, vertical: true) - Text("OpenClaw supports any model — we strongly recommend Opus 4.6 for the best experience.") - .font(.callout) - .foregroundStyle(.secondary) - .multilineTextAlignment(.center) - .frame(maxWidth: 540) - .fixedSize(horizontal: false, vertical: true) - - self.onboardingCard(spacing: 12, padding: 16) { - HStack(alignment: .center, spacing: 10) { - Circle() - .fill(self.anthropicAuthVerified ? Color.green : Color.orange) - .frame(width: 10, height: 10) - Text( - self.anthropicAuthConnected - ? (self.anthropicAuthVerified - ? "Claude connected (OAuth) — verified" - : "Claude connected (OAuth)") - : "Not connected yet") - .font(.headline) - Spacer() - } - - if self.anthropicAuthConnected, self.anthropicAuthVerifying { - Text("Verifying OAuth…") - .font(.caption) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - } else if !self.anthropicAuthConnected { - Text(self.anthropicAuthDetectedStatus.shortDescription) - .font(.caption) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - } else if self.anthropicAuthVerified, let date = self.anthropicAuthVerifiedAt { - Text("Detected working OAuth (\(date.formatted(date: .abbreviated, time: .shortened))).") - .font(.caption) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - } - - Text( - "This lets OpenClaw use Claude immediately. Credentials are stored at " + - "`~/.openclaw/credentials/oauth.json` (owner-only).") - .font(.subheadline) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - - HStack(spacing: 12) { - Text(OpenClawOAuthStore.oauthURL().path) - .font(.caption) - .foregroundStyle(.secondary) - .lineLimit(1) - .truncationMode(.middle) - - Spacer() - - Button("Reveal") { - NSWorkspace.shared.activateFileViewerSelecting([OpenClawOAuthStore.oauthURL()]) - } - .buttonStyle(.bordered) - - Button("Refresh") { - self.refreshAnthropicOAuthStatus() - } - .buttonStyle(.bordered) - } - - Divider().padding(.vertical, 2) - - HStack(spacing: 12) { - if !self.anthropicAuthVerified { - if self.anthropicAuthConnected { - Button("Verify") { - Task { await self.verifyAnthropicOAuthIfNeeded(force: true) } - } - .buttonStyle(.borderedProminent) - .disabled(self.anthropicAuthBusy || self.anthropicAuthVerifying) - - if self.anthropicAuthVerificationFailed { - Button("Re-auth (OAuth)") { - self.startAnthropicOAuth() - } - .buttonStyle(.bordered) - .disabled(self.anthropicAuthBusy || self.anthropicAuthVerifying) - } - } else { - Button { - self.startAnthropicOAuth() - } label: { - if self.anthropicAuthBusy { - ProgressView() - } else { - Text("Open Claude sign-in (OAuth)") - } - } - .buttonStyle(.borderedProminent) - .disabled(self.anthropicAuthBusy) - } - } - } - - if !self.anthropicAuthVerified, self.anthropicAuthPKCE != nil { - VStack(alignment: .leading, spacing: 8) { - Text("Paste the `code#state` value") - .font(.headline) - TextField("code#state", text: self.$anthropicAuthCode) - .textFieldStyle(.roundedBorder) - - Toggle("Auto-detect from clipboard", isOn: self.$anthropicAuthAutoDetectClipboard) - .font(.caption) - .foregroundStyle(.secondary) - .disabled(self.anthropicAuthBusy) - - Toggle("Auto-connect when detected", isOn: self.$anthropicAuthAutoConnectClipboard) - .font(.caption) - .foregroundStyle(.secondary) - .disabled(self.anthropicAuthBusy) - - Button("Connect") { - Task { await self.finishAnthropicOAuth() } - } - .buttonStyle(.bordered) - .disabled( - self.anthropicAuthBusy || - self.anthropicAuthCode.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty) - } - .onReceive(Self.clipboardPoll) { _ in - self.pollAnthropicClipboardIfNeeded() - } - } - - self.onboardingCard(spacing: 8, padding: 12) { - Text("API key (advanced)") - .font(.headline) - Text( - "You can also use an Anthropic API key, but this UI is instructions-only for now " + - "(GUI apps don’t automatically inherit your shell env vars like `ANTHROPIC_API_KEY`).") - .font(.subheadline) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - } - .shadow(color: .clear, radius: 0) - .background(Color.clear) - - if let status = self.anthropicAuthStatus { - Text(status) - .font(.caption) - .foregroundStyle(.secondary) - .fixedSize(horizontal: false, vertical: true) - } - } - } - .task { await self.verifyAnthropicOAuthIfNeeded() } - } - func permissionsPage() -> some View { self.onboardingPage { Text("Grant permissions") diff --git a/apps/macos/Sources/OpenClaw/OnboardingView+Testing.swift b/apps/macos/Sources/OpenClaw/OnboardingView+Testing.swift index cf8c3d0c78f..2bd9c525ad4 100644 --- a/apps/macos/Sources/OpenClaw/OnboardingView+Testing.swift +++ b/apps/macos/Sources/OpenClaw/OnboardingView+Testing.swift @@ -37,18 +37,9 @@ extension OnboardingView { view.cliStatus = "Installed" view.workspacePath = "/tmp/openclaw" view.workspaceStatus = "Saved workspace" - view.anthropicAuthPKCE = AnthropicOAuth.PKCE(verifier: "verifier", challenge: "challenge") - view.anthropicAuthCode = "code#state" - view.anthropicAuthStatus = "Connected" - view.anthropicAuthDetectedStatus = .connected(expiresAtMs: 1_700_000_000_000) - view.anthropicAuthConnected = true - view.anthropicAuthAutoDetectClipboard = false - view.anthropicAuthAutoConnectClipboard = false - view.state.connectionMode = .local _ = view.welcomePage() _ = view.connectionPage() - _ = view.anthropicAuthPage() _ = view.wizardPage() _ = view.permissionsPage() _ = view.cliPage() diff --git a/apps/macos/Sources/OpenClaw/OnboardingView+Workspace.swift b/apps/macos/Sources/OpenClaw/OnboardingView+Workspace.swift index 1895b2af94f..7538f846b89 100644 --- a/apps/macos/Sources/OpenClaw/OnboardingView+Workspace.swift +++ b/apps/macos/Sources/OpenClaw/OnboardingView+Workspace.swift @@ -13,8 +13,10 @@ extension OnboardingView { guard self.state.connectionMode == .local else { return } let configured = await self.loadAgentWorkspace() let url = AgentWorkspace.resolveWorkspaceURL(from: configured) - switch AgentWorkspace.bootstrapSafety(for: url) { - case .safe: + let safety = AgentWorkspace.bootstrapSafety(for: url) + if let reason = safety.unsafeReason { + self.workspaceStatus = "Workspace not touched: \(reason)" + } else { do { _ = try AgentWorkspace.bootstrap(workspaceURL: url) if (configured ?? "").trimmingCharacters(in: .whitespacesAndNewlines).isEmpty { @@ -23,8 +25,6 @@ extension OnboardingView { } catch { self.workspaceStatus = "Failed to create workspace: \(error.localizedDescription)" } - case let .unsafe (reason): - self.workspaceStatus = "Workspace not touched: \(reason)" } self.refreshBootstrapStatus() } @@ -54,7 +54,7 @@ extension OnboardingView { do { let url = AgentWorkspace.resolveWorkspaceURL(from: self.workspacePath) - if case let .unsafe (reason) = AgentWorkspace.bootstrapSafety(for: url) { + if let reason = AgentWorkspace.bootstrapSafety(for: url).unsafeReason { self.workspaceStatus = "Workspace not created: \(reason)" return } diff --git a/apps/macos/Sources/OpenClaw/OpenClawConfigFile.swift b/apps/macos/Sources/OpenClaw/OpenClawConfigFile.swift index f49f2b7e0d4..35744baeda5 100644 --- a/apps/macos/Sources/OpenClaw/OpenClawConfigFile.swift +++ b/apps/macos/Sources/OpenClaw/OpenClawConfigFile.swift @@ -223,6 +223,19 @@ enum OpenClawConfigFile { } } + static func clearRemoteGatewayUrl() { + self.updateGatewayDict { gateway in + guard var remote = gateway["remote"] as? [String: Any] else { return } + guard remote["url"] != nil else { return } + remote.removeValue(forKey: "url") + if remote.isEmpty { + gateway.removeValue(forKey: "remote") + } else { + gateway["remote"] = remote + } + } + } + private static func remoteGatewayUrl() -> URL? { let root = self.loadDict() guard let gateway = root["gateway"] as? [String: Any], diff --git a/apps/macos/Sources/OpenClaw/Resources/Info.plist b/apps/macos/Sources/OpenClaw/Resources/Info.plist index 8d32bbe09a6..8271e10b917 100644 --- a/apps/macos/Sources/OpenClaw/Resources/Info.plist +++ b/apps/macos/Sources/OpenClaw/Resources/Info.plist @@ -15,9 +15,9 @@ CFBundlePackageType APPL CFBundleShortVersionString - 2026.2.18 + 2026.2.27 CFBundleVersion - 202602180 + 202602270 CFBundleIconFile OpenClaw CFBundleURLTypes diff --git a/apps/macos/Sources/OpenClaw/SystemRunSettingsView.swift b/apps/macos/Sources/OpenClaw/SystemRunSettingsView.swift index b9bd6bd0c8c..7c047e01d03 100644 --- a/apps/macos/Sources/OpenClaw/SystemRunSettingsView.swift +++ b/apps/macos/Sources/OpenClaw/SystemRunSettingsView.swift @@ -105,16 +105,24 @@ struct SystemRunSettingsView: View { .foregroundStyle(.secondary) } else { HStack(spacing: 8) { - TextField("Add allowlist pattern (case-insensitive globs)", text: self.$newPattern) + TextField("Add allowlist path pattern (case-insensitive globs)", text: self.$newPattern) .textFieldStyle(.roundedBorder) Button("Add") { - let pattern = self.newPattern.trimmingCharacters(in: .whitespacesAndNewlines) - guard !pattern.isEmpty else { return } - self.model.addEntry(pattern) - self.newPattern = "" + if self.model.addEntry(self.newPattern) == nil { + self.newPattern = "" + } } .buttonStyle(.bordered) - .disabled(self.newPattern.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty) + .disabled(!self.model.isPathPattern(self.newPattern)) + } + + Text("Path patterns only. Basename entries like \"echo\" are ignored.") + .font(.footnote) + .foregroundStyle(.secondary) + if let validationMessage = self.model.allowlistValidationMessage { + Text(validationMessage) + .font(.footnote) + .foregroundStyle(.orange) } if self.model.entries.isEmpty { @@ -234,6 +242,7 @@ final class ExecApprovalsSettingsModel { var autoAllowSkills = false var entries: [ExecAllowlistEntry] = [] var skillBins: [String] = [] + var allowlistValidationMessage: String? var agentPickerIds: [String] { [Self.defaultsScopeId] + self.agentIds @@ -289,6 +298,7 @@ final class ExecApprovalsSettingsModel { func selectAgent(_ id: String) { self.selectedAgentId = id + self.allowlistValidationMessage = nil self.loadSettings(for: id) Task { await self.refreshSkillBins() } } @@ -301,6 +311,7 @@ final class ExecApprovalsSettingsModel { self.askFallback = defaults.askFallback self.autoAllowSkills = defaults.autoAllowSkills self.entries = [] + self.allowlistValidationMessage = nil return } let resolved = ExecApprovalsStore.resolve(agentId: agentId) @@ -310,6 +321,7 @@ final class ExecApprovalsSettingsModel { self.autoAllowSkills = resolved.agent.autoAllowSkills self.entries = resolved.allowlist .sorted { $0.pattern.localizedCaseInsensitiveCompare($1.pattern) == .orderedAscending } + self.allowlistValidationMessage = nil } func setSecurity(_ security: ExecSecurity) { @@ -367,32 +379,55 @@ final class ExecApprovalsSettingsModel { Task { await self.refreshSkillBins(force: enabled) } } - func addEntry(_ pattern: String) { - guard !self.isDefaultsScope else { return } - let trimmed = pattern.trimmingCharacters(in: .whitespacesAndNewlines) - guard !trimmed.isEmpty else { return } - self.entries.append(ExecAllowlistEntry(pattern: trimmed, lastUsedAt: nil)) - ExecApprovalsStore.updateAllowlist(agentId: self.selectedAgentId, allowlist: self.entries) + @discardableResult + func addEntry(_ pattern: String) -> ExecAllowlistPatternValidationReason? { + guard !self.isDefaultsScope else { return nil } + switch ExecApprovalHelpers.validateAllowlistPattern(pattern) { + case let .valid(normalizedPattern): + self.entries.append(ExecAllowlistEntry(pattern: normalizedPattern, lastUsedAt: nil)) + let rejected = ExecApprovalsStore.updateAllowlist(agentId: self.selectedAgentId, allowlist: self.entries) + self.allowlistValidationMessage = rejected.first?.reason.message + return rejected.first?.reason + case let .invalid(reason): + self.allowlistValidationMessage = reason.message + return reason + } } - func updateEntry(_ entry: ExecAllowlistEntry, id: UUID) { - guard !self.isDefaultsScope else { return } - guard let index = self.entries.firstIndex(where: { $0.id == id }) else { return } - self.entries[index] = entry - ExecApprovalsStore.updateAllowlist(agentId: self.selectedAgentId, allowlist: self.entries) + @discardableResult + func updateEntry(_ entry: ExecAllowlistEntry, id: UUID) -> ExecAllowlistPatternValidationReason? { + guard !self.isDefaultsScope else { return nil } + guard let index = self.entries.firstIndex(where: { $0.id == id }) else { return nil } + var next = entry + switch ExecApprovalHelpers.validateAllowlistPattern(next.pattern) { + case let .valid(normalizedPattern): + next.pattern = normalizedPattern + case let .invalid(reason): + self.allowlistValidationMessage = reason.message + return reason + } + self.entries[index] = next + let rejected = ExecApprovalsStore.updateAllowlist(agentId: self.selectedAgentId, allowlist: self.entries) + self.allowlistValidationMessage = rejected.first?.reason.message + return rejected.first?.reason } func removeEntry(id: UUID) { guard !self.isDefaultsScope else { return } guard let index = self.entries.firstIndex(where: { $0.id == id }) else { return } self.entries.remove(at: index) - ExecApprovalsStore.updateAllowlist(agentId: self.selectedAgentId, allowlist: self.entries) + let rejected = ExecApprovalsStore.updateAllowlist(agentId: self.selectedAgentId, allowlist: self.entries) + self.allowlistValidationMessage = rejected.first?.reason.message } func entry(for id: UUID) -> ExecAllowlistEntry? { self.entries.first(where: { $0.id == id }) } + func isPathPattern(_ pattern: String) -> Bool { + ExecApprovalHelpers.isPathPattern(pattern) + } + func refreshSkillBins(force: Bool = false) async { guard self.autoAllowSkills else { self.skillBins = [] diff --git a/apps/macos/Sources/OpenClaw/TalkModeRuntime.swift b/apps/macos/Sources/OpenClaw/TalkModeRuntime.swift index 47b041a5873..a8d8008c653 100644 --- a/apps/macos/Sources/OpenClaw/TalkModeRuntime.swift +++ b/apps/macos/Sources/OpenClaw/TalkModeRuntime.swift @@ -11,6 +11,7 @@ actor TalkModeRuntime { private let logger = Logger(subsystem: "ai.openclaw", category: "talk.runtime") private let ttsLogger = Logger(subsystem: "ai.openclaw", category: "talk.tts") private static let defaultModelIdFallback = "eleven_v3" + private static let defaultTalkProvider = "elevenlabs" private final class RMSMeter: @unchecked Sendable { private let lock = NSLock() @@ -184,6 +185,12 @@ actor TalkModeRuntime { } guard let audioEngine = self.audioEngine else { return } + guard AudioInputDeviceObserver.hasUsableDefaultInputDevice() else { + self.audioEngine = nil + self.logger.error("talk mode: no usable audio input device") + return + } + let input = audioEngine.inputNode let format = input.outputFormat(forBus: 0) input.removeTap(onBus: 0) @@ -792,6 +799,82 @@ extension TalkModeRuntime { let apiKey: String? } + struct TalkProviderConfigSelection { + let provider: String + let config: [String: AnyCodable] + let normalizedPayload: Bool + } + + private static func normalizedTalkProviderID(_ raw: String?) -> String? { + let trimmed = raw?.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() ?? "" + return trimmed.isEmpty ? nil : trimmed + } + + private static func normalizedTalkProviderConfig(_ value: AnyCodable) -> [String: AnyCodable]? { + if let typed = value.value as? [String: AnyCodable] { + return typed + } + if let foundation = value.value as? [String: Any] { + return foundation.mapValues(AnyCodable.init) + } + if let nsDict = value.value as? NSDictionary { + var converted: [String: AnyCodable] = [:] + for case let (key as String, raw) in nsDict { + converted[key] = AnyCodable(raw) + } + return converted + } + return nil + } + + private static func normalizedTalkProviders(_ raw: AnyCodable?) -> [String: [String: AnyCodable]] { + guard let raw else { return [:] } + var providerMap: [String: AnyCodable] = [:] + if let typed = raw.value as? [String: AnyCodable] { + providerMap = typed + } else if let foundation = raw.value as? [String: Any] { + providerMap = foundation.mapValues(AnyCodable.init) + } else if let nsDict = raw.value as? NSDictionary { + for case let (key as String, value) in nsDict { + providerMap[key] = AnyCodable(value) + } + } else { + return [:] + } + + return providerMap.reduce(into: [String: [String: AnyCodable]]()) { acc, entry in + guard + let providerID = Self.normalizedTalkProviderID(entry.key), + let providerConfig = Self.normalizedTalkProviderConfig(entry.value) + else { return } + acc[providerID] = providerConfig + } + } + + static func selectTalkProviderConfig( + _ talk: [String: AnyCodable]?) -> TalkProviderConfigSelection? + { + guard let talk else { return nil } + let rawProvider = talk["provider"]?.stringValue + let rawProviders = talk["providers"] + let hasNormalizedPayload = rawProvider != nil || rawProviders != nil + if hasNormalizedPayload { + let normalizedProviders = Self.normalizedTalkProviders(rawProviders) + let providerID = + Self.normalizedTalkProviderID(rawProvider) ?? + normalizedProviders.keys.min() ?? + Self.defaultTalkProvider + return TalkProviderConfigSelection( + provider: providerID, + config: normalizedProviders[providerID] ?? [:], + normalizedPayload: true) + } + return TalkProviderConfigSelection( + provider: Self.defaultTalkProvider, + config: talk, + normalizedPayload: false) + } + private func fetchTalkConfig() async -> TalkRuntimeConfig { let env = ProcessInfo.processInfo.environment let envVoice = env["ELEVENLABS_VOICE_ID"]?.trimmingCharacters(in: .whitespacesAndNewlines) @@ -804,13 +887,16 @@ extension TalkModeRuntime { params: ["includeSecrets": AnyCodable(true)], timeoutMs: 8000) let talk = snap.config?["talk"]?.dictionaryValue + let selection = Self.selectTalkProviderConfig(talk) + let activeProvider = selection?.provider ?? Self.defaultTalkProvider + let activeConfig = selection?.config let ui = snap.config?["ui"]?.dictionaryValue let rawSeam = ui?["seamColor"]?.stringValue?.trimmingCharacters(in: .whitespacesAndNewlines) ?? "" await MainActor.run { AppStateStore.shared.seamColorHex = rawSeam.isEmpty ? nil : rawSeam } - let voice = talk?["voiceId"]?.stringValue - let rawAliases = talk?["voiceAliases"]?.dictionaryValue + let voice = activeConfig?["voiceId"]?.stringValue + let rawAliases = activeConfig?["voiceAliases"]?.dictionaryValue let resolvedAliases: [String: String] = rawAliases?.reduce(into: [:]) { acc, entry in let key = entry.key.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() @@ -818,18 +904,30 @@ extension TalkModeRuntime { guard !key.isEmpty, !value.isEmpty else { return } acc[key] = value } ?? [:] - let model = talk?["modelId"]?.stringValue?.trimmingCharacters(in: .whitespacesAndNewlines) + let model = activeConfig?["modelId"]?.stringValue?.trimmingCharacters(in: .whitespacesAndNewlines) let resolvedModel = (model?.isEmpty == false) ? model! : Self.defaultModelIdFallback - let outputFormat = talk?["outputFormat"]?.stringValue + let outputFormat = activeConfig?["outputFormat"]?.stringValue let interrupt = talk?["interruptOnSpeech"]?.boolValue - let apiKey = talk?["apiKey"]?.stringValue - let resolvedVoice = + let apiKey = activeConfig?["apiKey"]?.stringValue + let resolvedVoice: String? = if activeProvider == Self.defaultTalkProvider { (voice?.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty == false ? voice : nil) ?? - (envVoice?.isEmpty == false ? envVoice : nil) ?? - (sagVoice?.isEmpty == false ? sagVoice : nil) - let resolvedApiKey = + (envVoice?.isEmpty == false ? envVoice : nil) ?? + (sagVoice?.isEmpty == false ? sagVoice : nil) + } else { + (voice?.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty == false ? voice : nil) + } + let resolvedApiKey: String? = if activeProvider == Self.defaultTalkProvider { (envApiKey?.isEmpty == false ? envApiKey : nil) ?? - (apiKey?.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty == false ? apiKey : nil) + (apiKey?.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty == false ? apiKey : nil) + } else { + nil + } + if activeProvider != Self.defaultTalkProvider { + self.ttsLogger + .info("talk provider \(activeProvider, privacy: .public) unsupported; using system voice") + } else if selection?.normalizedPayload == true { + self.ttsLogger.info("talk config provider elevenlabs") + } return TalkRuntimeConfig( voiceId: resolvedVoice, voiceAliases: resolvedAliases, diff --git a/apps/macos/Sources/OpenClaw/VoicePushToTalk.swift b/apps/macos/Sources/OpenClaw/VoicePushToTalk.swift index e535ebd6616..6eaa45e0675 100644 --- a/apps/macos/Sources/OpenClaw/VoicePushToTalk.swift +++ b/apps/macos/Sources/OpenClaw/VoicePushToTalk.swift @@ -244,6 +244,14 @@ actor VoicePushToTalk { } guard let audioEngine = self.audioEngine else { return } + guard AudioInputDeviceObserver.hasUsableDefaultInputDevice() else { + self.audioEngine = nil + throw NSError( + domain: "VoicePushToTalk", + code: 1, + userInfo: [NSLocalizedDescriptionKey: "No usable audio input device available"]) + } + let input = audioEngine.inputNode let format = input.outputFormat(forBus: 0) if self.tapInstalled { diff --git a/apps/macos/Sources/OpenClaw/VoiceWakeForwarder.swift b/apps/macos/Sources/OpenClaw/VoiceWakeForwarder.swift index ee634a628ed..0c6ea54c90e 100644 --- a/apps/macos/Sources/OpenClaw/VoiceWakeForwarder.swift +++ b/apps/macos/Sources/OpenClaw/VoiceWakeForwarder.swift @@ -37,7 +37,7 @@ enum VoiceWakeForwarder { var thinking: String = "low" var deliver: Bool = true var to: String? - var channel: GatewayAgentChannel = .last + var channel: GatewayAgentChannel = .webchat } @discardableResult diff --git a/apps/macos/Sources/OpenClaw/VoiceWakeOverlayTextViews.swift b/apps/macos/Sources/OpenClaw/VoiceWakeOverlayTextViews.swift index 8e88c86d45d..bbbed72926b 100644 --- a/apps/macos/Sources/OpenClaw/VoiceWakeOverlayTextViews.swift +++ b/apps/macos/Sources/OpenClaw/VoiceWakeOverlayTextViews.swift @@ -185,6 +185,11 @@ private final class TranscriptNSTextView: NSTextView { self.onEscape?() return } + // Keep IME candidate confirmation behavior: Return should commit marked text first. + if isReturn, self.hasMarkedText() { + super.keyDown(with: event) + return + } if isReturn, event.modifierFlags.contains(.command) { self.onSend?() return diff --git a/apps/macos/Sources/OpenClaw/VoiceWakeRuntime.swift b/apps/macos/Sources/OpenClaw/VoiceWakeRuntime.swift index 61f913b9da8..b7e2d329b82 100644 --- a/apps/macos/Sources/OpenClaw/VoiceWakeRuntime.swift +++ b/apps/macos/Sources/OpenClaw/VoiceWakeRuntime.swift @@ -166,6 +166,14 @@ actor VoiceWakeRuntime { } guard let audioEngine = self.audioEngine else { return } + guard AudioInputDeviceObserver.hasUsableDefaultInputDevice() else { + self.audioEngine = nil + throw NSError( + domain: "VoiceWakeRuntime", + code: 1, + userInfo: [NSLocalizedDescriptionKey: "No usable audio input device available"]) + } + let input = audioEngine.inputNode let format = input.outputFormat(forBus: 0) guard format.channelCount > 0, format.sampleRate > 0 else { diff --git a/apps/macos/Sources/OpenClaw/VoiceWakeTester.swift b/apps/macos/Sources/OpenClaw/VoiceWakeTester.swift index b3d0c58d90c..063fea826ab 100644 --- a/apps/macos/Sources/OpenClaw/VoiceWakeTester.swift +++ b/apps/macos/Sources/OpenClaw/VoiceWakeTester.swift @@ -89,6 +89,14 @@ final class VoiceWakeTester { self.logInputSelection(preferredMicID: micID) self.configureSession(preferredMicID: micID) + guard AudioInputDeviceObserver.hasUsableDefaultInputDevice() else { + self.audioEngine = nil + throw NSError( + domain: "VoiceWakeTester", + code: 1, + userInfo: [NSLocalizedDescriptionKey: "No usable audio input device available"]) + } + let engine = AVAudioEngine() self.audioEngine = engine diff --git a/apps/macos/Sources/OpenClaw/WebChatSwiftUI.swift b/apps/macos/Sources/OpenClaw/WebChatSwiftUI.swift index 5b866304b09..46e5d80a01e 100644 --- a/apps/macos/Sources/OpenClaw/WebChatSwiftUI.swift +++ b/apps/macos/Sources/OpenClaw/WebChatSwiftUI.swift @@ -316,7 +316,12 @@ final class WebChatSwiftUIWindowController { let controller = NSViewController() let effectView = NSVisualEffectView() effectView.material = .sidebar - effectView.blendingMode = .behindWindow + effectView.blendingMode = switch presentation { + case .panel: + .withinWindow + case .window: + .behindWindow + } effectView.state = .active effectView.wantsLayer = true effectView.layer?.cornerCurve = .continuous @@ -328,6 +333,7 @@ final class WebChatSwiftUIWindowController { } effectView.layer?.cornerRadius = cornerRadius effectView.layer?.masksToBounds = true + effectView.layer?.backgroundColor = NSColor.clear.cgColor effectView.translatesAutoresizingMaskIntoConstraints = true effectView.autoresizingMask = [.width, .height] @@ -335,6 +341,9 @@ final class WebChatSwiftUIWindowController { hosting.view.translatesAutoresizingMaskIntoConstraints = false hosting.view.wantsLayer = true + hosting.view.layer?.cornerCurve = .continuous + hosting.view.layer?.cornerRadius = cornerRadius + hosting.view.layer?.masksToBounds = true hosting.view.layer?.backgroundColor = NSColor.clear.cgColor controller.addChild(hosting) diff --git a/apps/macos/Sources/OpenClawDiscovery/TailscaleNetwork.swift b/apps/macos/Sources/OpenClawDiscovery/TailscaleNetwork.swift index 60b11306d05..ef78e6f400f 100644 --- a/apps/macos/Sources/OpenClawDiscovery/TailscaleNetwork.swift +++ b/apps/macos/Sources/OpenClawDiscovery/TailscaleNetwork.swift @@ -44,4 +44,3 @@ public enum TailscaleNetwork { return nil } } - diff --git a/apps/macos/Sources/OpenClawMacCLI/ConnectCommand.swift b/apps/macos/Sources/OpenClawMacCLI/ConnectCommand.swift index 0989164a01e..151b7fdda94 100644 --- a/apps/macos/Sources/OpenClawMacCLI/ConnectCommand.swift +++ b/apps/macos/Sources/OpenClawMacCLI/ConnectCommand.swift @@ -15,7 +15,7 @@ struct ConnectOptions { var clientMode: String = "ui" var displayName: String? var role: String = "operator" - var scopes: [String] = ["operator.admin", "operator.approvals", "operator.pairing"] + var scopes: [String] = defaultOperatorConnectScopes var help: Bool = false static func parse(_ args: [String]) -> ConnectOptions { diff --git a/apps/macos/Sources/OpenClawMacCLI/GatewayScopes.swift b/apps/macos/Sources/OpenClawMacCLI/GatewayScopes.swift new file mode 100644 index 00000000000..479c176d5d8 --- /dev/null +++ b/apps/macos/Sources/OpenClawMacCLI/GatewayScopes.swift @@ -0,0 +1,7 @@ +let defaultOperatorConnectScopes: [String] = [ + "operator.admin", + "operator.read", + "operator.write", + "operator.approvals", + "operator.pairing", +] diff --git a/apps/macos/Sources/OpenClawMacCLI/WizardCommand.swift b/apps/macos/Sources/OpenClawMacCLI/WizardCommand.swift index 0a73fc2108c..f75ef05fdb2 100644 --- a/apps/macos/Sources/OpenClawMacCLI/WizardCommand.swift +++ b/apps/macos/Sources/OpenClawMacCLI/WizardCommand.swift @@ -251,7 +251,7 @@ actor GatewayWizardClient { let clientMode = "ui" let role = "operator" // Explicit scopes; gateway no longer defaults empty scopes to admin. - let scopes: [String] = ["operator.admin", "operator.approvals", "operator.pairing"] + let scopes = defaultOperatorConnectScopes let client: [String: ProtoAnyCodable] = [ "id": ProtoAnyCodable(clientId), "displayName": ProtoAnyCodable(Host.current().localizedName ?? "OpenClaw macOS Wizard CLI"), @@ -280,33 +280,27 @@ actor GatewayWizardClient { let connectNonce = try await self.waitForConnectChallenge() let identity = DeviceIdentityStore.loadOrCreate() let signedAtMs = Int(Date().timeIntervalSince1970 * 1000) - let scopesValue = scopes.joined(separator: ",") - var payloadParts = [ - connectNonce == nil ? "v1" : "v2", - identity.deviceId, - clientId, - clientMode, - role, - scopesValue, - String(signedAtMs), - self.token ?? "", - ] - if let connectNonce { - payloadParts.append(connectNonce) - } - let payload = payloadParts.joined(separator: "|") + let payload = GatewayDeviceAuthPayload.buildV3( + deviceId: identity.deviceId, + clientId: clientId, + clientMode: clientMode, + role: role, + scopes: scopes, + signedAtMs: signedAtMs, + token: self.token, + nonce: connectNonce, + platform: platform, + deviceFamily: "Mac") if let signature = DeviceIdentityStore.signPayload(payload, identity: identity), let publicKey = DeviceIdentityStore.publicKeyBase64Url(identity) { - var device: [String: ProtoAnyCodable] = [ + let device: [String: ProtoAnyCodable] = [ "id": ProtoAnyCodable(identity.deviceId), "publicKey": ProtoAnyCodable(publicKey), "signature": ProtoAnyCodable(signature), "signedAt": ProtoAnyCodable(signedAtMs), + "nonce": ProtoAnyCodable(connectNonce), ] - if let connectNonce { - device["nonce"] = ProtoAnyCodable(connectNonce) - } params["device"] = ProtoAnyCodable(device) } @@ -333,29 +327,24 @@ actor GatewayWizardClient { } } - private func waitForConnectChallenge() async throws -> String? { - guard let task = self.task else { return nil } - do { - return try await AsyncTimeout.withTimeout( - seconds: self.connectChallengeTimeoutSeconds, - onTimeout: { ConnectChallengeError.timeout }, - operation: { - while true { - let message = try await task.receive() - let frame = try await self.decodeFrame(message) - if case let .event(evt) = frame, evt.event == "connect.challenge" { - if let payload = evt.payload?.value as? [String: ProtoAnyCodable], - let nonce = payload["nonce"]?.value as? String - { - return nonce - } - } + private func waitForConnectChallenge() async throws -> String { + guard let task = self.task else { throw ConnectChallengeError.timeout } + return try await AsyncTimeout.withTimeout( + seconds: self.connectChallengeTimeoutSeconds, + onTimeout: { ConnectChallengeError.timeout }, + operation: { + while true { + let message = try await task.receive() + let frame = try await self.decodeFrame(message) + if case let .event(evt) = frame, evt.event == "connect.challenge", + let payload = evt.payload?.value as? [String: ProtoAnyCodable], + let nonce = payload["nonce"]?.value as? String, + nonce.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty == false + { + return nonce } - }) - } catch { - if error is ConnectChallengeError { return nil } - throw error - } + } + }) } } diff --git a/apps/macos/Sources/OpenClawProtocol/GatewayModels.swift b/apps/macos/Sources/OpenClawProtocol/GatewayModels.swift index ced5df13a8f..a7aaa7d3914 100644 --- a/apps/macos/Sources/OpenClawProtocol/GatewayModels.swift +++ b/apps/macos/Sources/OpenClawProtocol/GatewayModels.swift @@ -40,8 +40,8 @@ public struct ConnectParams: Codable, Sendable { device: [String: AnyCodable]?, auth: [String: AnyCodable]?, locale: String?, - useragent: String? - ) { + useragent: String?) + { self.minprotocol = minprotocol self.maxprotocol = maxprotocol self.client = client @@ -56,6 +56,7 @@ public struct ConnectParams: Codable, Sendable { self.locale = locale self.useragent = useragent } + private enum CodingKeys: String, CodingKey { case minprotocol = "minProtocol" case maxprotocol = "maxProtocol" @@ -91,8 +92,8 @@ public struct HelloOk: Codable, Sendable { snapshot: Snapshot, canvashosturl: String?, auth: [String: AnyCodable]?, - policy: [String: AnyCodable] - ) { + policy: [String: AnyCodable]) + { self.type = type self._protocol = _protocol self.server = server @@ -102,6 +103,7 @@ public struct HelloOk: Codable, Sendable { self.auth = auth self.policy = policy } + private enum CodingKeys: String, CodingKey { case type case _protocol = "protocol" @@ -124,13 +126,14 @@ public struct RequestFrame: Codable, Sendable { type: String, id: String, method: String, - params: AnyCodable? - ) { + params: AnyCodable?) + { self.type = type self.id = id self.method = method self.params = params } + private enum CodingKeys: String, CodingKey { case type case id @@ -151,14 +154,15 @@ public struct ResponseFrame: Codable, Sendable { id: String, ok: Bool, payload: AnyCodable?, - error: [String: AnyCodable]? - ) { + error: [String: AnyCodable]?) + { self.type = type self.id = id self.ok = ok self.payload = payload self.error = error } + private enum CodingKeys: String, CodingKey { case type case id @@ -180,14 +184,15 @@ public struct EventFrame: Codable, Sendable { event: String, payload: AnyCodable?, seq: Int?, - stateversion: [String: AnyCodable]? - ) { + stateversion: [String: AnyCodable]?) + { self.type = type self.event = event self.payload = payload self.seq = seq self.stateversion = stateversion } + private enum CodingKeys: String, CodingKey { case type case event @@ -231,8 +236,8 @@ public struct PresenceEntry: Codable, Sendable { deviceid: String?, roles: [String]?, scopes: [String]?, - instanceid: String? - ) { + instanceid: String?) + { self.host = host self.ip = ip self.version = version @@ -250,6 +255,7 @@ public struct PresenceEntry: Codable, Sendable { self.scopes = scopes self.instanceid = instanceid } + private enum CodingKeys: String, CodingKey { case host case ip @@ -276,11 +282,12 @@ public struct StateVersion: Codable, Sendable { public init( presence: Int, - health: Int - ) { + health: Int) + { self.presence = presence self.health = health } + private enum CodingKeys: String, CodingKey { case presence case health @@ -296,6 +303,7 @@ public struct Snapshot: Codable, Sendable { public let statedir: String? public let sessiondefaults: [String: AnyCodable]? public let authmode: AnyCodable? + public let updateavailable: [String: AnyCodable]? public init( presence: [PresenceEntry], @@ -305,8 +313,9 @@ public struct Snapshot: Codable, Sendable { configpath: String?, statedir: String?, sessiondefaults: [String: AnyCodable]?, - authmode: AnyCodable? - ) { + authmode: AnyCodable?, + updateavailable: [String: AnyCodable]?) + { self.presence = presence self.health = health self.stateversion = stateversion @@ -315,7 +324,9 @@ public struct Snapshot: Codable, Sendable { self.statedir = statedir self.sessiondefaults = sessiondefaults self.authmode = authmode + self.updateavailable = updateavailable } + private enum CodingKeys: String, CodingKey { case presence case health @@ -325,6 +336,7 @@ public struct Snapshot: Codable, Sendable { case statedir = "stateDir" case sessiondefaults = "sessionDefaults" case authmode = "authMode" + case updateavailable = "updateAvailable" } } @@ -340,14 +352,15 @@ public struct ErrorShape: Codable, Sendable { message: String, details: AnyCodable?, retryable: Bool?, - retryafterms: Int? - ) { + retryafterms: Int?) + { self.code = code self.message = message self.details = details self.retryable = retryable self.retryafterms = retryafterms } + private enum CodingKeys: String, CodingKey { case code case message @@ -369,14 +382,15 @@ public struct AgentEvent: Codable, Sendable { seq: Int, stream: String, ts: Int, - data: [String: AnyCodable] - ) { + data: [String: AnyCodable]) + { self.runid = runid self.seq = seq self.stream = stream self.ts = ts self.data = data } + private enum CodingKeys: String, CodingKey { case runid = "runId" case seq @@ -394,6 +408,7 @@ public struct SendParams: Codable, Sendable { public let gifplayback: Bool? public let channel: String? public let accountid: String? + public let agentid: String? public let threadid: String? public let sessionkey: String? public let idempotencykey: String @@ -406,10 +421,11 @@ public struct SendParams: Codable, Sendable { gifplayback: Bool?, channel: String?, accountid: String?, + agentid: String?, threadid: String?, sessionkey: String?, - idempotencykey: String - ) { + idempotencykey: String) + { self.to = to self.message = message self.mediaurl = mediaurl @@ -417,10 +433,12 @@ public struct SendParams: Codable, Sendable { self.gifplayback = gifplayback self.channel = channel self.accountid = accountid + self.agentid = agentid self.threadid = threadid self.sessionkey = sessionkey self.idempotencykey = idempotencykey } + private enum CodingKeys: String, CodingKey { case to case message @@ -429,6 +447,7 @@ public struct SendParams: Codable, Sendable { case gifplayback = "gifPlayback" case channel case accountid = "accountId" + case agentid = "agentId" case threadid = "threadId" case sessionkey = "sessionKey" case idempotencykey = "idempotencyKey" @@ -461,8 +480,8 @@ public struct PollParams: Codable, Sendable { threadid: String?, channel: String?, accountid: String?, - idempotencykey: String - ) { + idempotencykey: String) + { self.to = to self.question = question self.options = options @@ -476,6 +495,7 @@ public struct PollParams: Codable, Sendable { self.accountid = accountid self.idempotencykey = idempotencykey } + private enum CodingKeys: String, CodingKey { case to case question @@ -511,6 +531,7 @@ public struct AgentParams: Codable, Sendable { public let groupchannel: String? public let groupspace: String? public let timeout: Int? + public let besteffortdeliver: Bool? public let lane: String? public let extrasystemprompt: String? public let inputprovenance: [String: AnyCodable]? @@ -537,13 +558,14 @@ public struct AgentParams: Codable, Sendable { groupchannel: String?, groupspace: String?, timeout: Int?, + besteffortdeliver: Bool?, lane: String?, extrasystemprompt: String?, inputprovenance: [String: AnyCodable]?, idempotencykey: String, label: String?, - spawnedby: String? - ) { + spawnedby: String?) + { self.message = message self.agentid = agentid self.to = to @@ -562,6 +584,7 @@ public struct AgentParams: Codable, Sendable { self.groupchannel = groupchannel self.groupspace = groupspace self.timeout = timeout + self.besteffortdeliver = besteffortdeliver self.lane = lane self.extrasystemprompt = extrasystemprompt self.inputprovenance = inputprovenance @@ -569,6 +592,7 @@ public struct AgentParams: Codable, Sendable { self.label = label self.spawnedby = spawnedby } + private enum CodingKeys: String, CodingKey { case message case agentid = "agentId" @@ -588,6 +612,7 @@ public struct AgentParams: Codable, Sendable { case groupchannel = "groupChannel" case groupspace = "groupSpace" case timeout + case besteffortdeliver = "bestEffortDeliver" case lane case extrasystemprompt = "extraSystemPrompt" case inputprovenance = "inputProvenance" @@ -603,11 +628,12 @@ public struct AgentIdentityParams: Codable, Sendable { public init( agentid: String?, - sessionkey: String? - ) { + sessionkey: String?) + { self.agentid = agentid self.sessionkey = sessionkey } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" case sessionkey = "sessionKey" @@ -624,13 +650,14 @@ public struct AgentIdentityResult: Codable, Sendable { agentid: String, name: String?, avatar: String?, - emoji: String? - ) { + emoji: String?) + { self.agentid = agentid self.name = name self.avatar = avatar self.emoji = emoji } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" case name @@ -645,11 +672,12 @@ public struct AgentWaitParams: Codable, Sendable { public init( runid: String, - timeoutms: Int? - ) { + timeoutms: Int?) + { self.runid = runid self.timeoutms = timeoutms } + private enum CodingKeys: String, CodingKey { case runid = "runId" case timeoutms = "timeoutMs" @@ -662,11 +690,12 @@ public struct WakeParams: Codable, Sendable { public init( mode: AnyCodable, - text: String - ) { + text: String) + { self.mode = mode self.text = text } + private enum CodingKeys: String, CodingKey { case mode case text @@ -699,8 +728,8 @@ public struct NodePairRequestParams: Codable, Sendable { caps: [String]?, commands: [String]?, remoteip: String?, - silent: Bool? - ) { + silent: Bool?) + { self.nodeid = nodeid self.displayname = displayname self.platform = platform @@ -714,6 +743,7 @@ public struct NodePairRequestParams: Codable, Sendable { self.remoteip = remoteip self.silent = silent } + private enum CodingKeys: String, CodingKey { case nodeid = "nodeId" case displayname = "displayName" @@ -730,17 +760,17 @@ public struct NodePairRequestParams: Codable, Sendable { } } -public struct NodePairListParams: Codable, Sendable { -} +public struct NodePairListParams: Codable, Sendable {} public struct NodePairApproveParams: Codable, Sendable { public let requestid: String public init( - requestid: String - ) { + requestid: String) + { self.requestid = requestid } + private enum CodingKeys: String, CodingKey { case requestid = "requestId" } @@ -750,10 +780,11 @@ public struct NodePairRejectParams: Codable, Sendable { public let requestid: String public init( - requestid: String - ) { + requestid: String) + { self.requestid = requestid } + private enum CodingKeys: String, CodingKey { case requestid = "requestId" } @@ -765,11 +796,12 @@ public struct NodePairVerifyParams: Codable, Sendable { public init( nodeid: String, - token: String - ) { + token: String) + { self.nodeid = nodeid self.token = token } + private enum CodingKeys: String, CodingKey { case nodeid = "nodeId" case token @@ -782,28 +814,29 @@ public struct NodeRenameParams: Codable, Sendable { public init( nodeid: String, - displayname: String - ) { + displayname: String) + { self.nodeid = nodeid self.displayname = displayname } + private enum CodingKeys: String, CodingKey { case nodeid = "nodeId" case displayname = "displayName" } } -public struct NodeListParams: Codable, Sendable { -} +public struct NodeListParams: Codable, Sendable {} public struct NodeDescribeParams: Codable, Sendable { public let nodeid: String public init( - nodeid: String - ) { + nodeid: String) + { self.nodeid = nodeid } + private enum CodingKeys: String, CodingKey { case nodeid = "nodeId" } @@ -821,14 +854,15 @@ public struct NodeInvokeParams: Codable, Sendable { command: String, params: AnyCodable?, timeoutms: Int?, - idempotencykey: String - ) { + idempotencykey: String) + { self.nodeid = nodeid self.command = command self.params = params self.timeoutms = timeoutms self.idempotencykey = idempotencykey } + private enum CodingKeys: String, CodingKey { case nodeid = "nodeId" case command @@ -852,8 +886,8 @@ public struct NodeInvokeResultParams: Codable, Sendable { ok: Bool, payload: AnyCodable?, payloadjson: String?, - error: [String: AnyCodable]? - ) { + error: [String: AnyCodable]?) + { self.id = id self.nodeid = nodeid self.ok = ok @@ -861,6 +895,7 @@ public struct NodeInvokeResultParams: Codable, Sendable { self.payloadjson = payloadjson self.error = error } + private enum CodingKeys: String, CodingKey { case id case nodeid = "nodeId" @@ -879,12 +914,13 @@ public struct NodeEventParams: Codable, Sendable { public init( event: String, payload: AnyCodable?, - payloadjson: String? - ) { + payloadjson: String?) + { self.event = event self.payload = payload self.payloadjson = payloadjson } + private enum CodingKeys: String, CodingKey { case event case payload @@ -906,8 +942,8 @@ public struct NodeInvokeRequestEvent: Codable, Sendable { command: String, paramsjson: String?, timeoutms: Int?, - idempotencykey: String? - ) { + idempotencykey: String?) + { self.id = id self.nodeid = nodeid self.command = command @@ -915,6 +951,7 @@ public struct NodeInvokeRequestEvent: Codable, Sendable { self.timeoutms = timeoutms self.idempotencykey = idempotencykey } + private enum CodingKeys: String, CodingKey { case id case nodeid = "nodeId" @@ -935,13 +972,14 @@ public struct PushTestParams: Codable, Sendable { nodeid: String, title: String?, body: String?, - environment: String? - ) { + environment: String?) + { self.nodeid = nodeid self.title = title self.body = body self.environment = environment } + private enum CodingKeys: String, CodingKey { case nodeid = "nodeId" case title @@ -966,8 +1004,8 @@ public struct PushTestResult: Codable, Sendable { reason: String?, tokensuffix: String, topic: String, - environment: String - ) { + environment: String) + { self.ok = ok self.status = status self.apnsid = apnsid @@ -976,6 +1014,7 @@ public struct PushTestResult: Codable, Sendable { self.topic = topic self.environment = environment } + private enum CodingKeys: String, CodingKey { case ok case status @@ -1009,8 +1048,8 @@ public struct SessionsListParams: Codable, Sendable { label: String?, spawnedby: String?, agentid: String?, - search: String? - ) { + search: String?) + { self.limit = limit self.activeminutes = activeminutes self.includeglobal = includeglobal @@ -1022,6 +1061,7 @@ public struct SessionsListParams: Codable, Sendable { self.agentid = agentid self.search = search } + private enum CodingKeys: String, CodingKey { case limit case activeminutes = "activeMinutes" @@ -1044,12 +1084,13 @@ public struct SessionsPreviewParams: Codable, Sendable { public init( keys: [String], limit: Int?, - maxchars: Int? - ) { + maxchars: Int?) + { self.keys = keys self.limit = limit self.maxchars = maxchars } + private enum CodingKeys: String, CodingKey { case keys case limit @@ -1073,8 +1114,8 @@ public struct SessionsResolveParams: Codable, Sendable { agentid: String?, spawnedby: String?, includeglobal: Bool?, - includeunknown: Bool? - ) { + includeunknown: Bool?) + { self.key = key self.sessionid = sessionid self.label = label @@ -1083,6 +1124,7 @@ public struct SessionsResolveParams: Codable, Sendable { self.includeglobal = includeglobal self.includeunknown = includeunknown } + private enum CodingKeys: String, CodingKey { case key case sessionid = "sessionId" @@ -1128,8 +1170,8 @@ public struct SessionsPatchParams: Codable, Sendable { spawnedby: AnyCodable?, spawndepth: AnyCodable?, sendpolicy: AnyCodable?, - groupactivation: AnyCodable? - ) { + groupactivation: AnyCodable?) + { self.key = key self.label = label self.thinkinglevel = thinkinglevel @@ -1147,6 +1189,7 @@ public struct SessionsPatchParams: Codable, Sendable { self.sendpolicy = sendpolicy self.groupactivation = groupactivation } + private enum CodingKeys: String, CodingKey { case key case label @@ -1173,11 +1216,12 @@ public struct SessionsResetParams: Codable, Sendable { public init( key: String, - reason: AnyCodable? - ) { + reason: AnyCodable?) + { self.key = key self.reason = reason } + private enum CodingKeys: String, CodingKey { case key case reason @@ -1187,17 +1231,22 @@ public struct SessionsResetParams: Codable, Sendable { public struct SessionsDeleteParams: Codable, Sendable { public let key: String public let deletetranscript: Bool? + public let emitlifecyclehooks: Bool? public init( key: String, - deletetranscript: Bool? - ) { + deletetranscript: Bool?, + emitlifecyclehooks: Bool?) + { self.key = key self.deletetranscript = deletetranscript + self.emitlifecyclehooks = emitlifecyclehooks } + private enum CodingKeys: String, CodingKey { case key case deletetranscript = "deleteTranscript" + case emitlifecyclehooks = "emitLifecycleHooks" } } @@ -1207,11 +1256,12 @@ public struct SessionsCompactParams: Codable, Sendable { public init( key: String, - maxlines: Int? - ) { + maxlines: Int?) + { self.key = key self.maxlines = maxlines } + private enum CodingKeys: String, CodingKey { case key case maxlines = "maxLines" @@ -1222,6 +1272,8 @@ public struct SessionsUsageParams: Codable, Sendable { public let key: String? public let startdate: String? public let enddate: String? + public let mode: AnyCodable? + public let utcoffset: String? public let limit: Int? public let includecontextweight: Bool? @@ -1229,26 +1281,32 @@ public struct SessionsUsageParams: Codable, Sendable { key: String?, startdate: String?, enddate: String?, + mode: AnyCodable?, + utcoffset: String?, limit: Int?, - includecontextweight: Bool? - ) { + includecontextweight: Bool?) + { self.key = key self.startdate = startdate self.enddate = enddate + self.mode = mode + self.utcoffset = utcoffset self.limit = limit self.includecontextweight = includecontextweight } + private enum CodingKeys: String, CodingKey { case key case startdate = "startDate" case enddate = "endDate" + case mode + case utcoffset = "utcOffset" case limit case includecontextweight = "includeContextWeight" } } -public struct ConfigGetParams: Codable, Sendable { -} +public struct ConfigGetParams: Codable, Sendable {} public struct ConfigSetParams: Codable, Sendable { public let raw: String @@ -1256,11 +1314,12 @@ public struct ConfigSetParams: Codable, Sendable { public init( raw: String, - basehash: String? - ) { + basehash: String?) + { self.raw = raw self.basehash = basehash } + private enum CodingKeys: String, CodingKey { case raw case basehash = "baseHash" @@ -1279,14 +1338,15 @@ public struct ConfigApplyParams: Codable, Sendable { basehash: String?, sessionkey: String?, note: String?, - restartdelayms: Int? - ) { + restartdelayms: Int?) + { self.raw = raw self.basehash = basehash self.sessionkey = sessionkey self.note = note self.restartdelayms = restartdelayms } + private enum CodingKeys: String, CodingKey { case raw case basehash = "baseHash" @@ -1308,14 +1368,15 @@ public struct ConfigPatchParams: Codable, Sendable { basehash: String?, sessionkey: String?, note: String?, - restartdelayms: Int? - ) { + restartdelayms: Int?) + { self.raw = raw self.basehash = basehash self.sessionkey = sessionkey self.note = note self.restartdelayms = restartdelayms } + private enum CodingKeys: String, CodingKey { case raw case basehash = "baseHash" @@ -1325,8 +1386,7 @@ public struct ConfigPatchParams: Codable, Sendable { } } -public struct ConfigSchemaParams: Codable, Sendable { -} +public struct ConfigSchemaParams: Codable, Sendable {} public struct ConfigSchemaResponse: Codable, Sendable { public let schema: AnyCodable @@ -1338,13 +1398,14 @@ public struct ConfigSchemaResponse: Codable, Sendable { schema: AnyCodable, uihints: [String: AnyCodable], version: String, - generatedat: String - ) { + generatedat: String) + { self.schema = schema self.uihints = uihints self.version = version self.generatedat = generatedat } + private enum CodingKeys: String, CodingKey { case schema case uihints = "uiHints" @@ -1359,11 +1420,12 @@ public struct WizardStartParams: Codable, Sendable { public init( mode: AnyCodable?, - workspace: String? - ) { + workspace: String?) + { self.mode = mode self.workspace = workspace } + private enum CodingKeys: String, CodingKey { case mode case workspace @@ -1376,11 +1438,12 @@ public struct WizardNextParams: Codable, Sendable { public init( sessionid: String, - answer: [String: AnyCodable]? - ) { + answer: [String: AnyCodable]?) + { self.sessionid = sessionid self.answer = answer } + private enum CodingKeys: String, CodingKey { case sessionid = "sessionId" case answer @@ -1391,10 +1454,11 @@ public struct WizardCancelParams: Codable, Sendable { public let sessionid: String public init( - sessionid: String - ) { + sessionid: String) + { self.sessionid = sessionid } + private enum CodingKeys: String, CodingKey { case sessionid = "sessionId" } @@ -1404,10 +1468,11 @@ public struct WizardStatusParams: Codable, Sendable { public let sessionid: String public init( - sessionid: String - ) { + sessionid: String) + { self.sessionid = sessionid } + private enum CodingKeys: String, CodingKey { case sessionid = "sessionId" } @@ -1433,8 +1498,8 @@ public struct WizardStep: Codable, Sendable { initialvalue: AnyCodable?, placeholder: String?, sensitive: Bool?, - executor: AnyCodable? - ) { + executor: AnyCodable?) + { self.id = id self.type = type self.title = title @@ -1445,6 +1510,7 @@ public struct WizardStep: Codable, Sendable { self.sensitive = sensitive self.executor = executor } + private enum CodingKeys: String, CodingKey { case id case type @@ -1468,13 +1534,14 @@ public struct WizardNextResult: Codable, Sendable { done: Bool, step: [String: AnyCodable]?, status: AnyCodable?, - error: String? - ) { + error: String?) + { self.done = done self.step = step self.status = status self.error = error } + private enum CodingKeys: String, CodingKey { case done case step @@ -1495,14 +1562,15 @@ public struct WizardStartResult: Codable, Sendable { done: Bool, step: [String: AnyCodable]?, status: AnyCodable?, - error: String? - ) { + error: String?) + { self.sessionid = sessionid self.done = done self.step = step self.status = status self.error = error } + private enum CodingKeys: String, CodingKey { case sessionid = "sessionId" case done @@ -1518,11 +1586,12 @@ public struct WizardStatusResult: Codable, Sendable { public init( status: AnyCodable, - error: String? - ) { + error: String?) + { self.status = status self.error = error } + private enum CodingKeys: String, CodingKey { case status case error @@ -1535,11 +1604,12 @@ public struct TalkModeParams: Codable, Sendable { public init( enabled: Bool, - phase: String? - ) { + phase: String?) + { self.enabled = enabled self.phase = phase } + private enum CodingKeys: String, CodingKey { case enabled case phase @@ -1550,10 +1620,11 @@ public struct TalkConfigParams: Codable, Sendable { public let includesecrets: Bool? public init( - includesecrets: Bool? - ) { + includesecrets: Bool?) + { self.includesecrets = includesecrets } + private enum CodingKeys: String, CodingKey { case includesecrets = "includeSecrets" } @@ -1563,10 +1634,11 @@ public struct TalkConfigResult: Codable, Sendable { public let config: [String: AnyCodable] public init( - config: [String: AnyCodable] - ) { + config: [String: AnyCodable]) + { self.config = config } + private enum CodingKeys: String, CodingKey { case config } @@ -1578,11 +1650,12 @@ public struct ChannelsStatusParams: Codable, Sendable { public init( probe: Bool?, - timeoutms: Int? - ) { + timeoutms: Int?) + { self.probe = probe self.timeoutms = timeoutms } + private enum CodingKeys: String, CodingKey { case probe case timeoutms = "timeoutMs" @@ -1609,8 +1682,8 @@ public struct ChannelsStatusResult: Codable, Sendable { channelmeta: [[String: AnyCodable]]?, channels: [String: AnyCodable], channelaccounts: [String: AnyCodable], - channeldefaultaccountid: [String: AnyCodable] - ) { + channeldefaultaccountid: [String: AnyCodable]) + { self.ts = ts self.channelorder = channelorder self.channellabels = channellabels @@ -1621,6 +1694,7 @@ public struct ChannelsStatusResult: Codable, Sendable { self.channelaccounts = channelaccounts self.channeldefaultaccountid = channeldefaultaccountid } + private enum CodingKeys: String, CodingKey { case ts case channelorder = "channelOrder" @@ -1640,11 +1714,12 @@ public struct ChannelsLogoutParams: Codable, Sendable { public init( channel: String, - accountid: String? - ) { + accountid: String?) + { self.channel = channel self.accountid = accountid } + private enum CodingKeys: String, CodingKey { case channel case accountid = "accountId" @@ -1661,13 +1736,14 @@ public struct WebLoginStartParams: Codable, Sendable { force: Bool?, timeoutms: Int?, verbose: Bool?, - accountid: String? - ) { + accountid: String?) + { self.force = force self.timeoutms = timeoutms self.verbose = verbose self.accountid = accountid } + private enum CodingKeys: String, CodingKey { case force case timeoutms = "timeoutMs" @@ -1682,11 +1758,12 @@ public struct WebLoginWaitParams: Codable, Sendable { public init( timeoutms: Int?, - accountid: String? - ) { + accountid: String?) + { self.timeoutms = timeoutms self.accountid = accountid } + private enum CodingKeys: String, CodingKey { case timeoutms = "timeoutMs" case accountid = "accountId" @@ -1701,12 +1778,13 @@ public struct AgentSummary: Codable, Sendable { public init( id: String, name: String?, - identity: [String: AnyCodable]? - ) { + identity: [String: AnyCodable]?) + { self.id = id self.name = name self.identity = identity } + private enum CodingKeys: String, CodingKey { case id case name @@ -1724,13 +1802,14 @@ public struct AgentsCreateParams: Codable, Sendable { name: String, workspace: String, emoji: String?, - avatar: String? - ) { + avatar: String?) + { self.name = name self.workspace = workspace self.emoji = emoji self.avatar = avatar } + private enum CodingKeys: String, CodingKey { case name case workspace @@ -1749,13 +1828,14 @@ public struct AgentsCreateResult: Codable, Sendable { ok: Bool, agentid: String, name: String, - workspace: String - ) { + workspace: String) + { self.ok = ok self.agentid = agentid self.name = name self.workspace = workspace } + private enum CodingKeys: String, CodingKey { case ok case agentid = "agentId" @@ -1776,14 +1856,15 @@ public struct AgentsUpdateParams: Codable, Sendable { name: String?, workspace: String?, model: String?, - avatar: String? - ) { + avatar: String?) + { self.agentid = agentid self.name = name self.workspace = workspace self.model = model self.avatar = avatar } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" case name @@ -1799,11 +1880,12 @@ public struct AgentsUpdateResult: Codable, Sendable { public init( ok: Bool, - agentid: String - ) { + agentid: String) + { self.ok = ok self.agentid = agentid } + private enum CodingKeys: String, CodingKey { case ok case agentid = "agentId" @@ -1816,11 +1898,12 @@ public struct AgentsDeleteParams: Codable, Sendable { public init( agentid: String, - deletefiles: Bool? - ) { + deletefiles: Bool?) + { self.agentid = agentid self.deletefiles = deletefiles } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" case deletefiles = "deleteFiles" @@ -1835,12 +1918,13 @@ public struct AgentsDeleteResult: Codable, Sendable { public init( ok: Bool, agentid: String, - removedbindings: Int - ) { + removedbindings: Int) + { self.ok = ok self.agentid = agentid self.removedbindings = removedbindings } + private enum CodingKeys: String, CodingKey { case ok case agentid = "agentId" @@ -1862,8 +1946,8 @@ public struct AgentsFileEntry: Codable, Sendable { missing: Bool, size: Int?, updatedatms: Int?, - content: String? - ) { + content: String?) + { self.name = name self.path = path self.missing = missing @@ -1871,6 +1955,7 @@ public struct AgentsFileEntry: Codable, Sendable { self.updatedatms = updatedatms self.content = content } + private enum CodingKeys: String, CodingKey { case name case path @@ -1885,10 +1970,11 @@ public struct AgentsFilesListParams: Codable, Sendable { public let agentid: String public init( - agentid: String - ) { + agentid: String) + { self.agentid = agentid } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" } @@ -1902,12 +1988,13 @@ public struct AgentsFilesListResult: Codable, Sendable { public init( agentid: String, workspace: String, - files: [AgentsFileEntry] - ) { + files: [AgentsFileEntry]) + { self.agentid = agentid self.workspace = workspace self.files = files } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" case workspace @@ -1921,11 +2008,12 @@ public struct AgentsFilesGetParams: Codable, Sendable { public init( agentid: String, - name: String - ) { + name: String) + { self.agentid = agentid self.name = name } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" case name @@ -1940,12 +2028,13 @@ public struct AgentsFilesGetResult: Codable, Sendable { public init( agentid: String, workspace: String, - file: AgentsFileEntry - ) { + file: AgentsFileEntry) + { self.agentid = agentid self.workspace = workspace self.file = file } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" case workspace @@ -1961,12 +2050,13 @@ public struct AgentsFilesSetParams: Codable, Sendable { public init( agentid: String, name: String, - content: String - ) { + content: String) + { self.agentid = agentid self.name = name self.content = content } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" case name @@ -1984,13 +2074,14 @@ public struct AgentsFilesSetResult: Codable, Sendable { ok: Bool, agentid: String, workspace: String, - file: AgentsFileEntry - ) { + file: AgentsFileEntry) + { self.ok = ok self.agentid = agentid self.workspace = workspace self.file = file } + private enum CodingKeys: String, CodingKey { case ok case agentid = "agentId" @@ -1999,8 +2090,7 @@ public struct AgentsFilesSetResult: Codable, Sendable { } } -public struct AgentsListParams: Codable, Sendable { -} +public struct AgentsListParams: Codable, Sendable {} public struct AgentsListResult: Codable, Sendable { public let defaultid: String @@ -2012,13 +2102,14 @@ public struct AgentsListResult: Codable, Sendable { defaultid: String, mainkey: String, scope: AnyCodable, - agents: [AgentSummary] - ) { + agents: [AgentSummary]) + { self.defaultid = defaultid self.mainkey = mainkey self.scope = scope self.agents = agents } + private enum CodingKeys: String, CodingKey { case defaultid = "defaultId" case mainkey = "mainKey" @@ -2039,14 +2130,15 @@ public struct ModelChoice: Codable, Sendable { name: String, provider: String, contextwindow: Int?, - reasoning: Bool? - ) { + reasoning: Bool?) + { self.id = id self.name = name self.provider = provider self.contextwindow = contextwindow self.reasoning = reasoning } + private enum CodingKeys: String, CodingKey { case id case name @@ -2056,17 +2148,17 @@ public struct ModelChoice: Codable, Sendable { } } -public struct ModelsListParams: Codable, Sendable { -} +public struct ModelsListParams: Codable, Sendable {} public struct ModelsListResult: Codable, Sendable { public let models: [ModelChoice] public init( - models: [ModelChoice] - ) { + models: [ModelChoice]) + { self.models = models } + private enum CodingKeys: String, CodingKey { case models } @@ -2076,26 +2168,153 @@ public struct SkillsStatusParams: Codable, Sendable { public let agentid: String? public init( - agentid: String? - ) { + agentid: String?) + { self.agentid = agentid } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" } } -public struct SkillsBinsParams: Codable, Sendable { +public struct ToolsCatalogParams: Codable, Sendable { + public let agentid: String? + public let includeplugins: Bool? + + public init( + agentid: String?, + includeplugins: Bool?) + { + self.agentid = agentid + self.includeplugins = includeplugins + } + + private enum CodingKeys: String, CodingKey { + case agentid = "agentId" + case includeplugins = "includePlugins" + } } +public struct ToolCatalogProfile: Codable, Sendable { + public let id: AnyCodable + public let label: String + + public init( + id: AnyCodable, + label: String) + { + self.id = id + self.label = label + } + + private enum CodingKeys: String, CodingKey { + case id + case label + } +} + +public struct ToolCatalogEntry: Codable, Sendable { + public let id: String + public let label: String + public let description: String + public let source: AnyCodable + public let pluginid: String? + public let optional: Bool? + public let defaultprofiles: [AnyCodable] + + public init( + id: String, + label: String, + description: String, + source: AnyCodable, + pluginid: String?, + optional: Bool?, + defaultprofiles: [AnyCodable]) + { + self.id = id + self.label = label + self.description = description + self.source = source + self.pluginid = pluginid + self.optional = optional + self.defaultprofiles = defaultprofiles + } + + private enum CodingKeys: String, CodingKey { + case id + case label + case description + case source + case pluginid = "pluginId" + case optional + case defaultprofiles = "defaultProfiles" + } +} + +public struct ToolCatalogGroup: Codable, Sendable { + public let id: String + public let label: String + public let source: AnyCodable + public let pluginid: String? + public let tools: [ToolCatalogEntry] + + public init( + id: String, + label: String, + source: AnyCodable, + pluginid: String?, + tools: [ToolCatalogEntry]) + { + self.id = id + self.label = label + self.source = source + self.pluginid = pluginid + self.tools = tools + } + + private enum CodingKeys: String, CodingKey { + case id + case label + case source + case pluginid = "pluginId" + case tools + } +} + +public struct ToolsCatalogResult: Codable, Sendable { + public let agentid: String + public let profiles: [ToolCatalogProfile] + public let groups: [ToolCatalogGroup] + + public init( + agentid: String, + profiles: [ToolCatalogProfile], + groups: [ToolCatalogGroup]) + { + self.agentid = agentid + self.profiles = profiles + self.groups = groups + } + + private enum CodingKeys: String, CodingKey { + case agentid = "agentId" + case profiles + case groups + } +} + +public struct SkillsBinsParams: Codable, Sendable {} + public struct SkillsBinsResult: Codable, Sendable { public let bins: [String] public init( - bins: [String] - ) { + bins: [String]) + { self.bins = bins } + private enum CodingKeys: String, CodingKey { case bins } @@ -2109,12 +2328,13 @@ public struct SkillsInstallParams: Codable, Sendable { public init( name: String, installid: String, - timeoutms: Int? - ) { + timeoutms: Int?) + { self.name = name self.installid = installid self.timeoutms = timeoutms } + private enum CodingKeys: String, CodingKey { case name case installid = "installId" @@ -2132,13 +2352,14 @@ public struct SkillsUpdateParams: Codable, Sendable { skillkey: String, enabled: Bool?, apikey: String?, - env: [String: AnyCodable]? - ) { + env: [String: AnyCodable]?) + { self.skillkey = skillkey self.enabled = enabled self.apikey = apikey self.env = env } + private enum CodingKeys: String, CodingKey { case skillkey = "skillKey" case enabled @@ -2179,8 +2400,8 @@ public struct CronJob: Codable, Sendable { wakemode: AnyCodable, payload: AnyCodable, delivery: AnyCodable?, - state: [String: AnyCodable] - ) { + state: [String: AnyCodable]) + { self.id = id self.agentid = agentid self.sessionkey = sessionkey @@ -2197,6 +2418,7 @@ public struct CronJob: Codable, Sendable { self.delivery = delivery self.state = state } + private enum CodingKeys: String, CodingKey { case id case agentid = "agentId" @@ -2218,19 +2440,43 @@ public struct CronJob: Codable, Sendable { public struct CronListParams: Codable, Sendable { public let includedisabled: Bool? + public let limit: Int? + public let offset: Int? + public let query: String? + public let enabled: AnyCodable? + public let sortby: AnyCodable? + public let sortdir: AnyCodable? public init( - includedisabled: Bool? - ) { + includedisabled: Bool?, + limit: Int?, + offset: Int?, + query: String?, + enabled: AnyCodable?, + sortby: AnyCodable?, + sortdir: AnyCodable?) + { self.includedisabled = includedisabled + self.limit = limit + self.offset = offset + self.query = query + self.enabled = enabled + self.sortby = sortby + self.sortdir = sortdir } + private enum CodingKeys: String, CodingKey { case includedisabled = "includeDisabled" + case limit + case offset + case query + case enabled + case sortby = "sortBy" + case sortdir = "sortDir" } } -public struct CronStatusParams: Codable, Sendable { -} +public struct CronStatusParams: Codable, Sendable {} public struct CronAddParams: Codable, Sendable { public let name: String @@ -2256,8 +2502,8 @@ public struct CronAddParams: Codable, Sendable { sessiontarget: AnyCodable, wakemode: AnyCodable, payload: AnyCodable, - delivery: AnyCodable? - ) { + delivery: AnyCodable?) + { self.name = name self.agentid = agentid self.sessionkey = sessionkey @@ -2270,6 +2516,7 @@ public struct CronAddParams: Codable, Sendable { self.payload = payload self.delivery = delivery } + private enum CodingKeys: String, CodingKey { case name case agentid = "agentId" @@ -2285,6 +2532,60 @@ public struct CronAddParams: Codable, Sendable { } } +public struct CronRunsParams: Codable, Sendable { + public let scope: AnyCodable? + public let id: String? + public let jobid: String? + public let limit: Int? + public let offset: Int? + public let statuses: [AnyCodable]? + public let status: AnyCodable? + public let deliverystatuses: [AnyCodable]? + public let deliverystatus: AnyCodable? + public let query: String? + public let sortdir: AnyCodable? + + public init( + scope: AnyCodable?, + id: String?, + jobid: String?, + limit: Int?, + offset: Int?, + statuses: [AnyCodable]?, + status: AnyCodable?, + deliverystatuses: [AnyCodable]?, + deliverystatus: AnyCodable?, + query: String?, + sortdir: AnyCodable?) + { + self.scope = scope + self.id = id + self.jobid = jobid + self.limit = limit + self.offset = offset + self.statuses = statuses + self.status = status + self.deliverystatuses = deliverystatuses + self.deliverystatus = deliverystatus + self.query = query + self.sortdir = sortdir + } + + private enum CodingKeys: String, CodingKey { + case scope + case id + case jobid = "jobId" + case limit + case offset + case statuses + case status + case deliverystatuses = "deliveryStatuses" + case deliverystatus = "deliveryStatus" + case query + case sortdir = "sortDir" + } +} + public struct CronRunLogEntry: Codable, Sendable { public let ts: Int public let jobid: String @@ -2292,11 +2593,18 @@ public struct CronRunLogEntry: Codable, Sendable { public let status: AnyCodable? public let error: String? public let summary: String? + public let delivered: Bool? + public let deliverystatus: AnyCodable? + public let deliveryerror: String? public let sessionid: String? public let sessionkey: String? public let runatms: Int? public let durationms: Int? public let nextrunatms: Int? + public let model: String? + public let provider: String? + public let usage: [String: AnyCodable]? + public let jobname: String? public init( ts: Int, @@ -2305,24 +2613,39 @@ public struct CronRunLogEntry: Codable, Sendable { status: AnyCodable?, error: String?, summary: String?, + delivered: Bool?, + deliverystatus: AnyCodable?, + deliveryerror: String?, sessionid: String?, sessionkey: String?, runatms: Int?, durationms: Int?, - nextrunatms: Int? - ) { + nextrunatms: Int?, + model: String?, + provider: String?, + usage: [String: AnyCodable]?, + jobname: String?) + { self.ts = ts self.jobid = jobid self.action = action self.status = status self.error = error self.summary = summary + self.delivered = delivered + self.deliverystatus = deliverystatus + self.deliveryerror = deliveryerror self.sessionid = sessionid self.sessionkey = sessionkey self.runatms = runatms self.durationms = durationms self.nextrunatms = nextrunatms + self.model = model + self.provider = provider + self.usage = usage + self.jobname = jobname } + private enum CodingKeys: String, CodingKey { case ts case jobid = "jobId" @@ -2330,11 +2653,18 @@ public struct CronRunLogEntry: Codable, Sendable { case status case error case summary + case delivered + case deliverystatus = "deliveryStatus" + case deliveryerror = "deliveryError" case sessionid = "sessionId" case sessionkey = "sessionKey" case runatms = "runAtMs" case durationms = "durationMs" case nextrunatms = "nextRunAtMs" + case model + case provider + case usage + case jobname = "jobName" } } @@ -2346,12 +2676,13 @@ public struct LogsTailParams: Codable, Sendable { public init( cursor: Int?, limit: Int?, - maxbytes: Int? - ) { + maxbytes: Int?) + { self.cursor = cursor self.limit = limit self.maxbytes = maxbytes } + private enum CodingKeys: String, CodingKey { case cursor case limit @@ -2373,8 +2704,8 @@ public struct LogsTailResult: Codable, Sendable { size: Int, lines: [String], truncated: Bool?, - reset: Bool? - ) { + reset: Bool?) + { self.file = file self.cursor = cursor self.size = size @@ -2382,6 +2713,7 @@ public struct LogsTailResult: Codable, Sendable { self.truncated = truncated self.reset = reset } + private enum CodingKeys: String, CodingKey { case file case cursor @@ -2392,8 +2724,7 @@ public struct LogsTailResult: Codable, Sendable { } } -public struct ExecApprovalsGetParams: Codable, Sendable { -} +public struct ExecApprovalsGetParams: Codable, Sendable {} public struct ExecApprovalsSetParams: Codable, Sendable { public let file: [String: AnyCodable] @@ -2401,11 +2732,12 @@ public struct ExecApprovalsSetParams: Codable, Sendable { public init( file: [String: AnyCodable], - basehash: String? - ) { + basehash: String?) + { self.file = file self.basehash = basehash } + private enum CodingKeys: String, CodingKey { case file case basehash = "baseHash" @@ -2416,10 +2748,11 @@ public struct ExecApprovalsNodeGetParams: Codable, Sendable { public let nodeid: String public init( - nodeid: String - ) { + nodeid: String) + { self.nodeid = nodeid } + private enum CodingKeys: String, CodingKey { case nodeid = "nodeId" } @@ -2433,12 +2766,13 @@ public struct ExecApprovalsNodeSetParams: Codable, Sendable { public init( nodeid: String, file: [String: AnyCodable], - basehash: String? - ) { + basehash: String?) + { self.nodeid = nodeid self.file = file self.basehash = basehash } + private enum CodingKeys: String, CodingKey { case nodeid = "nodeId" case file @@ -2456,13 +2790,14 @@ public struct ExecApprovalsSnapshot: Codable, Sendable { path: String, exists: Bool, hash: String, - file: [String: AnyCodable] - ) { + file: [String: AnyCodable]) + { self.path = path self.exists = exists self.hash = hash self.file = file } + private enum CodingKeys: String, CodingKey { case path case exists @@ -2474,51 +2809,84 @@ public struct ExecApprovalsSnapshot: Codable, Sendable { public struct ExecApprovalRequestParams: Codable, Sendable { public let id: String? public let command: String + public let commandargv: [String]? + public let systemrunplanv2: [String: AnyCodable]? + public let env: [String: AnyCodable]? public let cwd: AnyCodable? + public let nodeid: AnyCodable? public let host: AnyCodable? public let security: AnyCodable? public let ask: AnyCodable? public let agentid: AnyCodable? public let resolvedpath: AnyCodable? public let sessionkey: AnyCodable? + public let turnsourcechannel: AnyCodable? + public let turnsourceto: AnyCodable? + public let turnsourceaccountid: AnyCodable? + public let turnsourcethreadid: AnyCodable? public let timeoutms: Int? public let twophase: Bool? public init( id: String?, command: String, + commandargv: [String]?, + systemrunplanv2: [String: AnyCodable]?, + env: [String: AnyCodable]?, cwd: AnyCodable?, + nodeid: AnyCodable?, host: AnyCodable?, security: AnyCodable?, ask: AnyCodable?, agentid: AnyCodable?, resolvedpath: AnyCodable?, sessionkey: AnyCodable?, + turnsourcechannel: AnyCodable?, + turnsourceto: AnyCodable?, + turnsourceaccountid: AnyCodable?, + turnsourcethreadid: AnyCodable?, timeoutms: Int?, - twophase: Bool? - ) { + twophase: Bool?) + { self.id = id self.command = command + self.commandargv = commandargv + self.systemrunplanv2 = systemrunplanv2 + self.env = env self.cwd = cwd + self.nodeid = nodeid self.host = host self.security = security self.ask = ask self.agentid = agentid self.resolvedpath = resolvedpath self.sessionkey = sessionkey + self.turnsourcechannel = turnsourcechannel + self.turnsourceto = turnsourceto + self.turnsourceaccountid = turnsourceaccountid + self.turnsourcethreadid = turnsourcethreadid self.timeoutms = timeoutms self.twophase = twophase } + private enum CodingKeys: String, CodingKey { case id case command + case commandargv = "commandArgv" + case systemrunplanv2 = "systemRunPlanV2" + case env case cwd + case nodeid = "nodeId" case host case security case ask case agentid = "agentId" case resolvedpath = "resolvedPath" case sessionkey = "sessionKey" + case turnsourcechannel = "turnSourceChannel" + case turnsourceto = "turnSourceTo" + case turnsourceaccountid = "turnSourceAccountId" + case turnsourcethreadid = "turnSourceThreadId" case timeoutms = "timeoutMs" case twophase = "twoPhase" } @@ -2530,28 +2898,29 @@ public struct ExecApprovalResolveParams: Codable, Sendable { public init( id: String, - decision: String - ) { + decision: String) + { self.id = id self.decision = decision } + private enum CodingKeys: String, CodingKey { case id case decision } } -public struct DevicePairListParams: Codable, Sendable { -} +public struct DevicePairListParams: Codable, Sendable {} public struct DevicePairApproveParams: Codable, Sendable { public let requestid: String public init( - requestid: String - ) { + requestid: String) + { self.requestid = requestid } + private enum CodingKeys: String, CodingKey { case requestid = "requestId" } @@ -2561,10 +2930,11 @@ public struct DevicePairRejectParams: Codable, Sendable { public let requestid: String public init( - requestid: String - ) { + requestid: String) + { self.requestid = requestid } + private enum CodingKeys: String, CodingKey { case requestid = "requestId" } @@ -2574,10 +2944,11 @@ public struct DevicePairRemoveParams: Codable, Sendable { public let deviceid: String public init( - deviceid: String - ) { + deviceid: String) + { self.deviceid = deviceid } + private enum CodingKeys: String, CodingKey { case deviceid = "deviceId" } @@ -2591,12 +2962,13 @@ public struct DeviceTokenRotateParams: Codable, Sendable { public init( deviceid: String, role: String, - scopes: [String]? - ) { + scopes: [String]?) + { self.deviceid = deviceid self.role = role self.scopes = scopes } + private enum CodingKeys: String, CodingKey { case deviceid = "deviceId" case role @@ -2610,11 +2982,12 @@ public struct DeviceTokenRevokeParams: Codable, Sendable { public init( deviceid: String, - role: String - ) { + role: String) + { self.deviceid = deviceid self.role = role } + private enum CodingKeys: String, CodingKey { case deviceid = "deviceId" case role @@ -2627,6 +3000,7 @@ public struct DevicePairRequestedEvent: Codable, Sendable { public let publickey: String public let displayname: String? public let platform: String? + public let devicefamily: String? public let clientid: String? public let clientmode: String? public let role: String? @@ -2643,6 +3017,7 @@ public struct DevicePairRequestedEvent: Codable, Sendable { publickey: String, displayname: String?, platform: String?, + devicefamily: String?, clientid: String?, clientmode: String?, role: String?, @@ -2651,13 +3026,14 @@ public struct DevicePairRequestedEvent: Codable, Sendable { remoteip: String?, silent: Bool?, isrepair: Bool?, - ts: Int - ) { + ts: Int) + { self.requestid = requestid self.deviceid = deviceid self.publickey = publickey self.displayname = displayname self.platform = platform + self.devicefamily = devicefamily self.clientid = clientid self.clientmode = clientmode self.role = role @@ -2668,12 +3044,14 @@ public struct DevicePairRequestedEvent: Codable, Sendable { self.isrepair = isrepair self.ts = ts } + private enum CodingKeys: String, CodingKey { case requestid = "requestId" case deviceid = "deviceId" case publickey = "publicKey" case displayname = "displayName" case platform + case devicefamily = "deviceFamily" case clientid = "clientId" case clientmode = "clientMode" case role @@ -2696,13 +3074,14 @@ public struct DevicePairResolvedEvent: Codable, Sendable { requestid: String, deviceid: String, decision: String, - ts: Int - ) { + ts: Int) + { self.requestid = requestid self.deviceid = deviceid self.decision = decision self.ts = ts } + private enum CodingKeys: String, CodingKey { case requestid = "requestId" case deviceid = "deviceId" @@ -2717,11 +3096,12 @@ public struct ChatHistoryParams: Codable, Sendable { public init( sessionkey: String, - limit: Int? - ) { + limit: Int?) + { self.sessionkey = sessionkey self.limit = limit } + private enum CodingKeys: String, CodingKey { case sessionkey = "sessionKey" case limit @@ -2744,8 +3124,8 @@ public struct ChatSendParams: Codable, Sendable { deliver: Bool?, attachments: [AnyCodable]?, timeoutms: Int?, - idempotencykey: String - ) { + idempotencykey: String) + { self.sessionkey = sessionkey self.message = message self.thinking = thinking @@ -2754,6 +3134,7 @@ public struct ChatSendParams: Codable, Sendable { self.timeoutms = timeoutms self.idempotencykey = idempotencykey } + private enum CodingKeys: String, CodingKey { case sessionkey = "sessionKey" case message @@ -2771,11 +3152,12 @@ public struct ChatAbortParams: Codable, Sendable { public init( sessionkey: String, - runid: String? - ) { + runid: String?) + { self.sessionkey = sessionkey self.runid = runid } + private enum CodingKeys: String, CodingKey { case sessionkey = "sessionKey" case runid = "runId" @@ -2790,12 +3172,13 @@ public struct ChatInjectParams: Codable, Sendable { public init( sessionkey: String, message: String, - label: String? - ) { + label: String?) + { self.sessionkey = sessionkey self.message = message self.label = label } + private enum CodingKeys: String, CodingKey { case sessionkey = "sessionKey" case message @@ -2821,8 +3204,8 @@ public struct ChatEvent: Codable, Sendable { message: AnyCodable?, errormessage: String?, usage: AnyCodable?, - stopreason: String? - ) { + stopreason: String?) + { self.runid = runid self.sessionkey = sessionkey self.seq = seq @@ -2832,6 +3215,7 @@ public struct ChatEvent: Codable, Sendable { self.usage = usage self.stopreason = stopreason } + private enum CodingKeys: String, CodingKey { case runid = "runId" case sessionkey = "sessionKey" @@ -2854,13 +3238,14 @@ public struct UpdateRunParams: Codable, Sendable { sessionkey: String?, note: String?, restartdelayms: Int?, - timeoutms: Int? - ) { + timeoutms: Int?) + { self.sessionkey = sessionkey self.note = note self.restartdelayms = restartdelayms self.timeoutms = timeoutms } + private enum CodingKeys: String, CodingKey { case sessionkey = "sessionKey" case note @@ -2873,10 +3258,11 @@ public struct TickEvent: Codable, Sendable { public let ts: Int public init( - ts: Int - ) { + ts: Int) + { self.ts = ts } + private enum CodingKeys: String, CodingKey { case ts } @@ -2888,11 +3274,12 @@ public struct ShutdownEvent: Codable, Sendable { public init( reason: String, - restartexpectedms: Int? - ) { + restartexpectedms: Int?) + { self.reason = reason self.restartexpectedms = restartexpectedms } + private enum CodingKeys: String, CodingKey { case reason case restartexpectedms = "restartExpectedMs" @@ -2914,11 +3301,11 @@ public enum GatewayFrame: Codable, Sendable { let type = try typeContainer.decode(String.self, forKey: .type) switch type { case "req": - self = .req(try RequestFrame(from: decoder)) + self = try .req(RequestFrame(from: decoder)) case "res": - self = .res(try ResponseFrame(from: decoder)) + self = try .res(ResponseFrame(from: decoder)) case "event": - self = .event(try EventFrame(from: decoder)) + self = try .event(EventFrame(from: decoder)) default: let container = try decoder.singleValueContainer() let raw = try container.decode([String: AnyCodable].self) @@ -2928,13 +3315,15 @@ public enum GatewayFrame: Codable, Sendable { public func encode(to encoder: Encoder) throws { switch self { - case .req(let v): try v.encode(to: encoder) - case .res(let v): try v.encode(to: encoder) - case .event(let v): try v.encode(to: encoder) - case .unknown(_, let raw): + case let .req(v): + try v.encode(to: encoder) + case let .res(v): + try v.encode(to: encoder) + case let .event(v): + try v.encode(to: encoder) + case let .unknown(_, raw): var container = encoder.singleValueContainer() try container.encode(raw) } } - } diff --git a/apps/macos/Tests/OpenClawIPCTests/AgentWorkspaceTests.swift b/apps/macos/Tests/OpenClawIPCTests/AgentWorkspaceTests.swift index 6d5e4a37efd..8794a3f22fc 100644 --- a/apps/macos/Tests/OpenClawIPCTests/AgentWorkspaceTests.swift +++ b/apps/macos/Tests/OpenClawIPCTests/AgentWorkspaceTests.swift @@ -59,12 +59,7 @@ struct AgentWorkspaceTests { try "hello".write(to: marker, atomically: true, encoding: .utf8) let result = AgentWorkspace.bootstrapSafety(for: tmp) - switch result { - case .unsafe: - break - case .safe: - #expect(Bool(false), "Expected unsafe bootstrap safety result.") - } + #expect(result.unsafeReason != nil) } @Test @@ -77,12 +72,7 @@ struct AgentWorkspaceTests { try "# AGENTS.md".write(to: agents, atomically: true, encoding: .utf8) let result = AgentWorkspace.bootstrapSafety(for: tmp) - switch result { - case .safe: - break - case .unsafe: - #expect(Bool(false), "Expected safe bootstrap safety result.") - } + #expect(result.unsafeReason == nil) } @Test diff --git a/apps/macos/Tests/OpenClawIPCTests/AnthropicAuthControlsSmokeTests.swift b/apps/macos/Tests/OpenClawIPCTests/AnthropicAuthControlsSmokeTests.swift deleted file mode 100644 index 84c61833932..00000000000 --- a/apps/macos/Tests/OpenClawIPCTests/AnthropicAuthControlsSmokeTests.swift +++ /dev/null @@ -1,29 +0,0 @@ -import Testing -@testable import OpenClaw - -@Suite(.serialized) -@MainActor -struct AnthropicAuthControlsSmokeTests { - @Test func anthropicAuthControlsBuildsBodyLocal() { - let pkce = AnthropicOAuth.PKCE(verifier: "verifier", challenge: "challenge") - let view = AnthropicAuthControls( - connectionMode: .local, - oauthStatus: .connected(expiresAtMs: 1_700_000_000_000), - pkce: pkce, - code: "code#state", - statusText: "Detected code", - autoDetectClipboard: false, - autoConnectClipboard: false) - _ = view.body - } - - @Test func anthropicAuthControlsBuildsBodyRemote() { - let view = AnthropicAuthControls( - connectionMode: .remote, - oauthStatus: .missingFile, - pkce: nil, - code: "", - statusText: nil) - _ = view.body - } -} diff --git a/apps/macos/Tests/OpenClawIPCTests/AnthropicAuthResolverTests.swift b/apps/macos/Tests/OpenClawIPCTests/AnthropicAuthResolverTests.swift deleted file mode 100644 index c41b7f64be4..00000000000 --- a/apps/macos/Tests/OpenClawIPCTests/AnthropicAuthResolverTests.swift +++ /dev/null @@ -1,52 +0,0 @@ -import Foundation -import Testing -@testable import OpenClaw - -@Suite -struct AnthropicAuthResolverTests { - @Test - func prefersOAuthFileOverEnv() throws { - let dir = FileManager().temporaryDirectory - .appendingPathComponent("openclaw-oauth-\(UUID().uuidString)", isDirectory: true) - try FileManager().createDirectory(at: dir, withIntermediateDirectories: true) - let oauthFile = dir.appendingPathComponent("oauth.json") - let payload = [ - "anthropic": [ - "type": "oauth", - "refresh": "r1", - "access": "a1", - "expires": 1_234_567_890, - ], - ] - let data = try JSONSerialization.data(withJSONObject: payload, options: [.prettyPrinted, .sortedKeys]) - try data.write(to: oauthFile, options: [.atomic]) - - let status = OpenClawOAuthStore.anthropicOAuthStatus(at: oauthFile) - let mode = AnthropicAuthResolver.resolve(environment: [ - "ANTHROPIC_API_KEY": "sk-ant-ignored", - ], oauthStatus: status) - #expect(mode == .oauthFile) - } - - @Test - func reportsOAuthEnvWhenPresent() { - let mode = AnthropicAuthResolver.resolve(environment: [ - "ANTHROPIC_OAUTH_TOKEN": "token", - ], oauthStatus: .missingFile) - #expect(mode == .oauthEnv) - } - - @Test - func reportsAPIKeyEnvWhenPresent() { - let mode = AnthropicAuthResolver.resolve(environment: [ - "ANTHROPIC_API_KEY": "sk-ant-key", - ], oauthStatus: .missingFile) - #expect(mode == .apiKeyEnv) - } - - @Test - func reportsMissingWhenNothingConfigured() { - let mode = AnthropicAuthResolver.resolve(environment: [:], oauthStatus: .missingFile) - #expect(mode == .missing) - } -} diff --git a/apps/macos/Tests/OpenClawIPCTests/AnthropicOAuthCodeStateTests.swift b/apps/macos/Tests/OpenClawIPCTests/AnthropicOAuthCodeStateTests.swift deleted file mode 100644 index 3d337c2b279..00000000000 --- a/apps/macos/Tests/OpenClawIPCTests/AnthropicOAuthCodeStateTests.swift +++ /dev/null @@ -1,31 +0,0 @@ -import Testing -@testable import OpenClaw - -@Suite -struct AnthropicOAuthCodeStateTests { - @Test - func parsesRawToken() { - let parsed = AnthropicOAuthCodeState.parse(from: "abcDEF1234#stateXYZ9876") - #expect(parsed == .init(code: "abcDEF1234", state: "stateXYZ9876")) - } - - @Test - func parsesBacktickedToken() { - let parsed = AnthropicOAuthCodeState.parse(from: "`abcDEF1234#stateXYZ9876`") - #expect(parsed == .init(code: "abcDEF1234", state: "stateXYZ9876")) - } - - @Test - func parsesCallbackURL() { - let raw = "https://console.anthropic.com/oauth/code/callback?code=abcDEF1234&state=stateXYZ9876" - let parsed = AnthropicOAuthCodeState.parse(from: raw) - #expect(parsed == .init(code: "abcDEF1234", state: "stateXYZ9876")) - } - - @Test - func extractsFromSurroundingText() { - let raw = "Paste the code#state value: abcDEF1234#stateXYZ9876 then return." - let parsed = AnthropicOAuthCodeState.parse(from: raw) - #expect(parsed == .init(code: "abcDEF1234", state: "stateXYZ9876")) - } -} diff --git a/apps/macos/Tests/OpenClawIPCTests/AudioInputDeviceObserverTests.swift b/apps/macos/Tests/OpenClawIPCTests/AudioInputDeviceObserverTests.swift new file mode 100644 index 00000000000..a175e5e1a0a --- /dev/null +++ b/apps/macos/Tests/OpenClawIPCTests/AudioInputDeviceObserverTests.swift @@ -0,0 +1,21 @@ +import Foundation +import Testing +@testable import OpenClaw + +@Suite struct AudioInputDeviceObserverTests { + @Test func hasUsableDefaultInputDeviceReturnsBool() { + // Smoke test: verifies the composition logic runs without crashing. + // Actual result depends on whether the host has an audio input device. + let result = AudioInputDeviceObserver.hasUsableDefaultInputDevice() + _ = result // suppress unused-variable warning; the assertion is "no crash" + } + + @Test func hasUsableDefaultInputDeviceConsistentWithComponents() { + // When no default UID exists, the method must return false. + // When a default UID exists, the result must match alive-set membership. + let uid = AudioInputDeviceObserver.defaultInputDeviceUID() + let alive = AudioInputDeviceObserver.aliveInputDeviceUIDs() + let expected = uid.map { alive.contains($0) } ?? false + #expect(AudioInputDeviceObserver.hasUsableDefaultInputDevice() == expected) + } +} diff --git a/apps/macos/Tests/OpenClawIPCTests/CommandResolverTests.swift b/apps/macos/Tests/OpenClawIPCTests/CommandResolverTests.swift index 7a71bc08b6e..d8470679183 100644 --- a/apps/macos/Tests/OpenClawIPCTests/CommandResolverTests.swift +++ b/apps/macos/Tests/OpenClawIPCTests/CommandResolverTests.swift @@ -66,6 +66,48 @@ import Testing } } + @Test func prefersOpenClawBinaryOverPnpm() async throws { + let defaults = self.makeDefaults() + defaults.set(AppState.ConnectionMode.local.rawValue, forKey: connectionModeKey) + + let tmp = try makeTempDir() + CommandResolver.setProjectRoot(tmp.path) + + let binDir = tmp.appendingPathComponent("bin") + let openclawPath = binDir.appendingPathComponent("openclaw") + let pnpmPath = binDir.appendingPathComponent("pnpm") + try self.makeExec(at: openclawPath) + try self.makeExec(at: pnpmPath) + + let cmd = CommandResolver.openclawCommand( + subcommand: "rpc", + defaults: defaults, + configRoot: [:], + searchPaths: [binDir.path]) + + #expect(cmd.prefix(2).elementsEqual([openclawPath.path, "rpc"])) + } + + @Test func usesOpenClawBinaryWithoutNodeRuntime() async throws { + let defaults = self.makeDefaults() + defaults.set(AppState.ConnectionMode.local.rawValue, forKey: connectionModeKey) + + let tmp = try makeTempDir() + CommandResolver.setProjectRoot(tmp.path) + + let binDir = tmp.appendingPathComponent("bin") + let openclawPath = binDir.appendingPathComponent("openclaw") + try self.makeExec(at: openclawPath) + + let cmd = CommandResolver.openclawCommand( + subcommand: "gateway", + defaults: defaults, + configRoot: [:], + searchPaths: [binDir.path]) + + #expect(cmd.prefix(2).elementsEqual([openclawPath.path, "gateway"])) + } + @Test func fallsBackToPnpm() async throws { let defaults = self.makeDefaults() defaults.set(AppState.ConnectionMode.local.rawValue, forKey: connectionModeKey) @@ -76,7 +118,11 @@ import Testing let pnpmPath = tmp.appendingPathComponent("node_modules/.bin/pnpm") try self.makeExec(at: pnpmPath) - let cmd = CommandResolver.openclawCommand(subcommand: "rpc", defaults: defaults, configRoot: [:]) + let cmd = CommandResolver.openclawCommand( + subcommand: "rpc", + defaults: defaults, + configRoot: [:], + searchPaths: [tmp.appendingPathComponent("node_modules/.bin").path]) #expect(cmd.prefix(4).elementsEqual([pnpmPath.path, "--silent", "openclaw", "rpc"])) } @@ -95,7 +141,8 @@ import Testing subcommand: "health", extraArgs: ["--json", "--timeout", "5"], defaults: defaults, - configRoot: [:]) + configRoot: [:], + searchPaths: [tmp.appendingPathComponent("node_modules/.bin").path]) #expect(cmd.prefix(5).elementsEqual([pnpmPath.path, "--silent", "openclaw", "health", "--json"])) #expect(cmd.suffix(2).elementsEqual(["--timeout", "5"])) diff --git a/apps/macos/Tests/OpenClawIPCTests/ExecAllowlistTests.swift b/apps/macos/Tests/OpenClawIPCTests/ExecAllowlistTests.swift index 7da886ea794..3b27740d066 100644 --- a/apps/macos/Tests/OpenClawIPCTests/ExecAllowlistTests.swift +++ b/apps/macos/Tests/OpenClawIPCTests/ExecAllowlistTests.swift @@ -2,7 +2,55 @@ import Foundation import Testing @testable import OpenClaw +/// These cases cover optional `security=allowlist` behavior. +/// Default install posture remains deny-by-default for exec on macOS node-host. struct ExecAllowlistTests { + private struct ShellParserParityFixture: Decodable { + struct Case: Decodable { + let id: String + let command: String + let ok: Bool + let executables: [String] + } + + let cases: [Case] + } + + private struct WrapperResolutionParityFixture: Decodable { + struct Case: Decodable { + let id: String + let argv: [String] + let expectedRawExecutable: String? + } + + let cases: [Case] + } + + private static func loadShellParserParityCases() throws -> [ShellParserParityFixture.Case] { + let fixtureURL = self.fixtureURL(filename: "exec-allowlist-shell-parser-parity.json") + let data = try Data(contentsOf: fixtureURL) + let fixture = try JSONDecoder().decode(ShellParserParityFixture.self, from: data) + return fixture.cases + } + + private static func loadWrapperResolutionParityCases() throws -> [WrapperResolutionParityFixture.Case] { + let fixtureURL = self.fixtureURL(filename: "exec-wrapper-resolution-parity.json") + let data = try Data(contentsOf: fixtureURL) + let fixture = try JSONDecoder().decode(WrapperResolutionParityFixture.self, from: data) + return fixture.cases + } + + private static func fixtureURL(filename: String) -> URL { + var repoRoot = URL(fileURLWithPath: #filePath) + for _ in 0..<5 { + repoRoot.deleteLastPathComponent() + } + return repoRoot + .appendingPathComponent("test") + .appendingPathComponent("fixtures") + .appendingPathComponent(filename) + } + @Test func matchUsesResolvedPath() { let entry = ExecAllowlistEntry(pattern: "/opt/homebrew/bin/rg") let resolution = ExecCommandResolution( @@ -14,7 +62,7 @@ struct ExecAllowlistTests { #expect(match?.pattern == entry.pattern) } - @Test func matchUsesBasenameForSimplePattern() { + @Test func matchIgnoresBasenamePattern() { let entry = ExecAllowlistEntry(pattern: "rg") let resolution = ExecCommandResolution( rawExecutable: "rg", @@ -22,11 +70,22 @@ struct ExecAllowlistTests { executableName: "rg", cwd: nil) let match = ExecAllowlistMatcher.match(entries: [entry], resolution: resolution) - #expect(match?.pattern == entry.pattern) + #expect(match == nil) + } + + @Test func matchIgnoresBasenameForRelativeExecutable() { + let entry = ExecAllowlistEntry(pattern: "echo") + let resolution = ExecCommandResolution( + rawExecutable: "./echo", + resolvedPath: "/tmp/oc-basename/echo", + executableName: "echo", + cwd: "/tmp/oc-basename") + let match = ExecAllowlistMatcher.match(entries: [entry], resolution: resolution) + #expect(match == nil) } @Test func matchIsCaseInsensitive() { - let entry = ExecAllowlistEntry(pattern: "RG") + let entry = ExecAllowlistEntry(pattern: "/OPT/HOMEBREW/BIN/RG") let resolution = ExecCommandResolution( rawExecutable: "rg", resolvedPath: "/opt/homebrew/bin/rg", @@ -46,4 +105,145 @@ struct ExecAllowlistTests { let match = ExecAllowlistMatcher.match(entries: [entry], resolution: resolution) #expect(match?.pattern == entry.pattern) } + + @Test func resolveForAllowlistSplitsShellChains() { + let command = ["/bin/sh", "-lc", "echo allowlisted && /usr/bin/touch /tmp/openclaw-allowlist-test"] + let resolutions = ExecCommandResolution.resolveForAllowlist( + command: command, + rawCommand: "echo allowlisted && /usr/bin/touch /tmp/openclaw-allowlist-test", + cwd: nil, + env: ["PATH": "/usr/bin:/bin"]) + #expect(resolutions.count == 2) + #expect(resolutions[0].executableName == "echo") + #expect(resolutions[1].executableName == "touch") + } + + @Test func resolveForAllowlistKeepsQuotedOperatorsInSingleSegment() { + let command = ["/bin/sh", "-lc", "echo \"a && b\""] + let resolutions = ExecCommandResolution.resolveForAllowlist( + command: command, + rawCommand: "echo \"a && b\"", + cwd: nil, + env: ["PATH": "/usr/bin:/bin"]) + #expect(resolutions.count == 1) + #expect(resolutions[0].executableName == "echo") + } + + @Test func resolveForAllowlistFailsClosedOnCommandSubstitution() { + let command = ["/bin/sh", "-lc", "echo $(/usr/bin/touch /tmp/openclaw-allowlist-test-subst)"] + let resolutions = ExecCommandResolution.resolveForAllowlist( + command: command, + rawCommand: "echo $(/usr/bin/touch /tmp/openclaw-allowlist-test-subst)", + cwd: nil, + env: ["PATH": "/usr/bin:/bin"]) + #expect(resolutions.isEmpty) + } + + @Test func resolveForAllowlistFailsClosedOnQuotedCommandSubstitution() { + let command = ["/bin/sh", "-lc", "echo \"ok $(/usr/bin/touch /tmp/openclaw-allowlist-test-quoted-subst)\""] + let resolutions = ExecCommandResolution.resolveForAllowlist( + command: command, + rawCommand: "echo \"ok $(/usr/bin/touch /tmp/openclaw-allowlist-test-quoted-subst)\"", + cwd: nil, + env: ["PATH": "/usr/bin:/bin"]) + #expect(resolutions.isEmpty) + } + + @Test func resolveForAllowlistFailsClosedOnQuotedBackticks() { + let command = ["/bin/sh", "-lc", "echo \"ok `/usr/bin/id`\""] + let resolutions = ExecCommandResolution.resolveForAllowlist( + command: command, + rawCommand: "echo \"ok `/usr/bin/id`\"", + cwd: nil, + env: ["PATH": "/usr/bin:/bin"]) + #expect(resolutions.isEmpty) + } + + @Test func resolveForAllowlistMatchesSharedShellParserFixture() throws { + let fixtures = try Self.loadShellParserParityCases() + for fixture in fixtures { + let resolutions = ExecCommandResolution.resolveForAllowlist( + command: ["/bin/sh", "-lc", fixture.command], + rawCommand: fixture.command, + cwd: nil, + env: ["PATH": "/usr/bin:/bin"]) + + #expect(!resolutions.isEmpty == fixture.ok) + if fixture.ok { + let executables = resolutions.map { $0.executableName.lowercased() } + let expected = fixture.executables.map { $0.lowercased() } + #expect(executables == expected) + } + } + } + + @Test func resolveMatchesSharedWrapperResolutionFixture() throws { + let fixtures = try Self.loadWrapperResolutionParityCases() + for fixture in fixtures { + let resolution = ExecCommandResolution.resolve( + command: fixture.argv, + cwd: nil, + env: ["PATH": "/usr/bin:/bin"]) + #expect(resolution?.rawExecutable == fixture.expectedRawExecutable) + } + } + + @Test func resolveForAllowlistTreatsPlainShInvocationAsDirectExec() { + let command = ["/bin/sh", "./script.sh"] + let resolutions = ExecCommandResolution.resolveForAllowlist( + command: command, + rawCommand: nil, + cwd: "/tmp", + env: ["PATH": "/usr/bin:/bin"]) + #expect(resolutions.count == 1) + #expect(resolutions[0].executableName == "sh") + } + + @Test func resolveForAllowlistUnwrapsEnvShellWrapperChains() { + let command = ["/usr/bin/env", "/bin/sh", "-lc", "echo allowlisted && /usr/bin/touch /tmp/openclaw-allowlist-test"] + let resolutions = ExecCommandResolution.resolveForAllowlist( + command: command, + rawCommand: nil, + cwd: nil, + env: ["PATH": "/usr/bin:/bin"]) + #expect(resolutions.count == 2) + #expect(resolutions[0].executableName == "echo") + #expect(resolutions[1].executableName == "touch") + } + + @Test func resolveForAllowlistUnwrapsEnvToEffectiveDirectExecutable() { + let command = ["/usr/bin/env", "FOO=bar", "/usr/bin/printf", "ok"] + let resolutions = ExecCommandResolution.resolveForAllowlist( + command: command, + rawCommand: nil, + cwd: nil, + env: ["PATH": "/usr/bin:/bin"]) + #expect(resolutions.count == 1) + #expect(resolutions[0].resolvedPath == "/usr/bin/printf") + #expect(resolutions[0].executableName == "printf") + } + + @Test func matchAllRequiresEverySegmentToMatch() { + let first = ExecCommandResolution( + rawExecutable: "echo", + resolvedPath: "/usr/bin/echo", + executableName: "echo", + cwd: nil) + let second = ExecCommandResolution( + rawExecutable: "/usr/bin/touch", + resolvedPath: "/usr/bin/touch", + executableName: "touch", + cwd: nil) + let resolutions = [first, second] + + let partial = ExecAllowlistMatcher.matchAll( + entries: [ExecAllowlistEntry(pattern: "/usr/bin/echo")], + resolutions: resolutions) + #expect(partial.isEmpty) + + let full = ExecAllowlistMatcher.matchAll( + entries: [ExecAllowlistEntry(pattern: "/USR/BIN/ECHO"), ExecAllowlistEntry(pattern: "/usr/bin/touch")], + resolutions: resolutions) + #expect(full.count == 2) + } } diff --git a/apps/macos/Tests/OpenClawIPCTests/ExecApprovalHelpersTests.swift b/apps/macos/Tests/OpenClawIPCTests/ExecApprovalHelpersTests.swift index 760d6c9178e..455b4296753 100644 --- a/apps/macos/Tests/OpenClawIPCTests/ExecApprovalHelpersTests.swift +++ b/apps/macos/Tests/OpenClawIPCTests/ExecApprovalHelpersTests.swift @@ -29,6 +29,24 @@ import Testing #expect(ExecApprovalHelpers.allowlistPattern(command: [], resolution: nil) == nil) } + @Test func validateAllowlistPatternReturnsReasons() { + #expect(ExecApprovalHelpers.isPathPattern("/usr/bin/rg")) + #expect(ExecApprovalHelpers.isPathPattern(" ~/bin/rg ")) + #expect(!ExecApprovalHelpers.isPathPattern("rg")) + + if case .invalid(let reason) = ExecApprovalHelpers.validateAllowlistPattern(" ") { + #expect(reason == .empty) + } else { + Issue.record("Expected empty pattern rejection") + } + + if case .invalid(let reason) = ExecApprovalHelpers.validateAllowlistPattern("echo") { + #expect(reason == .missingPathComponent) + } else { + Issue.record("Expected basename pattern rejection") + } + } + @Test func requiresAskMatchesPolicy() { let entry = ExecAllowlistEntry(pattern: "/bin/ls", lastUsedAt: nil, lastUsedCommand: nil, lastResolvedPath: nil) #expect(ExecApprovalHelpers.requiresAsk( diff --git a/apps/macos/Tests/OpenClawIPCTests/ExecApprovalsStoreRefactorTests.swift b/apps/macos/Tests/OpenClawIPCTests/ExecApprovalsStoreRefactorTests.swift new file mode 100644 index 00000000000..fa9eef87881 --- /dev/null +++ b/apps/macos/Tests/OpenClawIPCTests/ExecApprovalsStoreRefactorTests.swift @@ -0,0 +1,75 @@ +import Foundation +import Testing +@testable import OpenClaw + +@Suite(.serialized) +struct ExecApprovalsStoreRefactorTests { + @Test + func ensureFileSkipsRewriteWhenUnchanged() async throws { + let stateDir = FileManager().temporaryDirectory + .appendingPathComponent("openclaw-state-\(UUID().uuidString)", isDirectory: true) + defer { try? FileManager().removeItem(at: stateDir) } + + try await TestIsolation.withEnvValues(["OPENCLAW_STATE_DIR": stateDir.path]) { + _ = ExecApprovalsStore.ensureFile() + let url = ExecApprovalsStore.fileURL() + let firstWriteDate = try Self.modificationDate(at: url) + + try await Task.sleep(nanoseconds: 1_100_000_000) + _ = ExecApprovalsStore.ensureFile() + let secondWriteDate = try Self.modificationDate(at: url) + + #expect(firstWriteDate == secondWriteDate) + } + } + + @Test + func updateAllowlistReportsRejectedBasenamePattern() async throws { + let stateDir = FileManager().temporaryDirectory + .appendingPathComponent("openclaw-state-\(UUID().uuidString)", isDirectory: true) + defer { try? FileManager().removeItem(at: stateDir) } + + await TestIsolation.withEnvValues(["OPENCLAW_STATE_DIR": stateDir.path]) { + let rejected = ExecApprovalsStore.updateAllowlist( + agentId: "main", + allowlist: [ + ExecAllowlistEntry(pattern: "echo"), + ExecAllowlistEntry(pattern: "/bin/echo"), + ]) + #expect(rejected.count == 1) + #expect(rejected.first?.reason == .missingPathComponent) + #expect(rejected.first?.pattern == "echo") + + let resolved = ExecApprovalsStore.resolve(agentId: "main") + #expect(resolved.allowlist.map(\.pattern) == ["/bin/echo"]) + } + } + + @Test + func updateAllowlistMigratesLegacyPatternFromResolvedPath() async throws { + let stateDir = FileManager().temporaryDirectory + .appendingPathComponent("openclaw-state-\(UUID().uuidString)", isDirectory: true) + defer { try? FileManager().removeItem(at: stateDir) } + + await TestIsolation.withEnvValues(["OPENCLAW_STATE_DIR": stateDir.path]) { + let rejected = ExecApprovalsStore.updateAllowlist( + agentId: "main", + allowlist: [ + ExecAllowlistEntry(pattern: "echo", lastUsedAt: nil, lastUsedCommand: nil, lastResolvedPath: " /usr/bin/echo "), + ]) + #expect(rejected.isEmpty) + + let resolved = ExecApprovalsStore.resolve(agentId: "main") + #expect(resolved.allowlist.map(\.pattern) == ["/usr/bin/echo"]) + } + } + + private static func modificationDate(at url: URL) throws -> Date { + let attributes = try FileManager().attributesOfItem(atPath: url.path) + guard let date = attributes[.modificationDate] as? Date else { + struct MissingDateError: Error {} + throw MissingDateError() + } + return date + } +} diff --git a/apps/macos/Tests/OpenClawIPCTests/ExecHostRequestEvaluatorTests.swift b/apps/macos/Tests/OpenClawIPCTests/ExecHostRequestEvaluatorTests.swift new file mode 100644 index 00000000000..64ef6a21eda --- /dev/null +++ b/apps/macos/Tests/OpenClawIPCTests/ExecHostRequestEvaluatorTests.swift @@ -0,0 +1,76 @@ +import Foundation +import Testing +@testable import OpenClaw + +struct ExecHostRequestEvaluatorTests { + @Test func validateRequestRejectsEmptyCommand() { + let request = ExecHostRequest(command: [], rawCommand: nil, cwd: nil, env: nil, timeoutMs: nil, needsScreenRecording: nil, agentId: nil, sessionKey: nil, approvalDecision: nil) + switch ExecHostRequestEvaluator.validateRequest(request) { + case .success: + Issue.record("expected invalid request") + case .failure(let error): + #expect(error.code == "INVALID_REQUEST") + #expect(error.message == "command required") + } + } + + @Test func evaluateRequiresPromptOnAllowlistMissWithoutDecision() { + let context = Self.makeContext(security: .allowlist, ask: .onMiss, allowlistSatisfied: false, skillAllow: false) + let decision = ExecHostRequestEvaluator.evaluate(context: context, approvalDecision: nil) + switch decision { + case .requiresPrompt: + break + case .allow: + Issue.record("expected prompt requirement") + case .deny(let error): + Issue.record("unexpected deny: \(error.message)") + } + } + + @Test func evaluateAllowsAllowOnceDecisionOnAllowlistMiss() { + let context = Self.makeContext(security: .allowlist, ask: .onMiss, allowlistSatisfied: false, skillAllow: false) + let decision = ExecHostRequestEvaluator.evaluate(context: context, approvalDecision: .allowOnce) + switch decision { + case .allow(let approvedByAsk): + #expect(approvedByAsk) + case .requiresPrompt: + Issue.record("expected allow decision") + case .deny(let error): + Issue.record("unexpected deny: \(error.message)") + } + } + + @Test func evaluateDeniesOnExplicitDenyDecision() { + let context = Self.makeContext(security: .full, ask: .off, allowlistSatisfied: true, skillAllow: false) + let decision = ExecHostRequestEvaluator.evaluate(context: context, approvalDecision: .deny) + switch decision { + case .deny(let error): + #expect(error.reason == "user-denied") + case .requiresPrompt: + Issue.record("expected deny decision") + case .allow: + Issue.record("expected deny decision") + } + } + + private static func makeContext( + security: ExecSecurity, + ask: ExecAsk, + allowlistSatisfied: Bool, + skillAllow: Bool) -> ExecApprovalEvaluation + { + ExecApprovalEvaluation( + command: ["/usr/bin/echo", "hi"], + displayCommand: "/usr/bin/echo hi", + agentId: nil, + security: security, + ask: ask, + env: [:], + resolution: nil, + allowlistResolutions: [], + allowlistMatches: [], + allowlistSatisfied: allowlistSatisfied, + allowlistMatch: nil, + skillAllow: skillAllow) + } +} diff --git a/apps/macos/Tests/OpenClawIPCTests/ExecSystemRunCommandValidatorTests.swift b/apps/macos/Tests/OpenClawIPCTests/ExecSystemRunCommandValidatorTests.swift new file mode 100644 index 00000000000..ed3773a44ed --- /dev/null +++ b/apps/macos/Tests/OpenClawIPCTests/ExecSystemRunCommandValidatorTests.swift @@ -0,0 +1,76 @@ +import Foundation +import Testing +@testable import OpenClaw + +private struct SystemRunCommandContractFixture: Decodable { + let cases: [SystemRunCommandContractCase] +} + +private struct SystemRunCommandContractCase: Decodable { + let name: String + let command: [String] + let rawCommand: String? + let expected: SystemRunCommandContractExpected +} + +private struct SystemRunCommandContractExpected: Decodable { + let valid: Bool + let displayCommand: String? + let errorContains: String? +} + +struct ExecSystemRunCommandValidatorTests { + @Test func matchesSharedSystemRunCommandContractFixture() throws { + for entry in try Self.loadContractCases() { + let result = ExecSystemRunCommandValidator.resolve(command: entry.command, rawCommand: entry.rawCommand) + + if !entry.expected.valid { + switch result { + case .ok(let resolved): + Issue.record("\(entry.name): expected invalid result, got displayCommand=\(resolved.displayCommand)") + case .invalid(let message): + if let expected = entry.expected.errorContains { + #expect( + message.contains(expected), + "\(entry.name): expected error containing \(expected), got \(message)") + } + } + continue + } + + switch result { + case .ok(let resolved): + #expect( + resolved.displayCommand == entry.expected.displayCommand, + "\(entry.name): unexpected display command") + case .invalid(let message): + Issue.record("\(entry.name): unexpected invalid result: \(message)") + } + } + } + + private static func loadContractCases() throws -> [SystemRunCommandContractCase] { + let fixtureURL = try self.findContractFixtureURL() + let data = try Data(contentsOf: fixtureURL) + let decoded = try JSONDecoder().decode(SystemRunCommandContractFixture.self, from: data) + return decoded.cases + } + + private static func findContractFixtureURL() throws -> URL { + var cursor = URL(fileURLWithPath: #filePath).deletingLastPathComponent() + for _ in 0..<8 { + let candidate = cursor + .appendingPathComponent("test") + .appendingPathComponent("fixtures") + .appendingPathComponent("system-run-command-contract.json") + if FileManager.default.fileExists(atPath: candidate.path) { + return candidate + } + cursor.deleteLastPathComponent() + } + throw NSError( + domain: "ExecSystemRunCommandValidatorTests", + code: 1, + userInfo: [NSLocalizedDescriptionKey: "missing shared system-run command contract fixture"]) + } +} diff --git a/apps/macos/Tests/OpenClawIPCTests/GatewayChannelConfigureTests.swift b/apps/macos/Tests/OpenClawIPCTests/GatewayChannelConfigureTests.swift index 7200af03cdd..ec2caf6057c 100644 --- a/apps/macos/Tests/OpenClawIPCTests/GatewayChannelConfigureTests.swift +++ b/apps/macos/Tests/OpenClawIPCTests/GatewayChannelConfigureTests.swift @@ -45,12 +45,7 @@ import Testing // First send is the connect handshake request. Subsequent sends are request frames. if currentSendCount == 0 { - guard case let .data(data) = message else { return } - if let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any], - (obj["type"] as? String) == "req", - (obj["method"] as? String) == "connect", - let id = obj["id"] as? String - { + if let id = GatewayWebSocketTestSupport.connectRequestID(from: message) { self.connectRequestID.withLock { $0 = id } } return @@ -65,7 +60,7 @@ import Testing return } - let response = Self.responseData(id: id) + let response = GatewayWebSocketTestSupport.okResponseData(id: id) let handler = self.pendingReceiveHandler.withLock { $0 } handler?(Result.success(.data(response))) } @@ -75,7 +70,7 @@ import Testing try await Task.sleep(nanoseconds: UInt64(self.helloDelayMs) * 1_000_000) } let id = self.connectRequestID.withLock { $0 } ?? "connect" - return .data(Self.connectOkData(id: id)) + return .data(GatewayWebSocketTestSupport.connectOkData(id: id)) } func receive( @@ -89,41 +84,6 @@ import Testing handler?(Result.success(.data(data))) } - private static func connectOkData(id: String) -> Data { - let json = """ - { - "type": "res", - "id": "\(id)", - "ok": true, - "payload": { - "type": "hello-ok", - "protocol": 2, - "server": { "version": "test", "connId": "test" }, - "features": { "methods": [], "events": [] }, - "snapshot": { - "presence": [ { "ts": 1 } ], - "health": {}, - "stateVersion": { "presence": 0, "health": 0 }, - "uptimeMs": 0 - }, - "policy": { "maxPayload": 1, "maxBufferedBytes": 1, "tickIntervalMs": 30000 } - } - } - """ - return Data(json.utf8) - } - - private static func responseData(id: String) -> Data { - let json = """ - { - "type": "res", - "id": "\(id)", - "ok": true, - "payload": { "ok": true } - } - """ - return Data(json.utf8) - } } private final class FakeWebSocketSession: WebSocketSessioning, @unchecked Sendable { diff --git a/apps/macos/Tests/OpenClawIPCTests/GatewayChannelConnectTests.swift b/apps/macos/Tests/OpenClawIPCTests/GatewayChannelConnectTests.swift index bda06e9cf56..afe9dea9e2c 100644 --- a/apps/macos/Tests/OpenClawIPCTests/GatewayChannelConnectTests.swift +++ b/apps/macos/Tests/OpenClawIPCTests/GatewayChannelConnectTests.swift @@ -38,17 +38,7 @@ import Testing } func send(_ message: URLSessionWebSocketTask.Message) async throws { - let data: Data? = switch message { - case let .data(d): d - case let .string(s): s.data(using: .utf8) - @unknown default: nil - } - guard let data else { return } - if let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any], - obj["type"] as? String == "req", - obj["method"] as? String == "connect", - let id = obj["id"] as? String - { + if let id = GatewayWebSocketTestSupport.connectRequestID(from: message) { self.connectRequestID.withLock { $0 = id } } } @@ -60,7 +50,7 @@ import Testing case let .helloOk(ms): delayMs = ms let id = self.connectRequestID.withLock { $0 } ?? "connect" - msg = .data(Self.connectOkData(id: id)) + msg = .data(GatewayWebSocketTestSupport.connectOkData(id: id)) case let .invalid(ms): delayMs = ms msg = .string("not json") @@ -77,29 +67,6 @@ import Testing self.pendingReceiveHandler.withLock { $0 = completionHandler } } - private static func connectOkData(id: String) -> Data { - let json = """ - { - "type": "res", - "id": "\(id)", - "ok": true, - "payload": { - "type": "hello-ok", - "protocol": 2, - "server": { "version": "test", "connId": "test" }, - "features": { "methods": [], "events": [] }, - "snapshot": { - "presence": [ { "ts": 1 } ], - "health": {}, - "stateVersion": { "presence": 0, "health": 0 }, - "uptimeMs": 0 - }, - "policy": { "maxPayload": 1, "maxBufferedBytes": 1, "tickIntervalMs": 30000 } - } - } - """ - return Data(json.utf8) - } } private final class FakeWebSocketSession: WebSocketSessioning, @unchecked Sendable { diff --git a/apps/macos/Tests/OpenClawIPCTests/GatewayChannelRequestTests.swift b/apps/macos/Tests/OpenClawIPCTests/GatewayChannelRequestTests.swift index 94edb6ebf77..4c788a959f5 100644 --- a/apps/macos/Tests/OpenClawIPCTests/GatewayChannelRequestTests.swift +++ b/apps/macos/Tests/OpenClawIPCTests/GatewayChannelRequestTests.swift @@ -42,17 +42,7 @@ import Testing // First send is the connect handshake. Second send is the request frame. if currentSendCount == 0 { - let data: Data? = switch message { - case let .data(d): d - case let .string(s): s.data(using: .utf8) - @unknown default: nil - } - guard let data else { return } - if let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any], - obj["type"] as? String == "req", - obj["method"] as? String == "connect", - let id = obj["id"] as? String - { + if let id = GatewayWebSocketTestSupport.connectRequestID(from: message) { self.connectRequestID.withLock { $0 = id } } } @@ -64,7 +54,7 @@ import Testing func receive() async throws -> URLSessionWebSocketTask.Message { let id = self.connectRequestID.withLock { $0 } ?? "connect" - return .data(Self.connectOkData(id: id)) + return .data(GatewayWebSocketTestSupport.connectOkData(id: id)) } func receive( @@ -73,29 +63,6 @@ import Testing self.pendingReceiveHandler.withLock { $0 = completionHandler } } - private static func connectOkData(id: String) -> Data { - let json = """ - { - "type": "res", - "id": "\(id)", - "ok": true, - "payload": { - "type": "hello-ok", - "protocol": 2, - "server": { "version": "test", "connId": "test" }, - "features": { "methods": [], "events": [] }, - "snapshot": { - "presence": [ { "ts": 1 } ], - "health": {}, - "stateVersion": { "presence": 0, "health": 0 }, - "uptimeMs": 0 - }, - "policy": { "maxPayload": 1, "maxBufferedBytes": 1, "tickIntervalMs": 30000 } - } - } - """ - return Data(json.utf8) - } } private final class FakeWebSocketSession: WebSocketSessioning, @unchecked Sendable { diff --git a/apps/macos/Tests/OpenClawIPCTests/GatewayChannelShutdownTests.swift b/apps/macos/Tests/OpenClawIPCTests/GatewayChannelShutdownTests.swift index eea7774adf2..5f995cd394a 100644 --- a/apps/macos/Tests/OpenClawIPCTests/GatewayChannelShutdownTests.swift +++ b/apps/macos/Tests/OpenClawIPCTests/GatewayChannelShutdownTests.swift @@ -32,24 +32,14 @@ import Testing } func send(_ message: URLSessionWebSocketTask.Message) async throws { - let data: Data? = switch message { - case let .data(d): d - case let .string(s): s.data(using: .utf8) - @unknown default: nil - } - guard let data else { return } - if let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any], - obj["type"] as? String == "req", - obj["method"] as? String == "connect", - let id = obj["id"] as? String - { + if let id = GatewayWebSocketTestSupport.connectRequestID(from: message) { self.connectRequestID.withLock { $0 = id } } } func receive() async throws -> URLSessionWebSocketTask.Message { let id = self.connectRequestID.withLock { $0 } ?? "connect" - return .data(Self.connectOkData(id: id)) + return .data(GatewayWebSocketTestSupport.connectOkData(id: id)) } func receive( @@ -63,29 +53,6 @@ import Testing handler?(Result.failure(URLError(.networkConnectionLost))) } - private static func connectOkData(id: String) -> Data { - let json = """ - { - "type": "res", - "id": "\(id)", - "ok": true, - "payload": { - "type": "hello-ok", - "protocol": 2, - "server": { "version": "test", "connId": "test" }, - "features": { "methods": [], "events": [] }, - "snapshot": { - "presence": [ { "ts": 1 } ], - "health": {}, - "stateVersion": { "presence": 0, "health": 0 }, - "uptimeMs": 0 - }, - "policy": { "maxPayload": 1, "maxBufferedBytes": 1, "tickIntervalMs": 30000 } - } - } - """ - return Data(json.utf8) - } } private final class FakeWebSocketSession: WebSocketSessioning, @unchecked Sendable { diff --git a/apps/macos/Tests/OpenClawIPCTests/GatewayDiscoveryHelpersTests.swift b/apps/macos/Tests/OpenClawIPCTests/GatewayDiscoveryHelpersTests.swift new file mode 100644 index 00000000000..17ffec07d46 --- /dev/null +++ b/apps/macos/Tests/OpenClawIPCTests/GatewayDiscoveryHelpersTests.swift @@ -0,0 +1,98 @@ +import Foundation +import OpenClawDiscovery +import Testing +@testable import OpenClaw + +@Suite +struct GatewayDiscoveryHelpersTests { + private func makeGateway( + serviceHost: String?, + servicePort: Int?, + lanHost: String? = "txt-host.local", + tailnetDns: String? = "txt-host.ts.net", + sshPort: Int = 22, + gatewayPort: Int? = 18789) -> GatewayDiscoveryModel.DiscoveredGateway + { + GatewayDiscoveryModel.DiscoveredGateway( + displayName: "Gateway", + serviceHost: serviceHost, + servicePort: servicePort, + lanHost: lanHost, + tailnetDns: tailnetDns, + sshPort: sshPort, + gatewayPort: gatewayPort, + cliPath: "/tmp/openclaw", + stableID: UUID().uuidString, + debugID: UUID().uuidString, + isLocal: false) + } + + @Test func sshTargetUsesResolvedServiceHostOnly() { + let gateway = self.makeGateway( + serviceHost: "resolved.example.ts.net", + servicePort: 18789, + sshPort: 2201) + + guard let target = GatewayDiscoveryHelpers.sshTarget(for: gateway) else { + Issue.record("expected ssh target") + return + } + let parsed = CommandResolver.parseSSHTarget(target) + #expect(parsed?.host == "resolved.example.ts.net") + #expect(parsed?.port == 2201) + } + + @Test func sshTargetAllowsMissingResolvedServicePort() { + let gateway = self.makeGateway( + serviceHost: "resolved.example.ts.net", + servicePort: nil, + sshPort: 2201) + + guard let target = GatewayDiscoveryHelpers.sshTarget(for: gateway) else { + Issue.record("expected ssh target") + return + } + let parsed = CommandResolver.parseSSHTarget(target) + #expect(parsed?.host == "resolved.example.ts.net") + #expect(parsed?.port == 2201) + } + + @Test func sshTargetRejectsTxtOnlyGateways() { + let gateway = self.makeGateway( + serviceHost: nil, + servicePort: nil, + lanHost: "txt-only.local", + tailnetDns: "txt-only.ts.net", + sshPort: 2222) + + #expect(GatewayDiscoveryHelpers.sshTarget(for: gateway) == nil) + } + + @Test func directUrlUsesResolvedServiceEndpointOnly() { + let tlsGateway = self.makeGateway( + serviceHost: "resolved.example.ts.net", + servicePort: 443) + #expect(GatewayDiscoveryHelpers.directUrl(for: tlsGateway) == "wss://resolved.example.ts.net") + + let wsGateway = self.makeGateway( + serviceHost: "resolved.example.ts.net", + servicePort: 18789) + #expect(GatewayDiscoveryHelpers.directUrl(for: wsGateway) == "wss://resolved.example.ts.net:18789") + + let localGateway = self.makeGateway( + serviceHost: "127.0.0.1", + servicePort: 18789) + #expect(GatewayDiscoveryHelpers.directUrl(for: localGateway) == "ws://127.0.0.1:18789") + } + + @Test func directUrlRejectsTxtOnlyFallback() { + let gateway = self.makeGateway( + serviceHost: nil, + servicePort: nil, + lanHost: "txt-only.local", + tailnetDns: "txt-only.ts.net", + gatewayPort: 22222) + + #expect(GatewayDiscoveryHelpers.directUrl(for: gateway) == nil) + } +} diff --git a/apps/macos/Tests/OpenClawIPCTests/GatewayEndpointStoreTests.swift b/apps/macos/Tests/OpenClawIPCTests/GatewayEndpointStoreTests.swift index 44c464c449f..bb969aeaec9 100644 --- a/apps/macos/Tests/OpenClawIPCTests/GatewayEndpointStoreTests.swift +++ b/apps/macos/Tests/OpenClawIPCTests/GatewayEndpointStoreTests.swift @@ -218,9 +218,19 @@ import Testing #expect(url.absoluteString == "https://gateway.example:443/remote-ui/") } - @Test func normalizeGatewayUrlAddsDefaultPortForWs() { - let url = GatewayRemoteConfig.normalizeGatewayUrl("ws://gateway") + @Test func normalizeGatewayUrlAddsDefaultPortForLoopbackWs() { + let url = GatewayRemoteConfig.normalizeGatewayUrl("ws://127.0.0.1") #expect(url?.port == 18789) - #expect(url?.absoluteString == "ws://gateway:18789") + #expect(url?.absoluteString == "ws://127.0.0.1:18789") + } + + @Test func normalizeGatewayUrlRejectsNonLoopbackWs() { + let url = GatewayRemoteConfig.normalizeGatewayUrl("ws://gateway.example:18789") + #expect(url == nil) + } + + @Test func normalizeGatewayUrlRejectsPrefixBypassLoopbackHost() { + let url = GatewayRemoteConfig.normalizeGatewayUrl("ws://127.attacker.example") + #expect(url == nil) } } diff --git a/apps/macos/Tests/OpenClawIPCTests/GatewayProcessManagerTests.swift b/apps/macos/Tests/OpenClawIPCTests/GatewayProcessManagerTests.swift index f8b226ab277..dabb15f8bf1 100644 --- a/apps/macos/Tests/OpenClawIPCTests/GatewayProcessManagerTests.swift +++ b/apps/macos/Tests/OpenClawIPCTests/GatewayProcessManagerTests.swift @@ -39,12 +39,7 @@ struct GatewayProcessManagerTests { } if currentSendCount == 0 { - guard case let .data(data) = message else { return } - if let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any], - (obj["type"] as? String) == "req", - (obj["method"] as? String) == "connect", - let id = obj["id"] as? String - { + if let id = GatewayWebSocketTestSupport.connectRequestID(from: message) { self.connectRequestID.withLock { $0 = id } } return @@ -59,14 +54,14 @@ struct GatewayProcessManagerTests { return } - let response = Self.responseData(id: id) + let response = GatewayWebSocketTestSupport.okResponseData(id: id) let handler = self.pendingReceiveHandler.withLock { $0 } handler?(Result.success(.data(response))) } func receive() async throws -> URLSessionWebSocketTask.Message { let id = self.connectRequestID.withLock { $0 } ?? "connect" - return .data(Self.connectOkData(id: id)) + return .data(GatewayWebSocketTestSupport.connectOkData(id: id)) } func receive( @@ -75,41 +70,6 @@ struct GatewayProcessManagerTests { self.pendingReceiveHandler.withLock { $0 = completionHandler } } - private static func connectOkData(id: String) -> Data { - let json = """ - { - "type": "res", - "id": "\(id)", - "ok": true, - "payload": { - "type": "hello-ok", - "protocol": 2, - "server": { "version": "test", "connId": "test" }, - "features": { "methods": [], "events": [] }, - "snapshot": { - "presence": [ { "ts": 1 } ], - "health": {}, - "stateVersion": { "presence": 0, "health": 0 }, - "uptimeMs": 0 - }, - "policy": { "maxPayload": 1, "maxBufferedBytes": 1, "tickIntervalMs": 30000 } - } - } - """ - return Data(json.utf8) - } - - private static func responseData(id: String) -> Data { - let json = """ - { - "type": "res", - "id": "\(id)", - "ok": true, - "payload": { "ok": true } - } - """ - return Data(json.utf8) - } } private final class FakeWebSocketSession: WebSocketSessioning, @unchecked Sendable { diff --git a/apps/macos/Tests/OpenClawIPCTests/GatewayWebSocketTestSupport.swift b/apps/macos/Tests/OpenClawIPCTests/GatewayWebSocketTestSupport.swift new file mode 100644 index 00000000000..0ba41f2806b --- /dev/null +++ b/apps/macos/Tests/OpenClawIPCTests/GatewayWebSocketTestSupport.swift @@ -0,0 +1,63 @@ +import OpenClawKit +import Foundation + +extension WebSocketTasking { + // Keep unit-test doubles resilient to protocol additions. + func sendPing(pongReceiveHandler: @escaping @Sendable (Error?) -> Void) { + pongReceiveHandler(nil) + } +} + +enum GatewayWebSocketTestSupport { + static func connectRequestID(from message: URLSessionWebSocketTask.Message) -> String? { + let data: Data? = switch message { + case let .data(d): d + case let .string(s): s.data(using: .utf8) + @unknown default: nil + } + guard let data else { return nil } + guard let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any] else { + return nil + } + guard (obj["type"] as? String) == "req", (obj["method"] as? String) == "connect" else { + return nil + } + return obj["id"] as? String + } + + static func connectOkData(id: String) -> Data { + let json = """ + { + "type": "res", + "id": "\(id)", + "ok": true, + "payload": { + "type": "hello-ok", + "protocol": 2, + "server": { "version": "test", "connId": "test" }, + "features": { "methods": [], "events": [] }, + "snapshot": { + "presence": [ { "ts": 1 } ], + "health": {}, + "stateVersion": { "presence": 0, "health": 0 }, + "uptimeMs": 0 + }, + "policy": { "maxPayload": 1, "maxBufferedBytes": 1, "tickIntervalMs": 30000 } + } + } + """ + return Data(json.utf8) + } + + static func okResponseData(id: String) -> Data { + let json = """ + { + "type": "res", + "id": "\(id)", + "ok": true, + "payload": { "ok": true } + } + """ + return Data(json.utf8) + } +} diff --git a/apps/macos/Tests/OpenClawIPCTests/HostEnvSanitizerTests.swift b/apps/macos/Tests/OpenClawIPCTests/HostEnvSanitizerTests.swift new file mode 100644 index 00000000000..7ee15107f40 --- /dev/null +++ b/apps/macos/Tests/OpenClawIPCTests/HostEnvSanitizerTests.swift @@ -0,0 +1,36 @@ +import Testing +@testable import OpenClaw + +struct HostEnvSanitizerTests { + @Test func sanitizeBlocksShellTraceVariables() { + let env = HostEnvSanitizer.sanitize(overrides: [ + "SHELLOPTS": "xtrace", + "PS4": "$(touch /tmp/pwned)", + "OPENCLAW_TEST": "1", + ]) + #expect(env["SHELLOPTS"] == nil) + #expect(env["PS4"] == nil) + #expect(env["OPENCLAW_TEST"] == "1") + } + + @Test func sanitizeShellWrapperAllowsOnlyExplicitOverrideKeys() { + let env = HostEnvSanitizer.sanitize( + overrides: [ + "LANG": "C", + "LC_ALL": "C", + "OPENCLAW_TOKEN": "secret", + "PS4": "$(touch /tmp/pwned)", + ], + shellWrapper: true) + + #expect(env["LANG"] == "C") + #expect(env["LC_ALL"] == "C") + #expect(env["OPENCLAW_TOKEN"] == nil) + #expect(env["PS4"] == nil) + } + + @Test func sanitizeNonShellWrapperKeepsRegularOverrides() { + let env = HostEnvSanitizer.sanitize(overrides: ["OPENCLAW_TOKEN": "secret"]) + #expect(env["OPENCLAW_TOKEN"] == "secret") + } +} diff --git a/apps/macos/Tests/OpenClawIPCTests/MacGatewayChatTransportMappingTests.swift b/apps/macos/Tests/OpenClawIPCTests/MacGatewayChatTransportMappingTests.swift index 661382dda69..2d26b7c0538 100644 --- a/apps/macos/Tests/OpenClawIPCTests/MacGatewayChatTransportMappingTests.swift +++ b/apps/macos/Tests/OpenClawIPCTests/MacGatewayChatTransportMappingTests.swift @@ -13,7 +13,8 @@ import Testing configpath: nil, statedir: nil, sessiondefaults: nil, - authmode: nil) + authmode: nil, + updateavailable: nil) let hello = HelloOk( type: "hello", diff --git a/apps/macos/Tests/OpenClawIPCTests/MenuSessionsInjectorTests.swift b/apps/macos/Tests/OpenClawIPCTests/MenuSessionsInjectorTests.swift index 8395ed145ce..ff63673b9e0 100644 --- a/apps/macos/Tests/OpenClawIPCTests/MenuSessionsInjectorTests.swift +++ b/apps/macos/Tests/OpenClawIPCTests/MenuSessionsInjectorTests.swift @@ -93,4 +93,45 @@ struct MenuSessionsInjectorTests { #expect(menu.items.contains { $0.tag == 9_415_557 }) #expect(menu.items.contains { $0.tag == 9_415_557 && $0.isSeparatorItem }) } + + @Test func costUsageSubmenuDoesNotUseInjectorDelegate() { + let injector = MenuSessionsInjector() + injector.setTestingControlChannelConnected(true) + + let summary = GatewayCostUsageSummary( + updatedAt: Date().timeIntervalSince1970 * 1000, + days: 1, + daily: [ + GatewayCostUsageDay( + date: "2026-02-24", + input: 10, + output: 20, + cacheRead: 0, + cacheWrite: 0, + totalTokens: 30, + totalCost: 0.12, + missingCostEntries: 0), + ], + totals: GatewayCostUsageTotals( + input: 10, + output: 20, + cacheRead: 0, + cacheWrite: 0, + totalTokens: 30, + totalCost: 0.12, + missingCostEntries: 0)) + injector.setTestingCostUsageSummary(summary, errorText: nil) + + let menu = NSMenu() + menu.addItem(NSMenuItem(title: "Header", action: nil, keyEquivalent: "")) + menu.addItem(.separator()) + menu.addItem(NSMenuItem(title: "Send Heartbeats", action: nil, keyEquivalent: "")) + + injector.injectForTesting(into: menu) + + let usageCostItem = menu.items.first { $0.title == "Usage cost (30 days)" } + #expect(usageCostItem != nil) + #expect(usageCostItem?.submenu != nil) + #expect(usageCostItem?.submenu?.delegate == nil) + } } diff --git a/apps/macos/Tests/OpenClawIPCTests/OnboardingViewSmokeTests.swift b/apps/macos/Tests/OpenClawIPCTests/OnboardingViewSmokeTests.swift index 57912eb412d..b824b2b0835 100644 --- a/apps/macos/Tests/OpenClawIPCTests/OnboardingViewSmokeTests.swift +++ b/apps/macos/Tests/OpenClawIPCTests/OnboardingViewSmokeTests.swift @@ -1,3 +1,4 @@ +import Foundation import OpenClawDiscovery import SwiftUI import Testing @@ -25,4 +26,36 @@ struct OnboardingViewSmokeTests { let order = OnboardingView.pageOrder(for: .local, showOnboardingChat: false) #expect(!order.contains(8)) } + + @Test func selectRemoteGatewayClearsStaleSshTargetWhenEndpointUnresolved() async { + let override = FileManager().temporaryDirectory + .appendingPathComponent("openclaw-config-\(UUID().uuidString)") + .appendingPathComponent("openclaw.json") + .path + + await TestIsolation.withEnvValues(["OPENCLAW_CONFIG_PATH": override]) { + let state = AppState(preview: true) + state.remoteTransport = .ssh + state.remoteTarget = "user@old-host:2222" + let view = OnboardingView( + state: state, + permissionMonitor: PermissionMonitor.shared, + discoveryModel: GatewayDiscoveryModel(localDisplayName: InstanceIdentity.displayName)) + let gateway = GatewayDiscoveryModel.DiscoveredGateway( + displayName: "Unresolved", + serviceHost: nil, + servicePort: nil, + lanHost: "txt-host.local", + tailnetDns: "txt-host.ts.net", + sshPort: 22, + gatewayPort: 18789, + cliPath: "/tmp/openclaw", + stableID: UUID().uuidString, + debugID: UUID().uuidString, + isLocal: false) + + view.selectRemoteGateway(gateway) + #expect(state.remoteTarget.isEmpty) + } + } } diff --git a/apps/macos/Tests/OpenClawIPCTests/OpenClawConfigFileTests.swift b/apps/macos/Tests/OpenClawIPCTests/OpenClawConfigFileTests.swift index 98e4e8046d3..2cd9d6432e2 100644 --- a/apps/macos/Tests/OpenClawIPCTests/OpenClawConfigFileTests.swift +++ b/apps/macos/Tests/OpenClawIPCTests/OpenClawConfigFileTests.swift @@ -62,6 +62,31 @@ struct OpenClawConfigFileTests { } } + @MainActor + @Test + func clearRemoteGatewayUrlRemovesOnlyUrlField() async { + let override = FileManager().temporaryDirectory + .appendingPathComponent("openclaw-config-\(UUID().uuidString)") + .appendingPathComponent("openclaw.json") + .path + + await TestIsolation.withEnvValues(["OPENCLAW_CONFIG_PATH": override]) { + OpenClawConfigFile.saveDict([ + "gateway": [ + "remote": [ + "url": "wss://old-host:111", + "token": "tok", + ], + ], + ]) + OpenClawConfigFile.clearRemoteGatewayUrl() + let root = OpenClawConfigFile.loadDict() + let remote = ((root["gateway"] as? [String: Any])?["remote"] as? [String: Any]) ?? [:] + #expect((remote["url"] as? String) == nil) + #expect((remote["token"] as? String) == "tok") + } + } + @Test func stateDirOverrideSetsConfigPath() async { let dir = FileManager().temporaryDirectory diff --git a/apps/macos/Tests/OpenClawIPCTests/OpenClawOAuthStoreTests.swift b/apps/macos/Tests/OpenClawIPCTests/OpenClawOAuthStoreTests.swift deleted file mode 100644 index b34e9c3008a..00000000000 --- a/apps/macos/Tests/OpenClawIPCTests/OpenClawOAuthStoreTests.swift +++ /dev/null @@ -1,97 +0,0 @@ -import Foundation -import Testing -@testable import OpenClaw - -@Suite -struct OpenClawOAuthStoreTests { - @Test - func returnsMissingWhenFileAbsent() { - let url = FileManager().temporaryDirectory - .appendingPathComponent("openclaw-oauth-\(UUID().uuidString)") - .appendingPathComponent("oauth.json") - #expect(OpenClawOAuthStore.anthropicOAuthStatus(at: url) == .missingFile) - } - - @Test - func usesEnvOverrideForOpenClawOAuthDir() throws { - let key = "OPENCLAW_OAUTH_DIR" - let previous = ProcessInfo.processInfo.environment[key] - defer { - if let previous { - setenv(key, previous, 1) - } else { - unsetenv(key) - } - } - - let dir = FileManager().temporaryDirectory - .appendingPathComponent("openclaw-oauth-\(UUID().uuidString)", isDirectory: true) - setenv(key, dir.path, 1) - - #expect(OpenClawOAuthStore.oauthDir().standardizedFileURL == dir.standardizedFileURL) - } - - @Test - func acceptsPiFormatTokens() throws { - let url = try self.writeOAuthFile([ - "anthropic": [ - "type": "oauth", - "refresh": "r1", - "access": "a1", - "expires": 1_234_567_890, - ], - ]) - - #expect(OpenClawOAuthStore.anthropicOAuthStatus(at: url).isConnected) - } - - @Test - func acceptsTokenKeyVariants() throws { - let url = try self.writeOAuthFile([ - "anthropic": [ - "type": "oauth", - "refresh_token": "r1", - "access_token": "a1", - ], - ]) - - #expect(OpenClawOAuthStore.anthropicOAuthStatus(at: url).isConnected) - } - - @Test - func reportsMissingProviderEntry() throws { - let url = try self.writeOAuthFile([ - "other": [ - "type": "oauth", - "refresh": "r1", - "access": "a1", - ], - ]) - - #expect(OpenClawOAuthStore.anthropicOAuthStatus(at: url) == .missingProviderEntry) - } - - @Test - func reportsMissingTokens() throws { - let url = try self.writeOAuthFile([ - "anthropic": [ - "type": "oauth", - "refresh": "", - "access": "a1", - ], - ]) - - #expect(OpenClawOAuthStore.anthropicOAuthStatus(at: url) == .missingTokens) - } - - private func writeOAuthFile(_ json: [String: Any]) throws -> URL { - let dir = FileManager().temporaryDirectory - .appendingPathComponent("openclaw-oauth-\(UUID().uuidString)", isDirectory: true) - try FileManager().createDirectory(at: dir, withIntermediateDirectories: true) - - let url = dir.appendingPathComponent("oauth.json") - let data = try JSONSerialization.data(withJSONObject: json, options: [.prettyPrinted, .sortedKeys]) - try data.write(to: url, options: [.atomic]) - return url - } -} diff --git a/apps/macos/Tests/OpenClawIPCTests/TalkModeConfigParsingTests.swift b/apps/macos/Tests/OpenClawIPCTests/TalkModeConfigParsingTests.swift new file mode 100644 index 00000000000..5ee30af273d --- /dev/null +++ b/apps/macos/Tests/OpenClawIPCTests/TalkModeConfigParsingTests.swift @@ -0,0 +1,36 @@ +import OpenClawProtocol +import Testing + +@testable import OpenClaw + +@Suite struct TalkModeConfigParsingTests { + @Test func prefersNormalizedTalkProviderPayload() { + let talk: [String: AnyCodable] = [ + "provider": AnyCodable("elevenlabs"), + "providers": AnyCodable([ + "elevenlabs": [ + "voiceId": "voice-normalized", + ], + ]), + "voiceId": AnyCodable("voice-legacy"), + ] + + let selection = TalkModeRuntime.selectTalkProviderConfig(talk) + #expect(selection?.provider == "elevenlabs") + #expect(selection?.normalizedPayload == true) + #expect(selection?.config["voiceId"]?.stringValue == "voice-normalized") + } + + @Test func fallsBackToLegacyTalkFieldsWhenNormalizedPayloadMissing() { + let talk: [String: AnyCodable] = [ + "voiceId": AnyCodable("voice-legacy"), + "apiKey": AnyCodable("legacy-key"), + ] + + let selection = TalkModeRuntime.selectTalkProviderConfig(talk) + #expect(selection?.provider == "elevenlabs") + #expect(selection?.normalizedPayload == false) + #expect(selection?.config["voiceId"]?.stringValue == "voice-legacy") + #expect(selection?.config["apiKey"]?.stringValue == "legacy-key") + } +} diff --git a/apps/macos/Tests/OpenClawIPCTests/VoiceWakeForwarderTests.swift b/apps/macos/Tests/OpenClawIPCTests/VoiceWakeForwarderTests.swift index 46971ac314c..6640d526a74 100644 --- a/apps/macos/Tests/OpenClawIPCTests/VoiceWakeForwarderTests.swift +++ b/apps/macos/Tests/OpenClawIPCTests/VoiceWakeForwarderTests.swift @@ -17,6 +17,7 @@ import Testing #expect(opts.thinking == "low") #expect(opts.deliver == true) #expect(opts.to == nil) - #expect(opts.channel == .last) + #expect(opts.channel == .webchat) + #expect(opts.channel.shouldDeliver(opts.deliver) == false) } } diff --git a/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatComposer.swift b/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatComposer.swift index 95a5ac3e584..62714838177 100644 --- a/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatComposer.swift +++ b/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatComposer.swift @@ -180,10 +180,12 @@ struct OpenClawChatComposer: View { VStack(alignment: .leading, spacing: 8) { self.editorOverlay - Rectangle() - .fill(OpenClawChatTheme.divider) - .frame(height: 1) - .padding(.horizontal, 2) + if !self.isComposerCompacted { + Rectangle() + .fill(OpenClawChatTheme.divider) + .frame(height: 1) + .padding(.horizontal, 2) + } HStack(alignment: .center, spacing: 8) { if self.showsConnectionPill { @@ -308,7 +310,7 @@ struct OpenClawChatComposer: View { } private var showsToolbar: Bool { - self.style == .standard + self.style == .standard && !self.isComposerCompacted } private var showsAttachments: Bool { @@ -316,15 +318,15 @@ struct OpenClawChatComposer: View { } private var showsConnectionPill: Bool { - self.style == .standard + self.style == .standard && !self.isComposerCompacted } private var composerPadding: CGFloat { - self.style == .onboarding ? 5 : 6 + self.style == .onboarding ? 5 : (self.isComposerCompacted ? 4 : 6) } private var editorPadding: CGFloat { - self.style == .onboarding ? 5 : 6 + self.style == .onboarding ? 5 : (self.isComposerCompacted ? 4 : 6) } private var textMinHeight: CGFloat { @@ -335,6 +337,14 @@ struct OpenClawChatComposer: View { self.style == .onboarding ? 52 : 64 } + private var isComposerCompacted: Bool { + #if os(macOS) + false + #else + self.style == .standard && self.isFocused + #endif + } + #if os(macOS) private func pickFilesMac() { let panel = NSOpenPanel() @@ -476,6 +486,10 @@ private final class ChatComposerNSTextView: NSTextView { override func keyDown(with event: NSEvent) { let isReturn = event.keyCode == 36 if isReturn { + if self.hasMarkedText() { + super.keyDown(with: event) + return + } if event.modifierFlags.contains(.shift) { super.insertNewline(nil) return diff --git a/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatMarkdownPreprocessor.swift b/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatMarkdownPreprocessor.swift index f435eab2dca..0b012586672 100644 --- a/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatMarkdownPreprocessor.swift +++ b/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatMarkdownPreprocessor.swift @@ -1,6 +1,18 @@ import Foundation enum ChatMarkdownPreprocessor { + // Keep in sync with `src/auto-reply/reply/strip-inbound-meta.ts` + // (`INBOUND_META_SENTINELS`), and extend parser expectations in + // `ChatMarkdownPreprocessorTests` when sentinels change. + private static let inboundContextHeaders = [ + "Conversation info (untrusted metadata):", + "Sender (untrusted metadata):", + "Thread starter (untrusted, for context):", + "Replied message (untrusted, for context):", + "Forwarded message context (untrusted metadata):", + "Chat history since last reply (untrusted, for context):", + ] + struct InlineImage: Identifiable { let id = UUID() let label: String @@ -13,17 +25,21 @@ enum ChatMarkdownPreprocessor { } static func preprocess(markdown raw: String) -> Result { + let withoutContextBlocks = self.stripInboundContextBlocks(raw) + let withoutTimestamps = self.stripPrefixedTimestamps(withoutContextBlocks) let pattern = #"!\[([^\]]*)\]\((data:image\/[^;]+;base64,[^)]+)\)"# guard let re = try? NSRegularExpression(pattern: pattern) else { - return Result(cleaned: raw, images: []) + return Result(cleaned: self.normalize(withoutTimestamps), images: []) } - let ns = raw as NSString - let matches = re.matches(in: raw, range: NSRange(location: 0, length: ns.length)) - if matches.isEmpty { return Result(cleaned: raw, images: []) } + let ns = withoutTimestamps as NSString + let matches = re.matches( + in: withoutTimestamps, + range: NSRange(location: 0, length: ns.length)) + if matches.isEmpty { return Result(cleaned: self.normalize(withoutTimestamps), images: []) } var images: [InlineImage] = [] - var cleaned = raw + var cleaned = withoutTimestamps for match in matches.reversed() { guard match.numberOfRanges >= 3 else { continue } @@ -43,9 +59,67 @@ enum ChatMarkdownPreprocessor { cleaned.replaceSubrange(start.. String { + guard self.inboundContextHeaders.contains(where: raw.contains) else { + return raw + } + + let normalized = raw.replacingOccurrences(of: "\r\n", with: "\n") + var outputLines: [String] = [] + var inMetaBlock = false + var inFencedJson = false + + for line in normalized.split(separator: "\n", omittingEmptySubsequences: false) { + let currentLine = String(line) + + if !inMetaBlock && self.inboundContextHeaders.contains(where: currentLine.hasPrefix) { + inMetaBlock = true + inFencedJson = false + continue + } + + if inMetaBlock { + if !inFencedJson && currentLine.trimmingCharacters(in: .whitespacesAndNewlines) == "```json" { + inFencedJson = true + continue + } + + if inFencedJson { + if currentLine.trimmingCharacters(in: .whitespacesAndNewlines) == "```" { + inMetaBlock = false + inFencedJson = false + } + continue + } + + if currentLine.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty { + continue + } + + inMetaBlock = false + } + + outputLines.append(currentLine) + } + + return outputLines + .joined(separator: "\n") + .replacingOccurrences(of: #"^\n+"#, with: "", options: .regularExpression) + } + + private static func stripPrefixedTimestamps(_ raw: String) -> String { + let pattern = #"(?m)^\[[A-Za-z]{3}\s+\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}(?::\d{2})?\s+(?:GMT|UTC)[+-]?\d{0,2}\]\s*"# + return raw.replacingOccurrences(of: pattern, with: "", options: .regularExpression) + } + + private static func normalize(_ raw: String) -> String { + var output = raw + output = output.replacingOccurrences(of: "\r\n", with: "\n") + output = output.replacingOccurrences(of: "\n\n\n", with: "\n\n") + output = output.replacingOccurrences(of: "\n\n\n", with: "\n\n") + return output.trimmingCharacters(in: .whitespacesAndNewlines) } } diff --git a/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatMessageViews.swift b/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatMessageViews.swift index baa790dbf74..22f28517d64 100644 --- a/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatMessageViews.swift +++ b/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatMessageViews.swift @@ -173,7 +173,8 @@ private struct ChatMessageBody: View { ToolResultCard( title: self.toolResultTitle, text: text, - isUser: self.isUser) + isUser: self.isUser, + toolName: self.message.toolName) } } else if self.isUser { ChatMarkdownRenderer( @@ -207,7 +208,8 @@ private struct ChatMessageBody: View { ToolResultCard( title: "\(display.emoji) \(display.title)", text: toolResult.text ?? "", - isUser: self.isUser) + isUser: self.isUser, + toolName: toolResult.name) } } } @@ -402,47 +404,54 @@ private struct ToolResultCard: View { let title: String let text: String let isUser: Bool + let toolName: String? @State private var expanded = false var body: some View { - VStack(alignment: .leading, spacing: 8) { - HStack(spacing: 6) { - Text(self.title) - .font(.footnote.weight(.semibold)) - Spacer(minLength: 0) - } - - Text(self.displayText) - .font(.footnote.monospaced()) - .foregroundStyle(self.isUser ? OpenClawChatTheme.userText : OpenClawChatTheme.assistantText) - .lineLimit(self.expanded ? nil : Self.previewLineLimit) - - if self.shouldShowToggle { - Button(self.expanded ? "Show less" : "Show full output") { - self.expanded.toggle() + if !self.displayContent.isEmpty { + VStack(alignment: .leading, spacing: 8) { + HStack(spacing: 6) { + Text(self.title) + .font(.footnote.weight(.semibold)) + Spacer(minLength: 0) + } + + Text(self.displayText) + .font(.footnote.monospaced()) + .foregroundStyle(self.isUser ? OpenClawChatTheme.userText : OpenClawChatTheme.assistantText) + .lineLimit(self.expanded ? nil : Self.previewLineLimit) + + if self.shouldShowToggle { + Button(self.expanded ? "Show less" : "Show full output") { + self.expanded.toggle() + } + .buttonStyle(.plain) + .font(.caption) + .foregroundStyle(.secondary) } - .buttonStyle(.plain) - .font(.caption) - .foregroundStyle(.secondary) } + .padding(10) + .background( + RoundedRectangle(cornerRadius: 12, style: .continuous) + .fill(OpenClawChatTheme.subtleCard) + .overlay( + RoundedRectangle(cornerRadius: 12, style: .continuous) + .strokeBorder(Color.white.opacity(0.08), lineWidth: 1))) } - .padding(10) - .background( - RoundedRectangle(cornerRadius: 12, style: .continuous) - .fill(OpenClawChatTheme.subtleCard) - .overlay( - RoundedRectangle(cornerRadius: 12, style: .continuous) - .strokeBorder(Color.white.opacity(0.08), lineWidth: 1))) } private static let previewLineLimit = 8 + private var displayContent: String { + ToolResultTextFormatter.format(text: self.text, toolName: self.toolName) + } + private var lines: [Substring] { - self.text.components(separatedBy: .newlines).map { Substring($0) } + self.displayContent.components(separatedBy: .newlines).map { Substring($0) } } private var displayText: String { - guard !self.expanded, self.lines.count > Self.previewLineLimit else { return self.text } + guard !self.expanded, self.lines.count > Self.previewLineLimit else { return self.displayContent } return self.lines.prefix(Self.previewLineLimit).joined(separator: "\n") + "\n…" } @@ -458,12 +467,7 @@ struct ChatTypingIndicatorBubble: View { var body: some View { HStack(spacing: 10) { TypingDots() - if self.style == .standard { - Text("OpenClaw is thinking…") - .font(.subheadline) - .foregroundStyle(.secondary) - Spacer() - } + Spacer(minLength: 0) } .padding(.vertical, self.style == .standard ? 12 : 10) .padding(.horizontal, self.style == .standard ? 12 : 14) diff --git a/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatView.swift b/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatView.swift index 68f9ae2f311..0675ffc2139 100644 --- a/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatView.swift +++ b/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatView.swift @@ -1,4 +1,7 @@ import SwiftUI +#if canImport(UIKit) +import UIKit +#endif @MainActor public struct OpenClawChatView: View { @@ -105,6 +108,9 @@ public struct OpenClawChatView: View { .padding(.top, Layout.messageListPaddingTop) .padding(.horizontal, Layout.messageListPaddingHorizontal) } + #if !os(macOS) + .scrollDismissesKeyboard(.interactively) + #endif // Keep the scroll pinned to the bottom for new messages. .scrollPosition(id: self.$scrollPosition, anchor: .bottom) .onChange(of: self.scrollPosition) { _, position in @@ -123,6 +129,10 @@ public struct OpenClawChatView: View { // Ensure the message list claims vertical space on the first layout pass. .frame(maxHeight: .infinity, alignment: .top) .layoutPriority(1) + .simultaneousGesture( + TapGesture().onEnded { + self.dismissKeyboardIfNeeded() + }) .onChange(of: self.viewModel.isLoading) { _, isLoading in guard !isLoading, !self.hasPerformedInitialScroll else { return } self.scrollPosition = self.scrollerBottomID @@ -406,6 +416,16 @@ public struct OpenClawChatView: View { } return parts.joined(separator: "\n").trimmingCharacters(in: .whitespacesAndNewlines) } + + private func dismissKeyboardIfNeeded() { + #if canImport(UIKit) + UIApplication.shared.sendAction( + #selector(UIResponder.resignFirstResponder), + to: nil, + from: nil, + for: nil) + #endif + } } private struct ChatNoticeCard: View { diff --git a/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatViewModel.swift b/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatViewModel.swift index fc7b399353d..62cb97a0e2f 100644 --- a/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatViewModel.swift +++ b/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ChatViewModel.swift @@ -189,10 +189,43 @@ public final class OpenClawChatViewModel { private static func decodeMessages(_ raw: [AnyCodable]) -> [OpenClawChatMessage] { let decoded = raw.compactMap { item in (try? ChatPayloadDecoding.decode(item, as: OpenClawChatMessage.self)) + .map { Self.stripInboundMetadata(from: $0) } } return Self.dedupeMessages(decoded) } + private static func stripInboundMetadata(from message: OpenClawChatMessage) -> OpenClawChatMessage { + guard message.role.lowercased() == "user" else { + return message + } + + let sanitizedContent = message.content.map { content -> OpenClawChatMessageContent in + guard let text = content.text else { return content } + let cleaned = ChatMarkdownPreprocessor.preprocess(markdown: text).cleaned + return OpenClawChatMessageContent( + type: content.type, + text: cleaned, + thinking: content.thinking, + thinkingSignature: content.thinkingSignature, + mimeType: content.mimeType, + fileName: content.fileName, + content: content.content, + id: content.id, + name: content.name, + arguments: content.arguments) + } + + return OpenClawChatMessage( + id: message.id, + role: message.role, + content: sanitizedContent, + timestamp: message.timestamp, + toolCallId: message.toolCallId, + toolName: message.toolName, + usage: message.usage, + stopReason: message.stopReason) + } + private static func messageIdentityKey(for message: OpenClawChatMessage) -> String? { let role = message.role.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() guard !role.isEmpty else { return nil } @@ -435,8 +468,12 @@ public final class OpenClawChatViewModel { case let .agent(agent): self.handleAgentEvent(agent) case .seqGap: - self.errorText = "Event stream interrupted; try refreshing." + self.errorText = nil self.clearPendingRuns(reason: nil) + Task { + await self.refreshHistoryAfterRun() + await self.pollHealthIfNeeded(force: true) + } } } diff --git a/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ToolResultTextFormatter.swift b/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ToolResultTextFormatter.swift new file mode 100644 index 00000000000..719e82cdf15 --- /dev/null +++ b/apps/shared/OpenClawKit/Sources/OpenClawChatUI/ToolResultTextFormatter.swift @@ -0,0 +1,157 @@ +import Foundation + +enum ToolResultTextFormatter { + static func format(text: String, toolName: String?) -> String { + let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines) + guard !trimmed.isEmpty else { return "" } + + guard self.looksLikeJSON(trimmed), + let data = trimmed.data(using: .utf8), + let json = try? JSONSerialization.jsonObject(with: data) + else { + return trimmed + } + + let normalizedTool = toolName?.trimmingCharacters(in: .whitespacesAndNewlines).lowercased() + return self.renderJSON(json, toolName: normalizedTool) + } + + private static func looksLikeJSON(_ value: String) -> Bool { + guard let first = value.first else { return false } + return first == "{" || first == "[" + } + + private static func renderJSON(_ json: Any, toolName: String?) -> String { + if let dict = json as? [String: Any] { + return self.renderDictionary(dict, toolName: toolName) + } + if let array = json as? [Any] { + if array.isEmpty { return "No items." } + return "\(array.count) item\(array.count == 1 ? "" : "s")." + } + return "" + } + + private static func renderDictionary(_ dict: [String: Any], toolName: String?) -> String { + let status = (dict["status"] as? String)?.trimmingCharacters(in: .whitespacesAndNewlines) + let errorText = self.firstString(in: dict, keys: ["error", "reason"]) + let messageText = self.firstString(in: dict, keys: ["message", "result", "detail"]) + + if status?.lowercased() == "error" || errorText != nil { + if let errorText { + return "Error: \(self.sanitizeError(errorText))" + } + if let messageText { + return "Error: \(self.sanitizeError(messageText))" + } + return "Error" + } + + if toolName == "nodes", let summary = self.renderNodesSummary(dict) { + return summary + } + + if let message = messageText { + return message + } + + if let status, !status.isEmpty { + return "Status: \(status)" + } + + return "" + } + + private static func renderNodesSummary(_ dict: [String: Any]) -> String? { + if let nodes = dict["nodes"] as? [[String: Any]] { + if nodes.isEmpty { return "No nodes found." } + var lines: [String] = [] + lines.append("\(nodes.count) node\(nodes.count == 1 ? "" : "s") found.") + + for node in nodes.prefix(3) { + let label = self.firstString(in: node, keys: ["displayName", "name", "nodeId"]) ?? "Node" + var details: [String] = [] + + if let connected = node["connected"] as? Bool { + details.append(connected ? "connected" : "offline") + } + if let platform = self.firstString(in: node, keys: ["platform"]) { + details.append(platform) + } + if let version = self.firstString(in: node, keys: ["osVersion", "appVersion", "version"]) { + details.append(version) + } + if let pairing = self.pairingDetail(node) { + details.append(pairing) + } + + if details.isEmpty { + lines.append("• \(label)") + } else { + lines.append("• \(label) - \(details.joined(separator: ", "))") + } + } + + let extra = nodes.count - 3 + if extra > 0 { + lines.append("... +\(extra) more") + } + return lines.joined(separator: "\n") + } + + if let pending = dict["pending"] as? [Any], let paired = dict["paired"] as? [Any] { + return "Pairing requests: \(pending.count) pending, \(paired.count) paired." + } + + if let pending = dict["pending"] as? [Any] { + if pending.isEmpty { return "No pending pairing requests." } + return "\(pending.count) pending pairing request\(pending.count == 1 ? "" : "s")." + } + + return nil + } + + private static func pairingDetail(_ node: [String: Any]) -> String? { + if let paired = node["paired"] as? Bool, !paired { + return "pairing required" + } + + for key in ["status", "state", "deviceStatus"] { + if let raw = node[key] as? String, raw.lowercased().contains("pairing required") { + return "pairing required" + } + } + return nil + } + + private static func firstString(in dict: [String: Any], keys: [String]) -> String? { + for key in keys { + if let value = dict[key] as? String { + let trimmed = value.trimmingCharacters(in: .whitespacesAndNewlines) + if !trimmed.isEmpty { + return trimmed + } + } + } + return nil + } + + private static func sanitizeError(_ raw: String) -> String { + var cleaned = raw.trimmingCharacters(in: .whitespacesAndNewlines) + if cleaned.contains("agent="), + cleaned.contains("action="), + let marker = cleaned.range(of: ": ") + { + cleaned = String(cleaned[marker.upperBound...]).trimmingCharacters(in: .whitespacesAndNewlines) + } + + if let firstLine = cleaned.split(separator: "\n").first { + cleaned = String(firstLine).trimmingCharacters(in: .whitespacesAndNewlines) + } + + if cleaned.count > 220 { + cleaned = String(cleaned.prefix(217)) + "..." + } + return cleaned + } +} diff --git a/apps/shared/OpenClawKit/Sources/OpenClawKit/DeepLinks.swift b/apps/shared/OpenClawKit/Sources/OpenClawKit/DeepLinks.swift index 30606ca2671..50714884619 100644 --- a/apps/shared/OpenClawKit/Sources/OpenClawKit/DeepLinks.swift +++ b/apps/shared/OpenClawKit/Sources/OpenClawKit/DeepLinks.swift @@ -1,4 +1,5 @@ import Foundation +import Network public enum DeepLinkRoute: Sendable, Equatable { case agent(AgentDeepLink) @@ -20,6 +21,40 @@ public struct GatewayConnectDeepLink: Codable, Sendable, Equatable { self.password = password } + fileprivate static func isLoopbackHost(_ raw: String) -> Bool { + var host = raw + .trimmingCharacters(in: .whitespacesAndNewlines) + .lowercased() + .trimmingCharacters(in: CharacterSet(charactersIn: "[]")) + if host.hasSuffix(".") { + host.removeLast() + } + if let zoneIndex = host.firstIndex(of: "%") { + host = String(host[.. String + { + let scopeString = scopes.joined(separator: ",") + let authToken = token ?? "" + let normalizedPlatform = normalizeMetadataField(platform) + let normalizedDeviceFamily = normalizeMetadataField(deviceFamily) + return [ + "v3", + deviceId, + clientId, + clientMode, + role, + scopeString, + String(signedAtMs), + authToken, + nonce, + normalizedPlatform, + normalizedDeviceFamily, + ].joined(separator: "|") + } + + static func normalizeMetadataField(_ value: String?) -> String { + guard let value else { return "" } + let trimmed = value.trimmingCharacters(in: .whitespacesAndNewlines) + if trimmed.isEmpty { + return "" + } + // Keep cross-runtime normalization deterministic (TS/Swift/Kotlin): + // lowercase ASCII A-Z only for auth payload metadata fields. + var output = String() + output.reserveCapacity(trimmed.count) + for scalar in trimmed.unicodeScalars { + let codePoint = scalar.value + if codePoint >= 65, codePoint <= 90, let lowered = UnicodeScalar(codePoint + 32) { + output.unicodeScalars.append(lowered) + } else { + output.unicodeScalars.append(scalar) + } + } + return output + } +} diff --git a/apps/shared/OpenClawKit/Sources/OpenClawKit/GatewayChannel.swift b/apps/shared/OpenClawKit/Sources/OpenClawKit/GatewayChannel.swift index 9682a31aa46..e8a53412cd1 100644 --- a/apps/shared/OpenClawKit/Sources/OpenClawKit/GatewayChannel.swift +++ b/apps/shared/OpenClawKit/Sources/OpenClawKit/GatewayChannel.swift @@ -7,6 +7,7 @@ public protocol WebSocketTasking: AnyObject { func resume() func cancel(with closeCode: URLSessionWebSocketTask.CloseCode, reason: Data?) func send(_ message: URLSessionWebSocketTask.Message) async throws + func sendPing(pongReceiveHandler: @escaping @Sendable (Error?) -> Void) func receive() async throws -> URLSessionWebSocketTask.Message func receive(completionHandler: @escaping @Sendable (Result) -> Void) } @@ -40,6 +41,18 @@ public struct WebSocketTaskBox: @unchecked Sendable { { self.task.receive(completionHandler: completionHandler) } + + public func sendPing() async throws { + try await withCheckedThrowingContinuation { (continuation: CheckedContinuation) in + self.task.sendPing { error in + if let error { + continuation.resume(throwing: error) + } else { + continuation.resume(returning: ()) + } + } + } + } } public protocol WebSocketSessioning: AnyObject { @@ -72,9 +85,9 @@ public struct GatewayConnectOptions: Sendable { public var clientId: String public var clientMode: String public var clientDisplayName: String? - // When false, the connection omits the signed device identity payload. - // This is useful for secondary "operator" connections where the shared gateway token - // should authorize without triggering device pairing flows. + // When false, the connection omits the signed device identity payload and cannot use + // device-scoped auth (role/scope upgrades will require pairing). Keep this true for + // role/scoped sessions such as operator UI clients. public var includeDeviceIdentity: Bool public init( @@ -114,6 +127,14 @@ private enum ConnectChallengeError: Error { case timeout } +private let defaultOperatorConnectScopes: [String] = [ + "operator.admin", + "operator.read", + "operator.write", + "operator.approvals", + "operator.pairing", +] + public actor GatewayChannelActor { private let logger = Logger(subsystem: "ai.openclaw", category: "gateway") private var task: WebSocketTaskBox? @@ -133,8 +154,8 @@ public actor GatewayChannelActor { private var lastAuthSource: GatewayAuthSource = .none private let decoder = JSONDecoder() private let encoder = JSONEncoder() - // Remote gateways (tailscale/wan) can take a bit longer to deliver the connect.challenge event, - // and we must include the nonce once the gateway requires v2 signing. + // Remote gateways (tailscale/wan) can take longer to deliver connect.challenge. + // Connect now requires this nonce before we send device-auth. private let connectTimeoutSeconds: Double = 12 private let connectChallengeTimeoutSeconds: Double = 6.0 // Some networks will silently drop idle TCP/TLS flows around ~30s. The gateway tick is server->client, @@ -213,7 +234,7 @@ public actor GatewayChannelActor { private func watchdogLoop() async { // Keep nudging reconnect in case exponential backoff stalls. while self.shouldReconnect { - try? await Task.sleep(nanoseconds: 30 * 1_000_000_000) // 30s cadence + guard await self.sleepUnlessCancelled(nanoseconds: 30 * 1_000_000_000) else { return } // 30s cadence guard self.shouldReconnect else { return } if self.connected { continue } do { @@ -285,13 +306,15 @@ public actor GatewayChannelActor { private func keepaliveLoop() async { while self.shouldReconnect { - try? await Task.sleep(nanoseconds: UInt64(self.keepaliveIntervalSeconds * 1_000_000_000)) + guard await self.sleepUnlessCancelled( + nanoseconds: UInt64(self.keepaliveIntervalSeconds * 1_000_000_000)) + else { return } guard self.shouldReconnect else { return } guard self.connected else { continue } - // Best-effort outbound message to keep intermediate NAT/proxy state alive. - // We intentionally ignore the response. + guard let task = self.task else { continue } + // Best-effort ping keeps NAT/proxy state alive without generating RPC load. do { - try await self.send(method: "health", params: nil) + try await task.sendPing() } catch { // Avoid spamming logs; the reconnect paths will surface meaningful errors. } @@ -303,7 +326,7 @@ public actor GatewayChannelActor { let primaryLocale = Locale.preferredLanguages.first ?? Locale.current.identifier let options = self.connectOptions ?? GatewayConnectOptions( role: "operator", - scopes: ["operator.admin", "operator.approvals", "operator.pairing"], + scopes: defaultOperatorConnectScopes, caps: [], commands: [], permissions: [:], @@ -375,33 +398,27 @@ public actor GatewayChannelActor { } let signedAtMs = Int(Date().timeIntervalSince1970 * 1000) let connectNonce = try await self.waitForConnectChallenge() - let scopesValue = scopes.joined(separator: ",") - var payloadParts = [ - connectNonce == nil ? "v1" : "v2", - identity?.deviceId ?? "", - clientId, - clientMode, - role, - scopesValue, - String(signedAtMs), - authToken ?? "", - ] - if let connectNonce { - payloadParts.append(connectNonce) - } - let payload = payloadParts.joined(separator: "|") if includeDeviceIdentity, let identity { + let payload = GatewayDeviceAuthPayload.buildV3( + deviceId: identity.deviceId, + clientId: clientId, + clientMode: clientMode, + role: role, + scopes: scopes, + signedAtMs: signedAtMs, + token: authToken, + nonce: connectNonce, + platform: platform, + deviceFamily: InstanceIdentity.deviceFamily) if let signature = DeviceIdentityStore.signPayload(payload, identity: identity), let publicKey = DeviceIdentityStore.publicKeyBase64Url(identity) { - var device: [String: ProtoAnyCodable] = [ + let device: [String: ProtoAnyCodable] = [ "id": ProtoAnyCodable(identity.deviceId), "publicKey": ProtoAnyCodable(publicKey), "signature": ProtoAnyCodable(signature), "signedAt": ProtoAnyCodable(signedAtMs), + "nonce": ProtoAnyCodable(connectNonce), ] - if let connectNonce { - device["nonce"] = ProtoAnyCodable(connectNonce) - } params["device"] = ProtoAnyCodable(device) } } @@ -530,33 +547,26 @@ public actor GatewayChannelActor { } } - private func waitForConnectChallenge() async throws -> String? { - guard let task = self.task else { return nil } - do { - return try await AsyncTimeout.withTimeout( - seconds: self.connectChallengeTimeoutSeconds, - onTimeout: { ConnectChallengeError.timeout }, - operation: { [weak self] in - guard let self else { return nil } - while true { - let msg = try await task.receive() - guard let data = self.decodeMessageData(msg) else { continue } - guard let frame = try? self.decoder.decode(GatewayFrame.self, from: data) else { continue } - if case let .event(evt) = frame, evt.event == "connect.challenge" { - if let payload = evt.payload?.value as? [String: ProtoAnyCodable], - let nonce = payload["nonce"]?.value as? String { - return nonce - } - } + private func waitForConnectChallenge() async throws -> String { + guard let task = self.task else { throw ConnectChallengeError.timeout } + return try await AsyncTimeout.withTimeout( + seconds: self.connectChallengeTimeoutSeconds, + onTimeout: { ConnectChallengeError.timeout }, + operation: { [weak self] in + guard let self else { throw ConnectChallengeError.timeout } + while true { + let msg = try await task.receive() + guard let data = self.decodeMessageData(msg) else { continue } + guard let frame = try? self.decoder.decode(GatewayFrame.self, from: data) else { continue } + if case let .event(evt) = frame, evt.event == "connect.challenge", + let payload = evt.payload?.value as? [String: ProtoAnyCodable], + let nonce = payload["nonce"]?.value as? String, + nonce.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty == false + { + return nonce } - }) - } catch { - if error is ConnectChallengeError { - self.logger.warning("gateway connect challenge timed out") - return nil - } - throw error - } + } + }) } private func waitForConnectResponse(reqId: String) async throws -> ResponseFrame { @@ -593,7 +603,7 @@ public actor GatewayChannelActor { private func watchTicks() async { let tolerance = self.tickIntervalMs * 2 while self.connected { - try? await Task.sleep(nanoseconds: UInt64(tolerance * 1_000_000)) + guard await self.sleepUnlessCancelled(nanoseconds: UInt64(tolerance * 1_000_000)) else { return } guard self.connected else { return } if let last = self.lastTick { let delta = Date().timeIntervalSince(last) * 1000 @@ -616,7 +626,7 @@ public actor GatewayChannelActor { guard self.shouldReconnect else { return } let delay = self.backoffMs / 1000 self.backoffMs = min(self.backoffMs * 2, 30000) - try? await Task.sleep(nanoseconds: UInt64(delay * 1_000_000_000)) + guard await self.sleepUnlessCancelled(nanoseconds: UInt64(delay * 1_000_000_000)) else { return } guard self.shouldReconnect else { return } do { try await self.connect() @@ -627,6 +637,15 @@ public actor GatewayChannelActor { } } + private nonisolated func sleepUnlessCancelled(nanoseconds: UInt64) async -> Bool { + do { + try await Task.sleep(nanoseconds: nanoseconds) + } catch { + return false + } + return !Task.isCancelled + } + public func request( method: String, params: [String: AnyCodable]?, diff --git a/apps/shared/OpenClawKit/Sources/OpenClawKit/GatewayNodeSession.swift b/apps/shared/OpenClawKit/Sources/OpenClawKit/GatewayNodeSession.swift index d0303f7e997..7dd2fe1eee1 100644 --- a/apps/shared/OpenClawKit/Sources/OpenClawKit/GatewayNodeSession.swift +++ b/apps/shared/OpenClawKit/Sources/OpenClawKit/GatewayNodeSession.swift @@ -26,6 +26,7 @@ public actor GatewayNodeSession { private var onConnected: (@Sendable () async -> Void)? private var onDisconnected: (@Sendable (String) async -> Void)? private var onInvoke: (@Sendable (BridgeInvokeRequest) async -> BridgeInvokeResponse)? + private var hasEverConnected = false private var hasNotifiedConnected = false private var snapshotReceived = false private var snapshotWaiters: [CheckedContinuation] = [] @@ -214,6 +215,7 @@ public actor GatewayNodeSession { self.activeToken = nil self.activePassword = nil self.activeConnectOptionsKey = nil + self.hasEverConnected = false self.resetConnectionState() } @@ -274,6 +276,11 @@ public actor GatewayNodeSession { case let .snapshot(ok): let raw = ok.canvashosturl?.trimmingCharacters(in: .whitespacesAndNewlines) self.canvasHostUrl = (raw?.isEmpty == false) ? raw : nil + if self.hasEverConnected { + self.broadcastServerEvent( + EventFrame(type: "event", event: "seqGap", payload: nil, seq: nil, stateversion: nil)) + } + self.hasEverConnected = true self.markSnapshotReceived() await self.notifyConnectedIfNeeded() case let .event(evt): diff --git a/apps/shared/OpenClawKit/Sources/OpenClawKit/WatchCommands.swift b/apps/shared/OpenClawKit/Sources/OpenClawKit/WatchCommands.swift index 814efe68a88..0bd6990710c 100644 --- a/apps/shared/OpenClawKit/Sources/OpenClawKit/WatchCommands.swift +++ b/apps/shared/OpenClawKit/Sources/OpenClawKit/WatchCommands.swift @@ -5,6 +5,24 @@ public enum OpenClawWatchCommand: String, Codable, Sendable { case notify = "watch.notify" } +public enum OpenClawWatchRisk: String, Codable, Sendable, Equatable { + case low + case medium + case high +} + +public struct OpenClawWatchAction: Codable, Sendable, Equatable { + public var id: String + public var label: String + public var style: String? + + public init(id: String, label: String, style: String? = nil) { + self.id = id + self.label = label + self.style = style + } +} + public struct OpenClawWatchStatusPayload: Codable, Sendable, Equatable { public var supported: Bool public var paired: Bool @@ -31,11 +49,36 @@ public struct OpenClawWatchNotifyParams: Codable, Sendable, Equatable { public var title: String public var body: String public var priority: OpenClawNotificationPriority? + public var promptId: String? + public var sessionKey: String? + public var kind: String? + public var details: String? + public var expiresAtMs: Int? + public var risk: OpenClawWatchRisk? + public var actions: [OpenClawWatchAction]? - public init(title: String, body: String, priority: OpenClawNotificationPriority? = nil) { + public init( + title: String, + body: String, + priority: OpenClawNotificationPriority? = nil, + promptId: String? = nil, + sessionKey: String? = nil, + kind: String? = nil, + details: String? = nil, + expiresAtMs: Int? = nil, + risk: OpenClawWatchRisk? = nil, + actions: [OpenClawWatchAction]? = nil) + { self.title = title self.body = body self.priority = priority + self.promptId = promptId + self.sessionKey = sessionKey + self.kind = kind + self.details = details + self.expiresAtMs = expiresAtMs + self.risk = risk + self.actions = actions } } diff --git a/apps/shared/OpenClawKit/Sources/OpenClawProtocol/GatewayModels.swift b/apps/shared/OpenClawKit/Sources/OpenClawProtocol/GatewayModels.swift index ced5df13a8f..a7aaa7d3914 100644 --- a/apps/shared/OpenClawKit/Sources/OpenClawProtocol/GatewayModels.swift +++ b/apps/shared/OpenClawKit/Sources/OpenClawProtocol/GatewayModels.swift @@ -40,8 +40,8 @@ public struct ConnectParams: Codable, Sendable { device: [String: AnyCodable]?, auth: [String: AnyCodable]?, locale: String?, - useragent: String? - ) { + useragent: String?) + { self.minprotocol = minprotocol self.maxprotocol = maxprotocol self.client = client @@ -56,6 +56,7 @@ public struct ConnectParams: Codable, Sendable { self.locale = locale self.useragent = useragent } + private enum CodingKeys: String, CodingKey { case minprotocol = "minProtocol" case maxprotocol = "maxProtocol" @@ -91,8 +92,8 @@ public struct HelloOk: Codable, Sendable { snapshot: Snapshot, canvashosturl: String?, auth: [String: AnyCodable]?, - policy: [String: AnyCodable] - ) { + policy: [String: AnyCodable]) + { self.type = type self._protocol = _protocol self.server = server @@ -102,6 +103,7 @@ public struct HelloOk: Codable, Sendable { self.auth = auth self.policy = policy } + private enum CodingKeys: String, CodingKey { case type case _protocol = "protocol" @@ -124,13 +126,14 @@ public struct RequestFrame: Codable, Sendable { type: String, id: String, method: String, - params: AnyCodable? - ) { + params: AnyCodable?) + { self.type = type self.id = id self.method = method self.params = params } + private enum CodingKeys: String, CodingKey { case type case id @@ -151,14 +154,15 @@ public struct ResponseFrame: Codable, Sendable { id: String, ok: Bool, payload: AnyCodable?, - error: [String: AnyCodable]? - ) { + error: [String: AnyCodable]?) + { self.type = type self.id = id self.ok = ok self.payload = payload self.error = error } + private enum CodingKeys: String, CodingKey { case type case id @@ -180,14 +184,15 @@ public struct EventFrame: Codable, Sendable { event: String, payload: AnyCodable?, seq: Int?, - stateversion: [String: AnyCodable]? - ) { + stateversion: [String: AnyCodable]?) + { self.type = type self.event = event self.payload = payload self.seq = seq self.stateversion = stateversion } + private enum CodingKeys: String, CodingKey { case type case event @@ -231,8 +236,8 @@ public struct PresenceEntry: Codable, Sendable { deviceid: String?, roles: [String]?, scopes: [String]?, - instanceid: String? - ) { + instanceid: String?) + { self.host = host self.ip = ip self.version = version @@ -250,6 +255,7 @@ public struct PresenceEntry: Codable, Sendable { self.scopes = scopes self.instanceid = instanceid } + private enum CodingKeys: String, CodingKey { case host case ip @@ -276,11 +282,12 @@ public struct StateVersion: Codable, Sendable { public init( presence: Int, - health: Int - ) { + health: Int) + { self.presence = presence self.health = health } + private enum CodingKeys: String, CodingKey { case presence case health @@ -296,6 +303,7 @@ public struct Snapshot: Codable, Sendable { public let statedir: String? public let sessiondefaults: [String: AnyCodable]? public let authmode: AnyCodable? + public let updateavailable: [String: AnyCodable]? public init( presence: [PresenceEntry], @@ -305,8 +313,9 @@ public struct Snapshot: Codable, Sendable { configpath: String?, statedir: String?, sessiondefaults: [String: AnyCodable]?, - authmode: AnyCodable? - ) { + authmode: AnyCodable?, + updateavailable: [String: AnyCodable]?) + { self.presence = presence self.health = health self.stateversion = stateversion @@ -315,7 +324,9 @@ public struct Snapshot: Codable, Sendable { self.statedir = statedir self.sessiondefaults = sessiondefaults self.authmode = authmode + self.updateavailable = updateavailable } + private enum CodingKeys: String, CodingKey { case presence case health @@ -325,6 +336,7 @@ public struct Snapshot: Codable, Sendable { case statedir = "stateDir" case sessiondefaults = "sessionDefaults" case authmode = "authMode" + case updateavailable = "updateAvailable" } } @@ -340,14 +352,15 @@ public struct ErrorShape: Codable, Sendable { message: String, details: AnyCodable?, retryable: Bool?, - retryafterms: Int? - ) { + retryafterms: Int?) + { self.code = code self.message = message self.details = details self.retryable = retryable self.retryafterms = retryafterms } + private enum CodingKeys: String, CodingKey { case code case message @@ -369,14 +382,15 @@ public struct AgentEvent: Codable, Sendable { seq: Int, stream: String, ts: Int, - data: [String: AnyCodable] - ) { + data: [String: AnyCodable]) + { self.runid = runid self.seq = seq self.stream = stream self.ts = ts self.data = data } + private enum CodingKeys: String, CodingKey { case runid = "runId" case seq @@ -394,6 +408,7 @@ public struct SendParams: Codable, Sendable { public let gifplayback: Bool? public let channel: String? public let accountid: String? + public let agentid: String? public let threadid: String? public let sessionkey: String? public let idempotencykey: String @@ -406,10 +421,11 @@ public struct SendParams: Codable, Sendable { gifplayback: Bool?, channel: String?, accountid: String?, + agentid: String?, threadid: String?, sessionkey: String?, - idempotencykey: String - ) { + idempotencykey: String) + { self.to = to self.message = message self.mediaurl = mediaurl @@ -417,10 +433,12 @@ public struct SendParams: Codable, Sendable { self.gifplayback = gifplayback self.channel = channel self.accountid = accountid + self.agentid = agentid self.threadid = threadid self.sessionkey = sessionkey self.idempotencykey = idempotencykey } + private enum CodingKeys: String, CodingKey { case to case message @@ -429,6 +447,7 @@ public struct SendParams: Codable, Sendable { case gifplayback = "gifPlayback" case channel case accountid = "accountId" + case agentid = "agentId" case threadid = "threadId" case sessionkey = "sessionKey" case idempotencykey = "idempotencyKey" @@ -461,8 +480,8 @@ public struct PollParams: Codable, Sendable { threadid: String?, channel: String?, accountid: String?, - idempotencykey: String - ) { + idempotencykey: String) + { self.to = to self.question = question self.options = options @@ -476,6 +495,7 @@ public struct PollParams: Codable, Sendable { self.accountid = accountid self.idempotencykey = idempotencykey } + private enum CodingKeys: String, CodingKey { case to case question @@ -511,6 +531,7 @@ public struct AgentParams: Codable, Sendable { public let groupchannel: String? public let groupspace: String? public let timeout: Int? + public let besteffortdeliver: Bool? public let lane: String? public let extrasystemprompt: String? public let inputprovenance: [String: AnyCodable]? @@ -537,13 +558,14 @@ public struct AgentParams: Codable, Sendable { groupchannel: String?, groupspace: String?, timeout: Int?, + besteffortdeliver: Bool?, lane: String?, extrasystemprompt: String?, inputprovenance: [String: AnyCodable]?, idempotencykey: String, label: String?, - spawnedby: String? - ) { + spawnedby: String?) + { self.message = message self.agentid = agentid self.to = to @@ -562,6 +584,7 @@ public struct AgentParams: Codable, Sendable { self.groupchannel = groupchannel self.groupspace = groupspace self.timeout = timeout + self.besteffortdeliver = besteffortdeliver self.lane = lane self.extrasystemprompt = extrasystemprompt self.inputprovenance = inputprovenance @@ -569,6 +592,7 @@ public struct AgentParams: Codable, Sendable { self.label = label self.spawnedby = spawnedby } + private enum CodingKeys: String, CodingKey { case message case agentid = "agentId" @@ -588,6 +612,7 @@ public struct AgentParams: Codable, Sendable { case groupchannel = "groupChannel" case groupspace = "groupSpace" case timeout + case besteffortdeliver = "bestEffortDeliver" case lane case extrasystemprompt = "extraSystemPrompt" case inputprovenance = "inputProvenance" @@ -603,11 +628,12 @@ public struct AgentIdentityParams: Codable, Sendable { public init( agentid: String?, - sessionkey: String? - ) { + sessionkey: String?) + { self.agentid = agentid self.sessionkey = sessionkey } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" case sessionkey = "sessionKey" @@ -624,13 +650,14 @@ public struct AgentIdentityResult: Codable, Sendable { agentid: String, name: String?, avatar: String?, - emoji: String? - ) { + emoji: String?) + { self.agentid = agentid self.name = name self.avatar = avatar self.emoji = emoji } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" case name @@ -645,11 +672,12 @@ public struct AgentWaitParams: Codable, Sendable { public init( runid: String, - timeoutms: Int? - ) { + timeoutms: Int?) + { self.runid = runid self.timeoutms = timeoutms } + private enum CodingKeys: String, CodingKey { case runid = "runId" case timeoutms = "timeoutMs" @@ -662,11 +690,12 @@ public struct WakeParams: Codable, Sendable { public init( mode: AnyCodable, - text: String - ) { + text: String) + { self.mode = mode self.text = text } + private enum CodingKeys: String, CodingKey { case mode case text @@ -699,8 +728,8 @@ public struct NodePairRequestParams: Codable, Sendable { caps: [String]?, commands: [String]?, remoteip: String?, - silent: Bool? - ) { + silent: Bool?) + { self.nodeid = nodeid self.displayname = displayname self.platform = platform @@ -714,6 +743,7 @@ public struct NodePairRequestParams: Codable, Sendable { self.remoteip = remoteip self.silent = silent } + private enum CodingKeys: String, CodingKey { case nodeid = "nodeId" case displayname = "displayName" @@ -730,17 +760,17 @@ public struct NodePairRequestParams: Codable, Sendable { } } -public struct NodePairListParams: Codable, Sendable { -} +public struct NodePairListParams: Codable, Sendable {} public struct NodePairApproveParams: Codable, Sendable { public let requestid: String public init( - requestid: String - ) { + requestid: String) + { self.requestid = requestid } + private enum CodingKeys: String, CodingKey { case requestid = "requestId" } @@ -750,10 +780,11 @@ public struct NodePairRejectParams: Codable, Sendable { public let requestid: String public init( - requestid: String - ) { + requestid: String) + { self.requestid = requestid } + private enum CodingKeys: String, CodingKey { case requestid = "requestId" } @@ -765,11 +796,12 @@ public struct NodePairVerifyParams: Codable, Sendable { public init( nodeid: String, - token: String - ) { + token: String) + { self.nodeid = nodeid self.token = token } + private enum CodingKeys: String, CodingKey { case nodeid = "nodeId" case token @@ -782,28 +814,29 @@ public struct NodeRenameParams: Codable, Sendable { public init( nodeid: String, - displayname: String - ) { + displayname: String) + { self.nodeid = nodeid self.displayname = displayname } + private enum CodingKeys: String, CodingKey { case nodeid = "nodeId" case displayname = "displayName" } } -public struct NodeListParams: Codable, Sendable { -} +public struct NodeListParams: Codable, Sendable {} public struct NodeDescribeParams: Codable, Sendable { public let nodeid: String public init( - nodeid: String - ) { + nodeid: String) + { self.nodeid = nodeid } + private enum CodingKeys: String, CodingKey { case nodeid = "nodeId" } @@ -821,14 +854,15 @@ public struct NodeInvokeParams: Codable, Sendable { command: String, params: AnyCodable?, timeoutms: Int?, - idempotencykey: String - ) { + idempotencykey: String) + { self.nodeid = nodeid self.command = command self.params = params self.timeoutms = timeoutms self.idempotencykey = idempotencykey } + private enum CodingKeys: String, CodingKey { case nodeid = "nodeId" case command @@ -852,8 +886,8 @@ public struct NodeInvokeResultParams: Codable, Sendable { ok: Bool, payload: AnyCodable?, payloadjson: String?, - error: [String: AnyCodable]? - ) { + error: [String: AnyCodable]?) + { self.id = id self.nodeid = nodeid self.ok = ok @@ -861,6 +895,7 @@ public struct NodeInvokeResultParams: Codable, Sendable { self.payloadjson = payloadjson self.error = error } + private enum CodingKeys: String, CodingKey { case id case nodeid = "nodeId" @@ -879,12 +914,13 @@ public struct NodeEventParams: Codable, Sendable { public init( event: String, payload: AnyCodable?, - payloadjson: String? - ) { + payloadjson: String?) + { self.event = event self.payload = payload self.payloadjson = payloadjson } + private enum CodingKeys: String, CodingKey { case event case payload @@ -906,8 +942,8 @@ public struct NodeInvokeRequestEvent: Codable, Sendable { command: String, paramsjson: String?, timeoutms: Int?, - idempotencykey: String? - ) { + idempotencykey: String?) + { self.id = id self.nodeid = nodeid self.command = command @@ -915,6 +951,7 @@ public struct NodeInvokeRequestEvent: Codable, Sendable { self.timeoutms = timeoutms self.idempotencykey = idempotencykey } + private enum CodingKeys: String, CodingKey { case id case nodeid = "nodeId" @@ -935,13 +972,14 @@ public struct PushTestParams: Codable, Sendable { nodeid: String, title: String?, body: String?, - environment: String? - ) { + environment: String?) + { self.nodeid = nodeid self.title = title self.body = body self.environment = environment } + private enum CodingKeys: String, CodingKey { case nodeid = "nodeId" case title @@ -966,8 +1004,8 @@ public struct PushTestResult: Codable, Sendable { reason: String?, tokensuffix: String, topic: String, - environment: String - ) { + environment: String) + { self.ok = ok self.status = status self.apnsid = apnsid @@ -976,6 +1014,7 @@ public struct PushTestResult: Codable, Sendable { self.topic = topic self.environment = environment } + private enum CodingKeys: String, CodingKey { case ok case status @@ -1009,8 +1048,8 @@ public struct SessionsListParams: Codable, Sendable { label: String?, spawnedby: String?, agentid: String?, - search: String? - ) { + search: String?) + { self.limit = limit self.activeminutes = activeminutes self.includeglobal = includeglobal @@ -1022,6 +1061,7 @@ public struct SessionsListParams: Codable, Sendable { self.agentid = agentid self.search = search } + private enum CodingKeys: String, CodingKey { case limit case activeminutes = "activeMinutes" @@ -1044,12 +1084,13 @@ public struct SessionsPreviewParams: Codable, Sendable { public init( keys: [String], limit: Int?, - maxchars: Int? - ) { + maxchars: Int?) + { self.keys = keys self.limit = limit self.maxchars = maxchars } + private enum CodingKeys: String, CodingKey { case keys case limit @@ -1073,8 +1114,8 @@ public struct SessionsResolveParams: Codable, Sendable { agentid: String?, spawnedby: String?, includeglobal: Bool?, - includeunknown: Bool? - ) { + includeunknown: Bool?) + { self.key = key self.sessionid = sessionid self.label = label @@ -1083,6 +1124,7 @@ public struct SessionsResolveParams: Codable, Sendable { self.includeglobal = includeglobal self.includeunknown = includeunknown } + private enum CodingKeys: String, CodingKey { case key case sessionid = "sessionId" @@ -1128,8 +1170,8 @@ public struct SessionsPatchParams: Codable, Sendable { spawnedby: AnyCodable?, spawndepth: AnyCodable?, sendpolicy: AnyCodable?, - groupactivation: AnyCodable? - ) { + groupactivation: AnyCodable?) + { self.key = key self.label = label self.thinkinglevel = thinkinglevel @@ -1147,6 +1189,7 @@ public struct SessionsPatchParams: Codable, Sendable { self.sendpolicy = sendpolicy self.groupactivation = groupactivation } + private enum CodingKeys: String, CodingKey { case key case label @@ -1173,11 +1216,12 @@ public struct SessionsResetParams: Codable, Sendable { public init( key: String, - reason: AnyCodable? - ) { + reason: AnyCodable?) + { self.key = key self.reason = reason } + private enum CodingKeys: String, CodingKey { case key case reason @@ -1187,17 +1231,22 @@ public struct SessionsResetParams: Codable, Sendable { public struct SessionsDeleteParams: Codable, Sendable { public let key: String public let deletetranscript: Bool? + public let emitlifecyclehooks: Bool? public init( key: String, - deletetranscript: Bool? - ) { + deletetranscript: Bool?, + emitlifecyclehooks: Bool?) + { self.key = key self.deletetranscript = deletetranscript + self.emitlifecyclehooks = emitlifecyclehooks } + private enum CodingKeys: String, CodingKey { case key case deletetranscript = "deleteTranscript" + case emitlifecyclehooks = "emitLifecycleHooks" } } @@ -1207,11 +1256,12 @@ public struct SessionsCompactParams: Codable, Sendable { public init( key: String, - maxlines: Int? - ) { + maxlines: Int?) + { self.key = key self.maxlines = maxlines } + private enum CodingKeys: String, CodingKey { case key case maxlines = "maxLines" @@ -1222,6 +1272,8 @@ public struct SessionsUsageParams: Codable, Sendable { public let key: String? public let startdate: String? public let enddate: String? + public let mode: AnyCodable? + public let utcoffset: String? public let limit: Int? public let includecontextweight: Bool? @@ -1229,26 +1281,32 @@ public struct SessionsUsageParams: Codable, Sendable { key: String?, startdate: String?, enddate: String?, + mode: AnyCodable?, + utcoffset: String?, limit: Int?, - includecontextweight: Bool? - ) { + includecontextweight: Bool?) + { self.key = key self.startdate = startdate self.enddate = enddate + self.mode = mode + self.utcoffset = utcoffset self.limit = limit self.includecontextweight = includecontextweight } + private enum CodingKeys: String, CodingKey { case key case startdate = "startDate" case enddate = "endDate" + case mode + case utcoffset = "utcOffset" case limit case includecontextweight = "includeContextWeight" } } -public struct ConfigGetParams: Codable, Sendable { -} +public struct ConfigGetParams: Codable, Sendable {} public struct ConfigSetParams: Codable, Sendable { public let raw: String @@ -1256,11 +1314,12 @@ public struct ConfigSetParams: Codable, Sendable { public init( raw: String, - basehash: String? - ) { + basehash: String?) + { self.raw = raw self.basehash = basehash } + private enum CodingKeys: String, CodingKey { case raw case basehash = "baseHash" @@ -1279,14 +1338,15 @@ public struct ConfigApplyParams: Codable, Sendable { basehash: String?, sessionkey: String?, note: String?, - restartdelayms: Int? - ) { + restartdelayms: Int?) + { self.raw = raw self.basehash = basehash self.sessionkey = sessionkey self.note = note self.restartdelayms = restartdelayms } + private enum CodingKeys: String, CodingKey { case raw case basehash = "baseHash" @@ -1308,14 +1368,15 @@ public struct ConfigPatchParams: Codable, Sendable { basehash: String?, sessionkey: String?, note: String?, - restartdelayms: Int? - ) { + restartdelayms: Int?) + { self.raw = raw self.basehash = basehash self.sessionkey = sessionkey self.note = note self.restartdelayms = restartdelayms } + private enum CodingKeys: String, CodingKey { case raw case basehash = "baseHash" @@ -1325,8 +1386,7 @@ public struct ConfigPatchParams: Codable, Sendable { } } -public struct ConfigSchemaParams: Codable, Sendable { -} +public struct ConfigSchemaParams: Codable, Sendable {} public struct ConfigSchemaResponse: Codable, Sendable { public let schema: AnyCodable @@ -1338,13 +1398,14 @@ public struct ConfigSchemaResponse: Codable, Sendable { schema: AnyCodable, uihints: [String: AnyCodable], version: String, - generatedat: String - ) { + generatedat: String) + { self.schema = schema self.uihints = uihints self.version = version self.generatedat = generatedat } + private enum CodingKeys: String, CodingKey { case schema case uihints = "uiHints" @@ -1359,11 +1420,12 @@ public struct WizardStartParams: Codable, Sendable { public init( mode: AnyCodable?, - workspace: String? - ) { + workspace: String?) + { self.mode = mode self.workspace = workspace } + private enum CodingKeys: String, CodingKey { case mode case workspace @@ -1376,11 +1438,12 @@ public struct WizardNextParams: Codable, Sendable { public init( sessionid: String, - answer: [String: AnyCodable]? - ) { + answer: [String: AnyCodable]?) + { self.sessionid = sessionid self.answer = answer } + private enum CodingKeys: String, CodingKey { case sessionid = "sessionId" case answer @@ -1391,10 +1454,11 @@ public struct WizardCancelParams: Codable, Sendable { public let sessionid: String public init( - sessionid: String - ) { + sessionid: String) + { self.sessionid = sessionid } + private enum CodingKeys: String, CodingKey { case sessionid = "sessionId" } @@ -1404,10 +1468,11 @@ public struct WizardStatusParams: Codable, Sendable { public let sessionid: String public init( - sessionid: String - ) { + sessionid: String) + { self.sessionid = sessionid } + private enum CodingKeys: String, CodingKey { case sessionid = "sessionId" } @@ -1433,8 +1498,8 @@ public struct WizardStep: Codable, Sendable { initialvalue: AnyCodable?, placeholder: String?, sensitive: Bool?, - executor: AnyCodable? - ) { + executor: AnyCodable?) + { self.id = id self.type = type self.title = title @@ -1445,6 +1510,7 @@ public struct WizardStep: Codable, Sendable { self.sensitive = sensitive self.executor = executor } + private enum CodingKeys: String, CodingKey { case id case type @@ -1468,13 +1534,14 @@ public struct WizardNextResult: Codable, Sendable { done: Bool, step: [String: AnyCodable]?, status: AnyCodable?, - error: String? - ) { + error: String?) + { self.done = done self.step = step self.status = status self.error = error } + private enum CodingKeys: String, CodingKey { case done case step @@ -1495,14 +1562,15 @@ public struct WizardStartResult: Codable, Sendable { done: Bool, step: [String: AnyCodable]?, status: AnyCodable?, - error: String? - ) { + error: String?) + { self.sessionid = sessionid self.done = done self.step = step self.status = status self.error = error } + private enum CodingKeys: String, CodingKey { case sessionid = "sessionId" case done @@ -1518,11 +1586,12 @@ public struct WizardStatusResult: Codable, Sendable { public init( status: AnyCodable, - error: String? - ) { + error: String?) + { self.status = status self.error = error } + private enum CodingKeys: String, CodingKey { case status case error @@ -1535,11 +1604,12 @@ public struct TalkModeParams: Codable, Sendable { public init( enabled: Bool, - phase: String? - ) { + phase: String?) + { self.enabled = enabled self.phase = phase } + private enum CodingKeys: String, CodingKey { case enabled case phase @@ -1550,10 +1620,11 @@ public struct TalkConfigParams: Codable, Sendable { public let includesecrets: Bool? public init( - includesecrets: Bool? - ) { + includesecrets: Bool?) + { self.includesecrets = includesecrets } + private enum CodingKeys: String, CodingKey { case includesecrets = "includeSecrets" } @@ -1563,10 +1634,11 @@ public struct TalkConfigResult: Codable, Sendable { public let config: [String: AnyCodable] public init( - config: [String: AnyCodable] - ) { + config: [String: AnyCodable]) + { self.config = config } + private enum CodingKeys: String, CodingKey { case config } @@ -1578,11 +1650,12 @@ public struct ChannelsStatusParams: Codable, Sendable { public init( probe: Bool?, - timeoutms: Int? - ) { + timeoutms: Int?) + { self.probe = probe self.timeoutms = timeoutms } + private enum CodingKeys: String, CodingKey { case probe case timeoutms = "timeoutMs" @@ -1609,8 +1682,8 @@ public struct ChannelsStatusResult: Codable, Sendable { channelmeta: [[String: AnyCodable]]?, channels: [String: AnyCodable], channelaccounts: [String: AnyCodable], - channeldefaultaccountid: [String: AnyCodable] - ) { + channeldefaultaccountid: [String: AnyCodable]) + { self.ts = ts self.channelorder = channelorder self.channellabels = channellabels @@ -1621,6 +1694,7 @@ public struct ChannelsStatusResult: Codable, Sendable { self.channelaccounts = channelaccounts self.channeldefaultaccountid = channeldefaultaccountid } + private enum CodingKeys: String, CodingKey { case ts case channelorder = "channelOrder" @@ -1640,11 +1714,12 @@ public struct ChannelsLogoutParams: Codable, Sendable { public init( channel: String, - accountid: String? - ) { + accountid: String?) + { self.channel = channel self.accountid = accountid } + private enum CodingKeys: String, CodingKey { case channel case accountid = "accountId" @@ -1661,13 +1736,14 @@ public struct WebLoginStartParams: Codable, Sendable { force: Bool?, timeoutms: Int?, verbose: Bool?, - accountid: String? - ) { + accountid: String?) + { self.force = force self.timeoutms = timeoutms self.verbose = verbose self.accountid = accountid } + private enum CodingKeys: String, CodingKey { case force case timeoutms = "timeoutMs" @@ -1682,11 +1758,12 @@ public struct WebLoginWaitParams: Codable, Sendable { public init( timeoutms: Int?, - accountid: String? - ) { + accountid: String?) + { self.timeoutms = timeoutms self.accountid = accountid } + private enum CodingKeys: String, CodingKey { case timeoutms = "timeoutMs" case accountid = "accountId" @@ -1701,12 +1778,13 @@ public struct AgentSummary: Codable, Sendable { public init( id: String, name: String?, - identity: [String: AnyCodable]? - ) { + identity: [String: AnyCodable]?) + { self.id = id self.name = name self.identity = identity } + private enum CodingKeys: String, CodingKey { case id case name @@ -1724,13 +1802,14 @@ public struct AgentsCreateParams: Codable, Sendable { name: String, workspace: String, emoji: String?, - avatar: String? - ) { + avatar: String?) + { self.name = name self.workspace = workspace self.emoji = emoji self.avatar = avatar } + private enum CodingKeys: String, CodingKey { case name case workspace @@ -1749,13 +1828,14 @@ public struct AgentsCreateResult: Codable, Sendable { ok: Bool, agentid: String, name: String, - workspace: String - ) { + workspace: String) + { self.ok = ok self.agentid = agentid self.name = name self.workspace = workspace } + private enum CodingKeys: String, CodingKey { case ok case agentid = "agentId" @@ -1776,14 +1856,15 @@ public struct AgentsUpdateParams: Codable, Sendable { name: String?, workspace: String?, model: String?, - avatar: String? - ) { + avatar: String?) + { self.agentid = agentid self.name = name self.workspace = workspace self.model = model self.avatar = avatar } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" case name @@ -1799,11 +1880,12 @@ public struct AgentsUpdateResult: Codable, Sendable { public init( ok: Bool, - agentid: String - ) { + agentid: String) + { self.ok = ok self.agentid = agentid } + private enum CodingKeys: String, CodingKey { case ok case agentid = "agentId" @@ -1816,11 +1898,12 @@ public struct AgentsDeleteParams: Codable, Sendable { public init( agentid: String, - deletefiles: Bool? - ) { + deletefiles: Bool?) + { self.agentid = agentid self.deletefiles = deletefiles } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" case deletefiles = "deleteFiles" @@ -1835,12 +1918,13 @@ public struct AgentsDeleteResult: Codable, Sendable { public init( ok: Bool, agentid: String, - removedbindings: Int - ) { + removedbindings: Int) + { self.ok = ok self.agentid = agentid self.removedbindings = removedbindings } + private enum CodingKeys: String, CodingKey { case ok case agentid = "agentId" @@ -1862,8 +1946,8 @@ public struct AgentsFileEntry: Codable, Sendable { missing: Bool, size: Int?, updatedatms: Int?, - content: String? - ) { + content: String?) + { self.name = name self.path = path self.missing = missing @@ -1871,6 +1955,7 @@ public struct AgentsFileEntry: Codable, Sendable { self.updatedatms = updatedatms self.content = content } + private enum CodingKeys: String, CodingKey { case name case path @@ -1885,10 +1970,11 @@ public struct AgentsFilesListParams: Codable, Sendable { public let agentid: String public init( - agentid: String - ) { + agentid: String) + { self.agentid = agentid } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" } @@ -1902,12 +1988,13 @@ public struct AgentsFilesListResult: Codable, Sendable { public init( agentid: String, workspace: String, - files: [AgentsFileEntry] - ) { + files: [AgentsFileEntry]) + { self.agentid = agentid self.workspace = workspace self.files = files } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" case workspace @@ -1921,11 +2008,12 @@ public struct AgentsFilesGetParams: Codable, Sendable { public init( agentid: String, - name: String - ) { + name: String) + { self.agentid = agentid self.name = name } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" case name @@ -1940,12 +2028,13 @@ public struct AgentsFilesGetResult: Codable, Sendable { public init( agentid: String, workspace: String, - file: AgentsFileEntry - ) { + file: AgentsFileEntry) + { self.agentid = agentid self.workspace = workspace self.file = file } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" case workspace @@ -1961,12 +2050,13 @@ public struct AgentsFilesSetParams: Codable, Sendable { public init( agentid: String, name: String, - content: String - ) { + content: String) + { self.agentid = agentid self.name = name self.content = content } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" case name @@ -1984,13 +2074,14 @@ public struct AgentsFilesSetResult: Codable, Sendable { ok: Bool, agentid: String, workspace: String, - file: AgentsFileEntry - ) { + file: AgentsFileEntry) + { self.ok = ok self.agentid = agentid self.workspace = workspace self.file = file } + private enum CodingKeys: String, CodingKey { case ok case agentid = "agentId" @@ -1999,8 +2090,7 @@ public struct AgentsFilesSetResult: Codable, Sendable { } } -public struct AgentsListParams: Codable, Sendable { -} +public struct AgentsListParams: Codable, Sendable {} public struct AgentsListResult: Codable, Sendable { public let defaultid: String @@ -2012,13 +2102,14 @@ public struct AgentsListResult: Codable, Sendable { defaultid: String, mainkey: String, scope: AnyCodable, - agents: [AgentSummary] - ) { + agents: [AgentSummary]) + { self.defaultid = defaultid self.mainkey = mainkey self.scope = scope self.agents = agents } + private enum CodingKeys: String, CodingKey { case defaultid = "defaultId" case mainkey = "mainKey" @@ -2039,14 +2130,15 @@ public struct ModelChoice: Codable, Sendable { name: String, provider: String, contextwindow: Int?, - reasoning: Bool? - ) { + reasoning: Bool?) + { self.id = id self.name = name self.provider = provider self.contextwindow = contextwindow self.reasoning = reasoning } + private enum CodingKeys: String, CodingKey { case id case name @@ -2056,17 +2148,17 @@ public struct ModelChoice: Codable, Sendable { } } -public struct ModelsListParams: Codable, Sendable { -} +public struct ModelsListParams: Codable, Sendable {} public struct ModelsListResult: Codable, Sendable { public let models: [ModelChoice] public init( - models: [ModelChoice] - ) { + models: [ModelChoice]) + { self.models = models } + private enum CodingKeys: String, CodingKey { case models } @@ -2076,26 +2168,153 @@ public struct SkillsStatusParams: Codable, Sendable { public let agentid: String? public init( - agentid: String? - ) { + agentid: String?) + { self.agentid = agentid } + private enum CodingKeys: String, CodingKey { case agentid = "agentId" } } -public struct SkillsBinsParams: Codable, Sendable { +public struct ToolsCatalogParams: Codable, Sendable { + public let agentid: String? + public let includeplugins: Bool? + + public init( + agentid: String?, + includeplugins: Bool?) + { + self.agentid = agentid + self.includeplugins = includeplugins + } + + private enum CodingKeys: String, CodingKey { + case agentid = "agentId" + case includeplugins = "includePlugins" + } } +public struct ToolCatalogProfile: Codable, Sendable { + public let id: AnyCodable + public let label: String + + public init( + id: AnyCodable, + label: String) + { + self.id = id + self.label = label + } + + private enum CodingKeys: String, CodingKey { + case id + case label + } +} + +public struct ToolCatalogEntry: Codable, Sendable { + public let id: String + public let label: String + public let description: String + public let source: AnyCodable + public let pluginid: String? + public let optional: Bool? + public let defaultprofiles: [AnyCodable] + + public init( + id: String, + label: String, + description: String, + source: AnyCodable, + pluginid: String?, + optional: Bool?, + defaultprofiles: [AnyCodable]) + { + self.id = id + self.label = label + self.description = description + self.source = source + self.pluginid = pluginid + self.optional = optional + self.defaultprofiles = defaultprofiles + } + + private enum CodingKeys: String, CodingKey { + case id + case label + case description + case source + case pluginid = "pluginId" + case optional + case defaultprofiles = "defaultProfiles" + } +} + +public struct ToolCatalogGroup: Codable, Sendable { + public let id: String + public let label: String + public let source: AnyCodable + public let pluginid: String? + public let tools: [ToolCatalogEntry] + + public init( + id: String, + label: String, + source: AnyCodable, + pluginid: String?, + tools: [ToolCatalogEntry]) + { + self.id = id + self.label = label + self.source = source + self.pluginid = pluginid + self.tools = tools + } + + private enum CodingKeys: String, CodingKey { + case id + case label + case source + case pluginid = "pluginId" + case tools + } +} + +public struct ToolsCatalogResult: Codable, Sendable { + public let agentid: String + public let profiles: [ToolCatalogProfile] + public let groups: [ToolCatalogGroup] + + public init( + agentid: String, + profiles: [ToolCatalogProfile], + groups: [ToolCatalogGroup]) + { + self.agentid = agentid + self.profiles = profiles + self.groups = groups + } + + private enum CodingKeys: String, CodingKey { + case agentid = "agentId" + case profiles + case groups + } +} + +public struct SkillsBinsParams: Codable, Sendable {} + public struct SkillsBinsResult: Codable, Sendable { public let bins: [String] public init( - bins: [String] - ) { + bins: [String]) + { self.bins = bins } + private enum CodingKeys: String, CodingKey { case bins } @@ -2109,12 +2328,13 @@ public struct SkillsInstallParams: Codable, Sendable { public init( name: String, installid: String, - timeoutms: Int? - ) { + timeoutms: Int?) + { self.name = name self.installid = installid self.timeoutms = timeoutms } + private enum CodingKeys: String, CodingKey { case name case installid = "installId" @@ -2132,13 +2352,14 @@ public struct SkillsUpdateParams: Codable, Sendable { skillkey: String, enabled: Bool?, apikey: String?, - env: [String: AnyCodable]? - ) { + env: [String: AnyCodable]?) + { self.skillkey = skillkey self.enabled = enabled self.apikey = apikey self.env = env } + private enum CodingKeys: String, CodingKey { case skillkey = "skillKey" case enabled @@ -2179,8 +2400,8 @@ public struct CronJob: Codable, Sendable { wakemode: AnyCodable, payload: AnyCodable, delivery: AnyCodable?, - state: [String: AnyCodable] - ) { + state: [String: AnyCodable]) + { self.id = id self.agentid = agentid self.sessionkey = sessionkey @@ -2197,6 +2418,7 @@ public struct CronJob: Codable, Sendable { self.delivery = delivery self.state = state } + private enum CodingKeys: String, CodingKey { case id case agentid = "agentId" @@ -2218,19 +2440,43 @@ public struct CronJob: Codable, Sendable { public struct CronListParams: Codable, Sendable { public let includedisabled: Bool? + public let limit: Int? + public let offset: Int? + public let query: String? + public let enabled: AnyCodable? + public let sortby: AnyCodable? + public let sortdir: AnyCodable? public init( - includedisabled: Bool? - ) { + includedisabled: Bool?, + limit: Int?, + offset: Int?, + query: String?, + enabled: AnyCodable?, + sortby: AnyCodable?, + sortdir: AnyCodable?) + { self.includedisabled = includedisabled + self.limit = limit + self.offset = offset + self.query = query + self.enabled = enabled + self.sortby = sortby + self.sortdir = sortdir } + private enum CodingKeys: String, CodingKey { case includedisabled = "includeDisabled" + case limit + case offset + case query + case enabled + case sortby = "sortBy" + case sortdir = "sortDir" } } -public struct CronStatusParams: Codable, Sendable { -} +public struct CronStatusParams: Codable, Sendable {} public struct CronAddParams: Codable, Sendable { public let name: String @@ -2256,8 +2502,8 @@ public struct CronAddParams: Codable, Sendable { sessiontarget: AnyCodable, wakemode: AnyCodable, payload: AnyCodable, - delivery: AnyCodable? - ) { + delivery: AnyCodable?) + { self.name = name self.agentid = agentid self.sessionkey = sessionkey @@ -2270,6 +2516,7 @@ public struct CronAddParams: Codable, Sendable { self.payload = payload self.delivery = delivery } + private enum CodingKeys: String, CodingKey { case name case agentid = "agentId" @@ -2285,6 +2532,60 @@ public struct CronAddParams: Codable, Sendable { } } +public struct CronRunsParams: Codable, Sendable { + public let scope: AnyCodable? + public let id: String? + public let jobid: String? + public let limit: Int? + public let offset: Int? + public let statuses: [AnyCodable]? + public let status: AnyCodable? + public let deliverystatuses: [AnyCodable]? + public let deliverystatus: AnyCodable? + public let query: String? + public let sortdir: AnyCodable? + + public init( + scope: AnyCodable?, + id: String?, + jobid: String?, + limit: Int?, + offset: Int?, + statuses: [AnyCodable]?, + status: AnyCodable?, + deliverystatuses: [AnyCodable]?, + deliverystatus: AnyCodable?, + query: String?, + sortdir: AnyCodable?) + { + self.scope = scope + self.id = id + self.jobid = jobid + self.limit = limit + self.offset = offset + self.statuses = statuses + self.status = status + self.deliverystatuses = deliverystatuses + self.deliverystatus = deliverystatus + self.query = query + self.sortdir = sortdir + } + + private enum CodingKeys: String, CodingKey { + case scope + case id + case jobid = "jobId" + case limit + case offset + case statuses + case status + case deliverystatuses = "deliveryStatuses" + case deliverystatus = "deliveryStatus" + case query + case sortdir = "sortDir" + } +} + public struct CronRunLogEntry: Codable, Sendable { public let ts: Int public let jobid: String @@ -2292,11 +2593,18 @@ public struct CronRunLogEntry: Codable, Sendable { public let status: AnyCodable? public let error: String? public let summary: String? + public let delivered: Bool? + public let deliverystatus: AnyCodable? + public let deliveryerror: String? public let sessionid: String? public let sessionkey: String? public let runatms: Int? public let durationms: Int? public let nextrunatms: Int? + public let model: String? + public let provider: String? + public let usage: [String: AnyCodable]? + public let jobname: String? public init( ts: Int, @@ -2305,24 +2613,39 @@ public struct CronRunLogEntry: Codable, Sendable { status: AnyCodable?, error: String?, summary: String?, + delivered: Bool?, + deliverystatus: AnyCodable?, + deliveryerror: String?, sessionid: String?, sessionkey: String?, runatms: Int?, durationms: Int?, - nextrunatms: Int? - ) { + nextrunatms: Int?, + model: String?, + provider: String?, + usage: [String: AnyCodable]?, + jobname: String?) + { self.ts = ts self.jobid = jobid self.action = action self.status = status self.error = error self.summary = summary + self.delivered = delivered + self.deliverystatus = deliverystatus + self.deliveryerror = deliveryerror self.sessionid = sessionid self.sessionkey = sessionkey self.runatms = runatms self.durationms = durationms self.nextrunatms = nextrunatms + self.model = model + self.provider = provider + self.usage = usage + self.jobname = jobname } + private enum CodingKeys: String, CodingKey { case ts case jobid = "jobId" @@ -2330,11 +2653,18 @@ public struct CronRunLogEntry: Codable, Sendable { case status case error case summary + case delivered + case deliverystatus = "deliveryStatus" + case deliveryerror = "deliveryError" case sessionid = "sessionId" case sessionkey = "sessionKey" case runatms = "runAtMs" case durationms = "durationMs" case nextrunatms = "nextRunAtMs" + case model + case provider + case usage + case jobname = "jobName" } } @@ -2346,12 +2676,13 @@ public struct LogsTailParams: Codable, Sendable { public init( cursor: Int?, limit: Int?, - maxbytes: Int? - ) { + maxbytes: Int?) + { self.cursor = cursor self.limit = limit self.maxbytes = maxbytes } + private enum CodingKeys: String, CodingKey { case cursor case limit @@ -2373,8 +2704,8 @@ public struct LogsTailResult: Codable, Sendable { size: Int, lines: [String], truncated: Bool?, - reset: Bool? - ) { + reset: Bool?) + { self.file = file self.cursor = cursor self.size = size @@ -2382,6 +2713,7 @@ public struct LogsTailResult: Codable, Sendable { self.truncated = truncated self.reset = reset } + private enum CodingKeys: String, CodingKey { case file case cursor @@ -2392,8 +2724,7 @@ public struct LogsTailResult: Codable, Sendable { } } -public struct ExecApprovalsGetParams: Codable, Sendable { -} +public struct ExecApprovalsGetParams: Codable, Sendable {} public struct ExecApprovalsSetParams: Codable, Sendable { public let file: [String: AnyCodable] @@ -2401,11 +2732,12 @@ public struct ExecApprovalsSetParams: Codable, Sendable { public init( file: [String: AnyCodable], - basehash: String? - ) { + basehash: String?) + { self.file = file self.basehash = basehash } + private enum CodingKeys: String, CodingKey { case file case basehash = "baseHash" @@ -2416,10 +2748,11 @@ public struct ExecApprovalsNodeGetParams: Codable, Sendable { public let nodeid: String public init( - nodeid: String - ) { + nodeid: String) + { self.nodeid = nodeid } + private enum CodingKeys: String, CodingKey { case nodeid = "nodeId" } @@ -2433,12 +2766,13 @@ public struct ExecApprovalsNodeSetParams: Codable, Sendable { public init( nodeid: String, file: [String: AnyCodable], - basehash: String? - ) { + basehash: String?) + { self.nodeid = nodeid self.file = file self.basehash = basehash } + private enum CodingKeys: String, CodingKey { case nodeid = "nodeId" case file @@ -2456,13 +2790,14 @@ public struct ExecApprovalsSnapshot: Codable, Sendable { path: String, exists: Bool, hash: String, - file: [String: AnyCodable] - ) { + file: [String: AnyCodable]) + { self.path = path self.exists = exists self.hash = hash self.file = file } + private enum CodingKeys: String, CodingKey { case path case exists @@ -2474,51 +2809,84 @@ public struct ExecApprovalsSnapshot: Codable, Sendable { public struct ExecApprovalRequestParams: Codable, Sendable { public let id: String? public let command: String + public let commandargv: [String]? + public let systemrunplanv2: [String: AnyCodable]? + public let env: [String: AnyCodable]? public let cwd: AnyCodable? + public let nodeid: AnyCodable? public let host: AnyCodable? public let security: AnyCodable? public let ask: AnyCodable? public let agentid: AnyCodable? public let resolvedpath: AnyCodable? public let sessionkey: AnyCodable? + public let turnsourcechannel: AnyCodable? + public let turnsourceto: AnyCodable? + public let turnsourceaccountid: AnyCodable? + public let turnsourcethreadid: AnyCodable? public let timeoutms: Int? public let twophase: Bool? public init( id: String?, command: String, + commandargv: [String]?, + systemrunplanv2: [String: AnyCodable]?, + env: [String: AnyCodable]?, cwd: AnyCodable?, + nodeid: AnyCodable?, host: AnyCodable?, security: AnyCodable?, ask: AnyCodable?, agentid: AnyCodable?, resolvedpath: AnyCodable?, sessionkey: AnyCodable?, + turnsourcechannel: AnyCodable?, + turnsourceto: AnyCodable?, + turnsourceaccountid: AnyCodable?, + turnsourcethreadid: AnyCodable?, timeoutms: Int?, - twophase: Bool? - ) { + twophase: Bool?) + { self.id = id self.command = command + self.commandargv = commandargv + self.systemrunplanv2 = systemrunplanv2 + self.env = env self.cwd = cwd + self.nodeid = nodeid self.host = host self.security = security self.ask = ask self.agentid = agentid self.resolvedpath = resolvedpath self.sessionkey = sessionkey + self.turnsourcechannel = turnsourcechannel + self.turnsourceto = turnsourceto + self.turnsourceaccountid = turnsourceaccountid + self.turnsourcethreadid = turnsourcethreadid self.timeoutms = timeoutms self.twophase = twophase } + private enum CodingKeys: String, CodingKey { case id case command + case commandargv = "commandArgv" + case systemrunplanv2 = "systemRunPlanV2" + case env case cwd + case nodeid = "nodeId" case host case security case ask case agentid = "agentId" case resolvedpath = "resolvedPath" case sessionkey = "sessionKey" + case turnsourcechannel = "turnSourceChannel" + case turnsourceto = "turnSourceTo" + case turnsourceaccountid = "turnSourceAccountId" + case turnsourcethreadid = "turnSourceThreadId" case timeoutms = "timeoutMs" case twophase = "twoPhase" } @@ -2530,28 +2898,29 @@ public struct ExecApprovalResolveParams: Codable, Sendable { public init( id: String, - decision: String - ) { + decision: String) + { self.id = id self.decision = decision } + private enum CodingKeys: String, CodingKey { case id case decision } } -public struct DevicePairListParams: Codable, Sendable { -} +public struct DevicePairListParams: Codable, Sendable {} public struct DevicePairApproveParams: Codable, Sendable { public let requestid: String public init( - requestid: String - ) { + requestid: String) + { self.requestid = requestid } + private enum CodingKeys: String, CodingKey { case requestid = "requestId" } @@ -2561,10 +2930,11 @@ public struct DevicePairRejectParams: Codable, Sendable { public let requestid: String public init( - requestid: String - ) { + requestid: String) + { self.requestid = requestid } + private enum CodingKeys: String, CodingKey { case requestid = "requestId" } @@ -2574,10 +2944,11 @@ public struct DevicePairRemoveParams: Codable, Sendable { public let deviceid: String public init( - deviceid: String - ) { + deviceid: String) + { self.deviceid = deviceid } + private enum CodingKeys: String, CodingKey { case deviceid = "deviceId" } @@ -2591,12 +2962,13 @@ public struct DeviceTokenRotateParams: Codable, Sendable { public init( deviceid: String, role: String, - scopes: [String]? - ) { + scopes: [String]?) + { self.deviceid = deviceid self.role = role self.scopes = scopes } + private enum CodingKeys: String, CodingKey { case deviceid = "deviceId" case role @@ -2610,11 +2982,12 @@ public struct DeviceTokenRevokeParams: Codable, Sendable { public init( deviceid: String, - role: String - ) { + role: String) + { self.deviceid = deviceid self.role = role } + private enum CodingKeys: String, CodingKey { case deviceid = "deviceId" case role @@ -2627,6 +3000,7 @@ public struct DevicePairRequestedEvent: Codable, Sendable { public let publickey: String public let displayname: String? public let platform: String? + public let devicefamily: String? public let clientid: String? public let clientmode: String? public let role: String? @@ -2643,6 +3017,7 @@ public struct DevicePairRequestedEvent: Codable, Sendable { publickey: String, displayname: String?, platform: String?, + devicefamily: String?, clientid: String?, clientmode: String?, role: String?, @@ -2651,13 +3026,14 @@ public struct DevicePairRequestedEvent: Codable, Sendable { remoteip: String?, silent: Bool?, isrepair: Bool?, - ts: Int - ) { + ts: Int) + { self.requestid = requestid self.deviceid = deviceid self.publickey = publickey self.displayname = displayname self.platform = platform + self.devicefamily = devicefamily self.clientid = clientid self.clientmode = clientmode self.role = role @@ -2668,12 +3044,14 @@ public struct DevicePairRequestedEvent: Codable, Sendable { self.isrepair = isrepair self.ts = ts } + private enum CodingKeys: String, CodingKey { case requestid = "requestId" case deviceid = "deviceId" case publickey = "publicKey" case displayname = "displayName" case platform + case devicefamily = "deviceFamily" case clientid = "clientId" case clientmode = "clientMode" case role @@ -2696,13 +3074,14 @@ public struct DevicePairResolvedEvent: Codable, Sendable { requestid: String, deviceid: String, decision: String, - ts: Int - ) { + ts: Int) + { self.requestid = requestid self.deviceid = deviceid self.decision = decision self.ts = ts } + private enum CodingKeys: String, CodingKey { case requestid = "requestId" case deviceid = "deviceId" @@ -2717,11 +3096,12 @@ public struct ChatHistoryParams: Codable, Sendable { public init( sessionkey: String, - limit: Int? - ) { + limit: Int?) + { self.sessionkey = sessionkey self.limit = limit } + private enum CodingKeys: String, CodingKey { case sessionkey = "sessionKey" case limit @@ -2744,8 +3124,8 @@ public struct ChatSendParams: Codable, Sendable { deliver: Bool?, attachments: [AnyCodable]?, timeoutms: Int?, - idempotencykey: String - ) { + idempotencykey: String) + { self.sessionkey = sessionkey self.message = message self.thinking = thinking @@ -2754,6 +3134,7 @@ public struct ChatSendParams: Codable, Sendable { self.timeoutms = timeoutms self.idempotencykey = idempotencykey } + private enum CodingKeys: String, CodingKey { case sessionkey = "sessionKey" case message @@ -2771,11 +3152,12 @@ public struct ChatAbortParams: Codable, Sendable { public init( sessionkey: String, - runid: String? - ) { + runid: String?) + { self.sessionkey = sessionkey self.runid = runid } + private enum CodingKeys: String, CodingKey { case sessionkey = "sessionKey" case runid = "runId" @@ -2790,12 +3172,13 @@ public struct ChatInjectParams: Codable, Sendable { public init( sessionkey: String, message: String, - label: String? - ) { + label: String?) + { self.sessionkey = sessionkey self.message = message self.label = label } + private enum CodingKeys: String, CodingKey { case sessionkey = "sessionKey" case message @@ -2821,8 +3204,8 @@ public struct ChatEvent: Codable, Sendable { message: AnyCodable?, errormessage: String?, usage: AnyCodable?, - stopreason: String? - ) { + stopreason: String?) + { self.runid = runid self.sessionkey = sessionkey self.seq = seq @@ -2832,6 +3215,7 @@ public struct ChatEvent: Codable, Sendable { self.usage = usage self.stopreason = stopreason } + private enum CodingKeys: String, CodingKey { case runid = "runId" case sessionkey = "sessionKey" @@ -2854,13 +3238,14 @@ public struct UpdateRunParams: Codable, Sendable { sessionkey: String?, note: String?, restartdelayms: Int?, - timeoutms: Int? - ) { + timeoutms: Int?) + { self.sessionkey = sessionkey self.note = note self.restartdelayms = restartdelayms self.timeoutms = timeoutms } + private enum CodingKeys: String, CodingKey { case sessionkey = "sessionKey" case note @@ -2873,10 +3258,11 @@ public struct TickEvent: Codable, Sendable { public let ts: Int public init( - ts: Int - ) { + ts: Int) + { self.ts = ts } + private enum CodingKeys: String, CodingKey { case ts } @@ -2888,11 +3274,12 @@ public struct ShutdownEvent: Codable, Sendable { public init( reason: String, - restartexpectedms: Int? - ) { + restartexpectedms: Int?) + { self.reason = reason self.restartexpectedms = restartexpectedms } + private enum CodingKeys: String, CodingKey { case reason case restartexpectedms = "restartExpectedMs" @@ -2914,11 +3301,11 @@ public enum GatewayFrame: Codable, Sendable { let type = try typeContainer.decode(String.self, forKey: .type) switch type { case "req": - self = .req(try RequestFrame(from: decoder)) + self = try .req(RequestFrame(from: decoder)) case "res": - self = .res(try ResponseFrame(from: decoder)) + self = try .res(ResponseFrame(from: decoder)) case "event": - self = .event(try EventFrame(from: decoder)) + self = try .event(EventFrame(from: decoder)) default: let container = try decoder.singleValueContainer() let raw = try container.decode([String: AnyCodable].self) @@ -2928,13 +3315,15 @@ public enum GatewayFrame: Codable, Sendable { public func encode(to encoder: Encoder) throws { switch self { - case .req(let v): try v.encode(to: encoder) - case .res(let v): try v.encode(to: encoder) - case .event(let v): try v.encode(to: encoder) - case .unknown(_, let raw): + case let .req(v): + try v.encode(to: encoder) + case let .res(v): + try v.encode(to: encoder) + case let .event(v): + try v.encode(to: encoder) + case let .unknown(_, raw): var container = encoder.singleValueContainer() try container.encode(raw) } } - } diff --git a/apps/shared/OpenClawKit/Tests/OpenClawKitTests/ChatMarkdownPreprocessorTests.swift b/apps/shared/OpenClawKit/Tests/OpenClawKitTests/ChatMarkdownPreprocessorTests.swift index 808f74af64f..781a325f3cf 100644 --- a/apps/shared/OpenClawKit/Tests/OpenClawKitTests/ChatMarkdownPreprocessorTests.swift +++ b/apps/shared/OpenClawKit/Tests/OpenClawKitTests/ChatMarkdownPreprocessorTests.swift @@ -17,4 +17,91 @@ struct ChatMarkdownPreprocessorTests { #expect(result.images.count == 1) #expect(result.images.first?.image != nil) } + + @Test func stripsInboundUntrustedContextBlocks() { + let markdown = """ + Conversation info (untrusted metadata): + ```json + { + "message_id": "123", + "sender": "openclaw-ios" + } + ``` + + Sender (untrusted metadata): + ```json + { + "label": "Razor" + } + ``` + + Razor? + """ + + let result = ChatMarkdownPreprocessor.preprocess(markdown: markdown) + + #expect(result.cleaned == "Razor?") + } + + @Test func stripsSingleConversationInfoBlock() { + let text = """ + Conversation info (untrusted metadata): + ```json + {"x": 1} + ``` + + User message + """ + + let result = ChatMarkdownPreprocessor.preprocess(markdown: text) + + #expect(result.cleaned == "User message") + } + + @Test func stripsAllKnownInboundMetadataSentinels() { + let sentinels = [ + "Conversation info (untrusted metadata):", + "Sender (untrusted metadata):", + "Thread starter (untrusted, for context):", + "Replied message (untrusted, for context):", + "Forwarded message context (untrusted metadata):", + "Chat history since last reply (untrusted, for context):", + ] + + for sentinel in sentinels { + let markdown = """ + \(sentinel) + ```json + {"x": 1} + ``` + + User content + """ + let result = ChatMarkdownPreprocessor.preprocess(markdown: markdown) + #expect(result.cleaned == "User content") + } + } + + @Test func preservesNonMetadataJsonFence() { + let markdown = """ + Here is some json: + ```json + {"x": 1} + ``` + """ + + let result = ChatMarkdownPreprocessor.preprocess(markdown: markdown) + + #expect(result.cleaned == markdown.trimmingCharacters(in: .whitespacesAndNewlines)) + } + + @Test func stripsLeadingTimestampPrefix() { + let markdown = """ + [Fri 2026-02-20 18:45 GMT+1] How's it going? + """ + + let result = ChatMarkdownPreprocessor.preprocess(markdown: markdown) + + #expect(result.cleaned == "How's it going?") + } } diff --git a/apps/shared/OpenClawKit/Tests/OpenClawKitTests/ChatViewModelTests.swift b/apps/shared/OpenClawKit/Tests/OpenClawKitTests/ChatViewModelTests.swift index ff7caabf381..147b80e5be1 100644 --- a/apps/shared/OpenClawKit/Tests/OpenClawKitTests/ChatViewModelTests.swift +++ b/apps/shared/OpenClawKit/Tests/OpenClawKitTests/ChatViewModelTests.swift @@ -416,6 +416,48 @@ extension TestChatTransportState { #expect(await MainActor.run { vm.pendingToolCalls.isEmpty }) } + @Test func seqGapClearsPendingRunsAndAutoRefreshesHistory() async throws { + let now = Date().timeIntervalSince1970 * 1000 + let history1 = OpenClawChatHistoryPayload( + sessionKey: "main", + sessionId: "sess-main", + messages: [], + thinkingLevel: "off") + let history2 = OpenClawChatHistoryPayload( + sessionKey: "main", + sessionId: "sess-main", + messages: [ + AnyCodable([ + "role": "assistant", + "content": [["type": "text", "text": "resynced after gap"]], + "timestamp": now, + ]), + ], + thinkingLevel: "off") + + let transport = TestChatTransport(historyResponses: [history1, history2]) + let vm = await MainActor.run { OpenClawChatViewModel(sessionKey: "main", transport: transport) } + + await MainActor.run { vm.load() } + try await waitUntil("bootstrap") { await MainActor.run { vm.healthOK } } + + await MainActor.run { + vm.input = "hello" + vm.send() + } + try await waitUntil("pending run starts") { await MainActor.run { vm.pendingRunCount == 1 } } + + transport.emit(.seqGap) + + try await waitUntil("pending run clears on seqGap") { + await MainActor.run { vm.pendingRunCount == 0 } + } + try await waitUntil("history refreshes on seqGap") { + await MainActor.run { vm.messages.contains(where: { $0.role == "assistant" }) } + } + #expect(await MainActor.run { vm.errorText == nil }) + } + @Test func sessionChoicesPreferMainAndRecent() async throws { let now = Date().timeIntervalSince1970 * 1000 let recent = now - (2 * 60 * 60 * 1000) @@ -605,6 +647,35 @@ extension TestChatTransportState { try await waitUntil("streaming cleared") { await MainActor.run { vm.streamingAssistantText == nil } } } + @Test func stripsInboundMetadataFromHistoryMessages() async throws { + let history = OpenClawChatHistoryPayload( + sessionKey: "main", + sessionId: "sess-main", + messages: [ + AnyCodable([ + "role": "user", + "content": [["type": "text", "text": """ +Conversation info (untrusted metadata): +```json +{ \"sender\": \"openclaw-ios\" } +``` + +Hello? +"""]], + "timestamp": Date().timeIntervalSince1970 * 1000, + ]), + ], + thinkingLevel: "off") + let transport = TestChatTransport(historyResponses: [history]) + let vm = await MainActor.run { OpenClawChatViewModel(sessionKey: "main", transport: transport) } + + await MainActor.run { vm.load() } + try await waitUntil("history loaded") { await MainActor.run { !vm.messages.isEmpty } } + + let sanitized = await MainActor.run { vm.messages.first?.content.first?.text } + #expect(sanitized == "Hello?") + } + @Test func abortRequestsDoNotClearPendingUntilAbortedEvent() async throws { let sessionId = "sess-main" let history = OpenClawChatHistoryPayload( diff --git a/apps/shared/OpenClawKit/Tests/OpenClawKitTests/DeepLinksSecurityTests.swift b/apps/shared/OpenClawKit/Tests/OpenClawKitTests/DeepLinksSecurityTests.swift new file mode 100644 index 00000000000..8bbf4f8a650 --- /dev/null +++ b/apps/shared/OpenClawKit/Tests/OpenClawKitTests/DeepLinksSecurityTests.swift @@ -0,0 +1,61 @@ +import Foundation +import OpenClawKit +import Testing + +@Suite struct DeepLinksSecurityTests { + @Test func gatewayDeepLinkRejectsInsecureNonLoopbackWs() { + let url = URL( + string: "openclaw://gateway?host=attacker.example&port=18789&tls=0&token=abc")! + #expect(DeepLinkParser.parse(url) == nil) + } + + @Test func gatewayDeepLinkRejectsInsecurePrefixBypassHost() { + let url = URL( + string: "openclaw://gateway?host=127.attacker.example&port=18789&tls=0&token=abc")! + #expect(DeepLinkParser.parse(url) == nil) + } + + @Test func gatewayDeepLinkAllowsLoopbackWs() { + let url = URL( + string: "openclaw://gateway?host=127.0.0.1&port=18789&tls=0&token=abc")! + #expect( + DeepLinkParser.parse(url) == .gateway( + .init(host: "127.0.0.1", port: 18789, tls: false, token: "abc", password: nil))) + } + + @Test func setupCodeRejectsInsecureNonLoopbackWs() { + let payload = #"{"url":"ws://attacker.example:18789","token":"tok"}"# + let encoded = Data(payload.utf8) + .base64EncodedString() + .replacingOccurrences(of: "+", with: "-") + .replacingOccurrences(of: "/", with: "_") + .replacingOccurrences(of: "=", with: "") + #expect(GatewayConnectDeepLink.fromSetupCode(encoded) == nil) + } + + @Test func setupCodeRejectsInsecurePrefixBypassHost() { + let payload = #"{"url":"ws://127.attacker.example:18789","token":"tok"}"# + let encoded = Data(payload.utf8) + .base64EncodedString() + .replacingOccurrences(of: "+", with: "-") + .replacingOccurrences(of: "/", with: "_") + .replacingOccurrences(of: "=", with: "") + #expect(GatewayConnectDeepLink.fromSetupCode(encoded) == nil) + } + + @Test func setupCodeAllowsLoopbackWs() { + let payload = #"{"url":"ws://127.0.0.1:18789","token":"tok"}"# + let encoded = Data(payload.utf8) + .base64EncodedString() + .replacingOccurrences(of: "+", with: "-") + .replacingOccurrences(of: "/", with: "_") + .replacingOccurrences(of: "=", with: "") + #expect( + GatewayConnectDeepLink.fromSetupCode(encoded) == .init( + host: "127.0.0.1", + port: 18789, + tls: false, + token: "tok", + password: nil)) + } +} diff --git a/apps/shared/OpenClawKit/Tests/OpenClawKitTests/DeviceAuthPayloadTests.swift b/apps/shared/OpenClawKit/Tests/OpenClawKitTests/DeviceAuthPayloadTests.swift new file mode 100644 index 00000000000..46a814f81a6 --- /dev/null +++ b/apps/shared/OpenClawKit/Tests/OpenClawKitTests/DeviceAuthPayloadTests.swift @@ -0,0 +1,30 @@ +import Testing +@testable import OpenClawKit + +@Suite("DeviceAuthPayload") +struct DeviceAuthPayloadTests { + @Test("builds canonical v3 payload vector") + func buildsCanonicalV3PayloadVector() { + let payload = GatewayDeviceAuthPayload.buildV3( + deviceId: "dev-1", + clientId: "openclaw-macos", + clientMode: "ui", + role: "operator", + scopes: ["operator.admin", "operator.read"], + signedAtMs: 1_700_000_000_000, + token: "tok-123", + nonce: "nonce-abc", + platform: " IOS ", + deviceFamily: " iPhone ") + #expect( + payload + == "v3|dev-1|openclaw-macos|ui|operator|operator.admin,operator.read|1700000000000|tok-123|nonce-abc|ios|iphone") + } + + @Test("normalizes metadata with ASCII-only lowercase") + func normalizesMetadataWithAsciiLowercase() { + #expect(GatewayDeviceAuthPayload.normalizeMetadataField(" İOS ") == "İos") + #expect(GatewayDeviceAuthPayload.normalizeMetadataField(" MAC ") == "mac") + #expect(GatewayDeviceAuthPayload.normalizeMetadataField(nil) == "") + } +} diff --git a/apps/shared/OpenClawKit/Tests/OpenClawKitTests/GatewayNodeSessionTests.swift b/apps/shared/OpenClawKit/Tests/OpenClawKitTests/GatewayNodeSessionTests.swift index 91e30961591..08a6ea2162a 100644 --- a/apps/shared/OpenClawKit/Tests/OpenClawKitTests/GatewayNodeSessionTests.swift +++ b/apps/shared/OpenClawKit/Tests/OpenClawKitTests/GatewayNodeSessionTests.swift @@ -3,6 +3,182 @@ import Testing @testable import OpenClawKit import OpenClawProtocol +private struct TimeoutError: Error, CustomStringConvertible { + let label: String + var description: String { "Timeout waiting for: \(self.label)" } +} + +private func waitUntil( + _ label: String, + timeoutSeconds: Double = 3.0, + pollMs: UInt64 = 10, + _ condition: @escaping @Sendable () async -> Bool) async throws +{ + let deadline = Date().addingTimeInterval(timeoutSeconds) + while Date() < deadline { + if await condition() { + return + } + try await Task.sleep(nanoseconds: pollMs * 1_000_000) + } + throw TimeoutError(label: label) +} + +private extension NSLock { + func withLock(_ body: () -> T) -> T { + self.lock() + defer { self.unlock() } + return body() + } +} + +private final class FakeGatewayWebSocketTask: WebSocketTasking, @unchecked Sendable { + private let lock = NSLock() + private var _state: URLSessionTask.State = .suspended + private var connectRequestId: String? + private var receivePhase = 0 + private var pendingReceiveHandler: + (@Sendable (Result) -> Void)? + + var state: URLSessionTask.State { + get { self.lock.withLock { self._state } } + set { self.lock.withLock { self._state = newValue } } + } + + func resume() { + self.state = .running + } + + func cancel(with closeCode: URLSessionWebSocketTask.CloseCode, reason: Data?) { + _ = (closeCode, reason) + self.state = .canceling + let handler = self.lock.withLock { () -> (@Sendable (Result) -> Void)? in + defer { self.pendingReceiveHandler = nil } + return self.pendingReceiveHandler + } + handler?(Result.failure(URLError(.cancelled))) + } + + func send(_ message: URLSessionWebSocketTask.Message) async throws { + let data: Data? = switch message { + case let .data(d): d + case let .string(s): s.data(using: .utf8) + @unknown default: nil + } + guard let data else { return } + if let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any], + obj["type"] as? String == "req", + obj["method"] as? String == "connect", + let id = obj["id"] as? String + { + self.lock.withLock { self.connectRequestId = id } + } + } + + func sendPing(pongReceiveHandler: @escaping @Sendable (Error?) -> Void) { + pongReceiveHandler(nil) + } + + func receive() async throws -> URLSessionWebSocketTask.Message { + let phase = self.lock.withLock { () -> Int in + let current = self.receivePhase + self.receivePhase += 1 + return current + } + if phase == 0 { + return .data(Self.connectChallengeData(nonce: "nonce-1")) + } + for _ in 0..<50 { + let id = self.lock.withLock { self.connectRequestId } + if let id { + return .data(Self.connectOkData(id: id)) + } + try await Task.sleep(nanoseconds: 1_000_000) + } + return .data(Self.connectOkData(id: "connect")) + } + + func receive( + completionHandler: @escaping @Sendable (Result) -> Void) + { + self.lock.withLock { self.pendingReceiveHandler = completionHandler } + } + + func emitReceiveFailure() { + let handler = self.lock.withLock { () -> (@Sendable (Result) -> Void)? in + self._state = .canceling + defer { self.pendingReceiveHandler = nil } + return self.pendingReceiveHandler + } + handler?(Result.failure(URLError(.networkConnectionLost))) + } + + private static func connectChallengeData(nonce: String) -> Data { + let json = """ + { + "type": "event", + "event": "connect.challenge", + "payload": { "nonce": "\(nonce)" } + } + """ + return Data(json.utf8) + } + + private static func connectOkData(id: String) -> Data { + let json = """ + { + "type": "res", + "id": "\(id)", + "ok": true, + "payload": { + "type": "hello-ok", + "protocol": 2, + "server": { "version": "test", "connId": "test" }, + "features": { "methods": [], "events": [] }, + "snapshot": { + "presence": [ { "ts": 1 } ], + "health": {}, + "stateVersion": { "presence": 0, "health": 0 }, + "uptimeMs": 0 + }, + "policy": { "maxPayload": 1, "maxBufferedBytes": 1, "tickIntervalMs": 30000 } + } + } + """ + return Data(json.utf8) + } +} + +private final class FakeGatewayWebSocketSession: WebSocketSessioning, @unchecked Sendable { + private let lock = NSLock() + private var tasks: [FakeGatewayWebSocketTask] = [] + private var makeCount = 0 + + func snapshotMakeCount() -> Int { + self.lock.withLock { self.makeCount } + } + + func latestTask() -> FakeGatewayWebSocketTask? { + self.lock.withLock { self.tasks.last } + } + + func makeWebSocketTask(url: URL) -> WebSocketTaskBox { + _ = url + return self.lock.withLock { + self.makeCount += 1 + let task = FakeGatewayWebSocketTask() + self.tasks.append(task) + return WebSocketTaskBox(task: task) + } + } +} + +private actor SeqGapProbe { + private var saw = false + func mark() { self.saw = true } + func value() -> Bool { self.saw } +} + struct GatewayNodeSessionTests { @Test func invokeWithTimeoutReturnsUnderlyingResponseBeforeTimeout() async { @@ -53,4 +229,56 @@ struct GatewayNodeSessionTests { #expect(response.ok == true) #expect(response.error == nil) } + + @Test + func emitsSyntheticSeqGapAfterReconnectSnapshot() async throws { + let session = FakeGatewayWebSocketSession() + let gateway = GatewayNodeSession() + let options = GatewayConnectOptions( + role: "operator", + scopes: ["operator.read"], + caps: [], + commands: [], + permissions: [:], + clientId: "openclaw-ios-test", + clientMode: "ui", + clientDisplayName: "iOS Test", + includeDeviceIdentity: false) + + let stream = await gateway.subscribeServerEvents(bufferingNewest: 32) + let probe = SeqGapProbe() + let listenTask = Task { + for await evt in stream { + if evt.event == "seqGap" { + await probe.mark() + return + } + } + } + + try await gateway.connect( + url: URL(string: "ws://example.invalid")!, + token: nil, + password: nil, + connectOptions: options, + sessionBox: WebSocketSessionBox(session: session), + onConnected: {}, + onDisconnected: { _ in }, + onInvoke: { req in + BridgeInvokeResponse(id: req.id, ok: true, payloadJSON: nil, error: nil) + }) + + let firstTask = try #require(session.latestTask()) + firstTask.emitReceiveFailure() + + try await waitUntil("reconnect socket created") { + session.snapshotMakeCount() >= 2 + } + try await waitUntil("synthetic seqGap broadcast") { + await probe.value() + } + + listenTask.cancel() + await gateway.disconnect() + } } diff --git a/apps/shared/OpenClawKit/Tests/OpenClawKitTests/ToolResultTextFormatterTests.swift b/apps/shared/OpenClawKit/Tests/OpenClawKitTests/ToolResultTextFormatterTests.swift new file mode 100644 index 00000000000..1688725c850 --- /dev/null +++ b/apps/shared/OpenClawKit/Tests/OpenClawKitTests/ToolResultTextFormatterTests.swift @@ -0,0 +1,54 @@ +import Testing +@testable import OpenClawChatUI + +@Suite("ToolResultTextFormatter") +struct ToolResultTextFormatterTests { + @Test func leavesPlainTextUntouched() { + let result = ToolResultTextFormatter.format(text: "All good", toolName: "nodes") + #expect(result == "All good") + } + + @Test func summarizesNodesListJSON() { + let json = """ + { + "ts": 1771610031380, + "nodes": [ + { + "displayName": "iPhone 16 Pro Max", + "connected": true, + "platform": "ios" + } + ] + } + """ + + let result = ToolResultTextFormatter.format(text: json, toolName: "nodes") + #expect(result.contains("1 node found.")) + #expect(result.contains("iPhone 16 Pro Max")) + #expect(result.contains("connected")) + } + + @Test func summarizesErrorJSONAndDropsAgentPrefix() { + let json = """ + { + "status": "error", + "tool": "nodes", + "error": "agent=main node=iPhone gateway=default action=invoke: pairing required" + } + """ + + let result = ToolResultTextFormatter.format(text: json, toolName: "nodes") + #expect(result == "Error: pairing required") + } + + @Test func suppressesUnknownStructuredPayload() { + let json = """ + { + "foo": "bar" + } + """ + + let result = ToolResultTextFormatter.format(text: json, toolName: "nodes") + #expect(result.isEmpty) + } +} diff --git a/apps/shared/OpenClawKit/Tools/CanvasA2UI/bootstrap.js b/apps/shared/OpenClawKit/Tools/CanvasA2UI/bootstrap.js index a9cb659876a..530287ca21d 100644 --- a/apps/shared/OpenClawKit/Tools/CanvasA2UI/bootstrap.js +++ b/apps/shared/OpenClawKit/Tools/CanvasA2UI/bootstrap.js @@ -32,6 +32,66 @@ if (modalElement && Array.isArray(modalElement.styles)) { modalElement.styles = [...modalElement.styles, modalStyles]; } +const appendComponentStyles = (tagName, extraStyles) => { + const component = customElements.get(tagName); + if (!component) { + return; + } + + const current = component.styles; + if (!current) { + component.styles = [extraStyles]; + return; + } + + component.styles = Array.isArray(current) ? [...current, extraStyles] : [current, extraStyles]; +}; + +appendComponentStyles( + "a2ui-row", + css` + @media (max-width: 860px) { + section { + flex-wrap: wrap; + align-content: flex-start; + } + + ::slotted(*) { + flex: 1 1 100%; + min-width: 100%; + width: 100%; + max-width: 100%; + } + } + `, +); + +appendComponentStyles( + "a2ui-column", + css` + :host { + min-width: 0; + } + + section { + min-width: 0; + } + `, +); + +appendComponentStyles( + "a2ui-card", + css` + :host { + min-width: 0; + } + + section { + min-width: 0; + } + `, +); + const emptyClasses = () => ({}); const textHintStyles = () => ({ h1: {}, h2: {}, h3: {}, h4: {}, h5: {}, body: {}, caption: {} }); diff --git a/apps/shared/OpenClawKit/Tools/CanvasA2UI/rolldown.config.mjs b/apps/shared/OpenClawKit/Tools/CanvasA2UI/rolldown.config.mjs index dbd4b86fff6..ccf1683d565 100644 --- a/apps/shared/OpenClawKit/Tools/CanvasA2UI/rolldown.config.mjs +++ b/apps/shared/OpenClawKit/Tools/CanvasA2UI/rolldown.config.mjs @@ -1,9 +1,10 @@ import path from "node:path"; +import { existsSync } from "node:fs"; import { fileURLToPath } from "node:url"; -import { defineConfig } from "rolldown"; const here = path.dirname(fileURLToPath(import.meta.url)); const repoRoot = path.resolve(here, "../../../../.."); +const uiRoot = path.resolve(repoRoot, "ui"); const fromHere = (p) => path.resolve(here, p); const outputFile = path.resolve( here, @@ -16,8 +17,28 @@ const outputFile = path.resolve( const a2uiLitDist = path.resolve(repoRoot, "vendor/a2ui/renderers/lit/dist/src"); const a2uiThemeContext = path.resolve(a2uiLitDist, "0.8/ui/context/theme.js"); +const uiNodeModules = path.resolve(uiRoot, "node_modules"); +const repoNodeModules = path.resolve(repoRoot, "node_modules"); -export default defineConfig({ +function resolveUiDependency(moduleId) { + const candidates = [ + path.resolve(uiNodeModules, moduleId), + path.resolve(repoNodeModules, moduleId), + ]; + for (const candidate of candidates) { + if (existsSync(candidate)) { + return candidate; + } + } + + const fallbackCandidates = candidates.join(", "); + throw new Error( + `A2UI bundle config cannot resolve ${moduleId}. Checked: ${fallbackCandidates}. ` + + "Keep dependency installed in ui workspace or repo root before bundling.", + ); +} + +export default { input: fromHere("bootstrap.js"), experimental: { attachDebugInfo: "none", @@ -28,12 +49,13 @@ export default defineConfig({ "@a2ui/lit": path.resolve(a2uiLitDist, "index.js"), "@a2ui/lit/ui": path.resolve(a2uiLitDist, "0.8/ui/ui.js"), "@openclaw/a2ui-theme-context": a2uiThemeContext, - "@lit/context": path.resolve(repoRoot, "node_modules/@lit/context/index.js"), - "@lit/context/": path.resolve(repoRoot, "node_modules/@lit/context/"), - "@lit-labs/signals": path.resolve(repoRoot, "node_modules/@lit-labs/signals/index.js"), - "@lit-labs/signals/": path.resolve(repoRoot, "node_modules/@lit-labs/signals/"), - lit: path.resolve(repoRoot, "node_modules/lit/index.js"), - "lit/": path.resolve(repoRoot, "node_modules/lit/"), + "@lit/context": resolveUiDependency("@lit/context"), + "@lit/context/": resolveUiDependency("@lit/context/"), + "@lit-labs/signals": resolveUiDependency("@lit-labs/signals"), + "@lit-labs/signals/": resolveUiDependency("@lit-labs/signals/"), + lit: resolveUiDependency("lit"), + "lit/": resolveUiDependency("lit/"), + "signal-utils/": resolveUiDependency("signal-utils/"), }, }, output: { @@ -42,4 +64,4 @@ export default defineConfig({ codeSplitting: false, sourcemap: false, }, -}); +}; diff --git a/assets/chrome-extension/README.md b/assets/chrome-extension/README.md index 2a2a11a3be5..4ee072c1f2b 100644 --- a/assets/chrome-extension/README.md +++ b/assets/chrome-extension/README.md @@ -20,3 +20,4 @@ Purpose: attach OpenClaw to an existing Chrome tab so the Gateway can automate i ## Options - `Relay port`: defaults to `18792`. +- `Gateway token`: required. Set this to `gateway.auth.token` (or `OPENCLAW_GATEWAY_TOKEN`). diff --git a/assets/chrome-extension/background-utils.js b/assets/chrome-extension/background-utils.js new file mode 100644 index 00000000000..fe32d2c0616 --- /dev/null +++ b/assets/chrome-extension/background-utils.js @@ -0,0 +1,48 @@ +export function reconnectDelayMs( + attempt, + opts = { baseMs: 1000, maxMs: 30000, jitterMs: 1000, random: Math.random }, +) { + const baseMs = Number.isFinite(opts.baseMs) ? opts.baseMs : 1000; + const maxMs = Number.isFinite(opts.maxMs) ? opts.maxMs : 30000; + const jitterMs = Number.isFinite(opts.jitterMs) ? opts.jitterMs : 1000; + const random = typeof opts.random === "function" ? opts.random : Math.random; + const safeAttempt = Math.max(0, Number.isFinite(attempt) ? attempt : 0); + const backoff = Math.min(baseMs * 2 ** safeAttempt, maxMs); + return backoff + Math.max(0, jitterMs) * random(); +} + +export async function deriveRelayToken(gatewayToken, port) { + const enc = new TextEncoder(); + const key = await crypto.subtle.importKey( + "raw", + enc.encode(gatewayToken), + { name: "HMAC", hash: "SHA-256" }, + false, + ["sign"], + ); + const sig = await crypto.subtle.sign( + "HMAC", + key, + enc.encode(`openclaw-extension-relay-v1:${port}`), + ); + return [...new Uint8Array(sig)].map((b) => b.toString(16).padStart(2, "0")).join(""); +} + +export async function buildRelayWsUrl(port, gatewayToken) { + const token = String(gatewayToken || "").trim(); + if (!token) { + throw new Error( + "Missing gatewayToken in extension settings (chrome.storage.local.gatewayToken)", + ); + } + const relayToken = await deriveRelayToken(token, port); + return `ws://127.0.0.1:${port}/extension?token=${encodeURIComponent(relayToken)}`; +} + +export function isRetryableReconnectError(err) { + const message = err instanceof Error ? err.message : String(err || ""); + if (message.includes("Missing gatewayToken")) { + return false; + } + return true; +} diff --git a/assets/chrome-extension/background.js b/assets/chrome-extension/background.js index 31ba401bddc..c78f2c7c452 100644 --- a/assets/chrome-extension/background.js +++ b/assets/chrome-extension/background.js @@ -1,3 +1,5 @@ +import { buildRelayWsUrl, isRetryableReconnectError, reconnectDelayMs } from './background-utils.js' + const DEFAULT_PORT = 18792 const BADGE = { @@ -11,8 +13,9 @@ const BADGE = { let relayWs = null /** @type {Promise|null} */ let relayConnectPromise = null - -let debuggerListenersInstalled = false +let relayGatewayToken = '' +/** @type {string|null} */ +let relayConnectRequestId = null let nextSession = 1 @@ -26,6 +29,18 @@ const childSessionToTab = new Map() /** @type {Mapvoid, reject:(e:Error)=>void}>} */ const pending = new Map() +// Per-tab operation locks prevent double-attach races. +/** @type {Set} */ +const tabOperationLocks = new Set() + +// Tabs currently in a detach/re-attach cycle after navigation. +/** @type {Set} */ +const reattachPending = new Set() + +// Reconnect state for exponential backoff. +let reconnectAttempt = 0 +let reconnectTimer = null + function nowStack() { try { return new Error().stack || '' @@ -42,6 +57,12 @@ async function getRelayPort() { return n } +async function getGatewayToken() { + const stored = await chrome.storage.local.get(['gatewayToken']) + const token = String(stored.gatewayToken || '').trim() + return token || '' +} + function setBadge(tabId, kind) { const cfg = BADGE[kind] void chrome.action.setBadgeText({ tabId, text: cfg.text }) @@ -49,14 +70,72 @@ function setBadge(tabId, kind) { void chrome.action.setBadgeTextColor({ tabId, color: '#FFFFFF' }).catch(() => {}) } +// Persist attached tab state to survive MV3 service worker restarts. +async function persistState() { + try { + const tabEntries = [] + for (const [tabId, tab] of tabs.entries()) { + if (tab.state === 'connected' && tab.sessionId && tab.targetId) { + tabEntries.push({ tabId, sessionId: tab.sessionId, targetId: tab.targetId, attachOrder: tab.attachOrder }) + } + } + await chrome.storage.session.set({ + persistedTabs: tabEntries, + nextSession, + }) + } catch { + // chrome.storage.session may not be available in all contexts. + } +} + +// Rehydrate tab state on service worker startup. Fast path — just restores +// maps and badges. Relay reconnect happens separately in background. +async function rehydrateState() { + try { + const stored = await chrome.storage.session.get(['persistedTabs', 'nextSession']) + if (stored.nextSession) { + nextSession = Math.max(nextSession, stored.nextSession) + } + const entries = stored.persistedTabs || [] + // Phase 1: optimistically restore state and badges. + for (const entry of entries) { + tabs.set(entry.tabId, { + state: 'connected', + sessionId: entry.sessionId, + targetId: entry.targetId, + attachOrder: entry.attachOrder, + }) + tabBySession.set(entry.sessionId, entry.tabId) + setBadge(entry.tabId, 'on') + } + // Phase 2: validate asynchronously, remove dead tabs. + for (const entry of entries) { + try { + await chrome.tabs.get(entry.tabId) + await chrome.debugger.sendCommand({ tabId: entry.tabId }, 'Runtime.evaluate', { + expression: '1', + returnByValue: true, + }) + } catch { + tabs.delete(entry.tabId) + tabBySession.delete(entry.sessionId) + setBadge(entry.tabId, 'off') + } + } + } catch { + // Ignore rehydration errors. + } +} + async function ensureRelayConnection() { if (relayWs && relayWs.readyState === WebSocket.OPEN) return if (relayConnectPromise) return await relayConnectPromise relayConnectPromise = (async () => { const port = await getRelayPort() + const gatewayToken = await getGatewayToken() const httpBase = `http://127.0.0.1:${port}` - const wsUrl = `ws://127.0.0.1:${port}/extension` + const wsUrl = await buildRelayWsUrl(port, gatewayToken) // Fast preflight: is the relay server up? try { @@ -67,6 +146,13 @@ async function ensureRelayConnection() { const ws = new WebSocket(wsUrl) relayWs = ws + relayGatewayToken = gatewayToken + // Bind message handler before open so an immediate first frame (for example + // gateway connect.challenge) cannot be missed. + ws.onmessage = (event) => { + if (ws !== relayWs) return + void whenReady(() => onRelayMessage(String(event.data || ''))) + } await new Promise((resolve, reject) => { const t = setTimeout(() => reject(new Error('WebSocket connect timeout')), 5000) @@ -84,42 +170,142 @@ async function ensureRelayConnection() { } }) - ws.onmessage = (event) => void onRelayMessage(String(event.data || '')) - ws.onclose = () => onRelayClosed('closed') - ws.onerror = () => onRelayClosed('error') - - if (!debuggerListenersInstalled) { - debuggerListenersInstalled = true - chrome.debugger.onEvent.addListener(onDebuggerEvent) - chrome.debugger.onDetach.addListener(onDebuggerDetach) + // Bind permanent handlers. Guard against stale socket: if this WS was + // replaced before its close fires, the handler is a no-op. + ws.onclose = () => { + if (ws !== relayWs) return + onRelayClosed('closed') + } + ws.onerror = () => { + if (ws !== relayWs) return + onRelayClosed('error') } })() try { await relayConnectPromise + reconnectAttempt = 0 } finally { relayConnectPromise = null } } +// Relay closed — update badges, reject pending requests, auto-reconnect. +// Debugger sessions are kept alive so they survive transient WS drops. function onRelayClosed(reason) { relayWs = null + relayGatewayToken = '' + relayConnectRequestId = null + for (const [id, p] of pending.entries()) { pending.delete(id) p.reject(new Error(`Relay disconnected (${reason})`)) } - for (const tabId of tabs.keys()) { - void chrome.debugger.detach({ tabId }).catch(() => {}) - setBadge(tabId, 'connecting') - void chrome.action.setTitle({ - tabId, - title: 'OpenClaw Browser Relay: disconnected (click to re-attach)', - }) + reattachPending.clear() + + for (const [tabId, tab] of tabs.entries()) { + if (tab.state === 'connected') { + setBadge(tabId, 'connecting') + void chrome.action.setTitle({ + tabId, + title: 'OpenClaw Browser Relay: relay reconnecting…', + }) + } } - tabs.clear() - tabBySession.clear() - childSessionToTab.clear() + + scheduleReconnect() +} + +function scheduleReconnect() { + if (reconnectTimer) { + clearTimeout(reconnectTimer) + reconnectTimer = null + } + + const delay = reconnectDelayMs(reconnectAttempt) + reconnectAttempt++ + + console.log(`Scheduling reconnect attempt ${reconnectAttempt} in ${Math.round(delay)}ms`) + + reconnectTimer = setTimeout(async () => { + reconnectTimer = null + try { + await ensureRelayConnection() + reconnectAttempt = 0 + console.log('Reconnected successfully') + await reannounceAttachedTabs() + } catch (err) { + const message = err instanceof Error ? err.message : String(err) + console.warn(`Reconnect attempt ${reconnectAttempt} failed: ${message}`) + if (!isRetryableReconnectError(err)) { + return + } + scheduleReconnect() + } + }, delay) +} + +function cancelReconnect() { + if (reconnectTimer) { + clearTimeout(reconnectTimer) + reconnectTimer = null + } + reconnectAttempt = 0 +} + +// Re-announce all attached tabs to the relay after reconnect. +async function reannounceAttachedTabs() { + for (const [tabId, tab] of tabs.entries()) { + if (tab.state !== 'connected' || !tab.sessionId || !tab.targetId) continue + + // Verify debugger is still attached. + try { + await chrome.debugger.sendCommand({ tabId }, 'Runtime.evaluate', { + expression: '1', + returnByValue: true, + }) + } catch { + tabs.delete(tabId) + if (tab.sessionId) tabBySession.delete(tab.sessionId) + setBadge(tabId, 'off') + void chrome.action.setTitle({ + tabId, + title: 'OpenClaw Browser Relay (click to attach/detach)', + }) + continue + } + + // Send fresh attach event to relay. + try { + const info = /** @type {any} */ ( + await chrome.debugger.sendCommand({ tabId }, 'Target.getTargetInfo') + ) + const targetInfo = info?.targetInfo + + sendToRelay({ + method: 'forwardCDPEvent', + params: { + method: 'Target.attachedToTarget', + params: { + sessionId: tab.sessionId, + targetInfo: { ...targetInfo, attached: true }, + waitingForDebugger: false, + }, + }, + }) + + setBadge(tabId, 'on') + void chrome.action.setTitle({ + tabId, + title: 'OpenClaw Browser Relay: attached (click to detach)', + }) + } catch { + setBadge(tabId, 'on') + } + } + + await persistState() } function sendToRelay(payload) { @@ -130,6 +316,33 @@ function sendToRelay(payload) { ws.send(JSON.stringify(payload)) } +function ensureGatewayHandshakeStarted(payload) { + if (relayConnectRequestId) return + const nonce = typeof payload?.nonce === 'string' ? payload.nonce.trim() : '' + relayConnectRequestId = `ext-connect-${Date.now()}-${Math.random().toString(16).slice(2, 8)}` + sendToRelay({ + type: 'req', + id: relayConnectRequestId, + method: 'connect', + params: { + minProtocol: 3, + maxProtocol: 3, + client: { + id: 'chrome-relay-extension', + version: '1.0.0', + platform: 'chrome-extension', + mode: 'webchat', + }, + role: 'operator', + scopes: ['operator.read', 'operator.write'], + caps: [], + commands: [], + nonce: nonce || undefined, + auth: relayGatewayToken ? { token: relayGatewayToken } : undefined, + }, + }) +} + async function maybeOpenHelpOnce() { try { const stored = await chrome.storage.local.get(['helpOnErrorShown']) @@ -144,10 +357,18 @@ async function maybeOpenHelpOnce() { function requestFromRelay(command) { const id = command.id return new Promise((resolve, reject) => { - pending.set(id, { resolve, reject }) + const timer = setTimeout(() => { + pending.delete(id) + reject(new Error('Relay request timeout (30s)')) + }, 30000) + pending.set(id, { + resolve: (v) => { clearTimeout(timer); resolve(v) }, + reject: (e) => { clearTimeout(timer); reject(e) }, + }) try { sendToRelay(command) } catch (err) { + clearTimeout(timer) pending.delete(id) reject(err instanceof Error ? err : new Error(String(err))) } @@ -163,6 +384,33 @@ async function onRelayMessage(text) { return } + if (msg && msg.type === 'event' && msg.event === 'connect.challenge') { + try { + ensureGatewayHandshakeStarted(msg.payload) + } catch (err) { + console.warn('gateway connect handshake start failed', err instanceof Error ? err.message : String(err)) + relayConnectRequestId = null + const ws = relayWs + if (ws && ws.readyState === WebSocket.OPEN) { + ws.close(1008, 'gateway connect failed') + } + } + return + } + + if (msg && msg.type === 'res' && relayConnectRequestId && msg.id === relayConnectRequestId) { + relayConnectRequestId = null + if (!msg.ok) { + const detail = msg?.error?.message || msg?.error || 'gateway connect failed' + console.warn('gateway connect handshake rejected', String(detail)) + const ws = relayWs + if (ws && ws.readyState === WebSocket.OPEN) { + ws.close(1008, 'gateway connect failed') + } + } + return + } + if (msg && msg.method === 'ping') { try { sendToRelay({ method: 'pong' }) @@ -218,8 +466,9 @@ async function attachTab(tabId, opts = {}) { throw new Error('Target.getTargetInfo returned no targetId') } - const sessionId = `cb-tab-${nextSession++}` - const attachOrder = nextSession + const sid = nextSession++ + const sessionId = `cb-tab-${sid}` + const attachOrder = sid tabs.set(tabId, { state: 'connected', sessionId, targetId, attachOrder }) tabBySession.set(sessionId, tabId) @@ -243,11 +492,33 @@ async function attachTab(tabId, opts = {}) { } setBadge(tabId, 'on') + await persistState() + return { sessionId, targetId } } async function detachTab(tabId, reason) { const tab = tabs.get(tabId) + + // Send detach events for child sessions first. + for (const [childSessionId, parentTabId] of childSessionToTab.entries()) { + if (parentTabId === tabId) { + try { + sendToRelay({ + method: 'forwardCDPEvent', + params: { + method: 'Target.detachedFromTarget', + params: { sessionId: childSessionId, reason: 'parent_detached' }, + }, + }) + } catch { + // Relay may be down. + } + childSessionToTab.delete(childSessionId) + } + } + + // Send detach event for main session. if (tab?.sessionId && tab?.targetId) { try { sendToRelay({ @@ -258,21 +529,17 @@ async function detachTab(tabId, reason) { }, }) } catch { - // ignore + // Relay may be down. } } if (tab?.sessionId) tabBySession.delete(tab.sessionId) tabs.delete(tabId) - for (const [childSessionId, parentTabId] of childSessionToTab.entries()) { - if (parentTabId === tabId) childSessionToTab.delete(childSessionId) - } - try { await chrome.debugger.detach({ tabId }) } catch { - // ignore + // May already be detached. } setBadge(tabId, 'off') @@ -280,6 +547,8 @@ async function detachTab(tabId, reason) { tabId, title: 'OpenClaw Browser Relay (click to attach/detach)', }) + + await persistState() } async function connectOrToggleForActiveTab() { @@ -287,33 +556,53 @@ async function connectOrToggleForActiveTab() { const tabId = active?.id if (!tabId) return - const existing = tabs.get(tabId) - if (existing?.state === 'connected') { - await detachTab(tabId, 'toggle') - return - } - - tabs.set(tabId, { state: 'connecting' }) - setBadge(tabId, 'connecting') - void chrome.action.setTitle({ - tabId, - title: 'OpenClaw Browser Relay: connecting to local relay…', - }) + // Prevent concurrent operations on the same tab. + if (tabOperationLocks.has(tabId)) return + tabOperationLocks.add(tabId) try { - await ensureRelayConnection() - await attachTab(tabId) - } catch (err) { - tabs.delete(tabId) - setBadge(tabId, 'error') + if (reattachPending.has(tabId)) { + reattachPending.delete(tabId) + setBadge(tabId, 'off') + void chrome.action.setTitle({ + tabId, + title: 'OpenClaw Browser Relay (click to attach/detach)', + }) + return + } + + const existing = tabs.get(tabId) + if (existing?.state === 'connected') { + await detachTab(tabId, 'toggle') + return + } + + // User is manually connecting — cancel any pending reconnect. + cancelReconnect() + + tabs.set(tabId, { state: 'connecting' }) + setBadge(tabId, 'connecting') void chrome.action.setTitle({ tabId, - title: 'OpenClaw Browser Relay: relay not running (open options for setup)', + title: 'OpenClaw Browser Relay: connecting to local relay…', }) - void maybeOpenHelpOnce() - // Extra breadcrumbs in chrome://extensions service worker logs. - const message = err instanceof Error ? err.message : String(err) - console.warn('attach failed', message, nowStack()) + + try { + await ensureRelayConnection() + await attachTab(tabId) + } catch (err) { + tabs.delete(tabId) + setBadge(tabId, 'error') + void chrome.action.setTitle({ + tabId, + title: 'OpenClaw Browser Relay: relay not running (open options for setup)', + }) + void maybeOpenHelpOnce() + const message = err instanceof Error ? err.message : String(err) + console.warn('attach failed', message, nowStack()) + } + } finally { + tabOperationLocks.delete(tabId) } } @@ -322,14 +611,12 @@ async function handleForwardCdpCommand(msg) { const params = msg?.params?.params || undefined const sessionId = typeof msg?.params?.sessionId === 'string' ? msg.params.sessionId : undefined - // Map command to tab const bySession = sessionId ? getTabBySessionId(sessionId) : null const targetId = typeof params?.targetId === 'string' ? params.targetId : undefined const tabId = bySession?.tabId || (targetId ? getTabByTargetId(targetId) : null) || (() => { - // No sessionId: pick the first connected tab (stable-ish). for (const [id, tab] of tabs.entries()) { if (tab.state === 'connected') return id } @@ -419,20 +706,256 @@ function onDebuggerEvent(source, method, params) { }, }) } catch { - // ignore + // Relay may be down. } } -function onDebuggerDetach(source, reason) { +async function onDebuggerDetach(source, reason) { const tabId = source.tabId if (!tabId) return if (!tabs.has(tabId)) return - void detachTab(tabId, reason) + + // User explicitly cancelled or DevTools replaced the connection — respect their intent + if (reason === 'canceled_by_user' || reason === 'replaced_with_devtools') { + void detachTab(tabId, reason) + return + } + + // Check if tab still exists — distinguishes navigation from tab close + let tabInfo + try { + tabInfo = await chrome.tabs.get(tabId) + } catch { + // Tab is gone (closed) — normal cleanup + void detachTab(tabId, reason) + return + } + + if (tabInfo.url?.startsWith('chrome://') || tabInfo.url?.startsWith('chrome-extension://')) { + void detachTab(tabId, reason) + return + } + + if (reattachPending.has(tabId)) return + + const oldTab = tabs.get(tabId) + const oldSessionId = oldTab?.sessionId + const oldTargetId = oldTab?.targetId + + if (oldSessionId) tabBySession.delete(oldSessionId) + tabs.delete(tabId) + for (const [childSessionId, parentTabId] of childSessionToTab.entries()) { + if (parentTabId === tabId) childSessionToTab.delete(childSessionId) + } + + if (oldSessionId && oldTargetId) { + try { + sendToRelay({ + method: 'forwardCDPEvent', + params: { + method: 'Target.detachedFromTarget', + params: { sessionId: oldSessionId, targetId: oldTargetId, reason: 'navigation-reattach' }, + }, + }) + } catch { + // Relay may be down. + } + } + + reattachPending.add(tabId) + setBadge(tabId, 'connecting') + void chrome.action.setTitle({ + tabId, + title: 'OpenClaw Browser Relay: re-attaching after navigation…', + }) + + const delays = [300, 700, 1500] + for (let attempt = 0; attempt < delays.length; attempt++) { + await new Promise((r) => setTimeout(r, delays[attempt])) + + if (!reattachPending.has(tabId)) return + + try { + await chrome.tabs.get(tabId) + } catch { + reattachPending.delete(tabId) + setBadge(tabId, 'off') + return + } + + if (!relayWs || relayWs.readyState !== WebSocket.OPEN) { + reattachPending.delete(tabId) + setBadge(tabId, 'error') + void chrome.action.setTitle({ + tabId, + title: 'OpenClaw Browser Relay: relay disconnected during re-attach', + }) + return + } + + try { + await attachTab(tabId) + reattachPending.delete(tabId) + return + } catch { + // continue retries + } + } + + reattachPending.delete(tabId) + setBadge(tabId, 'off') + void chrome.action.setTitle({ + tabId, + title: 'OpenClaw Browser Relay: re-attach failed (click to retry)', + }) } -chrome.action.onClicked.addListener(() => void connectOrToggleForActiveTab()) +// Tab lifecycle listeners — clean up stale entries. +chrome.tabs.onRemoved.addListener((tabId) => void whenReady(() => { + reattachPending.delete(tabId) + if (!tabs.has(tabId)) return + const tab = tabs.get(tabId) + if (tab?.sessionId) tabBySession.delete(tab.sessionId) + tabs.delete(tabId) + for (const [childSessionId, parentTabId] of childSessionToTab.entries()) { + if (parentTabId === tabId) childSessionToTab.delete(childSessionId) + } + if (tab?.sessionId && tab?.targetId) { + try { + sendToRelay({ + method: 'forwardCDPEvent', + params: { + method: 'Target.detachedFromTarget', + params: { sessionId: tab.sessionId, targetId: tab.targetId, reason: 'tab_closed' }, + }, + }) + } catch { + // Relay may be down. + } + } + void persistState() +})) + +chrome.tabs.onReplaced.addListener((addedTabId, removedTabId) => void whenReady(() => { + const tab = tabs.get(removedTabId) + if (!tab) return + tabs.delete(removedTabId) + tabs.set(addedTabId, tab) + if (tab.sessionId) { + tabBySession.set(tab.sessionId, addedTabId) + } + for (const [childSessionId, parentTabId] of childSessionToTab.entries()) { + if (parentTabId === removedTabId) { + childSessionToTab.set(childSessionId, addedTabId) + } + } + setBadge(addedTabId, 'on') + void persistState() +})) + +// Register debugger listeners at module scope so detach/event handling works +// even when the relay WebSocket is down. +chrome.debugger.onEvent.addListener((...args) => void whenReady(() => onDebuggerEvent(...args))) +chrome.debugger.onDetach.addListener((...args) => void whenReady(() => onDebuggerDetach(...args))) + +chrome.action.onClicked.addListener(() => void whenReady(() => connectOrToggleForActiveTab())) + +// Refresh badge after navigation completes — service worker may have restarted +// during navigation, losing ephemeral badge state. +chrome.webNavigation.onCompleted.addListener(({ tabId, frameId }) => void whenReady(() => { + if (frameId !== 0) return + const tab = tabs.get(tabId) + if (tab?.state === 'connected') { + setBadge(tabId, relayWs && relayWs.readyState === WebSocket.OPEN ? 'on' : 'connecting') + } +})) + +// Refresh badge when user switches to an attached tab. +chrome.tabs.onActivated.addListener(({ tabId }) => void whenReady(() => { + const tab = tabs.get(tabId) + if (tab?.state === 'connected') { + setBadge(tabId, relayWs && relayWs.readyState === WebSocket.OPEN ? 'on' : 'connecting') + } +})) chrome.runtime.onInstalled.addListener(() => { - // Useful: first-time instructions. void chrome.runtime.openOptionsPage() }) + +// MV3 keepalive via chrome.alarms — more reliable than setInterval across +// service worker restarts. Checks relay health and refreshes badges. +chrome.alarms.create('relay-keepalive', { periodInMinutes: 0.5 }) + +chrome.alarms.onAlarm.addListener(async (alarm) => { + if (alarm.name !== 'relay-keepalive') return + await initPromise + + if (tabs.size === 0) return + + // Refresh badges (ephemeral in MV3). + for (const [tabId, tab] of tabs.entries()) { + if (tab.state === 'connected') { + setBadge(tabId, relayWs && relayWs.readyState === WebSocket.OPEN ? 'on' : 'connecting') + } + } + + // If relay is down and no reconnect is in progress, trigger one. + if (!relayWs || relayWs.readyState !== WebSocket.OPEN) { + if (!relayConnectPromise && !reconnectTimer) { + console.log('Keepalive: WebSocket unhealthy, triggering reconnect') + await ensureRelayConnection().catch(() => { + // ensureRelayConnection may throw without triggering onRelayClosed + // (e.g. preflight fetch fails before WS is created), so ensure + // reconnect is always scheduled on failure. + if (!reconnectTimer) { + scheduleReconnect() + } + }) + } + } +}) + +// Rehydrate state on service worker startup. Split: rehydration is the gate +// (fast), relay reconnect runs in background (slow, non-blocking). +const initPromise = rehydrateState() + +initPromise.then(() => { + if (tabs.size > 0) { + ensureRelayConnection().then(() => { + reconnectAttempt = 0 + return reannounceAttachedTabs() + }).catch(() => { + scheduleReconnect() + }) + } +}) + +// Shared gate: all state-dependent handlers await this before accessing maps. +async function whenReady(fn) { + await initPromise + return fn() +} + +// Relay check handler for the options page. The service worker has +// host_permissions and bypasses CORS preflight, so the options page +// delegates token-validation requests here. +chrome.runtime.onMessage.addListener((msg, _sender, sendResponse) => { + if (msg?.type !== 'relayCheck') return false + const { url, token } = msg + const headers = token ? { 'x-openclaw-relay-token': token } : {} + fetch(url, { method: 'GET', headers, signal: AbortSignal.timeout(2000) }) + .then(async (res) => { + const contentType = String(res.headers.get('content-type') || '') + let json = null + if (contentType.includes('application/json')) { + try { + json = await res.json() + } catch { + json = null + } + } + sendResponse({ status: res.status, ok: res.ok, contentType, json }) + }) + .catch((err) => sendResponse({ status: 0, ok: false, error: String(err) })) + return true +}) diff --git a/assets/chrome-extension/manifest.json b/assets/chrome-extension/manifest.json index d6b593990de..62038276cd7 100644 --- a/assets/chrome-extension/manifest.json +++ b/assets/chrome-extension/manifest.json @@ -9,7 +9,7 @@ "48": "icons/icon48.png", "128": "icons/icon128.png" }, - "permissions": ["debugger", "tabs", "activeTab", "storage"], + "permissions": ["debugger", "tabs", "activeTab", "storage", "alarms", "webNavigation"], "host_permissions": ["http://127.0.0.1/*", "http://localhost/*"], "background": { "service_worker": "background.js", "type": "module" }, "action": { diff --git a/assets/chrome-extension/options-validation.js b/assets/chrome-extension/options-validation.js new file mode 100644 index 00000000000..53e2cd55014 --- /dev/null +++ b/assets/chrome-extension/options-validation.js @@ -0,0 +1,57 @@ +const PORT_GUIDANCE = 'Use gateway port + 3 (for gateway 18789, relay is 18792).' + +function hasCdpVersionShape(data) { + return !!data && typeof data === 'object' && 'Browser' in data && 'Protocol-Version' in data +} + +export function classifyRelayCheckResponse(res, port) { + if (!res) { + return { action: 'throw', error: 'No response from service worker' } + } + + if (res.status === 401) { + return { action: 'status', kind: 'error', message: 'Gateway token rejected. Check token and save again.' } + } + + if (res.error) { + return { action: 'throw', error: res.error } + } + + if (!res.ok) { + return { action: 'throw', error: `HTTP ${res.status}` } + } + + const contentType = String(res.contentType || '') + if (!contentType.includes('application/json')) { + return { + action: 'status', + kind: 'error', + message: `Wrong port: this is likely the gateway, not the relay. ${PORT_GUIDANCE}`, + } + } + + if (!hasCdpVersionShape(res.json)) { + return { + action: 'status', + kind: 'error', + message: `Wrong port: expected relay /json/version response. ${PORT_GUIDANCE}`, + } + } + + return { action: 'status', kind: 'ok', message: `Relay reachable and authenticated at http://127.0.0.1:${port}/` } +} + +export function classifyRelayCheckException(err, port) { + const message = String(err || '').toLowerCase() + if (message.includes('json') || message.includes('syntax')) { + return { + kind: 'error', + message: `Wrong port: this is not a relay endpoint. ${PORT_GUIDANCE}`, + } + } + + return { + kind: 'error', + message: `Relay not reachable/authenticated at http://127.0.0.1:${port}/. Start OpenClaw browser relay and verify token.`, + } +} diff --git a/assets/chrome-extension/options.html b/assets/chrome-extension/options.html index 14704d65cf0..17fc6a79eed 100644 --- a/assets/chrome-extension/options.html +++ b/assets/chrome-extension/options.html @@ -176,15 +176,19 @@
-

Relay port

+

Relay connection

+
+ +
+
- Default: 18792. Extension connects to: http://127.0.0.1:<port>/. - Only change this if your OpenClaw profile uses a different cdpUrl port. + Default port: 18792. Extension connects to: http://127.0.0.1:<port>/. + Gateway token must match gateway.auth.token (or OPENCLAW_GATEWAY_TOKEN).
diff --git a/assets/chrome-extension/options.js b/assets/chrome-extension/options.js index 5b558ddccf2..aa6fcc4901f 100644 --- a/assets/chrome-extension/options.js +++ b/assets/chrome-extension/options.js @@ -1,3 +1,6 @@ +import { deriveRelayToken } from './background-utils.js' +import { classifyRelayCheckException, classifyRelayCheckResponse } from './options-validation.js' + const DEFAULT_PORT = 18792 function clampPort(value) { @@ -20,39 +23,51 @@ function setStatus(kind, message) { status.textContent = message || '' } -async function checkRelayReachable(port) { - const url = `http://127.0.0.1:${port}/` - const ctrl = new AbortController() - const t = setTimeout(() => ctrl.abort(), 900) +async function checkRelayReachable(port, token) { + const url = `http://127.0.0.1:${port}/json/version` + const trimmedToken = String(token || '').trim() + if (!trimmedToken) { + setStatus('error', 'Gateway token required. Save your gateway token to connect.') + return + } try { - const res = await fetch(url, { method: 'HEAD', signal: ctrl.signal }) - if (!res.ok) throw new Error(`HTTP ${res.status}`) - setStatus('ok', `Relay reachable at ${url}`) - } catch { - setStatus( - 'error', - `Relay not reachable at ${url}. Start OpenClaw’s browser relay on this machine, then click the toolbar button again.`, - ) - } finally { - clearTimeout(t) + const relayToken = await deriveRelayToken(trimmedToken, port) + // Delegate the fetch to the background service worker to bypass + // CORS preflight on the custom x-openclaw-relay-token header. + const res = await chrome.runtime.sendMessage({ + type: 'relayCheck', + url, + token: relayToken, + }) + const result = classifyRelayCheckResponse(res, port) + if (result.action === 'throw') throw new Error(result.error) + setStatus(result.kind, result.message) + } catch (err) { + const result = classifyRelayCheckException(err, port) + setStatus(result.kind, result.message) } } async function load() { - const stored = await chrome.storage.local.get(['relayPort']) + const stored = await chrome.storage.local.get(['relayPort', 'gatewayToken']) const port = clampPort(stored.relayPort) + const token = String(stored.gatewayToken || '').trim() document.getElementById('port').value = String(port) + document.getElementById('token').value = token updateRelayUrl(port) - await checkRelayReachable(port) + await checkRelayReachable(port, token) } async function save() { - const input = document.getElementById('port') - const port = clampPort(input.value) - await chrome.storage.local.set({ relayPort: port }) - input.value = String(port) + const portInput = document.getElementById('port') + const tokenInput = document.getElementById('token') + const port = clampPort(portInput.value) + const token = String(tokenInput.value || '').trim() + await chrome.storage.local.set({ relayPort: port, gatewayToken: token }) + portInput.value = String(port) + tokenInput.value = token updateRelayUrl(port) - await checkRelayReachable(port) + await checkRelayReachable(port, token) } document.getElementById('save').addEventListener('click', () => void save()) diff --git a/changelog/fragments/README.md b/changelog/fragments/README.md new file mode 100644 index 00000000000..93bb5b65d70 --- /dev/null +++ b/changelog/fragments/README.md @@ -0,0 +1,13 @@ +# Changelog Fragments + +Use this directory when a PR should not edit `CHANGELOG.md` directly. + +- One fragment file per PR. +- File name recommendation: `pr-.md`. +- Include at least one line with both `#` and `thanks @`. + +Example: + +```md +- Fix LINE monitor lifecycle wait ownership (#27001) (thanks @alice) +``` diff --git a/docker-setup.sh b/docker-setup.sh index 1d2f5e53fd1..1f6e51cd75d 100755 --- a/docker-setup.sh +++ b/docker-setup.sh @@ -8,6 +8,11 @@ IMAGE_NAME="${OPENCLAW_IMAGE:-openclaw:local}" EXTRA_MOUNTS="${OPENCLAW_EXTRA_MOUNTS:-}" HOME_VOLUME_NAME="${OPENCLAW_HOME_VOLUME:-}" +fail() { + echo "ERROR: $*" >&2 + exit 1 +} + require_cmd() { if ! command -v "$1" >/dev/null 2>&1; then echo "Missing dependency: $1" >&2 @@ -15,6 +20,116 @@ require_cmd() { fi } +read_config_gateway_token() { + local config_path="$OPENCLAW_CONFIG_DIR/openclaw.json" + if [[ ! -f "$config_path" ]]; then + return 0 + fi + if command -v python3 >/dev/null 2>&1; then + python3 - "$config_path" <<'PY' +import json +import sys + +path = sys.argv[1] +try: + with open(path, "r", encoding="utf-8") as f: + cfg = json.load(f) +except Exception: + raise SystemExit(0) + +gateway = cfg.get("gateway") +if not isinstance(gateway, dict): + raise SystemExit(0) +auth = gateway.get("auth") +if not isinstance(auth, dict): + raise SystemExit(0) +token = auth.get("token") +if isinstance(token, str): + token = token.strip() + if token: + print(token) +PY + return 0 + fi + if command -v node >/dev/null 2>&1; then + node - "$config_path" <<'NODE' +const fs = require("node:fs"); +const configPath = process.argv[2]; +try { + const cfg = JSON.parse(fs.readFileSync(configPath, "utf8")); + const token = cfg?.gateway?.auth?.token; + if (typeof token === "string" && token.trim().length > 0) { + process.stdout.write(token.trim()); + } +} catch { + // Keep docker-setup resilient when config parsing fails. +} +NODE + fi +} + +ensure_control_ui_allowed_origins() { + if [[ "${OPENCLAW_GATEWAY_BIND}" == "loopback" ]]; then + return 0 + fi + + local allowed_origin_json + local current_allowed_origins + allowed_origin_json="$(printf '["http://127.0.0.1:%s"]' "$OPENCLAW_GATEWAY_PORT")" + current_allowed_origins="$( + docker compose "${COMPOSE_ARGS[@]}" run --rm openclaw-cli \ + config get gateway.controlUi.allowedOrigins 2>/dev/null || true + )" + current_allowed_origins="${current_allowed_origins//$'\r'/}" + + if [[ -n "$current_allowed_origins" && "$current_allowed_origins" != "null" && "$current_allowed_origins" != "[]" ]]; then + echo "Control UI allowlist already configured; leaving gateway.controlUi.allowedOrigins unchanged." + return 0 + fi + + docker compose "${COMPOSE_ARGS[@]}" run --rm openclaw-cli \ + config set gateway.controlUi.allowedOrigins "$allowed_origin_json" --strict-json >/dev/null + echo "Set gateway.controlUi.allowedOrigins to $allowed_origin_json for non-loopback bind." +} + +contains_disallowed_chars() { + local value="$1" + [[ "$value" == *$'\n'* || "$value" == *$'\r'* || "$value" == *$'\t'* ]] +} + +validate_mount_path_value() { + local label="$1" + local value="$2" + if [[ -z "$value" ]]; then + fail "$label cannot be empty." + fi + if contains_disallowed_chars "$value"; then + fail "$label contains unsupported control characters." + fi + if [[ "$value" =~ [[:space:]] ]]; then + fail "$label cannot contain whitespace." + fi +} + +validate_named_volume() { + local value="$1" + if [[ ! "$value" =~ ^[A-Za-z0-9][A-Za-z0-9_.-]*$ ]]; then + fail "OPENCLAW_HOME_VOLUME must match [A-Za-z0-9][A-Za-z0-9_.-]* when using a named volume." + fi +} + +validate_mount_spec() { + local mount="$1" + if contains_disallowed_chars "$mount"; then + fail "OPENCLAW_EXTRA_MOUNTS entries cannot contain control characters." + fi + # Keep mount specs strict to avoid YAML structure injection. + # Expected format: source:target[:options] + if [[ ! "$mount" =~ ^[^[:space:],:]+:[^[:space:],:]+(:[^[:space:],:]+)?$ ]]; then + fail "Invalid mount format '$mount'. Expected source:target[:options] without spaces." + fi +} + require_cmd docker if ! docker compose version >/dev/null 2>&1; then echo "Docker Compose not available (try: docker compose version)" >&2 @@ -24,8 +139,24 @@ fi OPENCLAW_CONFIG_DIR="${OPENCLAW_CONFIG_DIR:-$HOME/.openclaw}" OPENCLAW_WORKSPACE_DIR="${OPENCLAW_WORKSPACE_DIR:-$HOME/.openclaw/workspace}" +validate_mount_path_value "OPENCLAW_CONFIG_DIR" "$OPENCLAW_CONFIG_DIR" +validate_mount_path_value "OPENCLAW_WORKSPACE_DIR" "$OPENCLAW_WORKSPACE_DIR" +if [[ -n "$HOME_VOLUME_NAME" ]]; then + if [[ "$HOME_VOLUME_NAME" == *"/"* ]]; then + validate_mount_path_value "OPENCLAW_HOME_VOLUME" "$HOME_VOLUME_NAME" + else + validate_named_volume "$HOME_VOLUME_NAME" + fi +fi +if contains_disallowed_chars "$EXTRA_MOUNTS"; then + fail "OPENCLAW_EXTRA_MOUNTS cannot contain control characters." +fi + mkdir -p "$OPENCLAW_CONFIG_DIR" mkdir -p "$OPENCLAW_WORKSPACE_DIR" +# Seed device-identity parent eagerly for Docker Desktop/Windows bind mounts +# that reject creating new subdirectories from inside the container. +mkdir -p "$OPENCLAW_CONFIG_DIR/identity" export OPENCLAW_CONFIG_DIR export OPENCLAW_WORKSPACE_DIR @@ -38,7 +169,11 @@ export OPENCLAW_EXTRA_MOUNTS="$EXTRA_MOUNTS" export OPENCLAW_HOME_VOLUME="$HOME_VOLUME_NAME" if [[ -z "${OPENCLAW_GATEWAY_TOKEN:-}" ]]; then - if command -v openssl >/dev/null 2>&1; then + EXISTING_CONFIG_TOKEN="$(read_config_gateway_token || true)" + if [[ -n "$EXISTING_CONFIG_TOKEN" ]]; then + OPENCLAW_GATEWAY_TOKEN="$EXISTING_CONFIG_TOKEN" + echo "Reusing gateway token from $OPENCLAW_CONFIG_DIR/openclaw.json" + elif command -v openssl >/dev/null 2>&1; then OPENCLAW_GATEWAY_TOKEN="$(openssl rand -hex 32)" else OPENCLAW_GATEWAY_TOKEN="$(python3 - <<'PY' @@ -57,6 +192,9 @@ write_extra_compose() { local home_volume="$1" shift local mount + local gateway_home_mount + local gateway_config_mount + local gateway_workspace_mount cat >"$EXTRA_COMPOSE_FILE" <<'YAML' services: @@ -65,12 +203,19 @@ services: YAML if [[ -n "$home_volume" ]]; then - printf ' - %s:/home/node\n' "$home_volume" >>"$EXTRA_COMPOSE_FILE" - printf ' - %s:/home/node/.openclaw\n' "$OPENCLAW_CONFIG_DIR" >>"$EXTRA_COMPOSE_FILE" - printf ' - %s:/home/node/.openclaw/workspace\n' "$OPENCLAW_WORKSPACE_DIR" >>"$EXTRA_COMPOSE_FILE" + gateway_home_mount="${home_volume}:/home/node" + gateway_config_mount="${OPENCLAW_CONFIG_DIR}:/home/node/.openclaw" + gateway_workspace_mount="${OPENCLAW_WORKSPACE_DIR}:/home/node/.openclaw/workspace" + validate_mount_spec "$gateway_home_mount" + validate_mount_spec "$gateway_config_mount" + validate_mount_spec "$gateway_workspace_mount" + printf ' - %s\n' "$gateway_home_mount" >>"$EXTRA_COMPOSE_FILE" + printf ' - %s\n' "$gateway_config_mount" >>"$EXTRA_COMPOSE_FILE" + printf ' - %s\n' "$gateway_workspace_mount" >>"$EXTRA_COMPOSE_FILE" fi for mount in "$@"; do + validate_mount_spec "$mount" printf ' - %s\n' "$mount" >>"$EXTRA_COMPOSE_FILE" done @@ -80,16 +225,18 @@ YAML YAML if [[ -n "$home_volume" ]]; then - printf ' - %s:/home/node\n' "$home_volume" >>"$EXTRA_COMPOSE_FILE" - printf ' - %s:/home/node/.openclaw\n' "$OPENCLAW_CONFIG_DIR" >>"$EXTRA_COMPOSE_FILE" - printf ' - %s:/home/node/.openclaw/workspace\n' "$OPENCLAW_WORKSPACE_DIR" >>"$EXTRA_COMPOSE_FILE" + printf ' - %s\n' "$gateway_home_mount" >>"$EXTRA_COMPOSE_FILE" + printf ' - %s\n' "$gateway_config_mount" >>"$EXTRA_COMPOSE_FILE" + printf ' - %s\n' "$gateway_workspace_mount" >>"$EXTRA_COMPOSE_FILE" fi for mount in "$@"; do + validate_mount_spec "$mount" printf ' - %s\n' "$mount" >>"$EXTRA_COMPOSE_FILE" done if [[ -n "$home_volume" && "$home_volume" != *"/"* ]]; then + validate_named_volume "$home_volume" cat >>"$EXTRA_COMPOSE_FILE" < Building Docker image: $IMAGE_NAME" -docker build \ - --build-arg "OPENCLAW_DOCKER_APT_PACKAGES=${OPENCLAW_DOCKER_APT_PACKAGES}" \ - -t "$IMAGE_NAME" \ - -f "$ROOT_DIR/Dockerfile" \ - "$ROOT_DIR" +if [[ "$IMAGE_NAME" == "openclaw:local" ]]; then + echo "==> Building Docker image: $IMAGE_NAME" + docker build \ + --build-arg "OPENCLAW_DOCKER_APT_PACKAGES=${OPENCLAW_DOCKER_APT_PACKAGES}" \ + -t "$IMAGE_NAME" \ + -f "$ROOT_DIR/Dockerfile" \ + "$ROOT_DIR" +else + echo "==> Pulling Docker image: $IMAGE_NAME" + if ! docker pull "$IMAGE_NAME"; then + echo "ERROR: Failed to pull image $IMAGE_NAME. Please check the image name and your access permissions." >&2 + exit 1 + fi +fi echo "" echo "==> Onboarding (interactive)" @@ -194,6 +349,10 @@ echo " - Install Gateway daemon: No" echo "" docker compose "${COMPOSE_ARGS[@]}" run --rm openclaw-cli onboard --no-install-daemon +echo "" +echo "==> Control UI origin allowlist" +ensure_control_ui_allowed_origins + echo "" echo "==> Provider setup (optional)" echo "WhatsApp (QR):" diff --git a/docs/assets/sponsors/blacksmith.svg b/docs/assets/sponsors/blacksmith.svg new file mode 100644 index 00000000000..5bb1bc2e72c --- /dev/null +++ b/docs/assets/sponsors/blacksmith.svg @@ -0,0 +1,14 @@ + + + + + + + + + + + + + + diff --git a/docs/assets/sponsors/convex.svg b/docs/assets/sponsors/convex.svg new file mode 100644 index 00000000000..bd884e9ba65 --- /dev/null +++ b/docs/assets/sponsors/convex.svg @@ -0,0 +1,16 @@ + + + + + + + + + + + + + + + + diff --git a/docs/assets/sponsors/openai.svg b/docs/assets/sponsors/openai.svg new file mode 100644 index 00000000000..1c3491b9be9 --- /dev/null +++ b/docs/assets/sponsors/openai.svg @@ -0,0 +1,3 @@ + + + diff --git a/docs/automation/cron-jobs.md b/docs/automation/cron-jobs.md index aae5f58fdf2..8d140192607 100644 --- a/docs/automation/cron-jobs.md +++ b/docs/automation/cron-jobs.md @@ -349,7 +349,8 @@ Notes: ## Storage & history - Job store: `~/.openclaw/cron/jobs.json` (Gateway-managed JSON). -- Run history: `~/.openclaw/cron/runs/.jsonl` (JSONL, auto-pruned). +- Run history: `~/.openclaw/cron/runs/.jsonl` (JSONL, auto-pruned by size and line count). +- Isolated cron run sessions in `sessions.json` are pruned by `cron.sessionRetention` (default `24h`; set `false` to disable). - Override store path: `cron.store` in config. ## Configuration @@ -362,10 +363,21 @@ Notes: maxConcurrentRuns: 1, // default 1 webhook: "https://example.invalid/legacy", // deprecated fallback for stored notify:true jobs webhookToken: "replace-with-dedicated-webhook-token", // optional bearer token for webhook mode + sessionRetention: "24h", // duration string or false + runLog: { + maxBytes: "2mb", // default 2_000_000 bytes + keepLines: 2000, // default 2000 + }, }, } ``` +Run-log pruning behavior: + +- `cron.runLog.maxBytes`: max run-log file size before pruning. +- `cron.runLog.keepLines`: when pruning, keep only the newest N lines. +- Both apply to `cron/runs/.jsonl` files. + Webhook behavior: - Preferred: set `delivery.mode: "webhook"` with `delivery.to: "https://..."` per job. @@ -380,6 +392,85 @@ Disable cron entirely: - `cron.enabled: false` (config) - `OPENCLAW_SKIP_CRON=1` (env) +## Maintenance + +Cron has two built-in maintenance paths: isolated run-session retention and run-log pruning. + +### Defaults + +- `cron.sessionRetention`: `24h` (set `false` to disable run-session pruning) +- `cron.runLog.maxBytes`: `2_000_000` bytes +- `cron.runLog.keepLines`: `2000` + +### How it works + +- Isolated runs create session entries (`...:cron::run:`) and transcript files. +- The reaper removes expired run-session entries older than `cron.sessionRetention`. +- For removed run sessions no longer referenced by the session store, OpenClaw archives transcript files and purges old deleted archives on the same retention window. +- After each run append, `cron/runs/.jsonl` is size-checked: + - if file size exceeds `runLog.maxBytes`, it is trimmed to the newest `runLog.keepLines` lines. + +### Performance caveat for high volume schedulers + +High-frequency cron setups can generate large run-session and run-log footprints. Maintenance is built in, but loose limits can still create avoidable IO and cleanup work. + +What to watch: + +- long `cron.sessionRetention` windows with many isolated runs +- high `cron.runLog.keepLines` combined with large `runLog.maxBytes` +- many noisy recurring jobs writing to the same `cron/runs/.jsonl` + +What to do: + +- keep `cron.sessionRetention` as short as your debugging/audit needs allow +- keep run logs bounded with moderate `runLog.maxBytes` and `runLog.keepLines` +- move noisy background jobs to isolated mode with delivery rules that avoid unnecessary chatter +- review growth periodically with `openclaw cron runs` and adjust retention before logs become large + +### Customize examples + +Keep run sessions for a week and allow bigger run logs: + +```json5 +{ + cron: { + sessionRetention: "7d", + runLog: { + maxBytes: "10mb", + keepLines: 5000, + }, + }, +} +``` + +Disable isolated run-session pruning but keep run-log pruning: + +```json5 +{ + cron: { + sessionRetention: false, + runLog: { + maxBytes: "5mb", + keepLines: 3000, + }, + }, +} +``` + +Tune for high-volume cron usage (example): + +```json5 +{ + cron: { + sessionRetention: "12h", + runLog: { + maxBytes: "3mb", + keepLines: 1500, + }, + }, +} +``` + ## CLI quickstart One-shot reminder (UTC ISO, auto-delete after success): diff --git a/docs/automation/cron-vs-heartbeat.md b/docs/automation/cron-vs-heartbeat.md index a138e721ae4..9676d960d23 100644 --- a/docs/automation/cron-vs-heartbeat.md +++ b/docs/automation/cron-vs-heartbeat.md @@ -62,7 +62,7 @@ The agent reads this on each heartbeat and handles all items in one turn. defaults: { heartbeat: { every: "30m", // interval - target: "last", // where to deliver alerts + target: "last", // explicit alert delivery target (default is "none") activeHours: { start: "08:00", end: "22:00" }, // optional }, }, @@ -211,7 +211,7 @@ For ad-hoc workflows, call Lobster directly. - Lobster runs as a **local subprocess** (`lobster` CLI) in tool mode and returns a **JSON envelope**. - If the tool returns `needs_approval`, you resume with a `resumeToken` and `approve` flag. - The tool is an **optional plugin**; enable it additively via `tools.alsoAllow: ["lobster"]` (recommended). -- If you pass `lobsterPath`, it must be an **absolute path**. +- Lobster expects the `lobster` CLI to be available on `PATH`. See [Lobster](/tools/lobster) for full usage and examples. diff --git a/docs/automation/hooks.md b/docs/automation/hooks.md index 55c04e9990b..0f561741d9a 100644 --- a/docs/automation/hooks.md +++ b/docs/automation/hooks.md @@ -119,6 +119,8 @@ Example `package.json`: Each entry points to a hook directory containing `HOOK.md` and `handler.ts` (or `index.ts`). Hook packs can ship dependencies; they will be installed under `~/.openclaw/hooks/`. +Each `openclaw.hooks` entry must stay inside the package directory after symlink +resolution; entries that escape are rejected. Security note: `openclaw hooks install` installs dependencies with `npm install --ignore-scripts` (no lifecycle scripts). Keep hook pack dependency trees "pure JS/TS" and avoid packages that rely @@ -180,9 +182,7 @@ The `metadata.openclaw` object supports: The `handler.ts` file exports a `HookHandler` function: ```typescript -import type { HookHandler } from "../../src/hooks/hooks.js"; - -const myHandler: HookHandler = async (event) => { +const myHandler = async (event) => { // Only trigger on 'new' command if (event.type !== "command" || event.action !== "new") { return; @@ -303,13 +303,15 @@ Message events include rich context about the message: #### Example: Message Logger Hook ```typescript -import type { HookHandler } from "../../src/hooks/hooks.js"; -import { isMessageReceivedEvent, isMessageSentEvent } from "../../src/hooks/internal-hooks.js"; +const isMessageReceivedEvent = (event: { type: string; action: string }) => + event.type === "message" && event.action === "received"; +const isMessageSentEvent = (event: { type: string; action: string }) => + event.type === "message" && event.action === "sent"; -const handler: HookHandler = async (event) => { - if (isMessageReceivedEvent(event)) { +const handler = async (event) => { + if (isMessageReceivedEvent(event as { type: string; action: string })) { console.log(`[message-logger] Received from ${event.context.from}: ${event.context.content}`); - } else if (isMessageSentEvent(event)) { + } else if (isMessageSentEvent(event as { type: string; action: string })) { console.log(`[message-logger] Sent to ${event.context.to}: ${event.context.content}`); } }; @@ -362,9 +364,7 @@ This hook does something useful when you issue `/new`. ### 4. Create handler.ts ```typescript -import type { HookHandler } from "../../src/hooks/hooks.js"; - -const handler: HookHandler = async (event) => { +const handler = async (event) => { if (event.type !== "command" || event.action !== "new") { return; } @@ -791,13 +791,17 @@ Test your handlers in isolation: ```typescript import { test } from "vitest"; -import { createHookEvent } from "./src/hooks/hooks.js"; import myHandler from "./hooks/my-hook/handler.js"; test("my handler works", async () => { - const event = createHookEvent("command", "new", "test-session", { - foo: "bar", - }); + const event = { + type: "command", + action: "new", + sessionKey: "test-session", + timestamp: new Date(), + messages: [], + context: { foo: "bar" }, + }; await myHandler(event); diff --git a/docs/automation/troubleshooting.md b/docs/automation/troubleshooting.md index 51f2aa209cf..9190855dd59 100644 --- a/docs/automation/troubleshooting.md +++ b/docs/automation/troubleshooting.md @@ -89,7 +89,7 @@ Common signatures: - `heartbeat skipped` with `reason=quiet-hours` → outside `activeHours`. - `requests-in-flight` → main lane busy; heartbeat deferred. -- `empty-heartbeat-file` → `HEARTBEAT.md` exists but has no actionable content. +- `empty-heartbeat-file` → interval heartbeat skipped because `HEARTBEAT.md` has no actionable content and no tagged cron event is queued. - `alerts-disabled` → visibility settings suppress outbound heartbeat messages. ## Timezone and activeHours gotchas diff --git a/docs/channels/bluebubbles.md b/docs/channels/bluebubbles.md index fd677a1d585..8c8267498b7 100644 --- a/docs/channels/bluebubbles.md +++ b/docs/channels/bluebubbles.md @@ -46,7 +46,8 @@ Status: bundled plugin that talks to the BlueBubbles macOS server over HTTP. **R Security note: -- Always set a webhook password. If you expose the gateway through a reverse proxy (Tailscale Serve/Funnel, nginx, Cloudflare Tunnel, ngrok), the proxy may connect to the gateway over loopback. The BlueBubbles webhook handler treats requests with forwarding headers as proxied and will not accept passwordless webhooks. +- Always set a webhook password. +- Webhook authentication is always required. OpenClaw rejects BlueBubbles webhook requests unless they include a password/guid that matches `channels.bluebubbles.password` (for example `?password=` or `x-password`), regardless of loopback/proxy topology. ## Keeping Messages.app alive (VM / headless setups) diff --git a/docs/channels/discord.md b/docs/channels/discord.md index 774a0eba1a8..392c734be94 100644 --- a/docs/channels/discord.md +++ b/docs/channels/discord.md @@ -258,6 +258,29 @@ Now create some channels on your Discord server and start chatting. Your agent c - Group DMs are ignored by default (`channels.discord.dm.groupEnabled=false`). - Native slash commands run in isolated command sessions (`agent::discord:slash:`), while still carrying `CommandTargetSessionKey` to the routed conversation session. +## Forum channels + +Discord forum and media channels only accept thread posts. OpenClaw supports two ways to create them: + +- Send a message to the forum parent (`channel:`) to auto-create a thread. The thread title uses the first non-empty line of your message. +- Use `openclaw message thread create` to create a thread directly. Do not pass `--message-id` for forum channels. + +Example: send to forum parent to create a thread + +```bash +openclaw message send --channel discord --target channel: \ + --message "Topic title\nBody of the post" +``` + +Example: create a forum thread explicitly + +```bash +openclaw message thread create --channel discord --target channel: \ + --thread-name "Topic title" --message "Body of the post" +``` + +Forum parents do not accept Discord components. If you need components, send to the thread itself (`channel:`). + ## Interactive components OpenClaw supports Discord components v2 containers for agent messages. Use the message tool with a `components` payload. Interaction results are routed back to the agent as normal inbound messages and follow the existing Discord `replyToMode` settings. @@ -272,6 +295,8 @@ By default, components are single use. Set `components.reusable=true` to allow b To restrict who can click a button, set `allowedUsers` on that button (Discord user IDs, tags, or `*`). When configured, unmatched users receive an ephemeral denial. +The `/model` and `/models` slash commands open an interactive model picker with provider and model dropdowns plus a Submit step. The picker reply is ephemeral and only the invoking user can use it. + File attachments: - `file` blocks must point to an attachment reference (`attachment://`) @@ -351,6 +376,12 @@ Example: If DM policy is not open, unknown users are blocked (or prompted for pairing in `pairing` mode). + Multi-account precedence: + + - `channels.discord.accounts.default.allowFrom` applies only to the `default` account. + - Named accounts inherit `channels.discord.allowFrom` when their own `allowFrom` is unset. + - Named accounts do not inherit `channels.discord.accounts.default.allowFrom`. + DM target format for delivery: - `user:` @@ -372,7 +403,9 @@ Example: `allowlist` behavior: - guild must match `channels.discord.guilds` (`id` preferred, slug accepted) - - optional sender allowlists: `users` (IDs or names) and `roles` (role IDs only); if either is configured, senders are allowed when they match `users` OR `roles` + - optional sender allowlists: `users` (stable IDs recommended) and `roles` (role IDs only); if either is configured, senders are allowed when they match `users` OR `roles` + - direct name/tag matching is disabled by default; enable `channels.discord.dangerouslyAllowNameMatching: true` only as break-glass compatibility mode + - names/tags are supported for `users`, but IDs are safer; `openclaw security audit` warns when name/tag entries are used - if a guild has `channels` configured, non-listed channels are denied - if a guild has no `channels` block, all channels in that allowlisted guild are allowed @@ -399,7 +432,7 @@ Example: } ``` - If you only set `DISCORD_BOT_TOKEN` and do not create a `channels.discord` block, runtime fallback is `groupPolicy="open"` (with a warning in logs). + If you only set `DISCORD_BOT_TOKEN` and do not create a `channels.discord` block, runtime fallback is `groupPolicy="allowlist"` (with a warning in logs), even if `channels.defaults.groupPolicy` is `open`. @@ -509,6 +542,10 @@ Use `bindings[].match.roles` to route Discord guild members to different agents See [Slash commands](/tools/slash-commands) for command catalog and behavior. +Default slash command settings: + +- `ephemeral: true` + ## Feature details @@ -530,6 +567,51 @@ See [Slash commands](/tools/slash-commands) for command catalog and behavior. + + OpenClaw can stream draft replies by sending a temporary message and editing it as text arrives. + + - `channels.discord.streaming` controls preview streaming (`off` | `partial` | `block` | `progress`, default: `off`). + - `progress` is accepted for cross-channel consistency and maps to `partial` on Discord. + - `channels.discord.streamMode` is a legacy alias and is auto-migrated. + - `partial` edits a single preview message as tokens arrive. + - `block` emits draft-sized chunks (use `draftChunk` to tune size and breakpoints). + + Example: + +```json5 +{ + channels: { + discord: { + streaming: "partial", + }, + }, +} +``` + + `block` mode chunking defaults (clamped to `channels.discord.textChunkLimit`): + +```json5 +{ + channels: { + discord: { + streaming: "block", + draftChunk: { + minChars: 200, + maxChars: 800, + breakPreference: "paragraph", + }, + }, + }, +} +``` + + Preview streaming is text-only; media replies fall back to normal delivery. + + Note: preview streaming is separate from block streaming. When block streaming is explicitly + enabled for Discord, OpenClaw skips the preview stream to avoid double streaming. + + + Guild history context: @@ -552,6 +634,53 @@ See [Slash commands](/tools/slash-commands) for command catalog and behavior. + + Discord can bind a thread to a session target so follow-up messages in that thread keep routing to the same session (including subagent sessions). + + Commands: + + - `/focus ` bind current/new thread to a subagent/session target + - `/unfocus` remove current thread binding + - `/agents` show active runs and binding state + - `/session idle ` inspect/update inactivity auto-unfocus for focused bindings + - `/session max-age ` inspect/update hard max age for focused bindings + + Config: + +```json5 +{ + session: { + threadBindings: { + enabled: true, + idleHours: 24, + maxAgeHours: 0, + }, + }, + channels: { + discord: { + threadBindings: { + enabled: true, + idleHours: 24, + maxAgeHours: 0, + spawnSubagentSessions: false, // opt-in + }, + }, + }, +} +``` + + Notes: + + - `session.threadBindings.*` sets global defaults. + - `channels.discord.threadBindings.*` overrides Discord behavior. + - `spawnSubagentSessions` must be true to auto-create/bind threads for `sessions_spawn({ thread: true })`. + - `spawnAcpSessions` must be true to auto-create/bind threads for ACP (`/acp spawn ... --thread ...` or `sessions_spawn({ runtime: "acp", thread: true })`). + - If thread bindings are disabled for an account, `/focus` and related thread binding operations are unavailable. + + See [Sub-agents](/tools/subagents), [ACP Agents](/tools/acp-agents), and [Configuration Reference](/gateway/configuration-reference). + + + Per-guild reaction notification mode: @@ -650,7 +779,7 @@ See [Slash commands](/tools/slash-commands) for command catalog and behavior. Notes: - allowlists can use `pk:` - - member display names are matched by name/slug + - member display names are matched by name/slug only when `channels.discord.dangerouslyAllowNameMatching: true` - lookups use original message ID and are time-window constrained - if lookup fails, proxied messages are treated as bot messages and dropped unless `allowBots=true` @@ -774,6 +903,53 @@ Example: } ``` +## Voice channels + +OpenClaw can join Discord voice channels for realtime, continuous conversations. This is separate from voice message attachments. + +Requirements: + +- Enable native commands (`commands.native` or `channels.discord.commands.native`). +- Configure `channels.discord.voice`. +- The bot needs Connect + Speak permissions in the target voice channel. + +Use the Discord-only native command `/vc join|leave|status` to control sessions. The command uses the account default agent and follows the same allowlist and group policy rules as other Discord commands. + +Auto-join example: + +```json5 +{ + channels: { + discord: { + voice: { + enabled: true, + autoJoin: [ + { + guildId: "123456789012345678", + channelId: "234567890123456789", + }, + ], + daveEncryption: true, + decryptionFailureTolerance: 24, + tts: { + provider: "openai", + openai: { voice: "alloy" }, + }, + }, + }, + }, +} +``` + +Notes: + +- `voice.tts` overrides `messages.tts` for voice playback only. +- Voice is enabled by default; set `channels.discord.voice.enabled=false` to disable it. +- `voice.daveEncryption` and `voice.decryptionFailureTolerance` pass through to `@discordjs/voice` join options. +- `@discordjs/voice` defaults are `daveEncryption=true` and `decryptionFailureTolerance=24` if unset. +- OpenClaw also watches receive decrypt failures and auto-recovers by leaving/rejoining the voice channel after repeated failures in a short window. +- If receive logs repeatedly show `DecryptionFailed(UnencryptedWhenPassthroughDisabled)`, this may be the upstream `@discordjs/voice` receive bug tracked in [discord.js #11419](https://github.com/discordjs/discord.js/issues/11419). + ## Voice messages Discord voice messages show a waveform preview and require OGG/Opus audio plus metadata. OpenClaw generates the waveform automatically, but it needs `ffmpeg` and `ffprobe` available on the gateway host to inspect and convert audio files. @@ -848,6 +1024,18 @@ openclaw logs --follow If you set `channels.discord.allowBots=true`, use strict mention and allowlist rules to avoid loop behavior. + + + + - keep OpenClaw current (`openclaw update`) so the Discord voice receive recovery logic is present + - confirm `channels.discord.voice.daveEncryption=true` (default) + - start from `channels.discord.voice.decryptionFailureTolerance=24` (upstream default) and tune only if needed + - watch logs for: + - `discord voice: DAVE decrypt failures detected` + - `discord voice: repeated decrypt failures; attempting rejoin` + - if failures continue after automatic rejoin, collect logs and compare against [discord.js #11419](https://github.com/discordjs/discord.js/issues/11419) + + ## Configuration reference pointers @@ -860,9 +1048,10 @@ High-signal Discord fields: - startup/auth: `enabled`, `token`, `accounts.*`, `allowBots` - policy: `groupPolicy`, `dm.*`, `guilds.*`, `guilds.*.channels.*` -- command: `commands.native`, `commands.useAccessGroups`, `configWrites` +- command: `commands.native`, `commands.useAccessGroups`, `configWrites`, `slashCommand.*` - reply/history: `replyToMode`, `historyLimit`, `dmHistoryLimit`, `dms.*.historyLimit` - delivery: `textChunkLimit`, `chunkMode`, `maxLinesPerMessage` +- streaming: `streaming` (legacy alias: `streamMode`), `draftChunk`, `blockStreaming`, `blockStreamingCoalesce` - media/retry: `mediaMaxMb`, `retry` - actions: `actions.*` - presence: `activity`, `status`, `activityType`, `activityUrl` diff --git a/docs/channels/feishu.md b/docs/channels/feishu.md index 461facdbb27..077ddcf723a 100644 --- a/docs/channels/feishu.md +++ b/docs/channels/feishu.md @@ -109,6 +109,8 @@ On **Permissions**, click **Batch import** and paste: "application:application.app_message_stats.overview:readonly", "application:application:self_manage", "application:bot.menu:write", + "cardkit:card:read", + "cardkit:card:write", "contact:user.employee_id:readonly", "corehr:file:download", "event:ip_list", @@ -193,6 +195,8 @@ Edit `~/.openclaw/openclaw.json`: } ``` +If you use `connectionMode: "webhook"`, set `verificationToken`. The Feishu webhook server binds to `127.0.0.1` by default; set `webhookHost` only if you intentionally need a different bind address. + ### Configure via environment variables ```bash @@ -220,6 +224,34 @@ If your tenant is on Lark (international), set the domain to `lark` (or a full d } ``` +### Quota optimization flags + +You can reduce Feishu API usage with two optional flags: + +- `typingIndicator` (default `true`): when `false`, skip typing reaction calls. +- `resolveSenderNames` (default `true`): when `false`, skip sender profile lookup calls. + +Set them at top level or per account: + +```json5 +{ + channels: { + feishu: { + typingIndicator: false, + resolveSenderNames: false, + accounts: { + main: { + appId: "cli_xxx", + appSecret: "xxx", + typingIndicator: true, + resolveSenderNames: false, + }, + }, + }, + }, +} +``` + --- ## Step 3: Start + test @@ -313,14 +345,36 @@ After approval, you can chat normally. } ``` -### Allow specific users in groups only +### Allow specific groups only ```json5 { channels: { feishu: { groupPolicy: "allowlist", - groupAllowFrom: ["ou_xxx", "ou_yyy"], + // Feishu group IDs (chat_id) look like: oc_xxx + groupAllowFrom: ["oc_xxx", "oc_yyy"], + }, + }, +} +``` + +### Allow specific users to run control commands in a group (e.g. /reset, /new) + +In addition to allowing the group itself, control commands are gated by the **sender** open_id. + +```json5 +{ + channels: { + feishu: { + groupPolicy: "allowlist", + groupAllowFrom: ["oc_xxx"], + groups: { + oc_xxx: { + // Feishu user IDs (open_id) look like: ou_xxx + allowFrom: ["ou_user1", "ou_user2"], + }, + }, }, }, } @@ -527,23 +581,28 @@ Full configuration: [Gateway configuration](/gateway/configuration) Key options: -| Setting | Description | Default | -| ------------------------------------------------- | ------------------------------- | --------- | -| `channels.feishu.enabled` | Enable/disable channel | `true` | -| `channels.feishu.domain` | API domain (`feishu` or `lark`) | `feishu` | -| `channels.feishu.accounts..appId` | App ID | - | -| `channels.feishu.accounts..appSecret` | App Secret | - | -| `channels.feishu.accounts..domain` | Per-account API domain override | `feishu` | -| `channels.feishu.dmPolicy` | DM policy | `pairing` | -| `channels.feishu.allowFrom` | DM allowlist (open_id list) | - | -| `channels.feishu.groupPolicy` | Group policy | `open` | -| `channels.feishu.groupAllowFrom` | Group allowlist | - | -| `channels.feishu.groups..requireMention` | Require @mention | `true` | -| `channels.feishu.groups..enabled` | Enable group | `true` | -| `channels.feishu.textChunkLimit` | Message chunk size | `2000` | -| `channels.feishu.mediaMaxMb` | Media size limit | `30` | -| `channels.feishu.streaming` | Enable streaming card output | `true` | -| `channels.feishu.blockStreaming` | Enable block streaming | `true` | +| Setting | Description | Default | +| ------------------------------------------------- | ------------------------------- | ---------------- | +| `channels.feishu.enabled` | Enable/disable channel | `true` | +| `channels.feishu.domain` | API domain (`feishu` or `lark`) | `feishu` | +| `channels.feishu.connectionMode` | Event transport mode | `websocket` | +| `channels.feishu.verificationToken` | Required for webhook mode | - | +| `channels.feishu.webhookPath` | Webhook route path | `/feishu/events` | +| `channels.feishu.webhookHost` | Webhook bind host | `127.0.0.1` | +| `channels.feishu.webhookPort` | Webhook bind port | `3000` | +| `channels.feishu.accounts..appId` | App ID | - | +| `channels.feishu.accounts..appSecret` | App Secret | - | +| `channels.feishu.accounts..domain` | Per-account API domain override | `feishu` | +| `channels.feishu.dmPolicy` | DM policy | `pairing` | +| `channels.feishu.allowFrom` | DM allowlist (open_id list) | - | +| `channels.feishu.groupPolicy` | Group policy | `open` | +| `channels.feishu.groupAllowFrom` | Group allowlist | - | +| `channels.feishu.groups..requireMention` | Require @mention | `true` | +| `channels.feishu.groups..enabled` | Enable group | `true` | +| `channels.feishu.textChunkLimit` | Message chunk size | `2000` | +| `channels.feishu.mediaMaxMb` | Media size limit | `30` | +| `channels.feishu.streaming` | Enable streaming card output | `true` | +| `channels.feishu.blockStreaming` | Enable block streaming | `true` | --- diff --git a/docs/channels/googlechat.md b/docs/channels/googlechat.md index 818a8288f5d..8281d0fb0d2 100644 --- a/docs/channels/googlechat.md +++ b/docs/channels/googlechat.md @@ -153,7 +153,8 @@ Configure your tunnel's ingress rules to only route the webhook path: Use these identifiers for delivery and allowlists: -- Direct messages: `users/` (recommended) or raw email `name@example.com` (mutable principal). +- Direct messages: `users/` (recommended). +- Raw email `name@example.com` is mutable and only used for direct allowlist matching when `channels.googlechat.dangerouslyAllowNameMatching: true`. - Deprecated: `users/` is treated as a user id, not an email allowlist. - Spaces: `spaces/`. @@ -165,13 +166,14 @@ Use these identifiers for delivery and allowlists: googlechat: { enabled: true, serviceAccountFile: "/path/to/service-account.json", + // or serviceAccountRef: { source: "file", provider: "filemain", id: "/channels/googlechat/serviceAccount" } audienceType: "app-url", audience: "https://gateway.example.com/googlechat", webhookPath: "/googlechat", botUser: "users/1234567890", // optional; helps mention detection dm: { policy: "pairing", - allowFrom: ["users/1234567890", "name@example.com"], + allowFrom: ["users/1234567890"], }, groupPolicy: "allowlist", groups: { @@ -193,11 +195,15 @@ Use these identifiers for delivery and allowlists: Notes: - Service account credentials can also be passed inline with `serviceAccount` (JSON string). +- `serviceAccountRef` is also supported (env/file SecretRef), including per-account refs under `channels.googlechat.accounts..serviceAccountRef`. - Default webhook path is `/googlechat` if `webhookPath` isn’t set. +- `dangerouslyAllowNameMatching` re-enables mutable email principal matching for allowlists (break-glass compatibility mode). - Reactions are available via the `reactions` tool and `channels action` when `actions.reactions` is enabled. - `typingIndicator` supports `none`, `message` (default), and `reaction` (reaction requires user OAuth). - Attachments are downloaded through the Chat API and stored in the media pipeline (size capped by `mediaMaxMb`). +Secrets reference details: [Secrets Management](/gateway/secrets). + ## Troubleshooting ### 405 Method Not Allowed diff --git a/docs/channels/grammy.md b/docs/channels/grammy.md deleted file mode 100644 index ae92c5292b0..00000000000 --- a/docs/channels/grammy.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -summary: "Telegram Bot API integration via grammY with setup notes" -read_when: - - Working on Telegram or grammY pathways -title: grammY ---- - -# grammY Integration (Telegram Bot API) - -# Why grammY - -- TS-first Bot API client with built-in long-poll + webhook helpers, middleware, error handling, rate limiter. -- Cleaner media helpers than hand-rolling fetch + FormData; supports all Bot API methods. -- Extensible: proxy support via custom fetch, session middleware (optional), type-safe context. - -# What we shipped - -- **Single client path:** fetch-based implementation removed; grammY is now the sole Telegram client (send + gateway) with the grammY throttler enabled by default. -- **Gateway:** `monitorTelegramProvider` builds a grammY `Bot`, wires mention/allowlist gating, media download via `getFile`/`download`, and delivers replies with `sendMessage/sendPhoto/sendVideo/sendAudio/sendDocument`. Supports long-poll or webhook via `webhookCallback`. -- **Proxy:** optional `channels.telegram.proxy` uses `undici.ProxyAgent` through grammY’s `client.baseFetch`. -- **Webhook support:** `webhook-set.ts` wraps `setWebhook/deleteWebhook`; `webhook.ts` hosts the callback with health + graceful shutdown. Gateway enables webhook mode when `channels.telegram.webhookUrl` + `channels.telegram.webhookSecret` are set (otherwise it long-polls). -- **Sessions:** direct chats collapse into the agent main session (`agent::`); groups use `agent::telegram:group:`; replies route back to the same channel. -- **Config knobs:** `channels.telegram.botToken`, `channels.telegram.dmPolicy`, `channels.telegram.groups` (allowlist + mention defaults), `channels.telegram.allowFrom`, `channels.telegram.groupAllowFrom`, `channels.telegram.groupPolicy`, `channels.telegram.mediaMaxMb`, `channels.telegram.linkPreview`, `channels.telegram.proxy`, `channels.telegram.webhookSecret`, `channels.telegram.webhookUrl`, `channels.telegram.webhookHost`. -- **Live stream preview:** optional `channels.telegram.streamMode` sends a temporary message and updates it with `editMessageText`. This is separate from channel block streaming. -- **Tests:** grammy mocks cover DM + group mention gating and outbound send; more media/webhook fixtures still welcome. - -Open questions - -- Optional grammY plugins (throttler) if we hit Bot API 429s. -- Add more structured media tests (stickers, voice notes). -- Make webhook listen port configurable (currently fixed to 8787 unless wired through the gateway). diff --git a/docs/channels/groups.md b/docs/channels/groups.md index 6bd278846c5..3f9df076454 100644 --- a/docs/channels/groups.md +++ b/docs/channels/groups.md @@ -1,5 +1,5 @@ --- -summary: "Group chat behavior across surfaces (WhatsApp/Telegram/Discord/Slack/Signal/iMessage/Microsoft Teams)" +summary: "Group chat behavior across surfaces (WhatsApp/Telegram/Discord/Slack/Signal/iMessage/Microsoft Teams/Zalo)" read_when: - Changing group chat behavior or mention gating title: "Groups" @@ -7,7 +7,7 @@ title: "Groups" # Groups -OpenClaw treats group chats consistently across surfaces: WhatsApp, Telegram, Discord, Slack, Signal, iMessage, Microsoft Teams. +OpenClaw treats group chats consistently across surfaces: WhatsApp, Telegram, Discord, Slack, Signal, iMessage, Microsoft Teams, Zalo. ## Beginner intro (2 minutes) @@ -183,13 +183,15 @@ Control how group/room messages are handled per channel: Notes: - `groupPolicy` is separate from mention-gating (which requires @mentions). -- WhatsApp/Telegram/Signal/iMessage/Microsoft Teams: use `groupAllowFrom` (fallback: explicit `allowFrom`). +- WhatsApp/Telegram/Signal/iMessage/Microsoft Teams/Zalo: use `groupAllowFrom` (fallback: explicit `allowFrom`). +- DM pairing approvals (`*-allowFrom` store entries) apply to DM access only; group sender authorization stays explicit to group allowlists. - Discord: allowlist uses `channels.discord.guilds..channels`. - Slack: allowlist uses `channels.slack.channels`. - Matrix: allowlist uses `channels.matrix.groups` (room IDs, aliases, or names). Use `channels.matrix.groupAllowFrom` to restrict senders; per-room `users` allowlists are also supported. - Group DMs are controlled separately (`channels.discord.dm.*`, `channels.slack.dm.*`). - Telegram allowlist can match user IDs (`"123456789"`, `"telegram:123456789"`, `"tg:123456789"`) or usernames (`"@alice"` or `"alice"`); prefixes are case-insensitive. - Default is `groupPolicy: "allowlist"`; if your group allowlist is empty, group messages are blocked. +- Runtime safety: when a provider block is completely missing (`channels.` absent), group policy falls back to a fail-closed mode (typically `allowlist`) instead of inheriting `channels.defaults.groupPolicy`. Quick mental model (evaluation order for group messages): @@ -253,7 +255,10 @@ Notes: Some channel configs support restricting which tools are available **inside a specific group/room/channel**. - `tools`: allow/deny tools for the whole group. -- `toolsBySender`: per-sender overrides within the group (keys are sender IDs/usernames/emails/phone numbers depending on the channel). Use `"*"` as a wildcard. +- `toolsBySender`: per-sender overrides within the group. + Use explicit key prefixes: + `id:`, `e164:`, `username:`, `name:`, and `"*"` wildcard. + Legacy unprefixed keys are still accepted and matched as `id:` only. Resolution order (most specific wins): @@ -273,7 +278,7 @@ Example (Telegram): "-1001234567890": { tools: { deny: ["exec", "read", "write"] }, toolsBySender: { - "123456789": { alsoAllow: ["exec"] }, + "id:123456789": { alsoAllow: ["exec"] }, }, }, }, diff --git a/docs/channels/imessage.md b/docs/channels/imessage.md index 2876be31372..5720da1714a 100644 --- a/docs/channels/imessage.md +++ b/docs/channels/imessage.md @@ -97,12 +97,19 @@ exec ssh -T gateway-host imsg "$@" cliPath: "~/.openclaw/scripts/imsg-ssh", remoteHost: "user@gateway-host", // used for SCP attachment fetches includeAttachments: true, + // Optional: override allowed attachment roots. + // Defaults include /Users/*/Library/Messages/Attachments + attachmentRoots: ["/Users/*/Library/Messages/Attachments"], + remoteAttachmentRoots: ["/Users/*/Library/Messages/Attachments"], }, }, } ``` If `remoteHost` is not set, OpenClaw attempts to auto-detect it by parsing the SSH wrapper script. + `remoteHost` must be `host` or `user@host` (no spaces or SSH options). + OpenClaw uses strict host-key checking for SCP, so the relay host key must already exist in `~/.ssh/known_hosts`. + Attachment paths are validated against allowed roots (`attachmentRoots` / `remoteAttachmentRoots`). @@ -151,6 +158,7 @@ imsg send "test" Group sender allowlist: `channels.imessage.groupAllowFrom`. Runtime fallback: if `groupAllowFrom` is unset, iMessage group sender checks fall back to `allowFrom` when available. + Runtime note: if `channels.imessage` is completely missing, runtime falls back to `groupPolicy="allowlist"` and logs a warning (even if `channels.defaults.groupPolicy` is set). Mention gating for groups: @@ -224,13 +232,14 @@ exec ssh -T bot@mac-mini.tailnet-1234.ts.net imsg "$@" ``` Use SSH keys so both SSH and SCP are non-interactive. + Ensure the host key is trusted first (for example `ssh bot@mac-mini.tailnet-1234.ts.net`) so `known_hosts` is populated. iMessage supports per-account config under `channels.imessage.accounts`. - Each account can override fields such as `cliPath`, `dbPath`, `allowFrom`, `groupPolicy`, `mediaMaxMb`, and history settings. + Each account can override fields such as `cliPath`, `dbPath`, `allowFrom`, `groupPolicy`, `mediaMaxMb`, history settings, and attachment root allowlists. @@ -241,6 +250,11 @@ exec ssh -T bot@mac-mini.tailnet-1234.ts.net imsg "$@" - inbound attachment ingestion is optional: `channels.imessage.includeAttachments` - remote attachment paths can be fetched via SCP when `remoteHost` is set + - attachment paths must match allowed roots: + - `channels.imessage.attachmentRoots` (local) + - `channels.imessage.remoteAttachmentRoots` (remote SCP mode) + - default root pattern: `/Users/*/Library/Messages/Attachments` + - SCP uses strict host-key checking (`StrictHostKeyChecking=yes`) - outbound media size uses `channels.imessage.mediaMaxMb` (default 16 MB) @@ -325,7 +339,9 @@ openclaw channels status --probe Check: - `channels.imessage.remoteHost` + - `channels.imessage.remoteAttachmentRoots` - SSH/SCP key auth from the gateway host + - host key exists in `~/.ssh/known_hosts` on the gateway host - remote path readability on the Mac running Messages diff --git a/docs/channels/index.md b/docs/channels/index.md index 181b8d080aa..ff827d20f45 100644 --- a/docs/channels/index.md +++ b/docs/channels/index.md @@ -25,6 +25,7 @@ Text is supported everywhere; media and reactions vary by channel. - [BlueBubbles](/channels/bluebubbles) — **Recommended for iMessage**; uses the BlueBubbles macOS server REST API with full feature support (edit, unsend, effects, reactions, group management — edit currently broken on macOS 26 Tahoe). - [iMessage (legacy)](/channels/imessage) — Legacy macOS integration via imsg CLI (deprecated, use BlueBubbles for new setups). - [Microsoft Teams](/channels/msteams) — Bot Framework; enterprise support (plugin, installed separately). +- [Synology Chat](/channels/synology-chat) — Synology NAS Chat via outgoing+incoming webhooks (plugin, installed separately). - [LINE](/channels/line) — LINE Messaging API bot (plugin, installed separately). - [Nextcloud Talk](/channels/nextcloud-talk) — Self-hosted chat via Nextcloud Talk (plugin, installed separately). - [Matrix](/channels/matrix) — Matrix protocol (plugin, installed separately). @@ -42,6 +43,5 @@ Text is supported everywhere; media and reactions vary by channel. stores more state on disk. - Group behavior varies by channel; see [Groups](/channels/groups). - DM pairing and allowlists are enforced for safety; see [Security](/gateway/security). -- Telegram internals: [grammY notes](/channels/grammy). - Troubleshooting: [Channel troubleshooting](/channels/troubleshooting). - Model providers are documented separately; see [Model Providers](/providers/models). diff --git a/docs/channels/irc.md b/docs/channels/irc.md index 2bf6fb4eb4f..00403b6f92d 100644 --- a/docs/channels/irc.md +++ b/docs/channels/irc.md @@ -1,6 +1,10 @@ --- title: IRC description: Connect OpenClaw to IRC channels and direct messages. +summary: "IRC plugin setup, access controls, and troubleshooting" +read_when: + - You want to connect OpenClaw to IRC channels or DMs + - You are configuring IRC allowlists, group policy, or mention gating --- Use IRC when you want OpenClaw in classic channels (`#room`) and direct messages. @@ -53,7 +57,8 @@ Config keys: - Per-channel controls (channel + sender + mention rules): `channels.irc.groups["#channel"]` - `channels.irc.groupPolicy="open"` allows unconfigured channels (**still mention-gated by default**) -Allowlist entries can use nick or `nick!user@host` forms. +Allowlist entries should use stable sender identities (`nick!user@host`). +Bare nick matching is mutable and only enabled when `channels.irc.dangerouslyAllowNameMatching: true`. ### Common gotcha: `allowFrom` is for DMs, not channels @@ -159,7 +164,7 @@ Use `toolsBySender` to apply a stricter policy to `"*"` and a looser one to your "*": { deny: ["group:runtime", "group:fs", "gateway", "nodes", "cron", "browser"], }, - eigen: { + "id:eigen": { deny: ["gateway", "nodes", "cron"], }, }, @@ -172,7 +177,9 @@ Use `toolsBySender` to apply a stricter policy to `"*"` and a looser one to your Notes: -- `toolsBySender` keys can be a nick (e.g. `"eigen"`) or a full hostmask (`"eigen!~eigen@174.127.248.171"`) for stronger identity matching. +- `toolsBySender` keys should use `id:` for IRC sender identity values: + `id:eigen` or `id:eigen!~eigen@174.127.248.171` for stronger matching. +- Legacy unprefixed keys are still accepted and matched as `id:` only. - The first matching sender policy wins; `"*"` is the wildcard fallback. For more on group access vs mention-gating (and how they interact), see: [/channels/groups](/channels/groups). diff --git a/docs/channels/line.md b/docs/channels/line.md index d32e683fbeb..b87cbd3f5fb 100644 --- a/docs/channels/line.md +++ b/docs/channels/line.md @@ -118,6 +118,7 @@ Allowlists and policies: - `channels.line.groupPolicy`: `allowlist | open | disabled` - `channels.line.groupAllowFrom`: allowlisted LINE user IDs for groups - Per-group overrides: `channels.line.groups..allowFrom` +- Runtime note: if `channels.line` is completely missing, runtime falls back to `groupPolicy="allowlist"` for group checks (even if `channels.defaults.groupPolicy` is set). LINE IDs are case-sensitive. Valid IDs look like: diff --git a/docs/channels/matrix.md b/docs/channels/matrix.md index 04205d94971..9bb56d1ddb7 100644 --- a/docs/channels/matrix.md +++ b/docs/channels/matrix.md @@ -195,6 +195,7 @@ Notes: ## Rooms (groups) - Default: `channels.matrix.groupPolicy = "allowlist"` (mention-gated). Use `channels.defaults.groupPolicy` to override the default when unset. +- Runtime note: if `channels.matrix` is completely missing, runtime falls back to `groupPolicy="allowlist"` for room checks (even if `channels.defaults.groupPolicy` is set). - Allowlist rooms with `channels.matrix.groups` (room IDs or aliases; names are resolved to IDs when directory search finds a single exact match): ```json5 diff --git a/docs/channels/mattermost.md b/docs/channels/mattermost.md index fa0d9393e0f..702f72cc01f 100644 --- a/docs/channels/mattermost.md +++ b/docs/channels/mattermost.md @@ -101,8 +101,10 @@ Notes: ## Channels (groups) - Default: `channels.mattermost.groupPolicy = "allowlist"` (mention-gated). -- Allowlist senders with `channels.mattermost.groupAllowFrom` (user IDs or `@username`). +- Allowlist senders with `channels.mattermost.groupAllowFrom` (user IDs recommended). +- `@username` matching is mutable and only enabled when `channels.mattermost.dangerouslyAllowNameMatching: true`. - Open channels: `channels.mattermost.groupPolicy="open"` (mention-gated). +- Runtime note: if `channels.mattermost` is completely missing, runtime falls back to `groupPolicy="allowlist"` for group checks (even if `channels.defaults.groupPolicy` is set). ## Targets for outbound delivery diff --git a/docs/channels/msteams.md b/docs/channels/msteams.md index 2232582610a..9c4a583e1b5 100644 --- a/docs/channels/msteams.md +++ b/docs/channels/msteams.md @@ -87,7 +87,9 @@ Disable with: **DM access** - Default: `channels.msteams.dmPolicy = "pairing"`. Unknown senders are ignored until approved. -- `channels.msteams.allowFrom` accepts AAD object IDs, UPNs, or display names. The wizard resolves names to IDs via Microsoft Graph when credentials allow. +- `channels.msteams.allowFrom` should use stable AAD object IDs. +- UPNs/display names are mutable; direct matching is disabled by default and only enabled with `channels.msteams.dangerouslyAllowNameMatching: true`. +- The wizard can resolve names to IDs via Microsoft Graph when credentials allow. **Group access** @@ -454,7 +456,8 @@ Key settings (see `/gateway/configuration` for shared channel patterns): - `channels.msteams.webhook.port` (default `3978`) - `channels.msteams.webhook.path` (default `/api/messages`) - `channels.msteams.dmPolicy`: `pairing | allowlist | open | disabled` (default: pairing) -- `channels.msteams.allowFrom`: allowlist for DMs (AAD object IDs, UPNs, or display names). The wizard resolves names to IDs during setup when Graph access is available. +- `channels.msteams.allowFrom`: DM allowlist (AAD object IDs recommended). The wizard resolves names to IDs during setup when Graph access is available. +- `channels.msteams.dangerouslyAllowNameMatching`: break-glass toggle to re-enable mutable UPN/display-name matching. - `channels.msteams.textChunkLimit`: outbound text chunk size. - `channels.msteams.chunkMode`: `length` (default) or `newline` to split on blank lines (paragraph boundaries) before length chunking. - `channels.msteams.mediaAllowHosts`: allowlist for inbound attachment hosts (defaults to Microsoft/Teams domains). @@ -469,6 +472,8 @@ Key settings (see `/gateway/configuration` for shared channel patterns): - `channels.msteams.teams..channels..requireMention`: per-channel override. - `channels.msteams.teams..channels..tools`: per-channel tool policy overrides (`allow`/`deny`/`alsoAllow`). - `channels.msteams.teams..channels..toolsBySender`: per-channel per-sender tool policy overrides (`"*"` wildcard supported). +- `toolsBySender` keys should use explicit prefixes: + `id:`, `e164:`, `username:`, `name:` (legacy unprefixed keys still map to `id:` only). - `channels.msteams.sharePointSiteId`: SharePoint site ID for file uploads in group chats/channels (see [Sending files in group chats](#sending-files-in-group-chats)). ## Routing & Sessions diff --git a/docs/channels/pairing.md b/docs/channels/pairing.md index 4b575eb87c7..d402de16662 100644 --- a/docs/channels/pairing.md +++ b/docs/channels/pairing.md @@ -43,7 +43,14 @@ Supported channels: `telegram`, `whatsapp`, `signal`, `imessage`, `discord`, `sl Stored under `~/.openclaw/credentials/`: - Pending requests: `-pairing.json` -- Approved allowlist store: `-allowFrom.json` +- Approved allowlist store: + - Default account: `-allowFrom.json` + - Non-default account: `--allowFrom.json` + +Account scoping behavior: + +- Non-default accounts read/write only their scoped allowlist file. +- Default account uses the channel-scoped unscoped allowlist file. Treat these as sensitive (they gate access to your assistant). diff --git a/docs/channels/signal.md b/docs/channels/signal.md index 60bb5f7ce92..b216af120ce 100644 --- a/docs/channels/signal.md +++ b/docs/channels/signal.md @@ -195,6 +195,7 @@ Groups: - `channels.signal.groupPolicy = open | allowlist | disabled`. - `channels.signal.groupAllowFrom` controls who can trigger in groups when `allowlist` is set. +- Runtime note: if `channels.signal` is completely missing, runtime falls back to `groupPolicy="allowlist"` for group checks (even if `channels.defaults.groupPolicy` is set). ## How it works (behavior) diff --git a/docs/channels/slack.md b/docs/channels/slack.md index 9fdd3fb89a2..4fa23cf1ee7 100644 --- a/docs/channels/slack.md +++ b/docs/channels/slack.md @@ -152,6 +152,12 @@ For actions/directory reads, user token can be preferred when configured. For wr - `dm.groupEnabled` (group DMs default false) - `dm.groupChannels` (optional MPIM allowlist) + Multi-account precedence: + + - `channels.slack.accounts.default.allowFrom` applies only to the `default` account. + - Named accounts inherit `channels.slack.allowFrom` when their own `allowFrom` is unset. + - Named accounts do not inherit `channels.slack.accounts.default.allowFrom`. + Pairing in DMs uses `openclaw pairing approve slack `. @@ -165,12 +171,13 @@ For actions/directory reads, user token can be preferred when configured. For wr Channel allowlist lives under `channels.slack.channels`. - Runtime note: if `channels.slack` is completely missing (env-only setup) and `channels.defaults.groupPolicy` is unset, runtime falls back to `groupPolicy="open"` and logs a warning. + Runtime note: if `channels.slack` is completely missing (env-only setup), runtime falls back to `groupPolicy="allowlist"` and logs a warning (even if `channels.defaults.groupPolicy` is set). Name/ID resolution: - channel allowlist entries and DM allowlist entries are resolved at startup when token access allows - unresolved entries are kept as configured + - inbound authorization matching is ID-first by default; direct username/slug matching requires `channels.slack.dangerouslyAllowNameMatching: true` @@ -191,6 +198,8 @@ For actions/directory reads, user token can be preferred when configured. For wr - `skills` - `systemPrompt` - `tools`, `toolsBySender` + - `toolsBySender` key format: `id:`, `e164:`, `username:`, `name:`, or `"*"` wildcard + (legacy unprefixed keys still map to `id:` only) @@ -199,7 +208,8 @@ For actions/directory reads, user token can be preferred when configured. For wr - Native command auto-mode is **off** for Slack (`commands.native: "auto"` does not enable Slack native commands). - Enable native Slack command handlers with `channels.slack.commands.native: true` (or global `commands.native: true`). -- When native commands are enabled, register matching slash commands in Slack (`/` names). +- When native commands are enabled, register matching slash commands in Slack (`/` names), with one exception: + - register `/agentstatus` for the status command (Slack reserves `/status`) - If native commands are not enabled, you can run a single configured slash command via `channels.slack.slashCommand`. - Native arg menus now adapt their rendering strategy: - up to 5 options: button blocks @@ -241,7 +251,7 @@ Manual reply tags are supported: - `[[reply_to_current]]` - `[[reply_to:]]` -Note: `replyToMode="off"` disables implicit reply threading. Explicit `[[reply_to_*]]` tags are still honored. +Note: `replyToMode="off"` disables **all** reply threading in Slack, including explicit `[[reply_to_*]]` tags. This differs from Telegram, where explicit tags are still honored in `"off"` mode. The difference reflects the platform threading models: Slack threads hide messages from the channel, while Telegram replies remain visible in the main chat flow. ## Media, chunking, and delivery @@ -465,14 +475,29 @@ openclaw pairing list slack OpenClaw supports Slack native text streaming via the Agents and AI Apps API. -By default, streaming is enabled. Disable it per account: +`channels.slack.streaming` controls live preview behavior: + +- `off`: disable live preview streaming. +- `partial` (default): replace preview text with the latest partial output. +- `block`: append chunked preview updates. +- `progress`: show progress status text while generating, then send final text. + +`channels.slack.nativeStreaming` controls Slack's native streaming API (`chat.startStream` / `chat.appendStream` / `chat.stopStream`) when `streaming` is `partial` (default: `true`). + +Disable native Slack streaming (keep draft preview behavior): ```yaml channels: slack: - streaming: false + streaming: partial + nativeStreaming: false ``` +Legacy keys: + +- `channels.slack.streamMode` (`replace | status_final | append`) is auto-migrated to `channels.slack.streaming`. +- boolean `channels.slack.streaming` is auto-migrated to `channels.slack.nativeStreaming`. + ### Requirements 1. Enable **Agents and AI Apps** in your Slack app settings. @@ -496,9 +521,10 @@ Primary reference: High-signal Slack fields: - mode/auth: `mode`, `botToken`, `appToken`, `signingSecret`, `webhookPath`, `accounts.*` - DM access: `dm.enabled`, `dmPolicy`, `allowFrom` (legacy: `dm.policy`, `dm.allowFrom`), `dm.groupEnabled`, `dm.groupChannels` + - compatibility toggle: `dangerouslyAllowNameMatching` (break-glass; keep off unless needed) - channel access: `groupPolicy`, `channels.*`, `channels.*.users`, `channels.*.requireMention` - threading/history: `replyToMode`, `replyToModeByChatType`, `thread.*`, `historyLimit`, `dmHistoryLimit`, `dms.*.historyLimit` - - delivery: `textChunkLimit`, `chunkMode`, `mediaMaxMb` + - delivery: `textChunkLimit`, `chunkMode`, `mediaMaxMb`, `streaming`, `nativeStreaming` - ops/features: `configWrites`, `commands.native`, `slashCommand.*`, `actions.*`, `userToken`, `userTokenReadOnly` ## Related diff --git a/docs/channels/synology-chat.md b/docs/channels/synology-chat.md new file mode 100644 index 00000000000..89e96b318a3 --- /dev/null +++ b/docs/channels/synology-chat.md @@ -0,0 +1,128 @@ +--- +summary: "Synology Chat webhook setup and OpenClaw config" +read_when: + - Setting up Synology Chat with OpenClaw + - Debugging Synology Chat webhook routing +title: "Synology Chat" +--- + +# Synology Chat (plugin) + +Status: supported via plugin as a direct-message channel using Synology Chat webhooks. +The plugin accepts inbound messages from Synology Chat outgoing webhooks and sends replies +through a Synology Chat incoming webhook. + +## Plugin required + +Synology Chat is plugin-based and not part of the default core channel install. + +Install from a local checkout: + +```bash +openclaw plugins install ./extensions/synology-chat +``` + +Details: [Plugins](/tools/plugin) + +## Quick setup + +1. Install and enable the Synology Chat plugin. +2. In Synology Chat integrations: + - Create an incoming webhook and copy its URL. + - Create an outgoing webhook with your secret token. +3. Point the outgoing webhook URL to your OpenClaw gateway: + - `https://gateway-host/webhook/synology` by default. + - Or your custom `channels.synology-chat.webhookPath`. +4. Configure `channels.synology-chat` in OpenClaw. +5. Restart gateway and send a DM to the Synology Chat bot. + +Minimal config: + +```json5 +{ + channels: { + "synology-chat": { + enabled: true, + token: "synology-outgoing-token", + incomingUrl: "https://nas.example.com/webapi/entry.cgi?api=SYNO.Chat.External&method=incoming&version=2&token=...", + webhookPath: "/webhook/synology", + dmPolicy: "allowlist", + allowedUserIds: ["123456"], + rateLimitPerMinute: 30, + allowInsecureSsl: false, + }, + }, +} +``` + +## Environment variables + +For the default account, you can use env vars: + +- `SYNOLOGY_CHAT_TOKEN` +- `SYNOLOGY_CHAT_INCOMING_URL` +- `SYNOLOGY_NAS_HOST` +- `SYNOLOGY_ALLOWED_USER_IDS` (comma-separated) +- `SYNOLOGY_RATE_LIMIT` +- `OPENCLAW_BOT_NAME` + +Config values override env vars. + +## DM policy and access control + +- `dmPolicy: "allowlist"` is the recommended default. +- `allowedUserIds` accepts a list (or comma-separated string) of Synology user IDs. +- In `allowlist` mode, an empty `allowedUserIds` list is treated as misconfiguration and the webhook route will not start (use `dmPolicy: "open"` for allow-all). +- `dmPolicy: "open"` allows any sender. +- `dmPolicy: "disabled"` blocks DMs. +- Pairing approvals work with: + - `openclaw pairing list synology-chat` + - `openclaw pairing approve synology-chat ` + +## Outbound delivery + +Use numeric Synology Chat user IDs as targets. + +Examples: + +```bash +openclaw message send --channel synology-chat --target 123456 --text "Hello from OpenClaw" +openclaw message send --channel synology-chat --target synology-chat:123456 --text "Hello again" +``` + +Media sends are supported by URL-based file delivery. + +## Multi-account + +Multiple Synology Chat accounts are supported under `channels.synology-chat.accounts`. +Each account can override token, incoming URL, webhook path, DM policy, and limits. + +```json5 +{ + channels: { + "synology-chat": { + enabled: true, + accounts: { + default: { + token: "token-a", + incomingUrl: "https://nas-a.example.com/...token=...", + }, + alerts: { + token: "token-b", + incomingUrl: "https://nas-b.example.com/...token=...", + webhookPath: "/webhook/synology-alerts", + dmPolicy: "allowlist", + allowedUserIds: ["987654"], + }, + }, + }, + }, +} +``` + +## Security notes + +- Keep `token` secret and rotate it if leaked. +- Keep `allowInsecureSsl: false` unless you explicitly trust a self-signed local NAS cert. +- Inbound webhook requests are token-verified and rate-limited per sender. +- Prefer `dmPolicy: "allowlist"` for production. diff --git a/docs/channels/telegram.md b/docs/channels/telegram.md index 7e1d95d2feb..880941edd9c 100644 --- a/docs/channels/telegram.md +++ b/docs/channels/telegram.md @@ -47,6 +47,7 @@ Status: production-ready for bot DMs + groups via grammY. Long polling is the de ``` Env fallback: `TELEGRAM_BOT_TOKEN=...` (default account only). + Telegram does **not** use `openclaw channels login telegram`; configure token in config/env, then start gateway. @@ -108,13 +109,15 @@ Token resolution order is account-aware. In practice, config values win over env `channels.telegram.dmPolicy` controls direct message access: - `pairing` (default) - - `allowlist` + - `allowlist` (requires at least one sender ID in `allowFrom`) - `open` (requires `allowFrom` to include `"*"`) - `disabled` `channels.telegram.allowFrom` accepts numeric Telegram user IDs. `telegram:` / `tg:` prefixes are accepted and normalized. + `dmPolicy: "allowlist"` with empty `allowFrom` blocks all DMs and is rejected by config validation. The onboarding wizard accepts `@username` input and resolves it to numeric IDs. If you upgraded and your config contains `@username` allowlist entries, run `openclaw doctor --fix` to resolve them (best-effort; requires a Telegram bot token). + If you previously relied on pairing-store allowlist files, `openclaw doctor --fix` can recover entries into `channels.telegram.allowFrom` in allowlist flows (for example when `dmPolicy: "allowlist"` has no explicit IDs yet). ### Finding your Telegram user ID @@ -135,10 +138,12 @@ curl "https://api.telegram.org/bot/getUpdates" - There are two independent controls: + Two controls apply together: 1. **Which groups are allowed** (`channels.telegram.groups`) - - no `groups` config: all groups allowed + - no `groups` config: + - with `groupPolicy: "open"`: any group can pass group-ID checks + - with `groupPolicy: "allowlist"` (default): groups are blocked until you add `groups` entries (or `"*"`) - `groups` configured: acts as allowlist (explicit IDs or `"*"`) 2. **Which senders are allowed in groups** (`channels.telegram.groupPolicy`) @@ -147,7 +152,11 @@ curl "https://api.telegram.org/bot/getUpdates" - `disabled` `groupAllowFrom` is used for group sender filtering. If not set, Telegram falls back to `allowFrom`. - `groupAllowFrom` entries must be numeric Telegram user IDs. + `groupAllowFrom` entries should be numeric Telegram user IDs (`telegram:` / `tg:` prefixes are normalized). + Non-numeric entries are ignored for sender authorization. + Security boundary (`2026.2.25+`): group sender auth does **not** inherit DM pairing-store approvals. + Pairing stays DM-only. For groups, set `groupAllowFrom` or per-group/per-topic `allowFrom`. + Runtime note: if `channels.telegram` is completely missing, runtime defaults to fail-closed `groupPolicy="allowlist"` unless `channels.defaults.groupPolicy` is explicitly set. Example: allow any member in one specific group: @@ -226,21 +235,9 @@ curl "https://api.telegram.org/bot/getUpdates" Requirement: - - `channels.telegram.streamMode` is not `"off"` (default: `"partial"`) - - Modes: - - - `off`: no live preview - - `partial`: frequent preview updates from partial text - - `block`: chunked preview updates using `channels.telegram.draftChunk` - - `draftChunk` defaults for `streamMode: "block"`: - - - `minChars: 200` - - `maxChars: 800` - - `breakPreference: "paragraph"` - - `maxChars` is clamped by `channels.telegram.textChunkLimit`. + - `channels.telegram.streaming` is `off | partial | block | progress` (default: `off`) + - `progress` maps to `partial` on Telegram (compat with cross-channel naming) + - legacy `channels.telegram.streamMode` and boolean `streaming` values are auto-mapped This works in direct chats and groups/topics. @@ -248,7 +245,7 @@ curl "https://api.telegram.org/bot/getUpdates" For complex replies (for example media payloads), OpenClaw falls back to normal final delivery and then cleans up the preview message. - `streamMode` is separate from block streaming. When block streaming is explicitly enabled for Telegram, OpenClaw skips the preview stream to avoid double-streaming. + Preview streaming is separate from block streaming. When block streaming is explicitly enabled for Telegram, OpenClaw skips the preview stream to avoid double-streaming. Telegram-only reasoning stream: @@ -393,17 +390,19 @@ curl "https://api.telegram.org/bot/getUpdates" - `react` (`chatId`, `messageId`, `emoji`) - `deleteMessage` (`chatId`, `messageId`) - `editMessage` (`chatId`, `messageId`, `content`) + - `createForumTopic` (`chatId`, `name`, optional `iconColor`, `iconCustomEmojiId`) - Channel message actions expose ergonomic aliases (`send`, `react`, `delete`, `edit`, `sticker`, `sticker-search`). + Channel message actions expose ergonomic aliases (`send`, `react`, `delete`, `edit`, `sticker`, `sticker-search`, `topic-create`). Gating controls: - `channels.telegram.actions.sendMessage` - - `channels.telegram.actions.editMessage` - `channels.telegram.actions.deleteMessage` - `channels.telegram.actions.reactions` - `channels.telegram.actions.sticker` (default: disabled) + Note: `edit` and `topic-create` are currently enabled by default and do not have separate `channels.telegram.actions.*` toggles. + Reaction removal semantics: [/tools/reactions](/tools/reactions) @@ -563,6 +562,7 @@ curl "https://api.telegram.org/bot/getUpdates" Notes: - `own` means user reactions to bot-sent messages only (best-effort via sent-message cache). + - Reaction events still respect Telegram access controls (`dmPolicy`, `allowFrom`, `groupPolicy`, `groupAllowFrom`); unauthorized senders are dropped. - Telegram does not provide thread IDs in reaction updates. - non-forum groups route to group chat session - forum groups route to the group general-topic session (`:topic:1`), not the exact originating topic @@ -619,6 +619,7 @@ curl "https://api.telegram.org/bot/getUpdates" - set `channels.telegram.webhookSecret` (required when webhook URL is set) - optional `channels.telegram.webhookPath` (default `/telegram-webhook`) - optional `channels.telegram.webhookHost` (default `127.0.0.1`) + - optional `channels.telegram.webhookPort` (default `8787`) Default local listener for webhook mode binds to `127.0.0.1:8787`. @@ -636,7 +637,7 @@ curl "https://api.telegram.org/bot/getUpdates" - DM history controls: - `channels.telegram.dmHistoryLimit` - `channels.telegram.dms[""].historyLimit` - - outbound Telegram API retries are configurable via `channels.telegram.retry`. + - `channels.telegram.retry` config applies to Telegram send helpers (CLI/tools/actions) for recoverable outbound API errors. CLI send target can be numeric chat ID or username: @@ -682,6 +683,29 @@ openclaw message send --channel telegram --target @name --message "hi" - Node 22+ + custom fetch/proxy can trigger immediate abort behavior if AbortSignal types mismatch. - Some hosts resolve `api.telegram.org` to IPv6 first; broken IPv6 egress can cause intermittent Telegram API failures. + - If logs include `TypeError: fetch failed` or `Network request for 'getUpdates' failed!`, OpenClaw now retries these as recoverable network errors. + - On VPS hosts with unstable direct egress/TLS, route Telegram API calls through `channels.telegram.proxy`: + +```yaml +channels: + telegram: + proxy: socks5://user:pass@proxy-host:1080 +``` + + - Node 22+ defaults to `autoSelectFamily=true` (except WSL2) and `dnsResultOrder=ipv4first`. + - If your host is WSL2 or explicitly works better with IPv4-only behavior, force family selection: + +```yaml +channels: + telegram: + network: + autoSelectFamily: false +``` + + - Environment overrides (temporary): + - `OPENCLAW_TELEGRAM_DISABLE_AUTO_SELECT_FAMILY=1` + - `OPENCLAW_TELEGRAM_ENABLE_AUTO_SELECT_FAMILY=1` + - `OPENCLAW_TELEGRAM_DNS_RESULT_ORDER=ipv4first` - Validate DNS answers: ```bash @@ -702,9 +726,14 @@ Primary reference: - `channels.telegram.botToken`: bot token (BotFather). - `channels.telegram.tokenFile`: read token from file path. - `channels.telegram.dmPolicy`: `pairing | allowlist | open | disabled` (default: pairing). -- `channels.telegram.allowFrom`: DM allowlist (numeric Telegram user IDs). `open` requires `"*"`. `openclaw doctor --fix` can resolve legacy `@username` entries to IDs. +- `channels.telegram.allowFrom`: DM allowlist (numeric Telegram user IDs). `allowlist` requires at least one sender ID. `open` requires `"*"`. `openclaw doctor --fix` can resolve legacy `@username` entries to IDs and can recover allowlist entries from pairing-store files in allowlist migration flows. +- `channels.telegram.defaultTo`: default Telegram target used by CLI `--deliver` when no explicit `--reply-to` is provided. - `channels.telegram.groupPolicy`: `open | allowlist | disabled` (default: allowlist). -- `channels.telegram.groupAllowFrom`: group sender allowlist (numeric Telegram user IDs). `openclaw doctor --fix` can resolve legacy `@username` entries to IDs. +- `channels.telegram.groupAllowFrom`: group sender allowlist (numeric Telegram user IDs). `openclaw doctor --fix` can resolve legacy `@username` entries to IDs. Non-numeric entries are ignored at auth time. Group auth does not use DM pairing-store fallback (`2026.2.25+`). +- Multi-account precedence: + - `channels.telegram.accounts.default.allowFrom` and `channels.telegram.accounts.default.groupAllowFrom` apply only to the `default` account. + - Named accounts inherit `channels.telegram.allowFrom` and `channels.telegram.groupAllowFrom` when account-level values are unset. + - Named accounts do not inherit `channels.telegram.accounts.default.allowFrom` / `groupAllowFrom`. - `channels.telegram.groups`: per-group defaults + allowlist (use `"*"` for global defaults). - `channels.telegram.groups..groupPolicy`: per-group override for groupPolicy (`open | allowlist | disabled`). - `channels.telegram.groups..requireMention`: mention gating default. @@ -717,19 +746,22 @@ Primary reference: - `channels.telegram.groups..topics..requireMention`: per-topic mention gating override. - `channels.telegram.capabilities.inlineButtons`: `off | dm | group | all | allowlist` (default: allowlist). - `channels.telegram.accounts..capabilities.inlineButtons`: per-account override. +- `channels.telegram.commands.nativeSkills`: enable/disable Telegram native skills commands. - `channels.telegram.replyToMode`: `off | first | all` (default: `off`). - `channels.telegram.textChunkLimit`: outbound chunk size (chars). - `channels.telegram.chunkMode`: `length` (default) or `newline` to split on blank lines (paragraph boundaries) before length chunking. - `channels.telegram.linkPreview`: toggle link previews for outbound messages (default: true). -- `channels.telegram.streamMode`: `off | partial | block` (live stream preview). -- `channels.telegram.mediaMaxMb`: inbound/outbound media cap (MB). -- `channels.telegram.retry`: retry policy for outbound Telegram API calls (attempts, minDelayMs, maxDelayMs, jitter). -- `channels.telegram.network.autoSelectFamily`: override Node autoSelectFamily (true=enable, false=disable). Defaults to disabled on Node 22 to avoid Happy Eyeballs timeouts. +- `channels.telegram.streaming`: `off | partial | block | progress` (live stream preview; default: `off`; `progress` maps to `partial`; `block` is legacy preview mode compatibility). +- `channels.telegram.mediaMaxMb`: inbound Telegram media download/processing cap (MB). +- `channels.telegram.retry`: retry policy for Telegram send helpers (CLI/tools/actions) on recoverable outbound API errors (attempts, minDelayMs, maxDelayMs, jitter). +- `channels.telegram.network.autoSelectFamily`: override Node autoSelectFamily (true=enable, false=disable). Defaults to enabled on Node 22+, with WSL2 defaulting to disabled. +- `channels.telegram.network.dnsResultOrder`: override DNS result order (`ipv4first` or `verbatim`). Defaults to `ipv4first` on Node 22+. - `channels.telegram.proxy`: proxy URL for Bot API calls (SOCKS/HTTP). - `channels.telegram.webhookUrl`: enable webhook mode (requires `channels.telegram.webhookSecret`). - `channels.telegram.webhookSecret`: webhook secret (required when webhookUrl is set). - `channels.telegram.webhookPath`: local webhook path (default `/telegram-webhook`). - `channels.telegram.webhookHost`: local webhook bind host (default `127.0.0.1`). +- `channels.telegram.webhookPort`: local webhook bind port (default `8787`). - `channels.telegram.actions.reactions`: gate Telegram tool reactions. - `channels.telegram.actions.sendMessage`: gate Telegram tool message sends. - `channels.telegram.actions.deleteMessage`: gate Telegram tool message deletes. @@ -743,9 +775,9 @@ Telegram-specific high-signal fields: - startup/auth: `enabled`, `botToken`, `tokenFile`, `accounts.*` - access control: `dmPolicy`, `allowFrom`, `groupPolicy`, `groupAllowFrom`, `groups`, `groups.*.topics.*` -- command/menu: `commands.native`, `customCommands` +- command/menu: `commands.native`, `commands.nativeSkills`, `customCommands` - threading/replies: `replyToMode` -- streaming: `streamMode` (preview), `draftChunk`, `blockStreaming` +- streaming: `streaming` (preview), `blockStreaming` - formatting/delivery: `textChunkLimit`, `chunkMode`, `linkPreview`, `responsePrefix` - media/network: `mediaMaxMb`, `timeoutSeconds`, `retry`, `network.autoSelectFamily`, `proxy` - webhook: `webhookUrl`, `webhookSecret`, `webhookPath`, `webhookHost` diff --git a/docs/channels/whatsapp.md b/docs/channels/whatsapp.md index 88251bd8454..d92dfda9c75 100644 --- a/docs/channels/whatsapp.md +++ b/docs/channels/whatsapp.md @@ -169,8 +169,9 @@ OpenClaw recommends running WhatsApp on a separate number when possible. (The ch Sender allowlist fallback: - if `groupAllowFrom` is unset, runtime falls back to `allowFrom` when available + - sender allowlists are evaluated before mention/reply activation - Note: if no `channels.whatsapp` block exists at all, runtime group-policy fallback is effectively `open`. + Note: if no `channels.whatsapp` block exists at all, runtime group-policy fallback is `allowlist` (with a warning log), even if `channels.defaults.groupPolicy` is set. @@ -183,6 +184,11 @@ OpenClaw recommends running WhatsApp on a separate number when possible. (The ch - configured mention regex patterns (`agents.list[].groupChat.mentionPatterns`, fallback `messages.groupChat.mentionPatterns`) - implicit reply-to-bot detection (reply sender matches bot identity) + Security note: + + - quote/reply only satisfies mention gating; it does **not** grant sender authorization + - with `groupPolicy: "allowlist"`, non-allowlisted senders are still blocked even if they reply to an allowlisted user's message + Session-level activation command: - `/activation mention` @@ -407,6 +413,7 @@ Behavior notes: - `groupAllowFrom` / `allowFrom` - `groups` allowlist entries - mention gating (`requireMention` + mention patterns) + - duplicate keys in `openclaw.json` (JSON5): later entries override earlier ones, so keep a single `groupPolicy` per scope diff --git a/docs/channels/zalo.md b/docs/channels/zalo.md index c595c5e6dde..8e5d8ab0382 100644 --- a/docs/channels/zalo.md +++ b/docs/channels/zalo.md @@ -7,7 +7,7 @@ title: "Zalo" # Zalo (Bot API) -Status: experimental. Direct messages only; groups coming soon per Zalo docs. +Status: experimental. DMs are supported; group handling is available with explicit group policy controls. ## Plugin required @@ -51,7 +51,7 @@ It is a good fit for support or notifications where you want deterministic routi - A Zalo Bot API channel owned by the Gateway. - Deterministic routing: replies go back to Zalo; the model never chooses channels. - DMs share the agent's main session. -- Groups are not yet supported (Zalo docs state "coming soon"). +- Groups are supported with policy controls (`groupPolicy` + `groupAllowFrom`) and default to fail-closed allowlist behavior. ## Setup (fast path) @@ -107,6 +107,16 @@ Multi-account support: use `channels.zalo.accounts` with per-account tokens and - Pairing is the default token exchange. Details: [Pairing](/channels/pairing) - `channels.zalo.allowFrom` accepts numeric user IDs (no username lookup available). +## Access control (Groups) + +- `channels.zalo.groupPolicy` controls group inbound handling: `open | allowlist | disabled`. +- Default behavior is fail-closed: `allowlist`. +- `channels.zalo.groupAllowFrom` restricts which sender IDs can trigger the bot in groups. +- If `groupAllowFrom` is unset, Zalo falls back to `allowFrom` for sender checks. +- `groupPolicy: "disabled"` blocks all group messages. +- `groupPolicy: "open"` allows any group member (mention-gated). +- Runtime note: if `channels.zalo` is missing entirely, runtime still falls back to `groupPolicy="allowlist"` for safety. + ## Long-polling vs webhook - Default: long-polling (no public URL required). @@ -115,6 +125,9 @@ Multi-account support: use `channels.zalo.accounts` with per-account tokens and - Webhook URL must use HTTPS. - Zalo sends events with `X-Bot-Api-Secret-Token` header for verification. - Gateway HTTP handles webhook requests at `channels.zalo.webhookPath` (defaults to the webhook URL path). + - Requests must use `Content-Type: application/json` (or `+json` media types). + - Duplicate events (`event_name + message_id`) are ignored for a short replay window. + - Burst traffic is rate-limited per path/source and may return HTTP 429. **Note:** getUpdates (polling) and webhook are mutually exclusive per Zalo API docs. @@ -127,16 +140,16 @@ Multi-account support: use `channels.zalo.accounts` with per-account tokens and ## Capabilities -| Feature | Status | -| --------------- | ------------------------------ | -| Direct messages | ✅ Supported | -| Groups | ❌ Coming soon (per Zalo docs) | -| Media (images) | ✅ Supported | -| Reactions | ❌ Not supported | -| Threads | ❌ Not supported | -| Polls | ❌ Not supported | -| Native commands | ❌ Not supported | -| Streaming | ⚠️ Blocked (2000 char limit) | +| Feature | Status | +| --------------- | -------------------------------------------------------- | +| Direct messages | ✅ Supported | +| Groups | ⚠️ Supported with policy controls (allowlist by default) | +| Media (images) | ✅ Supported | +| Reactions | ❌ Not supported | +| Threads | ❌ Not supported | +| Polls | ❌ Not supported | +| Native commands | ❌ Not supported | +| Streaming | ⚠️ Blocked (2000 char limit) | ## Delivery targets (CLI/cron) @@ -169,6 +182,8 @@ Provider options: - `channels.zalo.tokenFile`: read token from file path. - `channels.zalo.dmPolicy`: `pairing | allowlist | open | disabled` (default: pairing). - `channels.zalo.allowFrom`: DM allowlist (user IDs). `open` requires `"*"`. The wizard will ask for numeric IDs. +- `channels.zalo.groupPolicy`: `open | allowlist | disabled` (default: allowlist). +- `channels.zalo.groupAllowFrom`: group sender allowlist (user IDs). Falls back to `allowFrom` when unset. - `channels.zalo.mediaMaxMb`: inbound/outbound media cap (MB, default 5). - `channels.zalo.webhookUrl`: enable webhook mode (HTTPS required). - `channels.zalo.webhookSecret`: webhook secret (8-256 chars). @@ -183,6 +198,8 @@ Multi-account options: - `channels.zalo.accounts..enabled`: enable/disable account. - `channels.zalo.accounts..dmPolicy`: per-account DM policy. - `channels.zalo.accounts..allowFrom`: per-account allowlist. +- `channels.zalo.accounts..groupPolicy`: per-account group policy. +- `channels.zalo.accounts..groupAllowFrom`: per-account group sender allowlist. - `channels.zalo.accounts..webhookUrl`: per-account webhook URL. - `channels.zalo.accounts..webhookSecret`: per-account webhook secret. - `channels.zalo.accounts..webhookPath`: per-account webhook path. diff --git a/docs/ci.md b/docs/ci.md index cdf5b126a28..51643c87001 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -1,6 +1,10 @@ --- title: CI Pipeline description: How the OpenClaw CI pipeline works +summary: "CI job graph, scope gates, and local command equivalents" +read_when: + - You need to understand why a CI job did or did not run + - You are debugging failing GitHub Actions checks --- # CI Pipeline @@ -34,12 +38,11 @@ Jobs are ordered so cheap checks fail before expensive ones run: ## Runners -| Runner | Jobs | -| ------------------------------- | ----------------------------- | -| `blacksmith-4vcpu-ubuntu-2404` | Most Linux jobs | -| `blacksmith-4vcpu-windows-2025` | `checks-windows` | -| `macos-latest` | `macos`, `ios` | -| `ubuntu-latest` | Scope detection (lightweight) | +| Runner | Jobs | +| -------------------------------- | ------------------------------------------ | +| `blacksmith-16vcpu-ubuntu-2404` | Most Linux jobs, including scope detection | +| `blacksmith-16vcpu-windows-2025` | `checks-windows` | +| `macos-latest` | `macos`, `ios` | ## Local Equivalents diff --git a/docs/cli/acp.md b/docs/cli/acp.md index 46b78cce6f5..3367173ace0 100644 --- a/docs/cli/acp.md +++ b/docs/cli/acp.md @@ -8,7 +8,7 @@ title: "acp" # acp -Run the ACP (Agent Client Protocol) bridge that talks to a OpenClaw Gateway. +Run the [Agent Client Protocol (ACP)](https://agentclientprotocol.com/) bridge that talks to a OpenClaw Gateway. This command speaks ACP over stdio for IDEs and forwards prompts to the Gateway over WebSocket. It keeps ACP sessions mapped to Gateway session keys. @@ -21,6 +21,9 @@ openclaw acp # Remote Gateway openclaw acp --url wss://gateway-host:18789 --token +# Remote Gateway (token from file) +openclaw acp --url wss://gateway-host:18789 --token-file ~/.openclaw/gateway.token + # Attach to an existing session key openclaw acp --session agent:main:main @@ -40,12 +43,19 @@ It spawns the ACP bridge and lets you type prompts interactively. openclaw acp client # Point the spawned bridge at a remote Gateway -openclaw acp client --server-args --url wss://gateway-host:18789 --token +openclaw acp client --server-args --url wss://gateway-host:18789 --token-file ~/.openclaw/gateway.token # Override the server command (default: openclaw) openclaw acp client --server "node" --server-args openclaw.mjs acp --url ws://127.0.0.1:19001 ``` +Permission model (client debug mode): + +- Auto-approval is allowlist-based and only applies to trusted core tool IDs. +- `read` auto-approval is scoped to the current working directory (`--cwd` when set). +- Unknown/non-core tool names, out-of-scope reads, and dangerous tools always require explicit prompt approval. +- Server-provided `toolCall.kind` is treated as untrusted metadata (not an authorization source). + ## How to use this Use ACP when an IDE (or other client) speaks Agent Client Protocol and you want @@ -66,6 +76,8 @@ Example direct run (no config write): ```bash openclaw acp --url wss://gateway-host:18789 --token +# preferred for local process safety +openclaw acp --url wss://gateway-host:18789 --token-file ~/.openclaw/gateway.token ``` ## Selecting agents @@ -153,7 +165,9 @@ Learn more about session keys at [/concepts/session](/concepts/session). - `--url `: Gateway WebSocket URL (defaults to gateway.remote.url when configured). - `--token `: Gateway auth token. +- `--token-file `: read Gateway auth token from file. - `--password `: Gateway auth password. +- `--password-file `: read Gateway auth password from file. - `--session `: default session key. - `--session-label