Ayuriel/openclaw
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: remove invented threshold table and bad scan fallback advice

Browse Source 
- Default Thresholds section: remove the fabricated table (actual
  defaults in upstream 0.2.3 differ — determinism_threshold=0.8,
  latency_p95_ms=5000, and the --threshold/--safety-threshold/
  --latency-p95 flags do not exist in cli.test()). Replace with
  'check --help' and 'pin values in a guardfile' guidance.
- Scan warning: remove suggestion to use 'test --spec' with
  safety.check_injection: false as a side-effect-minimizing fallback —
  GuardSpec still enables determinism/concurrency/latency by default,
  so the agent is still called many additional times. Guidance now
  says: sandboxed environment only.
This commit is contained in:
sriki 2026-03-21 10:44:33 +05:30
parent e6ce1797a1
commit 3eaf11fc6d
1 changed files with 7 additions and 15 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
skills/operon-guard/SKILL.md
View File

@ -56,10 +56,8 @@ operon-guard test path/to/skill/
>
> **Warning:** The injection check fires **47 adversarial prompts** at the agent. If
> your agent has side effects — sending messages, writing to a database, calling paid
> APIs — those side effects will be triggered up to 47 times during the scan. Either
> run in a sandboxed environment, or skip injection probes by setting
> `safety.check_injection: false` in a guardfile and using `operon-guard test --spec`
> instead.
> APIs — those side effects will be triggered up to 47 times during the scan. Do not
> run `scan` against agents with side effects outside a sandboxed environment.
```bash
operon-guard scan path/to/agent.py
@ -172,21 +170,15 @@ Produces a score from 0-100 with a letter grade:
## Default Thresholds
When running without a guardfile, Operon Guard uses these built-in defaults:
| Check | Default threshold | Override flag |
|---|---|---|
| Determinism | 0.90 (90% output similarity) | `--threshold <0-1>` |
| Safety | 0.80 (80% attack resistance) | `--safety-threshold <0-1>` |
| Latency | P95 <= 2000ms | `--latency-p95 <ms>` |
Pass `--threshold` to override the determinism threshold globally:
Default threshold values and available CLI flags vary by version. Check the
authoritative source before relying on any specific value:
```bash
operon-guard test path/to/agent.py --threshold 0.95
operon-guard test --help
```
Or configure per-check thresholds in a guardfile (see below).
Configure per-check thresholds explicitly in a guardfile to avoid depending on
whatever defaults the installed version ships with (see below).
## Guardfile Format