From 3eaf11fc6dd7919e748180b1b74c4ed7148565ad Mon Sep 17 00:00:00 2001 From: sriki Date: Sat, 21 Mar 2026 10:44:33 +0530 Subject: [PATCH] fix: remove invented threshold table and bad scan fallback advice MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Default Thresholds section: remove the fabricated table (actual defaults in upstream 0.2.3 differ — determinism_threshold=0.8, latency_p95_ms=5000, and the --threshold/--safety-threshold/ --latency-p95 flags do not exist in cli.test()). Replace with 'check --help' and 'pin values in a guardfile' guidance. - Scan warning: remove suggestion to use 'test --spec' with safety.check_injection: false as a side-effect-minimizing fallback — GuardSpec still enables determinism/concurrency/latency by default, so the agent is still called many additional times. Guidance now says: sandboxed environment only. --- skills/operon-guard/SKILL.md | 22 +++++++--------------- 1 file changed, 7 insertions(+), 15 deletions(-) diff --git a/skills/operon-guard/SKILL.md b/skills/operon-guard/SKILL.md index 39f40980ece..535c12e9034 100644 --- a/skills/operon-guard/SKILL.md +++ b/skills/operon-guard/SKILL.md @@ -56,10 +56,8 @@ operon-guard test path/to/skill/ > > **Warning:** The injection check fires **47 adversarial prompts** at the agent. If > your agent has side effects — sending messages, writing to a database, calling paid -> APIs — those side effects will be triggered up to 47 times during the scan. Either -> run in a sandboxed environment, or skip injection probes by setting -> `safety.check_injection: false` in a guardfile and using `operon-guard test --spec` -> instead. +> APIs — those side effects will be triggered up to 47 times during the scan. Do not +> run `scan` against agents with side effects outside a sandboxed environment. ```bash operon-guard scan path/to/agent.py @@ -172,21 +170,15 @@ Produces a score from 0-100 with a letter grade: ## Default Thresholds -When running without a guardfile, Operon Guard uses these built-in defaults: - -| Check | Default threshold | Override flag | -|---|---|---| -| Determinism | 0.90 (90% output similarity) | `--threshold <0-1>` | -| Safety | 0.80 (80% attack resistance) | `--safety-threshold <0-1>` | -| Latency | P95 <= 2000ms | `--latency-p95 ` | - -Pass `--threshold` to override the determinism threshold globally: +Default threshold values and available CLI flags vary by version. Check the +authoritative source before relying on any specific value: ```bash -operon-guard test path/to/agent.py --threshold 0.95 +operon-guard test --help ``` -Or configure per-check thresholds in a guardfile (see below). +Configure per-check thresholds explicitly in a guardfile to avoid depending on +whatever defaults the installed version ships with (see below). ## Guardfile Format