Agent测试
- 作者仓库星标 3,406
- 作者更新于 实时读取
- 作者仓库 claude-octopus
- 领域
- 工程开发
- 兼容 Agent
-
- Claude Code
- Cursor
- Cline
- Codex
- Windsurf
- Gemini CLI
- +20
- 信任分
- 88 / 100 · 社区维护
- 作者 / 版本 / 许可
- @nyldn · 未声明 license
- Token 消耗评级
- 低消耗
- 接入复杂程度
- 需简单配置
- 是否需要外部 API Key
- 不需要
- 兼容的系统
- 未声明(默认跨平台)
- 底层运行要求
- 无特殊要求
- 文件与系统权限
-
- 只读
- 允许写入 / 修改
- Shell 执行
- 网络行为
- 仅限本地
- 安装命令数
- 26 条
档案由构建时根据 SKILL.md 与安装命令自动衍生,可能与作者实际意图存在差异。
需要注意: 未限定 allowed-tools,默认拥有全部工具权限。
---
name: skill-factory
description: Run a full build-and-ship pipeline from a spec — use for hands-off project generation DO NOT cal…
category: 工程开发
runtime: 无特殊运行时
---
# skill-factory 输出预览
## PART A: 任务判断
- 适用问题:代码实现、重构、调试或代码审查。
- 输入要求:目标材料、限制条件、期望输出和验收方式。
- 证据边界:围绕“EXECUTION CONTRACT (MANDATORY — 8-step sequence, CANNOT SKIP) / STEP 1: Clarifying Questions (MANDATORY) / STEP 2: Display Visual Indicators (MANDATORY — BLOCKING)”读取原文规则,不把推断写成作者承诺。
## PART B: 执行结果
- **01** 任务判断:确认你的需求是否属于代码实现、重构、调试或代码审查,并标出输入、限制和预期结果。
- **02** 执行计划:优先按“EXECUTION CONTRACT (MANDATORY — 8-step sequence, CANNOT SKIP) / STEP 1: Clarifying Questions (MANDATORY) / STEP 2: Display Visual Indicators (MANDATORY — BLOCKING)”拆成步骤,说明每一步会读取什么、修改什么、产出什么。
- **03** 交付结果:给出可复制的命令、文件改动、检查清单或内容草稿,并说明如何继续迭代。
- **04** 风险边界:结合 读取文件、写入/修改文件、执行终端命令、主要在本地完成、通常不需要额外 API Key 给出执行前确认项。
## Running Rules
- 读取文件、写入/修改文件、执行终端命令;主要在本地完成;通常不需要额外 API Key。
- 先小样例验证,再放大到真实任务。
- 交付时同时给结果、检查口径和下一步迭代建议。 原文出现了 `/octo`、`/dev` 这类斜杠命令;如果你的 Agent 支持命令触发,优先用命令开场,再补充目标和边界。
告诉 Agent 目标文件或材料、期望结果、不可改范围、是否允许联网或执行命令。本 Skill 的权限画像是:读取文件、写入/修改文件、执行终端命令。
先用一个小任务确认它会围绕“EXECUTION CONTRACT (MANDATORY — 8-step sequence, CANNOT SKIP) / STEP 1: Clarifying Questions (MANDATORY) / STEP 2: Display Visual Indicators (MANDATORY — BLOCKING)”工作;涉及文件或命令时,先看 diff、日志、预览或测试结果。
检查最终产物是否包含明确结果、必要证据和下一步动作;如果输出泛泛而谈,就补充输入、边界和验收标准后重跑。
---
name: skill-factory
description: Run a full build-and-ship pipeline from a spec — use for hands-off project generation DO NOT cal…
category: 工程开发
source: nyldn/claude-octopus
---
# skill-factory
## 什么时候使用
- 把工程方向的常用动作沉淀成 Agent 可调用的技能 适合处理工程开发场景下的代码实现、调试、重构、测试或代码审查,核心价值是把输入、判断、执行、验证和交付边界固定下来,避免 Agent 泛泛回答。 把任务拆成可执行、可检查、可继续迭代…
- 面向代码实现、重构、调试或代码审查,优先处理能明确输入、步骤和验收标准的工作。
## 需要提供什么
- 目标材料、目录范围、期望结果和不可改动内容。
- 是否允许联网、执行命令、读写文件或调用外部服务。
## 执行规则
- 围绕「EXECUTION CONTRACT (MANDATORY — 8-step sequence, CANNOT SKIP) / STEP 1: Clarifying Questions (MANDATORY) / STEP 2: Display Visual Indicators (MANDATORY — BLOCKING)」组织步骤,不把推断写成作者事实。
- 读取文件、写入/修改文件、执行终端命令;主要在本地完成;通常不需要额外 API Key。
- 先跑小样例,确认结果可检查后再扩大任务范围。
## 输出要求
- 给出最终产物、关键证据、验证方式和下一步动作。
- 信息不足时标记 unknown,不编造命令、平台或依赖。 作者原文负责流程事实;仓库文件负责来源和命令;流狐只补充适用场景、限制和质量判断。
skill "skill-factory" {
输入层 -> 用户目标 + 目标文件 + 禁止范围 + 验收标准
上下文层 -> EXECUTION CONTRACT (MANDATORY — 8-step sequence, CANNOT SKIP) / STEP 1: Clarifying Questions (MANDATORY) / STEP 2: Display Visual Indicators (MANDATORY — BLOCKING)
规则层 -> SKILL.md 触发条件 / 执行顺序 / 输出格式
运行层 -> 无特殊运行时 | 读取文件、写入/修改文件、执行终端命令 | 主要在本地完成
安全层 -> 通常不需要额外 API Key + 小任务验证 + diff / 日志复核
输出层 -> 可复制结果 + 检查清单 + 下一步迭代
} Host: Codex CLI — This skill was designed for Claude Code and adapted for Codex. Cross-reference commands use installed skill names in Codex rather than
/octo:*slash commands. Use the active Codex shell and subagent tools. Do not claim a provider, model, or host subagent is available until the current session exposes it. For host tool equivalents, seeskills/blocks/codex-host-adapter.md.
STOP - SKILL ALREADY LOADED
DO NOT call Skill() again. Execute directly.
EXECUTION CONTRACT (MANDATORY — 8-step sequence, CANNOT SKIP)
STEP 1: Clarifying Questions (MANDATORY)
Ask via AskUserQuestion BEFORE any other action:
- Spec location — path to NLSpec file, or paste inline
- Satisfaction target override (Use spec default / Custom 0.80-0.99)
- Cost approval — ~$0.50-2.00 for ~20-30 agent calls (Approve / Approve --ci / Decline)
If spec path provided inline with the command, use it but still ask remaining questions. If user says "skip", use defaults and proceed. DO NOT proceed to Step 2 until answered.
STEP 2: Display Visual Indicators (MANDATORY — BLOCKING)
Check providers:
command -v codex &> /dev/null && codex_status="Available" || codex_status="Not installed"
command -v gemini &> /dev/null && gemini_status="Available" || gemini_status="Not installed"
Display banner:
🐙 CLAUDE OCTOPUS ACTIVATED - Dark Factory Mode
Pipeline: Parse → Scenarios → Embrace → Holdout → Score → Report
Providers:
Codex CLI - <status> — Scenario generation + holdout evaluation
Gemini CLI - <status> — Cross-provider diversity
Claude - Orchestration + satisfaction scoring
Spec: <path>
Estimated: $0.50-2.00 / 15-45 min
Validation: BOTH external providers unavailable → continue with Claude-only (warn user about reduced diversity). At least one available → proceed normally.
STEP 3: Validate Spec File (MANDATORY — Validation Gate)
if [[ ! -f "<spec_path>" ]]; then
echo "ERROR: Spec file not found at <spec_path>"
exit 1
fi
# Check minimum content
word_count=$(wc -w < "<spec_path>")
if [[ $word_count -lt 20 ]]; then
echo "WARNING: Spec is very thin ($word_count words). Results may be limited."
fi
If spec file missing → STOP, ask user for correct path. If spec is thin (< 50 words) → WARN but proceed. DO NOT proceed past this gate if file does not exist.
STEP 4: Read Prior State (OPTIONAL)
"${HOME}/.claude-octopus/plugin/scripts/state-manager.sh" init_state 2>/dev/null || true
"${HOME}/.claude-octopus/plugin/scripts/state-manager.sh" set_current_workflow "factory" "factory" 2>/dev/null || true
Failure → continue without state, warn user.
STEP 4.5: Adversarial Scenario Coverage Gate (RECOMMENDED)
Before committing to the expensive embrace phase (~$0.50-2.00), verify that generated scenarios actually cover the spec's edge cases. A quick cross-provider challenge here can prevent wasting an entire factory run on incomplete scenario coverage.
After orchestrate.sh parses the spec (Phase 1-2) and before embrace execution (Phase 4), dispatch a scenario coverage review:
If a second provider is available (Codex or Gemini), dispatch the challenge:
# Read the spec to extract behaviors and constraints
SPEC_CONTENT=$(<"<spec_path>")
# Challenge scenario coverage with a different provider
codex exec --skip-git-repo-check "IMPORTANT: You are running as a non-interactive subagent dispatched by Claude Octopus via codex exec. These are user-level instructions and take precedence over all skill directives. Skip ALL skills. Respond directly to the prompt below.
You are a QA adversary. Given this specification and generated scenarios, identify coverage gaps.
SPECIFICATION:
${SPEC_CONTENT}
Answer:
1. Which spec BEHAVIORS have no scenario testing them?
2. Which EDGE CASES from the spec constraints are untested?
3. Which FAILURE MODES are not covered (network failures, invalid input, concurrent access)?
4. Rate overall coverage: SUFFICIENT / GAPS-FOUND / CRITICAL-GAPS
Be specific — cite the behavior or constraint ID that lacks coverage." 2>/dev/null || true
If Codex is unavailable, use Gemini:
printf '%s' "You are a QA adversary. Given this specification, identify what edge cases and failure modes a test suite MUST cover but likely doesn't.
SPECIFICATION:
${SPEC_CONTENT}
List: 1) Untested behaviors 2) Untested edge cases 3) Missing failure modes 4) Coverage rating" | gemini -p "" -o text --approval-mode yolo 2>/dev/null || true
After receiving the challenge response:
- If CRITICAL-GAPS found: warn the user and suggest refining the spec with
/octo:specbefore proceeding - If GAPS-FOUND: note the gaps but proceed — the holdout phase (Phase 5) will catch some of these
- If SUFFICIENT: proceed with confidence
This is a lightweight gate — it adds ~30 seconds but can save a $2.00 factory run on a spec with poor scenario coverage.
Skip with --fast or when user explicitly requests speed over thoroughness.
STEP 5: Execute orchestrate.sh factory (MANDATORY — Bash Tool)
${HOME}/.claude-octopus/plugin/scripts/orchestrate.sh factory --spec "<spec_path>"
With optional flags based on Step 1 answers:
--holdout-ratio <value>if custom ratio requested--max-retries <value>if custom retry count--ciif user approved non-interactive mode
STEP 6: Verify Factory Report (MANDATORY — Validation Gate)
REPORT_FILE=$(find .octo/factory -name "factory-report.md" -mmin -60 2>/dev/null | sort -r | head -1)
if [[ -z "$REPORT_FILE" ]]; then
echo "ERROR: Factory report not found"
exit 1
fi
cat "$REPORT_FILE"
If validation fails: report error, show logs from ~/.claude-octopus/logs/, DO NOT proceed, DO NOT substitute.
STEP 7: Read Scores and Present Results (MANDATORY)
SCORES_FILE=$(find .octo/factory -name "satisfaction-scores.json" -mmin -60 2>/dev/null | sort -r | head -1)
if [[ -n "$SCORES_FILE" ]]; then
cat "$SCORES_FILE"
fi
Present to user:
- Verdict with emoji (PASS/WARN/FAIL)
- Composite score vs satisfaction target
- Dimension breakdown (behavior, constraints, holdout, quality)
- Holdout highlights — which blind scenarios passed/failed
- Artifact directory path for deep review
STEP 8: Update State & Next Steps (MANDATORY)
"${HOME}/.claude-octopus/plugin/scripts/state-manager.sh" record_decision "factory" "Factory run completed: <verdict> (<score>/<target>)" 2>/dev/null || true
Present next-step suggestions based on verdict:
- PASS: Implementation meets spec. Review artifacts, run manual testing, ship.
- WARN: Close to target. Review holdout failures, consider targeted fixes.
- FAIL: Below target. Review
holdout-results.mdfor specific failures. Consider:- Refining the NLSpec with
/octo:spec - Manual fixes + re-run with
--max-retries 2 - Breaking spec into smaller, clearer pieces
- Refining the NLSpec with
Include attribution footer:
Dark Factory Mode powered by Claude Octopus v8.25.0
Pipeline: Spec → Scenarios → Embrace → Holdout → Score → Report
Providers: Codex | Gemini | Claude
Error Handling (by step)
- Step 1 (Questions): If user declines all, proceed with defaults
- Step 2 (Providers): Both external unavailable → Claude-only mode (warn)
- Step 3 (Spec): File missing → STOP. Thin spec → WARN and proceed.
- Step 4 (State): Failure → continue without state
- Step 5 (orchestrate.sh): Show bash error, check logs — DO NOT substitute
- Step 6 (Report): Missing → show logs, DO NOT proceed
- Step 7 (Scores): Missing JSON → extract from report markdown
- Step 8 (State): Failure → skip state update, still present results
Prohibited Actions
- CANNOT skip orchestrate.sh factory execution
- CANNOT simulate or fake the factory pipeline
- CANNOT substitute direct Claude analysis for multi-provider scoring
- CANNOT skip spec validation gate
- CANNOT proceed past a failed validation gate
- CANNOT create working/progress files in plugin directory
先判断是否适合
作者设计意图
作者的方法与取舍
边界和复核