技能进化

工程开发 社区

技能解读

当前原文的自动摘译,尚未人工审校。

5 个评分维度

每个技能执行在五个维度上得分为 0-100:尺寸 · 重量 · 衡量的内容 准确性 · 25% · 该技能是否为任务产生了正确的结果?相关性 · 20% · 技能内容是否适用于实际用例?代币效率 · 20% · 该技能是否指导代理没有臃肿或重复?

吃水

新技能以草案形式进入。它们没有受到特殊保护,并且在首次使用时会受到严格评估。首次运行时得分低于 30 的轮抽技能将被丢弃而不是修复。

活跃

经常使用的技巧。分数记录在~/.claude/skill-scores.jsonl中。除非在 5 次运行的滚动窗口中分数趋势低于 30 或高于 90,否则不会采取行动。

结晶化

在 5 次或更多连续运行中保持平均综合得分高于 90 的技能得到了具体体现: 应用了 Git 标签:skill/<name>/crystallized-v<N> 在 frontmatter 中添加了只读标志:locked: true 技能被排除在自动修复之外 更改需要明确的人工解锁 + PR

已存档

自动修复失败 3 次的技能已存档:移至skills/_archived/<name>/ 应用的 Git 标记:skill/<name>/archived 如果仍需要该功能,则由catalyst 代理起草替换技能

流狐档案 运行与权限由流狐静态检查推断,不代表作者声明或实际测试结果。
领域
工程开发
作者声明的智能体
未找到明确声明;不据此推断已兼容或已测试
静态检查
88 / 100 · 启发式扫描,不代表运行安全
运行环境证据
未执行;当前没有隔离运行测试证据
作者 / 版本 / 许可证
@vibeeval · 未声明许可证
上下文用量评级
低消耗
接入复杂程度
需简单配置
是否需要外部接口密钥
不需要
兼容的系统
macOS · Linux · Windows
运行环境要求
Node.js
检测到的文件与系统行为
  • 只读
  • 允许写入 / 修改
  • 命令执行
网络行为
仅限本地
安装命令数
无(仅作为资料)

需要注意: 未声明工具权限范围;实际权限由运行环境决定。

作者说明文件(原文) SKILL.md

# Skill Evolution

Darwinian selection for skills. Skills that produce good outcomes are crystallized and protected. Skills that produce poor outcomes are repaired or archived. Every execution generates a score that drives the next generation of the skill.

## The 5 Scoring Dimensions

Each skill execution is scored 0-100 on five dimensions:

| Dimension | Weight | What It Measures |
|-----------|--------|-----------------|
| Accuracy | 25% | Did the skill produce the correct result for the task? |
| Relevance | 20% | Was the skill content applicable to the actual use case? |
| Token Efficiency | 20% | Did the skill guide the agent without bloat or repetition? |
| User Satisfaction | 20% | Did the outcome meet or exceed user expectations? |
| Reusability | 15% | Could another agent use this skill in a similar situation? |

**Composite score** = weighted average of all five dimensions (0-100).

### Scoring Rubric

```
90-100: Excellent -- candidate for crystallization
70-89:  Good -- active skill, no action needed
50-69:  Adequate -- flag for review after 3 more runs
30-49:  Poor -- schedule auto-repair attempt
0-29:   Critical -- immediate auto-repair or archive
```

## Skill Lifecycle

```
DRAFT          ACTIVE         CRYSTALLIZED      ARCHIVED
  |               |                |                |
New skill   In regular use   Proven stable    Deprecated/replaced
  |               |                |                |
  +-- first run ->+-- score >90   ++-- score <30    |
                  |   for 5+ runs  |   (3 attempts)  |
                  +-- score <30 -->+ auto-repair      |
                  |   auto-repair  |   fails 3x -->--+
                  +-- score >90 -->+
```

### Draft
New skills enter as Draft. They receive no special protection and are evaluated critically on first use. A Draft skill that scores below 30 on its very first run is discarded rather than repaired.

### Active
Skills in regular use. Scores are tracked in `~/.claude/skill-scores.jsonl`. No action unless scores trend below 30 or above 90 over a rolling window of 5 runs.

### Crystallized
A skill that maintains an average composite score above 90 over 5 or more consecutive runs is crystallized:
- Git tag applied: `skill/<name>/crystallized-v<N>`
- Read-only flag added to frontmatter: `locked: true`
- Skill is excluded from auto-repair
- Changes require explicit human unlock + PR

### Archived
A skill that fails auto-repair 3 times is archived:
- Moved to `skills/_archived/<name>/`
- Git tag applied: `skill/<name>/archived`
- Replacement skill drafted by `catalyst` agent if the capability is still needed

## Score Storage Format

Append one record per execution to `~/.claude/skill-scores.jsonl`:

```jsonl
{"skill":"experiment-loop","ts":"2026-04-07T10:00:00Z","session":"abc123","scores":{"accuracy":88,"relevance":92,"token_efficiency":75,"user_satisfaction":90,"reusability":85},"composite":86.5,"feedback":"Loop ran 4 iterations successfully, target nearly met"}
{"skill":"experiment-loop","ts":"2026-04-07T14:30:00Z","session":"def456","scores":{"accuracy":95,"relevance":90,"token_efficiency":82,"user_satisfaction":95,"reusability":88},"composite":90.4,"feedback":"Bundle size reduced 28%, target exceeded"}
```

### Score CLI (quick check)

```bash
# Average scores for a skill (last 10 runs)
cat ~/.claude/skill-scores.jsonl | python3 -c "
import sys, json, statistics
skill = '$1'
runs = [json.loads(l) for l in sys.stdin if json.loads(l).get('skill') == skill][-10:]
if runs:
    avg = statistics.mean(r['composite'] for r in runs)
    print(f'{skill}: {avg:.1f} avg over {len(runs)} runs')
"
```

## Crystallization Protocol

When a skill reaches 90+ composite score over 5+ consecutive runs:

1. Verify scores in `~/.claude/skill-scores.jsonl` -- confirm no outliers inflating the average
2. Add `locked: true` to the skill's frontmatter
3. Apply git tag:
   ```bash
   git tag skill/<name>/crystallized-v1 -m "Crystallized: avg score 92.3 over 7 runs"
   git push origin skill/<name>/crystallized-v1
   ```
4. Log the crystallization in `thoughts/SKILL-EVOLUTION.md`
5. Notify via canavar cross-training so all agents know this skill is stable

## Auto-Repair Protocol

When a skill's composite score drops below 30:

### Diagnosis
1. Identify the lowest-scoring dimension (the primary failure mode)
2. Read the last 3 session feedback notes from `~/.claude/skill-scores.jsonl`
3. Summarize what went wrong (specific, not vague)

### Repair
The `catalyst` agent rewrites the failing section(s) of the skill:
- Only the sections relevant to the low-scoring dimension
- Preserve all high-scoring sections unchanged
- Add a concrete example for the repaired section

### Validation
After repair, the skill is re-scored on a synthetic test case by the `verifier` agent:
- Synthetic score must be 50+ to proceed to Active state
- If synthetic score < 50, attempt 2 of 3 repairs begins

### Escalation
After 3 failed auto-repairs:
- Archive the skill
- Alert via `thoughts/SKILL-EVOLUTION.md`
- Spawn `catalyst` to draft a replacement from scratch

## Evolution Log Format

Append events to `thoughts/SKILL-EVOLUTION.md`:

```markdown
## 2026-04-07

### skill: experiment-loop
- Status change: Active -> Crystallized
- Trigger: avg composite 91.2 over 6 consecutive runs
- Git tag: skill/experiment-loop/crystallized-v1
- Notable strength: Token Efficiency dimension consistently 85+

### skill: legacy-deploy-helper
- Status change: Active -> Auto-Repair (attempt 1/3)
- Trigger: composite 24 on last run
- Lowest dimension: Relevance (12) -- skill referenced outdated Heroku patterns
- Repair: catalyst rewrote "Deployment Targets" section with Vercel/Railway focus
- Post-repair synthetic score: 71 -- promoted back to Active
```

## Integration with Canavar Cross-Training

Skill evolution data feeds into canavar's cross-training pipeline:

- A crystallized skill is injected into canavar's `skill-matrix.json` with `trust: locked`
- An archived skill is marked `trust: deprecated` -- agents stop referencing it
- Auto-repair failures are logged to `error-ledger.jsonl` with `source: skill-evolution`
- The canavar leaderboard tracks which agents most frequently produce high-scoring skill executions

```bash
# View crystallized skills
node ~/.claude/hooks/dist/canavar-cli.mjs leaderboard --filter crystallized

# View skills needing repair
cat ~/.claude/skill-scores.jsonl | python3 -c "
import sys, json, collections
runs = [json.loads(l) for l in sys.stdin]
low = {r['skill'] for r in runs if r['composite'] < 30}
print('Skills needing repair:', low)
"
```

## Activation

This skill activates automatically when:
- A skill completes an execution (PostToolUse hook)
- A skill is referenced in a session that ends with user dissatisfaction
- The `verifier` agent reports a skill-guided task as failed

Agents involved: `catalyst` (repair), `verifier` (validation), `self-learner` (feedback extraction), `canavar` (cross-training propagation).

讨论

基于 GitHub Discussions。登录 GitHub 即可参与讨论、点赞、订阅更新。