skill-benchmark

Engineering Community

Repo stars 498
Author repo agent-skills-standard

Interpretation is structured for decision-making; original keeps the upstream SKILL.md unchanged.

Decide Fit First

Core job: Benchmark AI skill effectiveness by measuring implementation quality against legacy constraints. When the user asks to perform t…
Best fit: Use it when the task has reusable inputs, steps, and validation criteria rather than a one-off answer.
Avoid forcing it: If the source lacks commands, platform support, or external-service evidence, keep those fields unknown instead of guessing.

Design Intent

Structure: The skill is organized around “Instructions”, “Step 1 — Project Context & Active Skills”, “Step 2 — Auto-Select a Legacy Trap”, “Step 3 — Build Eval-Driven Scorecard”, showing how the author expects the agent to judge fit, collect context, and produce verifiable output.
Trigger evidence: Prioritize the author’s wording around when to use it, what context to collect, and what output shape to produce.
Evidence boundary: Author text states facts, repository files prove commands and paths, and Fluxly only adds fit, limits, and usage judgment.

How To Use It

Inputs: Provide target material, scope, expected result, forbidden changes, and validation method.
Invocation: Name skill-benchmark directly; if the source includes slash commands, start with the command and then add task context.
Validation: Start small and check whether the result follows “Instructions / Step 1 — Project Context & Active Skills / Step 2 — Auto-Select a Legacy Trap” before expanding.

Boundaries And Review

Dependencies: It usually needs no extra API key, so start with a small validation task.
Permissions: Declared permissions include read / write / shell-exec; ask the agent to state file, command, and rollback boundaries before acting.
Quality bar: A useful result names the deliverable, evidence, and next action. Generic prose means the task needs tighter context.

Skill Benchmark Skill

[!IMPORTANT] Benchmark AI skill effectiveness by measuring implementation quality against legacy constraints.

Instructions

When the user asks to perform this workflow, execute the following steps:

📊 Skill Benchmark Orchestrator

Goal: Quantify how much active skills improve implementation quality. Deliver a prioritized compliance delta and skill applicability report.

Step 1 — Project Context & Active Skills

Identify the tech stack and all active skills in AGENTS.md.

# 1. Total source files and lines changed
find src -name "*.ts" -o -name "*.tsx" | xargs wc -l 2>/dev/null | sort -rn | head -20
# 2. Check active skill registry
cat AGENTS.md | head -80

Step 2 — Auto-Select a Legacy Trap

Pick the file automatically. Rank candidates by the severity of anti-patterns:

🔴 P0: Hardcoded secrets; Logic inside UI components.
🟠 P1: Wrong Router pattern; Global state for local concerns; Missing design tokens.
🟡 P2: Raw user-facing strings (i18n).

Step 3 — Build Eval-Driven Scorecard

Source your scorecard from evals/evals.json, not from hardcoded patterns. Follow the Scorecard Rubric in <SKILLS>/common/common-skill-creator/references/benchmark.md when synced:

Read <SKILLS>/<category>/<skill>/evals/evals.json.
Generate columns for Failure Pattern and Success Pattern.
Refactor the file, citing the exact skill rule for each change.

Step 4 — Benchmark Report & Compliance Delta

Output the scorecard and compliant score using the templates in <SKILLS>/common/common-skill-creator/references/benchmark.md when synced.

Compliance Score Before vs After.
Δ Delta: +Z% 🚀.
Eval Alignment: How well does the skill teach what the eval tests?

Step 5 — Skill Applicability & Iteration

For every ❌ FAIL, identify the root cause using the Iteration Table in: <SKILLS>/common/common-skill-creator/references/benchmark.md when synced.

Signal not matching file? → Refine trigger.
Rule too vague? → Add Anti-Pattern rule.
Conflict? → Ensure P0 overrides P1.

Suggested .skillsrc Exclusions

Recommend any skills that are noisy or non-applicable for the project.

exclude:
  - [skill-id] # reason

Fluxly profile Author and license come from source; runtime, permissions, and network are Fluxly detections or estimates

Fluxly category: Engineering
Author-declared agents: No explicit declaration found; this is not inferred or tested compatibility
Static check: 88 / 100 · heuristic scan, not runtime safety proof
Author / version / license: @HoangNguyen0403 · no license declared
Fluxly token estimate: Lean
Fluxly setup estimate: Guided setup
External API key: No requirement detected
Detected OS requirements: Unspecified
Runtime requirements: Unspecified
Detected file/system behavior: Read-only
Write / modify
Shell exec
Detected network behavior: Local-only
Install commands: None (reference only)

Profile is derived at build time from SKILL.md and install vectors. Subject to drift from author intent.

Heads up: 未限定 allowed-tools，默认拥有全部工具权限。

Output preview skill-benchmark.preview

The current SKILL.md does not define a fixed output example.

SKILL.md SKILL.md

# Skill Benchmark Skill

> [!IMPORTANT]
> Benchmark AI skill effectiveness by measuring implementation quality against legacy constraints.

## Instructions

When the user asks to perform this workflow, execute the following steps:


# 📊 Skill Benchmark Orchestrator

> **Goal**: Quantify how much active skills improve implementation quality. Deliver a prioritized compliance delta and skill applicability report.

---

## Step 1 — Project Context & Active Skills

Identify the tech stack and all active skills in `AGENTS.md`.

```bash
# 1. Total source files and lines changed
find src -name "*.ts" -o -name "*.tsx" | xargs wc -l 2>/dev/null | sort -rn | head -20
# 2. Check active skill registry
cat AGENTS.md | head -80
```

---

## Step 2 — Auto-Select a Legacy Trap

Pick the file automatically. Rank candidates by the severity of anti-patterns:

- 🔴 **P0**: Hardcoded secrets; Logic inside UI components.
- 🟠 **P1**: Wrong Router pattern; Global state for local concerns; Missing design tokens.
- 🟡 **P2**: Raw user-facing strings (i18n).

---

## Step 3 — Build Eval-Driven Scorecard

Source your scorecard from `evals/evals.json`, not from hardcoded patterns.
Follow the Scorecard Rubric in `<SKILLS>/common/common-skill-creator/references/benchmark.md` when synced:

1. Read `<SKILLS>/<category>/<skill>/evals/evals.json`.
2. Generate columns for **Failure Pattern** and **Success Pattern**.
3. Refactor the file, citing the exact skill rule for each change.

---

## Step 4 — Benchmark Report & Compliance Delta

Output the scorecard and compliant score using the templates in `<SKILLS>/common/common-skill-creator/references/benchmark.md` when synced.

- **Compliance Score Before vs After**.
- **Δ Delta: +Z%** 🚀.

…

Evidence boundary and execution chain

Author text anchors workflow facts; Fluxly only indexes current sections, terms, files, and commands.

sections -> Instructions → Step 1 — Project Context & Active Skills → Step 2 — Auto-Select a Legacy Trap → Step 3 — Build Eval-Driven Scorecard → Step 4 — Benchmark Report & Compliance Delta → Step 5 — Skill Applicability & Iteration
terms    -> Goal · P0 · P1 · P2 · Failure Pattern · Success Pattern · Compliance Score Before vs After · Δ Delta: +Z%
files/cmd -> AGENTS.md · evals/evals.json · <SKILLS>/common/common-skill-creator/references/benchmark.md · <SKILLS>/<category>/<skill>/evals/evals.json · ❌ FAIL
body sha256 -> 7e3e4a28268c