skill-evaluator
- Repo stars 386
- Author updated Live
- Author repo skillport
- Domain
- Security
- Compatible agents
-
- Claude Code
- Cursor
- Cline
- Codex
- Windsurf
- Gemini CLI
- +20
- Trust score
- 88 / 100 · community maintained
- Author / version / license
- @gotalab · no license declared
- Token usage
- Lean
- Setup complexity
- Plug-and-play
- External API key
- Not required
- Operating systems
- Windows
- Runtime requirements
- No special requirements
- Permissions
-
- Read-only
- Write / modify
- Network behavior
- Local-only
- Install commands
- 26 variants
Profile is derived at build time from SKILL.md and install vectors. Subject to drift from author intent.
Heads up: 未限定 allowed-tools,默认拥有全部工具权限。
---
name: skill-evaluator
description: Evaluates agent skills against Anthropic's best practices. Use when asked to review, evaluate, a…
category: security
runtime: no special runtime
---
# skill-evaluator output preview
## PART A: Task fit
- Use case: Evaluates agent skills against Anthropic's best practices. Use when asked to review, evaluate, assess, or audit a skill for quality. Analyzes SKILL.md structure, naming conventions, description quality, content organization, and identifies anti-patterns. Produces actionable improvement recommendations..
- Inputs: target material, constraints, expected output, and acceptance criteria.
- Evidence boundary: follow “Quick Start / Evaluation Workflow / Step 1: Automated Validation” and do not present inference as author intent.
## PART B: Execution result
- **01** The card summarizes the use case; runtime output centers on “Evaluates agent skills against Anthropic's best practices. Use when asked to review, evaluate, assess, or audit a skill for quality. Analyzes SKILL.md structure, naming conventions, description quality, content organization, and identifies anti-patterns. Produces actionable improvement recommendations.”.
- **02** When the source has headings, the agent prioritizes “Quick Start / Evaluation Workflow / Step 1: Automated Validation” so the result follows the author’s structure.
- **03** Typical output includes task judgment, concrete steps, required commands or file edits, validation, and follow-up options.
- **04** Risk context follows the fingerprint: read files, write/modify files; mostly runs locally; usually needs no extra API key.
## Running Rules
- read files, write/modify files; mostly runs locally; usually needs no extra API key.
- Validate with a small sample before expanding scope.
- Return the result, validation criteria, and next iteration options. The source does not require a stable slash command. After installation, invoke the skill by name and describe the task.
Name target files or source material, expected output, forbidden changes, and whether network or shell access is allowed. Permission fingerprint: read files, write/modify files.
Start with a small task and check whether the result follows “Quick Start / Evaluation Workflow / Step 1: Automated Validation”. Inspect diffs, logs, previews, or tests before expanding scope.
Confirm the final output includes a concrete result, evidence, and next action. If it stays generic, tighten inputs, boundaries, and acceptance criteria.
---
name: skill-evaluator
description: Evaluates agent skills against Anthropic's best practices. Use when asked to review, evaluate, a…
category: security
source: gotalab/skillport
---
# skill-evaluator
## When to use
- Evaluates agent skills against Anthropic's best practices. Use when asked to review, evaluate, assess, or audit a skil…
- Use it when the task has clear inputs, repeatable steps, and validation criteria.
## What to provide
- Target material, scope, expected result, and forbidden changes.
- Whether network, commands, file writes, or external services are allowed.
## Execution rules
- Organize steps around “Quick Start / Evaluation Workflow / Step 1: Automated Validation” and keep inference separate from source facts.
- read files, write/modify files; mostly runs locally; usually needs no extra API key.
- Validate with a small sample before expanding the task.
## Output requirements
- Return the deliverable, key evidence, validation method, and next action.
- Mark missing information as unknown; do not invent commands, platforms, or dependencies. The author source anchors workflow facts; repository files anchor sources and commands; Fluxly only adds fit, limitations, and quality judgment.
skill "skill-evaluator" {
input -> user goal + target files + boundaries + acceptance criteria
context -> Quick Start / Evaluation Workflow / Step 1: Automated Validation
rules -> SKILL.md triggers / order / output contract
runtime -> no special runtime | read files, write/modify files | mostly runs locally
guardrails -> usually needs no extra API key + small-sample validation + diff/log review
output -> copyable result + checklist + next iteration
} Skill Evaluator (WIP)
Evaluates skills against Anthropic's official best practices for agent skill authoring. Produces structured evaluation reports with scores and actionable recommendations.
Quick Start
- Read the skill's SKILL.md and understand its purpose
- Run automated validation:
scripts/validate_skill.py <skill-path> - Perform manual evaluation against criteria below
- Generate evaluation report with scores and recommendations
Evaluation Workflow
Step 1: Automated Validation
Run the validation script first:
scripts/validate_skill.py <path/to/skill>
This checks:
- SKILL.md exists with valid YAML frontmatter
- Name follows conventions (lowercase, hyphens, max 64 chars)
- Description is present and under 1024 chars
- Body is under 500 lines
- File references are one-level deep
Step 2: Manual Evaluation
Evaluate each dimension and assign a score (1-5):
A. Naming (Weight: 10%)
| Score | Criteria |
|---|---|
| 5 | Gerund form (-ing), clear purpose, memorable |
| 4 | Descriptive, follows conventions |
| 3 | Acceptable but could be clearer |
| 2 | Vague or misleading |
| 1 | Violates naming rules |
Rules: Max 64 chars, lowercase + numbers + hyphens only, no reserved words (anthropic, claude), no XML tags.
Good: processing-pdfs, analyzing-spreadsheets, building-dashboards
Bad: pdf, my-skill, ClaudeHelper, anthropic-tools
B. Description (Weight: 20%)
| Score | Criteria |
|---|---|
| 5 | Clear functionality + specific activation triggers + third person |
| 4 | Good description with some triggers |
| 3 | Adequate but missing triggers or vague |
| 2 | Too brief or unclear purpose |
| 1 | Missing or unhelpful |
Must include: What the skill does AND when to use it. Good: "Extracts text from PDFs. Use when working with PDF documents for text extraction, form parsing, or content analysis." Bad: "A skill for PDFs." or "Helps with documents."
C. Content Quality (Weight: 30%)
| Score | Criteria |
|---|---|
| 5 | Concise, assumes Claude intelligence, actionable instructions |
| 4 | Generally good, minor verbosity |
| 3 | Some unnecessary explanations or redundancy |
| 2 | Overly verbose or confusing |
| 1 | Bloated, explains obvious concepts |
Ask: "Does Claude really need this explanation?" Remove anything Claude already knows.
D. Structure & Organization (Weight: 25%)
| Score | Criteria |
|---|---|
| 5 | Excellent progressive disclosure, clear navigation, optimal length |
| 4 | Good organization, appropriate file splits |
| 3 | Acceptable but could be better organized |
| 2 | Poor organization, missing references, or bloated SKILL.md |
| 1 | No structure, everything dumped in SKILL.md |
Check:
- SKILL.md under 500 lines
- References are one-level deep (no nested chains)
- Long reference files (>100 lines) have table of contents
- Uses forward slashes in all paths
E. Degrees of Freedom (Weight: 10%)
| Score | Criteria |
|---|---|
| 5 | Perfect match: high freedom for flexible tasks, low for fragile operations |
| 4 | Generally appropriate freedom levels |
| 3 | Acceptable but could be better calibrated |
| 2 | Mismatched: too rigid or too loose |
| 1 | Completely wrong freedom level for the task type |
Guideline:
- High freedom (text): Multiple valid approaches, context-dependent
- Medium freedom (parameterized): Preferred pattern exists, some variation OK
- Low freedom (specific scripts): Fragile operations, exact sequence required
F. Anti-Pattern Check (Weight: 5%)
Deduct points for each anti-pattern found:
- Too many options without clear recommendation (-1)
- Time-sensitive information with date conditionals (-1)
- Inconsistent terminology (-1)
- Windows-style paths (backslashes) (-1)
- Deeply nested references (more than one level) (-2)
- Scripts that punt error handling to Claude (-1)
- Magic numbers without justification (-1)
Step 3: Generate Report
Use this template:
# Skill Evaluation Report: [skill-name]
## Summary
- **Overall Score**: X.X/5.0
- **Recommendation**: [Ready for publication / Needs minor improvements / Needs major revision]
## Dimension Scores
| Dimension | Score | Weight | Weighted |
|-----------|-------|--------|----------|
| Naming | X/5 | 10% | X.XX |
| Description | X/5 | 20% | X.XX |
| Content Quality | X/5 | 30% | X.XX |
| Structure | X/5 | 25% | X.XX |
| Degrees of Freedom | X/5 | 10% | X.XX |
| Anti-Patterns | X/5 | 5% | X.XX |
| **Total** | | 100% | **X.XX** |
## Strengths
- [List 2-3 things done well]
## Areas for Improvement
- [List specific issues with actionable fixes]
## Anti-Patterns Found
- [List any anti-patterns detected]
## Recommendations
1. [Priority 1 fix]
2. [Priority 2 fix]
3. [Priority 3 fix]
## Pre-Publication Checklist
- [ ] Description is specific with activation triggers
- [ ] SKILL.md under 500 lines
- [ ] One-level-deep file references
- [ ] Forward slashes in all paths
- [ ] No time-sensitive information
- [ ] Consistent terminology
- [ ] Concrete examples provided
- [ ] Scripts handle errors explicitly
- [ ] All configuration values justified
- [ ] Required packages listed
- [ ] Tested with Haiku, Sonnet, Opus
Score Interpretation
| Score Range | Rating | Action |
|---|---|---|
| 4.5 - 5.0 | Excellent | Ready for publication |
| 4.0 - 4.4 | Good | Minor improvements recommended |
| 3.0 - 3.9 | Acceptable | Several improvements needed |
| 2.0 - 2.9 | Needs Work | Major revision required |
| 1.0 - 1.9 | Poor | Fundamental redesign needed |
References
- references/evaluation-criteria.md - Detailed evaluation criteria with examples
- references/scoring-rubric.md - Complete scoring rubric and edge cases
Examples
See evaluations/ for example evaluation scenarios.
Decide Fit First
Design Intent
How To Use It
Boundaries And Review