技能评估员
- 作者仓库星标 386
- 作者仓库 skillport
- 领域
- 安全
- 作者声明的智能体
- 未找到明确声明;不据此推断已兼容或已测试
- 静态检查
- 88 / 100 · 启发式扫描,不代表运行安全
- 运行环境证据
- 未执行;当前没有隔离运行测试证据
- 作者 / 版本 / 许可证
- @gotalab · 未声明许可证
- 上下文用量评级
- 低消耗
- 接入复杂程度
- 即装即用
- 是否需要外部接口密钥
- 不需要
- 兼容的系统
- Windows
- 运行环境要求
- 未检出明确要求
- 检测到的文件与系统行为
-
- 只读
- 允许写入 / 修改
- 网络行为
- 仅限本地
- 安装命令数
- 无(仅作为资料)
需要注意: 未声明工具权限范围;实际权限由运行环境决定。
第 1 步:自动验证
首先运行验证脚本:这会检查: SKILL.md 存在且具有有效的 YAML frontmatter 名称遵循约定(小写、连字符,最多 64 个字符) 描述存在且少于 1024 个字符
第2步:手动评估
评估每个维度并分配分数(1-5): A. 命名(权重:10%) 分数 · 标准 5 · 动名词形式(-ing),目的明确,易于记忆 4 · 描述性,遵循惯例
第 3 步:生成报告
使用这个模板:
# Skill Evaluator (WIP)
Evaluates skills against Anthropic's official best practices for agent skill authoring. Produces structured evaluation reports with scores and actionable recommendations.
## Quick Start
1. Read the skill's SKILL.md and understand its purpose
2. Run automated validation: `scripts/validate_skill.py <skill-path>`
3. Perform manual evaluation against criteria below
4. Generate evaluation report with scores and recommendations
## Evaluation Workflow
### Step 1: Automated Validation
Run the validation script first:
```bash
scripts/validate_skill.py <path/to/skill>
```
This checks:
- SKILL.md exists with valid YAML frontmatter
- Name follows conventions (lowercase, hyphens, max 64 chars)
- Description is present and under 1024 chars
- Body is under 500 lines
- File references are one-level deep
### Step 2: Manual Evaluation
Evaluate each dimension and assign a score (1-5):
#### A. Naming (Weight: 10%)
| Score | Criteria |
|-------|----------|
| 5 | Gerund form (-ing), clear purpose, memorable |
| 4 | Descriptive, follows conventions |
| 3 | Acceptable but could be clearer |
| 2 | Vague or misleading |
| 1 | Violates naming rules |
**Rules**: Max 64 chars, lowercase + numbers + hyphens only, no reserved words (anthropic, claude), no XML tags.
**Good**: `processing-pdfs`, `analyzing-spreadsheets`, `building-dashboards`
**Bad**: `pdf`, `my-skill`, `ClaudeHelper`, `anthropic-tools`
#### B. Description (Weight: 20%)
| Score | Criteria |
|-------|----------|
| 5 | Clear functionality + specific activation triggers + third person |
| 4 | Good description with some triggers |
| 3 | Adequate but missing triggers or vague |
| 2 | Too brief or unclear purpose |
| 1 | Missing or unhelpful |
**Must include**: What the skill does AND when to use it.
**Good**: "Extracts text from PDFs. Use when working with PDF documents for text extraction, form parsing, or content analysis."
**Bad**: "A skill for PDFs." or "Helps with documents."
#### C. Content Quality (Weight: 30%)
| Score | Criteria |
|-------|----------|
| 5 | Concise, assumes Claude intelligence, actionable instructions |
| 4 | Generally good, minor verbosity |
| 3 | Some unnecessary explanations or redundancy |
| 2 | Overly verbose or confusing |
| 1 | Bloated, explains obvious concepts |
**Ask**: "Does Claude really need this explanation?" Remove anything Claude already knows.
#### D. Structure & Organization (Weight: 25%)
| Score | Criteria |
|-------|----------|
| 5 | Excellent progressive disclosure, clear navigation, optimal length |
| 4 | Good organization, appropriate file splits |
| 3 | Acceptable but could be better organized |
| 2 | Poor organization, missing references, or bloated SKILL.md |
| 1 | No structure, everything dumped in SKILL.md |
**Check**:
- SKILL.md under 500 lines
- References are one-level deep (no nested chains)
- Long reference files (>100 lines) have table of contents
- Uses forward slashes in all paths
#### E. Degrees of Freedom (Weight: 10%)
| Score | Criteria |
|-------|----------|
| 5 | Perfect match: high freedom for flexible tasks, low for fragile operations |
| 4 | Generally appropriate freedom levels |
| 3 | Acceptable but could be better calibrated |
| 2 | Mismatched: too rigid or too loose |
| 1 | Completely wrong freedom level for the task type |
**Guideline**:
- High freedom (text): Multiple valid approaches, context-dependent
- Medium freedom (parameterized): Preferred pattern exists, some variation OK
- Low freedom (specific scripts): Fragile operations, exact sequence required
#### F. Anti-Pattern Check (Weight: 5%)
Deduct points for each anti-pattern found:
- [ ] Too many options without clear recommendation (-1)
- [ ] Time-sensitive information with date conditionals (-1)
- [ ] Inconsistent terminology (-1)
- [ ] Windows-style paths (backslashes) (-1)
- [ ] Deeply nested references (more than one level) (-2)
- [ ] Scripts that punt error handling to Claude (-1)
- [ ] Magic numbers without justification (-1)
### Step 3: Generate Report
Use this template:
```markdown
# Skill Evaluation Report: [skill-name]
## Summary
- **Overall Score**: X.X/5.0
- **Recommendation**: [Ready for publication / Needs minor improvements / Needs major revision]
## Dimension Scores
| Dimension | Score | Weight | Weighted |
|-----------|-------|--------|----------|
| Naming | X/5 | 10% | X.XX |
| Description | X/5 | 20% | X.XX |
| Content Quality | X/5 | 30% | X.XX |
| Structure | X/5 | 25% | X.XX |
| Degrees of Freedom | X/5 | 10% | X.XX |
| Anti-Patterns | X/5 | 5% | X.XX |
| **Total** | | 100% | **X.XX** |
## Strengths
- [List 2-3 things done well]
## Areas for Improvement
- [List specific issues with actionable fixes]
## Anti-Patterns Found
- [List any anti-patterns detected]
## Recommendations
1. [Priority 1 fix]
2. [Priority 2 fix]
3. [Priority 3 fix]
## Pre-Publication Checklist
- [ ] Description is specific with activation triggers
- [ ] SKILL.md under 500 lines
- [ ] One-level-deep file references
- [ ] Forward slashes in all paths
- [ ] No time-sensitive information
- [ ] Consistent terminology
- [ ] Concrete examples provided
- [ ] Scripts handle errors explicitly
- [ ] All configuration values justified
- [ ] Required packages listed
- [ ] Tested with Haiku, Sonnet, Opus
```
## Score Interpretation
| Score Range | Rating | Action |
|-------------|--------|--------|
| 4.5 - 5.0 | Excellent | Ready for publication |
| 4.0 - 4.4 | Good | Minor improvements recommended |
| 3.0 - 3.9 | Acceptable | Several improvements needed |
| 2.0 - 2.9 | Needs Work | Major revision required |
| 1.0 - 1.9 | Poor | Fundamental redesign needed |
## References
- [references/evaluation-criteria.md](references/evaluation-criteria.md) - Detailed evaluation criteria with examples
- [references/scoring-rubric.md](references/scoring-rubric.md) - Complete scoring rubric and edge cases
## Examples
See [evaluations/](evaluations/) for example evaluation scenarios.
章节与文件索引
按说明文件中的顺序排列。
快速入门
第 1 步:自动验证
第2步:手动评估
第 3 步:生成报告
分数解读
文件/命令 -> scripts/validate_skill.py <skill-path> · processing-pdfs · analyzing-spreadsheets · building-dashboards · pdf · my-skill · ClaudeHelper · anthropic-tools
内容校验值 -> 47e0b2e5d7de
当前原文的自动摘译,尚未人工审校。
快速入门
阅读技能的 SKILL.md 并了解其目的 运行自动验证:
scripts/validate_skill.py <skill-path>根据以下标准执行手动评估 生成包含分数和建议的评估报告分数解读
分数范围 · 评级 · 行动 4.5 - 5.0 · 优秀 · 准备出版 4.0 - 4.4 · 良好 · 建议进行细微改进 3.0 - 3.9 · 可接受 · 需要进行一些改进 2.0 - 2.9 · 需要工作 · 需要进行重大修改