skill tester ci
- Repo stars 180
- Author repo pskoett-ai-skills
- Domain
- AI
- Author-declared agents
- No explicit declaration found; this is not inferred or tested compatibility
- Static check
- 88 / 100 · heuristic scan, not runtime safety proof
- Runtime evidence
- Not run; no isolated runtime test evidence is currently recorded
- Author / version / license
- @pskoett · no license declared
- Token usage
- Lean
- Setup complexity
- Guided setup
- External API key
- Not required
- Operating systems
- Unspecified (assume cross-platform)
- Runtime requirements
- No explicit requirements detected
- Detected file/system behavior
-
- Read-only
- Write / modify
- Shell exec
- Network behavior
- Local-only
- Install commands
- None (reference only)
Heads up: Tool permissions are not declared; actual access depends on the runtime.
# 2. Workflow Example Compilation
# Extract markdown code blocks with frontmatter
# Copy to temp .github/workflows/
# Run: gh aw compile
# Skill Tester CI
Validates all CI skills (`*-ci`) in this repo. Runs spec validation, compiles gh-aw workflows, and checks CI-specific conventions.
## When to Use
- After adding or modifying a CI skill
- After upgrading `gh-aw` to a new version (API changes may break workflows)
- Before committing CI skill changes
- Before submitting the plugin for Anthropic review
## Prerequisites
- `gh-aw` CLI installed (`gh extension install github/gh-aw`)
- `python3` with `pyyaml` installed
- `quick_validate.py` available at `.claude/skills/skill-creator/scripts/`
## Checks
### 1. Anthropic Spec Validation
Run `quick_validate.py` on every CI skill:
```bash
for d in skills/*-ci/; do
python3 .claude/skills/skill-creator/scripts/quick_validate.py "$d"
done
```
### 2. Workflow Example Compilation
Extract workflow blocks from each CI skill's `references/workflow-example.md` and compile with `gh aw compile`:
```bash
# Extract markdown code blocks with frontmatter
# Copy to temp .github/workflows/
# Run: gh aw compile
```
**Pass criteria:** Zero compilation errors. Warnings are reported but don't fail.
### 3. Permission Checks
CI workflows in strict mode must NOT use write permissions directly. Verify:
- No `issues: write` (use `safe-outputs: add-comment` instead)
- No `pull-requests: write` (use `safe-outputs: create-pull-request-review-comment` instead)
- No `contents: write` unless the workflow creates files (eval-creator create mode)
- Required read permissions present for declared toolsets
### 4. Structural Checks
| Check | Rule | Severity |
|-------|------|----------|
| Has `references/workflow-example.md` | Required for all CI skills | Error |
| Workflow example has frontmatter block | At least one `` ```markdown `` block with `---` | Error |
| Name matches folder | Frontmatter `name` == directory name | Error |
| Description mentions gh-aw | CI skills should reference gh-aw | Warning |
| Has corresponding interactive skill | `foo-ci` should have a `foo` counterpart | Warning |
### 5. Cross-Workflow Validation
For workflows that use `call-workflow`:
- Verify the target workflow exists and has `workflow_call` in its `on:` section
- Verify the target's inputs match what the caller provides
## Output Format
```markdown
## CI Skill Test Results
**Date:** YYYY-MM-DD
**gh-aw version:** vX.Y.Z
**Skills tested:** N
**Passed:** N
**Compile errors:** N
**Spec failures:** N
### Compilation Results
- [skill-name]: ✓ compiled (N KB) | ✗ error: [message]
### Spec Results
- [skill-name]: ✓ valid | ✗ [error]
### Permission Issues
- [skill-name]: [issue]
```
## Running
Invoke manually:
```
/skill-tester-ci
```
Or run the script directly:
```bash
bash skills/skill-tester-ci/scripts/run-tests.sh
```
## What This Skill Does NOT Do
- Does not test interactive skills (use `skill-tester` for those)
- Does not execute workflows against real repos — only compiles them
- Does not modify skills — reports findings only
- Does not test workflow runtime behavior — compilation validates structure and permissions only
Sections and files
Listed in source order.
sections -> When to Use → Prerequisites → Checks → 1. Anthropic Spec Validation → 2. Workflow Example Compilation → 3. Permission Checks
terms -> Pass criteria · Date · gh-aw version · Skills tested · Passed · Compile errors · Spec failures
files/cmd -> *-ci · gh-aw · gh extension install github/gh-aw · python3 · pyyaml · quick_validate.py · .claude/skills/skill-creator/scripts/ · references/workflow-example.md
body sha256 -> 6bbde45e9d07
When to Use
After adding or modifying a CI skill After upgrading
gh-awto a new version (API changes may break workflows) Before committing CI skill changes Before submitting the plugin for Anthropic reviewPrerequisites
gh-awCLI installed (gh extension install github/gh-aw)python3withpyyamlinstalledquick_validate.pyavailable at.claude/skills/skill-creator/scripts/1. Anthropic Spec Validation
Run
quick_validate.pyon every CI skill:2. Workflow Example Compilation
Extract workflow blocks from each CI skill's
references/workflow-example.mdand compile withgh aw compile: Pass criteria: Zero compilation errors. Warnings are reported but don't fail.3. Permission Checks
CI workflows in strict mode must NOT use write permissions directly. Verify: No
issues: write(usesafe-outputs: add-commentinstead) Nopull-requests: write(usesafe-outputs: create-pull-request-review-commentinstead) Nocontents: writeunless the workflow creates files (eval-creator create mode) Required read permissions present for declared toolsets