skill audit

Security Community

Skill notes

Step 1: Discover Skill Files

Find all markdown files under .claude/skills/: All SKILL.md files Companion files: DOSSIER-TEMPLATE.md, REPORT-TEMPLATE.md, and any other .md files If $ARGUMENTS names a specific skill (e.g., test), only audit .claude/skills/$ARGUMENTS/.

Step 2: Extract and Verify File Path References

For each skill file, find references to files/directories. Look for: Backtick-quoted paths containing / or ending in known extensions (.md, .sh, .ts, .tsx, .json, .js, .yaml, .yml) Bare references to known doc files: AGENTS.md, CLAUDE.md, STYLEGUIDE.md, TESTING.md, INTERNATIONALIZATION.md For each path found: Resolve relative to the repo root

Step 3: Verify @console/ Import Paths

Skills contain example import statements that reference real packages. Extract all @console/* import paths from import statements. For each path: @console/internal → verify the subpath exists under frontend/public/ (e.g., @console/internal/module/k8s → frontend/public/module/k8s) @console/shared → verify the subpath exists under frontend/packages/console-shared/ (e.g., @console/shared/src/test-utils/unit-test-utils → frontend/packages/console-shared/src/test-utils/unit-test-utils)

Step 3a: Verify PatternFly Import Paths

Skills may reference PatternFly components in example code. Extract all @patternfly/* import paths. For each path: Check whether the import uses a /deprecated subpath (e.g., @patternfly/react-core/deprecated) — report as a warning since these are slated for removal in future PF versions Verify the package exists in frontend/node_modules/@patternfly/ (or check frontend/package.json dependencies)

Step 3b: Verify $codeRef and Exposed Module References

Skills may contain examples of console-extensions.json entries with $codeRef values, or reference exposed module patterns. For each $codeRef found: Parse the format moduleName.exportName (e.g., exampleFlag.handler) If the skill references a specific plugin, check that plugin's package.json for a matching exposed module name under consolePlugin.exposedModules If a file path is associated with the exposed module, verify the file exists and exports the referenced name

Fluxly profile Runtime and permissions are inferred by Fluxly static checks, not author declarations or runtime test results.
Domain
Security
Author-declared agents
No explicit declaration found; this is not inferred or tested compatibility
Static check
88 / 100 · heuristic scan, not runtime safety proof
Runtime evidence
Not run; no isolated runtime test evidence is currently recorded
Author / version / license
@openshift · no license declared
Token usage
Lean
Setup complexity
Guided setup
External API key
Not required
Operating systems
macOS · Linux · Windows
Runtime requirements
No explicit requirements detected
Detected file/system behavior
  • Read-only
  • Write / modify
  • Shell exec
Network behavior
Local-only
Install commands
None (reference only)

Heads up: Tool permissions are not declared; actual access depends on the runtime.

01
Step 1: Discover Skill Files

Find all markdown files under `.claude/skills/`: All `SKILL.md` files Companion files: `DOSSIER-TEMPLATE.md`, `REPORT-TEMPLATE.md`, and any other `.md` files If `$ARGUMENTS` names a specific skill (e.g., `test`), only audit `.claude/skills/$ARGUMENTS/`.

02
Step 2: Extract and Verify File Path References

For each skill file, find references to files/directories. Look for: Backtick-quoted paths containing `/` or ending in known extensions (`.md`, `.sh`, `.ts`, `.tsx`, `.json`, `.js`, `.yaml`, `.yml`) Bare references to known doc files: `AGENTS.md`, `CLAUDE.md`, `STYLEGUIDE.md`, `TESTING.md`, `INTERNATIONALIZATION.md` For each path found: Resolve relative to the repo root

03
Step 3: Verify @console/ Import Paths

Skills contain example import statements that reference real packages. Extract all `@console/*` import paths from `import` statements. For each path: `@console/internal` → verify the subpath exists under `frontend/public/` (e.g., `@console/internal/module/k8s` → `frontend/public/module/k8s`) `@console/shared` → verify the subpath exists under `frontend/packages/console-shared/` (e.g., `@console/shared/src/test-utils/unit-test-utils` → `frontend/packages/console-shared/src/test-utils/unit-test-utils`)

04
Step 3a: Verify PatternFly Import Paths

Skills may reference PatternFly components in example code. Extract all `@patternfly/*` import paths. For each path: Check whether the import uses a `/deprecated` subpath (e.g., `@patternfly/react-core/deprecated`) — report as a warning since these are slated for removal in future PF versions Verify the package exists in `frontend/node_modules/@patternfly/` (or check `frontend/package.json` dependencies)

05
Step 3b: Verify $codeRef and Exposed Module References

Skills may contain examples of `console-extensions.json` entries with `$codeRef` values, or reference exposed module patterns. For each `$codeRef` found: Parse the format `moduleName.exportName` (e.g., `exampleFlag.handler`) If the skill references a specific plugin, check that plugin's `package.json` for a matching exposed module name under `consolePlugin.exposedModules` If a file path is associated with the exposed module, verify the file exists and exports the referenced name

06
Step 4: Verify yarn/npm Script References

Extract all `yarn <script>` and `npm run <script>` references from each skill file. Compare each script name against the `scripts` object in `frontend/package.json`. Report any that do not exist. Skip: yarn built-in commands (`install`, `add`, `up`, `npm info`, `why`, `--version`).

07
Step 5: Verify Shell Script References

Extract references matching `./something.sh` patterns. Verify each `.sh` file exists at the repo root. Report missing scripts with skill name and line number.

08
Step 6: Verify Claude Code Tool Name References

Skills may reference Claude Code tools by name (e.g., in instructions like "use the Read tool" or "call Grep"). Extract all words that appear to be tool references — look for capitalized names matching the pattern of Claude Code tool names, especially when preceded by words like "use", "call", "run", "the … tool", or appearing in backtick-quoted form. Canonical tool names — self-maintaining approach:

09
Step 7: Verify Hook Command Path References

`.claude/settings.json` and `.claude/settings.local.json` may contain hook definitions with shell commands that reference file paths (e.g., `cat .claude/skills/...`, `bash .claude/scripts/...`). For each hook command: Extract file paths from the command string (look for paths starting with `.claude/`, `frontend/`, `./`, or other repo-relative prefixes) Verify each referenced path exists on disk Report missing paths with the hook name and the full command for context

10
Step 8: Cross-Reference Skill Names in Project Config

Skills are referenced by name in `CLAUDE.md`, `AGENTS.md` (if present), `STYLEGUIDE.md`, `TESTING.md`, and `.claude/settings.json` / `.claude/settings.local.json`. Verify that every skill name mentioned in those files actually exists as a directory under `.claude/skills/`. How to find skill references: Scan each config file for patterns like: Backtick-quoted names followed by "skill" (e.g., `` `gen-rtl-test` skill ``)

11
Step 9: Report Findings

Output a summary grouped by skill, using severity levels to help prioritize fixes: Severity · Meaning · Examples ERROR · Broken reference that will cause the skill to malfunction or give wrong guidance · Deprecated tool name in executable instructions, missing file that the skill tells Claude to read, invalid script reference WARN · Likely stale but not immediately breaking · `/deprecated` PatternFly import, `@console/internal` path that may be alias-resolved, orphaned skill not referenced in config

Discussion

Powered by GitHub Discussions. Sign in with GitHub to comment, react, or subscribe.