skill coverage audit

Security Community

Skill notes

Overview

Trace every codepath in a diff, map each path against existing tests, visualize coverage gaps, and auto-generate tests for uncovered paths. Core principle: Trace codepaths in changed files -> Map against existing tests -> Score coverage quality -> Generate tests for gaps -> Report before/after counts.

Caps and Limits

These hard limits prevent runaway analysis: 30 code paths max per audit. If a diff yields more than 30, prioritize by complexity and risk (error paths, security-sensitive branches, public API surfaces first). 20 tests generated max per audit. Focus on highest-impact gaps first. 2-minute per-test exploration cap. If understanding a single test path takes longer than 2 minutes, mark it as "needs manual review" and move on.

Step 1: Identify Changed Files

Determine the diff scope. Use the most relevant source: Filter to source code files only (exclude configs, docs, generated files).

Step 2: Trace Data Flow Through Every Branch

For each changed file, you MUST trace: Conditionals -- Every if/else, switch/case, ternary, and pattern match. Each branch is a separate codepath. Error paths -- Every catch, throw, error return, validation failure, and early return with error. WHY: Error paths are the most common source of untested bugs. Function calls -- Every function invoked from changed code. Trace one level deep into callees to identify integration boundaries.

Step 3: Build the Codepath Inventory

Produce a structured inventory: Type categories: conditional, error, guard, loop-boundary, integration, async Risk assessment: high = user-facing failure or data loss, medium = degraded behavior, low = cosmetic or logging

Fluxly profile Runtime and permissions are inferred by Fluxly static checks, not author declarations or runtime test results.
Domain
Security
Author-declared agents
No explicit declaration found; this is not inferred or tested compatibility
Static check
88 / 100 · heuristic scan, not runtime safety proof
Runtime evidence
Not run; no isolated runtime test evidence is currently recorded
Author / version / license
@nyldn · no license declared
Token usage
Lean
Setup complexity
Guided setup
External API key
Not required
Operating systems
Unspecified (assume cross-platform)
Runtime requirements
No explicit requirements detected
Detected file/system behavior
  • Read-only
  • Write / modify
  • Shell exec
Network behavior
Local-only
Install commands
None (reference only)

Heads up: Tool permissions are not declared; actual access depends on the runtime.

01
Phase 1: Codepath Tracing

Phase 1: Codepath Tracing

02
Step 1: Identify Changed Files

Determine the diff scope. Use the most relevant source: Filter to source code files only (exclude configs, docs, generated files).

03
Step 2: Trace Data Flow Through Every Branch

For each changed file, you MUST trace: Conditionals -- Every `if/else`, `switch/case`, ternary, and pattern match. Each branch is a separate codepath. Error paths -- Every `catch`, `throw`, error return, validation failure, and early return with error. WHY: Error paths are the most common source of untested bugs. Function calls -- Every function invoked from changed code. Trace one level deep into callees to identify integration boundaries.

04
Step 3: Build the Codepath Inventory

Produce a structured inventory: Type categories: `conditional`, `error`, `guard`, `loop-boundary`, `integration`, `async` Risk assessment: `high` = user-facing failure or data loss, `medium` = degraded behavior, `low` = cosmetic or logging

05
Phase 2: Test Mapping and Quality Scoring

Phase 2: Test Mapping and Quality Scoring

06
Step 1: Search for Existing Tests

For each file in the diff, search the test directory for related tests:

07
Step 2: Score Test Quality

For each codepath, assess existing test coverage with this rubric: Rating · Meaning · Criteria ★★★ · Behavior + edge cases tested · Tests assert behavior AND cover boundary conditions, error cases, and edge inputs ★★ · Happy path tested · Tests cover the success path but miss error branches or edge cases ★ · Smoke test only · Test exists but only checks the function runs without error (no meaningful assertions)

08
Step 3: Produce Coverage Map

Map each codepath to its test coverage:

09
Phase 3: Coverage Diagram

After completing the map, produce an ASCII coverage summary. This is the primary output artifact. Break down by category: Use full block for covered and light shade for uncovered. 10-character bar. Always show exact fractions and percentages.

10
Phase 4: Auto-Generate Tests

Phase 4: Auto-Generate Tests

11
Step 1: Detect Project Test Conventions

Before generating any tests, you MUST detect the project's testing patterns:

12
Step 2: Generate Tests for Uncovered Paths

For each no-test and smoke-only codepath, generate a test that: Follows project naming conventions -- same directory structure, same file naming pattern Uses existing test helpers -- import from the same test utilities the project already uses Tests behavior, not implementation -- assert observable outcomes, not internal state Covers the specific gap -- targets the exact branch or error path identified in Phase 1

13
Step 3: Present Generated Tests

For each generated test, show:

14
Step 4: Report Before/After

After generating all tests, show the coverage change:

Discussion

Powered by GitHub Discussions. Sign in with GitHub to comment, react, or subscribe.