skillshare cli e2e test

Engineering Community

Skill notes

Phase 0: Environment Check

Confirm devcontainer is running and get container ID: If empty → prompt user: docker compose -f .devcontainer/docker-compose.yml up -d Ensure CONTAINER is set for all subsequent docker exec calls. Confirm Linux binary is available: Confirm mdproof is installed:

Phase 1: Detect Scope

Preview all available runbooks via the container: This returns JSON with every runbook's steps, commands, and expected assertions — no manual markdown parsing needed. Use this to understand what each runbook covers. Identify recent changes (unstaged + recent commits): Match changes to relevant runbooks (compare changed file paths against step commands in the JSON output).

Phase 2: Select Tests

Prompt user (via AskUserQuestion): Option A: Run existing runbook (list all available + mark those related to recent changes) Option B: Auto-generate new test script based on recent changes Option C: If $ARGUMENTS specifies a runbook, skip to Phase 3

Phase 3: Prepare & Execute

Running existing runbook: Create isolated environment with auto-initialization: Execute the entire runbook via mdproof inside the container: mdproof executes each step (bash -c <command>) in the ssenv-isolated HOME, then returns structured JSON: Analyze the JSON output:

Phase 4: Cleanup & Report

Ask user before cleanup (via AskUserQuestion): Option A: Delete ssenv environment now Option B: Keep for manual debugging (print env name for later ssenv delete) If user chose Option A: Output summary (derived from the runbook JSON output):

Fluxly profile Runtime and permissions are inferred by Fluxly static checks, not author declarations or runtime test results.
Domain
Engineering
Author-declared agents
No explicit declaration found; this is not inferred or tested compatibility
Static check
88 / 100 · heuristic scan, not runtime safety proof
Runtime evidence
Not run; no isolated runtime test evidence is currently recorded
Author / version / license
@JetBrains · no license declared
Token usage
Heavy
Setup complexity
Manual integration
External API key
Not required
Operating systems
Linux · Docker
Runtime requirements
Docker
Detected file/system behavior
  • Read-only
  • Write / modify
  • Shell exec
  • Env read
Network behavior
Local-only
Install commands
None (reference only)

Heads up: Tool permissions are not declared; actual access depends on the runtime.

01
Phase 0: Environment Check

Confirm devcontainer is running and get container ID: If empty → prompt user: `docker compose -f .devcontainer/docker-compose.yml up -d` Ensure `CONTAINER` is set for all subsequent `docker exec` calls. Confirm Linux binary is available: Confirm mdproof is installed:

02
Phase 1: Detect Scope

Preview all available runbooks via the container: This returns JSON with every runbook's steps, commands, and expected assertions — no manual markdown parsing needed. Use this to understand what each runbook covers. Identify recent changes (unstaged + recent commits): Match changes to relevant runbooks (compare changed file paths against step commands in the JSON output).

03
Phase 2: Select Tests

Prompt user (via AskUserQuestion): Option A: Run existing runbook (list all available + mark those related to recent changes) Option B: Auto-generate new test script based on recent changes Option C: If $ARGUMENTS specifies a runbook, skip to Phase 3

04
Phase 3: Prepare & Execute

Running existing runbook: Create isolated environment with auto-initialization: Execute the entire runbook via mdproof inside the container: mdproof executes each step (`bash -c <command>`) in the ssenv-isolated HOME, then returns structured JSON: Analyze the JSON output:

05
Phase 4: Cleanup & Report

Ask user before cleanup (via AskUserQuestion): Option A: Delete ssenv environment now Option B: Keep for manual debugging (print env name for later `ssenv delete`) If user chose Option A: Output summary (derived from the runbook JSON output):

Discussion

Powered by GitHub Discussions. Sign in with GitHub to comment, react, or subscribe.