skill-integration-tester

Data Community

Repo stars 1,568
Author repo claude-trading-skills

Interpretation is structured for decision-making; original keeps the upstream SKILL.md unchanged.

Decide Fit First

Core job: Validate multi-skill workflows defined in CLAUDE.md by checking skill existence, inter-skill data contracts (JSON schema compati…
Best fit: Use it when the task has reusable inputs, steps, and validation criteria rather than a one-off answer.
Avoid forcing it: If the source lacks commands, platform support, or external-service evidence, keep those fields unknown instead of guessing.

Design Intent

Structure: The skill is organized around “Overview”, “When to Use”, “Prerequisites”, “Workflow”, showing how the author expects the agent to judge fit, collect context, and produce verifiable output.
Trigger evidence: Prioritize the author’s wording around when to use it, what context to collect, and what output shape to produce.
Evidence boundary: Author text states facts, repository files prove commands and paths, and Fluxly only adds fit, limits, and usage judgment.

How To Use It

Inputs: Provide target material, scope, expected result, forbidden changes, and validation method.
Invocation: Name skill-integration-tester directly; if the source includes slash commands, start with the command and then add task context.
Validation: Start small and check whether the result follows “Overview / When to Use / Prerequisites” before expanding.

Boundaries And Review

Dependencies: It usually needs no extra API key, so start with a small validation task.
Permissions: Declared permissions include read / write / shell-exec; ask the agent to state file, command, and rollback boundaries before acting.
Quality bar: A useful result names the deliverable, evidence, and next action. Generic prose means the task needs tighter context.

Skill Integration Tester

Overview

Validate multi-skill workflows defined in CLAUDE.md (Daily Market Monitoring, Weekly Strategy Review, Earnings Momentum Trading, etc.) by executing each step in sequence. Check inter-skill data contracts for JSON schema compatibility between output of step N and input of step N+1, verify file naming conventions, and report broken handoffs. Supports dry-run mode with synthetic fixtures.

When to Use

After adding or modifying a multi-skill workflow in CLAUDE.md
After changing a skill's output format (JSON schema, file naming)
Before releasing new skills to verify pipeline compatibility
When debugging broken handoffs between consecutive workflow steps
As a CI pre-check for pull requests touching skill scripts

Prerequisites

Python 3.9+
No API keys required
No third-party Python packages required (uses only standard library)

Workflow

Step 1: Run Integration Validation

Execute the validation script against the project's CLAUDE.md:

python3 skills/skill-integration-tester/scripts/validate_workflows.py \
  --output-dir reports/

This parses all **Workflow Name:** blocks from the Multi-Skill Workflows section, resolves each step's display name to a skill directory, and validates existence, contracts, and naming.

Step 2: Validate a Specific Workflow

Target a single workflow by name substring:

python3 skills/skill-integration-tester/scripts/validate_workflows.py \
  --workflow "Earnings Momentum" \
  --output-dir reports/

Step 3: Dry-Run with Synthetic Fixtures

Create synthetic fixture JSON files for each skill's expected output and validate contract compatibility without real data:

python3 skills/skill-integration-tester/scripts/validate_workflows.py \
  --dry-run \
  --output-dir reports/

Fixture files are written to reports/fixtures/ with _fixture flag set.

Step 4: Review Results

Open the generated Markdown report for a human-readable summary, or parse the JSON report for programmatic consumption. Each workflow shows:

Step-by-step skill existence checks
Handoff contract validation (PASS / FAIL / N/A)
File naming convention violations
Overall workflow status (valid / broken / warning)

Step 5: Fix Broken Handoffs

For each FAIL handoff, verify that:

The producer skill's output contains all required fields
The consumer skill's input parameter accepts the producer's output format
File naming patterns are consistent between producer output and consumer input

Output Format

JSON Report

{
  "schema_version": "1.0",
  "generated_at": "2026-03-01T12:00:00+00:00",
  "dry_run": false,
  "summary": {
    "total_workflows": 8,
    "valid": 6,
    "broken": 1,
    "warnings": 1
  },
  "workflows": [
    {
      "workflow": "Daily Market Monitoring",
      "step_count": 4,
      "status": "valid",
      "steps": [...],
      "handoffs": [...],
      "naming_violations": []
    }
  ]
}

Markdown Report

Structured report with per-workflow sections showing step validation, handoff status, and naming violations.

Reports are saved to reports/ with filenames integration_test_YYYY-MM-DD_HHMMSS.{json,md}.

Resources

scripts/validate_workflows.py -- Main validation script
references/workflow_contracts.md -- Contract definitions and handoff patterns

Key Principles

No API keys required -- all validation is local and offline
Non-destructive -- reads SKILL.md and CLAUDE.md only, never modifies skills
Deterministic -- same inputs always produce same validation results

Fluxly profile Author and license come from source; runtime, permissions, and network are Fluxly detections or estimates

Fluxly category: Data
Author-declared agents: No explicit declaration found; this is not inferred or tested compatibility
Static check: 88 / 100 · heuristic scan, not runtime safety proof
Author / version / license: @tradermonty · no license declared
Fluxly token estimate: Lean
Fluxly setup estimate: Guided setup
External API key: No requirement detected
Detected OS requirements: Unspecified
Runtime requirements: Python >=3.9
Detected file/system behavior: Read-only
Write / modify
Shell exec
Detected network behavior: Local-only
Install commands: None (reference only)

Profile is derived at build time from SKILL.md and install vectors. Subject to drift from author intent.

Heads up: 未限定 allowed-tools，默认拥有全部工具权限。

Output preview skill-integration-tester.preview

# Step 4: Review Results

- Step-by-step skill existence checks
- Handoff contract validation (PASS / FAIL / N/A)
- File naming convention violations
- Overall workflow status (valid / broken / warning)

SKILL.md SKILL.md

# Skill Integration Tester

## Overview

Validate multi-skill workflows defined in CLAUDE.md (Daily Market Monitoring,
Weekly Strategy Review, Earnings Momentum Trading, etc.) by executing each step
in sequence. Check inter-skill data contracts for JSON schema compatibility
between output of step N and input of step N+1, verify file naming conventions,
and report broken handoffs. Supports dry-run mode with synthetic fixtures.

## When to Use

- After adding or modifying a multi-skill workflow in CLAUDE.md
- After changing a skill's output format (JSON schema, file naming)
- Before releasing new skills to verify pipeline compatibility
- When debugging broken handoffs between consecutive workflow steps
- As a CI pre-check for pull requests touching skill scripts

## Prerequisites

- Python 3.9+
- No API keys required
- No third-party Python packages required (uses only standard library)

## Workflow

### Step 1: Run Integration Validation

Execute the validation script against the project's CLAUDE.md:

```bash
python3 skills/skill-integration-tester/scripts/validate_workflows.py \
  --output-dir reports/
```

This parses all `**Workflow Name:**` blocks from the Multi-Skill Workflows
section, resolves each step's display name to a skill directory, and validates
existence, contracts, and naming.

### Step 2: Validate a Specific Workflow

Target a single workflow by name substring:

```bash
python3 skills/skill-integration-tester/scripts/validate_workflows.py \
  --workflow "Earnings Momentum" \
  --output-dir reports/
```

### Step 3: Dry-Run with Synthetic Fixtures

Create synthetic fixture JSON files for each skill's expected output and
validate contract compatibility without real data:

```bash
python3 skills/skill-integration-tester/scripts/validate_workflows.py \

…

Evidence boundary and execution chain

Author text anchors workflow facts; Fluxly only indexes current sections, terms, files, and commands.

sections -> Overview → When to Use → Prerequisites → Workflow → Step 1: Run Integration Validation → Step 2: Validate a Specific Workflow
terms    -> Workflow Name · Fixture files are written to reports/fixtures/ with fixture flag set. · Open the generated Markdown report for a human-readable summary, or parse the JSON report for programmatic consumption. · For each FAIL handoff, verify that: 1. · Structured report with per-workflow sections showing step validation, handoff status, and naming violations. · Reports are saved to reports/ with filenames integrationtestYYYY-MM-DDHHMMSS.{json,md}. · 1. No API keys required -- all validation is local and offline 2.
files/cmd -> Workflow Name: · reports/fixtures/ · fixture · FAIL · reports/ · integrationtestYYYY-MM-DDHHMMSS.{json,md} · scripts/validateworkflows.py · references/workflowcontracts.md
body sha256 -> b055ca45b738