dev
- Repo stars 3,367
- License MIT
- Author repo atopile
- Domain
- Engineering
- Author-declared agents
- No explicit declaration found; this is not inferred or tested compatibility
- Static check
- 94 / 100 · heuristic scan, not runtime safety proof
- Runtime evidence
- Not run; no isolated runtime test evidence is currently recorded
- Author / version / license
- @atopile · MIT
- Token usage
- Lean
- Setup complexity
- Plug-and-play
- External API key
- Not required
- Operating systems
- Unspecified (assume cross-platform)
- Runtime requirements
- Python
- Detected file/system behavior
-
- Read-only
- Write / modify
- Env read
- Network behavior
- External requests
- Install commands
- None (reference only)
Heads up: Tool permissions are not declared; actual access depends on the runtime.
# Dev Module
This skill is written for LLMs working inside this repo. It focuses on the fastest, most reliable inner loop:
- rebuild Zig bindings when needed
- run the repo’s orchestrated test runner (`ato dev test --llm`, not raw test output)
- use the generated test reports (`artifacts/test-report.json`, `artifacts/test-report.html`, `artifacts/test-report.llm.json`)
- discover and use `ConfigFlag`s correctly (and inventory them repo-wide)
## Quick Start
```bash
source .venv/bin/activate
ato dev compile
ato dev test --llm -k solver
ato dev test --llm --view HEAD --open
ato dev test --reuse --baseline HEAD~1
ato dev flags
```
## Relevant Files
- CLI commands: `src/atopile/cli/dev.py`
- `ato dev compile` (triggers Zig build via `import faebryk.core.zig`)
- `ato dev test --llm` (runs `test/runner/main.py` with args; supports baseline/CI report helpers)
- Zig build-on-import glue: `src/faebryk/core/zig/__init__.py` (`ZIG_NORECOMPILE`, `ZIG_RELEASEMODE`)
- Config flags utility: `src/faebryk/libs/util.py` (`ConfigFlag`, `ConfigFlagInt`, …)
- Test runner + reports: `test/runner/main.py` (`artifacts/test-report.json`, `artifacts/test-report.html`, `artifacts/test-report.llm.json`)
- CI artifacts definition: `.github/workflows/pytest.yml` (`test-report.json`, `test-report.html`)
## Dependants (Call Sites)
- **CI/CD**: The `dev` commands are the primary interface for GitHub Actions workflows.
- **Local Development**: Developers use `ato dev compile` after modifying Zig code.
## How to Work With / Develop / Test
### Core Commands
- `ato dev compile`: compile native extensions (graph/typegraph/sexp bindings).
- `ato dev test --llm`: runs the orchestrated test runner (defaults to `-p test -p src`); supports:
- `-k` filter (`-- -k ...` also works via passthrough args)
- `--baseline` comparisons (commit hash or `HEAD~N` style)
- `--view` / `--open` to fetch and open the `test-report.html` artifact from GitHub Actions (requires `gh` CLI)
- `--ci` to apply the CI marker expression (`not not_in_ci and not regression and not slow`)
- `--direct -k <testname>` to run a single test via `test/runtest.py` (tight single-test loops)
### Test Reports (JSON as source of truth)
Local test runs write:
- `artifacts/test-report.json` (single source of truth; outcomes/durations/memory/baseline compare status + stdout/stderr/logs/tracebacks; see `tests[].output_full`)
- `artifacts/test-report.html` (human dashboard; derived from JSON; controlled by `FBRK_TEST_GENERATE_HTML=1`)
- `artifacts/test-report.llm.json` (LLM-friendly; derived from JSON; ANSI stripped logs)
CI uploads both artifacts (see `.github/workflows/pytest.yml`):
- `test-report.json`
- `test-report.html`
Notes for LLM debugging:
- Prefer `artifacts/test-report.json` or `artifacts/test-report.llm.json` over raw output; they include structured failures, logs, baseline compare, and collection errors.
- The HTML is best for quickly scanning long-running tests, worker crashes, and per-test output.
Remote/baseline behavior:
- `ato dev test --llm --baseline <commit>` uses the **CI `test-report.json` artifact** as the baseline (requires `gh` CLI).
- `ato dev test --llm --view <commit> --open` currently fetches/opens **only** the HTML artifact; for JSON, download the `test-report.json` artifact via `gh run download`.
- `ato dev test --reuse --baseline <commit>` rebuilds JSON/HTML/LLM against a baseline without rerunning tests.
- `ato dev test --keep-open` keeps the live report server running after tests finish.
Useful test-runner environment variables (see `test/runner/main.py`):
- `FBRK_TEST_REPORT_INTERVAL` (seconds; report refresh cadence)
- `FBRK_TEST_LONG_THRESHOLD` (seconds; “long test” threshold)
- `FBRK_TEST_WORKERS` (`0` = cpu count, negative scales workers)
- `FBRK_TEST_GENERATE_HTML` (`1/0`)
- `FBRK_TEST_PERIODIC_HTML` (`1/0`)
- `FBRK_TEST_OUTPUT_MAX_BYTES` (truncate preview output used by HTML; `tests[].output_full` remains complete)
- `FBRK_TEST_OUTPUT_TRUNCATE_MODE` (`head` or `tail`)
- `FBRK_TEST_BIND_HOST` (orchestrator bind host; default `0.0.0.0`)
- `FBRK_TEST_REPORT_HOST` (host used in printed report URL; default bind host)
- `FBRK_TEST_PERF_THRESHOLD_PERCENT` (default `0.30`)
- `FBRK_TEST_PERF_MIN_TIME_DIFF_S` (default `1.0`)
- `FBRK_TEST_PERF_MIN_MEMORY_DIFF_MB` (default `50.0`)
LLM quick usage:
- `artifacts/test-report.llm.json` is always generated (ANSI stripped, full tests + logs).
- `ato dev test --llm` prints a concise summary + schema + jq hints (stdout only).
- jq recipes are embedded in the report under `llm.jq_recipes`.
- Auto-LLM: `ato dev test` enables the summary automatically when running under claude-code/codex-cli/cursor.
- Force on/off via `FBRK_TEST_LLM=1` or `FBRK_TEST_LLM=0`.
### ConfigFlags (how to use + how to inventory)
`ConfigFlag` is the repo’s “toggle-by-env-var” mechanism. The environment variable name is the first argument to `ConfigFlag(...)`.
Usage:
```bash
export SOME_FLAG=1
```
Inventory all ConfigFlags in-tree (preferred over trying to maintain a manual list):
```bash
ato dev flags
```
Prefer using `ato dev flags` when you want the full picture (types/defaults/descriptions + callsite counts) in one place.
High-leverage flags you’ll use often:
- Zig build: `ZIG_NORECOMPILE`, `ZIG_RELEASEMODE`
- Solver debug: `SLOG`, `SVERBOSE_TABLE`, `SPRINT_START`, `SMAX_ITERATIONS`, `SSHOW_SS_IS`
- Logs: `COLOR_LOGS`, `LOG_TIME`, `LOG_FILEINFO`
### Development Workflow
1. **Zig Changes**: Edit files under `src/faebryk/core/zig/src/` -> Run `ato dev compile`.
2. **Profiling**: If something is slow, use `ato dev profile <command>` to generate a flamegraph or stats.
### Testing
- Main test entrypoint: `ato dev test --llm`.
- If you change CLI behavior, add/adjust tests under `test/` that exercise the command surface.
## Best Practices
- **Use ConfigFlags**: For experimental features or verbose debugging, use a `ConfigFlag` instead of commenting out code.
- **Compile often**: Zig errors won’t be caught by Python tooling.
Sections and files
Listed in source order.
sections -> Quick Start → Relevant Files → Dependants (Call Sites) → How to Work With / Develop / Test → Core Commands → Test Reports (JSON as source of truth)
terms -> CI/CD · Local Development · CI test-report.json artifact · only · Zig Changes · Profiling · Use ConfigFlags · Compile often
files/cmd -> ato dev test --llm · artifacts/test-report.json · artifacts/test-report.html · artifacts/test-report.llm.json · ConfigFlag · src/atopile/cli/dev.py · ato dev compile · import faebryk.core.zig
body sha256 -> 9b071861e237
Relevant Files
CLI commands:
src/atopile/cli/dev.pyato dev compile(triggers Zig build viaimport faebryk.core.zig)ato dev test --llm(runstest/runner/main.pywith args; supports baseline/CI report helpers) Zig build-on-import glue:src/faebryk/core/zig/__init__.py(ZIG_NORECOMPILE,ZIG_RELEASEMODE) Config flags utility:src/faebryk/libs/util.py(ConfigFlag,ConfigFlagInt, …)Dependants (Call Sites)
CI/CD: The
devcommands are the primary interface for GitHub Actions workflows. Local Development: Developers useato dev compileafter modifying Zig code.Core Commands
ato dev compile: compile native extensions (graph/typegraph/sexp bindings).ato dev test --llm: runs the orchestrated test runner (defaults to-p test -p src); supports:-kfilter (-- -k ...also works via passthrough args)--baselinecomparisons (commit hash orHEAD~Nstyle)--view/--opento fetch and open thetest-report.htmlartifact from GitHub Actions (requiresghCLI)Test Reports (JSON as source of truth)
Local test runs write:
artifacts/test-report.json(single source of truth; outcomes/durations/memory/baseline compare status + stdout/stderr/logs/tracebacks; seetests[].output_full)artifacts/test-report.html(human dashboard; derived from JSON; controlled byFBRK_TEST_GENERATE_HTML=1)artifacts/test-report.llm.json(LLM-friendly; derived from JSON; ANSI stripped logs) CI uploads both artifacts (see.github/workflows/pytest.yml):ConfigFlags (how to use + how to inventory)
ConfigFlagis the repo’s “toggle-by-env-var” mechanism. The environment variable name is the first argument toConfigFlag(...). Usage: Inventory all ConfigFlags in-tree (preferred over trying to maintain a manual list): Prefer usingato dev flagswhen you want the full picture (types/defaults/descriptions + callsite counts) in one place. High-leverage flags you’ll use often: