benchmark

Other Community

Skill notes

Usage

/benchmark asr — transcribe test audio, report RTF /benchmark tts — synthesize test text, report RTF /benchmark vad — VAD on VoxConverse (all engines) /benchmark diarize — DER on VoxConverse (requires downloaded test set)

Performance targets (M2 Max)

Module · Metric · Target ASR (Qwen3) · RTF · ~0.06 ASR (Parakeet) · RTF · ~0.025 TTS · RTF · ~0.7 VAD (Silero) · RTF · >20x real-time

Fluxly profile Runtime and permissions are inferred by Fluxly static checks, not author declarations or runtime test results.
Domain
Other
Author-declared agents
No explicit declaration found; this is not inferred or tested compatibility
Static check
94 / 100 · heuristic scan, not runtime safety proof
Runtime evidence
Not run; no isolated runtime test evidence is currently recorded
Author / version / license
@soniqo · Apache-2.0
Token usage
Lean
Setup complexity
Plug-and-play
External API key
Not required
Operating systems
macOS
Runtime requirements
No explicit requirements detected
Detected file/system behavior
  • Read-only
Network behavior
Local-only
Install commands
None (reference only)

Heads up: Tool permissions are not declared; actual access depends on the runtime.

Author source file SKILL.md

# Benchmark

Run benchmarks using the release build. Build first with `/build`.

## Usage

- `/benchmark asr` — transcribe test audio, report RTF
- `/benchmark tts` — synthesize test text, report RTF
- `/benchmark vad` — VAD on VoxConverse (all engines)
- `/benchmark diarize` — DER on VoxConverse (requires downloaded test set)

```bash
module="$ARGUMENTS"
cli=".build/release/speech"

case "$module" in
  asr)
    $cli transcribe Tests/Qwen3ASRTests/Resources/test_audio.wav 2>&1
    ;;
  tts)
    $cli speak "The quick brown fox jumps over the lazy dog." --output /tmp/bench_tts.wav 2>&1
    ;;
  vad)
    python3 scripts/benchmark_vad.py --compare --num-files 5 2>&1
    ;;
  diarize)
    python3 scripts/benchmark_diarization.py --num-files 5 2>&1
    ;;
  *)
    echo "Usage: /benchmark [asr|tts|vad|diarize]"
    ;;
esac
```

## Performance targets (M2 Max)

| Module | Metric | Target |
|--------|--------|--------|
| ASR (Qwen3) | RTF | ~0.06 |
| ASR (Parakeet) | RTF | ~0.025 |
| TTS | RTF | ~0.7 |
| VAD (Silero) | RTF | >20x real-time |
| Diarization | DER | <10% (VoxConverse) |

Discussion

Powered by GitHub Discussions. Sign in with GitHub to comment, react, or subscribe.