benchmark
- Repo stars 749
- License Apache-2.0
- Author repo speech-swift
Benchmark
Run benchmarks using the release build. Build first with /build.
Usage
/benchmark asr— transcribe test audio, report RTF/benchmark tts— synthesize test text, report RTF/benchmark vad— VAD on VoxConverse (all engines)/benchmark diarize— DER on VoxConverse (requires downloaded test set)
module="$ARGUMENTS"
cli=".build/release/speech"
case "$module" in
asr)
$cli transcribe Tests/Qwen3ASRTests/Resources/test_audio.wav 2>&1
;;
tts)
$cli speak "The quick brown fox jumps over the lazy dog." --output /tmp/bench_tts.wav 2>&1
;;
vad)
python3 scripts/benchmark_vad.py --compare --num-files 5 2>&1
;;
diarize)
python3 scripts/benchmark_diarization.py --num-files 5 2>&1
;;
*)
echo "Usage: /benchmark [asr|tts|vad|diarize]"
;;
esac
Performance targets (M2 Max)
| Module | Metric | Target |
|---|---|---|
| ASR (Qwen3) | RTF | ~0.06 |
| ASR (Parakeet) | RTF | ~0.025 |
| TTS | RTF | ~0.7 |
| VAD (Silero) | RTF | >20x real-time |
| Diarization | DER | <10% (VoxConverse) |
- Fluxly category
- Other
- Author-declared agents
- No explicit declaration found; this is not inferred or tested compatibility
- Static check
- 94 / 100 · heuristic scan, not runtime safety proof
- Author / version / license
- @soniqo · Apache-2.0
- Fluxly token estimate
- Lean
- Fluxly setup estimate
- Plug-and-play
- External API key
- No requirement detected
- Detected OS requirements
- macOS
- Runtime requirements
- Unspecified
- Detected file/system behavior
-
- Read-only
- Detected network behavior
- Local-only
- Install commands
- None (reference only)
Profile is derived at build time from SKILL.md and install vectors. Subject to drift from author intent.
Heads up: 未限定 allowed-tools,默认拥有全部工具权限。
The current SKILL.md does not define a fixed output example. Usage
/benchmark asr — transcribe test audio, report RTF /benchmark tts — synthesize test text, report RTF /benchmark vad — VAD on VoxConverse (all engines)
Performance targets (M2 Max)
Module · Metric · Target ASR (Qwen3) · RTF · ~0.06 ASR (Parakeet) · RTF · ~0.025
# Benchmark
Run benchmarks using the release build. Build first with `/build`.
## Usage
- `/benchmark asr` — transcribe test audio, report RTF
- `/benchmark tts` — synthesize test text, report RTF
- `/benchmark vad` — VAD on VoxConverse (all engines)
- `/benchmark diarize` — DER on VoxConverse (requires downloaded test set)
```bash
module="$ARGUMENTS"
cli=".build/release/speech"
case "$module" in
asr)
$cli transcribe Tests/Qwen3ASRTests/Resources/test_audio.wav 2>&1
;;
tts)
$cli speak "The quick brown fox jumps over the lazy dog." --output /tmp/bench_tts.wav 2>&1
;;
vad)
python3 scripts/benchmark_vad.py --compare --num-files 5 2>&1
;;
diarize)
python3 scripts/benchmark_diarization.py --num-files 5 2>&1
;;
*)
echo "Usage: /benchmark [asr|tts|vad|diarize]"
;;
esac
```
## Performance targets (M2 Max)
| Module | Metric | Target |
|--------|--------|--------|
| ASR (Qwen3) | RTF | ~0.06 |
| ASR (Parakeet) | RTF | ~0.025 |
| TTS | RTF | ~0.7 |
| VAD (Silero) | RTF | >20x real-time |
| Diarization | DER | <10% (VoxConverse) | Evidence boundary and execution chain
Author text anchors workflow facts; Fluxly only indexes current sections, terms, files, and commands.
sections -> Usage → Performance targets (M2 Max)
terms -> Run benchmarks using the release build.
files/cmd -> /build · /benchmark asr · /benchmark tts · /benchmark vad · /benchmark diarize
body sha256 -> 06446d3f01a4
Decide Fit First
Design Intent
How To Use It
Boundaries And Review