benchmark
- Repo stars 749
- License Apache-2.0
- Author repo speech-swift
- Domain
- Other
- Author-declared agents
- No explicit declaration found; this is not inferred or tested compatibility
- Static check
- 94 / 100 · heuristic scan, not runtime safety proof
- Runtime evidence
- Not run; no isolated runtime test evidence is currently recorded
- Author / version / license
- @soniqo · Apache-2.0
- Token usage
- Lean
- Setup complexity
- Plug-and-play
- External API key
- Not required
- Operating systems
- macOS
- Runtime requirements
- No explicit requirements detected
- Detected file/system behavior
-
- Read-only
- Network behavior
- Local-only
- Install commands
- None (reference only)
Heads up: Tool permissions are not declared; actual access depends on the runtime.
# Benchmark
Run benchmarks using the release build. Build first with `/build`.
## Usage
- `/benchmark asr` — transcribe test audio, report RTF
- `/benchmark tts` — synthesize test text, report RTF
- `/benchmark vad` — VAD on VoxConverse (all engines)
- `/benchmark diarize` — DER on VoxConverse (requires downloaded test set)
```bash
module="$ARGUMENTS"
cli=".build/release/speech"
case "$module" in
asr)
$cli transcribe Tests/Qwen3ASRTests/Resources/test_audio.wav 2>&1
;;
tts)
$cli speak "The quick brown fox jumps over the lazy dog." --output /tmp/bench_tts.wav 2>&1
;;
vad)
python3 scripts/benchmark_vad.py --compare --num-files 5 2>&1
;;
diarize)
python3 scripts/benchmark_diarization.py --num-files 5 2>&1
;;
*)
echo "Usage: /benchmark [asr|tts|vad|diarize]"
;;
esac
```
## Performance targets (M2 Max)
| Module | Metric | Target |
|--------|--------|--------|
| ASR (Qwen3) | RTF | ~0.06 |
| ASR (Parakeet) | RTF | ~0.025 |
| TTS | RTF | ~0.7 |
| VAD (Silero) | RTF | >20x real-time |
| Diarization | DER | <10% (VoxConverse) |
Sections and files
Listed in source order.
sections -> Usage → Performance targets (M2 Max)
terms -> Run benchmarks using the release build.
files/cmd -> /build · /benchmark asr · /benchmark tts · /benchmark vad · /benchmark diarize
body sha256 -> 06446d3f01a4
Usage
/benchmark asr— transcribe test audio, report RTF/benchmark tts— synthesize test text, report RTF/benchmark vad— VAD on VoxConverse (all engines)/benchmark diarize— DER on VoxConverse (requires downloaded test set)Performance targets (M2 Max)
Module · Metric · Target ASR (Qwen3) · RTF · ~0.06 ASR (Parakeet) · RTF · ~0.025 TTS · RTF · ~0.7 VAD (Silero) · RTF · >20x real-time