What it is
Reproducible evaluation for voice agents. Score recorded calls, run scripted conversations over WebSocket, and turn transcription, latency, interruptions and task outcomes into CI gates.
Watch the demo · Quick start · Kaggle notebooks · Live probe · Documentation
How it works
View full-size diagram · Mermaid source
One evaluator, two entry points. A scored probe exports the same replay format used for offline calls. Replay reproduces the legacy scores from that recording; a new live call can vary with the agent, provider and network.
Results
| You have… | Start here | What you get |
|---|---|---|
| Recorded transcripts and timings | voice-eval run calls.jsonl |
Metrics, per-call details and configurable gates |
| A live WebSocket voice agent | voice-eval probe scenario.json |
A scripted call, interruption observations and replayable artifacts |
| No agent or credentials yet | Offline quick start or Kaggle | A complete offline run you can reproduce yourself |
Try it
git clone https://github.com/Gjusev/voice-evals.git
cd voice-evals
python -m pip install voice-evals
voice-eval run evals/data/demo_calls.jsonl

