What it is
nitelos checks whether an AI agent really did what it says it did. It is a
small Python SDK: you describe what “done” means, hand it the evidence the agent
left behind, and it answers success, failure or abstained.
How it works
- Your agent runs a task, on whatever model or framework you use, and
reports a result, for example
registration succeeded. nitelos never calls that model and treats the report as context, never as proof. - You write down what “done” means as criteria: “no validation error remains
on the form”, expressed as
/validation_error equals false. - Your runner hands over the evidence it captured: the final page state, screenshots, what each tool returned, records read from external systems.
- nitelos compares criteria and evidence and answers
success,failureorabstained, with the evidence behind every decided criterion.
nitelos only judges. It never clicks, fetches or retries for the agent; the runner that collects the evidence stays yours (an agent harness, a test suite, a CI job).
Results
| Integration | What it does | Guide |
|---|---|---|
| Claude Code plugin | Stop hook that blocks a stop when verification fails; /nitelos:init, /nitelos:check |
docs/claude-code.md |
| Collectors: git, junit, github, http, Playwright | Read-only evidence collection with caps, redaction and provenance | docs/collectors.md |
| MCP server | verify_outcome, host-pinned collect, read-only describe_criteria over stdio |
docs/mcp.md |
| Inspect AI scorer | Verification as a C/I/A score with abstention-aware accuracy |
docs/inspect.md |
| DeepEval metric | Deterministic 1/0 NitelosOutcomeMetric |
docs/deepeval.md |
| LangGraph node | verify_outcome_node with routing on the decision |
docs/langgraph.md |
| OpenTelemetry | One span per verification with allowlisted attributes | docs/opentelemetry.md |
| Langfuse reporting (v0.3) | Categorical nitelos_outcome score per verdict, plus trace backfill from the public API |
README section below |
| Hook adapters: Codex, OpenCode (v0.3) | The verify pipeline behind Codex Stop hooks; observe-only reporting on OpenCode | README section below |
| Pre-action gate (experimental) | Act/ask decision math from host-supplied confidence signals | docs/gate.md |
Try it
pip install nitelos

