Youssef

nitelos

Nitelos, verify the outcome, not the attempt (Python SDK)

Code on GitHub PyPI

Language
Python
Licence
Apache-2.0
Last push
13 Nov 2026
Stars
1
nitelos: Nitelos, verify the outcome, not the attempt (Python SDK)

What it is

nitelos checks whether an AI agent really did what it says it did. It is a small Python SDK: you describe what “done” means, hand it the evidence the agent left behind, and it answers success, failure or abstained.

How it works

  1. Your agent runs a task, on whatever model or framework you use, and reports a result, for example registration succeeded. nitelos never calls that model and treats the report as context, never as proof.
  2. You write down what “done” means as criteria: “no validation error remains on the form”, expressed as /validation_error equals false.
  3. Your runner hands over the evidence it captured: the final page state, screenshots, what each tool returned, records read from external systems.
  4. nitelos compares criteria and evidence and answers success, failure or abstained, with the evidence behind every decided criterion.

nitelos only judges. It never clicks, fetches or retries for the agent; the runner that collects the evidence stays yours (an agent harness, a test suite, a CI job).

Results

Integration What it does Guide
Claude Code plugin Stop hook that blocks a stop when verification fails; /nitelos:init, /nitelos:check docs/claude-code.md
Collectors: git, junit, github, http, Playwright Read-only evidence collection with caps, redaction and provenance docs/collectors.md
MCP server verify_outcome, host-pinned collect, read-only describe_criteria over stdio docs/mcp.md
Inspect AI scorer Verification as a C/I/A score with abstention-aware accuracy docs/inspect.md
DeepEval metric Deterministic 1/0 NitelosOutcomeMetric docs/deepeval.md
LangGraph node verify_outcome_node with routing on the decision docs/langgraph.md
OpenTelemetry One span per verification with allowlisted attributes docs/opentelemetry.md
Langfuse reporting (v0.3) Categorical nitelos_outcome score per verdict, plus trace backfill from the public API README section below
Hook adapters: Codex, OpenCode (v0.3) The verify pipeline behind Codex Stop hooks; observe-only reporting on OpenCode README section below
Pre-action gate (experimental) Act/ask decision math from host-supplied confidence signals docs/gate.md

Try it

pip install nitelos