Skeptic · open source

Trust, but verify.

Coding agents say the work is done. Skeptic runs it — drives your real, live app the way a user would — and proves each claim is true, or shows you exactly where it breaks.

$ npm i -g @theasanai/skepticGitHub ↗
Why Skeptic

A claim is not a result.

“Done,” “fixed,” “all tests green” — a coding agent writes the report on its own work, and reading its own source code is not proof. Skeptic settles it the only way that counts: it starts your actual app, drives it, and reports back with evidence on disk. CONFIRMED, REFUTED, or UNVERIFIABLE — never a shrug.

How it works

Five stages, one verdict.

Wired into Claude Code as a Stop hook, it runs on every turn the agent calls done — no command to remember.

  1. 01

    The Stop hook fires

    Claude Code finishes a turn and runs skeptic run, handing over the transcript and project directory. Verification starts the moment the agent says it’s done.

  2. 02

    Claims get extracted

    A small model reads the turn’s final message and the files it touched, and lists every verifiable claim about the running app — UI, API, test, or build — marking which ones are actually checkable.

  3. 03

    An independent agent tries to refute them

    A fresh, isolated run (launched with --safe-mode, so it never inherits the project’s own hooks or config) is told to disprove each claim against the live app: start it, drive the UI with skeptic probe on Playwright, curlthe API, run the project’s own tests. Reading source doesn’t count as verification.

  4. 04

    The evidence gate rules

    A verdict only stands if it cites a real artifact on disk inside .skeptic/evidence/— no file, no verdict, it downgrades to UNVERIFIABLE. And a CONFIRMED that the probe’s own captured facts contradict — a 4xx/5xx response, a console error, a failed step — is downgraded to REFUTED.

  5. 05

    You get a verdict with receipts

    Results are written to .skeptic/last-run.json and .skeptic/report.md, and a terminal summary prints in the same Stop-hook output you already watch — each claim with the repro steps and evidence files behind it.

    • ✓ CONFIRMED
    • ✗ REFUTED
    • ? UNVERIFIABLE
    • ⏱ TIMEOUT
Install & requirements

Global install, one browser.

A single global CLI plus Playwright’s Chromium. Then let skeptic doctor confirm the machine is ready.

$ npm i -g @theasanai/skeptic$ npx playwright install chromium$ skeptic doctor

The package isn’t on npm yet — until it publishes, install from source: clone the repo, then npm i && npm run build && npm link.

Trust boundary.Skeptic runs the target repository’s own code — it starts the app and runs its test and build commands — and launches an agent with shell and file tools against it. Only run Skeptic on repositories you trust.

Using it

One init, then it just runs.

initwires the hook and writes the config; from there every “done” gets checked automatically.

Run skeptic init inside your project. It installs a Stop hook in .claude/settings.json, writes a skeptic.json with the dev command and URL autodetected from your package.json (Vite :5173, Astro :4321, Next :3000, and more — it prints the URL it chose so you can confirm it), and adds .skeptic/ to your .gitignore. Then work with Claude Code as usual.

$ skeptic init

One-time setup — installs the Stop hook, writes skeptic.json, and ignores the .skeptic/ evidence directory.

$ skeptic run

What the hook calls on every turn. It reads the transcript on stdin, extracts and verifies the claims, and prints the verdict. It never gates — it always exits 0, so a failed check reports loudly but never blocks your agent.

$ skeptic report

Re-print the last run’s verdict in the terminal — the same summary the hook showed.

$ skeptic report --json

The same run as machine-readable JSON on stdout for CI. Exits non-zero if there’s no run or the report is corrupt, so a pipeline gets a clean signal.

$ skeptic doctor

Preflight check: the claude CLI with --safe-mode, Playwright’s Chromium, and whether your dev-server URL is reachable. Exits non-zero if a blocking dependency is missing.

$skeptic probe <url>

Drive a page by hand. Opens <url> in headless Chromium, runs your --steps steps.json, and prints facts as JSON — final URL, console errors, request statuses — plus a full-page screenshot into --out as --name.png.

Drive a page and capture the facts

$ skeptic probe http://localhost:4321 --steps steps.json --out .skeptic/evidence --name claim-1
$ skeptic probe --help  # step syntax: goto / fill / click / waitFor

skeptic.json — the config init writes

{
  "startCommand": "npm run dev",
  "url": "http://localhost:4321",
  "timeoutMs": 300000,
  "claudeArgs": ["--allowedTools", "Bash(node:*) Bash(npm:*) …"]
}

The headless verifier runs non-interactively, so nobody is there to approve commands. claudeArgs grants it — in plain sight — exactly what verification needs: common runtimes and package managers to start your app and run its tests, curl/wget to hit it, skeptic probe to drive the page, and read-only inspection. Deliberately absent: a bare Bash rule, rm, sudo, chmod, Docker, and any git write. Edit the list for your stack; [] opts out entirely.

Gate a CI job on the verdicts

$ skeptic doctor || exit 1
$ skeptic report --json > report.json
$ node -e 'const r=require("./report.json");process.exit(r.verdicts.every(v=>v.verdict==="CONFIRMED")?0:1)'
Open source

Read it. Fork it. Trust it because you can see it.

Skeptic is MIT-licensed and public — the whole verifier, the evidence gate, the permission defaults, all of it. A tool that judges your agent’s honesty shouldn’t ask you to take its own on faith.

Read the source on GitHub ↗

An agent that doesn’t take your agent’s word for it.

Skeptic is open source today. Join the waitlist and we’ll reach out first with the managed verifier and what’s next.