Verify in CI

ctrlrun verify runs the kernel’s own failure scenarios against your policy, in a scratch store, with fake executors and nothing reached outside the runner. It reports what passed, what failed, and what it could not check. The badge it writes means one thing: the declared guarantees pass. Prerequisites: a repository with a ctrlrun.yaml, and GitHub Actions.

Run it locally first

runnable

schema: ctrlrun.policy/v2

actions:
  stripe.refund:
    effect: "refund:{payment_id}"
    rules:
      - when: { amount_gte: 0, amount_lte: 50000 }
        decision: allow
      - decision: approve
  k8s.delete_namespace:
    effect: "namespace:{cluster}:{name}"
    decision: approve

runnable

ctrlrun verify

CTRLRun verify — ctrlrun 0.10.0, catalogue ctrlrun.guarantees/v6
policy     ./ctrlrun.yaml (ctrlrun.policy/v2, mode: enforce)
authority  none
store      sqlite, scratch (created and destroyed for this run)

G1   mutated approval refused         PASS  k8s.delete_namespace
G2   replayed approval refused        PASS  k8s.delete_namespace
G3   duplicate effect refused         PASS  k8s.delete_namespace
G4   one winner under concurrency     PASS  k8s.delete_namespace (8 processes)
G5   ambiguous blocks a blind retry   PASS  k8s.delete_namespace
G6   unknown action refused           PASS
G7   no principal refused             PASS  k8s.delete_namespace
G8   expired authority refused        N/A   no authority section
G9   delegation cannot escalate       N/A   no authority section
G10  unknown exception is ambiguous   PASS  k8s.delete_namespace
G11  an altered receipt is detected   PASS  k8s.delete_namespace
G12  a byte written is ambiguous      PASS  k8s.delete_namespace
G13  clock divergence is named        N/A   the store verify was given reads only the application's clock, so there is no second clock to diverge from; pass --store-url postgresql://… to grade this
G14  token changes across a renewal   PASS  k8s.delete_namespace (attempt 1 and its renewal carry different tokens)
G15  renewal past the ceiling refused N/A   no action verify can drive to allow or approve declares both `effect:` and `max_attempts`
G16  a moved fingerprint is refused   PASS  k8s.delete_namespace
                                            (verify supplies its own precondition provider; whether
                                            your @protect declares one is in your code, which verify
                                            does not read. The gateway and the ACS hook cannot name a
                                            provider at all, and refuse an approval that carries a
                                            fingerprint)
G17  an unentitled approver refused   N/A   no cited control names an approver role
G18  the requester cannot approve     PASS  k8s.delete_namespace
G19  one principal counts once        N/A   no action requires more than one approval
G20  revoked before its exp: no       PASS  k8s.delete_namespace
                                            (G20 is graded against a revocation feed verify supplies:
                                            whether this deployment configures one is a fact about its
                                            own code, which verify cannot read)
G21  unapproved policy decides no     PASS  stripe.refund
                                            (G21 is graded with require_approved_policy set by verify:
                                            whether this deployment sets it is a fact about its own
                                            code, which verify cannot read)
G22  held budget refuses next reserve N/A   no authority section
G23  a failing scope provider refuses N/A   no action this configuration admits carries a resource
G24  grant refused off its task       N/A   no authority section
G25  a hop narrows or it is refused   N/A   no authority section
G26  a hop is named on both sides     N/A   no authority section
G27  a swapped upstream is denied     N/A   no action entry pins an upstream
                                            (a token is unique only as far as your effect keys are:
                                            two stores sharing a provider account must not produce the
                                            same effect-key string for different effects, and nothing
                                            here can check that)

15/15 declared guarantees pass. 12 not applicable: G8, G9, G13, G15, G17, G19, G22, G23, G24, G25, G26, G27.

The first line is on stderr, from G7’s own scenario: an action with no principal is refused and the refusal logs, which is the guarantee passing. Every row names k8s.delete_namespace because verify takes the first action that fits, in alphabetical order.Four guarantees are not applicable: no authority: section (G8, G9), no second clock in a scratch store (G13), no max_attempts to refuse past (G15). Each is reported with its reason and excluded from the denominator. Never 16/16, and no flag folds them in.

Add the action

name: CTRLRun verify

on: [push, pull_request]

jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: CTRLRun/ctrlrun@v0.6.1
        with:
          policy: ctrlrun.yaml

The action installs ctrlrun, runs ctrlrun verify --json --junit, renders the job summary and the badge JSON from that one report, and uploads the three as an artifact. It fails the job when a guarantee failed or the configuration was refused, and succeeds when guarantees are N/A: green means nothing that could be checked was wrong.

Read the outputs

Input	Default
`policy`	`ctrlrun.yaml`	the document to verify
`authority`	the policy’s own section	a standalone authority document
`only`	all	comma-separated guarantee ids; a partial run writes no badge
`install`	`ctrlrun`	the pip requirement; `.` verifies with the checkout
`badge-path`	`verify-badge.json`	where the Shields endpoint JSON goes

Outputs: passed, failed, applicable, not-applicable, badge-message, report-path. The ref pins the action’s steps and not the package they install: install is unpinned by default, so @v0.6.1 still takes whatever PyPI serves on the day. Pin both — install: ctrlrun==0.6.1 — where the run has to be reproducible, and pin the action by commit rather than by tag where the ref has to be immovable.

Publish the badge, if you want it

The action writes the badge JSON and never publishes it, because publishing needs contents: write and asking every user for that is a bad trade for a tool about least privilege. Publish it yourself from a job that runs only on pushes to your default branch, with contents: write on that job alone, and point Shields’ endpoint badge at the raw file. Get the badge has the copy-paste workflow.

What verify cannot see

Your executors, your reconcile hooks, where you put the decorator, your deployment, and whether your policy is the right policy. An executor that raises NotExecuted after the remote acted is invisible to it. The badge does not mean secure, safe, compliant, certified or audited.

If it didn’t work

Exit 2, mode: observe: verify refuses an observed configuration, because observe mode executes what enforce mode would refuse and nothing can be proved about it.
Exit 2, nothing was checked and nothing is claimed: no guarantee was applicable, so 0/0 — which is never a pass. In practice that means an actions: map with nothing in it; a policy with one allow action still exercises the fail-closed and unknown-outcome guarantees.
Exit 1: a guarantee failed. The report names the scenario and the action; that is a bug report, not a configuration problem.

Exit codes.
What the badge means: the guarantee catalogue in full.
Get started · Why.

Start

Get started

Production

Concepts

Guides

MCP

Cookbook

Verify

Compare

FAQ

What verify cannot see

If it didn’t work

Next

​What verify cannot see

​If it didn’t work

​Next

What verify cannot see

If it didn’t work

Next