Checks

Checks are the commands a change has to pass before it merges: lint, tests, a build. You declare them once, in .ryph/checks.yml. Ryph runs them on the commits agents ask about, hands back what failed, and reports the result to GitHub, GitLab or Radicle.

Ryph also remembers every check that passed, and on which commit. When an agent fixes a failure, only the checks that still need an answer run. When its pull request opens, the results are already there.

Agents work until it’s green

An agent checks its own work as it goes. It commits, calls ryph__run_checks, and stops. When the checks finish, the result arrives as its next message, with the last lines of output from every check that failed. It fixes the cause, commits, and asks again.

While it works, an agent can ask for only the checks its change is most likely to break. Before it submits, it runs all of them. Its prompt lists what each repository’s checks run, and tells it to make a failing test pass by fixing the code.

A run that asked for checks stays open until the results arrive. If the agent finishes early, it gets one more turn to deal with them.

Passes carry forward

Ryph groups check results into suites. A suite is identified by what was checked:

• the commit
• the version of .ryph/checks.yml it was read from
• the environment image the checks ran in
• the network access and services they had

When a new suite starts on a commit, every check that already passed on that commit, under the same checks file, environment and access, is copied into it with its log. It shows as Inherited, and only the remaining checks start. If every check already passed, the suite passes straight away without starting a container.

Only passes carry forward. A failed check always runs again, since asking again is how you retry a flaky test. Run again, on the suite’s page in Ryph, or a re-run on GitHub, runs everything from scratch. GitLab has no re-run button for Ryph’s checks, so there you run them again from Ryph.

Every suite is listed under Checks in the sidebar, newest first, with filters for repository, state and who asked. A repository’s page under Repos shows its declared checks in the Checks section.

The pull request opens green

When an agent submits its patch, Ryph asks for the full suite on its commit. That’s the suite the agent has just run, so the request joins it. Ryph reports the results as soon as the pull request opens: a check run per check on GitHub, named ryph / <name>, a commit status of the same name on GitLab, or a job run on the commit on Radicle.

Reviewers see green checks the moment the pull request appears. Nothing waits on a second CI run that would only confirm what’s already known.

People’s pull requests get the same checks. Every push to a pull request, GitLab merge request or Radicle patch is checked on its newest commit, as long as its author, and whoever pushed, can command agents. A newer push cancels the checks of the one before. A commit someone already checked from their own machine opens green, too.

Why Ryph can vouch for the result

Usually, CI runs the full suite again even when an agent says its tests passed. That’s reasonable: the agent ran them in its own sandbox, where it could have edited a test, skipped a suite, tested uncommitted changes, or misread the output.

Ryph can reuse its results because it controls everything that decides them:

So a passing check means the commands the base branch declares exited 0 against this commit, in the base’s environment, in a container the agent never touched. That stays true wherever the same commit turns up again, so the result can be reused.

Set up checks

Turn them on

The ryph binary on Docker and Docker Compose run checks by default. On Kubernetes, set checks.enabled: true in the Helm values. Anywhere else, set RYPH_CHECKS=on.

To show results on pull requests, Ryph’s GitHub App needs the Checks: write permission. Without it, checks still run and agents still get their results, but GitHub doesn’t show them.

Declare your checks

Add .ryph/checks.yml to your default branch:

setup:
  - run: npm ci
keep: [node_modules]
checks:
  - name: lint
    run: npm run lint
  - name: test
    run: npm test
    timeout: 20m
    watch: [package.json, 'test/**']
  - name: e2e
    run: npm run e2e
    services: [postgres]

Up to 32 checks run side by side, one container each. The file can also be called .ryph/checks.yaml. Unknown keys are refused with a message that says what’s wrong.

Since the checks file is read from the branch a change goes into, checks apply to pull requests opened once the file is on their target branch.

Flag changes to the tests themselves

Ryph reads each check’s command from the base branch, but the command runs the commit’s code. npm test runs whatever the commit’s package.json says test is, against the commit’s test files. So a patch can delete a failing test, add .skip, or rewrite the test script, and the check passes.

To make edits like that easy to spot, list the files that decide what a check actually tests under watch:

checks:
  - name: test
    run: npm test
    watch: [package.json, 'test/**', '**/*.test.ts', vitest.config.ts]
  - name: lint
    run: npm run lint
    watch: [package.json, eslint.config.js]

When a change touches any of them, the check’s result on GitHub carries a Watched paths changed warning that lists them, and the agent’s results say so too. A reviewer then knows to read those files closely. Everything in .ryph/ is watched for every check.

Patterns match the whole path from the repository root. * and ? match within one directory, and ** matches any number of directories.

Give checks a database

A check that needs Postgres or Redis names a service from .ryph/services.yml:

services:
  postgres:
    uses: postgres@18
    env:
      POSTGRES_PASSWORD: password
      POSTGRES_DB: app_test
    port: 5432

Services start once setup has finished, so setup can’t use them. Running services needs RYPH_SERVICES=on on the deployment.

Keep build output warm

When the repository’s environment image already has the code checked out and built, each check starts from that checkout, moved to the commit under test. keep lists what the check keeps of the image’s ignored files. Everything else ignored is removed.

Only keep what your build tool rebuilds when the sources change. Otherwise a check can pass on output built from older code.

Without keep, the default is [node_modules, target, .venv]. Run your install in setup, so a change to the lockfile is picked up.

Require checks on GitHub

In a branch ruleset, add ryph / <name> for each check you want as a required status check, and set its source to the Ryph app. A pull request then can’t merge until that check has passed on its newest commit.

Require checks on GitLab

Ryph’s commit statuses join the merge request’s pipeline as an external stage. Turn on Pipelines must succeed in the project’s merge request settings, and a merge request can’t merge until they’ve passed.

Ask from your own machine

ryph checks run asks a deployment to run the checks on your current commit, on the same machines a pull request’s checks run on. When you then open the pull request on that commit, it opens green.

ryph checks run
ryph checks run --check test

It needs RYPH_URL and RYPH_TOKEN, an API token. Mint one in Ryph under Account settings → API tokens → Mint a token, and tick checks:request and repos:read. ryph checks wait <id> and ryph checks status <id> follow a suite you started. For a coding agent in your terminal, ryph mcp serves the same as the checks_run and checks_status tools.

If you run Ryph on your laptop, it can send its checks to a deployment and have them run there.

Settings