Pipeline file reference

A pipeline is a file in the repository at .overlane/ci.yml. It is read from the commit being built, never uploaded, so a change to the pipeline goes through review like any other change, and an agent that edits it needs an approval.

An example

version: 1
name: ci
on:
  push:
    branches: [main, "feature/**"]
  pull_request:
concurrency: cancel-superseded
egress:
  - registry.npmjs.org
jobs:
  build:
    runsOn: standard
    timeoutMinutes: 15
    cache:
      keyFiles: [pnpm-lock.yaml]
      paths: [node_modules]
    secrets: [NPM_TOKEN]
    steps:
      - name: install
        run: pnpm install --frozen-lockfile
      - name: bundle
        run: pnpm build
        artifact: dist/app.js
  test:
    needs: [build]
    affectedTests: true
    steps:
      - name: unit
        test: true
        run: pnpm vitest run {affected}
  docs:
    paths: ["docs/**"]
    steps:
      - run: pnpm docs:build
  nightly-soak:
    runsOn: large
    timeoutMinutes: 45
    steps:
      - run: pnpm test:soak

Check a file before you push it, offline, with the same parser the service runs:

overlane ci validate
overlane ci validate --file ci/test.yml

Top-level keys

Job keys

Step keys

The file is a strict subset of YAML: block maps and lists, | blocks, quoted and plain scalars, and lists of scalars on one line. Anchors, tags, several documents and one-line maps are refused. The spellings runs-on, timeout-minutes, key-files, pull_request and affected-tests are accepted too.

Databases for tests

A job that needs a database for its tests declares one with a database block. Overlane provides it, hands your steps its address as DATABASE_URL and deletes it when the job finishes, so you install and start nothing.

version: 1
on:
  push:
jobs:
  test:
    database:
      engine: sqlite
      sizeMb: 256
    steps:
      - name: create and read
        run: |
          curl -s "$DATABASE_URL" -d '{"sql": "CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT NOT NULL)"}'
          curl -s "$DATABASE_URL" -d '{"sql": "INSERT INTO users (name) VALUES (?)", "params": ["ada"]}'
          curl -s "$DATABASE_URL" -d '{"sql": "SELECT id, name FROM users"}'

How a SQLite database works

• A fresh database for every job. Each job in each run starts with an empty database that no other job, run or team can reach. A re-run starts empty again.
• Talk to it over HTTP. DATABASE_URL is an address on the job’s own machine. Send a POST with JSON {"sql": "...", "params": [...]} and read back {"columns": [...], "rows": [[...]], "rowsRead": 0, "rowsWritten": 0}. One statement per request; bind values as params (text, numbers or null) rather than writing them into the statement. A statement that fails answers with a 400 and {"error": {"code": "sql_error", "message": "..."}}, so curl --fail fails the step.
• Values. Rows come back as text, numbers or null. To read binary data, select hex(column).
• It has no credential in it. The address is bare, and the job credential that opens the door stays outside your steps.
• Gone when the job ends. The database is deleted when the job passes, fails, is cancelled or times out. If that ever fails, it deletes itself when the job’s time is up, and a deleted database refuses every later request.

Limits

A write that would pass the size limit is undone and answers 413 with “the job’s database is over its N MB limit; raise database.sizeMb or write less”. A request to a database that is no longer open answers 410.

The job log (overlane ci logs) has a database: line saying where the job’s database runs and why. Every provided database also records how long it took to provision, how much it stored and how it was deleted (and, for SQLite on Workers, how many statements ran), so the cost of database jobs can be tracked.

Postgres

A Postgres block is parsed and checked like any other, so a mistake in it is caught before the run starts. The job gets a fresh PostgreSQL 18 database on its own machine, started before the first step:

version: 1
on:
  push:
jobs:
  test:
    database:
      engine: postgres
      version: "18"
      extensions: [pgcrypto]
    steps:
      - run: pnpm test

• Your usual client. DATABASE_URL is a normal connection string, postgres://postgres:<password>@127.0.0.1:<port>/postgres, so psql, an ORM or a test framework connect as they would to any Postgres. The user is a superuser, so your tests can create databases, schemas and extensions.
• Reachable only from the job. The server listens on 127.0.0.1 only, with a password made for this job alone. The password is masked as *** in the job log, and a job variable or secret named DATABASE_URL never replaces the address Overlane sets.
• A fresh database for every job. Each job starts with an empty cluster. It is stopped and its files deleted when the job passes, fails, is cancelled or times out.
• Built for tests, not for keeping data. Writes are not flushed to disk (fsync is off), which makes tests faster; the data only lives as long as the job.
• Version and extensions. The version is PostgreSQL 18, so version may be left out or set to "18". extensions may list any extension that ships with PostgreSQL, such as pgcrypto, uuid-ossp, pg_trgm, citext, hstore, ltree or postgres_fdw; create it in your tests with CREATE EXTENSION. A job that asks for another version, or for an extension that does not ship with PostgreSQL (such as vector or postgis), is marked unavailable and nothing runs, with the reason in its log.
• Limits. The job’s machine: 8 GiB of memory and 16 GB of disk on standard, 12 GiB and 20 GB on large, shared with your steps. connections and sizeMb are recorded, not enforced.
• If it does not start. The job fails before any step runs, and its log says so: for example the job's Postgres database did not start: it did not accept connections within 30 s; no step ran.
• Where it runs. Every Postgres job runs it on the job’s own machine. Its log says database: runs in the job's container: Postgres on Workers is not available yet.

In the job’s container

placement: container runs the database on the job’s own machine even when it could run on Workers. For Postgres this is what already happens. For SQLite, DATABASE_URL is then a file address, file:/<path>/database.sqlite, for your own SQLite library to open (the HTTP interface above is for SQLite on Workers only). The file starts absent, is yours alone, and is deleted when the job finishes. The job log says database: runs in the job's container because the pipeline asks for placement: container.

Your own database

external connects the job to a Postgres database you run yourself, such as a staging database, instead of one Overlane provides. Store its connection string as a secret, list the secret in the job’s secrets, name it in external, and add the database’s host to egress:

version: 1
egress:
  - db.example.com
on:
  push:
jobs:
  test:
    secrets: [STAGING_DATABASE_URL]
    database:
      engine: postgres
      external: STAGING_DATABASE_URL
    steps:
      - run: pnpm test

printf %s "$STAGING_DATABASE_URL" | overlane ci secret set STAGING_DATABASE_URL --stdin

• The address. The secret holds a connection string such as postgres://user:password@db.example.com:5432/app. Your steps get it as DATABASE_URL, set after every other variable, so a job variable or another secret named DATABASE_URL never replaces it. The secret is also available under its own name.
• Never in the log. The whole connection string is masked as *** in the job log, and so is its password on its own. Overlane never prints the secret; when a check fails, the log names only the secret and the database’s host.
• Your network, checked first. The job connects to the database directly, so its host must be in the pipeline’s egress list and allowed by your workspace’s rules. If it is not, the job is marked unavailable before anything runs and the log says how to fix it, for example the job's network does not allow db.example.com, the host in secret STAGING_DATABASE_URL; add db.example.com to egress in .overlane/ci.yml; nothing ran. A secret that does not hold a postgres:// or postgresql:// connection string is refused the same way.
• Not for pull requests from forks. A fork never receives your secrets, so it never receives your database either: such a job is marked unavailable with a pull request from a fork never receives your own database or its secret STAGING_DATABASE_URL, so nothing ran. Leave external out to give fork pull requests a fresh database instead.
• Yours to manage. Overlane creates, empties and deletes nothing in your database, and sizeMb, version, extensions and connections are not checked against it; its own limits apply. Tests that write to a shared database should clean up after themselves.
• placement cannot be combined with external; the run is refused before it starts, with the line.

Where database jobs ran this week

overlane ci usage shows, for the last 7 days, how many of the project’s jobs had a database, the share that ran on each placement (durable-object-sqlite for SQLite on Workers, container for the job’s own machine, external for your own database) and the most common reasons a database ran in the job’s container:

databases in 7 days: 12 job(s), container 50% (6), durable-object-sqlite 41.7% (5), external 8.3% (1)
in the job's container 6x: runs in the job's container: Postgres on Workers is not available yet

With --json, the same figures are under databases.

Errors cite the line

A file is validated before it runs. Every problem is reported as path:line: message, and a file with an error starts nothing.

version: 1
jobs:
  build:
    timeoutMinutes: 0
    steps:
      - run: pnpm build
        shell: bash

.overlane/ci.yml:4: jobs.build.timeoutMinutes: Too small: expected number to be >=1
.overlane/ci.yml:7: unknown key 'shell'

A secret-looking value in the file is refused as well: declare the secret by name and set it with ci secret set.

Where a job runs

Every job runs on a Container runner: standard on a 2 vCPU, 8 GiB instance and large on a 4 vCPU, 12 GiB instance. Workers Builds stops a build at 20 minutes and is used only for deploys, so no CI job goes there; a job with a timeout over 20 minutes records that as the reason for its runner. overlane ci show prints the runner and the reason for each job.

Runners execute your repository’s code and nothing else. They do not run subscription clients or models.

Command reference: ci validate, ci run, ci show, ci logs and ci list.