---
title: Workflows
description: Write CI/CD pipelines for your repositories using spindle workflows.
---


## Pipelines

Spindle workflows allow you to write CI/CD pipelines in a
simple format. They're located in the `.tangled/workflows`
directory at the root of your repository, and are defined
using YAML.

A workflow has a set of common fields that apply no matter
which engine you pick:

- [Trigger](#trigger): A **required** field that defines
  when a workflow should be triggered.
- [Engine](#engine): A **required** field that defines which
  engine a workflow should run on.
- [Clone options](#clone-options): An **optional** field
  that defines how the repository should be cloned.
- [Environment](#environment): An **optional** field that
  allows you to define environment variables.
- [Steps](#steps): An **optional** field that allows you to
  define what steps should run in the workflow.

On top of these, each engine has its own options for things
like dependencies and images. See [Engines](#engines) for
the per-engine fields.

### Trigger

The first thing to add to a workflow is the trigger, which
defines when a workflow runs. This is defined using a `when`
field, which takes in a list of conditions. Each condition
has the following fields:

- `event`: This is a **required** field that defines when
  your workflow should run. It's a list that can take one or
  more of the following values:
  - `push`: The workflow should run every time a commit is
    pushed to the repository.
  - `pull_request`: The workflow should run every time a
    pull request is made or updated.
  - `manual`: The workflow can be triggered manually.
- `branch`: Defines which branches the workflow should run
  for. If used with the `push` event, commits to the
  branch(es) listed here will trigger the workflow. If used
  with the `pull_request` event, updates to pull requests
  targeting the branch(es) listed here will trigger the
  workflow. This field has no effect with the `manual`
  event. Supports glob patterns using `*` and `**` (e.g.,
  `main`, `develop`, `release-*`). Either `branch` or `tag`
  (or both) must be specified for `push` events.
- `tag`: Defines which tags the workflow should run for.
  Only used with the `push` event - when tags matching the
  pattern(s) listed here are pushed, the workflow will
  trigger. This field has no effect with `pull_request` or
  `manual` events. Supports glob patterns using `*` and `**`
  (e.g., `v*`, `v1.*`, `release-**`). Either `branch` or
  `tag` (or both) must be specified for `push` events.

For example, if you'd like to define a workflow that runs
when commits are pushed to the `main` and `develop`
branches, or when pull requests that target the `main`
branch are updated, or manually, you can do so with:

```yaml
when:
  - event: ["push", "manual"]
    branch: ["main", "develop"]
  - event: ["pull_request"]
    branch: ["main"]
```

You can also trigger workflows on tag pushes. For instance,
to run a deployment workflow when tags matching `v*` are
pushed:

```yaml
when:
  - event: ["push"]
    tag: ["v*"]
```

You can even combine branch and tag patterns in a single
constraint (the workflow triggers if either matches):

```yaml
when:
  - event: ["push"]
    branch: ["main", "release-*"]
    tag: ["v*", "stable"]
```

To skip CI for a push, pass a Git push option:

```sh
git push -o skip-ci
```

`ci-skip` is also accepted.

### Engine

Next is the engine on which the workflow should run, defined
using the **required** `engine` field. The currently
supported engines are:

- `nixery`: This uses an instance of
  [Nixery](https://nixery.dev) to run steps, which allows
  you to add [dependencies](#dependencies) from
  Nixpkgs (https://github.com/NixOS/nixpkgs). You can
  search for packages on https://search.nixos.org, and
  there's a pretty good chance the package(s) you're looking
  for will be there.
  See [Nixery engine](#nixery-engine).
- `microvm`: Runs the whole workflow inside its own
  microVM. Has configuration features for NixOS images
  that will let you enable services, do Docker-in-VM, etc.
  See [microVM engine](#microvm-engine).

Example:

```yaml
engine: "nixery"
```

Each engine also adds its own workflow fields (dependencies,
images, services, and so on). These are documented under
[Engines](#engines).

### Clone options

When a workflow starts, the first step is to clone the
repository. You can customize this behavior using the
**optional** `clone` field. It has the following fields:

- `skip`: Setting this to `true` will skip cloning the
  repository. This can be useful if your workflow is doing
  something that doesn't require anything from the
  repository itself. This is `false` by default.
- `depth`: This sets the number of commits, or the "clone
  depth", to fetch from the repository. For example, if you
  set this to 2, the last 2 commits will be fetched. By
  default, the depth is set to 1, meaning only the most
  recent commit will be fetched, which is the commit that
  triggered the workflow.
- `submodules`: If you use Git submodules
  (https://git-scm.com/book/en/v2/Git-Tools-Submodules)
  in your repository, setting this field to `true` will
  recursively fetch all submodules. This is `false` by
  default.

The default settings are:

```yaml
clone:
  skip: false
  depth: 1
  submodules: false
```

### Environment

The `environment` field allows you define environment
variables that will be available throughout the entire
workflow. **Do not put secrets here, these environment
variables are visible to anyone viewing the repository. You
can add secrets for pipelines in your repository's
settings.**

Example:

```yaml
environment:
  GOOS: "linux"
  GOARCH: "arm64"
  NODE_ENV: "production"
  MY_ENV_VAR: "MY_ENV_VALUE"
```

By default, the following environment variables are set:

- `CI` - Always set to `true` to indicate a CI environment
- `TANGLED_PIPELINE_ID` - The AT URI of the current pipeline
- `TANGLED_PIPELINE_KIND` - One of `push`, `pull_request` or
  `manual`
- `TANGLED_REPO_KNOT` - The repository's knot hostname
- `TANGLED_REPO_DID` - The DID of the repository owner
- `TANGLED_REPO_NAME` - The name of the repository
- `TANGLED_REPO_DEFAULT_BRANCH` - The default branch of the
  repository
- `TANGLED_REPO_URL` - The full URL to the repository

These variables are only available when the pipeline is
triggered by a push:

- `TANGLED_REF` - The full git reference (e.g.,
  `refs/heads/main` or `refs/tags/v1.0.0`)
- `TANGLED_REF_NAME` - The short name of the reference
  (e.g., `main` or `v1.0.0`)
- `TANGLED_REF_TYPE` - The type of reference, either
  `branch` or `tag`
- `TANGLED_SHA` - The commit SHA that triggered the pipeline
- `TANGLED_COMMIT_SHA` - Alias for `TANGLED_SHA`

These variables are only available when the pipeline is
triggered by a pull request:

- `TANGLED_PR_SOURCE_BRANCH` - The source branch of the pull
  request
- `TANGLED_PR_TARGET_BRANCH` - The target branch of the pull
  request
- `TANGLED_PR_SOURCE_SHA` - The commit SHA of the source
  branch

### Steps

The `steps` field allows you to define what steps should run
in the workflow. It's a list of step objects, each with the
following fields:

- `name`: This field allows you to give your step a name.
  This name is visible in your workflow runs, and is used to
  describe what the step is doing.
- `command`: This field allows you to define a command to
  run in that step. The step is run in a Bash shell, and the
  logs from the command will be visible in the pipelines
  page on the Tangled website. Any dependencies you added in
  your engine's section (see [Engines](#engines)) will be
  available to use here.
- `environment`: Similar to the global
  [environment](#environment) config, this **optional**
  field is a key-value map that allows you to set
  environment variables for the step. **Do not put secrets
  here, these environment variables are visible to anyone
  viewing the repository. You can add secrets for pipelines
  in your repository's settings.**

Example:

```yaml
steps:
  - name: "Build backend"
    command: "go build"
    environment:
      GOOS: "darwin"
      GOARCH: "arm64"
  - name: "Build frontend"
    command: "npm run build"
    environment:
      NODE_ENV: "production"
```

## Engines

The common fields above apply to every workflow. Each engine
then adds its own fields on top. Pick an engine with the
[`engine`](#engine) field and use the matching section below.

### Nixery engine

#### Dependencies

When you're running a workflow you'll usually need additional
dependencies. The `dependencies` field lets you define which
dependencies to get, and from where. It's a key-value map,
with the key being the registry to fetch dependencies from,
and the value being the list of dependencies to fetch.

The registry URL syntax can be found [on the nix
manual](https://nix.dev/manual/nix/2.18/command-ref/new-cli/nix3-registry-add).

Say you want to fetch Node.js and Go from `nixpkgs`, and a
package called `my_pkg` you've made from your own registry
at your repository at
`https://tangled.org/@example.com/my_pkg`. You can define
those dependencies like so:

```yaml
dependencies:
  # nixpkgs
  nixpkgs:
    - nodejs
    - go
  # unstable
  nixpkgs/nixpkgs-unstable:
    - bun
  # custom registry
  git+https://tangled.org/@example.com/my_pkg:
    - my_pkg
```

Now these dependencies are available to use in your
workflow!

#### Complete nixery workflow

```yaml
# .tangled/workflows/build.yml

when:
  - event: ["push", "manual"]
    branch: ["main", "develop"]
  - event: ["pull_request"]
    branch: ["main"]

engine: "nixery"

# using the default values
clone:
  skip: false
  depth: 1
  submodules: false

dependencies:
  # nixpkgs
  nixpkgs:
    - nodejs
    - go
  # custom registry
  git+https://tangled.org/@example.com/my_pkg:
    - my_pkg

environment:
  GOOS: "linux"
  GOARCH: "arm64"
  NODE_ENV: "production"
  MY_ENV_VAR: "MY_ENV_VALUE"

steps:
  - name: "Build backend"
    command: "go build"
    environment:
      GOOS: "darwin"
      GOARCH: "arm64"
  - name: "Build frontend"
    command: "npm run build"
    environment:
      NODE_ENV: "production"
```

If you want another example of a workflow, you can look at
the one [Tangled uses to build the
project](https://tangled.org/@tangled.org/core/blob/master/.tangled/workflows/build.yml).

### microVM engine

#### Image

A workflow picks the image to boot with the top-level `image`
field:

```yaml
engine: microvm
image: nixos
```

There are two flavours of images:

- **NixOS images** (e.g. `nixos`): the whole guest is built
  with Nix, so you can configure it from the workflow file
  itself. The `dependencies`, `services`, `virtualisation`,
  `registry` and `caches` fields below are all understood
  here, and the guest builds and activates that configuration
  before any of your steps run.
- **Non-NixOS images** (e.g. `alpine`): there's no NixOS to
  configure, so the workflow-level config fields above have
  no effect. You still get a full machine to run steps in.

The available image names depend on what the spindle operator
has installed. `nixos` and `alpine` are examples. If `image`
is omitted, the spindle's configured default image is used.

#### Dependencies

On the microVM engine, `dependencies` is a flat list of
packages that are made available to every step. This field
only applies to **NixOS images**; for other images you can
use the package manager included in a step.

The guest builds a [`nix develop`](https://nix.dev/manual/nix/2.18/command-ref/new-cli/nix3-develop)-style
devshell from your dependencies and uses it for each step,
so you can, for example, add `pkg-config` and `openssl` and
have the `openssl-sys` crate while compiling a Rust project
just work.

A bare name like `go` is looked up in nixpkgs. You can also
point at any flake with the `flakeref#attr` syntax, so
`github:nixos/nixpkgs#hello` pulls `hello` straight out of
that flake.

```yaml
dependencies:
  - go
  - github:nixos/nixpkgs#hello
```

#### Registry

The `registry` field remaps flake references, the same way
`nix registry` does. This lets you pin or alias the flakes
used by `dependencies`.

For example, pin `nixpkgs` to `nixos-unstable` so that the
bare `go` above resolves from unstable, and alias your own
flake so you can use `myflake#tool` in `dependencies`:

```yaml
registry:
  nixpkgs: github:nixos/nixpkgs/nixos-unstable
  myflake: github:me/x
```

#### Caches

The `caches` field is a map of Nix binary cache URL to its
trusted public key. These are fed into the spindle's read
proxy, so the guest can substitute prebuilt paths from them
instead of building everything from scratch.

```yaml
caches:
  https://nix-community.cachix.org: "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs="
```

#### Services and virtualisation

The `services` and `virtualisation` fields are passed straight
through to NixOS. Anything you could write under
`services.*` or `virtualisation.*` in a NixOS configuration,
you can write here, and it's brought up before any of your
steps run.

As a convenience, `true` works as shorthand for
`.enable = true` anywhere an `enable` option exists (e.g.
`virtualisation.docker: true`).

```yaml
services:
  postgresql:
    enable: true
    ensureDatabases: ["spindle-workflow"]
    ensureUsers:
      - name: spindle-workflow
        ensureDBOwnership: true

virtualisation:
  docker: true
```

#### Recipes

##### Lint, test and build a Node project

```yaml
when:
  - event: ["push", "pull_request"]
    branch: ["main"]

engine: microvm
image: nixos

dependencies:
  - pnpm

steps:
  - name: "Install dependencies"
    command: pnpm install --frozen-lockfile
  - name: "Lint and test"
    command: |
      pnpm run lint
      pnpm test
  - name: "Build"
    command: pnpm run build
```

##### Check formatting

```yaml
when:
  - event: ["push", "pull_request"]
    branch: ["main"]

engine: microvm
image: alpine # slimmer image for checking the formatting

steps:
  - name: "Install go"
    command: apk add go
  - name: "Check formatting"
    command: test -z $(gofmt -l .)
```

##### Build a Rust project that links OpenSSL

```yaml
when:
  - event: ["push", "pull_request"]
    branch: ["main"]

engine: microvm
image: nixos

dependencies:
  - gcc
  - cargo
  - rustc
  - clippy
  - rustfmt
  - pkg-config # exports PKG_CONFIG_PATH for the libraries below
  - openssl # the C library + headers openssl-sys links against

steps:
  - name: "Check formatting"
    command: cargo fmt --check
  - name: "Clippy"
    command: cargo clippy --all-targets -- -D warnings
  - name: "Test"
    command: cargo test --all
  - name: "Release build"
    command: cargo build --release
```

##### Run migrations and integration tests against PostgreSQL

```yaml
when:
  - event: ["push", "pull_request"]
    branch: ["main"]

engine: microvm
image: nixos

environment:
  DATABASE_URL: "postgresql:///spindle-workflow?host=/run/postgresql"

dependencies:
  - gcc
  - cargo
  - rustc
  - pkg-config
  - openssl
  - sqlx-cli

services:
  postgresql:
    enable: true
    # has to be same name as the user for peer auth to work automatically
    ensureDatabases: ["spindle-workflow"]
    ensureUsers:
      - name: spindle-workflow
        ensureDBOwnership: true

steps:
  - name: "Run migrations"
    command: sqlx migrate run
  - name: "Integration tests"
    command: cargo test --all
```

##### Build and push a Docker image on tag

```yaml
when:
  - event: ["push"]
    tag: ["v*"]

engine: microvm
image: nixos

virtualisation:
  docker: true

steps:
  - name: "Build and push to ghcr.io"
    command: |
      set -euo pipefail

      echo "$REGISTRY_TOKEN" | docker login ghcr.io -u "$REGISTRY_USER" --password-stdin
      image="ghcr.io/$REGISTRY_USER/myapp:$TANGLED_REF_NAME"

      docker build -t "$image" -t "ghcr.io/$REGISTRY_USER/myapp:latest" .
      docker push "$image"
      docker push "ghcr.io/$REGISTRY_USER/myapp:latest"
```

##### Deploy to Cloudflare Workers on tag

```yaml
# .tangled/workflows/deploy.yml
when:
  - event: ["push"]
    tag: ["v*"]

engine: microvm
image: nixos

dependencies:
  - pnpm

steps:
  - name: "Install dependencies"
    command: pnpm install --frozen-lockfile
  - name: "Deploy worker"
    # `wrangler` picks up `CLOUDFLARE_API_TOKEN` from the env.
    # set it under **Settings → Secrets**.
    command: pnpm exec wrangler deploy
```

##### Publish a release artifact

```yaml
when:
  - event: ["push"]
    tag: ["v*"] # trigger on versions

engine: microvm
image: nixos

dependencies:
  - go

steps:
  - name: "Build release binary"
    command: |
      mkdir -p dist
      CGO_ENABLED=0 go build -trimpath -ldflags "-s -w" -o dist/myapp ./cmd/myapp

  - name: "Publish artifact record"
    command: |
      set -euo pipefail
      # change this if you're not on `tngl.sh`
      PDS="https://tngl.sh"
      # also update this to your handle or did
      ATP_IDENTIFIER="user.tngl.sh"
      ARTIFACT_PATH="dist/myapp"
      ARTIFACT_NAME="myapp"

      # set `ATP_APP_PASSWORD` under **Settings → Secrets**
      session=$(curl -fsS -X POST "$PDS/xrpc/com.atproto.server.createSession" \
        -H "Content-Type: application/json" \
        -d "{\"identifier\":\"$ATP_IDENTIFIER\",\"password\":\"$ATP_APP_PASSWORD\"}")
      jwt=$(echo "$session" | jq -r .accessJwt)
      did=$(echo "$session" | jq -r .did)

      # upload the binary as a blob
      blob=$(curl -fsS -X POST "$PDS/xrpc/com.atproto.repo.uploadBlob" \
        -H "Authorization: Bearer $jwt" \
        -H "Content-Type: application/octet-stream" \
        --data-binary @"$ARTIFACT_PATH")

      # note that this requires an annotated tag (`git tag -a v1.0.0 -m ...`)
      tag_hash=$(git rev-parse "$TANGLED_REF_NAME^{tag}")
      tag_bytes=$(printf '%s' "$tag_hash" | xxd -r -p | base64 | tr -d '=')

      # the sh.tangled.repo.artifact record for your artifact
      record=$(jq -n \
        --arg did "$did" \
        --arg tag "$tag_bytes" \
        --arg name "$ARTIFACT_NAME" \
        --arg repo "$TANGLED_REPO_URL" \
        --arg created "$(date -Iseconds)" \
        --argjson blob "$(echo "$blob" | jq .blob)" '{
          repo: $did,
          collection: "sh.tangled.repo.artifact",
          validate: false,
          record: {
            "$type": "sh.tangled.repo.artifact",
            tag: {"$bytes": $tag},
            name: $name,
            repo: $repo,
            artifact: $blob,
            createdAt: $created
          }
        }')

      # create the record on the PDS
      curl -fsS -X POST "$PDS/xrpc/com.atproto.repo.createRecord" \
        -H "Authorization: Bearer $jwt" \
        -H "Content-Type: application/json" \
        -d "$record"
```

