alice.pds.demo.boltless.dev / core
This repository has no description
0
Fork 0

Configure Feed

Select the types of activity you want to include in your feed.

core / docs / DOCS.md
89 kB 2964 lines
1--- 2title: Tangled docs 3author: The Tangled Contributors 4date: 21 Sun, Dec 2025 5abstract: | 6 Tangled is a decentralized code hosting and collaboration 7 platform. Every component of Tangled is open-source and 8 self-hostable. [tangled.org](https://tangled.org) also 9 provides hosting and CI services that are free to use. 10 11 There are several models for decentralized code 12 collaboration platforms, ranging from ActivityPub’s 13 (Forgejo) federated model, to Radicle’s entirely P2P model. 14 Our approach attempts to be the best of both worlds by 15 adopting the AT Protocol—a protocol for building decentralized 16 social applications with a central identity 17 18 Our approach to this is the idea of “knots”. Knots are 19 lightweight, headless servers that enable users to host Git 20 repositories with ease. Knots are designed for either single 21 or multi-tenant use which is perfect for self-hosting on a 22 Raspberry Pi at home, or larger “community” servers. By 23 default, Tangled provides managed knots where you can host 24 your repositories for free. 25 26 The appview at tangled.org acts as a consolidated "view" 27 into the whole network, allowing users to access, clone and 28 contribute to repositories hosted across different knots 29 seamlessly. 30--- 31 32# Quick start guide 33 34## Login or sign up 35 36You can [login](https://tangled.org) by using your AT Protocol 37account. If you are unclear on what that means, simply head 38to the [signup](https://tangled.org/signup) page and create 39an account. By doing so, you will be choosing Tangled as 40your account provider (you will be granted a handle of the 41form `user.tngl.sh`). 42 43In the AT Protocol network, users are free to choose their account 44provider (known as a "Personal Data Service", or PDS), and 45login to applications that support AT accounts. 46 47You can think of it as "one account for all of the atmosphere"! 48 49If you already have an AT account (you may have one if you 50signed up to Bluesky, for example), you can login with the 51same handle on Tangled (so just use `user.bsky.social` on 52the login page). 53 54## Add an SSH key 55 56Once you are logged in, you can start creating repositories 57and pushing code. Tangled supports pushing git repositories 58over SSH. 59 60First, you'll need to generate an SSH key if you don't 61already have one: 62 63```bash 64ssh-keygen -t ed25519 -C "foo@bar.com" 65``` 66 67When prompted, save the key to the default location 68(`~/.ssh/id_ed25519`) and optionally set a passphrase. 69 70Copy your public key to your clipboard: 71 72```bash 73# on X11 74cat ~/.ssh/id_ed25519.pub | xclip -sel c 75 76# on wayland 77cat ~/.ssh/id_ed25519.pub | wl-copy 78 79# on macos 80cat ~/.ssh/id_ed25519.pub | pbcopy 81``` 82 83Now, navigate to 'Settings' -> 'Keys' and hit 'Add Key', 84paste your public key, give it a descriptive name, and hit 85save. 86 87## Create a repository 88 89Once your SSH key is added, create your first repository: 90 911. Hit the green `+` icon on the topbar, and select 92 repository 932. Enter a repository name 943. Add a description 954. Choose a knotserver to host this repository on 965. Hit create 97 98Knots are self-hostable, lightweight Git servers that can 99host your repository. Unlike traditional code forges, your 100code can live on any server. Read the [Knots](TODO) section 101for more. 102 103## Configure SSH 104 105To ensure Git uses the correct SSH key and connects smoothly 106to Tangled, add this configuration to your `~/.ssh/config` 107file: 108 109``` 110Host tangled.org 111 Hostname tangled.org 112 User git 113 IdentityFile ~/.ssh/id_ed25519 114 AddressFamily inet 115``` 116 117This tells SSH to use your specific key when connecting to 118Tangled and prevents authentication issues if you have 119multiple SSH keys. 120 121Note that this configuration only works for knotservers that 122are hosted by tangled.org. If you use a custom knot, refer 123to the [Knots](TODO) section. 124 125## Push your first repository 126 127Initialize a new Git repository: 128 129```bash 130mkdir my-project 131cd my-project 132 133git init 134echo "# My Project" > README.md 135``` 136 137Add some content and push! 138 139```bash 140git add README.md 141git commit -m "Initial commit" 142git remote add origin git@tangled.org:user.tngl.sh/my-project 143git push -u origin main 144``` 145 146That's it! Your code is now hosted on Tangled. 147 148## Migrating an existing repository 149 150Moving your repositories from GitHub, GitLab, Bitbucket, or 151any other Git forge to Tangled is straightforward. You'll 152simply change your repository's remote URL. At the moment, 153Tangled does not have any tooling to migrate data such as 154GitHub issues or pull requests. 155 156First, create a new repository on tangled.org as described 157in the [Quick Start Guide](#create-a-repository). 158 159Navigate to your existing local repository: 160 161```bash 162cd /path/to/your/existing/repo 163``` 164 165You can inspect your existing Git remote like so: 166 167```bash 168git remote -v 169``` 170 171You'll see something like: 172 173```bash 174origin git@github.com:username/my-project.git (fetch) 175origin git@github.com:username/my-project.git (push) 176``` 177 178Update the remote URL to point to tangled: 179 180```bash 181git remote set-url origin git@tangled.org:user.tngl.sh/my-project 182``` 183 184Verify the change: 185 186```bash 187git remote -v 188``` 189 190You should now see: 191 192```bash 193origin git@tangled.org:user.tngl.sh/my-project (fetch) 194origin git@tangled.org:user.tngl.sh/my-project (push) 195``` 196 197Push all your branches and tags to Tangled: 198 199```bash 200git push -u origin --all 201git push -u origin --tags 202``` 203 204Your repository is now migrated to Tangled! All commit 205history, branches, and tags have been preserved. 206 207## Mirroring a repository to Tangled 208 209If you want to maintain your repository on multiple forges 210simultaneously, for example, keeping your primary repository 211on GitHub while mirroring to Tangled for backup or 212redundancy, you can do so by adding [multiple remotes](https://git-scm.com/docs/git-push#_remotes). 213 214You can configure your local repository to push to both 215Tangled and, say, GitHub. You may already have the following 216setup: 217 218```bash 219$ git remote -v 220origin git@github.com:username/my-project.git (fetch) 221origin git@github.com:username/my-project.git (push) 222``` 223 224Now add Tangled as an additional push URL to the same 225remote: 226 227```bash 228git remote set-url --add --push origin git@tangled.org:user.tngl.sh/my-project 229``` 230 231You also need to re-add the original URL as a push 232destination (Git will now use the original URL to fetch only): 233 234```bash 235git remote set-url --add --push origin git@github.com:username/my-project.git 236``` 237 238Verify your configuration: 239 240```bash 241$ git remote -v 242origin git@github.com:username/my-project.git (fetch) 243origin git@tangled.org:user.tngl.sh/my-project (push) 244origin git@github.com:username/my-project.git (push) 245``` 246 247Notice that there's one fetch URL (the primary remote) and 248two push URLs. Now, whenever you push, Git will 249automatically push to both remotes: 250 251```bash 252git push origin main 253``` 254 255This single command pushes your `main` branch to both GitHub 256and Tangled simultaneously. 257 258To push all branches and tags: 259 260```bash 261git push origin --all 262git push origin --tags 263``` 264 265If you prefer more control over which remote you push to, 266you can maintain separate remotes: 267 268```bash 269git remote add github git@github.com:username/my-project.git 270git remote add tangled git@tangled.org:user.tngl.sh/my-project 271``` 272 273Then push to each explicitly: 274 275```bash 276git push github main 277git push tangled main 278``` 279 280# Hosting websites on Tangled 281 282You can serve static websites directly from your git repositories on 283Tangled. If you've used GitHub Pages or Codeberg Pages, this should feel 284familiar. 285 286## Overview 287 288Every user gets a sites domain. If you signed up through Tangled's own 289PDS (`tngl.sh`), your sites domain is automatically 290`<your-handle>.tngl.sh` no setup needed. Otherwise, you can claim a 291`<subdomain>.tngl.io` domain from your settings. 292 293You can serve multiple sites per domain: 294 295- One **index site** served at the root of your domain (e.g. 296 `alice.tngl.sh`) 297- Any number of **sub-path sites** served under the repository name 298 (e.g. `alice.tngl.sh/my-project`) 299 300## Claiming a domain 301 302If you don't have a `tngl.sh` handle, you need to claim a domain before 303publishing sites: 304 3051. Go to **Settings → Sites** 3062. Enter a subdomain (e.g. `alice` to claim `alice.tngl.io`) 3073. Click **claim** 308 309You can only hold one domain at a time. Releasing a domain puts it in a 31030-day cooldown before anyone else can claim it. 311 312## Configuring a site for a repository 313 3141. Navigate to your repository 3152. Go to **Settings → Sites** 3163. Choose a **branch** to deploy from 3174. Set the **deploy directory** — the path within the repository 318 containing your `index.html`. Use `/` for the root, or a subdirectory 319 like `/docs` or `/public` 3205. Choose the **site type**: 321 - **Index site** — served at the root of your domain (e.g. 322 `alice.tngl.sh`) 323 - **Sub-path site** — served under the repository name (e.g. 324 `alice.tngl.sh/my-project`) 3256. Click **save** 326 327The site will be deployed automatically. You can see the status of your 328previous deploys in the **Recent Deploys** section at the bottom of the 329page. 330 331Sites are redeployed automatically on every push to the configured 332branch. 333 334## Custom domains 335 336Tangled currently doesn't support custom domains for sites. This will be 337added in a future update. 338 339## Deploy directory 340 341The deploy directory is the path within your repository that Tangled 342serves as the site root. It must contain an `index.html`. 343 344| Deploy directory | Result | 345|---|---| 346| `/` | Serves the repository root | 347| `/docs` | Serves the `docs/` subdirectory | 348| `/public` | Serves the `public/` subdirectory | 349 350Directories are served with automatic `index.html` resolution -- a 351request to `/about` will serve `/about/index.html` if it exists. 352 353## Site types 354 355| Type | URL | 356|---|---| 357| Index site | `alice.tngl.sh` | 358| Sub-path site | `alice.tngl.sh/my-project` | 359 360Only one repository can be the index site for a given domain at a time. 361If another repository already holds the index site, you will see a 362notice in the settings and only the sub-path option will be available. 363 364## Deploy triggers 365 366A deployment is triggered automatically when: 367 368- You push to the configured branch 369- You change the site configuration (branch, deploy directory, or site 370 type) 371 372## Disabling a site 373 374To stop serving a site, go to **Settings → Sites** in your repository 375and click **Disable**. This removes the site configuration and stops 376serving the site. The deployed files are also deleted from storage. 377 378Releasing your domain from **Settings → Sites** at the account level 379will disable all sites associated with it and delete their files. 380 381 382# Knot self-hosting guide 383 384So you want to run your own knot server? Great! Here are a few prerequisites: 385 3861. A server of some kind (a VPS, a Raspberry Pi, etc.). Preferably running a Linux distribution of some kind. 3872. A (sub)domain name. People generally use `knot.example.com`. 3883. A valid SSL certificate for your domain. 389 390## NixOS 391 392Refer to the [knot 393module](https://tangled.org/tangled.org/core/blob/master/nix/modules/knot.nix) 394for a full list of options. Sample configurations: 395 396- [The test VM](https://tangled.org/tangled.org/core/blob/master/nix/vm.nix#L85) 397- [@pyrox.dev/nix](https://tangled.org/pyrox.dev/nix/blob/c2b644c214d278af12523618de952ee2eab1af3d/hosts/marvin/services/tangled.nix#L15-26) 398 399## Docker 400 401Refer to 402[@tangled.org/knot-docker](https://tangled.org/@tangled.org/knot-docker). 403Note that this is community maintained. 404 405## Manual setup 406 407First, clone this repository: 408 409``` 410git clone https://tangled.org/@tangled.org/core 411``` 412 413Then, build the `knot` CLI. This is the knot administration 414and operation tool. For the purpose of this guide, we're 415only concerned with these subcommands: 416 417- `knot server`: the main knot server process, typically 418 run as a supervised service 419- `knot guard`: handles role-based access control for git 420 over SSH (you'll never have to run this yourself) 421- `knot keys`: fetches SSH keys associated with your knot; 422 we'll use this to generate the SSH 423 `AuthorizedKeysCommand` 424 425``` 426cd core 427export CGO_ENABLED=1 428go build -o knot ./cmd/knot 429``` 430 431Next, move the `knot` binary to a location owned by `root` -- 432`/usr/local/bin/` is a good choice. Make sure the binary itself is also owned by `root`: 433 434``` 435sudo mv knot /usr/local/bin/knot 436sudo chown root:root /usr/local/bin/knot 437``` 438 439This is necessary because SSH `AuthorizedKeysCommand` requires [really 440specific permissions](https://stackoverflow.com/a/27638306). The 441`AuthorizedKeysCommand` specifies a command that is run by `sshd` to 442retrieve a user's public SSH keys dynamically for authentication. Let's 443set that up. 444 445``` 446sudo tee /etc/ssh/sshd_config.d/authorized_keys_command.conf <<EOF 447Match User git 448 AuthorizedKeysCommand /usr/local/bin/knot keys -o authorized-keys 449 AuthorizedKeysCommandUser nobody 450EOF 451``` 452 453Then, reload `sshd`: 454 455``` 456sudo systemctl reload ssh 457``` 458 459Next, create the `git` user. We'll use the `git` user's home directory 460to store repositories: 461 462``` 463sudo adduser git 464``` 465 466Create `/home/git/.knot.env` with the following, updating the values as 467necessary. The `KNOT_SERVER_OWNER` should be set to your 468DID, you can find your DID in the [Settings](https://tangled.sh/settings) page. 469 470``` 471KNOT_REPO_SCAN_PATH=/home/git 472KNOT_SERVER_HOSTNAME=knot.example.com 473APPVIEW_ENDPOINT=https://tangled.org 474KNOT_SERVER_OWNER=did:plc:foobar 475KNOT_SERVER_INTERNAL_LISTEN_ADDR=127.0.0.1:5444 476KNOT_SERVER_LISTEN_ADDR=127.0.0.1:5555 477``` 478 479If you run a Linux distribution that uses systemd, you can 480use the provided service file to run the server. Copy 481[`knotserver.service`](https://tangled.org/tangled.org/core/blob/master/systemd/knotserver.service) 482to `/etc/systemd/system/`. Then, run: 483 484``` 485systemctl enable knotserver 486systemctl start knotserver 487``` 488 489The last step is to configure a reverse proxy like Nginx or Caddy to front your 490knot. Here's an example configuration for Nginx: 491 492``` 493server { 494 listen 80; 495 listen [::]:80; 496 server_name knot.example.com; 497 498 location / { 499 proxy_pass http://localhost:5555; 500 proxy_set_header Host $host; 501 proxy_set_header X-Real-IP $remote_addr; 502 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 503 proxy_set_header X-Forwarded-Proto $scheme; 504 } 505 506 # wss endpoint for git events 507 location /events { 508 proxy_set_header X-Forwarded-For $remote_addr; 509 proxy_set_header Host $http_host; 510 proxy_set_header Upgrade websocket; 511 proxy_set_header Connection Upgrade; 512 proxy_pass http://localhost:5555; 513 } 514 # additional config for SSL/TLS go here. 515} 516 517``` 518 519Remember to use Let's Encrypt or similar to procure a certificate for your 520knot domain. 521 522You should now have a running knot server! You can finalize 523your registration by hitting the `verify` button on the 524[/settings/knots](https://tangled.org/settings/knots) page. This simply creates 525a record on your PDS to announce the existence of the knot. 526 527### Custom paths 528 529(This section applies to manual setup only. Docker users should edit the mounts 530in `docker-compose.yml` instead.) 531 532Right now, the database and repositories of your knot lives in `/home/git`. You 533can move these paths if you'd like to store them in another folder. Be careful 534when adjusting these paths: 535 536- Stop your knot when moving data (e.g. `systemctl stop knotserver`) to prevent 537 any possible side effects. Remember to restart it once you're done. 538- Make backups before moving in case something goes wrong. 539- Make sure the `git` user can read and write from the new paths. 540 541#### Database 542 543As an example, let's say the current database is at `/home/git/knotserver.db`, 544and we want to move it to `/home/git/database/knotserver.db`. 545 546Copy the current database to the new location. Make sure to copy the `.db-shm` 547and `.db-wal` files if they exist. 548 549``` 550mkdir /home/git/database 551cp /home/git/knotserver.db* /home/git/database 552``` 553 554In the environment (e.g. `/home/git/.knot.env`), set `KNOT_SERVER_DB_PATH` to 555the new file path (_not_ the directory): 556 557``` 558KNOT_SERVER_DB_PATH=/home/git/database/knotserver.db 559``` 560 561#### Repositories 562 563As an example, let's say the repositories are currently in `/home/git`, and we 564want to move them into `/home/git/repositories`. 565 566Create the new folder, then move the existing repositories (if there are any): 567 568``` 569mkdir /home/git/repositories 570# move all DIDs into the new folder; these will vary for you! 571mv /home/git/did:plc:wshs7t2adsemcrrd4snkeqli /home/git/repositories 572``` 573 574In the environment (e.g. `/home/git/.knot.env`), update `KNOT_REPO_SCAN_PATH` 575to the new directory: 576 577``` 578KNOT_REPO_SCAN_PATH=/home/git/repositories 579``` 580 581Similarly, update your `sshd` `AuthorizedKeysCommand` to use the updated 582repository path: 583 584``` 585sudo tee /etc/ssh/sshd_config.d/authorized_keys_command.conf <<EOF 586Match User git 587 AuthorizedKeysCommand /usr/local/bin/knot keys -o authorized-keys -git-dir /home/git/repositories 588 AuthorizedKeysCommandUser nobody 589EOF 590``` 591 592Make sure to restart your SSH server! 593 594#### MOTD (message of the day) 595 596To configure the MOTD used ("Welcome to this knot!" by default), edit the 597`/home/git/motd` file: 598 599``` 600printf "Hi from this knot!\n" > /home/git/motd 601``` 602 603Note that you should add a newline at the end if setting a non-empty message 604since the knot won't do this for you. 605 606## Secure Mode 607 608Secure Mode isolates each `git` subprocess to the repository it is 609operating on, using two mechanisms: 610 611- **Linux Landlock** restricts the filesystem paths the subprocess 612 can access -- it can only read/write its own repository and the 613 system directories it needs to run. 614- **UID isolation** runs each subprocess as a virtual UID assigned 615 to the repository owner, so that repositories belonging to 616 different owners are isolated from each other at the OS level 617 even if Landlock were somehow bypassed. 618 619Secure Mode requires: 620 621- Linux kernel >= 5.19 (Landlock V2). This is the minimum needed 622 for `git push` to work, because receive-pack's quarantine 623 migration uses cross-directory rename which requires the 624 Landlock `REFER` access right (added in V2). Kernels 5.13-5.18 625 support Landlock V1 and clones will work, but pushes will fail 626 with cross-device link errors. On kernels without any Landlock 627 support (< 5.13), the sandbox call is a no-op: UID isolation 628 still applies but no filesystem restriction is enforced. 629- `CAP_SETUID`, `CAP_SETGID`, and `CAP_CHOWN` available to the 630 knot process. The NixOS module grants these automatically; for 631 manual setups see the `setcap` step below. 632 633### NixOS 634 635Add `server.secureMode = true;` to your knot module configuration: 636 637```nix 638services.tangled.knot = { 639 server.secureMode = true; 640 # ... other options 641}; 642``` 643 644The NixOS module handles everything else automatically: 645 646- Grants the required capabilities to the knot service via 647 `AmbientCapabilities` in the systemd unit. 648- Installs a capability-bearing wrapper at 649 `/run/wrappers/bin/knot` via `security.wrappers`, so that 650 SSH-invoked git operations (pushes) also run under the correct 651 UID without requiring the service to run as root. 652- Runs `knot migrate-isolation` at service start to chown 653 existing repositories to their virtual UIDs. 654 655### Manual setup 656 657**Step 1.** Grant the required capabilities to the knot binary. 658This allows the knot process to switch to virtual UIDs at runtime 659without running as root. You will need to repeat this step 660whenever the binary is updated. 661 662``` 663sudo setcap cap_setuid,cap_setgid,cap_chown+eip /usr/local/bin/knot 664``` 665 666**Step 2.** Run the migration tool to assign virtual UIDs to all 667existing repositories and set their filesystem permissions. This 668must be run as root: 669 670``` 671sudo knot migrate-isolation \ 672 --git-dir /home/git \ 673 --db /home/git/knotserver.db \ 674 --internal-api 127.0.0.1:5444 675``` 676 677You can re-run this at any time with `--force` to reapply 678permissions (e.g. after a manual repair or after updating the 679binary). 680 681**Step 2a.** Ensure the home directory is traversable by 682non-group users. Git subprocesses run as virtual UIDs that are 683not in the git group, and they need to resolve 684`$HOME/.config/git/config` to load the global config: 685 686``` 687sudo chmod o+x /home/git 688``` 689 690This adds only the execute bit, not read -- the virtual UIDs can 691traverse to known paths but cannot list directory contents. 692 693**Step 3.** Enable Secure Mode in your environment file: 694 695``` 696KNOT_SERVER_SECURE_MODE=true 697``` 698 699Or pass it as a flag: 700 701``` 702knot server --secure-mode 703``` 704 705**Step 4.** Regenerate the `AuthorizedKeysCommand` with the 706`-secure-mode` flag. This causes `knot keys` to emit guard 707command lines that include `-secure-mode`, so SSH pushes also 708get UID isolation: 709 710``` 711sudo tee /etc/ssh/sshd_config.d/authorized_keys_command.conf <<EOF 712Match User git 713 AuthorizedKeysCommand /usr/local/bin/knot keys \ 714 -o authorized-keys -secure-mode 715 AuthorizedKeysCommandUser nobody 716EOF 717``` 718 719Reload `sshd` after making this change. 720 721> **Note:** the server will refuse to start in Secure Mode if any 722> repositories have not yet been isolation-migrated. Re-run 723> `migrate-isolation` if you see this error. 724 725## Troubleshooting 726 727If you run your own knot, you may run into some of these 728common issues. You can always join the 729[IRC](https://web.libera.chat/#tangled) or 730[Discord](https://chat.tangled.org/) if this section does 731not help. 732 733### Unable to push 734 735If you are unable to push to your knot or repository: 736 7371. First, ensure that you have added your SSH public key to 738 your account 7392. Check to see that your knot has synced the key by running 740 `knot keys` 7413. Check to see if git is supplying the correct private key 742 when pushing: `GIT_SSH_COMMAND="ssh -v" git push ...` 7434. Check to see if `sshd` on the knot is rejecting the push 744 for some reason: `journalctl -xeu ssh` (or `sshd`, 745 depending on your machine). These logs are unavailable if 746 using docker. 7475. Check to see if the knot itself is rejecting the push, 748 depending on your setup, the logs might be in one of the 749 following paths: 750 - `/tmp/knotguard.log` 751 - `/home/git/log` 752 - `/home/git/guard.log` 753 754# Spindles 755 756## Pipelines 757 758Spindle workflows allow you to write CI/CD pipelines in a 759simple format. They're located in the `.tangled/workflows` 760directory at the root of your repository, and are defined 761using YAML. 762 763A workflow has a set of common fields that apply no matter 764which engine you pick: 765 766- [Trigger](#trigger): A **required** field that defines 767 when a workflow should be triggered. 768- [Engine](#engine): A **required** field that defines which 769 engine a workflow should run on. 770- [Clone options](#clone-options): An **optional** field 771 that defines how the repository should be cloned. 772- [Environment](#environment): An **optional** field that 773 allows you to define environment variables. 774- [Steps](#steps): An **optional** field that allows you to 775 define what steps should run in the workflow. 776 777On top of these, each engine has its own options for things 778like dependencies and images. See [Engines](#engines) for 779the per-engine fields. 780 781### Trigger 782 783The first thing to add to a workflow is the trigger, which 784defines when a workflow runs. This is defined using a `when` 785field, which takes in a list of conditions. Each condition 786has the following fields: 787 788- `event`: This is a **required** field that defines when 789 your workflow should run. It's a list that can take one or 790 more of the following values: 791 - `push`: The workflow should run every time a commit is 792 pushed to the repository. 793 - `pull_request`: The workflow should run every time a 794 pull request is made or updated. 795 - `manual`: The workflow can be triggered manually. 796- `branch`: Defines which branches the workflow should run 797 for. If used with the `push` event, commits to the 798 branch(es) listed here will trigger the workflow. If used 799 with the `pull_request` event, updates to pull requests 800 targeting the branch(es) listed here will trigger the 801 workflow. This field has no effect with the `manual` 802 event. Supports glob patterns using `*` and `**` (e.g., 803 `main`, `develop`, `release-*`). Either `branch` or `tag` 804 (or both) must be specified for `push` events. 805- `tag`: Defines which tags the workflow should run for. 806 Only used with the `push` event - when tags matching the 807 pattern(s) listed here are pushed, the workflow will 808 trigger. This field has no effect with `pull_request` or 809 `manual` events. Supports glob patterns using `*` and `**` 810 (e.g., `v*`, `v1.*`, `release-**`). Either `branch` or 811 `tag` (or both) must be specified for `push` events. 812 813For example, if you'd like to define a workflow that runs 814when commits are pushed to the `main` and `develop` 815branches, or when pull requests that target the `main` 816branch are updated, or manually, you can do so with: 817 818```yaml 819when: 820 - event: ["push", "manual"] 821 branch: ["main", "develop"] 822 - event: ["pull_request"] 823 branch: ["main"] 824``` 825 826You can also trigger workflows on tag pushes. For instance, 827to run a deployment workflow when tags matching `v*` are 828pushed: 829 830```yaml 831when: 832 - event: ["push"] 833 tag: ["v*"] 834``` 835 836You can even combine branch and tag patterns in a single 837constraint (the workflow triggers if either matches): 838 839```yaml 840when: 841 - event: ["push"] 842 branch: ["main", "release-*"] 843 tag: ["v*", "stable"] 844``` 845 846To skip CI for a push, pass a Git push option: 847 848```sh 849git push -o skip-ci 850``` 851 852`ci-skip` is also accepted. 853 854### Engine 855 856Next is the engine on which the workflow should run, defined 857using the **required** `engine` field. The currently 858supported engines are: 859 860- `nixery`: This uses an instance of 861 [Nixery](https://nixery.dev) to run steps, which allows 862 you to add [dependencies](#dependencies) from 863 Nixpkgs (https://github.com/NixOS/nixpkgs). You can 864 search for packages on https://search.nixos.org, and 865 there's a pretty good chance the package(s) you're looking 866 for will be there. 867 See [Nixery engine](#nixery-engine). 868- `microvm`: Runs the whole workflow inside its own 869 microVM. Has configuration features for NixOS images 870 that will let you enable services, do Docker-in-VM, etc. 871 See [microVM engine](#microvm-engine). 872 873Example: 874 875```yaml 876engine: "nixery" 877``` 878 879Each engine also adds its own workflow fields (dependencies, 880images, services, and so on). These are documented under 881[Engines](#engines). 882 883### Clone options 884 885When a workflow starts, the first step is to clone the 886repository. You can customize this behavior using the 887**optional** `clone` field. It has the following fields: 888 889- `skip`: Setting this to `true` will skip cloning the 890 repository. This can be useful if your workflow is doing 891 something that doesn't require anything from the 892 repository itself. This is `false` by default. 893- `depth`: This sets the number of commits, or the "clone 894 depth", to fetch from the repository. For example, if you 895 set this to 2, the last 2 commits will be fetched. By 896 default, the depth is set to 1, meaning only the most 897 recent commit will be fetched, which is the commit that 898 triggered the workflow. 899- `submodules`: If you use Git submodules 900 (https://git-scm.com/book/en/v2/Git-Tools-Submodules) 901 in your repository, setting this field to `true` will 902 recursively fetch all submodules. This is `false` by 903 default. 904 905The default settings are: 906 907```yaml 908clone: 909 skip: false 910 depth: 1 911 submodules: false 912``` 913 914### Environment 915 916The `environment` field allows you define environment 917variables that will be available throughout the entire 918workflow. **Do not put secrets here, these environment 919variables are visible to anyone viewing the repository. You 920can add secrets for pipelines in your repository's 921settings.** 922 923Example: 924 925```yaml 926environment: 927 GOOS: "linux" 928 GOARCH: "arm64" 929 NODE_ENV: "production" 930 MY_ENV_VAR: "MY_ENV_VALUE" 931``` 932 933By default, the following environment variables are set: 934 935- `CI` - Always set to `true` to indicate a CI environment 936- `TANGLED_PIPELINE_ID` - The AT URI of the current pipeline 937- `TANGLED_PIPELINE_KIND` - One of `push`, `pull_request` or 938 `manual` 939- `TANGLED_REPO_KNOT` - The repository's knot hostname 940- `TANGLED_REPO_DID` - The DID of the repository owner 941- `TANGLED_REPO_NAME` - The name of the repository 942- `TANGLED_REPO_DEFAULT_BRANCH` - The default branch of the 943 repository 944- `TANGLED_REPO_URL` - The full URL to the repository 945 946These variables are only available when the pipeline is 947triggered by a push: 948 949- `TANGLED_REF` - The full git reference (e.g., 950 `refs/heads/main` or `refs/tags/v1.0.0`) 951- `TANGLED_REF_NAME` - The short name of the reference 952 (e.g., `main` or `v1.0.0`) 953- `TANGLED_REF_TYPE` - The type of reference, either 954 `branch` or `tag` 955- `TANGLED_SHA` - The commit SHA that triggered the pipeline 956- `TANGLED_COMMIT_SHA` - Alias for `TANGLED_SHA` 957 958These variables are only available when the pipeline is 959triggered by a pull request: 960 961- `TANGLED_PR_SOURCE_BRANCH` - The source branch of the pull 962 request 963- `TANGLED_PR_TARGET_BRANCH` - The target branch of the pull 964 request 965- `TANGLED_PR_SOURCE_SHA` - The commit SHA of the source 966 branch 967 968### Cache 969 970The `cache` field lets a workflow persist directories across 971pipeline runs. Before the first step, the engine looks up 972each entry's key and extracts the matching archive into the 973workspace; after all steps succeed, the paths are archived 974again and stored back under the key. 975 976- `key`: name this cache is saved under. Keys are scoped to 977 the repository and the engine specified. 978- `hash`: **optional** list of repo files (lockfiles, 979 manifests) whose content is folded into the key. The entry 980 is stored as `<key>-<hash of those files>`, so editing 981 `go.sum` automatically rotates the cache without bumping 982 the key by hand. Paths are relative to the repository 983 root and are read from git at the commit being built. 984- `paths`: paths to archive. Relative paths are anchored at 985 the repository checkout, the directory steps start in 986 (`/workspace/repo` on microvm, `/tangled/workspace` on 987 nixery). Absolute paths work too, for caching directories 988 outside the checkout, but note they name engine-specific 989 locations. All paths must be writable by the CI user and 990 contain no spaces. 991- `compression-level`: **optional** zstd level, `1` 992 (fastest) to `19` (smallest). defaults to (`5`). 993- `when`: **optional** save policy: `on-success` (the 994 default) or `always`. 995 996When the exact key (or generation, with `hash`) misses, the 997newest older generation under the same key is restored. 998 999```yaml 1000cache: 1001 - key: go-mod 1002 hash: 1003 - go.sum 1004 - go.mod 1005 paths: 1006 - .gocache 1007``` 1008 1009Caches are only restored and saved for trusted pipelines 1010(pushes and same-repository pull requests). Pipelines 1011building untrusted code, like pull requests from forks, 1012skip the cache entirely. Saving follows each entry's `when` 1013policy, except on timeout, when nothing is saved. A cache 1014miss or failure never fails the workflow. 1015 1016The spindle operator chooses the storage backend. See 1017[Running spindle](#running-spindle). If no backend is 1018configured, `cache` entries are ignored. 1019 1020### Steps 1021 1022The `steps` field allows you to define what steps should run 1023in the workflow. It's a list of step objects, each with the 1024following fields: 1025 1026- `name`: This field allows you to give your step a name. 1027 This name is visible in your workflow runs, and is used to 1028 describe what the step is doing. 1029- `command`: This field allows you to define a command to 1030 run in that step. The step is run in a Bash shell, and the 1031 logs from the command will be visible in the pipelines 1032 page on the Tangled website. Any dependencies you added in 1033 your engine's section (see [Engines](#engines)) will be 1034 available to use here. 1035- `environment`: Similar to the global 1036 [environment](#environment) config, this **optional** 1037 field is a key-value map that allows you to set 1038 environment variables for the step. **Do not put secrets 1039 here, these environment variables are visible to anyone 1040 viewing the repository. You can add secrets for pipelines 1041 in your repository's settings.** 1042 1043Example: 1044 1045```yaml 1046steps: 1047 - name: "Build backend" 1048 command: "go build" 1049 environment: 1050 GOOS: "darwin" 1051 GOARCH: "arm64" 1052 - name: "Build frontend" 1053 command: "npm run build" 1054 environment: 1055 NODE_ENV: "production" 1056``` 1057 1058## Engines 1059 1060The common fields above apply to every workflow. Each engine 1061then adds its own fields on top. Pick an engine with the 1062[`engine`](#engine) field and use the matching section below. 1063 1064### Nixery engine 1065 1066#### Dependencies 1067 1068When you're running a workflow you'll usually need additional 1069dependencies. The `dependencies` field lets you define which 1070dependencies to get, and from where. It's a key-value map, 1071with the key being the registry to fetch dependencies from, 1072and the value being the list of dependencies to fetch. 1073 1074The registry URL syntax can be found [on the nix 1075manual](https://nix.dev/manual/nix/2.18/command-ref/new-cli/nix3-registry-add). 1076 1077Say you want to fetch Node.js and Go from `nixpkgs`, and a 1078package called `my_pkg` you've made from your own registry 1079at your repository at 1080`https://tangled.org/@example.com/my_pkg`. You can define 1081those dependencies like so: 1082 1083```yaml 1084dependencies: 1085 # nixpkgs 1086 nixpkgs: 1087 - nodejs 1088 - go 1089 # unstable 1090 nixpkgs/nixpkgs-unstable: 1091 - bun 1092 # custom registry 1093 git+https://tangled.org/@example.com/my_pkg: 1094 - my_pkg 1095``` 1096 1097Now these dependencies are available to use in your 1098workflow! 1099 1100#### Complete nixery workflow 1101 1102```yaml 1103# .tangled/workflows/build.yml 1104 1105when: 1106 - event: ["push", "manual"] 1107 branch: ["main", "develop"] 1108 - event: ["pull_request"] 1109 branch: ["main"] 1110 1111engine: "nixery" 1112 1113# using the default values 1114clone: 1115 skip: false 1116 depth: 1 1117 submodules: false 1118 1119dependencies: 1120 # nixpkgs 1121 nixpkgs: 1122 - nodejs 1123 - go 1124 # custom registry 1125 git+https://tangled.org/@example.com/my_pkg: 1126 - my_pkg 1127 1128environment: 1129 GOOS: "linux" 1130 GOARCH: "arm64" 1131 NODE_ENV: "production" 1132 MY_ENV_VAR: "MY_ENV_VALUE" 1133 1134steps: 1135 - name: "Build backend" 1136 command: "go build" 1137 environment: 1138 GOOS: "darwin" 1139 GOARCH: "arm64" 1140 - name: "Build frontend" 1141 command: "npm run build" 1142 environment: 1143 NODE_ENV: "production" 1144``` 1145 1146If you want another example of a workflow, you can look at 1147the one [Tangled uses to build the 1148project](https://tangled.org/@tangled.org/core/blob/master/.tangled/workflows/build.yml). 1149 1150### microVM engine 1151 1152#### Image 1153 1154A workflow picks the image to boot with the top-level `image` 1155field: 1156 1157```yaml 1158engine: microvm 1159image: nixos 1160``` 1161 1162There are two flavours of images: 1163 1164- **NixOS images** (e.g. `nixos`): the whole guest is built 1165 with Nix, so you can configure it from the workflow file 1166 itself. The `dependencies`, `services`, `virtualisation`, 1167 `registry` and `caches` fields below are all understood 1168 here, and the guest builds and activates that configuration 1169 before any of your steps run. 1170- **Non-NixOS images** (e.g. `alpine`): there's no NixOS to 1171 configure, so the workflow-level config fields above have 1172 no effect. You still get a full machine to run steps in. 1173 1174The available image names depend on what the spindle operator 1175has installed. `nixos` and `alpine` are examples. If `image` 1176is omitted, the spindle's configured default image is used. 1177 1178#### Dependencies 1179 1180On the microVM engine, `dependencies` is a flat list of 1181packages that are made available to every step. This field 1182only applies to **NixOS images**; for other images you can 1183use the package manager included in a step. 1184 1185The guest builds a [`nix develop`](https://nix.dev/manual/nix/2.18/command-ref/new-cli/nix3-develop)-style 1186devshell from your dependencies and uses it for each step, 1187so you can, for example, add `pkg-config` and `openssl` and 1188have the `openssl-sys` crate while compiling a Rust project 1189just work. 1190 1191A bare name like `go` is looked up in nixpkgs. You can also 1192point at any flake with the `flakeref#attr` syntax, so 1193`github:nixos/nixpkgs#hello` pulls `hello` straight out of 1194that flake. 1195 1196```yaml 1197dependencies: 1198 - go 1199 - github:nixos/nixpkgs#hello 1200``` 1201 1202#### Registry 1203 1204The `registry` field remaps flake references, the same way 1205`nix registry` does. This lets you pin or alias the flakes 1206used by `dependencies`. 1207 1208For example, pin `nixpkgs` to `nixos-unstable` so that the 1209bare `go` above resolves from unstable, and alias your own 1210flake so you can use `myflake#tool` in `dependencies`: 1211 1212```yaml 1213registry: 1214 nixpkgs: github:nixos/nixpkgs/nixos-unstable 1215 myflake: github:me/x 1216``` 1217 1218#### Substituters 1219 1220The `substituters` field is a map of Nix binary cache URL to its 1221trusted public key. These are fed into the spindle's read 1222proxy, so the guest can substitute prebuilt paths from them 1223instead of building everything from scratch. 1224 1225```yaml 1226substituters: 1227 https://nix-community.cachix.org: "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=" 1228``` 1229 1230#### Services and virtualisation 1231 1232The `services` and `virtualisation` fields are passed straight 1233through to NixOS. Anything you could write under 1234`services.*` or `virtualisation.*` in a NixOS configuration, 1235you can write here, and it's brought up before any of your 1236steps run. 1237 1238As a convenience, `true` works as shorthand for 1239`.enable = true` anywhere an `enable` option exists (e.g. 1240`virtualisation.docker: true`). 1241 1242```yaml 1243services: 1244 postgresql: 1245 enable: true 1246 ensureDatabases: ["spindle-workflow"] 1247 ensureUsers: 1248 - name: spindle-workflow 1249 ensureDBOwnership: true 1250 1251virtualisation: 1252 docker: true 1253``` 1254 1255#### Recipes 1256 1257##### Lint, test and build a Node project 1258 1259```yaml 1260when: 1261 - event: ["push", "pull_request"] 1262 branch: ["main"] 1263 1264engine: microvm 1265image: nixos 1266 1267dependencies: 1268 - pnpm 1269 1270steps: 1271 - name: "Install dependencies" 1272 command: pnpm install --frozen-lockfile 1273 - name: "Lint and test" 1274 command: | 1275 pnpm run lint 1276 pnpm test 1277 - name: "Build" 1278 command: pnpm run build 1279``` 1280 1281##### Check formatting 1282 1283```yaml 1284when: 1285 - event: ["push", "pull_request"] 1286 branch: ["main"] 1287 1288engine: microvm 1289image: alpine # slimmer image for checking the formatting 1290 1291steps: 1292 - name: "Install go" 1293 command: apk add go 1294 - name: "Check formatting" 1295 command: test -z $(gofmt -l .) 1296``` 1297 1298##### Build a Rust project that links OpenSSL 1299 1300```yaml 1301when: 1302 - event: ["push", "pull_request"] 1303 branch: ["main"] 1304 1305engine: microvm 1306image: nixos 1307 1308dependencies: 1309 - gcc 1310 - cargo 1311 - rustc 1312 - clippy 1313 - rustfmt 1314 - pkg-config # exports PKG_CONFIG_PATH for the libraries below 1315 - openssl # the C library + headers openssl-sys links against 1316 1317steps: 1318 - name: "Check formatting" 1319 command: cargo fmt --check 1320 - name: "Clippy" 1321 command: cargo clippy --all-targets -- -D warnings 1322 - name: "Test" 1323 command: cargo test --all 1324 - name: "Release build" 1325 command: cargo build --release 1326``` 1327 1328##### Run migrations and integration tests against PostgreSQL 1329 1330```yaml 1331when: 1332 - event: ["push", "pull_request"] 1333 branch: ["main"] 1334 1335engine: microvm 1336image: nixos 1337 1338environment: 1339 DATABASE_URL: "postgresql:///spindle-workflow?host=/run/postgresql" 1340 1341dependencies: 1342 - gcc 1343 - cargo 1344 - rustc 1345 - pkg-config 1346 - openssl 1347 - sqlx-cli 1348 1349services: 1350 postgresql: 1351 enable: true 1352 # has to be same name as the user for peer auth to work automatically 1353 ensureDatabases: ["spindle-workflow"] 1354 ensureUsers: 1355 - name: spindle-workflow 1356 ensureDBOwnership: true 1357 1358steps: 1359 - name: "Run migrations" 1360 command: sqlx migrate run 1361 - name: "Integration tests" 1362 command: cargo test --all 1363``` 1364 1365##### Build and push a Docker image on tag 1366 1367```yaml 1368when: 1369 - event: ["push"] 1370 tag: ["v*"] 1371 1372engine: microvm 1373image: nixos 1374 1375virtualisation: 1376 docker: true 1377 1378steps: 1379 - name: "Build and push to ghcr.io" 1380 command: | 1381 set -euo pipefail 1382 1383 echo "$REGISTRY_TOKEN" | docker login ghcr.io -u "$REGISTRY_USER" --password-stdin 1384 image="ghcr.io/$REGISTRY_USER/myapp:$TANGLED_REF_NAME" 1385 1386 docker build -t "$image" -t "ghcr.io/$REGISTRY_USER/myapp:latest" . 1387 docker push "$image" 1388 docker push "ghcr.io/$REGISTRY_USER/myapp:latest" 1389``` 1390 1391##### Deploy to Cloudflare Workers on tag 1392 1393```yaml 1394# .tangled/workflows/deploy.yml 1395when: 1396 - event: ["push"] 1397 tag: ["v*"] 1398 1399engine: microvm 1400image: nixos 1401 1402dependencies: 1403 - pnpm 1404 1405steps: 1406 - name: "Install dependencies" 1407 command: pnpm install --frozen-lockfile 1408 - name: "Deploy worker" 1409 # `wrangler` picks up `CLOUDFLARE_API_TOKEN` from the env. 1410 # set it under **Settings → Secrets**. 1411 command: pnpm exec wrangler deploy 1412``` 1413 1414##### Publish a release artifact 1415 1416```yaml 1417when: 1418 - event: ["push"] 1419 tag: ["v*"] # trigger on versions 1420 1421engine: microvm 1422image: nixos 1423 1424dependencies: 1425 - go 1426 1427steps: 1428 - name: "Build release binary" 1429 command: | 1430 mkdir -p dist 1431 CGO_ENABLED=0 go build -trimpath -ldflags "-s -w" -o dist/myapp ./cmd/myapp 1432 1433 - name: "Publish artifact record" 1434 command: | 1435 set -euo pipefail 1436 # change this if you're not on `tngl.sh` 1437 PDS="https://tngl.sh" 1438 # also update this to your handle or did 1439 ATP_IDENTIFIER="user.tngl.sh" 1440 ARTIFACT_PATH="dist/myapp" 1441 ARTIFACT_NAME="myapp" 1442 1443 # set `ATP_APP_PASSWORD` under **Settings → Secrets** 1444 session=$(curl -fsS -X POST "$PDS/xrpc/com.atproto.server.createSession" \ 1445 -H "Content-Type: application/json" \ 1446 -d "{\"identifier\":\"$ATP_IDENTIFIER\",\"password\":\"$ATP_APP_PASSWORD\"}") 1447 jwt=$(echo "$session" | jq -r .accessJwt) 1448 did=$(echo "$session" | jq -r .did) 1449 1450 # upload the binary as a blob 1451 blob=$(curl -fsS -X POST "$PDS/xrpc/com.atproto.repo.uploadBlob" \ 1452 -H "Authorization: Bearer $jwt" \ 1453 -H "Content-Type: application/octet-stream" \ 1454 --data-binary @"$ARTIFACT_PATH") 1455 1456 # note that this requires an annotated tag (`git tag -a v1.0.0 -m ...`) 1457 tag_hash=$(git rev-parse "$TANGLED_REF_NAME^{tag}") 1458 tag_bytes=$(printf '%s' "$tag_hash" | xxd -r -p | base64 | tr -d '=') 1459 1460 # the sh.tangled.repo.artifact record for your artifact 1461 record=$(jq -n \ 1462 --arg did "$did" \ 1463 --arg tag "$tag_bytes" \ 1464 --arg name "$ARTIFACT_NAME" \ 1465 --arg repo "$TANGLED_REPO_URL" \ 1466 --arg created "$(date -Iseconds)" \ 1467 --argjson blob "$(echo "$blob" | jq .blob)" '{ 1468 repo: $did, 1469 collection: "sh.tangled.repo.artifact", 1470 validate: false, 1471 record: { 1472 "$type": "sh.tangled.repo.artifact", 1473 tag: {"$bytes": $tag}, 1474 name: $name, 1475 repo: $repo, 1476 artifact: $blob, 1477 createdAt: $created 1478 } 1479 }') 1480 1481 # create the record on the PDS 1482 curl -fsS -X POST "$PDS/xrpc/com.atproto.repo.createRecord" \ 1483 -H "Authorization: Bearer $jwt" \ 1484 -H "Content-Type: application/json" \ 1485 -d "$record" 1486``` 1487 1488## Self-hosting guide 1489 1490### Prerequisites 1491 1492- Go 1493- For the **nixery** engine: Docker (or Podman with Docker 1494 compatibility enabled). 1495- For the **microVM** engine: a Linux host with KVM, plus the 1496 microVM host dependencies described in [Running microVM 1497 workflows](#running-microvm-workflows). 1498 1499### Configuration 1500 1501Spindle is configured using environment variables. The following environment variables are available: 1502 1503- `SPINDLE_SERVER_LISTEN_ADDR`: The address the server listens on (default: `"0.0.0.0:6555"`). 1504- `SPINDLE_SERVER_DB_PATH`: The path to the SQLite database file (default: `"spindle.db"`). 1505- `SPINDLE_SERVER_HOSTNAME`: The hostname of the server (required). 1506- `SPINDLE_SERVER_JETSTREAM_ENDPOINT`: The endpoint of the Jetstream server (default: `"wss://jetstream1.us-west.bsky.network/subscribe"`). 1507- `SPINDLE_SERVER_DEV`: A boolean indicating whether the server is running in development mode (default: `false`). 1508- `SPINDLE_SERVER_OWNER`: The DID of the owner (required). 1509- `SPINDLE_SERVER_LOG_DIR`: The directory to store workflow logs (default: `"/var/log/spindle"`). 1510- `SPINDLE_SERVER_DOCKER_SOCKET`: Path to Docker socket to expose to invoked Spindle containers (default: `""`). 1511- `SPINDLE_PIPELINES_NIXERY`: The Nixery URL (default: `"nixery.tangled.sh"`). 1512- `SPINDLE_PIPELINES_WORKFLOW_TIMEOUT`: The default workflow timeout (default: `"5m"`). 1513 1514For the microVM engine, the following are also available 1515(prefix `SPINDLE_MICROVM_PIPELINES_`): 1516 1517- `SPINDLE_MICROVM_PIPELINES_IMAGE_DIR`: Directory containing 1518 microVM images (**required** to use the engine). See 1519 [Running microVM workflows](#running-microvm-workflows). 1520- `SPINDLE_MICROVM_PIPELINES_DEFAULT_IMAGE`: Image used when a 1521 workflow doesn't set `image` (default: `"nixos-x86_64"`). 1522- `SPINDLE_MICROVM_PIPELINES_OVERLAY_DIR`: Where per-workflow 1523 temporary disks are created (default: the system temp dir). 1524- `SPINDLE_MICROVM_PIPELINES_ENABLE_KVM`: Use KVM hardware 1525 acceleration (default: `true`). Without KVM, guests fall 1526 back to slow software emulation. 1527- `SPINDLE_MICROVM_PIPELINES_WORKFLOW_TIMEOUT`: Default 1528 workflow timeout (default: `"5m"`). 1529 1530Optional resource limits (a value of `0` disables that 1531limit). The limits cap usage across all running microVM 1532workflows: 1533 1534- `SPINDLE_MICROVM_PIPELINES_MAX_TOTAL_MEMORY_MIB` 1535- `SPINDLE_MICROVM_PIPELINES_MAX_TOTAL_VCPUS` 1536- `SPINDLE_MICROVM_PIPELINES_MAX_TOTAL_DISK_MIB` 1537 1538Optional cgroup enforcement: 1539 1540- `SPINDLE_MICROVM_PIPELINES_ENABLE_CGROUPS`: Place each 1541 workflow's QEMU and slirp4netns in a per-workflow cgroup= 1542 (default: `false`). 1543- `SPINDLE_MICROVM_PIPELINES_CGROUP_PARENT`: Parent cgroup; 1544 `self` resolves the spindle service's own cgroup (default: 1545 `"self"`). 1546- `SPINDLE_MICROVM_PIPELINES_CGROUP_PIDS_MAX`: Max processes 1547 per workflow cgroup (default: `4096`). 1548- `SPINDLE_MICROVM_PIPELINES_CGROUP_SWAP_MAX_MIB`: Max swap 1549 per workflow cgroup (default: `0`, no swap). 1550- `SPINDLE_MICROVM_PIPELINES_CGROUP_SUPERVISOR_MEMORY_MIN_MIB`: 1551 Memory protected for spindle itself so it isn't OOM-killed 1552 before the workflows (default: `512`). 1553 1554To push paths built inside microVMs back to a shared Nix 1555cache (and read from it), configure the cache (prefix 1556`SPINDLE_NIX_CACHE_`): 1557 1558- `SPINDLE_NIX_CACHE_READ_URLS`: Comma-separated binary cache 1559 URLs the guest reads from. 1560- `SPINDLE_NIX_CACHE_TRUSTED_PUBLIC_KEYS`: Comma-separated 1561 trusted public keys for those caches. 1562- `SPINDLE_NIX_CACHE_UPLOAD_URL`: Cache URL that paths built 1563 in the guest are uploaded to. 1564 1565The generic CI cache (the workflow-level 1566[`cache`](#cache) field) is configured via prefix 1567`SPINDLE_CACHE_`. 1568 1569- `SPINDLE_CACHE_BACKEND`: Storage backend, `disk` or `s3` 1570 (default: `""`, caching disabled). 1571- `SPINDLE_CACHE_DISK_DIR`: Directory for the `disk` backend 1572 (default: a `cache` directory next to the spindle 1573 database). 1574- `SPINDLE_CACHE_S3_BUCKET`: Unversioned bucket for the `s3` 1575 backend. Credentials come from the standard AWS chain and 1576 need `s3:GetBucketVersioning` in addition to object access. 1577- `SPINDLE_CACHE_S3_PREFIX`: Key prefix inside the bucket 1578 (default: `"spindle/cache"`). 1579- `SPINDLE_CACHE_RETENTION`: Time since the last restore or 1580 save before an entry is deleted (default: `720h`, or 30 1581 days). Set to `0` to keep entries indefinitely. 1582- `SPINDLE_CACHE_PRUNE_INTERVAL`: How often expired entries 1583 are deleted (default: `1h`). 1584 1585Cache metadata and usage are tracked in spindle's SQLite 1586database. Storage backends contain opaque objects and are 1587never listed during lookup or cleanup. 1588 1589### Running spindle 1590 15911. **Set the environment variables.** For example: 1592 1593 ```shell 1594 export SPINDLE_SERVER_HOSTNAME="your-hostname" 1595 export SPINDLE_SERVER_OWNER="your-did" 1596 ``` 1597 15982. **Build the Spindle binary.** 1599 1600 ```shell 1601 cd core 1602 go mod download 1603 go build -o cmd/spindle/spindle cmd/spindle/main.go 1604 ``` 1605 16063. **Create the log directory.** 1607 1608 ```shell 1609 sudo mkdir -p /var/log/spindle 1610 sudo chown $USER:$USER -R /var/log/spindle 1611 ``` 1612 16134. **Run the Spindle binary.** 1614 1615 ```shell 1616 ./cmd/spindle/spindle 1617 ``` 1618 1619Spindle will now start, connect to the Jetstream server, and begin processing pipelines. 1620 1621### Running microVM workflows 1622 1623The microVM engine needs a few extra things on the host, and 1624it needs images to boot. 1625 1626#### Host dependencies 1627 1628microVM workflows depend on a handful of host tools and 1629devices. spindle checks for the ones an image needs right 1630before it launches, so a missing dependency surfaces as a 1631clear error. You'll need: 1632 1633- `qemu`: the runner. The QEMU binary for the image's arch 1634 must be present (e.g. `qemu-system-x86_64`). 1635- `mkfs.ext4` (from `e2fsprogs`): to format the per-workflow 1636 writable volumes. 1637- [`slirp4netns`](https://github.com/rootless-containers/slirp4netns#install), 1638 `ip` (from `iproute2`), `mount` and `unshare` (from `util-linux`): 1639 used to sandbox guest networking. 1640- `/dev/kvm`: for hardware acceleration (unless you disable 1641 KVM with `SPINDLE_MICROVM_PIPELINES_ENABLE_KVM=false`). 1642- `/dev/vhost-vsock`: used by QEMU to enable guest-to-host vsock 1643 communication. 1644- `/dev/vsock`: used by spindle on the host to listen on vsock ports 1645 and accept guest agent connections. 1646- `/dev/net/tun`: required by `slirp4netns` to set up tap devices 1647 for sandboxed guest networking. 1648 1649On NixOS, the [spindle 1650module](https://tangled.org/tangled.org/core/blob/master/nix/modules/spindle.nix) 1651puts `qemu`, `e2fsprogs`, `slirp4netns`, `iproute2` and 1652`util-linux` on the service's `PATH` for you. 1653 1654#### Container virtualization 1655 1656Running `spindle` inside a container (e.g., Docker, Podman, or LXD) requires passing host device nodes into the container and granting the runtime additional privileges. Because spindle uses nested namespaces (`unshare`) and helper tools (`slirp4netns`), the container configuration will require: 1657 1658- `/dev/vsock`, `/dev/vhost-vsock`, `/dev/kvm`, and `/dev/net/tun` mapped from the host. 1659- `NET_ADMIN` and `SYS_ADMIN` capabilities to manage network namespaces and mounts inside the container. 1660- Relaxed seccomp filters (e.g., `seccomp=unconfined`) and SELinux/AppArmor containment if they restrict namespace creation or device access. 1661 1662#### Building images 1663 1664Images are built with Nix. The flake exposes packages for the 1665two stock images (use the `-tarball` prefixed ones for a gzipped 1666tarball you can copy to another host): 1667 1668```shell 1669# a NixOS image 1670nix build .#spindle-nixos-image 1671# an Alpine image 1672nix build .#spindle-alpine-image 1673``` 1674 1675#### Installing images 1676 1677Spindle looks for images in 1678`SPINDLE_MICROVM_PIPELINES_IMAGE_DIR`. An image is resolved by 1679the name a workflow puts in its `image` field, matched 1680literally against what's on disk: 1681 16821. a directory `<name>/` containing a `spec.json` (next to the 1683 kernel/initrd/store-disk), or 16842. a flat `<name>.json` self-contained spec. 1685 1686Resolution depends only on the name and what's on disk, never 1687on the host doing the resolving, so the same workflow resolves 1688to the same image on every spindle. If you keep multiple 1689arches side by side, you can name them `<name>-<arch>` (e.g. 1690`nixos-x86_64`, `alpine-aarch64`); the suffix is just part of 1691the name. To make a name like `nixos` work if you are hosting 1692multiple arches, you can use symlinks. 1693 1694On NixOS, you'll most likely want to use `systemd.tmpfiles.rules` 1695to set these up declaratively. 1696 1697## Architecture 1698 1699Spindle is a small CI runner service. Here's a high-level overview of how it operates: 1700 1701- Listens for [`sh.tangled.spindle.member`](/lexicons/spindle/member.json) and 1702 [`sh.tangled.repo`](/lexicons/repo.json) records on the Jetstream. 1703- When a new repo record comes through (typically when you add a spindle to a 1704 repo from the settings), spindle then resolves the underlying knot and 1705 subscribes to repo events (see: 1706 [`sh.tangled.pipeline`](/lexicons/pipeline.json)). 1707- The spindle engine then handles execution of the pipeline, with results and 1708 logs beamed on the spindle event stream over WebSocket 1709 1710### The engines 1711 1712Spindle has two execution backends, picked per-workflow with 1713the [`engine`](#engine) field: 1714 1715- **nixery**: executes each step in a fresh Docker container 1716 (Podman works too, if Docker compatibility is enabled so 1717 that `/run/docker.sock` is created), with state persisted 1718 across steps within the `/tangled/workspace` directory. The 1719 base image for the container is constructed on the fly using 1720 [Nixery](https://nixery.dev), which is/rhandy for caching 1721 layers for frequently used packages. 1722- **microvm**: runs the whole workflow inside its own 1723 microVM, supporting different images, with extra 1724 configuration for NixOS images (e.g. services in workflow file) 1725 See the [engine 1726 README](https://tangled.org/tangled.org/core/blob/master/spindle/engines/microvm/README.md) 1727 for the architecture in depth. 1728 1729The pipeline manifest is [specified here](https://docs.tangled.org/spindles.html#pipelines). 1730 1731## Secrets with openbao 1732 1733This document covers setting up spindle to use OpenBao for secrets 1734management via OpenBao Proxy instead of the default SQLite backend. 1735 1736### Overview 1737 1738Spindle now uses OpenBao Proxy for secrets management. The proxy handles 1739authentication automatically using AppRole credentials, while spindle 1740connects to the local proxy instead of directly to the OpenBao server. 1741 1742This approach provides better security, automatic token renewal, and 1743simplified application code. 1744 1745### Installation 1746 1747Install OpenBao from Nixpkgs: 1748 1749```bash 1750nix shell nixpkgs#openbao # for a local server 1751``` 1752 1753### Setup 1754 1755The setup process can is documented for both local development and production. 1756 1757#### Local development 1758 1759Start OpenBao in dev mode: 1760 1761```bash 1762bao server -dev -dev-root-token-id="root" -dev-listen-address=127.0.0.1:8201 1763``` 1764 1765This starts OpenBao on `http://localhost:8201` with a root token. 1766 1767Set up environment for bao CLI: 1768 1769```bash 1770export BAO_ADDR=http://localhost:8200 1771export BAO_TOKEN=root 1772``` 1773 1774#### Production 1775 1776You would typically use a systemd service with a 1777configuration file. Refer to 1778[@tangled.org/infra](https://tangled.org/@tangled.org/infra) 1779for how this can be achieved using Nix. 1780 1781Then, initialize the bao server: 1782 1783```bash 1784bao operator init -key-shares=1 -key-threshold=1 1785``` 1786 1787This will print out an unseal key and a root key. Save them 1788somewhere (like a password manager). Then unseal the vault 1789to begin setting it up: 1790 1791```bash 1792bao operator unseal <unseal_key> 1793``` 1794 1795All steps below remain the same across both dev and 1796production setups. 1797 1798#### Configure openbao server 1799 1800Create the spindle KV mount: 1801 1802```bash 1803bao secrets enable -path=spindle -version=2 kv 1804``` 1805 1806Set up AppRole authentication and policy: 1807 1808Create a policy file `spindle-policy.hcl`: 1809 1810```hcl 1811# Full access to spindle KV v2 data 1812path "spindle/data/*" { 1813 capabilities = ["create", "read", "update", "delete"] 1814} 1815 1816# Access to metadata for listing and management 1817path "spindle/metadata/*" { 1818 capabilities = ["list", "read", "delete", "update"] 1819} 1820 1821# Allow listing at root level 1822path "spindle/" { 1823 capabilities = ["list"] 1824} 1825 1826# Required for connection testing and health checks 1827path "auth/token/lookup-self" { 1828 capabilities = ["read"] 1829} 1830``` 1831 1832Apply the policy and create an AppRole: 1833 1834```bash 1835bao policy write spindle-policy spindle-policy.hcl 1836bao auth enable approle 1837bao write auth/approle/role/spindle \ 1838 token_policies="spindle-policy" \ 1839 token_ttl=1h \ 1840 token_max_ttl=4h \ 1841 bind_secret_id=true \ 1842 secret_id_ttl=0 \ 1843 secret_id_num_uses=0 1844``` 1845 1846Get the credentials: 1847 1848```bash 1849# Get role ID (static) 1850ROLE_ID=$(bao read -field=role_id auth/approle/role/spindle/role-id) 1851 1852# Generate secret ID 1853SECRET_ID=$(bao write -f -field=secret_id auth/approle/role/spindle/secret-id) 1854 1855echo "Role ID: $ROLE_ID" 1856echo "Secret ID: $SECRET_ID" 1857``` 1858 1859#### Create proxy configuration 1860 1861Create the credential files: 1862 1863```bash 1864# Create directory for OpenBao files 1865mkdir -p /tmp/openbao 1866 1867# Save credentials 1868echo "$ROLE_ID" > /tmp/openbao/role-id 1869echo "$SECRET_ID" > /tmp/openbao/secret-id 1870chmod 600 /tmp/openbao/role-id /tmp/openbao/secret-id 1871``` 1872 1873Create a proxy configuration file `/tmp/openbao/proxy.hcl`: 1874 1875```hcl 1876# OpenBao server connection 1877vault { 1878 address = "http://localhost:8200" 1879} 1880 1881# Auto-Auth using AppRole 1882auto_auth { 1883 method "approle" { 1884 mount_path = "auth/approle" 1885 config = { 1886 role_id_file_path = "/tmp/openbao/role-id" 1887 secret_id_file_path = "/tmp/openbao/secret-id" 1888 } 1889 } 1890 1891 # Optional: write token to file for debugging 1892 sink "file" { 1893 config = { 1894 path = "/tmp/openbao/token" 1895 mode = 0640 1896 } 1897 } 1898} 1899 1900# Proxy listener for spindle 1901listener "tcp" { 1902 address = "127.0.0.1:8201" 1903 tls_disable = true 1904} 1905 1906# Enable API proxy with auto-auth token 1907api_proxy { 1908 use_auto_auth_token = true 1909} 1910 1911# Enable response caching 1912cache { 1913 use_auto_auth_token = true 1914} 1915 1916# Logging 1917log_level = "info" 1918``` 1919 1920#### Start the proxy 1921 1922Start OpenBao Proxy: 1923 1924```bash 1925bao proxy -config=/tmp/openbao/proxy.hcl 1926``` 1927 1928The proxy will authenticate with OpenBao and start listening on 1929`127.0.0.1:8201`. 1930 1931#### Configure spindle 1932 1933Set these environment variables for spindle: 1934 1935```bash 1936export SPINDLE_SERVER_SECRETS_PROVIDER=openbao 1937export SPINDLE_SERVER_SECRETS_OPENBAO_PROXY_ADDR=http://127.0.0.1:8201 1938export SPINDLE_SERVER_SECRETS_OPENBAO_MOUNT=spindle 1939``` 1940 1941On startup, spindle will now connect to the local proxy, 1942which handles all authentication automatically. 1943 1944### Production setup for proxy 1945 1946For production, you'll want to run the proxy as a service: 1947 1948Place your production configuration in 1949`/etc/openbao/proxy.hcl` with proper TLS settings for the 1950vault connection. 1951 1952### Verifying setup 1953 1954Test the proxy directly: 1955 1956```bash 1957# Check proxy health 1958curl -H "X-Vault-Request: true" http://127.0.0.1:8201/v1/sys/health 1959 1960# Test token lookup through proxy 1961curl -H "X-Vault-Request: true" http://127.0.0.1:8201/v1/auth/token/lookup-self 1962``` 1963 1964Test OpenBao operations through the server: 1965 1966```bash 1967# List all secrets 1968bao kv list spindle/ 1969 1970# Add a test secret via the spindle API, then check it exists 1971bao kv list spindle/repos/ 1972 1973# Get a specific secret 1974bao kv get spindle/repos/your_repo_path/SECRET_NAME 1975``` 1976 1977### How it works 1978 1979- Spindle connects to OpenBao Proxy on localhost (typically 1980 port 8200 or 8201) 1981- The proxy authenticates with OpenBao using AppRole 1982 credentials 1983- All spindle requests go through the proxy, which injects 1984 authentication tokens 1985- Secrets are stored at 1986 `spindle/repos/{sanitized_repo_path}/{secret_key}` 1987- Repository paths like `did:plc:alice/myrepo` become 1988 `did_plc_alice_myrepo` 1989- The proxy handles all token renewal automatically 1990- Spindle no longer manages tokens or authentication 1991 directly 1992 1993### Troubleshooting 1994 1995**Connection refused**: Check that the OpenBao Proxy is 1996running and listening on the configured address. 1997 1998**403 errors**: Verify the AppRole credentials are correct 1999and the policy has the necessary permissions. 2000 2001**404 route errors**: The spindle KV mount probably doesn't 2002exist—run the mount creation step again. 2003 2004**Proxy authentication failures**: Check the proxy logs and 2005verify the role-id and secret-id files are readable and 2006contain valid credentials. 2007 2008**Secret not found after writing**: This can indicate policy 2009permission issues. Verify the policy includes both 2010`spindle/data/*` and `spindle/metadata/*` paths with 2011appropriate capabilities. 2012 2013Check proxy logs: 2014 2015```bash 2016# If running as systemd service 2017journalctl -u openbao-proxy -f 2018 2019# If running directly, check the console output 2020``` 2021 2022Test AppRole authentication manually: 2023 2024```bash 2025bao write auth/approle/login \ 2026 role_id="$(cat /tmp/openbao/role-id)" \ 2027 secret_id="$(cat /tmp/openbao/secret-id)" 2028``` 2029 2030# Webhooks 2031 2032Webhooks allow you to receive HTTP POST notifications when events occur in your repositories. This enables you to integrate Tangled with external services, trigger CI/CD pipelines, send notifications, or automate workflows. 2033 2034## Overview 2035 2036Webhooks send HTTP POST requests to URLs you configure whenever specific events happen. Currently, Tangled supports push, repository rename, and pull request events, with more event types coming soon. 2037 2038## Configuring webhooks 2039 2040To set up a webhook for your repository: 2041 20421. Navigate to your repository 20432. Go to **Settings → Hooks** 20443. Click **new webhook** 20454. Configure your webhook: 2046 - **Payload URL**: The endpoint that will receive the webhook POST requests 2047 - **Secret**: An optional secret key for verifying webhook authenticity (leave blank to send unsigned webhooks) 2048 - **Events**: Select which events trigger the webhook 2049 - **Active**: Toggle whether the webhook is enabled 2050 2051## Webhook payload 2052 2053### Push 2054 2055When a push event occurs, Tangled sends a POST request with a JSON payload of the format: 2056 2057```json 2058{ 2059 "after": "7b320e5cbee2734071e4310c1d9ae401d8f6cab5", 2060 "before": "c04ddf64eddc90e4e2a9846ba3b43e67a0e2865e", 2061 "pusher": { 2062 "did": "did:plc:hwevmowznbiukdf6uk5dwrrq" 2063 }, 2064 "ref": "refs/heads/main", 2065 "repository": { 2066 "clone_url": "https://tangled.org/did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo", 2067 "created_at": "2025-09-15T08:57:23Z", 2068 "description": "an example repository", 2069 "fork": false, 2070 "full_name": "did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo", 2071 "html_url": "https://tangled.org/did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo", 2072 "name": "some-repo", 2073 "open_issues_count": 5, 2074 "owner": { 2075 "did": "did:plc:hwevmowznbiukdf6uk5dwrrq" 2076 }, 2077 "ssh_url": "ssh://git@tangled.org/did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo", 2078 "stars_count": 1, 2079 "updated_at": "2025-09-15T08:57:23Z" 2080 } 2081} 2082``` 2083 2084### Pull request 2085 2086Pull request events are sent as separate event types, so you can subscribe to 2087exactly the transitions you care about: 2088 2089- `pull_request:created` — a pull request was opened 2090- `pull_request:resubmitted` — a new round (revision) was pushed to a pull request 2091- `pull_request:merged` — a pull request was merged 2092- `pull_request:closed` — a pull request was closed 2093- `pull_request:reopened` — a closed pull request was reopened 2094 2095All pull request events share the same payload format: 2096 2097```json 2098{ 2099 "action": "created", 2100 "pull_request": { 2101 "number": 4, 2102 "title": "add dark mode", 2103 "body": "implements dark mode as discussed in #2", 2104 "state": "open", 2105 "target_branch": "main", 2106 "source": { 2107 "branch": "dark-mode", 2108 "sha": "7b320e5cbee2734071e4310c1d9ae401d8f6cab5" 2109 }, 2110 "round_number": 0, 2111 "owner": { 2112 "did": "did:plc:hwevmowznbiukdf6uk5dwrrq" 2113 }, 2114 "html_url": "https://tangled.org/did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo/pulls/4", 2115 "patch_url": "https://tangled.org/did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo/pulls/4/round/0.patch", 2116 "created_at": "2025-09-15T08:57:23Z" 2117 }, 2118 "repository": { ... }, 2119 "sender": { 2120 "did": "did:plc:hwevmowznbiukdf6uk5dwrrq" 2121 } 2122} 2123``` 2124 2125Notes: 2126 2127- `action` mirrors the event type suffix (`created`, `resubmitted`, `merged`, `closed`, `reopened`). 2128- `repository` has the same format as in the push payload. 2129- The patch itself is not embedded in the payload (patches can be large); fetch it from `patch_url` instead. `round_number` identifies the latest round, and `patch_url` always points at that round's patch. 2130- `source` is only present for branch-based and fork-based pull requests; it is omitted for patch-based pulls. For fork-based pulls, `source.repo` contains the DID of the source repository. 2131- `sender` is the user who performed the action. 2132 2133## HTTP headers 2134 2135Each webhook request includes the following headers: 2136 2137- `Content-Type: application/json` 2138- `User-Agent: Tangled-Hook/<short-sha>` — User agent with short SHA of the commit (push events); `Tangled-Hook/pull_request` for pull request events 2139- `X-Tangled-Event: push` — The full event type (e.g. `push`, `pull_request:merged`) 2140- `X-Tangled-Hook-ID: <webhook-id>` — The webhook ID 2141- `X-Tangled-Delivery: <uuid>` — Unique delivery ID 2142- `X-Tangled-Signature-256: sha256=<hmac>` — HMAC-SHA256 signature (if secret configured) 2143 2144## Verifying webhook signatures 2145 2146If you configured a secret, you should verify the webhook signature to ensure requests are authentic. For example, in Go: 2147 2148```go 2149package main 2150 2151import ( 2152 "crypto/hmac" 2153 "crypto/sha256" 2154 "encoding/hex" 2155 "io" 2156 "net/http" 2157 "strings" 2158) 2159 2160func verifySignature(payload []byte, signatureHeader, secret string) bool { 2161 // Remove 'sha256=' prefix from signature header 2162 signature := strings.TrimPrefix(signatureHeader, "sha256=") 2163 2164 // Compute expected signature 2165 mac := hmac.New(sha256.New, []byte(secret)) 2166 mac.Write(payload) 2167 expected := hex.EncodeToString(mac.Sum(nil)) 2168 2169 // Use constant-time comparison to prevent timing attacks 2170 return hmac.Equal([]byte(signature), []byte(expected)) 2171} 2172 2173func webhookHandler(w http.ResponseWriter, r *http.Request) { 2174 // Read the request body 2175 payload, err := io.ReadAll(r.Body) 2176 if err != nil { 2177 http.Error(w, "Bad request", http.StatusBadRequest) 2178 return 2179 } 2180 2181 // Get signature from header 2182 signatureHeader := r.Header.Get("X-Tangled-Signature-256") 2183 2184 // Verify signature 2185 if signatureHeader != "" && verifySignature(payload, signatureHeader, yourSecret) { 2186 // Webhook is authentic, process it 2187 processWebhook(payload) 2188 w.WriteHeader(http.StatusOK) 2189 } else { 2190 http.Error(w, "Invalid signature", http.StatusUnauthorized) 2191 } 2192} 2193``` 2194 2195## Delivery retries 2196 2197Webhooks are automatically retried on failure: 2198 2199- **3 total attempts** (1 initial + 2 retries) 2200- **Exponential backoff** starting at 1 second, max 10 seconds 2201- **Retried on**: 2202 - Network errors 2203 - HTTP 5xx server errors 2204- **Not retried on**: 2205 - HTTP 4xx client errors (bad request, unauthorized, etc.) 2206 2207### Timeouts 2208 2209Webhook requests timeout after 30 seconds. If your endpoint needs more time: 2210 22111. Respond with 200 OK immediately 22122. Process the webhook asynchronously in the background 2213 2214## Example integrations 2215 2216### Discord notifications 2217 2218```javascript 2219app.post("/webhook", (req, res) => { 2220 const payload = req.body; 2221 2222 fetch("https://discord.com/api/webhooks/...", { 2223 method: "POST", 2224 headers: { "Content-Type": "application/json" }, 2225 body: JSON.stringify({ 2226 content: `New push to ${payload.repository.full_name}`, 2227 embeds: [ 2228 { 2229 title: `${payload.pusher.did} pushed to ${payload.ref}`, 2230 url: payload.repository.html_url, 2231 color: 0x00ff00, 2232 }, 2233 ], 2234 }), 2235 }); 2236 2237 res.status(200).send("OK"); 2238}); 2239``` 2240 2241# Migrating knots and spindles 2242 2243Sometimes, non-backwards compatible changes are made to the 2244knot/spindle XRPC APIs. If you host a knot or a spindle, you 2245will need to follow this guide to upgrade. Typically, this 2246only requires you to deploy the newest version. 2247 2248This document is laid out in reverse-chronological order. 2249Newer migration guides are listed first, and older guides 2250are further down the page. 2251 2252## Upgrading to v1.16.0-alpha 2253 2254Starting with v1.16.0-alpha, spindles own CI pipeline data 2255directly. The appview no longer stores pipeline runs or follows 2256spindle event streams for pipeline history. Instead, it asks the 2257configured spindle for pipeline lists, single pipeline details, 2258workflow logs, retries, and cancellations over XRPC. 2259 2260This means that existing pipeline logs / runs won't appear after 2261you upgrade. Existing pipeline history from the appview cannot be 2262automatically migrated. If you want to migrate your data, you can 2263reach out to us and we will send you an SQL file that'll add the data 2264into your spindle. 2265 2266- Upgrade to the latest tag (v1.16.0 or above) 2267- Head to the [spindle 2268 dashboard](https://tangled.org/settings/spindles) and hit the 2269 "retry" button to verify your spindle 2270 2271## Upgrading to v1.15.0-alpha 2272 2273With v1.15.0-alpha, a knot itself owns its members and 2274per-repo collaborators directly. Previously this data was sourced from 2275PDS records (`sh.tangled.knot.member` and `sh.tangled.repo.collaborator`) 2276that the appview and the knot both read off the firehose. 2277The knot is now the source of truth and serves them over XRPC instead: 2278 2279- `sh.tangled.knot.addMember`, `sh.tangled.knot.removeMember`, `sh.tangled.knot.listMembers` 2280- `sh.tangled.repo.addCollaborator`, `sh.tangled.repo.removeCollaborator`, `sh.tangled.repo.listCollaborators` 2281 2282Until your knot is upgraded, the appview keeps reading its 2283members and collaborators from the old firehose-sourced records. 2284Upgrade to move your knot onto knot-owned access control. 2285 2286- Upgrade to the latest tag (v1.15.0 or above) 2287- Head to the [knot dashboard](https://tangled.org/settings/knots) and 2288 hit the "retry" button to verify your knot 2289 2290## Upgrading to v1.14.0-alpha 2291 2292Starting with v1.14.0-alpha, the fully knot uses the repoDID as its 2293canonical handle for repositories. This unlocks repository 2294renames from the appview UI and changes the wire format for 2295the following lexicons (`sh.tangled.repo.pull`, `sh.tangled.repo.collaborator`, 2296`sh.tangled.repo.issue`, `sh.tangled.git.refUpdate`). 2297 2298Knots that have not been upgraded may silently drop new push 2299events, pull requests, issues, and collaborator invites for 2300repositories they host until upgraded. So upgrade please!!! 2301 2302- Upgrade to the latest tag (v1.14.0 or above) 2303- Head to the [knot dashboard](https://tangled.org/settings/knots) and 2304 hit the "retry" button to verify your knot 2305 2306## Upgrading to v1.13.0-alpha 2307 2308Starting with v1.13.0-alpha, every repository on a knot is 2309assigned a DID. This makes repositories stable across 2310renames and transfers. 2311 2312When you upgrade your knot to this version, the server will 2313automatically mint DIDs for all existing repositories on 2314startup. This is a one-time process and you may see 2315additional log output during the first boot as DIDs are 2316assigned. 2317 2318- Upgrade to the latest tag (v1.13.0 or above) 2319- Head to the [knot dashboard](https://tangled.org/settings/knots) and 2320 hit the "retry" button to verify your knot 2321 2322## Upgrading from v1.8.x 2323 2324After v1.8.2, the HTTP API for knots and spindles has been 2325deprecated and replaced with XRPC. Repositories on outdated 2326knots will not be viewable from the appview. Upgrading is 2327straightforward however. 2328 2329For knots: 2330 2331- Upgrade to the latest tag (v1.9.0 or above) 2332- Head to the [knot dashboard](https://tangled.org/settings/knots) and 2333 hit the "retry" button to verify your knot 2334 2335For spindles: 2336 2337- Upgrade to the latest tag (v1.9.0 or above) 2338- Head to the [spindle 2339 dashboard](https://tangled.org/settings/spindles) and hit the 2340 "retry" button to verify your spindle 2341 2342## Upgrading from v1.7.x 2343 2344After v1.7.0, knot secrets have been deprecated. You no 2345longer need a secret from the appview to run a knot. All 2346authorized commands to knots are managed via [Inter-Service 2347Authentication](https://atproto.com/specs/xrpc#inter-service-authentication-jwt). 2348Knots will be read-only until upgraded. 2349 2350Upgrading is quite easy, in essence: 2351 2352- `KNOT_SERVER_SECRET` is no more, you can remove this 2353 environment variable entirely 2354- `KNOT_SERVER_OWNER` is now required on boot, set this to 2355 your DID. You can find your DID in the 2356 [settings](https://tangled.org/settings) page. 2357- Restart your knot once you have replaced the environment 2358 variable 2359- Head to the [knot dashboard](https://tangled.org/settings/knots) and 2360 hit the "retry" button to verify your knot. This simply 2361 writes a `sh.tangled.knot` record to your PDS. 2362 2363If you use the nix module, simply bump the flake to the 2364latest revision, and change your config block like so: 2365 2366```diff 2367 services.tangled.knot = { 2368 enable = true; 2369 server = { 2370- secretFile = /path/to/secret; 2371+ owner = "did:plc:foo"; 2372 }; 2373 }; 2374``` 2375 2376# Bobbin 2377 2378Bobbin is an API appview for Tangled records. It serves XRPC 2379endpoints for `sh.tangled.*`, with it you can get repos, 2380issues, pulls, comments, follows, stars, labels, pipelines, 2381and profiles. It is read-only, there is no auth, since that 2382should all be handled direct-to-PDS and knot respectively. 2383 2384**Bobbin has no permanent storage**. 2385 2386It is only a glorified edge index, in the graph theory 2387sense. Additionally it has a record cache, re-filled on 2388demand. All other data that Bobbin serves comes live from 2389PDSes & knots. 2390 2391## What Bobbin needs 2392 2393The way that Bobbin is able to pull off being 2394so stateless is by moving state upstream. 2395Primarily it depends on an instance of 2396[Hydrant](https://tangled.org/did:plc:6v3ul2ptnqctyxwkz5ti4amn) 2397, which is the service that gives an event stream 2398for Bobbin to quickly backfill from on every restart. 2399Backfilling ought to take less than a couple of minutes 2400maximum. If the upstream instance of Hydrant fails 2401while Bobbin is live, its list/count endpoints stop 2402advancing and report a stale cursor. Single-lookups 2403will continue working, due to the second dependency: 2404[Slingshot](https://tangled.org/did:plc:c7mc2fn47ihdihul4vjwsuy3/tree/main/slingshot). 2405Slingshot fetches individual records & resolves identities. 2406If the upstream instance of Slingshot fails, single-lookups 2407will fail with a `502` error. There are some aggregation 2408endpoints that use Slingshot for hydrating, which will also 2409fail. 2410 2411A soft dependency that ought to exist for Bobbin to operate 2412correctly is simply the plethora of knots that are out 2413there, that Bobbin talks to directly for git data and, for 2414knots at v1.15+, members & collaborators. 2415 2416## Building Bobbin 2417 2418Bobbin is under [Tangled's core monorepo, under bobbin/](https://tangled.org/did:plc:j5hmlfdrwkvtxm7cjmu7j2is/tree/master/bobbin). 2419Here's an easy local debug-build: 2420 2421```sh 2422cargo build -p bobbin 2423``` 2424 2425Bobbin loves being in a container. When using 2426`bobbin/containerfiles/bobbin.Containerfile`, it runs `cargo 2427build --release --bin bobbin --package bobbin` within a 2428little Debian runtime, exposing port 8090. 2429 2430## Configuration 2431 2432The best way to configure Bobbin is via a toml config file. 2433There's an `example.toml` in [Bobbin's subdir](https://tangled.org/did:plc:j5hmlfdrwkvtxm7cjmu7j2is/blob/master/bobbin/example.toml). 2434Every value is overridable by a `BOBBIN_*` env var. 2435The load order is env, then `--config <path>`, then 2436`/etc/bobbin/config.toml`, then built-in defaults. 2437 2438Load and check a config without starting the server: 2439 2440```sh 2441bobbin --config config.toml validate 2442``` 2443 2444Minimal config is the two upstream URLs. The hydrant URL 2445takes `ws://` or `wss://`. An `http://` or `https://` 2446URL is rewritten to the matching websocket scheme at 2447connection-time. 2448 2449```toml 2450[server] 2451binds = ["127.0.0.1:8090"] 2452 2453# Loopback-only & can leave empty to disable debug introspection. 2454debug_bind = "127.0.0.1:8091" 2455 2456[hydrant] 2457url = "https://hydrant.example.com" 2458 2459[slingshot] 2460url = "https://slingshot.example.com" 2461``` 2462 2463> 🦪 Lewis 2464> 2465> At time of writing, we (Tangled) don't host public 2466> instances of Hydrant or Slingshot. You will have to 2467> find public instances or spin these up yourself! :P 2468 2469Take a gander in the project's example.toml for an 2470exhaustive list of things to configure. 2471 2472You will discover fun things such as a configurable adaptive 2473loop that watches the cgroup memory limit & throttles heavy 2474requests under pressure. It only works if it detects a 2475cgroup limit is present. The config for that is in the 2476`[backpressure]` block of the config template. 2477 2478## Running Bobbin 2479 2480Start the server using a config toml: 2481 2482```bash 2483bobbin --config config.toml 2484``` 2485Bobbin wakes up in a cold sweat and immediately gets to 2486work: 24871. It binds its listeners, connects to the Hydrant stream 2488 in the background. 24892. It serves requests from the first 2490 moment it's alive, even before the Hydrant stream connects 2491 or finishes catching up. Having a cold Hydrant itself 2492 costs only latency and approximate counts. 2493 2494## The API 2495 2496**Single lookups** take a record's AT-URI. 2497 2498- `getRepo` takes the repo URI: 2499 2500```sh 2501curl "$BOBBIN/xrpc/sh.tangled.repo.getRepo?repo=at://did:plc:boltless/sh.tangled.repo/squid" 2502``` 2503```json 2504{ 2505 "uri": "at://did:plc:boltless/sh.tangled.repo/squid", 2506 "cid": "bafyrei...", 2507 "value": { "$type": "sh.tangled.repo", "knot": "knot1.tangled.sh", "description": "...", "createdAt": "..." } 2508} 2509``` 2510 2511- `getProfile` takes the full profile record URI, so a bare 2512 handle or DID will not resolve: 2513 2514```sh 2515curl "$BOBBIN/xrpc/sh.tangled.actor.getProfile?actor=at://did:plc:boltless/sh.tangled.actor.profile/self" 2516``` 2517 2518- If Slingshot cannot serve the record, the response is `502`: 2519 2520```json 2521{ "error": "UpstreamFailed", "message": "upstream unavailable: ..." } 2522``` 2523 2524**Aggregation** endpoints come in `list*` and `count*` pairs, 2525each with a `*By` sibling, and require a `subject` query param. 2526 2527- `listRepos` and `countRepos` key on the owner DID: 2528 2529```sh 2530curl "$BOBBIN/xrpc/sh.tangled.repo.countRepos?subject=did:plc:boltless" 2531``` 2532```json 2533{ "count": 7, "distinctAuthors": 1 } 2534``` 2535 2536```sh 2537curl "$BOBBIN/xrpc/sh.tangled.repo.listRepos?subject=did:plc:boltless&limit=3" 2538``` 2539```json 2540{ "items": [ { "uri": "at://did:plc:boltless/sh.tangled.repo/squid", "cid": "bafyrei...", "value": { } } ], "cursor": null } 2541``` 2542 2543- Bobbin validates the subject per collection. Here a repo URI 2544 is passed where a bare DID is required, so the call returns a 2545 `400`: 2546 2547```sh 2548curl "$BOBBIN/xrpc/sh.tangled.graph.listFollows?subject=at://did:plc:boltless/sh.tangled.repo/squid" 2549``` 2550```json 2551{ "error": "InvalidRequest", "message": "invalid request: subject must be a bare did, got at-uri with collection sh.tangled.repo" } 2552``` 2553 2554**Search** is a single endpoint over an in-mem full-text 2555index: 2556 2557```sh 2558curl "$BOBBIN/xrpc/sh.tangled.search.query?q=tangled&limit=2" 2559``` 2560```json 2561{ "hits": [ { "uri": "at://...", "cid": "...", "nsid": "sh.tangled.repo", "score": 27.1, "value": { } } ], "cursor": null } 2562``` 2563 2564**Git data** such as blob, tree, diff, log, and archive proxies 2565straight to the repo's knot, streamed back without caching. 2566 2567## Coverage and warm-up 2568 2569- While the edge index is catching up from Hydrant, 2570 the aggregation count is a lower bound & may still climb. 2571- One endpoint reports how far along the backfill it is: 2572 2573```sh 2574curl "$BOBBIN/xrpc/sh.tangled.bobbin.getCoverage" 2575``` 2576 2577While warming up: 2578 2579```json 2580{ "ready": false, "eventsProcessed": 45588, "lastCursor": 51658 } 2581``` 2582 2583Once caught up, Bobbin flips to ready: 2584 2585```json 2586{ "ready": true, "eventsProcessed": 106085, "lastCursor": 116527 } 2587``` 2588 2589If starting up Hydrant for the first time, Hydrant itself 2590will take a decent while (a couple of hours) to backfill 2591from PDSes. Hydrant stores its backfill on disk. Bobbin 2592restart reaches `ready` in minutes by replaying event from 2593an already-populated Hydrant. If your Hydrant is new, expect 2594Bobbin to backfill in that same couple of hours that Hydrant 2595takes. 2596 2597## Loose ends and not-gonna-impl 2598 2599- **No coverage signal for per-knot rosters yet.** 2600 Coverage tracks the hydrant stream only. A v1.15 knot 2601 that is unreachable serves a stale or empty member set 2602 with nothing to flag it. 2603- **Knot eventstream fan-out isn't pooled.** 2604 Bobbin opens one websocket per v1.15 2605 knot on top of the hydrant subscription. A network with 2606 thousands of knots wants pooling or a shared subscription. 2607- **No sequential issue or PR numbers.** bobbin returns rkeys, 2608 not `#42` style ids like the web appview. A client 2609 deriving a display number does it from creation order. But 2610 why bother? rkeys are the IDs. 2611 2612# Hacking on Tangled 2613 2614We highly recommend [installing 2615Nix](https://nixos.org/download/) (the package manager) 2616before working on the codebase. The Nix flake provides a lot 2617of helpers to get started and most importantly, builds and 2618dev shells are entirely deterministic. 2619 2620To set up your dev environment: 2621 2622```bash 2623nix develop 2624``` 2625 2626Non-Nix users can look at the `devShell` attribute in the 2627`flake.nix` file to determine necessary dependencies. 2628 2629## Running the appview 2630 2631The appview requires Redis and OAuth JWKs. Start these 2632first, before launching the appview itself. 2633 2634```bash 2635# OAuth JWKs should already be set up by the Nix devshell: 2636echo $TANGLED_OAUTH_CLIENT_SECRET 2637z42ty4RT1ovnTopY8B8ekz9NuziF2CuMkZ7rbRFpAR9jBqMc 2638 2639echo $TANGLED_OAUTH_CLIENT_KID 26401761667908 2641 2642# if not, you can set it up yourself: 2643goat key generate -t P-256 2644Key Type: P-256 / secp256r1 / ES256 private key 2645Secret Key (Multibase Syntax): save this securely (eg, add to password manager) 2646 z42tuPDKRfM2mz2Kv953ARen2jmrPA8S9LX9tRq4RVcUMwwL 2647Public Key (DID Key Syntax): share or publish this (eg, in DID document) 2648 did:key:zDnaeUBxtG6Xuv3ATJE4GaWeyXM3jyamJsZw3bSPpxx4bNXDR 2649 2650# the secret key from above 2651export TANGLED_OAUTH_CLIENT_SECRET="z42tuP..." 2652 2653# Run Redis in a new shell to store OAuth sessions 2654redis-server 2655``` 2656 2657The Nix flake exposes a few `app` attributes (run `nix 2658flake show` to see a full list of what the flake provides), 2659one of the apps runs the appview with the `air` 2660live-reloader: 2661 2662```bash 2663TANGLED_DEV=true nix run .#watch-appview 2664 2665# TANGLED_DB_PATH might be of interest to point to 2666# different sqlite DBs 2667 2668# in a separate shell, you can live-reload tailwind 2669nix run .#watch-tailwind 2670``` 2671 2672## Running knots and spindles 2673 2674An end-to-end knot setup requires setting up a machine with 2675`sshd`, `AuthorizedKeysCommand`, and a Git user, which is 2676quite cumbersome. So the Nix flake provides a 2677`nixosConfiguration` to do so. 2678 2679<details> 2680 <summary><strong>macOS users will have to set up a Nix Builder first</strong></summary> 2681 2682In order to build Tangled's dev VM on macOS, you will 2683first need to set up a Linux Nix builder. The recommended 2684way to do so is to run a [`darwin.linux-builder` 2685VM](https://nixos.org/manual/nixpkgs/unstable/#sec-darwin-builder) 2686and to register it in `nix.conf` as a builder for Linux 2687with the same architecture as your Mac (`linux-aarch64` if 2688you are using Apple Silicon). 2689 2690If you're on nix-darwin, you can simply add 2691 2692``` 2693nix.linux-builder.enable = true; 2694``` 2695 2696to your host's `configuration.nix`. 2697 2698Alternatively, you can use any other method to set up a 2699Linux machine with Nix installed that you can `sudo ssh` 2700into (in other words, root user on your Mac has to be able 2701to ssh into the Linux machine without entering a password) 2702and that has the same architecture as your Mac. See 2703[remote builder 2704instructions](https://nix.dev/manual/nix/2.28/advanced-topics/distributed-builds.html#requirements) 2705for how to register such a builder in `nix.conf`. 2706 2707> WARNING: If you'd like to use 2708> [`nixos-lima`](https://github.com/nixos-lima/nixos-lima) or 2709> [Orbstack](https://orbstack.dev/), note that setting them up so that `sudo 2710ssh` works can be tricky. It seems to be [possible with 2711> Orbstack](https://github.com/orgs/orbstack/discussions/1669). 2712 2713</details> 2714 2715To begin, grab your DID from http://localhost:3000/settings. 2716Then, set `TANGLED_VM_KNOT_OWNER` and 2717`TANGLED_VM_SPINDLE_OWNER` to your DID. You can now start a 2718lightweight NixOS VM like so: 2719 2720```bash 2721nix run --impure .#vm 2722 2723# type `poweroff` at the shell to exit the VM 2724``` 2725 2726This starts a knot on port 6444, a spindle on port 6555 2727with `ssh` exposed on port 2222. 2728 2729Once the services are running, head to 2730http://localhost:3000/settings/knots and hit "Verify". It should 2731verify the ownership of the services instantly if everything 2732went smoothly. 2733 2734You can push repositories to this VM with this ssh config 2735block on your main machine: 2736 2737```bash 2738Host nixos-shell 2739 Hostname localhost 2740 Port 2222 2741 User git 2742 IdentityFile ~/.ssh/my_tangled_key 2743``` 2744 2745Set up a remote called `local-dev` on a git repo: 2746 2747```bash 2748git remote add local-dev git@nixos-shell:user/repo 2749git push local-dev main 2750``` 2751 2752The above VM should already be running a spindle on 2753`localhost:6555`. Head to http://localhost:3000/settings/spindles and 2754hit "Verify". You can then configure each repository to use 2755this spindle and run CI jobs. 2756 2757Of interest when debugging spindles: 2758 2759``` 2760# Service logs from journald: 2761journalctl -xeu spindle 2762 2763# CI job logs from disk: 2764ls /var/log/spindle 2765 2766# Debugging spindle database: 2767sqlite3 /var/lib/spindle/spindle.db 2768 2769# litecli has a nicer REPL interface: 2770litecli /var/lib/spindle/spindle.db 2771``` 2772 2773If for any reason you wish to disable either one of the 2774services in the VM, modify [nix/vm.nix](/nix/vm.nix) and set 2775`services.tangled.spindle.enable` (or 2776`services.tangled.knot.enable`) to `false`. 2777 2778# Contribution guide 2779 2780## Commit guidelines 2781 2782We follow a commit style similar to the Go project. Please keep commits: 2783 2784- **atomic**: each commit should represent one logical change 2785- **descriptive**: the commit message should clearly describe what the 2786 change does and why it's needed 2787 2788### Message format 2789 2790``` 2791<service/top-level directory>/<affected package/directory>: <short summary of change> 2792 2793Optional longer description can go here, if necessary. Explain what the 2794change does and why, especially if not obvious. Reference relevant 2795issues or PRs when applicable. These can be links for now since we don't 2796auto-link issues/PRs yet. 2797``` 2798 2799Here are some examples: 2800 2801``` 2802appview/state: fix token expiry check in middleware 2803 2804The previous check did not account for clock drift, leading to premature 2805token invalidation. 2806``` 2807 2808``` 2809knotserver/git/service: improve error checking in upload-pack 2810``` 2811 2812### General notes 2813 2814- PRs get merged "as-is" (fast-forward)—like applying a patch-series 2815 using `git am`. At present, there is no squashing—so please author 2816 your commits as they would appear on `master`, following the above 2817 guidelines. 2818- If there is a lot of nesting, for example "appview: 2819 pages/templates/repo/fragments: ...", these can be truncated down to 2820 just "appview: repo/fragments: ...". If the change affects a lot of 2821 subdirectories, you may abbreviate to just the top-level names, e.g. 2822 "appview: ..." or "knotserver: ...". 2823- Keep commits lowercased with no trailing period. 2824- Use the imperative mood in the summary line (e.g., "fix bug" not 2825 "fixed bug" or "fixes bug"). 2826- Try to keep the summary line under 72 characters, but we aren't too 2827 fussed about this. 2828- Follow the same formatting for PR titles if filled manually. 2829- Don't include unrelated changes in the same commit. 2830- Avoid noisy commit messages like "wip" or "final fix"—rewrite history 2831 before submitting if necessary. 2832 2833## Code formatting 2834 2835We use a variety of tools to format our code, and multiplex them with 2836[`treefmt`](https://treefmt.com). All you need to do to format your changes 2837is run `nix run .#fmt` (or just `treefmt` if you're in the devshell). 2838 2839## Proposals for bigger changes 2840 2841Small fixes like typos, minor bugs, or trivial refactors can be 2842submitted directly as PRs. 2843 2844For larger changes—especially those introducing new features, significant 2845refactoring, or altering system behavior—please open a proposal first. This 2846helps us evaluate the scope, design, and potential impact before implementation. 2847 2848Create a new issue titled: 2849 2850``` 2851proposal: <affected scope>: <summary of change> 2852``` 2853 2854In the description, explain: 2855 2856- What the change is 2857- Why it's needed 2858- How you plan to implement it (roughly) 2859- Any open questions or tradeoffs 2860 2861We'll use the issue thread to discuss and refine the idea before moving 2862forward. 2863 2864## Developer Certificate of Origin (DCO) 2865 2866We require all contributors to certify that they have the right to 2867submit the code they're contributing. To do this, we follow the 2868[Developer Certificate of Origin 2869(DCO)](https://developercertificate.org/). 2870 2871By signing your commits, you're stating that the contribution is your 2872own work, or that you have the right to submit it under the project's 2873license. This helps us keep things clean and legally sound. 2874 2875To sign your commit, just add the `-s` flag when committing: 2876 2877```sh 2878git commit -s -m "your commit message" 2879``` 2880 2881This appends a line like: 2882 2883``` 2884Signed-off-by: Your Name <your.email@example.com> 2885``` 2886 2887We won't merge commits if they aren't signed off. If you forget, you can 2888amend the last commit like this: 2889 2890```sh 2891git commit --amend -s 2892``` 2893 2894If you're submitting a PR with multiple commits, make sure each one is 2895signed. 2896 2897For [jj](https://jj-vcs.github.io/jj/latest/) users, you can run the following command 2898to make it sign off commits in the tangled repo: 2899 2900```shell 2901# Safety check, should say "No matching config key..." 2902jj config list templates.commit_trailers 2903# The command below may need to be adjusted if the command above returned something. 2904jj config set --repo templates.commit_trailers "format_signed_off_by_trailer(self)" 2905``` 2906 2907Refer to the [jujutsu 2908documentation](https://jj-vcs.github.io/jj/latest/config/#commit-trailers) 2909for more information. 2910 2911# Troubleshooting guide 2912 2913## Login issues 2914 2915Owing to the distributed nature of OAuth on AT Protocol, you 2916may run into issues with logging in. If you run a 2917self-hosted PDS: 2918 2919- You may need to ensure that your PDS is timesynced using 2920 NTP: 2921 - Enable the `ntpd` service 2922 - Run `ntpd -qg` to synchronize your clock 2923- You may need to increase the default request timeout: 2924 `NODE_OPTIONS="--network-family-autoselection-attempt-timeout=500"` 2925 2926## Empty punchcard 2927 2928For Tangled to register commits that you make across the 2929network, you need to setup one of following: 2930 2931- The committer email should be a verified email associated 2932 to your account. You can add and verify emails on the 2933 settings page. 2934- Or, the committer email should be set to your account's 2935 DID: `git config user.email "did:plc:foobar"`. You can find 2936 your account's DID on the settings page 2937 2938## Commit is not marked as verified 2939 2940Tangled only supports SSH commit signatures. Ensure the SSH public key you use 2941for signing is uploaded to Tangled. 2942 2943To sign commits using an SSH key with git: 2944 2945``` 2946git config --global gpg.format ssh 2947git config --global user.signingkey ~/.ssh/tangled-key 2948``` 2949 2950To sign commits using an SSH key with jj, add this to your 2951config: 2952 2953``` 2954[signing] 2955behavior = "own" 2956backend = "ssh" 2957key = "~/.ssh/tangled-key" 2958``` 2959 2960## Self-hosted knot issues 2961 2962If you need help troubleshooting a self-hosted knot, check 2963out the [knot troubleshooting 2964guide](/knot-self-hosting-guide.html#troubleshooting).