alice.pds.demo.boltless.dev / core
This repository has no description
0
Fork 0

Configure Feed

Select the types of activity you want to include in your feed.

core / docs / DOCS.md
55 kB 1927 lines
1--- 2title: Tangled docs 3author: The Tangled Contributors 4date: 21 Sun, Dec 2025 5abstract: | 6 Tangled is a decentralized code hosting and collaboration 7 platform. Every component of Tangled is open-source and 8 self-hostable. [tangled.org](https://tangled.org) also 9 provides hosting and CI services that are free to use. 10 11 There are several models for decentralized code 12 collaboration platforms, ranging from ActivityPub’s 13 (Forgejo) federated model, to Radicle’s entirely P2P model. 14 Our approach attempts to be the best of both worlds by 15 adopting the AT Protocol—a protocol for building decentralized 16 social applications with a central identity 17 18 Our approach to this is the idea of “knots”. Knots are 19 lightweight, headless servers that enable users to host Git 20 repositories with ease. Knots are designed for either single 21 or multi-tenant use which is perfect for self-hosting on a 22 Raspberry Pi at home, or larger “community” servers. By 23 default, Tangled provides managed knots where you can host 24 your repositories for free. 25 26 The appview at tangled.org acts as a consolidated "view" 27 into the whole network, allowing users to access, clone and 28 contribute to repositories hosted across different knots 29 seamlessly. 30--- 31 32# Quick start guide 33 34## Login or sign up 35 36You can [login](https://tangled.org) by using your AT Protocol 37account. If you are unclear on what that means, simply head 38to the [signup](https://tangled.org/signup) page and create 39an account. By doing so, you will be choosing Tangled as 40your account provider (you will be granted a handle of the 41form `user.tngl.sh`). 42 43In the AT Protocol network, users are free to choose their account 44provider (known as a "Personal Data Service", or PDS), and 45login to applications that support AT accounts. 46 47You can think of it as "one account for all of the atmosphere"! 48 49If you already have an AT account (you may have one if you 50signed up to Bluesky, for example), you can login with the 51same handle on Tangled (so just use `user.bsky.social` on 52the login page). 53 54## Add an SSH key 55 56Once you are logged in, you can start creating repositories 57and pushing code. Tangled supports pushing git repositories 58over SSH. 59 60First, you'll need to generate an SSH key if you don't 61already have one: 62 63```bash 64ssh-keygen -t ed25519 -C "foo@bar.com" 65``` 66 67When prompted, save the key to the default location 68(`~/.ssh/id_ed25519`) and optionally set a passphrase. 69 70Copy your public key to your clipboard: 71 72```bash 73# on X11 74cat ~/.ssh/id_ed25519.pub | xclip -sel c 75 76# on wayland 77cat ~/.ssh/id_ed25519.pub | wl-copy 78 79# on macos 80cat ~/.ssh/id_ed25519.pub | pbcopy 81``` 82 83Now, navigate to 'Settings' -> 'Keys' and hit 'Add Key', 84paste your public key, give it a descriptive name, and hit 85save. 86 87## Create a repository 88 89Once your SSH key is added, create your first repository: 90 911. Hit the green `+` icon on the topbar, and select 92 repository 932. Enter a repository name 943. Add a description 954. Choose a knotserver to host this repository on 965. Hit create 97 98Knots are self-hostable, lightweight Git servers that can 99host your repository. Unlike traditional code forges, your 100code can live on any server. Read the [Knots](TODO) section 101for more. 102 103## Configure SSH 104 105To ensure Git uses the correct SSH key and connects smoothly 106to Tangled, add this configuration to your `~/.ssh/config` 107file: 108 109``` 110Host tangled.org 111 Hostname tangled.org 112 User git 113 IdentityFile ~/.ssh/id_ed25519 114 AddressFamily inet 115``` 116 117This tells SSH to use your specific key when connecting to 118Tangled and prevents authentication issues if you have 119multiple SSH keys. 120 121Note that this configuration only works for knotservers that 122are hosted by tangled.org. If you use a custom knot, refer 123to the [Knots](TODO) section. 124 125## Push your first repository 126 127Initialize a new Git repository: 128 129```bash 130mkdir my-project 131cd my-project 132 133git init 134echo "# My Project" > README.md 135``` 136 137Add some content and push! 138 139```bash 140git add README.md 141git commit -m "Initial commit" 142git remote add origin git@tangled.org:user.tngl.sh/my-project 143git push -u origin main 144``` 145 146That's it! Your code is now hosted on Tangled. 147 148## Migrating an existing repository 149 150Moving your repositories from GitHub, GitLab, Bitbucket, or 151any other Git forge to Tangled is straightforward. You'll 152simply change your repository's remote URL. At the moment, 153Tangled does not have any tooling to migrate data such as 154GitHub issues or pull requests. 155 156First, create a new repository on tangled.org as described 157in the [Quick Start Guide](#create-a-repository). 158 159Navigate to your existing local repository: 160 161```bash 162cd /path/to/your/existing/repo 163``` 164 165You can inspect your existing Git remote like so: 166 167```bash 168git remote -v 169``` 170 171You'll see something like: 172 173```bash 174origin git@github.com:username/my-project.git (fetch) 175origin git@github.com:username/my-project.git (push) 176``` 177 178Update the remote URL to point to tangled: 179 180```bash 181git remote set-url origin git@tangled.org:user.tngl.sh/my-project 182``` 183 184Verify the change: 185 186```bash 187git remote -v 188``` 189 190You should now see: 191 192```bash 193origin git@tangled.org:user.tngl.sh/my-project (fetch) 194origin git@tangled.org:user.tngl.sh/my-project (push) 195``` 196 197Push all your branches and tags to Tangled: 198 199```bash 200git push -u origin --all 201git push -u origin --tags 202``` 203 204Your repository is now migrated to Tangled! All commit 205history, branches, and tags have been preserved. 206 207## Mirroring a repository to Tangled 208 209If you want to maintain your repository on multiple forges 210simultaneously, for example, keeping your primary repository 211on GitHub while mirroring to Tangled for backup or 212redundancy, you can do so by adding [multiple remotes](https://git-scm.com/docs/git-push#_remotes). 213 214You can configure your local repository to push to both 215Tangled and, say, GitHub. You may already have the following 216setup: 217 218```bash 219$ git remote -v 220origin git@github.com:username/my-project.git (fetch) 221origin git@github.com:username/my-project.git (push) 222``` 223 224Now add Tangled as an additional push URL to the same 225remote: 226 227```bash 228git remote set-url --add --push origin git@tangled.org:user.tngl.sh/my-project 229``` 230 231You also need to re-add the original URL as a push 232destination (Git will now use the original URL to fetch only): 233 234```bash 235git remote set-url --add --push origin git@github.com:username/my-project.git 236``` 237 238Verify your configuration: 239 240```bash 241$ git remote -v 242origin git@github.com:username/my-project.git (fetch) 243origin git@tangled.org:user.tngl.sh/my-project (push) 244origin git@github.com:username/my-project.git (push) 245``` 246 247Notice that there's one fetch URL (the primary remote) and 248two push URLs. Now, whenever you push, Git will 249automatically push to both remotes: 250 251```bash 252git push origin main 253``` 254 255This single command pushes your `main` branch to both GitHub 256and Tangled simultaneously. 257 258To push all branches and tags: 259 260```bash 261git push origin --all 262git push origin --tags 263``` 264 265If you prefer more control over which remote you push to, 266you can maintain separate remotes: 267 268```bash 269git remote add github git@github.com:username/my-project.git 270git remote add tangled git@tangled.org:user.tngl.sh/my-project 271``` 272 273Then push to each explicitly: 274 275```bash 276git push github main 277git push tangled main 278``` 279 280# Hosting websites on Tangled 281 282You can serve static websites directly from your git repositories on 283Tangled. If you've used GitHub Pages or Codeberg Pages, this should feel 284familiar. 285 286## Overview 287 288Every user gets a sites domain. If you signed up through Tangled's own 289PDS (`tngl.sh`), your sites domain is automatically 290`<your-handle>.tngl.sh` no setup needed. Otherwise, you can claim a 291`<subdomain>.tngl.io` domain from your settings. 292 293You can serve multiple sites per domain: 294 295- One **index site** served at the root of your domain (e.g. 296 `alice.tngl.sh`) 297- Any number of **sub-path sites** served under the repository name 298 (e.g. `alice.tngl.sh/my-project`) 299 300## Claiming a domain 301 302If you don't have a `tngl.sh` handle, you need to claim a domain before 303publishing sites: 304 3051. Go to **Settings → Sites** 3062. Enter a subdomain (e.g. `alice` to claim `alice.tngl.io`) 3073. Click **claim** 308 309You can only hold one domain at a time. Releasing a domain puts it in a 31030-day cooldown before anyone else can claim it. 311 312## Configuring a site for a repository 313 3141. Navigate to your repository 3152. Go to **Settings → Sites** 3163. Choose a **branch** to deploy from 3174. Set the **deploy directory** — the path within the repository 318 containing your `index.html`. Use `/` for the root, or a subdirectory 319 like `/docs` or `/public` 3205. Choose the **site type**: 321 - **Index site** — served at the root of your domain (e.g. 322 `alice.tngl.sh`) 323 - **Sub-path site** — served under the repository name (e.g. 324 `alice.tngl.sh/my-project`) 3256. Click **save** 326 327The site will be deployed automatically. You can see the status of your 328previous deploys in the **Recent Deploys** section at the bottom of the 329page. 330 331Sites are redeployed automatically on every push to the configured 332branch. 333 334## Custom domains 335 336Tangled currently doesn't support custom domains for sites. This will be 337added in a future update. 338 339## Deploy directory 340 341The deploy directory is the path within your repository that Tangled 342serves as the site root. It must contain an `index.html`. 343 344| Deploy directory | Result | 345|---|---| 346| `/` | Serves the repository root | 347| `/docs` | Serves the `docs/` subdirectory | 348| `/public` | Serves the `public/` subdirectory | 349 350Directories are served with automatic `index.html` resolution -- a 351request to `/about` will serve `/about/index.html` if it exists. 352 353## Site types 354 355| Type | URL | 356|---|---| 357| Index site | `alice.tngl.sh` | 358| Sub-path site | `alice.tngl.sh/my-project` | 359 360Only one repository can be the index site for a given domain at a time. 361If another repository already holds the index site, you will see a 362notice in the settings and only the sub-path option will be available. 363 364## Deploy triggers 365 366A deployment is triggered automatically when: 367 368- You push to the configured branch 369- You change the site configuration (branch, deploy directory, or site 370 type) 371 372## Disabling a site 373 374To stop serving a site, go to **Settings → Sites** in your repository 375and click **Disable**. This removes the site configuration and stops 376serving the site. The deployed files are also deleted from storage. 377 378Releasing your domain from **Settings → Sites** at the account level 379will disable all sites associated with it and delete their files. 380 381 382# Knot self-hosting guide 383 384So you want to run your own knot server? Great! Here are a few prerequisites: 385 3861. A server of some kind (a VPS, a Raspberry Pi, etc.). Preferably running a Linux distribution of some kind. 3872. A (sub)domain name. People generally use `knot.example.com`. 3883. A valid SSL certificate for your domain. 389 390## NixOS 391 392Refer to the [knot 393module](https://tangled.org/tangled.org/core/blob/master/nix/modules/knot.nix) 394for a full list of options. Sample configurations: 395 396- [The test VM](https://tangled.org/tangled.org/core/blob/master/nix/vm.nix#L85) 397- [@pyrox.dev/nix](https://tangled.org/pyrox.dev/nix/blob/d19571cc1b5fe01035e1e6951ec8cf8a476b4dee/hosts/marvin/services/tangled.nix#L15-25) 398 399## Docker 400 401Refer to 402[@tangled.org/knot-docker](https://tangled.org/@tangled.org/knot-docker). 403Note that this is community maintained. 404 405## Manual setup 406 407First, clone this repository: 408 409``` 410git clone https://tangled.org/@tangled.org/core 411``` 412 413Then, build the `knot` CLI. This is the knot administration 414and operation tool. For the purpose of this guide, we're 415only concerned with these subcommands: 416 417- `knot server`: the main knot server process, typically 418 run as a supervised service 419- `knot guard`: handles role-based access control for git 420 over SSH (you'll never have to run this yourself) 421- `knot keys`: fetches SSH keys associated with your knot; 422 we'll use this to generate the SSH 423 `AuthorizedKeysCommand` 424 425``` 426cd core 427export CGO_ENABLED=1 428go build -o knot ./cmd/knot 429``` 430 431Next, move the `knot` binary to a location owned by `root` -- 432`/usr/local/bin/` is a good choice. Make sure the binary itself is also owned by `root`: 433 434``` 435sudo mv knot /usr/local/bin/knot 436sudo chown root:root /usr/local/bin/knot 437``` 438 439This is necessary because SSH `AuthorizedKeysCommand` requires [really 440specific permissions](https://stackoverflow.com/a/27638306). The 441`AuthorizedKeysCommand` specifies a command that is run by `sshd` to 442retrieve a user's public SSH keys dynamically for authentication. Let's 443set that up. 444 445``` 446sudo tee /etc/ssh/sshd_config.d/authorized_keys_command.conf <<EOF 447Match User git 448 AuthorizedKeysCommand /usr/local/bin/knot keys -o authorized-keys 449 AuthorizedKeysCommandUser nobody 450EOF 451``` 452 453Then, reload `sshd`: 454 455``` 456sudo systemctl reload ssh 457``` 458 459Next, create the `git` user. We'll use the `git` user's home directory 460to store repositories: 461 462``` 463sudo adduser git 464``` 465 466Create `/home/git/.knot.env` with the following, updating the values as 467necessary. The `KNOT_SERVER_OWNER` should be set to your 468DID, you can find your DID in the [Settings](https://tangled.sh/settings) page. 469 470``` 471KNOT_REPO_SCAN_PATH=/home/git 472KNOT_SERVER_HOSTNAME=knot.example.com 473APPVIEW_ENDPOINT=https://tangled.org 474KNOT_SERVER_OWNER=did:plc:foobar 475KNOT_SERVER_INTERNAL_LISTEN_ADDR=127.0.0.1:5444 476KNOT_SERVER_LISTEN_ADDR=127.0.0.1:5555 477``` 478 479If you run a Linux distribution that uses systemd, you can 480use the provided service file to run the server. Copy 481[`knotserver.service`](https://tangled.org/tangled.org/core/blob/master/systemd/knotserver.service) 482to `/etc/systemd/system/`. Then, run: 483 484``` 485systemctl enable knotserver 486systemctl start knotserver 487``` 488 489The last step is to configure a reverse proxy like Nginx or Caddy to front your 490knot. Here's an example configuration for Nginx: 491 492``` 493server { 494 listen 80; 495 listen [::]:80; 496 server_name knot.example.com; 497 498 location / { 499 proxy_pass http://localhost:5555; 500 proxy_set_header Host $host; 501 proxy_set_header X-Real-IP $remote_addr; 502 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; 503 proxy_set_header X-Forwarded-Proto $scheme; 504 } 505 506 # wss endpoint for git events 507 location /events { 508 proxy_set_header X-Forwarded-For $remote_addr; 509 proxy_set_header Host $http_host; 510 proxy_set_header Upgrade websocket; 511 proxy_set_header Connection Upgrade; 512 proxy_pass http://localhost:5555; 513 } 514 # additional config for SSL/TLS go here. 515} 516 517``` 518 519Remember to use Let's Encrypt or similar to procure a certificate for your 520knot domain. 521 522You should now have a running knot server! You can finalize 523your registration by hitting the `verify` button on the 524[/settings/knots](https://tangled.org/settings/knots) page. This simply creates 525a record on your PDS to announce the existence of the knot. 526 527### Custom paths 528 529(This section applies to manual setup only. Docker users should edit the mounts 530in `docker-compose.yml` instead.) 531 532Right now, the database and repositories of your knot lives in `/home/git`. You 533can move these paths if you'd like to store them in another folder. Be careful 534when adjusting these paths: 535 536- Stop your knot when moving data (e.g. `systemctl stop knotserver`) to prevent 537 any possible side effects. Remember to restart it once you're done. 538- Make backups before moving in case something goes wrong. 539- Make sure the `git` user can read and write from the new paths. 540 541#### Database 542 543As an example, let's say the current database is at `/home/git/knotserver.db`, 544and we want to move it to `/home/git/database/knotserver.db`. 545 546Copy the current database to the new location. Make sure to copy the `.db-shm` 547and `.db-wal` files if they exist. 548 549``` 550mkdir /home/git/database 551cp /home/git/knotserver.db* /home/git/database 552``` 553 554In the environment (e.g. `/home/git/.knot.env`), set `KNOT_SERVER_DB_PATH` to 555the new file path (_not_ the directory): 556 557``` 558KNOT_SERVER_DB_PATH=/home/git/database/knotserver.db 559``` 560 561#### Repositories 562 563As an example, let's say the repositories are currently in `/home/git`, and we 564want to move them into `/home/git/repositories`. 565 566Create the new folder, then move the existing repositories (if there are any): 567 568``` 569mkdir /home/git/repositories 570# move all DIDs into the new folder; these will vary for you! 571mv /home/git/did:plc:wshs7t2adsemcrrd4snkeqli /home/git/repositories 572``` 573 574In the environment (e.g. `/home/git/.knot.env`), update `KNOT_REPO_SCAN_PATH` 575to the new directory: 576 577``` 578KNOT_REPO_SCAN_PATH=/home/git/repositories 579``` 580 581Similarly, update your `sshd` `AuthorizedKeysCommand` to use the updated 582repository path: 583 584``` 585sudo tee /etc/ssh/sshd_config.d/authorized_keys_command.conf <<EOF 586Match User git 587 AuthorizedKeysCommand /usr/local/bin/knot keys -o authorized-keys -git-dir /home/git/repositories 588 AuthorizedKeysCommandUser nobody 589EOF 590``` 591 592Make sure to restart your SSH server! 593 594#### MOTD (message of the day) 595 596To configure the MOTD used ("Welcome to this knot!" by default), edit the 597`/home/git/motd` file: 598 599``` 600printf "Hi from this knot!\n" > /home/git/motd 601``` 602 603Note that you should add a newline at the end if setting a non-empty message 604since the knot won't do this for you. 605 606## Troubleshooting 607 608If you run your own knot, you may run into some of these 609common issues. You can always join the 610[IRC](https://web.libera.chat/#tangled) or 611[Discord](https://chat.tangled.org/) if this section does 612not help. 613 614### Unable to push 615 616If you are unable to push to your knot or repository: 617 6181. First, ensure that you have added your SSH public key to 619 your account 6202. Check to see that your knot has synced the key by running 621 `knot keys` 6223. Check to see if git is supplying the correct private key 623 when pushing: `GIT_SSH_COMMAND="ssh -v" git push ...` 6244. Check to see if `sshd` on the knot is rejecting the push 625 for some reason: `journalctl -xeu ssh` (or `sshd`, 626 depending on your machine). These logs are unavailable if 627 using docker. 6285. Check to see if the knot itself is rejecting the push, 629 depending on your setup, the logs might be in one of the 630 following paths: 631 - `/tmp/knotguard.log` 632 - `/home/git/log` 633 - `/home/git/guard.log` 634 635# Spindles 636 637## Pipelines 638 639Spindle workflows allow you to write CI/CD pipelines in a 640simple format. They're located in the `.tangled/workflows` 641directory at the root of your repository, and are defined 642using YAML. 643 644The fields are: 645 646- [Trigger](#trigger): A **required** field that defines 647 when a workflow should be triggered. 648- [Engine](#engine): A **required** field that defines which 649 engine a workflow should run on. 650- [Clone options](#clone-options): An **optional** field 651 that defines how the repository should be cloned. 652- [Dependencies](#dependencies): An **optional** field that 653 allows you to list dependencies you may need. 654- [Environment](#environment): An **optional** field that 655 allows you to define environment variables. 656- [Steps](#steps): An **optional** field that allows you to 657 define what steps should run in the workflow. 658 659### Trigger 660 661The first thing to add to a workflow is the trigger, which 662defines when a workflow runs. This is defined using a `when` 663field, which takes in a list of conditions. Each condition 664has the following fields: 665 666- `event`: This is a **required** field that defines when 667 your workflow should run. It's a list that can take one or 668 more of the following values: 669 - `push`: The workflow should run every time a commit is 670 pushed to the repository. 671 - `pull_request`: The workflow should run every time a 672 pull request is made or updated. 673 - `manual`: The workflow can be triggered manually. 674- `branch`: Defines which branches the workflow should run 675 for. If used with the `push` event, commits to the 676 branch(es) listed here will trigger the workflow. If used 677 with the `pull_request` event, updates to pull requests 678 targeting the branch(es) listed here will trigger the 679 workflow. This field has no effect with the `manual` 680 event. Supports glob patterns using `*` and `**` (e.g., 681 `main`, `develop`, `release-*`). Either `branch` or `tag` 682 (or both) must be specified for `push` events. 683- `tag`: Defines which tags the workflow should run for. 684 Only used with the `push` event - when tags matching the 685 pattern(s) listed here are pushed, the workflow will 686 trigger. This field has no effect with `pull_request` or 687 `manual` events. Supports glob patterns using `*` and `**` 688 (e.g., `v*`, `v1.*`, `release-**`). Either `branch` or 689 `tag` (or both) must be specified for `push` events. 690 691For example, if you'd like to define a workflow that runs 692when commits are pushed to the `main` and `develop` 693branches, or when pull requests that target the `main` 694branch are updated, or manually, you can do so with: 695 696```yaml 697when: 698 - event: ["push", "manual"] 699 branch: ["main", "develop"] 700 - event: ["pull_request"] 701 branch: ["main"] 702``` 703 704You can also trigger workflows on tag pushes. For instance, 705to run a deployment workflow when tags matching `v*` are 706pushed: 707 708```yaml 709when: 710 - event: ["push"] 711 tag: ["v*"] 712``` 713 714You can even combine branch and tag patterns in a single 715constraint (the workflow triggers if either matches): 716 717```yaml 718when: 719 - event: ["push"] 720 branch: ["main", "release-*"] 721 tag: ["v*", "stable"] 722``` 723 724### Engine 725 726Next is the engine on which the workflow should run, defined 727using the **required** `engine` field. The currently 728supported engines are: 729 730- `nixery`: This uses an instance of 731 [Nixery](https://nixery.dev) to run steps, which allows 732 you to add [dependencies](#dependencies) from 733 Nixpkgs (https://github.com/NixOS/nixpkgs). You can 734 search for packages on https://search.nixos.org, and 735 there's a pretty good chance the package(s) you're looking 736 for will be there. 737 738Example: 739 740```yaml 741engine: "nixery" 742``` 743 744### Clone options 745 746When a workflow starts, the first step is to clone the 747repository. You can customize this behavior using the 748**optional** `clone` field. It has the following fields: 749 750- `skip`: Setting this to `true` will skip cloning the 751 repository. This can be useful if your workflow is doing 752 something that doesn't require anything from the 753 repository itself. This is `false` by default. 754- `depth`: This sets the number of commits, or the "clone 755 depth", to fetch from the repository. For example, if you 756 set this to 2, the last 2 commits will be fetched. By 757 default, the depth is set to 1, meaning only the most 758 recent commit will be fetched, which is the commit that 759 triggered the workflow. 760- `submodules`: If you use Git submodules 761 (https://git-scm.com/book/en/v2/Git-Tools-Submodules) 762 in your repository, setting this field to `true` will 763 recursively fetch all submodules. This is `false` by 764 default. 765 766The default settings are: 767 768```yaml 769clone: 770 skip: false 771 depth: 1 772 submodules: false 773``` 774 775### Dependencies 776 777Usually when you're running a workflow, you'll need 778additional dependencies. The `dependencies` field lets you 779define which dependencies to get, and from where. It's a 780key-value map, with the key being the registry to fetch 781dependencies from, and the value being the list of 782dependencies to fetch. 783 784The registry URL syntax can be found [on the nix 785manual](https://nix.dev/manual/nix/2.18/command-ref/new-cli/nix3-registry-add). 786 787Say you want to fetch Node.js and Go from `nixpkgs`, and a 788package called `my_pkg` you've made from your own registry 789at your repository at 790`https://tangled.org/@example.com/my_pkg`. You can define 791those dependencies like so: 792 793```yaml 794dependencies: 795 # nixpkgs 796 nixpkgs: 797 - nodejs 798 - go 799 # unstable 800 nixpkgs/nixpkgs-unstable: 801 - bun 802 # custom registry 803 git+https://tangled.org/@example.com/my_pkg: 804 - my_pkg 805``` 806 807Now these dependencies are available to use in your 808workflow! 809 810### Environment 811 812The `environment` field allows you define environment 813variables that will be available throughout the entire 814workflow. **Do not put secrets here, these environment 815variables are visible to anyone viewing the repository. You 816can add secrets for pipelines in your repository's 817settings.** 818 819Example: 820 821```yaml 822environment: 823 GOOS: "linux" 824 GOARCH: "arm64" 825 NODE_ENV: "production" 826 MY_ENV_VAR: "MY_ENV_VALUE" 827``` 828 829By default, the following environment variables are set: 830 831- `CI` - Always set to `true` to indicate a CI environment 832- `TANGLED_PIPELINE_ID` - The AT URI of the current pipeline 833- `TANGLED_REPO_KNOT` - The repository's knot hostname 834- `TANGLED_REPO_DID` - The DID of the repository owner 835- `TANGLED_REPO_NAME` - The name of the repository 836- `TANGLED_REPO_DEFAULT_BRANCH` - The default branch of the 837 repository 838- `TANGLED_REPO_URL` - The full URL to the repository 839 840These variables are only available when the pipeline is 841triggered by a push: 842 843- `TANGLED_REF` - The full git reference (e.g., 844 `refs/heads/main` or `refs/tags/v1.0.0`) 845- `TANGLED_REF_NAME` - The short name of the reference 846 (e.g., `main` or `v1.0.0`) 847- `TANGLED_REF_TYPE` - The type of reference, either 848 `branch` or `tag` 849- `TANGLED_SHA` - The commit SHA that triggered the pipeline 850- `TANGLED_COMMIT_SHA` - Alias for `TANGLED_SHA` 851 852These variables are only available when the pipeline is 853triggered by a pull request: 854 855- `TANGLED_PR_SOURCE_BRANCH` - The source branch of the pull 856 request 857- `TANGLED_PR_TARGET_BRANCH` - The target branch of the pull 858 request 859- `TANGLED_PR_SOURCE_SHA` - The commit SHA of the source 860 branch 861 862### Steps 863 864The `steps` field allows you to define what steps should run 865in the workflow. It's a list of step objects, each with the 866following fields: 867 868- `name`: This field allows you to give your step a name. 869 This name is visible in your workflow runs, and is used to 870 describe what the step is doing. 871- `command`: This field allows you to define a command to 872 run in that step. The step is run in a Bash shell, and the 873 logs from the command will be visible in the pipelines 874 page on the Tangled website. The 875 [dependencies](#dependencies) you added will be available 876 to use here. 877- `environment`: Similar to the global 878 [environment](#environment) config, this **optional** 879 field is a key-value map that allows you to set 880 environment variables for the step. **Do not put secrets 881 here, these environment variables are visible to anyone 882 viewing the repository. You can add secrets for pipelines 883 in your repository's settings.** 884 885Example: 886 887```yaml 888steps: 889 - name: "Build backend" 890 command: "go build" 891 environment: 892 GOOS: "darwin" 893 GOARCH: "arm64" 894 - name: "Build frontend" 895 command: "npm run build" 896 environment: 897 NODE_ENV: "production" 898``` 899 900### Complete workflow 901 902```yaml 903# .tangled/workflows/build.yml 904 905when: 906 - event: ["push", "manual"] 907 branch: ["main", "develop"] 908 - event: ["pull_request"] 909 branch: ["main"] 910 911engine: "nixery" 912 913# using the default values 914clone: 915 skip: false 916 depth: 1 917 submodules: false 918 919dependencies: 920 # nixpkgs 921 nixpkgs: 922 - nodejs 923 - go 924 # custom registry 925 git+https://tangled.org/@example.com/my_pkg: 926 - my_pkg 927 928environment: 929 GOOS: "linux" 930 GOARCH: "arm64" 931 NODE_ENV: "production" 932 MY_ENV_VAR: "MY_ENV_VALUE" 933 934steps: 935 - name: "Build backend" 936 command: "go build" 937 environment: 938 GOOS: "darwin" 939 GOARCH: "arm64" 940 - name: "Build frontend" 941 command: "npm run build" 942 environment: 943 NODE_ENV: "production" 944``` 945 946If you want another example of a workflow, you can look at 947the one [Tangled uses to build the 948project](https://tangled.org/@tangled.org/core/blob/master/.tangled/workflows/build.yml). 949 950## Self-hosting guide 951 952### Prerequisites 953 954- Go 955- Docker (the only supported backend currently) 956 957### Configuration 958 959Spindle is configured using environment variables. The following environment variables are available: 960 961- `SPINDLE_SERVER_LISTEN_ADDR`: The address the server listens on (default: `"0.0.0.0:6555"`). 962- `SPINDLE_SERVER_DB_PATH`: The path to the SQLite database file (default: `"spindle.db"`). 963- `SPINDLE_SERVER_HOSTNAME`: The hostname of the server (required). 964- `SPINDLE_SERVER_JETSTREAM_ENDPOINT`: The endpoint of the Jetstream server (default: `"wss://jetstream1.us-west.bsky.network/subscribe"`). 965- `SPINDLE_SERVER_DEV`: A boolean indicating whether the server is running in development mode (default: `false`). 966- `SPINDLE_SERVER_OWNER`: The DID of the owner (required). 967- `SPINDLE_SERVER_LOG_DIR`: The directory to store workflow logs (default: `"/var/log/spindle"`). 968- `SPINDLE_PIPELINES_NIXERY`: The Nixery URL (default: `"nixery.tangled.sh"`). 969- `SPINDLE_PIPELINES_WORKFLOW_TIMEOUT`: The default workflow timeout (default: `"5m"`). 970 971### Running spindle 972 9731. **Set the environment variables.** For example: 974 975 ```shell 976 export SPINDLE_SERVER_HOSTNAME="your-hostname" 977 export SPINDLE_SERVER_OWNER="your-did" 978 ``` 979 9802. **Build the Spindle binary.** 981 982 ```shell 983 cd core 984 go mod download 985 go build -o cmd/spindle/spindle cmd/spindle/main.go 986 ``` 987 9883. **Create the log directory.** 989 990 ```shell 991 sudo mkdir -p /var/log/spindle 992 sudo chown $USER:$USER -R /var/log/spindle 993 ``` 994 9954. **Run the Spindle binary.** 996 997 ```shell 998 ./cmd/spindle/spindle 999 ``` 1000 1001Spindle will now start, connect to the Jetstream server, and begin processing pipelines. 1002 1003## Architecture 1004 1005Spindle is a small CI runner service. Here's a high-level overview of how it operates: 1006 1007- Listens for [`sh.tangled.spindle.member`](/lexicons/spindle/member.json) and 1008 [`sh.tangled.repo`](/lexicons/repo.json) records on the Jetstream. 1009- When a new repo record comes through (typically when you add a spindle to a 1010 repo from the settings), spindle then resolves the underlying knot and 1011 subscribes to repo events (see: 1012 [`sh.tangled.pipeline`](/lexicons/pipeline.json)). 1013- The spindle engine then handles execution of the pipeline, with results and 1014 logs beamed on the spindle event stream over WebSocket 1015 1016### The engine 1017 1018At present, the only supported backend is Docker (and Podman, if Docker 1019compatibility is enabled, so that `/run/docker.sock` is created). spindle 1020executes each step in the pipeline in a fresh container, with state persisted 1021across steps within the `/tangled/workspace` directory. 1022 1023The base image for the container is constructed on the fly using 1024[Nixery](https://nixery.dev), which is handy for caching layers for frequently 1025used packages. 1026 1027The pipeline manifest is [specified here](https://docs.tangled.org/spindles.html#pipelines). 1028 1029## Secrets with openbao 1030 1031This document covers setting up spindle to use OpenBao for secrets 1032management via OpenBao Proxy instead of the default SQLite backend. 1033 1034### Overview 1035 1036Spindle now uses OpenBao Proxy for secrets management. The proxy handles 1037authentication automatically using AppRole credentials, while spindle 1038connects to the local proxy instead of directly to the OpenBao server. 1039 1040This approach provides better security, automatic token renewal, and 1041simplified application code. 1042 1043### Installation 1044 1045Install OpenBao from Nixpkgs: 1046 1047```bash 1048nix shell nixpkgs#openbao # for a local server 1049``` 1050 1051### Setup 1052 1053The setup process can is documented for both local development and production. 1054 1055#### Local development 1056 1057Start OpenBao in dev mode: 1058 1059```bash 1060bao server -dev -dev-root-token-id="root" -dev-listen-address=127.0.0.1:8201 1061``` 1062 1063This starts OpenBao on `http://localhost:8201` with a root token. 1064 1065Set up environment for bao CLI: 1066 1067```bash 1068export BAO_ADDR=http://localhost:8200 1069export BAO_TOKEN=root 1070``` 1071 1072#### Production 1073 1074You would typically use a systemd service with a 1075configuration file. Refer to 1076[@tangled.org/infra](https://tangled.org/@tangled.org/infra) 1077for how this can be achieved using Nix. 1078 1079Then, initialize the bao server: 1080 1081```bash 1082bao operator init -key-shares=1 -key-threshold=1 1083``` 1084 1085This will print out an unseal key and a root key. Save them 1086somewhere (like a password manager). Then unseal the vault 1087to begin setting it up: 1088 1089```bash 1090bao operator unseal <unseal_key> 1091``` 1092 1093All steps below remain the same across both dev and 1094production setups. 1095 1096#### Configure openbao server 1097 1098Create the spindle KV mount: 1099 1100```bash 1101bao secrets enable -path=spindle -version=2 kv 1102``` 1103 1104Set up AppRole authentication and policy: 1105 1106Create a policy file `spindle-policy.hcl`: 1107 1108```hcl 1109# Full access to spindle KV v2 data 1110path "spindle/data/*" { 1111 capabilities = ["create", "read", "update", "delete"] 1112} 1113 1114# Access to metadata for listing and management 1115path "spindle/metadata/*" { 1116 capabilities = ["list", "read", "delete", "update"] 1117} 1118 1119# Allow listing at root level 1120path "spindle/" { 1121 capabilities = ["list"] 1122} 1123 1124# Required for connection testing and health checks 1125path "auth/token/lookup-self" { 1126 capabilities = ["read"] 1127} 1128``` 1129 1130Apply the policy and create an AppRole: 1131 1132```bash 1133bao policy write spindle-policy spindle-policy.hcl 1134bao auth enable approle 1135bao write auth/approle/role/spindle \ 1136 token_policies="spindle-policy" \ 1137 token_ttl=1h \ 1138 token_max_ttl=4h \ 1139 bind_secret_id=true \ 1140 secret_id_ttl=0 \ 1141 secret_id_num_uses=0 1142``` 1143 1144Get the credentials: 1145 1146```bash 1147# Get role ID (static) 1148ROLE_ID=$(bao read -field=role_id auth/approle/role/spindle/role-id) 1149 1150# Generate secret ID 1151SECRET_ID=$(bao write -f -field=secret_id auth/approle/role/spindle/secret-id) 1152 1153echo "Role ID: $ROLE_ID" 1154echo "Secret ID: $SECRET_ID" 1155``` 1156 1157#### Create proxy configuration 1158 1159Create the credential files: 1160 1161```bash 1162# Create directory for OpenBao files 1163mkdir -p /tmp/openbao 1164 1165# Save credentials 1166echo "$ROLE_ID" > /tmp/openbao/role-id 1167echo "$SECRET_ID" > /tmp/openbao/secret-id 1168chmod 600 /tmp/openbao/role-id /tmp/openbao/secret-id 1169``` 1170 1171Create a proxy configuration file `/tmp/openbao/proxy.hcl`: 1172 1173```hcl 1174# OpenBao server connection 1175vault { 1176 address = "http://localhost:8200" 1177} 1178 1179# Auto-Auth using AppRole 1180auto_auth { 1181 method "approle" { 1182 mount_path = "auth/approle" 1183 config = { 1184 role_id_file_path = "/tmp/openbao/role-id" 1185 secret_id_file_path = "/tmp/openbao/secret-id" 1186 } 1187 } 1188 1189 # Optional: write token to file for debugging 1190 sink "file" { 1191 config = { 1192 path = "/tmp/openbao/token" 1193 mode = 0640 1194 } 1195 } 1196} 1197 1198# Proxy listener for spindle 1199listener "tcp" { 1200 address = "127.0.0.1:8201" 1201 tls_disable = true 1202} 1203 1204# Enable API proxy with auto-auth token 1205api_proxy { 1206 use_auto_auth_token = true 1207} 1208 1209# Enable response caching 1210cache { 1211 use_auto_auth_token = true 1212} 1213 1214# Logging 1215log_level = "info" 1216``` 1217 1218#### Start the proxy 1219 1220Start OpenBao Proxy: 1221 1222```bash 1223bao proxy -config=/tmp/openbao/proxy.hcl 1224``` 1225 1226The proxy will authenticate with OpenBao and start listening on 1227`127.0.0.1:8201`. 1228 1229#### Configure spindle 1230 1231Set these environment variables for spindle: 1232 1233```bash 1234export SPINDLE_SERVER_SECRETS_PROVIDER=openbao 1235export SPINDLE_SERVER_SECRETS_OPENBAO_PROXY_ADDR=http://127.0.0.1:8201 1236export SPINDLE_SERVER_SECRETS_OPENBAO_MOUNT=spindle 1237``` 1238 1239On startup, spindle will now connect to the local proxy, 1240which handles all authentication automatically. 1241 1242### Production setup for proxy 1243 1244For production, you'll want to run the proxy as a service: 1245 1246Place your production configuration in 1247`/etc/openbao/proxy.hcl` with proper TLS settings for the 1248vault connection. 1249 1250### Verifying setup 1251 1252Test the proxy directly: 1253 1254```bash 1255# Check proxy health 1256curl -H "X-Vault-Request: true" http://127.0.0.1:8201/v1/sys/health 1257 1258# Test token lookup through proxy 1259curl -H "X-Vault-Request: true" http://127.0.0.1:8201/v1/auth/token/lookup-self 1260``` 1261 1262Test OpenBao operations through the server: 1263 1264```bash 1265# List all secrets 1266bao kv list spindle/ 1267 1268# Add a test secret via the spindle API, then check it exists 1269bao kv list spindle/repos/ 1270 1271# Get a specific secret 1272bao kv get spindle/repos/your_repo_path/SECRET_NAME 1273``` 1274 1275### How it works 1276 1277- Spindle connects to OpenBao Proxy on localhost (typically 1278 port 8200 or 8201) 1279- The proxy authenticates with OpenBao using AppRole 1280 credentials 1281- All spindle requests go through the proxy, which injects 1282 authentication tokens 1283- Secrets are stored at 1284 `spindle/repos/{sanitized_repo_path}/{secret_key}` 1285- Repository paths like `did:plc:alice/myrepo` become 1286 `did_plc_alice_myrepo` 1287- The proxy handles all token renewal automatically 1288- Spindle no longer manages tokens or authentication 1289 directly 1290 1291### Troubleshooting 1292 1293**Connection refused**: Check that the OpenBao Proxy is 1294running and listening on the configured address. 1295 1296**403 errors**: Verify the AppRole credentials are correct 1297and the policy has the necessary permissions. 1298 1299**404 route errors**: The spindle KV mount probably doesn't 1300exist—run the mount creation step again. 1301 1302**Proxy authentication failures**: Check the proxy logs and 1303verify the role-id and secret-id files are readable and 1304contain valid credentials. 1305 1306**Secret not found after writing**: This can indicate policy 1307permission issues. Verify the policy includes both 1308`spindle/data/*` and `spindle/metadata/*` paths with 1309appropriate capabilities. 1310 1311Check proxy logs: 1312 1313```bash 1314# If running as systemd service 1315journalctl -u openbao-proxy -f 1316 1317# If running directly, check the console output 1318``` 1319 1320Test AppRole authentication manually: 1321 1322```bash 1323bao write auth/approle/login \ 1324 role_id="$(cat /tmp/openbao/role-id)" \ 1325 secret_id="$(cat /tmp/openbao/secret-id)" 1326``` 1327 1328# Webhooks 1329 1330Webhooks allow you to receive HTTP POST notifications when events occur in your repositories. This enables you to integrate Tangled with external services, trigger CI/CD pipelines, send notifications, or automate workflows. 1331 1332## Overview 1333 1334Webhooks send HTTP POST requests to URLs you configure whenever specific events happen. Currently, Tangled supports push events, with more event types coming soon. 1335 1336## Configuring webhooks 1337 1338To set up a webhook for your repository: 1339 13401. Navigate to your repository 13412. Go to **Settings → Hooks** 13423. Click **new webhook** 13434. Configure your webhook: 1344 - **Payload URL**: The endpoint that will receive the webhook POST requests 1345 - **Secret**: An optional secret key for verifying webhook authenticity (leave blank to send unsigned webhooks) 1346 - **Events**: Select which events trigger the webhook (currently only push events) 1347 - **Active**: Toggle whether the webhook is enabled 1348 1349## Webhook payload 1350 1351### Push 1352 1353When a push event occurs, Tangled sends a POST request with a JSON payload of the format: 1354 1355```json 1356{ 1357 "after": "7b320e5cbee2734071e4310c1d9ae401d8f6cab5", 1358 "before": "c04ddf64eddc90e4e2a9846ba3b43e67a0e2865e", 1359 "pusher": { 1360 "did": "did:plc:hwevmowznbiukdf6uk5dwrrq" 1361 }, 1362 "ref": "refs/heads/main", 1363 "repository": { 1364 "clone_url": "https://tangled.org/did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo", 1365 "created_at": "2025-09-15T08:57:23Z", 1366 "description": "an example repository", 1367 "fork": false, 1368 "full_name": "did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo", 1369 "html_url": "https://tangled.org/did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo", 1370 "name": "some-repo", 1371 "open_issues_count": 5, 1372 "owner": { 1373 "did": "did:plc:hwevmowznbiukdf6uk5dwrrq" 1374 }, 1375 "ssh_url": "ssh://git@tangled.org/did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo", 1376 "stars_count": 1, 1377 "updated_at": "2025-09-15T08:57:23Z" 1378 } 1379} 1380``` 1381 1382## HTTP headers 1383 1384Each webhook request includes the following headers: 1385 1386- `Content-Type: application/json` 1387- `User-Agent: Tangled-Hook/<short-sha>` — User agent with short SHA of the commit 1388- `X-Tangled-Event: push` — The event type 1389- `X-Tangled-Hook-ID: <webhook-id>` — The webhook ID 1390- `X-Tangled-Delivery: <uuid>` — Unique delivery ID 1391- `X-Tangled-Signature-256: sha256=<hmac>` — HMAC-SHA256 signature (if secret configured) 1392 1393## Verifying webhook signatures 1394 1395If you configured a secret, you should verify the webhook signature to ensure requests are authentic. For example, in Go: 1396 1397```go 1398package main 1399 1400import ( 1401 "crypto/hmac" 1402 "crypto/sha256" 1403 "encoding/hex" 1404 "io" 1405 "net/http" 1406 "strings" 1407) 1408 1409func verifySignature(payload []byte, signatureHeader, secret string) bool { 1410 // Remove 'sha256=' prefix from signature header 1411 signature := strings.TrimPrefix(signatureHeader, "sha256=") 1412 1413 // Compute expected signature 1414 mac := hmac.New(sha256.New, []byte(secret)) 1415 mac.Write(payload) 1416 expected := hex.EncodeToString(mac.Sum(nil)) 1417 1418 // Use constant-time comparison to prevent timing attacks 1419 return hmac.Equal([]byte(signature), []byte(expected)) 1420} 1421 1422func webhookHandler(w http.ResponseWriter, r *http.Request) { 1423 // Read the request body 1424 payload, err := io.ReadAll(r.Body) 1425 if err != nil { 1426 http.Error(w, "Bad request", http.StatusBadRequest) 1427 return 1428 } 1429 1430 // Get signature from header 1431 signatureHeader := r.Header.Get("X-Tangled-Signature-256") 1432 1433 // Verify signature 1434 if signatureHeader != "" && verifySignature(payload, signatureHeader, yourSecret) { 1435 // Webhook is authentic, process it 1436 processWebhook(payload) 1437 w.WriteHeader(http.StatusOK) 1438 } else { 1439 http.Error(w, "Invalid signature", http.StatusUnauthorized) 1440 } 1441} 1442``` 1443 1444## Delivery retries 1445 1446Webhooks are automatically retried on failure: 1447 1448- **3 total attempts** (1 initial + 2 retries) 1449- **Exponential backoff** starting at 1 second, max 10 seconds 1450- **Retried on**: 1451 - Network errors 1452 - HTTP 5xx server errors 1453- **Not retried on**: 1454 - HTTP 4xx client errors (bad request, unauthorized, etc.) 1455 1456### Timeouts 1457 1458Webhook requests timeout after 30 seconds. If your endpoint needs more time: 1459 14601. Respond with 200 OK immediately 14612. Process the webhook asynchronously in the background 1462 1463## Example integrations 1464 1465### Discord notifications 1466 1467```javascript 1468app.post("/webhook", (req, res) => { 1469 const payload = req.body; 1470 1471 fetch("https://discord.com/api/webhooks/...", { 1472 method: "POST", 1473 headers: { "Content-Type": "application/json" }, 1474 body: JSON.stringify({ 1475 content: `New push to ${payload.repository.full_name}`, 1476 embeds: [ 1477 { 1478 title: `${payload.pusher.did} pushed to ${payload.ref}`, 1479 url: payload.repository.html_url, 1480 color: 0x00ff00, 1481 }, 1482 ], 1483 }), 1484 }); 1485 1486 res.status(200).send("OK"); 1487}); 1488``` 1489 1490# Migrating knots and spindles 1491 1492Sometimes, non-backwards compatible changes are made to the 1493knot/spindle XRPC APIs. If you host a knot or a spindle, you 1494will need to follow this guide to upgrade. Typically, this 1495only requires you to deploy the newest version. 1496 1497This document is laid out in reverse-chronological order. 1498Newer migration guides are listed first, and older guides 1499are further down the page. 1500 1501## Upgrading to v1.13.0-alpha 1502 1503Starting with v1.13.0-alpha, every repository on a knot is 1504assigned a DID. This makes repositories stable across 1505renames and transfers. 1506 1507When you upgrade your knot to this version, the server will 1508automatically mint DIDs for all existing repositories on 1509startup. This is a one-time process and you may see 1510additional log output during the first boot as DIDs are 1511assigned. 1512 1513- Upgrade to the latest tag (v1.13.0 or above) 1514- Head to the [knot dashboard](https://tangled.org/settings/knots) and 1515 hit the "retry" button to verify your knot 1516 1517## Upgrading from v1.8.x 1518 1519After v1.8.2, the HTTP API for knots and spindles has been 1520deprecated and replaced with XRPC. Repositories on outdated 1521knots will not be viewable from the appview. Upgrading is 1522straightforward however. 1523 1524For knots: 1525 1526- Upgrade to the latest tag (v1.9.0 or above) 1527- Head to the [knot dashboard](https://tangled.org/settings/knots) and 1528 hit the "retry" button to verify your knot 1529 1530For spindles: 1531 1532- Upgrade to the latest tag (v1.9.0 or above) 1533- Head to the [spindle 1534 dashboard](https://tangled.org/settings/spindles) and hit the 1535 "retry" button to verify your spindle 1536 1537## Upgrading from v1.7.x 1538 1539After v1.7.0, knot secrets have been deprecated. You no 1540longer need a secret from the appview to run a knot. All 1541authorized commands to knots are managed via [Inter-Service 1542Authentication](https://atproto.com/specs/xrpc#inter-service-authentication-jwt). 1543Knots will be read-only until upgraded. 1544 1545Upgrading is quite easy, in essence: 1546 1547- `KNOT_SERVER_SECRET` is no more, you can remove this 1548 environment variable entirely 1549- `KNOT_SERVER_OWNER` is now required on boot, set this to 1550 your DID. You can find your DID in the 1551 [settings](https://tangled.org/settings) page. 1552- Restart your knot once you have replaced the environment 1553 variable 1554- Head to the [knot dashboard](https://tangled.org/settings/knots) and 1555 hit the "retry" button to verify your knot. This simply 1556 writes a `sh.tangled.knot` record to your PDS. 1557 1558If you use the nix module, simply bump the flake to the 1559latest revision, and change your config block like so: 1560 1561```diff 1562 services.tangled.knot = { 1563 enable = true; 1564 server = { 1565- secretFile = /path/to/secret; 1566+ owner = "did:plc:foo"; 1567 }; 1568 }; 1569``` 1570 1571# Hacking on Tangled 1572 1573We highly recommend [installing 1574Nix](https://nixos.org/download/) (the package manager) 1575before working on the codebase. The Nix flake provides a lot 1576of helpers to get started and most importantly, builds and 1577dev shells are entirely deterministic. 1578 1579To set up your dev environment: 1580 1581```bash 1582nix develop 1583``` 1584 1585Non-Nix users can look at the `devShell` attribute in the 1586`flake.nix` file to determine necessary dependencies. 1587 1588## Running the appview 1589 1590The appview requires Redis and OAuth JWKs. Start these 1591first, before launching the appview itself. 1592 1593```bash 1594# OAuth JWKs should already be set up by the Nix devshell: 1595echo $TANGLED_OAUTH_CLIENT_SECRET 1596z42ty4RT1ovnTopY8B8ekz9NuziF2CuMkZ7rbRFpAR9jBqMc 1597 1598echo $TANGLED_OAUTH_CLIENT_KID 15991761667908 1600 1601# if not, you can set it up yourself: 1602goat key generate -t P-256 1603Key Type: P-256 / secp256r1 / ES256 private key 1604Secret Key (Multibase Syntax): save this securely (eg, add to password manager) 1605 z42tuPDKRfM2mz2Kv953ARen2jmrPA8S9LX9tRq4RVcUMwwL 1606Public Key (DID Key Syntax): share or publish this (eg, in DID document) 1607 did:key:zDnaeUBxtG6Xuv3ATJE4GaWeyXM3jyamJsZw3bSPpxx4bNXDR 1608 1609# the secret key from above 1610export TANGLED_OAUTH_CLIENT_SECRET="z42tuP..." 1611 1612# Run Redis in a new shell to store OAuth sessions 1613redis-server 1614``` 1615 1616The Nix flake exposes a few `app` attributes (run `nix 1617flake show` to see a full list of what the flake provides), 1618one of the apps runs the appview with the `air` 1619live-reloader: 1620 1621```bash 1622TANGLED_DEV=true nix run .#watch-appview 1623 1624# TANGLED_DB_PATH might be of interest to point to 1625# different sqlite DBs 1626 1627# in a separate shell, you can live-reload tailwind 1628nix run .#watch-tailwind 1629``` 1630 1631## Running knots and spindles 1632 1633An end-to-end knot setup requires setting up a machine with 1634`sshd`, `AuthorizedKeysCommand`, and a Git user, which is 1635quite cumbersome. So the Nix flake provides a 1636`nixosConfiguration` to do so. 1637 1638<details> 1639 <summary><strong>macOS users will have to set up a Nix Builder first</strong></summary> 1640 1641In order to build Tangled's dev VM on macOS, you will 1642first need to set up a Linux Nix builder. The recommended 1643way to do so is to run a [`darwin.linux-builder` 1644VM](https://nixos.org/manual/nixpkgs/unstable/#sec-darwin-builder) 1645and to register it in `nix.conf` as a builder for Linux 1646with the same architecture as your Mac (`linux-aarch64` if 1647you are using Apple Silicon). 1648 1649> IMPORTANT: You must build `darwin.linux-builder` somewhere other than inside 1650> the Tangled repo so that it doesn't conflict with the other VM. For example, 1651> you can do 1652> 1653> ```shell 1654> cd $(mktemp -d buildervm.XXXXX) && nix run nixpkgs#darwin.linux-builder 1655> ``` 1656> 1657> to store the builder VM in a temporary dir. 1658> 1659> You should read and follow [all the other instructions][darwin builder vm] to 1660> avoid subtle problems. 1661 1662Alternatively, you can use any other method to set up a 1663Linux machine with Nix installed that you can `sudo ssh` 1664into (in other words, root user on your Mac has to be able 1665to ssh into the Linux machine without entering a password) 1666and that has the same architecture as your Mac. See 1667[remote builder 1668instructions](https://nix.dev/manual/nix/2.28/advanced-topics/distributed-builds.html#requirements) 1669for how to register such a builder in `nix.conf`. 1670 1671> WARNING: If you'd like to use 1672> [`nixos-lima`](https://github.com/nixos-lima/nixos-lima) or 1673> [Orbstack](https://orbstack.dev/), note that setting them up so that `sudo 1674ssh` works can be tricky. It seems to be [possible with 1675> Orbstack](https://github.com/orgs/orbstack/discussions/1669). 1676 1677</details> 1678 1679To begin, grab your DID from http://localhost:3000/settings. 1680Then, set `TANGLED_VM_KNOT_OWNER` and 1681`TANGLED_VM_SPINDLE_OWNER` to your DID. You can now start a 1682lightweight NixOS VM like so: 1683 1684```bash 1685nix run --impure .#vm 1686 1687# type `poweroff` at the shell to exit the VM 1688``` 1689 1690This starts a knot on port 6444, a spindle on port 6555 1691with `ssh` exposed on port 2222. 1692 1693Once the services are running, head to 1694http://localhost:3000/settings/knots and hit "Verify". It should 1695verify the ownership of the services instantly if everything 1696went smoothly. 1697 1698You can push repositories to this VM with this ssh config 1699block on your main machine: 1700 1701```bash 1702Host nixos-shell 1703 Hostname localhost 1704 Port 2222 1705 User git 1706 IdentityFile ~/.ssh/my_tangled_key 1707``` 1708 1709Set up a remote called `local-dev` on a git repo: 1710 1711```bash 1712git remote add local-dev git@nixos-shell:user/repo 1713git push local-dev main 1714``` 1715 1716The above VM should already be running a spindle on 1717`localhost:6555`. Head to http://localhost:3000/settings/spindles and 1718hit "Verify". You can then configure each repository to use 1719this spindle and run CI jobs. 1720 1721Of interest when debugging spindles: 1722 1723``` 1724# Service logs from journald: 1725journalctl -xeu spindle 1726 1727# CI job logs from disk: 1728ls /var/log/spindle 1729 1730# Debugging spindle database: 1731sqlite3 /var/lib/spindle/spindle.db 1732 1733# litecli has a nicer REPL interface: 1734litecli /var/lib/spindle/spindle.db 1735``` 1736 1737If for any reason you wish to disable either one of the 1738services in the VM, modify [nix/vm.nix](/nix/vm.nix) and set 1739`services.tangled.spindle.enable` (or 1740`services.tangled.knot.enable`) to `false`. 1741 1742# Contribution guide 1743 1744## Commit guidelines 1745 1746We follow a commit style similar to the Go project. Please keep commits: 1747 1748- **atomic**: each commit should represent one logical change 1749- **descriptive**: the commit message should clearly describe what the 1750 change does and why it's needed 1751 1752### Message format 1753 1754``` 1755<service/top-level directory>/<affected package/directory>: <short summary of change> 1756 1757Optional longer description can go here, if necessary. Explain what the 1758change does and why, especially if not obvious. Reference relevant 1759issues or PRs when applicable. These can be links for now since we don't 1760auto-link issues/PRs yet. 1761``` 1762 1763Here are some examples: 1764 1765``` 1766appview/state: fix token expiry check in middleware 1767 1768The previous check did not account for clock drift, leading to premature 1769token invalidation. 1770``` 1771 1772``` 1773knotserver/git/service: improve error checking in upload-pack 1774``` 1775 1776### General notes 1777 1778- PRs get merged "as-is" (fast-forward)—like applying a patch-series 1779 using `git am`. At present, there is no squashing—so please author 1780 your commits as they would appear on `master`, following the above 1781 guidelines. 1782- If there is a lot of nesting, for example "appview: 1783 pages/templates/repo/fragments: ...", these can be truncated down to 1784 just "appview: repo/fragments: ...". If the change affects a lot of 1785 subdirectories, you may abbreviate to just the top-level names, e.g. 1786 "appview: ..." or "knotserver: ...". 1787- Keep commits lowercased with no trailing period. 1788- Use the imperative mood in the summary line (e.g., "fix bug" not 1789 "fixed bug" or "fixes bug"). 1790- Try to keep the summary line under 72 characters, but we aren't too 1791 fussed about this. 1792- Follow the same formatting for PR titles if filled manually. 1793- Don't include unrelated changes in the same commit. 1794- Avoid noisy commit messages like "wip" or "final fix"—rewrite history 1795 before submitting if necessary. 1796 1797## Code formatting 1798 1799We use a variety of tools to format our code, and multiplex them with 1800[`treefmt`](https://treefmt.com). All you need to do to format your changes 1801is run `nix run .#fmt` (or just `treefmt` if you're in the devshell). 1802 1803## Proposals for bigger changes 1804 1805Small fixes like typos, minor bugs, or trivial refactors can be 1806submitted directly as PRs. 1807 1808For larger changes—especially those introducing new features, significant 1809refactoring, or altering system behavior—please open a proposal first. This 1810helps us evaluate the scope, design, and potential impact before implementation. 1811 1812Create a new issue titled: 1813 1814``` 1815proposal: <affected scope>: <summary of change> 1816``` 1817 1818In the description, explain: 1819 1820- What the change is 1821- Why it's needed 1822- How you plan to implement it (roughly) 1823- Any open questions or tradeoffs 1824 1825We'll use the issue thread to discuss and refine the idea before moving 1826forward. 1827 1828## Developer Certificate of Origin (DCO) 1829 1830We require all contributors to certify that they have the right to 1831submit the code they're contributing. To do this, we follow the 1832[Developer Certificate of Origin 1833(DCO)](https://developercertificate.org/). 1834 1835By signing your commits, you're stating that the contribution is your 1836own work, or that you have the right to submit it under the project's 1837license. This helps us keep things clean and legally sound. 1838 1839To sign your commit, just add the `-s` flag when committing: 1840 1841```sh 1842git commit -s -m "your commit message" 1843``` 1844 1845This appends a line like: 1846 1847``` 1848Signed-off-by: Your Name <your.email@example.com> 1849``` 1850 1851We won't merge commits if they aren't signed off. If you forget, you can 1852amend the last commit like this: 1853 1854```sh 1855git commit --amend -s 1856``` 1857 1858If you're submitting a PR with multiple commits, make sure each one is 1859signed. 1860 1861For [jj](https://jj-vcs.github.io/jj/latest/) users, you can run the following command 1862to make it sign off commits in the tangled repo: 1863 1864```shell 1865# Safety check, should say "No matching config key..." 1866jj config list templates.commit_trailers 1867# The command below may need to be adjusted if the command above returned something. 1868jj config set --repo templates.commit_trailers "format_signed_off_by_trailer(self)" 1869``` 1870 1871Refer to the [jujutsu 1872documentation](https://jj-vcs.github.io/jj/latest/config/#commit-trailers) 1873for more information. 1874 1875# Troubleshooting guide 1876 1877## Login issues 1878 1879Owing to the distributed nature of OAuth on AT Protocol, you 1880may run into issues with logging in. If you run a 1881self-hosted PDS: 1882 1883- You may need to ensure that your PDS is timesynced using 1884 NTP: 1885 - Enable the `ntpd` service 1886 - Run `ntpd -qg` to synchronize your clock 1887- You may need to increase the default request timeout: 1888 `NODE_OPTIONS="--network-family-autoselection-attempt-timeout=500"` 1889 1890## Empty punchcard 1891 1892For Tangled to register commits that you make across the 1893network, you need to setup one of following: 1894 1895- The committer email should be a verified email associated 1896 to your account. You can add and verify emails on the 1897 settings page. 1898- Or, the committer email should be set to your account's 1899 DID: `git config user.email "did:plc:foobar"`. You can find 1900 your account's DID on the settings page 1901 1902## Commit is not marked as verified 1903 1904Presently, Tangled only supports SSH commit signatures. 1905 1906To sign commits using an SSH key with git: 1907 1908``` 1909git config --global gpg.format ssh 1910git config --global user.signingkey ~/.ssh/tangled-key 1911``` 1912 1913To sign commits using an SSH key with jj, add this to your 1914config: 1915 1916``` 1917[signing] 1918behavior = "own" 1919backend = "ssh" 1920key = "~/.ssh/tangled-key" 1921``` 1922 1923## Self-hosted knot issues 1924 1925If you need help troubleshooting a self-hosted knot, check 1926out the [knot troubleshooting 1927guide](/knot-self-hosting-guide.html#troubleshooting).