alice.pds.demo.boltless.dev
This repository has no description
4.4 kB
75 lines
1Heavily inspired by [frontpage dev environment](https://github.com/frontpagefyi/frontpage/blob/10678df9c3f72cbd82f0856a9f99c74dd22326d8/apps/frontpage/local-infra/README.md).
2Tangled's setup is slightly more involved because services inside the network need to reach the PDS over its **public** hostname with **valid TLS** — federation paths (DID resolution, OAuth, etc.) round-trip through the same URLs an external client would use.
3
4For example, resolving `alice.pds.tngl.boltless.dev` yields an `#atproto_pds` service pointing at `https://pds.tngl.boltless.dev`. Knot and spindle running inside docker must hit that exact URL and trust its cert.
5
6To make that work:
7
8- Caddy's dev root CA is mounted into every container that talks to another service over HTTPS.
9- The Docker network uses an unrouted "public" subnet so the SSRF dialer doesn't reject container IPs as private.
10
11## What's inside:
12
13- [did-method-plc](https://github.com/did-method-plc/did-method-plc) (<https://plc.tngl.boltless.dev>)
14- atproto_pds (<https://pds.tngl.boltless.dev>)
15- jetstream (<https://jetstream.tngl.boltless.dev>)
16- knot (<https://knot.tngl.boltless.dev>)
17- spindle (<https://spindle.tngl.boltless.dev>)
18- knotmirror (<https://knotmirror.tngl.boltless.dev>)
19- appview (<https://tngl.boltless.dev>) (live reloading)
20- [ncps](https://github.com/kalbasit/ncps) nix binary cache (internal, `http://ncps:8501`)
21- pdsls (<https://pdsls.tngl.boltless.dev>)
22- caddy reverse proxy
23
24## Setup
25
261. Generate the dev CA from the repo root:
27 ```bash
28 mkdir -p localinfra/certs &&
29 openssl req -x509 -newkey rsa:2048 \
30 -keyout localinfra/certs/root.key \
31 -out localinfra/certs/root.crt \
32 -days 3650 -nodes \
33 -subj "/CN=Tangled Dev CA" \
34 -addext "basicConstraints=critical,CA:TRUE,pathlen:1" \
35 -addext "keyUsage=critical,keyCertSign,cRLSign" \
36 -addext "nameConstraints=critical,permitted;DNS:tngl.boltless.dev"
37 ```
382. Trust generated `localinfra/certs/root.crt` in your system's trust store.
39 - For example in MacOS, run
40 ```bash
41 sudo security add-trusted-cert -d -r trustRoot \
42 -k /Library/Keychains/System.keychain \
43 ./localinfra/certs/root.crt
44 ```
45 - Depending on your browser you may have to import the certificate into your browser profiles too as some have their own certs do not use your system ones
463. run `./localinfra/scripts/appview-static-files.sh`
474. Prepare the spindle microVM images:
48 ```bash
49 ./localinfra/scripts/prepare-spindle-images.sh
50 ```
51 This writes the image directory under `out/localinfra-spindle-images`.
525. `docker compose up`
536. AppView will be running on `127.0.0.1:3000` with two test users: `alice.pds.tngl.boltless.dev` and `bob.pds.tngl.boltless.dev`. Both with password `password`.
54`TANGLED_APPVIEW_HOST` must be a loopback IP with the mapped port (`127.0.0.1:3000`), not `localhost`: atproto's dev OAuth client requires a loopback IP for the redirect URI. If you remap the published appview port, update `TANGLED_APPVIEW_HOST` in `docker-compose.yml` to match.
55
56
57## Mill mode
58
59The default stack runs one standalone spindle. `docker-compose.mill.yml` is an overlay that splits that into the distributed arch: the primary spindle becomes the mill host (`role=mill`, it only places jobs) and a three-executor fleet (`role=executor`) runs the engines.
60
61```bash
62docker compose -f docker-compose.yml -f docker-compose.mill.yml --profile linux up
63```
64
65The executors differ on the two axes placement cares about, so you can watch candidates get filtered and ranked:
66
67| executor | labels | images | runs |
68|------------|---------------|-------------|-----------------------------|
69| executor-a | `linux, fast` | full | `image/alpine, image/nixos` |
70| executor-b | `linux, slow` | full | `image/alpine, image/nixos` |
71| executor-c | `linux, gpu` | alpine only | `image/alpine` |
72
73So `image: nixos` has two candidates, `image: alpine` has three, and `runs_on: [gpu]` pins to executor-c. The alpine-only image set is staged by `prepare-spindle-images.sh` (step 4) alongside the full one, no extra step.
74
75Each executor needs its own identity (one live session per token), so the `mill-tokens` service registers a token per executor in the mill db and drops it into the shared volume for the executor to read.