alice.pds.demo.boltless.dev
This repository has no description
1---
2atroot: true
3template:
4slug: bobbin
5title: Introducing Bobbin
6subtitle: A diskless, API-only AppView for Tangled
7image: https://assets.tangled.network/blog/bobbin.png
8date: 2026-07-13
9authors:
10 - name: Lewis
11 email: lewis@tangled.org
12 handle: oyster.cafe
13---
14
15Hello! It is I, Lewis, with my first Tangled post.
16At Tangled my job involves a lot of Research & Development, by
17which I mean fucking around and finding out. Let me tell you about one
18such finding out, which turned into this new Tangled API.
19
20Bobbin is a read-only, API-only
21[AppView](https://atproto.com/guides/glossary#app-view)
22for Tangled. It serves the read side of the `sh.tangled.*`
23[lexicons](https://atproto.com/specs/lexicon)
24over [XRPC](https://atproto.com/specs/xrpc), so a
25client can render git repositories, issues, pulls,
26comments, stars, & follows, without having to ping N
27[PDSes](https://atproto.com/guides/glossary#pds-personal-data-server)
28for their data. It has no permanent storage. Everything it knows is only
29ever in RAM and gets backfilled from upstream every restart.
30
31> I, along with many others, find the term "AppView" to be too
32> abstract or perhaps even unfortunately-named. In atproto, datasets
33> are distributed across Personal Data Servers (PDSes) that users each
34> own, so an AppView is a single API through which one can query
35> a dataset, in contrast to all downstream clients each maintaining a
36> connection with every PDS. An AppView is one of the many programs in
37> use to reduce the strain on PDSes, so that a PDS is relatively cheap and
38> resource-efficient to run.
39
40Let me list some problems that I find Tangled's AppView to have, such
41that the journey of discovery that led me to Bobbin makes sense:
42
431. Tangled's AppView has no API; it doubles as a web client via SSR.
442. It is brittle in the sense that it's not able to backfill
45[atproto](https://atproto.com) data easily.
463. Its architecture from the ground up is inherently built for the kind
47of web program that assumes *it* is the source of truth of data.
484. Because of all of the above, and the fact that there *is* some
49private data such as email -> user [DID](https://atproto.com/specs/did)
50ties, there ought to only be one AppView for Tangled. This means that
51latency to Tangled is vaguely a function of distance from the server
52in Stockholm.
535. It is hungry for RAM during a RAM crisis.
54
55Flipping to the inverse of all of these items would yield me my ideal
56AppView. An API-only, easily-backfillable, easily-distributable,
57easy-on-RAM program.
58
59Why "API-only"? Once it's easy to retrieve Tangled data via generic
60XRPC API, more programs can be built on top of it; programs such
61as alternative frontends, third-party clients, one's own targeted
62dashboards, in my case it was to enable a good CLI (though I suppose
63that just comes under alternative frontends). The existing Tangled
64AppView being its own web client means there's nothing extendable to
65build on, unless one is a scraping wizard; one has to figure out their
66own way to go grab the Tangled dataset from the world of PDSes in the
67wild.
68
69While I was mulling how to go about creating such a program for fun, I
70have also been having a crisis about where to store state, after I've
71woken up in cold sweats wondering if Tranquil PDS' homegrown embedded
72DB could fail in new mystical ways. How could I avoid this next program
73having to write any durable, fallible state? Besides, AppViews are
74downstream of PDSes, which means the more we try and hold onto state in
75an AppView, the more chance there is that it'll diverge from reality.
76All it takes to have an AppView be wildly incorrect is missing a single
77record, so it is of upmost importance to have an AppView be able to
78reconcile what it thinks it knows with what's actually true.
79
80Being the simpleton that I am, I think that there is no better way
81to make a program whose only job is to cache and serve records than
82to make it never touch disk in the first place. I would want it to
83simply backfill all of its data from scratch on every restart. In that
84case, reconciliation with reality is just a restart away and happens
85automatically! Was that feasible? What if such a backfill took a whole
86day every time? That would sure make deployments slow, would it anyhow
87be worth it?
88
89My first tests of this concept gave promising results, I
90threw together a little server that could in fact backfill
91all of Tangled's atproto dataset in around 15 minutes, using
92[Hydrant](https://tangled.org/did:plc:6v3ul2ptnqctyxwkz5ti4amn) as its
93aggregator. I decided that an acceptable backfill time would be under
945 minutes or less, so that it would be feasible to switch out running
95instances on a blue/green deploy within a reasonable warming timeframe.
96
97> Hydrant is an aggregator for arbitrary atproto datasets that works
98> by tailing the live [firehose](https://atproto.com/specs/event-stream),
99> backfilling every [repo](https://atproto.com/guides/glossary#data-repo)
100> on the network by pulling their
101> [CARs](https://atproto.com/specs/repository#car-file-serialization),
102> and serving it back over a websocket. All Bobbin has to do is open a
103> connection, ask for cursor 0, and (re)build its index off the resulting
104> barrage.
105>
106> Massive gratitude for
107> [Dawn](https://tangled.org/did:plc:dfl62fgb7wtjj3fcbb72naae) for having
108> made Hydrant! Then for hearing my struggles and going and optimizing it
109> also for fun!
110
111Once I reached backfill in 5 minutes, I moved the goalpost for fun, that
112the acceptable backfill time should now be 90 seconds or less. This is
113around the number we are approaching today, depending on how warm the
114upstream Hydrant instance is; ranging from 30 seconds in best-case
115scenario to 20 minutes in a disaster where Hydrant is also pulling
116records from PDSes from scratch.
117
118> Yes, you too can spin these up & order the entirety of Tangled in 20
119> minutes or less, or you get a refund. (Of €0.)
120
121There's a little conundrum that I encountered: what to have the
122program serve while the backfill is still warming? Do I bother
123having the program serve requests while aggregate/linked data is
124possibly incorrect? I think not. So do I have a program serving
125nothing in the backfill time that is possibly 20 minutes at
126worst? I probably could have done that, but instead we connect to
127[Slingshot](https://tangled.org/did:plc:c7mc2fn47ihdihul4vjwsuy3/tree/main/slingshot)
128for point-lookups at least and immediately get accurate
129data in the meantime, even if the program itself hasn't technically seen
130that data yet.
131
132> Slingshot is a wonderful edge cache for atproto records
133> & identities. Bobbin asks it for single record and identity
134> lookups, so that a freshly-born Bobbin can answer individual
135> queries accurately before its own index has warmed up. Slingshot
136> is part of the [microcosm](https://www.microcosm.blue/) project,
137> a bundle of community-run atproto infrastructure. Thanks
138> [Fig](https://tangled.org/did:plc:hdhoaan3xa3jiuq4fg4mefid) for making
139> it!
140
141The program that was turning into Bobbin was able to hold all of
142Tangled's atproto dataset in around 200MB of RAM, which was good, but
143some compression led to 100MB. There is likely more I could compress
144that is yet to be done. "Wow, such a small amount of data for the entire
145social layer of a whole platform!" you remark, but on the contrary
146remember that this is the *smallest* the dataset will likely ever be
147from today onwards. Space-saving is of the essence.
148
149Bobbin uses Slingshot as a record/identity resolver in real time for
150anything it doesn't already hold in RAM or hasn't necessarily received
151from Hydrant. I love Fig's work, don't get me wrong, but my lean on
152Slingshot instead of just writing the pings myself was a cop-out to save
153time; if there's one major architectural change yet to come to Bobbin,
154it's inlining that away such that Bobbin never has to ping Slingshot and
155can figure out faster ways of getting the same data.
156
157One would be tempted to throw in an embedded instance of Hydrant to
158their AppView program itself, and have the serving of a particular
159atproto dataset be just a thin wrapper over the Hydrant data; but
160I want it to be easy to spin up possibly hundreds of instances of my
161program. Having hundreds of Hydrants all pinging PDSes all the time
162would be quite wasteful. One Hydrant instance can serve hundreds of
163Bobbins without a sweat.
164
165Since we don't have a Hydrant/relay equivalent for Knot-related data,
166Bobbin proxies Knot-related requests straight through to a given knot. I
167think a knotstream aggregator is in order, perhaps even just a plugin to
168Hydrant that has it serving both. I think Bobbin and Knot Mirror should
169merge into one service, but that's just my opinion.
170
171With a "stateless" AppView, we don't have to worry about databases and
172their woes: migrations, production massaging when things are slightly
173off compared to the "true" dataset, et cetera. Any time we want to
174change/add functionality, all we have to do is simply restart
175Bobbin and the operation is done. Another benefit is that Bobbin lends
176itself to "cloud-native deployments" as the corpos say. Tangled's own
177flagship instance runs on Cloudflare Containers.
178
179All this being said, I'm not necessarily married to the idea that Bobbin
180never touches disk, I think that one cute improvement could be to simply
181dump RAM-stored-data contents as a snapshot once every hour and backfill
182the difference at startup; but for now I think there's headroom to
183continue down this path, considering Bobbin can healthily run with 200MB
184given to it at time of writing. Additionally, my original requirement of
185being easy on RAM is still fulfilled for now, but the moment it becomes
186too heavy is another point at which Bobbin will offload parts of itself
187to disk dynamically during runtime. How do I define "too heavy", you
188ask? I look inside my heart, which tells me that 1GB of RAM is wasteful,
18910GB is unacceptable. Bobbin also must serve users, whose each API
190request puts more pressure on the RAM beyond just storing the Tangled
191dataset.
192
193There you have it! Now there's this fun AppView with no disk storage,
194sub-90-second backfill, that serves API requests as a little engine
195that could. With it, anyone can build programs using the Tangled dataset
196more easily.
197
198There is an instance of Bobbin on [api.tangled.org](https://api.tangled.org),
199and hopefully you'd find it easy to host your own. Bobbin's source lives in
200[our monorepo](https://tangled.org/tangled.org/core/tree/master/bobbin).