The Only Constant in Life is Mutability…
… that’s how the saying goes, right? 🤪
… that’s how the saying goes, right? 🤪
It is true though. People are constantly asking us how to change their data. Which is a bit of a wild ask considering IPFS is all about content addressed IMMUTABLE data 😅. Really though, it’s not so wild, because it’s something that us humans end up needing to do all the time. It’s also totally possible with immutable data, provided you’re willing to throw away the old version and accept a new (changed) version of your data…but then you have a new problem — how do you keep track of the latest version?
For me, mutability boils down to 3 big problems:
- Keeping track of what the latest version of the data is.
- Distributing the latest version. i.e. the fact that something changed, and also the changes themselves.
- Allowing multiple writers.
Keeping track of the latest version is actually pretty easy if there’s only one writer. Then the only issue is distributing the fact that things changed…which we make a little harder for ourselves by operating in a decentralized context. Luckily there’s a bunch of existing IPFS and libp2p tech to help us with that — libp2p helps with establishing connections between peers that wouldn’t otherwise be able to communicate directly with each other. Yeah I’m talking hole punching and the like. It also provides pubsub primitives that allow push based messaging within applications, backed by peers in the network. The IPFS Amino DHT can help with persisting and making data globally available over a longer period. There’s also fallbacks you can use — things like circuit relays or centralized rendezvous points to help with distribution in a more pull based approach. So honestly, we’re pretty well covered for distribution in the decentralized world, especially with just a single writer.
IPNS uses the DHT and pubsub to distribute updates, but it has one big problem — it doesn’t allow for multiple writers, unless you share your private key. Let me explain — IPNS uses a cryptographic key pair. The public key is used to access the “current” value. Updates to the “current” value are signed by the private key. So you know they’re legit — someone else cannot issue an update for your IPNS key, unless you share your private key with them.
I put multiple writers in the list because oftentimes you want multiple parties to be able to update something. Like, multiple people in a chat room, multiple writers to a bucket, multiple players in a game. Single player is just no fun!
So how do we enable multi-writer mutability? Well, UCAN (User Controlled Authorisation Networks) provides the perfect method of authorising another party to perform an action. With UCAN, you can (pun intended) delegate capabilities to another party by sending them a signed UCAN delegation. UCAN also supports “invocation” — the act of exercising a capability. In UCAN, when you invoke a capability you need to provide proof that you have the authority to do so — the signed delegation chain that can be verified by the recipient. What if…ok hear me out…what if…we could “publish” a mutation by invoking a UCAN? 🤯
TADA! Multi-writer!
Ok calm down. There’s another…slight…problem. In what order did stuff happen? Well, in single writer mode you just use a sequence number, or a timestamp. Anyone that receives two updates simultaneously just takes the one with the highest number as the most recent. For multi-writer, events can happen concurrently, and an absolute ordering needs to be established to determine the true current value. Sequence numbers don’t work, because, well, concurrency — two users might use the same sequence number. Same problem for timestamps, but not as bad. It’s still possible to create an update at the exact same time as someone else though. It also requires system clocks to be set correctly.
One more problem to throw into the mix with sequences/timestamps — if you know your ordering resolution is “most recent/biggest” and you want to make sure your change always takes priority over others, you use a big number or future date. There’s incentive to “cheat”! Oh noes! 😱
So, at Storacha we’ve been experimenting for a while now with a thing called Merkle Clocks as a log that can provide a partial ordering of events created concurrently by peers, without them needing to come to consensus by having direct connections to each other, or operate within a high bandwidth or low latency environment. You don’t need sequence numbers because each “tick” of the clock (a new event) points to the prior event, or multiple events if two or more happened concurrently. Any peer can determine the latest version of the data simply by receiving the latest event (or events). We call this the “head” of the clock.
The diagram above shows a simplified clock with a few example events. In this clock, the event ending in x7bq happened first. Then the two events ex5m and 6bim happened concurrently (we don’t know which happened first) and finally, the event btlq happened. There is an indisputable ordering of some of the events here, but there’s still resolution required for the ordering of the two concurrent events ex5m and 6bim.
Merkle Clocks provide a partially ordered log of events that actually happened and your application figures out what to do when multiple concurrent events occur at the same time. It could be “take the most recent”, or order by some property and apply sequentially. You might want to assign weights to nodes favouring longer (or shorter) divergent paths, or maybe your data allows merging them in any order. Either way, as long as your app defines the rules and applies them consistently your peers never need to talk to each other directly. Events can be created, proxied, discovered and applied by any peer in any order and, provided peers eventually receive all the events at the head of the clock they’ll all come to the same conclusion about what the current state of the data is. This…by the way…is CRDTs (Conflict-free replicated data types), and they are AWESOME!
If you want to read more about Merkle Clocks, check out the Merkle-CRDTs paper.
Ok! Now we can put this all together and use UCAN to provide access to add an event to the clock. UCAN invocations allow multiple parties to access the clock and they can be broadcasted or sent directly peer to peer. We also stood up a rendezvous clock for experimentations called w3clock.
A Merkle Clock can be used to effectively build any mutable data type. It could be a single hash (like IPNS) or something more complex like a bucket. We’ve built a bucket CRDT that uses all this tech in JS and Go and hope to release this in an application and in an SDK soon:
If you’re interested in learning more, check out this video.
So, there you have it! Mutability in a decentralized, content-addressed world might sound like a tricky puzzle, but with a little help from UCAN and Merkle Clocks, it’s totally doable — and dare we say, pretty cool. 🧩✨ These tools unlock the magic of multi-writer collaboration while keeping everything in sync, even when things get a little chaotic.
At Storacha, we’ve been cooking up some 🔥 solutions to make mutable data types a reality. Whether it’s a single hash, a bucket, or something wild we haven’t even thought of yet, the possibilities are endless. Keep an eye out for our Pail SDKs (links above 👆), and if you’re feeling curious, take a peek at the video for even more nerdy goodness.
Ready to dive in? Grab a Merkle Clock, flex those UCAN muscles, and let’s make some mutable magic together. 🚀
