How do Farcaster Hubs work? Hubble Architecture

This article assumes you know what a Farcaster Hub is and have some general intuition about software programming.

This is a practical overview, consult the official documentation and offical reference as well as source code if you want to dig deeper.

Starting Hubble, how do Hubs synchronize with the Farcaster network?

Up on starting Hubble synchronizing your Hub to hold the messages sent on Farcaster works the following:

1. Hubble recieves gossip messages sent from other Hubs on the Farcaster network and adds these to the local set of messages.
2. Hubble will load a snapshot and use it as starting point for Messages sent over the Farcaster protocol. (configurable on/off) - Warpcast Snapshot is daily 09:00 PT
3. Hubble asks a peer (another Hub) for messages it missed between Gossip messages recieved and the Snapshot.

What are Messages? A Message is the core data format sent across the peer to peer network Farcaster. It contains actions like adding a post (cast), following someone (add link), reactions (like a post) and more.

Though Hubs have onchain Events as well which include Storage, Signer and FID Registry. Full Reference of the contracts here

How do you know that your Hub is synchronized?

Your hub will have a peer and they both check the number of messages each has if you’re within 99% of the other hub you can consider yourself synced or 99.5% … whatever metric you want to use.

The Snapshot sync also gives you progress, though for a running hub number of messages compared to peers is the only way to check. If you’re running multiple hubs you can reference between them or use Hubs from API providers and other public Hubs to check outside of your Hubs peer too.

Firstly know that your Hub is ready right away, to verify messages for Frames or so for example, and as soon is it’s live it can also post to the network. It’s just reading all messages (past) that takes time to synchronize.

Why is this the case?

The Hub network guarantees eventual consistency, that means eventually the network will have the same state (same set of messages), though people constantly post new ones, so it takes time to synchronize. Therefore different hubs have different number of messages and exchange missing ones. This is by design.

Running Hubble when does Data get deleted?

Data deletion from Farcaster hubs is often referred to as “protocol pruning” or “pruning hub data” which in the end just means that not all the data on hubs will be on hubs forever.

Okay so when and what will be deleted?

You need to know that there are Storage Limits.

These specify how many of your data points are kept on Hubs, these limits exist for

• Casts (Posts, Replies)
• Reactions (e.g. Likes)
• Links (Follows)

You can check how much storage to have left in your client or on caststorage.com.

If your storage runs out or you go over the limit there is a grace period, but after that old Messages (think Data: casts, likes, …) are pruned (deleted) until you are within your limit again.

Why? Because this prevents spamming Hubs and just overloading the network, this way every message you post you pay for storage or have it pruned eventually. This also means that your first ever cast may not be available in clients anymore though.

To buy storage one interacts with Smart Contracts on Optimism via a transaction that is nothing other than you saying “this is my Farcaster ID (FID) and I want to buy storage”. Since Hubs synchronize the onchain state to their state, they will know if you have enough storage or not.