How does one use Farcaster Protocol data from Hubs?
Farcaster Hubs are the source of truth of Farcaster. It’s the data that is valid.
The following assumes you are familiar with what a Hub is. You do not need to know how they work.
First What Data is on Hubs?
Hubs contain everything that is sent over the Farcaster protocol (so called Events which include Messages).
What are Messages? A Message is the core data format sent across the peer to peer network Farcaster. It contains actions like adding a post (cast), following someone (add link), reactions (like a post) and more.
Though Hubs have onchain Events as well which include Storage, Signer and FID Registry. Full Reference of the contracts here
For more learn how Hubs work.
Now how do we get that data and work with it?
The Event Stream
With the Event Stream we use gRPC to subscribe to it and recieve each new Event arriving at this specific Hub.
Farcaster Indexers use this stream to write each new event into a datastore like a database.
What we effectivly get here is a firehose. This is the lowest latency way to process events since we get them as soon as they go through the protocol.
Reading the Messages from the Event we can check if @samuellhuber is mentioned in a cast or the text contains “dTech” and reply from our bot that says “hey cool that you are talking about us”. Or of course you can do other things like send push notifications or trade a ticker if there is specific people posting about it. Like “if xyz mentions $dTech then buy 100 $dTech” in an automated fashion because you get it directly as soon as it is published.
But that is all, you use the datatype of the message and process it.
The API
A Hub also stores messages in a local database. We can use a HTTP and gRPC API to leverage that storage to query this data.
On top of just querying that on Hub storage we can also let our Hub do verification of Messages to check if it is valid.
Leading to three functions of the Hub API
- Check status and get diagnostics of our Hub
- Query data on our Hub (messages)
- Use the Hub for verification
A full API reference is available on the Farcaster documentation:
The Database
To store data locally each Hub has it’s own local database (implemented with RocksDB here), but while this should not be relied up on as a permanent highly available datasource we can work directly against it.
It’s just a database, so use SQL and write queries to get data directly if you do not want to depend on the APIs