How do Cast Embeds (links, videos, images, ...) work?

You do not need to know how Hubs work or the Farcaster protocol work to understand how Embeds work. Start reading and in case you have advanced knowledge, jump directly to topics of interest. Though generally it is useful to know what Farcaster Messages are.

What is an Embed?

When talking about Embeds one refers to anything that is embedded with a post. Usually via some kind of link (url). When you paste dtech.vision into a text message, you expect a preview of that url (dtech.vision) to show up in the text editor. That preview is what we refer to as embed from a user standpoint, though from a developer standpoint the embed is the url (dtech.vision in our case) that is pasted and should be embedded with the message.

How do Embeds work on Farcaster?

On Farcaster Embeds are part of the Farcaster Messages. To add an image, video, mini app or whatever you want to have as embed on your post (cast) one adds an url to it. Yes this literally means to cast an image we as developers take the image from the user, and turn it into a link. Then we take this link and add it to their past as embed. The user things they have uploaded and posted to Farcaster, but in reality they have uploaded to wherever the developer chose and attached that url to their post.

The same goes for videos, mini apps and all kinds of urls like articles. Think of embeds more like pointers to media then true uploads to the protocol. Since the Farcaster protocol doesn’t store your image (or video) you need another solution for that. As developer that means you are tasked with providing that infrastructure and making sure that when someone wants to view the post and the url resolves and is available. Yes if anyone - no matter from your app or another client - wants to watch a video that means your video infrastructure (our your provider) gets traffic, which you most likely pay for.

A look at Farcaster protocol data - the message

Here is some representation of raw strings of what a message looks like that includes an embed so one gets an understanding of the underlying data type.

How do Farcaster clients render embeds?

When rendering casts a Farcaster client sees the embed in the message or API response they watch. Most Farcaster clients use some form of processed data instead of raw protocol messages. The data format they see likely is something akin to the following:

{
  text: "This is a sample illustration of a cast. look at the great video and mini app I am posting here"
  embeds: ["https://url-to.video/1", "https://miniapp.dtech.vision"]
  author: {
    // information about the author
  }
  signer: {}
  // more details
}

From this representation of a cast it’s very clear that you’d access the url’s in the embed array. But how do you know what kind they are? Either your API provider has done that work for you already and hydrates with that information. They’d give you the MIME type e.g. image/jpg. Or you’ll have to run a GET request against the URL, look at the header and multiplex over the returned content MIME-type to select the proper render in your user interface (e.g. your image or video preview tool or your mini app preview renderer).

If your API provider gives you the content type already, then you make this part of your cast rendering logic and directly pick the right visual representation. Though be aware, that in case there are indexing delays on your provider you may have to fallback for undefined content types and add the logic yourself.

Tipps and Tricks to upload videos to Farcaster (handling video embeds)

Be careful with total cost of ownership when supporting video upload and how to save money!

For efficient usage of videos the Base App and Farcaster App expect your video to be a stream and not a raw .mp4 file or similar!

This is important since your users will highly likely not upload a stream, but an mp4 or mov… You will handle the transcoding from their filetype to .mp3u8 streams. That requires paying a provider like Cloudflare Stream to do the transcoding, video serving and storage for you (they have a nice upload file, get stream link back API) or handling it yourself.

When doing it yourself or when saving money on CDN providers you’d host a VPS for little money with high CPU capacity to run handbrake/ffmpeg. These are powerful video transcoding programs that are super optimized and will do the conversion to a streaming format like mp3u8 for you. Your server will be the endpoint the video is uploaded to, converted and then posted to the CDN/streaming provider unless you also host these yourself. Here you could then host the stream on S3 or anywhere. What you are saving is a lot of money that is spent on transcoding (usually paid by the minute).

The total cost of ownership for video content is often cited as reason for why smaller Farcaster clients won’t provide video uploading support. The developers would have to suffer the cost of every single view the video gets. Their financial incentive is to never have a video shown. Though yours is to get as many views as possible. So there would need to be a way to pay for the cost of the client. When a client has enough revenue to offset the video cost, they usually do. For eaxmple the Farcaster app allows you to upload videos and social media companies like Facebook do so to as they make money with Ads and use that to pay for video hosting, transcoding etc.

Legalities of hosting videos

When a user uploads a video to your website there maybe a risk that you become liable for their upload. Of course every jurisdiction is differnt and we are no lawyers, though you’d want to research that. Most likely you need to filter videos when uploaded and screen them for illegal content. That’s a service that large providers likely provide to you (at a cost of course). LLMs becoming cheap may help you here. Where you’d develop a pipeline of getting the upload, LLM scanning it with a cheap model that supports video files to have the contents checked against your rules and laws, then if fine transcoding to a stream and pushing that stream to your hosting solution. The final stream url is what is then used in the Message that composes the cast.

So your user would wait for that process to finish and see an indicator of the video uploading & processing. You may do that in the background and notifiy the user in case of failure and on success so they can use the app without being blocked from your video processing pipeline.