Stream with HLS while recording to a custom S3 bucket using Daily

As use cases and customer needs have become more varied over time, we at Daily have been focused on shipping lots of new features related to recording and streaming video calls. Recently, we added support for storing your call recordings in your own S3 bucket. Today we are launching support for multi-bitrate HLS when live streaming your calls.

We are excited about these two features because they give you even more control over how you stream, distribute, and store your video. By using these features together, you can stream your calls directly from Daily and store a recording of the call on your own infrastructure at the same time.

In this post, we are going to use HLS to stream a WebRTC-powered low-latency video call directly to your S3 storage. We'll then stream from your storage to a large live stream audience using AWS Cloudfront. If you haven't yet, you will need to follow our guide to set up storing recordings in your own S3 bucket before continuing with the rest of this tutorial.

Why HLS?

If you are looking for a refresher on some of the current web-video standards, we have a post on the different use cases for RTMP, HLS, and WebRTC written by our CEO/engineer, Kwindla Kramer. This post highlights some of the reasons why we wanted to support HLS in addition to WebRTC and RTMP.

In short, we want Daily’s API to give developers the most flexibility while also being easy to use. HLS support gives you more control over your data, lowers the cost of storing data, creates streams and recordings simultaneously, and reduces the number of suppliers necessary to achieve your vision.

HLS support helps us accomplish all of those things using flexible technology that supports multiple bitrates and reduces latency between your real-time video call and your larger audience.

The HLS stream outputs 4 bitrates with the following specifications:

The specifications allow video players to adjust to the bandwidth needs or constraints of each viewer in real time.

Enter the CDN

In order to use HLS, we still need to use a “middleman” technology so that the HLS stream has a place to go. CDNs (content delivery networks) help ensure a fast, efficient, and secure delivery of all types of content, including streaming media. Some examples of CDNs include Akamai, Cloudflare, CloudFront, and Fastly. In our use case, we want to have a URL that can act as the streaming endpoint for our live streams. With this endpoint, we can quickly deliver our content to a web location that anyone can access.

Throughout this post, we’ll be using AWS CloudFront as our CDN of choice. CloudFront offers low latency, high transfer speeds, and encrypted media support. However, you can use any CDN that you prefer.

Getting your CDN credentials

To connect Daily’s infrastructure with your CDN, we need to generate a private and public key to associate your Daily subdomain with your AWS account.

Our engineers have put together an open source GitHub repo with scripts to help set up an HLS endpoint hosted on CloudFront.

To use it, clone the repository and navigate to its root directory before following the instructions below:

git clone git@github.com:daily-co/daily-cloudfront-hls-example.git
cd daily-cloudfront-hls-example

To get set up, you’ll need to have Node installed and add these two dependencies:

npm install -g typescript aws-cdk

You will also need OpenSSL installed on your machine.

Once you have everything installed, you can run the following commands, replacing [daily_subdomain] with your Daily subdomain (e.g. the this_is_me part of this_is_me.daily.co):

openssl genrsa -out private_key-[daily_subdomain].pem 2048

openssl rsa -pubout -in private_key-[daily_subdomain].pem -out public_key-[daily_subdomain].pem

cdk bootstrap -c dailySubdomain=[daily_subdomain]

cdk deploy --context dailySubdomain=[daily_subdomain]

The output of the cdk deploy command will include the names of the S3 bucket and the IAM role configured for Daily, as well as the DNS name of the CloudFront distribution. You'll use these to configure your Daily domain and/or rooms for outputting HLS streams and (optionally) recordings, outlined in the next section.

Access and security levels

There are several ways to configure CloudFront with Daily, depending on your security requirements.

1. Option 1: Use signed URLs
   Signed URLs have authentication information as part of the URL string. This information limits permission levels and works for only a set amount of time. A signature on the URL can apply to multiple files based on an attached policy, so you can use the same signature for all files in the video.
2. Option 2: Use signed cookies
   For this option, you will need an API that is attached to an origin of the CloudFront distribution. Clients will be able to hit this endpoint and that will set the cookie, including the signature, and then access the URL.
3. Option 3: Just don’t sign the URLs
   Use a private S3 bucket and a CloudFront origin access identity to make secure requests to S3. This enables the public access to the video, but only via CloudFront.

This repo also includes an example Lambda@Edge function to add the additional security level for your content. This part is not required, but it is recommended for production use. If you are just testing and the endpoint is temporary, you may decide to skip the extra step for now, as we will do in this blog post.

Setting up Daily’s HLS Configuration

Now that we have CloudFront configured with Daily, we can set up our HLS configuration.

As a reminder, you will need to have added permission settings between Daily and AWS to both accounts.

To set up HLS, we want to send a JSON blob with:

• A unique name for the streaming endpoint (hls_cloudfront)
• The type of endpoint (HLS or RTMP)
• The S3 configuration

{
 "properties": {
   "streaming_endpoints": [
     {
       "name": "hls_cloudfront",
       "type": "hls",
       "hls_config": {
         "storage": {
           "bucket_name": "daily-hls-streams",
           "bucket_region": "us-west-2",
           "assume_role_arn": "arn:aws:iam::999999999999:role/DailyS3AccessRole",
           "path": "my-path/my-prefix"
         }
       }
     }
   ]
 }
}

As a curl request, it would look like this:

curl --request POST \                                        
    --url https://api.daily.co/v1/rooms/your-room \
    --header 'Accept: application/json' \
    --header 'Authorization: Bearer $DAILY_API_KEY \
    --header 'Content-Type: application/json' \
    --data '{
    "properties": {
       "streaming_endpoints": [{
           "name": "hls_cloudfront",
           "type": "hls",
           "hls_config": {
               "storage": {
                   "bucket_name": "daily-hls-streams",
                   "bucket_region": "us-west-2",
                   "assume_role_arn": "arn:aws:iam::999999999999:role/DailyS3AccessRole",
                   "path": "my-path/my-prefix"
               }
           }
       }]
   }
}'

It is possible to set up and stream to multiple endpoints by adding more configurations to the streaming_endpoints array. However, only one endpoint can be configured with HLS at this time. Multiple RTMP endpoints can be added. For this example, we will only be setting up one.

Let’s stream!

Now that everything has been configured, starting HLS in your app is just one API call. You can start streaming to your HLS endpoint in your app with Daily’s startLiveStreaming() method:

await callObject.startLiveStreaming({
  endpoints: [{"endpoint":"your_hls_name_here"}],
  width: 1280, 
  height: 720,
});

You can listen for the "live-streaming-started" event to confirm that streaming has started.

When your call is streaming, you can visit a URL structured like this to see and share your stream: https://[cloudfront_domain_name]/index.html?prefix=[S3 object key prefix]&mtgSessionId=[your meeting session ID]

The Playing Streams section of the daily-cloudfront-hls-example repo’s README outlines more ways to customize your stream and playback using players other than the default AWS URL.

If your goal of using HLS is recording, starting this stream will result in a stored recording in your AWS S3 Bucket as soon as live streaming your call has ended (as long as recording is set to true in your HLS configuration).

Conclusion

And there you have it: a new way to stream your calls with the Daily API and a new way to store and retrieve your recordings.

To learn more about HLS, read our guide: Streaming with HLS.

Speaking of live streaming and recording, have you checked out VCS? It’s our API for customizing the layout of your streams and recordings. In addition to our guide, we have a few great blog posts that can help you get started with VCS. All of these features allow you to do more fun things with your video call experiences.