Adding video backgrounds to a Daily call in a browser app

Introduction

Have your users ever wished their video call background was as exciting as those people working from a literal beach? With Daily's WebRTC API, developers can let participants introduce a little more excitement (or privacy) in their lives by hiding their room in favor of another video background image.

We've talked a bit about Daily's virtual backgrounds feature before, which allows developers to add custom backgrounds to call participants' video tracks. In our last post on the topic, we went through how to extend an Electron application running Daily Prebuilt with a custom virtual backgrounds feature.

Now, let's take a look at how to build the same feature set in a vanilla JavaScript browser-based web app.

What we're building

We'll be building a small web app that embeds a Daily Prebuilt call frame onto a page. On the side of the call frame, we'll show a set of custom video backgrounds for the call participant to pick from:

We'll use Daily's updateInputSettings() API method to update and reset the video background. For this demo we'll be using Daily Prebuilt, but this method is supported in our custom Client SDKs as well.

A little… background

Daily Prebuilt offers some sample background images already built-in, but in this project, we will be showing you how you can add your own custom WebRTC virtual backgrounds. Under the hood, Daily uses mediapipe for the real-time semantic segmentation that in turn uses a TensorFlow WASM backend. We’ve abstracted the complexities away so that you can get chromakey effects without the need for a green screen just by using our JavaScript libraries. If you really want to get into the weeds, check out our FOSDEM 2022 talk about implementing blurred backgrounds for WebRTC by Ravi, one of our engineers.

In this post, we show you how to quickly implement virtual backgrounds for your own WebRTC based app without having to study machine learning or TensorFlow video processing.

Getting started

First, create a free Daily account and a room.

Next, clone the demo:

git clone git@github.com:daily-demos/virtual-backgrounds-prebuilt.git
cd virtual-backgrounds-prebuilt

Replace the placeholder room URL to that of the room you just created in index.js.

Install relevant dependencies and run the app:

npm i && npm run start

Go to localhost:8080 in your web browser (replace :8080 with a different port if another one is referenced in the terminal on startup).

Embedding Daily Prebuilt

We'll start by creating a call container in our index.html file:

      <div id="call"></div>

Then, in index.js, we'll create an initCall() function which is run when the DOM has loaded:

let callFrame; // This will be our Daily Prebuilt call frame

window.addEventListener("DOMContentLoaded", () => {
  initCall();
});

function initCall() {
  // Get call frame container
  const container = document.getElementById("call");

  // Create call frame
  callFrame = DailyIframe.createFrame(container, {
    showLeaveButton: true,
    iframeStyle: {
      position: "fixed",
      width: "500px",
      height: "550px",
      border: "1px solid #e6eaef",
      borderRadius: "6px",
      boxShadow: `0 1px 2px rgba(0, 0, 0, 0.02), 0 2px 4px rgba(0, 0, 0, 0.02),
      0 4px 8px rgba(0, 0, 0, 0.02), 0 8px 16px rgba(0, 0, 0, 0.02),
      0 16px 32px rgba(0, 0, 0, 0.02)`,
    },
  })
    .on("nonfatal-error", (e) => {
      console.warn("nonfatal error:", e);
    })
    .on("started-camera", () => {
      const backgrounds = document.getElementById("backgrounds");
      backgrounds.classList.remove("hidden");
    })
    .on("left-meeting", () => {
      initCall();
    });

  // TODO: Replace the following URL with your own room URL.
  callFrame.join({ url: "https://<your-domain>.daily.co/<room-name>" });
}

Above, we create a Daily Prebuilt iframe with our API createFrame() method, supplying some custom styling via the iframeStyle property. Check out our guide about styling Daily Prebuilt if you'd like to learn more about customizing the visuals of your WebRTC video call.

We then add a few handlers to relevant Daily API events:

• The "nonfatal-error" handler simply prints the given error message to the console. In a production application, you'd likely want to expose this to the user in the most relevant way instead.
• The "started-camera" handler shows our video background selection div when the participant's camera is ready to accept virtual background modifications.
• The "left-meeting" handler simply re-initializes the call for the purpose of our demo.

Defining our video background data

All of our custom virtual background images live in the "backgrounds" directory. Then, in index.js, we define our background data:

const backgroundPrefix = "backgrounds";
const backgroundsData = [
  {
    path: "cafe-table-with-coffee.jpg",
    alt: "A round coffee table with a cup of coffee",
  },
  {
    path: "northern-lights.jpg",
    alt: "Northern lights",
  },
  // ...Further video call backgrounds elements here…
];

As we can see, each background in our backgroundsData array contains the file name of the background image and descriptive alt text for each one.

Populating our video call background selection options

As we mentioned above, our custom background options will live in their own "backgrounds"div in index.html:

      <div id="backgrounds" class="hidden">
        <h1>Click on a background image to select</h1>
        <div id="bgImages">
          <button id="reset">Reset background</button>
        </div>
      </div>

We apply the "hidden" class by default, which will ensure that our user can't try to select a virtual background before the Daily call is appropriately loaded. We also add a reset button, which will be used to remove any currently applied background from the video call.

Next, in index.js, we'll add a call to a new function to our "DOMContentLoaded" event handler:

window.addEventListener("DOMContentLoaded", () => {
  initCall();
  // New call to initialize our virtual background options:
  initBackgrounds();
});

Let's take a look at what exactly our new function, initBackgrounds(), does:

function initBackgrounds() {
  const resetBtn = document.getElementById("reset");
  resetBtn.onclick = () => {
    resetBackground();
  };
  loadBackgrounds();
}

Above, we first get our "backgrounds" div, followed by configuring the reset button contained within (we'll go through the details of resetBackgrounds() shortly).

Finally, we call loadBackgrounds() to display our selection of custom call backgrounds. We've left some comments to guide us through the loading process inline:

async function loadBackgrounds() {
  const bgImages = document.getElementById("bgImages");

  // Loop through our data array
  for (let i = 0; i < backgroundsData.length; i++) {
    const data = backgroundsData[i];
    // Create a button element for each background
    const btn = document.createElement("button");
    btn.className = "bg";
    
    // Create an image element nested within each button
    const imgPath = `${window.location.protocol}//${window.location.host}/${backgroundPrefix}/${data.path}`;
    const img = document.createElement("img");
    img.src = imgPath;
    img.alt = data.alt;
    
    // Set the background on button click
    btn.onclick = () => {
      setBackground(imgPath);
    };

    btn.appendChild(img);
    bgImages.appendChild(btn);
  }
}

In the loading function above, we loop through each video background element in our previously defined data array. For each background, we create a new <button> element and an <img /> element to go within it. We then set each button to call our setBackground() when clicked.

Let's look at where the background-setting magic happens: the setBackground() function.

Setting and resetting a Daily video call background with updateInputSettings()

Setting a new virtual background for a Daily video call participant is done by calling the updateInputSettings() call object instance method. We do so in our setBackground() function as follows:

function setBackground(imgPath) {
  callFrame.updateInputSettings({
    video: {
      processor: {
        type: "background-image",
        config: {
          source: imgPath,
        },
      },
    },
  });
}

With the above call, our call participant's background can go from boring wall to Northern Lights:

Resetting works in much the same way. With a call to updateInputSettings() and the processor type set to "none", the video call background can be removed:

function resetBackground() {
  callFrame.updateInputSettings({
    video: {
      processor: {
        type: "none",
      },
    },
  });
}

And our wall is back:

Wrapping up

In this post, we learned how to add custom video call backgrounds to Daily Prebuilt in a vanilla JavaScript web app.

If you're curious to implement this kind of feature set in an Electron application, you're in luck! We have a tutorial just for that.

As always, please don't hesitate to reach out if you have any questions or feedback.