Introducing detailed participant tracks state

The Daily call object is one of the most flexible ways to build a custom video chat app from the ground up. Using the call object gives you direct access to call primitives, including other participants’ audio and video tracks. You can then manipulate those media streams however you'd like to build an interface that’s entirely your own.

As of today, the call object also provides fine-grained information about participant tracks state. Developers not only can see whether a participant’s tracks are available or not, but also know why they are (or aren’t!).

The Daily participants object now returns a tracks property. tracks is an object that contains audio, video, screenAudio, and screenVideo objects. Each of these contains its own details about the state of the track subscription, in addition to the MediaStreamTrack if available. It’s now possible to know call specifics like whether a track is loading, a participant has muted themselves, or if the local participant isn’t subscribed to a track.

tracks: {
  audio: {
    subscribed: boolean;
    state:
      | 'blocked'
      | 'off'
      | 'sendable'
      | 'loading'
      | 'interrupted'
      | 'playable';
    blocked?: {
      byDeviceMissing?: boolean;
      byPermissions?: boolean;
    };
    off?: {
      byUser?: boolean;
      byBandwidth?: boolean;
    };
track?: MediaStreamTrack;
  };
  video: { /* same */ };
  screenAudio: { /* same */ };
  screenVideo: { /* same */ };
}

Having all this fine-grained information makes it possible to build expressive video app interfaces that change as a participant subscribes or unsubscribes from tracks.

As a quick example, let’s look at how to use participant tracks state to display detailed messages if a track is unavailable in our React demo app.

Prerequisites and release versions

• daily-js-0.9.997
• If you’d like to reference a full working app, you might find it helpful to fork our React demo [0].
• Note: while these tracks-level state details are also available when using Daily’s prebuilt UI, we think this information comes in most handy when building custom call experiences with the Daily call object, so we’re focusing on that for this blog post.

Access and update track state

In callState.js, we store a list of callItems that details every participant on the call and their audio and video track state.

const initialCallState = {
  callItems: {
    local: {
      videoTrackState: null,
      audioTrackState: null,
    },
  },
};

The keys for the callItems object are local for the local participant, and the session_id for every other participant on the call. The videoTrackState and audioTrackState properties populate with the tracks.video or tracks.audio data from the Daily participant object where the corresponding participant’s session_id is the key.

Every time a participant joins, leaves, or any other participant-related event happens on a call, we extract the new tracks state from the Daily participants object.

Display custom track status messages depending on track state

If a video or audio track is available, our app sends it to the other participants on the call. When a video is unavailable for any reason, we want to let other participants know why.

In getTrackUnavailableMessage(kind, trackState), we display a different notification depending on the trackState. If the track is "playable", we don’t show a message at all.

function getTrackUnavailableMessage(kind, trackState) {
  if (!trackState) return;
  switch (trackState.state) {
    case 'blocked':
      if (trackState.blocked.byPermissions) {
        return `${kind} permission denied`;
      } else if (trackState.blocked.byDeviceMissing) {
        return `${kind} device missing`;
      }
    case 'off':
      if (trackState.off.byUser) {
        return `${kind} muted`;
      } else if (trackState.off.byBandwidth) {
        return `${kind} muted to save bandwidth`;
      }
    case 'sendable':
      return `${kind} not subscribed`;
    case 'loading':
      return `${kind} loading...`;
    case 'interrupted':
      return `${kind} interrupted`;
    case 'playable':
      return null;
  }
}

For details on how we’re rendering those notifications, take a look at the full demo code.

But, remember, our demo app is just that: a demo. There are loads of ways you could customize your app based on track state, and we hope you do!

The state of tracks state

Dynamically updating video chat interfaces depending on what a participant is doing on a call, like sharing or muting their video or audio, can make video apps stand out. We hope detailed tracks state helps you add new kinds of customizations to your video apps.

If you’re building something with Daily that you’re excited about, please let us know! And, stay tuned to the blog for another example of detailed tracks state in action.

Other resources

• Tracks details in the docs
• Keep reading about track subscriptions
• If you’re looking for other ways to keep building on the React demo app, add a text overlay with a participant’s username

[0] For a comprehensive walkthrough of the demo app, check out our original call object React blog post.