1. Miscellaneous
  2. Capture Screen Contents with loopback audio (requires ToDesktop Builder `v0.22.3` or higher)

Miscellaneous

WARNING

If you wish to support apps built using versions of ToDesktop Builder before v0.22.3, then you will need to use our older docs on capturing screen contents. See here for the older docs.

Capture Screen Contents

Get the user's media access status

        import { systemPreferences } from '@todesktop/client-core';

const status = await systemPreferences.getMediaAccessStatus('screen');

      

This will return a Promise which will resolve to not-determined, granted, denied, restricted or unknown

On Windows and macOS 10.14 Mojave or lower the response will always begranted . macOS 10.15 Catalina or higher requires consent for screen access.

To get that consent just run the code below, it will automatically ask for permission

Get the user's screen sources

Say we wanted to get a user's screens and windows

        import { app } from '@todesktop/client-core';
const sources = await app.getCaptureSources({
  types: ['window', 'screen'],
  thumbnailSize: {
    height: 720,
    width: 1000
  }
});

      

This will return an array of sources

        [
    {
        appIcon: null,
        display_id: "",
        id: "window:60230:0",
        name: "Developer Tools - http://localhost:3000/",
        thumbnail: "data:image/png;base64,iV..."
    },
    ...
]

      

Getting a source id

You'll need a way of choosing which screen/window to capture. Unlike in a browser, your Desktop app will need to implement its own UI for this.

For example if you wanted to reproduce a browsers selection feature you could loop through the sources and listen to click events on each.

        <div id="desktop-capturer-sources"></div>

<button id="get-sources">Get Sources</button>

<style>
  #desktop-capturer-sources {
    display: flex;
    width: 60%;
    height: 400px;
    background-color: white;
    box-shadow: #c4c4c4 2px 4px 5px 2px;
    flex-wrap: wrap;
    overflow: auto;
    justify-content: center;
  }

  .desktop-source {
    min-width: 30%;
    padding: 1rem;
    margin: 0.25rem;
  }

  .desktop-source > img {
    width: 250px;
  }
</style>

<script>
  window.onload = () => {
    document
      .getElementById("get-sources")
      .addEventListener("click", () => getSources());
  };
  async function getSources() {
    const sources = await ...;
    const container = document.getElementById("desktop-capturer-sources");
    sources.forEach((source) => {
      const el = document.createElement("div");
      el.className = "desktop-source";
      const img = document.createElement("img");
      img.src = source.thumbnail;
      el.appendChild(img);
      // We'll create the getStream function in the next code snippet
      el.addEventListener("click", () => getStream(source));
      container.appendChild(el);
    });
  };
</script>

      

This will give you something like:

Getting Screens / Windows in Desktop App

Get the stream to a source

After choosing a source we'll use its id and navigator.mediaDevices.getUserMedia to get the stream.

        function getStream(videoSource) {
  await session.setDisplayMediaRequestHandler(null, (request, cb) => {
    // Using "loopback" audio to capture system audio.
    // See note below this code snippet if you want to capture system audio.
    cb({ video: videoSource, audio: "loopback" });
  });

  const stream = await navigator.mediaDevices
    .getDisplayMedia({
      video: true,
      audio: true,
    })

  return stream;
}

      
NOTE

By default "loopback" audio is not enabled in ToDesktop apps because it requires us to add a flag of MacLoopbackAudioForScreenShare to your app.

If this is something that you want then get in contact with us and we can enable it for you.

Play the stream

If you have a <video> in your HTML you can then use

        const video = document.querySelector('video');

// The `stream` is the one we got from `getStream`
video.srcObject = stream;

video.onloadedmetadata = (e) => video.play();

      

API Reference

getCaptureSources

Exposes the destopCapturer.getSources electron API: https://www.electronjs.org/docs/api/desktop-capturer

Accepts an options object:

  • types String - The types of desktop sources to be captured, eitherscreen or window.
  • thumbnailSize - { width: number, height: number } (optional) - Default is 150 x 150.
  • fetchWindowIcons Boolean (optional) - Default value is false

Returns a Promise which resolves into an array of DesktopCapturerSource objects each having:

  • id String - The id of a window or screen that can be used as a chromeMediaSourceId later
  • name String - The name of the screen (Entire Screen or Screen <index>) or the window's title
  • thumbnail String - A data Url of the image
  • display_id String - A unique identifier that will correspond to the id property
  • appIcon String - A data Url of the app Icon or null if type is screen

setDisplayMediaRequestHandler

Exposes the session.setDisplayMediaRequestHandler electron API: https://www.electronjs.org/docs/api/session

Accepts a handler function which will be called with the request object and a callback function.

  • request Object - The request object (contains frame, securityOrigin, videoRequested, audioRequested and userGesture properties)
  • callback Function - The callback function