← All posts
Ch. 05By Kostas Oraiopoulos3 min read

Getting the picture out

Native monitor outputs, multiview, a GPU-fed virtual camera and NDI in both directions, plus a first look at audio and why it needed a different approach.

Everything so far lived inside one window. That is fine for building a show and useless for running one. A production has to leave the machine: to monitors in the room, to a projector, to other software, to the network.

This chapter is about the exits.

Outputs straight from the GPU

The rule from the first week still applies: the picture is composed on the GPU and leaves from the GPU. Outputs never go through the interface.

Monitor outputs came first. Program, Preview or any scene can be sent to a window or full screen on any connected display, including the main one when you need it. These are native outputs drawn directly by the compositor, at full quality and full frame rate, independent of what the interface is doing.

Multiview

A director needs to see everything at once: what is on air, what is next, and what each source is doing right now.

Multiview shows several of these in one output. There can be more than one multiview, each with its own layout and its own selection of sources, so the director, the audio desk and the graphics operator can each watch what matters to them. From the live workspace you can send a source straight into a multiview slot without opening a settings screen.

Getting this right meant being strict about identity. A multiview tile has to show the exact layer or source it was assigned, even when the same video is used in several scenes at once. Small bugs here are confusing on a live show, so they were fixed at the root rather than patched.

A virtual camera

Much of today’s live production ends up in video calls, webinars and streaming software that expects a webcam.

So Muxivo Studio exposes its output as a virtual camera. Any application that can pick a webcam can pick Muxivo Studio instead, and receives the composed program.

The first version converted frames on the CPU, and it showed. The final version converts the picture into the camera format on the GPU, as part of the composition, so the virtual camera costs very little and does not steal time from the render loop.

NDI, in and out

On a local network, the common language for moving live video between machines and applications is NDI.

Muxivo Studio speaks it in both directions:

  • NDI inputs appear as ordinary sources. They can be switched to a different stream on the fly, paused and resumed, and activated only when needed, so idle streams do not use bandwidth.
  • NDI outputs send the program or other selections to the network, with the choice saved in the project.

Sources that disappear are marked clearly. If a camera or a network stream goes offline, the source and the tabs that use it show a warning, so the operator finds out before the audience does.

A first look at audio

Then I started on audio, and quickly learned something.

The first pass worked in the familiar way: each source has a level, some sources go to a bus, the bus goes out. It played sound. It also made obvious that this model would not stretch to what a modern show needs: listening to a source privately before it goes on air, ducking music under a voice, feeding a remote guest everything except their own voice, processing on the right path and not on everything at once.

Those are not features to add to a mixer. They describe a signal graph. Audio needed the same kind of thinking that went into the picture, and that deserves its own chapter.

Taking stock

By this point the application could compose a show, put graphics on it, change scenes cleanly, and get the result out to screens, software and the network.

It was also getting big. Big enough that adding the next feature without a clear map of the whole system would start to cost more than it gained. So before writing more code, I stopped to draw that map. That is the next chapter.