Better video mixing with MMixer's "sync mode"

A good way to make your mixed scenes to perform better is to use MMixer's sync mode (available with MPlatform SDK). In this mode MMixer prepares frames from the sources synchronously to make a nicer picture of the mixed video.

What's the difference? 

When the sync mode is disabled, frames arrive to the MMixer object from the sources independently, each one with its own frame rate. If a frame does not arrive at the right time, it is replaced with a duplicate. This makes MMixer rather stable, but the mixed picture may not look that good as the input sources are not mixed in sync. The mixed audio can also contain some artefacts.

When the sync mode is enabled, MMixer does as much as it can to prepare frames from sources to get the best mixed picture:

  • It sets the FPS of the sources, so there's a stable frame flow on its inputs.
  • It buffers the prepared frames in sync packets ready to be mixed. 
  • It prepares the frames in the same thread where it does the mixing, which ensures that mixing starts only when frames from all sources are received.

Advantages of the sync mode

Here's what you gain with sync mode:

  • The same source can be added to MMixer several times, i.e. you can add one DeckLink source as two separate items. This makes sense in some use cases such as when you never want to change the audio source, but still mix or switch the video.
  • A lot less chance of frame drops and audio artefacts.
  • Its possible to achieve frame accurate mixing for both files and live sources.
  • The mixed stream will be in sync with the input.

Recommendations for using the sync mode

Please take the following into account when considering sync mode:

  • Always set the specific output resolution for MMixer. Do not use "Auto". It will help MMixer to prep the input frames correctly.
  • We don't recommend using sync mode with unstable sources (such as DirectShow cameras or network streams), because unstable frame rate of the input source will lead to unstable frame rate of the mixed stream. We recommend to mix sources with stable frame rates to achieve good results.
  • In Sync mode MMixer sets the output frame rate for the sources that are connected to it. If you don't want this to happen, you should add the source stream via mp:// protocol with "sync_mode.direct_mp_links = false".

Options and registry settings

  • sync_mode - turns the sync mode on. Enabled by default. 
  • sync_mode.direct_mp_links - once enabled, forces MMixer to use a source object directly (to get the control over it) instead of mp:// links (which is not controllable). Gives better performance and less frame drops in a resulting stream, but controls the original object, which can make it unusable by other sink objects. Enabled by default.  
  • sync_mode.external_exclusive -  once enabled, forces MMixer to control the source objects exclusively to get the best conditions for mixing. Makes source objects unaccessible by other sink objects. Disabled by default.
  • sync_mode.fixed_buffers - the number of buffered source frame packets. The source frames in these packets are in sync and ready for mixing, which eases the life for MMixer greatly. While having a fixed delay, it allows to overcome short CPU overloads without any effect on the output. And, in case of long CPU overload, it allows MMixer to drop the video of buffered pack, but keep and mix the audio which makes such drops much less visible on the output. Use this property instead of mixer.output_buffers in sync mode.