Multimedia Component Framework Overview

From MediaLooks Knowledge Base

Jump to: navigation, search

This is a basic overview of the MediaLooks Multimedia Components Architecture.

Overview

MultiGraph Objects or MG Objects are COM objects that support a number of common interfaces in order to communicate with each other. Each component contains within itself one or two (in case of file capture objects) DirectShow Graphs. The MultiGraph Sink/Source DirectShow filters are used to transfer data between the graphs (i.e. between the objects).

Note, that on our web site we use Multimedia Component as the main term. In many cases it corresponds directly to a certain MG Object, but not in all cases. It is important to understand that a Multimedia Component is a DLL that can be part of an SDK and can be licensed. While an MG Object is something that can be used in the code (initialized, killed, etc.). It is correct to say that certain MG objects are implemented in specific MM Components.

In a certain way, this architecture is similar to DirectShow filters (meaning that there are Source Components and Sink Components), but it lacks certain limitations and offers several advantages. For example, it is not possible to run into a dead lock.

One of the key advantages is that the MultiGraph Objects are designed to be with the most popular programming languages and integrated development environments such as:

  • Visual C++
  • .NET (C#, Visual Basic)
  • VB6
  • Delphi
  • C++Builder

The framework is also a way to enable rapid application development methodologies within your organization as it provides for a way to focus on the business specifics quickly buidling a prototype and use short iterations to tune to application to the requirements.

Types of MultiGraph Objects

Similarly to DirectShow filters, there are 3 types of MG Objects: Source Objects, Sink Objects and Transform Objects.

  • MultiGraph Source Objects are the source of data in your application and are suppliers of data for downstream objects. For example, if your application is capturing data from a DeckLink board, you would be using the DeckLink Source object and if you need to play a list of files, you would be using the Playout object.


Mg objects1.jpg


  • MultiGraph Sink Objects and MultiGraph File Capture Objects are the consumers (i.e. receivers) of data. They receive data from Source Objects and either render it on the screen, output to a device or write to a file. For example, if you need to output data to a DeckLink board, you would be using the DeckLink Sink component and if you need to render the video on a screen, you would utilize the MG Preview object. In DirectShow, these would be referenced to as renderer filters.


Mg objects2.jpg


  • Transform Objects receive data from MG Source Objects, process it and stream it to the MG Sink or MG File Capture Objects. This type of objects is rarely used because our goal was to keep the framework as simple as possible, and there was really little need for such objects. Currently, only the Vision Mixer Object can be used in this mode (i.e. receive data from MP Playout), while in fact is is a MG Source Object.

Advantages of the Framework/Using the Framework/Examples of Using the Objects Together

The component-based approach provides flexibility when solving particular tasks. For example:

MultiGraph Objects Features

  • Every MG Object instance can be independently managed (i.e. started, stopped, connected to another object, etc.).
  • Any number of MG Sink Objects can be connected to a single MG Source Object. For example, you can simultaneously output to 2 DeckLink boards and write to a file:


Mg objects3.jpg


This feature is used in the Virtual Device Splitter application - to share a live source between several encoding applications (see also below on the third-party apps feature).

  • An MG Sink Object can dynamically switch between different MG Source Objects in a frame accurate seamless manner (without delay, black frames, etc.). For example, MG DV Capture can write to a file alternating between two MG Live Source instances. When switching from one source to the other there will be no black frames or frame loss, i.e. the switch is seamless. In other words, if the two MG Live Source instances represent a single device, the switching would not be noticeable.


Mg objects4.jpg


  • Due to the use of two DirectShow graphs inside the MG Sink objects, they allow for seamless switching between files during capture (without loosing a single video frame or audio sample). For example, an application log capture video and audio data 24/7 splitting it into 500 MB files. Overlapping is also pissible - i.e. the next file can start 10 seconds before the previous file ends. This feature is critical for audio/video logging appllications such as in broadcasting, security and medicine.
  • There are no limitations on the audio/video formats (i.e. type of compression, frame rate, resoltuion).
  • Built-in video scaling and audio conversion. High-quality video scaling and audio resampling is applied to enable multiple audio/video output options.
  • Internal audio/video synchronization is achieved by deploying the Audio/Video Synchronizer filter for live sources and the Multiplexer for transferring data between graphs (and objects).


Mg objects5.jpg


  • Data can be transferred between objects both within a single application or between several applications.
  • Some objects (such as MG Live Source, MP Playout, MG Vision Mixer) allow their output to be used as input in third-party applications. This is achieved by presenting an object instance as a virtual audio/video device. For example, the MG Vision Mixer object can be connected to the Flash Media Encoder to perform live streaming in Flash:


Mg objects6.jpg


This feature is also used by our Virtual Device Splitter application.

  • Most MG objects support DirectShow filters as plugins - for example to perform addtional audio/video conversions such as text, graphics of Flash animation overlay, or audio effects:


Mg objects7.jpg


  • Integrated support for MediaLooks codecs (for both decoding and encoding) - helps avoid the "codec hell" problem.
  • A common set of interfaces for every MG object provides for better understanding and simplfies development.

Basic Principles In Programming

The algorythm of using the MG Objects is as follows:

  1. Declaration of a MG Objects.
  2. Creation of MG Objects and plugins.
  3. Initial configuration (optional), such as:
    • enable/disable preview, set preview window
    • set audio/video format
    • select device, device format, device line
  4. Add plugin filters (optional). Steps (3) and (4) can be coded in any order.
  5. MG Object initialisation. Important! In order to initialise an MG Sink object a reference should be passed to an already initiated MG Source object; for MG Source object initialisation an empty (fake) reference is ised (see C# sample code below). So, an MG Sink object can be initiated only after the MG Source object it will be connected to.
  6. Additional object configuration (only for MG File Capture objects) - such as setting of audio/video encoders.
  7. Object start.
  8. Object closure/termination.

C# Sample Code

// (1) Declare MG Live Source Object & MG WMV Sink Object:
private MGLIVESOURCELib.CoMGLiveSourceClass m_objMGLiveSource;
private MLCAPTUREASFLib.CoMLCaptureASFClass m_objWMVSink;
 
// (2) Create MG Live Source Object:
m_objMGLiveSource = new MGLIVESOURCELib.CoMGLiveSourceClass();
 
// (2) Create Text & Graphics Overlay DirectShow Filter (the plugin):
m_objTnGOverlay = new PICTUREMIXERLib.CoPictureMixerClass();
 
// (3) Configure MG Live Source Object:
m_objMGLiveSource.EnableVideo(0,1);
m_objMGLiveSource.SetOutputWindow(0, panelPreview.Handle.ToInt32(), 0 );
m_objMGLiveSource.SetDevice( comboVidDevice.SelectedItem.ToString(), eMGDeviceType.eMGDT_Video );
m_objMGLiveSource.SetFormat( comboFormatVid.SelectedIndex, eMGDeviceType.eMGDT_Video);
m_objMGLiveSource.SetLine( comboInput.SelectedIndex, eMGDeviceType.eMGDT_Video );
 
// (4) Add Text & Graphics Overlay DirectShow Filter as plugin:
m_objMGLiveSource.AddFilter(MGLIVESOURCELib.eMGFilterType.eMGFT_Video, m_objTnGOverlay, "MediaLooks T & G Overlay");
 
// (5) Initialise Live Source Object:
Object pFake = new Object();
m_objMGLiveSource.Init(pFake, "MG Live Source" );
 
// (7) Start MG Live Source Object:
m_objMGLiveSource.Start(0);
 
// (2) Create WMV Sink Object:
m_objWMVSink = new MLCAPTUREASFLib.CoMLCaptureASFClass();
 
// (3) Configure WMV Sink Object:
m_objWMVSink.EnableVideo(0,0);
 
// (5) Initialize MG WMV Sink Object (we are doing this only after we have initialized m_objMGLiveSource):
m_objWMVSink.Init(m_objMGLiveSource, "MG ASF Capture" );
 
// (6) Set encoding and publishing properties (asfProps and pblProps need to be configured previously):
m_objWMVSink.SetASFProps( ref asfProps );
m_objWMVSink.SetPublishProps( ref pblProps );
 
// (7) Start MG WMV Sink Object:
m_objWMVSink.Start(1);
 
// (8) Close MG WMV Sink Object and MG Live Source Object (it is a better practice to close objects in the opposite
// order then they were initiated.
m_objWMVSink.Close();
m_objMGLiveSource.Close();
Retrieved from "http://wiki.medialooks.com/Multimedia_Component_Framework_Overview"
Personal tools