HTML5 Plugin: getting started

With our new HTML5 Plugin (beta) you can use HTML pages as the primary way to design your on-screen graphics as well as overlay web pages on top of the video. The plugin will be available as a separate product (price to be announced later) and will be available in MPlatform version 1.6.2.8736 and MFormats version 1.2.2.8726.

Use the HTML5 Plugin to:

  • overlay HTML-based text and graphics on top of the video;
  • overlay data, that is sourced from the web;
  • show web pages from the Internet (in PiP mode or in full) and control them (scroll, zoom, navigate, interact);
  • design a UI that can be overlaid on top of the preview window.

Here's an example of how Google Maps looks on top of a video stream:



When using the plugin, follow these simple steps.

1. Create the object

Create a new object and add it to your processing logic – just like you would with any other plugin.

In MPlatform:

// Create a plugin object  
MFOverlayHTMLClass m_objOverlayHTML = new MFOverlayHTMLClass();  

// Add it to the master object   
m_objPlaylist.PluginsAdd(m_objOverlayHTML, 0);

In MFormats:

// Creare plugin object  
MFOverlayHTMLClass m_objOverlayHTML = new MFOverlayHTMLClass();  

 // ... some threading stuff here to start receiving frames from the source ...              
 // Process each frame with plugin to apply changes   
m_objOverlayHTML.ProcessFrame(pFrame, out pFrameOut, out nFramesRes, ""); 

2. Load the web page

Use local web pages or web URLs:

m_objOverlayHTML.BrowserPageLoad("google.com");

3. Set the page display parameters: position, size, zoom, etc.

Use the HTML Plugin to configure the page appearance and behavior. For example, you can display just a part of the page and scroll down the content. The display parameters are configured using IMProps interface:

// Set the display area to the right side of the stream 
m_objOverlayHTML.PropsSet("dst.x", "60%"); 
m_objOverlayHTML.PropsSet("dst.y", "5%"); 
m_objOverlayHTML.PropsSet("dst.w", "40%"); 
m_objOverlayHTML.PropsSet("dst.h", "90%"); 

// Crop the page from the left and right sides  
// to make it look nicely and focus on a center column of the page 
m_objOverlayHTML.PropsSet("crop", "50,0,50,0"); 

// Scroll down the page vertically to display the content 
// outside the display area 
m_objOverlayHTML.PropsSet("scroll.speed_v", "1"); 


Note: you can also get and update the properties right on the web page. Check this article for details.

4. Control the page with mouse and keyboard actions

Use the page control methods to operate the page just like you would in a browser. As the plugin is displayed on a preview panel, we can monitor the mouse moves and clicks on that panel and pass them to the plugin to use for page control – or just simulate it programmatically.

For example, let's simulate a left mouse button click:

// Let's pass the left mouse down event to the HTML plugin 
// Create mouse event structure to fill the event details 
MF_MOUSE_EVENT mouseEvent = new MF_MOUSE_EVENT(); 

// Specify the mouse position 
mouseEvent.dblPosX = 100; 
mouseEvent.dblPosY = 200; 
             
// Use the eFlags value to specify the button that is pressed now.  
// It can be a shift, alt, etc. 
// I will pass the left mouse button press click. 
mouseEvent.eFlags = eMFEventsFlags.eMFEF_LEFT_MOUSE_BUTTON; 

// Pass the mouse click to the plugin  
// 3-rd parameter here (0) means that the button was pressed down (in opposite to mouse up) 
// 4-th parameter here (1) means that the clicks count. We pass 1 for a single click  
m_objOverlayHTML.BrowserMouseClick(mouseEvent, eMFMouseButtons.eMFMB_Left, 0, 1);

You can also simulate keyboard buttons:

// Let's simulate the key press events
// For example, the "A" key press
MF_KEY_EVENT keyEvent = new MF_KEY_EVENT();

// We will use the Char event type to simulate a single keystroke
keyEvent.eType = eMFKeyEventType.eMFKT_Char;
keyEvent.windows_key_code = (int)Keys.A;
m_objOverlayHTML.BrowserKeyEvent(keyEvent)

// We can also pass a key down event
// For example the "Alt" key
keyEvent = new MF_KEY_EVENT();

// We will use a KeyDown event type to simulate a button press down
keyEvent.eType = eMFKeyEventType.eMFKT_KeyDown;
keyEvent.windows_key_code = (int)Keys.Alt;
m_objOverlayHTML.BrowserKeyEvent(keyEvent);

5. Use common browser commands: back, forward, reload

Just like in any browser, you can go forward, backwards or reload the page:

// Navigate backward  
m_objOverlayHTML.BrowserCommand(eMFBrowserCommand.eMFBC_GoBack, ""); 
// Navigate forward  
m_objOverlayHTML.BrowserCommand(eMFBrowserCommand.eMFBC_GoForward, ""); 
// Reload the page 
m_objOverlayHTML.BrowserCommand(eMFBrowserCommand.eMFBC_Reload, "");

6. Interact with the page via JavaScript code

Use JavaScript to interact with a web page, such as fill in an email and password and click the login button:

string myJavascript = "document.getElementById('email').value = 'YourEmail';"; 
myJavascript += "document.getElementById('pass').value = 'YourPass';"; 
myJavascript += "document.getElementById('loginbutton').click();"; 
m_objOverlayHTML.BrowserJavascriptExecute(myJavascript);

7. Handle the events on the page

Refer to this article for on-page events handling.

Try the HTML5 Plugin with the MPlatform Sample Playlist application:



See also