Frame accurate capture and playback of custom user data

Sometimes it's important to have some custom data to be stored with your video stream frame accurately. For example, camera ID, scene tags, coordinates, service information etc. With MPlatform and MFormats SDKs you can add this data to frames of the video stream.  Once you will add custom data to frames  you can capture them to an *.ANC file that should be stored next to original video file. This file will contain all your custom data that can be extracted out of the frames during the playback.  

NOTE: The data in *.ANC files is stored in Medialooks custom format. This format can be written or read with Medialooks products only. It is not standardized as closed captions and cannot be transmitted over the SDI. If you need to store data as closed captions and transmit over the SDI you should use custom ANC data packets.  

In general working with custom data can be separated in this simple steps: 

1.  Configure MWriter or MFWriter object for custom data capturing.

1.1. Enable writing of custom data to *.ANC file for  MWriter or MFWriter object by setting "anc_capture = true" property value. 
Please note that you can do it in 3 ways. Choose the best one for you: 
Specify this parameter value in a registry.  This value will be used by default for all newly created objects:
MFormats SDK:  [HKEY_CURRENT_USER\SOFTWARE\Medialooks\MFormats\MFWriter] "anc_capture = true"
MPlatform SDK:  [HKEY_CURRENT_USER\SOFTWARE\Medialooks\MPlatform\MWriter] "anc_capture = true" 

Specify  this parameter value using PropSet method. This value will be used within object's life period:
m_objWriter.PropsSet("anc_capture", "true");

Specify  this parameter value within capture configuration line. It will be used within a capture session: 
m_objWriter.WriterSet(sfd.FileName, 1, "format='mp4' video::codec='mpeg4' audio::codec='aac' anc_capture='true'");

1.2. Specify the data tags that should be captured by setting "anc_capture.tags" property value.
By default, the  MWriter or MFWriter objects capture these types of data:  C608, C708, GA94 and all ANC lines (the ones that are specified like ANC.Line.X). 
So your custom data will not be captured until you will specify your custom tags with "anc_capture.tags" property: 
// Capture С608 and ANC.Line.9 only. No custom frame data will be captured in this case! 
m_objWriter.PropsSet("anc_capture.tags", "С608, ANC.Line.9"); 

// Capture all default data (C608, C708, GA94 and all ANC.LineX) and custom data under "location" and "cameraId" tags.  
m_objWriter.PropsSet("anc_capture.tags", "default, location, cameraId"); 

// Capture custom data under "location" and "cameraId" tags only.  
m_objWriter.PropsSet("anc_capture.tags", "location, cameraId"); 

2. Get frames to write or read data.  

Now you need to retrieve MFFrame objects to be able to write and read custom data.

2.1. MPlatform SDK
The frames should be processed in sync and you need to get access to frame's data, be able to write a custom one. 
So the first thing is you need to do is set "on_frame.sync = true" and "on_frame.data = true" for the object that you use to handle events:
m_objFile.PropsSet("object::on_frame.sync", "true"); 
m_objFile.PropsSet("object::on_frame.data", "true"); 

Important! Processing frames in sync mode is a dangerous thing because you keep the frame and it cannot be processed by further pipeline until you exit the OnFrameSafe event. So please be sure that you do not perform any operations that will consume more than your frame rate period. 

Once you have configured the object, you can  handle it's OnFrameSafe event to get frames:
// Subscribe on MFile OnFrameSafe event 
m_objFile.OnFrameSafe += new IMEventsFrame_OnFrameSafeEventHandler(m_objPlaylist_OnFrameSafe); 

// ... some code lines later ... 

// Handle the event to get the frame 
void m_objPlaylist_OnFrameSafe(string bsChannelID, object pMFrame) 
{ 

    // ... play around with frame, for example, write or read it's data...  
     
    // Don't forget to release the frame!   
    Marshal.ReleaseComObject(pMFrame);   
}

2.2. MFormats SDK
Getting frames in MFromats SDK is a native process, so all you need to do is operate with frames normally, as you do for example on file playback

3. Write custom data to frames and capture video to file.

To write the data to frames you need to use MFDataSet and MFStrSet methods of the MFFrame object. 

Once you have got the MFFrame object you can add your custom data to it. There are 3 types of data that you can write: 
  • string data with MFStrSet method.
  • long integer data with MFDataSet method by specifying the 0 as data size value and numeric as data pointer value. 
  • raw data with MFDataSet method by specifying the data size and pointer.
// Write custom string data, for example - a location description.  
pFrame.MFStrSet("location", "The best camera spot ever.");  

// Write custom long data, for example - the ID. Just pass 0 as data size value.  
pFrame.MFDataSet("CameraId", 0, 123456789); 
             
// Write custom raw data.  
pFrame.MFDataSet("myCustomData", myDataSize, myDataPointer);  

After that, you can write your MPlatform stream or MFormats frames flow into the file with MWriter or MFWriter object normally. The *.ANC file with your custom data will be located in the same folder and have the same name as your captured video file. 

Important! The *.ANC file should be transported and stored in the same location as your captured video file in order to let Medialooks objects to get access to it.  

4. Playback the file and read your custom data.

Make sure that *.ANC file with custom data is stored in the same location as your captured video file and start the video file playback. 
Handle OnFrameSafe event for MPlatform to get frames or just check frames with MFormats to get MFFrame objects and read  your custom data. 

Just the same way you write your data, you can extract it: 
  • string data with MFStrGet method.
  • long integer data with MFDataGet method by getting your numeric data value out of data pointer. 
  • raw data with MFDataGet method by getting the data size and pointer.
// Read custom string data, for example, a location description.  
string strLocation; 
pFrame.MFStrGet("location", out strLocation); 

// Read custom long data, for example, the ID.  
int myDataSizeStub; 
long myId; 
pFrame.MFDataGet("CameraId", out myDataSizeStub, out myId); 

// Read custom raw data.  
int myDataSize; 
long myDataPointer; 
pFrame.MFDataGet("myCustomData", out myDataSize, out myDataPointer);

Try capture and playback custom data with our sample applications.

MPlatform SDK 
Capture custom data: C:\Program Files (x86)\Medialooks\MPlatform SDK\Samples Basic\C#\ANC Transmit Sample
Read custom data: C:\Program Files (x86)\Medialooks\MPlatform SDK\Samples Basic\C#\ANC Receive Sample

MFormats SDK  
Capture custom data: C:\Program Files (x86)\Medialooks\MFormats SDK\Samples\C#\Sample Custom ANC Capture
Read custom data: C:\Program Files (x86)\Medialooks\MFormats SDK\Samples\C#\Sample Custom ANC Receive



Note: this feature is not available in playback while recording mode now, so stop the capture process before starting the playback.