Intelligent Video Encoder with Advanced Analytics

Most movie buffs will be familiar with a variety of surveillance gadgets that are used to track/identify a person of interest and prevent access to restricted areas. One common scene in movies is that of personnel from the department of intelligence viewing recorded video and providing the location, time and description of the villain (usually) within a couple of minutes to the investigation team. In real life though, isolating the video of interest could take hours, if not days. This is especially true when the video is recorded on a digital video recorder or on a tape and no video analytics has been installed.

Today a majority of the cameras installed globally are analog cameras that are recording the video into a digital video recorder. MindTree offers an intelligent video encoder or video server, to such establishments who want to experience the benefits of moving to an IP environment, while protecting their analog investments. MindTree’s encoder with built in video analytics converts an analog video stream into an IP stream and simplifies search by generating metadata. For example a security officer will be able to search for directional violation by a black car between 9am to 10am. Commonly cited benefits of video encoders are centralized recording of video on a server or network storage using a network video recorder and eliminating the use of coaxial cables.

MindTree’s portfolio of “ready-to-brand” solutions is targeted at the physical safety and security market. It consists of a Surveillance Manager, suite of video analytics, video encoder, video decoder and digital video recorder. This article lists some of the design challenges faced during the development of the intelligent video encoder.

Design inputs
MindTree’s video surveillance solutions are all designed around five core themes – scalable, distributed, standards-based, intelligent and networked. In addition to these core themes, some of the key inputs provided to the design team include the following requirements:
• Provide storage on the encoder
• Work reliably in a mobile environment
• Provide flexibility to support up to four analog cameras, with the option of running advanced video analytics

Based on the core design themes and the above requirements, MindTree has designed and developed an intelligent video encoder that has several advanced and unique features – all in one small form factor box. Some of the unique and advanced features of this intelligent video encoder are as follows:
• One of the few encoders to be offered with advanced built-in video analytics. Customers have the option to run several advanced video analytics algorithms like sterile zone, foreign object detection and object classification. Customers can pick from a comprehensive suite of video analytics algorithms to meet their requirements. Analytics deployed will enable security officers to quickly isolate the video of interest through metadata search.
• The first video encoder to support the ONVIF standard. This ensures interoperability with video management software from vendors who support the ONVIF protocol.
• Supports recording of video on a SDHC when the network or server goes down. The recorded video can be synchronized with the centralized storage unit when the network or server comes up.
• Option to embed a GPS unit is provided. The GPS unit will provide the GPS co-ordinates so that the recorded video can be tagged with the GPS co-ordinates. This is especially useful when the encoder is used in a mobile environment.
• High quality video (4CIF resolution) at low bit rates and low latency
• Supports dual streams (4CIF and CIF) for each video channel at full frame rates (25/30fps)
• Provides users the flexibility to either encode four analog video streams while running basic video analytics on all four channels or encode two analog video streams while running advanced video analytics on both the channels

Challenges
The key challenge faced while building the video encoder was related to supporting dual streams (4CIF and CIF) for each video channel at full frame rate and running the advanced video analytics algorithms on each video channel based on Texas Instruments Inc.’s DM6467 (729MHz). Key task was optimization to get the right performance of our video analytics and the IP encoder application.

In order to achieve a full frame rate of 25/30fps on all four channels and support both 4CIF and CIF streams, we had to fine tune our video capture application, reduce the latency on our video streaming application and also use the latest H.264 codec from TI.

While integrating video analytics onto the encoder, we had to optimize on usage of the EDMA channels available for the H.264 codec and the video analytics. Though the DM6467 uses a coprocessor, the resizer that is available on the DM6467 is not as flexible as the one available on the DM365. Hence we had to modify the Linux kernel to capture the videos at YUV 420 instead of YUV422 thereby saving some cycles. Post these modifications we were able to get all the four video channels to run at full frame rate with dual streams.

Apart from the technical challenges listed above, the video encoder market is extremely cost competitive. Hence, the price-performance ratio of the intelligent video encoder had to be extremely competitive to be successful. MindTree’s intelligent video encoder that provides built-in analytics and ONVIF support is extremely cost competitive when compared to current products available in the market.

Design and development of the intelligent video encoder to an extremely demanding specification required a multi-disciplinary team with diverse skills ranging from system design to hardware/board design to image processing to BoM optimization. MindTree was able to put together a highly talented and experienced multi-disciplinary team together to build a highly competitive product.

Future Roadmap
MindTree’s intelligent video encoder roadmap will include migration to the DM6467 1GHz platform to support advanced analytics on all four channels. Customers will also have an option to pick from a wider range of analytics that include directional tripwire, object left behind and Automatic Number Plate Recognition (ANPR). MindTree’s intelligent video encoder will continue to support future revisions to the ONVIF standard such as the ONVIF 2.0. The encoder will also support the PSIA standard. Each of these new releases will bring new challenges and new opportunities to create more differentiators.

With MindTree’s intelligent video encoder, establishments will be able to isolate the video of interest as quickly as an officer from the department of intelligence does in “reel” life!

Caption

Figure 1: MindTree’s Intelligent Video Encoder.