Video analytics – scaling the design pyramid

The world has indeed flattened. When Tomas Friedman first alluded to this concept in his book “The World is Flat” in 2005, the world still considered India and Bangalore as the place to outsource everything else except their crown jewel technologies. Not anymore.

Today, India is considered as a powerful product design powerhouse. Some of the best products in the world are being designed from India. The Indian design eco-system is a combination of small and niche organizations targeted at specific segments of the market and large global corporations involved in design of products launched in the market today. The engineering talent and skills have enabled both ends of the spectrum to set up teams to develop their products quickly. In addition, India in itself is a growing market for various electronic and software products. The India design ecosystem has received a boost due to investments by Indian companies and Government organizations in improving the R&D infrastructure and deepening collaboration with the academia. It will not be too long before we see Indian products competing with the best in the world market.

MindTree was founded 10 years ago, with the vision to create differentiated technologies, solutions and Intellectual Property (IP) for the Indian and global markets. While MindTree helps its customers build software and hardware products, it also invests proactively in building IPs and technologies which can be licensed to customers worldwide. MindTree is an acknowledged leader in the Bluetooth space and continues to maintain its leadership while at the same time is investing in few new technologies. Our current focus is in the area of intelligent video surveillance technologies and specifically into video analytics.

What is intelligent video surveillance?
A video surveillance system usually refers to the concept of viewing and monitoring of public or private installations using cameras remotely. It attempts to augment the effectiveness of security personnel with “additional electronic eyes”. In the recent past, video surveillance has made phenomenal progress both in terms of its adoption and range of technology solutions. In addition to the technological advancements, recent geopolitical events have caused an increased global focus on security in both private and public environments leading to an increased demand for video surveillance systems. However, even today, most of the video is scrutinized manually and more often after the event has occurred. The manual operation is not scalable due to possible operator fatigue or oversight. With the growing need for more efficient systems, the recent trend is towards distributed, networked and intelligent surveillance systems. The “intelligence” in the video surveillance systems comes from video analytic algorithms and the device management protocols.

Video analytics is a technology that is used to analyze the content of the video for specific patterns, behavior and objects. The software algorithms run on processors inside a computer or on an embedded computer platform in video cameras, recording devices, or specialized video processing units. Examples include: counting the number of pedestrians entering a door or geographic region, determining the location, speed and direction of travel, identifying movement of people in specified areas, identifying an object left behind, a guard missing from the post and so on. The real value of video analytics can be realized when incidents can be detected pro-actively, providing real-time alerts and when relevant footage can be located quickly without having to scan the video manually.

MindTree’s video analytics portfolio
MindTree has developed a very advanced and rich portfolio of video analytic algorithms and intelligent video surveillance systems. The video analytic algorithms (Figure 1) include Display Enhancement Algorithms such as real-time video stitching, digital zoom, image stabilization; Video Content Analysis Algorithms such as view tamper detection, sterile zone monitoring, perimeter violation, object density detection, and automated number plate detection; and Video Search Algorithms. The video analytic algorithms cover a wide range of applications from traffic monitoring to specific algorithms for surveillance of high value installations.

The patent-pending video analytic algorithms are optimized and customized to run on embedded devices such as the MindTree Digital Video Encoder (Figure 2), shown alongside or on Video Analytic Servers. The video analytic algorithms along with the ONVIF protocol turn this device into a very cost-effective intelligent and remote manageable device.

Figure 3 below illustrates a panoramic view of a parking lot (150o field of view) obtained by “stitching” images from three Sony Cameras. This helps the viewer to obtain an uninterrupted and seamless view on a real-time basis (25fps to 30fps) without having to switch between cameras continuously. This algorithm can be used to stitch videos from up to four cameras either in the vertical or horizontal plane.

MindTree has deployed systems with its algorithms on a trial basis at a few installations both in India and abroad. MindTree has also implemented and licensed variations of these algorithms for specific scenarios such as red-light violation, guard inactivity detection and panoramic view stitching.

Design challenges
Although video analytics has outlived many hype curves, many skeptics still question the performance and accuracy of such systems. When it was projected over a decade ago that video analytics solutions would eventually eliminate the need of an operator, it created lot of hype and buzz. However, the fact remains that these solutions have been around for a while now and are definitely here to stay!

Video analytics offer some of the most challenging technical problems ever solved by man, given the sheer volume, speed and accuracy of the data that has to be handled. To illustrate, a D1 Resolution (704x480) video captured at 30fps for a single camera generates approximately 162Mb (HD Resolution data is about 440Mbps) of data. Typically each system handles anywhere between four to 32 cameras. This volume of data has to be encoded, displayed, streamed, stored and analyzed per second. Hence this puts a lot of demand on the algorithms and systems.

This is also a very cost competitive market and hence the price-performance ratio has to be extremely competitive to create a successful product. Customers are willing to pay a little more for added intelligence but not a huge differential.

Most of the algorithms are heavily patented and hence one of the main challenges was to come up with unique algorithms that do not violate other patents.

India and MindTree advantage
Video analytics require multi-disciplinary skills and one of the obvious challenges is to obtain the right talent. The premier engineering institutes of India such as Indian Institute of Science (IISC) Bangalore, IIIT Hyderabad, IIT Mumbai, and IIT Chennai, to name a few, have well established Computer Vision and Machine Learning Labs and departments, thus creating the talent pool. In addition, expert opinion is available due to the fact that vision related technologies are used quite extensively in the Indian Space and Defense programs.

MindTree has been able to tap experts from academia like IISC and Government institutions like CSIR (Council for Scientific and Industrial Research) to shape its programs. The Indian scenarios are quite unique in terms of scale and diversity and provide huge challenges for video analytics.

Video analytic algorithms popular elsewhere in the world, often fail to work in Indian conditions. With the twin focus on solving for Indian conditions and price-performance ratio, MindTree is confident that its algorithms will work very well in India and other environments as well.

The future
Video analytics is one of the hottest areas of research in the world today, and many companies and research institutions are trying to find optimal solutions for real-world problems. Fortunately, the evolution of this technology and a healthy product development environment is providing opportunity to companies like MindTree to be a leader in this area.


Author Information
Sharmila Saha is the VP and Technology Evangelist, R&D Services, of MindTree Ltd.
Captions

Figure 1: MindTree’s portfolio of video analytic algorithms.

Figure 2: The Intelligent Digital Video Encoder.

Figure 3: MindTree’s Real Time Video Stitching algorithm.