Video analytics software development kit supports AI models
Video decoding, pre-processing, artificial intelligence (AI) inference, tracking and post-processing are enabled with the latest Vitis Video Analytics software development kit (SDK), announced by Xilinx.
It is a complete software stack for intelligent video analytics applications on Xilinx platforms, deployed edge-to-cloud. The end-to-end video analytics supports widely used AI models such as YOLO, SSD, ResNet, Inception, RefindDet, ReID, FPN and OpenPose for real-time objection detection, classification and segmentation.
The software development kit provides the foundational layer for video analytic applications such as understanding vehicle and pedestrian traffic patterns in smart cities, health and safety monitoring in hospitals, self-checkout and analytics in retail, and detecting component defects at manufacturing facilities.
In provides a complete framework for developers to create generic infrastructure plugins, software acceleration libraries, and an interface to develop a custom acceleration library to control a custom hardware accelerator. The SDK builds on top of and abstracts the complex interfaces of Xilinx Run Time (XRT) and Vitis AI tools, making it easy for developers to build video analytics and transcoding pipelines without having to learn the complexities of XRT and Vitis AI tools.
To meet the evolving requirements and demands of video and vision applications such as smart cameras, industrial vision systems, automotive driver assistance and autonomous driving, smart retail and security, broadcast, professional AV and consumer products like TV and gaming applications, Xilinx says it has expanded its video and imaging IP portfolio for UltraScale+ and Versal devices.
The company enables 8K video capture and 8K display with what is believed to be the industry’s first HDMI 2.1 and DisplayPort (DP) 1.4 with UltraScale+ and Versal products. Combining HDMI 2.1 or DP 1.4 with the 8K scaler and mixer IP allows customers to build an end-to-end 8K video pipeline. For immersive gaming experiences, Xilinx’s video and imaging portfolio now includes high dynamic range (HDR), high frame rates and variable refresh rate formats and adaptive sync across the UltraScale+ and Versal families.
Xilinx has also added capabilities in the video IP, including 4k60 Warp with horizontal and vertical keystone correction, lens distortion correction, rotation and arbitrary warp for video walls, advanced conferencing cameras, projectors and applications in automotive.
There are also further Vitis Vision libraries for developers across automotive, industrial machine vision and smart camera applications, which includes image signal processing (ISP) with HDR and pre- and post-processing functions for machine learning (ML) neural networks with the OpenCV API.
The company has also announced the Vivado release 2021.2 will support 4:4:4 YUV colour space within the Xilinx Zynq UltraScale+ EV family. It is expected to be released by end of year.
