Click to enlarge 

The Vitis™ Video Analytics SDK is the complete software stack to build AI-powered intelligent video analytics solutions on AMD platforms. It takes input from USB/MIPI cameras, video from files, or streams over RTSP, and uses Vitis AI to generate insights from pixels for various use cases such as understanding traffic and pedestrians in smart cities, health and safety monitoring in hospitals, self-checkout, and analytics in retail, detecting component defects at a manufacturing facility, and others.

The core SDK consists of several hardware accelerator plug-ins that use various accelerators such as Video Encoder, Decoder, Multiscaler (for resize and color space conversion), Deep Learning Processing Unit (DPU) for AI inference etc. By performing all the compute-heavy operations in dedicated accelerators, it can achieve the highest performance for video analytics applications.

For the developer community, Vitis Video also provides a framework in the form of generic Infrastructure plugins, software acceleration libraries, and a simplified interface for users to develop their own acceleration library to control a custom hardware accelerator. With this framework, users can easily integrate their custom accelerators/kernels into Vitis Video Analytics SDK. It builds on top of Xilinx Run Time (XRT), Vitis, and Vitis AI and abstracts these complex interfaces, making it easier for developers to build video analytics applications.

Using Vitis Video Analytics SDK, applications can be deployed on Zynq™ UltraScale+™ MPSoC-based embedded platforms such as Kria™ SoM and ZCU104 or PCIe acceleration platforms like Alveo U30, U50, and VCK5000.

The Vitis Video Analytics SDK is an optimized graph architecture built using the open-source GStreamer framework. The graph below shows a typical video analytic application starting from input video to output metadata. All the individual blocks are various plug-ins that are used. At the bottom are the different hardware engines used throughout the application. Optimum memory management with zero-memory copy between plug-ins and the use of various accelerators ensure the highest performance.

Click to enlarge 

Click to enlarge 

Custom Plug-ins
Highly optimized GStreamer plug-ins developed to provide very specific functionality using optimized kernels and IPs on AMD platforms.

Infrastructure Plug-ins
These are generic infrastructure GStreamer plug-ins being developed to help users integrate their kernels into the GStreamer framework.

Acceleration Software libs
These are optimized acceleration s/w libs developed to manage the state machine of the acceleration kernels/IPs and expose the interface so that these Acceleration s/w libs can be hooked into VVAS generic infrastructure plug-ins. These can be used as a reference to develop a new acceleration s/w lib based on the VVAS framework.

Acceleration Hardware (kernels/IPs)
These are highly optimized kernels being developed by AMD.

Reference Platforms and Applications
VVAS provides several reference platforms catering to different applications/solutions needs.