Overview

Vitis™ Video Analytics SDK

 

The Vitis™ Video Analytics SDK is the complete software stack to build AI-powered intelligent video analytics solutions on Xilinx platforms. It takes input from USB/MIPI cameras, video from files, or streams over RTSP, and uses Vitis AI to generate insights from pixels for various use cases such as understanding traffic and pedestrians in smart cities, health and safety monitoring in hospitals, self-checkout, and analytics in retail, detecting component defects at a manufacturing facility, and others.

The core SDK consists of several hardware accelerator plug-ins that use various accelerators such as Video Encoder, Decoder, Multiscaler (for resize and color space conversion), Deep Learning Processing Unit (DPU) for AI inference etc. By performing all the compute-heavy operations in dedicated accelerators, it can achieve the highest performance for video analytics applications.

For the developer community, Vitis Video also provides a framework in the form of generic Infrastructure plugins, software acceleration libraries, and a simplified interface for users to develop their own acceleration library to control a custom hardware accelerator. With this framework, users can easily integrate their custom accelerators/kernels into Vitis Video Analytics SDK. It builds on top of Xilinx Run Time (XRT), Vitis, and Vitis AI and abstracts these complex interfaces, making it easier for developers to build video analytics applications.

Using Vitis Video Analytics SDK, applications can be deployed on Zynq® UltraScale+™ MPSoC-based embedded platforms such as Kria™ SoM and ZCU104 or PCIe acceleration platforms like Alveo U30, U50, and VCK5000.


Vitis Video Analytics SDK Graph Architecture

The Vitis Video Analytics SDK is an optimized graph architecture built using the open-source GStreamer framework. The graph below shows a typical video analytic application starting from input video to output metadata. All the individual blocks are various plug-ins that are used. At the bottom are the different hardware engines used throughout the application. Optimum memory management with zero-memory copy between plug-ins and the use of various accelerators ensure the highest performance.


Vitis Video Analytics SDK Core Components

Custom Plug-ins
Highly optimized GStreamer plug-ins developed to provide very specific functionality using optimized kernels and IPs on Xilinx platforms.

Infrastructure Plug-ins
These are generic infrastructure GStreamer plug-ins being developed to help users integrate their kernels into the GStreamer framework.

Acceleration Software libs
These are optimized acceleration s/w libs developed to manage the state machine of the acceleration kernels/IPs and expose the interface so that these Acceleration s/w libs can be hooked into VVAS generic infrastructure plug-ins. These can be used as a reference to develop a new acceleration s/w lib based on the VVAS framework.

Acceleration Hardware (kernels/IPs)
These are highly optimized kernels being developed by Xilinx.

Reference Platforms and Applications
VVAS provides several reference platforms catering to different applications/solutions needs.


Development Flows

Development Flows

  1. Download prebuilt images of Multichannel ML and Smart Model Select applications
  2. Exercise the existing functionalities/pipelines of tutorial and applications
  3. Develop Vitis Video Analytics SDK acceleration software library for your custom logic/kernel and integrate it with Vitis Video Analytics SDKinfrastructure plugin and verify the functional correctness.
  4. Vitis Video Analytics SDK ships 16 AI models with the current release. If you would like to use a different AI model, please either bring your own or download it from Xilinx Model Zoo and compile it using Vitis AI
  5. Run your pipeline/application with a newly built AI model

Deployment Options
KV260

Embedded Deployment

The Vitis Video Analytics SDK delivers best-in-class performance for end-to-end intelligent video analytics application on your edge devices while keeping flexibility in deployment and optimal power consumption.

Get hands-on with the Vitis Video Analytics SDK and choose the Xilinx edge platforms:


On-Premise Deployment Alveo Card

PCIe Acceleration Deployment

Empowered by the Vitis Video Analytics SDK, Xilinx Data Center accelerator cards efficiently accelerate the whole pipeline for intelligent video analytics applications, providing higher performance and lower TCO over modern CPUs and GPUs.

Get hands-on with the Vitis Video Analytics SDK and setup your Xilinx® Alveo™ Acceleration Cards:


Documentation
Getting Started

Get Started with PCIe Acceleration Platforms

  •  Step 1: Install the Vitis Video Analytics SDK docker and drivers
  •  Step 2: Install and set up PCIe Acceleration card
  •  Step 3: Run example application

For more info, please visit https://www.xilinx.com/products/acceleration-solutions/xilinx-aupera-vmss.html

Video