Turning Vision into Powerful Insights
Vitis™ Video Analytics SDK
The Vitis™ Video Analytics SDK is the complete software stack to build AI-powered intelligent video analytics solutions on AMD platforms. It takes input from USB/MIPI cameras, video from files, or streams over RTSP, and uses Vitis AI to generate insights from pixels for various use cases such as understanding traffic and pedestrians in smart cities, health and safety monitoring in hospitals, self-checkout, and analytics in retail, detecting component defects at a manufacturing facility, and others.
The core SDK consists of several hardware accelerator plug-ins that use various accelerators such as video encoder, decoder, multiscaler (for resize and color space conversion), deep learning processing unit (DPU) for AI inference etc. By performing all the compute-heavy operations in dedicated accelerators, it can achieve the highest performance for video analytics applications.
For the developer community, Vitis Video also provides a framework in the form of generic Infrastructure plugins, software acceleration libraries, and a simplified interface for users to develop their own acceleration library to control a custom hardware accelerator. With this framework, users can easily integrate their custom accelerators/kernels into Vitis Video Analytics SDK. It builds on top of Xilinx Run Time (XRT), Vitis, and Vitis AI and abstracts these complex interfaces, making it easier for developers to build video analytics applications.
Using Vitis Video Analytics SDK supports deployment on Zynq™ UltraScale+™ MPSoC-based embedded platforms such as Kria™ SoM and ZCU104 evaluation kit, as well as larger edge or data center platforms like Alveo™ V70.
Vitis Video Analytics SDK Graph Architecture
The Vitis™ Video Analytics SDK is an optimized graph architecture built using the open-source GStreamer framework. The graph below shows a typical video analytic application starting from input video to output metadata. All the individual blocks are various plug-ins that are used. At the bottom are the different hardware engines used throughout the application. Optimum memory management with zero-memory copy between plug-ins and the use of various accelerators ensure the highest performance.
Vitis Video Analytics SDK Core Components
Custom Plug-ins
Highly optimized GStreamer plug-ins developed to provide very specific functionality using optimized kernels and IPs on AMD platforms.
Infrastructure Plug-ins
These are generic infrastructure GStreamer plug-ins being developed to help users integrate their kernels into the GStreamer framework.
Acceleration Software libs
These are optimized acceleration s/w libs developed to manage the state machine of the acceleration kernels/IPs and expose the interface so that these Acceleration s/w libs can be hooked into VVAS generic infrastructure plug-ins. These can be used as a reference to develop a new acceleration s/w lib based on the VVAS framework.
Acceleration Hardware (Kernels/IPs)
These are highly optimized kernels being developed by AMD.
Reference Platforms and Applications
VVAS provides several reference platforms catering to different applications/solutions needs.
Development Flows
- Download prebuilt images of Multichannel ML and Smart Model Select applications
- Exercise the existing functionalities/pipelines of tutorial and applications
- Develop Vitis™ Video Analytics SDK acceleration software library for your custom logic/kernel and integrate it with Vitis Video Analytics SDKinfrastructure plugin and verify the functional correctness.
- Vitis Video Analytics SDK ships 16 AI models with the current release. If you would like to use a different AI model, please either bring your own or download it from AMD Model Zoo and compile it using Vitis AI
- Run your pipeline/application with a newly built AI model
Embedded Deployment
The Vitis™ Video Analytics SDK delivers best-in-class performance for end-to-end intelligent video analytics application on your edge devices while keeping flexibility in deployment and optimal power consumption.
Get hands-on with the Vitis Video Analytics SDK and choose the AMD edge platforms:
PCIe Acceleration Deployment
Empowered by the Vitis Video Analytics SDK, AMD Data Center accelerator cards efficiently accelerate the whole pipeline for intelligent video analytics applications, providing higher performance and lower TCO over modern CPUs and GPUs.
Get hands-on with the Vitis Video Analytics SDK and setup your AMD Alveo™ Acceleration Cards:
Vitis Video Analytics SDK Documentation
Get Started with Embedded Platforms
Get Started with PCIe Acceleration Platforms
- Step 1: Install the Vitis Video Analytics SDK docker and drivers
- Step 2: Install and set up PCIe Acceleration card
- Step 3: Run example application
Download the Binary files:
(Vitis Video Analytics SDK version 3.0, Tool version 2022.2)