Profiling the Model

This chapter describes the utility tools included within the Vitis™ AI Development Kit, most of them are only available for the Edge DPU, except for the Vitis AI Profiler, which is a set of tools to profile and visualize AI applications based on the VART. The kit consists of five tools, which can be used for DPU execution debugging, performance profiling, DPU runtime mode manipulation, and DPU configuration file generation. With the combined use of these tools, you can conduct DPU debugging and performance profiling independently.

Vitis AI Profiler

The Vitis AI Profiler is an all-in-one profiling solution for Vitis AI. It is an application level tool to profile and visualize AI applications based on VART. For an AI application, there are components that run on the hardware, for example, neural network computation usually runs on the DPU, and there are components that run on a CPU as a function that is implemented by C/C++ code-like image pre-processing. This tool helps you to put the running status of all these different components together.

  • Easy to use as it neither requires any change in the user code nor any re-compilation of the program.
  • Visualize system performance bottlenecks.
  • Illustrate the execution state of different compute units (CPU/GPU).

Vitis AI Profiler Architecture

The Vitis AI Profiler architecture is shown in the following figure:

Figure 1: Vitis AI Profiler Architecture

Vitis AI Profiler GUI Overview

Figure 2: Vitis AI Profiler GUI Overview
DPU Summary
A table of the number of runs and minimum/average/maximum times (ms) for each kernel.

DPU Throughput and DDR Transfer Rates
Line graphs of achieved FPS and read/write transfer rates (in MB/s) as sampled during the application.

Timeline Trace
This includes timed events from VART, HAL APIs, and the DPUs.

Note:
  1. The Vitis Analyzer is the default GUI for vaitrace for Vitis AI 1.3 and later releases.
  2. The legacy web based Vitis-AI Profiler works for Edge devices (Zynq UltraScale+ MPSoC) in Vitis AI 1.3. For more information, see Vitis-AI Profiler v1.2 README.

Getting Started with Vitis AI Profiler

System Requirements

Hardware
  • Support Zynq UltraScale+ MPSoC (DPUCZD series)
  • Support Alveo accelerator cards (DPUCAH series)
Software
  • Support VART v1.3+

Installing the Vitis AI Profiler

  1. Prepare the debug environment for vaitrace in the Zynq UltraScale+ MPSoC PetaLinux platform.
    1. Configure and build PetaLinux by running petalinux-config -c kernel.
    2. Enable the following settings for the Linux kernel.
      • General architecture-dependent options ---> [*] Kprobes
      • Kernel hacking ---> [*] Tracers
      • Kernel hacking ---> [*] Tracers --->

        [*] Kernel Function Tracer

        [*] Enable kprobes-based dynamic events

        [*] Enable uprobes-based dynamic events

    3. Run petalinux-config -c rootfs and enable the following setting for root-fs.

      user-packages ---> modules ---> [*] packagegroup-petalinux-self-hosted

    4. Run petalinux-build.
  2. Install vaitrace. vaitrace is integrated into the VART runtime. If VART runtime is installed, vaitrace will be installed into /usr/bin/vaitrace.

Starting a Simple Trace with vaitrace

The following example uses VART ResNet50 sample:
  1. Download and set up Vitis AI.
  2. Start testing and tracing.
    • For C++ programs, add vaitrace in front of the test command as follows:
      # cd ~/Vitis_AI/examples/VART/samples/resnet50
      # vaitrace ./resnet50 /usr/share/vitis_ai_library/models/resnet50/resnet50.xmodel
    • For Python programs, add -m vaitrace_py to the python interpreter command as follows:
      # cd ~/Vitis_AI/examples/VART/samples/resnet50_mt_py
      # python3 -m vaitrace_py ./resnet50.py 2 /usr/share/vitis_ai_library/models/resnet50/resnet50.xmodel
      

    vaitrace and XRT generates some files in the working directory.

  3. Copy all .csv files and xclbin.ex.run_summary to your system. You can open the xclbin.ex.run_summary using vitis_analyzer 2020.2 and above:
    • If using the command line, run # vitis_analyzer xclbin.ex.run_summary.
    • If using the GUI, select File > Open Summary > xclbin.ex.run_summary.

To know more about the Vitis Analyzer, see Using the Vitis Analyzer.

VAI Trace Usage

Command Line Usage

# vaitrace --help
usage: Xilinx Vitis AI Trace [-h] [-c [CONFIG]] [-d] [-o [TRACESAVETO]] [-t [TIMEOUT]] [-v]

  cmd		      Command to be traced
  -b                 Bypass mode, just run command and by pass vaitrace, for debug use
  -c [CONFIG]        Specify the configuration file
  -o [TRACESAVETO]   Save trace file to
  -t [TIMEOUT]       Tracing time limitation, default value is 30 for vitis analyzer format, and 5 for .xat format
  --va               Generate trace data for Vitis Analyzer
  --xat              Generate trace data in .xat, for the legacy web based Vitis-AI Profiler, only available for Zynq MPSoC devices

Following are some important and frequently-used arguments:

cmd
cmd is your executable program of Vitis AI that want to be traced.
-t
Controlling the tracing time (in seconds) starting from the [cmd] being launched, the default value is 30. In other words, if no -t is specified for vaitrace, the tracing will stop after [cmd] running for 30 seconds. The [cmd] will continue to run as normal, but it will stop collecting tracing data. It is recommended that trace is about 50~100 images at once because less then 50 may not be enough for some statistic information and more then 100 will slow down the system significantly.
-c
You can start a tracing with more custom options by writing these options on a JSON configuration file and specify the configuration by -c. Details of configuration file will be explained in the next section.

Others arguments are used for debug.

Configuration

It is recommended to use a configuration file to record trace options for vaitrace. You can start a trace with configuration by using vaitrace -c trace_cfg.json.

Configuration priority: Configuration File > Command Line > Default

Here is an example of vaitrace configuration file.

{
  "options”: {
      "runmode": "normal",
      "cmd": "/usr/share/vitis-ai-library/sample/classification/test_jpeg_classification resnet50 sample.jpg",
      "output": "./trace_resnet50.xat",
      "timeout": 3
  },
  "trace": {
      "enable_trace_list": ["vitis-ai-library", "vart", "custom"],
      "trace_custom": []
  }
}
Table 1. Contents of the Configuration File
Key Name Value Type Description
options object Vaitrace options
cmd string the same with command line argument cmd
output string the same with command line argument -o
timeout integer the same with command line argument -t
runmode string Xmodel run mode control, can be “debug” or “normal”, if runmode == “debug” VART will control xmodel run in a debug mode by using this, user can achieve fine-grained profiling for xmodel.
trace object
enable_trace_list list Built-in trace function list to be enabled, available value "vitis-ai-library", "vart", “opencv”, "custom", custom for function in trace_custom list
trace_custom list The list of functions to be traced that are implemented by user. For the name of function, naming space are supported. You can see an example of using custom trace function later in this document

DPU Profiling Examples

You can find more advanced DPU Profiling examples with Vitis-AI Profiler on the Vitis-AI-Profiler GitHub page.