MLPerf v1.0 Benchmark Demo on VCK5000


MLPerf v1.0 Benchmark Demo on VCK5000 issued its third round of inferencing result (MLPerf Inference v1.0). In this benchmark, Xilinx has achieved a result of ResNet-50 inference throughput at 5,921 FPS (Available catalog, server mode), using the latest Versal ACAP PCI-e card VCK5000 for Data Center. It showed the industry-leading performance acceleration with the Domain-specific Architecture optimized for AI inference on Xilinx adaptive computing platform. Now, the demo is available for free evaluation on the App Store. 

Vendor: Xilinx

Last Update: June 11, 2021

Size: 370MB

Container Version: xilinx/vitis-ai-cpu:1.3.598

Deployment Options

This application is containerized and can be easily run in a few minutes in the cloud, or on-premises.

On Premises
Alveo VCK5000
View & Buy Product
  • Xilinx Runtime: 2020.2_PU1
  • Target Platform: xilinx_vck5000-es1_gen3x16_base_2-3102046


Start Evaluation

Follow the instructions based on your deployment method.

Alveo VCK5000


Host Setup

1.1 Vitis AI 1.3.1 Docker Installation

    docker pull xilinx/vitis-ai-cpu:1.3.598

1.2 Install VCK5000 Shell/XRT packages

After the installation, cold reboot machine. For CentOS7.4/REHL7.4:

    cd shell_xrt/CentOS7.4       
sudo yum install xrt_202110.2.9.204_7.4.1708-x86_64-xrt.rpm
sudo yum install xilinx-sc-fw-vck5000-4.4.6-2.e1f5e26.noarch.rpm
sudo yum install xilinx-vck5000-es1-gen3x16-validate-2-3105556.noarch.rpm
sudo yum install xilinx-vck5000-es1-gen3x16-base-2-3102046.noarch.rpm
sudo /opt/xilinx/xrt/bin/xbmgmt flash --update --shell xilinx_vck5000-es1_gen3x16_base_2

1.3. Install XBulter on AIE5000 host machine

Please refer to XBulter Installation ( to install XBulter. After the installation of XBulter, reboot xbulter service.

    sudo service xbutler restart 


Running MLPerf

2.1 After downloaded Vitis-AI docker, enter Vitis-AI directory, and then start the Docker.

Update Vitis AI runtime libraries in docker

    cd vitis-ai-libs 
sudo dpkg -i librt-engine_1.3.0_amd64.deb
sudo dkpg -i libtarget-factory_1.3.0_amd64.deb

2.2 Dataset Downloading

Please download validation dataset from

2.3 Build and install the MLPerf benchmark:

    cd xilinx-mlperf-v1.0/mlperf-vitis-benchmark-app
make clean&& make -j

2.4 Running MLPerf benchmark

Note that MLPerf ResNet50 benchmarking results may slightly fluctuate due to different configurations of hardware resource. For example, advanced CPU processor and higher PCIe bandwidth will bring better performance result.

  • MLPerf server performance test
    • cd mlperf-vitis-benchmark-app
    • ./ -r [TARGET_QPS] --dir [IMG_DIR]

NOTE: default TARGET_QPS is 4000. To achieve better performance result, to change the value of TARGET_QPS.

--dir is to specify the location of imageNet validation dataset.

e.g. ./ -r 5200 --dir  /workspace/CK-TOOLS/dataset-imagenet-ilsvrc2012-val
The result is shown in mlperf_log_summary.txt

  • MLPerf server accuracy test
    • conda activate vitis-ai-tensorflow
    • ./ -m AccuracyOnly -n 50000 --dir [IMG_DIR]

  • MLPerf server accuracy test
    • cd mlperf-vitis-benchmark-app
    • ./ -r [TARGET_QPS] -s Offline --dir [IMG_DIR]
  • MLPerf offline accuracy test
    • conda activate vitis-ai-tensorflow
    • ./ -m AccuracyOnly -n 50000 -s Offline --dir [IMG_DIR]



Check the benchmark performance result in mlperf_log_summary.txt;
Check the accuracy result in accuracy.txt

** If you have any questions or technical inquiries, please contact with