Aupera

VMSS (Aupera Implementation) – Video Machine-Learning Streaming Server

Video Machine-learning Streaming Server (VMSS) is a software application designed to function as a ‘server’ process to provide video analytic services to multiple video streams and efficiently utilize multiple FPGA resources on a server system. Aupera has jointly developed VMSS framework with Xilinx to implement it on Alveo U30 and U50LV acceleration cards and will maintain VMSS to be more compatible for developers who would like to utilize FPGA platforms without prerequisite FPGA knowledge. The solution includes VMSS framework, preprocessing, machine learning inference and postprocessing plugins.

If you have additional questions and/or want to report an issue with your user experience, write to vmss@auperatech.com, or vmss@xilinx.com.

Vendor: Aupera

Last Update: March 31, 2021

Size: 4.3G

Container Version: general_vmss_v1.2

Try or Buy

Obtain an entitlement to evaluate or purchase this product.


Use this product free and deploy the application below.


Deployment Options

This application is containerized and can be easily run in a few minutes in the cloud, or on-premises.

On Premises
Alveo U30
View & Buy Product
  • Xilinx Runtime: 202010.2.6.655 and 202020.2.8.726
  • Target Platform:
    U30 - U30_Rel_v0.9 or RC5_5
Alveo U50LV
View & Buy Product
  • Xilinx Runtime:
    202010.2.6.655 and 202020.2.8.726
  • Target Platform: U50LV - xilinx_u50lv_gen3x4_xdma_base_2

 


Get Started

Follow the instructions based on your deployment method.

Alveo U30/U50

1.

VMSS Docker and Drivers Installation:

1.1 Prerequisites

  1. Aupera release package only contains one docker image file: xilinxpartners/aupera_general_vmss:v1.2
  2. One U30 board with two xcu30 devices.The Satellite Controller firmware version is 6.3.2 or greater.
  3. One U50LV board
  4. One x86 Ubuntu 16.04/18.04 machine supports one PCIe x16 for U50LV and one PCIe x8 as dual x4 bifurcation for U30, this document will call this computer as the “x86_host”
  5. Internet access for the x86_host

1.2 VMSS Drivers and Docker Installation Procedure

1. Prepare essential software in x86_host:

    $sudo apt update 
$sudo apt install make build-essential nfs-kernel-server docker docker-containerd docker.io 
$sudo docker pull mongo:latest 

2. Edit /etc/sysctl.conf to enable net.ipv4.ip_forward=1 for ip forwarding:

    $sudo vim /etc/sysctl.conf 
$sudo service networking restart 

3. Load docker image and go into docker:

    $sudo docker pull xilinxpartners/aupera_general_vmss:v1.2
$docker run --cap-add sys_admin --cap-add NET_ADMIN --cap-add NET_RAW --privileged=true --network=bridge --hostname=general -it -v $(pwd):$(pwd) -w $(pwd) -e NFS_ABS_PATH=$(pwd) --name=<CONTAINER_NAME> <REPOSITORY>:<TAG>  bash 

Here <REPOSITORY>:<TAG> is the repository name, ex: general_vmss:v1.0, which can get from command ‘sudo docker images’, <CONTAINER_NAME> is a user defined container name, ex: test, and <NFS_ABS_PATH> is the nfs share directory and also the workspace of docker container, we will use current directory as nfs directory.

Next time when x86_host reboot, this shell do not need to perform, just run commands below to go inside docker container:

    $sudo docker container  <CONTAINER NAME> start
$sudo docker container exec -it <CONTAINER NAME> /bin/bash 

4. Start nfs service and extract nfs files (inside the docker):

    /root/nfs.sh

Then you can get following files/directory in ${NFS_ABS_PATH}:

├── BOOT_full_RC5_5.bin
├── EasyDarwin
├── U30
├── VMSS_APP
├── vmss_drivers.tar.gz
├── xrt_202010.2.6.655-amd64.deb
└── xu30-qspi-burn-58553330_10120113-2.9.3-build111.bin

Note: this shell needs to be performed every time when x86_host reboot.

5. Install VMSS drivers (outside the docker):

          $tar -xzvf vmss_drivers.tar.gz 
      $cd vmss_drivers 
      $sudo ./install.sh 

6. Confirm VMSS drivers had loaded (outside the docker)

    $lsmod | grep xocl 
$lsmod | grep xdma 

Note: VMSS driver will auto load when the x86_host reboot.

7. Start MongoDB docker (download if it does not exist)

         $sudo docker run --name mongo --rm -d -p 27017:27017 mongo  

Note, you have to modify the file VMSS_APP/server/conf/plugin.cfg inside VMSS docker container to change the mongo_url to match your host ip address.
Note: this step also needs to be performed every time when x86_host reboot.


2.

Obtaining Evaluation License

2.1 Xilinx Appstore Account

  1. Setup a Xilinx Appstore account (https://appstore.xilinx.com/ -> Manage Account).
  2. Request for an evaluation license for the VMSS product from the Xilinx Appstore. The product can also be purchased with a floating or node-locked license.

    Please note
    : Each U30 board contains 2 devices and therefore consumes 2 nodes to be fully utilized.

  3. At least one cred.json file (access key) must be created for your account (Manage Account -> Access Key -> Create an Access Key -> Download JSON). This file identifies your account to the Appstore during FPGA runtime and must be placed in the directory: ${NFS_PATH}U30/drm/cred.json where NFS_PATH is the shared directory user specified for docker to communicate with x86_host.

2.2 Node-Locked License

  1. The node-locked license configuration file located in the docker image at ${NFS_PATH}U30/drm/license_conf/nodelocked/conf.json must be copied into ${NFS_PATH}U30/drm/conf.json.
  2. To use a node-locked license purchased from the store, the U30 must be connected to the internet during runtime to generate a license file. After the license file is generated for the first time, it can unlock the DRM protection while offline and will only work for the specific FPGA’s DNA it was generated for.

2.3 Floating License / Free Trial

  1. Floating license / free trial configuration file is license_conf/floatingThe floating license configuration file located in the docker image at: ${NFS_PATH}U30/drm/license_conf/floating/conf.json must be copied into ${NFS_PATH}U30/drm/conf.json.
  2. A floating license purchased from the store can be used immediately. A floating license cannot be used while offline.

3.

VMSS General Release Firmware and Shell Installation

3.1 U30 Firmware Installation

3.1.1 Prerequisites

  • An Aupera U30 firmware QSPI flash dump file. The firmware file is named as xu30-qspi-burn<version and datecode>.bin.
  • One U30 board with two xcu30 devices.
  • One x86 Ubuntu 16.04/18.04 machine which supports PCIe x8 bifurcation with Xilinx XRT-2.6.655 installed (run “sudo dpkg -i xrt_202010.2.6.655-xxx.deb” in the host), xrt package version 2.6.655 can be downloaded from link

3.1.2 U30 Firmware Installation procedure

1. Source the XRT environment:

    $cd /opt/xilinx/xrt/  
$source setup.sh

2. Check Satellite Controller (SC) firmware version:

    $sudo /opt/xilinx/xrt/bin/xbmgmt flash --scan

It will show something like:

    Card [0000:07:00.0]
    Card type:          u30
    Flash type:         QSPI_PS
    Flashable partition running on FPGA:
        xilinx_U30_xdma_1_1,[ID=0x5ea44206],[SC=6.3.2]
    Flashable partitions installed in system:   (None)

Card [0000:08:00.0]
    Card type:          u30
    Flash type:         QSPI_PS
    Flashable partition running on FPGA:
        xilinx_U30_xdma_1_1,[ID=0x5ea44206],[SC=6.3.2]
    Flashable partitions installed in system:   (None)

Please contact Xilinx or Aupera support to upgrade the SC firmware if the version is lower than 6.3.2


3. Read the PCIe bus of the board

    $sudo lspci -d 10ee:

It will show something like:

    07:00.0 Processing accelerators: Xilinx Corporation Device 503d (rev 02)
07:00.1 Processing accelerators: Xilinx Corporation Device 503c (rev 02)
08:00.0 Processing accelerators: Xilinx Corporation Device 503d (rev 02)
08:00.1 Processing accelerators: Xilinx Corporation Device 503c (rev 02)

The BDF IDs, 07:00.1 and 08:00.1, are the card_id used in the next step. One U30 board has 2 FPGA devices (PCIe endpoints).


4. Flash the U30 board using XRT xbmgmt utility 

    $sudo /opt/xilinx/xrt/bin/xbmgmt flash --shell --card <card_id> --path <binfile>.bin 

where <card_id> is the ID read from the step 2, like 07:00.0, and <binfile> is the file name of the Aupera firmware QSPI flash dump file.
It will show the process as below:

    Idcode byte[0]=20 
Idcode byte[1]=bb 
Idcode byte[2]=21 
Idcode byte[3]=10 
Idcode byte[4]=44 
Idcode byte[5]=0 
Erasing flash.............................  
Programming flash........................ 
Verifying........................ 
Shell is updated successfully 
Cold reboot machine to load new shell on card 

After completion, flash the second card_id (like 07:00:1) read in the step 2 with the same flash dump file.


5. Power down and power up (cold reboot) the host to update the FPGA image from flash.


3.2 U50LV Shell Installation

3.2.1 Prerequisites

  • One x86 Ubuntu 16.04/18.04 machine with Xilinx XRT-2.8.726 installed (run “sudo apt remove xrt“ to uninstall XRT 2.6.655 in step 3.1.1, and then run “sudo dpkg -i xrt_202020.2.8.726-xxx.deb” to install XRT 2.8.726 in the host) , xrt package version 2.6.655 can be downloaded from link
  • U50LV board installed in a PCIe x16 slot

3.2.2 U50LV Shell Installation procedure

1. Source the XRT environment:

    $cd /opt/xilinx/xrt/ 
$source setup.sh

2. Read the PCIe bus of the U50LV board

    $sudo lspci -d 10ee

Screen will show message like:

    07:00.0 Processing accelerators: Xilinx Corporation Device 5060 
07:00.1 Processing accelerators: Xilinx Corporation Device 5061 

07:00.1 Processing accelerators: Xilinx Corporation Device 5061
The BDF IDs, 07:00.1 is the card_id used in the next step. The U50LV board has only 1 FPGA device (PCIe endpoint).


3. For U50LV card, DPUCAHX8H use the gen3x4 version target platform instead of the standard gen3x16 platform. Please download and install the required gen3x4 target platform files.

CentOS/Redhat 7.4-7.7: Xilinx-u50lv-gen3x4-xdma-2-202010.1-2902115-noarch_rpm.tar.gz
Ubuntu 16.04: Xilinx-u50lv-gen3x4-xdma-2-202010.1-2902115-16.04_deb.tar.gz
Ubuntu 18.04: Xilinx-u50lv-gen3x4-xdma-2-202010.1-2902115-18.04_deb.tar.gz


4. Flash the U50 board using Alveo XRT xbmgmt utility

    $sudo /opt/xilinx/xrt/bin/xbmgmt flash --shell --path <firmware_file> --card <card_id> 

where <card_id> is the ID read from the step 2, like 07:00.0, and <firmware_file> is the file name xilinx_u50lv_gen3x4_xdma_base_2


5. Cold reboot the host to load the FPGA image from flash and verify whether the FPGA boards are detected.

    $sudo lspci -d 10ee:  

More detail U50LV shell installation steps, see Xilinx github document.


4.

VMSS Application

4.1 Prerequisites

  1. U30 and U50 firmware and drivers ready in x86_host with U30 and U50LV boards installed.
  2. Have a RTSP streaming source (such as a camera) with the format as: encode=h264/h265, resolution=1080p, fps=30, gop=3.
  3. Get RTSP URL from an IP camera, like "rtsp://admin:admin123@172.16.1.15:554", as an input, or stream a local file to a RTSP server with the required format or local mp4 file.

    Tips: The VMSS docker includes a rtsp streaming tool that can be used to stream a local file to a rtsp server, with following steps:
    make sure inside the VMSS docker.
    go into the $NFS_ABS_PATH/EasyDarwin directory in the docker.
    run "nohup ./easydarwin -c easydarwin.xml -d 1>/dev/null 2>/dev/null &" to start a rtsp server with the default rtsp port 554.
    wait for a few seconds.
    use ffmpeg to stream a local file, like "ffmpeg -stream_loop -1 -re -i test.mp4 -c copy -rtsp_transport tcp -f rtsp rtsp://localhost:554/vmss-in".

  4. VMSS docker is running in x86_host. (see 1.2 step 3 and 4)
  5. MongoDB docker running in x86_host. (see 1.2 step 7)

4.2 Install U30 Software (inside the docker):

Note: a credential file named cred.json and a configuration file name conf.json must put into the directory U30/drm. Please refer to the section 2 of this document to generate and copy them to U30/drm to replace the existing example one, also make sure two U30 devices all program to correct firmware and then execute command below: 

            $cd U30 && ./start 

If this step prints some DRM activate failed error message, check the conf.json and cred.json correct or not. If this step hangs for long time with no response, check the following two common reasons:

  • Check whether VMSS drivers are ready or not. Reinstall VMMS drivers (xocl and xdma) if they are missing, as sometimes rebooting a machine while the system is updating may cause incorrect driver load. After driver reinstallation, restart the VMSS docker container.
  • The nfs service was not started before running the U30 start script. The nfs service is started by running the “root/nfs.sh” script.

The following message should be displayed in the end if the start script runs correctly:

    ntpdate -u ntp.api.bz
22 Apr 08:15:06 ntpdate[2804]: step time server 114.118.7.161 offset -1530581455.452152 sec

4.3 Run VMSS Client in Docker Container

1. Go inside VMSS docker container:

         $docker container exec -it <CONTAINER NAME> /bin/bash 

2. Run VMSS server:

         $cd VMSS_APP/server
     $source set_env.sh
     $./vmss_server

4.4 Run VMSS client in docker container

1. Go inside VMSS docker container

         $docker container exec -it <CONTAINER NAME> /bin/bash 

2. Start the VMSS client to open a session:

    $cd VMSS_APP/client 
$./vmss_client -c open -p 8001 -f open_stream.json 

3. Command to close one session:

        $../vmss_client -c close -p 8001 -s 5fcdebd24e4aff0693054073 

4. Command to close one session:

    $./vmss_client -c clear -p 8001 

For detail usage of vmss client, please refer to README.md in VMSS_APP/client directory.


4.5 Check detection or classification result

1. login MongoDB docker (in host):

        $docker exec -it mongo mongo 

2. Open database vmss_db:

    Open database vmss_db: 

3. browse results in database vmss_db:

    db.retail.find({ session_uuid: ObjectId("5fc8b1306dedf93052684a49")})  

where retail is the network name, and 5fc8b1306dedf93052684a49 is the session id string. This command will show the results of a specified session.


5.

Result

Check detection or classification result.

1. Login MongoDB docker(in host):

        $docker exec -it mongo mongo  

2. Open database vmss_db:

         $use vmss_db

3. Browse results in database vmss_db:

       db.retail.find({ session_uuid: ObjectId("5fc8b1306dedf93052684a49")})  

where retail is the network name, and 5fc8b1306dedf93052684a49 is the session id string. This command will show the results of a specified session.