Xilinx VMSS – Video Machine-Learning Streaming Server
Xilinx VMSS – Video Machine-Learning Streaming Server
Xilinx’s Video Machine-learning Streaming Server (VMSS) is a software application designed to function as a ‘server’ process to provide video analytic services to multiple video streams and efficiently utilize multiple FPGA resources on a server system.
If you should have additional questions and/or want to report an issue with your user experience, please send your inquiry to vmss@xilinx.com.
Vendor: Xilinx
Last Update: August 15, 2020
Size: 5.26 GB
Container Version: vmss_v005_nimbix
Deploy Free
This Product is free to deploy from Xilinx.
Use this product free and deploy the application below.
Deployment Options
This application is containerized and can be easily run in a few minutes in the cloud, or on-premises.
| Cloud Services | |
|---|---|
| Nimbix U200 + U250 |
|
Start Free Evaluation
Follow the instructions based on your deployment method.
1.
Sign up for Nimbix Service
- Go to Nimbix Cloud site to sign up : https://www.nimbix.net/contact-us
- Once the sign-up form has been submitted, you will receive an e-mail with instructions for setting up your Nimbix account.
2.
Application Execution
2.1 Select the VMSS Application on the Nimbix Platform
Sign-in to the Nimbix Platform and search for VMSS application:
https://platform.jarvice.com/?page=compute&name=VMSS
You will see the results as shown in the figure.
Clicking on it will start the VMSS application.
There are 3 different modes in which you can run jobs in this application
- Click on “One Click Demo Mode” button for the execution of a preset job. “One Click Demo Mode” will execute the Cascaded Retail model on an included video file and display the results on the console. If you want to run a customized job, by supplying your own video file or image file, we recommend using either “Interactive Client Demo Mode” or “Full Desktop Mode". Go to step 2.2.1 for the next step on running “One Click Demo Mode”.
- Click on “Interactive Client Demo Mode” button if you wish to supply your own media input file and see it processed from one of several available pre-configured ML models. “Interactive Client Demo Mode” will allow you to pass in either a JPEG or MP4 file and to select a model from a list of pretrained models which will be executed on the input file you passed. Note that because each model is trained on a particular dataset, you will need to provide input with a resolution based on the model you have selected (e.g. 416x416 or 224x224) and, ideally, consisting of content likely to match what the model was trained evaluate. If you would like an even more customized experience using VMSS, we recommend using “Full Desktop Mode." Go to step 2.2.2 for the next steps on running “Interactive Client Demo Mode”.
- Click on “Full Desktop Mode” button for a full Xfce Linux environment. "Full Desktop Mode” is used to explore VMSS using an interactive terminal. Go to step 2.2.3 for the next step on running “Full Desktop Mode”.
2.2 Run the Application
2.2.1 Run in 'One Click Demo Mode'
Press the ‘One Click Demo Mode’ to run the VMSS Application with a predefined job. This will allow any new user to sit back and watch VMSS Application execute with one click. The predefined job will run a sequence of two ML models (a detector followed by a classifier) trained to recognize a select number of retail items, on a sample video file and print the results to the console.
VMSS Application allows you to select which node you want to run the job on. Currently, you can only run VMSS on “nx56u_202010” machine only, which is the default. You can also preview you job, under preview submission tab. Pressing Submit will submit this job to the VMSS application queue.
Once the job starts, you will be able to watch scripts start VMSS server, client and execute the job. Once the job finishes it will perform clean up and terminate the job.
Results
You can have access to the full terminal output (in a text file which can be saved) by pressing the yellow icon in the upper left corner of the terminal (shown in figure).
2.2.2 Run ‘Interactive Client Demo Mode’
Press the “Interactive Client Demo Mode” to enable the VMSS application to invoke like an executable accepting some configurable parameters. These configurable parameters include
- Selection of a pretrained model from a list of models
- Supplying an input image or video file to run the model on
On this screen you can pass the parameters you wish to execute the job with. Current VMSS application implementation supports following models
| No. | Model Supported |
String to use Model parameter |
|---|---|---|
| 1 | 416x416 TinyYoloV3 trained on custom retail-themed dataset | tinyyolov3-retail |
| 2 | 224x224 Resnet50 trained on custom retail-themed dataset | resnet50-retail |
| 3 | 416x416 YoloV2 trained on COCO dataset | yolov2-coco |
| 4 | 224x224 GoogleNet trained on ImageNet dataset | googlenetv1-imagenet |
| 5 | 416x416 Cascaded Model consisting of TinyYolov3 and Resent50 models trained on a custom retail-themed dataset. | cascade-retail |
To select a model, you have to set the Model parameter to one of the strings from above table.
To provide your input file, it must be first uploaded. Click here for instructions to upload the file. The file will be in user’s /data directory. You must specify the “Path to input file” parameter to be the full path of your file in /data directory. Input file can either be a JPEG or H.264 encoded MP4 file.
NOTE: At the moment, only jpeg files are supported for GoogleNet and YoloV2.
Once you have submitted your job, it will queued up for execution by the application.
When your submitted job gets picked up by the application, you can watch it get executed on the console window.
Results
Once the job finishes, it will release the resources and do a cleanup. Similar to section 2.2.1, you can download the full console output by pressing the yellow button in upper right corner and save it locally.
VMSS Application allows you to select which node you want to run the job on. Currently, you can only run VMSS on “nx56u_202010” machine only, which is the default. You can also preview you job, under preview submission tab. Pressing Submit will submit this job to the VMSS application queue.
Once the job gets pulled off the queue for execution, Nimbix will show you window on the right.
There are multiple ways to interact with the application.
- You can press “Click here to connect”, which will open up Xfce Linux environment in a new browser tab.
- Alternate ways to connect to the job can be found by pressing yellow-highlighted button (shown right). This allows you to either use a VNC client or SSH into the job.
Once you log in, you will be presented with the Xfce Linux environment above. You will have a /data directory, in which you can access user files. You can upload or download files from /data directory either using Nimbix’s File Manager or using FTP (see https://support.nimbix.net/hc/en-us/articles/208083526-How-do-I-transfer-files-to-and-from-JARVICE- )
You can now start a terminal and begin interacting with VMSS application.
Here is the directory structure:
Once you have a terminal up, to start the server, run
cd /home/VMSS/server
source ./vmss_server_launch.sh
The above steps will launch the VMSS server process. You will see following banner to indicate VMSS server has started successfully. You can now open a new terminal to start one of the included client applications.
For the client application, start a new terminal and run following commands. In this example, we will use the included Gstreamer client application which is used to send JPEG images to VMSS for ML processing. To use MP4 video files as input, use FFMPEG client application. You can find instructions to run other client applications in the README.txt within each client application directory.
cd /home/VMSS/client/gstreamer
source env.sh
./vmss_client 127.0.0.1 8001 data/commands/vmss_open_jpeg_retail_request.txt
This will open a VMSS session which will run Cascaded Retail models on images. Also, this will open a sub shell to interact with.
In the subshell, run
./scripts/gst_send_rtp.sh -f /data/<name_of_416x416_image.jpeg> -w 416 -h 416
You should then see detection and classification results, as json strings, displayed on the VMSS server terminal. Screenshots are attached, to show expected behavior.
Gstreamer client pipelines are designed to run indefinitely. So, you have to manually terminate the pipeline. Once you terminate the Gstreamer pipeline, you can exit the subshell, which will automatically send a close session command to the server and terminate the session on the server side.
Once you are done with the VMSS application job, you can go back to Nimbix’s Jarvice tab, and close the job, by pressing the power button in upper right corner of the window. It will safely terminate the job and release the resources.
Results:
For the desktop mode, result of your job can be seen on the terminal while running the commands. The results are parsed as JSON strings, and can be stored in any database or text file.
