2. System-on-Modules (SOM) Boards
  3. Kria Adaptive System-on-Modules
  4. Vision AI Starter Kit

Step 4. Launching the Smart Camera accelerated application (PetaLinux)

With the Kria™ KV260 Vision AI Starter Kit, AMD proudly introduced users to the concept of accelerated applications. These are pre-built applications for AI and SW developers, giving them a head start to develop their target end-applications

You can quickly run one of our accelerated apps and even customize it with different AI models or sensors. Sound exciting?! Let’s try the Smart Camera accelerated application.

xilinx-smart-camera-tile

The KV260 Vision AI Starter Kit is unique in its ability to load and run applications that configure the hardware to implement a variety of functions. The utility “xmutil” is used to load the various applications. One accelerated application can be loaded at a given time. We will first load an application that programs the SOM to behave as a smart camera, with the ability to process a 1080p video stream from either a MIPI or USB camera, and display it over HDMI or DisplayPort, or stream it over wired Ethernet.*

* You will need a viewer capable of displaying video streamed over Ethernet, for example ffplay or vlc.

About the Smart Camera accelerated application:

An Ultra-HD camera with built-in machine learning to do face/pedestrian detection, having support to capture from various video sources (MIPI camera, video files & USB camera) and can output processed video on HDMI, DisplayPort, or RTSP streaming of ROI-based encoded video.

Most popular use case: Smart city applications (face, pedestrian detection, and traffic management) and video analytics capabilities.

smart-camera-diagram

Launching the Smart Camera accelerated application:

You have already made the connections needed to launch this application in Step 3

Dynamically install the smart camera app package feeds on the running target

AMD provides package feeds in run-time package management (rpm) format for users to dynamically load AMD accelerated applications on top of running a Linux starter image using dnf package manager utility command “dnf install”. AMD also provides the xmutil utility (xmutil getpkgs) to search the package feed and query the available accelerated applications package groups for the Vision AI Starter Kit. Commands are given below:

1) Run the below command to get the list of available application package groups.
 
    sudo xmutil getpkgs

 

2) Run the below command to install Smart Camera accelerated application package group from the above listing. Press “y” when prompted and wait for approximately two minutes for 204 packages to install.

    sudo dnf install packagegroup-kv260-smartcam.noarch

Load Smart Camera accelerated application firmware

Prior to executing the newly installed application available at /opt/xilinx/bin, the FPGA firmware (PL bitstream + device tree overlay + PL drivers) needs to be loaded using xmutil utility commands. Follow the below steps to load Smart Camera accelerated application firmware on the Vision AI Starter Kit.

1) Run the below command to list the existing application firmware available on the Vision AI Starter Kit.

    sudo xmutil listapps

 

2) Run the below command to unload default “kv260-dp” application firmware.

    sudo xmutil unloadapp

 

3) Run the below command to load Smart Camera accelerated application firmware.

    sudo xmutil loadapp kv260-smartcam

Run the Smart Camera accelerated application.

Place the AR1335 camera module pointing to the users face and run the Smart Camera accelerated app. To run the accelerated app with the IAS camera module, use the following command:

    sudo smartcam --mipi -W 1920 -H 1080 -r 30 --target dp

 

We have successfully launched the Smart Camera accelerated application! A bounding box should appear around the face of the user in the output video displayed on the DisplayPort/HDMI monitor.

Alternatively, if you’re using a USB webcam, place the recommended USB webcam pointing to the users face and run the following command:

    sudo smartcam --usb 0 -W 1920 -H 1080 -r 30 --target dp

 

Note: The argument 0 for “--usb” depends on which media node the USB webcam was detected by running Linux on Vision AI Starter Kit. In this case, it was /dev/media0, so we used “--usb 0.

You can also control the Smart Camera application from a Jupyter notebook. Once the Linux boot is complete, launch the Jupyter notebook by entering the IP address of the SOM into your browser. Step through the cells of the notebook to exercise the Smart Camera functionality of the Kria SOM.

For more details about the Smart Camera accelerated application and customization options, visit the GitHub page

< Previous Step
Next Step >