Responsive and Reconfigurable Vision Systems
Leading system developers are using All Programmable Devices in next generation vision guided machine learning systems. To accelerate productivity, Xilinx has created the reVISION Zone to aggregate useful resources for software, hardware and system developers.
For developers who wish to share their reference designs, libraries, and experience, we have also included a section with community projects.
Begin today by exploring this zone and get started building responsive and reconfigurable vision guided systems.
Differentiating Advantages
More Responsive than typical
SoCs & Embedded GPUs
Reconfigurable to the Latest
Algorithms and Sensors
Software Defined &
Improved Ease of Use
reVISION Enables Responsive and Reconfigurable Vision Systems
More Responsive than Typical SoCs & Embedded GPUs:
- 6X better images/sec/Watt in machine learning
- 42X higher frames/sec/Watt for computer vision processing
- 1/5th the latency
Reconfigurable to the Latest Algorithms and Sensors:
- Continue to upgrade to the latest machine learning algorithms
- Support the latest sensor types and connectivity standards
- Support up to 8K and custom resolutions
Software Defined & Improved Ease of Use:
- Accelerate development with ready to use OpenCV libraries
- Leverage any combination of C/C++ and OpenCL languages
- Develop with popular Machine Learning frameworks including Caffe
Be among the first to be notified of Machine Learning news and updates from Xilinx.
View our Customers and Partners who endorse the reVISION Stack.
Featured Videos
Zynq® All Programmable SoCs and MPSoCs
Hundreds of Xilinx’s Embedded Vision customers target Zynq® All Programmable SoCs and MPSoCs in addition to FPGAs.
Zynq-based Platforms Enable:
- Acceleration of computer vision and machine learning algorithms for fast system response
- Uniquely provide the reconfigurability required for rapid upgrade to the best available type and mix of sensors
- Enable any-to-any connectivity to new machines and/or the cloud
To address the challenges mentioned above, Xilinx provides the reVISION stack which includes a broad range of development resources for platform, algorithm and application development.
This includes support for the most popular neural networks, including AlexNet, GoogLeNet, SqueezeNet, SSD, and FCN, the functional elements required to build custom neural networks (CNN/DNN), and leverage pre-defined and optimized CNN implementations for network layers. This is complemented by a broad set of acceleration-ready OpenCV functions for computer vision processing.
For application level development, Xilinx supports popular frameworks including Caffe for machine learning and OpenVX for computer vision (to be released in second half 2017). The reVISION stack also includes development platforms from Xilinx and ecosystem partners based on Zynq SoCs and MPSoCs.
The reVISION stack enables design teams without deep hardware expertise to use a software defined development flow to combine efficient implementations of machine learning and computer vision algorithms into highly responsive systems.
The reVISION flow starts with a familiar, eclipse-based environment using C, C++ and/or OpenCL languages and associated compiler technology; this is called the SDSoC environment.
Within the SDSoC environment, software and systems engineers can target reVISION hardware platforms, and draw from a pool of acceleration-ready computer vision libraries, and/or the OpenVX framework (late Summer 2017), to quickly build new applications.
For machine learning, popular frameworks like Caffe are used to train a neural network. The Caffe generated .prototxt file is run on an ARM® based scheduler that drives inference processing on pre-optimized implementations of CNN network layers.
Expert Xilinx users deploying traditional RTL-based design flows, working with ARM based software developers, spent considerable design time creating highly differentiated machine learning and computer vision applications.
To further speed design time and reduce the reliance on hardware experts, Xilinx introduced the SDSoC Development Environment, based on C, C++ and OpenCL. While this significantly reduces development cycles, it is not domain specific for Embedded Vision.
Xilinx’s new reVISION stack enables a much broader set of software and systems engineers, with little or no hardware design expertise, to develop intelligent Embedded Vision systems easier and faster.
Get started today designing your computer vision system around Zynq SoCs/MPSoCs and FPGAs by leveraging existing Xilinx and ecosystem design hardware, modules and production-ready Systems on Module (SOMs).
Join the discussion on Xilinx forums.
Computer Vision
Introducing Xilinx’s all new library for computer vision, xfOpenCV , accelerating the most critical OpenCV functions. xfOpenCV will allow you to easily compose and accelerate computer vision functions in the FPGA fabric through SDx or HLx environments. In addition, xfOpenCV library is consistent with OpenCV and are optimized for performance, resource utilization and ease of use. xfOpenCV is available to the public on github here: https://github.com/Xilinx/xfopencv
OpenCV library functions are essential to developing many computer vision applications. Xilinx’s xfOpenCV for computer vision, based on key OpenCV functions, will allow you to easily compose and accelerate computer vision functions in the FPGA fabric through SDx or HLx environments. In addition, xfOpenCV library functions are consistent with OpenCV and are optimized for performance, resource utilization and ease of use.
- Thousands of functions in the OpenCV 3.1 library are available to run on the ARM Cortex™-A9 and Cortex A53 cores in Zynq
- ~45 OpenCV functions (the OpenVX subset) are available as a library of RTL optimized functions for Xilinx SoCs
- Complete library user guide with device utilization and performance
- Support for 1 and 8 pixel parallel versions is available for most functions
reVISION Design Flow for Computer Vision
Library Functions
The functions are grouped into three levels, from simple (left) to more complex (right).
| Level 1 | Level 2 | Level 3 |
|---|---|---|
| Absolute difference | Channel combine | Box |
| Accumulate | Channel extract | Gaussian |
| Accumulate squared | Color convert | Median |
| Accumulate weighted | Convert bit depth | Sobel |
| Arithmetic addition | Table lookup | Custom convolution |
| Arithmetic subtraction | Histogram | |
| Bitwise: AND, OR, XOR, NOT | Gradient Phase | Dilate |
| Pixel-wise multiplication | Min/Max Location | Erode |
| Integral image | Mean & Standard Deviation | Bilateral |
| Gradient Magnitude | Thresholding |
Get started today designing your computer vision system around Zynq SoCs/MPSoCs and FPGAs by leveraging existing Xilinx and ecosystem design hardware, modules and production-ready Systems on Module (SOMs).
Join the discussion on Xilinx forums.
Machine Learning
Machine learning and deep learning have gained attention from the development community as a technique that provides enhanced intelligence to many applications including Embedded Vision. While not a new discipline, relatively new breakthroughs in algorithms, access to large data sets for algorithm training and efficient and economically more viable computing platforms have resulted in very rapid interest and adoption of the technology.
Xilinx’s Zynq SoCs/MPSoCs are an ideal fit for machine learning, achieving 6X better images/sec/Watt in machine learning inference relative to embedded GPUs and typical SoCs. Xilinx’s reVISION Stack removes traditional design barriers by allowing you to quickly take a trained network and deploy it on Zynq SoCs and MPSoCs for inference.
Features:
- Full software stack for deploying machine learning applications
- Hardware optimized, and supported layers include: Conv, ReLU, Pooling, Dilated conv, Deconv, FC, Detector & Classifier, SoftMax layers
- Caffe inter-operability allows easy porting from prototxt files for Network definition and trained weights
- Optimized reference models available for a wide range of network topologies, such as AlexNet, GoogLeNet, SqueezeNet, FCN and SSD
- Networks can be customized through software running on the ARM processor without lengthy compilation
Deploying Networks
Get started today designing your computer vision system around Zynq SoCs/MPSoCs and FPGAs by leveraging existing Xilinx and ecosystem design hardware, modules and production-ready Systems on Module (SOMs).
Join the discussion on Xilinx forums.
Connectivity & Sensor Support
The AI revolution has accelerated the development and evolution of sensor technologies across numerous categories. It has also resulted in a mandate for a new level of sensor fusion, combining multiple types of sensors in different combinations to create a full and complete view of the system’s environment and objects in that environment. Whatever sensor configuration is specified today, or implemented tomorrow, needs to be ‘future proofed’ through hardware reconfigurability. Only Xilinx All Programmable devices offer this level of reconfigurability.
Zynq-based vision platforms offer robust any-to-any connectivity and sensor interfaces. Zynq sensor and connectivity advantages include:
- Up to 12x more bandwidth relative to alternative SoCs currently in the market, including support for native 8K and custom resolutions
- Significantly more high and low bandwidth sensor interfaces and channels, enabling highly differentiated combinations of sensors such as RADAR, LiDAR, accelerometers and force torque sensors
- Industry leading support for the latest data transfer and storage interfaces, easily reconfigured for future standards
Get started today designing your computer vision system around Zynq SoCs/MPSoCs and FPGAs by leveraging existing Xilinx and ecosystem design hardware, modules and production-ready Systems on Module (SOMs).
Be among the first to be notified of Machine Learning news and updates from Xilinx.
Join the discussion on Xilinx forums.
Design Examples and Demos for Computer Vision and Machine Learning
The reVISION Stack includes five computer vision design examples and five machine learning demos, with more to come. These examples are intended to get you up-and-running in a very short period of time. These design examples will help you easily see the distinct advantage Xilinx All Programmable SoCs have in high performance Embedded Vision applications.To access these examples, follow the reVISION Getting Started
Computer Vision Design Examples
| Design Example Provided by Xilinx | Latest SDSoC Version Supported | Board & SOM Supported | Provider |
|---|---|---|---|
| LK Dense Optical Flow iterative and pyramidal based implementation doing motion segmentation |
2017.1 | ZCU102, ZC702, ZC706 | Xilinx |
| Stereo Disparity Map Calculates disparity map from two sensor inputs using local block matching |
2017.1 | ZCU102, ZC702, ZC706 | Xilinx |
| Warp Transform | 2017.1 | ZCU102, ZC702, ZC706 | Xilinx |
| Harris Corner | 2017.1 | ZCU102, ZC702, ZC706 | Xilinx |
| Bilateral Filter | 2017.1 | ZCU102, ZC702, ZC706 | Xilinx |
Machine Learning Demos
| Design Example & Descriptions | Latest SDSoC Version Supported | Board & SOM Supported | Provider |
|---|---|---|---|
| GoogLeNet | 2017.1 Download Package | ZCU102 | Xilinx |
| AlexNet | Xilinx | ||
| VGG-16 | Xilinx | ||
| SSD-300 | Xilinx | ||
| FCN-AlexNet | Xilinx |
Get started today designing your computer vision system around Zynq SoCs/MPSoCs and FPGAs by leveraging existing Xilinx and ecosystem design hardware, modules and production-ready Systems on Module (SOMs).
Join the discussion on Xilinx forums.
| Resource | Description |
|---|---|
| Papers & Tutorials | A collection of application notes, whitepapers, tutorials and user guides |
| Xilinx Embedded Vision Videos | Various demonstrations and videos on embedded vision |
| Xcell Daily and Featured Blogs | Daily blog articles from Xilinx and Industry |
| Powered By Xilinx | Showcase of Products Enabled by Xilinx Technology |
| Forums | Xilinx Community Forums |
Get started today designing your computer vision system around Zynq SoCs/MPSoCs and FPGAs by leveraging existing Xilinx and ecosystem design hardware, modules and production-ready Systems on Module (SOMs).
Join the discussion on Xilinx forums.
