Healthcare AI at the Edge - Machine Learning & Healthcare Analytics

Build your appliance for radiological, diagnostic and clinical applications at the Healthcare IoT edge and achieve the best performance per watt at lowest total cost of ownership


Xilinx machine learning (ML) inference enables the early detection of critical ailments by identifying anomalies in X-rays, ultrasound, digital pathology, dermatology, ophthalmology, and more. Other applications include surgical tool guidance, drug discovery, and genome analysis. Xilinx and its partner ecosystem can deliver significant advancements across a wide array of healthcare applications and design methodologies.

Healthcare AI Reference Design and Open Model for Edge and Cloud

Healthcare IoT is rapidly accelerating the opportunity for cloud-connected clinical, diagnostic, and radiological equipment asset and collaborative control systems to decipher their full capabilities using machine learning. Hospital administrators, IT, service providers, and medical equipment makers realize the benefits and understand the need for integrated edge to cloud solutions that will accelerate their time to market.

Xilinx,, and AWS IoT services have a fully functional Healthcare AI reference design Kit, and an example X-Ray detection model with incredibly high accuracy and low output latency running on the Zynq® UltraScale+™ MPSoC integrated on the ZCU104 platform as an Edge device. They are developed using PYNQ™, an open-source Python programming platform for the Xilinx Zynq architecture, and the AWS Lambda function that makes this integration easily adaptable for other clinical platforms.
The Xilinx deep learning processing Unit (DPU) integrated into the MPSoC accelerates the convolutional neural network (CNN) within the AWS IoT Greengrass. High performance at the edge combined with cloud scalability enables this solution to be available anywhere as a clinical or as a point-of-care (POC) solution. The solution can also be easily integrated with any existing healthcare applications at a large scale as a federated learning platform.

AI Toolkit for Vitis AI

Xilinx latest AI Toolkit Vitis™ AI version 1.1 was used to compile the deep learning models for running accelerated inference, making this solution very cost-effective.

Healthcare AI Starter Kit

Xilinx and have developed a smart and scalable solution for Pneumonia and COVID-19 prediction system using Vitis-AI and AWS IoT Greengrass with Xilinx ZCU104 FPGA board as the edge device.

X-Ray Images using Vitis-AI leveraged the real-time capabilities and image processing features of the Zynq UltraScale+ MPSoC to implement Pneumonia and Covid19 detection models, which is useful for understanding degree-of-infection, and for generating visual heatmaps.

VITIS AI Platform

The Vitis AI development environment is Xilinx’s development platform for AI inference on Xilinx hardware platforms, including both edge devices and Alveo™ cards. It consists of optimized IP, tools, libraries, models, and example designs. It is designed with high efficiency and ease of use in mind, unleashing the full potential of AI acceleration on Xilinx FPGA and ACAP.
Xilinx has developed a complete end-to-end flow, allowing software developers, hardware developers, and data scientists to leverage the existing machine learning ecosystem. In this paradigm, we have designed tools to enable our customers to directly parse the model graph and trained weights saved from popular ML frameworks.

PYNQ – Python on Zynq

Python powered edge analytics and machine learning is enabled by the "PYNQ" platform. PYNQ is a software-hardware framework for the Xilinx Zynq SoCs, leveraging the programmable hardware to pre-process sensors and other data types to make software analysis highly efficient in an embedded processor.  The PYNQ platform supports all major Python libraries like Numpy, Scikit-Learn, Pandas, and others.

Partner Solutions

Solution Provider
Description Supported Devices Pneumonia and COVID-19 Detection from X-Ray Images Zynq UltraScale+
Amazon Web Services (AWS) Xilinx Zynq UltraScale+ Healthcare AI Starter Kit Zynq UltraScale+

Links and Reference

Solution Provider Description Supported Devices
Xilinx - Vitis Unified Software Platform All Xilinx Platforms
Xilinx - Vitis AI Adaptable and Real-Time AI Inference Acceleration
AI Model Zoo GitHub
All Xilinx Platforms
Xilinx - PYNQ PYNQ Homepage
PYNQ Community Projects
Zynq UltraScale+
AWS IoT AWS Certified Xilinx Products
Xilinx – AWS Workshop
Zynq UltraScale+
Xilinx for Healthcare Smart Solutions for Healthcare: Imaging, Diagnostics, and Clinical Equipment All Xilinx Platforms
Additional Resources

Xilinx Unleashes the Power of Artificial Intelligence in Medical Imaging
The use of artificial intelligence (AI) – including machine learning (ML) and deep learning techniques (DL) is poised to become a transformational force in medical imaging.
Read More >


Analytics to X-Rays Unchained: Adaptive AI for Healthcare At The Edge
During this two-part webinar series, we will address the importance of in situ, in silico, inference for Healthcare.

Vitis AI ModelZoo Healthcare Models

2D Endoscopy Multi-Class Segmentation

Dataset: EDD2020
Model: Xilinx custom Feature Pyramid Network with ResNet18 feature extractor and multiple prediction heads
Image: Results image from our algorithm
Model: Download
Accuracy:  Dice = 80.45%, F2-score=79.15%
Performance: ZCU102 79ms latency, 40fps

Vitis™ AI Skin Lesion Classification Tutorial

Dataset: HAM10000

The Skin Lesion Tutorial

Model: View on Github