UPGRADE YOUR BROWSER

We have detected your current browser version is not the latest one. Xilinx.com uses the latest web technologies to bring you the best online experience possible. Please upgrade to a Xilinx.com supported browser:Chrome, Firefox, Internet Explorer 11, Safari. Thank you!

Diagnostic and Clinical Endoscopy Processing

Deliver intelligence and best performance-per-watt
on enhanced vision systems for
surgical and GI video endoscopes

Overview

Advances in preventive medicine and minimally invasive surgeries are driving the need for enhancements in endoscope systems. Traditional multichip implementations or general-purpose GPUs introduce latency which is counter-productive to the low-latency, high-bandwidth computationally intensive functions required in modern endoscopy.

A typical endoscopy system consists of an endoscope with a light source and a camera head, a camera control unit or CCU and an image management unit.

The camera head is connected to a light source. The endoscope with the camera and light source is inserted to take images of the concerned area.

In the Camera Control Unit or the CCU, after performing image signal processing, the raw uncompressed images are captured and displayed real-time or near-real-time on one or more display monitors. The video or images are compressed and stored either in a DVR or other storage medium in various formats for later use and record keeping.

In a diagnostic GI video endoscope, the latency requirement is 50-150 milliseconds for an image to be captured, transferred, pre-processed and displayed. Surgical procedures using endoscopes needs a real-time or close-to real-time response. Various picture correction, color and noise corrections take place here followed by edge enhancement and scaling.

Typically, the image management unit will perform video processing like encode and decode, jpeg conversion, do picture overlay, manipulation, picture-in-picture and other functions and compress and store the images for possible future use.


Design Example

This design example discusses architecting a 4K video endoscope with Xilinx solutions. As mentioned above, an endoscopy system typically has a processing system consisting of a light source and a camera head, a camera control unit or CCU and an image management unit. This system supports a dual 4K video stream.

The dual-4K image sensor on the camera head performs the image capture and the image signal processing is done in a Xilinx FPGA/SOC.

The video streams are then fed to a one or more cascaded FPGAs-SoCs like Xilinx UltraScale or UltraScale+ devices to perform image pre-processing and the streams are displayed near real-time using a DisplayPort or quad SDI interface to high definition 4K monitors for the surgeon to view. The Camera Control Unit performs picture, color and noise corrections including white balance, automatic brightness, gain control, IRIS control etc. Some of the typical pre-processing functions are shown here. There are different ways to partition the pre-processing functions depending on what FPGAs are used and what functions can be performed on the raw images.

Using Xilinx SoC-FPGA device like Zynq Ultrascale+ provides several distinct advantages:

  • Low latency, high bandwidth of the FPGA-SoC provides increased system performance with hardware acceleration for video analytics such as lens distortion correction, low latency rotation, point of view translation, object identification, comparison, counting, recognition, positioning and measurement
  • Integration of single-chip system processor and image processing reduces BOM cost and power consumption
  • Optimal hardware/software partitioning reduces latency and bottlenecks by combining image/video processing, analytics and camera control into a single device
  • Support for multiple connectivity standards – Quad SDI, HDMI, DisplayPort, SATA, USB, Ethernet and more

As part of post processing, various image management functions are performed on the raw video using an SOC like ZU+ MPSOC with built-in video codec engines. Xilinx solutions today have significant platform opportunities to be major part of a video endoscopy processing system.

  • 4K60 video encoder for network-based video transfer
  • Accelerate design productivity with a tightly integrated development tool flow using OpenCV, Vivado™ High-Level Synthesis, and IP Integrator tools
  • Reduce time to integration with specialized ecosystem IP and software, and operating systems

Typically, FPGAs can produce much higher performance, bandwidth and real-time capabilities for image pre-processing functions over an ASSP device. GPUs are widely used for image processing at the back-end, but Xilinx FPGAs can be significantly lower power than using GPUs.

Additionally, Xilinx is focused on reliability, safety, security and long-life support necessary for clinical equipment unlike competing GPGPUs or processors.

Documentation
Page Bookmarked