Various flavors of video and audio encoders, decoders and transcoders are found throughout the broadcast and pro AV delivery chain from acquisition to display. Compression codecs are implemented right at the camera sensor, with codecs now accepting RAW formats. They are used as intermediate or mezzanine codecs in post-production, remote production and contribution, cinema, and AV-over-IP. They’re also found in final distribution to the consumer, with lossy OTT streaming and DTH distribution encoding. In any use case, it is a fine balance of trade-offs to be made between power, cost, form factor, video quality, latency and channel density. Xilinx adaptable platforms can support and evolve with most existing and emerging codecs, with integrated codec units, IP cores from our Alliance partners, and hardware acceleration for cost-effective real-time implementations all the way to 8K.
H.264/AVC/MPEG-4 Part 10 is a digital video codec standard from the Joint Video Team made up of ITU-T and MPEG. H.264 aims to offer half the bitrate of MPEG-2 for the same image quality and is widely used in various HDTV distribution standards, IP cameras, PCs and web-based streaming platforms and mobile phones. H.265/HEVC is a newer video coding standard from ITU-T aimed at improved encoding quality further, with another 50% bitrate target for the same quality as H.264. H.265 also supports 4K and 8K UHD video. Recently many new codecs have been introduced, such as VVC and LC-EVC as just two examples, however AVC and HEVC are deeply established industry “workhorse” codecs that will remain entrenched and viable for years to come.
Xilinx offers a unique platform with the Zynq UltraScale+ MPSoC to implement 4K60 (or multichannel SD/HD) simultaneous encode and decode at 4:2:2 10-bit broadcast quality. Leveraging its industry leading low latency, it is ideal for a wide range of contribution and professional streaming applications, or high-density multichannel OTT encoders. The codec is highly configurable and can enable trade-offs between bitrate, quality, latency and power.
JPEG 2000 is a wavelet-based image coding system that offers high scalability and accessibility. Content can be coded once at any quality, up to lossless, but accessed and decoded at many other qualities and resolutions or by region of interest. It also offers graceful degradation, so will blur rather than introduce blocky artifacts like DCT-based codecs, as well as handle multi-generation encode/decode passes. Standardized by ISO in 2000, the standard continues to evolve, with later versions including ultra-low latency (ULL) and high-throughput (HT) modes. JPEG 2000 is the mandatory compression format for Digital Cinema, as well as widely used in archiving, contribution (with ST 2022 AV-over-IP and defined by VSF TR-01), and pro AV IP networking where 4K60 can be transported over a 1GE link. Implementing JPEG 2000 on Xilinx platforms enables support for 4K and 8K in single devices, previously not possible with standard ASSPs. Using Xilinx platforms with our partner’s award-winning IP cores, you can encode or decode JPEG 2000 images and video up to 8K60 4:4:4 10-bit with unrivalled quality, high-speed and compact footprint.
Due to increasing resolutions (4K/8K), higher frame rates, 360° capture and higher quality pixels (e.g. HDR), all devices and infrastructures must handle ever increasing data volumes. Whether AV-over-IP, virtual reality, gaming, live production, remote production or digital cinema workflows, the JPEG XS mezzanine codec standard can be applied wherever uncompressed video is currently used. As a lightweight image coding system, it also offers line-based latency with compression ratios from 2:1 to 12:1 (or even more) while maintaining lossless quality. Available from intoPIX, Xilinx partner and pioneer of JPEG XS, the small footprint means it can be targeted to virtually any Xilinx platform, and being adaptable offers futureproofing as the JPEG XS standard evolves.
The TicoXS FIP codec from partner intoPIX, is engineered to enable the best AV distribution standards and workflows in 4K and 8K. It enables transport of 4K & 8K over IP using Cat5e cables (1GbE) or wireless links (WiFi-6, 60Ghz,..), with perfect quality, zero latency and low footprint on Xilinx platforms. Combined with the support of JPEG XS, the FIP coding profile offers additional coding efficiency. It maintains lossless quality on complex desktop and natural content with line-based latency, high compression ratios, and error concealment for packet losses. Main applications include wireless AV & displays, KVM, AV-over-IP (e.g. IPMX) and gaming. Delivering the best performance in terms of quality, reliability, and user experience, users can now build more sustainable AV products and workflows.
The Colibri codec from Xilinx partner Audinate, was developed specifically for AV-over-IP and for flawless distribution and delivery of high-quality complex graphical content, including text, spreadsheets and animations. Colibri has been optimized for what is to be the main use case of professional AV: the delivery of 4K60 quality over 1GE and 2.5GE. At the same time, it is future-proof and ready to support 8K AV streaming. Besides impeccable image quality, the Colibri codec guarantees a near-zero latency of a mere 24 video lines making it ideal for KVM or control room applications. Due to its small footprint, Colibri can be implemented on almost all Xilinx platforms.
Apple ProRes™ is one of the most popular codecs used in professional post-production. Supporting various 4:2:2 and 4:4:4 formats up to 4K and 5K, Apple ProRes codecs are aimed at various bitrates and video quality for different use cases in offline media workflows.
Xilinx partner IBEX Technologies provides a range of Apple ProRes encoder and decoder IP cores and are on the Apple authorized product list to provide FPGA-based implementations.
The BBC launched Dirac in 2005, an open and patent-free wavelet-based video codec which was later standardized by SMPTE as VC-2. Audinate’s VC-2 HQ IP implements the VC-2 codec defined in SMPTE ST 2042-1:2017 and provides all the supporting mechanisms for handling the compressed bitstream. With no need for external DDR and a tiny footprint, VC-2 can be implemented on almost all Xilinx devices, including those in our cost-optimised portfolio. VC-2 HQ enables an efficient way to lightly compress professional video with extremely low latency and high video quality, particularly useful for moving 8K over 10GE, or 4K over 1GE/2.5GE for instance.
Launched in 2020, the baseline profile of MPEG-5 Part 1 EVC (Essential Video Coding) has been built as a royalty-free option for content producers, whilst still providing an enhanced tool set (subject to royalties) in the Main profile for higher compression performance.
In parallel, MPEG-5 Part 2 LCEVC (Low Complexity Enhancement Video Coding) aims to increase compression efficiency for existing codecs by using a base bitstream and an enhancement bitstream. An LCEVC codec’s output is essentially an enhancement layer which, when combined with video encoded with a separate codec (AVC, HEVC etc), produces an enhanced video stream with improved video quality.
VVC (Versatile Video Coding) is the new H.266 coding standard which has a goal of 30%-50% improvement in compression efficiency over HEVC. VVC is aimed at better for streaming UHD content in 4K, 8K and even 16K, along with improved support for High Dynamic Range (HDR) and 360° video.
EVC, VVC and LCEVC are still brand new and will take time to establish, but Xilinx partner V-Nova was instrumental in the development of the LCEVC standard and have already built a containerized app that can be evaluated and deployed on Xilinx Alveo PCIe accelerator cards today.
Xilinx platforms enable engineers to rapidly integrate audio functionality into their products, offering differentiation, channel density and fast time-to-market. Designers can take full advantage of the DSP performance, bandwidth, and features of Xilinx platforms to implement system-on-chip designs that eliminate the need for separate components to perform audio processing tasks, thereby reducing costs, particularly for multi-channel audio applications. Xilinx offers cost-effective and flexible alternatives to ASSPs and DSPs, along with a world-class partner ecosystem providing audio and speech codecs, echo cancellers and more. Alliance partner Ittiam offers a range of audio codecs from AAC and USAC to MP3 and WMA that run on ARM processor subsystems and are therefore ideal for integrated AV designs based on the Xilinx Zynq-7000 SoC or Zynq UltraScale+ MPSoC.