The Xilinx Serial RapidIO data sheet claims 10 Gbps max bandwidth. This is a "peak unidirectional bandwidth" as supported by the logical layer. This bandwidth is inclusive of header. This is the same bandwidth quoted by the RapidIO TA with the understanding that control symbols and packet headers will lower the effective bandwidth. RapidIO as a protocol is very traffic dependent as to the obtainable bandwidth. Our RapidIO Core can operate at this bandwidth; however, it can be throttled by the buffer design.
The buffer is a temporary storage for packets. Flow of traffic is based on the availability of buffer space and packets outstanding. If there is not enough space in the link partner's buffer, the buffer will not transmit packets. Also, if your receive buffer does not have sufficient space, it will not be able to accept packets from a linked device. If the link is operating in receiver controlled flow control, packets will be retried when there is insufficient space for the packets in either direction. Retries further restrict the throughput of the link. The Xilinx buffer reference design delivered with the core has some inefficiencies mentioned below, which might further lower the bandwidth:
- Cannot provide data back-to-back; there is a minimum four-cycle delay.
- The transmitter side buffer allows for only eight possible outstanding packets.
- The receive side buffer can accept only seven outstanding packets. Additionally, as mandated by the RapidIO protocol, some of these need to be reserved for higher priority packets, so not all of them can be used depending on the traffic type.
- The logical layer can only handle store and forward on the receive side. This limitation further eliminates some of those free spaces in the buffer.
Since all customers and data flow requirements are different, Xilinx has provided a bare bones buffer delivered as a modifiable example reference design. If you would like to tailor the buffer to accommodate heavy traffic flow and system needs, you may modify the buffer design yourself or you can contact Xilinx Design Services.
If you are considering buffer redesign, there are two options:
- Increase the capacity of receive buffer (rx.v) to allow for more receive buffer space. By incorporating more receive buffers, you allow the transmitter to continue to send packets as soon as they are available. This assumes that you are able to pull out packets as rapidly as they are put in. However, if the user side lags the buffers will once again fill up and you will again see a degraded bandwidth.
- Redesign receive buffer as above and also redesign transmitter side to allow more than eight outstanding packets, and to allow it to transmit packets back-to-back. With this solution, you might see improvements in the bandwidth in both directions. However, bidirectional traffic necessitates additional control symbols that can potentially affect the bandwidth.
In the future release of Xilinx SRIO core, the buffer design will be redesigned to improve overall bandwidth. This release is not scheduled until late 2008.