The CTLE in GTX DFE can be used in two ways: auto adapting and fixed.
The GTX IBIS-AMI model default is set manually, not adapting CTLE.
Especially in a case of low insertion losses, you should check if the default RX settings for AGC are saturating to minimum code quickly, which could suggest that RX input is too large.
The first hint in cases like this is to reduce the TX swing.
In the case of low insertion losses, RX input amplitude is already very large andyou do notneed extra DC gain. The default value for CTLE3 in AMI model is 0, which gives the max DC gain. By setting RX_DFE_KL_CFG2[26:23] to 4b0111, you are setting CTLE3=7 and providing lower DC gain. Since there are too many combinations of CTLE1/2/3 settings, you would need to edit the receiver AMI file to set CTLE3 in this case.
CTLE3 is always controlled manually, there is no auto mode for it. There aretwoways to freeze CTLE1/2:
Summary table:
Mid/High Frequency Boost = CTLE1 and CTLE2
Wide-Band Gain = CTLE3
CTLE1 = RX_DFE_KL_CFG2[8:5]
CTLE2 = RX_DFE_KL_CFG2[17:14]
CTLE3 = RX_DFE_KL_CFG2[26:23]