During auto-negotiation the elastic buffer might under or overflow, but the Auto-Neg state machine in the EMAC should be aware of when this occurs and will temporarily freeze until the GT11 receiver PCS is reset and has recovered. The RXBUFERR pin from the GT11 is not wired up correctly to the EMAC in the v4.1 wrappers and can cause auto-negotiation to fail.
A patch is available to fix this issue. See (Xilinx Answer 22332) to download the patch.
Currently PHYEMAC#RXBUFSTATUS[1:0] is connected to the GT11 RXSTATUS[1:0]. To fix this, route RXBUFERR from the GT11 in question to the PHYEMAC#RXBUFSTATUS input port of the Virtex-4 EMAC. PHYEMAC#RXBUFSTATUS is unconnected in the EMAC and can be grounded.