The above message means that the stacksize that is currently set is not sufficient for XST to complete the synthesis process.The stacksize should be changed to unlimited by using the following commands:
if the shell is sh, bash or ksh use:
ulimit -s unlimited
if the shell is csh, tcsh or zsh use:
limit stacksize unlimited
Run the following command on the Linux/Unix shell for checking the stacksize:
limit
The output of the above command should resemble the following message:
stacksize unlimited
Now, run XST again and this should fix the FATAL ERROR message. If for some reason there is a restriction on the stacksize or XST is running on Windows, then add the following lines to the xst file and run XST again:
set -checkcmdline no
set -use_limited_stack 1
Setting the above variables alters the behavior of XST and can degrade QoR.
The recommended flow is to increase the stacksize.