AR# 9244: 6.1i CORE Generator - How do I increase/decrease heap space for COREGen after "Fatal," "Out of Memory," or "Initialization of the Dynamic Link Library...failed" errors are reported?
AR# 9244
6.1i CORE Generator - How do I increase/decrease heap space for COREGen after "Fatal," "Out of Memory," or "Initialization of the Dynamic Link Library...failed" errors are reported?
Description
Keywords: 2.1i, 3.1i, 4.1i, 5.1i
Urgency: Standard
General Description: 1. How do I increase the amount of heap space for CORE Generator after "Out of Memory" or "FATAL" errors are reported?
See Solution 1.
2. How do I decrease the amount of heap space for CORE Generator after the following error is reported?
"java.exe - DLL Initialization Failed Initialization of the Dynamic Link Library C:\WINNT\System32\ddraw.dll failed. The process is terminating abnormally."
See Solution 2.
Solution
1
A "FATAL" error message can be caused by CORE Generator running out of memory while generating large cores (such as a DA FIR with 8 channels and 32-bit coefficients).
If the CORE Generator application startup script is modified to have access to more than the default 1024 MB of system memory (which equals the physical plus the virtual memory), the core generation has a better chance of succeeding.
The actual amount of memory available to the CORE Generator application might need to be to anywhere from 512 to 1024 MB for very large DA FIR Cores. Large Virtex-II Cores can take up to 4096 MB.
Here are some possible values that you can use: -Xmx512m, -Xmx768m, -Xmx2048m, or -Xmx4096m
NOTE: The total of the physical memory plus the virtual memory must be greater than or equal to the command line memory setting for CORE Generator. Setting the "-mxm" option to a larger value also ties up this amount of memory for CORE Generator use during your CORE Generator session. Other applications will not be able to utilize this memory until after the CORE Generator application is terminated.
The following examples illustrate the process for changing the available memory setting in CORE Generator to 1024 MB:
PCs: Modify "%XILINX%\bin\nt\coregen.bat", changing the "-Xmx384m" option in the last line of this batch file to "-Xmx1024m":
The following error indicates that Java was not able to allocate enough memory to generate the specified heap size:
"java.exe - DLL Initialization Failed "Initialization of the Dynamic Link Library C:\WINNT\System32\ddraw.dll failed. The process is terminating abnormally."
The maximum heap size appears to be machine-dependent. If you are generating large cores, Xilinx recommends that you use Solution 1 to set the default heap size. If this is still too large, continue decreasing the -mx1024m value until you are able to launch CORE Generator. (This is usually possible at about 512m.)