Memory issues
RAM issues
Q: I get a message "ERROR: G_realloc: unable to allocate 52980000 bytes of memory at lib/vector/diglib/cindex.c:113" or similar? What's up?
A: Is your local GRASS GIS built for 32-bit (x86) or 64-bit (x86-64 aka amd64)? Running the "file" command (Unix systems only) on e.g. r.contour will tell you:
# run next line in a GRASS GIS session: file `which r.contour`
A 32-bit process can't use more than 4 GiB of memory regardless of how much RAM or swap is available. The amount available to malloc() etc will be somewhat lower due to reservations for the kernel, stack, etc.
If you run a 64bit GRASS GIS version on a 64 bit system, then check Large raster data processing#Troubleshooting. Otherwise use less data or get a 64 bit system...
Memory limits on MS-Windows
- Please refer to: https://msdn.microsoft.com/en-us/library/aa366778.aspx
Adding temporary swap space in Linux
thanks to Gordon Keith for these tips
To monitor your swap usage you can use utilities like top, xosview etc
(they're the ones I use, but there are many others that will do the
job).
If you are running out of swap space it is fairly easy to create a swap file and add it to your current swap area:
su dd if=/dev/zero of=/var/swapfile bs=1024 count=2048000 mkswap /var/swapfile swapon /var/swapfile
Will create a 2GB (approx) swapfile called /var/swapfile, format it as swapfile and add it to the swap space.
The file can reside anywhere on the local file system, so long as it it created using the preceding commands. On a faster disk is of course better for performance. Swapfiles bigger than 2GB only work on recent kernels, so if you have an older system I'd recommend sticking with 2GB or less and use more than one if you want more than 2GB extra swap space. (I used 2x2GB swapfiles and a 1.5GB swap partition when I was playing with s.surf.idw).
If you no longer need the swap space but need the disk space you can free it with:
su swapoff /var/swapfile rm /var/swapfile
The swapoff command will only work when there are no pages currently in the swapfile. See man pages for mkswap, swapon and swapoff for details.
Limiting process memory usage in Linux
On Linux using bash it can be useful to limit the amount of memory any process can have so that if you get the region size wrong the job dies rather than tying up the machine.
For example my .bashrc on our shared server has:
ulimit -S -v 20000000
This sets the soft limit for virtual memory to around 20GB which is reasonable for a machine with 32GB RAM and 64GB swap. I use a soft limit so that if I have a job that needs more I can up the limit. But generally if a module wants more than 20GB something is wrong.
It doesn't solve all out-of-memory problems, but may make it less painful. There is probably something similar on other platforms, but the mechanics may differ (eg under csh: limit vm 20000m).