Working with GRASS without starting it explicitly

2015-03-20T15:45:38Z

⚠️Osfraid: /* Python: GRASS GIS 7 with existing location */

GRASS modules and imports of GRASS Python packages works only in a specific environmental settings (GRASS session). This settings is ensured by starting GRASS GIS application which prepares this GRASS session for you. It is possible to set up you system in the way GRASS session will be always active (GRASS is not running, just the environmental settings is ready). People often prefer to set up the GRASS environment in their script or program and this is what this article discuss. However, it must be noted that the the generally preferred (and easy) way is to create scripts and programs as GRASS modules which means that your don't have to bother with setting up the environment since the GRASS module should be always invoked only in GRASS session.

== GRASS sessions ==

It is possible to access GRASS modules without explicitly starting a "GRASS session". GRASS libraries require certain [http://grass.osgeo.org/grass70/manuals/variables.html environment variables] to be set. In fact a "GRASS session" is just a set of processes (e.g. a shell and/or GUI) which have the necessary environment settings, specifically:

* '''GISBASE''' needs to be set to the top-level directory of the GRASS installation.
* '''GISRC''' needs to contain the absolute path to a file containing settings for '''GISDBASE''', '''LOCATION_NAME''' and '''MAPSET'''.
* '''PATH''' needs to include '''$GISBASE/bin''' and '''$GISBASE/scripts'''.

If the GRASS libraries are shared libraries, the loader needs to be able to find them. This normally means that '''LD_LIBRARY_PATH''' (Linux, Solaris), '''DYLD_LIBRARY_PATH''' (MacOSX) or '''PATH''' (Windows) need to contain '''$GISBASE/lib''', although there are other means to the same end (e.g. on Linux, putting $GISBASE/lib (with $GISBASE replaced by its actual value) into /etc/ld.so.conf then running ldconfig).

Some libraries and modules use other variables. More information for most of them is available in the file ''$GISBASE/docs/html/variables.html''. The display libraries used by ''d.*'' commands use additional variables, which are documented along with the individual drivers.

=== Batch jobs ===

You can run GRASS scripts non-interactively from outside of a GRASS session with the GRASS_BATCH_JOB environment variable. When GRASS is started with this environment variable set it will automatically run the contents of the script given in the variable, then close the GRASS session when complete. In this way full lock-checking, GRASS variables setup, and temporary file cleaning tasks are still performed.

See the [[GRASS_and_Shell#GRASS_Batch_jobs|Batch jobs]] section of the GRASS shell-help wiki page for details.

=== Accessing GRASS modules without starting a GRASS session ===

==== Python examples ====

See also [[GRASS Python Scripting Library]]

===== Python: GRASS GIS 7 with existing location =====

The script initializes the session and lists available raster and vector maps:

<source lang=python>
#!/usr/bin/env python

import os
import sys
import subprocess

# path to the GRASS GIS launch script
# MS Windows
grass7bin_win = r'C:\OSGeo4W\bin\grass70svn.bat'
# uncomment when using standalone WinGRASS installer
# grass7bin_win = r'C:\Program Files (x86)\GRASS GIS 7.0.0beta3\grass70.bat'
# Linux
grass7bin_lin = 'grass70'
# Mac OS X
# this is TODO
grass7bin_mac = '/Applications/GRASS/GRASS-7.0.app/'

# DATA
# define GRASS DATABASE
# add your path to grassdata (GRASS GIS database) directory
gisdb = os.path.join(os.path.expanduser("~"), "grassdata")
# the following path is the default path on MS Windows
# gisdb = os.path.join(os.path.expanduser("~"), "Documents/grassdata")

# specify (existing) location and mapset
location = "nc_spm_08"
mapset = "user1"

########### SOFTWARE
if sys.platform.startswith('linux'):
# we assume that the GRASS GIS start script is available and in the PATH
# query GRASS 7 itself for its GISBASE
grass7bin = grass7bin_lin
elif sys.platform.startswith('win'):
grass7bin = grass7bin_win
else:
raise OSError('Platform not configured.')

# query GRASS 7 itself for its GISBASE
startcmd = [grass7bin, '--config', 'path']

p = subprocess.Popen(startcmd, shell=False,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
if p.returncode != 0:
print >>sys.stderr, "ERROR: Cannot find GRASS GIS 7 start script (%s)" % startcmd
sys.exit(-1)
gisbase = out.strip('\n\r')

# Set GISBASE environment variable
os.environ['GISBASE'] = gisbase
# the following not needed with trunk
os.environ['PATH'] += os.pathsep + os.path.join(gisbase, 'extrabin')
# add path to GRASS addons
home = os.path.expanduser("~")
os.environ['PATH'] += os.pathsep + os.path.join(home, '.grass7', 'addons', 'scripts')

# define GRASS-Python environment
gpydir = os.path.join(gisbase, "etc", "python")
sys.path.append(gpydir)

########### DATA
# Set GISDBASE environment variable
os.environ['GISDBASE'] = gisdb

# import GRASS Python bindings (see also pygrass)
import grass.script as gscript
import grass.script.setup as gsetup

###########
# launch session
gsetup.init(gisbase,
gisdb, location, mapset)

gscript.message('Current GRASS GIS 7 environment:')
print gscript.gisenv()

gscript.message('Available raster maps:')
for rast in gscript.list_strings(type = 'rast'):
print rast

gscript.message('Available vector maps:')
for vect in gscript.list_strings(type = 'vect'):
print vect
</source>

===== Python: GRASS GIS 7 without existing location using metadata only =====

The script initializes the session, creates a temporary GRASS location and lists available raster and vector maps:

<source lang=python>
#!/usr/bin/env python

# Python script to generate a new GRASS GIS 7 location simply from metadata
# Markus Neteler, 2014

# ?? LINUX USAGE: First set LIBRARY SEARCH PATH
#export LD_LIBRARY_PATH=$(grass70 --config path)/lib
#python start_grass7_create_new_location_ADVANCED.py

# some predefined variables

# Windows
grass7path = r'C:\OSGeo4W\apps\grass\grass-7.0.svn'
grass7bin_win = r'C:\OSGeo4W\bin\grass70svn.bat'
# Linux
grass7bin_lin = 'grass70'
# MacOSX
grass7bin_mac = '/Applications/GRASS/GRASS-7.1.app/'
#myepsg = '4326' # latlong
myepsg = '3044' # ETRS-TM32, http://spatialreference.org/ref/epsg/3044/
#myfile = '/home/neteler/markus_repo/books/kluwerbook/data3rd/lidar/lidar_raleigh_nc_spm.shp'
myfile = '/data/maps/world_natural_earth_250m/europe_north_east.tif'
#myfile = r'C:\Dati\Padergnone\square_p95.tif'

###########
import os
import sys
import subprocess
import shutil
import binascii
import tempfile

########### SOFTWARE
if sys.platform.startswith('linux'):
# we assume that the GRASS GIS start script is available and in the PATH
# query GRASS 7 itself for its GISBASE
grass7bin = grass7bin_lin
elif sys.platform.startswith('win'):
grass7bin = grass7bin_win
else:
OSError('Platform not configured.')

startcmd = grass7bin + ' --config path'

p = subprocess.Popen(startcmd, shell=True,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
if p.returncode != 0:
print >>sys.stderr, 'ERROR: %s' % err
print >>sys.stderr, "ERROR: Cannot find GRASS GIS 7 start script (%s)" % startcmd
sys.exit(-1)
if sys.platform.startswith('linux'):
gisbase = out.strip('\n')
elif sys.platform.startswith('win'):
if out.find("OSGEO4W home is") != -1:
gisbase = out.strip().split('\n')[1]
else:
gisbase = out.strip('\n')
os.environ['GRASS_SH'] = os.path.join(gisbase, 'msys', 'bin', 'sh.exe')

# Set GISBASE environment variable
os.environ['GISBASE'] = gisbase
# define GRASS-Python environment
gpydir = os.path.join(gisbase, "etc", "python")
sys.path.append(gpydir)
########
# define GRASS DATABASE
if sys.platform.startswith('win'):
gisdb = os.path.join(os.getenv('APPDATA', 'grassdata')
else:
gisdb = os.path.join(os.getenv('HOME', 'grassdata')

# override for now with TEMP dir
gisdb = os.path.join(tempfile.gettempdir(), 'grassdata')
try:
os.stat(gisdb)
except:
os.mkdir(gisdb)

# location/mapset: use random names for batch jobs
string_length = 16
location = binascii.hexlify(os.urandom(string_length))
mapset = 'PERMANENT'
location_path = os.path.join(gisdb, location)

# Create new location (we assume that grass7bin is in the PATH)
# from EPSG code:
startcmd = grass7bin + ' -c epsg:' + myepsg + ' -e ' + location_path
# from SHAPE or GeoTIFF file
#startcmd = grass7bin + ' -c ' + myfile + ' -e ' + location_path

print startcmd
p = subprocess.Popen(startcmd, shell=True,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
if p.returncode != 0:
print >>sys.stderr, 'ERROR: %s' % err
print >>sys.stderr, 'ERROR: Cannot generate location (%s)' % startcmd
sys.exit(-1)
else:
print 'Created location %s' % location_path

# Now the location with PERMANENT mapset exists.

########
# Now we can use PyGRASS or GRASS Scripting library etc. after
# having started the session with gsetup.init() etc

# Set GISDBASE environment variable
os.environ['GISDBASE'] = gisdb

# Linux: Set path to GRASS libs (TODO: NEEDED?)
path = os.getenv('LD_LIBRARY_PATH')
dir = os.path.join(gisbase, 'lib')
if path:
path = dir + os.pathsep + path
else:
path = dir
os.environ['LD_LIBRARY_PATH'] = path

# language
os.environ['LANG'] = 'en_US'
os.environ['LOCALE'] = 'C'

# Windows: NEEDED?
#path = os.getenv('PYTHONPATH')
#dirr = os.path.join(gisbase, 'etc', 'python')
#if path:
# path = dirr + os.pathsep + path
#else:
# path = dirr
#os.environ['PYTHONPATH'] = path

#print os.environ

## Import GRASS Python bindings
import grass.script as grass
import grass.script.setup as gsetup

###########
# Launch session and do something
gsetup.init(gisbase, gisdb, location, mapset)

# say hello
grass.message('--- GRASS GIS 7: Current GRASS GIS 7 environment:')
print grass.gisenv()

# do something in GRASS now...

grass.message('--- GRASS GIS 7: Checking projection info:')
in_proj = grass.read_command('g.proj', flags = 'jf')

# selective proj parameter printing
kv = grass.parse_key_val(in_proj)
print kv
print kv['+proj']

# print full proj parameter printing
in_proj = in_proj.strip()
grass.message("--- Found projection parameters: '%s'" % in_proj)

# show current region:
grass.message('--- GRASS GIS 7: Checking computational region info:')
in_region = grass.region()
grass.message("--- Computational region: '%s'" % in_region)

# do something else: r.mapcalc, v.rectify, ...

# Finally remove the temporary batch location from disk
print 'Removing location %s' % location_path
shutil.rmtree(location_path)

sys.exit(0)
</source>

===== Python: GRASS GIS 6 =====

The script initializes the session and lists available raster maps:

<source lang=python>
import os
import sys

gisbase = os.environ['GISBASE'] = "/usr/local/src/grass_trunk/dist.x86_64-unknown-linux-gnu"

gisdbase = os.path.join(os.environ['HOME'], "grassdata")
location = "nc_spm_08"
mapset = "user1"

sys.path.append(os.path.join(os.environ['GISBASE'], "etc", "python"))
import grass.script as grass
import grass.script.setup as gsetup

gsetup.init(gisbase,
gisdbase, location, mapset)

print grass.gisenv()

grass.message('Raster maps:')
for rast in grass.list_strings(type = 'rast'):
print rast
</source>

==== Bash examples (GNU/Linux) ====

Below an example of a '''~/.bash_profile''' script to set the required settings in order to access GRASS commands outside of a GRASS-session. Specifically it is for GRASS 7 (commented parts for GRASS 6.4.2):

<source lang=bash>
# example for GRASS 7
export GISBASE=/usr/local/grass-7.0.svn
# example for GRASS 6.4.2
### export GISBASE=/usr/local/grass-6.4.2svn

# GRASS 7
export GRASS_VERSION="7.0.svn"
# GRASS 6.4.2
### export GRASS_VERSION="6.4.2svn"

#generate GISRCRC
MYGISDBASE=$HOME/grassdata
MYLOC=nc_spm_08
MYMAPSET=user1

# Set the global grassrc file to individual file name
MYGISRC="$HOME/.grassrc.$GRASS_VERSION.$$"

echo "GISDBASE: $MYGISDBASE" > "$MYGISRC"
echo "LOCATION_NAME: $MYLOC" >> "$MYGISRC"
echo "MAPSET: $MYMAPSET" >> "$MYGISRC"
echo "GRASS_GUI: text" >> "$MYGISRC"

# path to GRASS settings file
export GISRC=$MYGISRC
export GRASS_PYTHON=python
export GRASS_MESSAGE_FORMAT=plain
export GRASS_TRUECOLOR=TRUE
export GRASS_TRANSPARENT=TRUE
export GRASS_PNG_AUTO_WRITE=TRUE
export GRASS_GNUPLOT='gnuplot -persist'
export GRASS_WIDTH=640
export GRASS_HEIGHT=480
export GRASS_HTML_BROWSER=firefox
export GRASS_PAGER=cat
export GRASS_WISH=wish

export PATH="$GISBASE/bin:$GISBASE/scripts:$PATH"
export LD_LIBRARY_PATH="$GISBASE/lib"
export GRASS_LD_LIBRARY_PATH="$LD_LIBRARY_PATH"
export PYTHONPATH="$GISBASE/etc/python:$PYTHONPATH"
export MANPATH=$MANPATH:$GISBASE/man

#For the temporal modules
export TGISDB_DRIVER=sqlite
export TGISDB_DATABASE=$MYGISDBASE/$MYLOC/PERMANENT/tgis/sqlite.db

# test a command
g.list rast

######### below not needed ########
# GRASS 7
tmp=/tmp/grass7-"`whoami`"-$GIS_LOCK
# GRASS 6.4.2
### tmp=/tmp/grass6-"`whoami`"-$GIS_LOCK

export GISRC="$tmp/rc"
mkdir "$tmp"

# GRASS 7
cp ~/.grass7/rc "$GISRC"
# GRASS 6.4.2
### cp ~/.grassrc6 "$GISRC"
######### END below not needed ########
</source>

The above script will allow GRASS commands to be used anywhere. Furthermore, if the '''~/.Xsession''' sources the bash startup scripts, the settings aren't limited to interactive shells, but also work for e.g. M-! in XEmacs). Each interactive shell gets a separate "session" (i.e. a separate $GISRC file), while GUI programs share a common session.

Note, however, that '''~/.bash_profile''' is a personal initialization startup script and, thus, read during the login process. Any modifications to it will be read after (re-)login or explicitly sourcing it.

At the end of the ~/.bash_profile file, the following lines can be added to ensure no duplication of entries in the PATH variable.

<source lang=bash>
PATH=`awk -F: '{for(i=1;i<=NF;i++){if(!($i in a)){a[$i];printf s$i;s=":"}}}'<<<$PATH`
PYTHONPATH=`awk -F: '{for(i=1;i<=NF;i++){if(!($i in a)){a[$i];printf s$i;s=":"}}}'<<<$PYTHONPATH`
</source>

=== Important notes ===

Launching a grassXY session could still permit access on GRASS modules of the version that is set with the above script. The reason being is that grassXY scripts (e.g.: grass64, grass65, grass70) prepend the GRASS directories to '''PATH''', '''LD_LIBRARY_PATH''', etc. They won't remove any entries which are already there. A solution to this is to ''switch'' between GRASS versions by using a script like the following:

<source lang="bash">
strippath()
{
(
oldpath="$1"
newpath=
IFS=:
for dir in $oldpath ; do
case "${dir}" in
*grass*)
;;
*)
newpath="$newpath:$dir"
;;
esac
done
echo "${newpath#:}"
)
}

PATH=`strippath $PATH`

export GISBASE=$PWD/dist.i686-pc-linux-gnu
export PATH="$GISBASE/bin:$GISBASE/scripts:$PATH"
export LD_LIBRARY_PATH="$GISBASE/lib"
export PYTHONPATH="$GISBASE/etc/python"
</source>

'''Note: the above script needs to be "source"d; executing it won't work. It might also require some adjustments (e.g. the ''GISBASE'' variable).'''

Using normal grass sessions commands are recorded in '''$GISDBASE/$LOCATION_NAME/$MAPSET/.bash_history'''. While working with "pure" bash, shell entries go in '''~/.bash_history'''. Merging existing "grass-bash_history(-ies)" with the bash history log, would probably be not a good idea. Perhaps it is wise(r) to use normal GRASS sessions when working on real projects. Nevertheless, it is upon the users preferences and expertise level to decide what is best.

=== References ===
* Instructions and discussion on ''how to access GRASS mapsets outside of a GRASS session'' in the grass-user mailing list [http://n2.nabble.com/can-I-access-mapset-outside-of-grass-by-using-python-td3405902.html#a3405902].
* removing duplicate entries in $PATH taken from [http://chunchung.blogspot.com/2007/11/remove-duplicate-paths-from-path-in.html]

== GRASS databases ==

(project file structure)

=== Minimal mapsets ===

within a functional LOCATION (see below) the minimal mapset is a subdirectory of the MAPSET's name, containing a WIND file. The WIND file can simply be copied from PERMANENT/DEFAULT_WIND. Optionally you can put a VAR file in there too to define the default database driver to use.

* You can create a new mapset when starting GRASS with the -c flag. e.g.
grass64 -c /path/to/location/new_mapset_name

=== Minimal locations ===

Within the GISDATABASE (which is simply a subdirectory some where), the minimum LOCATION consists of a directory giving the LOCATION its name, which in turn contains a PERMANENT subdirectory for the PERMANENT mapset. The PERMANENT mapset contains a few special files that regular mapsets don't. These are:
; PROJ_INFO: contains the location's projection information
; PROJ_UNITS: contains the location's map units definition
; DEFAULT_WIND: the default region (WINDow file). The format is identical to the WIND files created by g.region
; WIND: (optional?)

* see also {{src|demolocation}} in the GRASS source code

== See also ==
* [[GRASS_and_Shell#GRASS_Batch_jobs|GRASS Batch jobs]]

[[Category: FAQ]]
[[Category: Linking to other languages]]
[[Category: bash]]
[[Category:Scripting]]
[[Category: Python]]

GRASS-Wiki - User contributions [en]

Working with GRASS without starting it explicitly