GRASS and Python: Difference between revisions

From GRASS-Wiki
Jump to navigation Jump to search
(+ How to write a Python GRASS GIS 7 addon)
 
(75 intermediate revisions by 13 users not shown)
Line 1: Line 1:
''(for discussions on the new GRASS GUI, see [[GRASS GUI|here]])''
''This page discusses usage of GRASS GIS and Python in a general way. If you just want to write a script to run in GRASS GIS, go to [[GRASS Python Scripting Library]] wiki page or [https://grass.osgeo.org/grass-stable/manuals/libpython/ Python API for GRASS GIS 7] documentation.''
 
==Python SIGs==
Python Special Interest Groups are focused collaborative efforts to develop, improve, or maintain specific Python resources. Each SIG has a charter, a coordinator, a mailing list, and a directory on the Python website. SIG membership is informal, defined by subscription to the SIG's mailing list. Anyone can join a SIG, and participate in the development discussions via the SIG's mailing list. Below is the list of currently active Python SIGs, with links to their resources.
 
See more at http://www.python.org/community/sigs/


==Writing Python scripts in GRASS==
==Writing Python scripts in GRASS==


Python is a programming language which is more powerful than shell scripting but easier and more forgiving than C.
Python is a programming language which is more powerful than shell scripting but easier and more forgiving than C.
The Python script can contain simple module description definitions which will be processed with {{cmd|g.parser}}, as shown in the example below. In this way with no extra coding a GUI can be built, inputs checked, and a skeleton help page can be generated automatically. In addition it adds links to the GRASS message translation system.
The Python script can contain simple module description definitions which will be processed with {{cmd|g.parser}}, as shown in the example below. In this way, with no extra coding a GUI can be built, inputs checked, and a skeleton help page can be generated automatically. In addition, it adds links to the GRASS message translation system. The library for "scripting" is "grass.script", typically used as:
For code which needs access to the power of C, you can access the GRASS C library functions via the Python "ctypes" interface.
 
<source lang="python">
import grass.script as gscript
</source>


;GRASS Python Scripting Library
The related files are at $GISBASE/etc/python/grass/script/*.py. See below for more details.
* http://grass.osgeo.org/programming6/pythonlib.html (GRASS 6)
* http://grass.osgeo.org/programming7/pythonlib.html (GRASS 7)


Code style: Have a look at [http://trac.osgeo.org/grass/browser/grass/trunk/SUBMITTING_PYTHON SUBMITTING_PYTHON].
''Note: For code which needs access to the power of C, you can access the GRASS C library functions via [[GRASS and Python#Python Ctypes Interface|the Python "ctypes" interface]].''


=== Creating Python scripts that call GRASS functionality from outside ===
=== How to write a Python GRASS GIS 7 addon ===


In order to use GRASS from outside, some environment variables have to be set.
The tutorial "How to write a Python GRASS GIS 7 addon" is available here: https://github.com/wenzeslaus/python-grass-addon


==== MS-Windows ====
=== Integrated Python shell ===


GISBASE= C:\GRASS-64
The [[wxGUI]] Layer Manager in GRASS GIS comes with a "Python Shell" which enables users to type and execute Python commands directly in wxGUI environment.
GISRC= C:\Documents and Settings\user\.grassrc6
LD_LIBRARY_PATH= C:\GRASS-64\lib
PATH= C:\GRASS-64\etc;C:\GRASS-64\etc\python;C:\GRASS-64\lib;C:\GRASS-64\bin;C:\GRASS-64\extralib;C:\GRASS-64\msys\bin;C:\Python26;
PYTHONLIB= C:\Python26
PYTHONPATH= C:\GRASS-64\etc\python
GRASS_SH= C:\GRASS-64\msys\bin\sh.exe


Some hints:
[[Image:wxgui-pyshell.png|center|400px|Embedded interactive Python Shell in wxGUI Layer Manager]]


# The Python interpreter (python.exe) needs to be in the PATH
=== Integrated Python editor ===
# Python needs to be associated with the .py extension
# PATHEXT needs to include .py if you want to be able to omit the extension
# PYTHONPATH needs to be set to %WINGISBASE%\etc\python


1-3 should be taken care of by the Python installer. 4 needs to be done by the startup (currently, this doesn't appear to be the case on MS-Windows).
The [[wxGUI]] Layer Manager in GRASS GIS 7 comes with a "Simple Python Editor" which enables users to author Python scripts directly in the GRASS GIS GUI. Users can also run the script easily in the GRASS GIS environment with all the dependencies loaded. The editor comes with several examples, templates, and links to documentation.


Note:
[[File:Simple python editor v buffer.png|300px|thumb|right|Python interactive shell (console) and a simple Python editor are a powerful option for interacting with GRASS GIS as well as extending its functionality]]


Currently (as of 22 Feb 2011) if you want to use Python for scripting GRASS on Windows, the best solution is to delete the bundled version of Python 2.5 from the GRASS installation, install Python and the required add-ons (wxPython, NumPy, PyWin32) from their official installers,
=== External Python editors (IDE) ===
then edit the GRASS start-up script to remove any references to the bundled version.


==== Linux ====
Besides the wxGUI Python shell and script editor, also advanced editors like '''Spyder''' (The Scientific PYthon Development EnviRonment) can be used in connection with GRASS GIS. For details, see [[Tools for Python programming]].


The variables are set like this:
==== The correct editor settings for Python indentation ====


export GISBASE="/usr/local/grass-6.4.svn/"
Be sure to use only white spaces and no tabs to indent Python code!
export PATH="$PATH:$GISBASE/bin:$GISBASE/scripts"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$GISBASE/lib"
# for parallel session management, we use process ID (PID) as lock file number:
export GIS_LOCK=$$
# path to GRASS settings file
export GISRC="$HOME/.grassrc6"


=== Running external commands from Python ===
See: https://trac.osgeo.org/grass/wiki/Submitting/Python#Editorsettingsfor4-spaceindentation
For information on running external commands from Python, see:
http://docs.python.org/lib/module-subprocess.html


Avoid using the older os.* functions. Section 17.1.3 lists equivalents
Editor settings for 4-space indentation
using the Popen() interface, which is more robust (particularly on
* '''Geany''' editor:
Windows).
** Edit > Preferences > Editor > Intentation tab > Type: Spaces
* GNU Emacs:
** python-mode default
* spyder ...
* pycharm ...


=== Using the GRASS Python Scripting Library ===
=== Using the GRASS Python Scripting Library ===


See [[GRASS Python Scripting Library]] for notes and examples.
You can run Python scripts easily in a GRASS session.


=== Examples ===
To write these scripts,
* check the code in lib/python/ which provides grass.script in order to support GRASS scripts written in Python.<br>
  See the [[GRASS Python Scripting Library]] for notes and examples.
* The scripts/ directory of GRASS contains a series of examples actually provided to the end users.


==== Display example ====
For the desired Python code style, have a look at [https://trac.osgeo.org/grass/wiki/Submitting/Python Submitting Python].
Example of Python script, which is processed by g.parser:


<source lang="python">
=== Creating Python scripts that call GRASS functionality from outside ===
#!/usr/bin/env python
#
############################################################################
#
# MODULE:      d.shadedmap
# AUTHOR(S):  Unknown; updated to GRASS 5.7 by Michael Barton
#              Converted to Python by Glynn Clements
# PURPOSE:    Uses d.his to drape a color raster over a shaded relief map
# COPYRIGHT:  (C) 2004,2008,2009 by the GRASS Development Team
#
#              This program is free software under the GNU General Public
#              License (>=v2). Read the file COPYING that comes with GRASS
#              for details.
#
#############################################################################


#%Module
For calling GRASS functionality from outside, see the [https://grass.osgeo.org/grass-stable/manuals/libpython/script.html#module-script.setup documentation] and also [[Working with GRASS without starting it explicitly]].
#% description: Drapes a color raster over a shaded relief map using d.his
#%End
#%option
#% key: reliefmap
#% type: string
#% gisprompt: old,cell,raster
#% description: Name of shaded relief or aspect map
#% required : yes
#%end
#%option
#% key: drapemap
#% type: string
#% gisprompt: old,cell,raster
#% description: Name of raster to drape over relief map
#% required : yes
#%end
#%option
#% key: brighten
#% type: integer
#% description: Percent to brighten
#% options: -99-99
#% answer: 0
#%end


import sys
Note: This is a more advanced use case of using GRASS' functionality from outside via Python. Commonly, a user will run GRASS Python script from inside a GRASS session, i.e. either from the command line or from the Python shell embedded in the wxGUI ([[:File:Wxgui-pyshell.png|screenshot]]).
from grass.script import core as grass


def main():
=== Running external commands from Python ===
    drape_map = options['drapemap']
For information on running external commands from Python, see:
    relief_map = options['reliefmap']
http://docs.python.org/lib/module-subprocess.html
    brighten = options['brighten']
    ret = grass.run_command("d.his", h_map = drape_map,  i_map = relief_map, brighten = brighten)
    sys.exit(ret)


if __name__ == "__main__":
Avoid using the older os.* functions. Section 17.1.3 lists equivalents
    options, flags = grass.parser()
using the Popen() interface, which is more robust (particularly on
    main()
Windows).


</source>
=== Testing and installing Python extensions ===


==== Parsing the options and flags  ====
==== Debugging ====


grass.parser() is an interface to g.parser, and allows to parse the options and flags passed to your script on the command line. It is to be called at the top-level:
Make sure the script is executable:


<source lang="python">
     chmod +x /path/to/my.extension.py
if __name__ == "__main__":
     options, flags = grass.parser()
    main()
</source>


Global variables "options" and "flags" are Python dictionaries containing the options/flags values, keyed by lower-case option/flag names. The values in "options" are strings, those in "flags" are Python booleans. All those variables have to be previously declared in the header of your script.
During development, a Python script can be debugged using the Python Debugger (pdb):


<source lang="python">
    python -m pdb /path/to/my.extension.py input=my_input_layer output=my_output_layer option=value -f
>>> options, flags = grass.parser()
>>> options
{'input': 'my_map', 'output': 'map_out', 'option1': '21.472', 'option2': ''}
>>> flags
{'c': True, 'm': False}
</source>


==== Example for embedding r.mapcalc (map algebra) ====
==== Installation ====


grass.mapcalc() accepts a template string followed by keyword
Once you're happy with your script, you can put it in the scripts/ folder of your GRASS install. To do so, first create a directory named after your extension, then create a Makefile for it, and a HTML man page:
arguments for the substitutions, e.g. (code snippets):


<source lang="python">
    cd /path/to/grass_src/
grass.mapcalc("${out} = ${rast1} + ${rast2}",
    cd scripts
              out = options['output'],
    ls # it is useful to check out the existing scripts and their structure
              rast1 = options['raster1'],
    mkdir my.extension
              rast2 = options['raster2'])
    cd my.extension
</source>
    cp path/to/my.extension.py .
    touch my.extension.html
    touch Makefile


''Best practice'': first copy all of the options[] into separate variables at the beginning of main(), i.e.:
Next step is to edit the Makefile. It is a very simple text file, the only thing to check is to put the right extension name (WITHOUT the .py file extension) after PGM:


<source lang="python">
     MODULE_TOPDIR = ../..
def main():
    output = options['output']
    raster1 = options['raster1']
    raster2 = options['raster2']
    ...
    grass.mapcalc("${out} = ${rast1} + ${rast2}",
                  out = output,
                  rast1 = raster1,
                  rast2 = raster2)
</source>
 
==== Example for parsing raster category labels ====
 
How to obtain the text labels
<source lang="python">
    # dump cats to file to avoid "too many argument" problem:
    p = grass.pipe_command('r.category', map = rastertmp, fs = ';', quiet = True)
    cats = []
    for line in p.stdout:
        cats.append(line.rstrip('\r\n').split(';')[0])
    p.wait()
 
    number = len(cats)
     if number < 1:
        grass.fatal(_("No categories found in raster map"))
</source>
 
==== Example for parsing category numbers ====
 
Q: How to obtain the number of cells of a certain category?
 
A: It is recommended to use pipe_command() and parse the output, e.g.:
 
<source lang="python">
      p = grass.pipe_command('r.stats',flags='c',input='map')
      result = {}
      for line in p.stdout:
          val,count = line.strip().split()
          result[int(val)] = int(count)
      p.wait()
</source>
 
==== Example for getting the region's number of rows and columns ====
 
Q: How to obtain the number of rows and columns of the current region?
 
A: It is recommended to use the "grass.region()" function which will create a dictionary with values for extents and resolution, e.g.:
 
<source lang="python">
#!/usr/bin/env python
#-*- coding:utf-8 -*-
#
############################################################################
#
# MODULE:      g.region.resolution
# AUTHOR(S):    based on a post at GRASS-USER mailing list [1]             
# PURPOSE: Parses "g.region -g", prints out number of rows, cols
# COPYLEFT:    ;-)
# COMMENT:      ...a lot of comments to be easy-to-read for/by beginners
#
#############################################################################
#
#%Module
#% description: Print number of rows, cols of current geographic region
#% keywords: region
#%end
 
# importing required modules
import sys # the sys module [2]
from grass.script import core as grass # the core module [3]
 
# information about imported modules can be obtained using the dir() function
# e.g.: dir(sys)
 
# define the "main" function: get number of rows, cols of region
def main():
      
      
     # #######################################################################
     PGM = my.extension
    # the following commented code works but is kept only for learning purposes
   
    ## assigning the output of the command "g.region -g" in a string called "return_rows_x_cols"
    # return_rows_x_cols = grass.read_command('g.region', flags = 'g')
      
      
     ## parsing arguments of interest (rows, cols) in a dictionary named "rows_x_cols"
     include $(MODULE_TOPDIR)/include/Make/Script.make
    # rows_x_cols = grass.parse_key_val(return_rows_x_cols)
      
      
     ## selectively print rows, cols from the dictionary "rows_x_cols"
     default: script
    # print 'rows=%d \ncols=%d' % (int(rows_x_cols['rows']), int(rows_x_cols['cols']))
   
    # #######################################################################
   
    # faster/ easier way: use of the "grass.region()" function
    gregion = grass.region()
    rows = gregion['rows']
    cols = gregion['cols']
   
    # print rows, cols properly formated
    print 'rows=%d \ncols=%d' % (rows, cols)


# this "if" condition instructs execution of code contained in this script, *only* if the script is being executed directly
The HTML file would be generated automatically. If you want to add more precisions in it, you can do it (just make sure you start at DESCRIPTION. See existing scripts.)
if __name__ == "__main__": # this allows the script to be used as a module in other scripts or as a standalone script
    options, flags = grass.parser() #
    sys.exit(main()) #


# Links
In the /path/to/grass_src/scripts folder there is also a Makefile. Before building your new folder name should be added to the SUBDIRS list in this file to be built correctly.
# [1] http://n2.nabble.com/Getting-rows-cols-of-a-region-in-a-script-tp2787474p2787509.html
# [2] http://www.python.org/doc/2.5.2/lib/module-sys.html
# [3] http://download.osgeo.org/grass/grass6_progman/pythonlib.html#pythonCore
</source>


==== Managing mapsets ====
You can then run "make" within the my.extension folder. Running "make" in the extension directory places the resulting files in the staging directory (path/to/grass_src/dist.<YOUR_ARCH>/). If you're running GRASS from the staging directory (/path/to/grass_src/bin.<YOUR_ARCH>/grass7), subsequent commands will used the updated files.


To check if a certain mapset exists in the active location, use:
    # in your extension directory (/path/to/grass_src/scripts/my.extension/)
    make
    # Starting GRASS from the staging directory
    /path/to/grass_src/bin.<YOUR_ARCH>/grass7
    my.extension help


<source lang="python">
You can also run "make install" from the top level directory of your GRASS install (say /usr/local/src/grass_trunk/). Running "make install" from the top level just copies the whole of the dist.<YOUR_ARCH>/ directory to the installation directory (e.g. /usr/local/grass78) and the bin.<YOUR_ARCH>/grass78 bin file to the bin directory (e.g. /usr/local/bin), and fixes any embedded paths in scripts and configuration files.
      grass.script.mapsets(False)
</source>


... returns a list of mapsets in the current location.
    cd /path/to/grass_src
 
    make install
==== r.mapcalc example ====
    # Starting GRASS as usual would work and show your extension available
 
    grass7
Example of Python script, which is processed by {{cmd|g.parser}}:
    my.extension help
 
The shell script line:
 
<source lang="bash">
  r.mapcalc "MASK = if(($cloudResampName < 0.01000),1,null())"
</source>


would be written like this:
=== Using the parser to get GUI and standardized CLI ===


<source lang="python">
Read the documentation for [https://grass.osgeo.org/grass-stable/manuals/libpython/script_intro.html GRASS GIS Python scripting with script package].
      import grass.script as grass


      ...
For more details, see
* [[Module command line parser]]
* [[GRASS Python Scripting Library#Parsing the options and flags]]


      grass.mapcalc("MASK=if(($cloudResampName < 0.01000),1,null())",
Be sure to use the standardized options such as G_OPT_R_INPUT or G_OPT_V_OUTPUT if possible. See the overview table of [https://grass.osgeo.org/grass-stable/manuals/parser_standard_options.html parser standard options] in the documentation.
                    cloudResampName = cloudResampName)
</source>


The first argument to the mapcalc function is a template (see the Python library documentation for [http://docs.python.org/library/string.html string.Template]). Any keyword arguments (other than quiet, verbose or overwrite) specify substitutions.
=== Using a version of Python different from the default installation ===


==== Using output from GRASS modules in the script ====
Sometimes you don't want to use the default Python version installed on your system. You can control the interpreter used for building by overriding the PYTHON make variable, e.g.


Sometimes you need to use the output of a module for the next step. There are dedicated functions to obtain the result of, for example, a statistical analysis.
  make PYTHON=/path/to/python ...


Example: get the range of a raster map and use it in {{cmd|r.mapcalc}}. Here you can use <code>grass.script.raster_info()</code>, e.g.:
For controlling the interpreter used at run time:
* On Windows, Python scripts are invoked via %GRASS_PYTHON%, so changing that environment variable will change the interpreter.
* On Unix, GRASS_PYTHON is only used for components which use wxPython, e.g. wxGUI or the parameter dialogues which are displayed when modules are run without arguments.


<source lang="python">
Scripts rely upon the "#!/usr/bin/env python" line. You can control
      import grass.script as grass
the interpreter used by scripts by creating a directory containing a
symlink named "python" which refers to your preferred version of the
interpreter, and placing that directory at the front of $PATH.


      max = grass.raster_info(inmap)['max']
==Python extensions in GRASS GIS==
      grass.mapcalc("$outmap = $inmap / $max",
                    inmap = inmap, outmap = outmap, max = max)
</source>
 
==== Calling a GRASS module in Python ====
 
Imagine, you wanted to execute this command in Python:
<source lang="bash">
  r.profile -g input=mymap output=newfile profile=12244.256,-295112.597,12128.012,-295293.77
</source>
 
All arguments except the first (which is a flag) are keyword arguments, i.e. <tt>arg = val</tt>. For the flag, use <tt>flags = 'g'</tt> (note that "-g" would be the negative of a Python variable named "g"!). So:
 
<source lang="python">
      grass.run_command(
              'r.profile',
              input = input_map,
              output = output_file,
              profile = [12244.256,-295112.597,12128.012,-295293.77]
</source>
or:
<source lang="python">
              profile = [(12244.256,-295112.597),(12128.012,-295293.77)]
</source>


i.e. you need to provide the keyword, and the argument must be a valid Python expression. Function <code>run_command()</code> etc accept lists and tuples.
=== Python Scripting Library ===


'''Differences between ''run_command()'' and ''read_command()'':'''
* See [[GRASS Python Scripting Library]]


* run_command() executes the command and waits for it to terminate; it doesn't redirect any of the standard streams.
=== pyGRASS Library ===
* read_command() executes the command with stdout redirected to a pipe, and reads everything written to it. Once the command terminates, it returns the data written to stdout as a string.


'''How to retrieve error messages from ''read_command()'':'''
pyGRASS is an object oriented Python Application Programming Interface (API) for GRASS GIS. pyGRASS wants to improve the integration between GRASS and Python, render the use of Python under GRASS more consistent with the language itself and make the GRASS scripting and programming activity easier and more natural to the final users. Some simple examples below.


None of the existing *_command functions redirect stderr. You can do so with e.g.:
For more details, see '''[[Python/pygrass|pygrass]]'''


'''Example 1''':
<source lang="python">
<source lang="python">
def read2_command(*args, **kwargs):
from grass.pygrass.modules import Module
  kwargs['stdout'] = grass.PIPE
  kwargs['stderr'] = grass.PIPE
  ps = grass.start_command(*args, **kwargs)
  return ps.communicate()
</source>
 
This behaves like read_command() except that it returns a tuple of (stdout,stderr) rather than just stdout.
 
==== Percentage output for progress of computation ====
 
A) Within a Python script, the grass.script.core.percent() module method wraps the <tt>g.message -p</tt> command.
 
B) If you call a GRASS command within the Python code, you have to parse the output by setting <tt>GRASS_MESSAGE_FORMAT=gui</tt> in the environment when running the command and read from the command's stderr; e.g.


<source lang="python">
slope_aspect = Module("r.slope.aspect")
      import grass.script as grass
slope_aspect(elevation='elevation', slope='slp',  aspect='asp',  
      env = os.environ.copy()
              format='degrees', overwrite=True)
      env['GRASS_MESSAGE_FORMAT'] = 'gui'
      p = grass.start_command(..., stderr = grass.PIPE, env = env)
      # read from p.stderr
      p.wait()
</source>
</source>


If you need to capture both stdout and stderr, you need to use threads, select, or non-blocking I/O to consume data from both streams as it is generated in order to avoid deadlock.
'''Example 2''', using "shortcuts":
 
ALTERNATIVE:
 
Redirect both stdout and stderr to the same pipe (and hope that the normal output doesn't include anything which will be mistaken for progress/error/etc messages):
 
<source lang="python">
<source lang="python">
      p = grass.start_command(..., stdout = grass.PIPE, stderr = grass.STDOUT, env = env)
#!/usr/bin/env python
</source>
 
==== NULL data management ====
 
How to analyse if there are only NULL cells in a map:
 
If a map contains only null cells, its minimum and maximum will be "NULL":
 
<source lang="bash">
      $ r.mapcalc 'foo = null()'
      $ r.info -r foo
      min=NULL
      max=NULL
</source>


Using the Python API, The 'min' and 'max' values in the result of the raster_info() function will be None.
# simple example for pyGRASS usage: raster processing via modules approach


==== Counting cells ====
from grass.pygrass.modules.shortcuts import general as g
from grass.pygrass.modules.shortcuts import raster as r


Counting cells is far more expensive than simply determining whether
g.message("Filter elevation map by a threshold...")
there are any non-null cells. Counting cells requires reading the
entire map, while the r.info approach only needs to read the metadata
files.


If you do need to count cells, {{cmd|r.stats}} is likely to be more efficient than {{cmd|r.univar}.
# set computational region
input = 'elevation'
g.region(raster=input)


A count loop:
# hardcoded:
# r.mapcalc('elev_100m = if(elevation > 100, elevation, null())', overwrite = True)
# with variables


<source lang="python">
output = 'elev_100m'
      while grass.raster_info(inmap)['max'] is not None:
thresh = 100.0
          ...
r.mapcalc("%s = if(%s > %d, %s, null())" % (output, input, thresh, input), overwrite = True)
r.colors(map=output, color="elevation")
</source>
</source>


==== Path to GISDBASE ====


In order to a avoid hardcoded paths to GRASS mapset files like the SQLite DB file, you can get the GISDBASE variable from the environment:
'''Example 3''', using "shortcuts" and external raster format support:
<source lang="python">
<source lang="python">
      import grass.script as grass
#!/usr/bin/env python
      import os.path
 
      env = grass.gisenv()
 
      gisdbase = env['GISDBASE']
      location = env['LOCATION_NAME']
      mapset = env['MAPSET']
 
      path = os.path.join(gisdbase, location, mapset, 'sqlite.db')
</source>
==== Use Python reserved keyword ====
 
''Question:'' r.resamp.bspline uses 'lambda' as a command line parameter name, but when you try to use it with grass.run_command() you get an error as lambda is a python reserved keyword. How to work around that?
 
''Answer:'' Prepend an underscore to the name, i.e.:
 
<source lang="python">
      grass.run_command('r.resamp.bspline', _lambda = ...)
</source>
 
==== Controlling the PNG display driver ====


Code fragment to control the {{cmd|pngdriver}} in Python:
# simple example for pyGRASS usage: raster processing via Modules approach
 
# Read GeoTIFF directly, write out GeoTIFF directly
<source lang="python">
import os
import os
import sys
import tempfile
from grass.script import core as grass
from grass.pygrass.modules.shortcuts import general as g
def main():
from grass.pygrass.modules.shortcuts import raster as r
      os.environ['GRASS_PNGFILE'] = filename
from grass.pygrass.modules import Module
      os.environ['GRASS_WIDTH'] = str(width)
      os.environ['GRASS_HEIGHT'] = str(height)
      grass.run_command('d.his', i='elevation_shade', h='elevation')
</source>


==== Sophisticated cleanup procedure ====
# use alias name for GRASS GIS commands with more than one dot in name
r.external_out = Module('r.external.out')


Scripts which create several temporary files need a more sophisticated cleanup procedure which deletes all the tmp maps which have been created. This procedure should also work if the script stops (e.g due to an error).
# get the raster elevation map into GRASS without import
geotiff = 'elevation.tif' # with path
curr_raster='myraster'
r.external(input=geotiff, output=curr_raster)


Solution: Define a list of map names which starts out empty and has names appended to it as the names are generated. Code fragment:
# define output directory for files resulting from GRASS calculation:
gisdb = os.path.join(tempfile.gettempdir(), 'gisoutput')
try:
    os.stat(gisdb)
except:
    os.mkdir(gisdb)
g.message("Output will be written to %s" % gisdb)


<source lang="python">
# define where to store the resulting files and format
      tmp_rast = []
r.external_out(directory=gisdb, format='GTiff')


      def cleanup():
# do the GRASS GIS job
          for rast in tmp_rast:
g.message("Filter elevation map by a threshold...")
              grass.run_command("g.remove",
# set computational region
                                rast = rast,
g.region(raster=curr_raster)
                                quiet = True)


      def main():
# generate GeoTIFF
          ...
output = 'elev_100m.tif'
          while ...:
thresh = 100.0
              next_rast = ...
r.mapcalc("%s = if(%s > %d, %s, null())" % (output, curr_raster, thresh, curr_raster), overwrite = True)
              tmp_rast.append(next_rast)
r.colors(map=output, color="elevation")
              ...


      if __name__ == "__main__":
# cease GDAL mode, back to GRASS data writing mode
          options, flags = grass.parser()
r.external_out(flags = 'r')
          atexit.register(cleanup)
# the resulting GeoTIFF file is in the gisdb directory
          sys.exit(main())
</source>
</source>


==Python extensions for GRASS GIS==
'''Example 4''':
=== wxPython GUI development for GRASS ===
 
* See the [[wxGUI]] wiki page
 
=== GRASS Python Scripting Library ===
 
See [http://grass.osgeo.org/programming7/pythonlib.html GRASS Python Scripting Library] (Programmer's manual). See also [[Converting Bash scripts to Python]], and [http://trac.osgeo.org/grass/browser/grass/trunk/scripts sample Python scripts in GRASS 7]
 
==== Uses for read, feed and pipe, start and exec commands ====
 
All of the *_command functions use make_command to construct a command
line for a program which uses the GRASS parser. Most of them then pass
that command line to ''subprocess.Popen()'' via ''start_command()'', except
for ''exec_command()'' which uses ''os.execvpe()''.
 
[To be precise, they use grass.Popen(), which just calls
subprocess.Popen() with shell=True on Windows and shell=False
otherwise. On Windows, you need to use shell=True to be able to
execute scripts (including batch files); shell=False only works with
binary executables.]
 
start_command() separates the arguments into those which
subprocess.Popen() understands and the rest. The rest are passed to
make_command() to construct a command line which is passed as the
"args" parameter to subprocess.Popen().
 
In other words, start_command() is a GRASS-oriented interface to
subprocess.Popen(). It should be suitable for any situation where you
would use subprocess.Popen() to execute a normal GRASS command (one
which uses the GRASS parser, which is almost all of them; the main
exception is r.mapcalc in 6.x).
 
Most of the others are convenience wrappers around start_command(), for common use cases.
 
* run_command() calls the wait() method on the process, so it doesn't return until the command has finished, and returns the command's exit code. Similar to system().
 
* pipe_command() calls start_command() with stdout=PIPE and returns the process object. You can use the process' .stdout member to read the command's stdout. Similar to popen(..., "r").
 
* feed_command() calls start_command() with stdin=PIPE and returns the process object. You can use the process' .stdin member to write to the command's stdout. Similar to popen(..., "w")
 
* read_command() calls pipe_command(), reads the data from the command's stdout, and returns it as a string. Similar to `backticks` in the shell.
 
* write_command() calls feed_command(), sends the string specified by the "stdin" argument to the command's stdin, waits for the command to finish and returns its exit code. Similar to "echo ... | command".
 
* parse_command() calls read_command() and parses its output as key-value pairs. Useful for obtaining information from g.region, g.proj, r.info, etc.
 
* exec_command() doesn't use start_command() but os.execvpe(). This causes the specified command to replace the current program (i.e. the Python script), so exec_command() never returns. Similar to bash's "exec" command. This can be useful if the script is a "wrapper" around a single command, where you construct the command line and execute the command as the final step.
 
If you have any other questions, you might want to look at the code ($GISBASE/etc/python/grass/script/core.py). Most of these functions are only a few lines long.
 
==== Interfacing with NumPy ====
 
The {{api|pythonlib.html#pythonArray|grass.script.array}} module defines a {{api|classpython_1_1array_1_1array.html|class array}} which is a subclass of [http://docs.scipy.org/doc/numpy/reference/generated/numpy.memmap.html numpy.memmap] with <code>.read()</code> and <code>.write()</code> methods to read/write the underlying file via {{cmd|r.out.bin}}/{{cmd|r.in.bin}}. Metadata can be read with {{api|namespacepython_1_1raster.html#a69e4a61eb59a31608410b733d174b8a7|raster::raster_info}}():
 
Example:
 
<source lang="python">
<source lang="python">
import grass.script as grass
#!/usr/bin/env python
import grass.script.array as garray
 
# Example for pyGRASS usage - vector API
def main():
    map = "elevation.dem"
 
    # read map
    a = garray.array()
    a.read(map)
 
    # get raster map info
    print grass.raster_info(map)['datatype']
    i = grass.raster_info(map)
   
    # get computational region info
    c = grass.region()
    print "rows: %d" % c['rows']
    print "cols: %d" % c['cols']
 
    # new array for result
    b = garray.array()
    # calculate new map from input map and store as GRASS raster map
    b[...] = (a / 50).astype(int) * 50
    b.write("elev.50m")
</source>


The size of the array is taken from the current region ([[computational region]]).
from grass.pygrass.modules.shortcuts import general as g
from grass.pygrass.vector import VectorTopo
from grass.pygrass.modules import Module


The main drawback of using numpy is that you're limited by available
vectmap = 'myzipcodes_wake'
memory. Using a subclass of <code>numpy.memmap</code> lets you use files which may
be much larger, but processing the entire array in one go is likely to
produce in-memory results of a similar size.


==== Interfacing with NumPy and SciPy  ====
g.message("Importing SHAPE file ...")
ogrimport = Module('v.in.ogr')
ogrimport('/home/neteler/gis_data/zipcodes_wake.shp', output=vectmap)


[http://docs.scipy.org/doc/scipy/reference/index.html SciPy] offers simple access to complex calculations. Example:
g.message("Assessing vector topology...")
zipcodes = VectorTopo(vectmap)


<source lang="python">
# Open the map with topology:
from scipy import stats
zipcodes.open()
import grass.script as grass
import grass.script.array as garray


def main():
# query number of topological features
    map = "elevation.dem"
areas  = zipcodes.number_of("areas")
islands = zipcodes.number_of("islands")
print 'Map: <' + vectmap + '> with %d areas and %d islands' % (areas, islands)


    x = garray.array()
dblink = zipcodes.dblinks[0]
    x.read(map)
print 'DB name:'
print dblink.database
table = dblink.table()
print 'Column names:'
print table.columns.names()
print 'Column types:'
print table.columns.types()


    # Descriptive Statistics:
zipcodes.close()
    print "max, min, mean, var:"
    print x.max(), x.min(), x.mean(), x.var()
    print "Skewness test: z-score and 2-sided p-value:"
    print stats.skewtest(stats.skew(x))
</source>
</source>


==== Interfacing with NumPy, SciPy and Matlab ====
* For more examples, see '''[[Python/pygrass|pygrass]]'''
 
One may also use the SciPy - Matlab interface:
   
    ### SH: in GRASS ###
    r.out.mat input=elevation output=elev.mat
 
<source lang="python">
    ### PY ###
    import scipy.io as sio
    # load data
    elev = sio.loadmat('elev.mat')
    # retrive the actual array. the data set contains also the spatial reference
    elev.get('map_data')
    data = elev.get('map_data')
    # a first simple plot
    import pylab
    pylab.plot(data)
    pylab.show()
    # the contour plot
    pylab.contour(data)
    # obviously data needs to ne reversed
    import numpy as np
    data_rev = data[::-1]
    pylab.contour(data_rev)
    # => this is a quick plot. basemap mapping may provide a nicer map!
    #######
</source>
 
=== Testing and installing Python extensions ===
 
==== Debugging ====
 
Make sure the script is executable:
 
    chmod +x /path/to/my.extension.py
 
During development, a Python script can be debugged using the Python Debugger (pdb):
 
    python -m pdb /path/to/my.extension.py input=my_input_layer output=my_output_layer option=value -f
 
==== Installation ====
 
Once you're happy with your script, you can put it in the scripts/ folder of your GRASS install. To do so, first create a directory named after your extension, then create a Makefile for it, and a HTML man page:
 
    cd /path/to/grass_src/
    cd scripts
    ls # It is useful to check out the existing scripts and their structure
    mkdir my.extension
    cd my.extension
    cp path/to/my.extension.py .
    touch my.extension.html
    touch Makefile
 
Next step is to edit the Makefile. It is a very simple text file, the only thing to check is to put the right extension name (WITHOUT the .py file extension) after PGM:
 
    MODULE_TOPDIR = ../..
   
    PGM = my.extension
   
    include $(MODULE_TOPDIR)/include/Make/Script.make
   
    default: script
 
The HTML file would be generated automatically. If you want to add more precisions in it, you can do it (just make sure you start at DESCRIPTION. See existing scripts.)
 
You can then run "make" within the my.extension folder. Running "make" in the extension directory places the resulting files in the staging directory (path/to/grass_src/dist.<YOUR_ARCH>/). If you're running GRASS from the staging directory (/path/to/grass_src/bin.<YOUR_ARCH>/grass7), subsequent commands will used the updated files.
 
    # in your extension directory (/path/to/grass_src/scripts/my.extension/)
    make
    # Starting GRASS from the staging directory
    /path/to/grass_src/bin.<YOUR_ARCH>/grass7
    my.extension help
 
You can also run "make install" from the top level directory of your GRASS install (say /usr/local/src/grass_trunk/). Running "make install" from the top level just copies the whole of the dist.<YOUR_ARCH>/ directory to the installation directory (e.g. /usr/local/grass70) and the bin.<YOUR_ARCH>/grass70 bin file to the bin directory (e.g. /usr/local/bin), and fixes any embedded paths in scripts and configuration files.
 
    cd /path/to/grass_src
    make install
    # Starting GRASS as usual would work and show your extension available
    grass7
    my.extension help


=== Python Ctypes Interface ===
=== Python Ctypes Interface ===
Line 694: Line 294:
Examples:
Examples:


* GRASS 7: [http://trac.osgeo.org/grass/browser/grass/trunk/doc/python/raster_example_ctypes.py raster], [http://trac.osgeo.org/grass/browser/grass/trunk/doc/python/vector_example_ctypes.py vector] example
* GRASS 7: [https://github.com/OSGeo/grass/blob/master/doc/python/raster_example_ctypes.py raster], [https://github.com/OSGeo/grass/blob/master/doc/python/vector_example_ctypes.py vector] example
 
 
* Latest and greatest: [[http://trac.osgeo.org/grass/browser/grass/trunk/scripts GRASS 7 Python scripts]]


* [[PythonSwigExamples|More complicated examples]] <<-- TODO: update to Ctypes
* Latest and greatest: GRASS 7 Python {{src|scripts}}


Sample script for GRASS 6 raster access (use within GRASS, Spearfish session):
Sample script for GRASS raster access (use within GRASS, Spearfish session):
<source lang="python">
<source lang="python">
#!/usr/bin/env python
#!/usr/bin/env python
Line 709: Line 306:
import os, sys
import os, sys
from grass.lib import grass
from grass.lib import grass
if "GISBASE" not in os.environ:
    print "You must be in GRASS GIS to run this program."
    sys.exit(1)


if len(sys.argv)==2:
if len(sys.argv)==2:
Line 749: Line 342:


# run within GRASS Spearfish session
# run within GRASS Spearfish session
# run this before starting python to append module search path:
#  export PYTHONPATH=/usr/src/grass70/swig/python
#  check with "import sys; sys.path"
# or:
#  sys.path.append("/usr/src/grass70/swig/python")
# FIXME: install the grass bindings in $GISBASE/lib/ ?
import os, sys
import os, sys
from grass.lib import grass
from grass.lib import grass
from grass.lib import vector as grassvect
from grass.lib import vector as grassvect
if "GISBASE" not in os.environ:
    print "You must be in GRASS GIS to run this program."
    sys.exit(1)


if len(sys.argv)==2:
if len(sys.argv)==2:
Line 795: Line 377:
</source>
</source>


=== Python-GRASS add-ons ===
=== Using ipython ===


Stand-alone addons:
Using [https://ipython.org/ ipython] is great due to the TAB completion. Don't remember the function name or curious about what's offered? Hit TAB (maybe twice for a list).


# Jáchym Čepický's G-ps.map, a GUI to typeset printable maps with ps.map (http://193.84.38.2/~jachym/index.py?cat=gpsmap)
Here an example how to connect to C functions directly from Python:
# Jáchym Čepický's v.pydigit, a GUI to v.edit (http://les-ejk.cz/?cat=vpydigit)
# Jáchym Čepický's PyWPS, GRASS-Web Processing Service (http://pywps.wald.intevation.org)


=== Using GRASS gui.tcl in Python ===
<source lang="python">
grass76
# ... in the GRASS GIS terminal
ipython
In [1]: from grass.lib import gis


Here is some example code to use the grass automatically generated guis in python code. This could (should) all be bundled up and abstracted away so that the implementation can be replaced later.
In [2]: gis.<TAB>
Display all 729 possibilities? (y or n)
gis.ARRAY                                gis.G_fatal_longjmp                      gis.G_set_key_value
gis.ArgumentError                        gis.G_file_name                          gis.G_set_keywords
gis.Array                                gis.G_file_name_misc                      gis.G_set_ls_exclude_filter
...
gis.G_fatal_error                        gis.G_set_gisrc_mode                      gis.wstring_at


<source lang="python">
In [2]: gis.G_message("Hello world")
import Tkinter
Hello world
import os
</source>


# Startup (once):
A full ipython workshop is available here: https://github.com/wenzeslaus/python-grass-addon


tk = Tkinter.Tk()
=== wxPython GUI development ===
tk.eval ("wm withdraw .")
tk.eval ("source $env(GISBASE)/etc/gui.tcl")
# Here you could do various things to change what the gui does
# See gui.tcl and README.GUI


# Make a gui (per dialog)
* See the [[wxGUI]] wiki page
# This sets up a window for the command.
# This can be different to integrate with tkinter:
tk.eval ('set path ".dialog$dlg"')
tk.eval ('toplevel .dialog$dlg')
# Load the code for this command:
fd = os.popen ("d.vect --tcltk")
gui = fd.read()
# Run it
tk.eval(gui)
dlg = tk.eval('set dlg') # This is used later to get and set


# Get the current command in the gui we just made:
=== 3rd party Python software connected to GRASS GIS ===
currentcommand = tk.eval ("dialog_get_command " + dlg)


# Set the command in the dialog we just made:
* PyWPS (http://pywps.org/)
tk.eval ("dialog_set_command " + dlg + " {d.vect map=roads}")
* QGIS Processing plugin
</source>


== FAQ ==
== FAQ ==
Line 847: Line 421:
=== General guides ===
=== General guides ===


* [http://en.wikibooks.org/wiki/Python_Programming/ Wikibook Python Programming]
* [https://en.wikibooks.org/wiki/Python_Programming/ Wikibook Python Programming]
* [http://www.poromenos.org/tutorials/python Quick Python tutorial] for programmers of other languages
* [https://www.poromenos.org/tutorials/python Quick Python tutorial] for programmers of other languages
*: [http://wiki.python.org/moin/BeginnersGuide/Programmers More Python tutorials] for programmers
*: [https://wiki.python.org/moin/BeginnersGuide/Programmers More Python tutorials] for programmers
* [http://www.python.org/dev/peps/pep-0008/ Python programming style guide]
* [https://www.python.org/dev/peps/pep-0008/ Python programming style guide]
* [http://wiki.python.org/moin/PythonEditors Python Editors]
* [https://wiki.python.org/moin/PythonEditors Python Editors]


=== Programming ===
=== Programming ===


* Python and GRASS:
* Python and GRASS:
** Library interfaces: [GRASS Python Scripting Library http://grass.osgeo.org/programming7/pythonlib.html]
** Library interfaces: [GRASS Python Scripting Library https://grass.osgeo.org/grass78/manuals/libpython/]
** Graphical user interface (GIU): [GRASS wxPython-based GUI http://grass.osgeo.org/programming7/wxpythonlib.html]
** Graphical user interface (GUI): [GRASS wxPython-based GUI https://grass.osgeo.org/grass-stable/manuals/wxGUI.html]
** PyWPS, GRASS-Web Processing Service: [[WPS]]
** PyWPS, GRASS-Web Processing Service: [[WPS]]


* Python and OSGeo:
* Python and OSGeo:
** [http://wiki.osgeo.org/wiki/OSGeo_Python_Library OSGeo Python Library]
** [https://wiki.osgeo.org/wiki/OSGeo_Python_Library OSGeo Python Library]


* Python and GDAL/OGR:
* Python and GDAL/OGR:
** [http://mapserver.gis.umn.edu/community/conferences/MUM3/workshop/python Open Source Python GIS Hacks Mum'03]
** [https://mapserver.gis.umn.edu/community/conferences/MUM3/workshop/python Open Source Python GIS Hacks Mum'03]
** http://hobu.biz/software/OSGIS_Hacks - Python OSGIS Hacks '05
** https://hobu.biz/software/OSGIS_Hacks - Python OSGIS Hacks '05
** http://zcologia.com/news/categorylist_html?cat_id=8
** https://zcologia.com/news/categorylist_html?cat_id=8
** http://www.perrygeo.net/wordpress/?p=4
** https://www.perrygeo.net/wordpress/?p=4


* Python bindings to PROJ:
* Python bindings to PROJ:
** http://www.cdc.noaa.gov/people/jeffrey.s.whitaker/python/pyproj.html
** https://www.cdc.noaa.gov/people/jeffrey.s.whitaker/python/pyproj.html


* Python and GIS:
* Python and GIS:
** [http://gispython.org/ Open Source GIS-Python Laboratory]
** [https://gispython.org/ Open Source GIS-Python Laboratory]


* Python and Statistics:
* Python and Statistics:
** [http://rpy.sourceforge.net/ RPy] - Python interface to the R-statistics programming language
** [https://rpy.sourceforge.net/ RPy] - Python interface to the R-statistics programming language


* Bindings:
* Bindings:
** SIP (C/C++ bindings generator) http://directory.fsf.org/all/Python-SIP.html
** SIP (C/C++ bindings generator) http://directory.fsf.org/all/Python-SIP.html
** [http://www.cython.org/ Cython] - C-Extensions for Python (compile where speed is needed)
** [https://www.cython.org/ Cython] - C-Extensions for Python (compile where speed is needed)


* Other external projects
* Other external projects
** [http://www.scipy.org Scientific Python]
** [https://www.scipy.org Scientific Python]
** [http://wiki.python.org/moin/NumericAndScientific Numeric and Scientific]
** [https://wiki.python.org/moin/NumericAndScientific Numeric and Scientific]
** [http://w3.pppl.gov/~hammett/comp/python/python.html Info on Python for Scientific Applications]
** [https://w3.pppl.gov/~hammett/comp/python/python.html Info on Python for Scientific Applications]


=== Presentations ===
=== Presentations ===


From FOSS4G2006:
* https://www.slideshare.net/search/slideshow?q=pygrass
* [http://www.foss4g2006.org/materialDisplay.py?contribId=136&amp;sessionId=48&amp;materialId=slides&amp;confId=1 A Python sweeps in the GRASS] - A. Frigeri 2006
 
* [http://www.foss4g2006.org/materialDisplay.py?contribId=67&amp;sessionId=48&amp;materialId=slides&amp;confId=1 GRASS goes web: PyWPS] - J. Cepicky 2006 (see also [[WPS]])
From FOSS4G2006 (initial historical steps):
* [https://www.foss4g2006.org/materialDisplay.py?contribId=136&amp;sessionId=48&amp;materialId=slides&amp;confId=1 A Python sweeps in the GRASS] - A. Frigeri 2006
* [https://www.foss4g2006.org/materialDisplay.py?contribId=67&amp;sessionId=48&amp;materialId=slides&amp;confId=1 GRASS goes web: PyWPS] - J. Cepicky 2006 (see also [[WPS]])
 
=== References ===
 
* Zambelli, P., Gebbert, S., Ciolli, M., 2013. ''Pygrass: An Object Oriented Python Application Programming Interface (API) for Geographic Resources Analysis Support System (GRASS) Geographic Information System (GIS)''. ISPRS International Journal of Geo-Information 2, 201–219. ([https://dx.doi.org/10.3390/ijgi2010201 DOI] | [https://www.mdpi.com/2220-9964/2/1/201/pdf PDF])
 
* GRASS GIS and the Python geospatial modules (Shapely, PySAL,...)
** See https://osgeo-org.1560.x6.nabble.com/GRASS-and-the-Python-geospatial-modules-Shapely-PySAL-td4985075.html


{{Python}}
{{Python}}

Latest revision as of 08:20, 8 December 2020

This page discusses usage of GRASS GIS and Python in a general way. If you just want to write a script to run in GRASS GIS, go to GRASS Python Scripting Library wiki page or Python API for GRASS GIS 7 documentation.

Writing Python scripts in GRASS

Python is a programming language which is more powerful than shell scripting but easier and more forgiving than C. The Python script can contain simple module description definitions which will be processed with g.parser, as shown in the example below. In this way, with no extra coding a GUI can be built, inputs checked, and a skeleton help page can be generated automatically. In addition, it adds links to the GRASS message translation system. The library for "scripting" is "grass.script", typically used as: 

import grass.script as gscript

The related files are at $GISBASE/etc/python/grass/script/*.py. See below for more details.

Note: For code which needs access to the power of C, you can access the GRASS C library functions via the Python "ctypes" interface.

How to write a Python GRASS GIS 7 addon

The tutorial "How to write a Python GRASS GIS 7 addon" is available here: https://github.com/wenzeslaus/python-grass-addon

Integrated Python shell

The wxGUI Layer Manager in GRASS GIS comes with a "Python Shell" which enables users to type and execute Python commands directly in wxGUI environment.

Embedded interactive Python Shell in wxGUI Layer Manager
Embedded interactive Python Shell in wxGUI Layer Manager

Integrated Python editor

The wxGUI Layer Manager in GRASS GIS 7 comes with a "Simple Python Editor" which enables users to author Python scripts directly in the GRASS GIS GUI. Users can also run the script easily in the GRASS GIS environment with all the dependencies loaded. The editor comes with several examples, templates, and links to documentation.

Python interactive shell (console) and a simple Python editor are a powerful option for interacting with GRASS GIS as well as extending its functionality

External Python editors (IDE)

Besides the wxGUI Python shell and script editor, also advanced editors like Spyder (The Scientific PYthon Development EnviRonment) can be used in connection with GRASS GIS. For details, see Tools for Python programming.

The correct editor settings for Python indentation

Be sure to use only white spaces and no tabs to indent Python code!

See: https://trac.osgeo.org/grass/wiki/Submitting/Python#Editorsettingsfor4-spaceindentation

Editor settings for 4-space indentation

  • Geany editor:
    • Edit > Preferences > Editor > Intentation tab > Type: Spaces
  • GNU Emacs:
    • python-mode default
  • spyder ...
  • pycharm ...

Using the GRASS Python Scripting Library

You can run Python scripts easily in a GRASS session.

To write these scripts,

  • check the code in lib/python/ which provides grass.script in order to support GRASS scripts written in Python.
 See the GRASS Python Scripting Library for notes and examples.
  • The scripts/ directory of GRASS contains a series of examples actually provided to the end users.

For the desired Python code style, have a look at Submitting Python.

Creating Python scripts that call GRASS functionality from outside

For calling GRASS functionality from outside, see the documentation and also Working with GRASS without starting it explicitly.

Note: This is a more advanced use case of using GRASS' functionality from outside via Python. Commonly, a user will run GRASS Python script from inside a GRASS session, i.e. either from the command line or from the Python shell embedded in the wxGUI (screenshot).

Running external commands from Python

For information on running external commands from Python, see: http://docs.python.org/lib/module-subprocess.html

Avoid using the older os.* functions. Section 17.1.3 lists equivalents using the Popen() interface, which is more robust (particularly on Windows).

Testing and installing Python extensions

Debugging

Make sure the script is executable: 

   chmod +x /path/to/my.extension.py

During development, a Python script can be debugged using the Python Debugger (pdb): 

   python -m pdb /path/to/my.extension.py input=my_input_layer output=my_output_layer option=value -f

Installation

Once you're happy with your script, you can put it in the scripts/ folder of your GRASS install. To do so, first create a directory named after your extension, then create a Makefile for it, and a HTML man page: 

   cd /path/to/grass_src/
   cd scripts
   ls # it is useful to check out the existing scripts and their structure
   mkdir my.extension
   cd my.extension
   cp path/to/my.extension.py .
   touch my.extension.html
   touch Makefile

Next step is to edit the Makefile. It is a very simple text file, the only thing to check is to put the right extension name (WITHOUT the .py file extension) after PGM: 

   MODULE_TOPDIR = ../..
   
   PGM = my.extension
   
   include $(MODULE_TOPDIR)/include/Make/Script.make
   
   default: script

The HTML file would be generated automatically. If you want to add more precisions in it, you can do it (just make sure you start at DESCRIPTION. See existing scripts.)

In the /path/to/grass_src/scripts folder there is also a Makefile. Before building your new folder name should be added to the SUBDIRS list in this file to be built correctly.

You can then run "make" within the my.extension folder. Running "make" in the extension directory places the resulting files in the staging directory (path/to/grass_src/dist.<YOUR_ARCH>/). If you're running GRASS from the staging directory (/path/to/grass_src/bin.<YOUR_ARCH>/grass7), subsequent commands will used the updated files. 

   # in your extension directory (/path/to/grass_src/scripts/my.extension/)
   make
   # Starting GRASS from the staging directory
   /path/to/grass_src/bin.<YOUR_ARCH>/grass7
   my.extension help

You can also run "make install" from the top level directory of your GRASS install (say /usr/local/src/grass_trunk/). Running "make install" from the top level just copies the whole of the dist.<YOUR_ARCH>/ directory to the installation directory (e.g. /usr/local/grass78) and the bin.<YOUR_ARCH>/grass78 bin file to the bin directory (e.g. /usr/local/bin), and fixes any embedded paths in scripts and configuration files. 

   cd /path/to/grass_src
   make install
   # Starting GRASS as usual would work and show your extension available
   grass7
   my.extension help

Using the parser to get GUI and standardized CLI

Read the documentation for GRASS GIS Python scripting with script package.

For more details, see

Be sure to use the standardized options such as G_OPT_R_INPUT or G_OPT_V_OUTPUT if possible. See the overview table of parser standard options in the documentation.

Using a version of Python different from the default installation

Sometimes you don't want to use the default Python version installed on your system. You can control the interpreter used for building by overriding the PYTHON make variable, e.g. 

 make PYTHON=/path/to/python ...

For controlling the interpreter used at run time:

  • On Windows, Python scripts are invoked via %GRASS_PYTHON%, so changing that environment variable will change the interpreter.
  • On Unix, GRASS_PYTHON is only used for components which use wxPython, e.g. wxGUI or the parameter dialogues which are displayed when modules are run without arguments.

Scripts rely upon the "#!/usr/bin/env python" line. You can control the interpreter used by scripts by creating a directory containing a symlink named "python" which refers to your preferred version of the interpreter, and placing that directory at the front of $PATH.

Python extensions in GRASS GIS

Python Scripting Library

pyGRASS Library

pyGRASS is an object oriented Python Application Programming Interface (API) for GRASS GIS. pyGRASS wants to improve the integration between GRASS and Python, render the use of Python under GRASS more consistent with the language itself and make the GRASS scripting and programming activity easier and more natural to the final users. Some simple examples below.

For more details, see pygrass

Example 1: 

from grass.pygrass.modules import Module

slope_aspect = Module("r.slope.aspect")
slope_aspect(elevation='elevation', slope='slp',  aspect='asp', 
               format='degrees', overwrite=True)

Example 2, using "shortcuts": 

#!/usr/bin/env python

# simple example for pyGRASS usage: raster processing via modules approach

from grass.pygrass.modules.shortcuts import general as g
from grass.pygrass.modules.shortcuts import raster as r

g.message("Filter elevation map by a threshold...")

# set computational region
input = 'elevation'
g.region(raster=input)

# hardcoded:
# r.mapcalc('elev_100m = if(elevation > 100, elevation, null())', overwrite = True)
# with variables

output = 'elev_100m'
thresh = 100.0
r.mapcalc("%s = if(%s > %d, %s, null())" % (output, input, thresh, input), overwrite = True)
r.colors(map=output, color="elevation")


Example 3, using "shortcuts" and external raster format support: 

#!/usr/bin/env python

# simple example for pyGRASS usage: raster processing via Modules approach
# Read GeoTIFF directly, write out GeoTIFF directly
import os
import tempfile
from grass.pygrass.modules.shortcuts import general as g
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules import Module

# use alias name for GRASS GIS commands with more than one dot in name
r.external_out = Module('r.external.out')

# get the raster elevation map into GRASS without import
geotiff = 'elevation.tif' # with path
curr_raster='myraster'
r.external(input=geotiff, output=curr_raster)

# define output directory for files resulting from GRASS calculation:
gisdb = os.path.join(tempfile.gettempdir(), 'gisoutput')
try:
    os.stat(gisdb)
except:
    os.mkdir(gisdb)
g.message("Output will be written to %s" % gisdb)

# define where to store the resulting files and format
r.external_out(directory=gisdb, format='GTiff')

# do the GRASS GIS job
g.message("Filter elevation map by a threshold...")
# set computational region
g.region(raster=curr_raster)

# generate GeoTIFF
output = 'elev_100m.tif'
thresh = 100.0
r.mapcalc("%s = if(%s > %d, %s, null())" % (output, curr_raster, thresh, curr_raster), overwrite = True)
r.colors(map=output, color="elevation")

# cease GDAL mode, back to GRASS data writing mode
r.external_out(flags = 'r')
# the resulting GeoTIFF file is in the gisdb directory

Example 4: 

#!/usr/bin/env python
 
# Example for pyGRASS usage - vector API

from grass.pygrass.modules.shortcuts import general as g
from grass.pygrass.vector import VectorTopo
from grass.pygrass.modules import Module

vectmap = 'myzipcodes_wake'

g.message("Importing SHAPE file ...")
ogrimport = Module('v.in.ogr')
ogrimport('/home/neteler/gis_data/zipcodes_wake.shp', output=vectmap)

g.message("Assessing vector topology...")
zipcodes = VectorTopo(vectmap)

# Open the map with topology:
zipcodes.open()

# query number of topological features
areas   = zipcodes.number_of("areas")
islands = zipcodes.number_of("islands")
print 'Map: <' + vectmap + '> with %d areas and %d islands' % (areas, islands)

dblink = zipcodes.dblinks[0]
print 'DB name:'
print dblink.database
table = dblink.table()
print 'Column names:'
print table.columns.names()
print 'Column types:'
print table.columns.types()

zipcodes.close()

Python Ctypes Interface

This interface allows calling GRASS library functions from Python scripts. See Python Ctypes Examples for details.

Examples:

  • Latest and greatest: GRASS 7 Python scripts

Sample script for GRASS raster access (use within GRASS, Spearfish session): 

#!/usr/bin/env python

## TODO: update example to Ctypes

import os, sys
from grass.lib import grass

if len(sys.argv)==2:
  input = sys.argv[1]
else:
  input = raw_input("Raster Map Name? ")

# initialize
grass.G_gisinit('')

# find map in search path
mapset = grass.G_find_cell2(input, '')

# determine the inputmap type (CELL/FCELL/DCELL) */
data_type = grass.G_raster_map_type(input, mapset)

infd = grass.G_open_cell_old(input, mapset)
inrast = grass.G_allocate_raster_buf(data_type)

rown = 0
while True:
    myrow = grass.G_get_raster_row(infd, inrast, rown, data_type)
    print rown, myrow[0:10]
    rown += 1
    if rown == 476:
        break

grass.G_close_cell(inrast)
grass.G_free(cell)

Sample script for vector access (use within GRASS, Spearfish session): 

#!/usr/bin/python

# run within GRASS Spearfish session
import os, sys
from grass.lib import grass
from grass.lib import vector as grassvect

if len(sys.argv)==2:
  input = sys.argv[1]
else:
  input = raw_input("Vector Map Name? ")

# initialize
grass.G_gisinit('')

# find map in search path
mapset = grass.G_find_vector2(input,'')

# define map structure
map = grassvect.Map_info()

# define open level (level 2: topology)
grassvect.Vect_set_open_level (2)

# open existing map
grassvect.Vect_open_old(map, input, mapset)

# query
print 'Vect map: ', input
print 'Vect is 3D: ', grassvect.Vect_is_3d (map)
print 'Vect DB links: ', grassvect.Vect_get_num_dblinks(map)
print 'Map Scale:  1:', grassvect.Vect_get_scale(map)
print 'Number of areas:', grassvect.Vect_get_num_areas(map)

# close map
grassvect.Vect_close(map)

Using ipython

Using ipython is great due to the TAB completion. Don't remember the function name or curious about what's offered? Hit TAB (maybe twice for a list).

Here an example how to connect to C functions directly from Python: 

grass76
# ... in the GRASS GIS terminal
ipython
In [1]: from grass.lib import gis

In [2]: gis.<TAB>
Display all 729 possibilities? (y or n)
gis.ARRAY                                 gis.G_fatal_longjmp                       gis.G_set_key_value
gis.ArgumentError                         gis.G_file_name                           gis.G_set_keywords
gis.Array                                 gis.G_file_name_misc                      gis.G_set_ls_exclude_filter
...
gis.G_fatal_error                         gis.G_set_gisrc_mode                      gis.wstring_at

In [2]: gis.G_message("Hello world")
Hello world

A full ipython workshop is available here: https://github.com/wenzeslaus/python-grass-addon

wxPython GUI development

3rd party Python software connected to GRASS GIS

FAQ

  • Q: Error message "execl() failed: Permission denied" - what to do?
A: Be sure that the execute bit of the script is set.

Links

General guides

Programming

  • Python and Statistics:
    • RPy - Python interface to the R-statistics programming language

Presentations

From FOSS4G2006 (initial historical steps):

References

  • Zambelli, P., Gebbert, S., Ciolli, M., 2013. Pygrass: An Object Oriented Python Application Programming Interface (API) for Geographic Resources Analysis Support System (GRASS) Geographic Information System (GIS). ISPRS International Journal of Geo-Information 2, 201–219. (DOI | PDF)
Retrieved from "https://grasswiki.osgeo.org/w/index.php?title=GRASS_and_Python&oldid=26418"