GRASS GIS APIs: Difference between revisions

From GRASS-Wiki
Jump to navigation Jump to search
(update to GitHub URLs; grass7x -> grass-stable)
 
(20 intermediate revisions by 3 users not shown)
Line 1: Line 1:
== Introduction to this document ==
== Introduction to this document ==


GRASS GIS is in continues development and this development is what keeps GRASS GIS alive and at the forefront of GIS evolution. From users stringing together a series of modules to build a novel processing chain to developers implementing new cutting-edge algorithms in the core of the GRASS GIS C source code there are many different ways to develop with and for GRASS GIS. This document aims at clarifying the role of the different application programming interfaces (APIs) available. For each of the different APIs, it presents typical use cases and introduces the basic logic of the API. For more detailed information, please look at more detailed introduction on [http://grasswiki.osgeo.org/wiki/Development GRASS GIS and Development ] and [http://grasswiki.osgeo.org/wiki/GRASS_and_Python GRASS GIS and Python] as well as the programming manuals for the [http://grass.osgeo.org/programming7/ C] and [http://grass.osgeo.org/grass71/manuals/libpython/ Python] APIs.
GRASS GIS is in continues development and this development is what keeps GRASS GIS alive and at the forefront of GIS evolution. From users stringing together a series of modules to build a novel processing chain to developers implementing new cutting-edge algorithms in the core of the GRASS GIS C source code there are many different ways to develop with and for GRASS GIS. This document aims at clarifying the role of the different application programming interfaces (APIs) available. For each of the different APIs, it presents typical use cases and introduces the basic logic of the API. For more detailed information, please look at more detailed introduction on [https://grasswiki.osgeo.org/wiki/Development GRASS GIS and Development ] and [https://grasswiki.osgeo.org/wiki/GRASS_and_Python GRASS GIS and Python] as well as the programming manuals for the [https://grass.osgeo.org/programming7/ C] and [https://grass.osgeo.org/grass-stable/manuals/libpython/ Python] APIs.
 
=== The GRASS GIS environment ===
 
'''Note that all of the below code examples require that they are run with necessary GRASS GIS environment variables defined''' in order to allow the system to find the relevant modules and libraries. You can either set up this environment by launching the GRASS GIS startup script or by [[Working_with_GRASS_without_starting_it_explicitly|manually setting up your environment]].


== GRASS GIS Modules as "functions" ==
== GRASS GIS Modules as "functions" ==


GRASS in itself is not a monolithic application, but rather a collection of over 300 applications, called modules, following the [http://en.wikipedia.org/wiki/Unix_philosophy Unix philosophy] with the idea that each module does one thing and does it well, even if that thing is trivial, and that real power comes from combining these tools.
GRASS in itself is not a monolithic application, but rather a collection of over 500 applications, called modules, following the [https://en.wikipedia.org/wiki/Unix_philosophy Unix philosophy] with the idea that each module does one thing and does it well, even if that thing is trivial, and that real power comes from combining these tools.


A consequence of this principle is that each module is autonomous in its functioning, including its memory management and error handling. In other words, as GRASS GIS is not a monolithic application, GRASS as such cannot crash, only individual modules can.
A consequence of this principle is that each module is autonomous in its functioning, including its memory management and error handling. In other words, as GRASS GIS is not a monolithic application, GRASS as such cannot crash, only individual modules can.


Module output in the form of new maps is automatically stored in the GRASS GIS Database and can thus be reused by follow-up modules. Other types of outputs can either be stored in temporary files to be read later by other modules or data can be communicated between modules via the [http://en.wikipedia.org/wiki/Standard_streams standard streams], i.e. the standard output and input.
Module output in the form of new maps is automatically stored in the GRASS GIS Database and can thus be reused by follow-up modules. Other types of outputs can either be stored in temporary files to be read later by other modules or data can be communicated between modules via the [https://en.wikipedia.org/wiki/Standard_streams standard streams], i.e. the standard output and input.


With this structure in mind, one can argue that GRASS GIS in itself is an application programming interface with each module playing the role of a "function". Chaining these functions thus allows any user to create an application of any level of desired complexity. The description of this API can be found in the [http://grass.osgeo.org/documentation/manuals/ module man pages] or on the command line via the --help parameter.
With this structure in mind, one can argue that GRASS GIS in itself is an application programming interface with each module playing the role of a "function". Chaining these functions thus allows any user to create an application of any level of desired complexity. The description of this API can be found in the [https://grass.osgeo.org/learn/manuals/ module man pages] or on the command line via the --help parameter.


Here is a simple example of such an application that extracts estimated stream locations from a digital elevation model, transforms them to vector format and the creates a 500m buffer around them:
Here is a simple example of such an application that extracts estimated stream locations from a digital elevation model, transforms them to vector format and the creates a 500m buffer around them:


g.region rast=elevation
<source lang="Bash">
r.watershed elevation=elevation threshold=10000 stream=raster_streams --o
g.region raster=elevation -p
r.to.vect intput=raster_streams output=vector_streams type=line --o
r.watershed elevation=elevation threshold=10000 stream=raster_streams
v.buffer input=vector_streams output=stream_buffers distance=500 --o
r.to.vect intput=raster_streams output=vector_streams type=line
v.buffer input=vector_streams output=stream_buffers distance=500
</source>


Just putting the above lines into one file that the operating system can read at the command line can already be considered "programming an application" and by just changing the name of the raster used as input the same program can be launched on different data. There are many advantages of creating such application instead of clicking through the different commands in a graphical user interface:
Just putting the above lines into one file that the operating system can read at the command line can already be considered "programming an application" and by just changing the name of the raster used as input the same program can be launched on different data. There are many advantages of creating such application instead of clicking through the different commands in a graphical user interface:
Line 28: Line 34:
However, in this form one is limited to just chaining modules without any control flow such as conditional execution of repeated execution of certain parts of the code. It can, thus be interesting to embed the calls to GRASS GIS modules in a program written in a programming language. Many programming language allow system calls which enable the programmer to call the GRASS GIS modules while having at the same time all the functions of the language at hand.
However, in this form one is limited to just chaining modules without any control flow such as conditional execution of repeated execution of certain parts of the code. It can, thus be interesting to embed the calls to GRASS GIS modules in a program written in a programming language. Many programming language allow system calls which enable the programmer to call the GRASS GIS modules while having at the same time all the functions of the language at hand.


Here are some examples of such system calls (using the first line of the above example) in different programming languages:
Here are some examples of such system calls (using the second line of the above example) in different programming languages. ['''Warning: In many cases, system calls such as these demand that you really know what you are doing. If you want to program in Python, you are encouraged to rather use the existing Python APIs explained below instead of such system calls.''']:


* Python
* Python


import os
<source lang="Python">
os.system('g.region rast=elevation')
import subprocess
 
subprocess.call(['r.watershed', 'elevation=elevation', 'threshold=10000', 'stream=raster_streams'])
* C
</source>
 
void main()
  {
    system("g.region rast=elevation");
  }


* Perl
* Perl


system "g.region rast=elevation";
<source lang="Perl">
@args = ("r.watershed", "elevation=elevation", "threshold=10000", "stream=raster_streams");
system(@args);
</source>


These system calls are easy to handle when no output is expected from the GRASS module. When output needs to be collected then the programming task already becomes a little harder unless you know what you are doing. It is for this reason that the Python GRASS libraries where developed that are explained in the next section.
These system calls are easy to handle when no output is expected from the GRASS module. When output needs to be collected then the programming task already becomes a little harder unless you know what you are doing. Equally, they can be tricky when introducing variable arguments and special characters. It is for this reason that the Python GRASS libraries where developed that are explained in the next section.


== The Python scripting library ==
== The Python scripting library ==


As was mentioned at the end of the previous section, just calling GRASS GIS modules via system calls out of a language can come very handy, but it is also not always straightforward to deal with the output of these modules. As a first solution to that, the [http://grasswiki.osgeo.org/wiki/GRASS_Python_Scripting_Library GRASS Python Scripting Library] was introduced in GRASS 6.
As was mentioned at the end of the previous section, just calling GRASS GIS modules via system calls out of a language can come very handy, but it is also not always straightforward to deal with the output of these modules. As a first solution to that, the [https://grasswiki.osgeo.org/wiki/GRASS_Python_Scripting_Library GRASS Python Scripting Library] was introduced in GRASS 6.


The aim of this library is to provide a light-weight, easy to learn toolkit that eases the interaction with GRASS GIS modules from Python. It mainly offers specialized wrapper functions to facilitate calling grass modules and capturing their output, as well as a few specific functions for frequent tasks (which themselves are again generally just wrapper functions around module calls). It allows to embed the use of GRASS GIS modules into the larger Python environment, without creating a complete pythonic access to the entire GRASS GIS API. It was used to transform all the shell scripts in the GRASS GIS source code to Python scripts in GRASS7.
The aim of this library is to provide a light-weight, easy to learn toolkit that eases the interaction with GRASS GIS modules from Python. It mainly offers specialized wrapper functions to facilitate calling grass modules and capturing their output, as well as a few specific functions for frequent tasks (which themselves are again generally just wrapper functions around module calls). It allows to embed the use of GRASS GIS modules into the larger Python environment, without creating a complete pythonic access to the entire GRASS GIS API. It was used to transform all the shell scripts in the GRASS GIS source code to Python scripts in GRASS7.
Line 58: Line 62:
Here is the same example as above using the run_command() function. This shows that the syntax stays very close to the actual GRASS GIS module calls.
Here is the same example as above using the run_command() function. This shows that the syntax stays very close to the actual GRASS GIS module calls.


import grass.script as grass  
<source lang="Python">
grass.run_command('g.region', rast='elevation')
import grass.script as grass  
grass.run_command('r.watershed', elevation='elevation', threshold=10000, stream='raster_streams', overwrite=True)
grass.run_command('g.region', raster='elevation')
grass.run_command('r.to.vect', input='raster_streams', output='vector_streams', type='line', overwrite=True
grass.run_command('r.watershed', elevation='elevation', threshold=10000, stream='raster_streams', overwrite=True)
grass.run_command('v.buffer', input='vector_streams', output='stream_buffers', distance=500, overwrite=True)
grass.run_command('r.to.vect', input='raster_streams', output='vector_streams', type='line', overwrite=True
grass.run_command('v.buffer', input='vector_streams', output='stream_buffers', distance=500, overwrite=True)
</source>


There are a series of different functions to call modules depending on how output should be handled:
There are a series of different functions to call modules depending on how output should be handled:


#get the current region settings as a list of parameters one per line (can then be treated with the Python splitlines() function
<source lang="Python">
>>> grass.read_command('g.region', flags='g')
>>> # get the current region settings as a list of parameters one per line (can then be treated with the Python splitlines() function
'n=228500\ns=215000\nw=630000\ne=645000\nnsres=10\newres=10\nrows=1350\ncols=1500\ncells=2025000\n'
>>> grass.read_command('g.region', flags='g')
'n=228500\ns=215000\nw=630000\ne=645000\nnsres=10\newres=10\nrows=1350\ncols=1500\ncells=2025000\n'
>>> # get current region settings as a python dictionary:
>>> grass.parse_command('g.region', flags='g')
{'rows': '1350', 'e': '645000', 'cells': '2025000', 'cols': '1500', 'n': '228500', 's': '215000', 'w': '630000', 'ewres': '10', 'nsres': '10'}
>>> # or using a dedicated function as a shortcut (which also directly transforms all numbers from strings to numeric format)
>>> grass.region()
{'rows': 1350, 'e': 645000.0, 'cells': 2025000, 'cols': 1500, 'n': 228500.0, 's': 215000.0, 'w': 630000.0, 'ewres': 10.0, 'nsres': 10.0}
</source>


#get current region settings as a python dictionary:
>>> grass.parse_command('g.region', flags='g')
{'rows': '1350', 'e': '645000', 'cells': '2025000', 'cols': '1500', 'n': '228500', 's': '215000', 'w': '630000', 'ewres': '10', 'nsres': '10'}
#or using a dedicated function as a shortcut (which also directly transforms all numbers from strings to numeric format)
>>> grass.region()
{'rows': 1350, 'e': 645000.0, 'cells': 2025000, 'cols': 1500, 'n': 228500.0, 's': 215000.0, 'w': 630000.0, 'ewres': 10.0, 'nsres': 10.0}


For a description of the different functions to call modules, see the [http://grasswiki.osgeo.org/wiki/GRASS_Python_Scripting_Library#Uses_for_read.2C_feed_and_pipe.2C_start_and_exec_commands relevant section on the dedicated wiki page].
For a description of the different functions to call modules, see the [https://grasswiki.osgeo.org/wiki/GRASS_Python_Scripting_Library#Uses_for_read.2C_feed_and_pipe.2C_start_and_exec_commands relevant section on the dedicated wiki page].


As mentioned, this library is meant as a small toolkit for making it easier to call GRASS GIS modules from Python. It does not, however, offer a real pythonic access to GRASS GIS, and no access to the underlying functions. This is the role of the pygrass library discussed below.
As mentioned, this library is meant as a small toolkit for making it easier to call GRASS GIS modules from Python. It does not, however, offer a real pythonic access to GRASS GIS, and no access to the underlying functions. This is the role of the pygrass library discussed below.
Line 83: Line 91:
=== Graphical programming with the GRASS GIS modeler ===
=== Graphical programming with the GRASS GIS modeler ===


Another easy entry into programming with the GRASS GIS Python scripting library is the [http://grass.osgeo.org/grass70/manuals/wxGUI.gmodeler.html graphical modeler]. Any model created graphically can also be saved as a Python script. For more details see the [http://grasswiki.osgeo.org/wiki/WxGUI_Graphical_Modeler dedicated wiki page].
Another easy entry into programming with the GRASS GIS Python scripting library is the [https://grass.osgeo.org/grass-stable/manuals/wxGUI.gmodeler.html graphical modeler]. Any model created graphically can also be saved as a Python script. For more details see the [httsp://grasswiki.osgeo.org/wiki/WxGUI_Graphical_Modeler dedicated wiki page].


== The PyGRASS API ==
== The PyGRASS API ==


The [http://grass.osgeo.org/grass71/manuals/libpython/pygrass_index.html PyGRASS] API provides two layers of access to GRASS:
The [https://grass.osgeo.org/grass-stable/manuals/libpython/pygrass_index.html PyGRASS] API provides two layers of access to GRASS:


# An access to GRASS modules in the same line as the Python scripting library
# An access to GRASS modules in the same line as the Python scripting library
Line 94: Line 102:
=== Accessing GRASS modules ===
=== Accessing GRASS modules ===


The first layer, the [http://grass.osgeo.org/grass71/manuals/libpython/pygrass_modules.html interface to GRASS GIS modules] provides a unified, object-oriented access to GRASS GIS modules. Through its design modules can be called quite similarly to how they would be called on the command line:
The first layer, the [https://grass.osgeo.org/grass-stable/manuals/libpython/pygrass_modules.html interface to GRASS GIS modules] provides a unified, object-oriented access to GRASS GIS modules. Through its design modules can be called quite similarly to how they would be called on the command line:


>>> from grass.pygrass.modules import Module
<source lang="Python">
>>> Module('g.region', flags='p')
>>> from grass.pygrass.modules import Module
projection: 99 (Lambert Conformal Conic)
>>> Module('g.region', flags='p')
zone:      0
projection: 99 (Lambert Conformal Conic)
datum:      nad83
zone:      0
ellipsoid:  a=6378137 es=0.006694380022900787
datum:      nad83
north:      318500
ellipsoid:  a=6378137 es=0.006694380022900787
south:      -16000
north:      318500
west:      124000
south:      -16000
east:      963000
west:      124000
nsres:      500
east:      963000
ewres:      500
nsres:      500
rows:      669
ewres:      500
cols:      1678
rows:      669
cells:      1122582
cols:      1678
Module('g.region')
cells:      1122582
Module('g.region')
</source>


This is quite similar to the GRASS GIS Python scripting library's run_command() function. However, PyGRASS does not provide wrappers such as the read_command() and parse_command() and thus the developer has to handle output in the same way as it would be handled at the command line, i.e. by manipulating the standard input and standard output streams.
This is quite similar to the GRASS GIS Python scripting library's run_command() function. However, PyGRASS does not provide wrappers such as the read_command() and parse_command() and thus the developer has to handle output in the same way as it would be handled at the command line, i.e. by manipulating the standard input and standard output streams.
In order to mimic even closer the GRASS GIS command line, PyGRASS offers shortcuts to modules which are equivalent to the GRASS GIS modules calls:
<source lang="Python">
>>> from grass.pygrass.modules.shortcuts import general as g
>>> g.region(flags='g')
</source>
<source lang="Python">
>>> from grass.pygrass.modules.shortcuts import vector as v
>>> v.info('schools')
</source>


At the same time, the PyGRASS library allows to treat modules as objects and provides a series of functions for these objects:
At the same time, the PyGRASS library allows to treat modules as objects and provides a series of functions for these objects:


>>> from grass.pygrass.modules import Module
<source lang="Python">
>>> gregion=Module('g.region', flags='g')
>>> from grass.pygrass.modules import Module
>>> gregion
>>> gregion=Module('g.region', flags='g')
Module('g.region')
>>> gregion
>>> gregion.name
Module('g.region')
'g.region'
>>> gregion.name
>>> gregion.description
'g.region'
'Manages the boundary definitions for the geographic region.'
>>> gregion.description
>>> gregion.run()
'Manages the boundary definitions for the geographic region.'
n=318500
>>> gregion.run()
s=-16000
n=318500
w=124000
s=-16000
e=963000
w=124000
nsres=500
e=963000
ewres=500
nsres=500
rows=669
ewres=500
cols=1678
rows=669
cells=1122582
cols=1678
Module('g.region')
cells=1122582
>>> gregion.inputs['raster'].value='elevation'
Module('g.region')
Module('g.region')
>>> gregion.inputs['raster'].value='elevation'
Module('g.region')
</source>


In all, this API to module calls thus allows to integrate modules into a Python object oriented logic and manage them in more flexible ways, but it also demands a bit more understanding from the programmer concerning output handling.
In all, this API to module calls thus allows to integrate modules into a Python object oriented logic and manage them in more flexible ways, but it also demands a bit more understanding from the programmer concerning output handling.
Line 145: Line 169:
It is the second layer of the PyGRASS library that sets it completely apart from the GRASS GIS Python scripting library. This makes it possible to directly access geometry features and a selection of low-level C-functions and thus to program a complex GRASS GIS application in Python.
It is the second layer of the PyGRASS library that sets it completely apart from the GRASS GIS Python scripting library. This makes it possible to directly access geometry features and a selection of low-level C-functions and thus to program a complex GRASS GIS application in Python.


In addition to the modules class mentioned above, the library is divided into different packages. The most important are [http://grass.osgeo.org/grass71/manuals/libpython/pygrass_raster.html raster], [http://grass.osgeo.org/grass71/manuals/libpython/pygrass_vector.html vector] and [http://grass.osgeo.org/grass71/manuals/libpython/pygrass_gis.html gis]. The latter offers the possibility to manage the GRASS database (location, mapset, etc) as well as the working region. The two other packages offer functions to handle the respective map types, but also a direct access to the content of such maps.
In addition to the modules class mentioned above, the library is divided into different packages. The most important are [https://grass.osgeo.org/grass-stable/manuals/libpython/pygrass_raster.html raster], [https://grass.osgeo.org/grass-stable/manuals/libpython/pygrass_vector.html vector] and [https://grass.osgeo.org/grass-stable/manuals/libpython/pygrass_gis.html gis]. The latter offers the possibility to manage the GRASS database (location, mapset, etc) as well as the working region. The two other packages offer functions to handle the respective map types, but also a direct access to the content of such maps.


Here is an example of accessing a vector polygon map:
Here is an example of accessing a vector point map and adding a new point:


  >>> from grass.pygrass.vector import VectorTopo
  >>> from grass.pygrass.vector import VectorTopo
Line 154: Line 178:
  >>> schools.num_primitive_of('point')
  >>> schools.num_primitive_of('point')
  167
  167
  >>> from grass.pygrass.vector.geometry import Point
  >>> from grass.pygrass.vector.geometry import Point
  >>> new_school = Point()
  >>> new_school = Point()
Line 160: Line 185:
  >>> schools.write(new_school)
  >>> schools.write(new_school)
  >>> schools.close()
  >>> schools.close()
  >>> schools.open(mode='r')
  >>> schools.open(mode='r')
  >>> schools.num_primitive_of('point')
  >>> schools.num_primitive_of('point')
  168
  168
  >>> for school in schools:
  >>> for school in schools:
  ...    print(school.id, school)
  ...    print(school.id, school)
Line 173: Line 200:
  >>> schools.close()
  >>> schools.close()


The [grass.osgeo.org/grass71/manuals/libpython/pygrass_raster.html raster package] of the library offers different access types to raster maps: by row, by segment, read-only, read-and-write, as [http://www.numpy.org/ NumPy] array. One can then treat the raster map as any matrix of numbers and write the results of treatments into a new, or the same raster map.
The [grass.osgeo.org/grass-stable/manuals/libpython/pygrass_raster.html raster package] of the library offers different access types to raster maps: by row, by segment, read-only, read-and-write, as [https://www.numpy.org/ NumPy] array. One can then treat the raster map as any matrix of numbers and write the results of treatments into a new, or the same raster map.
   
   
The PyGRASS library thus offers a more sophisticated access to GRASS GIS data and GRASS GIS functions. It contains the necessary functions to read and write raster and vector data that can then be treated with any Python tools the developer wants. It is conceived in a pythonic logic and thus allows a quite seamless integration of GRASS GIS into Python applications.
The PyGRASS library thus offers a more sophisticated access to GRASS GIS data and GRASS GIS functions. It contains the necessary functions to read and write raster and vector data that can then be treated with any Python tools the developer wants. It is conceived in a pythonic logic and thus allows a quite seamless integration of GRASS GIS into Python applications.
Line 179: Line 206:
== The C-API ==
== The C-API ==


* Core library is being developed for more than 30 years
The GRASS GIS core C-library has been in constant development for over 30 years. The fundamental functions were well-designed from the start and many have not changed much since then, but many more functionalities have been added. A selection of major improvements to he C-library were the introduction of floating-point and null support (GRASS 5), a completely new, quite sophisticated vector library including network analysis (GRASS 6), large-file support, significant performance optimization and much better cross-platform usability (GRASS 6-7).
* Different additional libraries added through time
 
The GRASS GIS C-API provides access to the different data types and related functions. There are three main core libraries:
 
* gis: fundamental operations, data structures and GRASS database management
* raster: creation, handling and processing of raster data
* vector: creation, handling and processing of vector data
 
The list of the other libraries is quite long (see the [https://grass.osgeo.org/programming7/ Programmer's Manual] for the complete list) and includes functions for different mathematical calculations, the treatment of satellite imagery, displaying maps, projection handling, 3D-support, etc.
 
A consistent naming scheme is used to ease the identification of the source of a particular function, e.g.
 
* '''G_''' = gis library
* '''Rast_''' = raster handling
* '''Vect_''' = vector handling
* '''I_''' = satellite imagery
* '''PNG_''' = png display driver
etc.


== Easy user interace creation ==
The libraries also define a whole host of data structures, of which some of the most important are:


Whatever the API you chose, when writing GRASS GIS scripts it is very easy to create a user interface, including a graphical user interface, to your script in a semi-automatic way. This interface will then handle all the input (choice of maps, values of parameters, names of outputs, etc) expected from the user. To see the details and examples for the use of this function, see the [http://grass.osgeo.org/grass70/manuals/g.parser.html manual page].
* Cell_head containing metadata concerning a raster map or a region
* Map_info containing metadata concerning a vector map
* field_info containing information about a vector map database connection
etc.


Here is an example showing the option descriptions of the [http://grass.osgeo.org/grass70/manuals/v.db.addcolumn.html v.db.addcolumn] module:
It is beyond the scope of this page to go into detail about the use of the the C-API. This is the role of the [https://grass.osgeo.org/programming7/index.html Programmer's Manual]. Using the C-API to program new modules for GRASS GIS demands a good knowledge of the C-language. The best way to get a first impression on how to use the C-API is to either find an existing module that is similar to the one you want to program, or to start with the two example modules [https://github.com/OSGeo/grass/tree/master/doc/raster/r.example r.example] and [https://github.com/OSGeo/grass/tree/master/doc/vector/v.example v.example], including the related Makefile. Another great starting point, combining both raster and vector library usage in a short program is the module [https://github.com/OSGeo/grass/tree/master/vector/v.to.rast3/main.c v.to.rast3] of which the core loop is reproduced below for illustration (note the use of the gis (G_), 3D-raster (Rast3d_), vector (Vect_), attribute database (db_ ) libraries.
 
    nlines = Vect_get_num_lines(&Map);
    for (line = 1; line <= nlines; line++) {
        int type, cat, depth, row, col, ret;
        double value;
 
        G_percent(line, nlines, 2);
 
        type = Vect_read_line(&Map, Points, Cats, line);
        if (!(type & GV_POINT))
            continue;
        Vect_cat_get(Cats, field, &cat);
        if (cat < 0) {
            continue;
        }
        /* Check if the coordinates are located in the cube */
        if (!Rast3d_is_valid_location(&(map->region), Points->y[0], Points->x[0], Points->z[0])) {
            continue;
        }
        /* Convert the north, east and top coorindate into row, col and depth*/
        Rast3d_location2coord2(&(map->region), Points->y[0], Points->x[0], Points->z[0], &col, &row, &depth);
        if (ctype == DB_C_TYPE_INT) {
            int ivalue;
            ret = db_CatValArray_get_value_int(&cvarr, cat, &ivalue);
            value = ivalue;
        }
        else if (ctype == DB_C_TYPE_DOUBLE) {
            ret = db_CatValArray_get_value_double(&cvarr, cat, &value);
        }
        if (ret != DB_OK) {
            G_warning(_("No record for line (cat = %d)"), cat);
            continue;
        }
        G_debug(3, "col,row,depth,val: %d %d %d %f", col, row, depth, value);
        Rast3d_put_float(map, col, row, depth, (float)value);
    }
 
The C-API is thus meant for people who are willing to learn C-language programming and to spend some time understanding the functions and data structures of the API. The big advantage of the use of this API is to be able access the data directly and to program fast routines for their processing, allowing the implementation of new algorithms, or avoiding a possible performance penalty that chaining existing modules might entail.
 
== Easy user interface creation ==
 
Whatever the API you chose, when writing GRASS GIS scripts it is very easy to create a user interface, including a graphical user interface, to your script in a semi-automatic way. This interface will then handle all the input (choice of maps, values of parameters, names of outputs, etc) expected from the user. To see the details and examples for the use of this function, see the [https://grass.osgeo.org/grass-stable/manuals/g.parser.html manual page].
 
Here is an example showing the option descriptions of the [https://grass.osgeo.org/grass-stable/manuals/v.db.addcolumn.html v.db.addcolumn] module:


  #%module
  #%module
Line 242: Line 338:


[[File:V db addcolumn gui.png|600x400px|framed|center|GUI interface resulting from above option definitions]]
[[File:V db addcolumn gui.png|600x400px|framed|center|GUI interface resulting from above option definitions]]
== Coding standards ==
A final word about coding standards: in order to keep the large code-base of GRASS GIS in a readable and understandable state, the GRASS GIS Development Team has developed over the time a [https://trac.osgeo.org/grass/wiki/Submitting series of coding standards] that any submission of code into the GRASS GIS code base should follow. These coding standards are defined for several different languages, but notably for C and Python code and you are encouraged to read and integrate them into your coding from the start in order to avoid the drag of cleanup later own.
== Third-Party APIs ==
Other software projects have developed APIs that allow using GRASS GIS in their context. The two most prominent are:
* [https://cran.r-project.org/web/packages/rgrass7/index.html rgrass]: R-package to access GRASS data and modules from inside R-code
* [https://docs.qgis.org/3.10/en/docs/user_manual/processing/index.html QGIS processing toolbox]: QGIS toolbox that allows accessing commands from other programs, including GRASS GIS.
== See also ==
* [[Development]]
* [[GRASS Programming Howto]]
* [[GRASS GIS Performance]]
* [[Software requirements specification]]

Latest revision as of 19:40, 31 August 2020

Introduction to this document

GRASS GIS is in continues development and this development is what keeps GRASS GIS alive and at the forefront of GIS evolution. From users stringing together a series of modules to build a novel processing chain to developers implementing new cutting-edge algorithms in the core of the GRASS GIS C source code there are many different ways to develop with and for GRASS GIS. This document aims at clarifying the role of the different application programming interfaces (APIs) available. For each of the different APIs, it presents typical use cases and introduces the basic logic of the API. For more detailed information, please look at more detailed introduction on GRASS GIS and Development and GRASS GIS and Python as well as the programming manuals for the C and Python APIs.

The GRASS GIS environment

Note that all of the below code examples require that they are run with necessary GRASS GIS environment variables defined in order to allow the system to find the relevant modules and libraries. You can either set up this environment by launching the GRASS GIS startup script or by manually setting up your environment.

GRASS GIS Modules as "functions"

GRASS in itself is not a monolithic application, but rather a collection of over 500 applications, called modules, following the Unix philosophy with the idea that each module does one thing and does it well, even if that thing is trivial, and that real power comes from combining these tools.

A consequence of this principle is that each module is autonomous in its functioning, including its memory management and error handling. In other words, as GRASS GIS is not a monolithic application, GRASS as such cannot crash, only individual modules can.

Module output in the form of new maps is automatically stored in the GRASS GIS Database and can thus be reused by follow-up modules. Other types of outputs can either be stored in temporary files to be read later by other modules or data can be communicated between modules via the standard streams, i.e. the standard output and input.

With this structure in mind, one can argue that GRASS GIS in itself is an application programming interface with each module playing the role of a "function". Chaining these functions thus allows any user to create an application of any level of desired complexity. The description of this API can be found in the module man pages or on the command line via the --help parameter.

Here is a simple example of such an application that extracts estimated stream locations from a digital elevation model, transforms them to vector format and the creates a 500m buffer around them:

g.region raster=elevation -p
r.watershed elevation=elevation threshold=10000 stream=raster_streams
r.to.vect intput=raster_streams output=vector_streams type=line
v.buffer input=vector_streams output=stream_buffers distance=500

Just putting the above lines into one file that the operating system can read at the command line can already be considered "programming an application" and by just changing the name of the raster used as input the same program can be launched on different data. There are many advantages of creating such application instead of clicking through the different commands in a graphical user interface:

  • archiving your work flow
  • making your work flow easily reproducible
  • being able to send your work flow to others

However, in this form one is limited to just chaining modules without any control flow such as conditional execution of repeated execution of certain parts of the code. It can, thus be interesting to embed the calls to GRASS GIS modules in a program written in a programming language. Many programming language allow system calls which enable the programmer to call the GRASS GIS modules while having at the same time all the functions of the language at hand.

Here are some examples of such system calls (using the second line of the above example) in different programming languages. [Warning: In many cases, system calls such as these demand that you really know what you are doing. If you want to program in Python, you are encouraged to rather use the existing Python APIs explained below instead of such system calls.]:

  • Python
import subprocess
subprocess.call(['r.watershed', 'elevation=elevation', 'threshold=10000', 'stream=raster_streams'])
  • Perl
@args = ("r.watershed", "elevation=elevation", "threshold=10000", "stream=raster_streams");
system(@args);

These system calls are easy to handle when no output is expected from the GRASS module. When output needs to be collected then the programming task already becomes a little harder unless you know what you are doing. Equally, they can be tricky when introducing variable arguments and special characters. It is for this reason that the Python GRASS libraries where developed that are explained in the next section.

The Python scripting library

As was mentioned at the end of the previous section, just calling GRASS GIS modules via system calls out of a language can come very handy, but it is also not always straightforward to deal with the output of these modules. As a first solution to that, the GRASS Python Scripting Library was introduced in GRASS 6.

The aim of this library is to provide a light-weight, easy to learn toolkit that eases the interaction with GRASS GIS modules from Python. It mainly offers specialized wrapper functions to facilitate calling grass modules and capturing their output, as well as a few specific functions for frequent tasks (which themselves are again generally just wrapper functions around module calls). It allows to embed the use of GRASS GIS modules into the larger Python environment, without creating a complete pythonic access to the entire GRASS GIS API. It was used to transform all the shell scripts in the GRASS GIS source code to Python scripts in GRASS7.

The library is a perfect starting point for beginners who just want to write simple scripts without wanting to dive too deeply into Python philosophy and style.

Here is the same example as above using the run_command() function. This shows that the syntax stays very close to the actual GRASS GIS module calls.

import grass.script as grass 
grass.run_command('g.region', raster='elevation')
grass.run_command('r.watershed', elevation='elevation', threshold=10000, stream='raster_streams', overwrite=True)
grass.run_command('r.to.vect', input='raster_streams', output='vector_streams', type='line', overwrite=True
grass.run_command('v.buffer', input='vector_streams', output='stream_buffers', distance=500, overwrite=True)

There are a series of different functions to call modules depending on how output should be handled:

>>> # get the current region settings as a list of parameters one per line (can then be treated with the Python splitlines() function
>>> grass.read_command('g.region', flags='g')
'n=228500\ns=215000\nw=630000\ne=645000\nnsres=10\newres=10\nrows=1350\ncols=1500\ncells=2025000\n'
>>> # get current region settings as a python dictionary:
>>> grass.parse_command('g.region', flags='g')
{'rows': '1350', 'e': '645000', 'cells': '2025000', 'cols': '1500', 'n': '228500', 's': '215000', 'w': '630000', 'ewres': '10', 'nsres': '10'}
>>> # or using a dedicated function as a shortcut (which also directly transforms all numbers from strings to numeric format)
>>> grass.region()
{'rows': 1350, 'e': 645000.0, 'cells': 2025000, 'cols': 1500, 'n': 228500.0, 's': 215000.0, 'w': 630000.0, 'ewres': 10.0, 'nsres': 10.0}


For a description of the different functions to call modules, see the relevant section on the dedicated wiki page.

As mentioned, this library is meant as a small toolkit for making it easier to call GRASS GIS modules from Python. It does not, however, offer a real pythonic access to GRASS GIS, and no access to the underlying functions. This is the role of the pygrass library discussed below.

Graphical programming with the GRASS GIS modeler

Another easy entry into programming with the GRASS GIS Python scripting library is the graphical modeler. Any model created graphically can also be saved as a Python script. For more details see the [httsp://grasswiki.osgeo.org/wiki/WxGUI_Graphical_Modeler dedicated wiki page].

The PyGRASS API

The PyGRASS API provides two layers of access to GRASS:

  1. An access to GRASS modules in the same line as the Python scripting library
  2. An access to the underlying functions of the C-library thus giving access to GRASS algorithms and geometries directly from Python

Accessing GRASS modules

The first layer, the interface to GRASS GIS modules provides a unified, object-oriented access to GRASS GIS modules. Through its design modules can be called quite similarly to how they would be called on the command line:

>>> from grass.pygrass.modules import Module
>>> Module('g.region', flags='p')
projection: 99 (Lambert Conformal Conic)
zone:       0
datum:      nad83
ellipsoid:  a=6378137 es=0.006694380022900787
north:      318500
south:      -16000
west:       124000
east:       963000
nsres:      500
ewres:      500
rows:       669
cols:       1678
cells:      1122582
Module('g.region')

This is quite similar to the GRASS GIS Python scripting library's run_command() function. However, PyGRASS does not provide wrappers such as the read_command() and parse_command() and thus the developer has to handle output in the same way as it would be handled at the command line, i.e. by manipulating the standard input and standard output streams.

In order to mimic even closer the GRASS GIS command line, PyGRASS offers shortcuts to modules which are equivalent to the GRASS GIS modules calls:

>>> from grass.pygrass.modules.shortcuts import general as g
>>> g.region(flags='g')
>>> from grass.pygrass.modules.shortcuts import vector as v
>>> v.info('schools')

At the same time, the PyGRASS library allows to treat modules as objects and provides a series of functions for these objects:

>>> from grass.pygrass.modules import Module
>>> gregion=Module('g.region', flags='g')
>>> gregion
Module('g.region')
>>> gregion.name
'g.region'
>>> gregion.description
'Manages the boundary definitions for the geographic region.'
>>> gregion.run()
n=318500
s=-16000
w=124000
e=963000
nsres=500
ewres=500
rows=669
cols=1678
cells=1122582
Module('g.region')
>>> gregion.inputs['raster'].value='elevation'
Module('g.region')

In all, this API to module calls thus allows to integrate modules into a Python object oriented logic and manage them in more flexible ways, but it also demands a bit more understanding from the programmer concerning output handling.

Accessing low-level C-functions

It is the second layer of the PyGRASS library that sets it completely apart from the GRASS GIS Python scripting library. This makes it possible to directly access geometry features and a selection of low-level C-functions and thus to program a complex GRASS GIS application in Python.

In addition to the modules class mentioned above, the library is divided into different packages. The most important are raster, vector and gis. The latter offers the possibility to manage the GRASS database (location, mapset, etc) as well as the working region. The two other packages offer functions to handle the respective map types, but also a direct access to the content of such maps.

Here is an example of accessing a vector point map and adding a new point:

>>> from grass.pygrass.vector import VectorTopo
>>> schools = VectorTopo('schools')
>>> schools.open(mode='rw')
>>> schools.num_primitive_of('point')
167

>>> from grass.pygrass.vector.geometry import Point
>>> new_school = Point()
>>> new_school.x = 643550
>>> new_school.y = 216200
>>> schools.write(new_school)
>>> schools.close()

>>> schools.open(mode='r')
>>> schools.num_primitive_of('point')
168

>>> for school in schools:
...     print(school.id, school)
...
(1, Point(633649.285674, 221412.944348))
(2, Point(628787.129283, 223961.620521))
[...]
(167, Point(650870.509540, 247064.343249))
(168, Point(643550.000000, 216200.000000))
>>> schools.close()

The [grass.osgeo.org/grass-stable/manuals/libpython/pygrass_raster.html raster package] of the library offers different access types to raster maps: by row, by segment, read-only, read-and-write, as NumPy array. One can then treat the raster map as any matrix of numbers and write the results of treatments into a new, or the same raster map.

The PyGRASS library thus offers a more sophisticated access to GRASS GIS data and GRASS GIS functions. It contains the necessary functions to read and write raster and vector data that can then be treated with any Python tools the developer wants. It is conceived in a pythonic logic and thus allows a quite seamless integration of GRASS GIS into Python applications.

The C-API

The GRASS GIS core C-library has been in constant development for over 30 years. The fundamental functions were well-designed from the start and many have not changed much since then, but many more functionalities have been added. A selection of major improvements to he C-library were the introduction of floating-point and null support (GRASS 5), a completely new, quite sophisticated vector library including network analysis (GRASS 6), large-file support, significant performance optimization and much better cross-platform usability (GRASS 6-7).

The GRASS GIS C-API provides access to the different data types and related functions. There are three main core libraries:

  • gis: fundamental operations, data structures and GRASS database management
  • raster: creation, handling and processing of raster data
  • vector: creation, handling and processing of vector data

The list of the other libraries is quite long (see the Programmer's Manual for the complete list) and includes functions for different mathematical calculations, the treatment of satellite imagery, displaying maps, projection handling, 3D-support, etc.

A consistent naming scheme is used to ease the identification of the source of a particular function, e.g.

  • G_ = gis library
  • Rast_ = raster handling
  • Vect_ = vector handling
  • I_ = satellite imagery
  • PNG_ = png display driver

etc.

The libraries also define a whole host of data structures, of which some of the most important are:

  • Cell_head containing metadata concerning a raster map or a region
  • Map_info containing metadata concerning a vector map
  • field_info containing information about a vector map database connection

etc.

It is beyond the scope of this page to go into detail about the use of the the C-API. This is the role of the Programmer's Manual. Using the C-API to program new modules for GRASS GIS demands a good knowledge of the C-language. The best way to get a first impression on how to use the C-API is to either find an existing module that is similar to the one you want to program, or to start with the two example modules r.example and v.example, including the related Makefile. Another great starting point, combining both raster and vector library usage in a short program is the module v.to.rast3 of which the core loop is reproduced below for illustration (note the use of the gis (G_), 3D-raster (Rast3d_), vector (Vect_), attribute database (db_ ) libraries.

   nlines = Vect_get_num_lines(&Map);
   for (line = 1; line <= nlines; line++) {
       int type, cat, depth, row, col, ret;
       double value;
 
       G_percent(line, nlines, 2);
 
       type = Vect_read_line(&Map, Points, Cats, line);
       if (!(type & GV_POINT))
           continue;

       Vect_cat_get(Cats, field, &cat);
       if (cat < 0) {
           continue;
       }
       /* Check if the coordinates are located in the cube */
       if (!Rast3d_is_valid_location(&(map->region), Points->y[0], Points->x[0], Points->z[0])) {
           continue;
       }
       /* Convert the north, east and top coorindate into row, col and depth*/
       Rast3d_location2coord2(&(map->region), Points->y[0], Points->x[0], Points->z[0], &col, &row, &depth);

       if (ctype == DB_C_TYPE_INT) {
           int ivalue;

           ret = db_CatValArray_get_value_int(&cvarr, cat, &ivalue);
           value = ivalue;
       }
       else if (ctype == DB_C_TYPE_DOUBLE) {
           ret = db_CatValArray_get_value_double(&cvarr, cat, &value);
       }

       if (ret != DB_OK) {
           G_warning(_("No record for line (cat = %d)"), cat);
           continue;
       }

       G_debug(3, "col,row,depth,val: %d %d %d %f", col, row, depth, value);

       Rast3d_put_float(map, col, row, depth, (float)value);
   }

The C-API is thus meant for people who are willing to learn C-language programming and to spend some time understanding the functions and data structures of the API. The big advantage of the use of this API is to be able access the data directly and to program fast routines for their processing, allowing the implementation of new algorithms, or avoiding a possible performance penalty that chaining existing modules might entail.

Easy user interface creation

Whatever the API you chose, when writing GRASS GIS scripts it is very easy to create a user interface, including a graphical user interface, to your script in a semi-automatic way. This interface will then handle all the input (choice of maps, values of parameters, names of outputs, etc) expected from the user. To see the details and examples for the use of this function, see the manual page.

Here is an example showing the option descriptions of the v.db.addcolumn module:

#%module
#% description: Adds one or more columns to the attribute table connected to a given vector map.
#% keyword: vector
#% keyword: attribute table
#% keyword: database
#%end

#%option G_OPT_V_MAP
#%end

#%option G_OPT_V_FIELD
#% label: Layer number where to add column(s)
#%end

#%option
#% key: columns
#% type: string
#% label: Name and type of the new column(s) ('name type [,name type, ...]')
#% description: Data types depend on database backend, but all support VARCHAR(), INT, DOUBLE PRECISION and DATE
#% required: yes
#%end

The result of this definition are the following command line parameters:

> v.db.addcolumn --help

Description:
 Adds one or more columns to the attribute table connected to a given vector map.

Keywords:
 vector, attribute table, database

Usage:
 v.db.addcolumn map=name [layer=string] columns=string [--help]
   [--verbose] [--quiet] [--ui]

Flags:
 --h   Print usage summary
 --v   Sortie du module en mode bavard
 --q   Sortie du module en mode silence
 --ui  Force launching GUI dialog

Parameters:
      map   Name of vector map
             Or data source for direct OGR access
    layer   Layer number where to add column(s)
             Vector features can have category values in different layers. This number determines which layer to use. When used with direct OGR access this is the layer name.
            par défaut: 1
  columns   Name and type of the new column(s) ('name type [,name type, ...]')
             Data types depend on database backend, but all support VARCHAR(), INT, DOUBLE PRECISION and DATE

The graphical result is this:

GUI interface resulting from above option definitions

Coding standards

A final word about coding standards: in order to keep the large code-base of GRASS GIS in a readable and understandable state, the GRASS GIS Development Team has developed over the time a series of coding standards that any submission of code into the GRASS GIS code base should follow. These coding standards are defined for several different languages, but notably for C and Python code and you are encouraged to read and integrate them into your coding from the start in order to avoid the drag of cleanup later own.

Third-Party APIs

Other software projects have developed APIs that allow using GRASS GIS in their context. The two most prominent are:

  • rgrass: R-package to access GRASS data and modules from inside R-code
  • QGIS processing toolbox: QGIS toolbox that allows accessing commands from other programs, including GRASS GIS.

See also