GRASS 7 ideas collection: Difference between revisions

From GRASS-Wiki
Jump to navigation Jump to search
(→‎remove: r.univar.sh is gone)
 
(95 intermediate revisions by 10 users not shown)
Line 1: Line 1:
{{MoveToTrac}}
{{MoveToTrac}}
  New home: https://trac.osgeo.org/grass/wiki/Grass7Planning
  See also: http://trac.osgeo.org/grass/wiki/Grass7/NewFeatures
== Maintenance of repository ==
== Maintenance of repository ==


For grass7 development will be used svn-trunk, for grass6 development is used separated branch [https://svn.osgeo.org/grass/grass/branches/develbranch_6 develbranch_6].
For GRASS 7 development is used <strike>[http://svn.osgeo.org/grass/grass/trunk/ svn-trunk]</strike> '''https://github.com/OSGeo/grass/''' <strike>, for GRASS 6 development is used separated SVN branch [https://svn.osgeo.org/grass/grass/branches/develbranch_6 develbranch_6]</strike>.


   Planning is continued here: http://trac.osgeo.org/grass/wiki/Grass7Planning
   '''Planning is continued here: http://trac.osgeo.org/grass/wiki/Grass7Planning'''


== Terminology ==
== Terminology ==


See [[GRASS 7 Terminology]].
See [[GRASS 7 Terminology]].
== List of new features in GRASS 7 (already implemented) ==
See http://trac.osgeo.org/grass/wiki/Grass7/NewFeatures
== API to access algorithms in modules ==
We need to better expose the "knowledge" which is contained at module level. We want to have it accessible via API, exposed in various programming languages such as C, Python, Perl, Java, ..
Update 1/2011: Python ctypes interface available, also PyGRASS, see http://grass.osgeo.org/grass71/manuals/libpython/
See how the temporal modules and temporal library is done.
== Dead code cleanup ==
''? unclear: Trash or move to addons?''
* general/g.setproj - partially replaced by `g.proj -c` but contains still useful logic
* vector/v.label.sa see {{trac|1942}} and {{rev|34923}}. It probably shouldn't be dealing with FreeType directly. Glynn suggests using the display API, specifically D_get_text_box() rather than trying to calculate a skyline.
''Keep:''
* imagery/i.ortho.photo/i.ortho.init - i.ortho.photo should remain:  {{trac|2461}}
* imagery/i.ortho.photo/i.ortho.transform - i.ortho.photo should remain: {{trac|2461}}
* raster/r.watershed/shed - keep for a future reference (e.g., for development of a wizard in a GSoC)
'''IMPORTANT:'''
* search and clean up manual references, too
* update accordingly http://trac.osgeo.org/grass/wiki/Grass7/NewFeatures#Removedmodules


== Raster ==
== Raster ==


See also [[Replacement raster format]]
The raster data API has been updated. See also
* [http://trac.osgeo.org/grass/wiki/Grass7/RasterLib/ListOfFunctions Function name changes from GRASS 6 to GRASS 7]


=== Library ===
=== Raster Library ===
* allowing nulls to be embedded
* allowing nulls to be embedded (Merge NULL file into main data array; this approach is implemented in the G3d library) --> [http://trac.osgeo.org/grass/wiki/Grass8Planning GRASS GIS 8]
* Split libgis into G_() part and Rast_() part
* Rewrite library from scratch. See [http://grass.itc.it/pipermail/grass-dev/2006-August/025025.html suggestions]
* Insert 'vertical' 2d rasters (e.g. [http://woodshole.er.usgs.gov/project-pages/longislandsound/images/Ghist_square2.jpg geophysical survey data])
* Insert 'vertical' 2d rasters (e.g. [http://woodshole.er.usgs.gov/project-pages/longislandsound/images/Ghist_square2.jpg geophysical survey data])
* [http://freegis.org/cgi-bin/viewcvs.cgi/grass/gips/gip-0002.txt?rev=HEAD&content-type=text/vnd.viewcvs-markup GRASS raster "live links" via GDAL]
* Rewrite library from scratch. See [http://lists.osgeo.org/pipermail/grass-dev/2006-August/025153.html suggestions] --> [http://trac.osgeo.org/grass/wiki/Grass8Planning GRASS GIS 8]
* <strike>Split libgis into [http://trac.osgeo.org/grass/wiki/Grass7RasterLib G_() part and Rast_() part]</strike> - done.
* <strike>[http://freegis.org/cgi-bin/viewcvs.cgi/grass/gips/gip-0002.txt?rev=HEAD&content-type=text/vnd.viewcvs-markup GRASS raster "live links" via GDAL]</strike> - done.
 
Raster map "history" metadata:
* In 7.0, the fields of the history structure are dynamically allocated. You have to use Rast_set_history() or Rast_format_history() to set fields. The the HIST_* constants have to be used.
 
About cell_misc/null file size:
 
The size of the cell_misc/<mapname>/null file depends only upon the number of rows and columns, not the data. It's one bit (null/non-null) for each cell, rounded to an integral number of bytes per row (i.e. the number of columns is rounded up to the nearest multiple of 8).
 
=== Segment Library ===
 
(from {{trac|2296}}): we should look into unifying all of the different "segment" libraries.
 
They all do essentially the same thing: provide a 2-dimensional array which may be too large to fit into RAM (or, more accurately, into the process' address space; if RAM was the issue, mmap() etc would suffice), and which can be accessed (more or less) randomly.
 
Apart from the "official" segment library (lib/segment), r.proj has its own, r.stream.* each have their own, r.grow.distance has something simpler (the temporary file is read row-by-row but in reverse).


=== Modules ===
=== Modules ===


==== rename ====
==== rename ====
* rename r.in.gdal to r.import
* <strike>rename r.in.gdal to r.import - see [http://lists.osgeo.org/pipermail/grass-dev/2008-October/thread.html#40450 discussion]</strike> - new {{cmd|r.import|version=71}}
* rename r.out.gdal to r.export
* rename r.out.gdal to r.export - see [http://lists.osgeo.org/pipermail/grass-dev/2008-October/thread.html#40450 discussion]
* rename r.volume 'data' parameter to something more standard like 'input' or 'map'
* <strike>rename r.volume 'data' parameter to something more standard like 'input'</strike> - '''done.'''
<strike>* r.stats vs r.statistics vs r.statistics2 vs r.statistics3:</strike>
** <strike>r.statistics2 has been renamed to r.stats.zonal</strike>
** <strike>r.statistics3 has been renamed to r.stats.quantile</strike>


==== remove ====
==== remove ====
Line 32: Line 83:
* Remove r.in.arc and r.out.arc, '''if''' a [http://intevation.de/rt/webrt?serial_num=4897 related bug in r.in.gdal] is fixed. The [http://bugzilla.remotesensing.org/show_bug.cgi?id=1071 integer/floating point detection for AAIGrid driver in GDAL] was fixed after 1.3.2 release, so r.in.gdal and r.out.gdal should be enough now.
* Remove r.in.arc and r.out.arc, '''if''' a [http://intevation.de/rt/webrt?serial_num=4897 related bug in r.in.gdal] is fixed. The [http://bugzilla.remotesensing.org/show_bug.cgi?id=1071 integer/floating point detection for AAIGrid driver in GDAL] was fixed after 1.3.2 release, so r.in.gdal and r.out.gdal should be enough now.
: see delta comment about r.out.tiff below, sometimes the simple stuff works best! --HB
: see delta comment about r.out.tiff below, sometimes the simple stuff works best! --HB
:: Ditto.


* remove '''r.resample''' and <strike>'''r.bilinear'''</strike> in favor of '''r.resamp.interp'''
* remove '''r.resample''' and <strike>'''r.bilinear'''</strike> in favor of '''r.resamp.interp'''
: TODO: double-check that r.resample is in fact fully replaced by 'r.resamp.interp's method=nearest'. ie check that an alternate useful method is not lost.
: TODO: double-check that r.resample is in fact fully replaced by 'r.resamp.interp's method=nearest'. ie check that an alternate useful method is not lost.
* <strike>remove '''r.univar.sh'''; newly implemented '''r.univar''' features cover it</strike> '''done.'''
* <strike>remove '''r.univar.sh'''; newly implemented '''r.univar''' features cover it</strike> - '''done.'''


* remove r.out.tiff. New C r.out.gdal should cover all it's option now (doublecheck!). See [http://intevation.de/rt/webrt?serial_num=3680 RT #3680] (starting with date Sun, Nov 26 2006 14:54:23).
* <strike>remove r.out.tiff</strike>. New C r.out.gdal should cover all it's option now (doublecheck!). See [http://intevation.de/rt/webrt?serial_num=3680 RT #3680] (starting with date Sun, Nov 26 2006 14:54:23).
: It might be worth keeping r.out.tiff! It makes a nice delta when things don't go well (eg [https://svn.qgis.org/trac/ticket/348 QGIS bug#348]) --HB
: It might be worth keeping r.out.tiff! It makes a nice delta when things don't go well (eg [https://svn.qgis.org/trac/ticket/348 QGIS bug#348]) --HB
:: However: code duplication, maintenance overhead, user confusion (more entries in GUI, more manual pages, why are there modules doing the same?).
::: They do not do the same. r.out.gdal outputs data values, r.out.tiff outputs color values. Two completely different things. --HB


* Remove remaining -v and -q flags for verbosity levels of modules.
* Remove remaining -v and -q flags for verbosity levels of modules.
Line 44: Line 98:
==== Merge ====
==== Merge ====


* merge '''r.surf.idw''' and '''r.surf.idw2'''
* merge {{cmd|r.surf.idw|version=70}} and {{cmd|r.surf.idw2|version=70}}
: while r.surf.idw will only output CELL maps, it is Very Fast.
: while r.surf.idw will only output CELL maps, it is Very Fast. (or that is to say, ''r.surf.idw2'' is Very Slow)
 
* {{cmd|r.mode|version=70}}, {{cmd|r.statistics|version=70}}, {{cmd|r.univar.sh}}, {{cmd|r.univar}}(2) - maybe they can be reduced to just r.statistics and r.univar? See [http://intevation.de/rt/webrt?serial_num=1848 RT #1848] and a [http://grass.itc.it/pipermail/grass-dev/2006-November/027665.html thread on the GRASS dev list]


* r.sum, r.mode, r.median, r.average, r.statistics, r.univar, r.univar2 - maybe they can be reduced to just r.statistics and r.univar? See [http://intevation.de/rt/webrt?serial_num=1848 RT #1848] and a [http://grass.itc.it/pipermail/grass-dev/2006-November/027665.html thread on the GRASS dev list]
* {{cmd|r.resamp.rst|version=70}}: merge into {{cmd|r.resamp.interp|version=70}} to make resolution management identical to the other modules
: HB: eh? the options and algorithm are nothing alike.
:: MS: I meant that r.resamp.rst could be a subset of r.resamp.interp (yet another resampling algorithm next to nearest, bilinear, bicubic). I haven't considered that the number of rst options would make r.resamp.interp user interface much less clear. Maybe not such a good idea after all - user interface wise.


* r.resamp.rst: merge into r.resamp.interp to make resolution management identical to the other modules
* {{cmd|r.in.wms|version=70}} and {{cmd|r.in.srtm|version=70}} into {{cmd|r.in.gdal|version=70}} thanks to native support for [http://www.gdal.org/frmt_wms.html WMS] and [http://www.gdal.org/frmt_various.html#SRTMHGT SRTM] in GDAL 1.5
: HB: just be careful that the GDAL version is as featureful and grid/cell center correct as the r.in.* versions. I suspect it might not be. r.in.wms needs further python rewrite with full XML and HTTP library support.


* r.in.wms and r.in.srtm into r.in.gdal thanks to native support for [http://www.gdal.org/frmt_wms.html WMS] and [http://www.gdal.org/frmt_various.html#SRTMHGT SRTM] in GDAL 1.5
* {{cmd|r.buffer|version=70}} and {{cmd|r.grow|version=70}}
* '''r.buffer''' and '''r.grow'''
: HB: why? they do two fundamentally different things, and both work quite nicely right now. One works in cell space, the other geographic space (especially for lat/lon).
:: MS: Right. The 2 modules do different things. But it would be usefull if r.grow supported distance in units and r.buffer in cells. Could both share same code for distance options?


==== fix ====
==== fix ====
* fix lseek() usage for Large File Support: see [http://grass.itc.it/pipermail/grass-dev/2006-December/028231.html list of affected modules]
* fix the raster map history management (truncating long history, odd storage). It should work like for vector maps in GRASS 6.
* fix the raster map history management (truncating long history, odd storage). It should work like for vector maps in GRASS 6.
* r.volume centroids parameter only makes a level one vector with no attribute table; module should be updated to GRASS 6 vector library)
* {{cmd|r.volume|version=70}} centroids parameter only makes a level one vector with no attribute table; module should be updated to GRASS 6 vector library)
* {{cmd|r.random|version=70}} should be split into 2 modules: one for generating a raster map with random points (alike v.random), and the other for sampling a raster map (alike {{cmd|v.what.rast|version=70}}). Vector functionality should be droped from r.random - a dupe of existing vector modules. -i and -z flags should be droped.
* {{cmd|v.random|version=70}} -z: read zmin and zmax from region settings, drop zmin and zmax. I.e. treat Z coord same as X,Y.


==== Good coding practice ====
==== Good coding practice ====
Line 122: Line 183:
[http://freegis.org/cgi-bin/viewcvs.cgi/grass6/doc/vector/TODO?rev=HEAD&content-type=text/vnd.viewcvs-markup Vector TODO list]
[http://freegis.org/cgi-bin/viewcvs.cgi/grass6/doc/vector/TODO?rev=HEAD&content-type=text/vnd.viewcvs-markup Vector TODO list]
* Particularly important: "Keep topology and spatial index in file instead of in memory" --ML
* Particularly important: "Keep topology and spatial index in file instead of in memory" --ML
* v.in.ogr: split long boundaries to speed up topology cleaning. Implemented in GRASS 7, partial backport to GRASS 6.5 possible.
{|
||
[[Image:V.in.ogr_speed.png|400px|thumb|left|v.in.ogr speed comparison]]
|}


=== Library ===
=== Library ===
Line 132: Line 198:


==== rename ====
==== rename ====
* rename v.in.ogr to v.import
* <strike>rename v.in.ogr to v.import - see [http://lists.osgeo.org/pipermail/grass-dev/2008-October/thread.html#40450 discussion]</strike> - new {{cmd|v.import|version=71}}
* rename v.out.ogr to v.export
* rename v.out.ogr to v.export - see [http://lists.osgeo.org/pipermail/grass-dev/2008-October/thread.html#40450 discussion]
* rename v.mkgrid to v.grid
* rename v.mkgrid to v.grid - see [http://lists.osgeo.org/pipermail/grass-dev/2008-October/thread.html#40450 discussion]
* rename v.univar.sh to v.db.univar ([http://grass.itc.it/pipermail/grass-dev/2007-May/030883.html comment])
* rename v.univar.sh to v.db.univar ([http://grass.itc.it/pipermail/grass-dev/2007-May/030883.html comment])


Line 145: Line 211:
: needs discussion, they are doing fundamentally different things --HB
: needs discussion, they are doing fundamentally different things --HB


* merge v.sample and v.what.rast
* merge {{cmd|v.sample}} and {{cmd|v.what.rast}}
: See a feature request [http://wald.intevation.org/tracker/index.php?func=detail&aid=506&group_id=21&atid=188 #506] in GForge.
: See a feature request <strike>[http://wald.intevation.org/tracker/index.php?func=detail&aid=506&group_id=21&atid=188 #506]</strike> in GForge.
: see also {{cmd|v.rast.stats}} and {{AddonCmd|v.what.rast.buffer}} addon


==== fix ====
==== fix ====
Line 162: Line 229:
* extend v.overlay to allow combination of all types (point, line, area)
* extend v.overlay to allow combination of all types (point, line, area)
* v.category: add possibility to create new layer categories on the basis of a column in the table linked to an existing layer
* v.category: add possibility to create new layer categories on the basis of a column in the table linked to an existing layer
* v.generalize: add point cloud data reduction, e.g. with http://www.cs.umd.edu/~mount/ANN/ (ANN: A Library for
Approximate Nearest Neighbor Searching)
=== 3D topology ===
Currently, GRASS' topological cleaning is 2D only. This will regularly corrupt 3D data that is actually topologically correct in 3D space, but appears incorrect in 2D space (e.g. 3D polygons that ''appear'' to overlap or have zero-area in the X-Y geographic reference plane). Therefore, ''v.clean'' in GRASS 7 should become more competent at handling 3D topology. This is a sketch of how to implement the same level of topology support that 2D geometries currently have in 3D space.
====Problem structure====
The classes of potential topological errors depend on the geometry type (point, line, polygon), in 3D just as in 2D. Each more complex geometry type inherits the potential problems of the less complex types, and adds its own.
*3D topological problems for points reduce to coincidence in 3D space.
*3D topological problems for lines are over- and undershoots as well as duplicate vertices. These are completely equivalent to the same problem in 2D space.
*3D topological problems for polygons appear much more complex, as they include areal overlap in 3D, and shared boundaries between adjacent polygons. But this complexity can be reduced significantly. Only polygons that lie in the same plane (including a small error margin -- could be implemented just like the minimum error option in ''v.clean'') can get into topological trouble with each other: if two polygons do not lie in the same plane, then by definition they cannot overlap, only intersect. (It is open to debate, whether an intersection of polygons in 3D space would constitute a topological error.) Likewise, adjacent polygons can only have a shared boundary if they are neighbors on the same plane.
====Basic approach====
As a result, the topological cleaning could proceed as follows.
#All 3D polygons need to be '''planar''', i.e. all of their vertices must lie on the same plane. If they are not, then this is a topological error, and the polygon needs to be planarized by ''v.clean'' (add a "planarize" action -- also potentially useful for other geometry types). Planarization is done by first calculating the plane equation of the polygon (a simple equation that defines the location and orientation of a best-fit 2D plane through all vertices) and then reprojecting each of its original vertices to the plane defined by the plane equation (it could make sense to store the plane equations as part of the topological data structure). Planarization needs to be carried out for both boundaries and islands.
#Once planarized, all polygons can be subjected to the exact same topological cleaning as 2D polygons:
##After planarization, group all polygons that lie on the same plane.
##For each "plane group": reproject all vertices to the geographic X-Y reference plane. Run the topological cleaning just like for 2D polygons, but preserve the Z coordinates.
##Then revert the projection of all vertices back to the group's original reference plane, using the plane equation.
##Do the same with the next "plane group" until all polygons have been processed.
#Faces (3D triangles) are primitives that are part of more complex 3D geometries (including TINs and 3D hulls). Topological problems involving such geometries are too complex to handle in GIS. Thus, faces should not be checked for overlap. GRASS GIS should just assume that faces are part of complex geometries that are not to be subjected to analytical treatment, but simple tasks, such as visualization, only. Therefore, only basic cleaning (check for duplicate coordinates, zero-area triangles, etc.) should be carried out. Faces are always planar by definition.
#The areas and centroids of both faces and planar polygons can be calculated in 3D space just as they can be calculated in 2D space.
====Implementation details====
A simple algorithm for computing the plane equation of a set of vertices is <missing URL> available in the form of Newell's Method. This method requires three points to define a plane. This requirement is met by even the most simply polygons. However <missing URL>: "Newell's method is known to fail if the 3 points are chosen around a concave corner - the normal of the resulting plane will point in the direction opposite to the expected one." (This should be avoidable by taking three vertices at the extreme edges.)
Since the planar equation is a best-fit model, not always an exact solution, a warning should be issued if an error threshold is reached.
=== 3D point clouds ===
* see http://pointclouds.org/documentation/ library
* evaluate change from http://liblas.org (BSD license) to http://lastools.org (license is LGPL but tools are not)


== General ==
== General ==
Line 179: Line 284:
=== Library ===
=== Library ===


* <strike>establish SQLite as default DBMI driver (DBF is too limited).</strike>  '''done May 2008.'''
* <strike>establish SQLite as default DBMI driver for vector attribute storage (DBF is too limited).</strike>  '''done May 2008.'''


=== Modules ===
=== Modules ===
* allow cross-mapset database access, i.e. parse the '@mapset' notation appended to vector names (requires access via possibly different DBMI drivers)
* allow cross-mapset database access, i.e. parse the '@mapset' notation appended to vector names (requires access via possibly different DBMI drivers)
==== rename ====
* rename db.in.ogr to db.import
* rename db.in.ogr to db.import


== Imagery ==
== Imagery ==
=== Library ===


Do merge of image libraries:
Page has been moved to Trac: http://trac.osgeo.org/grass/wiki/Grass7/ImageryLib
 
* A)
** lib/imagery/: standard lib, in use (i.* except for i.points3, i.rectify3)
** imagery/i.ortho.photo/libes/: standard lib, in use (i.ortho.photo, photo.*)
* B)
** lib/image3/: never finished improvement which integrated the standard lib and the ortho lib. Seems to provide also ortho rectification for satellite data (i.points3, i.rectify3)
 
=== Modules ===
 
* merge of i.points, i.vpoints, i.points3
: drop these altogether? (loss of interactive XDRIVER)
* merge of i.rectify and i.rectify3
: maybe depend on gdalwarp C API for this?
* addition of new resampling algorithms such as bilinear, cubic convolution (take from r.proj?)
* add other warping methods (maybe thin splines from GDAL?)
* implement/finish linewise ortho-rectification of satellite data
* Depreciate tape functions in next major revision of GRASS and create a tape module that accomplishes tape access.


== Raster3D ==
== Raster3D ==
Line 212: Line 301:
* extent/rewrite documentation  
* extent/rewrite documentation  
* localisation support (why wait for GRASS 7 ??)
* localisation support (why wait for GRASS 7 ??)
See http://trac.osgeo.org/grass/wiki/Grass7/G3dLib


=== Modules ===
=== Modules ===
Line 221: Line 312:
== Display ==
== Display ==


=== New display architecture ===
The page moved to Trac: http://trac.osgeo.org/grass/wiki/Grass7/DisplayLib
 
''Comments from Glynn Clements (from [http://www.nabble.com/Cairo-monitor-driver-tf4620004.html#a13206009 here]):''
 
# Eliminating separate driver processes. Having to extend the protocol for each new operation has been a major drag on development.
# Use of floating-point for all graphics coordinates. For geographic data, the expected approach will be to set an appropriate transform then use cartographic coordinates.
# Common architecture for both video and hardcopy output. ps.map should be redundant.
 
In retrospect, #1 means that we don't really need support for the Windows/MacOSX GUI, just the ability to generate image files.
 
With X, we could take the shortcut of having d.* commands draw directly into an existing window, but I don't know whether that's possible on other platforms.
 
OTOH, it might be useful to be able to use the same functions for self-contained GUI programs (e.g. vector digitising) as for d.* commands.
 
=== Library ===
* Drop support for interactive xmon modules
 
=== Modules ===
* d.font etc.
** Huidae Cho merged d.text.freetype and d.text into d.text.new; drop them and rename d.text.new into d.text
** merge d.font and d.font.freetype too
: now done in 6.3?
 
* d.vect
** consolidate parameter names (attrcol, wcolumn, rgb_column)
: how?
 
* remove d.ask, d.menu, d.linegraph(?), d.mapgraph, d.text.freetype, d.paint.labels (symlink), d.font.*
 
* remove d.m
 
* d.legend, d.barscale, d.text, etc: make at=0,0 origin identical!
: FWIW, I copied the d.legend at= syntax from d.frame --HB


== Postscript ==
== Postscript ==
Line 261: Line 320:
: -> from the command line, not the map instruction
: -> from the command line, not the map instruction
* rename sizecol to sizecolumn (remove the given warning)
* rename sizecol to sizecolumn (remove the given warning)
* use PAL/JPAL [http://geosysin.iict.ch/PAL cartographic labelling library] (GPL, C++ language, JNI wrapper)


See also "New display architecture" comments above.
See also "New display architecture" comments above.
Line 283: Line 343:


* Provide a mechanism (g.access option?) to enable r/w access for users in mapsets they don't own. So that it they don't need to hack lib/gis/mapset_msc.c. Glynn explains on [http://www.nabble.com/forum/ViewPost.jtp?post=12914361&framed=y GRASS user ML]: "AFAICT, that restriction has been unnecessary ever since the lockfile was moved from the user's home directory to the mapset directory."
* Provide a mechanism (g.access option?) to enable r/w access for users in mapsets they don't own. So that it they don't need to hack lib/gis/mapset_msc.c. Glynn explains on [http://www.nabble.com/forum/ViewPost.jtp?post=12914361&framed=y GRASS user ML]: "AFAICT, that restriction has been unnecessary ever since the lockfile was moved from the user's home directory to the mapset directory."
: Actually, I (Glynn) no longer think it's that simple. If other users can create directories within your mapset, they can create directories which you cannot remove, and in which you cannot add, remove or modify files. And this is quite likely to happen: most users will have a umask of 0022 or worse, meaning that other users (i.e. you) cannot modify any files or directories which they create.


== Data management ==
== Data management ==
Line 299: Line 360:


* Implement better [[Time series in GRASS]] support (series of satellite data etc)
* Implement better [[Time series in GRASS]] support (series of satellite data etc)
: for example?
: for example? [http://trac.osgeo.org/grass/browser/grass-addons/raster/r.rast4d r.rast4d]


== Visualization ==
== Visualization ==
Line 307: Line 368:


== CLI ==
== CLI ==
=== Parameters standardization ===


* Fix the parameters and flags. Make it a concept. See proposal in GRASS 5 [http://freegis.org/cgi-bin/viewcvs.cgi/grass/documents/parameter_proposal.txt?rev=HEAD&content-type=text/vnd.viewcvs-markup documents/parameter_proposal.txt]
* Fix the parameters and flags. Make it a concept. See proposal in GRASS 5 [http://freegis.org/cgi-bin/viewcvs.cgi/grass/documents/parameter_proposal.txt?rev=HEAD&content-type=text/vnd.viewcvs-markup documents/parameter_proposal.txt]
=== Flags standardization ===


* Get rid of 'quiet/verbose' flags, preparation in GRASS 6, e.g.:
* Get rid of 'quiet/verbose' flags, preparation in GRASS 6, e.g.:
Line 324: Line 389:
* Multiplatform
* Multiplatform
* Fast, minimalist, number of windows reduced
* Fast, minimalist, number of windows reduced
* Novice, ...., Expert profiles for the GUI (with reduced features), tailored for use cases, e.g. 3D models of a risk map,
* link from application to the relevant wiki online support, so that non-programmers can contribute to GRASS support
* compile wiki online content and help pages into a offline version of help pages for usage in GRASS without internet access. 
* Manageable also from command line via d.* modules
* Manageable also from command line via d.* modules
* Facilitating easy development of custom GUI application based on GRASS
* Facilitating easy development of custom GUI application based on GRASS
 
* <strike>drop '''d.m'''</strike> {{done}}
* <strike>drop '''gis.m'''</strike> {{done}}


* &rarr; [[WxPython-based GUI for GRASS]]
* &rarr; [[WxPython-based GUI for GRASS]]
Line 335: Line 404:
** the grass etc dir is a mess... module should maintain arch-deps and arch-indep things in different paths -- <cite> frankie at #grass irc</cite>
** the grass etc dir is a mess... module should maintain arch-deps and arch-indep things in different paths -- <cite> frankie at #grass irc</cite>
** it's basically a FHS violation, i dunno if it is reported by lintian, anyway /usr/lib/grass should be used for arch-deps data, not for mixed stuff -- <cite> frankie at #grass irc</cite>
** it's basically a FHS violation, i dunno if it is reported by lintian, anyway /usr/lib/grass should be used for arch-deps data, not for mixed stuff -- <cite> frankie at #grass irc</cite>
* Creating $HOME/.grass7 directory for
* Creating $HOME/.grass7 directory for {{done}}
** Custom fonts
** Custom fonts
** r.li and other modules temp. files
** r.li and other modules temp. files
** GEM addons installation
** GEM addons installation {{done}}
** Default path for custom scripts
** Default path for custom scripts
** Custom symbols and EPS fill patterns
** Custom symbols and EPS fill patterns
** Custom color maps
** Custom color maps
** Add here new item
** Add here new item...
 
* [[GRASS repository layout proposal]]


== User Wishes ==
== User Wishes ==
Line 357: Line 428:
* Or use [http://www.llnl.gov/visit/ VisIt software], it should be able to read GRASS maps directly via GDAL
* Or use [http://www.llnl.gov/visit/ VisIt software], it should be able to read GRASS maps directly via GDAL


== Complete GRASS Test Suite ==
== Complete GRASS Test Suite: see activity on [[Test_Suite | Test Suite development page]] ==
* base a comprehensive test suite on [http://www-pool.math.tu-berlin.de/~soeren/grass/GRASS_TestSuite/?C=M;O=D Soeren's GRASS Test Suite]
* base a comprehensive test suite on [http://www-pool.math.tu-berlin.de/~soeren/grass/GRASS_TestSuite/?C=M;O=D Soeren's GRASS Test Suite]
* automated error checking on all modules to catch data corruption issues
* automated error checking on all modules to catch data corruption issues

Latest revision as of 21:04, 23 May 2019

 New home: https://trac.osgeo.org/grass/wiki/Grass7Planning
 See also: http://trac.osgeo.org/grass/wiki/Grass7/NewFeatures

Maintenance of repository

For GRASS 7 development is used svn-trunk https://github.com/OSGeo/grass/ , for GRASS 6 development is used separated SVN branch develbranch_6.

  Planning is continued here: http://trac.osgeo.org/grass/wiki/Grass7Planning

Terminology

See GRASS 7 Terminology.

List of new features in GRASS 7 (already implemented)

See http://trac.osgeo.org/grass/wiki/Grass7/NewFeatures

API to access algorithms in modules

We need to better expose the "knowledge" which is contained at module level. We want to have it accessible via API, exposed in various programming languages such as C, Python, Perl, Java, ..

Update 1/2011: Python ctypes interface available, also PyGRASS, see http://grass.osgeo.org/grass71/manuals/libpython/

See how the temporal modules and temporal library is done.

Dead code cleanup

? unclear: Trash or move to addons?

  • general/g.setproj - partially replaced by `g.proj -c` but contains still useful logic
  • vector/v.label.sa see trac #1942 and r34923. It probably shouldn't be dealing with FreeType directly. Glynn suggests using the display API, specifically D_get_text_box() rather than trying to calculate a skyline.

Keep:

  • imagery/i.ortho.photo/i.ortho.init - i.ortho.photo should remain: trac #2461
  • imagery/i.ortho.photo/i.ortho.transform - i.ortho.photo should remain: trac #2461
  • raster/r.watershed/shed - keep for a future reference (e.g., for development of a wizard in a GSoC)

IMPORTANT:

Raster

The raster data API has been updated. See also

Raster Library

Raster map "history" metadata:

  • In 7.0, the fields of the history structure are dynamically allocated. You have to use Rast_set_history() or Rast_format_history() to set fields. The the HIST_* constants have to be used.

About cell_misc/null file size:

The size of the cell_misc/<mapname>/null file depends only upon the number of rows and columns, not the data. It's one bit (null/non-null) for each cell, rounded to an integral number of bytes per row (i.e. the number of columns is rounded up to the nearest multiple of 8).

Segment Library

(from trac #2296): we should look into unifying all of the different "segment" libraries.

They all do essentially the same thing: provide a 2-dimensional array which may be too large to fit into RAM (or, more accurately, into the process' address space; if RAM was the issue, mmap() etc would suffice), and which can be accessed (more or less) randomly.

Apart from the "official" segment library (lib/segment), r.proj has its own, r.stream.* each have their own, r.grow.distance has something simpler (the temporary file is read row-by-row but in reverse).

Modules

rename

  • rename r.in.gdal to r.import - see discussion - new r.import
  • rename r.out.gdal to r.export - see discussion
  • rename r.volume 'data' parameter to something more standard like 'input' - done.

* r.stats vs r.statistics vs r.statistics2 vs r.statistics3:

    • r.statistics2 has been renamed to r.stats.zonal
    • r.statistics3 has been renamed to r.stats.quantile

remove

see delta comment about r.out.tiff below, sometimes the simple stuff works best! --HB
Ditto.
  • remove r.resample and r.bilinear in favor of r.resamp.interp
TODO: double-check that r.resample is in fact fully replaced by 'r.resamp.interp's method=nearest'. ie check that an alternate useful method is not lost.
  • remove r.univar.sh; newly implemented r.univar features cover it - done.
  • remove r.out.tiff. New C r.out.gdal should cover all it's option now (doublecheck!). See RT #3680 (starting with date Sun, Nov 26 2006 14:54:23).
It might be worth keeping r.out.tiff! It makes a nice delta when things don't go well (eg QGIS bug#348) --HB
However: code duplication, maintenance overhead, user confusion (more entries in GUI, more manual pages, why are there modules doing the same?).
They do not do the same. r.out.gdal outputs data values, r.out.tiff outputs color values. Two completely different things. --HB
  • Remove remaining -v and -q flags for verbosity levels of modules.

Merge

while r.surf.idw will only output CELL maps, it is Very Fast. (or that is to say, r.surf.idw2 is Very Slow)
HB: eh? the options and algorithm are nothing alike.
MS: I meant that r.resamp.rst could be a subset of r.resamp.interp (yet another resampling algorithm next to nearest, bilinear, bicubic). I haven't considered that the number of rst options would make r.resamp.interp user interface much less clear. Maybe not such a good idea after all - user interface wise.
HB: just be careful that the GDAL version is as featureful and grid/cell center correct as the r.in.* versions. I suspect it might not be. r.in.wms needs further python rewrite with full XML and HTTP library support.
HB: why? they do two fundamentally different things, and both work quite nicely right now. One works in cell space, the other geographic space (especially for lat/lon).
MS: Right. The 2 modules do different things. But it would be usefull if r.grow supported distance in units and r.buffer in cells. Could both share same code for distance options?

fix

  • fix the raster map history management (truncating long history, odd storage). It should work like for vector maps in GRASS 6.
  • r.volume centroids parameter only makes a level one vector with no attribute table; module should be updated to GRASS 6 vector library)
  • r.random should be split into 2 modules: one for generating a raster map with random points (alike v.random), and the other for sampling a raster map (alike v.what.rast). Vector functionality should be droped from r.random - a dupe of existing vector modules. -i and -z flags should be droped.
  • v.random -z: read zmin and zmax from region settings, drop zmin and zmax. I.e. treat Z coord same as X,Y.

Good coding practice

  • Input handling:
/* Define the different options */
input1               = G_define_standard_option(G_OPT_R_INPUT) ;
input1->key          = _("albedo");
input1->description  =_("Name of the Albedo map [0.0-1.0]");
input1->answer       =_("albedo");
input1->guisection   = _("Required");

In here you can find G_define_standard_option(G_OPT_R_INPUT) assuming already those:

  input1->type       = TYPE_STRING;
  input1->required   = YES;
  input1->gisprompt  =_("old,cell,raster") ;

If your input is not required to run the module, you just create the following line:

 input1->required   = NO;
  • In a similar way, metadata/history storage:
      G_short_history(result1, "raster", &history);
      G_command_history(&history);
      G_write_history(result1,&history);
  • This is the standard incantation, but I have to find timestamp(), and

more details metadata maybe like sensor type for a start, or source/origin of data... Can we make metadata having elements (history->processing, history->timestamp, history->source(or history->origin), etc?) then it could be filled up specifically.

  • Or color palettes application is nice:
/* Color table for biomass */
      G_init_colors(&colors);
      G_add_color_rule(0,0,0,0,1,255,255,255,&colors);
  • Changes (from Glynn):

I would prefer it if G_open_*_new() simply called G_fatal_error() themselves if an error occurred. It's not as if there's any reasonable alternative way to handle the error.

  • Changes:
     input = G_define_option(G_OPT_F(D?)_INPUT) ;
     input->key        =_("parameter");
     input->type       = TYPE_DOUBLE;
     input->required   = YES;
     input->gisprompt  =_("value, parameter");
     input->description=_("Value of the parameter");
     /*input->answer     =_("0.000");*/
     input->guisection = _("Required");

I could also think similarly for non-GRASS files actually (configuration files sometimes need to be loaded separately)

Vector

Radim's TODO list

Vector TODO list

  • Particularly important: "Keep topology and spatial index in file instead of in memory" --ML
  • v.in.ogr: split long boundaries to speed up topology cleaning. Implemented in GRASS 7, partial backport to GRASS 6.5 possible.
v.in.ogr speed comparison

Library

  • 2d 'vertical' vector data (e.g. Geologic Cross Sections)
  • implement transactions for geometry handling (esp. v.edit, v.digit and to avoid leftover files when a vector command fails)
  • Assume cat 0 as the first possible, instead of 1. GRASS has supported cat 0 since around 2005, but it hasn't been widely used. According to Radim, using cat 0 allows for exact mapping from OGR FID (which can be 0) to GRASS cat.
  • support for elliptical arcs, quadratic and cubic splines. Elliptical arcs or circular arcs are very common in Swiss survey data (Amtliche Vermessung)

Modules

rename

remove

merge

  • merge v.select and v.overlay
needs discussion, they are doing fundamentally different things --HB
See a feature request #506 in GForge.
see also v.rast.stats and v.what.rast.buffer addon

fix

  • Fix the Column 'cat_' already exists (duplicate name) in v.in.ogr. Maybe by creating columns cat_1, cat_2 etc. each time a Grass vector is exported to shapefile and imported back to Grass?
  • write Vect_map_exists() and implement in g.remove and v.digit -n (why wait for GRASS 7 ??)
  • add '-d' dissolve to v.reclass
  • add 'where=' to v.to.rast (why wait for GRASS 7 ??)
  • implement Douglas-Peucker generalization (C code file)to substitute prune tool of v.clean (done?, see also GSoC)
  • Rewrite vector labeling. Needs more placement control options (may be db field value based), label overlapping prevention would be also good. May be we could borrow some ideas from MapServer? (ongoing: v.label.sa)
  • v.what.vect - rename parameters "vector" to "map", "qvector" to "qmap"
  • v.type - change type= option to from= and to=.(code's already in there)

enhance

  • extend v.overlay to allow combination of all types (point, line, area)
  • v.category: add possibility to create new layer categories on the basis of a column in the table linked to an existing layer
  • v.generalize: add point cloud data reduction, e.g. with http://www.cs.umd.edu/~mount/ANN/ (ANN: A Library for

Approximate Nearest Neighbor Searching)

3D topology

Currently, GRASS' topological cleaning is 2D only. This will regularly corrupt 3D data that is actually topologically correct in 3D space, but appears incorrect in 2D space (e.g. 3D polygons that appear to overlap or have zero-area in the X-Y geographic reference plane). Therefore, v.clean in GRASS 7 should become more competent at handling 3D topology. This is a sketch of how to implement the same level of topology support that 2D geometries currently have in 3D space.

Problem structure

The classes of potential topological errors depend on the geometry type (point, line, polygon), in 3D just as in 2D. Each more complex geometry type inherits the potential problems of the less complex types, and adds its own.

  • 3D topological problems for points reduce to coincidence in 3D space.
  • 3D topological problems for lines are over- and undershoots as well as duplicate vertices. These are completely equivalent to the same problem in 2D space.
  • 3D topological problems for polygons appear much more complex, as they include areal overlap in 3D, and shared boundaries between adjacent polygons. But this complexity can be reduced significantly. Only polygons that lie in the same plane (including a small error margin -- could be implemented just like the minimum error option in v.clean) can get into topological trouble with each other: if two polygons do not lie in the same plane, then by definition they cannot overlap, only intersect. (It is open to debate, whether an intersection of polygons in 3D space would constitute a topological error.) Likewise, adjacent polygons can only have a shared boundary if they are neighbors on the same plane.

Basic approach

As a result, the topological cleaning could proceed as follows.

  1. All 3D polygons need to be planar, i.e. all of their vertices must lie on the same plane. If they are not, then this is a topological error, and the polygon needs to be planarized by v.clean (add a "planarize" action -- also potentially useful for other geometry types). Planarization is done by first calculating the plane equation of the polygon (a simple equation that defines the location and orientation of a best-fit 2D plane through all vertices) and then reprojecting each of its original vertices to the plane defined by the plane equation (it could make sense to store the plane equations as part of the topological data structure). Planarization needs to be carried out for both boundaries and islands.
  2. Once planarized, all polygons can be subjected to the exact same topological cleaning as 2D polygons:
    1. After planarization, group all polygons that lie on the same plane.
    2. For each "plane group": reproject all vertices to the geographic X-Y reference plane. Run the topological cleaning just like for 2D polygons, but preserve the Z coordinates.
    3. Then revert the projection of all vertices back to the group's original reference plane, using the plane equation.
    4. Do the same with the next "plane group" until all polygons have been processed.
  3. Faces (3D triangles) are primitives that are part of more complex 3D geometries (including TINs and 3D hulls). Topological problems involving such geometries are too complex to handle in GIS. Thus, faces should not be checked for overlap. GRASS GIS should just assume that faces are part of complex geometries that are not to be subjected to analytical treatment, but simple tasks, such as visualization, only. Therefore, only basic cleaning (check for duplicate coordinates, zero-area triangles, etc.) should be carried out. Faces are always planar by definition.
  4. The areas and centroids of both faces and planar polygons can be calculated in 3D space just as they can be calculated in 2D space.

Implementation details

A simple algorithm for computing the plane equation of a set of vertices is <missing URL> available in the form of Newell's Method. This method requires three points to define a plane. This requirement is met by even the most simply polygons. However <missing URL>: "Newell's method is known to fail if the 3 points are chosen around a concave corner - the normal of the resulting plane will point in the direction opposite to the expected one." (This should be avoidable by taking three vertices at the extreme edges.)

Since the planar equation is a best-fit model, not always an exact solution, a warning should be issued if an error threshold is reached.

3D point clouds

General

Library

  • Add support for planetary bodies reference systems
  • Add new partial differential equation (PDE) library with OpenMP support (GRASS 6.3)

Modules

  • g.remove, g.mremove, g.rename, g.copy: don't allow for default datatype (which is currently raster) [1].
controversial, needs more discussion --HB
  • g.region
    • Glynn's notes - cleaning the print flags and new print= option
    • Support vector's 3rd dimension in g.region vect= [-a], [res=], like the 2d extents are (or should be) see trac #121

Database

Library

  • establish SQLite as default DBMI driver for vector attribute storage (DBF is too limited). done May 2008.

Modules

  • allow cross-mapset database access, i.e. parse the '@mapset' notation appended to vector names (requires access via possibly different DBMI drivers)

rename

  • rename db.in.ogr to db.import

Imagery

Page has been moved to Trac: http://trac.osgeo.org/grass/wiki/Grass7/ImageryLib

Raster3D

Library

  • renaming of all G3D library functions to fulfil the grass coding standard
  • extent/rewrite documentation
  • localisation support (why wait for GRASS 7 ??)

See http://trac.osgeo.org/grass/wiki/Grass7/G3dLib

Modules

  • report and support modules like r3.stats, r3.support
  • voxel -> vector (isosurfaces ...) and vector -> voxel (lines, faces, volumes) conversion modules
  • module for 3d Kriging interpolation based on vector points
  • a GRASS-Python/VTK visualisation/manipulation tool

Display

The page moved to Trac: http://trac.osgeo.org/grass/wiki/Grass7/DisplayLib

Postscript

Modules

ps.map

  • remove scale parameter
-> from the command line, not the map instruction

See also "New display architecture" comments above.

Parser

  • Making GRASS modules be less verbose. Use --verbose flag and GRASS_VERBOSE environment variable. All output (G_message, G_percetn, G_warning) should go to GRASS_LOG file which could be grassdata/location/mapset/.grass.log by default.
less verbose: this is well underway in 6.3
Note warning and errors are already logged to GIS_ERROR_LOG (see variables.html)

Init shell and startup

  • .grassrc6 is not what you expect. It holds the g.gisenv GIS variables, it's not a shell script containing commands like .bashrc is.
Suggestion: We should change the name for 7.x. It isn't an "rc" file in the conventional sense.
Suggestion: make it even a $HOME/.grass7/ directory to store settings etc (e.g., from r.li and others)
  • It is asked to run GRASS in its own shell to avoid portability issues 1.
  • Eliminate Init.sh. Glynn explains on GRASS user ML: "Most of the environment can be set up through an e.g. /etc/env.d/grass script. The database, location and mapset can be set through g.mapset. The only slight subtlety is if you want multiple independent sessions, but that can be done with a fraction of the code in Init.sh."
  • Provide a mechanism (g.access option?) to enable r/w access for users in mapsets they don't own. So that it they don't need to hack lib/gis/mapset_msc.c. Glynn explains on GRASS user ML: "AFAICT, that restriction has been unnecessary ever since the lockfile was moved from the user's home directory to the mapset directory."
Actually, I (Glynn) no longer think it's that simple. If other users can create directories within your mapset, they can create directories which you cannot remove, and in which you cannot add, remove or modify files. And this is quite likely to happen: most users will have a umask of 0022 or worse, meaning that other users (i.e. you) cannot modify any files or directories which they create.

Data management

  • store vertical units on per-map base, using code from units software
Support for free form unit meta-data added in 6.3. I don't mind it as a guide, but we shouldn't be limited to units found in units. --HB
  • store vertical map datum on per-location base (GDAL/OGR needs the same enhancement)
This requires more discussion. I'm not sure it's a good idea to do this location-wide. --HB
On a per raster map basis done in 6.3 cvs.
  • add versioning for maps (to recover previous map versions)
see "v.info -h" ?

Time series

for example? r.rast4d

Visualization

  • better support for faces and kernels in libgis
not really Visualization, but....

CLI

Parameters standardization

Flags standardization

  • Get rid of 'quiet/verbose' flags, preparation in GRASS 6, e.g.:
    /* please, remove before GRASS 7 released */
    if(q->answer) {
        putenv("GRASS_VERBOSE=0");
        G_warning(_("The '-q' flag is superseded and will be removed "
            "in future. Please use '--quiet' instead"));
    }

GUI

  • Multiplatform
  • Fast, minimalist, number of windows reduced
  • Novice, ...., Expert profiles for the GUI (with reduced features), tailored for use cases, e.g. 3D models of a risk map,
  • link from application to the relevant wiki online support, so that non-programmers can contribute to GRASS support
  • compile wiki online content and help pages into a offline version of help pages for usage in GRASS without internet access.
  • Manageable also from command line via d.* modules
  • Facilitating easy development of custom GUI application based on GRASS
  • drop d.m
  • drop gis.m

Conceptual changes

  • File organization in binaries:
    • the grass etc dir is a mess... module should maintain arch-deps and arch-indep things in different paths -- frankie at #grass irc
    • it's basically a FHS violation, i dunno if it is reported by lintian, anyway /usr/lib/grass should be used for arch-deps data, not for mixed stuff -- frankie at #grass irc
  • Creating $HOME/.grass7 directory for
    • Custom fonts
    • r.li and other modules temp. files
    • GEM addons installation
    • Default path for custom scripts
    • Custom symbols and EPS fill patterns
    • Custom color maps
    • Add here new item...

User Wishes

This section is not really development related...

  • Create 3D animation w nviz showing GRASS 3D coolness. MarisN 12:00, 4 August 2006 (CEST)
  • here are some examples to get inspired (apparently that's already possible):
  • Convince the users to use ParaView [2] for sophisticated animations --huhabla 20:47, 14 August 2006 (CEST)
    • (Add support for Paraview in GDAL/OGR or add GDAL/OGR support in ParaView to read directly data from GRASS) see discussion
  • Or use VisIt software, it should be able to read GRASS maps directly via GDAL

Complete GRASS Test Suite: see activity on Test Suite development page

  • base a comprehensive test suite on Soeren's GRASS Test Suite
  • automated error checking on all modules to catch data corruption issues