V.krige GSoC 2009: Difference between revisions

From GRASS-Wiki
Jump to navigation Jump to search
 
(24 intermediate revisions by 6 users not shown)
Line 1: Line 1:
=== v.krige: Python porting and wxPython GUI addition  ===
=== v.krige: Python porting and [[wxGUI]] addition  ===
Anne Ghisla's [http://socghop.appspot.com/student_project/show/google/gsoc2009/osgeo/t124023165994 Google Summer of Code 2009 project], mentored by Martin Landa and Michael Barton
Anne Ghisla's [http://socghop.appspot.com/student_project/show/google/gsoc2009/osgeo/t124023165994 Google Summer of Code 2009 project], mentored by [[User:Landa|Martin Landa]] and Michael Barton


== Aim of the project ==
== Aim of the project ==
Line 9: Line 9:
== State of the art ==
== State of the art ==


''' Breaking news: Ordinary kriging is up and running both from GUI and CLI!''' not all the options are available for gstat, anyway is it a great goal to see that the whole process ends in writing a GRASS raster map with the result. Note also that the module suggests a filename based on the input data map name. One step further towards the One-Click Interface.
'''V.krige is back to functionality in GRASS 7 trunk, from [http://trac.osgeo.org/grass/changeset/57146 r57146] - made during [[GRASS_Community_Sprint_Prague_2013]]'''
Soon more improvements and fixes will be available, and also be backported to GRASS 6.5. Until then, feel free to comment on [http://trac.osgeo.org/grass/query?status=assigned&status=new&status=reopened&description=~krig&order=priority the tickets on Trac] and on grass-dev mailing list.


[[File:Schermata-Kriging Module.png|450px|left]]


Have a look at it from svn:
[[File:Schermata-Kriging Module.png|600px]]


svn co https://svn.osgeo.org/grass/grass-addons/vector/v.krige v.krige


Dependency check is performed on grass-python module, R, rpy2 and R packages gstat and spgrass6. If something is missing, the module won't start. Some graceful popup window will be set up in the near future. At the moment it aborts with a message in the shell.
Dependency check is performed on grass-python module, R, rpy2 and R packages gstat and rgrass7. If something is missing, the module won't start. Some graceful popup window will be set up in the near future. At the moment it aborts with a message in the shell.


Kriging relies on R package gstat, the most widespread. geoR tab is now hidden as it doesn't provide any functionality, but if anyone is interested in using it instead of gstat I'll put this todo higher in the list.
Kriging relies on R package gstat, the most widespread. geoR tab is now hidden as it doesn't provide any functionality, but if anyone is interested in using it instead of gstat I'll put this todo higher in the list.
Line 25: Line 24:
== Installation ==  
== Installation ==  


Please refer to [[Compile_and_Install#Scripts]].
For GRASS 6.5 <r40048, it is easy to install it via [http://grass.osgeo.org/grass65/manuals/html65_user/g.extension.html g.extension] (for GRASS 6.4 it is an AddOn, from 6.5+/7 onwards included). Simply type in a GRASS shell:


=== Debian GNU/Linux ===
g.extension v.krige


apt-get install python-rpy
To install v.krige.py from source, please refer to [[Compile_and_Install#Scripts]]. Source is available on SVN:


TODO
svn co https://svn.osgeo.org/grass/grass-addons/vector/v.krige


== Bulletin Board ==
'''The module requires GRASS >= 6.4 RC5''' - it uses the new version of grass-python library. At the moment, no backward compatibility with previous versions is provided.


* '''TODOs'''
The module has dependencies:
** '''Document dependencies''' - continue fill up html template page
* R 2.x - no exhaustive tests on compatible versions. An up-to-date R package will do the job in 99% cases.
** Cleanup code! Refactoring brings refactoring.
* R packages spgrass6 (for GRASS 6.*) or rgrass7 (for GRASS 7.*), gstat and maptools
** '''Splash screen at module load'''.
* Python module rpy2
*** Check '''dependencies''' all before interface load. If something is missing, a popup window will give a clear message.
** Continue integrating '''gstat functions'''


* '''IDEAS TO DISCUSS'''
Be sure that ALL these dependencies are installed and operative. See below for platform-specific details.
** Create a '''package''' to delegate dependencies check!
** '''Variogram fit''': in a separated window, with allowed interactive model fit, using Python plot libraries.


* '''BLOCKING ISSUE(S)'''
=== Notes for Debian GNU/Linux ===
** NULL


== Documentation ==
Install the dependiencies:


Kriging theory from Isaaks and Srivastava's "An Introduction to Applied Geostatistics" [0]. Great book.
aptitude install R python-rpy2


== Users feedback and other kriging software ==
'''Attention! python-rpy IS NOT SUITABLE.'''


* Call for users has been done on gfoss.it, grass-it and grass-users, r-sig-geo and r-sig-ecology mailing lists. Thanks to Giovanni Manghi and João Tiago from GFOSS.pt and Giovanni Allegri for user advices, and to Edzer Pebesma, Paul Hiemstra, Ebrahim Jahanshiri, Dylan Beaudette for support on R side.
To install R packages, use either R's function (as root):


* There are (have been) some GRASS modules that perform kriging, like:
install.packages("gstat", dep=T)
** v.surf.krige, v.variogram: no longer developed by authors, difficult to port.
install.packages("spgrass6", dep=T)
** s.surf.krig: deprecated since GRASS 6 series.
install.packages("maptools", dep=T)
** scripts including R and spgrass6: are the concrete basis upon which build the module.


* Obtained ArcGIS availability for comparison of interface and functionality, because it is AFAIK the widespread tool used for kriging. Also Isats [1] is a valuable source of ispiration for interface design.  
If you are running GRASS 7.*, install rgrass7 instead of spgrass6:
install.packages("rgrass7", dep=T)


== Planned Timeline ==
Alternatively, you can install the packages using the brand new Debian packages [6]. To do so, add to repositories' list for 32bit:
deb http://debian.cran.r-project.org/cran2deb/debian-i386 testing/


Draft:
or, for 64bit:
* create a wxPython interface for ordinary kriging. Will follow Humane Interface rules [2].
* integrate R package ''automap''
* '''midterm deadline''': have a working module that performs ordinary kriging.
* integrate more R functions from ''gstat'' and ''geoR'', giving the user the choice between gstat and geoR (kriging results vary in respect of implementation)


== Weekly Reports, from SoC mailing list [3] ==
deb http://debian.cran.r-project.org/cran2deb/debian-amd64 testing/


=== Report #1, 29 May ===
and get the packages via


* Done
aptitude install r-cran-gstat r-cran-spgrass6 maptools
I dedicated this week to documentation and discussion with users and developers.
if you are using GRASS 6.*, or
** Documentation: I'm reading "An introduction to Applied Geostatistics" [0], as I need to understand the theory behind kriging functions provided by R. On wxPython side, wxPython wiki is the main source of information, together with the code of GRASS wxPython interface.
aptitude install r-cran-gstat r-cran-rgrass7 maptools
** Community discussion: Feedback on interface design and R has been collected on various mailing lists. Also, a group of Portuguese ArcGIS users is interested in giving advice and test the new module.
if you are using GRASS 7.*


* Planned for next week
=== Notes for Windows ===
I plan to define which functions (and consequently which R packages) are to be included into the module. I plan to first include package automap (a wrapper for gstat), then gstat advanced functions, then geoR (an "ecological vicariant" of gstat), all alvailable on CRAN [4]. This will allow R users to keep using their preferred functions, as kriging results are implementation-dependent.
At this very moment, no backward compatibility with OSGeo4W's packaged GRASS 6.4svn2 is provided, nor with WinGRASS. Testing is planned, as latest  WinGRASS includes the most recent python library.
Then I'll work on wxPython interface and get a draft as soon as possible. The interfaces that I use as model are ArcGIS' kriging module and Isatis. I won't replicate their structure, rather see what are the provided features and create v.autokrige interface following Humane Interface guidelines.


* Bottleneck(s)
If you really need to run v.krige right now on Windows, I suggest to compile GRASS following [http://trac.osgeo.org/grass/wiki/CompileOnWindows this guide].
I'm experiencing some difficulties, mainly with wxPython, as I never used it before, and also with kriging, for the same reason :) but I'm not worried nor blocked. Last year GSoC project was a bet and I succeeded, this year it's even harder but I can rely on some more experience and, as always, on mentor and community support.
You could also use Linux in a virtual machine. Or install Linux in a separate partition of the HD. This is not as painful as it appears, there are lots of guides over the Internet to help you.


=== Report #2, 5 June ===
== Documentation ==


* Done
Notes on v.krige usage, examples and tips are available in the module's help page, available via Help button if the GUI.
** Ended "An Introduction to Applied Geostatistics", great source of information of what the module is supposed to do. Now I can be both developer and user, not discarding any of the hints of other users.
** Some progress on the interface: I'm getting used of wxWidgets, even if the lack of a graphical designer slows down my work. wxDesigner seems good, but I think it is also good for me to learn the meaning of each line of code. The interface is quite ready for automap and GRASS integration.


* Planned for next week
Kriging theory is available in Isaaks and Srivastava's "An Introduction to Applied Geostatistics" [1]. Another great book.
End of interface essential features and integration of automap most automated functions - ideally, the user will pick up the point layer containing data and press OK.


* Bottleneck(s)
Developers documentation is available at Trac's wiki http://trac.osgeo.org/grass/wiki/wxGUIDevelopment/vkrige .
The only bottleneck is wxWidgets handling, but not so much as one week ago.


=== Report #2.5, 11 June ===
== Users feedback and other kriging software ==


First of all, the interface is almost finished. See a screenshot on the wiki page [0].
* Call for users has been done on gfoss.it, grass-it and grass-users, r-sig-geo and r-sig-ecology mailing lists. Thanks to Giovanni Manghi and João Tiago from GFOSS.pt and Giovanni Allegri for user advices, and to Edzer Pebesma, Paul Hiemstra, Ebrahim Jahanshiri, Dylan Beaudette for support on R side.
The layout includes a notebook with a page for each R package (automap, gstat and geoR), with the available options. Another layout could be a chiocebox wht the three packages on the top of Kriging section, that redraws the section accourding to the user's choice.
Let me know which one do you prefer, feel free also to suggest something new.


The R-GRASS integration is on its way: autofitting the variogram on a map obtained from a DEM sample [1] works in the proof of concept. rpy2 is extremely helpful and does not require much more code than the original R code. More to come in the next days.
* There are (have been) some GRASS modules that perform kriging, like:
** v.surf.krige, v.variogram: no longer developed by authors, difficult to port.
** s.surf.krig: deprecated since GRASS 6 series.
** scripts including R and spgrass6: are the concrete basis upon which build the module.


The idea for variogram fitting is to plot the variogram in a new frame and add some controls like sliders and/or text boxes to fill up with
* Obtained ArcGIS availability for comparison of interface and functionality, because it is AFAIK the widespread tool used for kriging. Also Isats [2] is a valuable source of ispiration for interface design.  
nugget, sill and range values. This will involve Python graphics, not R, as the former is more flexible.


=== Report #3, 12 June ===
== Planned Timeline ==


this weekly report will be very short, as yesterday I sent this email
Draft and definitive:
[0] and made little progress.
* create a wxPython interface for ordinary kriging. Will follow Humane Interface rules [3].
 
* integrate R package ''automap''
I'm having some problems in using autoKrige() function work, AFAIK
* '''midterm deadline''': have a working module that performs ordinary kriging.
because of projection information handling.
* integrate more R functions from ''gstat'' and ''geoR'', giving the user the choice between gstat and geoR (kriging results vary in respect of implementation)
 
=== Report #4, 19 June ===
 
this week I made steady progress on the module.
What I've done:
* renamed the module v.krige, as its features will go beyond automatic kriging.
* addition of choicebox with only numerical columns, as interpolation will be based on such variables
* refactoring on interface population - more to come
* started documentation page
 
Next week I plan to create the splashscreen during dependencies check
and data load, and examine how to integrate gstat functions.
 
=== Report #5, 26 June (at [http://wiki.osgeo.org/wiki/OSGeo_Hacking_Event_2009 OSGeo Bolsena Hacking Event]) ===
 
just few minutes ago I succeded in completing the kriging procedure with gstat functions. It runs with a proof-of-concept dataset based on
spearfish data, see this tutorial [0]. Martin helped me a lot in getting the standard wxGUI comboboxes to run properly with filters.
 
Next week I plan to adapt the interface to each R package's available options and consider how to solve lag issues in populating interface.
 
The blocking issue about autoKrige() and projections is no more valid as I create the grid myself based on GRASS region.
 
=== Report #6, 3 July ===
 
this week I worked on removing parameters hardwired in the code, binding them to the interface instead.
Therefore, now it is possible use all widgets in gstat and automap pages, i.e. pick up the model from a list, set sill, nugget and range;
set the output raster map name and eventually overwrite it.
I'm going to implement CLI in these next days, it will very likely need to reorganise the code and clearly split interface from model.
 
Blocking issues - none.
 
=== Report #7, midterm, 10 July ===
 
this week has been full of improvements:
* CLI with optional arguments is up and running
* interface creation is no more delayed by vector map filtering
* dependencies are checked all at the beginning - no risk of mid-process crashes if something is missing
* automap page merged with gstat: they share the same algorithms
* geoR page hidden, no implementation yet
* deep refactoring of the rpy code, moved into a separate controller class
* interface fills up all options except input data map - minimum interaction required (2 clicks)


next week I plan to implement the splashscreen and add further functionalities, hopefully different kriging techniques.


Blocking issues - none.


== References ==
== References ==
* [0] Isaaks and Srivastava, 1989: "An Introduction to Applied Geostatistics", (ISBN 0-19-505013-4)
* [0] Fowler, Beck, Brant, Opdyke, Roberts, 1999. "Refactoring. Improving the design of existing code" (ISBN-10: 0201485672; ISBN-13: 978-0201485677)
* [1] http://www.geovariances.com/software/video-data-investigation-with-isatis-exploratory-data-analysis-ar0353.html
* [1] Isaaks and Srivastava, 1989: "An Introduction to Applied Geostatistics"  (ISBN 0-19-505013-4)
* [2] Jef Raskin, 2000: "The Humane Interface: New Directions for Designing Interactive Systems" (ISBN 0-201-37937-6)  
* [2] http://www.geovariances.com/software/video-data-investigation-with-isatis-exploratory-data-analysis-ar0353.html
* [3] http://lists.osgeo.org/pipermail/soc/
* [3] Jef Raskin, 2000: "The Humane Interface: New Directions for Designing Interactive Systems" (ISBN 0-201-37937-6)  
* [4] http://cran.r-project.org/web/views/Spatial.html
* [4] OSGeo SoC mailing list http://lists.osgeo.org/pipermail/soc/
* [5] CRAN: Spatial view http://cran.r-project.org/web/views/Spatial.html
* [6] cran2deb repository https://stat.ethz.ch/pipermail/r-sig-debian/2009-July/000805.html


[[Category:GSoC]]
{{GSoC}}
[[Category:R]]

Latest revision as of 21:21, 30 September 2015

v.krige: Python porting and wxGUI addition

Anne Ghisla's Google Summer of Code 2009 project, mentored by Martin Landa and Michael Barton

Aim of the project

As GRASS presently lacks kriging capability, it is performed via an add-on, v.autokrige, that delegates analysis to R (package automap). This module is written in ksh and has the classical autogenerated GUI. The project aims to rewrite the module in Python, creating a new GUI in wxPython that allows the user to refine parameters. I therefore plan to examinate present v.autokrige code and port it into Python, possibly improving it at the same time, and add a wxPython GUI. All time left will be dedicated to addition of further functionality, in respect of the most needed.

State of the art

V.krige is back to functionality in GRASS 7 trunk, from r57146 - made during GRASS_Community_Sprint_Prague_2013 Soon more improvements and fixes will be available, and also be backported to GRASS 6.5. Until then, feel free to comment on the tickets on Trac and on grass-dev mailing list.



Dependency check is performed on grass-python module, R, rpy2 and R packages gstat and rgrass7. If something is missing, the module won't start. Some graceful popup window will be set up in the near future. At the moment it aborts with a message in the shell.

Kriging relies on R package gstat, the most widespread. geoR tab is now hidden as it doesn't provide any functionality, but if anyone is interested in using it instead of gstat I'll put this todo higher in the list.

The list of point layers is provided by VectorSelect class (gselect module) to provide uniformity of layout with wxGUI. The filtering is done once at the start of the module, then if the user adds a suitable point layer she/he will press the Refresh button to force addition to the list. Automatic refresh on popup has been tested as too time and resource consuming.

Installation

For GRASS 6.5 <r40048, it is easy to install it via g.extension (for GRASS 6.4 it is an AddOn, from 6.5+/7 onwards included). Simply type in a GRASS shell: 

g.extension v.krige

To install v.krige.py from source, please refer to Compile_and_Install#Scripts. Source is available on SVN: 

svn co https://svn.osgeo.org/grass/grass-addons/vector/v.krige

The module requires GRASS >= 6.4 RC5 - it uses the new version of grass-python library. At the moment, no backward compatibility with previous versions is provided.

The module has dependencies:

  • R 2.x - no exhaustive tests on compatible versions. An up-to-date R package will do the job in 99% cases.
  • R packages spgrass6 (for GRASS 6.*) or rgrass7 (for GRASS 7.*), gstat and maptools
  • Python module rpy2

Be sure that ALL these dependencies are installed and operative. See below for platform-specific details.

Notes for Debian GNU/Linux

Install the dependiencies: 

aptitude install R python-rpy2

Attention! python-rpy IS NOT SUITABLE.

To install R packages, use either R's function (as root): 

install.packages("gstat", dep=T)
install.packages("spgrass6", dep=T)
install.packages("maptools", dep=T)

If you are running GRASS 7.*, install rgrass7 instead of spgrass6: 

install.packages("rgrass7", dep=T)

Alternatively, you can install the packages using the brand new Debian packages [6]. To do so, add to repositories' list for 32bit: 

deb http://debian.cran.r-project.org/cran2deb/debian-i386 testing/

or, for 64bit: 

deb http://debian.cran.r-project.org/cran2deb/debian-amd64 testing/

and get the packages via 

aptitude install r-cran-gstat r-cran-spgrass6 maptools

if you are using GRASS 6.*, or 

aptitude install r-cran-gstat r-cran-rgrass7 maptools

if you are using GRASS 7.*

Notes for Windows

At this very moment, no backward compatibility with OSGeo4W's packaged GRASS 6.4svn2 is provided, nor with WinGRASS. Testing is planned, as latest WinGRASS includes the most recent python library.

If you really need to run v.krige right now on Windows, I suggest to compile GRASS following this guide. You could also use Linux in a virtual machine. Or install Linux in a separate partition of the HD. This is not as painful as it appears, there are lots of guides over the Internet to help you.

Documentation

Notes on v.krige usage, examples and tips are available in the module's help page, available via Help button if the GUI.

Kriging theory is available in Isaaks and Srivastava's "An Introduction to Applied Geostatistics" [1]. Another great book.

Developers documentation is available at Trac's wiki http://trac.osgeo.org/grass/wiki/wxGUIDevelopment/vkrige .

Users feedback and other kriging software

  • Call for users has been done on gfoss.it, grass-it and grass-users, r-sig-geo and r-sig-ecology mailing lists. Thanks to Giovanni Manghi and João Tiago from GFOSS.pt and Giovanni Allegri for user advices, and to Edzer Pebesma, Paul Hiemstra, Ebrahim Jahanshiri, Dylan Beaudette for support on R side.
  • There are (have been) some GRASS modules that perform kriging, like:
    • v.surf.krige, v.variogram: no longer developed by authors, difficult to port.
    • s.surf.krig: deprecated since GRASS 6 series.
    • scripts including R and spgrass6: are the concrete basis upon which build the module.
  • Obtained ArcGIS availability for comparison of interface and functionality, because it is AFAIK the widespread tool used for kriging. Also Isats [2] is a valuable source of ispiration for interface design.

Planned Timeline

Draft and definitive:

  • create a wxPython interface for ordinary kriging. Will follow Humane Interface rules [3].
  • integrate R package automap
  • midterm deadline: have a working module that performs ordinary kriging.
  • integrate more R functions from gstat and geoR, giving the user the choice between gstat and geoR (kriging results vary in respect of implementation)


References

Retrieved from "https://grasswiki.osgeo.org/w/index.php?title=V.krige_GSoC_2009&oldid=22040"