V.krige GSoC 2009

v.krige: Python porting and wxPython GUI addition

Anne Ghisla's Google Summer of Code 2009 project, mentored by Martin Landa and Michael Barton

Aim of the project

As GRASS presently lacks kriging capability, it is performed via an add-on, v.autokrige, that delegates analysis to R (package automap). This module is written in ksh and has the classical autogenerated GUI. The project aims to rewrite the module in Python, creating a new GUI in wxPython that allows the user to refine parameters. I therefore plan to examinate present v.autokrige code and port it into Python, possibly improving it at the same time, and add a wxPython GUI. All time left will be dedicated to addition of further functionality, in respect of the most needed.

State of the art

Breaking news: Ordinary kriging with gstat or automap is up and running! not all the options are available for gstat, anyway is it a great goal to see that the whole process ends in writing a GRASS raster map with the result. Note also that the module suggests a filename based on the input data map name. One step further towards the One-Click Interface.

Writing the wxPython interface, with GRASS bindings and part of R bindings - have a look at it from svn:

svn co https://svn.osgeo.org/grass/grass-addons/vector/v.krige v.krige

It is quite complete, allows the user to choose among automap (default choice), gstat (advanced functions) and geoR.

Poll: Automap and gstat pages are redundant. The underlying algorithms are for both from gstat. I therefore plan to remove automap page and set the "automatic variogram fit" checkbox in gstat's page.

These three packages are not all required; if the user already uses R for kriging, the interface will load the package(s) present in R library. If none of the packages is available a popup message will ask for installation of at least one of the packages and abort. That is, crash gracefully and with clear explanation. I know what it means to crawl among missing dependencies and I don't want my users to be puzzled by inexplicable error messages.

Question: another layout can be a Choicebox with the three packages and in respect of the choice the drawing of corresponding widgets. Opinions welcome. [Until now, all preferences went to the Notebook.]

The list of point layers is provided by VectorSelect class (gselect module) to provide uniformity of layout with wxGUI. The filtering is done once at the start of the module, then if the user adds a suitable point layer she/he will press a Refresh button to force addition to the list. Automatic refresh on popup has been tested as too time and resource consuming.

Installation

Please refer to Compile_and_Install#Scripts.

Bulletin Board

• TODOs
  • Document dependencies - continue fill up html template page
  • Set CLI - in progress!
  • Refactor all code. Interface must be separated from model (i.e. wx from R) in order to implement CLI
  • Remove automap page and move "automatic variogram fit" checkbox into gstat page, if no objection comes up
  • Continue integrating gstat functions

• IDEAS TO DISCUSS
  • Filter the maps after interface creation - wx.CallAfter/threads... have to grep documentation and examples
  • Initial sanity checks:
    • Check dependencies all before interface load. If something is missing, a popup window will give a clear message.
    • Create a package to delegate dependencies check!
  • Variogram fit: in a separated window, with allowed interactive model fit, using Python plot libraries.

• BLOCKING ISSUE(S)
  • The big refactorisation about interface-model separation. Will quite completely rewrite the module.

Documentation

Kriging theory from Isaaks and Srivastava's "An Introduction to Applied Geostatistics" [0]. Great book.

Users feedback and other kriging software

• Call for users has been done on gfoss.it, grass-it and grass-users, r-sig-geo and r-sig-ecology mailing lists. Thanks to Giovanni Manghi and João Tiago from GFOSS.pt and Giovanni Allegri for user advices, and to Edzer Pebesma, Paul Hiemstra, Ebrahim Jahanshiri, Dylan Beaudette for support on R side.

• There are (have been) some GRASS modules that perform kriging, like:
  • v.surf.krige, v.variogram: no longer developed by authors, difficult to port.
  • s.surf.krig: deprecated since GRASS 6 series.
  • scripts including R and spgrass6: are the concrete basis upon which build the module.

• Obtained ArcGIS availability for comparison of interface and functionality, because it is AFAIK the widespread tool used for kriging. Also Isats [1] is a valuable source of ispiration for interface design.

Planned Timeline

Draft:

• create a wxPython interface for ordinary kriging. Will follow Humane Interface rules [2].
• integrate R package automap
• midterm deadline: have a working module that performs ordinary kriging.
• integrate more R functions from gstat and geoR, giving the user the choice between gstat and geoR (kriging results vary in respect of implementation)

Weekly Reports, from SoC mailing list [3]

Report #1, 29 May

- Done

I dedicated this week to documentation and discussion with users and developers. - - Documentation: I'm reading "An introduction to Applied Geostatistics" [2], as I need to understand the theory behind kriging functions provided by R. On wxPython side, wxPython wiki is the main source of information, together with the code of GRASS wxPython interface. - - Community discussion: Feedback on interface design and R has been collected on various mailing lists, mainly in this thread [3]. Also, a group of Portuguese ArcGIS users is interested in giving advice and test the new module.

- Planned for next week

I plan to define which functions (and consequently which R packages) are to be included into the module. I plan to first include package automap (a wrapper for gstat), then gstat advanced functions, then geoR (an "ecological vicariant" of gstat), all alvailable on CRAN [4] . This will allow R users to keep using their preferred functions, as kriging results are implementation-dependent. Then I'll work on wxPython interface and get a draft as soon as possible. The interfaces that I use as model are ArcGIS' kriging module and Isatis. I won't replicate their structure, rather see what are the provided features and create v.autokrige interface following Humane Interface guidelines.

- Bottleneck(s)

I'm experiencing some difficulties, mainly with wxPython, as I never used it before, and also with kriging, for the same reason :) but I'm not worried nor blocked. Last year GSoC project was a bet and I succeeded, this year it's even harder but I can rely on some more experience and, as always, on mentor and community support.

[0] http://precisiongis.blogspot.com/2008/11/vautokrige.html [1] http://grass.osgeo.org/wiki/V.autokrige_GSoC_2009 [2] "An introduction to Applied Geostatistics", E. H. Isaaks and R. M Srivastava, 1989. [3] https://stat.ethz.ch/pipermail/r-sig-geo/2009-May/005786.html [4] http://cran.r-project.org/web/views/Spatial.html

Report #2, 5 June

- Done

- - Ended "An Introduction to Applied Geostatistics", great source of information of what the module is supposed to do. Now I can be both developer and user, not discarding any of the hints of other users. - - Some progress on the interface: I'm getting used of wxWidgets, even if the lack of a graphical designer slows down my work. wxDesigner seems good, but I think it is also good for me to learn the meaning of each line of code. The interface is quite ready for automap and GRASS integration.

- Planned for next week

End of interface essential features and integration of automap most automated functions - ideally, the user will pick up the point layer containing data and press OK.

- Bottleneck(s)

The only bottleneck is wxWidgets handling, but not so much as one week ago.

Report #2.5, 11 June

First of all, the interface is almost finished. See a screenshot on the wiki page [0]. The layout includes a notebook with a page for each R package (automap, gstat and geoR), with the available options. Another layout could be a chiocebox wht the three packages on the top of Kriging section, that redraws the section accourding to the user's choice. Let me know which one do you prefer, feel free also to suggest something new.

The R-GRASS integration is on its way: autofitting the variogram on a map obtained from a DEM sample [1] works in the proof of concept. rpy2 is extremely helpful and does not require much more code than the original R code. More to come in the next days.

The idea for variogram fitting is to plot the variogram in a new frame and add some controls like sliders and/or text boxes to fill up with nugget, sill and range values. This will involve Python graphics, not R, as the former is more flexible.

Comments, suggestions, critics are welcome!

Report #3, 12 June

this weekly report will be very short, as yesterday I sent this email [0] and made little progress.

I'm having some problems in using autoKrige() function work, AFAIK because of projection information handling [1].

Report #4, 19 June

this week I made steady progress on the module. What I've done:

• renamed the module v.krige, as its features will go beyond automatic

kriging.

• addition of choicebox with only numerical columns, as interpolation

will be based on such variables

• refactoring on interface population - more to come
• started documentation page

Next week I plan to create the splashscreen during dependencies check and data load, and examine how to integrate gstat functions.

Report #5, 26 June (at OSGeo Bolsena Hacking Event)

just few minutes ago I succeded in completing the kriging procedure with gstat functions. It runs with a proof-of-concept dataset based on spearfish data, see this tutorial [0]. Martin helped me a lot in getting the standard wxGUI comboboxes to run properly with filters.

Next week I plan to adapt the interface to each R package's available options and consider how to solve lag issues in populating interface.

The blocking issue about autoKrige() and projections is no more valid as I create the grid myself based on GRASS region.

References

• [0] Isaaks and Srivastava, 1989: "An Introduction to Applied Geostatistics", (ISBN 0-19-505013-4)
• [1] http://www.geovariances.com/software/video-data-investigation-with-isatis-exploratory-data-analysis-ar0353.html
• [2] Jef Raskin, 2000: "The Humane Interface: New Directions for Designing Interactive Systems" (ISBN 0-201-37937-6)
• [3] http://lists.osgeo.org/pipermail/soc/