GRASS GSoC 2013 GRASS GIS Interactive Scatter Plot Tool

From GRASS-Wiki
Revision as of 10:07, 23 September 2013 by ⚠️Turek (talk | contribs) (→‎Roadmap)
Jump to navigation Jump to search

(See also other GRASS GSoC 2013 projects)

Student Name: Stepan Turek, Czech Technical University in Prague
Organization: OSGeo - Open Source Geospatial Foundation
Mentor Name: Martin Landa Backup mentor: Michael Barton
Title: GRASS GIS Interactive Scatter Plot Tool
Repository: sandbox, browse at: scatter_plot
Abstract

Goal of the project is development of interactive scatter plot tool, which will be integrated into wxGUI. The tool will improve GRASS GIS supervised classification workflow. Also it will be useful for raster data analysis in general. Additional goal is extension of GRASS GIS plotting capability by integration of Matplotlib plotting library into wxGUI.

Application

Introduction

Common analysis, performed on remote sensing data, is supervised classification. GRASS GIS offers necessary tools needed for performing classification workflow. However, there are some weak spots, which can be improved.

One of such a spot is limited real time analyzing of classes (categories) according to chosen data in training areas. Recently, the process of training area selection has been significantly improved by wxGUI Supervised Classification Tool. This tool simplifies definition of classes and their training areas. It also allows to show some basic statistics for classes and bands.

GRASS GIS has Bivariate Scatterplot Tool, which creates scatter plot for selected bands combination. However this tool is not connected with the Supervised Classification Tool and therefore it can not be used in process of training areas selection.

The idea

Main idea of the project is development of tool, which would allow to look at raster data from new points of view unveiling relations, which otherwise would stay hidden.

The tool would be fully integrated into wxGUI therefore it will be connected with already existing components (Map display window and the Supervised Classification Tool).

Thanks to the connection with the Supervised Classification Tool it will be possible to highlight plotted points according to currently chosen pixels for classes by it's training areas. Thanks to this information user will be able to gradually change configuration of training areas achieving better classes separability during supervised classification. Also the tool will be able to plot confidence ellipses for classes.

The scatter plot tool will be able to work with multiple plots, which will represent multiple raster bands combinations. It will be useful for imagery data, which are usually comprised from more than two bands. All such a plots will be interactively connected, therefore user will be able to define areas in plots and the tool will highlight pixels in map display window and corresponding points in other plots. These areas will be grouped into categories and there will be kind of category manager, which will allow to define the categories plotting order, plotting style etc.

Also it can be used for pure pixels identification. If bands of principal component analysis of analysed imagery data will be added into the tool, it will be possible to identify endmember points, which lie in a corners of PCA space scatter plot. It will be possible to see the pure pixels in map display window and corresponding points in other plots by selection of these points. The pure pixels are suitable to be used as class training areas, because they are not mixed with other classes (endmembers) and therefore they represent only surface of individual class (endmember).

It will be possible to create training areas from scatter plot. This feature could be used for creation of training areas from pure pixels.

The tool could be useful for analyzing of raster data in general, therefore it will be also integrated into Map display window. For instance, it could help to see correlation of analyzed data or detect outliers. It will be possible to create mask from selected pixels belonging to some category. Using this feature it will be possible to easily get rid of the effect of the outliers in subsequent analyses.

Because analyzed data can have big size, significant attention will be paid to make the tool memory efficient and fast.

Plotting part of the tool will use Matplotlib library. The library will be integrated into wxGUI. The integration should make plotting in wxGUI easier and plotting code reusable e. g. settings of plot properties or implementation of more abstract functions.

Exact way of integration will be determined when I will be more experience with usage of Matplotlib.

Usage of integrated Matplotlib will be demonstrated by development of feature, which allows to create different graph types (e. g. line graph, bar graph) from data in vector maps or raster maps.

Project plan

Period Task Status Notes
May 27 - June 16 Bonding period
June 17 back-end integration with map window
June 24 back-end integration with map window
July 1 back-end integration with wxGUI Supervised Classification Tool
July 8 back-end integration with wxGUI Supervised Classification Tool
July 15 back-end integration with wxGUI Supervised Classification Tool
July 22 testing and optimization of back-end
July 29 testing and optimization of back-end
August 2 Mid-term evaluation The goal: back-end is finished passed
August 5 design of Matplotlib integration into wxGUI
August 12 development of front end of the scatter plot tool
August 19 development of front end of the scatter plot tool
August 26 development of front end of the scatter plot tool
September 2 implementation of the plotting feature for vector/raster data
September 9 implementation of the plotting feature for vector/raster data
September 16 testing, documentation
September 23 Final evaluation

Testing

Currently testing is possible applying testing_patch.diff to GRASS 7 from sandbox repository.

cd grass_trunk
wget http://svn.osgeo.org/grass/sandbox/turek/scatter_plot/testing_patch.diff
patch -p0 < testing_patch.diff
make

Open wxGUI Supervised Classification Tool and from toolbar click on Open Scatter Plot Tool.

Roadmap

Week 1 (June 17)

  • worked on integration of the scatter plot tool with mapwindow

Week 2 (June 24)

  • implemented highlighting of pixels in mapwidnow according to selected areas in scatter plots
Highlighted pixels in mapwindow (red color) corresponding to selected areas in scatter plots (green color) (2013-7-21)

Week 3 (July 8)

  • studying raster library

Week 4 (July 15)

  • multiprocessing module, integration of raster library into the backend

Week 5 (July 22)

  • integration into wxIClass
Backend preview

Week 6 (July 29)

  • worked on releasing of alpha version -
    • bugs fixing
    • backend optimization
    • better integration with wxIClass
  • released all source code sandbox (alpha version)

Week 7 (August 5)

  • backend optimization + testing
  • clarified project plan for rest of GSoC with mentor

Week 8 (August 12)

  • worked on layout of frontend in wxIClass
New layout of the scatter plot tool in wx.IClass (2013-8-10)

Week 9 (August 19)

  • worked polygon selection

Week 10 (August 26)

  • implemented selection of areas by polygons in scatter plots and plotting of confidence ellipses
Selection of categories by polygons and plotting confidence ellipses (2013-8-26)

Week 11 (September 2)

  • plots rendering optimization including speeding up and reducing memory consumption
    • much better plotting separation from WxGUI thread. now only small part of plotting runs in wxGUI thread (because is directly connected to wxGUI)
    • creating composite image of categories by numpy (including transparency support), thanks to that only one merged image is added into scatter plot representing all categories
    • discussed and implemented advises from matplotlib comunity about memory reduction of plotting scatter plots for 12 bit data see
  • thanks to that it is possible to analyze 12 bit raster data

Week 12 (September 9)

  • bugs fixing, optimalization

Week 13 (September 16)

  • integration into main gui, bugs fixing

Week 14 (September 23)

  • documentation, video tutorial, various fixes and improvements


Provided ID could not be validated.

Scatter plot tool integrated into main gui (2013-9-13)

Week 14 (September 23)

  • documentation, video tutorial, various fixes and improvements
  • tool wiki page

Provided ID could not be validated.

Weekly Reports