GRASS GSoC 2024 Improve user experience in Jupyter Notebooks

From GRASS-Wiki
Jump to navigation Jump to search
Student Name Riya
Organization ​OSGeo - Open Source Geospatial Foundation
Mentor Name Anna Petrasova, Corey White
GitHub Fork View Repo
GSoC proposal View Proposal

Abstract

The grass.jupyter.InteractiveMap() tool offers basic map visualization features like zooming and layer addition but suffers from suboptimal rendering speed (approximately 7.55 seconds per loop) and cumbersome data integration, particularly with large datasets. Additionally, its reliance on the Pseudo-Mercator projection limits its versatility. This abstract highlights the need for enhancements to improve rendering efficiency, streamline data integration, and broaden projection support for increased usability and functionality.

Goal

Main Goals:

  1. Enhance InteractiveMap with interactive data retrieval features, enabling users to query attribute values and draw geometries directly on the map.
  2. Implement support for custom coordinate reference systems (CRS) in InteractiveMap to eliminate the need for layer reprojection and improve visualization speed.
  3. Optimize rendering processes for SeriesMap and TimeseriesMap by parallelizing map rendering using standard Python libraries like multiprocessing.

Stretch Goals:

  1. Improve integration between GRASS and Pandas for smoother data handling and analysis.
  2. Incorporate tile-based rendering to efficiently display extensive datasets without compromising performance.
  3. Integrate a graphical user interface (GUI) within a Jupyter Notebook for enhanced user experience and ease of interaction with InteractiveMap functionalities.

Timeline

Period Timeline Tasks Status
Bonding Period

May 1 - May 26

  1. Initial research
  2. Proposal discussion with mentor
  3. Work on this PR
OK
Official Coding Period
Week 1 (May 27 - June 1) Write initial function for adding button to grass.jupyter.InteractiveMap() OK
Week 2 (June 2 - June 8) Add click to see attributes features to the button OK
Week 3 (June 9 - June 15) Add button to allow users to create drawings in the displayed InteractiveMap() OK
Week 4 (June 16 - June 22) Add tools to the button to create different drawings OK
Week 5 (June 23 - June 29) Add functionality to retrieve points from user drawn drawings and add them to the data, if the user wants OK
Week 6 (June 30 - July 6) Speeding up the rendering of InteractiveMap() by skipping reprojection Ongoing
Week 7 (July 7 - July 13)
  1. Code refactoring, testing, documentation
  2. Mid-term Evaluation: July 12
OK
Week 8 (July 14 - July 20) Adding parallelization to TimeseriesMap and SeriesMap OK
Week 9 (July 21 - July 27) Documentation, testing, writing tutorial OK
Week 10 (July 28 - August 3) Working on a selected stretch goal Worked on Handling Toggle Buttons OK
Week 11 (August 4 - August 10) Working on a selected stretch goal Worked on Handling Toggle Buttons OK
Week 12 (August 11 - August 17) Write project summary
Evaluation Period
August 26 - September 2 Submit code and final evaluation

Reports

Bonding Period

  1. Contribution Guidelines: Spent time familiarizing myself with the project's contribution guidelines to ensure that my work aligns with the required standards.
  2. Introduction: Introduced myself to the discourse mailing list.
  3. WikiPages Setup: Created and organized my personal WikiPage and the project WikiPage, providing necessary documentation.
  4. Development Environment Configuration: Successfully set up and configured the development environment to streamline my coding and testing processes.
  5. GRASS GIS Exploration: Learnt more about GRASS GIS.
  6. Remove Redundancy in the code: Continued working on the pull request to remove redundancies in the SeriesMap and TimeSeriesMap modules opened by me before the Contributor Proposals Opened.

Week-1

  1. Created the Class BaseSeriesMap (PR)
    • The BaseSeriesMap class serves as a base class for SeriesMap and TimeSeriesMap.
    • It eliminates code redundancy and offers a unified interface for rendering images and creating interactive timeline sliders.
  2. Query Button (PR)
    • Enabled users to retrieve the Latitude and Longitude of raster data upon activation of Query Button.

Week-2

  1. Query Button (PR)
    • Added functionality to handle both raster and vector data.
    • The button now returns output to the user in the Coordinate Reference System (CRS).

Week-3

  1. Query Button (PR)
    • Added a Popup button to display the queried point.
    • Enabled support for multiple raster and vector layers.
  2. View/Update Computational Region Button (PR)
    • Began initial work on this feature, for example, adding the button, getting the region, etc.

Week-4

  1. Query Button (PR)
    • Explored methods to dynamically calculate the query threshold distance.
    • Made slight modifications to the display of the Query button.
  2. View/Update Computational Region Button (PR)
    • Attempted to use ipyleaflet.Rectangle as an alternative to DrawControl, but this method was unsuccessful.

Week-5

  1. Query Button (PR)
    • Added test case for query button.
  2. View/Update Computational Region Button (PR)
    • Added a rectangle to display the current computational region when this button is activated.
    • User can modify the current computational region by adjusting the rectangle.

Week-6

  1. Query Button (PR)
    • Enhanced the appearance of the popup button that will display the output.
  2. View/Update Computational Region Button (PR)
    • Worked on enhancing the display of the computational region.

Week-7

  1. Query Button (PR)
    • Finalized the appearance of the popup button.
  2. View/Update Computational Region Button (PR)
    • Added functionality for zooming into the computational region.
  3. Drawing and Saving of Simple Geometries (PR)
    • Users can now draw simple geometries in InteractiveMap and save them as GRASS native vector maps.
    • Transitioned from DrawControl (deprecated) to GeomanDrawControl in ipyleaflet.

Week-8

  1. Query Button (PR)
    • Worked on pylint errors.
  2. Drawing and Saving of Simple Geometries (PR)
    • Discovered limitations of GeomanDrawControl and reverted to DrawControl.
  3. Parallelization (PR)
    • Began working on parallelization for TimeseriesMap and SeriesMap.

Week-9

  1. Drawing and Saving of Simple Geometries (PR)
    • Updated the display of buttons.
  2. Parallelization (PR)
    • Continued working on parallelization for TimeseriesMap and SeriesMap.

Week-10

  1. Drawing and Saving of Simple Geometries (PR)
    • Did some final edits in the code.
  2. Parallelization (PR)
    • Added parallelization in the render function for TimeseriesMap and SeriesMap, and completed this task.

Week-11

  1. Managing Multiple Toggle Buttons
  2. Modified Tutorials
  3. Updated Descriptions

Week-12

  1. Wrote Report
  2. Started working on custom projections in InteractiveMap(PR)

Final Report

Title: Improve GRASS user experience in Jupyter Notebook

Organization: GRASS GIS - OSGeo


Abstract

This project introduces three significant enhancements to the InteractiveMap: a Query Button for querying specific points, a View/Update Computational Region Button for directly viewing and updating the computational region, and a Draw Geometries Button for creating and adding simple geometries as GRASS native vector maps. Additionally, the project introduces the BaseSeriesMap to reduce redundancy between SeriesMap and TimeSeriesMap, and adds parallelization to both SeriesMap and TimeSeriesMap for improved performance.


The state of integration BEFORE the start of GSoC

Previously, it was not possible to query vector/raster data, draw geometries, or view the computational region in InteractiveMap. While updating the region was possible, it required running additional commands. Additionally, improving the rendering speed of images in SeriesMap and TimeSeriesMap was not feasible.


The state of integration AFTER GSoC

  1. Features in grass.InteractiveMap(): The grass.jupyter.interactivemap.py module includes three key features: the Query button (ℹ️), the View/Update Computational Region button (◻️), and the Draw Geometries button (🖉).
    1. Querying Raster/Vector Information at a Specific Point (ℹ️)
      1. Activate the Query Tool: Click the info button (ℹ️) to enable the query mode.
      2. Select a Point on the Map: Click on the map to retrieve raster/vector information for that specific point.
      3. Deactivate the Tool: Toggle the button off when you're done.
    2. Drawing and Saving Geometries as GRASS Native Vector Maps (🖉)
      1. Activate the Drawing Tool: Click the Pencil button (🖉) to start drawing on the map.
      2. Draw Your Geometry: You can draw a polyline, polygon, or circle marker directly on the map.
      3. Name the Vector Map: Enter the name for the new vector map in the New vector map name text box.
      4. Save the Geometry: Click the Save button to add the geometry to the map.
      5. Finalize and Close: The geometry is now added as a new layer on the map, and the drawing interface will automatically close.
    3. Viewing and Updating the Computational Region (◻️)
      1. Activate the Computational Region Tool: Click the View/Update Computational Region button (◻️).
      2. Visualize the Current Region: The current computational region is displayed as a rectangle on the map.
      3. Adjust the Region: Move the rectangle to a new location or resize it by dragging its vertices.
      4. Update the Region: Click Update Region to apply your changes to the computational region.
      5. Deactivate the Tool: Toggle the button off when you're done.
  2. Removal of redundancy in the TimeSeriesMap and SeriesMap: A new class, BaseSeriesMap, has been added to serve as a parent class to both TimeSeriesMap and SeriesMap. This enhancement reduces redundancy and streamlines the codebase.
  3. Parallelization in SeriesMap and TimeSeriesMap
    • Benchmark for TimeSeriesMap and SeriesMap: The following tables present the performance results obtained using 1-12 cores, as tested on the examples in temporal.ipynb and jupyter_tutorial.ipynb. Below is an example of how to reproduce this benchmark:
cores_range = range(1, 13)
benchmark_results = {}
for cores in cores_range:
    gs.run_command("g.gisenv", set=f"NPROCS={cores}")
    start_time = time.time()
    series = gj.SeriesMap(height=500)
    series.add_rasters(["elevation", "elevation_shade", "slope"])
    series.add_vectors(["streams", "streets", "viewpoints"])
    series.d_vect(map="streets")
    series.d_barscale()
    series.show()  # Create Slider
    end_time = time.time()
    elapsed_time = end_time - start_time
    benchmark_results[cores] = elapsed_time
    print(f"Cores: {cores}, Time: {elapsed_time:.2f} seconds")


Performance Results - TimeSeriesMap
Cores 1 2 3 4 5 6 7 8 9 10 11 12
Time (sec) 53.42 26.24 18.99 18.50 15.08 14.04 17.68 15.75 16.00 14.12 14.22 14.76
Performance Results - SeriesMap
Cores 1 2 3 4 5 6 7 8 9 10 11 12
Time (sec) 4.88 2.47 1.58 1.67 1.74 1.73 1.68 1.64 1.65 1.64 1.62 1.61

Conclusion

This project has successfully improved the user experience of GRASS GIS in Jupyter Notebook. Significant enhancements have been made to the InteractiveMap, SeriesMap, and TimeSeriesMap. The new features, such as the Query Button, View/Update Computational Region Button, and Draw Geometries Button, have streamlined data querying, visualization, and editing processes. Introducing the BaseSeriesMap has reduced redundancy, and parallelization has substantially improved performance. Benchmark results show notable speed improvements, making GRASS GIS more efficient and user-friendly. These advancements contribute to a more seamless and productive experience for users.

I appreciate the support and opportunity to contribute to GRASS GIS this summer. Thanks to my mentors and the GRASS Development Team for their guidance. Anna Petrasova's contributions were pivotal to the project's success, and I'm grateful for her support. Looking forward to contributing more.

Future Work

  1. Improve the distance used in Query Button functionality.
  2. Use GeomanDrawControl in the Draw Button functionality.
  3. Enhanced connectivity with Pandas will enable users to display data with a single command.
  4. Large, high-resolution image display will be enabled by implementing tiling techniques, reducing memory consumption and enabling smooth rendering of large images.
  5. Implementing a graphical user interface (GUI) for tool presentation within Jupyter Notebooks would simplify usability and enhance user experience, facilitating smoother workflows within the platform.

Links

Pull requests related to grass.jupyter:

Ttile Pull Request
Create BaseSeriesMap to remove redundancies in SeriesMap and TimeSeriesMap 3468
Add Query Button to InteractiveMap 3793
Allow Users to view/update computational region 3838
Allow users to draw simple geometries and save it as vector 4003
Add Parallelization to TimeSeriesMap and SeriesMap 4097
Use InteractiveMap with custom projections 4204
Add documentation of InteractiveMap Features and Parallelization in jupyter_tutorial.ipynb 4164
Modify Descriptions 4159

Some additional links:

Ttile Link
OSGeo Wiki Page Improve User Experience in Jupyter Notebooks
Github Fork GitHub Fork
Example jupyter_tutorial

Images

Draw and Query Functioning
Draw and Query
Shows the original computational region
Original Computational Region
Shows the modified computational region
Modified Computational Region


Picture showing the various draw controls of the Draw Button
Shows the various draw controls of the Draw Button
Displays benchmark on SeriesMap
Benchmark on SeriesMap
Displays benchmark on TimeSeriesMap
Benchmark on TimeSeriesMap

Media

GIF showing the function of update region button
GIF showing the function of update region button
GIF showing the working of the Draw Button
Working of the Draw Button
GIF showing the working of the Query Button
Working of the Query Button


Working of Draw Geometries Button
Working of Query Button
Working of View/Update Region Button