GRASS GSoC 2024 Improve user experience in Jupyter Notebooks

Student Name	Riya
Organization	​OSGeo - Open Source Geospatial Foundation
Mentor Name	Anna Petrasova, Corey White
GitHub Fork	View Repo
GSoC proposal	View Proposal

Abstract

The grass.jupyter.InteractiveMap() tool offers basic map visualization features like zooming and layer addition but suffers from suboptimal rendering speed (approximately 7.55 seconds per loop) and cumbersome data integration, particularly with large datasets. Additionally, its reliance on the Pseudo-Mercator projection limits its versatility. This abstract highlights the need for enhancements to improve rendering efficiency, streamline data integration, and broaden projection support for increased usability and functionality.

Goal

Main Goals:

1. Enhance InteractiveMap with interactive data retrieval features, enabling users to query attribute values and draw geometries directly on the map.
2. Implement support for custom coordinate reference systems (CRS) in InteractiveMap to eliminate the need for layer reprojection and improve visualization speed.
3. Optimize rendering processes for SeriesMap and TimeseriesMap by parallelizing map rendering using standard Python libraries like multiprocessing.

Stretch Goals:

1. Improve integration between GRASS and Pandas for smoother data handling and analysis.
2. Incorporate tile-based rendering to efficiently display extensive datasets without compromising performance.
3. Integrate a graphical user interface (GUI) within a Jupyter Notebook for enhanced user experience and ease of interaction with InteractiveMap functionalities.

Timeline

Period	Timeline	Tasks	Status
Bonding Period	May 1 - May 26	Initial research Proposal discussion with mentor Work on this PR	OK
Official Coding Period
	Week 1 (May 27 - June 1)	Write initial function for adding button to grass.jupyter.InteractiveMap()	OK
	Week 2 (June 2 - June 8)	Add click to see attributes features to the button	OK
	Week 3 (June 9 - June 15)	Add button to allow users to create drawings in the displayed InteractiveMap()	OK
	Week 4 (June 16 - June 22)	Add tools to the button to create different drawings	OK
	Week 5 (June 23 - June 29)	Add functionality to retrieve points from user drawn drawings and add them to the data, if the user wants	OK
	Week 6 (June 30 - July 6)	Speeding up the rendering of InteractiveMap() by skipping reprojection	In Progress
	Week 7 (July 7 - July 13)	Code refactoring, testing, documentation Mid-term Evaluation: July 12	OK
	Week 8 (July 14 - July 20)	Adding parallelization to TimeseriesMap and SeriesMap	OK
	Week 9 (July 21 - July 27)	Documentation, testing, writing tutorial	OK
	Week 10 (July 28 - August 3)	Working on a selected stretch goal Worked on Handling Toggle Buttons	OK
	Week 11 (August 4 - August 10)	Working on a selected stretch goal Worked on Handling Toggle Buttons	OK
	Week 12 (August 11 - August 17)	Write project summary	OK
Evaluation Period
	August 26 - September 2	Submit code and final evaluation	OK

Reports

Bonding Period

1. Contribution Guidelines: Spent time familiarizing myself with the project's contribution guidelines to ensure that my work aligns with the required standards.
2. Introduction: Introduced myself to the discourse mailing list.
3. WikiPages Setup: Created and organized my personal WikiPage and the project WikiPage, providing necessary documentation.
4. Development Environment Configuration: Successfully set up and configured the development environment to streamline my coding and testing processes.
5. GRASS GIS Exploration: Learnt more about GRASS GIS.
6. Remove Redundancy in the code: Continued working on the pull request to remove redundancies in the SeriesMap and TimeSeriesMap modules opened by me before the Contributor Proposals Opened.

Week-1

1. Created the Class BaseSeriesMap (PR)

   • The BaseSeriesMap class serves as a base class for SeriesMap and TimeSeriesMap.
   • It eliminates code redundancy and offers a unified interface for rendering images and creating interactive timeline sliders.

2. Query Button (PR)

   • Enabled users to retrieve the Latitude and Longitude of raster data upon activation of Query Button.

Week-2

1. Query Button (PR)

   • Added functionality to handle both raster and vector data.
   • The button now returns output to the user in the Coordinate Reference System (CRS).

Week-3

1. Query Button (PR)

   • Added a Popup button to display the queried point.
   • Enabled support for multiple raster and vector layers.

2. View/Update Computational Region Button (PR)

   • Began initial work on this feature, for example, adding the button, getting the region, etc.

Week-4

1. Query Button (PR)

   • Explored methods to dynamically calculate the query threshold distance.
   • Made slight modifications to the display of the Query button.

2. View/Update Computational Region Button (PR)

   • Attempted to use ipyleaflet.Rectangle as an alternative to DrawControl, but this method was unsuccessful.

Week-5

1. Query Button (PR)

   • Added test case for query button.

2. View/Update Computational Region Button (PR)

   • Added a rectangle to display the current computational region when this button is activated.
   • User can modify the current computational region by adjusting the rectangle.

Week-6

1. Query Button (PR)

   • Enhanced the appearance of the popup button that will display the output.

2. View/Update Computational Region Button (PR)

   • Worked on enhancing the display of the computational region.

Week-7

1. Query Button (PR)

   • Finalized the appearance of the popup button.

2. View/Update Computational Region Button (PR)

   • Added functionality for zooming into the computational region.

3. Drawing and Saving of Simple Geometries (PR)

   • Users can now draw simple geometries in InteractiveMap and save them as GRASS native vector maps.
   • Transitioned from DrawControl (deprecated) to GeomanDrawControl in ipyleaflet.

Week-8

1. Query Button (PR)

   • Worked on pylint errors.

2. Drawing and Saving of Simple Geometries (PR)

   • Discovered limitations of GeomanDrawControl and reverted to DrawControl.

3. Parallelization (PR)

   • Began working on parallelization for TimeseriesMap and SeriesMap.

Week-9

1. Drawing and Saving of Simple Geometries (PR)

   • Updated the display of buttons.

2. Parallelization (PR)

   • Continued working on parallelization for TimeseriesMap and SeriesMap.

Week-10

1. Drawing and Saving of Simple Geometries (PR)

   • Did some final edits in the code.

2. Parallelization (PR)

   • Added parallelization in the render function for TimeseriesMap and SeriesMap, and completed this task.

Week-11

1. Managing Multiple Toggle Buttons

   • Implemented InteractiveDrawController and InteractiveQueryController to handle individual events.

2. Modified Tutorials
3. Updated Descriptions

Week-12

1. Wrote Report
2. Started working on custom projections in InteractiveMap(PR)

Final Report

Title: Improve GRASS user experience in Jupyter Notebook

Organization: GRASS GIS - OSGeo

Abstract

This project introduces three significant enhancements to the InteractiveMap: a Query Button for querying specific points, a View/Update Computational Region Button for directly viewing and updating the computational region, and a Draw Geometries Button for creating and adding simple geometries as GRASS native vector maps. Additionally, the project introduces the BaseSeriesMap to reduce redundancy between SeriesMap and TimeSeriesMap, and adds parallelization to both SeriesMap and TimeSeriesMap for improved performance.

The state of integration BEFORE the start of GSoC

Previously, it was not possible to query vector/raster data, draw geometries, or view the computational region in InteractiveMap. While updating the region was possible, it required running additional commands. Additionally, improving the rendering speed of images in SeriesMap and TimeSeriesMap was not feasible.

The state of integration AFTER GSoC

1. Features in grass.InteractiveMap(): The grass.jupyter.interactivemap.py module includes three key features: the Query button (ℹ️), the View/Update Computational Region button (◻️), and the Draw Geometries button (🖉).
   1. Querying Raster/Vector Information at a Specific Point (ℹ️)
      1. Activate the Query Tool: Click the info button (ℹ️) to enable the query mode.
      2. Select a Point on the Map: Click on the map to retrieve raster/vector information for that specific point.
      3. Deactivate the Tool: Toggle the button off when you're done.
   2. Drawing and Saving Geometries as GRASS Native Vector Maps (🖉)
      1. Activate the Drawing Tool: Click the Pencil button (🖉) to start drawing on the map.
      2. Draw Your Geometry: You can draw a polyline, polygon, or circle marker directly on the map.
      3. Name the Vector Map: Enter the name for the new vector map in the New vector map name text box.
      4. Save the Geometry: Click the Save button to add the geometry to the map.
      5. Finalize and Close: The geometry is now added as a new layer on the map, and the drawing interface will automatically close.
   3. Viewing and Updating the Computational Region (◻️)
      1. Activate the Computational Region Tool: Click the View/Update Computational Region button (◻️).
      2. Visualize the Current Region: The current computational region is displayed as a rectangle on the map.
      3. Adjust the Region: Move the rectangle to a new location or resize it by dragging its vertices.
      4. Update the Region: Click Update Region to apply your changes to the computational region.
      5. Deactivate the Tool: Toggle the button off when you're done.
2. Removal of redundancy in the TimeSeriesMap and SeriesMap: A new class, BaseSeriesMap, has been added to serve as a parent class to both TimeSeriesMap and SeriesMap. This enhancement reduces redundancy and streamlines the codebase.
3. Parallelization in SeriesMap and TimeSeriesMap

   • Benchmark for TimeSeriesMap and SeriesMap: The following tables present the performance results obtained using 1-12 cores, as tested on the examples in temporal.ipynb and jupyter_tutorial.ipynb. Below is an example of how to reproduce this benchmark:

cores_range = range(1, 13)
benchmark_results = {}
for cores in cores_range:
    gs.run_command("g.gisenv", set=f"NPROCS={cores}")
    start_time = time.time()
    series = gj.SeriesMap(height=500)
    series.add_rasters(["elevation", "elevation_shade", "slope"])
    series.add_vectors(["streams", "streets", "viewpoints"])
    series.d_vect(map="streets")
    series.d_barscale()
    series.show()  # Create Slider
    end_time = time.time()
    elapsed_time = end_time - start_time
    benchmark_results[cores] = elapsed_time
    print(f"Cores: {cores}, Time: {elapsed_time:.2f} seconds")

Performance Results - TimeSeriesMap

Cores	1	2	3	4	5	6	7	8	9	10	11	12
Time (sec)	53.42	26.24	18.99	18.50	15.08	14.04	17.68	15.75	16.00	14.12	14.22	14.76

Performance Results - SeriesMap

Cores	1	2	3	4	5	6	7	8	9	10	11	12
Time (sec)	4.88	2.47	1.58	1.67	1.74	1.73	1.68	1.64	1.65	1.64	1.62	1.61

Conclusion

This project has successfully improved the user experience of GRASS GIS in Jupyter Notebook. Significant enhancements have been made to the InteractiveMap, SeriesMap, and TimeSeriesMap. The new features, such as the Query Button, View/Update Computational Region Button, and Draw Geometries Button, have streamlined data querying, visualization, and editing processes. Introducing the BaseSeriesMap has reduced redundancy, and parallelization has substantially improved performance. Benchmark results show notable speed improvements, making GRASS GIS more efficient and user-friendly. These advancements contribute to a more seamless and productive experience for users.

I appreciate the support and opportunity to contribute to GRASS GIS this summer. Thanks to my mentors and the GRASS Development Team for their guidance. Anna Petrasova's contributions were pivotal to the project's success, and I'm grateful for her support. Looking forward to contributing more.

Future Work

1. Improve the distance used in Query Button functionality.
2. Use GeomanDrawControl in the Draw Button functionality.
3. Enhanced connectivity with Pandas will enable users to display data with a single command.
4. Large, high-resolution image display will be enabled by implementing tiling techniques, reducing memory consumption and enabling smooth rendering of large images.
5. Implementing a graphical user interface (GUI) for tool presentation within Jupyter Notebooks would simplify usability and enhance user experience, facilitating smoother workflows within the platform.

Links

Pull requests related to grass.jupyter:

Ttile	Pull Request	Status
Create BaseSeriesMap to remove redundancies in SeriesMap and TimeSeriesMap	3468	Merged
Add Query Button to InteractiveMap	3793	Merged
Allow Users to view/update computational region	3838	Merged
Allow users to draw simple geometries and save it as vector	4003	Merged
Add Parallelization to TimeSeriesMap and SeriesMap	4097	Merged
Use InteractiveMap with custom projections	4204	Open
Add documentation of InteractiveMap Features and Parallelization in jupyter_tutorial.ipynb	4164	Merged
Modify Descriptions	4159	Merged

Some additional links:

Ttile	Link
OSGeo Wiki Page	Improve User Experience in Jupyter Notebooks
Github Fork	GitHub Fork
Example	jupyter_tutorial

Images

Draw and Query

Original Computational Region

Modified Computational Region

Shows the various draw controls of the Draw Button

Benchmark on SeriesMap

Benchmark on TimeSeriesMap

Media

GIF showing the function of update region button

Working of the Draw Button

Working of the Query Button

Working of Draw Geometries Button

Working of Query Button

Working of View/Update Region Button