GRASS GSoC 2024 Improve user experience in Jupyter Notebooks
Student Name | Riya |
Organization | OSGeo - Open Source Geospatial Foundation |
Mentor Name | Anna Petrasova, Corey White |
GitHub Fork | View Repo |
GSoC proposal | View Proposal |
Abstract
The grass.jupyter.InteractiveMap() tool offers basic map visualization features like zooming and layer addition but suffers from suboptimal rendering speed (approximately 7.55 seconds per loop) and cumbersome data integration, particularly with large datasets. Additionally, its reliance on the Pseudo-Mercator projection limits its versatility. This abstract highlights the need for enhancements to improve rendering efficiency, streamline data integration, and broaden projection support for increased usability and functionality.
Goal
Main Goals:
- Enhance InteractiveMap with interactive data retrieval features, enabling users to query attribute values and draw geometries directly on the map.
- Implement support for custom coordinate reference systems (CRS) in InteractiveMap to eliminate the need for layer reprojection and improve visualization speed.
- Optimize rendering processes for SeriesMap and TimeseriesMap by parallelizing map rendering using standard Python libraries like multiprocessing.
Stretch Goals:
- Improve integration between GRASS and Pandas for smoother data handling and analysis.
- Incorporate tile-based rendering to efficiently display extensive datasets without compromising performance.
- Integrate a graphical user interface (GUI) within a Jupyter Notebook for enhanced user experience and ease of interaction with InteractiveMap functionalities.
Timeline
Period | Timeline | Tasks | Status |
---|---|---|---|
Bonding Period |
May 1 - May 26 |
|
OK |
Official Coding Period | |||
Week 1 (May 27 - June 1) | Write initial function for adding button to grass.jupyter.InteractiveMap() | OK | |
Week 2 (June 2 - June 8) | Add click to see attributes features to the button | OK | |
Week 3 (June 9 - June 15) | Add button to allow users to create drawings in the displayed InteractiveMap() | OK | |
Week 4 (June 16 - June 22) | Add tools to the button to create different drawings | OK | |
Week 5 (June 23 - June 29) | Add functionality to retrieve points from user drawn drawings and add them to the data, if the user wants | OK | |
Week 6 (June 30 - July 6) | Speeding up the rendering of InteractiveMap() by skipping reprojection | In Progress | |
Week 7 (July 7 - July 13) |
|
OK | |
Week 8 (July 14 - July 20) | Adding parallelization to TimeseriesMap and SeriesMap | OK | |
Week 9 (July 21 - July 27) | Documentation, testing, writing tutorial | OK | |
Week 10 (July 28 - August 3) | OK | ||
Week 11 (August 4 - August 10) | OK | ||
Week 12 (August 11 - August 17) | Write project summary | OK | |
Evaluation Period | |||
August 26 - September 2 | Submit code and final evaluation | OK |
Reports
Bonding Period
- Contribution Guidelines: Spent time familiarizing myself with the project's contribution guidelines to ensure that my work aligns with the required standards.
- Introduction: Introduced myself to the discourse mailing list.
- WikiPages Setup: Created and organized my personal WikiPage and the project WikiPage, providing necessary documentation.
- Development Environment Configuration: Successfully set up and configured the development environment to streamline my coding and testing processes.
- GRASS GIS Exploration: Learnt more about GRASS GIS.
- Remove Redundancy in the code: Continued working on the pull request to remove redundancies in the
SeriesMap
andTimeSeriesMap
modules opened by me before the Contributor Proposals Opened.
Week-1
- Created the Class
BaseSeriesMap
(PR)- The
BaseSeriesMap
class serves as a base class forSeriesMap
andTimeSeriesMap
. - It eliminates code redundancy and offers a unified interface for rendering images and creating interactive timeline sliders.
- The
- Query Button (PR)
- Enabled users to retrieve the Latitude and Longitude of raster data upon activation of Query Button.
Week-2
- Query Button (PR)
- Added functionality to handle both raster and vector data.
- The button now returns output to the user in the Coordinate Reference System (CRS).
Week-3
- Query Button (PR)
- Added a Popup button to display the queried point.
- Enabled support for multiple raster and vector layers.
- View/Update Computational Region Button (PR)
- Began initial work on this feature, for example, adding the button, getting the region, etc.
Week-4
- Query Button (PR)
- Explored methods to dynamically calculate the query threshold distance.
- Made slight modifications to the display of the Query button.
- View/Update Computational Region Button (PR)
- Attempted to use
ipyleaflet.Rectangle
as an alternative toDrawControl
, but this method was unsuccessful.
- Attempted to use
Week-5
- Query Button (PR)
- Added test case for query button.
- View/Update Computational Region Button (PR)
- Added a rectangle to display the current computational region when this button is activated.
- User can modify the current computational region by adjusting the rectangle.
Week-6
- Query Button (PR)
- Enhanced the appearance of the popup button that will display the output.
- View/Update Computational Region Button (PR)
- Worked on enhancing the display of the computational region.
Week-7
- Query Button (PR)
- Finalized the appearance of the popup button.
- View/Update Computational Region Button (PR)
- Added functionality for zooming into the computational region.
- Drawing and Saving of Simple Geometries (PR)
- Users can now draw simple geometries in
InteractiveMap
and save them as GRASS native vector maps. - Transitioned from
DrawControl
(deprecated) toGeomanDrawControl
inipyleaflet
.
- Users can now draw simple geometries in
Week-8
- Query Button (PR)
- Worked on pylint errors.
- Drawing and Saving of Simple Geometries (PR)
- Discovered limitations of
GeomanDrawControl
and reverted toDrawControl
.
- Discovered limitations of
- Parallelization (PR)
- Began working on parallelization for
TimeseriesMap
andSeriesMap
.
- Began working on parallelization for
Week-9
- Drawing and Saving of Simple Geometries (PR)
- Updated the display of buttons.
- Parallelization (PR)
- Continued working on parallelization for
TimeseriesMap
andSeriesMap
.
- Continued working on parallelization for
Week-10
- Drawing and Saving of Simple Geometries (PR)
- Did some final edits in the code.
- Parallelization (PR)
- Added parallelization in the render function for
TimeseriesMap
andSeriesMap
, and completed this task.
- Added parallelization in the render function for
Week-11
- Managing Multiple Toggle Buttons
- Implemented
InteractiveDrawController
andInteractiveQueryController
to handle individual events.
- Implemented
- Modified Tutorials
- Updated Descriptions
Week-12
- Wrote Report
- Started working on custom projections in
InteractiveMap
(PR)
Final Report
Title: Improve GRASS user experience in Jupyter Notebook
Organization: GRASS GIS - OSGeo
Abstract
This project introduces three significant enhancements to the InteractiveMap
: a Query Button for querying specific points, a View/Update Computational Region Button for directly viewing and updating the computational region, and a Draw Geometries Button for creating and adding simple geometries as GRASS native vector maps. Additionally, the project introduces the BaseSeriesMap
to reduce redundancy between SeriesMap
and TimeSeriesMap
, and adds parallelization to both SeriesMap
and TimeSeriesMap
for improved performance.
The state of integration BEFORE the start of GSoC
Previously, it was not possible to query vector/raster data, draw geometries, or view the computational region in InteractiveMap
. While updating the region was possible, it required running additional commands. Additionally, improving the rendering speed of images in SeriesMap
and TimeSeriesMap
was not feasible.
The state of integration AFTER GSoC
- Features in
grass.InteractiveMap()
: Thegrass.jupyter.interactivemap.py
module includes three key features: the Query button (ℹ️), the View/Update Computational Region button (◻️), and the Draw Geometries button (🖉).- Querying Raster/Vector Information at a Specific Point (ℹ️)
- Activate the Query Tool: Click the info button (ℹ️) to enable the query mode.
- Select a Point on the Map: Click on the map to retrieve raster/vector information for that specific point.
- Deactivate the Tool: Toggle the button off when you're done.
- Drawing and Saving Geometries as GRASS Native Vector Maps (🖉)
- Activate the Drawing Tool: Click the Pencil button (🖉) to start drawing on the map.
- Draw Your Geometry: You can draw a polyline, polygon, or circle marker directly on the map.
- Name the Vector Map: Enter the name for the new vector map in the New vector map name text box.
- Save the Geometry: Click the Save button to add the geometry to the map.
- Finalize and Close: The geometry is now added as a new layer on the map, and the drawing interface will automatically close.
- Viewing and Updating the Computational Region (◻️)
- Activate the Computational Region Tool: Click the View/Update Computational Region button (◻️).
- Visualize the Current Region: The current computational region is displayed as a rectangle on the map.
- Adjust the Region: Move the rectangle to a new location or resize it by dragging its vertices.
- Update the Region: Click Update Region to apply your changes to the computational region.
- Deactivate the Tool: Toggle the button off when you're done.
- Querying Raster/Vector Information at a Specific Point (ℹ️)
- Removal of redundancy in the
TimeSeriesMap
andSeriesMap
: A new class,BaseSeriesMap
, has been added to serve as a parent class to bothTimeSeriesMap
andSeriesMap
. This enhancement reduces redundancy and streamlines the codebase. - Parallelization in
SeriesMap
andTimeSeriesMap
- Benchmark for
TimeSeriesMap
andSeriesMap
: The following tables present the performance results obtained using 1-12 cores, as tested on the examples intemporal.ipynb
andjupyter_tutorial.ipynb
. Below is an example of how to reproduce this benchmark:
- Benchmark for
cores_range = range(1, 13)
benchmark_results = {}
for cores in cores_range:
gs.run_command("g.gisenv", set=f"NPROCS={cores}")
start_time = time.time()
series = gj.SeriesMap(height=500)
series.add_rasters(["elevation", "elevation_shade", "slope"])
series.add_vectors(["streams", "streets", "viewpoints"])
series.d_vect(map="streets")
series.d_barscale()
series.show() # Create Slider
end_time = time.time()
elapsed_time = end_time - start_time
benchmark_results[cores] = elapsed_time
print(f"Cores: {cores}, Time: {elapsed_time:.2f} seconds")
Cores | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 |
---|---|---|---|---|---|---|---|---|---|---|---|---|
Time (sec) | 53.42 | 26.24 | 18.99 | 18.50 | 15.08 | 14.04 | 17.68 | 15.75 | 16.00 | 14.12 | 14.22 | 14.76 |
Cores | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 |
---|---|---|---|---|---|---|---|---|---|---|---|---|
Time (sec) | 4.88 | 2.47 | 1.58 | 1.67 | 1.74 | 1.73 | 1.68 | 1.64 | 1.65 | 1.64 | 1.62 | 1.61 |
Conclusion
This project has successfully improved the user experience of GRASS GIS
in Jupyter Notebook
. Significant enhancements have been made to the InteractiveMap
, SeriesMap
, and TimeSeriesMap
. The new features, such as the Query Button, View/Update Computational Region Button, and Draw Geometries Button, have streamlined data querying, visualization, and editing processes. Introducing the BaseSeriesMap
has reduced redundancy, and parallelization has substantially improved performance. Benchmark results show notable speed improvements, making GRASS GIS
more efficient and user-friendly. These advancements contribute to a more seamless and productive experience for users.
I appreciate the support and opportunity to contribute to GRASS GIS this summer. Thanks to my mentors and the GRASS Development Team for their guidance. Anna Petrasova's contributions were pivotal to the project's success, and I'm grateful for her support. Looking forward to contributing more.
Future Work
- Improve the distance used in Query Button functionality.
- Use
GeomanDrawControl
in the Draw Button functionality. - Enhanced connectivity with Pandas will enable users to display data with a single command.
- Large, high-resolution image display will be enabled by implementing tiling techniques, reducing memory consumption and enabling smooth rendering of large images.
- Implementing a graphical user interface (GUI) for tool presentation within Jupyter Notebooks would simplify usability and enhance user experience, facilitating smoother workflows within the platform.
Links
Pull requests related to grass.jupyter
:
Ttile | Pull Request |
---|---|
Create BaseSeriesMap to remove redundancies in SeriesMap and TimeSeriesMap | 3468 |
Add Query Button to InteractiveMap | 3793 |
Allow Users to view/update computational region | 3838 |
Allow users to draw simple geometries and save it as vector | 4003 |
Add Parallelization to TimeSeriesMap and SeriesMap | 4097 |
Use InteractiveMap with custom projections
|
4204 |
Add documentation of InteractiveMap Features and Parallelization in jupyter_tutorial.ipynb | 4164 |
Modify Descriptions | 4159 |
Some additional links:
Ttile | Link |
---|---|
OSGeo Wiki Page | Improve User Experience in Jupyter Notebooks |
Github Fork | GitHub Fork |
Example | jupyter_tutorial
|
Images
Media