NSF POSE Project 2023-2025 Timeline

To inform GRASS community about the work done as part of the award 2303651 from the U.S. National Science Foundation (NSF) to enhance GRASS ecosystem, the following report summarizes the progress and outcomes in quarterly reports and presents the roadmap for the project.

The two main goals of the project are: 1) to facilitate the adoption of GRASS GIS as a key geoprocessing engine by a growing number of researchers and geospatial practitioners in academia, governments, and industry; and 2) to expand and diversify the developer community, especially through supporting next-generation scientists to gain expertise to maintain and innovate GRASS software.

Principal investigators and senior personnel:

• Helena Mitasova, North Carolina State University (Principal Investigator)
• Vaclav Petras, North Carolina State University (Co-Principal Investigator)
• Anna Petrasova, North Carolina State University (Co-Principal Investigator)
• C Michael Barton, Arizona State University (Co-Principal Investigator)
• Huidae Cho, New Mexico State University (Co-Principal Investigator)
• Giuseppe Amatulli, Yale (Senior Personnel)

Other personnel who is joining for the execution of the grant is listed below when relevant.

Y1 1st quarter report

Project Announcements and Communication

• Announcement on GRASS website.
• Info session to explain the POSE Project (Announcement on GRASS user mailing list, 2 sessions for different time zones).
• Collected statistics from GitHub, X and others and set up automated collection for GitHub.

Conference Presentations

• Parallelization Tips for Geoprocessing with GRASS GIS by Anna Petrasova at FOSS4GNA, October 2023.
• GRASS GIS: Not What You Think by Vaclav Petras at FOSS4GNA, October 2023.
• Computational Notebooks for Reproducible Geospatial Computation in Research and Education by Caitlin Haedrich at FOSS4GNA, October 2023.
• State of GRASS GIS: 40 Years Strong and Counting by Huidae Cho at FOSS4G Asia, November 2023.

Training and Workshops

• Unlock the power of GRASS GIS by Anna Petrasova and Caitlin Haedrich at FOSS4GNA, October 2023.
• Develop Geospatial Workflows and Custom Tools with GRASS GIS by Vaclav Petras and Corey White at FOSS4GNA, October 2023.
• Big Data Computing with GRASS GIS by Anna Petrasova and Vaclav Petras at NC State GIS week, November 2023.
• GIS Week Workshop: High Performance Computing for Geospatial Analysis (GRASS GIS part) by Vaclav Petras (main lecture lead by Andrew Petersen from NCSU OIT) at NC State GIS week, November 2023.
• Geocomputation and Machine Learning for Environmental Applications training course announced. GRASS GIS part led by Giuseppe Amatulli is free of charge thanks to the support by NSF.

Community Sprint Participation

Anna Petrasova and Vaclav Petras remotely participated in OSGeo 2023 code sprint doing PR reviews and maintenance of GRASS GIS code, documentation, and website.

Mentoring Program

• Started a development-oriented mentoring program announced with details available on a wiki page.
• Program now has 4 participants (from 12 responses) supported by email and video calls.
• Program already resulted in improvements to GRASS GIS core and addons:
  • merged PR: grass.script: ensure memmap content is flushed before writing to a raster
  • radar addon toolset PR: Add i.saocom and i.sar toolsets

Mentoring Program is a new effort started and so-far performed only by the POSE team.

Student Grants Program

Student grants announced with topics at a wiki page. Student Grants Program is an existing effort which the POSE team participates in.

Working Groups

Four working groups created (scope, wiki page, coordinator, and a list of people for each).

Citations

• GRASS GIS Google Scholar account updated with more complete records.
• A form was created to collect new and missing records.
• Profile link added to GRASS website Contribute and Citation page.

Y1 2nd quarter report

NSF I-Corps program

Corey White, Vaclav Petras, and Anna Petrasova participated in a 4-week long intensive mandatory training that draws on the methods, customer discovery, and curriculum of the NSF Innovation Corps and applies them to open-source projects.

The program included conducting 70+ 30-minute interviews with both GRASS users and non-users to better shape the future path of GRASS GIS and the POSE project. So far, the interviews focused on educators, companies, researchers, influencers and it focused more on non-users than users.

Next steps, include processing the collected qualitative data, developing possible future directions for sustainability of the project, discussing these with the Project Steering Committee, the core team of developers, and the broader community.

Mentoring Program

The development oriented mentoring program welcomed 5 new participants from various institutions including universities, USGS, US Army Corps of Engineers, and local municipalities. The discussed topics covered developing workflows for topography and network analysis, environmental modeling, and geophysics. Participants were supported by email and video calls.

Conference Presentations

• Modeling vegetated stream buffer impacts on water depth and discharge at the watershed scale with GRASS GIS and Jupyter Notebooks by Caitlin Haedrich at AGU, December 2023.
• OpenPlains: A Web-Based Platform to Democratize Geospatial Participatory Modeling on the Cloud by Corey White at AGU, December 2023.

Presentations and Workshops

• Intro to GRASS GIS by Vaclav Petras and Anna Petrasova was remotely delivered to geographers at Vanderbilt University in February 2024. The workshop was developed in Jupyter Lab and run using WholeTale online platform.
• Introduction to GRASS GIS Geospatial Processing by Corey White at SMathHacks hackathon, March 2nd (supported also by Veronica Andreo and Vaclav Petras). SMathHacks is the annual 36-hour hackathon hosted by the NC School of Science and Math, open to all North Carolina students. Students collaborate to build innovative software projects, explore new technologies, and have fun along the way. The workshop was delivered in person with students using Google Colab.

Student Grant Program

Linda Karlovska participates in the Student Grant program with her project Easy command history navigation through the History browser panel mentored by Anna Petrasova (POSE team) and Martin Landa (FCE CTU). Student Grants Program is an existing effort which the POSE team participates in by mentoring.

Code Security

Vaclav Petras mentors four students who create code security improvements in their NCSU class CSC 472: Cybersecurity Practicum. So far, the students created two PRs which were successfully merged:

• grass.script: Change insecure mktemp to NamedTemporaryFile #3444
• utils: fixed shell vulnerability in mkrest.py #3451

Two other PRs are open and more work is underway to do first steps for new approach to a more secure code.

Localized Basic Datasets

• NCSU and ASU teams worked on localized versions of GRASS basic data set following the North Carolina basic data set example, preparing Flagstaff, Arizona and Nepal draft version with basic layers

Y1 3rd quarter report

Mentoring Activities

The mentoring program form received 2 new responses and mentors were actively working with 3 mentees. It resulted in fixes in v.transects and r.accumulate.

Linda Kladivova finished her project developing History browser in the GRASS GUI as part of the GRASS Student Grant Program. Mentor's time was covered by POSE.

GRASS GIS has 3 students this year for Google Summer of Code. Time spent managing GSoC for GRASS GIS was covered by POSE.

The topics covered by GSoC projects are:

• Add EODAG support to GRASS GIS
• Improve GRASS user experience in Jupyter Notebook
• Add JSON output to different GRASS tools in C

Presentations, Workshops, Event Planning

• GRASS 101: From GUI Clicks to Writing Scripts (TNGIC 2024) by Veronica Andreo and Caitlin Haedrich
• Coastal evolution analysis and inundation modeling with GRASS GIS (CSDMS Workshop 2024) by Caitlin Haedrich and Pratikshya Regmi, see also blog post with images

• Organization of GRASS Community Meeting Prague 2024 and budget planning by Vaclav Petras

CMake transition

Ongoing work on CMake compilation was partially covered by POSE.

Code Quality and Security

Code quality improvements in Python code:

• PR addressing Flake8 E501 (long lines in 158 files not fixable by automated tools)
• PR addressing Flake8 E401 (unused imports)

Vaclav Petras mentored a team of four students who create code security improvements in their NCSU class CSC 472: Cybersecurity Practicum. The team created 12 PRs which were successfully merged, 7 additional experimental PRs (not merged) and 2 PRs which are still open at this time. Specifically, the team achieved the following:

• With the involvement and help from the other contributors, the team introduced Bandit and Coverity Scan as additional code scanning tools for Python and C, respectively.
• The team analyzed issues reported CodeQL, Bandit, and Coverity Scan and reported on relevance of different issues to the project (report privately shared with the GRASS Development Team).
• Vulnerability Triage Guide (also privately shared with the GRASS Development Team).
• The team also developed a security policy for the project.
• The vulnerability reporting and security advisories were enabled based on the team recommendations.
• The team fixed several specific security issues in the code:
  • packaging: Use subprocess instead of os.popen for change log creation #3469 (merged)
  • lib raster: fixed security vulnerabilities and weaknesses #3549 (merged)
  • utils: fixed shell vulnerability in mkrest.py #3451 (merged)
  • grass.script: Change insecure mktemp to NamedTemporaryFile #3444 (merged)
  • lib/psdriver: Fix issues with wrong type of arguments to printf #3551 (PR open)
  • g.region/r.to.rast3elev: fixed scanf error to recognize EOF as a possible return value #3452 (PR open)

Documentation

• Documentation and Education Working Group met and planned the development of new tutorials. Veronica Andreo met other GRASS developers and contributors to coordinate further tutorials development.
• Development of new tutorials in progress, topics include:
  • Getting started with GRASS GIS
  • GRASS GIS in Google Colab
  • GRASS GIS and Python
  • GRASS GIS and R
  • GRASS R and Python comparison
  • Making plots in GRASS GIS
• New GRASS Programming Style Guide
• New addon template powered by Cookiecutter
• Analysis of options names and functionality in plotting tools:
  • Addons for plotting: Standardize parameters and flags

Communication and outreach

• Creation of 4 news post in the GRASS website
• Social media:
  • Twitter: 9 new posts/threads
  • Fosstodon: 9 new posts/threads

Other sustainability activities

• Code of Conduct added to the main repository
• revised Issue template
• New RFC for a Release Policy
• Revised RFC for Release Procedure
• WIP: GRASS GIS Roadmap
• GRASS website: ongoing restructuring
• PSC page moved from trac wiki to media wiki

Y1 4th quarter report

Mentoring Activities

The mentoring program form received 3 new responses and mentors were actively working with 3 mentees. Results include parallelization of r.texture, r.horizon, improved grass.benchmarking library and a new tutorial on lidar data.

GSoC successfully completed their projects:

• Add EODAG support to GRASS GIS
• Improve GRASS user experience in Jupyter Notebook
• Add JSON output to different GRASS tools in C

Time spent by mentoring was covered by NSF.

Presentations, Workshops, Event Planning

• Let's combine GRASS, Python and R: Satellite time series data for species distribution modeling. Workshop by Veronica Andreo. FOSS4G Europe 2024, Tartu.
• Status of GRASS GIS project. Talk by Veronica Andreo. FOSS4G Europe 2024, Tartu.
• Boost Spatial Data Science Workflows with GRASS GIS and R. Talk by Veronica Andreo. useR Conference 2024, Salzburg.
• Geoprocessing with GRASS GIS and Jupyter Notebooks. Webinar by Corey White. U.S. DOI, August 2024, NCSU.
• GRASS Community Meeting Prague 2024

• GEO-OPEN-HACK-2024: Big Geospatial Data Hackathon with Open Infrastructure and Tools training course delivered (announcement). The GRASS material has been developed as part of the NSF grant.

• Planning GRASS Dev summit 2025 at NCSU

• Planning AGU (4 abstract submitted), AGU booth for GRASS GIS reserved and payed

• Submitted workshop, talk and keynote to FOSS4G 2024 - Brazil.

CMake transition

Ongoing work on CMake compilation was partially covered by POSE.

Code Quality and Security

• Code quality improvements in Python code:
  • Flake8 W605 Invalid escape sequence #3763
  • Flake8 E741 Do not use variables named 'I', 'O', or 'l' in wxGUI code #3926
  • Flake8 E741 for the rest of code base #3928
  • grass.temporal: Flake8 F841 Local variable assigned to but never used #4200
  • grass.pygrass: Remove unused arg in ctypes.CFUNCTYPE #4113
  • t.rast.to.vect: Fix passing column parameter to r.to.vect, fix flake8 F841 #4206

• Code quality and security improvements in C and C++ code are in 36 PRs with fixes for more than 25 tools, specifically in:
  • raster: r.path, r.out.mpeg, r.out.png, r.in.xyz, r.terraflow, r.statistics, r.spreadpath, r.sim.water, r.sim.erosion, r.object.geometry, r.coin
  • imagery: i.atcorr, i.aster.toar, i.ortho.photo, i.landsat.acca, i.segment, i.smap
  • display: d.linegraph, d.legend.vect, d.histogram, d.labels
  • raster 3D: r3.in.v5d, r3.info, r3.mapcalc
  • other: lib/gis, lib/vector/diglib, g.findfile, g.setproj, ps.map, lib/cdhc

Changes by author: ShubhamDesai, Makiko Shukunobe, ymdatta

Several reviewers, not supported by the NSF POSE project, played crucial role not only in reviewing, but also in determining the right fixes.

Fixed issues were reported by Flake8, Coverity Scan, and Cppcheck.

Documentation

• Veronica Andreo met some GRASS developers and contributors to follow up on tutorials development.
• 7 new tutorials on time series:
  • Management and visualization
  • Aggregation
  • Algebra
  • Accumulation
  • Gap filling
  • Querying raster time series with vectors
  • Subset, import and export
• Tutorial on how to set up Jupyter and GRASS in Windows
• New repository under OSGeo organization to host the upcoming quarto based website for GRASS tutorials
• PR template created - GitHub Community Standards all checked complete

Communication and outreach

• Creation of 4 news posts in the GRASS website
• Social media:
  • Twitter: 25 new posts
  • Fosstodon: 25 new posts
• New GRASS GIS LinkedIn page at: https://www.linkedin.com/company/grass-gis/
• Requested transition of grass-dev mailing list to discourse.osgeo.org
• Set up GRASS GIS swag shop on Redbubble

Governance

• Roadmap and Governance pages added to the website
• PSC elections:
  • Documented criteria to define people entitled to vote
  • Documented procedure to compile the list of voters
  • Elections announcement sent out

Y2 1st quarter report

Presentations, Workshops, Event Planning

FOSS4GNA 2024 in St. Louis, Missouri:

• Vaclav Petras. State of GRASS GIS. FOSS4G NA St. Louis, September 9-11, 2024 (slides)
• Huidae Cho. Evolution of GRASS GIS. FOSS4G NA St. Louis, September 9-11, 2024 (slides)
• Huidae Cho. An OpenMP Algorithm for Delineating a Large Number of Watersheds. FOSS4G NA St. Louis, September 9-11, 2024
• Corey T. White. Developing Web-Applications with GRASS GIS. Workshop. FOSS4G NA St. Louis, September 9-11, 2024
• Corey T. White. GRASS GIS as a Geospatial Computational Engine. FOSS4G NA St. Louis, September 9-11, 2024

Planning GRASS Dev summit 2025 at NCSU.

Communication and outreach

• Transition of grass-dev mailing list to discourse.osgeo.org finished.

Code Quality and Security

• Python code quality issues addressed in more than 70 PRs, reducing the list of per-file ignores by more than 60%.
• Code quality and security improvements in C and C++ code are in more than 80 PRs.
• In total, these changes improve more than 60 tools, specifically:
  • raster: r.water.outlet, r.watershed, r.mfilter, r.to.vect, r.in.poly, r.carve, r.thin, r.fill.dir, r.clump, r.in.gridatb, r.out.png, r.in.wms, r.viewshed, r.fillnulls, r.in.srtm, r.in.pdal, r.in.xyz
  • vector: v.out.dxf, v.build.polylines, v.external.out, v.net, v.univar, v.in.ascii, v.delaunay, v.hull , v.net.timetable, v.kernel, v.to.db, v.colors, v.rectify, v.reclass, v.generalize, v.transform, v.vol.rst, v.cluster, v.to.3d, v.out.ogr, v.external, v.in.dwg, v.report, v.unpack, v.import
  • imagery: i.topo.corr, i.attcorr, i.ortho.photo, i.segment, i.rectify, i.topo.corr, i.panshrapen
  • display: d.vect, d.mon, d.frame, d.rast.edit
  • other: GUI, grass main executable, g.proj, g.version, db.out.ogr, ps.map, r3.in.v5d, r3.in.v5d, lib/vector/Vlib, lib/ogsf, lib/gis, lib/db, lib/lidar, lib/linkm, grass.temporal, grass.script

Changes by author: ShubhamDesai, ymdatta, Arohan Ajit

Several reviewers, not supported by the NSF POSE project, played crucial role not only in reviewing, but also in determining the right fixes. The NSF POSE project personnel focused on fixing issues reported by Flake8, Coverity Scan, and Cppcheck, however also Ruff and Pylint are utilized in GRASS GIS thanks to other contributors.

A configure file for Cppcheck is now included in the repository highlighting the level of compliance and aiding contributors in local analysis and subsequent implementation of fixes (PR #4766).

Finally, we created a new public GitHub project in the OSGeo GitHub organization for GRASS GIS Security to track and communicate progress on the security front with 147 resolved items and 8 in-progress items at the time of writing.

Documentation, both Internal and External

• To improve indexing of GRASS GIS manual pages, we revised our current sitemap (grass-website/pull/482)), added a updated sitemap to Google Search Console and Bing, fixed robots.txt, and triggered and re-triggered the indexing many times to make Google consume the new content. While before, some searches gave outdated documentation, experimental websites, or unofficial resources only, now the official website and stable documentation is returned as the first search result.
• All older manuals, including version 6.4 and 6.5, are now available (restored) so that search engines can consume them and old links and indices continue to work. They now newly contain the canonical URLs so that search engines understand the structure and don't penalize duplicate content.
• On OSGeo server which hosts GRASS GIS website, we replaced some of the redirects in Apache by local soft links so that URL like grass.osgeo.org/grass-stable stay as is for both humans and machines and don't redirect to specific version (easier to copy-paste from a browser URL bar and easier for search engine to understand what is the page to show).
• We implemented the necessary changes into the cronjobs and did overall updates to enable build of the server-side documentation jobs locally for testing purposes. The most relevant PRs are:
  • Inject canonical URLs into older manual pages (SEO) #1241
  • Fix red new version box injection and script documentation #1173
  • Expand script to run stand-alone #1215
• We advanced the GRASS Markdown Documentation project.
• New and better wording for intro to imagery tools: Add intro text to imageryintro.html #4536
• List of GRASS GIS profiles on different websites is now in an internal PSC document.
• Documentation of all the internal infrastructure, especially that of grass.osgeo.org server, is now in an internal PSC document.
• We painstakingly identified and described all maintenance procedures performed by Markus Neteler in OSGeo SAC capacity for GRASS GIS and as a contributor to GRASS GIS for better work distribution.
• Most of the work on the documentation was done through a contract with Markus Neteler with additional help from Martin Landa for the transition to Markdown.

Mentoring Activities

Mentors were actively working with 3 mentees.

Developer Summit

• Spaces reserved (some since May 2024, other since September 2024)
• Save the date post for grass-dev, interest form, and first round of additional invites.
• Date established earlier in spring 2024 by NSF POSE team after a survey of PSC members and Core Development Team members.
• Address visa needs based on the responses to the interest form.
• Page created: GRASS Developer Summit Raleigh 2025.
• Initial Organizing Committee formed and agenda created.

Y2 2nd quarter report

Presentations, Workshops, Event Planning

GRASS GIS booth at AGU24

AGU Fall Meeting 2024, Washington, DC:

• GRASS GIS booth in the Exhibit Hall with a banner, new flyers, and a Tangible Landscape setup to demonstrate some of the capabilities of GRASS GIS (Monday afternoon - Thursday morning)
• GRASS GIS for Advanced Earth Systems Analysis and Modeling (eLightning) by Vaclav Petras
• Perspectives from an Open Source Geospatial Modeling Platform Maintainer (poster) by Anna Petrasova
• GRASS GIS Is FAIR: Its Evolution Towards Open Science (poster) by Huidae Cho
• GRASS GIS as a Platform for Dissemination of Reusable Geocomputational Research (poster) by Vaclav Petras

Documentation and Tutorials

We finalized and prepared for launch Quarto-based tutorial website editable through GitHub. OSGeo provided a new domain grass-tutorials.osgeo.org. Initial set of tutorials were created by Veronica Andreo when she was part of the NSF POSE team (some tutorials will be finalized and released later). Anna Petrasova did the reviews, curation, posts for external tutorials, and transition from list of tutorials at the main website.

Communication and outreach

Renewed Youtube channel @grass-gis by adding playlists, organizing videos, and updating branding.

Governance

Submitted application for GRASS GIS to join NumFocus.

Roadmap

Mentoring program (fall 2023 - summer 2025)

Free developer-oriented mentoring program to support researchers and software developers to integrate GRASS GIS into their workflows and contribute to GRASS GIS.

Streamline contributor onboarding (spring 2024 - fall 2024)

Includes updating and creating guidelines for code and non-code contributions.

Streamline user onboarding (spring 2024 - fall 2024)

Includes modernizing tool documentation, developing tutorials.

Creating sample datasets for localized tutorials.

Developing online training materials for big data processing.

Training sessions (fall 2023 - summer 2025)

Local training sessions (at universities and conferences) introducing GRASS GIS to students and researchers.

Training program for big data processing delivered online.

Engaging industry partners (January - August 2024)

Identify and engage industry partners to gather feedback and to start industry partnership program.

Conferences

Presence (talks, workshops, networking) at FOSS4GNA 2023, FOSS4G Asia 2023, AGU 2023, CSDMD 2024, FOSS4G Europe 2024, FOSS4G 2024, AGU 2024

Community sprints

Virtual or in-person participation in OSGeo community sprints, GRASS GIS community sprints

Software distribution (January 2024 - December 2024)

Transition to CMake build system

Develop conda package

Simplify maintenance of GRASS integrations with QGIS and R (2024-2025)

Virtual meetings with QGIS and R developers and GRASS GIS developers (spring 2024).

Identify and implement improvements to the integration.

Improve code quality, security, and quality assurance

Code quality (Flake8, Pylint, Cppcheck), first half of 2024.

Security (CodeQL), second half of 2024.

Quality assurance (better pytest integration), second half of 2024.

In-person developer summit (spring/summer 2025)

Hosted by NC State University, Raleigh, NC, USA.

Identify additional challenges for adoption of GRASS GIS as geoprocessing engine.

Long-term planning beyond the POSE project.

Acknowledgements

The funding is provided by the U.S. National Science Foundation, award 2303651.