GSoC 2026 Add Spatio-Temporal dataset support to datacatalog in GUI

From GRASS-Wiki
Jump to navigation Jump to search
Student Name Saket Kumar Mall
Organization NumFOCUS
Mentors Name Anna Petrasova, Stefan Blumentrath
GitHub Fork View Repo
LinkedIn Profile View LinkedIn


Abstract

Before this project, the GRASS Data Catalog queried only standard spatial maps and bypassed the temporal database entirely. Space-time datasets (STDS) were therefore not visible in the GUI and could only be managed from the command line. Maps that were already registered in an STDS were listed as ordinary standalone maps, with no indication that they belonged to a dataset.

In this project I extended the Data Catalog to support STDS natively. STRDS, STVDS and STR3DS are now listed inside their respective mapset trees alongside the standard spatial maps, registered maps are shown inside the dataset they belong to, and dedicated context menus let users create, inspect, modify and remove datasets and their registered maps directly from the GUI. Alongside the GUI work I also improved, tested and fixed the core temporal tools that the Data Catalog relies on.

Background

GRASS handles time series through its temporal framework. Maps are registered in space-time datasets, which are stored in a temporal database maintained per mapset. Three dataset types exist:

  • STRDS — space-time raster dataset
  • STR3DS — space-time 3D raster dataset
  • STVDS — space-time vector dataset

These datasets are created and manipulated with the t.* tools (t.create, t.register, t.list, and others). Because the Data Catalog did not query the temporal database, none of this structure was visible in the GUI: users could see the individual maps, but not the datasets they belonged to, and every temporal operation required leaving the GUI for the command line.

Project goals

The goals below come from my accepted project proposal. The status column reflects the state of the work at the end of the coding period.

Goal Status
Show STRDS, STVDS and STR3DS under each mapset in the Data Catalog Implemented
Nest registered maps inside their STDS Implemented
Add context menu operations (create, rename, delete, register/unregister, metadata, export, …) Implemented
Make the Data Catalog search aware of STDS Implemented
Lazy loading for large datasets Superseded — replaced by the temporal database queries described in Loading strategy and performance
Enhancements to core temporal tools (format/JSON output, t.list options, bug fixes) Implemented, see Temporal tool enhancements

Implementation

All Data Catalog work is contained in a single pull request, OSGeo/grass#7608, which I developed as a draft from Week 4 onwards and opened for review in Week 10.

Dataset listing and tree structure

  • STRDS, STVDS and STR3DS are listed per mapset, at the same level as the existing raster, 3D raster and vector map listings. The dataset types are not grouped under a separate node; users distinguish them by their icons.
  • Maps registered in a dataset are shown inside the corresponding dataset node.
  • Maps that are already registered in an STDS are hidden from the regular map listing of the mapset. They remain accessible from inside the dataset node.
  • I kept the existing Data Catalog hierarchy. Restructuring the whole tree was discussed with my mentors and deliberately rejected (see Design decisions).
  • Space-time datasets in the Data Catalog tree
  • Maps registered in a dataset, shown inside the dataset node

    Maps registered in a dataset, shown inside the dataset node

  • Datasets listed alongside the raster, 3D raster and vector maps

    Datasets listed alongside the raster, 3D raster and vector maps

Icons

New icons were added for the three dataset types. Each one reuses the existing raster, vector or 3D raster map icon with a clock drawn on top of it, so the dataset type stays recognisable and the temporal nature is immediately visible.


  • Icons for the three dataset types
  • STRDS listed under a mapset

    STRDS listed under a mapset

  • STVDS listed under a mapset

    STVDS listed under a mapset

  • STR3DS listed under a mapset

    STR3DS listed under a mapset


Loading strategy and performance

Datasets and their registered maps are loaded as follows:

  1. t.connect is run once to determine which mapsets have a temporal database connection.
  2. For those mapsets, the datasets and the maps registered in them are fetched through the grass.temporal.gui_support functions tlist_grouped() and registered_maps_grouped(), which query the temporal database.

registered_maps_grouped() returns a nested dictionary of dataset type → dataset id → registered maps, where each map carries its id, start time, end time and, for relative time, its unit. Maps are ordered by start time, and time stamped maps that are not registered in any dataset are not included. This is what makes it possible to hide already registered maps from the standalone map listing.

The result is one set of queries per mapset instead of one t.*.list call per dataset, which is considerably faster on mapsets holding many datasets.

Two earlier approaches were replaced during development:

  1. Search path manipulation. At the very start, retrieving datasets from all mapsets required temporarily extending the search path with g.mapset, running t.list, and restoring the user's original search path. This was a workaround and was dropped once t.list supported the mapset option (#7631), and later became unnecessary altogether.
  2. Per-dataset listing with lazy loading. Up to Week 7, only the datasets were listed initially and their maps were loaded with t.*.list when a dataset node was expanded. This worked but was slow on larger mapsets and complicated mapset reloading and keeping the tree in sync. The gui_support queries described above made the per-dataset calls unnecessary and simplified the reload logic.

Context menu operations

I added context menus for three nodes.

Mapset node

Action Description
Create temporal dataset Creates a new STDS in the selected mapset.

STDS node

Action Description
Register maps Registers maps in the dataset.
Unregister maps Unregisters maps from the dataset.
Merge dataset Merges datasets; cross-mapset merges are supported when the target mapset is in the search path.
Update metadata Updates the dataset metadata.
Delete dataset only Deletes the dataset.
Delete dataset and maps Deletes the dataset together with the maps registered in it (-fd flags).
Rename dataset Renames the dataset.
Export dataset Exports the dataset.
Show metadata Displays the dataset metadata.
Display temporal extent Shows the temporal extent of the dataset.

Map node inside an STDS

Action Description
Unregister map Unregisters the map from its dataset.
Copy Copies the map.
Copy name Copies the map name.
Display layer Adds the map to the current map display.
Show metadata Displays the map metadata.
  • Context menus
  • Mapset node — create a temporal dataset

    Mapset node — create a temporal dataset

  • STDS node — dataset operations

    STDS node — dataset operations

  • Map registered in an STDS

    Map registered in an STDS

Import options

I added the t.*.import tools to the Data Catalog toolbar under "Select another import option", so temporal import is reachable from the same place as the other import tools.

Search

The Data Catalog search was extended to cover STDS, so datasets are found together with the standard spatial maps.

Auto refresh

STDS entries are refreshed automatically across the GUI when datasets are created or modified through context menu, so the tree stays consistent with the temporal database without a manual reload.

Module dialogs

Context menu actions that run a GRASS tool open the corresponding module dialog on top of the existing GUI, instead of launching a separate module window through the --ui option. This keeps the interaction inside the GUI the user is already working in.

Temporal tool enhancements

Part of this work prepared the temporal tools for the GUI integration, since the Data Catalog depends on them; the rest improved the temporal framework independently.

Tool Change Pull request
t.list Added output format options #7466
t.list Added test files #7482
t.list Added a check on the temporal database connection so that a temporal database is no longer created for mapsets that have none. This first approach was later replaced by a more robust solution. #7580
t.list Added the mapset option, which accepts the current mapset ("."), all mapsets in the search path ("*"), or a comma-separated list of mapsets, together with tests #7631
t.list Added support for listing multiple dataset types in one call (e.g. type=strds,stvds), making the interface consistent with g.list #7731
t.rast.univar Added format options #7495
t.rast.univar Added the range to the output and corrected the cells values #7529
t.rast3d.list Added pytest files #7498
t.rast3d.list Added further checks and fixed the issues they exposed #7527
t.rast3d.list Added the format option and tests #7531
t.rast3d.univar Added format options and tests #7582
t.connect Added format options #7535
t.connect Added pytest coverage #7573
t.vect.db.select Added format options #7530
t.vect.db.select Added pytest files #7574
temporal framework Replaced the use of g.mapset during import with a temporary GISRC environment #7434
temporal test suite Removed needs_solo_run from t.vect.list and fixed typos in the t.vect.list and t.rast.list tests #7636

Design decisions

Data Catalog hierarchy kept unchanged
Changing the overall hierarchical structure of the Data Catalog was discussed with my mentors. After weighing the advantages and disadvantages, we kept the existing structure and built the STDS support on top of it.
Datasets at the same level, distinguished by icons
Instead of collecting the three dataset types under a separate group node, they are listed at the same level and told apart by their icons. This keeps the tree flat and consistent with how the existing map types are presented.
One map, one place in the tree
Maps registered in an STDS are hidden from the regular map listing rather than shown twice, and remain reachable inside their dataset node.

Current state

  • The Data Catalog work described above is complete. It is contained in #7608.
  • The temporal tool enhancements listed above are merged.

Reports

Period Timeline Tasks Pull requests
Community Bonding Period

May 1 – May 24, 2026

  1. Introductory meeting with the GRASS mentors and contributors, followed by a project-specific meeting on the scope of the project
  2. Studied the temporal framework, its documentation and the Data Catalog code base
  3. Set up a local environment with mapsets, temporal databases and registered/unregistered time series maps
  4. Fixed a broken link found while going through the time series tutorials

grass-tutorials#117

Week 1

May 25 – May 31, 2026

  1. Wrapped up open pull requests, including the JSON output rewrite of r.geomorphon
  2. Worked on GUI feature enhancements
  3. Worked on a temporal framework issue related to importing STDS: replaced the use of g.mapset during import with a temporary GISRC environment

#7122, #7434

Week 2

June 1 – June 7, 2026

  1. Added output format options to t.list and t.rast.univar
  2. Added test files for t.list and pytest files for t.rast3d.list
  3. Added further checks and fixes to t.rast3d.list
  4. Fixed pre-existing issues found while working on these modules
  5. Started weekly project meetings with my mentors

#7466, #7482, #7495, #7498, #7527

Week 3

June 8 – June 14, 2026

  1. Continued adding format and JSON output options to temporal tools (t.connect, t.rast3d.list, t.vect.db.select, t.rast.univar), with tests and documentation
  2. Discussed the runtime of the existing temporal modules with my mentors as preparation for the GUI integration
  3. Addressed review feedback on the open pull requests

#7529, #7530, #7531, #7535

Week 4

June 15 – June 21, 2026

  1. Added pytest coverage for t.connect and t.vect.db.select
  2. Implemented an initial check to prevent creating temporal databases for mapsets without a temporal database connection; this approach was later replaced by a more robust solution
  3. Profiled the import time of a module used by nearly every temporal tool, since slow imports would make the GUI feel unresponsive; the difference we saw turned out to be operating-system specific and was fixed by my mentors, so no lazy-import work was needed
  4. Opened the draft pull request for the main project
  5. Discussed how STDS should be represented in the Data Catalog, including the need for dedicated icons; continued with a temporary icon

#7573, #7574, #7580, #7608 (draft)

Week 5

June 22 – June 28, 2026

  1. Implemented the first listing of STDS in the Data Catalog
  2. Retrieved datasets from all mapsets by temporarily extending the search path with g.mapset, running t.list and restoring the original search path; replaced later in the week once t.list supported mapset="*"
  3. Implemented lazy loading of the maps registered in a dataset
  4. Added the first context menu actions: create dataset from the mapset node; rename, delete, merge (including cross-mapset merges when the target mapset is in the search path), show metadata, register and unregister maps; display and unregister for maps inside a dataset
  5. Added mapset options to t.list with tests

#7608, #7631

Week 6

June 29 – July 5, 2026

  1. Refined the structure of the draft implementation in preparation for wider community review
  2. Addressed community feedback on the icons and the presentation of the new STDS nodes
  3. Discussed restructuring the Data Catalog hierarchy with my mentors and decided to keep the existing structure
  4. Improved the temporal test suite
  5. Added format options and tests to t.rast3d.univar

#7582, #7636

Week 7

July 6 – July 12, 2026

  1. Extended the functionality available on STDS nodes and on the maps registered inside them
  2. Refined the context menu actions and the overall interaction with the Data Catalog based on mentor and community feedback
  3. Identified further optimisation opportunities in the implementation
  4. Prioritised the remaining features with my mentors, separating those essential for the initial implementation from later additions
  5. Submitted the midterm evaluation (July 6–10)

#7608

Week 8

July 13 – July 19, 2026

  1. Added support for listing multiple dataset types in a single t.list call, consistent with g.list
  2. Adopted the new gui_support functions introduced by my mentors, replacing the per-dataset t.*.list calls with temporal database queries executed once per mapset
  3. Removed the per-dataset lazy loading approach used until Week 7, which simplified mapset reloading and keeping the tree in sync
  4. Hid maps that are already registered in an STDS from the regular map listing
  5. Updated the context menu actions for mapset, dataset and map nodes
  6. Fixed a watchdog Observer RuntimeError on macOS

#7731, #7677, #7608

Week 9

July 20 – July 26, 2026

  1. Completed the remaining context menu options discussed with my mentors
  2. Addressed review feedback on the multiple dataset type support in t.list

#7608, #7731

Week 10

July 27 – August 2, 2026

  1. Added search and filtering support for STDS in the Data Catalog
  2. Added the t.*.import tools to the Data Catalog toolbar under "Select another import option"
  3. Changed context menu actions to open module dialogs on top of the existing GUI instead of launching separate module windows via --ui
  4. Reviewed the pull request, removed unnecessary code and simplified the implementation; converted the draft into a pull request ready for review
  5. Experimented with integrating g.gui.animation into the GUI as a tab; found a freeze and file descriptor exhaustion on large datasets and opened PR to address this performance issue.

#7608, #7811

Week 11

August 3 – August 9, 2026

  1. Made the g.gui.tplot window resizable
  2. Fixed the freeze and file descriptor exhaustion in g.gui.animation on large datasets

#7820, #7811

Week 12

August 10 – August 16, 2026

  1. Integrated g.gui.animation into the GUI as a tab
  2. Added a database path tooltip to the database nodes in the Data Catalog
  3. Removed the SQLite < 3.33 workaround from the temporal framework
  4. Completed the multiple dataset type support in t.list
  5. Made final changes to the main STDS Data Catalog pull request

#7823, #7824, #7825, #7731, #7608

Additional material:

Future work

  • env support in the temporal framework — allow an environment to be passed to grass.temporal, e.g. tgis.init(env=env). I started this during the coding period and paused it because it affects several parts of the framework.
  • Appending to an existing STDS — modify t.rast.algebra, t.rast.aggregate and similar tools so that their output can be appended to an existing dataset instead of always creating a new one.
  • Temporal import fixes — resolve issues in the temporal import workflow, such as t.rast.import failing when a new project is created and an STRDS is imported into it.
  • Further temporal tool enhancements — continue the work on format/JSON output and tool options started during the project.
  • Usability issues in the temporal workflow — address open issues that make common temporal tasks easier for users.

Pull requests

Data Catalog and GUI

  • #7608 — wxGUI/datacatalog: Add STDS in datacatalog
  • #7677 — wxGUI: Share watchdog Observer to fix FSEvents RuntimeError
  • #7811 — wxGUI/animation: Fix freeze and descriptor exhaustion on large datasets
  • #7820 — wxGUI/tplot: Make the window resizable
  • #7823 — wxGUI/animation: Integrate g.gui.animation into the GUI as a tab
  • #7824 — wxGUI/datacatalog: Add db path tooltip to nodes

Temporal tools and framework

  • #7434 — temporal: Use temporary GISRC env instead of g.mapset for import
  • #7466 — t.list: Add output format options in t.list
  • #7482 — t.list: Add tests files to t.list
  • #7495 — t.rast.univar: Add format options in t.rast.univar
  • #7498 — t.rast3d.list: Add pytest files
  • #7527 — t.rast3d.list: Add checks and some fixes
  • #7529 — t.rast.univar: Add range and correct cells values
  • #7530 — t.vect.db.select: Add format options
  • #7531 — t.rast3d.list: Add format option and tests
  • #7535 — t.connect: Add formats
  • #7573 — t.connect: Add pytest
  • #7574 — t.vect.db.select: Add pytest files
  • #7580 — t.list: Check connection to prevent creating temporal database
  • #7582 — t.rast3d.univar: Add format options and tests
  • #7631 — t.list: Add mapset options for t.list
  • #7636 — temporal: Remove needs_solo_run from t.vect.list and fix typos in t.vect.list & t.rast.list tests
  • #7731 — t.list: Add support for listing multiple dataset types
  • #7825 — temporal: Remove SQLite < 3.33 workaround

Earlier contributions to GRASS

Contributions made before the GSoC coding period.

  • #6719 — docs: Fixed broken tutorials link in README
  • #6745 — mkdocs: Fixed incorrect URL replacement in view source button
  • #6824 — wxGUI: Fixes UTM hemisphere selection in location wizard
  • #6864 — MkDocs: Fixes missing Parameter tabs in documentation
  • #6875 — v.profile: Add JSON output format
  • #6906 — MkDocs: Add title field to frontmatter and remove H1 headings
  • #6929 — v.profile: Fix compilation error when GEOS is not installed
  • #6949 — v.profile: Fixes incorrect quoting in JSON output
  • #6977 — r.kappa: Switch to JSON output using Parson library
  • #6979 — lib: Add wrapper for JSON output handling
  • #6985 — r.kappa: fix wrong reference data order in testcase
  • #6990 — wxGUI/datacatalog: Add database aliases for easier identification
  • #7031 — grass.script: Scan GRASS_ADDON_BASE in get_commands()
  • #7076 — v.db.select: Rewrite JSON output using parson library
  • #7107 — wxGUI/datacatalog: Add copy path option for databases and projects in datacatalog
  • #7122 — r.geomorphon: Rewrite with parson and add json test
  • #7297 — wxGUI/datacatalog: Add EPSG statusbar button and data catalog context menu options
  • grass-tutorials#117 — Tutorials: fix broken link in time series tutorials
Retrieved from "https://grasswiki.osgeo.org/w/index.php?title=GSoC_2026_Add_Spatio-Temporal_dataset_support_to_datacatalog_in_GUI&oldid=29132"