Time series development

This page is born out of discussions in Lausanne. A lot of people seem to be interested in having some standardized, documented format for dealing with time series in GRASS, so that we can deal with data (e.g. climate station, water gauges), link with models, etc., without having to come up with a custom solution each time. This could also lead to modules to help with interpolation to deal with missing values, etc.

There was also discussion of possibly setting up a mailing list. Do we want this?

I would vote against just-another-mailing list Neteler 23:51, 18 September 2006 (CEST)

Ideas from MN

• each imported raster map get's automatically "registered" in an SQL table
  • Table structure:
    • map name
    • map creator (optional)
    • time stamp of import
    • time stamp of map production (optional)
    • time range of validity (optional)
• g.list, g.rename etc. tools have a new "where" parameter do search maps in this table
• g.remove will also remove row from SQL table
• SQL commands can be used for search by time stamps

maxi's proposal

• what about a table with time series (sort of header of each time series) and another table for each time serie with a list of associated filename and related timestamp?
  • time series head table structure:
    • series name
    • creator name
    • timestamp of creation
    • range of validity
    • time step
    • ....
  • time serie associated map (table named like the related time serie):
    • timestamp
    • map name
    • timestamp of import

Soerens ideas

• the temporal gis extension of grass should manage:
  • raster, raster3d, vecter maps and groups of raster maps
  • the existing timestamps should be used for registration in a temporal database
  • timestamps support for groups must be added
  • the timestamp should be automatically created if a map is created
  • g.remove, g.rename and g.copy will need to have access to the temporal database
    • every map keeps the info in which temporal database it is registered (additional file or info in header)
  • g.list should list all temporal databases, not the registered maps (t.info should do this)

Temporal region and time frames with timesteps should be supported, like the region and window functionality in grass.

temporal database

• the temporal database in which the maps are registered should provide temporal queries
  • select all maps between 20 jan 2002 nad 13 feb 2004
  • select all maps with a valid range of 3 months
  • select all maps with datum < 13 sep 1999
  • select all maps with a valid range of 10 years and a value range between [20:45]
  • Postgresql is the only Open Source database which supports such queries and suitable temporal Datatypes [[1]] and [[2]]
  • another way is to implement this functionality in a double linked list

timestamps

• the following timestamps should be created in the database
  • a valid timestamp with startime or start- and endtime
  • creation time
  • modification time
• also statistical Informations about the maps (ranges) should be stored in the database

temporal management tools

• the temporal management should be done by specific commands starting with t.*
• tools for queries, register, unregister und extraction of maps from the temporal database have to be implemented, like t.info, t.register, t.unregister and t.extract ...

Example t.register:

GRASS 6.3.cvs > t.register help

Description:

Register groups, raster, raster3d and vector maps into the temporal database

Keywords:

temporal, time

Usage:

t.register [-s] [tgroup=string[,string,...]]
  [trast=string[,string,...]] [trast3d=string[,string,...]]
  [tvect=string[,string,...]] [group=string[,string,...]]
  [rast=string[,string,...]] [rast3d=string[,string,...]]
  [vect=string[,string,...]] [date=timestamp[,timestamp,...]]
  [timestep=timestep[,timestep,...]]

Flags:

 -s   Use the timestep between new maps.

Parameters:

   tgroup   Temporal group database(s) in which the group(s) should be registered
    trast   Temporal raster database(s) in which the raster(s) should be registered
  trast3d   Temporal raster3d database(s) in which the raster3d(s) should be registered
    tvect   Temporal vector database(s) in which the vector(s) should be registered
    group   Group(s) to be registered in the temporal group database
     rast   Raster map(s) to be registered in the temporal raster database
   rast3d   Raster3d map(s) to be registered in the temporal raster3d database
     vect   Vector map(s) to be registered in the temporal vector database
     date   datetime, datetime1/datetime2 for map(s)
 timestep   timestep between the maps

example

register 3 raster maps beginning from 20 jan 2001 with a timestep of 3 month in databse Landsat

• t.register -s trast=Landsat rast=LandsatJan,LandsatApr,LandsatJul data="20 jan 2001" timestep="3 mon"

Table:

1 LandsatJan 20 jan 2001 ... 
2 LandsatApr 20 apr 2001 ...
3 LandsatJul 20 jul 2001 ...

OO Layer

• it should be possible to have data access with spatial-temporal functions:
  • value = g4dDataObject->Get4DValue(x, y, z, timestamp) -- for volume maps
  • value = g4dDataObject->Get4DValue(x, y, timestamp) -- for raster maps

• based on this functionality a 4d animation tool based on VTK should be implemented

Open Issues

• will 'g.list rast' show also rasters belonging to time series?
• how to deal with huge file number in a folder? (very long serie often deals with a huge number of maps)
  • limitation: how many rasters can a folder contain ('fileno' in /etc/security/limits.conf).
  • efficiency: huge number of files will tremendiously slow down each map listing procedures.
• associated color table: only one color table should serve one series (avoid multiple color table for each map)