ISO/INSPIRE Metadata Support
ISO 19115:2003- Geographic information -- Metadata
ISO 19115:2003 defines the scheme required for description of geographic information and services. It provides information about the identification, the extent, the quality, the spatial and temporal scheme, spatial reference, and distribution of digital geographic data.
About wx.metadata
Wx.metadata which was developed during Google Summer of Code 2014 is currently available in GRASS add-ons. Package for creating and editing metadata according to EN ISO 19115, described here, consists of three modules. Modules r.info.iso and v.info.iso are command line based and support conversion from current native metadata management (r.info, v.info) into ISO based metadata. GUI based module - g.gui.metadata offers advanced tools for management of metadata.
Requirements and installation
Requirements
wx.metadata package has dependence on external libraries:
- pycsw package - development version(OGC CSW server implementation written in Python):
git clone https://github.com/geopython/pycsw.git && cd pycsw
pip install -e . && pip install -r requirements-standalone.txt
- owslib package - development version (OGC CSW client implementation written in Python):
git clone git://github.com/geopython/OWSLib.git
cd OWSLib && sudo python setup.py install
- Jinja2 (one of the most used template engines for Python)
git clone git://github.com/mitsuhiko/jinja2.git
cd jinja2 && sudo python setup.py install
Modules and usage
Naming of metadata files and storage
Default location for exported metadata files is metadata directory in the map's mapset. For raster maps, the prefix derived from the current nomenclature is cell, for vector maps vector. File ends with .xml suffix.
For example default metadata file name for vector map "roads" is vector_roads.xml.
Metadata profile
The metadata profile defines attributes, which can be edited in this editor. The editor offers the already mentioned two profiles(Basic and Inspire). Editor also supports loading of customized profiles. See Creation of custom ISO based profile.
Currently there are two build-in ISO profiles available:
- The basic profile is substituted from intersection between items stored in GRASS native metadata format and INSPIRE profile. The intersect (subset) includes all available GRASS metadata. Metadata which cannot be assigned to ISO based attributes are stored in metadata attribute sabstractact.
- The Inspire profile profile fulfills the criteria of INSPIRE profile. In case if is not possible to get the values from native GRASS metadata, these are filled with text string '$NULL'. This rule is applied for both profiles.
- The temporal profileis based on the Grass basic profile and includes additional specific attributes for Temporal framework.
Modules r.info.iso and v.info.iso
Command line based modules implements conversion of native GRASS metadata to ISO based XML metadata files. Identical configuration applies for both modules. Products of these modules differ partly in certain metadata attributes related to differences between raster (r.info) and vector (v.info) maps metadata.
Parameters of modules:
- map - selection of map for metadata creation
- profile - selection of metadata profile (basic or inspire)
- output - path to output XML metadata file. Default = according to naming&storage rules
Module g.gui.metadata
Metadata editor (editor) includes graphical interface for conversion metadata from r.info and v.info to ISO based metadata, editing metadata files and creating metadata templates. In addition, the graphical module allows validating of INSPIRE and Basic profile.
Information about metadata item
Tooltip function helps to fill in the form correctly.
Creation of metadata
Editor offers two editing modes. The first one allows to create metadata from selected GRASS maps from an active location. The second one allows editing of external metadata file.
GRASS metadata map editor
Combo box for selection of ISO profile:
- basic
- inspire
- load custom – allows to load metadata profile from file. For example in case of using predefined template. The intersection of GRASS Basic profile(all available metadata for selected map) and loaded custom profile will proced with this selection.
Video
External metadata editor
This editor mode allows to load own template and xml file for editing. This option does not support any connection with GRASS maps metadata.
Video
Defining templates
This function allows to create pattern of metadata profile. In other words it support defining fixed values to template. With using created template, the editor does not initialize texts inputs of defined values. These attributes is able to see only in Tree Browser in editor. Direct link on this function is multiple editing mode (see below).
- If check-box of metadata attribute is checked, the OWSLib object in template will be replaced by value. Unchecked attributes will not change this 'part' of the template.
- The yellow background of text fields indicates metadata attributes, which are not possible get from GRASS map information.
Usage
- If check-box of metadata attribute is checked, the OWSLib object in template will be replaced by value. Unchecked attributes will not change this 'part' of the template.
example: Template result of attribute which is checked
<gmd:dateStamp>
<gco:Date>2013-01-01</gco:Date>
</gmd:dateStamp>
example: Template result of attribute which is unchecked
<gmd:dateStamp>
<gco:Date>{{ md.datestamp }}{# tag="md.datestamp" , name="Metadata date" ,\
ref= "Part B 10.2" , desc= "Date that the metadata was created.",type = "date" ,
example = "2005-03-27" , multi= 0 , group= "Metadata" , multiline= False #}</gco:Date>
</gmd:dateStamp>
Video
Video shows record of simple practice of defining custom template
Editing metadata of multiple map selection
Advantages of this editing mode is in work using of defined template. This option can be used by selection of multiple maps in data catalog.
Validation of metadata profile
On the right side of editor there is a validator of metadata. Currently, validator is able to validate two build-in profiles. For selection of profile “Load custom” is set by default the inspire profile.
Updating GRASS native metadata
The g.gui.metadata module allows updating native GRASS metadata with using modules r.support and v.support. The intersection between ISO based metadata and r|v.support is limited by available parameters for these modules.
Creation of custom ISO based profile
Creation of own custom ISO based profile is with using Jinja template syntax and some addition rules. The main item of template is composite from two braces. Both of them are fully according to syntax of Jinja template. First holds information about statements of loop or condition and also simple Python object. Second braces according to Jinja template syntax is represented by comment {# #}. In this braces additional information for generating graphical interface of editor are stored. In addition also information about individual metadata attributes (name, example, description, multiplicity, etc) is included.
The first braces
Contain object of OWSLib which holds the metadata value. This values can be object which is describing single variable or list of variables. Value of some attribute can be defined many times(according to ISO). This attributes must be defined in block of template with starting statements and ending mark of block. Only allowed loop statement is FOR(see jinja2 manual). The XML based text of block will by duplicated according to number of items in the list.
Initialize of OWSLib objects
Most object are initialized in configuration of file which is stored in wx.metadat/config/init_md.txt example: Default configuration file for the build-in Basic and Inspire profile:
self.md.identification = MD_DataIdentification() self.md.dataquality = DQ_DataQuality() self.md.distribution = MD_Distribution() self.md.identification.extent = EX_Extent() self.md.identification.extent.boundingBox = EX_GeographicBoundingBox()
- on the left side of equation is obligatory prefix 'self.md'. Postfix(name of variable) is open. On right side is OWSLib object
Single element
In case if the value of metadata element is declared only once: example: Identifier
{{ md.identifier }}
example: Lineage
{{ md.dataquality.lineage }}
Loop or Condition statements
Loops and condition are defined in by starting braces and ending braces '{% -%}' . Start of block:
example: for statement
{% for co in md.contact -%}
Editor allows to use Python 'zip' function and 'len' function. Note- in Jinja syntax is Python len(length) function defined specifically.
example: zip
{% for (u,d) in zip(md.identification.uom,md.identification.distance) -%}
example: len
{% if md.identification.denominators|length > 0 -%}
End of block is defined by:
example: if statement
{% endif -%}
example: for statement
{% endfor -%}
The second braces
The second braces(by Jinja syntax comment) contains these parameters:
- tag - just copy of string from first braces
- object – for reading and holding metadata in python objects is used OWSLib library.
Most of object are initialized by configuration file wx.metadata/config/init_md.txt. Some metadata attributes are necessary initialized directly in this(object) parameter. Best example is the metadata contact object(organization,email) which can be initialized for many times.
example: responsible party
{% for co in md.contact -%}{# object="CI_ResponsibleParty()" #}
{{ co.organization }}{# object="CI_ResponsibleParty()" #}
{{ co.email }}{# object="CI_ResponsibleParty()" #}
{{ co.role }}{# object="CI_ResponsibleParty()" #}
{% endfor -%}
- name – free text string describing the name of metadata item
- ref – free additional information. By default should be reference of ISO attributes
- type – data type characteristic. Input text field validator for 'email' and 'date' is implemented.
- desc – free text which should describe or define the characteristic of metadata item
- example – free text with example of the value for metadata item
- group- label of the notebook list(tab) in editor.
- inbox- label of the box which includes attributes from the block of Jinja template. All items from one block of Jinja template must be in same box(same name).
- multi- 0 or 1. In case 0, the item cannot be duplicated= text field widget does not contain button for duplicating itself. Note that only OWSLib list based variables can be used with duplication button.
- inboxmulti- 0 or 1. In case 0, the items in box cannot be duplicated = box with items not dispose with button for cloning self.
- multiline – True = initialize text field with multilines editing. Suitable for longer texts input
example of second braces:
{# tag="md.dataquality.lineage" , name="Lineage" , ref= "Part B 6.1" ,/
desc= "General explanation of the data producers knowledge about the lineage of a dataset."/
,example = "Product 1 scenes correspond to the path/row of the Landsat orbit.." ,/
type ="string" , group= "Quality and Validity" , multiline= True #}
Syntax of Jinja template system and editor custom parser
- both of braces of one metadata element must be on same line
- after '{# ','{%' must be white space. Also before '#}' '-%}'
Examples of template
Video tutorial
Video tutorial shows practice of creation custom ISO profile.