Location and Mapsets: Difference between revisions

From GRASS-Wiki
Jump to navigation Jump to search
(Using GRASS GIS locations in a network)
m (remove hardcoded version specific urls, use latest version)
 
(5 intermediate revisions by the same user not shown)
Line 23: Line 23:
One of the main reasons why GRASS implements the ownership check is to prevent users from trying to create group-writable mapsets and encountering this situation.
One of the main reasons why GRASS implements the ownership check is to prevent users from trying to create group-writable mapsets and encountering this situation.


In GRASS GIS 7 versions, the ownership check can be suppressed by setting one of the specific environmental {{cmd|variables|version=70}}, but this is only intended as workaround for single-user systems with "foreign" filesystems (e.g. FAT, shared directory between virtual machine and hosting computer) which don't have ownership information. It is not safe on a multi-user system.
In GRASS GIS 7 versions, the ownership check can be suppressed by setting one of the specific environmental {{cmd|variables}}, but this is only intended as workaround for single-user systems with "foreign" filesystems (e.g. FAT, shared directory between virtual machine and hosting computer) which don't have ownership information. It is not safe on a multi-user system.


If you are on a single user environment and you need to suppressed ownership check, you might want to put the setting of the variable to your <tt>~/.profile</tt>, <tt>~/.bashrc</tt>, <tt>~/.pam_enviroment</tt> or whatever is the right way of setting variables session-wide on your system.
If you are on a single user environment and you need to suppressed ownership check, you might want to put the setting of the variable to your <tt>~/.profile</tt>, <tt>~/.bashrc</tt>, <tt>~/.pam_enviroment</tt> or whatever is the right way of setting variables session-wide on your system.
== Restrictions to the location/mapset names ==
They may not contain any of the characters:
        / " ' @ , = *
or space, DEL (code 127/0x7F), any control character or any 8-bit character ([https://trac.osgeo.org/grass/browser/grass/trunk/lib/gis/legal_name.c source]).
Also, because map, mapset and location names are used directly as file and/or directory names, they may not contain any character which is prohibited in a filename by the underlying OS. On Windows, this includes:
        \ / : * ? " < > |
(source: try entering any of these characters when renaming a file or directory on Windows.)
The OS may also impose other restrictions (e.g. you typically can't have a file or directory named "." or ".." although the "." character isn't itself prohibited).
=== Mapset name related user questions ===
'''Q:''' I just found out that if you use a numeric mapset (e.g. 2014_123) you have to use single quotes in mapcalc expressions.
'''A:''' Which names are legal and which names can be used in {{cmd|r.mapcalc}} without quoting are different. E.g. a "-" character (minus, hyphen, dash) is legal in a map, mapset or location name, but r.mapcalc requires the name to be quoted, otherwise it will be interpreted as a subtraction.
'''Q:''' ... e.g the following example will raise an error:
  r.mapcalc expression="foo=R1@2014_123"
results in:
  Syntax error, unexpected INTEGER, expecting VARNAME or NAME
'''A:''' An unquoted name consists of a sequence of characters other than space, control characters, or any of:
        ^ # @ , " ' ( ) [ ] + - * / % > < ! = & | ? : ; ~
([https://trac.osgeo.org/grass/browser/grass/trunk/raster/r.mapcalc/mapcalc.l#L97 source]:, definition of "N" (line 97) in r.mapcalc.l]
''However'': it may not have a prefix which is a valid integer or floating-point literal (which is why your example above generates an error), as those take precedence over names.
A quoted name can contain any character other than the quote character itself. Note that quoted names aren't always maps (or mapsets or locations); they can also be used for variables if you feel you have to use a name which isn't valid as an unquoted name.


== Best practice examples ==
== Best practice examples ==
Line 34: Line 74:


=== Variation for '''teams''' ===
=== Variation for '''teams''' ===
* The GRASS GIS software is installed in a network folder. It comes right away with multi-user support.
* The /grassdata/ directory is on a shared network drive (e.g. NFS).
* The /grassdata/ directory is on a shared network drive (e.g. NFS).
* LOCATIONs are named to reflect their coordinate reference system (''latlong_wgs84'', or ''utm32n'', or ...)
* LOCATIONs are named to reflect their coordinate reference system (''latlong_wgs84'', or ''utm32n'', or ...)
Line 41: Line 82:
** ''dataset names'' in case of larger datasets (e.g. MODIS or ECAD time series with > 10,000 maps)
** ''dataset names'' in case of larger datasets (e.g. MODIS or ECAD time series with > 10,000 maps)


=== Using GRASS GIS locations in a network ===
=== Multiuser support: Using GRASS GIS locations in a network ===


'''''Q:''' I am trying to figure out the best way to allow students to work on high resolution imagery in computer labs where their own local computers/sessions are limited in terms of disk space and where the file server is normally read-only for students.''
'''''Q:''' I am trying to figure out the best way to allow students to work on high resolution imagery in computer labs where their own local computers/sessions are limited in terms of disk space and where the file server is normally read-only for students.''
Line 50: Line 91:
* Clients: Linux, Windows, whatever
* Clients: Linux, Windows, whatever


'''Server setup:'''
==== Server setup ====
* Put the data on e.g. <tt>/storage/grassdata/<various locations here/></tt>
* Put the data on e.g. <tt>/storage/grassdata/<various locations here/></tt>
* export the directory tree <tt>/storage/grassdata/</tt> as read-only NFS share
* export the directory tree <tt>/storage/grassdata/</tt> as read-only NFS share
* Remember to put all data into the PERMANENT mapset.
* Remember to put all data into the PERMANENT mapset.


'''Student machine(s):'''
==== Student machine(s) ====
* as sysadmin (root) user, add to /etc/fstab (only one time needed):
* as sysadmin (root) user, add to /etc/fstab (only one time needed):
   mkdir -p /storage/grassdata/
   mkdir -p /storage/grassdata/
   mount via NFS /storage/grassdata/ into /storage/grassdata/
   mount via NFS /storage/grassdata/ into /storage/grassdata/
* for each user (Linux/Unix example):
* for each user
** Linux/Unix example:
   mkdir $USER/grassdata
   mkdir $USER/grassdata
   mkdir $USER/grassdata/location1
   mkdir $USER/grassdata/location1
   ln -s /storage/grassdata//location1/PERMANENT $USER/grassdata/location1/
   ln -s /storage/grassdata//location1/PERMANENT $USER/grassdata/location1/
** Windows example
*** Check http://en.wikipedia.org/wiki/NTFS_reparse_point and http://en.wikipedia.org/wiki/NTFS_symbolic_link
==== Permission management ====
Example for a Linux based permission management, server side (/storage/2/grassdata/ is NFS exported):
* all users belong to group "gis"
* "grassdata" is user + group writable but not world writable (don't enable world writable).
* the ''locations'' are user + group writable but not world writable (don't enable world writable). Each user can generate their own mapsets.
* the ''mapsets'' are user writable (don't enable group+world writable).
* hence the PERMANENT management is done by the owner of PERMANENT. Like this one person keeps PERMANENT up-to-date and clean, no confusion arises.
Example:
<pre>
ls -lart /storage/2/
total 4
drwxrwxr-x  2 neteler gis  72 May  8  2013 raw_data/
drwxrwxr-x  3 neteler gis  55 Dec 10 21:02 _never_backup_me_/
drwxrwxr-x 42 neteler gis 4096 Dec 11 15:08 grassdata/
</pre>
<pre>
ls -lart grassdata/
drwxrwxr-x  13 lucadelu gis      4096 Aug  8 14:45 xy/
drwxrwxr-x  9 neteler  gis      4096 Aug 28 11:46 switzerland_ch1903/
drwxrwxr-x  11 metz    gis      4096 Sep  1 15:43 conus_albers/
drwxrwxr-x  9 neteler  gis      4096 Nov  3 14:40 nc_spm_08_grass7/
drwxrwxr-x 105 neteler  gis      4096 Nov 27 10:30 patUTM32/
drwxrwxr-x 102 neteler  gis      4096 Nov 27 16:18 latlong/
drwxrwxr-x  8 lucadelu gis      4096 Nov 28 17:29 nc_spm_temporal_workshop/
drwxrwxr-x 119 neteler  gis      16384 Dec  8 10:11 eu_laea/
...
</pre>
<pre>
ls -lart eu_laea/
drwxr-xr-x 17 neteler  gis  4096 Oct 20 17:08 PERMANENT/
drwxr-xr-x 13 neteler  gis  4096 Oct 27 16:41 modis_lst_reconstructed_europe_monthly/
drwxr-xr-x 11 neteler  gis  4096 Nov  4 08:52 r_li_test/
drwxr-xr-x 18 neteler  gis  4096 Nov 20 17:29 neteler/
drwxr-xr-x 10 metz    gis  4096 Nov 24 11:28 sunhours_italy/
drwxr-xr-x 14 lucadelu gis  4096 Nov 26 22:23 lucadelu/
drwxr-xr-x 11 sajid    gis  4096 Nov 27 15:08 MYD09CMG/
drwxr-xr-x 18 matteo  gis  4096 Dec  1 10:19 matteo/
drwxr-xr-x  5 garzonc  gis    130 Dec  3 14:40 carol/
drwxr-xr-x 15 metz    gis  4096 Dec  5 15:03 modis_lst_europe/
...
</pre>


'''Usage:'''
==== Usage ====


Now start GRASS with $USER/grassdata/ and location1 and create a new user mapset within it.
Now start GRASS with $USER/grassdata/ and location1 and create a new user mapset within it.

Latest revision as of 10:15, 4 December 2018

A LOCATION is some geographic extent of interest that contains data sets that must all be in the same coordinate system. Every location has a PERMANENT directory which stores some basic information about the whole location, and is a good place to park base files. You can think of a location as a data library for a region of interest.

In a MAPSET you can organize GIS maps thematically or geographically or by project or whatever. Every GRASS session runs under the name of a MAPSET. A MAPSET may be a geographical subset or as large as the parent LOCATION. Technically they are subdirectories under any location. In a networked environment with several users working within the same location, mapsets play a special role. Users may only select (and thus modify) a mapset that they own (i.e., have created). However, data in all mapsets for a given location can be read by anyone (unless prevented by UNIX file permissions). The "PERMANENT" mapset usually contains the read-only base maps like the elevation model, while the other locations are readable and writable by their owners. The "PERMANENT" mapset also contains some information about the location itself that is not found in other mapsets (projection info etc.), thus it must exist in every location.

GRASS' logical database structure (one projection definition per LOCATION) avoids problems such as the ones introduced with the "on-the-fly" reprojection ability (typically problems caused by mixed datums) which is present in most GISes.

In depth explanations

In order to select a mapset as the current mapset (where newly created maps are stored), the user must be the owner of the mapset directory. Write permission of the directory is not sufficient.

A user can access maps in any mapset of the location for which s/he has the necessary permissions, i.e. read permission on files, read and execute permissions on directories.

Maps in a mapset other than the current mapset can be accessed using the map@mapset syntax, or by adding the other mapset(s) to the mapset search path using g.mapsets.

If you want to "take over" an existing location or mapset from a different user, you can change the owner with "chown" if you have root access. If you don't have root access but have write permission on the directory and all subdirectories, you can create a copy (which you will own) then delete the original and rename the copy.

If you have write permission on the mapset directory but lack write permission on any non-empty subdirectories, you can still make a copy (and rename both the original and the copy) but you will not be able to delete the original directory.

Rationale:

One of the main reasons why GRASS implements the ownership check is to prevent users from trying to create group-writable mapsets and encountering this situation.

In GRASS GIS 7 versions, the ownership check can be suppressed by setting one of the specific environmental variables, but this is only intended as workaround for single-user systems with "foreign" filesystems (e.g. FAT, shared directory between virtual machine and hosting computer) which don't have ownership information. It is not safe on a multi-user system.

If you are on a single user environment and you need to suppressed ownership check, you might want to put the setting of the variable to your ~/.profile, ~/.bashrc, ~/.pam_enviroment or whatever is the right way of setting variables session-wide on your system.

Restrictions to the location/mapset names

They may not contain any of the characters:

       / " ' @ , = *

or space, DEL (code 127/0x7F), any control character or any 8-bit character (source).

Also, because map, mapset and location names are used directly as file and/or directory names, they may not contain any character which is prohibited in a filename by the underlying OS. On Windows, this includes:

       \ / : * ? " < > |

(source: try entering any of these characters when renaming a file or directory on Windows.)

The OS may also impose other restrictions (e.g. you typically can't have a file or directory named "." or ".." although the "." character isn't itself prohibited).

Mapset name related user questions

Q: I just found out that if you use a numeric mapset (e.g. 2014_123) you have to use single quotes in mapcalc expressions.

A: Which names are legal and which names can be used in r.mapcalc without quoting are different. E.g. a "-" character (minus, hyphen, dash) is legal in a map, mapset or location name, but r.mapcalc requires the name to be quoted, otherwise it will be interpreted as a subtraction.

Q: ... e.g the following example will raise an error:

 r.mapcalc expression="foo=R1@2014_123"

results in:

 Syntax error, unexpected INTEGER, expecting VARNAME or NAME

A: An unquoted name consists of a sequence of characters other than space, control characters, or any of:

       ^ # @ , " ' ( ) [ ] + - * / % > < ! = & | ? : ; ~

(source:, definition of "N" (line 97) in r.mapcalc.l]

However: it may not have a prefix which is a valid integer or floating-point literal (which is why your example above generates an error), as those take precedence over names.

A quoted name can contain any character other than the quote character itself. Note that quoted names aren't always maps (or mapsets or locations); they can also be used for variables if you feel you have to use a name which isn't valid as an unquoted name.

Best practice examples

Here some examples:

  • A method is to make each LOCATION = a coordinate reference system, then use MAPSETs therein to hold all layers for some project.

Variation for teams

  • The GRASS GIS software is installed in a network folder. It comes right away with multi-user support.
  • The /grassdata/ directory is on a shared network drive (e.g. NFS).
  • LOCATIONs are named to reflect their coordinate reference system (latlong_wgs84, or utm32n, or ...)
  • the MAPSET names are usually
    • project names, or
    • user surnames (= mapset owner, so they have their personal playground), or
    • dataset names in case of larger datasets (e.g. MODIS or ECAD time series with > 10,000 maps)

Multiuser support: Using GRASS GIS locations in a network

Q: I am trying to figure out the best way to allow students to work on high resolution imagery in computer labs where their own local computers/sessions are limited in terms of disk space and where the file server is normally read-only for students.

A: You may set it up like this:

  • Server: Linux or other Unix-like system
  • Clients: Linux, Windows, whatever

Server setup

  • Put the data on e.g. /storage/grassdata/<various locations here/>
  • export the directory tree /storage/grassdata/ as read-only NFS share
  • Remember to put all data into the PERMANENT mapset.

Student machine(s)

  • as sysadmin (root) user, add to /etc/fstab (only one time needed):
 mkdir -p /storage/grassdata/
 mount via NFS /storage/grassdata/ into /storage/grassdata/
  • for each user
    • Linux/Unix example:
 mkdir $USER/grassdata
 mkdir $USER/grassdata/location1
 ln -s /storage/grassdata//location1/PERMANENT $USER/grassdata/location1/

Permission management

Example for a Linux based permission management, server side (/storage/2/grassdata/ is NFS exported):

  • all users belong to group "gis"
  • "grassdata" is user + group writable but not world writable (don't enable world writable).
  • the locations are user + group writable but not world writable (don't enable world writable). Each user can generate their own mapsets.
  • the mapsets are user writable (don't enable group+world writable).
  • hence the PERMANENT management is done by the owner of PERMANENT. Like this one person keeps PERMANENT up-to-date and clean, no confusion arises.

Example:

ls -lart /storage/2/
total 4
drwxrwxr-x  2 neteler gis   72 May  8  2013 raw_data/
drwxrwxr-x  3 neteler gis   55 Dec 10 21:02 _never_backup_me_/
drwxrwxr-x 42 neteler gis 4096 Dec 11 15:08 grassdata/
ls -lart grassdata/
drwxrwxr-x  13 lucadelu gis       4096 Aug  8 14:45 xy/
drwxrwxr-x   9 neteler  gis       4096 Aug 28 11:46 switzerland_ch1903/
drwxrwxr-x  11 metz     gis       4096 Sep  1 15:43 conus_albers/
drwxrwxr-x   9 neteler  gis       4096 Nov  3 14:40 nc_spm_08_grass7/
drwxrwxr-x 105 neteler  gis       4096 Nov 27 10:30 patUTM32/
drwxrwxr-x 102 neteler  gis       4096 Nov 27 16:18 latlong/
drwxrwxr-x   8 lucadelu gis       4096 Nov 28 17:29 nc_spm_temporal_workshop/
drwxrwxr-x 119 neteler  gis      16384 Dec  8 10:11 eu_laea/
...
ls -lart eu_laea/
drwxr-xr-x 17 neteler  gis   4096 Oct 20 17:08 PERMANENT/
drwxr-xr-x 13 neteler  gis   4096 Oct 27 16:41 modis_lst_reconstructed_europe_monthly/
drwxr-xr-x 11 neteler  gis   4096 Nov  4 08:52 r_li_test/
drwxr-xr-x 18 neteler  gis   4096 Nov 20 17:29 neteler/
drwxr-xr-x 10 metz     gis   4096 Nov 24 11:28 sunhours_italy/
drwxr-xr-x 14 lucadelu gis   4096 Nov 26 22:23 lucadelu/
drwxr-xr-x 11 sajid    gis   4096 Nov 27 15:08 MYD09CMG/
drwxr-xr-x 18 matteo   gis   4096 Dec  1 10:19 matteo/
drwxr-xr-x  5 garzonc  gis    130 Dec  3 14:40 carol/
drwxr-xr-x 15 metz     gis   4096 Dec  5 15:03 modis_lst_europe/
...

Usage

Now start GRASS with $USER/grassdata/ and location1 and create a new user mapset within it. That will stay locally while PERMANENT comes from the NFS server in read-only.

See also