Global datasets: Difference between revisions

From GRASS-Wiki
Jump to navigation Jump to search
mNo edit summary
mNo edit summary
Line 327: Line 327:


== Climatic data ==
== Climatic data ==
=== WorldClim maps ===
[http://www.worldclim.org/ WorldClim] is a set of global climate layers (climate grids) with a spatial resolution of a square kilometer.
* Load into a Lat/Lon WGS84 location (EPSG:4326)
* Import with {{cmd|r.in.bin}} or {{cmd|r.in.gdal}}
* {{cmd|r.in.gdal}} will read .BIL format, see the {{cmd|r.in.srtm}} shell script for hints
* binary format is 2 byte integer. Multiply by 10 using {{cmd|r.mapcalc}} to convert units. See http://www.worldclim.org/format.htm for more information and the [[MODIS]] help page for example of converting raw to data units.


=== OGC WCS - Albedo example ===
=== OGC WCS - Albedo example ===
Line 365: Line 357:
   anull=-9999 input=snowdas_input.dat output=snowdas
   anull=-9999 input=snowdas_input.dat output=snowdas


=== WorldClim maps ===
[http://www.worldclim.org/ WorldClim] is a set of global climate layers (climate grids) with a spatial resolution of a square kilometer.
* Load into a Lat/Lon WGS84 location (EPSG:4326)
* Import with {{cmd|r.in.bin}} or {{cmd|r.in.gdal}}
* {{cmd|r.in.gdal}} will read .BIL format, see the {{cmd|r.in.srtm}} shell script for hints
* binary format is 2 byte integer. Multiply by 10 using {{cmd|r.mapcalc}} to convert units. See http://www.worldclim.org/format.htm for more information and the [[MODIS]] help page for example of converting raw to data units.


== Population maps ==
== Population maps ==
Line 436: Line 435:
* http://www.geonames.org/data-sources.html
* http://www.geonames.org/data-sources.html
* [http://ckan.net/tag/read/geo Open Knowledge Foundation link collection]
* [http://ckan.net/tag/read/geo Open Knowledge Foundation link collection]
=== European datasets ===
* [http://grass.osgeo.org/wiki/European_datasets European datasets]
* [http://preview.grid.unep.ch/index3.php?preview=data&lang=eng Global Risk Data Platform]
* [http://open-data.europa.eu/open-data/data/ European Commission Opendata Portal]: 5800+ datasets


=== National datasets ===
=== National datasets ===
Line 444: Line 449:
* United States from NOAA/USGSs data portal (FIXME: link?)
* United States from NOAA/USGSs data portal (FIXME: link?)
* [http://geodata.gov.gr/geodata/ Greek Public Geodata] (in Greek)
* [http://geodata.gov.gr/geodata/ Greek Public Geodata] (in Greek)
=== European datasets ===
* [http://grass.osgeo.org/wiki/European_datasets European datasets]
* [http://preview.grid.unep.ch/index3.php?preview=data&lang=eng Global Risk Data Platform]
* [http://open-data.europa.eu/open-data/data/ European Commission Opendata Portal]: 5800+ datasets


=== Various datasets worldwide ===
=== Various datasets worldwide ===

Revision as of 15:12, 18 February 2013

Raster data

Elevation data

ASTER topography (GDEM V1)

Improved ASTER GDEM 1 from 2009:

GDEM global 30m elevation calculated from stereo-pair images collected by the Terra satellite. "This is the most complete, consistent global digital elevation data yet made available to the world." This is a very new dataset, at version 1 (treat as experimental). Accuracy will be improved in forthcoming versions (validation with SRTM, etc.; see assessment here and here).

Tutorial: ASTER topography.

See also: ASTER GDEM 30m quality assessment

ASTER topography (GDEM V2)

Improved ASTER GDEM 2 from 2011:

The ASTER GDEM covers land surfaces between 83°N and 83°S and is comprised of 22,702 tiles. Tiles that contain at least 0.01% land area are included. The ASTER GDEM is distributed as Geographic Tagged Image File Format (GeoTIFF) files with geographic coordinates (latitude, longitude). The data are posted on a 1 arc-second (approximately 30–m at the equator) grid and referenced to the 1984 World Geodetic System (WGS84)/ 1996 Earth Gravitational Model (EGM96) geoid.

Notes: this DEM can be rather well filtered and smoothed with the Sun's denoising algorithm (using GDAL and free / open source program <mdenoise> or simply GRASS add-on r.denoise.

Experiments showed that the best smoothing of ASTER GDEM 2 is reached with such parameters of <mdenoise>:

  • threshold = 0.8
  • iterations = 10-20

Also filtering with r.neighbors by "average" method and window size >=5 is quite useful to remove some noise from DEM.

See also: Validation of the ASTER Global Digital Elevation Model Version 2 over the Conterminous United States

ACE2

The ACE2 Global Digital Elevation Model is available at 3", 30" and 5' spatial resolutions.

Import example: 

 r.in.bin -f input="00N105E_3S.ACE2" output="ACE2_00N105E" bytes=4 \
          order="native" north=15 south=0 east=120 west=105 \
          rows=18000 cols=18000

CleanTOPO2 (DEM)

Import in GRASS: 

 r.in.gdal CleanTOPO2.tif out=cleanTOPO2.tmp -l -o
 g.region rast=cleanTOPO2  -p -g
 # rescale from odd integer values to true world values
 r.rescale cleanTOPO2.tmp out=cleanTOPO2 to=-10701,8248
 r.colors cleanTOPO2_final col=terrain
Rescaled ClearTOPO2 map

EGM2008 Geoid Data (Earth Gravitational Model)

Global 2.5 Minute Geoid Undulations:

Geoid undulations in Trentino, Italy

Verifications of points can be done with the http://geographiclib.sourceforge.net/cgi-bin/GeoidEval

ETOPO (DEM)

The ETOPO datasets provide global topography and bathymetry at 1', 2', and 5' per-cell resolutions.

ETOPO1 (DEM)

  • The cell registered version can be loaded directly into a lat/lon location. GRASS raster data is cell registered (see the GRASS raster semantics page)


  • Special care must be taken with the grid registered version. It can not be loaded directly into a lat/lon location as the parameters found in the .hdr file exceed the limits of polar coordinate space: they have N,S rows which go 1/2 a cell beyond 90 latitude, when considered in the cell registered convention.
So the data needs to have those 90deg N,S rows cropped away, and while we're at it we crop away a redundant overlapping column at 180 longitude. To do this we have to first tell the GIS a little fib during import to squeeze the data into lat/lon space, then crop away the spurious rows and column, then finally reset the resulting map's bounds to its true extent.
 # Import grid registered binary float, fibbing about its true extent
 r.in.bin -f in=etopo1_bed_g.flt out=etopo1_bed_g.raw \
    n=90 s=-90 e=180 w=-180 rows=10801 cols=21601 anull=-9999
 
 # reduce the working region by 1 cell
 g.region rast=etopo1_bed_g.raw
 eval `g.region -g`
 g.region n=n-$nsres s=s+$nsres e=e-$ewres -p
 
 # save smaller raster and remove original
 r.mapcalc "etopo1_bed_g.crop = etopo1_bed_g.raw"
 g.remove etopo1_bed_g.raw

 # re-establish the correct bounds, now that they'll fit
 r.region etopo1_bed_g.crop n=89:59:30N s=89:59:30S w=179:59:30E e=179:59:30E
 g.region rast=etopo1_bed_g.crop

 # check that N,S,E,W and Res are all nice and clean:
 r.info etopo1_bed_g.crop

 # looks good, so accept the results by resetting the map name
 g.rename etopo1_bed_g.crop,etopo1_bed_g

 # set to use appropriate color rules
 r.colors etopo1_bed_g color=etopo2

 # set the 'units' metadata field (for elevation data contained within the map)
 r.support etopo1_bed_g units=meters
  • For the problematic grid registered version, the resulting r.info report should look like:
|   Rows:         10799                                                      |
|   Columns:      21600                                                      |
|   Total Cells:  233258400                                                  |
|        Projection: Latitude-Longitude                                      |
|            N:  89:59:30N    S:  89:59:30S   Res:  0:01                     |
|            E: 179:59:30E    W: 179:59:30E   Res:  0:01                     |
|   Range of data:    min = -10898  max = 8271                               |

(the east and west bounds of the map touch 1/2 a cell west of 180 longitude)

  • For the problematic grid registered version, since the data's grid is 1/2 a cell shifted from nicely rounded 1 arc-minutes (0:01), you'll need to ensure that the mapset's region preserves that alignment after zooming or panning:
g.region align=etopo1_bed_g -p
(or oversample and set the region resolution to 1/2 arc-minutes (0:00:30), which will be four times as slow)

ETOPO2 (DEM)

ETOPO2v2 data download (take for example the ETOPO2v2g_f4_LSB.flt file)

GTOPO30 (DEM)

Note: To avoid that the GTOPO30 data are read incorrectly, you can add a new line "PIXELTYPE SIGNEDINT" in the .HDR to force interpretation of the file as signed rather than unsigned integers. Then the .DEM file can be imported. Finally, e.g. the 'terrain' color table can be assigned to the imported map with r.colors.


Global Multi-resolution Terrain Elevation Data 2010 (GMTED2010)

Import of GMTED2010 tiles in GRASS GIS: 

 r.in.gdal 30N000E_20101117_gmted_mea075.tif out=gmted2010_30N000E_20101117
 r.colors gmted2010_30N000E_20101117 color=elevation
 g.region rast=gmted2010_30N000E_20101117
 r.shaded.relief gmted2010_30N000E_20101117 units=meters
 d.mon x0
 d.rast gmted2010_30N000E_20101117.shade
 d.grid 1 color=red textcolor=red
GMTED2010 example: Trento - Garda Lake - Verona area (Northern Italy)

GEBCO Bathymetric Chart

  • The General Bathymetric Chart of the Oceans (original 1' release 2003, new 1' and 30" releases 2008)
http://www.gebco.net/data_and_products/gridded_bathymetry_data/
http://www.bodc.ac.uk/data/online_delivery/gebco/

r.in.gdal can be used to import the GMT netCDF files directly, or if that doesn't work you can use GMT tools to convert to an old-style native GMT format and import that with r.in.bin.

example: (GEBCO 2003 1' data)
# convert to an old style GMT binary .grd using grdreformat
$ grdreformat 3n24s47w14w.grd 3n24s47w14w_Native.grd=bs

# then import into GRASS,
GRASS> r.in.bin -h -s bytes=2 in=3n24s47w14w_Native.grd out=3n24s47w14w

# and set some nice colors
GRASS> r.colors 3n24s47w14w rules=- << EOF
nv magenta
0% black
-7740 0:0:168
0 84:176:248
0 40:124:0
522 68:148:24
1407 148:228:108
1929 232:228:108
2028 232:228:92
2550 228:160:32
2724 216:116:8
2730 grey
2754 grey
2760 252:252:252
2874 252:252:252
2883 192:192:192
2913 192:192:192
100% 252:252:252
EOF

Global Multi-Resolution Topography (GMRT DEM)

 export `g.region -g`
 wget "http://www.marine-geo.org/cgi-bin/getgridB?west=${w}&east=${e}&south=${s}&north=${n}&resolution=1" -O /tmp/test.grd
 r.in.gdal /tmp/test.grd output=GMRT -o
 rm /tmp/test.grd
  • Note: Downloaded file contains no projection information, but is EPSG:4326 (WGS84 Geographic). The file size is limited, but lower resolution (resolution=2,4,8) data can be downloaded for larger areas.


Smith and Sandwell DEM

SRTM DEM

Space Shuttle Radar Topography Mission

SRTM30plus data DEM

SRTM30plus data consists of 33 files of global topography in the same format as the SRTM30 products distributed by the USGS EROS data center. The grid resolution is 30 second which is roughly one kilometer.

Land data are based on the 1-km averages of topography derived from the USGS SRTM30 grided DEM data product created with data from the NASA Shuttle Radar Topography Mission. GTOPO30 data are used for high latitudes where SRTM data are not available.

Ocean data are based on the Smith and Sandwell global 2-minute grid between latitudes +/- 72 degrees. Higher resolution grids have been added from the LDEO Ridge Multibeam Synthesis Project and the NGDC Coastal Relief Model. Arctic bathymetry is from the International Bathymetric Chart of the Oceans (IBCAO).

All data are derived from public domain sources and these data are also in the public domain.

GRASS 6 script r.in.srtm described in GRASSNews vol. 3 won't work with this dataset (as it was made for the original SRTM HGT files). But you can import SRTM30plus tiles into GRASS this way: 

r.in.bin -sb input=e020n40.Bathmetry.srtm output=e020n40_topex bytes=2 north=40 south=-10 east=60 west=20 r=6000 c=4800
r.colors e020n40_topex rules=etopo2
Source
GRASS Users Mailing List http://lists.osgeo.org/pipermail/grass-user/2005-August/030063.html
Getting as SRTM30plus tiles
ftp://topex.ucsd.edu/pub/srtm30_plus/srtm30/data/
Getting as SRTM30plus huge file
ftp://topex.ucsd.edu/pub/srtm30_plus/topo30/

Imagery

AVHRR

Blue Marble imagery

NASA's Blue Marble is a 500m-8 degree per-cell world wide visual image of the Earth from space, with the clouds removed.

EO-1 imagery

(Earth Observing-1)

  • "Advanced Land Imager (ALI) provides image data from ten spectral bands (band designations). The instrument operates in a pushbroom fashion, with a spatial resolution of 30 meters for the multispectral bands and 10 meters for the panchromatic band."
-- http://eros.usgs.gov/products/satellite/eo1.php
  • On-board Atmospheric Corrections

Global Land Cover Characteristics

USGS et al. generated dataset at 1km resolution. Provides global landcover characteristics.

LANDSAT imagery

Since October 1, 2008 all Landsat 7 ETM+ scenes held in the USGS EROS archive are available for download at no charge.

  • Download via the Glovis online search tool (req. Java)
  • Download via the USGS's EarthExplorer interface

Import Modules

  • r.in.gdal - Main import tool for complete multiband scenes
  • r.in.wms - Download data covering current map region via WMS server
  • r.in.onearth - WMS frontend for NASA's OnEarth Global Landsat Mosaic
  • i.landsat.rgb - Color balancing/enhancement tool

See also

  • Processing tips can be found on the LANDSAT wiki page

Miscellaneous

Data sources

Import Modules

  • The r.in.gdal modules may be used to import data of many formats, including GMT netCDF
  • The r.in.bin module may be used to import raw binary files

MODIS imagery

Natural Earth imagery

Orthoimagery

Pathfinder AVHRR SST imagery

  • see the Pathfinder AVHRR SST wiki page

QuickBird imagery

SeaWiFS imagery

SPOT Vegetation imagery

SPOT Vegetation (1km) global: NDVI data sets

True Marble imagery

  • True Marble: 250m world wide visual image of the Earth from space, with the clouds removed. GeoTIFF (use the r.in.gdal module)

Climatic data

OGC WCS - Albedo example

GRASS imports OGC Web Coverage Service data. Example server (please suggest a better one!) 

 <WCS_GDAL>
 <ServiceURL>http://laits.gmu.edu/cgi-bin/NWGISS/NWGISS?</ServiceURL>
 <CoverageName>AUTUMN.hdf</CoverageName>
 <Timeout>90</Timeout>
 <Resample>nearest</Resample>
 </WCS_GDAL>

Save this as albedo.xml. Import into a LatLong WGS84 location: 

 r.in.gdal albedo.xml out=albedo

Unfortunately this server sends out the map shifted by 0.5 pixel. This requires a fix to the map boundary coordinates: 

 r.region albedo n=90 s=-90 w=-180 e=180

Now apply color table and look at the map: 

 r.colors albedo color=byr
 d.mon x0
 d.rast albedo

SNODAS maps

Snow Data Assimilation System data that support hydrological modeling and analysis. First download the data, and untar them (once for each month, and once for each day), and you should get pairs of “.dat” and “.Hdr” files. The data files are stored in flat 16-bit binary format, so assuming that “snowdas_in.dat” is the name of the input file, at the GRASS prompt: 

  r.in.bin -bs bytes=2 rows=3351 cols=6935 north=52.874583333332339 \
  south=24.949583333333454 east=-66.942083333334011 west=-124.733749999998366 \
  anull=-9999 input=snowdas_input.dat output=snowdas

WorldClim maps

WorldClim is a set of global climate layers (climate grids) with a spatial resolution of a square kilometer.

Population maps

Gridded Population of the World

Import with r.in.gdal, assign population color table with r.colors

Topographic maps

Soviet topographic maps

Vector data

CDC Geographic Boundary and Public Health Maps

Global Administrative Areas

  • GADM is a database of the location of the world's administrative areas (boundaries) available in shapefiles.
http://gadm.org (extracted by country here)
  • World Borders Dataset including ISO 3166-1 Country codes available in shapefiles.
http://thematicmapping.org/downloads/world_borders.php
  • Free GIS data from Mapping Hacks
http://mappinghacks.com/data/

GSHHS World Coastline

GSHHS is a high resolution shoreline dataset. It is derived from data in the public domain and licensed as GPL. The shorelines are constructed entirely from hierarchically arranged closed polygons. It is closely linked to the GMT project.

For GRASS 6 you can download 1:250,000 shoreline data from NOAA's site in Mapgen format, which can be imported with the v.in.mapgen module.

OpenStreetMap

See the OpenStreetMap wiki page.

SALB

Second Administrative Level Boundaries: "The SALB dataset is a global digital dataset consisting of digital maps and codes that can be downloaded on a country by country basis."

VMap0

1:1 million vector data. Formerly known as Digital Chart of the World

Check the Wikipedia page on VMAP, see the links at the bottom of that article to shapefile versions of VMAP0 and VMAP1. Those look like the versions that were, several years ago, on a NIMA (predecessor to NGA, and successor to the Defense Mapping Agency that managed the Digital Chart of the World and VMAP project) Website. Many GRASS users may prefer the shapefiles to the original Vector Product Format data.

See also

European datasets

National datasets

Various datasets worldwide

WMS servers

Retrieved from "https://grasswiki.osgeo.org/w/index.php?title=Global_datasets&oldid=17931"