GRASS-Wiki - User contributions [en]

Supercomputing

2018-05-29T13:16:42Z

⚠️NikosA: Link to "European High-Performance Computing Joint Undertaking"

This page aims to

* '''overview''' installations of GRASS GIS in [https://en.wikipedia.org/wiki/Supercomputer supercomputing] (HPC, HTC) environments and related applications
* '''document''' best practices of processing big geospatial data, common mistakes and errors and how to work-around them

= Overview of GRASS GIS in supercomputing environments =

* [https://vs.sav.sk/?lang=en&section=departments&sub=vvt&sub2=services Supercomputer "Aurel"], 4096 CPU cores (Power7 architecture), features GRASS GIS 7.4
* GRASS GIS in JRC's JEODPP, [https://doi.org/10.1016/j.future.2017.11.007 A versatile data-intensive computing platform for information retrieval from big geospatial data]
* [https://projects.ncsu.edu/hpc//Software/Software.php henry2 at NCSU] (Intel Xeon based Linux cluster; GRASS GIS 7.2.0 installed around 2017)
* [https://wiki.ncsa.illinois.edu/pages/viewpage.action?pageId=47294247 ROGER, the CyberGIS supercomputer] at NCSA UIUC (batch compute nodes: 24x, 10 cores, 2.6 GHz, 256 GB RAM, 500 GB of local storage, cluster-wide General Parallel File System (GPFS) 4.5PB; GRASS GIS available alongside GDAL, PDAL, Geotools, and R)

= Processing Practices =

See page [[Parallel GRASS jobs]] for Cluster and Grid computing with parallelized code, Job scheduler, and GRASS on a cluster

= See also =

== Related wiki pages ==

* https://grasswiki.osgeo.org/wiki/Parallel_GRASS_jobs
* https://grasswiki.osgeo.org/wiki/Working_with_GRASS_without_starting_it_explicitly
* https://grasswiki.osgeo.org/wiki/GRASS_and_Shell
* https://grasswiki.osgeo.org/wiki/GRASS_GIS_Performance
* https://grasswiki.osgeo.org/wiki/Large_raster_data_processing
* https://grasswiki.osgeo.org/wiki/Large_vector_data_processing

== Presentations ==

* https://fosdem.org/2018/schedule/event/geo_grass/
* https://archive.fosdem.org/2015/schedule/event/grass_7/
* [http://gfzpublic.gfz-potsdam.de/pubman/item/escidoc:100071:1/component/escidoc:100070/5_GISDAY-2012_loewe_thaler_State_of_the_Cluster_bib.pdf%3Bjsessionid=87D8757B6257885C6 State of GIS at the High Performance Computing Cluster (2012)]

== Publications ==
* Alvioli, M., A. C. Mondini, F. Fiorucci, M. Cardinali & I. Marchesini (2018) Topography-driven satellite imagery analysis for landslide mapping, Geomatics, Natural Hazards and Risk, 9:1, 544-567, DOI: [https://doi.org/10.1080/19475705.2018.1458050 10.1080/19475705.2018.1458050]
* Delucchi, L., Neteler, M. (2011): g.cloud module for GRASS GIS, FOSS4G 2011 Denver, Slides: https://www.slideshare.net/lucadelu/grass-cloud
* Neteler, M. (2008): Building a cluster for GRASS GIS and other software from the OSGeo stack, https://courses.neteler.org/building-a-cluster-for-grass-gis-and-other-software-from-the-osgeo-stack/

Upcoming in 2018:
* [http://www.mdpi.com/si/15134 Special issue "High-Performance Computing in Geoscience and Remote Sensing"], Sensors (ISSN 1424-8220; CODEN: SENSC9)

== Miscellaneous ==

* [http://europa.eu/!qk37Tr The European High-Performance Computing Joint Undertaking - EuroHPC]

Supercomputing

2018-05-23T12:28:44Z

⚠️NikosA: More links collected from related thread in the grass-user mailing list

Supercomputing

2018-05-23T12:13:52Z

⚠️NikosA: Creation of a page for GRASS GIS and supercomputing (HPC, HTC)

Working with external data in GRASS

2018-05-08T09:15:28Z

⚠️NikosA: Updated link to OGR's PostgreSQL / PostGIS driver page

This page explains how to effectively work with external raster and vector data in GRASS 7. See also [[PostGIS]] and [[PostGIS Topology]].
__TOC__
== Raster data ==

External raster data can be linked via {{cmd|r.external}}. List of supported formats can be determined by

{{cmd|r.external}} -f

To link file-based data formats, eg. GeoTiff

{{cmd|r.external}} input=ncrast/urban.tif output=urban

=== Raster data processing workflow when keeping data external ===

External raster maps can directly be linked into GRASS with {{cmd|r.external}} which saves time and disk space. Additionally, there is no more a need to store results in the internal GRASS format - with {{cmd|r.external.out}} the resulting maps are directly written to a GDAL supported format.

'''Preparations'''

GRASS can be used in an automated way by just defining a set of variables. See here for [[GRASS_and_Shell#Setting_the_variables|GRASS and Shell]] settings and [[GRASS_and_Python#Creating_Python_scripts_that_call_GRASS_functionality_from_outside|GRASS and Python]].

'''Data flow example'''

The script content could be looking like this:

<source lang="bash">
# register (rather than import) a GeoTIFF file in GRASS GIS:
r.external input=terra_lst1km20030314.LST_Day.tif output=modis_celsius

# define output directory for files resulting from subsequent calculations:
r.external.out directory=$HOME/gisoutput/ format="GTiff"

# perform calculations (here: extract pixels > 20 deg C)
# store output directly as GeoTIFF file, hence add the .tif extension:
r.mapcalc "warm.tif = if(modis_celsius > 20.0, modis_celsius, null() )"

# cease GDAL output connection and turn back to write standard GRASS raster files:
r.external.out -r

# use the result elsewhere
qgis $HOME/gisoutput/warm.tif
</source>

== Vector data ==

See trac {{twiki|Grass7/VectorLib/OGRInterface|OGR interface}} and {{twiki|Grass7/VectorLib/PostGISEditing| PG interface}} page for development issues.

=== Link external data ===

External vector data can be linked via {{cmd|v.external}} using [http://gdal.org/ogr OGR library] or ''GRASS-PostGIS data driver'' (<tt>format=PostgreSQL</tt>). List of supported formats can be determined by

{{cmd|v.external}} -f

To link ''file-based data formats'', eg. [http://gdal.osgeo.org/ogr/drv_shapefile.html ESRI Shapefile] using OGR library

{{cmd|v.external}} input=ncshape/ layer=railroads

Assuming that <tt>railroads.shp</tt> is located in directory <tt>ncshape</tt>.

To link ''database-based data formats'', eg. [http://gdal.osgeo.org/ogr/drv_pg.html PostGIS] using GRASS-PostGIS data driver

{{cmd|v.external}} input=PG:dbname=pgis_nc -l

PostGIS database <pgis_nc> contains 55 feature table(s):
...
bridges
...

{{cmd|v.external}} input=PG:dbname=pgis_nc layer=bridges output=b

Assuming that PostGIS layer named <tt>bridges</tt> is located in database <tt>pgis_nc</tt>. This layer is linked to GRASS mapset as vector map with name <tt>b</tt>. {{cmd|v.external}} also builds pseudo-topology over simple features which enables GRASS to access linked vector data on level 2. Note that data are stored as simple features, so no full topology support can be build based on this data.

By default PostGIS feature tables are linked using built-in '''GRASS-PostGIS data driver''' - it means that PostGIS geometry data are accessed by GRASS directly without any abstract level (like OGR library). When GRASS is built without PostgreSQL support or environmental variable <tt>GRASS_VECTOR_OGR</tt> exists, then GRASS will access PostGIS data using [http://www.gdal.org/drv_pg.html OGR-PostgreSQL] driver.

=== Direct access to external data ===

External data can be accessed using OGR library via ''virtual'' mapset <tt>OGR</tt>.

{{cmd|v.info}} map=PG:dbname=pgis_nc@OGR layer=bridges

Direct access using GRASS-PostGIS data driver is not currently possible.

=== Create new OGR layers using GRASS modules ===

==== v.extract showcase ====

GRASS 7 also supports write access to the external data using OGR or GRASS-PostGIS data driver. Showcase bellow:

{{cmd|v.external}} input=PG:dbname=pgis_nc layer=bridges output=b

{{cmd|v.out.ascii}} input=b where="cat < 10" --q

375171.4992779|317756.72097616|1
374247.5192779|317487.13697616|2
380230.2292779|316900.97897616|3
379191.4162779|316419.09697616|4
388958.8222779|316332.04697616|5
375875.2662779|316319.89597616|6
376393.5282779|316155.96797616|7
380647.5282779|316022.61797616|8
376739.6982779|315970.62597616|9

{{cmd|v.external.out}} input=PG:dbname=pgis_nc format=PostgreSQL

{{cmd|v.extract}} input=b output=b_9 where="cat < 10"

{{cmd|v.external}} input=PG:dbname=pgis_nc -l

PostGIS database <pgis_nc> contains 56 feature table(s):
b_9
bridges
...

Example of direct access to external data without creating a link (only OGR data driver)

{{cmd|v.out.ascii}} input=PG:dbname=pgis_nc@OGR layer=b_9

375171.4992779|317756.72097616|1
374247.5192779|317487.13697616|2
380230.2292779|316900.97897616|3
379191.4162779|316419.09697616|4
388958.8222779|316332.04697616|5
375875.2662779|316319.89597616|6
376393.5282779|316155.96797616|7
380647.5282779|316022.61797616|8
376739.6982779|315970.62597616|9

==== Create new empty OGR/PostGIS layer ====

Check connection settings:

(for GRASS-OGR data driver)

{{cmd|v.external.out}} -p

dsn: /path/to/shapefiles
format: ESRI Shapefile

(for GRASS-PostGIS data driver)

{{cmd|v.external.out}} -p

conninfo: dbname=pgis_nc
format: PostGIS

New OGR/PostGIS layer can be created using {{cmd|v.edit}}, note that you need to specify feature type for newly created OGR/PostGIS layer (point, line or boundary).

{{cmd|v.edit}} map=pmap tool=create type=point

Check created OGR layer (direct access, OGR data driver only):

{{cmd|v.info}} -t map=PG:dbname=pgis_nc@OGR layer=pmap

...
points=0
...

Check created OGR/PostGIS layer (link):

{{cmd|v.external}} input=PG:dbname=pgis_nc layer=pmap

{{cmd|v.info}} -t map=pmap

Adding new point feature:

cat point.txt

P 1 1
375171.4992779 317756.72097616
1 1

{{cmd|v.edit}} -n map=pmap tool=add input=point.txt
{{cmd|v.info}} -t map=pmap

...
points=1
...

==== v.select speed test ====

Testing data:

* DBF input (attributes only)

{{cmd|g.mapset}} user1
{{cmd|db.connect}} -p

driver:dbf
database:$GISDBASE/$LOCATION_NAME/$MAPSET/dbf/

{{cmd|g.copy}} vect=bridges,b --o
{{cmd|g.copy}} vect=urbanarea,u --o

* SQLite input (attributes only)

{{cmd|g.mapset}} sqlite
{{cmd|db.connect}} -p

driver:sqlite
database:$GISDBASE/$LOCATION_NAME/$MAPSET/sqlite.db

{{cmd|g.copy}} vect=bridges,b --o
{{cmd|g.copy}} vect=urbanarea,u --o

* Export data to PostGIS database (from mapset 'sqlite')

{{cmd|v.out.ogr}} input=b input=PG:dbname=pgis_nc format=PostgreSQL olayer=bridges --o
{{cmd|v.out.ogr}} input=u input=PG:dbname=pgis_nc format=PostgreSQL olayer=urbanarea --o

* PostGIS input (geometry + attributes) - OGR data driver

{{cmd|g.mapset}} pg_ogr
export GRASS_VECTOR_OGR=1
{{cmd|v.external}} input=PG:dbname=pgis_nc layer=bridges output=b --o
{{cmd|v.external}} input=PG:dbname=pgis_nc layer=urbanarea output=u --o

* PostGIS input (geometry + attributes) - GRASS-PostGIS data driver

{{cmd|g.mapset}} pg
{{cmd|v.external}} input=PG:dbname=pgis_nc layer=bridges output=b --o
{{cmd|v.external}} input=PG:dbname=pgis_nc layer=urbanarea output=u --o

Writing output directly using OGR library (mapset 'pg_ogr'):

{{cmd|v.external.out}} -p

dsn: PG:dbname=pgis_nc
format: PostgreSQL

Writing output directly using PostGIS data driver (mapset 'pg'):

{{cmd|v.external.out}} -p

conninfo: dbname=pgis_nc
format: PostgreSQL

* DBF input

time {{cmd|v.select}} ain=b@user1 atype=point bin=u@user1 btype=area out=b_u ope=overlap --o

real 0m6.059s
user 0m4.780s
sys 0m0.588s

* SQLite input

time {{cmd|v.select}} ain=b@sqlite atype=point bin=u@sqlite btype=area out=b_u ope=overlap --o

real 0m2.239s
user 0m1.084s
sys 0m0.524s

* PostGIS input

time {{cmd|v.select}} ain=b atype=point bin=u btype=area out=b_u ope=overlap --o

real 0m20.609s
user 0m7.920s
sys 0m1.644s

''Note:'' Main reason of worse speed is random access used by v.select which is quite costly for OGR layers.

Time consumption for native output:

real 0m1.631s
user 0m0.860s
sys 0m0.768s

=== Digitize OGR layer using wxGUI ===

[[Image:wxgui-digitize-postgis.png|center|thumb|640px|Example of digitizing PostGIS layer (linked via {{cmd|v.external}}) in wxGUI]]

{{YouTube|7XKMGTcje-Q|desc=wxGUI - create and edit new OGR layer}}

{{YouTube|cyy-1HPQR5Y|desc=wxGUI - create and edit new PostGIS layer}}

== Using wxGUI ==

From ''menu''

File -> Link external formats

[[Image:wxgui-link-menu.png|center|frame|Link external data - Layer Manager menu]]

or from ''toolbar'' in Layer Manager.

[[Image:wxgui-link-tooltip.png|center|frame|Link external data - Layer Manager toolbar (1)]]
[[Image:wxgui-link-tool.png|center|frame|Link external data - Layer Manager toolbar (2)]]
[[Image:wxgui-v.external.png|center|thumb|400px|wxGUI {{cmd|v.external}} dialog]]
[[Image:wxgui-v.external.out.png|center|thumb|400px|wxGUI {{cmd|v.external.out}} dialog]]

== See also ==

* [[PostGIS]]
* [[PostGIS Topology]]

[[Category:Vector]]
[[Category:FAQ]]
[[Category:GRASS 7]]

Development

2018-04-10T16:52:37Z

⚠️NikosA: Replaced broken link on Adding modules in QGIS

__TOC__

= GRASS License =

The GRASS GIS project is developed under the terms of the [http://www.gnu.org/copyleft/gpl.html GNU General Public License] (the GPL) [http://grass.ibiblio.org/devel/index.php in the open] by [http://grass.ibiblio.org/community/index.php volunteers] the [http://mapserver.gdf-hannover.de/grassusers/map.phtml world over].

* [[How the Open Source software development model works]]

* [http://en.wikipedia.org/wiki/Gpl Wikipedia entry discussing the GPL]
* The Software Freedom Law Center's [http://www.softwarefreedom.org/resources/2008/foss-primer.html Legal Issues Primer for Open Source and Free Software Projects]

= Resources for Developers =

=== Communication ===
* You can contact GRASS folks in [[How to participate in IRC communication|IRC]]
* [http://lists.osgeo.org/mailman/listinfo/grass-dev Developer mailing list]

=== Documentation ===
* [http://grass.osgeo.org/programming6/ GRASS 6 Programming Manual]
: GRASS libraries are self-documenting using Doxygen header comments.
:* [http://www.stack.nl/~dimitri/doxygen/manual.html Doxygen manual]

* [http://grass.osgeo.org/programming7/ GRASS 7 Programming Manual]

* [[GRASS Programming Howto]] (partially outdated)
* [[Scripting]] tips
* [[Updating GRASS Documentation]] How-To

* [[GIS Concepts]] and how they are implemented in GRASS
* [[Large File Support]] (LFS) implementation
* [[Eclipse]] tips how to use the Eclipse IDE for GRASS development

=== Debugging ===

* [[GRASS Debugging]]

=== Code ===

* [[Tracking]] bugs, patches and feature requests
* [[GRASS AddOns]] - User code contributions (custom scripts, modules, icons, etc)
* [[Compile and Install]] hints
* Lines of source code:
** [https://lists.osgeo.org/pipermail/grass-dev/2004-August/015245.html GRASS SLOC analysis 2004]

==== Code submission procedure ====

* Creating [[Patches]]
* [http://trac.osgeo.org/grass/newticket Submit patches to the trac system]
* [http://trac.osgeo.org/grass/wiki/HowToContribute SVN write access]

==== Code submission standards ====

* [http://svn.osgeo.org/grass/grass/trunk/SUBMITTING C language coding standards]
* [http://svn.osgeo.org/grass/grass/trunk/SUBMITTING_SCRIPTS Shell script coding standards]
* [http://svn.osgeo.org/grass/grass/trunk/SUBMITTING_TCLTK Tcl/Tk script coding standards]
* [http://svn.osgeo.org/grass/grass/trunk/SUBMITTING_PYTHON Python script coding standards]
* [http://svn.osgeo.org/grass/grass/trunk/SUBMITTING_DOCS Documentation coding standards] (manual pages)

===== Explanation of C indentation rules =====
(''see [http://svn.osgeo.org/grass/grass/trunk/SUBMITTING C language coding standards]'')

:{| class="wikitable" border="1"
| -bap||Force blank lines after procedure bodies.
|-
| -bbb||Force blank lines before block comments.
|-
| -bli1||Indent braces 1 space.
|-
| -bls||Put braces on the line after struct declaration lines.
|-
| -br||Put braces on line with if, etc.
|-
| -cbi0||Indent braces after a case label 0 spaces.
|-
| -ci4||Continuation indent of 4 spaces.
|-
| -cli0||Case label indent of 0 spaces.
|-
| -d0||Set indentation of comments not to the right of code to 0 spaces.
|-
| -di0||Put variables in column 0.
|-
| -fc1||Format comments in the first column.
|-
| -hnl||Prefer to break long lines at the position of newlines in the input.
|-
| -i4||Set indentation level to 4 spaces.
|-
| -ip4||Indent parameter types in old-style function definitions by 4 spaces.
|-
| -l80||Set maximum line length for non-comment lines to 80.
|-
| -lc80||Set maximum line length for comment formatting to 80.
|-
| -lp||Line up continued lines at parentheses.
|-
| -nbad||Do not force blank lines after declarations.
|-
| -nbbo||Do not prefer to break long lines before boolean operators.
|-
| -nbc||Do not force newlines after commas in declarations.
|-
| -ncdb||Do not put comment delimiters on blank lines.
|-
| -nce||Do not cuddle } and else.
|-
| -ncs||Do not put a space after cast operators.
|-
| -nfca||Do not format any comments.
|-
| -npcs||Do not put space after the function in function calls.
|-
| -nprs||Do not put a space after every '(' and before every ')'.
|-
| -npsl||Put the type of a procedure on the same line as its name.
|-
| -nsob||Do not swallow optional blank lines.
|-
| -pi4||Specify the extra indentation per open parentheses '(' when a statement is broken.
|-
| -sbi0||Indent braces of a struct, union or enum 0 spaces.
|-
| -sc||Put the `*' character at the left of comments.
|-
| -ss||On one-line for and while statments, force a blank before the semicolon.
|-
| -ts8||Set tab size to 8 spaces.
|}

===== GRASS Makefile writing =====

PGM must be set before including Module.make.

In general, the point at which variables get defined doesn't matter if they are only used in commands, but it matters if they are used in the dependency line. Essentially, dependency lines get expanded at the point they are read, while commands are expanded at the point they are executed.

The list of variables where the order is significant is:

PGM
LIB_NAME
SUBDIRS
CMD_OBJS
LIB_OBJS
LOCAL_HEADERS
EXTRA_HEADERS
DEPENDENCIES

The "documentation" for writing Makefiles boils down to: try to find
something similar in the main GRASS tree and use its Makefile as a
reference; if there isn't one, or if that doesn't work, ask on the
developers' list.

Writing documentation for this will result in either wasted effort
(from describing situations which never actually happen) or inadequate
documentation (from failing to describe situations which actually
happen) or (most likely) both.

A couple of points about wx.metadata specifically:

1. Using "parsubdirs" won't work as, because mdlib (presumably) needs
to be built before any of the modules.

2. Files should not be installed using "cp"; use $(INSTALL) for
anything which should be installed with execute permission or
$(INSTALL_DATA) for anything without it.

3. Directories shouldn't be copied with "cp -r". Rather, the
individual destination files should be listed as pre-requisites, so
that "building" the target "builds" the destination files (typically
via pattern rules).

wx.metadata/Makefile should probably include "templates" and "config"
in $(SUBDIRS). These would have their own Makefiles; e.g. for
templates/Makefile, something like:

include $(MODULE_TOPDIR)/include/Make/Other.make

DSTDIR = $(ETC)/mdlib/templates

SRCFILES := $(wildcard *.xml)
DSTFILES := $(patsubst %.xml,$(DSTDIR)/%.xml,$(SRCFILES))

default: $(DSTFILES)

$(DSTDIR)/%.xml: %.xml:
$(INSTALL_DATA) $< $@

This should allow wx.metadata/Makefile to be a simple "directory"
Makefile (i.e. set SUBDIRS, include Dir.make, "default: subdirs").

==== SVN/CVS ====
* New [http://trac.osgeo.org/grass/wiki/DownloadSource SVN instructions]
* Live GRASS [http://trac.osgeo.org/grass/browser/grass/trunk TracBrowser] and [http://josef.fsv.cvut.cz/cgi-bin/viewcvs.cgi/grass/trunk/?root=grass-osgeo ViewCVS] browseable web interface to the source code repository
* Hints for [[Working with SVN]]
* [http://lists.osgeo.org/pipermail/grass-commit/ SVN commit mailing list archive]

* [[Migration from CVS to SVN]] notes

==== GIT ====

* Some [http://grass.itc.it/pipermail/grass-dev/2007-October/033478.html experiments] with GRASS managed in GIT repository

==== QA ====

* Official document about Quality Control measures: http://grass.osgeo.org/development/code-submission/
* Automated [http://fatra.cnr.ncsu.edu/grassgistests/ GRASS GIS testing suite]
* SVN-commit reports into [irc://irc.freenode.net/grass IRC '#grass' channel] via [http://cia.vc/stats/project/GRASS CIA - The open source informant]
** [http://cia.vc/stats/project/GRASS CIA commit bot] for realtime monitoring along with [http://lists.osgeo.org/mailman/listinfo/grass-commit grass-commit] mailing list (showing the diff's)
* [https://scan.coverity.com/projects/1038 Coverity Scan]
* Historical QA:
** [http://lists.osgeo.org/mailman/listinfo/grass-qa/ Code Quality Control System] Mailing list
** [[Testing GRASS software]] - a test protocol based on the Spearfish sample data set
** External [http://www-pool.math.tu-berlin.de/~soeren/grass/GRASS_TestSuite/html_grass-6.2/ GRASS test suite] (TU Berlin)
*** [http://www-pool.math.tu-berlin.de/~soeren/grass/GRASS_TestSuite/ GRASS Test Suite] a small test suite for GRASS, the current html output is available [http://www-pool.math.tu-berlin.de/~soeren/grass/GRASS_TestSuite/html/ here] and with memory check [http://www-pool.math.tu-berlin.de/~soeren/grass/GRASS_TestSuite/html_memcheck/ here]
** External [http://web.soccerlab.polymtl.ca/project-manager/grass-website/index.html GRASS Quality Assessment and monitoring system] (École Polytechnique de Montréal and FEM-CRI (since 2008; from 2005-2007 ITC-irst was the partner))

==== Code Search and Metrics ====

* [http://www.google.com/codesearch?q=package%3Agrass Google Code Search]

* [http://www.ohloh.net/projects/3666 Ohloh Metrics]

* [http://koders.com Koders Code Search]

= Ongoing and Plans =

=== Overview ===

* Reports:
** [[GRASS Project Update 2007Q1]]
* [[Release Roadmap]] (old [http://grass.ibiblio.org/devel/roadmap.php Development Roadmap] which needs freshening) - please help with testing release candidates
* [[GRASS Module Porting List]] (check here if you don't find a certain command in GRASS 6)
* [[GRASS ToDo List]] (overview of GRASS related community projects, see also [[Release Roadmap]])
* [http://svn.osgeo.org/grass/grass/trunk/doc/vector/TODO GRASS 6 Vector TODO]
* [[Vector network analysis ideas]] (new ideas)
* [http://trac.osgeo.org/grass/wiki/Grass7/NewFeatures List of new features in GRASS 7 (and renamed options)]

=== Works in Progress ===

* Next generation [[GRASS GUI]] development

* [[WinGRASS_Current_Status|Native Windows development]]

* Message [[Development_Specs#Message_standardization|standardization]] and [[GRASS_messages_translation|translation]]

* [[Cairo_driver|Cairo display driver]]

=== Sandbox (ideas section) ===

* [[GRASS 7 ideas collection]]
* [[Image processing]]
* [http://wgbis.ces.iisc.ernet.in/grdss/index.php GRDSS (Geographic Resources Decision Support System)]
* Ideas on a [[replacement raster format]] specification
* add support (at least storage!) of vertical datum and units
* Discussion on [[Development Specs]] for standardized messages
* Discussion on add-on manager repository setup: [[Development GEM|GEM repository]]
* Discussion of support for [[time series in GRASS]], e.g. for linking to data and models
* [[GRASS Metadata Management|Metadata Management]] ideas
* [[GRASS SoC Ideas|GRASS Summer of Code]] ideas

= Linking GRASS to external languages =
* [http://mpa.itc.it/markus/grass63progman/swig/ GRASS-SWIG interface]: generic interface to various languages
* [[GRASS and PHP]]
* [[GRASS and Python]]
* [[GRASS and Shell]]: Starting and running GRASS from a script
* [[GRASS and ABM]]
* [http://www.jgrass.org JGRASS]

= GRASS and QGIS =

* [http://wiki.qgis.org/qgiswiki/BuildingWindowsBinaryOnLinux Building QGIS/GRASS Windows Binary On Linux] (using MinGW)

* [https://qgis.org/en/site/getinvolved/development/addinggrasstools.html Adding GRASS Tools]

* [[QGIS GRASS Cookbook]] - Recipes for common tasks

* [[GRASS-QGIS relevant module list]]

= Commercial development for GRASS =

This is an unsorted (and most probably incomplete) list of paid development for GRASS GIS
* Comune di Trento (Italy): better vector editing, printing
* [http://mpa.itc.it/ ITC-irst]:
** WILMA - [http://wilmaproject.unitn.it/ Trento Wireless Projekt] -> babyGRASS
** MITRIS [http://mpa.itc.it/index.php?option=com_content&task=view&id=104&Itemid=127 traffic-project]
** GRASS 6 vector engine, vector network analysis
* [http://www.faunalia.it/index_en.php Faunalia, Italy]:
** r.li suite
** v.in.gpsbabel
** OGR GRASS vector driver
** MySQL spatial GRASS connector

* [http://www.tekmap.ns.ca/ Tekmap Consulting, Canada]: NVIZ and many more
* [http://navicon.dk/web/normal.php?pageid=92 NAVICON CSE, Denmark: GRASS/Qt]
* [http://www.acsys.it/riade/upload/testo_r/home_link/RIADE_ACS_GRASS_article.pdf Advanced Computer Systems (ACS), Italy]: NVIZ 3D Histograms, Fly-through
* [http://www.orkney.co.jp/english/software/index.html Orkney Inc., Japan]: i18N Infrastructure in GRASS 5.0
* [http://www.geomodel.sk/index-en.htm Geomodel, Slovakia]:
** r.sun
** r.flow etc.
* Lockheed Martin LMCO, USA:
** d.rast.proj
** d.mon.proj (GRASS 5 contrib)

= Related projects =

== With connections to GRASS ==
* [http://www.jgrass.org JGrass] - Java based frontend for GRASS incuding extended hydrological modelling tools
* [http://www.kergis.com/en/index.html KerGIS] - BSD-like licensed fork of GRASS 4.1.5
* [http://gal-framework.no-ip.org/ GAL] - Project to reimplement a GRASS-like GIS using a modern object oriented approach

* [http://pywps.wald.intevation.org pyWPS] - Python Web Processing Service (GRASS on the web)
* [http://proj.maptools.org PROJ.4] - Cartographic Projections Library
* [http://www.gdal.org GDAL] - Geospatial Data Abstraction Library
* [http://www.qgis.org QGIS] - Quantum GIS

* [http://www.osgeo.org OSGeo]- The Open Source Geospatial Foundation
* [http://freegis.org FreeGIS.org] - Interactive information base for the GIS Free Software world
* [http://pkg-grass.alioth.debian.org DebianGIS] - Project coordinating geospatial software for Debian GNU/Linux

* [http://grass.itc.it/statsgrass/grass_geostats.html R statistics] - High powered geostatistical analysis engine
* [http://www.gstat.org/grass.html gstat] - Multivariable geostatistical modelling, prediction and simulation

== Not connected to GRASS ==

* [http://gmt.soest.hawaii.edu GMT] - The Generic Mapping Tools advanced cartography package
* [http://www.saga-gis.uni-goettingen.de SAGA GIS] - A modern programming method GIS for the geosciences
* [http://udig.refractions.net uDig] - User-friendly Desktop Internet GIS (uDig) web portal
* [http://www.gvsig.gva.es/index.php?id=que-es-gvsig&L=2 gvSIG] - Similar in nature to QGIS, but written in Java
* [http://jump-project.org JUMP] - For viewing and manipulating spatial data-sets, using Java
* [http://starspan.casil.ucdavis.edu Starspan] - Mixed raster and vector spatial analysis
* [http://openev.sourceforge.net/ OpenEV] - Raster and vector GIS with good support for image analysis

[[Category:Development]]

Talk:GRASS Community Sprint Bonn 2018

2018-03-25T12:27:24Z

⚠️NikosA: Move last entry under Nikos Alexandris

= Participants and Reports =

== Martin Landa ==

* [https://trac.osgeo.org/grass/timeline?from=Mar+25%2C+2018&daysback=7&authors=martinl&milestone=on&ticket=on&changeset=on&wiki=on&update=Update SVN log] overview
* Show computation region extent settings, see {{trac|3519}}
* wxGUI/datacatalog: zoom to first added map (sync with lmgr behaviour)
* wxGUI/preferences: add new option for random colors (vector)
* Read EPSG codes with PROJ v.5 fails, see {{trac|3514}}
* wxGUI/datacatalog: display layer on double click
* disable to_text_string() in grass.py, see {{trac|3508}}
* rename r.sentinel -> i.sentinel (work in progress), see {{trac|3522}}
* v.random restrict wrong categories assigned, see {{trac|3524}}
* Currently G7:v.random with restrict parameter transfers categories {{trac|3528}}
* wxGUI: set up default map display properties when new display is open
* libgis: improve parser UI description error reporting
* v.to.rast: add support for centroids
* wxGUI/dbmgr: better handling of text values in SQLBuilder
* wxGUI/dbmgr: get sample/all values tuning
* wxGUI: implement widget for SQL WHERE params
[[File:Wxgui-sqlwhere-select.png|500px|thumb|center]]

== Moritz Lennert ==

* [https://trac.osgeo.org/grass/timeline?from=Mar+25%2C+2018&daysback=7&authors=mlennert&milestone=on&ticket=on&changeset=on&wiki=on&update=Update SVN log] overview
* Committed fix for r.texture to avoid overwrite of existing output maps
* Committed fix for i.segment.uspo to avoid zero division error
* Worked on and discussed new startup mockup, created [https://trac.osgeo.org/grass/wiki/wxGUIDevelopment/New_Startup wiki page] for brainstorming
* Use of '_' in i.zc module parameter in i.cutlines to enable parallelization of i.zc. Thanks to Pietro for making this possible in pygrass.
* Some testing for [https://trac.osgeo.org/grass/ticket/3361 #3361] on slowness of v.select with GEOS. Added explanation to man page to clarify that GRASS' "overlap' = GEOS 'intersects'.
* Testing Ondrej' artificial neural networks modules + discussions on further improvements
* Discussion of new website design
* Discussion on move to github.

== Markus Metz ==
* [https://trac.osgeo.org/grass/timeline?from=Mar+25%2C+2018&daysback=7&authors=mmetz&milestone=on&ticket=on&changeset=on&wiki=on&update=Update%20SVN%20log SVN log] overview
* Support of new PROJ 5 API

== Markus Neteler ==

* [https://trac.osgeo.org/grass/timeline?from=Mar+25%2C+2018&daysback=7&authors=neteler&milestone=on&ticket=on&changeset=on&wiki=on&update=Update%20SVN%20log SVN log] overview
* set up [https://docs.google.com/spreadsheets/d/1BgFMiYlLosFjsVUlUCWwssoxiQB_G7gpkVFgG29gZBM/edit?usp=sharing Budget] for this community sprint
* testing of {{AddonCmd|i.nightlights.intercalibration}} with DMSP-OLS data, parser fix (contributed by Anika Bettge) and HTML fixes
* Web: activation of roles in http://staging.grass.osgeo.org/
* svn to git migration discussions
** Following https://trac.osgeo.org/gdal/wiki/rfc71_github_migration with interest
* Ticket {{trac|3527}} (i.atcorr: discrepancy in sensor band numbering in the manual)
* added "How to add keywords in code" submitting rules for [https://trac.osgeo.org/grass/wiki/Submitting/Python#Addingdescriptionandkeywords Python] and [https://trac.osgeo.org/grass/wiki/Submitting/C#Addingdescriptionandkeywords C]
* work together with Vero on https://grasswiki.osgeo.org/wiki/GRASS-Wiki/CMS_migration_plan_2018
* i.atcorr: sync to trunk (updated spectral curves for various satellites; fix band order for Worldview2; addition of Sentinel-2B, improvement of numerical stability {{rev|72557}}

== Veronica Andreo ==

* [https://trac.osgeo.org/grass/timeline?from=Mar+25%2C+2018&daysback=7&authors=veroandreo&milestone=on&ticket=on&changeset=on&wiki=on&update=Update%20SVN%20log SVN log] overview
* tested {{cmd|r.texture}}, see {{trac|3492}}
* committed {{cmd|i.atcorr}} manual example and figure contributed by Zofie Cimburova to perform atmospheric correction on Sentinel-2 bands
* performed general clean-up on i.atcorr manual page
* completed [https://grasswiki.osgeo.org/wiki/SENTINEL Sentinel wiki] (kinda). Editions, corrections and more examples are welcome.
* discussion about new flag and flag behavior for {{cmd|t.remove}} with Luca {{trac|3362}}
* revised and tested old t.* related tickets; closed some old ones as wontfix or worksforme accordingly.
* created patch for {{trac|3264}} with Luca's help
* tested -h flag in {{cmd|t.info}}, see {{trac|3322}}
* added example to {{cmd|t.register}} (contributed by Ondrej Pesek) + correction of typos and minor text edition
* tweeted and RT from GRASS account and posted in FB GRASS GIS group
* completed revision and clean-up of the [https://grasswiki.osgeo.org/wiki/Temporal_data_processing Temporal data processing] wiki
* opened tickets for {{cmd|g.gui.tplot}} (after discussions with Nikos and Luca)
* tested {{trac|3223}} and {{trac|3302}}
* prepared small extra [https://grass.osgeo.org/sampledata/north_carolina/nc_spm_mapset_modis2015_2016_lst.zip mapset] with MODIS LST time series data (2015-2016) for NC location
* created draft roadmap of priorities for the migration of the web site (work together with MarkusN): see [https://grasswiki.osgeo.org/wiki/GRASS-Wiki/CMS_migration_plan_2018 wiki]
* created draft [https://trac.osgeo.org/grass/wiki/Grass7/NewFeatures76 GRASS 7.6. New Features] page to start adding the cool stuff that will be shipped with 76
* added examples for the new features implemented in {{cmd|g.gui.tplot}}
* edited the GRASS GIS page at: https://www.osgeo.org/projects/grass-gis/
* Spanish translations in transifex

== Ondřej Pešek ==

* Working on artificial neural network modules (see [https://github.com/ctu-geoforall-lab-projects/dp-pesek-2018/compare/master@%7B18/3/2018%7D...master github commits] overview)
* g.gui.gmodeler: Support parameterization and GUI generation in python script; export to PyWPS script (see [https://github.com/ctu-yfsg/2017-d-grass-modeler-pywps/compare/master@%7B18/03/2018%7D...master github commits] overview)
* Working on SOS modules (see [https://github.com/pesekon2/GRASS-GIS-SOS-tools/compare/master@%7B17/03/2018%7D...master github commits] overview)

== Luca Delucchi ==

* [https://trac.osgeo.org/grass/timeline?from=Mar+25%2C+2018&daysback=7&authors=lucadelu&milestone=on&ticket=on&changeset=on&wiki=on&update=Update%20SVN%20log SVN log] overview
* {{cmd|g.region}} improved grow option
* renamed r.modis addons to {{AddonCmd|i.modis}}
* working on temporal framework open tickets
* discussion about new flag and flag behavior for {{cmd|t.remove}} with Vero {{trac|3362}}
* created three new modules to work with signature files {{AddonCmd|i.signature.list}}, {{AddonCmd|i.signature.copy}}, {{AddonCmd|i.signature.remove}}
* worked with Markus Metz on {{cmd|i.ortho.transform}} command
* fixed some bug in {{cmd|g.gui.tplot}} {{rev|72485}} {{rev|72489}}, one related to {{trac|3294}}
* {{cmd|g.gui.tplot}} added capabilities to set labels for title, x and y axes {{rev|72523}} {{rev|72526}}
* {{cmd|g.gui.tplot}} added capabilities to export plotted data to CSV file {{rev|72528}} {{rev|72529}} related to {{trac|3533}}
* review GSoC proposals

== Nikos Alexandris ==

* Discussion about GRASS GIS, HPC and distributed file systems with Markus Neteler, Soeren Gebert, Markus Metz. Respective wiki pages to be updated.
* Tests related to https://trac.osgeo.org/grass/ticket/774 available at https://github.com/NikosAlexandris/test_color_space_conversions
* Support for vector input in r.clip, see https://trac.osgeo.org/grass/attachment/ticket/3538/r.clip_patch_for_vector_input

== Stefan Blumentrath ==

* Made his first commit to trunk {{rev|72444}} and fixed {{trac|3322}}
* worked on an AddOn [https://trac.osgeo.org/grass/browser/sandbox/sbl/t.rast.aggregate.update/t.rast.aggregate.update.py t.rast.aggegate.update] for updating aggregated STRDS from t.rast.aggregate (please test)
* wrote a patch for improving performance of v.rast.stats that allows multiple input {{trac|3523}} (testing/feedback very welcome)
* [https://trac.osgeo.org/grass/timeline?from=Mar+25%2C+2018&daysback=7&authors=sbl&milestone=on&ticket=on&changeset=on&wiki=on&update=Update%20SVN%20log SVN log] overview

== Sören Gebbert ==
* [https://trac.osgeo.org/grass/timeline?from=Mar+25%2C+2018&daysback=7&authors=huhabla&milestone=on&ticket=on&changeset=on&wiki=on&update=Update%20SVN%20log SVN log] overview
* temporal algebra bugfixing

== Radek Novotný ==

* Implementation of new --exec funcionalities (--tmp-location,--no-clean), see {{trac|3537}}
* Creating [https://docs.google.com/document/d/1XlxRHvtaiebVIOvlq8G6Q5FuUkFigZ8DlQbaQ-y9keQ/edit?ts=5ab51252 GSoC poroposal]

Talk:GRASS Community Sprint Bonn 2018

2018-03-25T12:26:24Z

⚠️NikosA: Added support for vector input map in r.clip

= Participants and Reports =

== Martin Landa ==

* [https://trac.osgeo.org/grass/timeline?from=Mar+25%2C+2018&daysback=7&authors=martinl&milestone=on&ticket=on&changeset=on&wiki=on&update=Update SVN log] overview
* Show computation region extent settings, see {{trac|3519}}
* wxGUI/datacatalog: zoom to first added map (sync with lmgr behaviour)
* wxGUI/preferences: add new option for random colors (vector)
* Read EPSG codes with PROJ v.5 fails, see {{trac|3514}}
* wxGUI/datacatalog: display layer on double click
* disable to_text_string() in grass.py, see {{trac|3508}}
* rename r.sentinel -> i.sentinel (work in progress), see {{trac|3522}}
* v.random restrict wrong categories assigned, see {{trac|3524}}
* Currently G7:v.random with restrict parameter transfers categories {{trac|3528}}
* wxGUI: set up default map display properties when new display is open
* libgis: improve parser UI description error reporting
* v.to.rast: add support for centroids
* wxGUI/dbmgr: better handling of text values in SQLBuilder
* wxGUI/dbmgr: get sample/all values tuning
* wxGUI: implement widget for SQL WHERE params
[[File:Wxgui-sqlwhere-select.png|500px|thumb|center]]

== Moritz Lennert ==

* [https://trac.osgeo.org/grass/timeline?from=Mar+25%2C+2018&daysback=7&authors=mlennert&milestone=on&ticket=on&changeset=on&wiki=on&update=Update SVN log] overview
* Committed fix for r.texture to avoid overwrite of existing output maps
* Committed fix for i.segment.uspo to avoid zero division error
* Worked on and discussed new startup mockup, created [https://trac.osgeo.org/grass/wiki/wxGUIDevelopment/New_Startup wiki page] for brainstorming
* Use of '_' in i.zc module parameter in i.cutlines to enable parallelization of i.zc. Thanks to Pietro for making this possible in pygrass.
* Some testing for [https://trac.osgeo.org/grass/ticket/3361 #3361] on slowness of v.select with GEOS. Added explanation to man page to clarify that GRASS' "overlap' = GEOS 'intersects'.
* Testing Ondrej' artificial neural networks modules + discussions on further improvements
* Discussion of new website design
* Discussion on move to github.

== Markus Metz ==
* [https://trac.osgeo.org/grass/timeline?from=Mar+25%2C+2018&daysback=7&authors=mmetz&milestone=on&ticket=on&changeset=on&wiki=on&update=Update%20SVN%20log SVN log] overview
* Support of new PROJ 5 API

== Markus Neteler ==

* [https://trac.osgeo.org/grass/timeline?from=Mar+25%2C+2018&daysback=7&authors=neteler&milestone=on&ticket=on&changeset=on&wiki=on&update=Update%20SVN%20log SVN log] overview
* set up [https://docs.google.com/spreadsheets/d/1BgFMiYlLosFjsVUlUCWwssoxiQB_G7gpkVFgG29gZBM/edit?usp=sharing Budget] for this community sprint
* testing of {{AddonCmd|i.nightlights.intercalibration}} with DMSP-OLS data, parser fix (contributed by Anika Bettge) and HTML fixes
* Web: activation of roles in http://staging.grass.osgeo.org/
* svn to git migration discussions
** Following https://trac.osgeo.org/gdal/wiki/rfc71_github_migration with interest
* Ticket {{trac|3527}} (i.atcorr: discrepancy in sensor band numbering in the manual)
* added "How to add keywords in code" submitting rules for [https://trac.osgeo.org/grass/wiki/Submitting/Python#Addingdescriptionandkeywords Python] and [https://trac.osgeo.org/grass/wiki/Submitting/C#Addingdescriptionandkeywords C]
* work together with Vero on https://grasswiki.osgeo.org/wiki/GRASS-Wiki/CMS_migration_plan_2018
* i.atcorr: sync to trunk (updated spectral curves for various satellites; fix band order for Worldview2; addition of Sentinel-2B, improvement of numerical stability {{rev|72557}}

== Veronica Andreo ==

* [https://trac.osgeo.org/grass/timeline?from=Mar+25%2C+2018&daysback=7&authors=veroandreo&milestone=on&ticket=on&changeset=on&wiki=on&update=Update%20SVN%20log SVN log] overview
* tested {{cmd|r.texture}}, see {{trac|3492}}
* committed {{cmd|i.atcorr}} manual example and figure contributed by Zofie Cimburova to perform atmospheric correction on Sentinel-2 bands
* performed general clean-up on i.atcorr manual page
* completed [https://grasswiki.osgeo.org/wiki/SENTINEL Sentinel wiki] (kinda). Editions, corrections and more examples are welcome.
* discussion about new flag and flag behavior for {{cmd|t.remove}} with Luca {{trac|3362}}
* revised and tested old t.* related tickets; closed some old ones as wontfix or worksforme accordingly.
* created patch for {{trac|3264}} with Luca's help
* tested -h flag in {{cmd|t.info}}, see {{trac|3322}}
* added example to {{cmd|t.register}} (contributed by Ondrej Pesek) + correction of typos and minor text edition
* tweeted and RT from GRASS account and posted in FB GRASS GIS group
* completed revision and clean-up of the [https://grasswiki.osgeo.org/wiki/Temporal_data_processing Temporal data processing] wiki
* opened tickets for {{cmd|g.gui.tplot}} (after discussions with Nikos and Luca)
* tested {{trac|3223}} and {{trac|3302}}
* prepared small extra [https://grass.osgeo.org/sampledata/north_carolina/nc_spm_mapset_modis2015_2016_lst.zip mapset] with MODIS LST time series data (2015-2016) for NC location
* created draft roadmap of priorities for the migration of the web site (work together with MarkusN): see [https://grasswiki.osgeo.org/wiki/GRASS-Wiki/CMS_migration_plan_2018 wiki]
* created draft [https://trac.osgeo.org/grass/wiki/Grass7/NewFeatures76 GRASS 7.6. New Features] page to start adding the cool stuff that will be shipped with 76
* added examples for the new features implemented in {{cmd|g.gui.tplot}}
* edited the GRASS GIS page at: https://www.osgeo.org/projects/grass-gis/
* Spanish translations in transifex

== Ondřej Pešek ==

* Working on artificial neural network modules (see [https://github.com/ctu-geoforall-lab-projects/dp-pesek-2018/compare/master@%7B18/3/2018%7D...master github commits] overview)
* g.gui.gmodeler: Support parameterization and GUI generation in python script; export to PyWPS script (see [https://github.com/ctu-yfsg/2017-d-grass-modeler-pywps/compare/master@%7B18/03/2018%7D...master github commits] overview)
* Working on SOS modules (see [https://github.com/pesekon2/GRASS-GIS-SOS-tools/compare/master@%7B17/03/2018%7D...master github commits] overview)

== Luca Delucchi ==

* [https://trac.osgeo.org/grass/timeline?from=Mar+25%2C+2018&daysback=7&authors=lucadelu&milestone=on&ticket=on&changeset=on&wiki=on&update=Update%20SVN%20log SVN log] overview
* {{cmd|g.region}} improved grow option
* renamed r.modis addons to {{AddonCmd|i.modis}}
* working on temporal framework open tickets
* discussion about new flag and flag behavior for {{cmd|t.remove}} with Vero {{trac|3362}}
* created three new modules to work with signature files {{AddonCmd|i.signature.list}}, {{AddonCmd|i.signature.copy}}, {{AddonCmd|i.signature.remove}}
* worked with Markus Metz on {{cmd|i.ortho.transform}} command
* fixed some bug in {{cmd|g.gui.tplot}} {{rev|72485}} {{rev|72489}}, one related to {{trac|3294}}
* {{cmd|g.gui.tplot}} added capabilities to set labels for title, x and y axes {{rev|72523}} {{rev|72526}}
* {{cmd|g.gui.tplot}} added capabilities to export plotted data to CSV file {{rev|72528}} {{rev|72529}} related to {{trac|3533}}
* review GSoC proposals

== Nikos Alexandris ==

* Discussion about GRASS GIS, HPC and distributed file systems with Markus Neteler, Soeren Gebert, Markus Metz. Respective wiki pages to be updated.
* Tests related to https://trac.osgeo.org/grass/ticket/774 available at https://github.com/NikosAlexandris/test_color_space_conversions

== Stefan Blumentrath ==

* Made his first commit to trunk {{rev|72444}} and fixed {{trac|3322}}
* worked on an AddOn [https://trac.osgeo.org/grass/browser/sandbox/sbl/t.rast.aggregate.update/t.rast.aggregate.update.py t.rast.aggegate.update] for updating aggregated STRDS from t.rast.aggregate (please test)
* wrote a patch for improving performance of v.rast.stats that allows multiple input {{trac|3523}} (testing/feedback very welcome)
* [https://trac.osgeo.org/grass/timeline?from=Mar+25%2C+2018&daysback=7&authors=sbl&milestone=on&ticket=on&changeset=on&wiki=on&update=Update%20SVN%20log SVN log] overview

== Sören Gebbert ==
* [https://trac.osgeo.org/grass/timeline?from=Mar+25%2C+2018&daysback=7&authors=huhabla&milestone=on&ticket=on&changeset=on&wiki=on&update=Update%20SVN%20log SVN log] overview
* temporal algebra bugfixing

== Radek Novotný ==

* Implementation of new --exec funcionalities (--tmp-location,--no-clean), see {{trac|3537}}
* Creating [https://docs.google.com/document/d/1XlxRHvtaiebVIOvlq8G6Q5FuUkFigZ8DlQbaQ-y9keQ/edit?ts=5ab51252 GSoC poroposal]
* Support for vector input in r.clip, see https://trac.osgeo.org/grass/attachment/ticket/3538/r.clip_patch_for_vector_input

Talk:GRASS Community Sprint Bonn 2018

2018-03-22T19:02:10Z

⚠️NikosA: Nikos' activities

GRASS GIS Community Sprint Bonn 2018

2018-03-18T14:04:12Z

⚠️NikosA: NikosA: participation details

{{toc|right}}

The '''GRASS GIS community''' will join the general '''OSGeo Code Sprint''' at [http://www.basecamp-bonn.com/ BaseCamp in Bonn], following the annual FOSSGIS conference (German Speaking OSGeo Local Chapter conference in Bonn, Germany)

<center>
[[File:GRASS_GIS_Code_Sprint_2018.png|320px]]
</center>

== Timing, Venue ==

OSGeo Code Sprint: '''March 18 - 25, 2018 at [http://www.basecamp-bonn.com/ BaseCamp, Bonn], Germany'''.

Location: [http://www.openstreetmap.org/way/44335958#map=18/50.70763/7.12316| OpenStreetMap]

== Sponsors ==

Please check more info on the Bonn Code Sprint 2018 [https://wiki.osgeo.org/wiki/OSGeo_Code_Sprint_2018 here].

We also welcome direct '''financial contributions''' to help reducing travelling and accommodation expenses for GRASS developers with far arrival If you are interested to sponsor the GRASS Community Sprint, please read about

:::'''sponsoring the GRASS project''' at '''[http://grass.osgeo.org/donations http://grass.osgeo.org/donations]'''

Note that it is also possible to buy a round of beer for the developers with a quick click using the PayPal "Buy [pizza/beer/...] Now" button [[Image:Btn_donate_SM.gif|center|link=http://grass.osgeo.org/donations/]]

For any questions, please contact [[User:Neteler|Markus Neteler]] <tt><neteler at osgeo.org></tt>. Any surplus at the end of the event will be turned over to the GRASS GIS project.

This GRASS GIS Community Sprint is a great occasion for you to support the development of GRASS. With your contribution you'll enable more developers to meet. The community sprint is an important opportunity for the GRASS developers to discuss and collaboratively resolve bugs, plan the direction for the project and work on new features. Please see below for the more detailed agenda. The developers and contributors are donating their valuable time, so it would be great if in-kind funding can be made available from within the community to cover out-of-pocket expenses. All of the work that takes place at the community sprint will be directly contributed back into the GRASS project to the benefit of everyone who uses it.
This Bonn event is especially interesting since '''important OSGeo projects come together'''!

== Agenda - What we plan to do ==

=== New GRASS GIS startup style ===

* To simplify GRASS GIS startup,
** first the full GUI should be shown and immediately the location dialog popped up _inside_ GRASS GIS
** default SRS: long-lat 4326
* Rationale: don't continue to be different from the rest of the world when not needed
** New solution needs to cover following scenarios:
*** GISDBASE does not exist;
*** GISDBASE exists, but there is no location with EPSG:4326;
*** Selected location/mapset must be propagated down to background CLI process too.

=== Move to Git ===

* Choose among different options: OSGeo Git, Github, GitLab
* Participants are invited to study Git in advance :-)
* Consider moving only GRASS 8 (or 7.7) without history and push history into a separate repository. Rationale – repository checkout will have to contain all branches thus blowing-up checkout size. There is an option to use several git repos "chained" together to have access to full history without having it to live in a single repository. Here's an example: https://archive.org/details/git-history-of-linux https://stackoverflow.com/questions/161928/what-are-git-info-grafts-for

=== New website ===

* <del>Choose among different options</del>: '''Wordpress''' <del>,Sphinx</del>, ... ? A new Wordpress installation is requested in SAC ([https://trac.osgeo.org/osgeo/ticket/2131 ticket]). {{done}}
** https://staging.grass.osgeo.org/ (login with OSGeo-ID)

=== Python interface for external usage ===

* complete the new "grass-session" pip package (https://github.com/zarch/grass-session)
* change packaging to for better support of site-package

=== Integration with QGIS Processing ===

After some intensive discussions the decision of the QGIS devs is to continue with the "Processing" support (see [https://lists.osgeo.org/pipermail/grass-dev/2018-February/087557.html summary email by P Cavallini])

Action item: update the GRASS GIS processing provider. A boilerplate could be the new [https://github.com/alexbruy/processing-saga SAGA processing provider].

=== Port wxGUI to wxPython 4 ===

* Finish porting wxGUI code to the new wxPython 4
* It also includes porting code to Python 3

=== Finish the full Python 3 support ===

* Includes also wxGUI support for wxPython 4
* ...

== Planned participation ==

For organizational reasons, please '''additionally register''' at: https://wiki.osgeo.org/wiki/OSGeo_Code_Sprint_2018#Participants <<--- important!

=== In person ===

Please add your name here or contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail com></tt>:

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|width=100px|'''Arrival'''
|width=100px|'''Departure'''
|'''Topic'''
|width=75px|'''T-Shirt'''
|'''Notes'''
|-
|1
|[[User:Neteler|Markus Neteler]]
|Germany
|18.3.
|25.3.
|Migration to git discussion, GRASS GIS 8, new GRASS GIS startup
|M
|
|-
|2
|[[User:Landa|Martin Landa]]
|Czech Republic
|18.3.
|25.3.
|Migration to git discussion, website discussion, GRASS GIS 8
|L
|
|-
|3
|Stefan Blumentrath
|Germany/Norway
|20.03.
|25.03.
|Flexible
|L
|
|-
|4
|[[User:Veroandreo|Veronica Andreo]]
|Argentina/Netherlands
|18.3
|??.??
|TGRASS, Documentation, website discussion
|S
|
|-
|5
|[[User:Lucadelu|Luca Delucchi]]
|Italy
|18.3
|??.??
|Migration to git discussion, TGRASS, testsuite, website discussion
|L
|
|-
|6
|[[User:Mlennert|Moritz Lennert]]
|Belgium
|18.3
|21.3
|Migration to git discussion, website discussion, GRASS 8, new GRASS GIS startup, work on OBIA addons, ...
|M
|
|-
|7
|Markus Metz
|Germany
|18.3.
|22.3.
|Migration to git discussion, website discussion, GRASS 8, new GRASS GIS startup
|M
|
|-
|8
|Ondřej Pešek
|Czech Republic
|18.3.
|25.3.
|Artificial neural networks in GRASS GIS, g.gui.modeler, SOS tools
|M
|
|-
|9
|[[User:Radeknovotny94|Radek Novotný]]
|Czech Republic
|18.3.
|23.3.
|Integration with QGIS Processing
|L
|
|-
|10
|Anika Bettge
|Germany
|20.3.
|20.3.
|Segmentaion and classification
|
|
|-
|11
|Nikos Alexandris
|Germany
|19.3.
|23.3.
| GRASS workflows + distributed file system (i.e. https://github.com/cern-eos/eos), Publishing r.estimap ([[Talk:GRASS_GIS_Community_Sprint_Autumn_2017#Nikos_Alexandris|started in November 2017]]), Tests suggested in https://trac.osgeo.org/grass/ticket/774
|S/M
|
|-
|#
|...
|
|
|
|
|
|
|-
|}

=== Via IRC chat ===

(IRC is a nice archaic chat system, see [[IRC]])

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|'''Note'''
|-
|1
|Yann Chemin
|Sri Lanka
|
|-
|2
|[[User:hellik|Helmut Kudrnovsky]]
|Austria
|
|-
|3
|[[User:Wenzeslaus|Vaclav Petras]]
|USA
|Available for video call on Monday, Mar 19
|-
|4
|[[User:Annakrat|Anna Petrasova]]
|USA
|Available for video call on Monday, Mar 19
|-
|}

== Individual Preparation ==

* Bring your own computer
* Bring [http://en.wikipedia.org/wiki/Mains_electricity_by_country your power connector adapter] if needed (photo: [https://en.wikipedia.org/wiki/File:Schuko_plug_and_socket.png Germany]). See also http://www.power-plugs-sockets.com/
* Install subversion and the compiler tools, and come with a working GRASS development environment if possible.

== Photos ==

We have to remember to take at least one group photo! :-)

== Reports ==

See [[Talk:GRASS_Community_Sprint_Bonn_2018]]

== FAQ ==

* ''Is the GRASS Community Sprint just a coding event?''
** It is mainly a coding and documentation event. It is a working session for people who are already participants in the GRASS project and/or are committed to improving the GRASS project.
** On demand we can do some presentations of current working GRASS implementation and new upcoming features to spread the idea of Open Source GIS software
* ''Is the GRASS Community Sprint for developers only?''
** No: anybody can help, with testing, checking out bugs and fixes, documentation and more.
* ''Where can I get help and more information about the community sprint?''
** Contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail com></tt>

[[Category: Workshops]]
[[Category: Code Sprint]]
[[Category: 2018]]

Talk:GRASS GIS Community Sprint Autumn 2017

2017-11-15T20:05:21Z

⚠️NikosA: r.estimap in private repository, will open once ready

= Photos of the event =

<gallery widths="300px" heights="225px" perrow="2">
File:Sprint NL autumn 2017 1.jpg
File:Sprint NL autumn 2017 2.jpg
</gallery>

= Summary =

To be added

= Participants and Reports =

Changes in trac: [https://trac.osgeo.org/grass/timeline?from=Nov+12%2C+2017&daysback=3&authors=&milestone=on&ticket=on&changeset=on&wiki=on&update=Update see here]

== Veronica Andreo ==

* Add/enhance descriptions of tasks for Google Code-in (GCI) in shared [https://docs.google.com/spreadsheets/d/1N9jY_VIRchWvZSGa2P5_j0XP5uhasDShDqVDs-oTqC4/edit?usp=sharing spreadsheet]
* Discussions about GRASS GIS GCI tasks with Luca Delucchi
* Create list of core modules and add-ons missing examples and screenshots (by means of: [https://trac.osgeo.org/grass/browser/grass-addons/tools/check_examples_screenshots.py check_examples_screenshots.py])
* Add new and improved modules to [https://trac.osgeo.org/grass/wiki/Grass7/NewFeatures74 NewFeatures74] page and re-write it nicely
* Upload of GRASS GIS flyer in Spanish with help of MN and svg support of Vincent Bain: [https://github.com/OSGeo/osgeo/tree/master/marketing/collateral/one-page-info/grass-gis/es svg], [https://github.com/OSGeo/osgeo/tree/master/marketing/print/one-page-info/es pdf]
* Test r.hants and r.series.lwr after r71682, see {{trac|2790}}
* Closed ticket {{trac|3301}}
* (re)Tweeting as https://twitter.com/GRASSGIS
* Text and screenshot update of lib/init/helptext for trunk and release branch
* Told MN about Add-ons manuals outdated and missing or wrong description in meta-pages (r.li, r.modis, r.green)
* Discussions about the new website
* Add missing height of figures in {{AddonCmd|r.series.lwr}} and {{AddonCmd|r.seasons}} add-ons manual pages

== Markus Neteler ==
* GRASS GIS 8 discussions: improvements in the raster library: tile support
** see: https://trac.osgeo.org/grass/wiki/Grass8Planning
* Assist in production and upload of GRASS GIS flyer in Spanish with VA
* manual: fix missing height HTML tags according to https://trac.osgeo.org/grass/wiki/Submitting/Docs#Images
* Told Nikos about an ultra-fast and cool cmd line search tool (so much faster than "grep"): https://github.com/ggreer/the_silver_searcher
** Search for a function in the source code: <tt>ag grass_to_osr</tt>
** Search for a function in the source code restricted to C files only: <tt>ag --cc grass_to_osr</tt>
** <tt>ag --list-file-types</tt>
* various module manual fixes and ticket cleanup
* Re(Tweeting) to https://twitter.com/GRASSGIS (a bit of marketing :-)
* GRASS GIS 7.4.x release branch has been created in {{rev|71701}}
** Remember that trunk is now called 7.5.svn, i.e. the trunk binary is called "grass75" and no longer "grass73".
** many updates of grass72 --> grass74 in relbranch74
* Web site:
** cronjobs for G74 created and activated
** a few Web pages updated for new upcoming release

== Markus Metz ==

* Some discussions about raster format and grass 8
* made NULL file compression the default
* fixing 7.4-tagged bugs:
** fix vector support in GCP manager
** r.out.gdal: export inf values
** r.out.gdal: +option to add overviews
** work on datum check at data import
* more, see [https://trac.osgeo.org/grass/timeline?from=Nov+12%2C+2017&daysback=3&authors=mmetz&milestone=on&ticket=on&changeset=on&wiki=on&update=Update trac list]

== Moritz Lennert ==

* Added tests to r.object.geometry and fixed a bug noticed through the test ;-)
* Bug triaging
* 7.4 release preparation:
** [https://trac.osgeo.org/grass/wiki/Release/7.4.0-News 7.4.0-News] page
** Writing press release
* Some discussions about raster format and grass 8
* Work on improving r.neighborhoodmatrix
* Discussions on new web site
* (unsuccessful) attempts at debugging g.gui.iclass

== Nikos Alexandris ==

* Minor assistance in adding new/improved modules at [https://trac.osgeo.org/grass/wiki/Grass7/NewFeatures74 NewFeatures74]
* Implemented suggested tests for NULL file compression {{trac|2750}} (see comment [https://trac.osgeo.org/grass/ticket/2750#comment:66 66], see also https://github.com/NikosAlexandris/test_r_compress) | Thank you Moritz, Markus M
* Basic structure for an r.estimap addon, implementation of an algorithm for producing (European) recreation potential and opportunity maps | For the moment, still in a private repository. Will open once... ready!
* I suggest https://getgrav.org/, a flat-file CMS

Talk:GRASS GIS Community Sprint Autumn 2017

2017-11-14T00:17:12Z

⚠️NikosA: Added link to https://getgrav.org/

GRASS GIS Community Sprint Autumn 2017

2017-11-14T00:15:47Z

⚠️NikosA: Added link to grav, a flat file cms (that supports Bootstrap as well)

{{toc|right}}
GRASS people, let's meet! Because until the [[GRASS Community Sprint Bonn 2018|GRASS Community Sprint in March 2018]] there's still a looong way to go and there are topics that have been on the agenda for quite some time already... but especially because it is always so nice and fun to meet :D

'''Important dates:'''

* Fri, 10 Nov 2017, 12:00 - Mon 13 Nov 2017 (open end)

== Purpose and Agenda ==

Discuss and tackle topics such as:

* Specs for '''GRASS GIS 8'''
** please edit here: https://trac.osgeo.org/grass/wiki/Grass8Planning
* What's missing for the '''7.4.0 release'''
** See https://trac.osgeo.org/grass/wiki/Grass7Planning#a7.4.0
* Set up '''new website''' (asap) with responsive design
** Bootstrap?
*** http://www.keycloak.org/
*** https://getgrav.org/
** Wordpress?
*** like https://osgeo.nl/ or the new OSGeo site?
* Development:
** Migration to '''git'''
** Include/invite more core developers
** have more grass-dev meetings
* '''Funding''':
** define concrete goals to be funded
** discuss measures and overall system
** rewrite list of sponsors
* '''GSoC''':
** Review and (possible) integration of GSoC 2017 work
* '''Outreach''':
** offer more GRASS GIS courses, like on osgeo-nl
** more online material, use Sphinx
** generate tons of 1min videos
** Initiatives: one new user a day
** launch a lime survey to know who uses GRASS GIS
** convert manual to Sphinx
** Docker image with tests & benchmarks

== Sponsors ==

If you are interested to sponsor the GRASS Community Sprint, please read about 
    '''sponsoring the GRASS project at [http://grass.osgeo.org/donations/ http://grass.osgeo.org/donations/]''' 
and, if needed, contact [[User:Neteler|Markus Neteler]] <tt><neteler at osgeo.org></tt>. Any surplus at the end of the event will be turned over to the GRASS GIS project.

The GRASS Community Sprint is a great occasion for you to support the development of GRASS. With your contribution you'll enable more developers to meet. The community sprint is an important opportunity for the GRASS developers to discuss and collaboratively resolve bugs, plan the direction for the project and work on new features. The developers and contributors are donating their valuable time, so it would be great if in-kind funding can be made available from within the community to cover out-of-pocket expenses. All of the work that takes place at the community sprint will be directly contributed back into the GRASS project to the benefit of everyone who uses it.

=== Thanks to our sponsors ===

== Venue ==

* Enschede, Vero's home

== Accommodation and Costs ==

* Enschede, Vero's home - self-funded so far (HEY, consider to donate!)



== Planned participation ==

=== In person ===

Please add your name here:

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|width=100px|'''Arrival'''
|width=100px|'''Departure'''
|'''Topic'''
|width=75px|'''T-Shirt'''
|'''Notes'''
|-
|1
|[[User:Veroandreo|Veronica Andreo]]
|Netherlands
|Fri 10
|Mon 13
|Doc, testing, GCI
|S
|
|-
|2
|[[User:Neteler|Markus Neteler]]
|Germany
|Sat 11th
|Sun 12th
|See agenda :)
|M
|Will be travelling back from SFCon Bolzano, hence arriving on Sat
|-
|3
|[https://wiki.osgeo.org/wiki/User:Mmetz Markus Metz]
|Germany
|Fri 10
|Mon 13
|GRASS 7.4, 8
|M
|
|-
|4
|[[User:Mlennert|Moritz Lennert]]
|Belgium
|Fri 10
|Sun 12
|GRASS 7.4, 8, Web site, Docs/Tutorials
|M
|Will have to take a train at around 18:00 on Sunday night
|-
|5
|[[User:NikosA|Nikos Alexandris]]
|Italy
|Fri 10
|Mon 13
|Allarounder, as usual + Checking old posts of mine in mailing lists and file tickets where applicable + Implementing parts of [[chrome-extension://oemmndcbldboiebfnladdacbdfmadadm/http://publications.jrc.ec.europa.eu/repository/bitstream/JRC87585/lb-na-26474-en-n.pdf|ESTIMAP]] as GRASS-GIS addon
|M
|
|-
|6
|
|
|
|
|
|
|
|-
|}

=== Via IRC chat ===

(IRC is a nice archaic chat system, see [[IRC]])

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|-
|1
|[[User:Ychemin|Yann Chemin]]
|France
|-
|2
|[[User:hellik|Helmut Kudrnovsky]]
|Austria
|-
|3
|
|
|}

== Individual Preparation ==

* Bring your own computer
* Bring [http://en.wikipedia.org/wiki/Mains_electricity_by_country your power connector adapter] if needed. See also http://www.power-plugs-sockets.com/
* Install subversion and the compiler tools, and come with a working GRASS development environment if possible.

== Photos ==

Remember to take at least one group photo! :-)

== Reports ==

See [[Talk:GRASS_GIS_Community_Sprint_Autumn_2017]]

== FAQ ==

* ''Is the GRASS Community Sprint just a coding event?''
** It is mainly a coding and documentation event. It is a working session for people who are already participants in the GRASS project and/or are committed to improving the GRASS project.
** On demand we can do some presentations of current working GRASS implementation and new upcoming features to spread the idea of Open Source GIS software
* ''Is the GRASS Community Sprint for developers only?''
** No: anybody can help, with testing, checking out bugs and fixes, documentation and more.
* ''Where can I get help and more information about the community sprint?''
** Contact [[User:Veroandreo|Veronica Andreo]] <tt><veroandreo at gmail com></tt>

[[Category: Workshops]]
[[Category: Code Sprint]]
[[Category: 2017]]

Talk:GRASS GIS Community Sprint Autumn 2017

2017-11-12T22:06:52Z

⚠️NikosA: Minor syntactic correction

= Photos of the event =

To be added

= Summary =

To be added

= Participants and Reports =

Changes in trac: [https://trac.osgeo.org/grass/timeline?from=Nov+12%2C+2017&daysback=3&authors=&milestone=on&ticket=on&changeset=on&wiki=on&update=Update see here]

== Veronica Andreo ==

* Add/enhance descriptions of tasks for Google Code-in (GCI) in shared [https://docs.google.com/spreadsheets/d/1N9jY_VIRchWvZSGa2P5_j0XP5uhasDShDqVDs-oTqC4/edit?usp=sharing spreadsheet]
* Discussions about GRASS GIS GCI tasks with Luca Delucchi
* Create list of core modules and add-ons missing examples and screenshots (by means of: [https://trac.osgeo.org/grass/browser/grass-addons/tools/check_examples_screenshots.py check_examples_screenshots.py])
* Add new and improved modules to [https://trac.osgeo.org/grass/wiki/Grass7/NewFeatures74 NewFeatures74] page and re-write it nicely
* Upload of GRASS GIS flyer in Spanish with help of MN: [https://github.com/OSGeo/osgeo/tree/master/marketing/collateral/one-page-info/grass-gis/es svg], [https://github.com/OSGeo/osgeo/tree/master/marketing/print/one-page-info/es pdf]
* (re)Tweeting as https://twitter.com/GRASSGIS

== Markus Neteler ==
* GRASS GIS 8 discussions: improvements in the raster library: tile support
** see: https://trac.osgeo.org/grass/wiki/Grass8Planning
* Assist in production and upload of GRASS GIS flyer in Spanish with VA
* manual: fix missing height HTML tags according to https://trac.osgeo.org/grass/wiki/Submitting/Docs#Images
* Told Nikos about an ultra-fast and cool cmd line search tool (so much faster than "grep"): https://github.com/ggreer/the_silver_searcher
** Search for a function in the source code: <tt>ag grass_to_osr</tt>
** Search for a function in the source code restricted to C files only: <tt>ag --cc grass_to_osr</tt>
** <tt>ag --list-file-types</tt>
* various module manual fixes and ticket cleanup
* Re(Tweeting) to https://twitter.com/GRASSGIS
* GRASS GIS 7.4.x release branch has been created in {{rev|71701}}
** Remember that trunk is now called 7.5.svn, i.e. the trunk binary is called "grass75" and no longer "grass73".

== Markus Metz ==

* Some discussions about raster format and grass 8
* made NULL file compression the default
* fixing 7.4-tagged bugs:
** fix vector support in GCP manager
** r.out.gdal: export inf values
** r.out.gdal: +option to add overviews
** work on datum check at data import

== Moritz Lennert ==

* Added tests to r.object.geometry and fixed a bug noticed through the test ;-)
* Bug triaging
* 7.4 release preparation:
** [https://trac.osgeo.org/grass/wiki/Release/7.4.0-News 7.4.0-News] page
** Writing press release
* Some discussions about raster format and grass 8
* Work on improving r.neighborhoodmatrix
* Discussions on new web site
* (unsuccessful) attempts at debugging g.gui.iclass

== Nikos Alexandris ==

* Minor assistance in adding new/improved modules at [https://trac.osgeo.org/grass/wiki/Grass7/NewFeatures74 NewFeatures74]
* Implemented suggested tests for NULL file compression {{trac|2750}} (see comment [https://trac.osgeo.org/grass/ticket/2750#comment:66 66], see also https://github.com/NikosAlexandris/test_r_compress) | Thank you Moritz, Markus M
* Basic structure for an r.estimap addon, implementation of an algorithm for producing (European) recreation potential and opportunity maps

Talk:GRASS GIS Community Sprint Autumn 2017

2017-11-12T22:05:48Z

⚠️NikosA: Nikos' activities

= Photos of the event =

To be added

= Summary =

To be added

= Participants and Reports =

Changes in trac: [https://trac.osgeo.org/grass/timeline?from=Nov+12%2C+2017&daysback=3&authors=&milestone=on&ticket=on&changeset=on&wiki=on&update=Update see here]

== Veronica Andreo ==

* Add/enhance descriptions of tasks for Google Code-in (GCI) in shared [https://docs.google.com/spreadsheets/d/1N9jY_VIRchWvZSGa2P5_j0XP5uhasDShDqVDs-oTqC4/edit?usp=sharing spreadsheet]
* Discussions about GRASS GIS GCI tasks with Luca Delucchi
* Create list of core modules and add-ons missing examples and screenshots (by means of: [https://trac.osgeo.org/grass/browser/grass-addons/tools/check_examples_screenshots.py check_examples_screenshots.py])
* Add new and improved modules to [https://trac.osgeo.org/grass/wiki/Grass7/NewFeatures74 NewFeatures74] page and re-write it nicely
* Upload of GRASS GIS flyer in Spanish with help of MN: [https://github.com/OSGeo/osgeo/tree/master/marketing/collateral/one-page-info/grass-gis/es svg], [https://github.com/OSGeo/osgeo/tree/master/marketing/print/one-page-info/es pdf]
* (re)Tweeting as https://twitter.com/GRASSGIS

== Markus Neteler ==
* GRASS GIS 8 discussions: improvements in the raster library: tile support
** see: https://trac.osgeo.org/grass/wiki/Grass8Planning
* Assist in production and upload of GRASS GIS flyer in Spanish with VA
* manual: fix missing height HTML tags according to https://trac.osgeo.org/grass/wiki/Submitting/Docs#Images
* Told Nikos about an ultra-fast and cool cmd line search tool (so much faster than "grep"): https://github.com/ggreer/the_silver_searcher
** Search for a function in the source code: <tt>ag grass_to_osr</tt>
** Search for a function in the source code restricted to C files only: <tt>ag --cc grass_to_osr</tt>
** <tt>ag --list-file-types</tt>
* various module manual fixes and ticket cleanup
* Re(Tweeting) to https://twitter.com/GRASSGIS
* GRASS GIS 7.4.x release branch has been created in {{rev|71701}}
** Remember that trunk is now called 7.5.svn, i.e. the trunk binary is called "grass75" and no longer "grass73".

== Markus Metz ==

* Some discussions about raster format and grass 8
* made NULL file compression the default
* fixing 7.4-tagged bugs:
** fix vector support in GCP manager
** r.out.gdal: export inf values
** r.out.gdal: +option to add overviews
** work on datum check at data import

== Moritz Lennert ==

* Added tests to r.object.geometry and fixed a bug noticed through the test ;-)
* Bug triaging
* 7.4 release preparation:
** [https://trac.osgeo.org/grass/wiki/Release/7.4.0-News 7.4.0-News] page
** Writing press release
* Some discussions about raster format and grass 8
* Work on improving r.neighborhoodmatrix
* Discussions on new web site
* (unsuccessful) attempts at debugging g.gui.iclass

== Nikos Alexandris ==

* Minor assistance in adding new/improved modules at [https://trac.osgeo.org/grass/wiki/Grass7/NewFeatures74 NewFeatures74]
* Implemented suggested tests for NULL file compression {{trac|2750}} (see comment [https://trac.osgeo.org/grass/ticket/2750#comment:66 66], see also https://github.com/NikosAlexandris/test_r_compress) | Thank you Moritz, Markus M
* Basic structure for an r.estimap addon, an implementation of algorithm for producing (European) recreation potential and opportunity maps

GRASS GIS Community Sprint Autumn 2017

2017-11-08T23:25:32Z

⚠️NikosA: Added in my to do: Check for stuff I should have done!

{{toc|right}}
GRASS people, let's meet! Because until the [[GRASS Community Sprint Bonn 2018|GRASS Community Sprint in March 2018]] there's still a looong way to go and there are topics that have been on the agenda for quite some time already... but especially because it is always so nice and fun to meet :D

== Purpose and Agenda ==

Discuss and tackle topics such as:

* Specs for GRASS GIS 8
* What's missing for 7.4.0
* Set up new website (asap) with responsive design
* Migration to git
* Include/invite more core devs
* Review and (possible) integration of GSoC 2017 work
* Outreach:
** possibility of a survey to know who uses GRASS GIS
** offer more GRASS GIS courses
** more online material, use Sphinx
** generate tons of 1min videos

== Sponsors ==

If you are interested to sponsor the GRASS Community Sprint, please read about 
    '''sponsoring the GRASS project at [http://grass.osgeo.org/donations/ http://grass.osgeo.org/donations/]''' 
and, if needed, contact [[User:Neteler|Markus Neteler]] <tt><neteler at osgeo.org></tt>. Any surplus at the end of the event will be turned over to the GRASS GIS project.

The GRASS Community Sprint is a great occasion for you to support the development of GRASS. With your contribution you'll enable more developers to meet. The community sprint is an important opportunity for the GRASS developers to discuss and collaboratively resolve bugs, plan the direction for the project and work on new features. The developers and contributors are donating their valuable time, so it would be great if in-kind funding can be made available from within the community to cover out-of-pocket expenses. All of the work that takes place at the community sprint will be directly contributed back into the GRASS project to the benefit of everyone who uses it.

=== Thanks to our sponsors ===

== Venue ==

* To be defined

== Accommodation and Costs ==

* To be defined


== Planned participation ==

For organizational reasons (i.e. define place and date), please '''additionally register''' at: https://beta.doodle.com/poll/p4a8wmtcpw6c9m9s

=== In person ===

Please add your name here:

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|width=100px|'''Arrival'''
|width=100px|'''Departure'''
|'''Topic'''
|width=75px|'''T-Shirt'''
|'''Notes'''
|-
|1
|[[User:Veroandreo|Veronica Andreo]]
|Netherlands
|Fri 10
|Mon 13
|Doc, testing, GCI
|S
|
|-
|2
|[[User:Neteler|Markus Neteler]]
|Germany
|Sat 11th
|Mon 13th
|See agenda :)
|M
|Will be travelling back from SFCon Bolzano, hence arriving on Sat
|-
|3
|[https://wiki.osgeo.org/wiki/User:Mmetz Markus Metz]
|Germany
|Fri 10
|Mon 13
|GRASS 7.4, 8
|M
|
|-
|4
|[[User:Mlennert|Moritz Lennert]]
|Belgium
|Fri 10
|Sun 12
|GRASS 7.4, 8, Web site, Docs/Tutorials
|M
|Will have to take a train at around 18:00 on Sunday night
|-
|5
|[[User:NikosA|Nikos Alexandris]]
|Italy
|Fri 10
|Mon 13
|Allarounder, as usual + Checking old posts of mine in mailing lists and file tickets where applicable + Implementing parts of [[chrome-extension://oemmndcbldboiebfnladdacbdfmadadm/http://publications.jrc.ec.europa.eu/repository/bitstream/JRC87585/lb-na-26474-en-n.pdf|ESTIMAP]] as GRASS-GIS addon
|M
|
|-
|6
|
|
|
|
|
|
|
|-
|}

=== Via IRC chat ===

(IRC is a nice archaic chat system, see [[IRC]])

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|-
|1
|[[User:Ychemin|Yann Chemin]]
|France
|-
|2
|[[User:hellik|Helmut Kudrnovsky]]
|Austria
|-
|3
|
|
|}

== Individual Preparation ==

* Bring your own computer
* Bring [http://en.wikipedia.org/wiki/Mains_electricity_by_country your power connector adapter] if needed. See also http://www.power-plugs-sockets.com/
* Install subversion and the compiler tools, and come with a working GRASS development environment if possible.

== Photos ==

Remember to take at least one group photo! :-)

== Reports ==

See [[Talk:GRASS_GIS_Community_Sprint_Autumn_2017]]

== FAQ ==

* ''Is the GRASS Community Sprint just a coding event?''
** It is mainly a coding and documentation event. It is a working session for people who are already participants in the GRASS project and/or are committed to improving the GRASS project.
** On demand we can do some presentations of current working GRASS implementation and new upcoming features to spread the idea of Open Source GIS software
* ''Is the GRASS Community Sprint for developers only?''
** No: anybody can help, with testing, checking out bugs and fixes, documentation and more.
* ''Where can I get help and more information about the community sprint?''
** Contact [[User:Veroandreo|Veronica Andreo]] <tt><veroandreo at gmail com></tt>

[[Category: Workshops]]
[[Category: Code Sprint]]
[[Category: 2017]]

GRASS GIS Community Sprint Autumn 2017

2017-10-27T21:11:13Z

⚠️NikosA: +NikosA

{{toc|right}}
GRASS people, let's meet! Because until the [[GRASS Community Sprint Bonn 2018|GRASS Community Sprint in March 2018]] there's still a looong way to go and there are topics that have been on the agenda for quite some time already... but especially because it is always so nice and fun to meet :D

== Purpose and Agenda ==

Discuss and tackle topics such as:

* Specs for GRASS GIS 8
* What's missing for 7.4.0
* Set up new website (asap) with responsive design
* Migration to git
* Include/invite more core devs
* Review and (possible) integration of GSoC 2017 work
* Outreach:
** possibility of a survey to know who uses GRASS GIS
** offer more GRASS GIS courses
** more online material, use Sphinx
** generate tons of 1min videos

== Sponsors ==

If you are interested to sponsor the GRASS Community Sprint, please read about 
    '''sponsoring the GRASS project at [http://grass.osgeo.org/donations/ http://grass.osgeo.org/donations/]''' 
and, if needed, contact [[User:Neteler|Markus Neteler]] <tt><neteler at osgeo.org></tt>. Any surplus at the end of the event will be turned over to the GRASS GIS project.

The GRASS Community Sprint is a great occasion for you to support the development of GRASS. With your contribution you'll enable more developers to meet. The community sprint is an important opportunity for the GRASS developers to discuss and collaboratively resolve bugs, plan the direction for the project and work on new features. The developers and contributors are donating their valuable time, so it would be great if in-kind funding can be made available from within the community to cover out-of-pocket expenses. All of the work that takes place at the community sprint will be directly contributed back into the GRASS project to the benefit of everyone who uses it.

=== Thanks to our sponsors ===

== Venue ==

* To be defined

== Accommodation and Costs ==

* To be defined


== Planned participation ==

For organizational reasons (i.e. define place and date), please '''additionally register''' at: https://beta.doodle.com/poll/p4a8wmtcpw6c9m9s

=== In person ===

Please add your name here:

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|width=100px|'''Arrival'''
|width=100px|'''Departure'''
|'''Topic'''
|width=75px|'''T-Shirt'''
|'''Notes'''
|-
|1
|[[User:Veroandreo|Veronica Andreo]]
|Netherlands
|Fri 10
|Mon 13
|Doc, testing, GCI
|S
|
|-
|2
|[[User:Neteler|Markus Neteler]]
|Germany
|Sat 11th
|Mon 13th
|See agenda :)
|M
|Will be travelling back from SFCon Bolzano, hence arriving on Sat
|-
|3
|[https://wiki.osgeo.org/wiki/User:Mmetz Markus Metz]
|Germany
|Fri 10
|Mon 13
|GRASS 7.4, 8
|M
|
|-
|4
|[[User:Mlennert|Moritz Lennert]]
|Belgium
|Fri 10
|Sun 12
|GRASS 7.4, 8, Web site, Docs/Tutorials
|M
|Will have to take a train at around 18:00 on Sunday night
|-
|5
|[[User:NikosA|Nikos Alexandris]]
|Italy
|Fri 10
|Mon 13
|Allarounder, as usual + Implementing parts of [[chrome-extension://oemmndcbldboiebfnladdacbdfmadadm/http://publications.jrc.ec.europa.eu/repository/bitstream/JRC87585/lb-na-26474-en-n.pdf|ESTIMAP]] as GRASS-GIS addon
|M
|
|-
|6
|
|
|
|
|
|
|
|-
|}

=== Via IRC chat ===

(IRC is a nice archaic chat system, see [[IRC]])

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|-
|1
|[[User:Ychemin|Yann Chemin]]
|France
|-
|2
|[[User:hellik|Helmut Kudrnovsky]]
|Austria
|-
|3
|
|
|}

== Individual Preparation ==

* Bring your own computer
* Bring [http://en.wikipedia.org/wiki/Mains_electricity_by_country your power connector adapter] if needed. See also http://www.power-plugs-sockets.com/
* Install subversion and the compiler tools, and come with a working GRASS development environment if possible.

== Photos ==

Remember to take at least one group photo! :-)

== Reports ==

See [[Talk:GRASS_GIS_Community_Sprint_Autumn_2017]]

== FAQ ==

* ''Is the GRASS Community Sprint just a coding event?''
** It is mainly a coding and documentation event. It is a working session for people who are already participants in the GRASS project and/or are committed to improving the GRASS project.
** On demand we can do some presentations of current working GRASS implementation and new upcoming features to spread the idea of Open Source GIS software
* ''Is the GRASS Community Sprint for developers only?''
** No: anybody can help, with testing, checking out bugs and fixes, documentation and more.
* ''Where can I get help and more information about the community sprint?''
** Contact [[User:Veroandreo|Veronica Andreo]] <tt><veroandreo at gmail com></tt>

[[Category: Workshops]]
[[Category: Code Sprint]]
[[Category: 2017]]

Temporal data processing

2017-10-21T06:58:26Z

⚠️NikosA: s/t.rast.series}/t.rast.series}}/

== Introduction ==

TGRASS is the temporal enabled GRASS GIS. It is available from [http://grass.osgeo.org/grass7/ GRASS GIS 7] onwards.
TGRASS is completely metadata based, i.e. it does not change any data but simply handles the organization of raster, vector, raster3D maps actually stored in a GRASS GIS mapset by registering in an additional internal database. This is done specifically for managing temporal and spatial extent including temporal topology.

Manual overview: https://grass.osgeo.org/grass72/manuals/temporalintro.html

== Terminology Overview ==

* Space time raster datasets ('''strds''') are designed to manage raster map time series. Modules that process strds have the naming prefix ''t.rast''.
* Space time 3D raster datasets ('''str3ds''') are designed to manage 3D raster map time series. Modules that process str3ds have the naming prefix ''t.rast3d''.
* Space time vector datasets ('''stvds''') are designed to manage vector map time series. Modules that process stvds have the naming prefix ''t.vect''.

== Workflow overview ==

# create an empty space time datasets: strds, str3ds, or stvds ({{cmd|t.create}})
# register the GRASS GIS maps ({{cmd|t.register}})
# check the generated space time datasets ({{cmd|t.list}}, {{cmd|t.info}})
# do your analysis: {{cmd|t.rast.aggregate}}, {{cmd|t.info}}, {{cmd|t.rast.univar}}, {{cmd|t.vect.univar}}, ... so many more!

== Example workflow for a Chlorophyll-a MODIS time series ==

The following examples are based on a series of MODIS L3 Chlorophyll-a product that is freely available at the [http://oceancolor.gsfc.nasa.gov/cgi/l3?sen=A&per=MO&prd=CHL_chlor_a ocean color] site. So, say we download the SMI 8-day composite product at 4.6 km resolution for the period 2003-2013. That is a 506 set of images, 46 per year. Data comes as compressed HDF4 files. Chlorophyll products filenames look like this:

A20030012003008.L3m_8D_CHL_chlor_a_4km

A: MODIS/Aqua
2003: Year at start
001: Julian day at start
2003: Year at end
008: Julian day at end
L3m: Level 3 data, mapped (Projection: Plate carrée)
8D: 8 day composition
CHL: Chlorophyll a concentration product
chlor_a: algorithm used 10^(a0 + a1*X + a2*X^2 + a3*X^3 + a4*X^4)
4km: 4.6km pixel size (8640x4320 image, 2.5 minute resolution)

We now decompress files and check metadata

<source lang="bash">
# Go to where the data is and decompress
find -iname '*.bz2' -exec bzip2 -d {} \;
# Check file meta-data (GDAL utilities)
gdalinfo A20030012003008.L3m_8D_CHL_chlor_a_4km
</source>

Next step is to import all 506 images into GRASS. You can use {{cmd|r.in.gdal}} or {{cmd|r.external}} for that.
Note that global Cl-a images as downloaded from [http://oceancolor.gsfc.nasa.gov/cgi/l3?sen=A&per=MO&prd=CHL_chlor_a ocean color] site are ~150 mb each (disk space issues!). Here, 3 different options:

* import global images (506 images, 150 Mb each) with r.in.gdal, resize to study area and remove global files
* set projection and extension, and resize to study area with gdal_translate, and import already resized images with r.in.gdal
* import global images (506 images, 150 Mb each) with r.external, resize to study area and remove global files, as showed next:

<source lang="bash">
# define region extension
g.region -p n=-38 s=-55 w=-70 e=-56

suffix=_tmp
for map in *chlor* ; do
r.external input=$map output=${map}${suffix} -o
r.mapcalc expression="$map = ${map}${suffix}"
g.remove type=raster name=${map}${suffix}
done
</source>

Once we have maps inside GRASS we can start the temporal processing. If this is the first time you'll use temporal modules,
you need to run

<source lang="bash">
t.connect -d
</source>

to set the default temporal GIS database connection for the current mapset. The default TGIS database of type ''sqlite3'' is located in the PERMANENT mapset directory. Temporal GIS content from all created mapsets will be stored there.

=== Creating a STRDS and registering maps ===

'''Creating a STRDS'''

First step is to '''create a space time dataset''' by means of {{cmd|t.create}}. As an example, let us create a strds for the Chlorophyll-a (Cl-a) time series. We need to define the name and semantic type of the new space time dataset, its title and a description. By default a raster dataset is created with absolute time (Gregorian calendar). We can change that according to our needs with type and temporaltype options.

<source lang="bash">
t.create type=strds temporaltype=absolute output=cla title="Chlorophyll-a concentration" \
description="MODIS L3 Chlorophyll-a concentration for Argentinian sea" semantictype=mean
</source>

'''Registering maps'''

Then, we '''register the maps''' in the strds using {{cmd|t.register}}. This module assigns time stamps to raster, 3D raster and vector maps and register them into space time datasets. Existing timestamps can be read and used by t.register. Note that this is a metadata based registration, nothing is imported into GRASS GIS since the maps are already present in the location (hence, no duplication occurs).

This module (and TGRASS in general) supports '''absolute time''' and '''relative time'''. The ''absolute'' temporal type refers to a fixed date or interval in the Gregorian calendar, while the ''relative'' temporal type refers to data without fixed time stamps (e.g., sequential maps used to calculate multi-decadal averages). Refer to the [http://www.geostat-course.org/system/files/Presentation_0.pdf TGRASS overview slides] for background.

Maps can be registered by command line argument (a list of comma separated map names) or using an input file. The start time, end time and a temporal increment can be provided either by command line or in the input file. End time and increment are mutual exclusive. Maps can be registered in several space time datasets using the same timestamp. For a more detailed explanation and examples on how to register maps in stds, see also [https://grasswiki.osgeo.org/wiki/Temporal_data_processing/maps_registration maps registration] wiki.

Start time and end time with absolute time must be provided using the format yyyy-mm-dd HH:MM:SS +HHMM. It is also supported to specify the date yyyy-mm-dd only. In case of relative time the temporal unit (years, months, days, hours, minutes or seconds) must be provided. The relative start time, end time and the increment are integers (see also {{cmd|t.register}}).

<source lang="bash">
# note: with -i we create intervals (start and end time), for the given increment, and starting from start.
t.register -i type=raster input=cla maps=`g.list raster pattern="*_chlor_*" separator=comma` \
start="2003-01-01" increment="8 days"
</source>

'''Dealing with weekly data'''

While the former would have been the simplest solution, our 8-day products have a problem: in this example the last image of each year is not a product of an 8-day composition, but 4 or 5-day. Then, when using the increment parameter, dates (and consequently, intervals) are not set properly.
The solution is to create a list of maps, with their respective start and end date. As the filenames contain information regarding year and DOY (day of year), we can use the following Python script to '''read filenames and transform DOY to calendar dates''' (Thanks Soeren!).

<source lang="python">
# Script to extract the input table for t.register from map names
# run in python
# modify to your needs

from os import walk
f = []
for (dirpath, dirnames, filenames) in walk("/path/to/the/maps"):
f.extend(filenames)
break
# to order the list
f.sort()

input_list=[]

import datetime
for map_name in f:
start_year = int(map_name[1:5])
start_day = int(map_name[5:8])
end_year = int(map_name[8:12])
end_day = int(map_name[12:15])
start = datetime.datetime(start_year, 1, 1) + datetime.timedelta(start_day - 1)
end = datetime.datetime(int(end_year), 1, 1) + datetime.timedelta(end_day)
map_list = map_name + '|' + str(start) + '|' + str(end)
print map_list
input_list.append(map_list)

print input_list

txt = '\n' .join(input_list)
f = open("input_list_cla.txt","w")
f.write(txt)
f.close()
</source>

Using the number of characters in the filenames and datetime library in Python, you can convert DOY in the filenames into start_time and end_time as in the list you need to pass to {{cmd|t.register}}. The resulting list
looks like this:

A20030012003008.L3m_8D_CHL_chlor_a_4km_arg|2003-01-01 00:00:00|2003-01-09 00:00:00
A20030092003016.L3m_8D_CHL_chlor_a_4km_arg|2003-01-09 00:00:00|2003-01-17 00:00:00
A20030172003024.L3m_8D_CHL_chlor_a_4km_arg|2003-01-17 00:00:00|2003-01-25 00:00:00
...
A20133452013352.L3m_8D_CHL_chlor_a_4km_arg|2013-12-11 00:00:00|2013-12-19 00:00:00
A20133532013360.L3m_8D_CHL_chlor_a_4km_arg|2013-12-19 00:00:00|2013-12-27 00:00:00
A20133612013365.L3m_8D_CHL_chlor_a_4km_arg|2013-12-27 00:00:00|2014-01-01 00:00:00

and then, the command would be:

<source lang="bash">
t.register --o type=raster input=cla file=input_list_cla.txt
</source>

As we are providing start and end time along with map names, we don't need to use the -i flag, neither set start nor increment options because that information is all there in the file.

'''Assign a color table'''

We can also set a color table for all maps in the strds with {{cmd|t.rast.colors}}:

<source lang="bash">
# using a predetermined color table
t.rast.colors input=cla color=bcyr
#using your dedicated color table
t.rast.colors input=cla rules=path/to/your/color_table
</source>

=== Getting some basic info and statistics ===

We now check the space time data sets we have in our mapset with {{cmd|t.list}}:

<source lang="bash">
t.list type=strds
</source>

and get information about our recently created strds. See {{cmd|t.info}} for additional uses.

<source lang="bash">
t.info type=strds input=cla
</source>

+-------------------- Space Time Raster Dataset -----------------------------+
| |
+-------------------- Basic information -------------------------------------+
| Id: ........................ cla@clorofila
| Name: ...................... cla
| Mapset: .................... clorofila
| Creator: ................... veroandreo
| Temporal type: ............. absolute
| Creation time: ............. 2014-04-29 14:23:00.579342
| Modification time:.......... 2014-05-12 09:15:08.917309
| Semantic type:.............. mean
+-------------------- Absolute time -----------------------------------------+
| Start time:................. 2003-01-01 00:00:00
| End time:................... 2014-01-01 00:00:00
| Granularity:................ 1 day
| Temporal type of maps:...... interval
+-------------------- Spatial extent ----------------------------------------+
| North:...................... -38.0
| South:...................... -55.0
| East:.. .................... -55.0
| West:....................... -70.0
| Top:........................ 0.0
| Bottom:..................... 0.0
+-------------------- Metadata information ----------------------------------+
| Raster register table:...... raster_map_register_91ba57d5f0924f4fa0bd7176a1b39b2f
| North-South resolution min:. 0.041667
| North-South resolution max:. 0.041667
| East-west resolution min:... 0.041667
| East-west resolution max:... 0.041667
| Minimum value min:.......... 0.02925
| Minimum value max:.......... 0.26472
| Maximum value min:.......... 5.2104
| Maximum value max:.......... 99.953934
| Aggregation type:........... None
| Number of registered maps:.. 506
|
| Title:
| Chlorophyll-a
| Description:
| Concentracion de Clorofila a
| Command history:
| # 2014-04-29 14:23:00
| t.create type="strds" temporaltype="absolute"
| output="cla" title="Chlorophyll-a"
| description="Concentracion de Clorofila a" --o
| # 2014-04-29 14:23:23
| t.register --o type="rast" input="cla"
| file="map_list"
|
+----------------------------------------------------------------------------+

Now, we get univariate statistics from the non-null cells for each registered raster map of the strds. For that matter we use {{cmd|t.rast.univar}} which, by default, returns the name of the map, the start and end date of dataset and the following values: ''mean, minimum and maximum vale, mean_of_abs, standard deviation, variance, coeff_var, number of null cells, total number of cell''.

<source lang="bash">
t.rast.univar input=cla
</source>

Or, you can send the output to a text file using

<source lang="bash">
t.rast.univar input=cla separator=comma output=stats_cla.csv
</source>

This file "stats_cla.csv" you can now open in a spreadsheet or statistical software for inspection and plotting.

=== Listing maps and selections ===

The module {{cmd|t.rast.list}} allows you to list registered maps of a strds and provides several options to get the list of maps that you want. For example, you can select different granules and also perform different queries by means of the where parameter. Some examples are:

<source lang="bash">
t.rast.list input=cla method=gran granule="1 month"
# this will give one image every one month, 3 months, 1 year, or the granule you choose

t.rast.list input=cla order=min columns=id,name,start_time,min where="min <= '0.05'"
# this will order by minimum value all the maps in the strds that have a minimum value lower than or equal to 0.05

t.rast.list input=cla order=max columns=name,start_time,max where="max > '10.0'"
# maps ordered by maximum value in which maximum value is higher than 10.

t.rast.list input=cla where="start_time >= '2003-01' and start_time <= '2003-06'"
# all the maps in the first 6 month of the time series

t.rast.list input=cla where="strftime('%m', start_time)='01'"
# all the maps from January

t.rast.list input=cla where="strftime('%m-%d', start_time)='01-01'"
# all the maps from January, 1st

t.rast.list input=cla where="strftime('%w', start_time)='1'"
# all Mondays in the time series (Sunday is 0)
</source>

If you have monthly (instead of 8-day products) data and you want to list all January maps, then you can do:

<source lang="bash">
t.rast.list input=cla_orig where="start_time=datetime(start_time, 'start of year', ' 0 month')"
</source>

Note: '''Do not forget to use single quotes around date''', neither in {{cmd|t.rast.list}} nor in any other of the t.* modules offering the ''where'' option, because the clause will be ignored (i.e.: no selection performed) and no warning nor error will be printed.

=== Visualization ===

There are different visualization options for strds.

* {{cmd|g.gui.timeline}} allows to compare temporal datasets by displaying their temporal extents in a plot.

<source lang="bash">
# only temporal extent
g.gui.timeline inputs=cla
# temporal and spatio-temporal extent
g.gui.timeline -3 inputs=cla
</source>

[[Image:Cla strds.png|center|thumb|560px|Temporal and spatio-temporal extent of cla strds.]]

* {{cmd|g.gui.tplot}} allows to see the values of one or more temporal datasets for a queried point defined by a coordinate pair.

Steps to use this module are:

# Select strds
# Select pair of coordinates (east,north) or point in the map
# Hit Draw
# Customize plot as desired
# Save

[[Image:Output gguitplot.png|center|thumb|800px|Time series plot (Chlorophyll vs Time) for a certain coordinate pair in the study area]]

* {{cmd|g.gui.animation}} is the tool for animating a series of raster and vector maps or a space time raster or vector dataset.

<source lang="bash">
g.gui.animation strds=cla_monthly_average
</source>

[[Image:Grass70 temporal chlorophyll anim small.gif|framed|center|Monthly mean chlorophyll-a concentration]]

=== Aggregation ===

For aggregations of data with different methods and different granularities, there are two very useful commands:

* {{cmd|t.rast.series}} that performs different aggregation algorithms from r.series on all or a subset of raster maps in a strds, and

* {{cmd|t.rast.aggregate}} that temporally aggregates the maps in a strds by a user defined granularity.

With these modules it is very simple to get maps of basic statistical parameters for different temporal granules, and this permits the analysis of the spatio-temporal variability of the variable of interest.

Some examples:

<source lang="bash">
# yearly aggregation
t.rast.aggregate input=cla output=cla_yearly_average basename=cla_yearly_average \
granularity="1 years" method=average sampling=starts suffix=gran

# yearly aggregation with corresponding methods (output: 7 strds with 11 maps each)
for method in average median mode minimum maximum stddev range ; do
t.rast.aggregate input=cla output=cla_yearly_${method} \
basename=cla_yearly_${method} granularity="1 years" \
method=${method} sampling=start suffix=gran
done
</source>

With the where parameter, as exemplified before, we can select for example all 8-day products which start_time is 01 (January) over the years. Like this we can get the so-known climatologies.

<source lang="bash">
t.rast.series input=cla method=average where="strftime('%m', start_time)='01'" output=january_average
</source>

Generalizing a bit, we can:

<source lang="bash">
# monthly climatologies
for i in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
for m in average median stddev minimum maximum ; do
t.rast.series input=cla method=${m} where="strftime('%m', start_time)='${i}'" output=${m}_${i}
done
done

# seasonal climatologies
for i in "01 02 03" "04 05 06" "07 08 09" "10 11 12" ; do
set -- $i ; echo $1 $2 $3
for m in average median stddev minimum maximum ; do
t.rast.series input=cla method=${m} output=${m}_${1} \
where="strftime('%m',start_time)='"${1}"' or strftime('%m',start_time)='"${2}"' or strftime('%m', start_time)='"${3}"'"
done
done
</source>

Using the climatologies previously obtained, we'll now estimate monthly anomalies in the mean, max and min Cl-a concentration. First, we need to monthly aggregate data, and then do the difference between the monthly climatology and each respective monthly aggregate. We'll do the aggregation for the average, minimum and maximum of Cl-a concentration (from 506 input maps in cla strds, we'll get 132 maps in each monthly aggregated new strds).

<source lang="bash">
# monthly aggregates (132 maps)

for method in average minimum maximum ; do
t.rast.aggregate input=cla output=cla_monthly_${method} basename=cla_monthly_${method} \
granularity="1 months" method=${method} sampling=contains suffix=gran
done

# January anomalies in mean, min and max Cl-a

t.rast.list -s input=cla_monthly_average where="start_time=datetime(start_time, 'start of year', '0 month')" columns=name

for m in average minimum maximum ; do
for i in 1 13 25 37 49 61 73 85 97 109 121 ; do # these numbers correspond to all january monthly aggregates
r.mapcalc expression="Jan_${m}_anomaly_${i}=cla_monthly_${m}_${i}-01_${m}"
done
done
</source>

You can also use {{cmd|t.rast.list}} for looping, the same way you use {{cmd|g.list}}:

<source lang="bash">
for map in `t.rast.list -s input=cla_monthly_average where="strftime('%m', start_time)='01'" columns=name` ; do
r.mapcalc expression="anomaly_${map}=${map}-01_average"
done
</source>

Generalizing, we can estimate all monthly anomalies from mean like this:

<source lang="bash">
for i in `seq -w 1 12` ; do
for map in `t.rast.list -s input=cla_monthly_average columns=name where="strftime('%m', start_time)='"${i}"'"` ; do
r.mapcalc expression="anomaly_${map}=${map}-average_${i}"
done
done
</source>

Say we now need to know the date of the maximum value of Cl-a concentration over all the study period and/or on a yearly basis:

<source lang="bash">
# map index for the overall maximum Cl-a value
t.rast.series input=cla method=max_raster output=cla_max_index

# map index for the yearly maximum Cl-a value
t.rast.aggregate input=cla granularity="1 year" method=max_raster output=yearly_max_index basename=yearly_max_index suffix=gran
</source>

The outputs show (pixelwise) the map index in which the maximum value of Cl-a occurs (from 1 to 506 and from 1 to 46, for the whole time series and on a yearly basis, respectively). For relative time this is maybe enough, but you may then want to reclassify data to get DOY, for example. In that case, you may use {{cmd|r.reclass}}.

If you already have monthly data, you can get climatologies quite simply as follows:

<source lang="bash">
# January averages
t.rast.series input=cla_monthly method=average where="start_time=datetime(start_time, 'start of year', '0 month')" output=jan_average
</source>

=== Spatio-temporal algebra with STRDS ===

The module {{cmd|t.rast.mapcalc}} allows us to perform spatio-temporal mapcalc expressions on temporally sampled maps of strds. There are spatial and temporal operators available for the "expression" string. Spatial operators, functions and internal variables are those used in {{cmd|r.mapcalc}}. Temporal internal variables supported for both relative and absolute time include: td(), start_time() and end_time(). There are also several very useful internal variables supported especially for absolute time of the current sample interval or instance, e.g.: start_doy(), start_year(), start_month() and so on (see {{cmd|t.rast.mapcalc}} manual site for further details and examples).

Some examples now. Say we did some analysis and decided that we will only consider values higher than 0.05. Then, we need to set all values below that threshold to null.

<source lang="bash">
t.rast.mapcalc input=cla expression="if(cla < 0.05, null(), cla)" output=cla_corrected basename=cla_corrected
</source>

or we may also want to take negative erroneous values to the knowm minimum of the strds, so:

<source lang="bash">
t.rast.series input=cla method=minimum output=min_cla
t.rast.mapcalc input=cla expression="if(cla < 0.0, min_cla, cla)" output=cla_corrected basename=cla_corrected
</source>

We may also need to reclassify all maps in the strds according to a certain threshold, e.g.: a certain level of Cl-a that indicates bloom conditions, in order to get bloom frequency afterwards:

<source lang="bash">
# reclassify
t.rast.mapcalc -n input=cla output=cla_bloom basename=cla_bloom expression="if(cla > 0.75, 1, null())"
# bloom frequency
t.rast.series input=cla_bloom output=bloom_freq method=count
</source>

Do you remember we wanted to get the DOY of maximum Cl-a value before? Well, here's another way of doing it...

<source lang="bash">
# overall maximum value
t.rast.series input=cla method=maximum output=max_cla
# new strds with DOY of overall maximum
t.rast.mapcalc -n inputs=cla output=date_max_cla expression="if(cla == max_cla,start_doy(),null())" basename=date_max_cla
# map with DOY of overall maximum
t.rast.series input=date_max_cla method=maximum output=max_cla_date
# remove date_max_cla strds (we were only interested in the resulting aggregated map)
t.remove -rf inputs=date_max_cla
</source>

In the development version of GRASS GIS, there's also a {{cmd|t.rast.algebra}} module that allows for temporal and spatial operations on strds by means of temporal raster algebra. The module expects an expression as input parameter in the following form: "result = expression".

The statement structure is similar to r.mapcalc, the result is the name of a new strds that will contain the result of the calculation given as expression. Expressions can be any valid or nested combination of temporal operations and spatial overlay or buffer functions that are provided by the temporal algebra. See the manual for further details and explanations.

We'll use this module to estimate the rate of change (slope) between every pair of maps in the "cla" strds. The result will be a new strds consisting of maps with the slope value between every consecutive pair of maps in the original strds:

<source lang="bash">
t.rast.algebra expression="slope_cla = (cla[1]-cla[0])/8.0" basename=slope_cla
# we set 8 as fixed denominator, because products are 8-day compositions
</source>

We can then use any of the aggregation modules that we saw before to get the maximum or minimumm rate of change for different granularities.

===== Some known issues with t.rast.mapcalc =====

It has been observed that if the names of inputs and output space time datasets (partially) match each other, t.rast.mapcalc may not work properly (especially when several input strds are used). This is because the module uses a simple search and replace approach
to substitute the input STRDS with the corresponding map names. Eventualy, choosing a specific order for the input STRDS in the input parameter may reduce the risk of wrong substitution. Therefore, especially in operations involving several STRDS (with partially matching names) as inputs, it might be better to use t.rast.algebra that correctly recognizes spatio-temporal datasets. However, t.rast.algebra is still experimental and only available in development version.

=== Subseting and something else ===

{{cmd|t.rast.extract}} is another really useful module in temporal GRASS. It allows to extract a subset of a strds and store it in a different strds. You use the "where" condition to do the subset, but you can also specify a r.mapcalc sub-expression that performs operations on all maps of the selected subset. If no r.mapcalc expression is defined, the selected maps are simply registered in the new output strds.

Say we need to need to know in how many maps of the strds min values are below a threshold, and not only that, but you also want to know how many pixels per map meet that condition. Then, we can do

<source lang="bash">
t.rast.extract input=cla where="min <= '0.05'" output=cla_less_05 basename=cla_less_05 expression="if(cla<0.05,1,null())"
</source>

to extract those maps with a minimum value lower than 0.05, and in the same step put 1 in every cell meeting the criterium and null everywhere else. To get a count of maps and pixels meeting the condition we may use:

<source lang="bash">
# univariate stistics to get the count per map
t.rast.univar input=cla_less_05
# count map of cells with min value < 0.05
t.rast.series input=cla_less_05 output=count_cla_less_05 method=count
</source>

=== Importing / Exporting strds ===

Say we now need to do some processing in R (e.g.: run [http://menugget.blogspot.com.ar/2012/10/dineof-data-interpolating-empirical.html DINEOF] to fill gaps in data), so we need to export our strds, hence we use {{cmd|t.rast.export}}.

<source lang="bash">
t.rast.export input=cla output=cla4R compression=gzip
</source>

After we have done our analysis we can import the whole strds back to GRASS again, exporting it from R with read/write.tgrass from [http://cran.r-project.org/web/packages/spacetime/index.html spacetime] R package and using {{cmd|t.rast.import}} in GRASS.

<source lang="bash">
t.rast.import input=cla_from_R.tar.gz output=cla_from_R basename=new_map directory=/tmp
</source>

For a full example of how to handle raster time series between GRASS and R, please visit the [https://grasswiki.osgeo.org/wiki/Temporal_data_processing/raster_time_series_grass_R_statistics time series GRASS-R Statistics] wiki. The example is based on North Carolina climate location [http://courses.ncsu.edu/mea592/common/media/02/nc_climate_spm_2000_2012.zip] and includes the following steps:

# Exporting the strds out of GRASS
# Importing into R
# Re-formating the data
# Running [http://modb.oce.ulg.ac.be/mediawiki/index.php/DINEOF DINEOF]
# Re-constructing the raster time series
# Exporting out of R, and
# Importing the gap-filled new strds into GRASS.

=== Neighborhood analysis ===

{{cmd|t.rast.neighbors}} performs a neighborhood analysis for each map in a space time raster dataset. This module supports a subset of the options already available in r.neighbors. Both, size of the neighborhood and aggregation method can be chosen. As an example, we'll estimate mean Cl-a concentration in a 3x3 neighborhood for every map in the strds.

<source lang="bash">
t.rast.neighbors input=cla output=cla_smoth base=cla_smooth method=average size=3
</source>

=== Filling and reconstructing time series data with gaps ===

==== Harmonic ANalysis of Time Series - HANTS ====

{{AddonCmd|r.hants}} is an add-on not strictly within the temporal modules, but really useful when working with this kind of data, particularly with gappy data. Here's a simple example for generating a list of temporally ordered maps to use as input in '''r.hants''', running HANTS and getting a map of dominant frequencies, afterwards. See the manual for further information on parameter setting and explanations.

<source lang="bash">
# use t.rast.list to create a list of temporally ordered maps
t.rast.list input=cla order=start_time columns=name -u > map_list.csv

# a HANTS run using the list of maps
r.hants -l file=map_list.csv nf=5 fet=0.1 dod=11 base_period=46 suffix=_hants amplitude=amp_hants phase=pha_hants

# dominant frequency map (0 means the dominant freq is 1, 1 that dominant freq is 2, and so on...)
r.series input=`g.list raster pattern=amp_hants* separator=comma` output=dominant_freq_hants method=max_raster
</source>

==== Local Weighted Regression - LWR ====

Another very useful add-on for the reconstruction of gappy time series, such as those from remote sensing imagery, is {{AddonCmd|r.series.lwr}}. This module performs a local weighted regression (LWR) of the input time series of maps in order to estimate missing values and identify (and remove) outliers. See the manual for further information and explanations.

<source lang="bash">
# use same list as before
t.rast.list input=cla order=start_time columns=name -u > map_list.csv

# run r.series.lwr
r.series.lwr file=map_list.csv suffix=_lwr order=2 weight=tricube range=0.0,65.0 dod=16 fet=0.1 -l
</source>

==== Estimate Mean Absolute Error (MAE) ====

One of the measures used to assess how close forecasts or predictions might be to the observed values is the
[https://en.wikipedia.org/wiki/Mean_absolute_error mean absolute error]. We will use it to evaluate the performance of HANTS and LWR.

First, we will rebuild our time series with the corresponding output maps of r.hants and r.series.lwr. Here, only showed for LWR.

<source lang="bash">
# re-build time series
t.create type=strds temporaltype=absolute output=cla_lwr \
title="LWR output for Chl-a" \
description="MODIS Aqua L3 Chl-a 8-day 4km 2010-2013. Reconstruction with r.series.lwr"

# create list with filenames to parse
g.list type=raster pattern="*lwr" output=names_list

# parse filenames, convert YYYY-DOY to YYYY-MM-DD and write file to use in t.register
for mapname in `cat names_list` ; do
year_start=`echo ${mapname:1:4}`
doy_start=`echo ${mapname:5:3}`
year_end=`echo ${mapname:8:4}`
doy_end=`echo ${mapname:12:3}`
# convert YYYY-DOY to YYYY-MM-DD
#BEWARE: leading zeros make bash assume the number is in base 8 system, not base 10!
doy_start=`echo "$doy_start" | sed 's/^0*//'`
doy_end=`echo "$doy_end" | sed 's/^0*//'`
START_DATE=`date -d "${year_start}-01-01 +$(( ${doy_start} - 1 ))days" +%Y-%m-%d`
END_DATE=`date -d "${year_end}-01-01 +$(( ${doy_end} ))days" +%Y-%m-%d`
# print mapname, start and end date
echo "$mapname|$START_DATE|$END_DATE" >> map_list_start_and_end_time.txt
done

# register maps in strds
t.register input=cla_lwr type=raster file=map_list_start_and_end_time.txt
</source>

Now, we will estimate the MAE and use it to compare both methods. Again, only the example for LWR is showed.

<source lang="bash">
# obtain a strds of the absolute differences between predicted and observed Chl-a values
t.rast.algebra -n basename=lwr_minus_orig \
expression="abs_lwr_minus_orig = abs(cla_lwr - cla)"

# sum of the absolute differences (numerator) and count of non-null maps per pixel (denominator)
t.rast.series input=abs_lwr_minus_orig output=sum_abs_lwr_minus_orig method=sum
t.rast.series input=abs_lwr_minus_orig output=count_abs_lwr_minus_orig method=count

# MAE for LWR
r.mapcalc expression="mae_lwr = sum_abs_lwr_minus_orig / count_abs_lwr_minus_orig"

# remove intermediate strds and maps
t.remove -rf abs_lwr_minus_orig
g.remove -f type=raster name=sum_abs_lwr_minus_orig,count_abs_lwr_minus_orig
</source>

Let's now compare the predictions generated by both methods, HANTS and LWR, and the corresponding mean absolute error maps.

{|
|-
| [[Image:cla_vs_cla_hants.png|center|thumb|560px|Original Chl-a time series vs HANTS output.]] || [[Image:cla_vs_cla_lwr.png|center|thumb|560px|Original Chl-a time series vs LWR output.)]]
|-
| [[Image:mae_hants_map.png|center|thumb|560px|MAE of HANTS predictions.]] || [[Image:mae_lwr_map.png|center|thumb|560px|MAE of LWR predictions.]]
|}

=== Extract strds values for points in a vector ===

Let's say we need to extract values of the strds to check the behaviour of HANTS at given locations and compare it with the original values, you can graphically do that with g.gui.tplot. But, if you need to take data outside GRASS, say, read it into R and do some other analysis (See [http://grasswiki.osgeo.org/wiki/R_statistics/rgrass7#Reading_in_data R_Statistics wiki page]) you may use {{cmd|v.what.strds}}. This module retrieves raster values from a given strds to the attribute table of a point vector map.

<source lang="bash">
# original data
v.what.strds --overwrite input=points_cla strds=cla output=points_cla_out
v.db.select map=points_cla_out file=ts_points_cla.csv

# HANTS' reconstructed data (several runs)
for i in `seq 1 13` ; do
v.what.strds --overwrite input=points_cla strds=cla_hants_${i} output=points_hants_${i}_out
v.db.select map=points_cla file=ts_points_hants_${i}.csv
done
</source>

Another option would be to use [https://grass.osgeo.org/grass71/manuals/t.rast.what.html t.rast.what] that samples a space time raster dataset at specific vector point coordinates and write the output to stdout or text file using different layouts. You don't write the vector's attribute table in this case.

<source lang="bash">
t.rast.what points=points strds=cla output=cla_points.csv null_value=NA separator=comma
</source>

=== TODO ===

(add some more)

== FAQ ==

=== Best practice data organization ===

When it come to time series, often thousands of maps are involved. In order to keep control, here some suggestion

* separate original data from derived aggregates: store them in separate mapsets
* for specific projects, maintain them in individual mapsets. You can access the STDS stored in other mapsets (same location) through
t.some.module input=my_strds@the_other_mapset output=result ...
as long as mapsets are indeed accessible, i.e.: in the mapset's search path (See {{cmd|g.mapsets}}).

=== Best practice multi user management ===

GRASS GIS supports multi-user data management and offers the possibility that several users work in the same location within their own mapsets. Data can be read from other mapsets but not edited there. The same principle applies to space time datasets (STDS), i.e. in case of having a collection of maps corresponding to a time series in a dedicated mapset, you need to also create the STDS therein (See {{cmd|t.create}} and {{cmd|t.register}}), so it is visible to other users in other mapsets when they use {{cmd|t.list}}, for example.

=== Use of ''where'' parameter (from ticket [https://trac.osgeo.org/grass/ticket/2270 #2270]) ===

Pay attention when using the "where" option in t.* modules. In some occasions it may not yield the expected results. Given the backend database chosen for TGRASS implementation, you may not be selecting all maps you think if you forget to set time along with date in the where clause. For example, let's say we need to select maps from 2005-03-01 until 2005-05-31 included. You would think the following command would do the job:

<source lang="bash">
t.rast.list daily_temp where="start_time > '2005-03-01' and start_time <= '2005-05-31'"
</source>

but, no... The last map in the list is:

temp_0516|pruebas|2005-05-30 00:00:00|2005-05-31 00:00:00

which would be equivalent to:

<source lang="bash">
t.rast.list daily_temp where="start_time > '2005-03-01' and start_time < '2005-05-31'"
</source>

This is a product of sqlite. Therefore, if you need the last date included in your selection (map from 2005-05-31 in our example), you need to set time too.

<source lang="bash">
t.rast.list daily_temp where="start_time > '2005-03-01' and start_time <= '2005-05-31 00:00:00'" | tail -n1
temp_0517|pruebas|2005-05-31 00:00:00|2005-06-01 00:00:00
</source>

=== Aggregation with defined granularity ===

''Q: I need to aggregate a strds with a granularity of 1 year, but shifting the start day one month in each run, i.e.: changing the start_time to 2003-02-01, 2003-03-01, 2003-04-01 and so on... My question is: if i recursively change start_time with the 'where' parameter, will the module t.rast.aggregate "aggregate" to the next february, march, april (what i'd wish) or just till the end of 2003?''

A: If you specify a granularity of a year, then the start time to perform the aggregation will always be shifted to the 1st January of the current year and the end time the 1st January of the next year (eg. 2002-01-01 - 2003-01-01). If you wish to aggregate a full year but shifting one month forward then simply use a granularity of 12 months.

=== Listing maps with specific start month ===

''Q: I have a strds with 506 maps that correspond to 8-day composite products. I need to sequentially list all maps which "start_month" is January, February and so on... to use them as input in {{cmd|r.series}} (or {{cmd|t.rast.series}}). How can I achieve that?''

A: You can use the [https://www.sqlite.org/lang_datefunc.html datetime functionality of SQLite] to perform this task, this should work for January:

<source lang="bash">
t.rast.list input=cla_null_mayor65 \
where='start_time >= datetime(start_time, "start of year") and start_time <= datetime(start_time, "start of year", "1 month")'
</source>

=== Aggregation of seasonal data ===

''Q: How can I calculate average seasonal temperature starting from a daily temperatures temporal dataset?''

A: Use {{cmd|t.rast.aggregate.ds}}, the input is the daily strds, the sampling stds should have seasonal intervals. Then use average as method. The output will have seasonal aggregated temperatures. For a detailed workflow see: [https://grasswiki.osgeo.org/wiki/Temporal_data_processing/seasonal_aggregation seasonal aggregation example].

=== Aggregation of seasonal data using time ranges ===

A way to aggregate seasons from daily data without granularity but by using time ranges is shown here. The issue is to include for the season calculation also a month from the previous year. This can be addressed by some datetime calculations in SQLite:

<source lang="bash">
# We assume to have daily temperature data in DB "temp_daily_average"

# loop over seasons, generate aggregates:
### 'start_time' and 'end_time' are columns in TGRASS

for year in `seq 2004 2014` ; do

# we consider also a month of the previous year
for month in "12 01 02" "03 04 05" "06 07 08" "09 10 11" ; do
set -- $month ; echo $1 $2 $3

prevyear=$year
if [ $1 -eq 12 ] ; then
prevyear=`expr $year - 1`
fi

# prepare query strings in SQLite
MYSTART=`echo "SELECT strftime(datetime('${prevyear}-${1}-01'));" | sqlite3`
MYEND=`echo "SELECT strftime(datetime('${prevyear}-${1}-01','+3 month'));" | sqlite3`

# Debugging only, to see what it does:
echo "---- Querying ${prevyear}-${1}-01 ... ${year}-${3}-end:"
# we use start_time and end_time to get the proper time range
t.rast.list input=temp_daily_average@modis_lst_reconstructed_europe_daily \
where="start_time >= '$MYSTART' AND end_time <= '$MYEND'" > list_${prevyear}_${1}_${3}.csv

head -n 3 list_${prevyear}_${1}_${3}.csv
echo "..."
tail -n 3 list_${prevyear}_${1}_${3}.csv
echo "======="
rm -f list_${prevyear}_${1}_${3}.csv

# calculate aggregates:
method="average" # median mode minimum maximum stddev

t.rast.series input=temp_daily_average@modis_lst_reconstructed_europe_daily \
where="start_time >= '$MYSTART' AND end_time <= '$MYEND'" \
output=temp_${method}_${prevyear}_${1}
done
done
</source>

Much simpler '''alternative''' (which runs in parallel on multi-cores):
<source lang="bash">
t.rast.aggregate input=A output=B basename=b where="start_time >= '2004-03-01 00:00:00'" granularity="3 months"
</source>

== Online tutorials and courses ==

* [http://www.geostat-course.org/Topic_Gebbert The temporal GRASS GIS framework: Introduction and application] by Sören Gebbert (2012)
* [http://fatra.cnr.ncsu.edu/temporal-grass-workshop/ Spatio-temporal data handling and visualization in GRASS GIS] - FOSS4G 2014 workshop by Vaclav Petras, Anna Petrasova, Helena Mitasova, Markus Neteler (2014)
* [http://www.openniche.org/processing-era-interim-dataset-in-ncdf-using-gdal-and-grass-gis/ Processing ERA-Interim dataset in netcdf using GDAL and GRASS GIS] by Sajid Pareeth (2015)
* CZ: [http://training.gismentors.eu/grass-gis-pokrocily/tgrass/index.html Úvod do časoprostorových analýz] (Introduction to space-time analysis) by Martin Landa (2015)

== References ==
* Gebbert, S., Pebesma, E. 2014. ''TGRASS: A temporal GIS for field based environmental modeling''. Environmental Modelling & Software 53, 1-12 ([http://dx.doi.org/10.1016/j.envsoft.2013.11.001 DOI]) - [http://ifgi.uni-muenster.de/~epebe_01/tgrass.pdf preprint PDF]
* Gebbert, S., Pebesma, E. 2017. ''The GRASS GIS temporal framework''. International Journal of Geographical Information Science 31, 1273-1292 ([http://dx.doi.org/10.1080/13658816.2017.1306862 DOI])

== See also ==
* Introduction: [http://www.slideshare.net/Luis_de_Sousa/presentation-soeren GRASS as a Temporal GIS] by Sören Gebbert (slides)
* [https://archive.org/details/TheTemporalGrassGisFrameworkIntroductionAndApplication# The temporal GRASS GIS framework: Introduction and application] Video of Sören Gebbert's presentation in [http://www.geostat-course.org/Muenster_2012 GEOSTAT course] 2012
* [http://grass.osgeo.org/grass72/manuals/temporalintro.html Introduction to temporal modules] (GRASS GIS manual)
* [http://grass.osgeo.org/grass72/manuals/temporal.html Available temporal modules] (GRASS GIS manual)
* [http://trac.osgeo.org/grass/wiki/Grass7/TemporalExtension Temporal Extension] (background info in trac)
* [[GRASS GSoC 2013 Temporal GIS Algebra for raster and vector data in GRASS]]

[[Category: Documentation]]
[[Category: Tutorial]]
[[Category: Temporal]]

Temporal data processing

2017-10-20T17:40:47Z

⚠️NikosA: /* Aggregation */ s/sampling=start/sampling=starts/

== Introduction ==

TGRASS is the temporal enabled GRASS GIS. It is available from [http://grass.osgeo.org/grass7/ GRASS GIS 7] onwards.
TGRASS is completely metadata based, i.e. it does not change any data but simply handles the organization of raster, vector, raster3D maps actually stored in a GRASS GIS mapset by registering in an additional internal database. This is done specifically for managing temporal and spatial extent including temporal topology.

Manual overview: https://grass.osgeo.org/grass72/manuals/temporalintro.html

== Terminology Overview ==

* Space time raster datasets ('''strds''') are designed to manage raster map time series. Modules that process strds have the naming prefix ''t.rast''.
* Space time 3D raster datasets ('''str3ds''') are designed to manage 3D raster map time series. Modules that process str3ds have the naming prefix ''t.rast3d''.
* Space time vector datasets ('''stvds''') are designed to manage vector map time series. Modules that process stvds have the naming prefix ''t.vect''.

== Workflow overview ==

# create an empty space time datasets: strds, str3ds, or stvds ({{cmd|t.create}})
# register the GRASS GIS maps ({{cmd|t.register}})
# check the generated space time datasets ({{cmd|t.list}}, {{cmd|t.info}})
# do your analysis: {{cmd|t.rast.aggregate}}, {{cmd|t.info}}, {{cmd|t.rast.univar}}, {{cmd|t.vect.univar}}, ... so many more!

== Example workflow for a Chlorophyll-a MODIS time series ==

The following examples are based on a series of MODIS L3 Chlorophyll-a product that is freely available at the [http://oceancolor.gsfc.nasa.gov/cgi/l3?sen=A&per=MO&prd=CHL_chlor_a ocean color] site. So, say we download the SMI 8-day composite product at 4.6 km resolution for the period 2003-2013. That is a 506 set of images, 46 per year. Data comes as compressed HDF4 files. Chlorophyll products filenames look like this:

A20030012003008.L3m_8D_CHL_chlor_a_4km

A: MODIS/Aqua
2003: Year at start
001: Julian day at start
2003: Year at end
008: Julian day at end
L3m: Level 3 data, mapped (Projection: Plate carrée)
8D: 8 day composition
CHL: Chlorophyll a concentration product
chlor_a: algorithm used 10^(a0 + a1*X + a2*X^2 + a3*X^3 + a4*X^4)
4km: 4.6km pixel size (8640x4320 image, 2.5 minute resolution)

We now decompress files and check metadata

<source lang="bash">
# Go to where the data is and decompress
find -iname '*.bz2' -exec bzip2 -d {} \;
# Check file meta-data (GDAL utilities)
gdalinfo A20030012003008.L3m_8D_CHL_chlor_a_4km
</source>

Next step is to import all 506 images into GRASS. You can use {{cmd|r.in.gdal}} or {{cmd|r.external}} for that.
Note that global Cl-a images as downloaded from [http://oceancolor.gsfc.nasa.gov/cgi/l3?sen=A&per=MO&prd=CHL_chlor_a ocean color] site are ~150 mb each (disk space issues!). Here, 3 different options:

* import global images (506 images, 150 Mb each) with r.in.gdal, resize to study area and remove global files
* set projection and extension, and resize to study area with gdal_translate, and import already resized images with r.in.gdal
* import global images (506 images, 150 Mb each) with r.external, resize to study area and remove global files, as showed next:

<source lang="bash">
# define region extension
g.region -p n=-38 s=-55 w=-70 e=-56

suffix=_tmp
for map in *chlor* ; do
r.external input=$map output=${map}${suffix} -o
r.mapcalc expression="$map = ${map}${suffix}"
g.remove type=raster name=${map}${suffix}
done
</source>

Once we have maps inside GRASS we can start the temporal processing. If this is the first time you'll use temporal modules,
you need to run

<source lang="bash">
t.connect -d
</source>

to set the default temporal GIS database connection for the current mapset. The default TGIS database of type ''sqlite3'' is located in the PERMANENT mapset directory. Temporal GIS content from all created mapsets will be stored there.

=== Creating a STRDS and registering maps ===

'''Creating a STRDS'''

First step is to '''create a space time dataset''' by means of {{cmd|t.create}}. As an example, let us create a strds for the Chlorophyll-a (Cl-a) time series. We need to define the name and semantic type of the new space time dataset, its title and a description. By default a raster dataset is created with absolute time (Gregorian calendar). We can change that according to our needs with type and temporaltype options.

<source lang="bash">
t.create type=strds temporaltype=absolute output=cla title="Chlorophyll-a concentration" \
description="MODIS L3 Chlorophyll-a concentration for Argentinian sea" semantictype=mean
</source>

'''Registering maps'''

Then, we '''register the maps''' in the strds using {{cmd|t.register}}. This module assigns time stamps to raster, 3D raster and vector maps and register them into space time datasets. Existing timestamps can be read and used by t.register. Note that this is a metadata based registration, nothing is imported into GRASS GIS since the maps are already present in the location (hence, no duplication occurs).

This module (and TGRASS in general) supports '''absolute time''' and '''relative time'''. The ''absolute'' temporal type refers to a fixed date or interval in the Gregorian calendar, while the ''relative'' temporal type refers to data without fixed time stamps (e.g., sequential maps used to calculate multi-decadal averages). Refer to the [http://www.geostat-course.org/system/files/Presentation_0.pdf TGRASS overview slides] for background.

Maps can be registered by command line argument (a list of comma separated map names) or using an input file. The start time, end time and a temporal increment can be provided either by command line or in the input file. End time and increment are mutual exclusive. Maps can be registered in several space time datasets using the same timestamp. For a more detailed explanation and examples on how to register maps in stds, see also [https://grasswiki.osgeo.org/wiki/Temporal_data_processing/maps_registration maps registration] wiki.

Start time and end time with absolute time must be provided using the format yyyy-mm-dd HH:MM:SS +HHMM. It is also supported to specify the date yyyy-mm-dd only. In case of relative time the temporal unit (years, months, days, hours, minutes or seconds) must be provided. The relative start time, end time and the increment are integers (see also {{cmd|t.register}}).

<source lang="bash">
# note: with -i we create intervals (start and end time), for the given increment, and starting from start.
t.register -i type=raster input=cla maps=`g.list raster pattern="*_chlor_*" separator=comma` \
start="2003-01-01" increment="8 days"
</source>

'''Dealing with weekly data'''

While the former would have been the simplest solution, our 8-day products have a problem: in this example the last image of each year is not a product of an 8-day composition, but 4 or 5-day. Then, when using the increment parameter, dates (and consequently, intervals) are not set properly.
The solution is to create a list of maps, with their respective start and end date. As the filenames contain information regarding year and DOY (day of year), we can use the following Python script to '''read filenames and transform DOY to calendar dates''' (Thanks Soeren!).

<source lang="python">
# Script to extract the input table for t.register from map names
# run in python
# modify to your needs

from os import walk
f = []
for (dirpath, dirnames, filenames) in walk("/path/to/the/maps"):
f.extend(filenames)
break
# to order the list
f.sort()

input_list=[]

import datetime
for map_name in f:
start_year = int(map_name[1:5])
start_day = int(map_name[5:8])
end_year = int(map_name[8:12])
end_day = int(map_name[12:15])
start = datetime.datetime(start_year, 1, 1) + datetime.timedelta(start_day - 1)
end = datetime.datetime(int(end_year), 1, 1) + datetime.timedelta(end_day)
map_list = map_name + '|' + str(start) + '|' + str(end)
print map_list
input_list.append(map_list)

print input_list

txt = '\n' .join(input_list)
f = open("input_list_cla.txt","w")
f.write(txt)
f.close()
</source>

Using the number of characters in the filenames and datetime library in Python, you can convert DOY in the filenames into start_time and end_time as in the list you need to pass to {{cmd|t.register}}. The resulting list
looks like this:

A20030012003008.L3m_8D_CHL_chlor_a_4km_arg|2003-01-01 00:00:00|2003-01-09 00:00:00
A20030092003016.L3m_8D_CHL_chlor_a_4km_arg|2003-01-09 00:00:00|2003-01-17 00:00:00
A20030172003024.L3m_8D_CHL_chlor_a_4km_arg|2003-01-17 00:00:00|2003-01-25 00:00:00
...
A20133452013352.L3m_8D_CHL_chlor_a_4km_arg|2013-12-11 00:00:00|2013-12-19 00:00:00
A20133532013360.L3m_8D_CHL_chlor_a_4km_arg|2013-12-19 00:00:00|2013-12-27 00:00:00
A20133612013365.L3m_8D_CHL_chlor_a_4km_arg|2013-12-27 00:00:00|2014-01-01 00:00:00

and then, the command would be:

<source lang="bash">
t.register --o type=raster input=cla file=input_list_cla.txt
</source>

As we are providing start and end time along with map names, we don't need to use the -i flag, neither set start nor increment options because that information is all there in the file.

'''Assign a color table'''

We can also set a color table for all maps in the strds with {{cmd|t.rast.colors}}:

<source lang="bash">
# using a predetermined color table
t.rast.colors input=cla color=bcyr
#using your dedicated color table
t.rast.colors input=cla rules=path/to/your/color_table
</source>

=== Getting some basic info and statistics ===

We now check the space time data sets we have in our mapset with {{cmd|t.list}}:

<source lang="bash">
t.list type=strds
</source>

and get information about our recently created strds. See {{cmd|t.info}} for additional uses.

<source lang="bash">
t.info type=strds input=cla
</source>

+-------------------- Space Time Raster Dataset -----------------------------+
| |
+-------------------- Basic information -------------------------------------+
| Id: ........................ cla@clorofila
| Name: ...................... cla
| Mapset: .................... clorofila
| Creator: ................... veroandreo
| Temporal type: ............. absolute
| Creation time: ............. 2014-04-29 14:23:00.579342
| Modification time:.......... 2014-05-12 09:15:08.917309
| Semantic type:.............. mean
+-------------------- Absolute time -----------------------------------------+
| Start time:................. 2003-01-01 00:00:00
| End time:................... 2014-01-01 00:00:00
| Granularity:................ 1 day
| Temporal type of maps:...... interval
+-------------------- Spatial extent ----------------------------------------+
| North:...................... -38.0
| South:...................... -55.0
| East:.. .................... -55.0
| West:....................... -70.0
| Top:........................ 0.0
| Bottom:..................... 0.0
+-------------------- Metadata information ----------------------------------+
| Raster register table:...... raster_map_register_91ba57d5f0924f4fa0bd7176a1b39b2f
| North-South resolution min:. 0.041667
| North-South resolution max:. 0.041667
| East-west resolution min:... 0.041667
| East-west resolution max:... 0.041667
| Minimum value min:.......... 0.02925
| Minimum value max:.......... 0.26472
| Maximum value min:.......... 5.2104
| Maximum value max:.......... 99.953934
| Aggregation type:........... None
| Number of registered maps:.. 506
|
| Title:
| Chlorophyll-a
| Description:
| Concentracion de Clorofila a
| Command history:
| # 2014-04-29 14:23:00
| t.create type="strds" temporaltype="absolute"
| output="cla" title="Chlorophyll-a"
| description="Concentracion de Clorofila a" --o
| # 2014-04-29 14:23:23
| t.register --o type="rast" input="cla"
| file="map_list"
|
+----------------------------------------------------------------------------+

Now, we get univariate statistics from the non-null cells for each registered raster map of the strds. For that matter we use {{cmd|t.rast.univar}} which, by default, returns the name of the map, the start and end date of dataset and the following values: ''mean, minimum and maximum vale, mean_of_abs, standard deviation, variance, coeff_var, number of null cells, total number of cell''.

<source lang="bash">
t.rast.univar input=cla
</source>

Or, you can send the output to a text file using

<source lang="bash">
t.rast.univar input=cla separator=comma output=stats_cla.csv
</source>

This file "stats_cla.csv" you can now open in a spreadsheet or statistical software for inspection and plotting.

=== Listing maps and selections ===

The module {{cmd|t.rast.list}} allows you to list registered maps of a strds and provides several options to get the list of maps that you want. For example, you can select different granules and also perform different queries by means of the where parameter. Some examples are:

<source lang="bash">
t.rast.list input=cla method=gran granule="1 month"
# this will give one image every one month, 3 months, 1 year, or the granule you choose

t.rast.list input=cla order=min columns=id,name,start_time,min where="min <= '0.05'"
# this will order by minimum value all the maps in the strds that have a minimum value lower than or equal to 0.05

t.rast.list input=cla order=max columns=name,start_time,max where="max > '10.0'"
# maps ordered by maximum value in which maximum value is higher than 10.

t.rast.list input=cla where="start_time >= '2003-01' and start_time <= '2003-06'"
# all the maps in the first 6 month of the time series

t.rast.list input=cla where="strftime('%m', start_time)='01'"
# all the maps from January

t.rast.list input=cla where="strftime('%m-%d', start_time)='01-01'"
# all the maps from January, 1st

t.rast.list input=cla where="strftime('%w', start_time)='1'"
# all Mondays in the time series (Sunday is 0)
</source>

If you have monthly (instead of 8-day products) data and you want to list all January maps, then you can do:

<source lang="bash">
t.rast.list input=cla_orig where="start_time=datetime(start_time, 'start of year', ' 0 month')"
</source>

Note: '''Do not forget to use single quotes around date''', neither in {{cmd|t.rast.list}} nor in any other of the t.* modules offering the ''where'' option, because the clause will be ignored (i.e.: no selection performed) and no warning nor error will be printed.

=== Visualization ===

There are different visualization options for strds.

* {{cmd|g.gui.timeline}} allows to compare temporal datasets by displaying their temporal extents in a plot.

<source lang="bash">
# only temporal extent
g.gui.timeline inputs=cla
# temporal and spatio-temporal extent
g.gui.timeline -3 inputs=cla
</source>

[[Image:Cla strds.png|center|thumb|560px|Temporal and spatio-temporal extent of cla strds.]]

* {{cmd|g.gui.tplot}} allows to see the values of one or more temporal datasets for a queried point defined by a coordinate pair.

Steps to use this module are:

# Select strds
# Select pair of coordinates (east,north) or point in the map
# Hit Draw
# Customize plot as desired
# Save

[[Image:Output gguitplot.png|center|thumb|800px|Time series plot (Chlorophyll vs Time) for a certain coordinate pair in the study area]]

* {{cmd|g.gui.animation}} is the tool for animating a series of raster and vector maps or a space time raster or vector dataset.

<source lang="bash">
g.gui.animation strds=cla_monthly_average
</source>

[[Image:Grass70 temporal chlorophyll anim small.gif|framed|center|Monthly mean chlorophyll-a concentration]]

=== Aggregation ===

For aggregations of data with different methods and different granularities, there are two very useful commands:

* {{cmd|t.rast.series}} that performs different aggregation algorithms from r.series on all or a subset of raster maps in a strds, and

* {{cmd|t.rast.aggregate}} that temporally aggregates the maps in a strds by a user defined granularity.

With these modules it is very simple to get maps of basic statistical parameters for different temporal granules, and this permits the analysis of the spatio-temporal variability of the variable of interest.

Some examples:

<source lang="bash">
# yearly aggregation
t.rast.aggregate input=cla output=cla_yearly_average basename=cla_yearly_average \
granularity="1 years" method=average sampling=starts suffix=gran

# yearly aggregation with corresponding methods (output: 7 strds with 11 maps each)
for method in average median mode minimum maximum stddev range ; do
t.rast.aggregate input=cla output=cla_yearly_${method} \
basename=cla_yearly_${method} granularity="1 years" \
method=${method} sampling=start suffix=gran
done
</source>

With the where parameter, as exemplified before, we can select for example all 8-day products which start_time is 01 (January) over the years. Like this we can get the so-known climatologies.

<source lang="bash">
t.rast.series input=cla method=average where="strftime('%m', start_time)='01'" output=january_average
</source>

Generalizing a bit, we can:

<source lang="bash">
# monthly climatologies
for i in 01 02 03 04 05 06 07 08 09 10 11 12 ; do
for m in average median stddev minimum maximum ; do
t.rast.series input=cla method=${m} where="strftime('%m', start_time)='${i}'" output=${m}_${i}
done
done

# seasonal climatologies
for i in "01 02 03" "04 05 06" "07 08 09" "10 11 12" ; do
set -- $i ; echo $1 $2 $3
for m in average median stddev minimum maximum ; do
t.rast.series input=cla method=${m} output=${m}_${1} \
where="strftime('%m',start_time)='"${1}"' or strftime('%m',start_time)='"${2}"' or strftime('%m', start_time)='"${3}"'"
done
done
</source>

Using the climatologies previously obtained, we'll now estimate monthly anomalies in the mean, max and min Cl-a concentration. First, we need to monthly aggregate data, and then do the difference between the monthly climatology and each respective monthly aggregate. We'll do the aggregation for the average, minimum and maximum of Cl-a concentration (from 506 input maps in cla strds, we'll get 132 maps in each monthly aggregated new strds).

<source lang="bash">
# monthly aggregates (132 maps)

for method in average minimum maximum ; do
t.rast.aggregate input=cla output=cla_monthly_${method} basename=cla_monthly_${method} \
granularity="1 months" method=${method} sampling=contains suffix=gran
done

# January anomalies in mean, min and max Cl-a

t.rast.list -s input=cla_monthly_average where="start_time=datetime(start_time, 'start of year', '0 month')" columns=name

for m in average minimum maximum ; do
for i in 1 13 25 37 49 61 73 85 97 109 121 ; do # these numbers correspond to all january monthly aggregates
r.mapcalc expression="Jan_${m}_anomaly_${i}=cla_monthly_${m}_${i}-01_${m}"
done
done
</source>

You can also use {{cmd|t.rast.list}} for looping, the same way you use {{cmd|g.list}}:

<source lang="bash">
for map in `t.rast.list -s input=cla_monthly_average where="strftime('%m', start_time)='01'" columns=name` ; do
r.mapcalc expression="anomaly_${map}=${map}-01_average"
done
</source>

Generalizing, we can estimate all monthly anomalies from mean like this:

<source lang="bash">
for i in `seq -w 1 12` ; do
for map in `t.rast.list -s input=cla_monthly_average columns=name where="strftime('%m', start_time)='"${i}"'"` ; do
r.mapcalc expression="anomaly_${map}=${map}-average_${i}"
done
done
</source>

Say we now need to know the date of the maximum value of Cl-a concentration over all the study period and/or on a yearly basis:

<source lang="bash">
# map index for the overall maximum Cl-a value
t.rast.series input=cla method=max_raster output=cla_max_index

# map index for the yearly maximum Cl-a value
t.rast.aggregate input=cla granularity="1 year" method=max_raster output=yearly_max_index basename=yearly_max_index suffix=gran
</source>

The outputs show (pixelwise) the map index in which the maximum value of Cl-a occurs (from 1 to 506 and from 1 to 46, for the whole time series and on a yearly basis, respectively). For relative time this is maybe enough, but you may then want to reclassify data to get DOY, for example. In that case, you may use {{cmd|r.reclass}}.

If you already have monthly data, you can get climatologies quite simply as follows:

<source lang="bash">
# January averages
t.rast.series input=cla_monthly method=average where="start_time=datetime(start_time, 'start of year', '0 month')" output=jan_average
</source>

=== Spatio-temporal algebra with STRDS ===

The module {{cmd|t.rast.mapcalc}} allows us to perform spatio-temporal mapcalc expressions on temporally sampled maps of strds. There are spatial and temporal operators available for the "expression" string. Spatial operators, functions and internal variables are those used in {{cmd|r.mapcalc}}. Temporal internal variables supported for both relative and absolute time include: td(), start_time() and end_time(). There are also several very useful internal variables supported especially for absolute time of the current sample interval or instance, e.g.: start_doy(), start_year(), start_month() and so on (see {{cmd|t.rast.mapcalc}} manual site for further details and examples).

Some examples now. Say we did some analysis and decided that we will only consider values higher than 0.05. Then, we need to set all values below that threshold to null.

<source lang="bash">
t.rast.mapcalc input=cla expression="if(cla < 0.05, null(), cla)" output=cla_corrected basename=cla_corrected
</source>

or we may also want to take negative erroneous values to the knowm minimum of the strds, so:

<source lang="bash">
t.rast.series input=cla method=minimum output=min_cla
t.rast.mapcalc input=cla expression="if(cla < 0.0, min_cla, cla)" output=cla_corrected basename=cla_corrected
</source>

We may also need to reclassify all maps in the strds according to a certain threshold, e.g.: a certain level of Cl-a that indicates bloom conditions, in order to get bloom frequency afterwards:

<source lang="bash">
# reclassify
t.rast.mapcalc -n input=cla output=cla_bloom basename=cla_bloom expression="if(cla > 0.75, 1, null())"
# bloom frequency
t.rast.series input=cla_bloom output=bloom_freq method=count
</source>

Do you remember we wanted to get the DOY of maximum Cl-a value before? Well, here's another way of doing it...

<source lang="bash">
# overall maximum value
t.rast.series input=cla method=maximum output=max_cla
# new strds with DOY of overall maximum
t.rast.mapcalc -n inputs=cla output=date_max_cla expression="if(cla == max_cla,start_doy(),null())" basename=date_max_cla
# map with DOY of overall maximum
t.rast.series input=date_max_cla method=maximum output=max_cla_date
# remove date_max_cla strds (we were only interested in the resulting aggregated map)
t.remove -rf inputs=date_max_cla
</source>

In the development version of GRASS GIS, there's also a {{cmd|t.rast.algebra}} module that allows for temporal and spatial operations on strds by means of temporal raster algebra. The module expects an expression as input parameter in the following form: "result = expression".

The statement structure is similar to r.mapcalc, the result is the name of a new strds that will contain the result of the calculation given as expression. Expressions can be any valid or nested combination of temporal operations and spatial overlay or buffer functions that are provided by the temporal algebra. See the manual for further details and explanations.

We'll use this module to estimate the rate of change (slope) between every pair of maps in the "cla" strds. The result will be a new strds consisting of maps with the slope value between every consecutive pair of maps in the original strds:

<source lang="bash">
t.rast.algebra expression="slope_cla = (cla[1]-cla[0])/8.0" basename=slope_cla
# we set 8 as fixed denominator, because products are 8-day compositions
</source>

We can then use any of the aggregation modules that we saw before to get the maximum or minimumm rate of change for different granularities.

===== Some known issues with t.rast.mapcalc =====

It has been observed that if the names of inputs and output space time datasets (partially) match each other, t.rast.mapcalc may not work properly (especially when several input strds are used). This is because the module uses a simple search and replace approach
to substitute the input STRDS with the corresponding map names. Eventualy, choosing a specific order for the input STRDS in the input parameter may reduce the risk of wrong substitution. Therefore, especially in operations involving several STRDS (with partially matching names) as inputs, it might be better to use t.rast.algebra that correctly recognizes spatio-temporal datasets. However, t.rast.algebra is still experimental and only available in development version.

=== Subseting and something else ===

{{cmd|t.rast.extract}} is another really useful module in temporal GRASS. It allows to extract a subset of a strds and store it in a different strds. You use the "where" condition to do the subset, but you can also specify a r.mapcalc sub-expression that performs operations on all maps of the selected subset. If no r.mapcalc expression is defined, the selected maps are simply registered in the new output strds.

Say we need to need to know in how many maps of the strds min values are below a threshold, and not only that, but you also want to know how many pixels per map meet that condition. Then, we can do

<source lang="bash">
t.rast.extract input=cla where="min <= '0.05'" output=cla_less_05 basename=cla_less_05 expression="if(cla<0.05,1,null())"
</source>

to extract those maps with a minimum value lower than 0.05, and in the same step put 1 in every cell meeting the criterium and null everywhere else. To get a count of maps and pixels meeting the condition we may use:

<source lang="bash">
# univariate stistics to get the count per map
t.rast.univar input=cla_less_05
# count map of cells with min value < 0.05
t.rast.series input=cla_less_05 output=count_cla_less_05 method=count
</source>

=== Importing / Exporting strds ===

Say we now need to do some processing in R (e.g.: run [http://menugget.blogspot.com.ar/2012/10/dineof-data-interpolating-empirical.html DINEOF] to fill gaps in data), so we need to export our strds, hence we use {{cmd|t.rast.export}}.

<source lang="bash">
t.rast.export input=cla output=cla4R compression=gzip
</source>

After we have done our analysis we can import the whole strds back to GRASS again, exporting it from R with read/write.tgrass from [http://cran.r-project.org/web/packages/spacetime/index.html spacetime] R package and using {{cmd|t.rast.import}} in GRASS.

<source lang="bash">
t.rast.import input=cla_from_R.tar.gz output=cla_from_R basename=new_map directory=/tmp
</source>

For a full example of how to handle raster time series between GRASS and R, please visit the [https://grasswiki.osgeo.org/wiki/Temporal_data_processing/raster_time_series_grass_R_statistics time series GRASS-R Statistics] wiki. The example is based on North Carolina climate location [http://courses.ncsu.edu/mea592/common/media/02/nc_climate_spm_2000_2012.zip] and includes the following steps:

# Exporting the strds out of GRASS
# Importing into R
# Re-formating the data
# Running [http://modb.oce.ulg.ac.be/mediawiki/index.php/DINEOF DINEOF]
# Re-constructing the raster time series
# Exporting out of R, and
# Importing the gap-filled new strds into GRASS.

=== Neighborhood analysis ===

{{cmd|t.rast.neighbors}} performs a neighborhood analysis for each map in a space time raster dataset. This module supports a subset of the options already available in r.neighbors. Both, size of the neighborhood and aggregation method can be chosen. As an example, we'll estimate mean Cl-a concentration in a 3x3 neighborhood for every map in the strds.

<source lang="bash">
t.rast.neighbors input=cla output=cla_smoth base=cla_smooth method=average size=3
</source>

=== Filling and reconstructing time series data with gaps ===

==== Harmonic ANalysis of Time Series - HANTS ====

{{AddonCmd|r.hants}} is an add-on not strictly within the temporal modules, but really useful when working with this kind of data, particularly with gappy data. Here's a simple example for generating a list of temporally ordered maps to use as input in '''r.hants''', running HANTS and getting a map of dominant frequencies, afterwards. See the manual for further information on parameter setting and explanations.

<source lang="bash">
# use t.rast.list to create a list of temporally ordered maps
t.rast.list input=cla order=start_time columns=name -u > map_list.csv

# a HANTS run using the list of maps
r.hants -l file=map_list.csv nf=5 fet=0.1 dod=11 base_period=46 suffix=_hants amplitude=amp_hants phase=pha_hants

# dominant frequency map (0 means the dominant freq is 1, 1 that dominant freq is 2, and so on...)
r.series input=`g.list raster pattern=amp_hants* separator=comma` output=dominant_freq_hants method=max_raster
</source>

==== Local Weighted Regression - LWR ====

Another very useful add-on for the reconstruction of gappy time series, such as those from remote sensing imagery, is {{AddonCmd|r.series.lwr}}. This module performs a local weighted regression (LWR) of the input time series of maps in order to estimate missing values and identify (and remove) outliers. See the manual for further information and explanations.

<source lang="bash">
# use same list as before
t.rast.list input=cla order=start_time columns=name -u > map_list.csv

# run r.series.lwr
r.series.lwr file=map_list.csv suffix=_lwr order=2 weight=tricube range=0.0,65.0 dod=16 fet=0.1 -l
</source>

==== Estimate Mean Absolute Error (MAE) ====

One of the measures used to assess how close forecasts or predictions might be to the observed values is the
[https://en.wikipedia.org/wiki/Mean_absolute_error mean absolute error]. We will use it to evaluate the performance of HANTS and LWR.

First, we will rebuild our time series with the corresponding output maps of r.hants and r.series.lwr. Here, only showed for LWR.

<source lang="bash">
# re-build time series
t.create type=strds temporaltype=absolute output=cla_lwr \
title="LWR output for Chl-a" \
description="MODIS Aqua L3 Chl-a 8-day 4km 2010-2013. Reconstruction with r.series.lwr"

# create list with filenames to parse
g.list type=raster pattern="*lwr" output=names_list

# parse filenames, convert YYYY-DOY to YYYY-MM-DD and write file to use in t.register
for mapname in `cat names_list` ; do
year_start=`echo ${mapname:1:4}`
doy_start=`echo ${mapname:5:3}`
year_end=`echo ${mapname:8:4}`
doy_end=`echo ${mapname:12:3}`
# convert YYYY-DOY to YYYY-MM-DD
#BEWARE: leading zeros make bash assume the number is in base 8 system, not base 10!
doy_start=`echo "$doy_start" | sed 's/^0*//'`
doy_end=`echo "$doy_end" | sed 's/^0*//'`
START_DATE=`date -d "${year_start}-01-01 +$(( ${doy_start} - 1 ))days" +%Y-%m-%d`
END_DATE=`date -d "${year_end}-01-01 +$(( ${doy_end} ))days" +%Y-%m-%d`
# print mapname, start and end date
echo "$mapname|$START_DATE|$END_DATE" >> map_list_start_and_end_time.txt
done

# register maps in strds
t.register input=cla_lwr type=raster file=map_list_start_and_end_time.txt
</source>

Now, we will estimate the MAE and use it to compare both methods. Again, only the example for LWR is showed.

<source lang="bash">
# obtain a strds of the absolute differences between predicted and observed Chl-a values
t.rast.algebra -n basename=lwr_minus_orig \
expression="abs_lwr_minus_orig = abs(cla_lwr - cla)"

# sum of the absolute differences (numerator) and count of non-null maps per pixel (denominator)
t.rast.series input=abs_lwr_minus_orig output=sum_abs_lwr_minus_orig method=sum
t.rast.series input=abs_lwr_minus_orig output=count_abs_lwr_minus_orig method=count

# MAE for LWR
r.mapcalc expression="mae_lwr = sum_abs_lwr_minus_orig / count_abs_lwr_minus_orig"

# remove intermediate strds and maps
t.remove -rf abs_lwr_minus_orig
g.remove -f type=raster name=sum_abs_lwr_minus_orig,count_abs_lwr_minus_orig
</source>

Let's now compare the predictions generated by both methods, HANTS and LWR, and the corresponding mean absolute error maps.

{|
|-
| [[Image:cla_vs_cla_hants.png|center|thumb|560px|Original Chl-a time series vs HANTS output.]] || [[Image:cla_vs_cla_lwr.png|center|thumb|560px|Original Chl-a time series vs LWR output.)]]
|-
| [[Image:mae_hants_map.png|center|thumb|560px|MAE of HANTS predictions.]] || [[Image:mae_lwr_map.png|center|thumb|560px|MAE of LWR predictions.]]
|}

=== Extract strds values for points in a vector ===

Let's say we need to extract values of the strds to check the behaviour of HANTS at given locations and compare it with the original values, you can graphically do that with g.gui.tplot. But, if you need to take data outside GRASS, say, read it into R and do some other analysis (See [http://grasswiki.osgeo.org/wiki/R_statistics/rgrass7#Reading_in_data R_Statistics wiki page]) you may use {{cmd|v.what.strds}}. This module retrieves raster values from a given strds to the attribute table of a point vector map.

<source lang="bash">
# original data
v.what.strds --overwrite input=points_cla strds=cla output=points_cla_out
v.db.select map=points_cla_out file=ts_points_cla.csv

# HANTS' reconstructed data (several runs)
for i in `seq 1 13` ; do
v.what.strds --overwrite input=points_cla strds=cla_hants_${i} output=points_hants_${i}_out
v.db.select map=points_cla file=ts_points_hants_${i}.csv
done
</source>

Another option would be to use [https://grass.osgeo.org/grass71/manuals/t.rast.what.html t.rast.what] that samples a space time raster dataset at specific vector point coordinates and write the output to stdout or text file using different layouts. You don't write the vector's attribute table in this case.

<source lang="bash">
t.rast.what points=points strds=cla output=cla_points.csv null_value=NA separator=comma
</source>

=== TODO ===

(add some more)

== FAQ ==

=== Best practice data organization ===

When it come to time series, often thousands of maps are involved. In order to keep control, here some suggestion

* separate original data from derived aggregates: store them in separate mapsets
* for specific projects, maintain them in individual mapsets. You can access the STDS stored in other mapsets (same location) through
t.some.module input=my_strds@the_other_mapset output=result ...
as long as mapsets are indeed accessible, i.e.: in the mapset's search path (See {{cmd|g.mapsets}}).

=== Best practice multi user management ===

GRASS GIS supports multi-user data management and offers the possibility that several users work in the same location within their own mapsets. Data can be read from other mapsets but not edited there. The same principle applies to space time datasets (STDS), i.e. in case of having a collection of maps corresponding to a time series in a dedicated mapset, you need to also create the STDS therein (See {{cmd|t.create}} and {{cmd|t.register}}), so it is visible to other users in other mapsets when they use {{cmd|t.list}}, for example.

=== Use of ''where'' parameter (from ticket [https://trac.osgeo.org/grass/ticket/2270 #2270]) ===

Pay attention when using the "where" option in t.* modules. In some occasions it may not yield the expected results. Given the backend database chosen for TGRASS implementation, you may not be selecting all maps you think if you forget to set time along with date in the where clause. For example, let's say we need to select maps from 2005-03-01 until 2005-05-31 included. You would think the following command would do the job:

<source lang="bash">
t.rast.list daily_temp where="start_time > '2005-03-01' and start_time <= '2005-05-31'"
</source>

but, no... The last map in the list is:

temp_0516|pruebas|2005-05-30 00:00:00|2005-05-31 00:00:00

which would be equivalent to:

<source lang="bash">
t.rast.list daily_temp where="start_time > '2005-03-01' and start_time < '2005-05-31'"
</source>

This is a product of sqlite. Therefore, if you need the last date included in your selection (map from 2005-05-31 in our example), you need to set time too.

<source lang="bash">
t.rast.list daily_temp where="start_time > '2005-03-01' and start_time <= '2005-05-31 00:00:00'" | tail -n1
temp_0517|pruebas|2005-05-31 00:00:00|2005-06-01 00:00:00
</source>

=== Aggregation with defined granularity ===

''Q: I need to aggregate a strds with a granularity of 1 year, but shifting the start day one month in each run, i.e.: changing the start_time to 2003-02-01, 2003-03-01, 2003-04-01 and so on... My question is: if i recursively change start_time with the 'where' parameter, will the module t.rast.aggregate "aggregate" to the next february, march, april (what i'd wish) or just till the end of 2003?''

A: If you specify a granularity of a year, then the start time to perform the aggregation will always be shifted to the 1st January of the current year and the end time the 1st January of the next year (eg. 2002-01-01 - 2003-01-01). If you wish to aggregate a full year but shifting one month forward then simply use a granularity of 12 months.

=== Listing maps with specific start month ===

''Q: I have a strds with 506 maps that correspond to 8-day composite products. I need to sequentially list all maps which "start_month" is January, February and so on... to use them as input in {{cmd|r.series}} (or {{cmd|t.rast.series}). How can I achieve that?''

A: You can use the [https://www.sqlite.org/lang_datefunc.html datetime functionality of SQLite] to perform this task, this should work for January:

<source lang="bash">
t.rast.list input=cla_null_mayor65 \
where='start_time >= datetime(start_time, "start of year") and start_time <= datetime(start_time, "start of year", "1 month")'
</source>

=== Aggregation of seasonal data ===

''Q: How can I calculate average seasonal temperature starting from a daily temperatures temporal dataset?''

A: Use {{cmd|t.rast.aggregate.ds}}, the input is the daily strds, the sampling stds should have seasonal intervals. Then use average as method. The output will have seasonal aggregated temperatures. For a detailed workflow see: [https://grasswiki.osgeo.org/wiki/Temporal_data_processing/seasonal_aggregation seasonal aggregation example].

=== Aggregation of seasonal data using time ranges ===

A way to aggregate seasons from daily data without granularity but by using time ranges is shown here. The issue is to include for the season calculation also a month from the previous year. This can be addressed by some datetime calculations in SQLite:

<source lang="bash">
# We assume to have daily temperature data in DB "temp_daily_average"

# loop over seasons, generate aggregates:
### 'start_time' and 'end_time' are columns in TGRASS

for year in `seq 2004 2014` ; do

# we consider also a month of the previous year
for month in "12 01 02" "03 04 05" "06 07 08" "09 10 11" ; do
set -- $month ; echo $1 $2 $3

prevyear=$year
if [ $1 -eq 12 ] ; then
prevyear=`expr $year - 1`
fi

# prepare query strings in SQLite
MYSTART=`echo "SELECT strftime(datetime('${prevyear}-${1}-01'));" | sqlite3`
MYEND=`echo "SELECT strftime(datetime('${prevyear}-${1}-01','+3 month'));" | sqlite3`

# Debugging only, to see what it does:
echo "---- Querying ${prevyear}-${1}-01 ... ${year}-${3}-end:"
# we use start_time and end_time to get the proper time range
t.rast.list input=temp_daily_average@modis_lst_reconstructed_europe_daily \
where="start_time >= '$MYSTART' AND end_time <= '$MYEND'" > list_${prevyear}_${1}_${3}.csv

head -n 3 list_${prevyear}_${1}_${3}.csv
echo "..."
tail -n 3 list_${prevyear}_${1}_${3}.csv
echo "======="
rm -f list_${prevyear}_${1}_${3}.csv

# calculate aggregates:
method="average" # median mode minimum maximum stddev

t.rast.series input=temp_daily_average@modis_lst_reconstructed_europe_daily \
where="start_time >= '$MYSTART' AND end_time <= '$MYEND'" \
output=temp_${method}_${prevyear}_${1}
done
done
</source>

Much simpler '''alternative''' (which runs in parallel on multi-cores):
<source lang="bash">
t.rast.aggregate input=A output=B basename=b where="start_time >= '2004-03-01 00:00:00'" granularity="3 months"
</source>

== Online tutorials and courses ==

* [http://www.geostat-course.org/Topic_Gebbert The temporal GRASS GIS framework: Introduction and application] by Sören Gebbert (2012)
* [http://fatra.cnr.ncsu.edu/temporal-grass-workshop/ Spatio-temporal data handling and visualization in GRASS GIS] - FOSS4G 2014 workshop by Vaclav Petras, Anna Petrasova, Helena Mitasova, Markus Neteler (2014)
* [http://www.openniche.org/processing-era-interim-dataset-in-ncdf-using-gdal-and-grass-gis/ Processing ERA-Interim dataset in netcdf using GDAL and GRASS GIS] by Sajid Pareeth (2015)
* CZ: [http://training.gismentors.eu/grass-gis-pokrocily/tgrass/index.html Úvod do časoprostorových analýz] (Introduction to space-time analysis) by Martin Landa (2015)

== References ==
* Gebbert, S., Pebesma, E. 2014. ''TGRASS: A temporal GIS for field based environmental modeling''. Environmental Modelling & Software 53, 1-12 ([http://dx.doi.org/10.1016/j.envsoft.2013.11.001 DOI]) - [http://ifgi.uni-muenster.de/~epebe_01/tgrass.pdf preprint PDF]
* Gebbert, S., Pebesma, E. 2017. ''The GRASS GIS temporal framework''. International Journal of Geographical Information Science 31, 1273-1292 ([http://dx.doi.org/10.1080/13658816.2017.1306862 DOI])

== See also ==
* Introduction: [http://www.slideshare.net/Luis_de_Sousa/presentation-soeren GRASS as a Temporal GIS] by Sören Gebbert (slides)
* [https://archive.org/details/TheTemporalGrassGisFrameworkIntroductionAndApplication# The temporal GRASS GIS framework: Introduction and application] Video of Sören Gebbert's presentation in [http://www.geostat-course.org/Muenster_2012 GEOSTAT course] 2012
* [http://grass.osgeo.org/grass72/manuals/temporalintro.html Introduction to temporal modules] (GRASS GIS manual)
* [http://grass.osgeo.org/grass72/manuals/temporal.html Available temporal modules] (GRASS GIS manual)
* [http://trac.osgeo.org/grass/wiki/Grass7/TemporalExtension Temporal Extension] (background info in trac)
* [[GRASS GSoC 2013 Temporal GIS Algebra for raster and vector data in GRASS]]

[[Category: Documentation]]
[[Category: Tutorial]]
[[Category: Temporal]]

LANDSAT

2017-06-22T07:54:40Z

⚠️NikosA: Updated text in link to band designations

== Data Availability==

Landsat imagery can be obtained from [http://landsat.usgs.gov/ USGS' LANDSAT archive]:

* [http://glovis.usgs.gov USGS GloVis]
* [http://earthexplorer.usgs.gov/ EarthExplorer]
* [http://earthexplorer.usgs.gov/bulk/help Bulk Download Orders] using the [http://earthexplorer.usgs.gov/bulk/ Bulk Download Application]
* [http://landsatlook.usgs.gov/ LandsatLook Viewer]
* [https://landsatonaws.com/ Landsat on Amazon Web Services]

See also

* the [[Global datasets]] wiki page
* [http://landsat.usgs.gov/band_designations_landsat_satellites.php Landsat satellites band designations].

== Modules overview ==

GRASS GIS features a complete set of native modules and various add-ons for pre- and post-processing Landsat satellite imagery. The following lists offer an overview of related modules and add-ons.

=== Generic modules applicable to Landsat ===

* {{cmd|d.rgb}} → display 3-band data
* {{cmd|r.composite}} → flatten 3-bands of data into a single image (lossy, maybe used in combination with {{cmd|i.landsat.rgb|version=64}} (GRASS 6.x) | {{cmd|i.colors.enhance|version=70}} (GRASS 7.x)
* {{cmd|i.atcorr}} → correct top of atmosphere to surface reflectance - see also the [[Atmospheric correction]] wiki page
* {{cmd|i.topo.corr}} → used to topographically correct reflectance from imagery files, e.g. obtained with {{cmd|i.landsat.toar}}, using a sun illumination terrain model
* {{cmd|i.oif}} → calculate the 3 bands showing the greatest difference (for use as R,G,B bands)
* {{AddonCmd|i.histo.match}} (addon) → histogram matching of two or more raster maps (in grass 7) ''Note, the module works with integer values and does not accept the "." character as part of the raster map's name!''

=== Landsat specific modules ===

* {{cmd|i.landsat.rgb|version=64}} (GRASS 6.x) | {{cmd|i.colors.enhance|version=70}} (GRASS 7.x) → auto-enhance colors
* {{cmd|i.landsat.toar}} → convert DN to top of atmosphere radiance
* {{cmd|i.landsat.acca}} → cloud cover assessment
* {{cmd|i.tasscap}} → Performs Tasseled Cap (Kauth Thomas) transformation

=== Landsat specific GRASS AddOns ===

* {{AddonCmd|i.landsat.trim|version=64}} → trims border fringes for each band separately or with the MASK where coverage exists for all bands
* {{AddonCmd|i.landsat.dehaze|version=64}} → haze removal
* {{AddonCmd|i.landsat8.qc|version=70}} → Reclass Landsat8 QA band according to acceptable pixel quality as defined by the user
* {{AddonCmd|i.landsat8.swlst|version=70}} → Practical split-window algorithm estimating Land Surface Temperature from Landsat 8 OLI/TIRS imagery
* {{AddonCmd|i.wi|version=70}} → Calculate different water indices

== Pre-Processing ==

=== Overview ===

Typically, pre-processing Landsat imagery comprises the following steps:

# '''import''' in the database → {{cmd|r.in.gdal}}
# geometrically & orthometrically correct imagery
#* already done for L1T products, read more at [http://landsat.usgs.gov//Landsat_Processing_Details.php USGS' Landsat Information Products] webpage
# optionally, automatically '''cut-off border fringes''' → {{AddonCmd|i.landsat.trim|version=64}}
#* of course one can use the official WRS2 Path/Row vector tiles to manually trim border fringes
# optionally, '''denoise''' for obvious/intensive salt & pepper effects, stripes, etc.
#* for example by applying Principal Components Analysis as a denoising technique → {{cmd|i.pca}} ''<<< Re-order this step?''
# '''convert''' the '''Digital Numbers''' (DNs) to '''Top-of-Atmosphere Radiances/Reflectances''' (ToARs) → {{cmd|i.landsat.toar}}
# optionally, '''correct for atmospheric effects''' → {{cmd|i.atcorr}}
#* that is, accounting for distorting atmospheric effects and estimating actual reflectances as they would have been measured on the ground
#* also described as conversion to Top-of-Canopy Reflectances (ToCRs)
# '''assessing cloud cover''' → {{cmd|i.landsat.acca}}
#* optionally, detect and remove clouds shadows as well
# '''topographically normalise''' imagery → {{cmd|i.topo.corr}}
#* also known as topographic correction, that is, accounting for illumination differences due to the acquisition's geometry
# '''radiometrically normalise''' → one approach via {{AddonCmd|i.histo.match}} (in '''grass 7''', addon), also known as relative radiometric normalisation -- one approach is the ''histogram matching'' technique of two or more raster maps
#* '''ToDo/More techniques?'''

See also
* [http://courses.neteler.org/processing-landsat8-data-in-grass-gis-7/ Processing Landsat 8 data in GRASS GIS 7: Import and visualization]

=== Importing data ===

Importing Landsat spectral bands in GRASS GIS' data base can be done both from the Graphical User Interface or from the command line

# Open GRASS GIS, select ''Location Wizard'' in order to create a new location from georeferenced file
# Use the ''Import file tool'', or the {{cmd|r.in.gdal}} module, to import the GeoTIFF files into GRASS GIS.

See also [http://grasswiki.osgeo.org/wiki/LANDSAT#Automated_data_import Automated data import] below.

==== Notes ====

# Most Landsat scenes are delivered in ''north-is-up'' orientation, hence the import process is straightforward.
# If you get <source lang="bash" enclose="none">ERROR: Input map is rotated - cannot import.</source>, the image must be first rotated externally, applying the rotation info stored in the metadata field of the raster image file. For example, the <code>gdalwarp</code> software can be used to transform the map to North-up (note, there are several <code>gdalwarp</code> parameters to select the resampling algorithm):

<ul>
<source lang="bash">
gdalwarp rotated.tif northup.tif
</source>
</ul>

==== Hint: Minimal disk space copies ====

Here's a little trick using {{cmd|r.reclass}} to rename maps (for example, from ''L71074092_09220040924_B'''10''''' to ''L71074092_09220040924_B'''1''''') without touching the data or wasting disk space:

<source lang="bash">
BASE=L71074092_09220040924

for BAND in 10 20 30 40 50 61 70 80; do
BAND1st=`echo $BAND | sed -e 's/0$//'`
r.reclass in="${BASE}_B$BAND" out=$BASE.$BAND1st << EOF
* = *
EOF
done
</source>

==== Automated data import ====

The following ''Python'' script imports Landsat imagery into GRASS' data base. Specifically, the script

* creates an independent Mapset for each Landsat scene
* imports and renames bands of a scene as <tt>B<id></tt>, e.g. B10, B20, ..., B80.
* additionaly sets up the timestamp based on MTL metadata file

'''Note,''' the (newest) official naming pattern for Landsat scenes -- explained in [https://lta.cr.usgs.gov/landsat_dictionary.html USGS' Landsat Data Dictionary] as the [https://lta.cr.usgs.gov/landsat_dictionary.html#entity_id Landsat Scene Identifier] -- and all individual bands that compose a scene -- bands have a suffix which is like _B10, _B20, _B30, etc. -- differ from what some Landsat specific modules expect. For example, the modules {{cmd|i.landsat.toar}} and {{cmd|i.landsat.acca}} expect the bands to follow a naming pattern such as "scenename.1, .2, .3", etc.

To use the script save it as <source lang="bash" enclose="none">import_landsat.py</source> file and make sure it is granted the execution permission.

<source lang=python>
#!/usr/bin/python

import os
import sys
import glob
import grass.script as grass

def get_timestamp(mapset):
try:
metafile = glob.glob(mapset + '/*MTL.txt')[0]
except IndexError:
return

result = dict()
try:
fd = open(metafile)
for line in fd.readlines():
line = line.rstrip('\n')
if len(line) == 0:
continue
if any(x in line for x in ('DATE_ACQUIRED', 'ACQUISITION_DATE')):
result['date'] = line.strip().split('=')[1].strip()
finally:
fd.close()

return result

def import_tifs(mapset):
meta = get_timestamp(mapset)
for file in os.listdir(mapset):
if os.path.splitext(file)[-1] != '.TIF':
continue
ffile = os.path.join(mapset, file)
if ('VCID') in ffile:
name = "".join((os.path.splitext(file)[0].split('_'))[1::2])
else:
name = os.path.splitext(file)[0].split('_')[-1]
if len(name) == 3 and name[-1] == '0':
band = int(name[1:2])
elif len(name) == 3 and name[-1] != '0':
band = int(name[1:3])
else:
band = int(name[-1:])
grass.message('Importing %s -> %s@%s...' % (file, name, mapset))
grass.run_command('g.mapset',
flags = 'c',
mapset = mapset,
quiet = True,
stderr = open(os.devnull, 'w'))
grass.run_command('r.in.gdal',
input = ffile,
output = name,
quiet = True,
overwrite = True,
title = 'band %d' % band)
if meta:
year, month, day = meta['date'].split('-')
if month == '01':
month = 'jan'
elif month == '02':
month = 'feb'
elif month == '03':
month = 'mar'
elif month == '04':
month = 'apr'
elif month == '05':
month = 'may'
elif month == '06':
month = 'jun'
elif month == '07':
month = 'jul'
elif month == '08':
month = 'aug'
elif month == '09':
month = 'sep'
elif month == '10':
month = 'oct'
elif month == '11':
month = 'nov'
elif month == '12':
month = 'dec'

grass.run_command('r.timestamp',
map = name,
date = ' '.join((day, month, year)))

def main():
if len(sys.argv) == 1:
for directory in filter(os.path.isdir, os.listdir(os.getcwd())):
import_tifs(directory)
else:
import_tifs(sys.argv[1])

if __name__ == "__main__":
main()
</source>

'''Example of usage'''

After having collected the Landsat scenes of interest in one directory, the script can be used as follows:

* <source lang="python" enclose="none">./import_landsat.py</source> → the script will walk through a ''pool'' directory that contains unique Landsat scene directories (e.g. three directories named after the official naming pattern: LT51800342011158MOR00 LT51810352009079MTI00 LT51820352009326MTI00) and import all bands of each individual scene in their own Mapset
* <source lang="python" enclose="none">./import_landsat.py LM41890261983200FFF03</source> → the scrip will import only bands of the specified Landsat scene directory

== Post-Processing ==

=== Natural color composites ===

Creating natural (also known as true-) color composites, can be done by

a) using {{cmd|i.landsat.rgb|version=64}} (GRASS 6.x) | {{cmd|i.colors.enhance|version=70}} (GRASS 7.x) to automatically balance the colors for the Red, Green and Blue bands

or

b) equalizing the colors on each of the Red, Green and Blue bands

<ul>
<source lang="bash">
r.colors -e map=band1 color=grey
</source>
</ul>

'''Composites''' can then be produced with the {{cmd|r.composite}} module.

'''Resetting the color tables''' of all bands back to normal greyscale can be done with a for loop:

<source lang="bash">
BASE=L71074092_09220040924

for map in `g.mlist pat="$BASE.[0-8]*"` ; do
r.colors $map color=grey255
done
</source>

See also:
* [http://courses.neteler.org/processing-landsat-8-data-in-grass-gis-7-rgb-composites-and-pan-sharpening/ Processing Landsat 8 data in GRASS GIS 7: RGB composites and pan sharpening]

=== Create a MASK to only show data where coverage exists for all bands ===

<source lang="bash">
BASE=L71074092_09220040924

g.region rast=$BASE.1
r.series in=`g.mlist pat="$BASE.[0-8]*" sep=,` -n out=$BASE.thresh method=threshold thresh=1
r.mapcalc "$BASE.mask = if(isnull($BASE.thresh))"
g.remove $BASE.thresh
</source>

=== Calculate Top-of-Atmosphere Reflectance and band-6 Temperature ===

Calculate ''Top-of-Atmosphere'' reflectance and band-6 temperature using the {{cmd|i.landsat.toar}} module. For details, refer to USGS' [http://landsat.usgs.gov/Landsat_Metadata_Changes.php Landsat Filename and Metadata Changes] dedicated page. For Landsat 8, see also [http://landsat.usgs.gov/about_LU_Vol_7_Issue_4.php#3a here].
<source lang="bash">
i.landsat.toar input_prefix=$BASE output_prefix=${BASE}_toar metfile=${BASE}_MTL.txt
</source>

Note, the resulting temperature map is in Kelvin:
<source lang="bash">
# convert to degree Celsius
r.mapcalc "$BASE.temp_celsius = ${BASE}_toar.6 - 273.15"
r.info -r $BASE.temp_celsius
</source>

=== Haze removal ===

Simple haze removal can be done with {{AddonCmd|i.landsat.dehaze|version=64}}. This addons applies a bandwise haze correction using tasscap4 (haze) and linear regression.

During the 2000s, prior acquired Landsat data were reprocessed to LPGS (Level 1 Product Generation System). Seems that with this level of processing, haze effect in raw data was removed and now is sufficient to apply {{Cmd|i.colors.enhance}} to obtain a good image with high contrast.

=== Atmospheric correction ===

See [[Atmospheric correction]]

=== Cloud identification ===

Identify clouds in the image with {{cmd|i.landsat.acca}}:
<source lang="bash">i.landsat.acca -f input_prefix=226_62_toar. output=226_62.acca_cloudmask</source>

Mask out the clouds:
<source lang="bash">r.mapcalc "MASK = if(isnull($BASE.acca_cloudmask))"</source>

== Download sample data ==

=== Preprocessed Landsat-7 data for North Carolina ===

The [[Sample_datasets|North Carolina 2008 sample dataset]] comes with 3 different Landsat scenes:
: (''Wake County -- path: 16 row: 35 for various dates'')

The above import efforts are not needed since the data are already in a GRASS location.

=== Landsat-5: Oct 14, 1987 ===

;Glovis download: LT50160351987287XXX08
: http://edcsns17.cr.usgs.gov/cgi-bin/EarthExplorer/run-phtml/results/download.phtml?node=GV&ordered=LT50160351987287XXX08&dataset_name=LANDSAT_TM



Provided metadata:
IMAGE_ID=P016R35_5T871014
PATH=16
ROW=35
DATE=10/14/87
PLATFORM=LANDSAT5

=== Landsat-7: Mar 31, 2000 ===

;Glovis download: LE70160352000091EDC00
: http://edcsns17.cr.usgs.gov/cgi-bin/EarthExplorer/run-phtml/results/download.phtml?node=GV&ordered=LE70160352000091EDC00&dataset_name=LANDSAT_ETM

Values in the metadata below indicate that the version provided with the NC 2008 dataset's production date was after July 1, 2000, and that the channel gains were <tt>HHHLHLHHL</tt>.

Convert DNs to radiance/temperatures:<source lang="bash">
GRASS> i.landsat.toar -v band=lsat7_2000 sensor=7 date=2000-03-31 \
product_date=2000-07-02 solar_elevation=51.5246529 gain=HHHLHLHHL</source>

Provided metadata:
SPACECRAFT_ID=Landsat7
SENSOR_ID=ETM+
'''ACQUISITION_DATE=2000-03-31'''
WRS_PATH=16
CPF_FILE_NAME=L7CPF20000101_20000331_12
SUN_AZIMUTH=139.6033279
'''SUN_ELEVATION=51.5246529'''
LMAX_BAND1=191.600
LMIN_BAND1=-6.200
LMAX_BAND2=196.500
LMIN_BAND2=-6.400
LMAX_BAND3=152.900
LMIN_BAND3=-5.000
LMAX_BAND4=241.100
LMIN_BAND4=-5.100
LMAX_BAND5=31.060
LMIN_BAND5=-1.000
LMAX_BAND61=17.040
LMIN_BAND61=0.000
LMAX_BAND62=12.650
LMIN_BAND62=3.200
LMAX_BAND7=10.800
LMIN_BAND7=-0.350
LMAX_BAND8=243.100
LMIN_BAND8=-4.700
QCALMAX_BAND1=255.0
QCALMIN_BAND1=1.0
QCALMAX_BAND2=255.0
QCALMIN_BAND2=1.0
QCALMAX_BAND3=255.0
QCALMIN_BAND3=1.0
QCALMAX_BAND4=255.0
QCALMIN_BAND4=1.0
QCALMAX_BAND5=255.0
QCALMIN_BAND5=1.0
QCALMAX_BAND61=255.0
QCALMIN_BAND61=1.0
QCALMAX_BAND62=255.0
QCALMIN_BAND62=1.0
QCALMAX_BAND7=255.0
QCALMIN_BAND7=1.0
QCALMAX_BAND8=255.0
QCALMIN_BAND8=1.0

=== Landsat-7: May 24, 2002 ===

;Glovis download: LE70160352002144EDC00
: http://edcsns17.cr.usgs.gov/cgi-bin/EarthExplorer/run-phtml/results/download.phtml?node=GV&ordered=LE70160352002144EDC00&dataset_name=LANDSAT_ETM



Provided metadata: (`p016r035_7x20020524.met`)
SPACECRAFT_ID=Landsat7
SENSOR_ID=ETM+
'''ACQUISITION_DATE=2002-05-24'''
WRS_PATH=016
WRS_ROW=035
SUN_AZIMUTH=120.8810347
'''SUN_ELEVATION=64.7730999'''
QA_PERCENT_MISSING_DATA=0
CLOUD_COVER=0
CPF_FILE_NAME=L7CPF20020401_20020630_03
LMAX_BAND1=191.600
LMIN_BAND1=-6.200
LMAX_BAND2=196.500
LMIN_BAND2=-6.400
LMAX_BAND3=152.900
LMIN_BAND3=-5.000
LMAX_BAND4=241.100
LMIN_BAND4=-5.100
LMAX_BAND5=31.060
LMIN_BAND5=-1.000
LMAX_BAND61=17.040
LMIN_BAND61=0.000
LMAX_BAND62=12.650
LMIN_BAND62=3.200
LMAX_BAND7=10.800
LMIN_BAND7=-0.350
LMAX_BAND8=243.100
LMIN_BAND8=-4.700
QCALMAX_BAND1=255.0
QCALMIN_BAND1=1.0
QCALMAX_BAND2=255.0
QCALMIN_BAND2=1.0
QCALMAX_BAND3=255.0
QCALMIN_BAND3=1.0
QCALMAX_BAND4=255.0
QCALMIN_BAND4=1.0
QCALMAX_BAND5=255.0
QCALMIN_BAND5=1.0
QCALMAX_BAND61=255.0
QCALMIN_BAND61=1.0
QCALMAX_BAND62=255.0
QCALMIN_BAND62=1.0
QCALMAX_BAND7=255.0
QCALMIN_BAND7=1.0
QCALMAX_BAND8=255.0
QCALMIN_BAND8=1.0

== LANDSAT Image classification ==

See [[Image classification]]

== Time series analysis ==

See [[Time series]]

== Sources ==

* [http://landsat.usgs.gov/tools_access_all_faqs.php Frequently Asked Questions about the Landsat Missions]
* [https://landsat8portal.eo.esa.int/portal/ European LANDSAT 8 data in near-realtime]
* [http://courses.neteler.org/processing-landsat8-data-in-grass-gis-7/ Processing Landsat 8 data in GRASS GIS 7: Import and visualization]
* [http://courses.neteler.org/processing-landsat-8-data-in-grass-gis-7-rgb-composites-and-pan-sharpening/ Processing Landsat 8 data in GRASS GIS 7: RGB composites and pan sharpening]

[[Category: Documentation]]
[[Category: FAQ]]
[[Category: Landsat]]
[[Category: Image processing]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Geodata]]

LANDSAT

2017-06-22T07:52:20Z

⚠️NikosA: Landsat on AWS

== Data Availability==

Landsat imagery can be obtained from [http://landsat.usgs.gov/ USGS' LANDSAT archive]:

* [http://glovis.usgs.gov USGS GloVis]
* [http://earthexplorer.usgs.gov/ EarthExplorer]
* [http://earthexplorer.usgs.gov/bulk/help Bulk Download Orders] using the [http://earthexplorer.usgs.gov/bulk/ Bulk Download Application]
* [http://landsatlook.usgs.gov/ LandsatLook Viewer]
* [https://landsatonaws.com/ Landsat on Amazon Web Services]

See also
* the [[Global datasets]] wiki page
* [http://landsat.usgs.gov/band_designations_landsat_satellites.php this page].

== Modules overview ==

GRASS GIS features a complete set of native modules and various add-ons for pre- and post-processing Landsat satellite imagery. The following lists offer an overview of related modules and add-ons.

=== Generic modules applicable to Landsat ===

* {{cmd|d.rgb}} → display 3-band data
* {{cmd|r.composite}} → flatten 3-bands of data into a single image (lossy, maybe used in combination with {{cmd|i.landsat.rgb|version=64}} (GRASS 6.x) | {{cmd|i.colors.enhance|version=70}} (GRASS 7.x)
* {{cmd|i.atcorr}} → correct top of atmosphere to surface reflectance - see also the [[Atmospheric correction]] wiki page
* {{cmd|i.topo.corr}} → used to topographically correct reflectance from imagery files, e.g. obtained with {{cmd|i.landsat.toar}}, using a sun illumination terrain model
* {{cmd|i.oif}} → calculate the 3 bands showing the greatest difference (for use as R,G,B bands)
* {{AddonCmd|i.histo.match}} (addon) → histogram matching of two or more raster maps (in grass 7) ''Note, the module works with integer values and does not accept the "." character as part of the raster map's name!''

=== Landsat specific modules ===

* {{cmd|i.landsat.rgb|version=64}} (GRASS 6.x) | {{cmd|i.colors.enhance|version=70}} (GRASS 7.x) → auto-enhance colors
* {{cmd|i.landsat.toar}} → convert DN to top of atmosphere radiance
* {{cmd|i.landsat.acca}} → cloud cover assessment
* {{cmd|i.tasscap}} → Performs Tasseled Cap (Kauth Thomas) transformation

=== Landsat specific GRASS AddOns ===

* {{AddonCmd|i.landsat.trim|version=64}} → trims border fringes for each band separately or with the MASK where coverage exists for all bands
* {{AddonCmd|i.landsat.dehaze|version=64}} → haze removal
* {{AddonCmd|i.landsat8.qc|version=70}} → Reclass Landsat8 QA band according to acceptable pixel quality as defined by the user
* {{AddonCmd|i.landsat8.swlst|version=70}} → Practical split-window algorithm estimating Land Surface Temperature from Landsat 8 OLI/TIRS imagery
* {{AddonCmd|i.wi|version=70}} → Calculate different water indices

== Pre-Processing ==

=== Overview ===

Typically, pre-processing Landsat imagery comprises the following steps:

# '''import''' in the database → {{cmd|r.in.gdal}}
# geometrically & orthometrically correct imagery
#* already done for L1T products, read more at [http://landsat.usgs.gov//Landsat_Processing_Details.php USGS' Landsat Information Products] webpage
# optionally, automatically '''cut-off border fringes''' → {{AddonCmd|i.landsat.trim|version=64}}
#* of course one can use the official WRS2 Path/Row vector tiles to manually trim border fringes
# optionally, '''denoise''' for obvious/intensive salt & pepper effects, stripes, etc.
#* for example by applying Principal Components Analysis as a denoising technique → {{cmd|i.pca}} ''<<< Re-order this step?''
# '''convert''' the '''Digital Numbers''' (DNs) to '''Top-of-Atmosphere Radiances/Reflectances''' (ToARs) → {{cmd|i.landsat.toar}}
# optionally, '''correct for atmospheric effects''' → {{cmd|i.atcorr}}
#* that is, accounting for distorting atmospheric effects and estimating actual reflectances as they would have been measured on the ground
#* also described as conversion to Top-of-Canopy Reflectances (ToCRs)
# '''assessing cloud cover''' → {{cmd|i.landsat.acca}}
#* optionally, detect and remove clouds shadows as well
# '''topographically normalise''' imagery → {{cmd|i.topo.corr}}
#* also known as topographic correction, that is, accounting for illumination differences due to the acquisition's geometry
# '''radiometrically normalise''' → one approach via {{AddonCmd|i.histo.match}} (in '''grass 7''', addon), also known as relative radiometric normalisation -- one approach is the ''histogram matching'' technique of two or more raster maps
#* '''ToDo/More techniques?'''

See also
* [http://courses.neteler.org/processing-landsat8-data-in-grass-gis-7/ Processing Landsat 8 data in GRASS GIS 7: Import and visualization]

=== Importing data ===

Importing Landsat spectral bands in GRASS GIS' data base can be done both from the Graphical User Interface or from the command line

# Open GRASS GIS, select ''Location Wizard'' in order to create a new location from georeferenced file
# Use the ''Import file tool'', or the {{cmd|r.in.gdal}} module, to import the GeoTIFF files into GRASS GIS.

See also [http://grasswiki.osgeo.org/wiki/LANDSAT#Automated_data_import Automated data import] below.

==== Notes ====

# Most Landsat scenes are delivered in ''north-is-up'' orientation, hence the import process is straightforward.
# If you get <source lang="bash" enclose="none">ERROR: Input map is rotated - cannot import.</source>, the image must be first rotated externally, applying the rotation info stored in the metadata field of the raster image file. For example, the <code>gdalwarp</code> software can be used to transform the map to North-up (note, there are several <code>gdalwarp</code> parameters to select the resampling algorithm):

<ul>
<source lang="bash">
gdalwarp rotated.tif northup.tif
</source>
</ul>

==== Hint: Minimal disk space copies ====

Here's a little trick using {{cmd|r.reclass}} to rename maps (for example, from ''L71074092_09220040924_B'''10''''' to ''L71074092_09220040924_B'''1''''') without touching the data or wasting disk space:

<source lang="bash">
BASE=L71074092_09220040924

for BAND in 10 20 30 40 50 61 70 80; do
BAND1st=`echo $BAND | sed -e 's/0$//'`
r.reclass in="${BASE}_B$BAND" out=$BASE.$BAND1st << EOF
* = *
EOF
done
</source>

==== Automated data import ====

The following ''Python'' script imports Landsat imagery into GRASS' data base. Specifically, the script

* creates an independent Mapset for each Landsat scene
* imports and renames bands of a scene as <tt>B<id></tt>, e.g. B10, B20, ..., B80.
* additionaly sets up the timestamp based on MTL metadata file

'''Note,''' the (newest) official naming pattern for Landsat scenes -- explained in [https://lta.cr.usgs.gov/landsat_dictionary.html USGS' Landsat Data Dictionary] as the [https://lta.cr.usgs.gov/landsat_dictionary.html#entity_id Landsat Scene Identifier] -- and all individual bands that compose a scene -- bands have a suffix which is like _B10, _B20, _B30, etc. -- differ from what some Landsat specific modules expect. For example, the modules {{cmd|i.landsat.toar}} and {{cmd|i.landsat.acca}} expect the bands to follow a naming pattern such as "scenename.1, .2, .3", etc.

To use the script save it as <source lang="bash" enclose="none">import_landsat.py</source> file and make sure it is granted the execution permission.

<source lang=python>
#!/usr/bin/python

import os
import sys
import glob
import grass.script as grass

def get_timestamp(mapset):
try:
metafile = glob.glob(mapset + '/*MTL.txt')[0]
except IndexError:
return

result = dict()
try:
fd = open(metafile)
for line in fd.readlines():
line = line.rstrip('\n')
if len(line) == 0:
continue
if any(x in line for x in ('DATE_ACQUIRED', 'ACQUISITION_DATE')):
result['date'] = line.strip().split('=')[1].strip()
finally:
fd.close()

return result

def import_tifs(mapset):
meta = get_timestamp(mapset)
for file in os.listdir(mapset):
if os.path.splitext(file)[-1] != '.TIF':
continue
ffile = os.path.join(mapset, file)
if ('VCID') in ffile:
name = "".join((os.path.splitext(file)[0].split('_'))[1::2])
else:
name = os.path.splitext(file)[0].split('_')[-1]
if len(name) == 3 and name[-1] == '0':
band = int(name[1:2])
elif len(name) == 3 and name[-1] != '0':
band = int(name[1:3])
else:
band = int(name[-1:])
grass.message('Importing %s -> %s@%s...' % (file, name, mapset))
grass.run_command('g.mapset',
flags = 'c',
mapset = mapset,
quiet = True,
stderr = open(os.devnull, 'w'))
grass.run_command('r.in.gdal',
input = ffile,
output = name,
quiet = True,
overwrite = True,
title = 'band %d' % band)
if meta:
year, month, day = meta['date'].split('-')
if month == '01':
month = 'jan'
elif month == '02':
month = 'feb'
elif month == '03':
month = 'mar'
elif month == '04':
month = 'apr'
elif month == '05':
month = 'may'
elif month == '06':
month = 'jun'
elif month == '07':
month = 'jul'
elif month == '08':
month = 'aug'
elif month == '09':
month = 'sep'
elif month == '10':
month = 'oct'
elif month == '11':
month = 'nov'
elif month == '12':
month = 'dec'

grass.run_command('r.timestamp',
map = name,
date = ' '.join((day, month, year)))

def main():
if len(sys.argv) == 1:
for directory in filter(os.path.isdir, os.listdir(os.getcwd())):
import_tifs(directory)
else:
import_tifs(sys.argv[1])

if __name__ == "__main__":
main()
</source>

'''Example of usage'''

After having collected the Landsat scenes of interest in one directory, the script can be used as follows:

* <source lang="python" enclose="none">./import_landsat.py</source> → the script will walk through a ''pool'' directory that contains unique Landsat scene directories (e.g. three directories named after the official naming pattern: LT51800342011158MOR00 LT51810352009079MTI00 LT51820352009326MTI00) and import all bands of each individual scene in their own Mapset
* <source lang="python" enclose="none">./import_landsat.py LM41890261983200FFF03</source> → the scrip will import only bands of the specified Landsat scene directory

== Post-Processing ==

=== Natural color composites ===

Creating natural (also known as true-) color composites, can be done by

a) using {{cmd|i.landsat.rgb|version=64}} (GRASS 6.x) | {{cmd|i.colors.enhance|version=70}} (GRASS 7.x) to automatically balance the colors for the Red, Green and Blue bands

or

b) equalizing the colors on each of the Red, Green and Blue bands

<ul>
<source lang="bash">
r.colors -e map=band1 color=grey
</source>
</ul>

'''Composites''' can then be produced with the {{cmd|r.composite}} module.

'''Resetting the color tables''' of all bands back to normal greyscale can be done with a for loop:

<source lang="bash">
BASE=L71074092_09220040924

for map in `g.mlist pat="$BASE.[0-8]*"` ; do
r.colors $map color=grey255
done
</source>

See also:
* [http://courses.neteler.org/processing-landsat-8-data-in-grass-gis-7-rgb-composites-and-pan-sharpening/ Processing Landsat 8 data in GRASS GIS 7: RGB composites and pan sharpening]

=== Create a MASK to only show data where coverage exists for all bands ===

<source lang="bash">
BASE=L71074092_09220040924

g.region rast=$BASE.1
r.series in=`g.mlist pat="$BASE.[0-8]*" sep=,` -n out=$BASE.thresh method=threshold thresh=1
r.mapcalc "$BASE.mask = if(isnull($BASE.thresh))"
g.remove $BASE.thresh
</source>

=== Calculate Top-of-Atmosphere Reflectance and band-6 Temperature ===

Calculate ''Top-of-Atmosphere'' reflectance and band-6 temperature using the {{cmd|i.landsat.toar}} module. For details, refer to USGS' [http://landsat.usgs.gov/Landsat_Metadata_Changes.php Landsat Filename and Metadata Changes] dedicated page. For Landsat 8, see also [http://landsat.usgs.gov/about_LU_Vol_7_Issue_4.php#3a here].
<source lang="bash">
i.landsat.toar input_prefix=$BASE output_prefix=${BASE}_toar metfile=${BASE}_MTL.txt
</source>

Note, the resulting temperature map is in Kelvin:
<source lang="bash">
# convert to degree Celsius
r.mapcalc "$BASE.temp_celsius = ${BASE}_toar.6 - 273.15"
r.info -r $BASE.temp_celsius
</source>

=== Haze removal ===

Simple haze removal can be done with {{AddonCmd|i.landsat.dehaze|version=64}}. This addons applies a bandwise haze correction using tasscap4 (haze) and linear regression.

During the 2000s, prior acquired Landsat data were reprocessed to LPGS (Level 1 Product Generation System). Seems that with this level of processing, haze effect in raw data was removed and now is sufficient to apply {{Cmd|i.colors.enhance}} to obtain a good image with high contrast.

=== Atmospheric correction ===

See [[Atmospheric correction]]

=== Cloud identification ===

Identify clouds in the image with {{cmd|i.landsat.acca}}:
<source lang="bash">i.landsat.acca -f input_prefix=226_62_toar. output=226_62.acca_cloudmask</source>

Mask out the clouds:
<source lang="bash">r.mapcalc "MASK = if(isnull($BASE.acca_cloudmask))"</source>

== Download sample data ==

=== Preprocessed Landsat-7 data for North Carolina ===

The [[Sample_datasets|North Carolina 2008 sample dataset]] comes with 3 different Landsat scenes:
: (''Wake County -- path: 16 row: 35 for various dates'')

The above import efforts are not needed since the data are already in a GRASS location.

=== Landsat-5: Oct 14, 1987 ===

;Glovis download: LT50160351987287XXX08
: http://edcsns17.cr.usgs.gov/cgi-bin/EarthExplorer/run-phtml/results/download.phtml?node=GV&ordered=LT50160351987287XXX08&dataset_name=LANDSAT_TM



Provided metadata:
IMAGE_ID=P016R35_5T871014
PATH=16
ROW=35
DATE=10/14/87
PLATFORM=LANDSAT5

=== Landsat-7: Mar 31, 2000 ===

;Glovis download: LE70160352000091EDC00
: http://edcsns17.cr.usgs.gov/cgi-bin/EarthExplorer/run-phtml/results/download.phtml?node=GV&ordered=LE70160352000091EDC00&dataset_name=LANDSAT_ETM

Values in the metadata below indicate that the version provided with the NC 2008 dataset's production date was after July 1, 2000, and that the channel gains were <tt>HHHLHLHHL</tt>.

Convert DNs to radiance/temperatures:<source lang="bash">
GRASS> i.landsat.toar -v band=lsat7_2000 sensor=7 date=2000-03-31 \
product_date=2000-07-02 solar_elevation=51.5246529 gain=HHHLHLHHL</source>

Provided metadata:
SPACECRAFT_ID=Landsat7
SENSOR_ID=ETM+
'''ACQUISITION_DATE=2000-03-31'''
WRS_PATH=16
CPF_FILE_NAME=L7CPF20000101_20000331_12
SUN_AZIMUTH=139.6033279
'''SUN_ELEVATION=51.5246529'''
LMAX_BAND1=191.600
LMIN_BAND1=-6.200
LMAX_BAND2=196.500
LMIN_BAND2=-6.400
LMAX_BAND3=152.900
LMIN_BAND3=-5.000
LMAX_BAND4=241.100
LMIN_BAND4=-5.100
LMAX_BAND5=31.060
LMIN_BAND5=-1.000
LMAX_BAND61=17.040
LMIN_BAND61=0.000
LMAX_BAND62=12.650
LMIN_BAND62=3.200
LMAX_BAND7=10.800
LMIN_BAND7=-0.350
LMAX_BAND8=243.100
LMIN_BAND8=-4.700
QCALMAX_BAND1=255.0
QCALMIN_BAND1=1.0
QCALMAX_BAND2=255.0
QCALMIN_BAND2=1.0
QCALMAX_BAND3=255.0
QCALMIN_BAND3=1.0
QCALMAX_BAND4=255.0
QCALMIN_BAND4=1.0
QCALMAX_BAND5=255.0
QCALMIN_BAND5=1.0
QCALMAX_BAND61=255.0
QCALMIN_BAND61=1.0
QCALMAX_BAND62=255.0
QCALMIN_BAND62=1.0
QCALMAX_BAND7=255.0
QCALMIN_BAND7=1.0
QCALMAX_BAND8=255.0
QCALMIN_BAND8=1.0

=== Landsat-7: May 24, 2002 ===

;Glovis download: LE70160352002144EDC00
: http://edcsns17.cr.usgs.gov/cgi-bin/EarthExplorer/run-phtml/results/download.phtml?node=GV&ordered=LE70160352002144EDC00&dataset_name=LANDSAT_ETM



Provided metadata: (`p016r035_7x20020524.met`)
SPACECRAFT_ID=Landsat7
SENSOR_ID=ETM+
'''ACQUISITION_DATE=2002-05-24'''
WRS_PATH=016
WRS_ROW=035
SUN_AZIMUTH=120.8810347
'''SUN_ELEVATION=64.7730999'''
QA_PERCENT_MISSING_DATA=0
CLOUD_COVER=0
CPF_FILE_NAME=L7CPF20020401_20020630_03
LMAX_BAND1=191.600
LMIN_BAND1=-6.200
LMAX_BAND2=196.500
LMIN_BAND2=-6.400
LMAX_BAND3=152.900
LMIN_BAND3=-5.000
LMAX_BAND4=241.100
LMIN_BAND4=-5.100
LMAX_BAND5=31.060
LMIN_BAND5=-1.000
LMAX_BAND61=17.040
LMIN_BAND61=0.000
LMAX_BAND62=12.650
LMIN_BAND62=3.200
LMAX_BAND7=10.800
LMIN_BAND7=-0.350
LMAX_BAND8=243.100
LMIN_BAND8=-4.700
QCALMAX_BAND1=255.0
QCALMIN_BAND1=1.0
QCALMAX_BAND2=255.0
QCALMIN_BAND2=1.0
QCALMAX_BAND3=255.0
QCALMIN_BAND3=1.0
QCALMAX_BAND4=255.0
QCALMIN_BAND4=1.0
QCALMAX_BAND5=255.0
QCALMIN_BAND5=1.0
QCALMAX_BAND61=255.0
QCALMIN_BAND61=1.0
QCALMAX_BAND62=255.0
QCALMIN_BAND62=1.0
QCALMAX_BAND7=255.0
QCALMIN_BAND7=1.0
QCALMAX_BAND8=255.0
QCALMIN_BAND8=1.0

== LANDSAT Image classification ==

See [[Image classification]]

== Time series analysis ==

See [[Time series]]

== Sources ==

* [http://landsat.usgs.gov/tools_access_all_faqs.php Frequently Asked Questions about the Landsat Missions]
* [https://landsat8portal.eo.esa.int/portal/ European LANDSAT 8 data in near-realtime]
* [http://courses.neteler.org/processing-landsat8-data-in-grass-gis-7/ Processing Landsat 8 data in GRASS GIS 7: Import and visualization]
* [http://courses.neteler.org/processing-landsat-8-data-in-grass-gis-7-rgb-composites-and-pan-sharpening/ Processing Landsat 8 data in GRASS GIS 7: RGB composites and pan sharpening]

[[Category: Documentation]]
[[Category: FAQ]]
[[Category: Landsat]]
[[Category: Image processing]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Geodata]]

LANDSAT

2017-06-22T07:50:52Z

⚠️NikosA: Landsat on AWS

== Data Availability==

Landsat imagery can be obtained from [http://landsat.usgs.gov/ USGS' LANDSAT archive]:

* [http://glovis.usgs.gov USGS GloVis]
* [http://earthexplorer.usgs.gov/ EarthExplorer]
* [http://earthexplorer.usgs.gov/bulk/help Bulk Download Orders] using the [http://earthexplorer.usgs.gov/bulk/ Bulk Download Application]
* [http://landsatlook.usgs.gov/ LandsatLook Viewer]

See also the [[Global datasets]] wiki page

On 06/12/13, USGS [http://landsat.usgs.gov/about_LU_Vol_7_Issue_4.php announced] the availability of [http://landsat.usgs.gov/landsat8.php LANDSAT 8]. See also [http://landsat.usgs.gov/band_designations_landsat_satellites.php this page].

== Modules overview ==

GRASS GIS features a complete set of native modules and various add-ons for pre- and post-processing Landsat satellite imagery. The following lists offer an overview of related modules and add-ons.

=== Generic modules applicable to Landsat ===

* {{cmd|d.rgb}} → display 3-band data
* {{cmd|r.composite}} → flatten 3-bands of data into a single image (lossy, maybe used in combination with {{cmd|i.landsat.rgb|version=64}} (GRASS 6.x) | {{cmd|i.colors.enhance|version=70}} (GRASS 7.x)
* {{cmd|i.atcorr}} → correct top of atmosphere to surface reflectance - see also the [[Atmospheric correction]] wiki page
* {{cmd|i.topo.corr}} → used to topographically correct reflectance from imagery files, e.g. obtained with {{cmd|i.landsat.toar}}, using a sun illumination terrain model
* {{cmd|i.oif}} → calculate the 3 bands showing the greatest difference (for use as R,G,B bands)
* {{AddonCmd|i.histo.match}} (addon) → histogram matching of two or more raster maps (in grass 7) ''Note, the module works with integer values and does not accept the "." character as part of the raster map's name!''

=== Landsat specific modules ===

* {{cmd|i.landsat.rgb|version=64}} (GRASS 6.x) | {{cmd|i.colors.enhance|version=70}} (GRASS 7.x) → auto-enhance colors
* {{cmd|i.landsat.toar}} → convert DN to top of atmosphere radiance
* {{cmd|i.landsat.acca}} → cloud cover assessment
* {{cmd|i.tasscap}} → Performs Tasseled Cap (Kauth Thomas) transformation

=== Landsat specific GRASS AddOns ===

* {{AddonCmd|i.landsat.trim|version=64}} → trims border fringes for each band separately or with the MASK where coverage exists for all bands
* {{AddonCmd|i.landsat.dehaze|version=64}} → haze removal
* {{AddonCmd|i.landsat8.qc|version=70}} → Reclass Landsat8 QA band according to acceptable pixel quality as defined by the user
* {{AddonCmd|i.landsat8.swlst|version=70}} → Practical split-window algorithm estimating Land Surface Temperature from Landsat 8 OLI/TIRS imagery
* {{AddonCmd|i.wi|version=70}} → Calculate different water indices

== Pre-Processing ==

=== Overview ===

Typically, pre-processing Landsat imagery comprises the following steps:

# '''import''' in the database → {{cmd|r.in.gdal}}
# geometrically & orthometrically correct imagery
#* already done for L1T products, read more at [http://landsat.usgs.gov//Landsat_Processing_Details.php USGS' Landsat Information Products] webpage
# optionally, automatically '''cut-off border fringes''' → {{AddonCmd|i.landsat.trim|version=64}}
#* of course one can use the official WRS2 Path/Row vector tiles to manually trim border fringes
# optionally, '''denoise''' for obvious/intensive salt & pepper effects, stripes, etc.
#* for example by applying Principal Components Analysis as a denoising technique → {{cmd|i.pca}} ''<<< Re-order this step?''
# '''convert''' the '''Digital Numbers''' (DNs) to '''Top-of-Atmosphere Radiances/Reflectances''' (ToARs) → {{cmd|i.landsat.toar}}
# optionally, '''correct for atmospheric effects''' → {{cmd|i.atcorr}}
#* that is, accounting for distorting atmospheric effects and estimating actual reflectances as they would have been measured on the ground
#* also described as conversion to Top-of-Canopy Reflectances (ToCRs)
# '''assessing cloud cover''' → {{cmd|i.landsat.acca}}
#* optionally, detect and remove clouds shadows as well
# '''topographically normalise''' imagery → {{cmd|i.topo.corr}}
#* also known as topographic correction, that is, accounting for illumination differences due to the acquisition's geometry
# '''radiometrically normalise''' → one approach via {{AddonCmd|i.histo.match}} (in '''grass 7''', addon), also known as relative radiometric normalisation -- one approach is the ''histogram matching'' technique of two or more raster maps
#* '''ToDo/More techniques?'''

See also
* [http://courses.neteler.org/processing-landsat8-data-in-grass-gis-7/ Processing Landsat 8 data in GRASS GIS 7: Import and visualization]

=== Importing data ===

Importing Landsat spectral bands in GRASS GIS' data base can be done both from the Graphical User Interface or from the command line

# Open GRASS GIS, select ''Location Wizard'' in order to create a new location from georeferenced file
# Use the ''Import file tool'', or the {{cmd|r.in.gdal}} module, to import the GeoTIFF files into GRASS GIS.

See also [http://grasswiki.osgeo.org/wiki/LANDSAT#Automated_data_import Automated data import] below.

==== Notes ====

# Most Landsat scenes are delivered in ''north-is-up'' orientation, hence the import process is straightforward.
# If you get <source lang="bash" enclose="none">ERROR: Input map is rotated - cannot import.</source>, the image must be first rotated externally, applying the rotation info stored in the metadata field of the raster image file. For example, the <code>gdalwarp</code> software can be used to transform the map to North-up (note, there are several <code>gdalwarp</code> parameters to select the resampling algorithm):

<ul>
<source lang="bash">
gdalwarp rotated.tif northup.tif
</source>
</ul>

==== Hint: Minimal disk space copies ====

Here's a little trick using {{cmd|r.reclass}} to rename maps (for example, from ''L71074092_09220040924_B'''10''''' to ''L71074092_09220040924_B'''1''''') without touching the data or wasting disk space:

<source lang="bash">
BASE=L71074092_09220040924

for BAND in 10 20 30 40 50 61 70 80; do
BAND1st=`echo $BAND | sed -e 's/0$//'`
r.reclass in="${BASE}_B$BAND" out=$BASE.$BAND1st << EOF
* = *
EOF
done
</source>

==== Automated data import ====

The following ''Python'' script imports Landsat imagery into GRASS' data base. Specifically, the script

* creates an independent Mapset for each Landsat scene
* imports and renames bands of a scene as <tt>B<id></tt>, e.g. B10, B20, ..., B80.
* additionaly sets up the timestamp based on MTL metadata file

'''Note,''' the (newest) official naming pattern for Landsat scenes -- explained in [https://lta.cr.usgs.gov/landsat_dictionary.html USGS' Landsat Data Dictionary] as the [https://lta.cr.usgs.gov/landsat_dictionary.html#entity_id Landsat Scene Identifier] -- and all individual bands that compose a scene -- bands have a suffix which is like _B10, _B20, _B30, etc. -- differ from what some Landsat specific modules expect. For example, the modules {{cmd|i.landsat.toar}} and {{cmd|i.landsat.acca}} expect the bands to follow a naming pattern such as "scenename.1, .2, .3", etc.

To use the script save it as <source lang="bash" enclose="none">import_landsat.py</source> file and make sure it is granted the execution permission.

<source lang=python>
#!/usr/bin/python

import os
import sys
import glob
import grass.script as grass

def get_timestamp(mapset):
try:
metafile = glob.glob(mapset + '/*MTL.txt')[0]
except IndexError:
return

result = dict()
try:
fd = open(metafile)
for line in fd.readlines():
line = line.rstrip('\n')
if len(line) == 0:
continue
if any(x in line for x in ('DATE_ACQUIRED', 'ACQUISITION_DATE')):
result['date'] = line.strip().split('=')[1].strip()
finally:
fd.close()

return result

def import_tifs(mapset):
meta = get_timestamp(mapset)
for file in os.listdir(mapset):
if os.path.splitext(file)[-1] != '.TIF':
continue
ffile = os.path.join(mapset, file)
if ('VCID') in ffile:
name = "".join((os.path.splitext(file)[0].split('_'))[1::2])
else:
name = os.path.splitext(file)[0].split('_')[-1]
if len(name) == 3 and name[-1] == '0':
band = int(name[1:2])
elif len(name) == 3 and name[-1] != '0':
band = int(name[1:3])
else:
band = int(name[-1:])
grass.message('Importing %s -> %s@%s...' % (file, name, mapset))
grass.run_command('g.mapset',
flags = 'c',
mapset = mapset,
quiet = True,
stderr = open(os.devnull, 'w'))
grass.run_command('r.in.gdal',
input = ffile,
output = name,
quiet = True,
overwrite = True,
title = 'band %d' % band)
if meta:
year, month, day = meta['date'].split('-')
if month == '01':
month = 'jan'
elif month == '02':
month = 'feb'
elif month == '03':
month = 'mar'
elif month == '04':
month = 'apr'
elif month == '05':
month = 'may'
elif month == '06':
month = 'jun'
elif month == '07':
month = 'jul'
elif month == '08':
month = 'aug'
elif month == '09':
month = 'sep'
elif month == '10':
month = 'oct'
elif month == '11':
month = 'nov'
elif month == '12':
month = 'dec'

grass.run_command('r.timestamp',
map = name,
date = ' '.join((day, month, year)))

def main():
if len(sys.argv) == 1:
for directory in filter(os.path.isdir, os.listdir(os.getcwd())):
import_tifs(directory)
else:
import_tifs(sys.argv[1])

if __name__ == "__main__":
main()
</source>

'''Example of usage'''

After having collected the Landsat scenes of interest in one directory, the script can be used as follows:

* <source lang="python" enclose="none">./import_landsat.py</source> → the script will walk through a ''pool'' directory that contains unique Landsat scene directories (e.g. three directories named after the official naming pattern: LT51800342011158MOR00 LT51810352009079MTI00 LT51820352009326MTI00) and import all bands of each individual scene in their own Mapset
* <source lang="python" enclose="none">./import_landsat.py LM41890261983200FFF03</source> → the scrip will import only bands of the specified Landsat scene directory

== Post-Processing ==

=== Natural color composites ===

Creating natural (also known as true-) color composites, can be done by

a) using {{cmd|i.landsat.rgb|version=64}} (GRASS 6.x) | {{cmd|i.colors.enhance|version=70}} (GRASS 7.x) to automatically balance the colors for the Red, Green and Blue bands

or

b) equalizing the colors on each of the Red, Green and Blue bands

<ul>
<source lang="bash">
r.colors -e map=band1 color=grey
</source>
</ul>

'''Composites''' can then be produced with the {{cmd|r.composite}} module.

'''Resetting the color tables''' of all bands back to normal greyscale can be done with a for loop:

<source lang="bash">
BASE=L71074092_09220040924

for map in `g.mlist pat="$BASE.[0-8]*"` ; do
r.colors $map color=grey255
done
</source>

See also:
* [http://courses.neteler.org/processing-landsat-8-data-in-grass-gis-7-rgb-composites-and-pan-sharpening/ Processing Landsat 8 data in GRASS GIS 7: RGB composites and pan sharpening]

=== Create a MASK to only show data where coverage exists for all bands ===

<source lang="bash">
BASE=L71074092_09220040924

g.region rast=$BASE.1
r.series in=`g.mlist pat="$BASE.[0-8]*" sep=,` -n out=$BASE.thresh method=threshold thresh=1
r.mapcalc "$BASE.mask = if(isnull($BASE.thresh))"
g.remove $BASE.thresh
</source>

=== Calculate Top-of-Atmosphere Reflectance and band-6 Temperature ===

Calculate ''Top-of-Atmosphere'' reflectance and band-6 temperature using the {{cmd|i.landsat.toar}} module. For details, refer to USGS' [http://landsat.usgs.gov/Landsat_Metadata_Changes.php Landsat Filename and Metadata Changes] dedicated page. For Landsat 8, see also [http://landsat.usgs.gov/about_LU_Vol_7_Issue_4.php#3a here].
<source lang="bash">
i.landsat.toar input_prefix=$BASE output_prefix=${BASE}_toar metfile=${BASE}_MTL.txt
</source>

Note, the resulting temperature map is in Kelvin:
<source lang="bash">
# convert to degree Celsius
r.mapcalc "$BASE.temp_celsius = ${BASE}_toar.6 - 273.15"
r.info -r $BASE.temp_celsius
</source>

=== Haze removal ===

Simple haze removal can be done with {{AddonCmd|i.landsat.dehaze|version=64}}. This addons applies a bandwise haze correction using tasscap4 (haze) and linear regression.

During the 2000s, prior acquired Landsat data were reprocessed to LPGS (Level 1 Product Generation System). Seems that with this level of processing, haze effect in raw data was removed and now is sufficient to apply {{Cmd|i.colors.enhance}} to obtain a good image with high contrast.

=== Atmospheric correction ===

See [[Atmospheric correction]]

=== Cloud identification ===

Identify clouds in the image with {{cmd|i.landsat.acca}}:
<source lang="bash">i.landsat.acca -f input_prefix=226_62_toar. output=226_62.acca_cloudmask</source>

Mask out the clouds:
<source lang="bash">r.mapcalc "MASK = if(isnull($BASE.acca_cloudmask))"</source>

== Download sample data ==

=== Preprocessed Landsat-7 data for North Carolina ===

The [[Sample_datasets|North Carolina 2008 sample dataset]] comes with 3 different Landsat scenes:
: (''Wake County -- path: 16 row: 35 for various dates'')

The above import efforts are not needed since the data are already in a GRASS location.

=== Landsat-5: Oct 14, 1987 ===

;Glovis download: LT50160351987287XXX08
: http://edcsns17.cr.usgs.gov/cgi-bin/EarthExplorer/run-phtml/results/download.phtml?node=GV&ordered=LT50160351987287XXX08&dataset_name=LANDSAT_TM



Provided metadata:
IMAGE_ID=P016R35_5T871014
PATH=16
ROW=35
DATE=10/14/87
PLATFORM=LANDSAT5

=== Landsat-7: Mar 31, 2000 ===

;Glovis download: LE70160352000091EDC00
: http://edcsns17.cr.usgs.gov/cgi-bin/EarthExplorer/run-phtml/results/download.phtml?node=GV&ordered=LE70160352000091EDC00&dataset_name=LANDSAT_ETM

Values in the metadata below indicate that the version provided with the NC 2008 dataset's production date was after July 1, 2000, and that the channel gains were <tt>HHHLHLHHL</tt>.

Convert DNs to radiance/temperatures:<source lang="bash">
GRASS> i.landsat.toar -v band=lsat7_2000 sensor=7 date=2000-03-31 \
product_date=2000-07-02 solar_elevation=51.5246529 gain=HHHLHLHHL</source>

Provided metadata:
SPACECRAFT_ID=Landsat7
SENSOR_ID=ETM+
'''ACQUISITION_DATE=2000-03-31'''
WRS_PATH=16
CPF_FILE_NAME=L7CPF20000101_20000331_12
SUN_AZIMUTH=139.6033279
'''SUN_ELEVATION=51.5246529'''
LMAX_BAND1=191.600
LMIN_BAND1=-6.200
LMAX_BAND2=196.500
LMIN_BAND2=-6.400
LMAX_BAND3=152.900
LMIN_BAND3=-5.000
LMAX_BAND4=241.100
LMIN_BAND4=-5.100
LMAX_BAND5=31.060
LMIN_BAND5=-1.000
LMAX_BAND61=17.040
LMIN_BAND61=0.000
LMAX_BAND62=12.650
LMIN_BAND62=3.200
LMAX_BAND7=10.800
LMIN_BAND7=-0.350
LMAX_BAND8=243.100
LMIN_BAND8=-4.700
QCALMAX_BAND1=255.0
QCALMIN_BAND1=1.0
QCALMAX_BAND2=255.0
QCALMIN_BAND2=1.0
QCALMAX_BAND3=255.0
QCALMIN_BAND3=1.0
QCALMAX_BAND4=255.0
QCALMIN_BAND4=1.0
QCALMAX_BAND5=255.0
QCALMIN_BAND5=1.0
QCALMAX_BAND61=255.0
QCALMIN_BAND61=1.0
QCALMAX_BAND62=255.0
QCALMIN_BAND62=1.0
QCALMAX_BAND7=255.0
QCALMIN_BAND7=1.0
QCALMAX_BAND8=255.0
QCALMIN_BAND8=1.0

=== Landsat-7: May 24, 2002 ===

;Glovis download: LE70160352002144EDC00
: http://edcsns17.cr.usgs.gov/cgi-bin/EarthExplorer/run-phtml/results/download.phtml?node=GV&ordered=LE70160352002144EDC00&dataset_name=LANDSAT_ETM



Provided metadata: (`p016r035_7x20020524.met`)
SPACECRAFT_ID=Landsat7
SENSOR_ID=ETM+
'''ACQUISITION_DATE=2002-05-24'''
WRS_PATH=016
WRS_ROW=035
SUN_AZIMUTH=120.8810347
'''SUN_ELEVATION=64.7730999'''
QA_PERCENT_MISSING_DATA=0
CLOUD_COVER=0
CPF_FILE_NAME=L7CPF20020401_20020630_03
LMAX_BAND1=191.600
LMIN_BAND1=-6.200
LMAX_BAND2=196.500
LMIN_BAND2=-6.400
LMAX_BAND3=152.900
LMIN_BAND3=-5.000
LMAX_BAND4=241.100
LMIN_BAND4=-5.100
LMAX_BAND5=31.060
LMIN_BAND5=-1.000
LMAX_BAND61=17.040
LMIN_BAND61=0.000
LMAX_BAND62=12.650
LMIN_BAND62=3.200
LMAX_BAND7=10.800
LMIN_BAND7=-0.350
LMAX_BAND8=243.100
LMIN_BAND8=-4.700
QCALMAX_BAND1=255.0
QCALMIN_BAND1=1.0
QCALMAX_BAND2=255.0
QCALMIN_BAND2=1.0
QCALMAX_BAND3=255.0
QCALMIN_BAND3=1.0
QCALMAX_BAND4=255.0
QCALMIN_BAND4=1.0
QCALMAX_BAND5=255.0
QCALMIN_BAND5=1.0
QCALMAX_BAND61=255.0
QCALMIN_BAND61=1.0
QCALMAX_BAND62=255.0
QCALMIN_BAND62=1.0
QCALMAX_BAND7=255.0
QCALMIN_BAND7=1.0
QCALMAX_BAND8=255.0
QCALMIN_BAND8=1.0

== LANDSAT Image classification ==

See [[Image classification]]

== Time series analysis ==

See [[Time series]]

== Sources ==

* [http://landsat.usgs.gov/tools_access_all_faqs.php Frequently Asked Questions about the Landsat Missions]
* [https://landsat8portal.eo.esa.int/portal/ European LANDSAT 8 data in near-realtime]
* [http://courses.neteler.org/processing-landsat8-data-in-grass-gis-7/ Processing Landsat 8 data in GRASS GIS 7: Import and visualization]
* [http://courses.neteler.org/processing-landsat-8-data-in-grass-gis-7-rgb-composites-and-pan-sharpening/ Processing Landsat 8 data in GRASS GIS 7: RGB composites and pan sharpening]

[[Category: Documentation]]
[[Category: FAQ]]
[[Category: Landsat]]
[[Category: Image processing]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Geodata]]

LANDSAT

2017-06-22T07:50:17Z

⚠️NikosA: Landsat on AWS

== Data Availability==

Landsat imagery can be obtained from [http://landsat.usgs.gov/ USGS' LANDSAT archive]:

* [http://glovis.usgs.gov USGS GloVis]
* [http://earthexplorer.usgs.gov/ EarthExplorer]
* [http://earthexplorer.usgs.gov/bulk/help Bulk Download Orders] using the [http://earthexplorer.usgs.gov/bulk/ Bulk Download Application]
* [http://landsatlook.usgs.gov/ LandsatLook Viewer]
* [https://landsatonaws.com/ Landsat on Amazon Web Services]

See also
* the [[Global datasets]] wiki page
* [http://landsat.usgs.gov/band_designations_landsat_satellites.php this page].

== Modules overview ==

GRASS GIS features a complete set of native modules and various add-ons for pre- and post-processing Landsat satellite imagery. The following lists offer an overview of related modules and add-ons.

=== Generic modules applicable to Landsat ===

* {{cmd|d.rgb}} → display 3-band data
* {{cmd|r.composite}} → flatten 3-bands of data into a single image (lossy, maybe used in combination with {{cmd|i.landsat.rgb|version=64}} (GRASS 6.x) | {{cmd|i.colors.enhance|version=70}} (GRASS 7.x)
* {{cmd|i.atcorr}} → correct top of atmosphere to surface reflectance - see also the [[Atmospheric correction]] wiki page
* {{cmd|i.topo.corr}} → used to topographically correct reflectance from imagery files, e.g. obtained with {{cmd|i.landsat.toar}}, using a sun illumination terrain model
* {{cmd|i.oif}} → calculate the 3 bands showing the greatest difference (for use as R,G,B bands)
* {{AddonCmd|i.histo.match}} (addon) → histogram matching of two or more raster maps (in grass 7) ''Note, the module works with integer values and does not accept the "." character as part of the raster map's name!''

=== Landsat specific modules ===

* {{cmd|i.landsat.rgb|version=64}} (GRASS 6.x) | {{cmd|i.colors.enhance|version=70}} (GRASS 7.x) → auto-enhance colors
* {{cmd|i.landsat.toar}} → convert DN to top of atmosphere radiance
* {{cmd|i.landsat.acca}} → cloud cover assessment
* {{cmd|i.tasscap}} → Performs Tasseled Cap (Kauth Thomas) transformation

=== Landsat specific GRASS AddOns ===

* {{AddonCmd|i.landsat.trim|version=64}} → trims border fringes for each band separately or with the MASK where coverage exists for all bands
* {{AddonCmd|i.landsat.dehaze|version=64}} → haze removal
* {{AddonCmd|i.landsat8.qc|version=70}} → Reclass Landsat8 QA band according to acceptable pixel quality as defined by the user
* {{AddonCmd|i.landsat8.swlst|version=70}} → Practical split-window algorithm estimating Land Surface Temperature from Landsat 8 OLI/TIRS imagery
* {{AddonCmd|i.wi|version=70}} → Calculate different water indices

== Pre-Processing ==

=== Overview ===

Typically, pre-processing Landsat imagery comprises the following steps:

# '''import''' in the database → {{cmd|r.in.gdal}}
# geometrically & orthometrically correct imagery
#* already done for L1T products, read more at [http://landsat.usgs.gov//Landsat_Processing_Details.php USGS' Landsat Information Products] webpage
# optionally, automatically '''cut-off border fringes''' → {{AddonCmd|i.landsat.trim|version=64}}
#* of course one can use the official WRS2 Path/Row vector tiles to manually trim border fringes
# optionally, '''denoise''' for obvious/intensive salt & pepper effects, stripes, etc.
#* for example by applying Principal Components Analysis as a denoising technique → {{cmd|i.pca}} ''<<< Re-order this step?''
# '''convert''' the '''Digital Numbers''' (DNs) to '''Top-of-Atmosphere Radiances/Reflectances''' (ToARs) → {{cmd|i.landsat.toar}}
# optionally, '''correct for atmospheric effects''' → {{cmd|i.atcorr}}
#* that is, accounting for distorting atmospheric effects and estimating actual reflectances as they would have been measured on the ground
#* also described as conversion to Top-of-Canopy Reflectances (ToCRs)
# '''assessing cloud cover''' → {{cmd|i.landsat.acca}}
#* optionally, detect and remove clouds shadows as well
# '''topographically normalise''' imagery → {{cmd|i.topo.corr}}
#* also known as topographic correction, that is, accounting for illumination differences due to the acquisition's geometry
# '''radiometrically normalise''' → one approach via {{AddonCmd|i.histo.match}} (in '''grass 7''', addon), also known as relative radiometric normalisation -- one approach is the ''histogram matching'' technique of two or more raster maps
#* '''ToDo/More techniques?'''

See also
* [http://courses.neteler.org/processing-landsat8-data-in-grass-gis-7/ Processing Landsat 8 data in GRASS GIS 7: Import and visualization]

=== Importing data ===

Importing Landsat spectral bands in GRASS GIS' data base can be done both from the Graphical User Interface or from the command line

# Open GRASS GIS, select ''Location Wizard'' in order to create a new location from georeferenced file
# Use the ''Import file tool'', or the {{cmd|r.in.gdal}} module, to import the GeoTIFF files into GRASS GIS.

See also [http://grasswiki.osgeo.org/wiki/LANDSAT#Automated_data_import Automated data import] below.

==== Notes ====

# Most Landsat scenes are delivered in ''north-is-up'' orientation, hence the import process is straightforward.
# If you get <source lang="bash" enclose="none">ERROR: Input map is rotated - cannot import.</source>, the image must be first rotated externally, applying the rotation info stored in the metadata field of the raster image file. For example, the <code>gdalwarp</code> software can be used to transform the map to North-up (note, there are several <code>gdalwarp</code> parameters to select the resampling algorithm):

<ul>
<source lang="bash">
gdalwarp rotated.tif northup.tif
</source>
</ul>

==== Hint: Minimal disk space copies ====

Here's a little trick using {{cmd|r.reclass}} to rename maps (for example, from ''L71074092_09220040924_B'''10''''' to ''L71074092_09220040924_B'''1''''') without touching the data or wasting disk space:

<source lang="bash">
BASE=L71074092_09220040924

for BAND in 10 20 30 40 50 61 70 80; do
BAND1st=`echo $BAND | sed -e 's/0$//'`
r.reclass in="${BASE}_B$BAND" out=$BASE.$BAND1st << EOF
* = *
EOF
done
</source>

==== Automated data import ====

The following ''Python'' script imports Landsat imagery into GRASS' data base. Specifically, the script

* creates an independent Mapset for each Landsat scene
* imports and renames bands of a scene as <tt>B<id></tt>, e.g. B10, B20, ..., B80.
* additionaly sets up the timestamp based on MTL metadata file

'''Note,''' the (newest) official naming pattern for Landsat scenes -- explained in [https://lta.cr.usgs.gov/landsat_dictionary.html USGS' Landsat Data Dictionary] as the [https://lta.cr.usgs.gov/landsat_dictionary.html#entity_id Landsat Scene Identifier] -- and all individual bands that compose a scene -- bands have a suffix which is like _B10, _B20, _B30, etc. -- differ from what some Landsat specific modules expect. For example, the modules {{cmd|i.landsat.toar}} and {{cmd|i.landsat.acca}} expect the bands to follow a naming pattern such as "scenename.1, .2, .3", etc.

To use the script save it as <source lang="bash" enclose="none">import_landsat.py</source> file and make sure it is granted the execution permission.

<source lang=python>
#!/usr/bin/python

import os
import sys
import glob
import grass.script as grass

def get_timestamp(mapset):
try:
metafile = glob.glob(mapset + '/*MTL.txt')[0]
except IndexError:
return

result = dict()
try:
fd = open(metafile)
for line in fd.readlines():
line = line.rstrip('\n')
if len(line) == 0:
continue
if any(x in line for x in ('DATE_ACQUIRED', 'ACQUISITION_DATE')):
result['date'] = line.strip().split('=')[1].strip()
finally:
fd.close()

return result

def import_tifs(mapset):
meta = get_timestamp(mapset)
for file in os.listdir(mapset):
if os.path.splitext(file)[-1] != '.TIF':
continue
ffile = os.path.join(mapset, file)
if ('VCID') in ffile:
name = "".join((os.path.splitext(file)[0].split('_'))[1::2])
else:
name = os.path.splitext(file)[0].split('_')[-1]
if len(name) == 3 and name[-1] == '0':
band = int(name[1:2])
elif len(name) == 3 and name[-1] != '0':
band = int(name[1:3])
else:
band = int(name[-1:])
grass.message('Importing %s -> %s@%s...' % (file, name, mapset))
grass.run_command('g.mapset',
flags = 'c',
mapset = mapset,
quiet = True,
stderr = open(os.devnull, 'w'))
grass.run_command('r.in.gdal',
input = ffile,
output = name,
quiet = True,
overwrite = True,
title = 'band %d' % band)
if meta:
year, month, day = meta['date'].split('-')
if month == '01':
month = 'jan'
elif month == '02':
month = 'feb'
elif month == '03':
month = 'mar'
elif month == '04':
month = 'apr'
elif month == '05':
month = 'may'
elif month == '06':
month = 'jun'
elif month == '07':
month = 'jul'
elif month == '08':
month = 'aug'
elif month == '09':
month = 'sep'
elif month == '10':
month = 'oct'
elif month == '11':
month = 'nov'
elif month == '12':
month = 'dec'

grass.run_command('r.timestamp',
map = name,
date = ' '.join((day, month, year)))

def main():
if len(sys.argv) == 1:
for directory in filter(os.path.isdir, os.listdir(os.getcwd())):
import_tifs(directory)
else:
import_tifs(sys.argv[1])

if __name__ == "__main__":
main()
</source>

'''Example of usage'''

After having collected the Landsat scenes of interest in one directory, the script can be used as follows:

* <source lang="python" enclose="none">./import_landsat.py</source> → the script will walk through a ''pool'' directory that contains unique Landsat scene directories (e.g. three directories named after the official naming pattern: LT51800342011158MOR00 LT51810352009079MTI00 LT51820352009326MTI00) and import all bands of each individual scene in their own Mapset
* <source lang="python" enclose="none">./import_landsat.py LM41890261983200FFF03</source> → the scrip will import only bands of the specified Landsat scene directory

== Post-Processing ==

=== Natural color composites ===

Creating natural (also known as true-) color composites, can be done by

a) using {{cmd|i.landsat.rgb|version=64}} (GRASS 6.x) | {{cmd|i.colors.enhance|version=70}} (GRASS 7.x) to automatically balance the colors for the Red, Green and Blue bands

or

b) equalizing the colors on each of the Red, Green and Blue bands

<ul>
<source lang="bash">
r.colors -e map=band1 color=grey
</source>
</ul>

'''Composites''' can then be produced with the {{cmd|r.composite}} module.

'''Resetting the color tables''' of all bands back to normal greyscale can be done with a for loop:

<source lang="bash">
BASE=L71074092_09220040924

for map in `g.mlist pat="$BASE.[0-8]*"` ; do
r.colors $map color=grey255
done
</source>

See also:
* [http://courses.neteler.org/processing-landsat-8-data-in-grass-gis-7-rgb-composites-and-pan-sharpening/ Processing Landsat 8 data in GRASS GIS 7: RGB composites and pan sharpening]

=== Create a MASK to only show data where coverage exists for all bands ===

<source lang="bash">
BASE=L71074092_09220040924

g.region rast=$BASE.1
r.series in=`g.mlist pat="$BASE.[0-8]*" sep=,` -n out=$BASE.thresh method=threshold thresh=1
r.mapcalc "$BASE.mask = if(isnull($BASE.thresh))"
g.remove $BASE.thresh
</source>

=== Calculate Top-of-Atmosphere Reflectance and band-6 Temperature ===

Calculate ''Top-of-Atmosphere'' reflectance and band-6 temperature using the {{cmd|i.landsat.toar}} module. For details, refer to USGS' [http://landsat.usgs.gov/Landsat_Metadata_Changes.php Landsat Filename and Metadata Changes] dedicated page. For Landsat 8, see also [http://landsat.usgs.gov/about_LU_Vol_7_Issue_4.php#3a here].
<source lang="bash">
i.landsat.toar input_prefix=$BASE output_prefix=${BASE}_toar metfile=${BASE}_MTL.txt
</source>

Note, the resulting temperature map is in Kelvin:
<source lang="bash">
# convert to degree Celsius
r.mapcalc "$BASE.temp_celsius = ${BASE}_toar.6 - 273.15"
r.info -r $BASE.temp_celsius
</source>

=== Haze removal ===

Simple haze removal can be done with {{AddonCmd|i.landsat.dehaze|version=64}}. This addons applies a bandwise haze correction using tasscap4 (haze) and linear regression.

During the 2000s, prior acquired Landsat data were reprocessed to LPGS (Level 1 Product Generation System). Seems that with this level of processing, haze effect in raw data was removed and now is sufficient to apply {{Cmd|i.colors.enhance}} to obtain a good image with high contrast.

=== Atmospheric correction ===

See [[Atmospheric correction]]

=== Cloud identification ===

Identify clouds in the image with {{cmd|i.landsat.acca}}:
<source lang="bash">i.landsat.acca -f input_prefix=226_62_toar. output=226_62.acca_cloudmask</source>

Mask out the clouds:
<source lang="bash">r.mapcalc "MASK = if(isnull($BASE.acca_cloudmask))"</source>

== Download sample data ==

=== Preprocessed Landsat-7 data for North Carolina ===

The [[Sample_datasets|North Carolina 2008 sample dataset]] comes with 3 different Landsat scenes:
: (''Wake County -- path: 16 row: 35 for various dates'')

The above import efforts are not needed since the data are already in a GRASS location.

=== Landsat-5: Oct 14, 1987 ===

;Glovis download: LT50160351987287XXX08
: http://edcsns17.cr.usgs.gov/cgi-bin/EarthExplorer/run-phtml/results/download.phtml?node=GV&ordered=LT50160351987287XXX08&dataset_name=LANDSAT_TM



Provided metadata:
IMAGE_ID=P016R35_5T871014
PATH=16
ROW=35
DATE=10/14/87
PLATFORM=LANDSAT5

=== Landsat-7: Mar 31, 2000 ===

;Glovis download: LE70160352000091EDC00
: http://edcsns17.cr.usgs.gov/cgi-bin/EarthExplorer/run-phtml/results/download.phtml?node=GV&ordered=LE70160352000091EDC00&dataset_name=LANDSAT_ETM

Values in the metadata below indicate that the version provided with the NC 2008 dataset's production date was after July 1, 2000, and that the channel gains were <tt>HHHLHLHHL</tt>.

Convert DNs to radiance/temperatures:<source lang="bash">
GRASS> i.landsat.toar -v band=lsat7_2000 sensor=7 date=2000-03-31 \
product_date=2000-07-02 solar_elevation=51.5246529 gain=HHHLHLHHL</source>

Provided metadata:
SPACECRAFT_ID=Landsat7
SENSOR_ID=ETM+
'''ACQUISITION_DATE=2000-03-31'''
WRS_PATH=16
CPF_FILE_NAME=L7CPF20000101_20000331_12
SUN_AZIMUTH=139.6033279
'''SUN_ELEVATION=51.5246529'''
LMAX_BAND1=191.600
LMIN_BAND1=-6.200
LMAX_BAND2=196.500
LMIN_BAND2=-6.400
LMAX_BAND3=152.900
LMIN_BAND3=-5.000
LMAX_BAND4=241.100
LMIN_BAND4=-5.100
LMAX_BAND5=31.060
LMIN_BAND5=-1.000
LMAX_BAND61=17.040
LMIN_BAND61=0.000
LMAX_BAND62=12.650
LMIN_BAND62=3.200
LMAX_BAND7=10.800
LMIN_BAND7=-0.350
LMAX_BAND8=243.100
LMIN_BAND8=-4.700
QCALMAX_BAND1=255.0
QCALMIN_BAND1=1.0
QCALMAX_BAND2=255.0
QCALMIN_BAND2=1.0
QCALMAX_BAND3=255.0
QCALMIN_BAND3=1.0
QCALMAX_BAND4=255.0
QCALMIN_BAND4=1.0
QCALMAX_BAND5=255.0
QCALMIN_BAND5=1.0
QCALMAX_BAND61=255.0
QCALMIN_BAND61=1.0
QCALMAX_BAND62=255.0
QCALMIN_BAND62=1.0
QCALMAX_BAND7=255.0
QCALMIN_BAND7=1.0
QCALMAX_BAND8=255.0
QCALMIN_BAND8=1.0

=== Landsat-7: May 24, 2002 ===

;Glovis download: LE70160352002144EDC00
: http://edcsns17.cr.usgs.gov/cgi-bin/EarthExplorer/run-phtml/results/download.phtml?node=GV&ordered=LE70160352002144EDC00&dataset_name=LANDSAT_ETM



Provided metadata: (`p016r035_7x20020524.met`)
SPACECRAFT_ID=Landsat7
SENSOR_ID=ETM+
'''ACQUISITION_DATE=2002-05-24'''
WRS_PATH=016
WRS_ROW=035
SUN_AZIMUTH=120.8810347
'''SUN_ELEVATION=64.7730999'''
QA_PERCENT_MISSING_DATA=0
CLOUD_COVER=0
CPF_FILE_NAME=L7CPF20020401_20020630_03
LMAX_BAND1=191.600
LMIN_BAND1=-6.200
LMAX_BAND2=196.500
LMIN_BAND2=-6.400
LMAX_BAND3=152.900
LMIN_BAND3=-5.000
LMAX_BAND4=241.100
LMIN_BAND4=-5.100
LMAX_BAND5=31.060
LMIN_BAND5=-1.000
LMAX_BAND61=17.040
LMIN_BAND61=0.000
LMAX_BAND62=12.650
LMIN_BAND62=3.200
LMAX_BAND7=10.800
LMIN_BAND7=-0.350
LMAX_BAND8=243.100
LMIN_BAND8=-4.700
QCALMAX_BAND1=255.0
QCALMIN_BAND1=1.0
QCALMAX_BAND2=255.0
QCALMIN_BAND2=1.0
QCALMAX_BAND3=255.0
QCALMIN_BAND3=1.0
QCALMAX_BAND4=255.0
QCALMIN_BAND4=1.0
QCALMAX_BAND5=255.0
QCALMIN_BAND5=1.0
QCALMAX_BAND61=255.0
QCALMIN_BAND61=1.0
QCALMAX_BAND62=255.0
QCALMIN_BAND62=1.0
QCALMAX_BAND7=255.0
QCALMIN_BAND7=1.0
QCALMAX_BAND8=255.0
QCALMIN_BAND8=1.0

== LANDSAT Image classification ==

See [[Image classification]]

== Time series analysis ==

See [[Time series]]

== Sources ==

* [http://landsat.usgs.gov/tools_access_all_faqs.php Frequently Asked Questions about the Landsat Missions]
* [https://landsat8portal.eo.esa.int/portal/ European LANDSAT 8 data in near-realtime]
* [http://courses.neteler.org/processing-landsat8-data-in-grass-gis-7/ Processing Landsat 8 data in GRASS GIS 7: Import and visualization]
* [http://courses.neteler.org/processing-landsat-8-data-in-grass-gis-7-rgb-composites-and-pan-sharpening/ Processing Landsat 8 data in GRASS GIS 7: RGB composites and pan sharpening]

[[Category: Documentation]]
[[Category: FAQ]]
[[Category: Landsat]]
[[Category: Image processing]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Geodata]]

Talk:Compile and Install

2017-03-31T07:07:29Z

⚠️NikosA: Remove AIX info from Discussion? Please move compilation material to trac.

__TOC__

* Can we remove the "AIX" related info below?
* Please, if taking time to update the compilation related sections, why not move over to somewhere under https://trac.osgeo.org/grass/wiki/BuildHints?
--Nikos Alexandris 00:07, 31 March 2017 (PDT)

=== Ubuntu compilation hints ===

''feedback on following the same directions under Ubuntu 8.10 amd64''

* apt-get says "Couldn't find package bpng12-dev"
: ''HB: try "apt-cache search bpng*-dev". Probably they bumped the version number.

=== AIX 7.x compilation hints ===

Hints: [http://www.ibm.com/developerworks/aix/library/au-gnu.html Using the GNU C/C++ compiler on AIX]

* conf_proj_AIX.sh:
sh configure --prefix=$HOME/bin --without-mutex

* conf_sqlite3.sh:
sh configure --prefix=$HOME/bin \
--disable-shared --disable-threadsafe --disable-tcl

* conf_gdal.sh:
<pre>
export LIBICONV=/opt/freeware
export CC="gcc"
export CXX="g++"

PREFIX=$HOME/bin
CFLAGS='-DHAVE_INTTYPES_H=0' ./configure \
--prefix=$PREFIX --with-libz=internal --with-threads=no --with-geotiff=internal --with-libtiff=internal

echo "Now manually edit: vim port/cpl_config.h and disable use of AIX's <inttypes.h> header file"
echo "#define HAVE_INTTYPES_H 0"
echo "See bug http://trac.osgeo.org/gdal/ticket/4809"
</pre>

* Fix $HOME/bin/gdal-config issue (bug in GRASS GIS 7 or GDAL?): add to the 'CONFIG_LIBS' line the entry
-lstdc++

* If you want r.terraflow, patch the source code for a workaround of a [http://gcc.gnu.org/ml/gcc-patches/2012-09/msg01957.html gcc bug on AIX]:
<pre>
r.terraflow_AIX_gcc_fix.diff
Index: raster/r.terraflow/stats.h
===================================================================
--- raster/r.terraflow/stats.h (revision 56936)
+++ raster/r.terraflow/stats.h (working copy)
@@ -22,6 +22,16 @@

#include <sys/types.h>

+/* http://gcc.gnu.org/ml/gcc-patches/2012-09/msg01957.html */
+#if defined(_AIX) && defined(_LARGE_FILES)
+#include <fcntl.h>
+#undef open
+/* Alias the symbols using asm */
+extern "C" {
+extern int open(const char *, int, ...) __asm__("open64");
+}
+#endif
+
#include <fstream>
#include <iostream>
using namespace std;
</pre>

=== AIX 5.x compilation hints ===

* conf_wget_AIX.sh:
sh configure --prefix=/afs/cluster/grid/user/private/bin/ --disable-nls

* conf_proj_AIX.sh:
sh configure --prefix=/afs/cluster/grid/user/private/bin --without-mutex

* conf_gdal_AIX.sh:
export LIBICONV=/opt/freeware
export CC="gcc"
export CXX="g++"
PREFIX=/afs/cluster/grid/user/private/bin
sh configure \
--prefix=$PREFIX \
--with-libz=internal \
--with-threads=no \
--with-geotiff=internal --with-libtiff=internal

* conf_grass6_AIX.sh
<pre>
#!/bin/sh

# optimization
# http://en.gentoo-wiki.com/wiki/Safe_Cflags/Intel
MYCFLAGS="-O2 $XEON"
MYCXXFLAGS="-O2"

# debug
#MYCFLAGS="-g"
#MYCXXFLAGS="-g"

########
MYBIN=$HOME/private/bin
MYNAD2BIN=$MYBIN/bin/nad2bin

# 64bit:
CFLAGS="$MYCFLAGS" CXXFLAGS="$MYCXXFLAGS" NAD2BIN=$MYNAD2BIN ./configure \
--prefix=$MYBIN \
--with-cxx \
--enable-largefile \
--with-gdal=$MYBIN/bin/gdal-config \
--with-includes=$MYBIN/include \
--with-proj \
--with-proj-includes=$MYBIN/include \
--with-proj-libs=$MYBIN/lib \
--with-proj-share=$MYBIN/share/proj \
--with-png=yes \
--with-tiff=yes \
--with-tcltk=yes \
--with-sqlite=yes \
--with-opengl=yes \
--with-x=yes \
--with-nls=no \
--with-freetype=no \
--with-jpeg=no \
--with-postgres=no \
--with-mysql=no \
--with-ffmpeg=no \
--with-odbc=no \
--with-fftw=no \
--with-blas=no \
--with-lapack=no \
--with-motif=no \
--with-glw=no \
--with-readline=no \
--with-opendwg=no \
--with-python=no
</pre>

* conf_grass7_AIX.sh:
<pre>
export LIBICONV=/opt/freeware
# comment next two lines for using the IBM xlc compiler:
export CC="gcc"
export CXX="g++"

PREFIX=$HOME/private/bin

CFLAGS='-ansi -D_ALL_SOURCE=1 -D_POSIX_SOURCE=1 -D_POSIX_C_SOURCE=200809L -Dinline=' ./configure \
--prefix=$PREFIX \
--disable-shared \
--enable-largefile \
--with-cxx \
--with-proj-includes=$PREFIX/include/ \
--with-proj-libs=$PREFIX/lib/ \
--with-proj-share=$PREFIX/share/proj/ \
--with-sqlite \
--with-sqlite-includes=$PREFIX/include/ \
--with-sqlite-libs=$PREFIX/lib/ \
--with-tiff=no \
--with-png=no \
--with-fftw=no \
--with-cairo=no \
--with-freetype=no
</pre>

GRASS Community Sprint Genova 2017

2017-02-06T07:44:16Z

⚠️NikosA: NikosA not joining the sprint in Genova 2017

The GRASS GIS team will organize a '''GRASS Developer and Power User Meeting, aka 'GRASS Community Sprint'''' from '''6-12 Feb, 2017'''. The sprint is at the same time of the XVIII Meeting degli Utenti Italiani Grass e Gfoss, 9-10 Feb 2017, University of Genova, Polytechnic School of Engineering, [http://en.wikipedia.org/wiki/Genoa Genova], [http://en.wikipedia.org/wiki/Italy Italy].

== Purpose ==

This next edition of the GRASS GIS community sprint 2017 is a great occasion for folks to support the development by actively contributing to the source code, manuals or likewise. The '''community''' sprint is a get-together for GRASS project members and supporters and related [http://www.osgeo.org/ OSGeo] projects to make decisions and tackle larger problems. For this meeting, we welcome people committed to improving the GRASS GIS project and the interfaces to [[QGIS GRASS Cookbook|QGIS]], [[GDAL]], [[PostGIS]], [[R statistics]], [[GRASS and Sextante|Sextante, gvSIG]], OGC Services (esp. [[WPS]]) and more. This includes developers, documenters, bug reporters, translators and others.

For this meeting, we welcome people committed to improving the GRASS GIS and related projects. This includes developers, document writers, wish and bug reporters, translators etc.

== Sponsors ==

We welcome '''financial contributions''' to support the meeting and we are looking for '''sponsors''' to cover costs such as meals or to help reducing travelling and accommodation expenses for GRASS developers with far arrival. If you are interested to sponsor the GRASS Community Sprint, please read about

:::'''sponsoring the GRASS project at [http://grass.osgeo.org/donations/ http://grass.osgeo.org/donations/]'''

For questions, please contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail.com></tt>. Any surplus at the end of the event will be turned over to the GRASS GIS project.

The third GRASS Community Sprint is a great occasion for you to support the development of GRASS. With your contribution you'll enable more developers to meet. The community sprint is an important opportunity for the GRASS developers to discuss and collaboratively resolve bugs, plan the direction for the project and work on new features. Please see below for the more detailed agenda. The developers and contributors are donating their valuable time, so it would be great if in-kind funding can be made available from within the community to cover out-of-pocket expenses. All of the work that takes place at the community sprint will be directly contributed back into the GRASS project to the benefit of everyone who uses it.

== Timing ==

'''When''': 6-12 Feb, 2017

Of course you are free to join or leave the community sprint whenever you want.

'''Duration''': 7 days

== Venue ==

Two rooms (A8 and A9) are booked from 6th to 10th February at University of Genova, Polytechnic School of Engineering in [http://osm.org/go/xX0mu8L~Z?m= Villa Cambiaso]

11th and 12th February TBD

== Accommodation and Costs ==

Some beds are available in Luca's granma house, others are available in the FOSS4G.it website

== Agenda ==

=== Timeline ===

==== Monday, 6 February ====

* End Transifex setting

==== Tuesday, 7 February ====

==== Wednesday, 8 February ====

==== Thursday, 9 February ====

* Official FOSS4G.it dinner

==== Friday, 10 February ====

* Unofficial FOSS4G.it dinner

==== Saturday, 11 February ====

* Santa Margherita and Portofino visit related to OSM mapping party. capabilities to work there

== Participation ==

=== In person (planned) ===

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
!width=50px|'''Number'''
!width=130px|'''Participant '''
!width=100px|'''Country'''
!width=100px|'''Arrival'''
!width=100px|'''Departure'''
!'''Topic'''
!width=75px|'''T-Shirt'''
!'''Notes'''
|-
|1
|[[User:Lucadelu|Luca Delucchi]]
|Italy
| Feb 3
| Feb 12
|
| L
|
|-
|2
|[[User:Landa|Martin Landa]]
|Czech Republic
| Feb 6
| Feb 12
| Various
| L
|
|-
|3
|<s>[[User:NikosAlexandris|Nikos Alexandris]]</s>
|?
| Feb 3
| Feb 12
| Various
| M
| Unfortunately, not available
|-
|4
|[[User:Elena84m|Elena Mezzini]]
|Italy
| Feb 7 evening
| Feb 10 afternoon
| IT Translator
| M
|}

=== Via IRC chat or hangout===

: [irc://freenode/grass #grass] on Freenode
: [http://www.google.com/+/learnmore/hangouts/ G+ hangout] (to be investigated; smartphones only? is Jitsi on a desktop computer compatible?)

For details, see [[IRC]]

Please add yourself here below (please add also your gmail account for G+ hangout):
* ADD YOUR SELF

== Individual Preparation ==

* Bring your own computer
* Bring [http://en.wikipedia.org/wiki/Mains_electricity_by_country your power connector adapter] if needed ([http://en.wikipedia.org/wiki/File:L_plug.jpg Italy])
* Install subversion and the compiler tools, and come with a working GRASS development environment if possible.

== Photos ==

== FAQ ==

* '''How was it last time?'''
** Very cool, see [[GRASS Community Sprint Bonn 2016]]!

* '''Is the GRASS Community Sprint just a coding event?'''
** It is mainly a coding and documentation event. It is a working session for people who are already participants in the GRASS project and/or are committed to improving the GRASS project.
** On demand, however, we can do some presentations of current working GRASS implementations and new upcoming features to spread the idea of Open Source GIS software.

* '''Is the GRASS Community Sprint for developers only?'''
** ''Not at all!'' Anybody can help with testing, checking out bugs and fixes, documentation and more.

* '''Where can I get help and more information about the community sprint?'''
** Contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail dot com></tt> or [[User:Neteler|Markus Neteler]]

[[Category: Workshops]]
[[Category: Code Sprint]]
[[Category: 2017]]

GRASS Community Sprint Genova 2017

2016-12-10T00:32:05Z

⚠️NikosA:

The GRASS GIS team will organize a '''GRASS Developer and Power User Meeting, aka 'GRASS Community Sprint'''' from '''6-12 Feb, 2017'''. The sprint is at the same time of the XVIII Meeting degli Utenti Italiani Grass e Gfoss, 9-10 Feb 2017, University of Genova, Polytechnic School of Engineering, [http://en.wikipedia.org/wiki/Genoa Genova], [http://en.wikipedia.org/wiki/Italy Italy].

== Purpose ==

This next edition of the GRASS GIS community sprint 2017 is a great occasion for folks to support the development by actively contributing to the source code, manuals or likewise. The '''community''' sprint is a get-together for GRASS project members and supporters and related [http://www.osgeo.org/ OSGeo] projects to make decisions and tackle larger problems. For this meeting, we welcome people committed to improving the GRASS GIS project and the interfaces to [[QGIS GRASS Cookbook|QGIS]], [[GDAL]], [[PostGIS]], [[R statistics]], [[GRASS and Sextante|Sextante, gvSIG]], OGC Services (esp. [[WPS]]) and more. This includes developers, documenters, bug reporters, translators and others.

For this meeting, we welcome people committed to improving the GRASS GIS and related projects. This includes developers, document writers, wish and bug reporters, translators etc.

== Sponsors ==

We welcome '''financial contributions''' to support the meeting and we are looking for '''sponsors''' to cover costs such as meals or to help reducing travelling and accommodation expenses for GRASS developers with far arrival. If you are interested to sponsor the GRASS Community Sprint, please read about

:::'''sponsoring the GRASS project at [http://grass.osgeo.org/donations/ http://grass.osgeo.org/donations/]'''

For questions, please contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail.com></tt>. Any surplus at the end of the event will be turned over to the GRASS GIS project.

The third GRASS Community Sprint is a great occasion for you to support the development of GRASS. With your contribution you'll enable more developers to meet. The community sprint is an important opportunity for the GRASS developers to discuss and collaboratively resolve bugs, plan the direction for the project and work on new features. Please see below for the more detailed agenda. The developers and contributors are donating their valuable time, so it would be great if in-kind funding can be made available from within the community to cover out-of-pocket expenses. All of the work that takes place at the community sprint will be directly contributed back into the GRASS project to the benefit of everyone who uses it.

== Timing ==

'''When''': 6-12 Feb, 2017

Of course you are invited to join or leave the community sprint whenever you want.

'''Duration''': 7 days

== Venue ==

Two rooms (A8 and A9) are booked from 6th to 10th February at University of Genova, Polytechnic School of Engineering in [http://osm.org/go/xX0mu8L~Z?m= Villa Cambiaso]

11th and 12th February TBD

== Accommodation and Costs ==

== Agenda ==

=== Timeline ===

== Participation ==

=== In person (planned) ===

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
!width=50px|'''Number'''
!width=130px|'''Participant '''
!width=100px|'''Country'''
!width=100px|'''Arrival'''
!width=100px|'''Departure'''
!'''Topic'''
!width=75px|'''T-Shirt'''
!'''Notes'''
|-
|1
|[[User:Lucadelu|Luca Delucchi]]
|Italy
| Feb 3
| Feb 12
|
| L
|
|-
|2
|[[User:Ychemin|Yann Chemin]]
|Italy
| Feb 5
| Feb 12
| GDAL-GRASS planetary IAU2000/2009 integration
| XXL
|
|-
|3
|[[User:Landa|Martin Landa]]
|Czech Republic
| Feb 6
| Feb 12
| Various
| L
| Not yet confirmed
|-
|4
|[[User:NikosAlexandris|Nikos Alexandris]]
|?
| Feb 3
| Feb 12
| Various
| M
| Not yet confirmed
|}

=== Via IRC chat or hangout===

: [irc://freenode/grass #grass] on Freenode
: [http://www.google.com/+/learnmore/hangouts/ G+ hangout] (to be investigated; smartphones only? is Jitsi on a desktop computer compatible?)

For details, see [[IRC]]

Please add yourself here below (please add also your gmail account for G+ hangout):
* ADD YOUR SELF

== Individual Preparation ==

* Bring your own computer
* Bring [http://en.wikipedia.org/wiki/Mains_electricity_by_country your power connector adapter] if needed ([http://en.wikipedia.org/wiki/File:L_plug.jpg Italy])
* Install subversion and the compiler tools, and come with a working GRASS development environment if possible.

== Photos ==

== FAQ ==

* '''How was it last time?'''
** Very cool, see [[GRASS Community Sprint Bonn 2016]]!

* '''Is the GRASS Community Sprint just a coding event?'''
** It is mainly a coding and documentation event. It is a working session for people who are already participants in the GRASS project and/or are committed to improving the GRASS project.
** On demand, however, we can do some presentations of current working GRASS implementations and new upcoming features to spread the idea of Open Source GIS software.

* '''Is the GRASS Community Sprint for developers only?'''
** ''Not at all!'' Anybody can help with testing, checking out bugs and fixes, documentation and more.

* '''Where can I get help and more information about the community sprint?'''
** Contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail dot com></tt> or [[User:Neteler|Markus Neteler]]

[[Category: Workshops]]
[[Category: Code Sprint]]
[[Category: 2017]]

Compile and Install

2016-09-11T05:42:29Z

⚠️NikosA:

'''Disclaimer''': This page explains how to turn the GRASS GIS source code into an installable binary package ("compilation") for different operating systems. If you just want to get ready-to-use binaries, go [http://grass.osgeo.org/download/ here], otherwise read on...

== How to do compilation and installation of GRASS GIS? ==

Here we explain the procedure to compile GRASS from SVN, but it also applies to official GRASS 6 releases.

''For installation of precompiled binary packages, see the main [[Installation Guide]].''

For detailed information on compilation, please see the [http://grass.osgeo.org/grass64/source/INSTALL INSTALL] file in the source code.

=== Prerequisites ===

==== Extra libraries ====
GRASS needs at least two extra libraries: PROJ and GDAL/OGR

''Note: if you want to have DBMS support in GDAL (subsequently in GRASS) you have to perform the "Optional" steps below as well.''

* [http://trac.osgeo.org/proj/ PROJ4] for management of projections (with proj-datumgrid-1.3.zip support)
* Optional: [http://trac.osgeo.org/geos/ GEOS]
* Optional: [http://www.postgresql.org PostgreSQL], [http://www.mysql.org mySQL], [http://www.unixodbc.org unixODBC], [http://www.sqlite.org SQLite] (SQLite is needed for [[QGIS]])
* [http://www.gdal.org GDAL/OGR] for reading and writing various GIS data formats (interoperability)

You have to install these two libraries '''first''' (see below how to get them precompiled for your system).

It is easiest to obtain a prepackaged version of these libraries (e.g., .rpm; .deb) for your particular operating system and run the corresponding package installation (e.g., rpm -Uhv packagename.rpm; apt-get) in a terminal window. Take care to also install the development packages of these libraries (...-devel packages). If there is no prepackage version, then you will have to download the source code (see links above, source code packages usually ends in .tar.gz or .zip) and compile it (you must have a C compiler installed as part of your operating system). The Web sites show the steps to compile the libraries.

Other libraries needed to run GRASS are listed on the {{website|grass64/source/REQUIREMENTS.html|requirements page}}.

To compile, you will also need the respective "-devel" packages.

==== Download GRASS GIS source code ====
Then [http://grass.osgeo.org/download/software/sources/ download the GRASS GIS source code] of course.

=== Generic Compilation and installation procedure ===

* It is wise that compilation processes are carried out as a normal user: If you want to get the source code in a place where you do not have write permissions (e.g. in /usr/local/src/) just follow this:
cd /usr/local/src/
su -c 'mkdir grass6'
su -c 'chown yourlogin:yourgroup grass6'

Otherwise if you have permissions just continue as a normal user:
cd /usr/local/src/
svn checkout ...

* do a code checkout from the SVN source code repository
: checkout the latest GRASS 6.x from SVN (see: {{twiki|DownloadSource}})

* in the grass6 directory, you will find the precious INSTALL file, open it with your favourite pager/editor and read it carefully!

* run configure with parameters to adapt the compile process to your own system. To see what options can be passed to it, run:
./configure --help | less


It may (!) look like this:

./configure \
--with-cxx \
--with-sqlite \
--with-postgres-libs=/usr/include/pgsql/libpq \
--with-postgres-includes=/usr/include/pgsql \
--with-freetype \
--with-freetype-includes=/usr/include/freetype2 \
--with-motif \
--with-proj-share=/usr/share/proj

You may have to explicitly state the path for certain packages (i.e., gdal). The Unix 'locate' command will come in handy for finding the path of the package you need (you may have to run locate as root ex: sudo locate gdal-config).

Please note that the paths mentioned may widely vary due to the distribution used.
See [[Compile_and_Install#Platform_Specific_Notes|Platform Specific Notes]] below.

Depending on your needs it may be a good idea to include debugging hooks.
: See [[GRASS_Debugging#Compile_Time_Setup]].
CFLAGS="-ggdb -Wall -Werror-implicit-function-declaration" ./configure ...

At the end of configuration process you should get report not much different from this:

GRASS is now configured for: i686-pc-linux-gnu

Source directory: /usr/src/grass6
Build directory: /usr/src/grass6
Installation directory: /usr/local/grass-6.3.svn
Startup script in directory: ${exec_prefix}/bin
C compiler: gcc -g -O2
C++ compiler: c++ -g -O2
FORTRAN compiler:
Building shared libraries: yes
64bit support: no

NVIZ: yes

BLAS support: no
C++ support: yes
DWG support: no
FFMPEG support: no
FFTW support: yes
FreeType support: yes
GDAL support: yes
GLw support: no
LAPACK support: no
Large File Support (LFS): no
Motif support: no
MySQL support: no
NLS support: no
ODBC support: no
OGR support: yes
OpenGL(R) support: yes
PNG support: yes
PostgreSQL support: yes
Readline support: no
SQLite support: no
Tcl/Tk support: yes
TIFF support: yes
X11 support: yes

* Let's compile it (takes a little while...)!
make
* At the end, you should get report not much different from this:
----------------------------------------------------------------------
Following modules are missing the 'description.html' file in src code:
----------------------------------------------------------------------
GRASS GIS compilation log
-------------------------
Started compilation: Ne kvě 28 13:18:43 CEST 2006
--
Errors in:
--
Finished compilation: Ne kvě 28 13:43:40 CEST 2006
(In case of errors please change into the directory with error and run 'make')

* If there is any error, change directory to directory with error and run "make" again. Report occuring bug to grass mailing list
* Once the installation process is finished, you're ready to install GRASS system wide.
su -c 'make install'
* enjoy GRASS:
grass64

=== What else? ===

If you want to use [http://www.qgis.org QGIS], then also compile the GRASS-GDAL/OGR plugin. This is also useful to access your GRASS-data
from other application using GDAL/OGR like [http://thuban.intevation.de thuban].
* [[Compile and install GRASS and QGIS with GDAL/OGR Plugin]] (enables QGIS to read GRASS data directly)

=== Compile and install GDAL-GRASS plugin ===

* See [[Compile and install GDAL-GRASS plugin]]

=== Platform Specific Notes ===

==== Linux ====

Linux cames in various flavours, i.e. distributions.

===== Debian =====

Read the instructions here:
: http://trac.osgeo.org/grass/browser/grass/branches/develbranch_6/debian/README.debian

# first install PROJ, GDAL, etc.
cd grass64/
# follow instructions in debian/README.debian
fakeroot buildpackage

* Official [http://wiki.debian.org/DebianGis DebianGIS] packaging control files:

<strike>
http://svn.debian.org/viewsvn/pkg-grass/packages/grass/
svn co svn://svn.debian.org/svn/pkg-grass/packages/grass/trunk/

or

svn co svn://svn.debian.org/svn/pkg-grass/packages/grass/branches/<GRASS Version>
</strike>

Available on git server:
git://git.debian.org/git/pkg-grass/grass.git

http://anonscm.debian.org/cgit/pkg-grass/grass.git/

====== GRASS 6.1 on Debian Sarge ======

* [http://hamish.bowman.googlepages.com/debiangisfiles#compile Compiling GRASS 6.1-CVS on Debian/OldStable (aka 3.1, Sarge)]

====== GRASS 6.4 on Debian Lenny ======

Install needed packages:
apt-get install flex bison libreadline-dev libncurses5-dev lesstif2-dev debhelper dpatch libtiff4-dev \
tcl-dev tk-dev libfftw3-dev libxmu-dev libfreetype6-dev autoconf2.13 autotools-dev doxygen \
libmysqlclient15-dev graphviz libsqlite3-dev python-wxgtk2.8 libcairo2-dev libwxgtk2.8-dev \
python-dev libgdal1-dev libgdal1-1.5.0 libproj-dev libproj0 proj-bin proj-data mysql

# Important: avoid using CFLAGS="... -Werror-implicit-function-declaration" (also on later Debian versions), see {{trac|1684}}

Configure:
./configure \
--with-cxx \
--with-sqlite \
--with-postgres --with-postgres-includes=/usr/include/postgresql \
--with-mysql --with-mysql-includes=/usr/include/mysql --with-mysql-libs=/usr/lib/mysql \
--with-odbc \
--with-cairo \
--with-proj-share=/usr/share/proj \
--with-tcltk-includes=/usr/include/tcl8.4/ \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--with-motif --with-fftw --with-nls --with-python \
--with-x --x-libraries=/usr/lib/x86_64-linux-gnu

Compile:
make

Install:
sudo make install

====== GRASS 6.4 on Debian Jessie ======

Install needed packages:
apt-get install flex bison debhelper dpatch autoconf2.13 autotools-dev \
python-dev g++ gcc gettext graphviz libcairo2-dev libfftw3-dev tcl-dev tk-dev \
libfreetype6-dev libgdal1h libgdal1-dev libglu1-mesa-dev libglw1-mesa-dev \
libncurses5-dev libreadline-dev libsqlite3-dev libtiff5-dev \
libwxgtk3.0-dev libxmu-dev libxmu-headers libxt-dev mesa-common-dev \
libproj-dev proj-bin proj-data python-numpy python-wxgtk3.0 subversion wx-common zlib1g-dev \
netcdf-bin libnetcdf-dev libatlas-dev libgegl-dev doxygen postgresql mysql-server \
libgeotiff-dev libblas-dev libgeotiff-epsg libxmu-dev libxt-dev libx11-dev

# Important: avoid using CFLAGS="... -Werror-implicit-function-declaration" (also on later Debian versions), see {{trac|1684}}

Configure:
./configure \
--with-cxx \
--with-sqlite \
--with-postgres \
--with-odbc \
--with-cairo \
--with-geos \
--with-proj-share=/usr/share/proj \
--with-tcltk-includes=/usr/include/tcl8.6/ \
--with-mysql --with-mysql-includes=/usr/include/mysql --with-mysql-libs=/usr/lib/mysql \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--with-motif \
--with-fftw \
--with-nls \
--with-postgres-includes=/usr/include/postgresql \
--with-python \
--with-x --x-libraries=/usr/lib/x86_64-linux-gnu

Compile:
make

Install:
sudo make install

====== GRASS 7 on Debian Squeeze ======

Install needed packages:
apt-get install flex bison debhelper dpatch autoconf2.13 autotools-dev python-dev \
g++ gcc gettext graphviz libcairo2-dev libfftw3-dev libfreetype6-dev \
libgdal1 libgdal1-dev libglu1-mesa-dev libglw1-mesa-dev \
libncurses5-dev libproj-dev libreadline-dev libsqlite3-dev libtiff4-dev \
libwxgtk2.8-dev libxmu-dev libxmu-headers libxt-dev mesa-common-dev \
proj-bin proj-data python-numpy python-wxgtk2.8 subversion wx-common zlib1g-dev

Download source code:

svn checkout https://svn.osgeo.org/grass/grass/trunk grass_trunk

Configure:

cd grass_trunk/
CFLAGS="-Wall -Werror-implicit-function-declaration -fno-common -Wextra -Wunused" \
CXXFLAGS="-Wall" \
./configure --prefix=/usr/local \
--with-gdal --with-proj --with-proj-share=/usr/share \
--with-glw --with-nls --with-readline \
--without-tcltk \
--with-cxx \
--enable-largefile \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--with-sqlite \
--with-cairo --with-python=/usr/bin/python2.6-config --with-wxwidgets \
--with-geos --with-pthread

Compile:
make

Install:
sudo make install # installs as /usr/local/bin/grass71

====== GRASS 7 on Debian Wheezy ======

Install needed packages:

# note: cmake and libboost-all-dev only needed for optional libLAS
sudo apt-get install cmake libboost-all-dev

sudo apt-get install flex bison debhelper dpatch autoconf2.13 autotools-dev \
python-dev g++ gcc gettext graphviz libcairo2-dev libfftw3-dev libfreetype6-dev \
libgdal1 libgdal1-dev libglu1-mesa-dev libglw1-mesa-dev libncurses5-dev libproj-dev \
libreadline-dev libtiff4-dev libwxgtk2.8-dev libxmu-dev libxmu-headers \
libxt-dev mesa-common-dev proj-bin proj-data python-numpy python-wxgtk2.8 subversion wx-common \
zlib1g-dev netcdf-bin libatlas-dev libgegl-dev opencl-headers ocl-icd-libopencl1 \
libsqlite3-dev postgresql python-sphinx python-gdal

Download and Install libLAS
wget http://download.osgeo.org/liblas/libLAS-1.7.0.tar.gz
tar -zxvf libLAS-1.7.0.tar.gz
cd libLAS-1.7.0/
mkdir makefiles
cd makefiles
cmake -G "Unix Makefiles" ../
make
sudo make install
sudo ldconfig
cd ../..

Download of GRASS GIS 7 source code:
wget http://grass.osgeo.org/grass70/source/grass-7.0.0.tar.gz
tar -zxvf grass-7.0.0.tar.gz

Configure:
cd grass-7.0.0
CFLAGS="-g -Wall -Werror-implicit-function-declaration -fno-common -Wextra -Wunused" CXXFLAGS="-g -Wall" \
./configure \
--prefix=/usr/local \
--with-gdal \
--with-proj --with-proj-share=/usr/share \
--with-glw --with-nls --with-readline \
--with-cxx --enable-largefile \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--with-sqlite --with-cairo --with-python=/usr/bin/python-config \
--with-wxwidgets --with-geos --with-blas \
--with-lapack-includes=/usr/lib/lapack --with-liblas=yes \
--with-netcdf=/usr/bin/nc-config --with-odbc=yes \
--with-openmp=yes --with-pthread=yes --with-postgres=yes \
--with-postgres-includes=/usr/include/postgresql \
--with-postgres-libs=/usr/lib/postgresql

# --with-mysql=yes --with-mysql-includes=/usr/include/mysql \
# --with-mysql-libs=/usr/lib/mysql \

GRASS GIS 7 is now configured (example):

Source directory: /home/user/grass-7.0.0
Build directory: /home/user/grass-7.0.0
Installation directory: ${prefix}/grass-7.0.0
Startup script in directory:${exec_prefix}/bin
C compiler: gcc -g -Wall -Werror-implicit-function-declaration -fno-common -Wextra -Wunused
C++ compiler: c++ -g -Wall
Building shared libraries: yes
OpenGL platform: X11
MacOSX application: no
MacOSX architectures:
MacOSX SDK:
BLAS support: yes
C++ support: yes
Cairo support: yes
DWG support: no
FFTW support: yes
FreeType support: yes
GDAL support: yes
GEOS support: yes
LAPACK support: no
Large File support (LFS): yes
libLAS support: yes
MySQL support: no
NetCDF support: yes
NLS support: yes
ODBC support: yes
OGR support: yes
OpenCL support: no
OpenGL support: yes
OpenMP support: yes
PNG support: yes
POSIX thread support: yes
PostgreSQL support: yes
Readline support: yes
Regex support: yes
SQLite support: yes
TIFF support: yes
wxWidgets support: yes
X11 support: no

Compile:
make -j2

Install:
sudo make install # installs as /usr/local/bin/grass70

Launch:
grass70

====== GRASS 7 on Debian Jessie ======

Install needed packages:
sudo apt-get install cmake libboost-all-dev flex bison debhelper dpatch autoconf2.13 \
autotools-dev python-dev g++ gcc gettext graphviz libcairo2-dev libfftw3-dev \
libfreetype6-dev libgdal1h libgdal1-dev libglu1-mesa-dev libglw1-mesa-dev \
libncurses5-dev libproj-dev libreadline-dev libsqlite3-dev libtiff5-dev \
libwxgtk3.0-dev libxmu-dev libxmu-headers libxt-dev mesa-common-dev \
proj-bin proj-data python-numpy python-wxgtk3.0 subversion wx-common zlib1g-dev \
netcdf-bin libnetcdf-dev libgegl-dev doxygen python-sphinx \
postgresql libgeotiff-dev libblas-dev mysql-server \
libatlas-dev libblas-dev liblapack3gf liblapack-dev
# opencl-headers ocl-icd-libopencl1 liblas-bin liblas-c-dev python-gdal

Download and compile libLAS (or simply install 'liblas-c-dev ', see above):
wget http://download.osgeo.org/liblas/libLAS-1.8.0.tar.bz2
tar xjf libLAS-1.8.0.tar.bz2
mkdir libLAS-1.8.0/makefiles
cd libLAS-1.8.0/makefiles
cmake -G "Unix Makefiles" ../
make
sudo make install

Download and compile GRASS 7:
# (check there for later version)
wget http://grass.osgeo.org/grass70/source/grass-7.0.0.tar.gz
tar xzfv grass-7.0.0.tar.gz
cd grass-7.0.0
CFLAGS="-g -Wall -Werror-implicit-function-declaration -fno-common -Wextra -Wunused" \
CXXFLAGS="-g -Wall" \
./configure \
--prefix=/usr/local \
--with-gdal --with-proj --with-proj-share=/usr/share \
--with-glw --with-nls --with-readline \
--with-cxx --enable-largefile \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--with-sqlite --with-cairo --with-python=/usr/bin/python-config \
--with-wxwidgets --with-geos \
--with-blas --with-blas-includes=/usr/include/atlas/ \
--with-lapack --with-lapack-includes=/usr/include/atlas/ \
--with-netcdf=/usr/bin/nc-config --with-odbc=yes \
--with-openmp=yes --with-pthread=no --with-postgres=yes \
--with-postgres-includes=/usr/include/postgresql \
--with-postgres-libs=/usr/lib/postgresql \
--with-mysql=yes --with-mysql-includes=/usr/include/mysql \
--with-mysql-libs=/usr/lib/mysql
make -j2

Install:
sudo make install # installs as /usr/local/bin/grass70

====== GRASS 7 on Debian Stretch ======
(Updated from above with package changes required for Debian Stretch)

Install needed packages:
sudo apt-get install cmake libboost-all-dev flex bison debhelper dpatch autoconf2.13 \
autotools-dev python-dev g++ gcc gettext graphviz libcairo2-dev libfftw3-dev \
libfreetype6-dev libgdal1i libgdal1-dev libglu1-mesa-dev libglw1-mesa-dev \
libncurses5-dev libproj-dev libreadline-dev libsqlite3-dev libtiff5-dev \
libwxgtk3.0-dev libxmu-dev libxmu-headers libxt-dev mesa-common-dev \
proj-bin proj-data python-numpy python-wxgtk3.0 subversion wx-common zlib1g-dev \
netcdf-bin libnetcdf-dev libgegl-dev doxygen python-sphinx \
postgresql libgeotiff-dev libblas-dev mysql-server \
libatlas-dev libblas-dev liblapack3 liblapack-dev \
opencl-headers ocl-icd-libopencl1 liblas-bin liblas-c-dev python-gdal

Configuration:

CFLAGS="-g -Wall -Werror-implicit-function-declaration -fno-common -Wextra -Wunused" \
CXXFLAGS="-g -Wall" \
./configure \
--prefix=/usr/local \
--with-gdal --with-proj --with-proj-share=/usr/share \
--with-glw --with-nls --with-readline \
--with-cxx --enable-largefile \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--with-sqlite --with-cairo --with-python=/usr/bin/python-config \
--with-wxwidgets --with-geos \
--with-blas --with-blas-includes=/usr/include/atlas/ \
--with-lapack --with-lapack-includes=/usr/include/atlas/ \
--with-netcdf=/usr/bin/nc-config --with-odbc=yes \
--with-openmp=yes --with-pthread=no --with-postgres=yes \
--with-postgres-includes=/usr/include/postgresql \
--with-postgres-libs=/usr/lib/postgresql \
--with-mysql=yes --with-mysql-includes=/usr/include/mysql \
--with-mysql-libs=/usr/lib/mysql

Compile:

make -j2

Install:

make install

===== Ubuntu =====

There is a dedicated page on how to [[Compile_and_Install_Ubuntu | Compile the GRASS GIS source code & install it in Ubuntu]].

===== Linux Mint =====

Install packages needed for compilation:
<pre>
sudo apt-get install build-essential flex make bison gcc libgcc1 g++ \
cmake ccache python python-dev python-qt4 python-qt4-dev python-opengl \
python-wxversion python-wxtools python-wxgtk2.8 python-dateutil \
libgsl0-dev python-numpy wx2.8-headers wx-common libwxgtk2.8-dev \
libwxgtk2.8-dbg libwxbase2.8-dev libwxbase2.8-dbg \
zlib1g-dev gettext libtiff-dev libpnglite-dev \
libcairo2 libcairo2-dev sqlite3 libsqlite3-dev libpq-dev libreadline6 \
libreadline6-dev libfreetype6-dev txt2tags libfftw3-3 libfftw3-dev \
lsb-desktop libapt-pkg-perl resolvconf libjasper-dev subversion \
libswscale-dev checkinstall libglu1-mesa-dev libxmu-dev \
libproj-dev proj-bin proj-data gdal-bin python-gdal libgdal1-dev

# GRASS GIS 6 only: libncurses5-dev tcl8.5-dev tk8.5-dev
# Extra: ffmpeg ffmpeg2theora libffmpegthumbnailer-dev libavcodec-dev libavformat-dev

CFLAGS="-Wall" ./configure --with-cxx --with-gdal \
--with-proj --with-proj-share=/usr/share/proj/ \
--with-geos --without-postgres \
--with-nls --with-opengl-includes=/usr/include/GL/ \
--with-fftw --with-freetype \
--with-freetype-includes=/usr/include/freetype2 --with-sqlite \
--with-python=/usr/bin/python2.7-config --with-wx=/usr/bin/wx-config \
--with-wxwidgets --with-cairo --enable-largefile

# G6: --with-tcltk-includes=/usr/include/tcl8.5/ \

make

sudo make install
</pre>

===== Mandriva =====

Installation of dependencies (urpmi will ask you a few more):

'''Mandriva 2009:''' (take out the '64' everywhere if you are on 32bit)
# as root
urpmi flex bison zlib-devel tiff-devel png-devel tcl-devel tk-devel sqlite3-devel \
mesagl1-devel mesaglu1-devel lib64xmu6-devel gcc-c++ gettext \
lib64wxgtk2.8 lib64wxgtk2.8-devel lib64wxgtkgl2.8 wxgtk2.8 \
lib64wxPythonGTK2.8 lib64wxPythonGTK2.8-devel wxPythonGTK wxPythonGTK-wxversion
exit

'''Mandriva 2010:''' (take out the '64' everywhere if you are on 32bit) - see also [http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/grass/current/SPECS/ SPEC] file
# as root
# installation of PROJ and GDAL
urpmi proj proj-devel gdal gdal-devel gcc-gfortran lib64openssl1.0.0 \
lib64openssl1.0.0-devel postgresql8.4-devel lib64pq8.4

# installation of compilation environment
urpmi flex bison zlib-devel tiff-devel png-devel tcl-devel tk-devel sqlite3-devel \
lib64mesagl1-devel lib64mesaglu1-devel lib64xmu6-devel gcc-c++ gettext \
lib64wxgtk2.8 lib64wxgtk2.8-devel lib64wxgtkgl2.8 wxgtk2.8 \
lib64wxPythonGTK2.8 lib64wxPythonGTK2.8-devel wxPythonGTK wxPythonGTK-wxversion
exit

Then, to configure GRASS, run (64 bit stuff optional of course):
# as user
./configure \
--enable-64bit --with-libs=/usr/lib64 \
--with-cxx \
--with-gdal=/usr/local/bin/gdal-config \
--with-sqlite \
--with-nls \
--with-python \
--with-wxwidgets=/usr/lib/wxPython/bin/wx-config \
--with-fftw \
--with-ffmpeg --with-ffmpeg-includes="/usr/include/libav* /usr/include/libpostproc /usr/include/libswscale" \
--with-motif \
--with-mysql --with-mysql-includes=/usr/include/mysql --with-mysql-libs=/usr/lib64 \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--enable-largefile

# compilation (use -j2 ior -j4 parameter on multi-core CPUs to accelerate):
make

Installation:
su
# this will install into /usr/local/
make install
exit

===== Enterprise Linux =====

Enterprise Linux (EL) and derivatives (that is, Red Hat Enterprise Linux, CentOS and Scientific Linux) is a popular and robust platform for servers and computing-heavy workstations, and is therefore a good fit for GIS specific requirements.

See: http://wiki.osgeo.org/wiki/Enterprise_Linux_GIS

===== Scientific Linux =====

See Centos

===== CentOS =====

You first need to add the EPEL repository for PROJ.4 and GDAL, see [https://fedoraproject.org/wiki/EPEL/FAQ#How_can_I_install_the_packages_from_the_EPEL_software_repository.3F here]

<strike>
Preparation '''Centos 5''' (old):
yum install epel-release

# Note: CentOS 5 comes with Python 2.4 which lacks python-config, hence two extra tweaks are needed.
yum install flex bison zlib-devel tcl-devel tk-devel gcc-c++ gettext \
libtiff-devel libpng-devel sqlite-devel \
mesa-libGL-devel mesa-libGLU-devel mesa-libGLw-devel \
mesa-libOSMesa-devel libXmu-devel python-devel gtk2-devel\
ncurses-devel postgresql-devel make
</strike>

Preparation '''Centos 6''' | '''Scientific Linux 6''':
yum install epel-release

yum install flex bison make zlib-devel gcc-c++ gettext \
sqlite-devel mesa-libGL-devel mesa-libGLU-devel \
libXmu-devel libX11-devel fftw-devel libtiff-devel \
lesstif-devel python-devel numpy wxPython wxGTK-devel \
proj proj-devel proj-epsg proj-nad libxml2 gdal gdal-devel geos geos-devel \
python-dateutil python-imaging python-matplotlib
# only GRASS GIS 6: install also tcl-devel tk-devel

Preparation '''Centos 7''' | '''Scientific Linux 7''':
yum install epel-release

yum install flex bison make zlib-devel gcc-c++ gettext \
sqlite-devel mesa-libGL-devel mesa-libGLU-devel \
libXmu-devel libX11-devel fftw-devel libtiff-devel \
lesstif-devel python-devel numpy wxPython wxGTK-devel \
proj proj-devel proj-epsg proj-nad libxml2 gdal gdal-devel geos geos-devel \
netcdf netcdf-devel blas-devel lapack-devel atlas-devel \
python-dateutil python-imaging python-matplotlib python-sphinx \
doxygen subversion

'''GRASS GIS 7 compilation and installation'''

[http://grass.osgeo.org/download/software/sources/ Download source code] (e.g., --> "Download latest 7.0-svn code (SVN))" and configure GRASS 7 (suggestion: save this as script):

./configure \
--with-cxx \
--enable-largefile \
--with-proj --with-proj-share=/usr/share/proj \
--with-gdal=/usr/bin/gdal-config \
--with-sqlite \
--with-python \
--with-cairo --with-cairo-ldflags=-lfontconfig \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--with-wxwidgets=/usr/bin/wx-config \
--with-openmp \
--with-blas --with-blas-includes=/usr/include/atlas-x86_64-base/ \
--with-lapack --with-lapack-includes=/usr/include/atlas-x86_64-base/ \
--with-fftw \
--with-geos \
--with-netcdf \
--without-ffmpeg \
--without-mysql \
--without-postgres \
--without-odbc \
--without-fftw

Compile:
make
or on a multicore system (number depends of available cores):
make -j4

For a system wide installation, run as root user:
make install
OR simply run it directly from the source code directory (substitute ARCH with i686 or x86_64):

bin.$ARCH/grass70 -gui

In this case, for convenience, add it to the search path:
mkdir $HOME/bin
cd $HOME/bin
# example:
ln -s $HOME/software/grass70_release/bin.i686-pc-linux-gnu/grass70 .

Now use it subsequently with
grass70 -gui

===== Gentoo =====

./configure \
--with-freetype=yes --with-freetype-includes="/usr/include/freetype2/"

See also http://packages.gentoo.org/package/sci-geosciences/grass

===== Funtoo =====

{{MovedToTrac|FuntooLinux}}

===== Fedora =====

====== GRASS GIS 6 on Fedora ======

'''Preparation''' for the compilation of GRASS GIS 6 source code (F16-F21):

yum install proj-devel gdal-devel sqlite-devel ffmpeg-devel mesa-libGL-devel \
mesa-libGLU-devel libXmu-devel libX11-devel tcl-devel tk-devel geos \
fftw-devel libtiff-devel lesstif-devel python-devel numpy wxPython wxGTK-devel \
gcc gcc-c++ bison flex ncurses-devel proj-epsg proj-nad xml2 subversion

''Note 1: that currently gdal-devel has (too) many dependencies and may lead to a massive download of extra packages (200 on a fresh Fedora 16 install). Hopefully solved in later Fedora versions.''

''Note 2: the optional ffmpeg-devel comes from the rpmfusion-free repository ([http://rpmfusion.org/Configuration/ configuration]).''

'''Download''' the source code:
* GRASS GIS 6: [http://grass.osgeo.org/download/software/sources/ release package] - [http://grass.osgeo.org/grass64/source/snapshot/ weekly snapshot] - [http://trac.osgeo.org/grass/wiki/DownloadSource#GRASS6.4 SVN repository]

'''Configure''': This is an ''example'' how to configure the source code on a Fedora system:

./configure \
--with-cxx \
--with-gdal=/usr/bin/gdal-config \
--with-proj --with-proj-share=/usr/share/proj \
--with-sqlite \
--with-nls \
--with-geos \
--with-wxwidgets=/usr/bin/wx-config \
--with-python=/usr/bin/python-config \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--enable-largefile \
--without-odbc \
--with-fftw

Extra: If you also want '''FFMPEG''' support - it requires 'yum install fftw-devel'. Then add to the configuration lines above:

--with-ffmpeg --with-ffmpeg-includes="/usr/include/ffmpeg /usr/include/ffmpeg/libav* /usr/include/ffmpeg/libpostproc /usr/include/ffmpeg/libswscale" \

Finally '''compile''' the configured source code:
make
or on multicore (number depends of available cores):
make -j4
and
make install # requires root permissions unless you become owner of /usr/local/

Then use GRASS GIS 6 and enjoy!

====== GRASS GIS 7 on Fedora ======

'''Preparation''' for the compilation of GRASS GIS 7 source code (F20-F22):

dnf install gcc gcc-c++ bison flex ncurses-devel proj-epsg proj-nad xml2 \
proj-devel gdal gdal-devel gdal-python sqlite-devel mesa-libGL-devel \
fftw-devel mesa-libGLU-devel libXmu-devel libX11-devel geos geos-devel \
libtiff-devel lesstif-devel python-devel numpy wxPython wxGTK-devel \
python-dateutil python-imaging python-matplotlib-wx subversion doxygen python-sphinx

# optionally also: netcdf-devel postgresql-devel

''Note 1: the optional ffmpeg-devel comes from the rpmfusion-free repository ([http://rpmfusion.org/Configuration/ configuration]).''

'''Download''' the source code:
* GRASS GIS 7: [http://grass.osgeo.org/grass70/source/snapshot/ weekly snapshot] - [http://trac.osgeo.org/grass/wiki/DownloadSource#GRASS7 SVN repository]

'''Configure''': This is an example how to configure the source code on a Fedora system:

./configure \
--with-cxx \
--with-gdal=/usr/bin/gdal-config \
--with-proj --with-proj-share=/usr/share/proj \
--with-python=/usr/bin/python-config \
--with-geos \
--with-sqlite \
--with-nls \
--with-wxwidgets=/usr/bin/wx-config \
--with-fftw \
--with-cairo --with-cairo-ldflags=-lfontconfig \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--enable-largefile \
--without-odbc

''Extra 1:'' If you also want '''FFMPEG support''' - it requires 'dnf install fftw-devel' from the rpmfusion-free repository. Then add to the configuration lines above:

--with-ffmpeg --with-ffmpeg-includes="/usr/include/ffmpeg /usr/include/ffmpeg/libav* /usr/include/ffmpeg/libpostproc /usr/include/ffmpeg/libswscale" \

''Extra 2:'' For '''LAPACK/BLAS support''' (addons i.spec.unmix, v.kriging etc), install "dnf install lapack lapack-devel blas blas-devel atlas atlas-devel". Then add to the configuration lines above:
--with-blas --with-blas-includes=/usr/include/atlas-x86_64-base/ \
--with-lapack --with-lapack-includes=/usr/include/atlas-x86_64-base/ \

Finally '''compile''' the configured source code:
make
or on multicore (number depends of available cores):
make -j4
and
make install # requires root permissions unless you become owner of /usr/local/

Then use GRASS GIS 7 and enjoy!

===== openSUSE =====

To build GRASS on openSUSE:

RPM packages to be installed:
<source lang="bash">
sudo zypper install bison flex freetype2-devel fftw3-devel gcc-c++ \
libgdal-devel libgeos-devel libjpeg-devel libpng-devel libtiff-devel \
man proj libproj-devel readline-devel netcdf-devel ncurses-devel \
mysql-devel postgresql-devel sqlite-devel unixODBC-devel \
tcl-devel tk-devel xorg-x11-Mesa-devel libXmu-devel \
python-numpy python-dateutil python-devel python-opengl \
python-wxWidgets python-xml python-dateutil wxWidgets-devel \
zlib-devel doxygen python-sphinx
</source>

Source code configuration:
<source lang="bash">
./configure \
--enable-largefile \
--with-proj-share=/usr/share/proj \
--with-cxx \
--with-lapack=yes \
--with-x \
--with-motif \
--with-gdal=/usr/bin/gdal-config \
--with-postgres --with-postgres-includes=/usr/include/pgsql \
--with-mysql --with-mysql-includes=/usr/include/mysql \
--with-fftw \
--with-readline \
--with-netcdf \
--with-curses \
--with-geos \
--with-nls \
--with-sqlite \
--with-freetype \
--with-freetype-includes=/usr/include/freetype2 \
--with-wxwidgets \
--with-odbc \
--with-python
</source>

Then compile with "make [-j2]".

===== Arch Linux =====

The easiest/fastest way is to build GRASS GIS using AUR:

https://aur.archlinux.org/packages/?O=0&K=grass (msieczka: I would recommend my set of PKBUILDs - https://aur.archlinux.org/packages/?K=czk&SeB=m. They are most feature-rich, up-to-date and allow to have various versions installed alongside each other without conflicts.)

But if you want to compile it yourself you have to keep in mind that in Arch Linux the default Python version 3.

python --version

Python 3.3.1

Therefore, to build GRASS GIS (version 7 in this example), we can use either use virtualenv or symlink python2 and python2-config in a given directory.
Let's use the latter approach

sudo ln -s /usr/bin/python2 $HOME/usr/bin/python
sudo ln -s /usr/bin/python2-config $HOME/usr/bin/python-config

We can now start building GRASS 7

export PATH=~/usr/bin:$PATH && \
./configure \
--enable-debug \
--enable-64bit \
--with-libs=/usr/lib64 \
--with-cxx \
--with-readline \
--with-zlib-includes="/usr/include/" \
--with-freetype=yes \
--with-freetype-includes="/usr/include/freetype2/" \
--enable-largefile=yes \
--with-gdal=/usr/bin/gdal-config \
--with-proj-share=/usr/share/proj/ \
--with-geos=/usr/bin/geos-config \
--with-cairo \
--with-odbc \
--with-pthread \
--with-liblas=/usr/local/bin/liblas-config \
--with-fftw-includes="/usr/include/" \
--with-fftw-libs=/usr/lib/ \
--with-tcltk-includes="/usr/include/" \
--with-wxwidgets \
--with-postgres=yes \
--with-postgres-includes="/usr/include/postgresql/internal" \
--with-postgres-libs="/usr/include/postgresql/internal/libpq" \
--with-sqlite=yes \
--with-python=yes \
--with-liblas \
--with-netcdf

Now let's compile and install it

make
sudo make install

Now the final step required to use GRASS with python2 is to create a bash script "grass" (or the name you want)

#!/bin/bash
export PATH=~/usr/bin:$PATH
python2 /usr/local/bin/grass70

And make it executable and copy it somewhere in the PATH

chmod+x grass
sudo cp grass /usr/local/bin

It's done, you can enjoy GRASS 7 in Arch, just type "grass" (or the name you gave to the script) in the terminal to launch it.

===== RPM SPEC files =====
* ... can be found in the source code, rpm/ directory,
* or [https://build.opensuse.org/package/show?package=grass&project=Application%3AGeo OpenSuSe]
* or [https://admin.fedoraproject.org/pkgdb/acls/name/grass Fedora]
* or [http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/grass/ Mandriva] (there are also [http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/proj/ proj4], [http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/geos/ geos], [http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/gdal/ gdal], [http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/gdal-grass/ gdal-grass-plugin], [http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/qgis/ qgis] etc)
* ...

===== Zaurus =====

... see [http://wiki.debian.org/?GrassGISonZaurus here] for instructions

==== Mac OSX ====

===== Conventional installation =====
* see the source/macosx readme ([http://trac.osgeo.org/grass/browser/grass/trunk/macosx/ReadMe.rtf Trac link])
* main page [[Compiling on MacOSX]]
* main page [[Packaging on MacOSX]]
* [http://grass.osgeo.org/download/software/mac-osx/ official download site]
* [http://www.kyngchaos.com/software/frameworks#build_scripts build scripts for dependencies]
* solving errors at starting GRASS GIS: [[MacOSX GRASS errors]]
=====Compiling on MacOSX using homebrew =====

* See [[Compiling on MacOSX using homebrew]]

==== FreeBSD / NetBSD ====

The recommended compiler tools are GCC, GNU make, GNU coreutils (for install), and flex. All are available through the respective package managing tools (pkg_add for FreeBSD and pkgin install for NetBSD) and for recent *BSD versions most likely installed by default.

GRASS 6.x and GRASS 7 should compile on FreeBSD 8.0 or later and NetBSD 5.0 or later (maybe also on updated NetBSD 4.x).

It is highly recommended to install GDAL/OGR and PROJ4 first. These libraries and tools are available as [http://www.freebsd.org/ports/ ports for FreeBSD] and [http://www.pkgsrc.org/ packages for NetBSD].

Optional functionality is listed with ''./configure --help'', and related libraries and tools might need to be installed first.

You then may need to edit include/Make/Platform.make and set
ICONVLIB = -liconv

==== Solaris ====

* ''2008 Oct 15'': see [http://lists.osgeo.org/pipermail/grass-user/2008-October/047093.html this post on the grass mailing list]

===== 11 SPARC/i86pc =====

The recommended compiler tools are GCC, GNU make, GNU coreutils (for install), and flex. All are available through the Solaris package manager.

Most dependencies are available through the Solaris package manager. GDAL and proj4 can either be compiled from source or installed e.g. from [http://www.opencsw.org/ OpenCSW]. If packages are installed from OpenCSW, the linker flags need to be set with

LDFLAGS="-Wl,-R/opt/csw/lib -L/opt/csw/lib -Wl,-R/opt/csw/gxx/lib -L/opt/csw/gxx/lib"

See also the [http://www.opencsw.org/use-it/ OpenCSW documentation].

===== 10 SPARC/i86pc =====

* get gcc compiler and tools. There are several sources: Solaris Companion CD (SFW pkg, installs in /opt/sfw/), Blastwave ([http://www.blastwave.org], CSW pkg, installs in /opt/csw/) or Sunfreeware ([http://www.sunfreeware.com], SMC pkg, installs in /usr/local/).
Needed Packages from Sunfreeware: SMCbinut, SMCbison, SMCcoreu, SMCfindu, SMCflex, SMCgawk, SMCgcc, SMCgrep, SMCgzip, SMCless, SMClibt, SMClicon, SMCmake, SMCncurs, SMCproj, SMCsed, SMCtar, SMCtcl, SMCtiff, SMCtk, SMCunzip, SMCzlib.

* compile and install fftw-library ([http://www.fftw.org]). You need to re-compile the library with:

./configure --with-pic --enable-shared; make ; make install.

The pre-built packages don't work.

* compile and install gdal library (see documentation of gdal, [http://www.gdal.org]).

* compile and install any additional libraries (e. g. GEOS, [http://geos.refractions.net]).

* set compiler flags and path. e. g.:

# on ultra-sparc machine:
CFLAGS="-O3 -mcpu=v9"
CXXFLAGS="-O3 -mcpu=v9"
PATH="/usr/local/bin:/opt/sfw/bin:/usr/ccs/bin:/usr/bin:/usr/sbin"
export CFLAGS CXXFLAGS PATH

Path has to be changed for the packages (Sunfreeware: /usr/local/bin, Solaris Companion: /opt/sfw/bin, Blastwave: /opt/csw/bin).

* Next configure, e. g.:

./configure --with-postgres-includes=/usr/include/pgsql/ \
--with-postgres-libs=/usr/lib --with-postgres=yes \
--with-includes=/usr/local/include/ncurses

If you use n(ew)curses, you have to include the path /usr/local/include/ncurses.

then:

make
su
make install

If the shared libraries are not found at runtime of the modules, use 'crle' to add the paths of the libraries for the dynamic linker, e. g. as root:

crle -l /lib:/usr/lib:/usr/local/lib:/opt/sfw/lib:/usr/X11/lib

Be careful not to omit a library path, the system may be unusable if you forget the /lib path.

==== AIX ====

A recent [ftp://ftp.gnu.org/gnu/make/ GNU make] (>= 3.81) and GNU coreutils are required. These are available with the [http://www-03.ibm.com/systems/power/software/aix/linux/toolbox/download.html IBM AIX toolbox] or through third-party AIX software repositories, e.g. [http://www.bullfreeware.com/ bullfreeware] and [http://www.perzl.org/aix/ perzl.org]. Note that 'make' does not work, only 'gmake' works.

General instructions to compile on AIX are e.g. [http://www.perzl.org/aix/index.php?n=Main.Instructions here]

On AIX, compilation results by default in 32 bit applications and static libraries. The [http://www.ibm.com/developerworks/aix/library/au-gnu.html IBM documentation] explains how to build 64 bit applications and shared libraries with GCC.

'''GRASS 6: Using the IBM xlc compiler:'''
* ''see [http://thread.gmane.org/gmane.comp.gis.grass.user/32667 this mailing list thread]''

Mike wrote in 2009:

After attempting all the suggestions, I finally used
--disable-shared on the configure command, and all but
a handful of modules successfully compiled. I was able to
individually address the ones that failed through Makefile
edits and several small source code/header file edits.

The environment variables and configure command that worked were:
<pre>
# xlc compiler:
export PATH=/usr/local/bin:/opt/freeware/bin:$PATH
export OBJECT_MODE=64
export LIBICONV=/opt/freeware
export CC="xlc_r -q64"
export CFLAGS="-O -qstrict"
export CXX="xlC_r -q64"
export CXXFLAGS="-O -qstrict"
export AR="ar -X64"
export F77="xlf_r -q64"
export CPPFLAGS="-I/afs/isis/pkg/libpng/include -I/usr/local/include -I$LIBICONV/include -I/usr/lpp/X11/include/X11"
export LDFLAGS="-L/usr/local/lib -L$LIBICONV/lib -L/usr/lib -L/usr/X11R6/lib -lc"

./configure --prefix=/afs/isis/pkg/grass-6.4.0 \
--enable-64bit \
--disable-shared \
--with-includes="/usr/include/fontconfig /usr/include/X11 /usr/include/X11/Xft /usr/include/X11/ext" \
--x-includes=/usr/include/X11 \
--x-libraries=/usr/X11R6/lib \
--with-fftw-includes=/afs/isis/pkg/fftw-3.2.2/include \
--with-fftw-libs=/afs/isis/pkg/fftw-3.2.2/lib \
--with-gdal=/afs/isis/pkg/gdal/bin/gdal-config \
--with-proj-includes=/afs/isis/pkg/proj/include \
--with-proj-libs=/afs/isis/pkg/proj/lib \
--with-proj-share=/afs/isis/pkg/proj/share/proj \
--with-tcltk-includes=/usr/local/include \
--with-tcltk-libs=/usr/local/lib \
--with-opengl-includes=/usr/include/GL
</pre>

'''GRASS 7: Using the IBM xlc compiler:'''

Get and install (in this order):
* [ftp://ftp.gnu.org/gnu/tar/ GNU tar]
* [ftp://ftp.gnu.org/gnu/make/ GNU make] (>= 3.81)
* [http://www.sqlite.org SQLite]
* [http://trac.osgeo.org/proj/ PROJ.4] (for ./configure parameters, see [[Talk:Compile and Install|here]])
* [http://www.gdal.org GDAL] (for ./configure parameters, see [[Talk:Compile and Install|here]])
* [http://grass.osgeo.org/grass70/source/snapshot/ GRASS GIS 7]

The environment variables and configure command that worked:
<pre>
## AIX 5.x

export LIBPATH=/opt/freeware/lib64:/opt/freeware/lib
# make 'ar' happy
export OBJECT_MODE=64

PREFIX=$HOME/private/bin
./configure \
--prefix=$PREFIX \
--disable-shared \
--enable-largefile \
--with-cxx \
--with-proj-includes=$PREFIX/include/ \
--with-proj-libs=$PREFIX/lib/ \
--with-proj-share=$PREFIX/share/proj/ \
--with-sqlite \
--with-sqlite-includes=$PREFIX/include/ \
--with-sqlite-libs=$PREFIX/lib/ \
--with-tiff=no \
--with-png=no \
--with-fftw=no \
--with-cairo=no \
--with-freetype=no
</pre>

'''GRASS 7: Using the GNU gcc compiler:'''

Get and install (in this order):
* if needed, [ftp://ftp.gnu.org/gnu/tar/ GNU tar]
* if needed (the name is often 'gmake' and not 'make') [ftp://ftp.gnu.org/gnu/make/ GNU make] (>= 3.81)
* [http://www.sqlite.org SQLite]
* [http://trac.osgeo.org/proj/ PROJ.4] (for ./configure parameters, see [[Talk:Compile and Install|here]])
* [http://www.gdal.org GDAL] (for ./configure parameters, see [[Talk:Compile and Install|here]])
* [http://grass.osgeo.org/grass70/source/snapshot/ GRASS GIS 7]

The environment variables and configure command that worked for AIX 5.x:
<pre>
## AIX 5.x

export LIBPATH=/opt/freeware/lib64:/opt/freeware/lib
# make 'ar' happy
export OBJECT_MODE=64

PREFIX=$HOME/private/bin
CFLAGS='-ansi -D_ALL_SOURCE=1 -D_POSIX_SOURCE=1 -D_POSIX_C_SOURCE=200809L -Dinline=' ./configure \
--prefix=$PREFIX \
--disable-shared \
--enable-largefile \
--with-cxx \
--with-proj-includes=$PREFIX/include/ \
--with-proj-libs=$PREFIX/lib/ \
--with-proj-share=$PREFIX/share/proj/ \
--with-sqlite \
--with-sqlite-includes=$PREFIX/include/ \
--with-sqlite-libs=$PREFIX/lib/ \
--with-tiff=no \
--with-png=no \
--with-fftw=no \
--with-cairo=no \
--with-freetype=no
</pre>

The environment variables and configure command that worked for AIX 7.x:
<pre>
## AIX 7.x
export CC="gcc"
export CXX="g++"
PREFIX=$HOME/bin

LDFLAGS="-Wl,-bsvr4,-R,/opt/freeware/lib -L/opt/freeware/lib" CPPFLAGS="-I/opt/freeware/include" ./configure \
--prefix=$PREFIX \
--enable-largefile \
--enable-shared \
--with-includes=/opt/freeware/include --with-libs=/opt/freeware/lib \
--with-cxx \
--with-proj-includes=$PREFIX/include/ \
--with-proj-libs=$PREFIX/lib/ \
--with-proj-share=$PREFIX/share/proj/ \
--with-gdal=$PREFIX/bin/gdal-config \
--with-sqlite \
--with-sqlite-libs=$PREFIX/lib --with-sqlite-includes=$PREFIX/include \
--with-png=no \
--with-tiff=no \
--with-fftw=no \
--with-cairo=no \
--with-opengl=no \
--with-freetype=no
</pre>

==== MS-Windows ====

===== MS-Windows/Cygwin =====

* See the [[Cygwin]] wiki pages

===== MS-Windows/native =====

====== Compile ======

* [http://trac.osgeo.org/grass/wiki/CompileOnWindows GRASS Windows Native Binary Building Guide] (GRASS 6.4.x + 7.x)
* <strike>[http://www.webalice.it/marco.pasetti/grass/BuildFromSource.html GRASS Windows Native Binary Building Guide] (GRASS 6.3.x) </strike>
* See/adapt [http://blog.qgis.org/node/124 idea] for unattended install of QGIS (et al) from [http://trac.osgeo.org/osgeo4w/ OSGeo4W] from the QuantumGIS Blog.

See also [[WinGRASS Current Status]] for latest updates.

=== Common problems and solutions ===

During compilation, error can occur if certain packages are not installed. Here a list of problems with solution:

* error: X11/Xlib.h: No such file or directory
** this suggests that you don't have the X headers installed
** Solution: Install the libx11-dev package

* error: g.list: error while loading shared libraries: libgdal1.6.0.so.1: cannot open shared object file: No such file or directory
** this error appears in the shell right after the user clicks GUI's "Start GRASS" button. The GUI shows an error about geographic extent and gets closed afterwards.
** It happens when you launch bin.i686 executable on 64bit system. Be careful and choose the right architecture.

=== Static compilation ===

In order to get static rather than dynamically linked binaries, configure like this:

./configure --disable-shared --enable-static

This will however break the wxGUI and GRASS 7 completely because "ctypes" wants to link against shared libs, or there is something in the static libs that "ctypes" does not like.

=== Optimization ===

GCC and other compilers support [http://gcc.gnu.org/onlinedocs/gcc-4.3.3/gcc/Optimize-Options.html#Optimize-Options optimization]

If you would like to set compiler optimisations, for a possibly faster binary, type (don't enter a ";" anywhere):

CFLAGS=-O ./configure
or,
setenv CFLAGS -O
./configure

whichever works on your shell. Use -O2 instead of -O if your compiler supports this (note: O is the letter, not zero). Using the "gcc" compiler, you can also specify processor specific flags (examples, please suggest better settings to us):

CFLAGS="-mcpu=athlon -O2" # AMD Athlon processor with code optimisations
CFLAGS="-march=amdfam10" # AMD Phenom II X4 64bit processor with gcc >=4.3
CFLAGS="-mcpu=pentium" # Intel Pentium processor
CFLAGS="-mcpu=pentium4" # Intel Pentium4 processor
CFLAGS="-O2 -msse -msse2 -mfpmath=sse -minline-all-stringops" # Intel XEON 64bit processor
CFLAGS="-mtune=nocona -m64 -minline-all-stringops" # Intel Pentium 64bit processor

To find out optional CFLAGS for your platform, enter:
gcc -dumpspecs

See also: http://gcc.gnu.org/

A real fast GRASS version (and small binaries) will be created with LDFLAGS set to "stripping" (but this disables debugging):

CFLAGS="-O2 -mcpu=<cpu_see_above> -Wall" LDFLAGS="-s" ./configure

=== Configure options and their meanings ===

For configure there are many options and some GRASS modules are built only if some options are set. Here are listed common configuration options with short explanation.

* --prefix=/path - Sets path where GRASS will be installed. GRASS will reside in /path/grass-version.
* --enable-largefile - Enables large (>2Gb on 32bit systems) support. For current large file support status look at [[Large File Support]] page.
* --with-cxx - Enables compilation of C++ code. Required for r.terraflow module.
* --with-readline - Enables readline support. If readline is enabled, you can use its history/editing facilities when entering r.mapcalc expressions on stdin.
* --with-glw - Enables GLw support. The GLw library provides OpenGL "canvas" widgets for Athena and Motif.

That switch is unnecessary for normal compilation. It's only
required for r3.showdspf, which isn't normally built; if you
want it, you have build it manually
(e.g. "make -C raster3d/r3.showdspf").
As similar functionality is now provided by NVIZ, r3.showdspf
is deprecated.
r3.showdspf uses the Motif widget (so you also need a
Motif library, e.g. Lesstif or OpenMotif).
[http://grass.itc.it/pipermail/grassuser/2006-December/037475.html Glynn Clements at GRASS-user mailing list]

=== Parallelized compilation on multi-core CPUs ===

You can dramatically accelerate the compilation of the GRASS code with the -j flag of "make" if you have a multi-core CPU system. This determines the maximum number of jobs to have running at once, so cores don't have to sit idle waiting for jobs on other cores to complete. A good rule of thumb for this value is <tt>number_of_cores * 1.5</tt>, but note that setting any higher than the actual number of cores will only affect the timing slightly. For example, on a dual-core processor:
make -j 4



== GRASS-GDAL plugin ==

* see [[Compile and install GRASS and QGIS with GDAL/OGR Plugin]]

== Addons ==

Please note that the installation of [[AddOns/GRASS_6|Addons]] can be easily done with the {{cmd|g.extension}} manager. The compile instructions below are aiming at own development.

=== Compiled modules ===

This is the general way which works for C modules as well as scripts (Python or Shell/Bash modules) which has HTML documentation.

'''Requirements:'''

Either:
* a binary GRASS package, or
* source code which has been prepared with:
./configure [opionally flags]
make libs

Each of the [[GRASS_AddOns|addon]] modules should come with a Makefile. To compile it, just run:
make MODULE_TOPDIR=/path/to/grass64/

Note that the <tt>/path/to/grass64</tt> has to be absolute, not relative.

If using Bash it may be useful to set that up as an alias:
alias gmake64='make MODULE_TOPDIR=/path/to/grass64/'

Installation (perhaps requires "sudo"):
make MODULE_TOPDIR=/path/to/grass64/ install

Note: Compiled addons may require a re-compilation if you changed/updated your GRASS standard binaries.

==== If binary comes with a -dev package ====

''(work in progress, this text states how it eventually will be :)''
Nowadays one does not need to the source code, nor compiling GRASS by oneself to be able to add add-ons. On Debian, you can just install the grass-dev package and then run:
make MODULE_TOPDIR=/usr/lib/grass64/ INST_DIR=/usr/lib/grass64/

The grass-dev package essentially provides GRASS's <tt>include</tt> header files and Make configuration files.

=== Scripts ===

If the addon module is a script, it is sufficient to copy it into the (GRASS binaries) path somewhere. Alternatively, install addons into a separate GRASS addons binaries/scripts directory which is easier to maintain. It avoids getting clobbered every time you reinstall GRASS. To use these separately stored scripts, set and export the GRASS_ADDON_PATH environment variable before starting GRASS and it will automatically be added to the module search path (see the {{cmd|variables}} help page). To simplify this, do for example:

# add in $HOME/.bashrc:
GRASS_ADDON_PATH=/usr/local/grass/addons/
export GRASS_ADDON_PATH

Make sure that the script is executable, then just call it in GRASS typing the filename. Python scripts need to be called writing the extension as well, like:

GRASS 6.5.svn (spearfish60):~ > v.krige.py

[[Category:Development]]
[[Category:Documentation]]
[[Category:Installation]]

Compile and Install

2016-09-11T05:41:35Z

⚠️NikosA: Funtoo MovedToTrac

'''Disclaimer''': This page explains how to turn the GRASS GIS source code into an installable binary package ("compilation") for different operating systems. If you just want to get ready-to-use binaries, go [http://grass.osgeo.org/download/ here], otherwise read on...

== How to do compilation and installation of GRASS GIS? ==

Here we explain the procedure to compile GRASS from SVN, but it also applies to official GRASS 6 releases.

''For installation of precompiled binary packages, see the main [[Installation Guide]].''

For detailed information on compilation, please see the [http://grass.osgeo.org/grass64/source/INSTALL INSTALL] file in the source code.

=== Prerequisites ===

==== Extra libraries ====
GRASS needs at least two extra libraries: PROJ and GDAL/OGR

''Note: if you want to have DBMS support in GDAL (subsequently in GRASS) you have to perform the "Optional" steps below as well.''

* [http://trac.osgeo.org/proj/ PROJ4] for management of projections (with proj-datumgrid-1.3.zip support)
* Optional: [http://trac.osgeo.org/geos/ GEOS]
* Optional: [http://www.postgresql.org PostgreSQL], [http://www.mysql.org mySQL], [http://www.unixodbc.org unixODBC], [http://www.sqlite.org SQLite] (SQLite is needed for [[QGIS]])
* [http://www.gdal.org GDAL/OGR] for reading and writing various GIS data formats (interoperability)

You have to install these two libraries '''first''' (see below how to get them precompiled for your system).

It is easiest to obtain a prepackaged version of these libraries (e.g., .rpm; .deb) for your particular operating system and run the corresponding package installation (e.g., rpm -Uhv packagename.rpm; apt-get) in a terminal window. Take care to also install the development packages of these libraries (...-devel packages). If there is no prepackage version, then you will have to download the source code (see links above, source code packages usually ends in .tar.gz or .zip) and compile it (you must have a C compiler installed as part of your operating system). The Web sites show the steps to compile the libraries.

Other libraries needed to run GRASS are listed on the {{website|grass64/source/REQUIREMENTS.html|requirements page}}.

To compile, you will also need the respective "-devel" packages.

==== Download GRASS GIS source code ====
Then [http://grass.osgeo.org/download/software/sources/ download the GRASS GIS source code] of course.

=== Generic Compilation and installation procedure ===

* It is wise that compilation processes are carried out as a normal user: If you want to get the source code in a place where you do not have write permissions (e.g. in /usr/local/src/) just follow this:
cd /usr/local/src/
su -c 'mkdir grass6'
su -c 'chown yourlogin:yourgroup grass6'

Otherwise if you have permissions just continue as a normal user:
cd /usr/local/src/
svn checkout ...

* do a code checkout from the SVN source code repository
: checkout the latest GRASS 6.x from SVN (see: {{twiki|DownloadSource}})

* in the grass6 directory, you will find the precious INSTALL file, open it with your favourite pager/editor and read it carefully!

* run configure with parameters to adapt the compile process to your own system. To see what options can be passed to it, run:
./configure --help | less


It may (!) look like this:

./configure \
--with-cxx \
--with-sqlite \
--with-postgres-libs=/usr/include/pgsql/libpq \
--with-postgres-includes=/usr/include/pgsql \
--with-freetype \
--with-freetype-includes=/usr/include/freetype2 \
--with-motif \
--with-proj-share=/usr/share/proj

You may have to explicitly state the path for certain packages (i.e., gdal). The Unix 'locate' command will come in handy for finding the path of the package you need (you may have to run locate as root ex: sudo locate gdal-config).

Please note that the paths mentioned may widely vary due to the distribution used.
See [[Compile_and_Install#Platform_Specific_Notes|Platform Specific Notes]] below.

Depending on your needs it may be a good idea to include debugging hooks.
: See [[GRASS_Debugging#Compile_Time_Setup]].
CFLAGS="-ggdb -Wall -Werror-implicit-function-declaration" ./configure ...

At the end of configuration process you should get report not much different from this:

GRASS is now configured for: i686-pc-linux-gnu

Source directory: /usr/src/grass6
Build directory: /usr/src/grass6
Installation directory: /usr/local/grass-6.3.svn
Startup script in directory: ${exec_prefix}/bin
C compiler: gcc -g -O2
C++ compiler: c++ -g -O2
FORTRAN compiler:
Building shared libraries: yes
64bit support: no

NVIZ: yes

BLAS support: no
C++ support: yes
DWG support: no
FFMPEG support: no
FFTW support: yes
FreeType support: yes
GDAL support: yes
GLw support: no
LAPACK support: no
Large File Support (LFS): no
Motif support: no
MySQL support: no
NLS support: no
ODBC support: no
OGR support: yes
OpenGL(R) support: yes
PNG support: yes
PostgreSQL support: yes
Readline support: no
SQLite support: no
Tcl/Tk support: yes
TIFF support: yes
X11 support: yes

* Let's compile it (takes a little while...)!
make
* At the end, you should get report not much different from this:
----------------------------------------------------------------------
Following modules are missing the 'description.html' file in src code:
----------------------------------------------------------------------
GRASS GIS compilation log
-------------------------
Started compilation: Ne kvě 28 13:18:43 CEST 2006
--
Errors in:
--
Finished compilation: Ne kvě 28 13:43:40 CEST 2006
(In case of errors please change into the directory with error and run 'make')

* If there is any error, change directory to directory with error and run "make" again. Report occuring bug to grass mailing list
* Once the installation process is finished, you're ready to install GRASS system wide.
su -c 'make install'
* enjoy GRASS:
grass64

=== What else? ===

If you want to use [http://www.qgis.org QGIS], then also compile the GRASS-GDAL/OGR plugin. This is also useful to access your GRASS-data
from other application using GDAL/OGR like [http://thuban.intevation.de thuban].
* [[Compile and install GRASS and QGIS with GDAL/OGR Plugin]] (enables QGIS to read GRASS data directly)

=== Compile and install GDAL-GRASS plugin ===

* See [[Compile and install GDAL-GRASS plugin]]

=== Platform Specific Notes ===

==== Linux ====

Linux cames in various flavours, i.e. distributions.

===== Debian =====

Read the instructions here:
: http://trac.osgeo.org/grass/browser/grass/branches/develbranch_6/debian/README.debian

# first install PROJ, GDAL, etc.
cd grass64/
# follow instructions in debian/README.debian
fakeroot buildpackage

* Official [http://wiki.debian.org/DebianGis DebianGIS] packaging control files:

<strike>
http://svn.debian.org/viewsvn/pkg-grass/packages/grass/
svn co svn://svn.debian.org/svn/pkg-grass/packages/grass/trunk/

or

svn co svn://svn.debian.org/svn/pkg-grass/packages/grass/branches/<GRASS Version>
</strike>

Available on git server:
git://git.debian.org/git/pkg-grass/grass.git

http://anonscm.debian.org/cgit/pkg-grass/grass.git/

====== GRASS 6.1 on Debian Sarge ======

* [http://hamish.bowman.googlepages.com/debiangisfiles#compile Compiling GRASS 6.1-CVS on Debian/OldStable (aka 3.1, Sarge)]

====== GRASS 6.4 on Debian Lenny ======

Install needed packages:
apt-get install flex bison libreadline-dev libncurses5-dev lesstif2-dev debhelper dpatch libtiff4-dev \
tcl-dev tk-dev libfftw3-dev libxmu-dev libfreetype6-dev autoconf2.13 autotools-dev doxygen \
libmysqlclient15-dev graphviz libsqlite3-dev python-wxgtk2.8 libcairo2-dev libwxgtk2.8-dev \
python-dev libgdal1-dev libgdal1-1.5.0 libproj-dev libproj0 proj-bin proj-data mysql

# Important: avoid using CFLAGS="... -Werror-implicit-function-declaration" (also on later Debian versions), see {{trac|1684}}

Configure:
./configure \
--with-cxx \
--with-sqlite \
--with-postgres --with-postgres-includes=/usr/include/postgresql \
--with-mysql --with-mysql-includes=/usr/include/mysql --with-mysql-libs=/usr/lib/mysql \
--with-odbc \
--with-cairo \
--with-proj-share=/usr/share/proj \
--with-tcltk-includes=/usr/include/tcl8.4/ \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--with-motif --with-fftw --with-nls --with-python \
--with-x --x-libraries=/usr/lib/x86_64-linux-gnu

Compile:
make

Install:
sudo make install

====== GRASS 6.4 on Debian Jessie ======

Install needed packages:
apt-get install flex bison debhelper dpatch autoconf2.13 autotools-dev \
python-dev g++ gcc gettext graphviz libcairo2-dev libfftw3-dev tcl-dev tk-dev \
libfreetype6-dev libgdal1h libgdal1-dev libglu1-mesa-dev libglw1-mesa-dev \
libncurses5-dev libreadline-dev libsqlite3-dev libtiff5-dev \
libwxgtk3.0-dev libxmu-dev libxmu-headers libxt-dev mesa-common-dev \
libproj-dev proj-bin proj-data python-numpy python-wxgtk3.0 subversion wx-common zlib1g-dev \
netcdf-bin libnetcdf-dev libatlas-dev libgegl-dev doxygen postgresql mysql-server \
libgeotiff-dev libblas-dev libgeotiff-epsg libxmu-dev libxt-dev libx11-dev

# Important: avoid using CFLAGS="... -Werror-implicit-function-declaration" (also on later Debian versions), see {{trac|1684}}

Configure:
./configure \
--with-cxx \
--with-sqlite \
--with-postgres \
--with-odbc \
--with-cairo \
--with-geos \
--with-proj-share=/usr/share/proj \
--with-tcltk-includes=/usr/include/tcl8.6/ \
--with-mysql --with-mysql-includes=/usr/include/mysql --with-mysql-libs=/usr/lib/mysql \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--with-motif \
--with-fftw \
--with-nls \
--with-postgres-includes=/usr/include/postgresql \
--with-python \
--with-x --x-libraries=/usr/lib/x86_64-linux-gnu

Compile:
make

Install:
sudo make install

====== GRASS 7 on Debian Squeeze ======

Install needed packages:
apt-get install flex bison debhelper dpatch autoconf2.13 autotools-dev python-dev \
g++ gcc gettext graphviz libcairo2-dev libfftw3-dev libfreetype6-dev \
libgdal1 libgdal1-dev libglu1-mesa-dev libglw1-mesa-dev \
libncurses5-dev libproj-dev libreadline-dev libsqlite3-dev libtiff4-dev \
libwxgtk2.8-dev libxmu-dev libxmu-headers libxt-dev mesa-common-dev \
proj-bin proj-data python-numpy python-wxgtk2.8 subversion wx-common zlib1g-dev

Download source code:

svn checkout https://svn.osgeo.org/grass/grass/trunk grass_trunk

Configure:

cd grass_trunk/
CFLAGS="-Wall -Werror-implicit-function-declaration -fno-common -Wextra -Wunused" \
CXXFLAGS="-Wall" \
./configure --prefix=/usr/local \
--with-gdal --with-proj --with-proj-share=/usr/share \
--with-glw --with-nls --with-readline \
--without-tcltk \
--with-cxx \
--enable-largefile \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--with-sqlite \
--with-cairo --with-python=/usr/bin/python2.6-config --with-wxwidgets \
--with-geos --with-pthread

Compile:
make

Install:
sudo make install # installs as /usr/local/bin/grass71

====== GRASS 7 on Debian Wheezy ======

Install needed packages:

# note: cmake and libboost-all-dev only needed for optional libLAS
sudo apt-get install cmake libboost-all-dev

sudo apt-get install flex bison debhelper dpatch autoconf2.13 autotools-dev \
python-dev g++ gcc gettext graphviz libcairo2-dev libfftw3-dev libfreetype6-dev \
libgdal1 libgdal1-dev libglu1-mesa-dev libglw1-mesa-dev libncurses5-dev libproj-dev \
libreadline-dev libtiff4-dev libwxgtk2.8-dev libxmu-dev libxmu-headers \
libxt-dev mesa-common-dev proj-bin proj-data python-numpy python-wxgtk2.8 subversion wx-common \
zlib1g-dev netcdf-bin libatlas-dev libgegl-dev opencl-headers ocl-icd-libopencl1 \
libsqlite3-dev postgresql python-sphinx python-gdal

Download and Install libLAS
wget http://download.osgeo.org/liblas/libLAS-1.7.0.tar.gz
tar -zxvf libLAS-1.7.0.tar.gz
cd libLAS-1.7.0/
mkdir makefiles
cd makefiles
cmake -G "Unix Makefiles" ../
make
sudo make install
sudo ldconfig
cd ../..

Download of GRASS GIS 7 source code:
wget http://grass.osgeo.org/grass70/source/grass-7.0.0.tar.gz
tar -zxvf grass-7.0.0.tar.gz

Configure:
cd grass-7.0.0
CFLAGS="-g -Wall -Werror-implicit-function-declaration -fno-common -Wextra -Wunused" CXXFLAGS="-g -Wall" \
./configure \
--prefix=/usr/local \
--with-gdal \
--with-proj --with-proj-share=/usr/share \
--with-glw --with-nls --with-readline \
--with-cxx --enable-largefile \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--with-sqlite --with-cairo --with-python=/usr/bin/python-config \
--with-wxwidgets --with-geos --with-blas \
--with-lapack-includes=/usr/lib/lapack --with-liblas=yes \
--with-netcdf=/usr/bin/nc-config --with-odbc=yes \
--with-openmp=yes --with-pthread=yes --with-postgres=yes \
--with-postgres-includes=/usr/include/postgresql \
--with-postgres-libs=/usr/lib/postgresql

# --with-mysql=yes --with-mysql-includes=/usr/include/mysql \
# --with-mysql-libs=/usr/lib/mysql \

GRASS GIS 7 is now configured (example):

Source directory: /home/user/grass-7.0.0
Build directory: /home/user/grass-7.0.0
Installation directory: ${prefix}/grass-7.0.0
Startup script in directory:${exec_prefix}/bin
C compiler: gcc -g -Wall -Werror-implicit-function-declaration -fno-common -Wextra -Wunused
C++ compiler: c++ -g -Wall
Building shared libraries: yes
OpenGL platform: X11
MacOSX application: no
MacOSX architectures:
MacOSX SDK:
BLAS support: yes
C++ support: yes
Cairo support: yes
DWG support: no
FFTW support: yes
FreeType support: yes
GDAL support: yes
GEOS support: yes
LAPACK support: no
Large File support (LFS): yes
libLAS support: yes
MySQL support: no
NetCDF support: yes
NLS support: yes
ODBC support: yes
OGR support: yes
OpenCL support: no
OpenGL support: yes
OpenMP support: yes
PNG support: yes
POSIX thread support: yes
PostgreSQL support: yes
Readline support: yes
Regex support: yes
SQLite support: yes
TIFF support: yes
wxWidgets support: yes
X11 support: no

Compile:
make -j2

Install:
sudo make install # installs as /usr/local/bin/grass70

Launch:
grass70

====== GRASS 7 on Debian Jessie ======

Install needed packages:
sudo apt-get install cmake libboost-all-dev flex bison debhelper dpatch autoconf2.13 \
autotools-dev python-dev g++ gcc gettext graphviz libcairo2-dev libfftw3-dev \
libfreetype6-dev libgdal1h libgdal1-dev libglu1-mesa-dev libglw1-mesa-dev \
libncurses5-dev libproj-dev libreadline-dev libsqlite3-dev libtiff5-dev \
libwxgtk3.0-dev libxmu-dev libxmu-headers libxt-dev mesa-common-dev \
proj-bin proj-data python-numpy python-wxgtk3.0 subversion wx-common zlib1g-dev \
netcdf-bin libnetcdf-dev libgegl-dev doxygen python-sphinx \
postgresql libgeotiff-dev libblas-dev mysql-server \
libatlas-dev libblas-dev liblapack3gf liblapack-dev
# opencl-headers ocl-icd-libopencl1 liblas-bin liblas-c-dev python-gdal

Download and compile libLAS (or simply install 'liblas-c-dev ', see above):
wget http://download.osgeo.org/liblas/libLAS-1.8.0.tar.bz2
tar xjf libLAS-1.8.0.tar.bz2
mkdir libLAS-1.8.0/makefiles
cd libLAS-1.8.0/makefiles
cmake -G "Unix Makefiles" ../
make
sudo make install

Download and compile GRASS 7:
# (check there for later version)
wget http://grass.osgeo.org/grass70/source/grass-7.0.0.tar.gz
tar xzfv grass-7.0.0.tar.gz
cd grass-7.0.0
CFLAGS="-g -Wall -Werror-implicit-function-declaration -fno-common -Wextra -Wunused" \
CXXFLAGS="-g -Wall" \
./configure \
--prefix=/usr/local \
--with-gdal --with-proj --with-proj-share=/usr/share \
--with-glw --with-nls --with-readline \
--with-cxx --enable-largefile \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--with-sqlite --with-cairo --with-python=/usr/bin/python-config \
--with-wxwidgets --with-geos \
--with-blas --with-blas-includes=/usr/include/atlas/ \
--with-lapack --with-lapack-includes=/usr/include/atlas/ \
--with-netcdf=/usr/bin/nc-config --with-odbc=yes \
--with-openmp=yes --with-pthread=no --with-postgres=yes \
--with-postgres-includes=/usr/include/postgresql \
--with-postgres-libs=/usr/lib/postgresql \
--with-mysql=yes --with-mysql-includes=/usr/include/mysql \
--with-mysql-libs=/usr/lib/mysql
make -j2

Install:
sudo make install # installs as /usr/local/bin/grass70

====== GRASS 7 on Debian Stretch ======
(Updated from above with package changes required for Debian Stretch)

Install needed packages:
sudo apt-get install cmake libboost-all-dev flex bison debhelper dpatch autoconf2.13 \
autotools-dev python-dev g++ gcc gettext graphviz libcairo2-dev libfftw3-dev \
libfreetype6-dev libgdal1i libgdal1-dev libglu1-mesa-dev libglw1-mesa-dev \
libncurses5-dev libproj-dev libreadline-dev libsqlite3-dev libtiff5-dev \
libwxgtk3.0-dev libxmu-dev libxmu-headers libxt-dev mesa-common-dev \
proj-bin proj-data python-numpy python-wxgtk3.0 subversion wx-common zlib1g-dev \
netcdf-bin libnetcdf-dev libgegl-dev doxygen python-sphinx \
postgresql libgeotiff-dev libblas-dev mysql-server \
libatlas-dev libblas-dev liblapack3 liblapack-dev \
opencl-headers ocl-icd-libopencl1 liblas-bin liblas-c-dev python-gdal

Configuration:

CFLAGS="-g -Wall -Werror-implicit-function-declaration -fno-common -Wextra -Wunused" \
CXXFLAGS="-g -Wall" \
./configure \
--prefix=/usr/local \
--with-gdal --with-proj --with-proj-share=/usr/share \
--with-glw --with-nls --with-readline \
--with-cxx --enable-largefile \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--with-sqlite --with-cairo --with-python=/usr/bin/python-config \
--with-wxwidgets --with-geos \
--with-blas --with-blas-includes=/usr/include/atlas/ \
--with-lapack --with-lapack-includes=/usr/include/atlas/ \
--with-netcdf=/usr/bin/nc-config --with-odbc=yes \
--with-openmp=yes --with-pthread=no --with-postgres=yes \
--with-postgres-includes=/usr/include/postgresql \
--with-postgres-libs=/usr/lib/postgresql \
--with-mysql=yes --with-mysql-includes=/usr/include/mysql \
--with-mysql-libs=/usr/lib/mysql

Compile:

make -j2

Install:

make install

===== Ubuntu =====

There is a dedicated page on how to [[Compile_and_Install_Ubuntu | Compile the GRASS GIS source code & install it in Ubuntu]].

===== Linux Mint =====

Install packages needed for compilation:
<pre>
sudo apt-get install build-essential flex make bison gcc libgcc1 g++ \
cmake ccache python python-dev python-qt4 python-qt4-dev python-opengl \
python-wxversion python-wxtools python-wxgtk2.8 python-dateutil \
libgsl0-dev python-numpy wx2.8-headers wx-common libwxgtk2.8-dev \
libwxgtk2.8-dbg libwxbase2.8-dev libwxbase2.8-dbg \
zlib1g-dev gettext libtiff-dev libpnglite-dev \
libcairo2 libcairo2-dev sqlite3 libsqlite3-dev libpq-dev libreadline6 \
libreadline6-dev libfreetype6-dev txt2tags libfftw3-3 libfftw3-dev \
lsb-desktop libapt-pkg-perl resolvconf libjasper-dev subversion \
libswscale-dev checkinstall libglu1-mesa-dev libxmu-dev \
libproj-dev proj-bin proj-data gdal-bin python-gdal libgdal1-dev

# GRASS GIS 6 only: libncurses5-dev tcl8.5-dev tk8.5-dev
# Extra: ffmpeg ffmpeg2theora libffmpegthumbnailer-dev libavcodec-dev libavformat-dev

CFLAGS="-Wall" ./configure --with-cxx --with-gdal \
--with-proj --with-proj-share=/usr/share/proj/ \
--with-geos --without-postgres \
--with-nls --with-opengl-includes=/usr/include/GL/ \
--with-fftw --with-freetype \
--with-freetype-includes=/usr/include/freetype2 --with-sqlite \
--with-python=/usr/bin/python2.7-config --with-wx=/usr/bin/wx-config \
--with-wxwidgets --with-cairo --enable-largefile

# G6: --with-tcltk-includes=/usr/include/tcl8.5/ \

make

sudo make install
</pre>

===== Mandriva =====

Installation of dependencies (urpmi will ask you a few more):

'''Mandriva 2009:''' (take out the '64' everywhere if you are on 32bit)
# as root
urpmi flex bison zlib-devel tiff-devel png-devel tcl-devel tk-devel sqlite3-devel \
mesagl1-devel mesaglu1-devel lib64xmu6-devel gcc-c++ gettext \
lib64wxgtk2.8 lib64wxgtk2.8-devel lib64wxgtkgl2.8 wxgtk2.8 \
lib64wxPythonGTK2.8 lib64wxPythonGTK2.8-devel wxPythonGTK wxPythonGTK-wxversion
exit

'''Mandriva 2010:''' (take out the '64' everywhere if you are on 32bit) - see also [http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/grass/current/SPECS/ SPEC] file
# as root
# installation of PROJ and GDAL
urpmi proj proj-devel gdal gdal-devel gcc-gfortran lib64openssl1.0.0 \
lib64openssl1.0.0-devel postgresql8.4-devel lib64pq8.4

# installation of compilation environment
urpmi flex bison zlib-devel tiff-devel png-devel tcl-devel tk-devel sqlite3-devel \
lib64mesagl1-devel lib64mesaglu1-devel lib64xmu6-devel gcc-c++ gettext \
lib64wxgtk2.8 lib64wxgtk2.8-devel lib64wxgtkgl2.8 wxgtk2.8 \
lib64wxPythonGTK2.8 lib64wxPythonGTK2.8-devel wxPythonGTK wxPythonGTK-wxversion
exit

Then, to configure GRASS, run (64 bit stuff optional of course):
# as user
./configure \
--enable-64bit --with-libs=/usr/lib64 \
--with-cxx \
--with-gdal=/usr/local/bin/gdal-config \
--with-sqlite \
--with-nls \
--with-python \
--with-wxwidgets=/usr/lib/wxPython/bin/wx-config \
--with-fftw \
--with-ffmpeg --with-ffmpeg-includes="/usr/include/libav* /usr/include/libpostproc /usr/include/libswscale" \
--with-motif \
--with-mysql --with-mysql-includes=/usr/include/mysql --with-mysql-libs=/usr/lib64 \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--enable-largefile

# compilation (use -j2 ior -j4 parameter on multi-core CPUs to accelerate):
make

Installation:
su
# this will install into /usr/local/
make install
exit

===== Enterprise Linux =====

Enterprise Linux (EL) and derivatives (that is, Red Hat Enterprise Linux, CentOS and Scientific Linux) is a popular and robust platform for servers and computing-heavy workstations, and is therefore a good fit for GIS specific requirements.

See: http://wiki.osgeo.org/wiki/Enterprise_Linux_GIS

===== Scientific Linux =====

See Centos

===== CentOS =====

You first need to add the EPEL repository for PROJ.4 and GDAL, see [https://fedoraproject.org/wiki/EPEL/FAQ#How_can_I_install_the_packages_from_the_EPEL_software_repository.3F here]

<strike>
Preparation '''Centos 5''' (old):
yum install epel-release

# Note: CentOS 5 comes with Python 2.4 which lacks python-config, hence two extra tweaks are needed.
yum install flex bison zlib-devel tcl-devel tk-devel gcc-c++ gettext \
libtiff-devel libpng-devel sqlite-devel \
mesa-libGL-devel mesa-libGLU-devel mesa-libGLw-devel \
mesa-libOSMesa-devel libXmu-devel python-devel gtk2-devel\
ncurses-devel postgresql-devel make
</strike>

Preparation '''Centos 6''' | '''Scientific Linux 6''':
yum install epel-release

yum install flex bison make zlib-devel gcc-c++ gettext \
sqlite-devel mesa-libGL-devel mesa-libGLU-devel \
libXmu-devel libX11-devel fftw-devel libtiff-devel \
lesstif-devel python-devel numpy wxPython wxGTK-devel \
proj proj-devel proj-epsg proj-nad libxml2 gdal gdal-devel geos geos-devel \
python-dateutil python-imaging python-matplotlib
# only GRASS GIS 6: install also tcl-devel tk-devel

Preparation '''Centos 7''' | '''Scientific Linux 7''':
yum install epel-release

yum install flex bison make zlib-devel gcc-c++ gettext \
sqlite-devel mesa-libGL-devel mesa-libGLU-devel \
libXmu-devel libX11-devel fftw-devel libtiff-devel \
lesstif-devel python-devel numpy wxPython wxGTK-devel \
proj proj-devel proj-epsg proj-nad libxml2 gdal gdal-devel geos geos-devel \
netcdf netcdf-devel blas-devel lapack-devel atlas-devel \
python-dateutil python-imaging python-matplotlib python-sphinx \
doxygen subversion

'''GRASS GIS 7 compilation and installation'''

[http://grass.osgeo.org/download/software/sources/ Download source code] (e.g., --> "Download latest 7.0-svn code (SVN))" and configure GRASS 7 (suggestion: save this as script):

./configure \
--with-cxx \
--enable-largefile \
--with-proj --with-proj-share=/usr/share/proj \
--with-gdal=/usr/bin/gdal-config \
--with-sqlite \
--with-python \
--with-cairo --with-cairo-ldflags=-lfontconfig \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--with-wxwidgets=/usr/bin/wx-config \
--with-openmp \
--with-blas --with-blas-includes=/usr/include/atlas-x86_64-base/ \
--with-lapack --with-lapack-includes=/usr/include/atlas-x86_64-base/ \
--with-fftw \
--with-geos \
--with-netcdf \
--without-ffmpeg \
--without-mysql \
--without-postgres \
--without-odbc \
--without-fftw

Compile:
make
or on a multicore system (number depends of available cores):
make -j4

For a system wide installation, run as root user:
make install
OR simply run it directly from the source code directory (substitute ARCH with i686 or x86_64):

bin.$ARCH/grass70 -gui

In this case, for convenience, add it to the search path:
mkdir $HOME/bin
cd $HOME/bin
# example:
ln -s $HOME/software/grass70_release/bin.i686-pc-linux-gnu/grass70 .

Now use it subsequently with
grass70 -gui

===== Gentoo =====

./configure \
--with-freetype=yes --with-freetype-includes="/usr/include/freetype2/"

See also http://packages.gentoo.org/package/sci-geosciences/grass

{{MovedToTrac|FuntooLinux}}

===== Fedora =====

====== GRASS GIS 6 on Fedora ======

'''Preparation''' for the compilation of GRASS GIS 6 source code (F16-F21):

yum install proj-devel gdal-devel sqlite-devel ffmpeg-devel mesa-libGL-devel \
mesa-libGLU-devel libXmu-devel libX11-devel tcl-devel tk-devel geos \
fftw-devel libtiff-devel lesstif-devel python-devel numpy wxPython wxGTK-devel \
gcc gcc-c++ bison flex ncurses-devel proj-epsg proj-nad xml2 subversion

''Note 1: that currently gdal-devel has (too) many dependencies and may lead to a massive download of extra packages (200 on a fresh Fedora 16 install). Hopefully solved in later Fedora versions.''

''Note 2: the optional ffmpeg-devel comes from the rpmfusion-free repository ([http://rpmfusion.org/Configuration/ configuration]).''

'''Download''' the source code:
* GRASS GIS 6: [http://grass.osgeo.org/download/software/sources/ release package] - [http://grass.osgeo.org/grass64/source/snapshot/ weekly snapshot] - [http://trac.osgeo.org/grass/wiki/DownloadSource#GRASS6.4 SVN repository]

'''Configure''': This is an ''example'' how to configure the source code on a Fedora system:

./configure \
--with-cxx \
--with-gdal=/usr/bin/gdal-config \
--with-proj --with-proj-share=/usr/share/proj \
--with-sqlite \
--with-nls \
--with-geos \
--with-wxwidgets=/usr/bin/wx-config \
--with-python=/usr/bin/python-config \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--enable-largefile \
--without-odbc \
--with-fftw

Extra: If you also want '''FFMPEG''' support - it requires 'yum install fftw-devel'. Then add to the configuration lines above:

--with-ffmpeg --with-ffmpeg-includes="/usr/include/ffmpeg /usr/include/ffmpeg/libav* /usr/include/ffmpeg/libpostproc /usr/include/ffmpeg/libswscale" \

Finally '''compile''' the configured source code:
make
or on multicore (number depends of available cores):
make -j4
and
make install # requires root permissions unless you become owner of /usr/local/

Then use GRASS GIS 6 and enjoy!

====== GRASS GIS 7 on Fedora ======

'''Preparation''' for the compilation of GRASS GIS 7 source code (F20-F22):

dnf install gcc gcc-c++ bison flex ncurses-devel proj-epsg proj-nad xml2 \
proj-devel gdal gdal-devel gdal-python sqlite-devel mesa-libGL-devel \
fftw-devel mesa-libGLU-devel libXmu-devel libX11-devel geos geos-devel \
libtiff-devel lesstif-devel python-devel numpy wxPython wxGTK-devel \
python-dateutil python-imaging python-matplotlib-wx subversion doxygen python-sphinx

# optionally also: netcdf-devel postgresql-devel

''Note 1: the optional ffmpeg-devel comes from the rpmfusion-free repository ([http://rpmfusion.org/Configuration/ configuration]).''

'''Download''' the source code:
* GRASS GIS 7: [http://grass.osgeo.org/grass70/source/snapshot/ weekly snapshot] - [http://trac.osgeo.org/grass/wiki/DownloadSource#GRASS7 SVN repository]

'''Configure''': This is an example how to configure the source code on a Fedora system:

./configure \
--with-cxx \
--with-gdal=/usr/bin/gdal-config \
--with-proj --with-proj-share=/usr/share/proj \
--with-python=/usr/bin/python-config \
--with-geos \
--with-sqlite \
--with-nls \
--with-wxwidgets=/usr/bin/wx-config \
--with-fftw \
--with-cairo --with-cairo-ldflags=-lfontconfig \
--with-freetype --with-freetype-includes=/usr/include/freetype2 \
--enable-largefile \
--without-odbc

''Extra 1:'' If you also want '''FFMPEG support''' - it requires 'dnf install fftw-devel' from the rpmfusion-free repository. Then add to the configuration lines above:

--with-ffmpeg --with-ffmpeg-includes="/usr/include/ffmpeg /usr/include/ffmpeg/libav* /usr/include/ffmpeg/libpostproc /usr/include/ffmpeg/libswscale" \

''Extra 2:'' For '''LAPACK/BLAS support''' (addons i.spec.unmix, v.kriging etc), install "dnf install lapack lapack-devel blas blas-devel atlas atlas-devel". Then add to the configuration lines above:
--with-blas --with-blas-includes=/usr/include/atlas-x86_64-base/ \
--with-lapack --with-lapack-includes=/usr/include/atlas-x86_64-base/ \

Finally '''compile''' the configured source code:
make
or on multicore (number depends of available cores):
make -j4
and
make install # requires root permissions unless you become owner of /usr/local/

Then use GRASS GIS 7 and enjoy!

===== openSUSE =====

To build GRASS on openSUSE:

RPM packages to be installed:
<source lang="bash">
sudo zypper install bison flex freetype2-devel fftw3-devel gcc-c++ \
libgdal-devel libgeos-devel libjpeg-devel libpng-devel libtiff-devel \
man proj libproj-devel readline-devel netcdf-devel ncurses-devel \
mysql-devel postgresql-devel sqlite-devel unixODBC-devel \
tcl-devel tk-devel xorg-x11-Mesa-devel libXmu-devel \
python-numpy python-dateutil python-devel python-opengl \
python-wxWidgets python-xml python-dateutil wxWidgets-devel \
zlib-devel doxygen python-sphinx
</source>

Source code configuration:
<source lang="bash">
./configure \
--enable-largefile \
--with-proj-share=/usr/share/proj \
--with-cxx \
--with-lapack=yes \
--with-x \
--with-motif \
--with-gdal=/usr/bin/gdal-config \
--with-postgres --with-postgres-includes=/usr/include/pgsql \
--with-mysql --with-mysql-includes=/usr/include/mysql \
--with-fftw \
--with-readline \
--with-netcdf \
--with-curses \
--with-geos \
--with-nls \
--with-sqlite \
--with-freetype \
--with-freetype-includes=/usr/include/freetype2 \
--with-wxwidgets \
--with-odbc \
--with-python
</source>

Then compile with "make [-j2]".

===== Arch Linux =====

The easiest/fastest way is to build GRASS GIS using AUR:

https://aur.archlinux.org/packages/?O=0&K=grass (msieczka: I would recommend my set of PKBUILDs - https://aur.archlinux.org/packages/?K=czk&SeB=m. They are most feature-rich, up-to-date and allow to have various versions installed alongside each other without conflicts.)

But if you want to compile it yourself you have to keep in mind that in Arch Linux the default Python version 3.

python --version

Python 3.3.1

Therefore, to build GRASS GIS (version 7 in this example), we can use either use virtualenv or symlink python2 and python2-config in a given directory.
Let's use the latter approach

sudo ln -s /usr/bin/python2 $HOME/usr/bin/python
sudo ln -s /usr/bin/python2-config $HOME/usr/bin/python-config

We can now start building GRASS 7

export PATH=~/usr/bin:$PATH && \
./configure \
--enable-debug \
--enable-64bit \
--with-libs=/usr/lib64 \
--with-cxx \
--with-readline \
--with-zlib-includes="/usr/include/" \
--with-freetype=yes \
--with-freetype-includes="/usr/include/freetype2/" \
--enable-largefile=yes \
--with-gdal=/usr/bin/gdal-config \
--with-proj-share=/usr/share/proj/ \
--with-geos=/usr/bin/geos-config \
--with-cairo \
--with-odbc \
--with-pthread \
--with-liblas=/usr/local/bin/liblas-config \
--with-fftw-includes="/usr/include/" \
--with-fftw-libs=/usr/lib/ \
--with-tcltk-includes="/usr/include/" \
--with-wxwidgets \
--with-postgres=yes \
--with-postgres-includes="/usr/include/postgresql/internal" \
--with-postgres-libs="/usr/include/postgresql/internal/libpq" \
--with-sqlite=yes \
--with-python=yes \
--with-liblas \
--with-netcdf

Now let's compile and install it

make
sudo make install

Now the final step required to use GRASS with python2 is to create a bash script "grass" (or the name you want)

#!/bin/bash
export PATH=~/usr/bin:$PATH
python2 /usr/local/bin/grass70

And make it executable and copy it somewhere in the PATH

chmod+x grass
sudo cp grass /usr/local/bin

It's done, you can enjoy GRASS 7 in Arch, just type "grass" (or the name you gave to the script) in the terminal to launch it.

===== RPM SPEC files =====
* ... can be found in the source code, rpm/ directory,
* or [https://build.opensuse.org/package/show?package=grass&project=Application%3AGeo OpenSuSe]
* or [https://admin.fedoraproject.org/pkgdb/acls/name/grass Fedora]
* or [http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/grass/ Mandriva] (there are also [http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/proj/ proj4], [http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/geos/ geos], [http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/gdal/ gdal], [http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/gdal-grass/ gdal-grass-plugin], [http://svn.mandriva.com/cgi-bin/viewvc.cgi/packages/cooker/qgis/ qgis] etc)
* ...

===== Zaurus =====

... see [http://wiki.debian.org/?GrassGISonZaurus here] for instructions

==== Mac OSX ====

===== Conventional installation =====
* see the source/macosx readme ([http://trac.osgeo.org/grass/browser/grass/trunk/macosx/ReadMe.rtf Trac link])
* main page [[Compiling on MacOSX]]
* main page [[Packaging on MacOSX]]
* [http://grass.osgeo.org/download/software/mac-osx/ official download site]
* [http://www.kyngchaos.com/software/frameworks#build_scripts build scripts for dependencies]
* solving errors at starting GRASS GIS: [[MacOSX GRASS errors]]
=====Compiling on MacOSX using homebrew =====

* See [[Compiling on MacOSX using homebrew]]

==== FreeBSD / NetBSD ====

The recommended compiler tools are GCC, GNU make, GNU coreutils (for install), and flex. All are available through the respective package managing tools (pkg_add for FreeBSD and pkgin install for NetBSD) and for recent *BSD versions most likely installed by default.

GRASS 6.x and GRASS 7 should compile on FreeBSD 8.0 or later and NetBSD 5.0 or later (maybe also on updated NetBSD 4.x).

It is highly recommended to install GDAL/OGR and PROJ4 first. These libraries and tools are available as [http://www.freebsd.org/ports/ ports for FreeBSD] and [http://www.pkgsrc.org/ packages for NetBSD].

Optional functionality is listed with ''./configure --help'', and related libraries and tools might need to be installed first.

You then may need to edit include/Make/Platform.make and set
ICONVLIB = -liconv

==== Solaris ====

* ''2008 Oct 15'': see [http://lists.osgeo.org/pipermail/grass-user/2008-October/047093.html this post on the grass mailing list]

===== 11 SPARC/i86pc =====

The recommended compiler tools are GCC, GNU make, GNU coreutils (for install), and flex. All are available through the Solaris package manager.

Most dependencies are available through the Solaris package manager. GDAL and proj4 can either be compiled from source or installed e.g. from [http://www.opencsw.org/ OpenCSW]. If packages are installed from OpenCSW, the linker flags need to be set with

LDFLAGS="-Wl,-R/opt/csw/lib -L/opt/csw/lib -Wl,-R/opt/csw/gxx/lib -L/opt/csw/gxx/lib"

See also the [http://www.opencsw.org/use-it/ OpenCSW documentation].

===== 10 SPARC/i86pc =====

* get gcc compiler and tools. There are several sources: Solaris Companion CD (SFW pkg, installs in /opt/sfw/), Blastwave ([http://www.blastwave.org], CSW pkg, installs in /opt/csw/) or Sunfreeware ([http://www.sunfreeware.com], SMC pkg, installs in /usr/local/).
Needed Packages from Sunfreeware: SMCbinut, SMCbison, SMCcoreu, SMCfindu, SMCflex, SMCgawk, SMCgcc, SMCgrep, SMCgzip, SMCless, SMClibt, SMClicon, SMCmake, SMCncurs, SMCproj, SMCsed, SMCtar, SMCtcl, SMCtiff, SMCtk, SMCunzip, SMCzlib.

* compile and install fftw-library ([http://www.fftw.org]). You need to re-compile the library with:

./configure --with-pic --enable-shared; make ; make install.

The pre-built packages don't work.

* compile and install gdal library (see documentation of gdal, [http://www.gdal.org]).

* compile and install any additional libraries (e. g. GEOS, [http://geos.refractions.net]).

* set compiler flags and path. e. g.:

# on ultra-sparc machine:
CFLAGS="-O3 -mcpu=v9"
CXXFLAGS="-O3 -mcpu=v9"
PATH="/usr/local/bin:/opt/sfw/bin:/usr/ccs/bin:/usr/bin:/usr/sbin"
export CFLAGS CXXFLAGS PATH

Path has to be changed for the packages (Sunfreeware: /usr/local/bin, Solaris Companion: /opt/sfw/bin, Blastwave: /opt/csw/bin).

* Next configure, e. g.:

./configure --with-postgres-includes=/usr/include/pgsql/ \
--with-postgres-libs=/usr/lib --with-postgres=yes \
--with-includes=/usr/local/include/ncurses

If you use n(ew)curses, you have to include the path /usr/local/include/ncurses.

then:

make
su
make install

If the shared libraries are not found at runtime of the modules, use 'crle' to add the paths of the libraries for the dynamic linker, e. g. as root:

crle -l /lib:/usr/lib:/usr/local/lib:/opt/sfw/lib:/usr/X11/lib

Be careful not to omit a library path, the system may be unusable if you forget the /lib path.

==== AIX ====

A recent [ftp://ftp.gnu.org/gnu/make/ GNU make] (>= 3.81) and GNU coreutils are required. These are available with the [http://www-03.ibm.com/systems/power/software/aix/linux/toolbox/download.html IBM AIX toolbox] or through third-party AIX software repositories, e.g. [http://www.bullfreeware.com/ bullfreeware] and [http://www.perzl.org/aix/ perzl.org]. Note that 'make' does not work, only 'gmake' works.

General instructions to compile on AIX are e.g. [http://www.perzl.org/aix/index.php?n=Main.Instructions here]

On AIX, compilation results by default in 32 bit applications and static libraries. The [http://www.ibm.com/developerworks/aix/library/au-gnu.html IBM documentation] explains how to build 64 bit applications and shared libraries with GCC.

'''GRASS 6: Using the IBM xlc compiler:'''
* ''see [http://thread.gmane.org/gmane.comp.gis.grass.user/32667 this mailing list thread]''

Mike wrote in 2009:

After attempting all the suggestions, I finally used
--disable-shared on the configure command, and all but
a handful of modules successfully compiled. I was able to
individually address the ones that failed through Makefile
edits and several small source code/header file edits.

The environment variables and configure command that worked were:
<pre>
# xlc compiler:
export PATH=/usr/local/bin:/opt/freeware/bin:$PATH
export OBJECT_MODE=64
export LIBICONV=/opt/freeware
export CC="xlc_r -q64"
export CFLAGS="-O -qstrict"
export CXX="xlC_r -q64"
export CXXFLAGS="-O -qstrict"
export AR="ar -X64"
export F77="xlf_r -q64"
export CPPFLAGS="-I/afs/isis/pkg/libpng/include -I/usr/local/include -I$LIBICONV/include -I/usr/lpp/X11/include/X11"
export LDFLAGS="-L/usr/local/lib -L$LIBICONV/lib -L/usr/lib -L/usr/X11R6/lib -lc"

./configure --prefix=/afs/isis/pkg/grass-6.4.0 \
--enable-64bit \
--disable-shared \
--with-includes="/usr/include/fontconfig /usr/include/X11 /usr/include/X11/Xft /usr/include/X11/ext" \
--x-includes=/usr/include/X11 \
--x-libraries=/usr/X11R6/lib \
--with-fftw-includes=/afs/isis/pkg/fftw-3.2.2/include \
--with-fftw-libs=/afs/isis/pkg/fftw-3.2.2/lib \
--with-gdal=/afs/isis/pkg/gdal/bin/gdal-config \
--with-proj-includes=/afs/isis/pkg/proj/include \
--with-proj-libs=/afs/isis/pkg/proj/lib \
--with-proj-share=/afs/isis/pkg/proj/share/proj \
--with-tcltk-includes=/usr/local/include \
--with-tcltk-libs=/usr/local/lib \
--with-opengl-includes=/usr/include/GL
</pre>

'''GRASS 7: Using the IBM xlc compiler:'''

Get and install (in this order):
* [ftp://ftp.gnu.org/gnu/tar/ GNU tar]
* [ftp://ftp.gnu.org/gnu/make/ GNU make] (>= 3.81)
* [http://www.sqlite.org SQLite]
* [http://trac.osgeo.org/proj/ PROJ.4] (for ./configure parameters, see [[Talk:Compile and Install|here]])
* [http://www.gdal.org GDAL] (for ./configure parameters, see [[Talk:Compile and Install|here]])
* [http://grass.osgeo.org/grass70/source/snapshot/ GRASS GIS 7]

The environment variables and configure command that worked:
<pre>
## AIX 5.x

export LIBPATH=/opt/freeware/lib64:/opt/freeware/lib
# make 'ar' happy
export OBJECT_MODE=64

PREFIX=$HOME/private/bin
./configure \
--prefix=$PREFIX \
--disable-shared \
--enable-largefile \
--with-cxx \
--with-proj-includes=$PREFIX/include/ \
--with-proj-libs=$PREFIX/lib/ \
--with-proj-share=$PREFIX/share/proj/ \
--with-sqlite \
--with-sqlite-includes=$PREFIX/include/ \
--with-sqlite-libs=$PREFIX/lib/ \
--with-tiff=no \
--with-png=no \
--with-fftw=no \
--with-cairo=no \
--with-freetype=no
</pre>

'''GRASS 7: Using the GNU gcc compiler:'''

Get and install (in this order):
* if needed, [ftp://ftp.gnu.org/gnu/tar/ GNU tar]
* if needed (the name is often 'gmake' and not 'make') [ftp://ftp.gnu.org/gnu/make/ GNU make] (>= 3.81)
* [http://www.sqlite.org SQLite]
* [http://trac.osgeo.org/proj/ PROJ.4] (for ./configure parameters, see [[Talk:Compile and Install|here]])
* [http://www.gdal.org GDAL] (for ./configure parameters, see [[Talk:Compile and Install|here]])
* [http://grass.osgeo.org/grass70/source/snapshot/ GRASS GIS 7]

The environment variables and configure command that worked for AIX 5.x:
<pre>
## AIX 5.x

export LIBPATH=/opt/freeware/lib64:/opt/freeware/lib
# make 'ar' happy
export OBJECT_MODE=64

PREFIX=$HOME/private/bin
CFLAGS='-ansi -D_ALL_SOURCE=1 -D_POSIX_SOURCE=1 -D_POSIX_C_SOURCE=200809L -Dinline=' ./configure \
--prefix=$PREFIX \
--disable-shared \
--enable-largefile \
--with-cxx \
--with-proj-includes=$PREFIX/include/ \
--with-proj-libs=$PREFIX/lib/ \
--with-proj-share=$PREFIX/share/proj/ \
--with-sqlite \
--with-sqlite-includes=$PREFIX/include/ \
--with-sqlite-libs=$PREFIX/lib/ \
--with-tiff=no \
--with-png=no \
--with-fftw=no \
--with-cairo=no \
--with-freetype=no
</pre>

The environment variables and configure command that worked for AIX 7.x:
<pre>
## AIX 7.x
export CC="gcc"
export CXX="g++"
PREFIX=$HOME/bin

LDFLAGS="-Wl,-bsvr4,-R,/opt/freeware/lib -L/opt/freeware/lib" CPPFLAGS="-I/opt/freeware/include" ./configure \
--prefix=$PREFIX \
--enable-largefile \
--enable-shared \
--with-includes=/opt/freeware/include --with-libs=/opt/freeware/lib \
--with-cxx \
--with-proj-includes=$PREFIX/include/ \
--with-proj-libs=$PREFIX/lib/ \
--with-proj-share=$PREFIX/share/proj/ \
--with-gdal=$PREFIX/bin/gdal-config \
--with-sqlite \
--with-sqlite-libs=$PREFIX/lib --with-sqlite-includes=$PREFIX/include \
--with-png=no \
--with-tiff=no \
--with-fftw=no \
--with-cairo=no \
--with-opengl=no \
--with-freetype=no
</pre>

==== MS-Windows ====

===== MS-Windows/Cygwin =====

* See the [[Cygwin]] wiki pages

===== MS-Windows/native =====

====== Compile ======

* [http://trac.osgeo.org/grass/wiki/CompileOnWindows GRASS Windows Native Binary Building Guide] (GRASS 6.4.x + 7.x)
* <strike>[http://www.webalice.it/marco.pasetti/grass/BuildFromSource.html GRASS Windows Native Binary Building Guide] (GRASS 6.3.x) </strike>
* See/adapt [http://blog.qgis.org/node/124 idea] for unattended install of QGIS (et al) from [http://trac.osgeo.org/osgeo4w/ OSGeo4W] from the QuantumGIS Blog.

See also [[WinGRASS Current Status]] for latest updates.

=== Common problems and solutions ===

During compilation, error can occur if certain packages are not installed. Here a list of problems with solution:

* error: X11/Xlib.h: No such file or directory
** this suggests that you don't have the X headers installed
** Solution: Install the libx11-dev package

* error: g.list: error while loading shared libraries: libgdal1.6.0.so.1: cannot open shared object file: No such file or directory
** this error appears in the shell right after the user clicks GUI's "Start GRASS" button. The GUI shows an error about geographic extent and gets closed afterwards.
** It happens when you launch bin.i686 executable on 64bit system. Be careful and choose the right architecture.

=== Static compilation ===

In order to get static rather than dynamically linked binaries, configure like this:

./configure --disable-shared --enable-static

This will however break the wxGUI and GRASS 7 completely because "ctypes" wants to link against shared libs, or there is something in the static libs that "ctypes" does not like.

=== Optimization ===

GCC and other compilers support [http://gcc.gnu.org/onlinedocs/gcc-4.3.3/gcc/Optimize-Options.html#Optimize-Options optimization]

If you would like to set compiler optimisations, for a possibly faster binary, type (don't enter a ";" anywhere):

CFLAGS=-O ./configure
or,
setenv CFLAGS -O
./configure

whichever works on your shell. Use -O2 instead of -O if your compiler supports this (note: O is the letter, not zero). Using the "gcc" compiler, you can also specify processor specific flags (examples, please suggest better settings to us):

CFLAGS="-mcpu=athlon -O2" # AMD Athlon processor with code optimisations
CFLAGS="-march=amdfam10" # AMD Phenom II X4 64bit processor with gcc >=4.3
CFLAGS="-mcpu=pentium" # Intel Pentium processor
CFLAGS="-mcpu=pentium4" # Intel Pentium4 processor
CFLAGS="-O2 -msse -msse2 -mfpmath=sse -minline-all-stringops" # Intel XEON 64bit processor
CFLAGS="-mtune=nocona -m64 -minline-all-stringops" # Intel Pentium 64bit processor

To find out optional CFLAGS for your platform, enter:
gcc -dumpspecs

See also: http://gcc.gnu.org/

A real fast GRASS version (and small binaries) will be created with LDFLAGS set to "stripping" (but this disables debugging):

CFLAGS="-O2 -mcpu=<cpu_see_above> -Wall" LDFLAGS="-s" ./configure

=== Configure options and their meanings ===

For configure there are many options and some GRASS modules are built only if some options are set. Here are listed common configuration options with short explanation.

* --prefix=/path - Sets path where GRASS will be installed. GRASS will reside in /path/grass-version.
* --enable-largefile - Enables large (>2Gb on 32bit systems) support. For current large file support status look at [[Large File Support]] page.
* --with-cxx - Enables compilation of C++ code. Required for r.terraflow module.
* --with-readline - Enables readline support. If readline is enabled, you can use its history/editing facilities when entering r.mapcalc expressions on stdin.
* --with-glw - Enables GLw support. The GLw library provides OpenGL "canvas" widgets for Athena and Motif.

That switch is unnecessary for normal compilation. It's only
required for r3.showdspf, which isn't normally built; if you
want it, you have build it manually
(e.g. "make -C raster3d/r3.showdspf").
As similar functionality is now provided by NVIZ, r3.showdspf
is deprecated.
r3.showdspf uses the Motif widget (so you also need a
Motif library, e.g. Lesstif or OpenMotif).
[http://grass.itc.it/pipermail/grassuser/2006-December/037475.html Glynn Clements at GRASS-user mailing list]

=== Parallelized compilation on multi-core CPUs ===

You can dramatically accelerate the compilation of the GRASS code with the -j flag of "make" if you have a multi-core CPU system. This determines the maximum number of jobs to have running at once, so cores don't have to sit idle waiting for jobs on other cores to complete. A good rule of thumb for this value is <tt>number_of_cores * 1.5</tt>, but note that setting any higher than the actual number of cores will only affect the timing slightly. For example, on a dual-core processor:
make -j 4



== GRASS-GDAL plugin ==

* see [[Compile and install GRASS and QGIS with GDAL/OGR Plugin]]

== Addons ==

Please note that the installation of [[AddOns/GRASS_6|Addons]] can be easily done with the {{cmd|g.extension}} manager. The compile instructions below are aiming at own development.

=== Compiled modules ===

This is the general way which works for C modules as well as scripts (Python or Shell/Bash modules) which has HTML documentation.

'''Requirements:'''

Either:
* a binary GRASS package, or
* source code which has been prepared with:
./configure [opionally flags]
make libs

Each of the [[GRASS_AddOns|addon]] modules should come with a Makefile. To compile it, just run:
make MODULE_TOPDIR=/path/to/grass64/

Note that the <tt>/path/to/grass64</tt> has to be absolute, not relative.

If using Bash it may be useful to set that up as an alias:
alias gmake64='make MODULE_TOPDIR=/path/to/grass64/'

Installation (perhaps requires "sudo"):
make MODULE_TOPDIR=/path/to/grass64/ install

Note: Compiled addons may require a re-compilation if you changed/updated your GRASS standard binaries.

==== If binary comes with a -dev package ====

''(work in progress, this text states how it eventually will be :)''
Nowadays one does not need to the source code, nor compiling GRASS by oneself to be able to add add-ons. On Debian, you can just install the grass-dev package and then run:
make MODULE_TOPDIR=/usr/lib/grass64/ INST_DIR=/usr/lib/grass64/

The grass-dev package essentially provides GRASS's <tt>include</tt> header files and Make configuration files.

=== Scripts ===

If the addon module is a script, it is sufficient to copy it into the (GRASS binaries) path somewhere. Alternatively, install addons into a separate GRASS addons binaries/scripts directory which is easier to maintain. It avoids getting clobbered every time you reinstall GRASS. To use these separately stored scripts, set and export the GRASS_ADDON_PATH environment variable before starting GRASS and it will automatically be added to the module search path (see the {{cmd|variables}} help page). To simplify this, do for example:

# add in $HOME/.bashrc:
GRASS_ADDON_PATH=/usr/local/grass/addons/
export GRASS_ADDON_PATH

Make sure that the script is executable, then just call it in GRASS typing the filename. Python scripts need to be called writing the extension as well, like:

GRASS 6.5.svn (spearfish60):~ > v.krige.py

[[Category:Development]]
[[Category:Documentation]]
[[Category:Installation]]

Large File Support

2016-09-11T05:22:10Z

⚠️NikosA: Moved to Trac

{{MovedToTrac|LargeFileSupport}}

[[Category: Development]]
[[Category: massive data analysis]]

Image processing

2016-08-31T06:55:14Z

⚠️NikosA: added i.landsat8.swlst

== Introduction ==

For a general overview, see [http://grass.osgeo.org/grass70/manuals/imageryintro.html Introductiojn: image processing] in GRASS GIS

=== General introduction ===

'''Digital numbers and physical values (reflection/radiance-at-sensor):'''

Satellite imagery is commonly stored in Digital Numbers (DN) for minimizing the storage volume, i.e. the originally sampled analog physical value (color, temperature, etc) is stored a discrete representation in 8-16 bits. For example, Landsat data are stored in 8bit values (i.e., ranging from 0 to 255); other satellite data may be stored in 10 or 16 bits. Having data stored in DN, it implies that these data are not yet the observed ground reality. Such data are called "at-satellite", for example the amount of energy sensed by the sensor of the satellite platform is encoded in 8 or more bits. This energy is called radiance-at-sensor. To obtain physical values from DNs, satellite image providers use a linear transform equation <tt>(y = a * x + b)</tt> to encode the radiance-at-sensor in 8 to 16 bits. DNs can be turned back into physical values by applying the reverse formula <tt>(x = (y - b) / a)</tt>.

The GRASS GIS module {{cmd|i.landsat.toar}} easily transforms Landsat DN to radiance-at-sensor. The equivalent module for ASTER data is {{cmd|i.aster.toar}}. For other satellites, {{cmd|r.mapcalc}} can be employed.

'''Reflection/radiance-at-sensor and surface reflectance'''

When radiance-at-sensor has been obtained, still the atmosphere influences the signal as recorded at the sensor. This atmospheric interaction with the sun energy reflected back into space by ground/vegetation/soil needs to be corrected. There are two ways to apply atmospheric correction for satellite imagery. The simple way for Landsat is with {{cmd|i.landsat.toar}}, using the DOS correction method. The more accurate way is using {{cmd|i.atcorr}} (which works for many satellite sensors). The atmospherically corrected sensor data represent surface {{wikipedia|reflectance}}, which ranges theoretically from 0 % to 100 %. Note that this level of data correction is the proper level of correction to calculate vegetation indices.

=== Image processing in GRASS GIS ===
Satellite imagery and orthophotos (aerial photographs) are handled in GRASS as raster maps and specialized tasks are performed using the '''imagery''' (i.*) modules. All general operations are handled by the raster modules.

* {{cmd|imageryintro}}: A short introduction to image processing in GRASS 6
* Full [http://grass.osgeo.org/gdp/imagery/grass4_image_processing.pdf GRASS 4.0 Image Processing manual] (PDF, 47 pages)
* {{cmd|imagery}}: Imagery module help pages

* Data import is generally handled by the {{cmd|r.in.gdal}} module

== Screenshots ==

* The [http://grass.osgeo.org/screenshots/remote-sensing/ imagery screenshots] page

== Importing ==

The wxGUI offers a convenient tool for single map and bulk import:

* see [[Importing data]]

=== Satellite Data ===

==== Ocean Color ====
* [[MODIS]]
* [[MODIS#SeaWiFS|SeaWiFS]]

==== Sea Surface Temperature (SST) ====

* [[MODIS]]
* [[AVHRR]]

==== High Resolution Data ====

Commercial satellite imagery

* [[IKONOS]]
* [[QuickBird]]
* [[WorldView]]

See also,

* [http://www.digitalglobe.com/sites/default/files/DigitalGlobe_Spectral_Response_1.pdf Spectral Response specifications for IKONOS, GeoEye, QuickBird, WorldView]
* various [http://apollomapping.com/about-us/whitepapers Whitepapers] on High Resolution Satellite Imagery

=== Orthophotos ===

The wxGUI offers a convenient tool for single map and bulk import:

* see [[Importing data]]

== Preprocessing ==

See also [http://cloudsgate2.larc.nasa.gov/cgi-bin/predict/predict.cgi NASA LaRC Satellite Overpass Predictor]

=== Geometric preprocessing/Georectification ===

* Georectification tool is available from the File menu in the GUI.
* {{cmd|i.points}}, {{cmd|i.vpoints}} (scanned maps, satellite images)
* {{cmd|i.ortho.photo}} (aerial images)

A multi-band image may be grouped and georectified with a single set of ground control points ({{cmd|i.group}}, {{cmd|i.target}}, {{cmd|i.rectify}}).

See also the [[Georeferencing]] wiki page

=== Orthorectification ===

* see [[Orthorectification]]
* see [[Orthorectification digital camera|Ortho-rectification of oblique photographs]]

=== Radiometric preprocessing ===

* use {{cmd|r.mapcalc}} to apply gain/bias formula
* [[LANDSAT]]: you can also use {{cmd|i.landsat.toar}} <strike>from [[GRASS AddOns]]</strike> (included since 6.4)

=== Correction for atmospheric effects ===

Visit the dedicated page on [[Atmospheric correction]]

==== Related Modules ====

* {{addonCmd|i.landsat.dehaze}}: simple dark-object/Tasseled Cap based haze minimization (from [[GRASS AddOns]])
* {{cmd|i.atcorr}}: more complex correction but based on atmospheric models

=== Correction for topographic/terrain effects ===

In rugged terrain, such correction might be useful to minimize negative effects.

* simple "cosine correction" using {{cmd|r.sunmask}}, {{cmd|r.mapcalc}} (tends to overshoot when slopes are high)
* In {{cmd|i.topo.corr}} the following correction methods are implemented: cosine, minnaert, percent, c-factor.
** '''Note,''' that for the sun's zenith (in degrees) parameter, the equation "'''Sun's Zenith''' = '''90''' - '''Sun's Elevation'''" is generally valid

Examples:
* [http://sylla-consult.de/en/topographic-correction-of-landsat-imagery-using-grass-gis/ Topographic correction of Landsat imagery using GRASS GIS] (Blog article)

=== Cloud removal ===

* with {{cmd|i.landsat.acca}}

== Image classification ==

See the dedicated [[Image classification]] page.

== Image segmentation ==

* {{cmd|i.smap}}: Performs contextual image classification using sequential maximum a posteriori (SMAP) estimation.
* {{AddonCmd|r.smooth.seg}}: Performs image segmentation and discontinuity detection (based on the Mumford-Shah variational model).
* [[GRASS GSoC 2012 Image Segmentation|i.segment]]: Image Segmentation

== Filtering ==

* {{cmd|r.mfilter}}, {{cmd|r.neighbors}}

== Fourier Transform ==

* {{cmd|i.fft}}, {{cmd|i.ifft}}
* see also [[Image destriping]]
* see also [[Fourier transforms for multitemporal analysis]]

== Canonical Component Analysis ==

* {{cmd|i.cca}}

== Principal Component Analysis ==

* {{cmd|i.pca}}
* see also [[Principal Components Analysis]]

== Texture ==

A series of commonly used texture measures (derived from the Grey Level Co-occurrence Matrix, GLCM), also called Haralick's texture features are available:

* {{cmd|r.texture}}: In case of panchromatic maps or limited amount of channels, it is often recommended to generate synthetic channels through texture analysis

See [http://www.fp.ucalgary.ca/mhallbey/texture_calculations.htm here] and [http://murphylab.web.cmu.edu/publications/boland/boland_node26.html here] for the formulas to calculate texture. See also [http://wiki.landscapetoolbox.org/doku.php/remote_sensing_methods:canopy_texture_mapping canopy texture mapping].

== Spectral unmixing ==

* {{AddonCmd|i.spec.unmix}} is used to perform Spectral Unmixing ([https://svn.osgeo.org/grass/grass-addons/grass6/imagery/i.spec.unmix/description.html manual])
* {{AddonSrc|imagery|i.spec.unmix|version=7}}: version for GRASS GIS 7 (manual: {{addonCmd|i.spec.unmix}})

== Thermal remote sensing ==

* {{cmd|r.mapcalc}} can be used to convert from DN (digital number) of arbitrary sensors to Kelvin/Celsius/...
* {{cmd|i.landsat.toar}} - Calculates top-of-atmosphere radiance or reflectance and temperature for Landsat MSS/TM/ETM+/OLI
* {{AddonSrc|imagery|i.landsat8.swlst|version=7}} Practical split-window algorithm estimating Land Surface Temperature from Landsat 8 OLI/TIRS imagery
* {{cmd|i.aster.toar|version=70}} - Calculates top-of-atmosphere radiance or reflectance and temperature for ASTER
* [[MODIS#Advanced_MODIS_LST_time_series_reconstruction|MODIS]]

== Time series analysis ==

* {{cmd|r.series}}- Makes each output cell value a function of the values assigned to the corresponding cells in the input raster map layers.
* see also [[Time series]]
* see also [[Time series development]]
* see also [[MODIS#Advanced_MODIS_LST_time_series_reconstruction|MODIS]]

== Enhancements ==

=== Radiometric Enhancements ===
* {{cmd|i.landsat.rgb}} (GRASS 6.x) | {{cmd|i.colors.enhance|version=70}} (GRASS 7.x)
* Decorrelation stretching with {{cmd|r.colors}} or {{cmd|r.mapcalc}}
* Density slicing with {{cmd|r.colors}}
* [[Principal Component Analysis]] with {{cmd|i.pca}}

=== Geometric Enhancements - Image Fusion - Pansharpening - Image Segmentation ===

Image fusion and Pansharpening:
* {{cmd|i.rgb.his}} and {{cmd|i.his.rgb}}: can be used for image fusion
* {{cmd|i.fusion.brovey}}: image fusion of pan-chromatic and color channels
* {{cmd|i.pansharpen|version=70}}: Image fusion algorithms to sharpen multispectral with high-res panchromatic channels (GRASS 7)

Segmentation:
* {{AddonCmd|r.smooth.seg}} which performs image segmentation and discontinuity detection (based on the Mumford-Shah variational model). The module generates a piece-wise smooth approximation of the input raster map and a raster map of the discontinuities of the output approximation. The discontinuities of the output approximation are preserved from being smoothed. (Addons)
* {{cmd|i.segment|version=70}}: Identifies segments (objects) from imagery data (GRASS 7)

=== Optimal channel selection for color composites ===
* {{cmd|i.oif}}

== Vegetation indices ==

* {{cmd|r.mapcalc}} can be used to calculate vegetation indices
* In GRASS GIS 7, see also {{cmd|i.vi|version=70}}
* Search this wiki for NDVI (see top right search field)

== Evapotranspiration ==

* In GRASS GIS 7, i.eb.* and i.evapo.* are modules dedicated to evapotranspiration

Please look at [[Image_processing/Evapotranspiration]] for some background information.

== Stereo anaglyphs ==

* see [[Stereo anaglyphs]]

== Ideas collection for improving GRASS' Image processing capabilities ==

Below modules need some tuning before being added to GRASS 6. Volunteers welcome.

=== libCTL - Library for affine, Helmert and projective transformations in 2D ===

To be evaluated: plain C translation
https://svn.code.sf.net/p/gvsigce/code/trunk/libraries/libCTL/

* requires GNU Scienfic Library for the matrix algebra
* It is a small library that provides a handful of transformation methods from source to target (2D) coordinates. Currently, this includes affine, Helmert and projective transformations in 2D.
* The main library is written in plain C and the transformation functions are a plain C conversion of the methods found in the QGIS (www.qgis.org) Georeferencer Plugin (projective and Helmert transformations) and Olivier Dalang's "worldfile transform" (https://gist.github.com/olivierdalang/ba97fc986ade4545068d).
* author: B Ducke

=== Spectral unmixing ideas for processing hyperspectral image data ===

* Make use of the [http://spectralpython.sourceforge.net/ Spectral Python] (SPy) which is a pure Python module for processing hyperspectral image data

=== Spectral angle mapping ideas ===

* {{AddonCmd|i.spec.sam}}

=== Geocoding ideas ===

* {{AddonCmd|i.homography}}: geocoding with lines (instead of points) with homography (as improved i.points; it was formerly called i.linespoints)
* support splines from GDAL (see [[GRASS_AddOns#Imagery_add-ons]])
* New [[WxGUI#Georectifier|Georectifier]]: see also http://gama.fsv.cvut.cz/~landa/grass/swf/georect.html

=== Image matching ideas ===
* {{AddonCmd|i.points.auto}}: automated search of GCPs based on FFT correlation (as improved i.points)
: ''Reference:'' M. Neteler, D. Grasso, I. Michelazzi, L. Miori, S. Merler, and C. Furlanello, 2005: An integrated toolbox for image registration, fusion and classification. International Journal of Geoinformatics, 1(1), pp. 51-61 [http://www.grassbook.org/neteler/papers/neteler2005_IJG_051-061_draft.pdf PDF]

=== Image classification ideas ===

* [http://mpa.itc.it/merler/index.html#ml pr: C code for classification problems]
* GRASS implementation: i.pr.* source code is available [http://trac.osgeo.org/grass/browser/grass-addons/grass6/imagery/i.pr here])

=== Stereo ideas ===

This is stand-alone stereo modeling software (DEM extraction etc). Waits for integration into GRASS.
* http://grass.osgeo.org/outgoing/grass5/stereo-0.2b.tar.gz
* [http://grass.osgeo.org/gdp/stereo-grass/index.html Stereo Tutorial]

=== Bundle block adjustment ===

Needed to orthorectify a series aerial images taken sequentially with overlap. "Historical" method which is nowadays interesting for UAV flights with octocopters and such.

Automatec GPC search could be done by "auto-sift".

Available: Octave code which prepares input to an {{cmd|i.ortho.photo}} batch job (contact Markus Neteler).

=== Lidar LAS format ===

LAS Tools by M. Isenburg, Howard Butler et al.: http://www.liblas.org

las2txt | r.in.xyz in=- fs=" "

''Update:'' {{cmd|r.in.lidar|version=70}} and {{cmd|v.in.lidar|version=70}} implemented by Markus Metz (GRASS 7), but las2txt is still useful for added control such as filtering or importing secondary data streams such as intensity.

(see [[LIDAR]])

=== Improving the existing code ===

It might be sensible to merge the various image libraries:

* GRASS 6 standard libs:
** lib/imagery/: standard lib, in use (i.* except for i.points3, i.rectify3, see below)
** imagery/i.ortho.photo/libes/: standard lib, in use ({{cmd|i.ortho.photo}}, photo.*)
* GRASS 5 (! only) image3 lib:
** [http://trac.osgeo.org/grass/browser/grass/branches/releasebranch_5_5/src/libes/image3 libes/image3/]: never finished improvement which integrated the standard lib and the ortho lib. Seems to provide also ortho rectification for satellite data ([http://trac.osgeo.org/grass/browser/grass/branches/releasebranch_5_5/src/imagery/i.points3 i.points3], [http://trac.osgeo.org/grass/browser/grass/branches/releasebranch_5_5/src/imagery/i.rectify3 i.rectify3])
* GRASS 5/6 image proc commands:
** merge of {{cmd|i.points}}, {{cmd|i.vpoints}}, i.points3 (see above)
** merge of {{cmd|i.rectify}} and i.rectify3 (see above)
** <strike>addition of new resampling algorithms such as bilinear, cubic convolution (take from {{cmd|r.proj}} or {{cmd|r.resamp.aggreg}})</strike> (done 10/2010)
** add other warping methods (maybe lanczos or thin splines from GDAL?): [[Addons#i.warp]]
** implement/finish linewise ortho-rectification of satellite data

== Bibliography ==

* Search for [http://scholar.google.com/scholar?sourceid=mozclient&ie=utf-8&oe=utf-8&q=grass+gis+image+processing "GRASS GIS Image processing"] - Google Scholar
* Search for [http://scholar.google.com/scholar?sourceid=mozclient&ie=utf-8&oe=utf-8&q=grass+gis+remote+sensing "GRASS GIS Remote Sensing"] - Google Scholar

[[Category:Development]]
[[Category:Documentation]]
[[Category:Image processing]]

GRASS Community Sprint Bonn 2016

2016-08-27T21:50:05Z

⚠️NikosA: Updated activity

{{toc|right}}
During the [http://2016.foss4g.org/ FOSS4G 2016] , 21-28 August 2016, [http://en.wikipedia.org/wiki/Bonn Bonn], [https://en.wikipedia.org/wiki/Germany Germany], the GRASS GIS project will join the code sprint

'''Important dates:'''

* CodeSprint I: Sun 21.08.2016 12:00 - Tue 23.08.2016 (open end) (see also [http://2016.foss4g.org/programme.html#code-sprints here])
* [http://2016.foss4g.org/#page9 FOSS4G 2016 Schedule]
* CodeSprint II: Sat 27.08.2016 - Sun 28.08.2016 12:00 p.m. (see also [http://2016.foss4g.org/programme.html#code-sprints here])

== Purpose ==

Get together of GRASS GIS project members to take decisions and tackle small and larger problems.

For this meeting, we welcome people committed to improving the GRASS GIS project. This includes developers, documenters, bug reporters, translators, and users!

== Sponsors ==

If you are interested to sponsor the GRASS Community Sprint, please read about 
    '''sponsoring the GRASS project at [http://grass.osgeo.org/donations/ http://grass.osgeo.org/donations/]''' 
and, if needed, contact [[User:Neteler|Markus Neteler]] <tt><neteler at osgeo.org></tt>. Any surplus at the end of the event will be turned over to the GRASS GIS project.

The GRASS Community Sprint is a great occasion for you to support the development of GRASS. With your contribution you'll enable more developers to meet in Bonn. The community sprint is an important opportunity for the GRASS developers to discuss and collaboratively resolve bugs, plan the direction for the project and work on new features. Please see below for the more detailed agenda. The developers and contributors are donating their valuable time, so it would be great if in-kind funding can be made available from within the community to cover out-of-pocket expenses. All of the work that takes place at the community sprint will be directly contributed back into the GRASS project to the benefit of everyone who uses it.

=== Thanks to our sponsors ===

== Venue ==

Bonn, Germany: Basecamp

How to get to FOSS4G2016 OSGeo Code Sprint?
* http://alienus.de/basecamp1/en/arrival-bonn-hostel/
* http://www.openstreetmap.org/search?query=raste%201%20bonn#map=18/50.70762/7.12383

By train
* From Bonn Main station take the U-Trains (Underground platform) 16, 63 or 68. Get off after 6 min at stop Dt. Telekom / Ollenhauerstr.
* http://alienus.de/basecamp1/en/arrival-bonn-hostel/
* http://www.openstreetmap.org/search?query=raste%201%20bonn#map=18/50.70762/7.12383
* [https://www.vrsinfo.de/englisch/the-vrs/vrs-about-us.html online public transport planner]

By Car
* Coming from the highway A59 Basecamp is located at the end of the highway A562. For navigation systems it is best to type Basecamp's address: "''In der Raste 3, 53129 Bonn''")

== Accommodation and Costs ==

* to be defined

== Weather ==

* https://en.wikipedia.org/wiki/Bonn#Climate

== Agenda - What we plan to do ==

'''Note:''' The program is generally open for your ideas. Please edit this wiki page!

=== More core developers with SVN write access ===
* Develop a strategy to refresh the group with core SVN write access

=== Web site ===
* Migrate the good old CMSMS to Wordpress
* Move development related stuff from CMS (https://grass.osgeo.org/development/) to trac Wiki

=== Promo ===
* Finalize the new GRASS GIS flyer prepared by Vincent Bain
** needed: catchy time series screenshot

=== New Releases ===
Minor releases:
* Release GRASS GIS 6.4.6:
** Coordination is done here: http://trac.osgeo.org/grass/wiki/Grass6Planning
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/6.4.6-News
* Release GRASS GIS 7.0.5:
** Coordination is done here: https://trac.osgeo.org/grass/wiki/Grass7Planning#a7.0.5tobebackported
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/7.0.5-News

'''Major release''':
* Release GRASS GIS 7.2.0:
** Coordination is done here: https://trac.osgeo.org/grass/wiki/Grass7Planning#a7.2.0
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/7.2.0-News

=== Documentation and translation ===
* Add examples to manual pages
* Add examples to Wiki pages
** GRASS-R/R-GRASS handling of raster time series
** Sentinel-2 processing
* Translation of messages
* Translation of Wiki pages

=== Source code Quality Assessment ===
* integrate "coverity scan" with Travis, https://scan.coverity.com/travis_ci
* check "coverity scan": copy-paste errors, https://scan.coverity.com/projects/1038

=== Bug squashing ===
* Open bugs concerning GRASS 7: https://trac.osgeo.org/grass/report/19
* {{trac|2864}}: Add link to source code in the documentation pages (almost done, let's finish it!)
* {{trac|3042}}: Patches to make the build reproducible (fileordering, randomness)
* fix i.histo.match: https://lists.osgeo.org/pipermail/grass-dev/2013-February/062179.html
* Numpy and matplotlib issues when installing/updating from OSGeo4W (ticket [https://trac.osgeo.org/grass/ticket/3057 #3057])

=== Testing ===

Please test in GRASS 7.2.svn!
* {{cmd|d.legend.vect|version=72}}

=== Improvements ===
* improve {{AddonCmd|r.modis}} addons
* integrate Google Summer of Code projects (where adequate)
* continue working on OBIA toolchain:
** Testing of mean shift in i.segment
** Finalization of {{AddonCmd|r.object.geometry}} and its integration into {{AddonCmd|i.segment.uspo}}
* add R-batch files to OSGeo4W-winGRASS7 see [https://trac.osgeo.org/osgeo4w/ticket/413 osgeo4w/ticket 413]
* GUI: Implement an optional session "kickstart" in GRASS with LatLong predefined, then open import wizard | switch projection inside (like most other GIS do!)
* TGRASS:
** suffix option for output mapnames in t.rast.mapcalc and t.rast.algebra (tickets [https://trac.osgeo.org/grass/ticket/2294 #2294] and [https://trac.osgeo.org/grass/ticket/2736 #2736])
** pattern option to list and remove space-time data sets (ticket [https://trac.osgeo.org/grass/ticket/2804 #2804])
** t.rast.what: input file with coordinates (ticket [https://trac.osgeo.org/grass/ticket/2721 #2721])
** t.remove: dual step removal (ticket [https://trac.osgeo.org/grass/ticket/2752 #2752])
** support for space-time datasets in g.gui.gmodeler (ticket [https://trac.osgeo.org/grass/ticket/2977 #2977])

=== Better integration of git into development ===
* since github/gitlab/bitbucket etc are a valuable and recognized source of new source code, we should be able to receive such contributions easily
** OSGeo currently deploys a self-hosted "gitlab" repo: https://lists.osgeo.org/pipermail/sac/2016-August/007345.html

... YOUR IDEA HERE ...

== Planned participation ==

For organizational reasons, please '''additionally register''' at: https://wiki.osgeo.org/wiki/FOSS4G_2016_Code_Sprint

=== In person ===

Please add your name here or contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail com></tt>:

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|width=100px|'''Arrival'''
|width=100px|'''Departure'''
|'''Topic'''
|width=75px|'''T-Shirt'''
|'''Notes'''
|-
|1
|[[User:Lucadelu|Luca Delucchi]]
|Italy
| August 21
| August 28
|
| L
|
|-
|2
|[[User:Landa|Martin Landa]]
|Czech Republic
| August 21
| August 28
|
| L
|
|-
|3
|[[User:Neteler|Markus Neteler]]
|Italy
| August 21
| August 28
| Releases, project coordination, bug hunting, getting more ppl involved
| M
|
|-
|4
|[[User:NikosA|Nikos Alexandris]]
|Switzerland
| August 21
| August 28
| CRO, create_iwave.py, i.<nowiki>[rgb|his].[his|rgb]</nowiki>
| S or M
|
|-
|5
|[[User:Veroandreo|Veronica Andreo]]
|Argentina
| August 26
| August 28
|
| S
|
|-
|6
|[[User:mlennert|Moritz Lennert]]
|Belgium
| August 21
| August 27
| Releases, OBIA toolchain, GSoC module integration
| M
|
|-
|7
|[[User:sbl|Stefan Blumentrath]]
|Norway
| August 26
| August 28
| Port some Addons to python? (or whatever you put me up to)
| L
|
|-
|8
|[[User:MilenaN|Milena Nowotarska]]
|Poland
| August 21
| August 28
|
| M
|
|-
|9
|[[User:annazanchetta|Anna Zanchetta]]
|Italy - Israel
| August 22
| August 27
| Implementing and improving some imagery modules, inshallah
| S
|
|-
|}

=== Via IRC chat ===

(IRC is a nice archaic chat system, see [[IRC]])

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|-
|1
|[[User:Ychemin|Yann Chemin]]
|France
|-
|2
|[[User:hellik|Helmut Kudrnovsky]]
|Austria
|-
|}

== Individual Preparation ==

* Bring your own computer
* Bring [http://en.wikipedia.org/wiki/Mains_electricity_by_country your power connector adapter] if needed (photo: [https://en.wikipedia.org/wiki/File:Schuko_plug_and_socket.png Germany]). See also http://www.power-plugs-sockets.com/
* Install subversion and the compiler tools, and come with a working GRASS development environment if possible.

== Photos ==

We have to remember to take at least one group photo! :-)

== FAQ ==

* ''Is the GRASS Community Sprint just a coding event?''
** It is mainly a coding and documentation event. It is a working session for people who are already participants in the GRASS project and/or are committed to improving the GRASS project.
** On demand we can do some presentations of current working GRASS implementation and new upcoming features to spread the idea of Open Source GIS software
* ''Is the GRASS Community Sprint for developers only?''
** No: anybody can help, with testing, checking out bugs and fixes, documentation and more.
* ''Where can I get help and more information about the community sprint?''
** Contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail com></tt>

== Participants and Reports ==

===Moritz===
* OBIA Toolchain
** Cleaned up r.object.geometry and moved it to addons {{rev|69187}} + {{rev|69188}}, {{AddonCmd|r.object.geometry}}
** Modified {{AddonCmd|i.segment.stats}} to user.object.geometry instead of v.to.db
** Improved error handling in i.segment.stats and use temporary files on disk to reduce memory use
** Modified {{AddonCmd|i.segment.uspo}} to also allow testing new mean shift algorithm (currently only available in sandbox) {{rev|69236}}
** Tested mean shift algorithm
* Working on 6.4.6 release
** General bug triaging
** Backported fix of {{trac|1444}} to release_64
** Backported fix of {{trac|2558}} to release_64
* Scratched an old itch and implement a flag in {{Cmd|g.region}} to allow shell style output in flat (one line) style {{rev|69240}}

*[https://trac.osgeo.org/grass/timeline?from=Aug+23%2C+2016&daysback=2&authors=mlennert&milestone=on&ticket=on&changeset=on&wiki=on&sfp_email=&sfph_mail=&update=Update Log overview]

=== Markus ===
* assist CRO in cleaning PSC election voter email addresses
* minor pseudo-mercator discussions
* major datum shift file discussions with Even Rouault (GDAL) and fellow GRASS sprinters: libproj cleanup of local copies of NAD and datum shift files which are indeed used from GDAL; extended documentation in README.txt ({{trac|2456}}, {{rev|2456}})

=== Anna Petrasova ===
* {{rev|69199}}: major backport of Adam's GSoC work to 7.2. (please test!)
* fix {{trac|3133}}, {{trac|2252}}

=== Martin Landa ===

* Fix {{trac|2480}}, {{trac|2765}}
* Bug reports for 7.0.5 milestone reviewed, outdated tickets closed
* [https://trac.osgeo.org/grass/timeline?from=Aug+28%2C+2016&daysback=7&authors=martinl&milestone=on&ticket=on&changeset=on&wiki=on&sfp_email=&sfph_mail=&update=Update Log overview]

=== Vero ===

* Tutorial: R-GRASS integration for raster time series handling/processing. Work in progress.

=== Nikos ===

===== Done =====

* CRO related tasks
* Patched [https://trac.osgeo.org/grass/changeset/69012/grass/trunk/imagery/i.atcorr/create_iwave.py create_iwave.py], see {{trac|3027}} -- mentored by Moritz.
* Rewrote {{Cmd|i.rgb.his}} to allow for 16-bit input data -- need to test more though.
* Corrected erroneous function in {{AddonCmd|i.landsat8.swlst}} -- thanks to Stefan Blumentrath!

===== On-Going =====

* Rewrite: {{Cmd|i.his.rgb}} -- see {{trac|774}}!

===== Ideas =====

* Scratching a new version for `create_iwave.py` -- vision: one Class to rule them all.
* Reviewing options to fuse {{Cmd|i.pansharpen}} with/into {{AddonCmd|i.fusion.hpf}}

[[Category: Workshops]]
[[Category: Code Sprint]]
[[Category: 2016]]

GRASS Community Sprint Bonn 2016

2016-08-25T09:31:27Z

⚠️NikosA: Updated activities

{{toc|right}}
During the [http://2016.foss4g.org/ FOSS4G 2016] , 21-28 August 2016, [http://en.wikipedia.org/wiki/Bonn Bonn], [https://en.wikipedia.org/wiki/Germany Germany], the GRASS GIS project will join the code sprint

'''Important dates:'''

* CodeSprint I: Sun 21.08.2016 12:00 - Tue 23.08.2016 (open end) (see also [http://2016.foss4g.org/programme.html#code-sprints here])
* [http://2016.foss4g.org/#page9 FOSS4G 2016 Schedule]
* CodeSprint II: Sat 27.08.2016 - Sun 28.08.2016 12:00 p.m. (see also [http://2016.foss4g.org/programme.html#code-sprints here])

== Purpose ==

Get together of GRASS GIS project members to take decisions and tackle small and larger problems.

For this meeting, we welcome people committed to improving the GRASS GIS project. This includes developers, documenters, bug reporters, translators, and users!

== Sponsors ==

If you are interested to sponsor the GRASS Community Sprint, please read about 
    '''sponsoring the GRASS project at [http://grass.osgeo.org/donations/ http://grass.osgeo.org/donations/]''' 
and, if needed, contact [[User:Neteler|Markus Neteler]] <tt><neteler at osgeo.org></tt>. Any surplus at the end of the event will be turned over to the GRASS GIS project.

The GRASS Community Sprint is a great occasion for you to support the development of GRASS. With your contribution you'll enable more developers to meet in Bonn. The community sprint is an important opportunity for the GRASS developers to discuss and collaboratively resolve bugs, plan the direction for the project and work on new features. Please see below for the more detailed agenda. The developers and contributors are donating their valuable time, so it would be great if in-kind funding can be made available from within the community to cover out-of-pocket expenses. All of the work that takes place at the community sprint will be directly contributed back into the GRASS project to the benefit of everyone who uses it.

=== Thanks to our sponsors ===

== Venue ==

Bonn, Germany: Basecamp

How to get to FOSS4G2016 OSGeo Code Sprint?
* http://alienus.de/basecamp1/en/arrival-bonn-hostel/
* http://www.openstreetmap.org/search?query=raste%201%20bonn#map=18/50.70762/7.12383

By train
* From Bonn Main station take the U-Trains (Underground platform) 16, 63 or 68. Get off after 6 min at stop Dt. Telekom / Ollenhauerstr.
* http://alienus.de/basecamp1/en/arrival-bonn-hostel/
* http://www.openstreetmap.org/search?query=raste%201%20bonn#map=18/50.70762/7.12383
* [https://www.vrsinfo.de/englisch/the-vrs/vrs-about-us.html online public transport planner]

By Car
* Coming from the highway A59 Basecamp is located at the end of the highway A562. For navigation systems it is best to type Basecamp's address: "''In der Raste 3, 53129 Bonn''")

== Accommodation and Costs ==

* to be defined

== Weather ==

* https://en.wikipedia.org/wiki/Bonn#Climate

== Agenda - What we plan to do ==

'''Note:''' The program is generally open for your ideas. Please edit this wiki page!

=== More core developers with SVN write access ===
* Develop a strategy to refresh the group with core SVN write access

=== Web site ===
* Migrate the good old CMSMS to Wordpress
* Move development related stuff from CMS (https://grass.osgeo.org/development/) to trac Wiki

=== Promo ===
* Finalize the new GRASS GIS flyer prepared by Vincent Bain
** needed: catchy time series screenshot

=== New Releases ===
Minor releases:
* Release GRASS GIS 6.4.6:
** Coordination is done here: http://trac.osgeo.org/grass/wiki/Grass6Planning
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/6.4.6-News
* Release GRASS GIS 7.0.5:
** Coordination is done here: https://trac.osgeo.org/grass/wiki/Grass7Planning#a7.0.5tobebackported
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/7.0.5-News

'''Major release''':
* Release GRASS GIS 7.2.0:
** Coordination is done here: https://trac.osgeo.org/grass/wiki/Grass7Planning#a7.2.0
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/7.2.0-News

=== Documentation and translation ===
* Add examples to manual pages
* Add examples to Wiki pages
** GRASS-R/R-GRASS handling of raster time series
** Sentinel-2 processing
* Translation of messages
* Translation of Wiki pages

=== Source code Quality Assessment ===
* integrate "coverity scan" with Travis, https://scan.coverity.com/travis_ci
* check "coverity scan": copy-paste errors, https://scan.coverity.com/projects/1038

=== Bug squashing ===
* Open bugs concerning GRASS 7: https://trac.osgeo.org/grass/report/19
* {{trac|2864}}: Add link to source code in the documentation pages (almost done, let's finish it!)
* {{trac|3042}}: Patches to make the build reproducible (fileordering, randomness)
* fix i.histo.match: https://lists.osgeo.org/pipermail/grass-dev/2013-February/062179.html
* Numpy and matplotlib issues when installing/updating from OSGeo4W (ticket [https://trac.osgeo.org/grass/ticket/3057 #3057])

=== Testing ===

Please test in GRASS 7.2.svn!
* {{cmd|d.legend.vect|version=72}}

=== Improvements ===
* improve {{AddonCmd|r.modis}} addons
* integrate Google Summer of Code projects (where adequate)
* continue working on OBIA toolchain:
** Testing of mean shift in i.segment
** Finalization of {{AddonCmd|r.object.geometry}} and its integration into {{AddonCmd|i.segment.uspo}}
* add R-batch files to OSGeo4W-winGRASS7 see [https://trac.osgeo.org/osgeo4w/ticket/413 osgeo4w/ticket 413]
* GUI: Implement an optional session "kickstart" in GRASS with LatLong predefined, then open import wizard | switch projection inside (like most other GIS do!)
* TGRASS:
** suffix option for output mapnames in t.rast.mapcalc and t.rast.algebra (tickets [https://trac.osgeo.org/grass/ticket/2294 #2294] and [https://trac.osgeo.org/grass/ticket/2736 #2736])
** pattern option to list and remove space-time data sets (ticket [https://trac.osgeo.org/grass/ticket/2804 #2804])
** t.rast.what: input file with coordinates (ticket [https://trac.osgeo.org/grass/ticket/2721 #2721])
** t.remove: dual step removal (ticket [https://trac.osgeo.org/grass/ticket/2752 #2752])
** support for space-time datasets in g.gui.gmodeler (ticket [https://trac.osgeo.org/grass/ticket/2977 #2977])

=== Better integration of git into development ===
* since github/gitlab/bitbucket etc are a valuable and recognized source of new source code, we should be able to receive such contributions easily
** OSGeo currently deploys a self-hosted "gitlab" repo: https://lists.osgeo.org/pipermail/sac/2016-August/007345.html

... YOUR IDEA HERE ...

== Planned participation ==

For organizational reasons, please '''additionally register''' at: https://wiki.osgeo.org/wiki/FOSS4G_2016_Code_Sprint

=== In person ===

Please add your name here or contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail com></tt>:

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|width=100px|'''Arrival'''
|width=100px|'''Departure'''
|'''Topic'''
|width=75px|'''T-Shirt'''
|'''Notes'''
|-
|1
|[[User:Lucadelu|Luca Delucchi]]
|Italy
| August 21
| August 28
|
| L
|
|-
|2
|[[User:Landa|Martin Landa]]
|Czech Republic
| August 21
| August 28
|
| L
|
|-
|3
|[[User:Neteler|Markus Neteler]]
|Italy
| August 21
| August 28
| Releases, project coordination, bug hunting, getting more ppl involved
| M
|
|-
|4
|[[User:NikosA|Nikos Alexandris]]
|Switzerland
| August 21
| August 28
| CRO, create_iwave.py, i.<nowiki>[rgb|his].[his|rgb]</nowiki>
| S or M
|
|-
|5
|[[User:Veroandreo|Veronica Andreo]]
|Argentina
| August 26
| August 28
|
| S
|
|-
|6
|[[User:mlennert|Moritz Lennert]]
|Belgium
| August 21
| August 27
| Releases, OBIA toolchain, GSoC module integration
| M
|
|-
|7
|[[User:sbl|Stefan Blumentrath]]
|Norway
| August 26
| August 28
| Port some Addons to python? (or whatever you put me up to)
| L
|
|-
|8
|[[User:MilenaN|Milena Nowotarska]]
|Poland
| August 21
| August 28
|
| M
|
|-
|9
|[[User:annazanchetta|Anna Zanchetta]]
|Italy - Israel
| August 22
| August 27
| Implementing and improving some imagery modules, inshallah
| S
|
|-
|}

=== Via IRC chat ===

(IRC is a nice archaic chat system, see [[IRC]])

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|-
|1
|[[User:Ychemin|Yann Chemin]]
|France
|-
|2
|[[User:hellik|Helmut Kudrnovsky]]
|Austria
|-
|}

== Individual Preparation ==

* Bring your own computer
* Bring [http://en.wikipedia.org/wiki/Mains_electricity_by_country your power connector adapter] if needed (photo: [https://en.wikipedia.org/wiki/File:Schuko_plug_and_socket.png Germany]). See also http://www.power-plugs-sockets.com/
* Install subversion and the compiler tools, and come with a working GRASS development environment if possible.

== Photos ==

We have to remember to take at least one group photo! :-)

== FAQ ==

* ''Is the GRASS Community Sprint just a coding event?''
** It is mainly a coding and documentation event. It is a working session for people who are already participants in the GRASS project and/or are committed to improving the GRASS project.
** On demand we can do some presentations of current working GRASS implementation and new upcoming features to spread the idea of Open Source GIS software
* ''Is the GRASS Community Sprint for developers only?''
** No: anybody can help, with testing, checking out bugs and fixes, documentation and more.
* ''Where can I get help and more information about the community sprint?''
** Contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail com></tt>

== Participants and Reports ==

===Moritz===
* OBIA Toolchain
** Cleaned up r.object.geometry and moved it to addons {{rev|69187}} + {{rev|69188}}, {{AddonCmd|r.object.geometry}}
** Modified {{AddonCmd|i.segment.stats}} to user.object.geometry instead of v.to.db
** Improved error handling in i.segment.stats and use temporary files on disk to reduce memory use
** Modified {{AddonCmd|i.segment.uspo}} to also allow testing new mean shift algorithm (currently only available in sandbox) {{rev|69236}}
** Tested mean shift algorithm
* Working on 6.4.6 release
** General bug triaging
** Backported fix of {{trac|1444}} to release_64
** Backported fix of {{trac|2558}} to release_64
* Scratched an old itch and implement a flag in {{Cmd|g.region}} to allow shell style output in flat (one line) style {{rev|69240}}

*[https://trac.osgeo.org/grass/timeline?from=Aug+23%2C+2016&daysback=2&authors=mlennert&milestone=on&ticket=on&changeset=on&wiki=on&sfp_email=&sfph_mail=&update=Update Log overview]

=== Markus ===
* assist CRO in cleaning PSC election voter email addresses
* minor pseudo-mercator discussions
* major datum shift file discussions with Even Rouault (GDAL) and fellow GRASS sprinters: libproj cleanup of local copies of NAD and datum shift files which are indeed used from GDAL; extended documentation in README.txt ({{trac|2456}}, {{rev|2456}})

=== Anna Petrasova ===
* {{rev|69199}}: major backport of Adam's GSoC work to 7.2. (please test!)
* fix {{trac|3133}}, {{trac|2252}}

=== Martin Landa ===

* Fix {{trac|2480}}, {{trac|2765}}
* Bug reports for 7.0.5 milestone reviewed, outdated tickets closed
* [https://trac.osgeo.org/grass/timeline?from=Aug+28%2C+2016&daysback=7&authors=martinl&milestone=on&ticket=on&changeset=on&wiki=on&sfp_email=&sfph_mail=&update=Update Log overview]

=== Vero ===

* Tutorial: R-GRASS integration for raster time series handling/processing. Work in progress.

=== Nikos ===

* CRO related tasks
* Patched [https://trac.osgeo.org/grass/changeset/69012/grass/trunk/imagery/i.atcorr/create_iwave.py create_iwave.py], see {{trac|3027}} -- mentored by Moritz. | Scratching a new version for it -- vision: one Class to rule them all.
* Working on {{Cmd|i.rgb.his}} and {{Cmd|i.his.rgb}} to allow for 16-bit input data (DCELL?) -- see latest diff(s) in {{trac|774}}!
* Reviewing options to fuse {{Cmd|i.pansharpen}} with/into {{AddonCmd|i.fusion.hpf}}

[[Category: Workshops]]
[[Category: Code Sprint]]
[[Category: 2016]]

GRASS Community Sprint Bonn 2016

2016-08-24T09:49:35Z

⚠️NikosA: trac|774

{{toc|right}}
During the [http://2016.foss4g.org/ FOSS4G 2016] , 21-28 August 2016, [http://en.wikipedia.org/wiki/Bonn Bonn], [https://en.wikipedia.org/wiki/Germany Germany], the GRASS GIS project will join the code sprint

'''Important dates:'''

* CodeSprint I: Sun 21.08.2016 12:00 - Tue 23.08.2016 (open end) (see also [http://2016.foss4g.org/programme.html#code-sprints here])
* [http://2016.foss4g.org/#page9 FOSS4G 2016 Schedule]
* CodeSprint II: Sat 27.08.2016 - Sun 28.08.2016 12:00 p.m. (see also [http://2016.foss4g.org/programme.html#code-sprints here])

== Purpose ==

Get together of GRASS GIS project members to take decisions and tackle small and larger problems.

For this meeting, we welcome people committed to improving the GRASS GIS project. This includes developers, documenters, bug reporters, translators, and users!

== Sponsors ==

If you are interested to sponsor the GRASS Community Sprint, please read about 
    '''sponsoring the GRASS project at [http://grass.osgeo.org/donations/ http://grass.osgeo.org/donations/]''' 
and, if needed, contact [[User:Neteler|Markus Neteler]] <tt><neteler at osgeo.org></tt>. Any surplus at the end of the event will be turned over to the GRASS GIS project.

The GRASS Community Sprint is a great occasion for you to support the development of GRASS. With your contribution you'll enable more developers to meet in Bonn. The community sprint is an important opportunity for the GRASS developers to discuss and collaboratively resolve bugs, plan the direction for the project and work on new features. Please see below for the more detailed agenda. The developers and contributors are donating their valuable time, so it would be great if in-kind funding can be made available from within the community to cover out-of-pocket expenses. All of the work that takes place at the community sprint will be directly contributed back into the GRASS project to the benefit of everyone who uses it.

=== Thanks to our sponsors ===

== Venue ==

Bonn, Germany: Basecamp

How to get to FOSS4G2016 OSGeo Code Sprint?
* http://alienus.de/basecamp1/en/arrival-bonn-hostel/
* http://www.openstreetmap.org/search?query=raste%201%20bonn#map=18/50.70762/7.12383

By train
* From Bonn Main station take the U-Trains (Underground platform) 16, 63 or 68. Get off after 6 min at stop Dt. Telekom / Ollenhauerstr.
* http://alienus.de/basecamp1/en/arrival-bonn-hostel/
* http://www.openstreetmap.org/search?query=raste%201%20bonn#map=18/50.70762/7.12383
* [https://www.vrsinfo.de/englisch/the-vrs/vrs-about-us.html online public transport planner]

By Car
* Coming from the highway A59 Basecamp is located at the end of the highway A562. For navigation systems it is best to type Basecamp's address: "''In der Raste 3, 53129 Bonn''")

== Accommodation and Costs ==

* to be defined

== Weather ==

* https://en.wikipedia.org/wiki/Bonn#Climate

== Agenda - What we plan to do ==

'''Note:''' The program is generally open for your ideas. Please edit this wiki page!

=== More core developers with SVN write access ===
* Develop a strategy to refresh the group with core SVN write access

=== Web site ===
* Migrate the good old CMSMS to Wordpress
* Move development related stuff from CMS (https://grass.osgeo.org/development/) to trac Wiki

=== Promo ===
* Finalize the new GRASS GIS flyer prepared by Vincent Bain
** needed: catchy time series screenshot

=== New Releases ===
Minor releases:
* Release GRASS GIS 6.4.6:
** Coordination is done here: http://trac.osgeo.org/grass/wiki/Grass6Planning
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/6.4.6-News
* Release GRASS GIS 7.0.5:
** Coordination is done here: https://trac.osgeo.org/grass/wiki/Grass7Planning#a7.0.5tobebackported
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/7.0.5-News

'''Major release''':
* Release GRASS GIS 7.2.0:
** Coordination is done here: https://trac.osgeo.org/grass/wiki/Grass7Planning#a7.2.0
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/7.2.0-News

=== Documentation and translation ===
* Add examples to manual pages
* Add examples to Wiki pages
** GRASS-R/R-GRASS handling of raster time series
** Sentinel-2 processing
* Translation of messages
* Translation of Wiki pages

=== Source code Quality Assessment ===
* integrate "coverity scan" with Travis, https://scan.coverity.com/travis_ci
* check "coverity scan": copy-paste errors, https://scan.coverity.com/projects/1038

=== Bug squashing ===
* Open bugs concerning GRASS 7: https://trac.osgeo.org/grass/report/19
* {{trac|2864}}: Add link to source code in the documentation pages (almost done, let's finish it!)
* {{trac|3042}}: Patches to make the build reproducible (fileordering, randomness)
* fix i.histo.match: https://lists.osgeo.org/pipermail/grass-dev/2013-February/062179.html
* Numpy and matplotlib issues when installing/updating from OSGeo4W (ticket [https://trac.osgeo.org/grass/ticket/3057 #3057])

=== Testing ===

Please test in GRASS 7.2.svn!
* {{cmd|d.legend.vect|version=72}}

=== Improvements ===
* improve {{AddonCmd|r.modis}} addons
* integrate Google Summer of Code projects (where adequate)
* continue working on OBIA toolchain:
** Testing of mean shift in i.segment
** Finalization of {{AddonCmd|r.object.geometry}} and its integration into {{AddonCmd|i.segment.uspo}}
* add R-batch files to OSGeo4W-winGRASS7 see [https://trac.osgeo.org/osgeo4w/ticket/413 osgeo4w/ticket 413]
* GUI: Implement an optional session "kickstart" in GRASS with LatLong predefined, then open import wizard | switch projection inside (like most other GIS do!)
* TGRASS:
** suffix option for output mapnames in t.rast.mapcalc and t.rast.algebra (tickets [https://trac.osgeo.org/grass/ticket/2294 #2294] and [https://trac.osgeo.org/grass/ticket/2736 #2736])
** pattern option to list and remove space-time data sets (ticket [https://trac.osgeo.org/grass/ticket/2804 #2804])
** t.rast.what: input file with coordinates (ticket [https://trac.osgeo.org/grass/ticket/2721 #2721])
** t.remove: dual step removal (ticket [https://trac.osgeo.org/grass/ticket/2752 #2752])
** support for space-time datasets in g.gui.gmodeler (ticket [https://trac.osgeo.org/grass/ticket/2977 #2977])

=== Better integration of git into development ===
* since github/gitlab/bitbucket etc are a valuable and recognized source of new source code, we should be able to receive such contributions easily
** OSGeo currently deploys a self-hosted "gitlab" repo: https://lists.osgeo.org/pipermail/sac/2016-August/007345.html

... YOUR IDEA HERE ...

== Planned participation ==

For organizational reasons, please '''additionally register''' at: https://wiki.osgeo.org/wiki/FOSS4G_2016_Code_Sprint

=== In person ===

Please add your name here or contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail com></tt>:

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|width=100px|'''Arrival'''
|width=100px|'''Departure'''
|'''Topic'''
|width=75px|'''T-Shirt'''
|'''Notes'''
|-
|1
|[[User:Lucadelu|Luca Delucchi]]
|Italy
| August 21
| August 28
|
| L
|
|-
|2
|[[User:Landa|Martin Landa]]
|Czech Republic
| August 21
| August 28
|
| L
|
|-
|3
|[[User:Neteler|Markus Neteler]]
|Italy
| August 21
| August 28
| Releases, project coordination, bug hunting, getting more ppl involved
| M
|
|-
|4
|[[User:NikosA|Nikos Alexandris]]
|Switzerland
| August 21
| August 28
| CRO, create_iwave.py, i.<nowiki>[rgb|his].[his|rgb]</nowiki>
| S or M
|
|-
|5
|[[User:Veroandreo|Veronica Andreo]]
|Argentina
| August 26
| August 28
|
| S
|
|-
|6
|[[User:mlennert|Moritz Lennert]]
|Belgium
| August 21
| August 27
| Releases, OBIA toolchain, GSoC module integration
| M
|
|-
|7
|[[User:sbl|Stefan Blumentrath]]
|Norway
| August 26
| August 28
| Port some Addons to python? (or whatever you put me up to)
| L
|
|-
|8
|[[User:MilenaN|Milena Nowotarska]]
|Poland
| August 21
| August 28
|
| M
|
|-
|9
|[[User:annazanchetta|Anna Zanchetta]]
|Italy - Israel
| August 22
| August 27
| Implementing and improving some imagery modules, inshallah
| S
|
|-
|}

=== Via IRC chat ===

(IRC is a nice archaic chat system, see [[IRC]])

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|-
|1
|[[User:Ychemin|Yann Chemin]]
|France
|-
|2
|[[User:hellik|Helmut Kudrnovsky]]
|Austria
|-
|}

== Individual Preparation ==

* Bring your own computer
* Bring [http://en.wikipedia.org/wiki/Mains_electricity_by_country your power connector adapter] if needed (photo: [https://en.wikipedia.org/wiki/File:Schuko_plug_and_socket.png Germany]). See also http://www.power-plugs-sockets.com/
* Install subversion and the compiler tools, and come with a working GRASS development environment if possible.

== Photos ==

We have to remember to take at least one group photo! :-)

== FAQ ==

* ''Is the GRASS Community Sprint just a coding event?''
** It is mainly a coding and documentation event. It is a working session for people who are already participants in the GRASS project and/or are committed to improving the GRASS project.
** On demand we can do some presentations of current working GRASS implementation and new upcoming features to spread the idea of Open Source GIS software
* ''Is the GRASS Community Sprint for developers only?''
** No: anybody can help, with testing, checking out bugs and fixes, documentation and more.
* ''Where can I get help and more information about the community sprint?''
** Contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail com></tt>

== Participants and Reports ==

===Moritz===
* OBIA Toolchain
** Cleaned up r.object.geometry and moved it to addons {{rev|69187}} + {{rev|69188}}, {{AddonCmd|r.object.geometry}}
** Modified {{AddonCmd|i.segment.stats}} to user.object.geometry instead of v.to.db
** Improved error handling in i.segment.stats and use temporary files on disk to reduce memory use
** Modified {{AddonCmd|i.segment.uspo}} to also allow testing new mean shift algorithm (currently only available in sandbox) {{rev|69236}}
** Tested mean shift algorithm
* Working on 6.4.6 release
** General bug triaging
** Backported fix of {{trac|1444}} to release_64
** Backported fix of {{trac|2558}} to release_64
* Scratched an old itch and implement a flag in {{Cmd|g.region}} to allow shell style output in flat (one line) style {{rev|69240}}

*[https://trac.osgeo.org/grass/timeline?from=Aug+23%2C+2016&daysback=2&authors=mlennert&milestone=on&ticket=on&changeset=on&wiki=on&sfp_email=&sfph_mail=&update=Update Log overview]

=== Markus ===
* assist CRO in cleaning PSC election voter email addresses
* minor pseudo-mercator discussions
* major datum shift file discussions with Even Rouault (GDAL) and fellow GRASS sprinters: libproj cleanup of local copies of NAD and datum shift files which are indeed used from GDAL; extended documentation in README.txt ({{trac|2456}}, {{rev|2456}})

=== Anna Petrasova ===
* {{rev|69199}}: major backport of Adam's GSoC work to 7.2. (please test!)
* fix {{trac|3133}}, {{trac|2252}}

=== Martin Landa ===

* Fix {{trac|2480}}, {{trac|2765}}
* Bug reports for 7.0.5 milestone reviewed, outdated tickets closed
* [https://trac.osgeo.org/grass/timeline?from=Aug+28%2C+2016&daysback=7&authors=martinl&milestone=on&ticket=on&changeset=on&wiki=on&sfp_email=&sfph_mail=&update=Update Log overview]

=== Vero ===

* Tutorial: R-GRASS integration for raster time series handling/processing. Work in progress.

=== Nikos ===

* CRO related tasks
* Patched [https://trac.osgeo.org/grass/changeset/69012/grass/trunk/imagery/i.atcorr/create_iwave.py create_iwave.py], see {{trac|3027}} -- mentored by Moritz.
* Working on {{Cmd|i.rgb.his}} and {{Cmd|i.his.rgb}} to allow for 16-bit input data (DCELL?) -- see {{trac|774}}!
* Reviewing options to fuse {{Cmd|i.pansharpen}} with/into {{AddonCmd|i.fusion.hpf}}

[[Category: Workshops]]
[[Category: Code Sprint]]
[[Category: 2016]]

GRASS Community Sprint Bonn 2016

2016-08-24T06:17:53Z

⚠️NikosA: /* In person */

{{toc|right}}
During the [http://2016.foss4g.org/ FOSS4G 2016] , 21-28 August 2016, [http://en.wikipedia.org/wiki/Bonn Bonn], [https://en.wikipedia.org/wiki/Germany Germany], the GRASS GIS project will join the code sprint

'''Important dates:'''

* CodeSprint I: Sun 21.08.2016 12:00 - Tue 23.08.2016 (open end) (see also [http://2016.foss4g.org/programme.html#code-sprints here])
* [http://2016.foss4g.org/#page9 FOSS4G 2016 Schedule]
* CodeSprint II: Sat 27.08.2016 - Sun 28.08.2016 12:00 p.m. (see also [http://2016.foss4g.org/programme.html#code-sprints here])

== Purpose ==

Get together of GRASS GIS project members to take decisions and tackle small and larger problems.

For this meeting, we welcome people committed to improving the GRASS GIS project. This includes developers, documenters, bug reporters, translators, and users!

== Sponsors ==

If you are interested to sponsor the GRASS Community Sprint, please read about 
    '''sponsoring the GRASS project at [http://grass.osgeo.org/donations/ http://grass.osgeo.org/donations/]''' 
and, if needed, contact [[User:Neteler|Markus Neteler]] <tt><neteler at osgeo.org></tt>. Any surplus at the end of the event will be turned over to the GRASS GIS project.

The GRASS Community Sprint is a great occasion for you to support the development of GRASS. With your contribution you'll enable more developers to meet in Bonn. The community sprint is an important opportunity for the GRASS developers to discuss and collaboratively resolve bugs, plan the direction for the project and work on new features. Please see below for the more detailed agenda. The developers and contributors are donating their valuable time, so it would be great if in-kind funding can be made available from within the community to cover out-of-pocket expenses. All of the work that takes place at the community sprint will be directly contributed back into the GRASS project to the benefit of everyone who uses it.

=== Thanks to our sponsors ===

== Venue ==

Bonn, Germany: Basecamp

How to get to FOSS4G2016 OSGeo Code Sprint?
* http://alienus.de/basecamp1/en/arrival-bonn-hostel/
* http://www.openstreetmap.org/search?query=raste%201%20bonn#map=18/50.70762/7.12383

By train
* From Bonn Main station take the U-Trains (Underground platform) 16, 63 or 68. Get off after 6 min at stop Dt. Telekom / Ollenhauerstr.
* http://alienus.de/basecamp1/en/arrival-bonn-hostel/
* http://www.openstreetmap.org/search?query=raste%201%20bonn#map=18/50.70762/7.12383
* [https://www.vrsinfo.de/englisch/the-vrs/vrs-about-us.html online public transport planner]

By Car
* Coming from the highway A59 Basecamp is located at the end of the highway A562. For navigation systems it is best to type Basecamp's address: "''In der Raste 3, 53129 Bonn''")

== Accommodation and Costs ==

* to be defined

== Weather ==

* https://en.wikipedia.org/wiki/Bonn#Climate

== Agenda - What we plan to do ==

'''Note:''' The program is generally open for your ideas. Please edit this wiki page!

=== More core developers with SVN write access ===
* Develop a strategy to refresh the group with core SVN write access

=== Web site ===
* Migrate the good old CMSMS to Wordpress
* Move development related stuff from CMS (https://grass.osgeo.org/development/) to trac Wiki

=== Promo ===
* Finalize the new GRASS GIS flyer prepared by Vincent Bain
** needed: catchy time series screenshot

=== New Releases ===
Minor releases:
* Release GRASS GIS 6.4.6:
** Coordination is done here: http://trac.osgeo.org/grass/wiki/Grass6Planning
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/6.4.6-News
* Release GRASS GIS 7.0.5:
** Coordination is done here: https://trac.osgeo.org/grass/wiki/Grass7Planning#a7.0.5tobebackported
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/7.0.5-News

'''Major release''':
* Release GRASS GIS 7.2.0:
** Coordination is done here: https://trac.osgeo.org/grass/wiki/Grass7Planning#a7.2.0
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/7.2.0-News

=== Documentation and translation ===
* Add examples to manual pages
* Add examples to Wiki pages
** GRASS-R/R-GRASS handling of raster time series
** Sentinel-2 processing
* Translation of messages
* Translation of Wiki pages

=== Source code Quality Assessment ===
* integrate "coverity scan" with Travis, https://scan.coverity.com/travis_ci
* check "coverity scan": copy-paste errors, https://scan.coverity.com/projects/1038

=== Bug squashing ===
* Open bugs concerning GRASS 7: https://trac.osgeo.org/grass/report/19
* {{trac|2864}}: Add link to source code in the documentation pages (almost done, let's finish it!)
* {{trac|3042}}: Patches to make the build reproducible (fileordering, randomness)
* fix i.histo.match: https://lists.osgeo.org/pipermail/grass-dev/2013-February/062179.html
* Numpy and matplotlib issues when installing/updating from OSGeo4W (ticket [https://trac.osgeo.org/grass/ticket/3057 #3057])

=== Testing ===

Please test in GRASS 7.2.svn!
* {{cmd|d.legend.vect|version=72}}

=== Improvements ===
* improve {{AddonCmd|r.modis}} addons
* integrate Google Summer of Code projects (where adequate)
* continue working on OBIA toolchain:
** Testing of mean shift in i.segment
** Finalization of {{AddonCmd|r.object.geometry}} and its integration into {{AddonCmd|i.segment.uspo}}
* add R-batch files to OSGeo4W-winGRASS7 see [https://trac.osgeo.org/osgeo4w/ticket/413 osgeo4w/ticket 413]
* GUI: Implement an optional session "kickstart" in GRASS with LatLong predefined, then open import wizard | switch projection inside (like most other GIS do!)
* TGRASS:
** suffix option for output mapnames in t.rast.mapcalc and t.rast.algebra (tickets [https://trac.osgeo.org/grass/ticket/2294 #2294] and [https://trac.osgeo.org/grass/ticket/2736 #2736])
** pattern option to list and remove space-time data sets (ticket [https://trac.osgeo.org/grass/ticket/2804 #2804])
** t.rast.what: input file with coordinates (ticket [https://trac.osgeo.org/grass/ticket/2721 #2721])
** t.remove: dual step removal (ticket [https://trac.osgeo.org/grass/ticket/2752 #2752])
** support for space-time datasets in g.gui.gmodeler (ticket [https://trac.osgeo.org/grass/ticket/2977 #2977])

=== Better integration of git into development ===
* since github/gitlab/bitbucket etc are a valuable and recognized source of new source code, we should be able to receive such contributions easily
** OSGeo currently deploys a self-hosted "gitlab" repo: https://lists.osgeo.org/pipermail/sac/2016-August/007345.html

... YOUR IDEA HERE ...

== Planned participation ==

For organizational reasons, please '''additionally register''' at: https://wiki.osgeo.org/wiki/FOSS4G_2016_Code_Sprint

=== In person ===

Please add your name here or contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail com></tt>:

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|width=100px|'''Arrival'''
|width=100px|'''Departure'''
|'''Topic'''
|width=75px|'''T-Shirt'''
|'''Notes'''
|-
|1
|[[User:Lucadelu|Luca Delucchi]]
|Italy
| August 21
| August 28
|
| L
|
|-
|2
|[[User:Landa|Martin Landa]]
|Czech Republic
| August 21
| August 28
|
| L
|
|-
|3
|[[User:Neteler|Markus Neteler]]
|Italy
| August 21
| August 28
| Releases, project coordination, bug hunting, getting more ppl involved
| M
|
|-
|4
|[[User:NikosA|Nikos Alexandris]]
|Switzerland
| August 21
| August 28
| CRO, create_iwave.py, i.<nowiki>[rgb|his].[his|rgb]</nowiki>
| S or M
|
|-
|5
|[[User:Veroandreo|Veronica Andreo]]
|Argentina
| August 26
| August 28
|
| S
|
|-
|6
|[[User:mlennert|Moritz Lennert]]
|Belgium
| August 21
| August 27
| Releases, OBIA toolchain, GSoC module integration
| M
|
|-
|7
|[[User:sbl|Stefan Blumentrath]]
|Norway
| August 26
| August 28
| Port some Addons to python? (or whatever you put me up to)
| L
|
|-
|8
|[[User:MilenaN|Milena Nowotarska]]
|Poland
| August 21
| August 28
|
| M
|
|-
|9
|[[User:annazanchetta|Anna Zanchetta]]
|Italy - Israel
| August 22
| August 27
| Implementing and improving some imagery modules, inshallah
| S
|
|-
|}

=== Via IRC chat ===

(IRC is a nice archaic chat system, see [[IRC]])

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|-
|1
|[[User:Ychemin|Yann Chemin]]
|France
|-
|2
|[[User:hellik|Helmut Kudrnovsky]]
|Austria
|-
|}

== Individual Preparation ==

* Bring your own computer
* Bring [http://en.wikipedia.org/wiki/Mains_electricity_by_country your power connector adapter] if needed (photo: [https://en.wikipedia.org/wiki/File:Schuko_plug_and_socket.png Germany]). See also http://www.power-plugs-sockets.com/
* Install subversion and the compiler tools, and come with a working GRASS development environment if possible.

== Photos ==

We have to remember to take at least one group photo! :-)

== FAQ ==

* ''Is the GRASS Community Sprint just a coding event?''
** It is mainly a coding and documentation event. It is a working session for people who are already participants in the GRASS project and/or are committed to improving the GRASS project.
** On demand we can do some presentations of current working GRASS implementation and new upcoming features to spread the idea of Open Source GIS software
* ''Is the GRASS Community Sprint for developers only?''
** No: anybody can help, with testing, checking out bugs and fixes, documentation and more.
* ''Where can I get help and more information about the community sprint?''
** Contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail com></tt>

== Participants and Reports ==

===Moritz===
* OBIA Toolchain
** Cleaned up r.object.geometry and moved it to addons {{rev|69187}} + {{rev|69188}}, {{AddonCmd|r.object.geometry}}
** Modified {{AddonCmd|i.segment.stats}} to user.object.geometry instead of v.to.db
** Improved error handling in i.segment.stats and use temporary files on disk to reduce memory use
** Modified {{AddonCmd|i.segment.uspo}} to also allow testing new mean shift algorithm (currently only available in sandbox) {{rev|69236}}
** Tested mean shift algorithm
* Working on 6.4.6 release
** General bug triaging
** Backported fix of {{trac|1444}} to release_64
** Backported fix of {{trac|2558}} to release_64
* Scratched an old itch and implement a flag in {{Cmd|g.region}} to allow shell style output in flat (one line) style {{rev|69240}}

*[https://trac.osgeo.org/grass/timeline?from=Aug+23%2C+2016&daysback=2&authors=mlennert&milestone=on&ticket=on&changeset=on&wiki=on&sfp_email=&sfph_mail=&update=Update Log overview]

=== Markus ===
* assist CRO in cleaning PSC election voter email addresses
* minor pseudo-mercator discussions
* major datum shift file discussions with Even Rouault (GDAL) and fellow GRASS sprinters: libproj cleanup of local copies of NAD and datum shift files which are indeed used from GDAL; extended documentation in README.txt ({{trac|2456}}, {{rev|2456}})

=== Anna Petrasova ===
* {{rev|69199}}: major backport of Adam's GSoC work to 7.2. (please test!)
* fix {{trac|3133}}, {{trac|2252}}

=== Martin Landa ===

* Fix {{trac|2480}}, {{trac|2765}}
* Bug reports for 7.0.5 milestone reviewed, outdated tickets closed
* [https://trac.osgeo.org/grass/timeline?from=Aug+28%2C+2016&daysback=7&authors=martinl&milestone=on&ticket=on&changeset=on&wiki=on&sfp_email=&sfph_mail=&update=Update Log overview]

=== Vero ===

* Tutorial: R-GRASS integration for raster time series handling/processing. Work in progress.

=== Nikos ===

* CRO related tasks
* Patched [https://trac.osgeo.org/grass/changeset/69012/grass/trunk/imagery/i.atcorr/create_iwave.py create_iwave.py], see {{trac|3027}} -- mentored by Moritz.
* Working on {{Cmd|i.rgb.his}} and {{Cmd|i.his.rgb}} to allow for 16-bit input data (DCELL?)
* Reviewing options to fuse {{Cmd|i.pansharpen}} with/into {{AddonCmd|i.fusion.hpf}}

[[Category: Workshops]]
[[Category: Code Sprint]]
[[Category: 2016]]

GRASS Community Sprint Bonn 2016

2016-08-24T06:12:16Z

⚠️NikosA: /* Planned participation */

{{toc|right}}
During the [http://2016.foss4g.org/ FOSS4G 2016] , 21-28 August 2016, [http://en.wikipedia.org/wiki/Bonn Bonn], [https://en.wikipedia.org/wiki/Germany Germany], the GRASS GIS project will join the code sprint

'''Important dates:'''

* CodeSprint I: Sun 21.08.2016 12:00 - Tue 23.08.2016 (open end) (see also [http://2016.foss4g.org/programme.html#code-sprints here])
* [http://2016.foss4g.org/#page9 FOSS4G 2016 Schedule]
* CodeSprint II: Sat 27.08.2016 - Sun 28.08.2016 12:00 p.m. (see also [http://2016.foss4g.org/programme.html#code-sprints here])

== Purpose ==

Get together of GRASS GIS project members to take decisions and tackle small and larger problems.

For this meeting, we welcome people committed to improving the GRASS GIS project. This includes developers, documenters, bug reporters, translators, and users!

== Sponsors ==

If you are interested to sponsor the GRASS Community Sprint, please read about 
    '''sponsoring the GRASS project at [http://grass.osgeo.org/donations/ http://grass.osgeo.org/donations/]''' 
and, if needed, contact [[User:Neteler|Markus Neteler]] <tt><neteler at osgeo.org></tt>. Any surplus at the end of the event will be turned over to the GRASS GIS project.

The GRASS Community Sprint is a great occasion for you to support the development of GRASS. With your contribution you'll enable more developers to meet in Bonn. The community sprint is an important opportunity for the GRASS developers to discuss and collaboratively resolve bugs, plan the direction for the project and work on new features. Please see below for the more detailed agenda. The developers and contributors are donating their valuable time, so it would be great if in-kind funding can be made available from within the community to cover out-of-pocket expenses. All of the work that takes place at the community sprint will be directly contributed back into the GRASS project to the benefit of everyone who uses it.

=== Thanks to our sponsors ===

== Venue ==

Bonn, Germany: Basecamp

How to get to FOSS4G2016 OSGeo Code Sprint?
* http://alienus.de/basecamp1/en/arrival-bonn-hostel/
* http://www.openstreetmap.org/search?query=raste%201%20bonn#map=18/50.70762/7.12383

By train
* From Bonn Main station take the U-Trains (Underground platform) 16, 63 or 68. Get off after 6 min at stop Dt. Telekom / Ollenhauerstr.
* http://alienus.de/basecamp1/en/arrival-bonn-hostel/
* http://www.openstreetmap.org/search?query=raste%201%20bonn#map=18/50.70762/7.12383
* [https://www.vrsinfo.de/englisch/the-vrs/vrs-about-us.html online public transport planner]

By Car
* Coming from the highway A59 Basecamp is located at the end of the highway A562. For navigation systems it is best to type Basecamp's address: "''In der Raste 3, 53129 Bonn''")

== Accommodation and Costs ==

* to be defined

== Weather ==

* https://en.wikipedia.org/wiki/Bonn#Climate

== Agenda - What we plan to do ==

'''Note:''' The program is generally open for your ideas. Please edit this wiki page!

=== More core developers with SVN write access ===
* Develop a strategy to refresh the group with core SVN write access

=== Web site ===
* Migrate the good old CMSMS to Wordpress
* Move development related stuff from CMS (https://grass.osgeo.org/development/) to trac Wiki

=== Promo ===
* Finalize the new GRASS GIS flyer prepared by Vincent Bain
** needed: catchy time series screenshot

=== New Releases ===
Minor releases:
* Release GRASS GIS 6.4.6:
** Coordination is done here: http://trac.osgeo.org/grass/wiki/Grass6Planning
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/6.4.6-News
* Release GRASS GIS 7.0.5:
** Coordination is done here: https://trac.osgeo.org/grass/wiki/Grass7Planning#a7.0.5tobebackported
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/7.0.5-News

'''Major release''':
* Release GRASS GIS 7.2.0:
** Coordination is done here: https://trac.osgeo.org/grass/wiki/Grass7Planning#a7.2.0
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/7.2.0-News

=== Documentation and translation ===
* Add examples to manual pages
* Add examples to Wiki pages
** GRASS-R/R-GRASS handling of raster time series
** Sentinel-2 processing
* Translation of messages
* Translation of Wiki pages

=== Source code Quality Assessment ===
* integrate "coverity scan" with Travis, https://scan.coverity.com/travis_ci
* check "coverity scan": copy-paste errors, https://scan.coverity.com/projects/1038

=== Bug squashing ===
* Open bugs concerning GRASS 7: https://trac.osgeo.org/grass/report/19
* {{trac|2864}}: Add link to source code in the documentation pages (almost done, let's finish it!)
* {{trac|3042}}: Patches to make the build reproducible (fileordering, randomness)
* fix i.histo.match: https://lists.osgeo.org/pipermail/grass-dev/2013-February/062179.html
* Numpy and matplotlib issues when installing/updating from OSGeo4W (ticket [https://trac.osgeo.org/grass/ticket/3057 #3057])

=== Testing ===

Please test in GRASS 7.2.svn!
* {{cmd|d.legend.vect|version=72}}

=== Improvements ===
* improve {{AddonCmd|r.modis}} addons
* integrate Google Summer of Code projects (where adequate)
* continue working on OBIA toolchain:
** Testing of mean shift in i.segment
** Finalization of {{AddonCmd|r.object.geometry}} and its integration into {{AddonCmd|i.segment.uspo}}
* add R-batch files to OSGeo4W-winGRASS7 see [https://trac.osgeo.org/osgeo4w/ticket/413 osgeo4w/ticket 413]
* GUI: Implement an optional session "kickstart" in GRASS with LatLong predefined, then open import wizard | switch projection inside (like most other GIS do!)
* TGRASS:
** suffix option for output mapnames in t.rast.mapcalc and t.rast.algebra (tickets [https://trac.osgeo.org/grass/ticket/2294 #2294] and [https://trac.osgeo.org/grass/ticket/2736 #2736])
** pattern option to list and remove space-time data sets (ticket [https://trac.osgeo.org/grass/ticket/2804 #2804])
** t.rast.what: input file with coordinates (ticket [https://trac.osgeo.org/grass/ticket/2721 #2721])
** t.remove: dual step removal (ticket [https://trac.osgeo.org/grass/ticket/2752 #2752])
** support for space-time datasets in g.gui.gmodeler (ticket [https://trac.osgeo.org/grass/ticket/2977 #2977])

=== Better integration of git into development ===
* since github/gitlab/bitbucket etc are a valuable and recognized source of new source code, we should be able to receive such contributions easily
** OSGeo currently deploys a self-hosted "gitlab" repo: https://lists.osgeo.org/pipermail/sac/2016-August/007345.html

... YOUR IDEA HERE ...

== Planned participation ==

For organizational reasons, please '''additionally register''' at: https://wiki.osgeo.org/wiki/FOSS4G_2016_Code_Sprint

=== In person ===

Please add your name here or contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail com></tt>:

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|width=100px|'''Arrival'''
|width=100px|'''Departure'''
|'''Topic'''
|width=75px|'''T-Shirt'''
|'''Notes'''
|-
|1
|[[User:Lucadelu|Luca Delucchi]]
|Italy
| August 21
| August 28
|
| L
|
|-
|2
|[[User:Landa|Martin Landa]]
|Czech Republic
| August 21
| August 28
|
| L
|
|-
|3
|[[User:Neteler|Markus Neteler]]
|Italy
| August 21
| August 28
| Releases, project coordination, bug hunting, getting more ppl involved
| M
|
|-
|4
|[[User:NikosA|Nikos Alexandris]]
|Switzerland
| August 21
| August 28
| CRO, create_iwave.py, i.[rgb|his].[his|rgb]
| S or M
|
|-
|5
|[[User:Veroandreo|Veronica Andreo]]
|Argentina
| August 26
| August 28
|
| S
|
|-
|6
|[[User:mlennert|Moritz Lennert]]
|Belgium
| August 21
| August 27
| Releases, OBIA toolchain, GSoC module integration
| M
|
|-
|7
|[[User:sbl|Stefan Blumentrath]]
|Norway
| August 26
| August 28
| Port some Addons to python? (or whatever you put me up to)
| L
|
|-
|8
|[[User:MilenaN|Milena Nowotarska]]
|Poland
| August 21
| August 28
|
| M
|
|-
|9
|[[User:annazanchetta|Anna Zanchetta]]
|Italy - Israel
| August 22
| August 27
| Implementing and improving some imagery modules, inshallah
| S
|
|-
|}

=== Via IRC chat ===

(IRC is a nice archaic chat system, see [[IRC]])

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|-
|1
|[[User:Ychemin|Yann Chemin]]
|France
|-
|2
|[[User:hellik|Helmut Kudrnovsky]]
|Austria
|-
|}

== Individual Preparation ==

* Bring your own computer
* Bring [http://en.wikipedia.org/wiki/Mains_electricity_by_country your power connector adapter] if needed (photo: [https://en.wikipedia.org/wiki/File:Schuko_plug_and_socket.png Germany]). See also http://www.power-plugs-sockets.com/
* Install subversion and the compiler tools, and come with a working GRASS development environment if possible.

== Photos ==

We have to remember to take at least one group photo! :-)

== FAQ ==

* ''Is the GRASS Community Sprint just a coding event?''
** It is mainly a coding and documentation event. It is a working session for people who are already participants in the GRASS project and/or are committed to improving the GRASS project.
** On demand we can do some presentations of current working GRASS implementation and new upcoming features to spread the idea of Open Source GIS software
* ''Is the GRASS Community Sprint for developers only?''
** No: anybody can help, with testing, checking out bugs and fixes, documentation and more.
* ''Where can I get help and more information about the community sprint?''
** Contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail com></tt>

== Participants and Reports ==

===Moritz===
* OBIA Toolchain
** Cleaned up r.object.geometry and moved it to addons {{rev|69187}} + {{rev|69188}}, {{AddonCmd|r.object.geometry}}
** Modified {{AddonCmd|i.segment.stats}} to user.object.geometry instead of v.to.db
** Improved error handling in i.segment.stats and use temporary files on disk to reduce memory use
** Modified {{AddonCmd|i.segment.uspo}} to also allow testing new mean shift algorithm (currently only available in sandbox) {{rev|69236}}
** Tested mean shift algorithm
* Working on 6.4.6 release
** General bug triaging
** Backported fix of {{trac|1444}} to release_64
** Backported fix of {{trac|2558}} to release_64
* Scratched an old itch and implement a flag in {{Cmd|g.region}} to allow shell style output in flat (one line) style {{rev|69240}}

*[https://trac.osgeo.org/grass/timeline?from=Aug+23%2C+2016&daysback=2&authors=mlennert&milestone=on&ticket=on&changeset=on&wiki=on&sfp_email=&sfph_mail=&update=Update Log overview]

=== Markus ===
* assist CRO in cleaning PSC election voter email addresses
* minor pseudo-mercator discussions
* major datum shift file discussions with Even Rouault (GDAL) and fellow GRASS sprinters: libproj cleanup of local copies of NAD and datum shift files which are indeed used from GDAL; extended documentation in README.txt ({{trac|2456}}, {{rev|2456}})

=== Anna Petrasova ===
* {{rev|69199}}: major backport of Adam's GSoC work to 7.2. (please test!)
* fix {{trac|3133}}, {{trac|2252}}

=== Martin Landa ===

* Fix {{trac|2480}}, {{trac|2765}}
* Bug reports for 7.0.5 milestone reviewed, outdated tickets closed
* [https://trac.osgeo.org/grass/timeline?from=Aug+28%2C+2016&daysback=7&authors=martinl&milestone=on&ticket=on&changeset=on&wiki=on&sfp_email=&sfph_mail=&update=Update Log overview]

=== Vero ===

* Tutorial: R-GRASS integration for raster time series handling/processing. Work in progress.

=== Nikos ===

* CRO related tasks
* Patched [https://trac.osgeo.org/grass/changeset/69012/grass/trunk/imagery/i.atcorr/create_iwave.py create_iwave.py], see {{trac|3027}} -- mentored by Moritz.
* Working on {{Cmd|i.rgb.his}} and {{Cmd|i.his.rgb}} to allow for 16-bit input data (DCELL?)
* Reviewing options to fuse {{Cmd|i.pansharpen}} with/into {{AddonCmd|i.fusion.hpf}}

[[Category: Workshops]]
[[Category: Code Sprint]]
[[Category: 2016]]

GRASS Community Sprint Bonn 2016

2016-08-23T23:30:53Z

⚠️NikosA: Added activity

{{toc|right}}
During the [http://2016.foss4g.org/ FOSS4G 2016] , 21-28 August 2016, [http://en.wikipedia.org/wiki/Bonn Bonn], [https://en.wikipedia.org/wiki/Germany Germany], the GRASS GIS project will join the code sprint

'''Important dates:'''

* CodeSprint I: Sun 21.08.2016 12:00 - Tue 23.08.2016 (open end) (see also [http://2016.foss4g.org/programme.html#code-sprints here])
* [http://2016.foss4g.org/#page9 FOSS4G 2016 Schedule]
* CodeSprint II: Sat 27.08.2016 - Sun 28.08.2016 12:00 p.m. (see also [http://2016.foss4g.org/programme.html#code-sprints here])

== Purpose ==

Get together of GRASS GIS project members to take decisions and tackle small and larger problems.

For this meeting, we welcome people committed to improving the GRASS GIS project. This includes developers, documenters, bug reporters, translators, and users!

== Sponsors ==

If you are interested to sponsor the GRASS Community Sprint, please read about 
    '''sponsoring the GRASS project at [http://grass.osgeo.org/donations/ http://grass.osgeo.org/donations/]''' 
and, if needed, contact [[User:Neteler|Markus Neteler]] <tt><neteler at osgeo.org></tt>. Any surplus at the end of the event will be turned over to the GRASS GIS project.

The GRASS Community Sprint is a great occasion for you to support the development of GRASS. With your contribution you'll enable more developers to meet in Bonn. The community sprint is an important opportunity for the GRASS developers to discuss and collaboratively resolve bugs, plan the direction for the project and work on new features. Please see below for the more detailed agenda. The developers and contributors are donating their valuable time, so it would be great if in-kind funding can be made available from within the community to cover out-of-pocket expenses. All of the work that takes place at the community sprint will be directly contributed back into the GRASS project to the benefit of everyone who uses it.

=== Thanks to our sponsors ===

== Venue ==

Bonn, Germany: Basecamp

How to get to FOSS4G2016 OSGeo Code Sprint?
* http://alienus.de/basecamp1/en/arrival-bonn-hostel/
* http://www.openstreetmap.org/search?query=raste%201%20bonn#map=18/50.70762/7.12383

By train
* From Bonn Main station take the U-Trains (Underground platform) 16, 63 or 68. Get off after 6 min at stop Dt. Telekom / Ollenhauerstr.
* http://alienus.de/basecamp1/en/arrival-bonn-hostel/
* http://www.openstreetmap.org/search?query=raste%201%20bonn#map=18/50.70762/7.12383
* [https://www.vrsinfo.de/englisch/the-vrs/vrs-about-us.html online public transport planner]

By Car
* Coming from the highway A59 Basecamp is located at the end of the highway A562. For navigation systems it is best to type Basecamp's address: "''In der Raste 3, 53129 Bonn''")

== Accommodation and Costs ==

* to be defined

== Weather ==

* https://en.wikipedia.org/wiki/Bonn#Climate

== Agenda - What we plan to do ==

'''Note:''' The program is generally open for your ideas. Please edit this wiki page!

=== More core developers with SVN write access ===
* Develop a strategy to refresh the group with core SVN write access

=== Web site ===
* Migrate the good old CMSMS to Wordpress
* Move development related stuff from CMS (https://grass.osgeo.org/development/) to trac Wiki

=== Promo ===
* Finalize the new GRASS GIS flyer prepared by Vincent Bain
** needed: catchy time series screenshot

=== New Releases ===
Minor releases:
* Release GRASS GIS 6.4.6:
** Coordination is done here: http://trac.osgeo.org/grass/wiki/Grass6Planning
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/6.4.6-News
* Release GRASS GIS 7.0.5:
** Coordination is done here: https://trac.osgeo.org/grass/wiki/Grass7Planning#a7.0.5tobebackported
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/7.0.5-News

'''Major release''':
* Release GRASS GIS 7.2.0:
** Coordination is done here: https://trac.osgeo.org/grass/wiki/Grass7Planning#a7.2.0
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/7.2.0-News

=== Documentation and translation ===
* Add examples to manual pages
* Add examples to Wiki pages
** GRASS-R/R-GRASS handling of raster time series
** Sentinel-2 processing
* Translation of messages
* Translation of Wiki pages

=== Source code Quality Assessment ===
* integrate "coverity scan" with Travis, https://scan.coverity.com/travis_ci
* check "coverity scan": copy-paste errors, https://scan.coverity.com/projects/1038

=== Bug squashing ===
* Open bugs concerning GRASS 7: https://trac.osgeo.org/grass/report/19
* {{trac|2864}}: Add link to source code in the documentation pages (almost done, let's finish it!)
* {{trac|3042}}: Patches to make the build reproducible (fileordering, randomness)
* fix i.histo.match: https://lists.osgeo.org/pipermail/grass-dev/2013-February/062179.html
* Numpy and matplotlib issues when installing/updating from OSGeo4W (ticket [https://trac.osgeo.org/grass/ticket/3057 #3057])

=== Testing ===

Please test in GRASS 7.2.svn!
* {{cmd|d.legend.vect|version=72}}

=== Improvements ===
* improve {{AddonCmd|r.modis}} addons
* integrate Google Summer of Code projects (where adequate)
* continue working on OBIA toolchain:
** Testing of mean shift in i.segment
** Finalization of {{AddonCmd|r.object.geometry}} and its integration into {{AddonCmd|i.segment.uspo}}
* add R-batch files to OSGeo4W-winGRASS7 see [https://trac.osgeo.org/osgeo4w/ticket/413 osgeo4w/ticket 413]
* GUI: Implement an optional session "kickstart" in GRASS with LatLong predefined, then open import wizard | switch projection inside (like most other GIS do!)
* TGRASS:
** suffix option for output mapnames in t.rast.mapcalc and t.rast.algebra (tickets [https://trac.osgeo.org/grass/ticket/2294 #2294] and [https://trac.osgeo.org/grass/ticket/2736 #2736])
** pattern option to list and remove space-time data sets (ticket [https://trac.osgeo.org/grass/ticket/2804 #2804])
** t.rast.what: input file with coordinates (ticket [https://trac.osgeo.org/grass/ticket/2721 #2721])
** t.remove: dual step removal (ticket [https://trac.osgeo.org/grass/ticket/2752 #2752])
** support for space-time datasets in g.gui.gmodeler (ticket [https://trac.osgeo.org/grass/ticket/2977 #2977])

=== Better integration of git into development ===
* since github/gitlab/bitbucket etc are a valuable and recognized source of new source code, we should be able to receive such contributions easily
** OSGeo currently deploys a self-hosted "gitlab" repo: https://lists.osgeo.org/pipermail/sac/2016-August/007345.html

... YOUR IDEA HERE ...

== Planned participation ==

For organizational reasons, please '''additionally register''' at: https://wiki.osgeo.org/wiki/FOSS4G_2016_Code_Sprint

=== In person ===

Please add your name here or contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail com></tt>:

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|width=100px|'''Arrival'''
|width=100px|'''Departure'''
|'''Topic'''
|width=75px|'''T-Shirt'''
|'''Notes'''
|-
|1
|[[User:Lucadelu|Luca Delucchi]]
|Italy
| August 21
| August 28
|
| L
|
|-
|2
|[[User:Landa|Martin Landa]]
|Czech Republic
| August 21
| August 28
|
| L
|
|-
|3
|[[User:Neteler|Markus Neteler]]
|Italy
| August 21
| August 28
| Releases, project coordination, bug hunting, getting more ppl involved
| M
|
|-
|4
|[[User:NikosA|Nikos Alexandris]]
|Switzerland
| August 21
| August 28
|
| S or M
|
|-
|5
|[[User:Veroandreo|Veronica Andreo]]
|Argentina
| August 26
| August 28
|
| S
|
|-
|6
|[[User:mlennert|Moritz Lennert]]
|Belgium
| August 21
| August 27
| Releases, OBIA toolchain, GSoC module integration
| M
|
|-
|7
|[[User:sbl|Stefan Blumentrath]]
|Norway
| August 26
| August 28
| Port some Addons to python? (or whatever you put me up to)
| L
|
|-
|8
|[[User:MilenaN|Milena Nowotarska]]
|Poland
| August 21
| August 28
|
| M
|
|-
|9
|[[User:annazanchetta|Anna Zanchetta]]
|Italy - Israel
| August 22
| August 27
| Implementing and improving some imagery modules, inshallah
| S
|
|-
|}

=== Via IRC chat ===

(IRC is a nice archaic chat system, see [[IRC]])

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|-
|1
|[[User:Ychemin|Yann Chemin]]
|France
|-
|2
|[[User:hellik|Helmut Kudrnovsky]]
|Austria
|-
|}

== Individual Preparation ==

* Bring your own computer
* Bring [http://en.wikipedia.org/wiki/Mains_electricity_by_country your power connector adapter] if needed (photo: [https://en.wikipedia.org/wiki/File:Schuko_plug_and_socket.png Germany]). See also http://www.power-plugs-sockets.com/
* Install subversion and the compiler tools, and come with a working GRASS development environment if possible.

== Photos ==

We have to remember to take at least one group photo! :-)

== FAQ ==

* ''Is the GRASS Community Sprint just a coding event?''
** It is mainly a coding and documentation event. It is a working session for people who are already participants in the GRASS project and/or are committed to improving the GRASS project.
** On demand we can do some presentations of current working GRASS implementation and new upcoming features to spread the idea of Open Source GIS software
* ''Is the GRASS Community Sprint for developers only?''
** No: anybody can help, with testing, checking out bugs and fixes, documentation and more.
* ''Where can I get help and more information about the community sprint?''
** Contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail com></tt>

== Participants and Reports ==

===Moritz===
* OBIA Toolchain
** Cleaned up r.object.geometry and moved it to addons {{rev|69187}} + {{rev|69188}}, {{AddonCmd|r.object.geometry}}
** Modified {{AddonCmd|i.segment.stats}} to user.object.geometry instead of v.to.db
** Improved error handling in i.segment.stats and use temporary files on disk to reduce memory use
** Modified {{AddonCmd|i.segment.uspo}} to also allow testing new mean shift algorithm (currently only available in sandbox) {{rev|69236}}
** Tested mean shift algorithm
* Working on 6.4.6 release
** General bug triaging
** Backported fix of {{trac|1444}} to release_64
** Backported fix of {{trac|2558}} to release_64
* Scratched an old itch and implement a flag in {{Cmd|g.region}} to allow shell style output in flat (one line) style {{rev|69240}}

*[https://trac.osgeo.org/grass/timeline?from=Aug+23%2C+2016&daysback=2&authors=mlennert&milestone=on&ticket=on&changeset=on&wiki=on&sfp_email=&sfph_mail=&update=Update Log overview]

=== Markus ===
* assist CRO in cleaning PSC election voter email addresses
* minor pseudo-mercator discussions
* major datum shift file discussions with Even Rouault (GDAL) and fellow GRASS sprinters: libproj cleanup of local copies of NAD and datum shift files which are indeed used from GDAL; extended documentation in README.txt ({{trac|2456}}, {{rev|2456}})

=== Anna Petrasova ===
* {{rev|69199}}: major backport of Adam's GSoC work to 7.2. (please test!)
* fix {{trac|3133}}, {{trac|2252}}

=== Martin Landa ===

* Fix {{trac|2480}}, {{trac|2765}}
* Bug reports for 7.0.5 milestone reviewed, outdated tickets closed
* [https://trac.osgeo.org/grass/timeline?from=Aug+28%2C+2016&daysback=7&authors=martinl&milestone=on&ticket=on&changeset=on&wiki=on&sfp_email=&sfph_mail=&update=Update Log overview]

=== Vero ===

* Tutorial: R-GRASS integration for raster time series handling/processing. Work in progress.

=== Nikos ===

* CRO related tasks
* Patched [https://trac.osgeo.org/grass/changeset/69012/grass/trunk/imagery/i.atcorr/create_iwave.py create_iwave.py], see {{trac|3027}} -- mentored by Moritz.
* Working on {{Cmd|i.rgb.his}} and {{Cmd|i.his.rgb}} to allow for 16-bit input data (DCELL?)
* Reviewing options to fuse {{Cmd|i.pansharpen}} with/into {{AddonCmd|i.fusion.hpf}}

[[Category: Workshops]]
[[Category: Code Sprint]]
[[Category: 2016]]

GRASS Community Sprint Bonn 2016

2016-08-23T15:24:14Z

⚠️NikosA: Added activity during code sprint in Bonn

{{toc|right}}
During the [http://2016.foss4g.org/ FOSS4G 2016] , 21-28 August 2016, [http://en.wikipedia.org/wiki/Bonn Bonn], [https://en.wikipedia.org/wiki/Germany Germany], the GRASS GIS project will join the code sprint

'''Important dates:'''

* CodeSprint I: Sun 21.08.2016 12:00 - Tue 23.08.2016 (open end) (see also [http://2016.foss4g.org/programme.html#code-sprints here])
* [http://2016.foss4g.org/#page9 FOSS4G 2016 Schedule]
* CodeSprint II: Sat 27.08.2016 - Sun 28.08.2016 12:00 p.m. (see also [http://2016.foss4g.org/programme.html#code-sprints here])

== Purpose ==

Get together of GRASS GIS project members to take decisions and tackle small and larger problems.

For this meeting, we welcome people committed to improving the GRASS GIS project. This includes developers, documenters, bug reporters, translators, and users!

== Sponsors ==

If you are interested to sponsor the GRASS Community Sprint, please read about 
    '''sponsoring the GRASS project at [http://grass.osgeo.org/donations/ http://grass.osgeo.org/donations/]''' 
and, if needed, contact [[User:Neteler|Markus Neteler]] <tt><neteler at osgeo.org></tt>. Any surplus at the end of the event will be turned over to the GRASS GIS project.

The GRASS Community Sprint is a great occasion for you to support the development of GRASS. With your contribution you'll enable more developers to meet in Bonn. The community sprint is an important opportunity for the GRASS developers to discuss and collaboratively resolve bugs, plan the direction for the project and work on new features. Please see below for the more detailed agenda. The developers and contributors are donating their valuable time, so it would be great if in-kind funding can be made available from within the community to cover out-of-pocket expenses. All of the work that takes place at the community sprint will be directly contributed back into the GRASS project to the benefit of everyone who uses it.

=== Thanks to our sponsors ===

== Venue ==

Bonn, Germany: Basecamp

How to get to FOSS4G2016 OSGeo Code Sprint?
* http://alienus.de/basecamp1/en/arrival-bonn-hostel/
* http://www.openstreetmap.org/search?query=raste%201%20bonn#map=18/50.70762/7.12383

By train
* From Bonn Main station take the U-Trains (Underground platform) 16, 63 or 68. Get off after 6 min at stop Dt. Telekom / Ollenhauerstr.
* http://alienus.de/basecamp1/en/arrival-bonn-hostel/
* http://www.openstreetmap.org/search?query=raste%201%20bonn#map=18/50.70762/7.12383
* [https://www.vrsinfo.de/englisch/the-vrs/vrs-about-us.html online public transport planner]

By Car
* Coming from the highway A59 Basecamp is located at the end of the highway A562. For navigation systems it is best to type Basecamp's address: "''In der Raste 3, 53129 Bonn''")

== Accommodation and Costs ==

* to be defined

== Weather ==

* https://en.wikipedia.org/wiki/Bonn#Climate

== Agenda - What we plan to do ==

'''Note:''' The program is generally open for your ideas. Please edit this wiki page!

=== More core developers with SVN write access ===
* Develop a strategy to refresh the group with core SVN write access

=== Web site ===
* Migrate the good old CMSMS to Wordpress
* Move development related stuff from CMS (https://grass.osgeo.org/development/) to trac Wiki

=== Promo ===
* Finalize the new GRASS GIS flyer prepared by Vincent Bain
** needed: catchy time series screenshot

=== New Releases ===
Minor releases:
* Release GRASS GIS 6.4.6:
** Coordination is done here: http://trac.osgeo.org/grass/wiki/Grass6Planning
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/6.4.6-News
* Release GRASS GIS 7.0.5:
** Coordination is done here: https://trac.osgeo.org/grass/wiki/Grass7Planning#a7.0.5tobebackported
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/7.0.5-News

'''Major release''':
* Release GRASS GIS 7.2.0:
** Coordination is done here: https://trac.osgeo.org/grass/wiki/Grass7Planning#a7.2.0
** Changes are tracked at: https://trac.osgeo.org/grass/wiki/Release/7.2.0-News

=== Documentation and translation ===
* Add examples to manual pages
* Add examples to Wiki pages
** GRASS-R/R-GRASS handling of raster time series
** Sentinel-2 processing
* Translation of messages
* Translation of Wiki pages

=== Source code Quality Assessment ===
* integrate "coverity scan" with Travis, https://scan.coverity.com/travis_ci
* check "coverity scan": copy-paste errors, https://scan.coverity.com/projects/1038

=== Bug squashing ===
* Open bugs concerning GRASS 7: https://trac.osgeo.org/grass/report/19
* {{trac|2864}}: Add link to source code in the documentation pages (almost done, let's finish it!)
* {{trac|3042}}: Patches to make the build reproducible (fileordering, randomness)
* fix i.histo.match: https://lists.osgeo.org/pipermail/grass-dev/2013-February/062179.html
* Numpy and matplotlib issues when installing/updating from OSGeo4W (ticket [https://trac.osgeo.org/grass/ticket/3057 #3057])

=== Testing ===

Please test in GRASS 7.2.svn!
* {{cmd|d.legend.vect|version=72}}

=== Improvements ===
* improve {{AddonCmd|r.modis}} addons
* integrate Google Summer of Code projects (where adequate)
* continue working on OBIA toolchain:
** Testing of mean shift in i.segment
** Finalization of {{AddonCmd|r.object.geometry}} and its integration into {{AddonCmd|i.segment.uspo}}
* add R-batch files to OSGeo4W-winGRASS7 see [https://trac.osgeo.org/osgeo4w/ticket/413 osgeo4w/ticket 413]
* GUI: Implement an optional session "kickstart" in GRASS with LatLong predefined, then open import wizard | switch projection inside (like most other GIS do!)
* TGRASS:
** suffix option for output mapnames in t.rast.mapcalc and t.rast.algebra (tickets [https://trac.osgeo.org/grass/ticket/2294 #2294] and [https://trac.osgeo.org/grass/ticket/2736 #2736])
** pattern option to list and remove space-time data sets (ticket [https://trac.osgeo.org/grass/ticket/2804 #2804])
** t.rast.what: input file with coordinates (ticket [https://trac.osgeo.org/grass/ticket/2721 #2721])
** t.remove: dual step removal (ticket [https://trac.osgeo.org/grass/ticket/2752 #2752])
** support for space-time datasets in g.gui.gmodeler (ticket [https://trac.osgeo.org/grass/ticket/2977 #2977])

=== Better integration of git into development ===
* since github/gitlab/bitbucket etc are a valuable and recognized source of new source code, we should be able to receive such contributions easily
** OSGeo currently deploys a self-hosted "gitlab" repo: https://lists.osgeo.org/pipermail/sac/2016-August/007345.html

... YOUR IDEA HERE ...

== Planned participation ==

For organizational reasons, please '''additionally register''' at: https://wiki.osgeo.org/wiki/FOSS4G_2016_Code_Sprint

=== In person ===

Please add your name here or contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail com></tt>:

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|width=100px|'''Arrival'''
|width=100px|'''Departure'''
|'''Topic'''
|width=75px|'''T-Shirt'''
|'''Notes'''
|-
|1
|[[User:Lucadelu|Luca Delucchi]]
|Italy
| August 21
| August 28
|
| L
|
|-
|2
|[[User:Landa|Martin Landa]]
|Czech Republic
| August 21
| August 28
|
| L
|
|-
|3
|[[User:Neteler|Markus Neteler]]
|Italy
| August 21
| August 28
| Releases, project coordination, bug hunting, getting more ppl involved
| M
|
|-
|4
|[[User:NikosA|Nikos Alexandris]]
|Switzerland
| August 21
| August 28
|
| S or M
|
|-
|5
|[[User:Veroandreo|Veronica Andreo]]
|Argentina
| August 26
| August 28
|
| S
|
|-
|6
|[[User:mlennert|Moritz Lennert]]
|Belgium
| August 21
| August 27
| Releases, OBIA toolchain, GSoC module integration
| M
|
|-
|7
|[[User:sbl|Stefan Blumentrath]]
|Norway
| August 26
| August 28
| Port some Addons to python? (or whatever you put me up to)
| L
|
|-
|8
|[[User:MilenaN|Milena Nowotarska]]
|Poland
| August 21
| August 28
|
| M
|
|-
|9
|[[User:annazanchetta|Anna Zanchetta]]
|Italy - Israel
| August 22
| August 27
| Implementing and improving some imagery modules, inshallah
| S
|
|-
|}

=== Via IRC chat ===

(IRC is a nice archaic chat system, see [[IRC]])

{|class="wikitable sortable" border="2" cellspacing="0" cellpadding="4" rules="all" style="margin:1em 1em 1em 0; border:solid 1px #AAAAAA; border-collapse:collapse; background-color:#edf9c7; font-size:95%; empty-cells:show;"
|width=50px|'''No'''
|width=130px|'''Participant '''
|width=100px|'''Country'''
|-
|1
|[[User:Ychemin|Yann Chemin]]
|France
|-
|2
|[[User:hellik|Helmut Kudrnovsky]]
|Austria
|-
|}

== Individual Preparation ==

* Bring your own computer
* Bring [http://en.wikipedia.org/wiki/Mains_electricity_by_country your power connector adapter] if needed (photo: [https://en.wikipedia.org/wiki/File:Schuko_plug_and_socket.png Germany]). See also http://www.power-plugs-sockets.com/
* Install subversion and the compiler tools, and come with a working GRASS development environment if possible.

== Photos ==

We have to remember to take at least one group photo! :-)

== FAQ ==

* ''Is the GRASS Community Sprint just a coding event?''
** It is mainly a coding and documentation event. It is a working session for people who are already participants in the GRASS project and/or are committed to improving the GRASS project.
** On demand we can do some presentations of current working GRASS implementation and new upcoming features to spread the idea of Open Source GIS software
* ''Is the GRASS Community Sprint for developers only?''
** No: anybody can help, with testing, checking out bugs and fixes, documentation and more.
* ''Where can I get help and more information about the community sprint?''
** Contact [[User:Lucadelu|Luca Delucchi]] <tt><lucadeluge at gmail com></tt>

== Participants and Reports ==

===Moritz===
* OBIA Toolchain
** Cleaned up r.object.geometry and moved it to addons {{rev|69187}} + {{rev|69188}}, {{AddonCmd|r.object.geometry}}
** Modified {{AddonCmd|i.segment.stats}} to user.object.geometry instead of v.to.db
** Improved error handling in i.segment.stats and use temporary files on disk to reduce memory use
** Modified {{AddonCmd|i.segment.uspo}} to also allow testing new mean shift algorithm (currently only available in sandbox) {{rev|69236}}
** Tested mean shift algorithm
* Working on 6.4.6 release
** General bug triaging
** Backported fix of {{trac|1444}} to release_64
** Backported fix of {{trac|2558}} to release_64
* Scratched an old itch and implement a flag in {{Cmd|g.region}} to allow shell style output in flat (one line) style {{rev|69240}}

*[https://trac.osgeo.org/grass/timeline?from=Aug+23%2C+2016&daysback=2&authors=mlennert&milestone=on&ticket=on&changeset=on&wiki=on&sfp_email=&sfph_mail=&update=Update Log overview]

=== Markus ===
* assist CRO in cleaning PSC election voter email addresses
* minor pseudo-mercator discussions
* major datum shift file discussions with Even Rouault (GDAL) and fellow GRASS sprinters: libproj cleanup of local copies of NAD and datum shift files which are indeed used from GDAL; extended documentation in README.txt ({{trac|2456}}, {{rev|2456}})

=== Anna Petrasova ===
* {{rev|69199}}: major backport of Adam's GSoC work to 7.2. (please test!)
* fix {{trac|3133}}, {{trac|2252}}

=== Martin Landa ===

* Fix {{trac|2480}}, {{trac|2765}}
* Bug reports for 7.0.5 milestone reviewed, outdated tickets closed
* [https://trac.osgeo.org/grass/timeline?from=Aug+28%2C+2016&daysback=7&authors=martinl&milestone=on&ticket=on&changeset=on&wiki=on&sfp_email=&sfph_mail=&update=Update Log overview]

=== Vero ===

* Tutorial: R-GRASS integration for raster time series handling/processing. Work in progress.

=== Nikos ===

* Patched [https://trac.osgeo.org/grass/changeset/69012/grass/trunk/imagery/i.atcorr/create_iwave.py create_iwave.py], see {{trac|3027}} -- mentored by Moritz.
* Working on {{Cmd|i.rgb.his}} and {{Cmd|i.his.rgb}} to allow for 16-bit input data (DCELL?)
* Reviewing options to fuse {{Cmd|i.pansharpen}} with/into {{AddonCmd|i.fusion.hpf}}

[[Category: Workshops]]
[[Category: Code Sprint]]
[[Category: 2016]]

QuickBird

2016-08-14T08:19:48Z

⚠️NikosA: added i.quickbird.toar

== Overview ==

QuickBird is a commercial earth observation satellite. Details about the sensor are provided at Digital Globe's [http://www.digitalglobe.com/sites/default/files/QuickBird-DS-QB-PROD.pdf QuickBird Data Sheet]. Wikipedia's article on {{wikipedia|QuickBird}} provides an informative overview as well.

=== Quick Facts ===

* Panchromatic (black and white) and Multi-Spectral imagery
* Multi-spectral bands are ordered as

band 1 = blue
2 = green
3 = red
4 = Near IR

* Scenes are 16.5km x 16.5km.

* Data is 11-bit integers stored in a 16-bit integer field. Thus band minimum = 0, band maximum = 2047. 

* Available pixel resolutions

<ul>
{|
! Product
! Black & White
! Multispectral
! Color & Pan-sharpened
|-
| Basic
| 61 cm to 72 cm as collected
| 2.44 m to 2.88 m as collected
| not available
|-
| Standard
| 60 cm or 70 cm
| 2.4 m or 2.8 m
| 60 cm or 70 cm
|-
| Orthorectified
| 60 cm or 70 cm
| 2.4 m or 2.8 m
| 60 cm or 70 cm
|}
</ul>

=== Notes ===

From the whitepaper [http://www.apollomapping.com/wp-content/user_uploads/2011/09/Radiance_Conversion_of_QuickBird_Data.pdf Radiance Conversion of QuickBird Data]:

<blockquote>QuickBird products are delivered to the customer as radiometrically corrected image pixels (qPixel,Band).
Radiometric correction includes a dark offset subtraction and a non-uniformity correction (detector-to-detector
relative gain). Corrected counts are specific to the QuickBird instrument and therefore QuickBird imagery MUST be
converted to spectral radiance before radiometric/spectral analysis or comparison with imagery from other sensors in
a radiometric/spectral manner.</blockquote>

== Availability (Sample Data) ==

* Search for commercial satellite image providers in the internet.
* Some sample QuickBird2 data are available at Digital Globe's collection of [http://www.digitalglobe.com/product-samples Imagery Product Samples]
* [http://www.landcover.org/ The Global Land Cover Facility (GLCF)] provides some [http://glcf.umd.edu/data/quickbird/ QuickBird] acqusitions for education or research purposes only.

== Modules overview ==

=== Outside of GRASS ===

* gdalinfo
* gdalwarp

=== Inside GRASS GIS ===

* {{cmd|r.in.gdal}}
* {{cmd|r.mapcalc}}
* {{cmd|r.colors}}
* {{cmd|i.colors.enhance|version=70}}
* {{AddonSrc|imagery|i.fusion.hpf|version=7}}
* {{cmd|i.pansharpen|version=70}} (GRASS 7.x)
* [https://github.com/NikosAlexandris/i.quickbird.toar i.quickbird.toar]
* {{cmd|i.vi|version=70}} (GRASS 7.x)
* {{cmd|i.segment|version=70}} (GRASS 7.x)

== Pre-Processing ==

=== Overview ===

Typically, multispectral satellite data are converted into physical quantities such as ''Radiance'' or ''Reflectance'' before they are subjected in multispectral analysis techniques (image interpretation, band arithmetic, vegetation indices, matrix transformations, etc.). The latter can be differentiated in ''Top of Atmosphere Reflectance'' (ToAR) which does not account for atmospheric effects (absorption or scattering) and in ''Top of Canopy Reflectance'' (ToCR) which introduces a "correction" for atmospheric effects.

In order to derive Reflectance values, likewise as with remotely sensed data acquired by other sensors, QuickBird raw image digital numbers (DNs), also referred to as ''radiometrically corrected image pixels'' need to be converted to ''Top of Atmosphere Radiance'' values. Spectral Radiance values are then converted to Reflectance values based on various acquisition related parameters. Note, ''Spectal Radiance'' is the measure of the quantity of radiation that hits the sensor and typically expressed in <math>\frac{W}{m^2*sr*nm}</math>, that is ''watts'' per ''unit source area'', per ''unit solid angle'', and per ''unit wavelength''.

<ul>

* Converting QuickBird DNs to ''Top of Atmosphere'' Radiance, as described by Keith Krause (2005) in ''Radiometric Use of QuickBird Imagery, Technical Note'', can be done by using the following equation:

<math>L_{\lambda\text{Pixel, Band}} = \frac{K_{\text{Band}} * q_{\text{Pixel, Band}}}{\Delta\lambda_{\text{Band}}}</math>

 where:

<ul>
* <math>L_{\lambda\text{Pixel,Band}}</math>: Top-of-Atmosphere Spectral Radiance image pixels [W-m-2-sr-1-μm-1]
* <math>K_{\text{Band}}</math>: the absolute radiometric calibration factor [W-m-2-sr-1-count-1] for a given band
* <math>q_{\text{Pixel,Band}}</math>: radiometrically corrected image pixels [counts or Digital Numbers]
* <math>\Delta_{\lambda_{\text{Band}}}</math>: the effective bandwidth [μm] for a given band
</ul>

* Converting to Top of Atmosphere Reflectance, also referred to as Planetary Reflectance, can be done by using the following equation:

<math>\rho_p = \frac{\pi * L\lambda * d^2}{ESUN\lambda * cos(\Theta_S)}</math>

 where:

<ul>
* <math>\rho</math> - Unitless Planetary Reflectance
* <math>\pi</math> - mathematical constant (3.14159265358)
* <math>L\lambda</math> spectral Radiance at the sensor's aperture, from equation... '''''ToADD'''''
* <math>d</math> - Earth-Sun distance in astronomical units, interpolated values
* <math>Esun</math> - Mean solar exoatmospheric irradiance(s) (W/m2/μm), interpolated values
* <math>cos(\theta_s)</math> - {{wikipedia||Solar_zenith_angle}} from the image acquisition's metadata
</ul>

</ul>

=== Importing data ===

==== File Formats & Metadata ====

* GeoTIFF, NITF 2.1, NITF 2.0
* Image bits / pixel: 8 or 16 bits

Setting up a Location in GRASS GIS requires to know the Spatial Reference System in which the data of interest are projected. Retrieving the projection parameters of QuickBird imagery, can be done by using the `gdalinfo` program.

''ToAdd''
<ul>
* Various ways of Location creation based on georeferenced data
</ul>

==== Importing ====

Once inside an appropriate Location, importing the data is done with the {{cmd|r.in.gdal}} module.

For example, GeoTIFF files/bands, can be imported in one go by looping {{cmd|r.in.gdal}} over all of them

<source lang="bash">
for TIF in `echo *.tif`; do r.in.gdal in=${TIF} out=${TIF%%.*}; done
</source>

=== Cleanup ===
Rename the NIR band
{{cmd|g.rename}} QBird_Multichrom.4,QBird_Multichrom.NIR

It is very likely that the Red and Blue bands are swapped (in older datasets), so fix that by renaming them. (see [http://osdir.com/ml/gis.gdal.devel/2004-03/msg00025.html])
g.rename QBird_Multichrom.blue,QBird_Multichrom.realred
g.rename QBird_Multichrom.red,QBird_Multichrom.blue
g.rename QBird_Multichrom.realred,QBird_Multichrom.red

No data is set to 0, so convert that to NULL with {{cmd|r.null}}:
for BAND in red green blue NIR ; do
r.null QBird_Multichrom.$BAND setnull=0
echo $BAND:
r.info -r QBird_Multichrom.$BAND
echo
done

Set color tables appropriately for 11-bit data:
for BAND in red green blue NIR ; do
{{cmd|r.colors}} QBird_Multichrom.$BAND color=rules << EOF
0 black
2047 white
EOF
done

It is not necessarily needed, but if you wish to rescale to 8-bit data you can use {{cmd|r.rescale}} as follows:
for BAND in red green blue NIR ; do
r.rescale in=QBird_Multichrom.$BAND from=0,2047 \
out=QBird_Multichrom255.$BAND to=0,255
done

Display the 3-band RGB image:
{{cmd|d.rgb}} r=QBird_Multichrom.red g=QBird_Multichrom.green b=QBird_Multichrom.blue

If the image is excessively dark you might try using the {{cmd|i.landsat.rgb}} module to auto-balance the colors and redraw:
i.landsat.rgb r=QBird_Multichrom.red g=QBird_Multichrom.green b=QBird_Multichrom.blue
d.redraw

=== Deriving Physical Quantities ===

Converting Digital Numbers to Radiance/Reflectance requires knowledge about the sensor's specific spectral band parameters. These are, as extracted from the document ''Radiometric Use of QuickBird Imagery, Technical Note. 2005-11-07, by Keith Krause.''

<source lang=bash>
# Band-Averaged Solar Spectral Irradiance [W/sq.m./μm]
Pan_Esun=1381.79
Blue_Esun=1924.59
Green_Esun=1843.08
Red_Esun=1574.77
NIR_Esun=1113.71

# Revised K Factors & Effective Bandwidths

# 1st column: K Conversion Factors (& Pan TDI Level) for 16-Bit products [W/sq.m./sr/count]
# 2nd column: Effective Bandwidth [μm] per Spectral Band

K16_Pan10=0.08381880 ; Pan10_Width=0.398
K16_Pan13=0.06447600 ; Pan13_Width=0.398
K16_Pan18=0.04656600 ; Pan18_Width=0.398
K16_Pan24=0.03494440 ; Pan24_Width=0.398
K16_Pan32=0.02618840 ; Pan32_Width=0.398
K16_Blue=0.01604120 ; Blue_Width=0.068
K16_Green=0.01438470 ; Green_Width=0.099
K16_Red=0.01267350 ; Red_Width=0.071
K16_NIR=0.01542420 ; NIR_Width=0.114

# 1st column: k′ Conversion Factors for 8-Bit products
# 2nd column being identical (as above) not repeated!

K8_Pan10=1.02681367
K8_Pan13=1.02848939
K8_Pan18=1.02794702
K8_Pan24=1.02989685
K8_Pan32=1.02739898
K8_Blue=1.12097834
K8_Green=1.37652632
K8_Red=1.30924587
K8_NIR=0.98368622
</source>

Attention for 32-bit floating point calculations is requried. As stated, in the same document, 

''NOTE: conversion equations are to be performed on all pixels in a given band of a QuickBird image and should use 32-bit floating point calculations.''

==== Spectral Radiance ====

Converting a Blue Band (Digital Numbers) in to Spectral Radiance in the ''correct'' units to be further used for the conversion in to unitless Reflectance:

<source lang="bash">
# set the region
g.region rast=Blue_DNs -p

# convert DNs to spectral Radiance values
r.mapcalc "Blue_Radiance = '''ToAdd''' "
</source>

==== Planetary Reflectance ====

The equation to derive Reflectance values incorporates in addition:

<ul>
* The mathematical constant {{wikipedia|Pi}} <source lang="bash" enclose=none>PI=3.14159265358</source>.
* The Earth-Sun distance in astronomical units which depends on the acquisition's day of year (DOY -- also referred to as Julian day, {{wikipedia|Ordinal_date}}) and can be retrieved from the following spreadsheet <http://landsathandbook.gsfc.nasa.gov/excel_docs/d.xls>.
* The mean solar exoatmospheric irradiance in <math>\frac{W}{m^2*\mu m}</math>. See 3rd column of (interplated) values given above.
* The cosine of the ''Solar Zenith Angle'' (SZA) at the time of the acquisition. The SZA can be calculated from its complementary ''Solar Elevation Angle'' (SEA) given in the image acquisition's metadata.
</ul>

In the following example we accept as the acquisition's <source lang="bash" enclose=none>DOY=...</source> and <source lang="bash" enclose=none>SEA=...</source>. Hence, we get the Earth-Sun distance <source lang="bash" enclose=none>ESD=...</source> and <source lang="bash" enclose=none>SZA = ... deg</source>.

Converting the in-Blue spectral band at-sensor Radiance in to Planerary Reflectance:

<source lang="bash" enclose=none>PI=3.14159265358; ESD=...; BAND_Esun=...; SZA=...</source>
{{cmd|r.mapcalc}} <source lang="bash" enclose=none>"Blue_Reflectance = ( ${PI} * Blue_Radiance * ${ESD}^2 ) / ( ${BAND_Esun} * cos(${SZA}) )"</source>

==== Automatising Conversions ====

Scripting the conversion process avoids repeating the same steps for each band separately.

===== in Python =====

A custom python script, performing the operations of interest, might be like [https://github.com/NikosAlexandris/i.quickbird.toar i.quickbird.toar (for GRASS 7.x)]

===== in bash =====

Am example bash shell script, might look like the one that follows. '''Note,''' however, constants, band parameters and acquisition related metadata are hard-coded! Reviewing the code and altering appropriately is required, i.e. checking for the parameters <code>ESD</code>, <code>SEA</code>, <code>PanTDI</code>, <code>K_BAND</code>.

<source lang="bash">
#!bin/bash

# Pi, first 11 decimals
PI=3.14159265358

# HardCoded MetaData!

# Acquisition's Day of Year and estimated Earth-Sun Distance
DOY=274; ESD=1.001190

# Sun Zenith Angle based on the acquisition's Sun Elevation Angle
SEA=67.8; SZA=$(echo "90 - ${SEA}" | bc )

# which Pan TDI level?
PanTDI="10" # evaluate below...

# Spectral Irradiance [W-m-2-μm-1]

Pan_Esun=1381.79
Blue_Esun=1924.59
Green_Esun=1843.08
Red_Esun=1574.77
NIR_Esun=1113.71

# 1st column: K Conversion Factors for 16-Bit products
# 2nd column: Effective Bandwidth [μm] per Spectral Band

K16_Pan10=0.08381880; Pan10_Width=0.398
K16_Pan13=0.06447600; Pan13_Width=0.398
K16_Pan18=0.04656600; Pan18_Width=0.398
K16_Pan24=0.03494440; Pan24_Width=0.398
K16_Pan32=0.02618840; Pan32_Width=0.398
K16_Blue=0.01604120 ; Blue_Width=0.068
K16_Green=0.01438470; Green_Width=0.099
K16_Red=0.01267350 ; Red_Width=0.071
K16_NIR=0.01542420 ; NIR_Width=0.114

# 1st column: k′ Conversion Factors for 8-Bit products
# Effective Bandwidths [μm] per Spectral Band: as above

K8_Pan10=1.02681367
K8_Pan13=1.02848939
K8_Pan18=1.02794702
K8_Pan24=1.02989685
K8_Pan32=1.02739898
K8_Blue=1.12097834
K8_Green=1.37652632
K8_Red=1.30924587
K8_NIR=0.98368622

# Bands
Spectral_Bands="Pan Blue Green Red NIR"

# loop over all bands
for BAND in ${Spectral_Bands}; do

# which Pan TDI level?
if [[ ${BAND} == Pan ]]
then
eval Pan="Pan${PanTDI}"; echo "Note, processing for ${Pan}"
eval BAND="${Pan}"
fi

# some echo...
echo "Processing the ${BAND} spectral band"

# set band parameters as variables -- using K16 for 16-bit data!
eval K_BAND="K16_${BAND}"
eval BAND_Width="${BAND}_Width"
echo "Band Parameters set to K=${!K_BAND}, Bandwidth=${!BAND_Width}"

if [[ ${BAND} == Pan* ]]
then
BAND="Pan" ; echo "Processing the ${BAND} spectral band"
fi

# set region
g.region rast=${BAND}_DNs #-pg
echo "Region matching the ${BAND} spectral band"

# conversion to Radiance based on (1) -- attention: 32-bit calculations required
r.mapcalc "${BAND}_Radiance = ( double(${!K_BAND}) * ${BAND}_DNs ) / ${!BAND_Width}"
r.info -r "${BAND}_Radiance"

# add info
r.support map=${BAND}_Radiance \
title="" \
units="W / sq.m. / μm / ster" \
description="Top-of-Atmosphere `echo ${BAND}` band spectral Radiance [W/m^2/sr/μm]" \
source1='"Radiometric Use of QuickBird Imagery, Technical Note (2005)," by Keith Krause, Digital Globe'

eval BAND_Esun="${BAND}_Esun"; echo "Using Esun=${!BAND_Esun}"

# calculate ToAR -- ${BAND}_Radiance is already 32-bit -- see above!
r.mapcalc "${BAND}_ToAR = \
( ${PI} * ${BAND}_Radiance * ${ESD}^2 ) / ( ${!BAND_Esun} * cos(${SZA}) )"
r.info -r ${BAND}_ToAR

# add some metadata
r.support map=${BAND}_ToAR \
title="echo ${BAND} band (Top of Atmosphere Reflectance)" \
units="Unitless planetary reflectance" \
description="Top of Atmosphere `echo ${BAND}` band spectral Reflectance (unitless)" \
source1='"Radiometric Use of QuickBird Imagery, Technical Note (2005)," by Keith Krause, Digital Globe' \
source2="Digital Globe" \
history="PI=3.14159265358; K=${!K_BAND}; Bandwidth=${!BAND_Width}; ESD=${ESD}; Esun=${!BAND_Esun}; SZA=${SZA}"

done
</source>

=== Atmospheric correction ===

See [[Atmospheric correction]]

== Post-Processing ==

Having beforehand satellite image data expressed in physical quantities (radiance or reflectance) is preferred to follow-up with digital image analysis techniques.

== Pan Sharpening ==

In GRASS ver. 6.x, one can use the {{cmd|i.fusion.brovey}} module.

<source lang="bash">
i.fusion.brovey -q ms1=qbird_green ms2=qbird_nir ms3=qbird_red pan=qbird_pan outputprefix=brov
</source>

[[Category: Documentation]]
[[Category: Geodata]]
[[Category: High Resolution]]
[[Category: QuikBird]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Image processing]]

=== HPFA based sharpening ===

The following screenshots exemplify the High Pass Filtering Addition fusion technique applied in a fragment of the QuickBird acquisition "04APR05050541-X2AS_R1C1-000000186011_01_P001-Sri_Lanka-Kokilai_Lagoon" which is publicly available via the GLCF. An implementation of this technique for GRASS-GIS is available as a grass-addon {{AddonSrc|imagery|i.fusion.hpf|version=7}}.

{|
|[[File:Pan_04APR05050541-M2AS-000000186011_01_P001.jpg|thumb|400px|border|center|Fragment from the high resolution (0.7m) panchromatic image]]
|[[File:RGB_04APR05050541-M2AS-000000186011_01_P001.jpg|thumb|400px|border|center|Fragment from the low resolution (2.8m) multi-spectral RGB composite image]]
|}

{|
|[[File:RGB_HPF_Sharpened_Default_Parameters.jpg|thumb|400px|border|center|Fragment from the HPFA Pan-Sharpened RGB composite image. Sharpening performed with default parameters.]]
|[[File:RGB_HPF_Sharpened_Center_Low_Modulator_Min.jpg|thumb|400px|border|center|Fragment from another HPFA Pan-Sharpened RGB composite image. Sharpening performed with ''center=low'' and ''modulator=min'' parameters.]]
|}

QuickBird

2016-08-14T08:17:17Z

⚠️NikosA: updated list of grass related modules

== Overview ==

QuickBird is a commercial earth observation satellite. Details about the sensor are provided at Digital Globe's [http://www.digitalglobe.com/sites/default/files/QuickBird-DS-QB-PROD.pdf QuickBird Data Sheet]. Wikipedia's article on {{wikipedia|QuickBird}} provides an informative overview as well.

=== Quick Facts ===

* Panchromatic (black and white) and Multi-Spectral imagery
* Multi-spectral bands are ordered as

band 1 = blue
2 = green
3 = red
4 = Near IR

* Scenes are 16.5km x 16.5km.

* Data is 11-bit integers stored in a 16-bit integer field. Thus band minimum = 0, band maximum = 2047. 

* Available pixel resolutions

<ul>
{|
! Product
! Black & White
! Multispectral
! Color & Pan-sharpened
|-
| Basic
| 61 cm to 72 cm as collected
| 2.44 m to 2.88 m as collected
| not available
|-
| Standard
| 60 cm or 70 cm
| 2.4 m or 2.8 m
| 60 cm or 70 cm
|-
| Orthorectified
| 60 cm or 70 cm
| 2.4 m or 2.8 m
| 60 cm or 70 cm
|}
</ul>

=== Notes ===

From the whitepaper [http://www.apollomapping.com/wp-content/user_uploads/2011/09/Radiance_Conversion_of_QuickBird_Data.pdf Radiance Conversion of QuickBird Data]:

<blockquote>QuickBird products are delivered to the customer as radiometrically corrected image pixels (qPixel,Band).
Radiometric correction includes a dark offset subtraction and a non-uniformity correction (detector-to-detector
relative gain). Corrected counts are specific to the QuickBird instrument and therefore QuickBird imagery MUST be
converted to spectral radiance before radiometric/spectral analysis or comparison with imagery from other sensors in
a radiometric/spectral manner.</blockquote>

== Availability (Sample Data) ==

* Search for commercial satellite image providers in the internet.
* Some sample QuickBird2 data are available at Digital Globe's collection of [http://www.digitalglobe.com/product-samples Imagery Product Samples]
* [http://www.landcover.org/ The Global Land Cover Facility (GLCF)] provides some [http://glcf.umd.edu/data/quickbird/ QuickBird] acqusitions for education or research purposes only.

== Modules overview ==

=== Outside of GRASS ===

* gdalinfo
* gdalwarp

=== Inside GRASS GIS ===

* {{cmd|r.in.gdal}}
* {{cmd|r.mapcalc}}
* {{cmd|r.colors}}
* {{cmd|i.colors.enhance|version=70}}
* {{AddonSrc|imagery|i.fusion.hpf|version=7}}
* {{cmd|i.pansharpen|version=70}} (GRASS 7.x)
* {{cmd|i.vi|version=70}} (GRASS 7.x)
* {{cmd|i.segment|version=70}} (GRASS 7.x)

== Pre-Processing ==

=== Overview ===

Typically, multispectral satellite data are converted into physical quantities such as ''Radiance'' or ''Reflectance'' before they are subjected in multispectral analysis techniques (image interpretation, band arithmetic, vegetation indices, matrix transformations, etc.). The latter can be differentiated in ''Top of Atmosphere Reflectance'' (ToAR) which does not account for atmospheric effects (absorption or scattering) and in ''Top of Canopy Reflectance'' (ToCR) which introduces a "correction" for atmospheric effects.

In order to derive Reflectance values, likewise as with remotely sensed data acquired by other sensors, QuickBird raw image digital numbers (DNs), also referred to as ''radiometrically corrected image pixels'' need to be converted to ''Top of Atmosphere Radiance'' values. Spectral Radiance values are then converted to Reflectance values based on various acquisition related parameters. Note, ''Spectal Radiance'' is the measure of the quantity of radiation that hits the sensor and typically expressed in <math>\frac{W}{m^2*sr*nm}</math>, that is ''watts'' per ''unit source area'', per ''unit solid angle'', and per ''unit wavelength''.

<ul>

* Converting QuickBird DNs to ''Top of Atmosphere'' Radiance, as described by Keith Krause (2005) in ''Radiometric Use of QuickBird Imagery, Technical Note'', can be done by using the following equation:

<math>L_{\lambda\text{Pixel, Band}} = \frac{K_{\text{Band}} * q_{\text{Pixel, Band}}}{\Delta\lambda_{\text{Band}}}</math>

 where:

<ul>
* <math>L_{\lambda\text{Pixel,Band}}</math>: Top-of-Atmosphere Spectral Radiance image pixels [W-m-2-sr-1-μm-1]
* <math>K_{\text{Band}}</math>: the absolute radiometric calibration factor [W-m-2-sr-1-count-1] for a given band
* <math>q_{\text{Pixel,Band}}</math>: radiometrically corrected image pixels [counts or Digital Numbers]
* <math>\Delta_{\lambda_{\text{Band}}}</math>: the effective bandwidth [μm] for a given band
</ul>

* Converting to Top of Atmosphere Reflectance, also referred to as Planetary Reflectance, can be done by using the following equation:

<math>\rho_p = \frac{\pi * L\lambda * d^2}{ESUN\lambda * cos(\Theta_S)}</math>

 where:

<ul>
* <math>\rho</math> - Unitless Planetary Reflectance
* <math>\pi</math> - mathematical constant (3.14159265358)
* <math>L\lambda</math> spectral Radiance at the sensor's aperture, from equation... '''''ToADD'''''
* <math>d</math> - Earth-Sun distance in astronomical units, interpolated values
* <math>Esun</math> - Mean solar exoatmospheric irradiance(s) (W/m2/μm), interpolated values
* <math>cos(\theta_s)</math> - {{wikipedia||Solar_zenith_angle}} from the image acquisition's metadata
</ul>

</ul>

=== Importing data ===

==== File Formats & Metadata ====

* GeoTIFF, NITF 2.1, NITF 2.0
* Image bits / pixel: 8 or 16 bits

Setting up a Location in GRASS GIS requires to know the Spatial Reference System in which the data of interest are projected. Retrieving the projection parameters of QuickBird imagery, can be done by using the `gdalinfo` program.

''ToAdd''
<ul>
* Various ways of Location creation based on georeferenced data
</ul>

==== Importing ====

Once inside an appropriate Location, importing the data is done with the {{cmd|r.in.gdal}} module.

For example, GeoTIFF files/bands, can be imported in one go by looping {{cmd|r.in.gdal}} over all of them

<source lang="bash">
for TIF in `echo *.tif`; do r.in.gdal in=${TIF} out=${TIF%%.*}; done
</source>

=== Cleanup ===
Rename the NIR band
{{cmd|g.rename}} QBird_Multichrom.4,QBird_Multichrom.NIR

It is very likely that the Red and Blue bands are swapped (in older datasets), so fix that by renaming them. (see [http://osdir.com/ml/gis.gdal.devel/2004-03/msg00025.html])
g.rename QBird_Multichrom.blue,QBird_Multichrom.realred
g.rename QBird_Multichrom.red,QBird_Multichrom.blue
g.rename QBird_Multichrom.realred,QBird_Multichrom.red

No data is set to 0, so convert that to NULL with {{cmd|r.null}}:
for BAND in red green blue NIR ; do
r.null QBird_Multichrom.$BAND setnull=0
echo $BAND:
r.info -r QBird_Multichrom.$BAND
echo
done

Set color tables appropriately for 11-bit data:
for BAND in red green blue NIR ; do
{{cmd|r.colors}} QBird_Multichrom.$BAND color=rules << EOF
0 black
2047 white
EOF
done

It is not necessarily needed, but if you wish to rescale to 8-bit data you can use {{cmd|r.rescale}} as follows:
for BAND in red green blue NIR ; do
r.rescale in=QBird_Multichrom.$BAND from=0,2047 \
out=QBird_Multichrom255.$BAND to=0,255
done

Display the 3-band RGB image:
{{cmd|d.rgb}} r=QBird_Multichrom.red g=QBird_Multichrom.green b=QBird_Multichrom.blue

If the image is excessively dark you might try using the {{cmd|i.landsat.rgb}} module to auto-balance the colors and redraw:
i.landsat.rgb r=QBird_Multichrom.red g=QBird_Multichrom.green b=QBird_Multichrom.blue
d.redraw

=== Deriving Physical Quantities ===

Converting Digital Numbers to Radiance/Reflectance requires knowledge about the sensor's specific spectral band parameters. These are, as extracted from the document ''Radiometric Use of QuickBird Imagery, Technical Note. 2005-11-07, by Keith Krause.''

<source lang=bash>
# Band-Averaged Solar Spectral Irradiance [W/sq.m./μm]
Pan_Esun=1381.79
Blue_Esun=1924.59
Green_Esun=1843.08
Red_Esun=1574.77
NIR_Esun=1113.71

# Revised K Factors & Effective Bandwidths

# 1st column: K Conversion Factors (& Pan TDI Level) for 16-Bit products [W/sq.m./sr/count]
# 2nd column: Effective Bandwidth [μm] per Spectral Band

K16_Pan10=0.08381880 ; Pan10_Width=0.398
K16_Pan13=0.06447600 ; Pan13_Width=0.398
K16_Pan18=0.04656600 ; Pan18_Width=0.398
K16_Pan24=0.03494440 ; Pan24_Width=0.398
K16_Pan32=0.02618840 ; Pan32_Width=0.398
K16_Blue=0.01604120 ; Blue_Width=0.068
K16_Green=0.01438470 ; Green_Width=0.099
K16_Red=0.01267350 ; Red_Width=0.071
K16_NIR=0.01542420 ; NIR_Width=0.114

# 1st column: k′ Conversion Factors for 8-Bit products
# 2nd column being identical (as above) not repeated!

K8_Pan10=1.02681367
K8_Pan13=1.02848939
K8_Pan18=1.02794702
K8_Pan24=1.02989685
K8_Pan32=1.02739898
K8_Blue=1.12097834
K8_Green=1.37652632
K8_Red=1.30924587
K8_NIR=0.98368622
</source>

Attention for 32-bit floating point calculations is requried. As stated, in the same document, 

''NOTE: conversion equations are to be performed on all pixels in a given band of a QuickBird image and should use 32-bit floating point calculations.''

==== Spectral Radiance ====

Converting a Blue Band (Digital Numbers) in to Spectral Radiance in the ''correct'' units to be further used for the conversion in to unitless Reflectance:

<source lang="bash">
# set the region
g.region rast=Blue_DNs -p

# convert DNs to spectral Radiance values
r.mapcalc "Blue_Radiance = '''ToAdd''' "
</source>

==== Planetary Reflectance ====

The equation to derive Reflectance values incorporates in addition:

<ul>
* The mathematical constant {{wikipedia|Pi}} <source lang="bash" enclose=none>PI=3.14159265358</source>.
* The Earth-Sun distance in astronomical units which depends on the acquisition's day of year (DOY -- also referred to as Julian day, {{wikipedia|Ordinal_date}}) and can be retrieved from the following spreadsheet <http://landsathandbook.gsfc.nasa.gov/excel_docs/d.xls>.
* The mean solar exoatmospheric irradiance in <math>\frac{W}{m^2*\mu m}</math>. See 3rd column of (interplated) values given above.
* The cosine of the ''Solar Zenith Angle'' (SZA) at the time of the acquisition. The SZA can be calculated from its complementary ''Solar Elevation Angle'' (SEA) given in the image acquisition's metadata.
</ul>

In the following example we accept as the acquisition's <source lang="bash" enclose=none>DOY=...</source> and <source lang="bash" enclose=none>SEA=...</source>. Hence, we get the Earth-Sun distance <source lang="bash" enclose=none>ESD=...</source> and <source lang="bash" enclose=none>SZA = ... deg</source>.

Converting the in-Blue spectral band at-sensor Radiance in to Planerary Reflectance:

<source lang="bash" enclose=none>PI=3.14159265358; ESD=...; BAND_Esun=...; SZA=...</source>
{{cmd|r.mapcalc}} <source lang="bash" enclose=none>"Blue_Reflectance = ( ${PI} * Blue_Radiance * ${ESD}^2 ) / ( ${BAND_Esun} * cos(${SZA}) )"</source>

==== Automatising Conversions ====

Scripting the conversion process avoids repeating the same steps for each band separately.

===== in Python =====

A custom python script, performing the operations of interest, might be like [https://github.com/NikosAlexandris/i.quickbird.toar i.quickbird.toar (for GRASS 7.x)]

===== in bash =====

Am example bash shell script, might look like the one that follows. '''Note,''' however, constants, band parameters and acquisition related metadata are hard-coded! Reviewing the code and altering appropriately is required, i.e. checking for the parameters <code>ESD</code>, <code>SEA</code>, <code>PanTDI</code>, <code>K_BAND</code>.

<source lang="bash">
#!bin/bash

# Pi, first 11 decimals
PI=3.14159265358

# HardCoded MetaData!

# Acquisition's Day of Year and estimated Earth-Sun Distance
DOY=274; ESD=1.001190

# Sun Zenith Angle based on the acquisition's Sun Elevation Angle
SEA=67.8; SZA=$(echo "90 - ${SEA}" | bc )

# which Pan TDI level?
PanTDI="10" # evaluate below...

# Spectral Irradiance [W-m-2-μm-1]

Pan_Esun=1381.79
Blue_Esun=1924.59
Green_Esun=1843.08
Red_Esun=1574.77
NIR_Esun=1113.71

# 1st column: K Conversion Factors for 16-Bit products
# 2nd column: Effective Bandwidth [μm] per Spectral Band

K16_Pan10=0.08381880; Pan10_Width=0.398
K16_Pan13=0.06447600; Pan13_Width=0.398
K16_Pan18=0.04656600; Pan18_Width=0.398
K16_Pan24=0.03494440; Pan24_Width=0.398
K16_Pan32=0.02618840; Pan32_Width=0.398
K16_Blue=0.01604120 ; Blue_Width=0.068
K16_Green=0.01438470; Green_Width=0.099
K16_Red=0.01267350 ; Red_Width=0.071
K16_NIR=0.01542420 ; NIR_Width=0.114

# 1st column: k′ Conversion Factors for 8-Bit products
# Effective Bandwidths [μm] per Spectral Band: as above

K8_Pan10=1.02681367
K8_Pan13=1.02848939
K8_Pan18=1.02794702
K8_Pan24=1.02989685
K8_Pan32=1.02739898
K8_Blue=1.12097834
K8_Green=1.37652632
K8_Red=1.30924587
K8_NIR=0.98368622

# Bands
Spectral_Bands="Pan Blue Green Red NIR"

# loop over all bands
for BAND in ${Spectral_Bands}; do

# which Pan TDI level?
if [[ ${BAND} == Pan ]]
then
eval Pan="Pan${PanTDI}"; echo "Note, processing for ${Pan}"
eval BAND="${Pan}"
fi

# some echo...
echo "Processing the ${BAND} spectral band"

# set band parameters as variables -- using K16 for 16-bit data!
eval K_BAND="K16_${BAND}"
eval BAND_Width="${BAND}_Width"
echo "Band Parameters set to K=${!K_BAND}, Bandwidth=${!BAND_Width}"

if [[ ${BAND} == Pan* ]]
then
BAND="Pan" ; echo "Processing the ${BAND} spectral band"
fi

# set region
g.region rast=${BAND}_DNs #-pg
echo "Region matching the ${BAND} spectral band"

# conversion to Radiance based on (1) -- attention: 32-bit calculations required
r.mapcalc "${BAND}_Radiance = ( double(${!K_BAND}) * ${BAND}_DNs ) / ${!BAND_Width}"
r.info -r "${BAND}_Radiance"

# add info
r.support map=${BAND}_Radiance \
title="" \
units="W / sq.m. / μm / ster" \
description="Top-of-Atmosphere `echo ${BAND}` band spectral Radiance [W/m^2/sr/μm]" \
source1='"Radiometric Use of QuickBird Imagery, Technical Note (2005)," by Keith Krause, Digital Globe'

eval BAND_Esun="${BAND}_Esun"; echo "Using Esun=${!BAND_Esun}"

# calculate ToAR -- ${BAND}_Radiance is already 32-bit -- see above!
r.mapcalc "${BAND}_ToAR = \
( ${PI} * ${BAND}_Radiance * ${ESD}^2 ) / ( ${!BAND_Esun} * cos(${SZA}) )"
r.info -r ${BAND}_ToAR

# add some metadata
r.support map=${BAND}_ToAR \
title="echo ${BAND} band (Top of Atmosphere Reflectance)" \
units="Unitless planetary reflectance" \
description="Top of Atmosphere `echo ${BAND}` band spectral Reflectance (unitless)" \
source1='"Radiometric Use of QuickBird Imagery, Technical Note (2005)," by Keith Krause, Digital Globe' \
source2="Digital Globe" \
history="PI=3.14159265358; K=${!K_BAND}; Bandwidth=${!BAND_Width}; ESD=${ESD}; Esun=${!BAND_Esun}; SZA=${SZA}"

done
</source>

=== Atmospheric correction ===

See [[Atmospheric correction]]

== Post-Processing ==

Having beforehand satellite image data expressed in physical quantities (radiance or reflectance) is preferred to follow-up with digital image analysis techniques.

== Pan Sharpening ==

In GRASS ver. 6.x, one can use the {{cmd|i.fusion.brovey}} module.

<source lang="bash">
i.fusion.brovey -q ms1=qbird_green ms2=qbird_nir ms3=qbird_red pan=qbird_pan outputprefix=brov
</source>

[[Category: Documentation]]
[[Category: Geodata]]
[[Category: High Resolution]]
[[Category: QuikBird]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Image processing]]

=== HPFA based sharpening ===

The following screenshots exemplify the High Pass Filtering Addition fusion technique applied in a fragment of the QuickBird acquisition "04APR05050541-X2AS_R1C1-000000186011_01_P001-Sri_Lanka-Kokilai_Lagoon" which is publicly available via the GLCF. An implementation of this technique for GRASS-GIS is available as a grass-addon {{AddonSrc|imagery|i.fusion.hpf|version=7}}.

{|
|[[File:Pan_04APR05050541-M2AS-000000186011_01_P001.jpg|thumb|400px|border|center|Fragment from the high resolution (0.7m) panchromatic image]]
|[[File:RGB_04APR05050541-M2AS-000000186011_01_P001.jpg|thumb|400px|border|center|Fragment from the low resolution (2.8m) multi-spectral RGB composite image]]
|}

{|
|[[File:RGB_HPF_Sharpened_Default_Parameters.jpg|thumb|400px|border|center|Fragment from the HPFA Pan-Sharpened RGB composite image. Sharpening performed with default parameters.]]
|[[File:RGB_HPF_Sharpened_Center_Low_Modulator_Min.jpg|thumb|400px|border|center|Fragment from another HPFA Pan-Sharpened RGB composite image. Sharpening performed with ''center=low'' and ''modulator=min'' parameters.]]
|}

WorldView

2016-08-14T08:16:44Z

⚠️NikosA: updated list of grass related modules

= This page is under construction =

= Overview =

WorldView-1, WorldView-2 and WorldView-3 are commercial earth observation satellites. Details about the sensors are provided at Digital Globe's collection of [http://www.digitalglobe.com/resources/satellite-information Spacecraft Data Sheets].

It is recommended -- actually it is a necessity! -- to have a look at the documentation explaining [http://www.digitalglobe.com/sites/default/files/Imagery_Support_Data_Documentation%20%281%29.pdf Digital Globe's products metadata]. Note, all of DigitalGlobe’s satellites collect data using an 11 bit dynamic range. For technical reasons, products are delivered as either 16-bit or 8-bit data.

== Availability (Sample Data) ==

* Search for commercial satellite image providers in the internet.

== Modules overview ==

=== Outside of GRASS ===

Satellite imagery can be managed and, at some extent, pre-processed with various [http://www.gdal.org/gdal_utilities.html GDAL utilities].

* [http://www.gdal.org/gdalmanage.html gdalmanage]
* [http://www.gdal.org/gdalinfo.html gdalinfo]
* [http://www.gdal.org/gdal_translate.html gdal_translate]
* [http://www.gdal.org/gdalbuildvrt.html gdalbuildvrt]
* [http://www.gdal.org/gdalwarp.html gdalwarp]

=== Inside GRASS GIS ===

GRASS GIS features a complete set of modules and various add-ons enabling pre- and post-processing of WorldView satellite imagery. The following lists offer an overview of related modules and add-ons.

* {{cmd|r.in.gdal}}
* {{cmd|r.mapcalc}}
* {{cmd|r.colors}}
* {{cmd|i.colors.enhance|version=70}}
* {{AddonSrc|imagery|i.fusion.hpf|version=7}}
* {{cmd|i.pansharpen}}
* [https://github.com/NikosAlexandris/i.worldview.toar i.worldview.toar]
* {{cmd|i.vi}}
* {{cmd|i.segment}}
* ...''more to add''

== Pre-Processing ==

=== Overview ===

Typically, multispectral satellite data are converted into physical quantities such as ''Radiance'' or ''Reflectance'' before they are subjected in multispectral analysis techniques (image interpretation, band arithmetic, vegetation indices, matrix transformations, etc.). The latter can be differentiated in ''Top of Atmosphere Reflectance'' (ToAR) which does not account for atmospheric effects (absorption or scattering) and in ''Top of Canopy Reflectance'' (ToCR) which introduces a "correction" for atmospheric effects.

In order to derive Reflectance values, likewise as with remotely sensed data acquired by other sensors, WorldView raw image digital numbers (DNs) need to be converted to ''at-sensor spectral Radiance'' values. At-sensor spectral Radiance values are an important input for the equation to derive Reflectance values. Note, ''Spectal Radiance'' is the measure of the quantity of radiation that hits the sensor and typically expressed in <math>\frac{W}{m^2*sr*nm}</math>, that is ''watts'' per ''unit source area'', per ''unit solid angle'', and per ''unit wavelength''.

<ul>

* Converting WorldView DNs to ''Top of Atmosphere'' Radiance, as described in [http://www.digitalglobe.com/downloads/Radiometric_Use_of_WorldView-2_Imagery.pdf Radiometric Use of WorldView-2 Imagery, Technical Note (2010)] by Todd Updike and Chris Comp, can be done by using the following equation:

<math>L_{\lambda\text{Pixel, Band}} = \frac{K_{\text{Band}} * q_{\text{Pixel, Band}}}{\Delta\lambda_{\text{Band}}}</math>

 where:

<ul>
* <math>L_{\lambda\text{Pixel,Band}}</math>: Top-of-Atmosphere Spectral Radiance image pixels [W-m-2-sr-1-μm-1]
* <math>K_{\text{Band}}</math>: the absolute radiometric calibration factor [W-m-2-sr-1-count-1] for a given band
* <math>q_{\text{Pixel,Band}}</math>: radiometrically corrected image pixels [counts or Digital Numbers]
* <math>\Delta_{\lambda_{\text{Band}}}</math>: the effective bandwidth [μm] for a given band
</ul>

* Converting to Top of Atmosphere Reflectance, also referred to as Planetary Reflectance, can be done by using the following equation:

<math>\rho_p = \frac{\pi * L\lambda * d^2}{ESUN\lambda * cos(\Theta_S)}</math>

 where:

<ul>
* <math>\rho</math> - Unitless Planetary Reflectance
* <math>\pi</math> - mathematical constant (3.14159265358)
* <math>L\lambda</math> spectral Radiance at the sensor's aperture, from equation... '''''ToADD'''''
* <math>d</math> - Earth-Sun distance in astronomical units, interpolated values
* <math>Esun</math> - Mean solar exoatmospheric irradiance(s) (W/m2/μm), interpolated values
* <math>cos(\theta_s)</math> - {{wikipedia||Solar_zenith_angle}} from the image acquisition's metadata
</ul>

</ul>

=== Importing data ===

==== File Formats & Metadata ====

''ToAdd''

==== Importing ====

''ToAdd''
<ul>
* Location creation based on georeferenced data
</ul>

Once inside a Location that is defined by the spatial reference system in which the bands of interest are projected, they can be imported with the {{cmd|r.in.gdal}} module.

For example, GeoTIFF files can be imported by looping {{cmd|r.in.gdal}} over all of them

<source lang="bash">
for TIF in `echo *.tif`; do r.in.gdal in=${TIF} out=${TIF%%.*}; done
</source>

To keep the raw material untouched, we create another Mapset inside the same Location and copy over the bands giving, ''optionally'' at the same time, a new name for each.

''ToAdd''

=== Deriving Physical Quantities ===

Conversions implemented in the GRASS module [https://github.com/NikosAlexandris/i.worldview.toar i.worldview.toar]

''ToCleanUp''

Converting Digital Numbers to Radiance/Reflectance requires knowledge about the sensor's specific spectral band parameters. In the following listed parameters (see below), the ''Effective Bandwidth''s and the ''Band-Averaged Solar Spectral Irradiance''s are extracted from the document [http://www.digitalglobe.com/downloads/Radiometric_Use_of_WorldView-2_Imagery.pdf Radiometric Use of WorldView-2 Imagery, Technical Note (2010)] by Todd Updike and Chris Comp. The Absolute Calibration Factors are extracted from a WorldView-2 product, specifically from an image metadata file (extension .IMD).

'''Note,''' however, as stated in the above document,

The absolute radiometric calibration factor is dependent on the specific band,
as well as the TDI exposure level, line rate, pixel aggregation, and bit depth
of the product. Based on these parameters, the appropriate value is provided in
the .IMD file. For this reason, care should be taken not to mix absolute radiometric
calibration factors between products that might have different collection conditions.

<source lang=bash>
Pan_CalCoef=0.05678345 ; Pan_Width=0.2846000 ; Pan_Esun=1580.8140
Coastal_CalCoef=0.009295654 ; Coastal_CalCoef=0.04730000 ; Coastal_Esun=1758.2229
Blue_CalCoef=0.01260825 ; Blue_Width=0.05430000 ; Blue_Esun=1974.2416
Green_CalCoef=0.009713071 ; Green_Width=0.06300000 ; Green_Esun=1856.4104
Yellow_CalCoef=0.005829815 ; Yellow_Width=0.03740000 ; Yellow_Esun=1738.4791
Red_CalCoef=0.01103623 ; Red_Width=0.05740000 ; Red_Esun=1559.4555
RedEdge_CalCoef=0.005188136 ; RedEdge_Width=0.03930000 ; Red_Edge_Esun=1342.0695
NIR1_CalCoef=0.01224380 ; NIR1_Width=0.09890000 ; NIR_Esun=1069.7302
NIR2_CalCoef=0.009042234 ; NIR2_Width=0.09960000 ; NIR2_Esun=861.2866
</source>

The following examples exemplify the conversion of raw Blue band digital numbers into Radiance and Reflectance.

==== Spectral Radiance ====

Converting a Blue Band (Digital Numbers) in to Spectral at-sensor Radiance in the ''correct'' units to be further used for the conversion in to unitless Reflectance:

<source lang="bash">
# set the region
g.region rast=Blue_DNs -p

# convert DNs to spectral Radiance values
r.mapcalc "Blue_Radiance = ( (10000 * WorldView_Blue_DNs) / (... * ...) )"
</source>

==== Planetary Reflectance ====

The equation to derive Reflectance values incorporates in addition:

<ul>
* The mathematical constant {{wikipedia|Pi}} <source lang="bash" enclose=none>PI=3.14159265358</source>.
* The Earth-Sun distance in astronomical units which depends on the acquisition's day of year (DOY -- also referred to as Julian day, {{wikipedia|Ordinal_date}}) and can be retrieved from the following spreadsheet <http://landsathandbook.gsfc.nasa.gov/excel_docs/d.xls>.
* The mean solar exoatmospheric irradiance in <math>\frac{W}{m^2*\mu m}</math>. See 3rd column of (interplated) values given above.
* The cosine of the ''Solar Zenith Angle'' (SZA) at the time of the acquisition. The SZA can be calculated from its complementary ''Solar Elevation Angle'' (SEA) given in the image acquisition's metadata.
</ul>

In the following example we accept as the acquisition's <source lang="bash" enclose=none>DOY=...</source> and <source lang="bash" enclose=none>SEA=...</source>. Hence, we get the Earth-Sun distance <source lang="bash" enclose=none>ESD=...</source> and <source lang="bash" enclose=none>SZA = ... deg</source>.

In any case, '''note''' that Top-of-Atmosphere Reflectance does not account for topographic, atmospheric, or BRDF differences.

Converting the in-Blue spectral band at-sensor Radiance in to Planerary Reflectance:

<source lang="bash" enclose=none>PI=...; ESD=...; BAND_Esun=...; SZA=...</source>
{{cmd|r.mapcalc}} <source lang="bash" enclose=none>"Blue_Reflectance = ( ${PI} * Blue_Radiance * ${ESD}^2 ) / ( ${BAND_Esun} * cos(${SZA}) )"</source>

==== Automatising Conversions ====

The conversion process can be scripted to avoid repeating the same steps for each band separately.

===== Python =====

A custom python script, performing the operations of interest, might be like [https://github.com/NikosAlexandris/i.worldview.toar i.worldview.toar (for GRASS 7.x)]

===== Bash =====
In bash, such a script might be as the following example. '''Note,''' however, in this example script constants, band parameters and acquisition related metadata are hard-coded!

<source lang="bash">
#!/bin/bash

# π, first 11 decimals
PI=3.14159265358

# HardCoded MetaData!

# Acquisition's Day of Year and estimated Earth-Sun Distance
DOY=100; ESD=1.00184

# Sun Zenith Angle based on the acquisition's Sun Elevation Angle
SEA=53.8; SZA=$(echo "90 - ${SEA}" | bc )

# some echo
echo "Acquisition-specific parameters"
echo "Day of Year: ${DOY}, Earth-Sun Distance: ${ESD}, Sun Zenith Angle: ${SZA}"
echo -e "\n"

# 1st column: Absolute Calibration Factors
# 2nd column: Spectral Band Effective Bandwidth, Δλ
# 3rd column: Band-Averaged Solar Spectral Irradiance [W/sq.m./micro-m]

K_Pan=0.05678345 ; Pan_Width=0.2846000 ; Pan_Esun=1580.8140
K_Coastal=0.009295654 ; Coastal_Width=0.04730000 ; Coastal_Esun=1758.2229
K_Blue=0.01260825 ; Blue_Width=0.05430000 ; Blue_Esun=1974.2416
K_Green=0.009713071 ; Green_Width=0.06300000 ; Green_Esun=1856.4104
K_Yellow=0.005829815 ; Yellow_Width=0.03740000 ; Yellow_Esun=1738.4791
K_Red=0.01103623 ; Red_Width=0.05740000 ; Red_Esun=1559.4555
K_RedEdge=0.005188136 ; RedEdge_Width=0.03930000 ; RedEdge_Esun=1342.0695
K_NIR1=0.01224380 ; NIR1_Width=0.09890000 ; NIR1_Esun=1069.7302
K_NIR2=0.009042234 ; NIR2_Width=0.09960000 ; NIR2_Esun=861.2866

# Bands
Spectral_Bands="Pan Coastal Blue Green Yellow Red RedEdge NIR1 NIR2"

# loop over all bands
for BAND in ${Spectral_Bands}; do

# some echo
echo -e "Processing the \"${BAND}\" spectral band"

# set band parameters as variables
eval BAND_Width="${BAND}_Width"
echo "* Band-specific radiometric parameters set to K=${!K_BAND}, Bandwidth=${!BAND_Width}"

# set region
g.region rast=${BAND}_DNs

# conversion to Radiance based on (1) -- attention: 32-bit calculations required
echo "* Converting ${BAND} Digital Numebrs to Radiance..." | tr -d "\n"
r.mapcalc "${BAND}_Radiance = ( double(${!K_BAND}) * ${BAND}_DNs ) / ${!BAND_Width}"

# report range
echo "* Reporting range of the ${BAND} spectral radiance: " | tr -d "\n"
# r.info -r "${BAND}_Radiance"
r.info -r "${BAND}_Radiance" | tr "\n" "," | cut -d"," -f1,2 | sed 's/,/,\ /'

# add info
r.support map=${BAND}_Radiance \
title="" \
units="W / sq.m. / μm / ster" \
description="Top-of-Atmosphere `echo ${BAND}` band spectral Radiance [W/m^2/sr/μm]" \
source1='"Radiometric Use of WorldView-2 Imagery, Technical Note (2010)", by Todd Updike & Chris Comp'

# Esun
eval BAND_Esun="${BAND}_Esun"

# some echo
echo "* Parameters required for the conversion: Esun= ${!BAND_Esun}, ESD= ${ESD}, ${SZA}"

# calculate ToAR
echo "* Converting ${BAND}_Radiance to Top of Atmosphere Reflectance..." | tr -d "\n"
r.mapcalc "${BAND}_ToAR = \
( ${PI} * ${BAND}_Radiance * ${ESD}^2 ) / ( ${!BAND_Esun} * cos(${SZA}) )"

# report range
echo "* Reporting range of the ${BAND} spectral radiance: " | tr -d "\n"
r.info -r ${BAND}_ToAR | tr "\n" "," | cut -d"," -f1,2 | sed 's/,/,\ /'

# add some metadata
r.support map=${BAND}_ToAR \
title="echo ${BAND} band (Top of Atmosphere Reflectance)" \
units="Unitless planetary reflectance" \
description="Top of Atmosphere `echo ${BAND}` band spectral Reflectance (unitless)" \
source1='"Radiometric Use of WorldView-2 Imagery, Technical Note (2010)", by Todd Updike & Chris Comp, Digital Globe' \
source2="Digital Globe" \
history="PI=3.14159265358; K=${!K_BAND}; Bandwidth=${!BAND_Width}; ESD=${ESD}; Esun=${!BAND_Esun}; SZA=${SZA}"

echo -e "\n"

done
</source>

=== Atmospheric correction ===

See [[Atmospheric correction]]

== Post-Processing ==

Having beforehand satellite image data expressed in physical quantities (radiance or reflectance) is preferred to follow-up with digital image analysis techniques. A few common post-processing practices are Contrast-Enhancement, Pan-Sharpening and creating Pseudo- or True-Color Composites. Other well known enhancing manipualtions to support the analyses of satellite imagery, include deriving Vegetation Indices, transforming multi-spectral data based on [[Principal_Components_Analysis | PCA]] and Segmenting images.

=== Contrast Enhancement ===

''ToAdd''

=== Pan-Sharpening ===

[http://en.wikipedia.org/wiki/Pansharpened_image Pan-Sharpening] / [http://en.wikipedia.org/wiki/Image_fusion Fusion] is the process of merging high-resolution panchromatic and lower resolution multi-spectral imagery. [http://grass.osgeo.org/grass70/ GRASS 7] holds a dedicated pan-sharpening module, {{cmd|i.pansharpen}} which features three techniques for sharpening, namely the [http://wiki.awf.forst.uni-goettingen.de/wiki/index.php/Brovey_Transformation Brovey transformation], the classical IHS method and one that is based on [[Principal Components Analysis]] (PCA).
Another algorithm deriving excellent detail and a realistic representation of original multispectral scene colors, is the High-Pass Filter Addition (HPFA) technique. It is implemented via the {{AddonSrc|imagery|i.fusion.hpf|version=7}} add-on (for GRASS 6, please refer to a bash shell script https://github.com/NikosAlexandris/i.fusion.hpf.sh which is, however, unmaintained).

One approach inside GRASS-GIS to get an acceptable color-balanced composite image after Pan-sharpening 11-bit IKONOS spectral bands comprises the following steps

<ul>
# rescale the 11-bit WorldView spectral bands to 8-bit ranging in [0, 255] ({{cmd|r.rescale}})
# pan-sharpen with any of the featured methods (Brovey, IHS, PCA) ({{cmd|i.pansharpen}})
# color-balance by using the {{cmd|i.landsat.rgb}} module or manually adjusting the color tables of the bands of interest
# create a composite image by using the {{cmd|r.compose}} module
</ul>

''ToAdd''

==== Automatising Sharpening & RGB Composition ====

An ''experimental'' bash script to automatise the sharpening/fusion and RGB composition process is demonstrated here. The script uses the Spectral Reflectance values (double precision values), gained from previous steps (see above), and produces Pan-sharpened images by applying all of the three methods that {{cmd|i.pansharpen}} offers. In addition, it attempts to produce True-Color composite images, without and with re-balancing the color tables by using the {{cmd|i.landsat.rgb}} module.

Initial tests indicate that at least one of the three sharpening methods, combined with re-balancing, produces very clear and balanced True-Color composites. Note, there is no 100% confidence that it will produce nice looking True-Color composites. Some of the methods might simply work, others might deliver fancy-colored images. As usual, some manual action might be required to get the re-balancing to work as best as possible.

The script can be expanded in terms of using more inputs by utilising bash's positional parameters. However, care has to be taken to alter the instruction that concerns the conversion from double precision values to 8-bit integers.

<source lang="bash">
...
</source>

=== Color Composites ===

''ToAdd''

== Vegetation Indices ==

''ToAdd''

== PCA ==

''ToAdd''

== WordView Image classification ==

See [[Image classification]]

== References / Sources ==

* ...
* [http://landsathandbook.gsfc.nasa.gov/data_prod/prog_sect11_3.html Landsat7 Science Data Users Handbook, Chapter 11, Section 3]
* [http://igett.delmar.edu/Resources/Remote%20Sensing%20Technology%20Training/Calculation-DN_to_Reflectance_Irish_20June08.pdf Calibrated Landsat Digital Number (DN) to Top of Atmosphere (TOA) Reflectance Conversion, by Richard Irish]
* from [http://landsat.usgs.gov/tools_access_all_faqs.php FAQs about the Landsat Missions]
** [http://landsat.usgs.gov/how_is_radiance_calculated.php How is radiance calculated?]
** [http://landsat.usgs.gov/at_sensor_reflectance_calculated.php How is at-sensor reflectance calculated?]

== See also ==

* GRASS-Wiki page about [http://grasswiki.osgeo.org/wiki/Image_processing Image Processing]

''ToAdd''

[[Category: Documentation]]
[[Category: FAQ]]
[[Category: WorldView]]
[[Category: High Resolution]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Image processing]]

IKONOS

2016-08-14T08:15:54Z

⚠️NikosA: updated list of grass related modules

= Overview =

IKONOS is a commercial earth observation satellite. Details about the sensor are provided at Digital Globe's [http://www.digitalglobe.com/sites/default/files/DG_IKONOS_DS.pdf IKONOS Data Sheet]. [http://en.wikipedia.org/wiki/Ikonos Wikipedia's article on IKONOS] provides a nice overview as well.

== Availability (Sample Data) ==

* Search for commercial satellite image providers in the internet.
* [http://www.landcover.org/ The Global Land Cover Facility (GLCF)] provides [ftp://ftp.glcf.umd.edu/glcf/China_earthquake_May_2008/IKONOS/ four openly available IKONOS scenes of western Sichuan].
* ISPRS provides a small [http://www.isprs.org/data/ikonos/ IKONOS data set], fragments from a Panchromatic image as well as from a Stereo product.

== Modules overview ==

=== Outside of GRASS ===

Satellite imagery can be managed and, at some extent, pre-processed with various [http://www.gdal.org/gdal_utilities.html GDAL utilities].

* [http://www.gdal.org/gdalmanage.html gdalmanage]
* [http://www.gdal.org/gdalinfo.html gdalinfo]
* [http://www.gdal.org/gdal_translate.html gdal_translate]
* [http://www.gdal.org/gdalbuildvrt.html gdalbuildvrt]
* [http://www.gdal.org/gdalwarp.html gdalwarp]

=== Inside GRASS GIS ===

GRASS GIS features a complete set of modules and various add-ons enabling pre- and post-processing of IKONOS satellite imagery. The following lists offer an overview of related modules and add-ons.

* {{cmd|r.in.gdal}}
* {{cmd|r.mapcalc}}
* {{cmd|r.colors}}
* {{cmd|i.colors.enhance|version=70}}
* {{AddonSrc|imagery|i.fusion.hpf|version=7}}
* {{cmd|i.pansharpen}}
* [https://github.com/NikosAlexandris/i.ikonos.toar i.ikonos.toar]
* {{cmd|i.vi}}
* {{cmd|i.segment}}

== Pre-Processing ==

=== Overview ===

Typically, multispectral satellite data are converted into physical quantities such as ''Radiance'' or ''Reflectance'' before they are subjected in multispectral analysis techniques (image interpretation, band arithmetic, vegetation indices, matrix transformations, etc.). The latter can be differentiated in ''Top of Atmosphere Reflectance'' (ToAR) which does not account for atmospheric effects (absorption or scattering) and in ''Top of Canopy Reflectance'' (ToCR) which introduces a "correction" for atmospheric effects.

In order to derive Reflectance values, likewise as with remotely sensed data acquired by other sensors, IKONOS raw image digital numbers (DNs) need to be converted to ''at-sensor spectral Radiance'' values. At-sensor spectral Radiance values are an important input for the equation to derive Reflectance values. Note, ''Spectal Radiance'' is the measure of the quantity of radiation that hits the sensor and typically expressed in <math>\frac{W}{m^2*sr*nm}</math>, that is ''watts'' per ''unit source area'', per ''unit solid angle'', and per ''unit wavelength''.

<ul>
* Converting DNs to at-sensor Radiance can be done by using the following equation:

<math>L\lambda = \frac{10^4 * DN\lambda}{CalCoef\lambda * Bandwidth\lambda}</math>

* Converting to Top of Atmosphere Reflectance, also referred to as Planetary Reflectance, can be done by using the following equation:

<math>\rho_p = \frac{\pi * L\lambda * d^2}{ESUN\lambda * cos(\Theta_S)}</math>

 where:

<ul>
* <math>\rho</math> - Unitless Planetary Reflectance
* <math>\pi</math> - mathematical constant (3.14159265358)
* <math>L\lambda</math> spectral Radiance at the sensor's aperture, from equation... '''''ToADD'''''
* <math>d</math> - Earth-Sun distance in astronomical units, interpolated values
* <math>Esun</math> - Mean solar exoatmospheric irradiance(s) (W/m2/μm), interpolated values
* <math>cos(\theta_s)</math> - {{wikipedia||Solar_zenith_angle}} from the image acquisition's metadata
</ul>

</ul>

=== Importing data ===

==== File Formats & Metadata ====

''ToAdd''

==== Importing ====

In the following example the publicly available IKONOS acqisition [ftp://ftp.glcf.umd.edu/glcf/China_earthquake_May_2008/IKONOS/po_58204_0000000.20001116.China-Sichuan/ po_58204_0000000.20001116.China-Sichuan] is used.

''ToAdd''
<ul>
* Location creation based on georeferenced data
</ul>

Once inside a Location that is defined by the spatial reference system in which the bands of interest are projected, they can be imported with the {{cmd|r.in.gdal}} module.

For example, GeoTIFF files can be imported by looping {{cmd|r.in.gdal}} over all of them

<source lang="bash">
for TIF in `echo *.tif`; do r.in.gdal in=${TIF} out=${TIF%%.*}; done
</source>

To keep the raw material untouched, we create another Mapset inside the same Location and copy over the bands giving, ''optionally'' at the same time, a new name for each.

<source lang="bash">
g.mapset -c pre-processing
g.copy rast=po_58204_blu_0000000,Blue_DNs
g.copy rast=po_58204_grn_0000000,Green_DNs
g.copy rast=po_58204_red_0000000,Red_DNs
g.copy rast=po_58204_nir_0000000,NIR_DNs
g.copy rast=po_58204_pan_0000000,Pan_DNs
</source>

=== Deriving Physical Quantities ===

Conversions are implemented in a GRASS module available at https://github.com/NikosAlexandris/i.ikonos.toar.

''Details''

Converting Digital Numbers to Radiance/Reflectance requires knowledge about the sensor's specific spectral band parameters. Those are, as extracted from the document ''IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance'', by Martin Taylor (see references):

<source lang=bash>
Pan_CalCoef=161 ; Pan_Width=403 ; Pan_Esun=1375.8
Blue_CalCoef=728 ; Blue_Width=71.3 ; Blue_Esun=1930.9
Green_CalCoef=720 ; Green_Width=88.6 ; Green_Esun=1854.8
Red_CalCoef=949 ; Red_Width=65.8 ; Red_Esun=1556.5
NIR_CalCoef=843 ; NIR_Width=95.4 ; NIR_Esun=1156.9
</source>

The following examples exemplify the conversion of raw Blue band digital numbers into Radiance and Reflectance.

==== Spectral Radiance ====

Converting a Blue Band (Digital Numbers) in to Spectral at-sensor Radiance in the ''correct'' units to be further used for the conversion in to unitless Reflectance:

<source lang="bash">
# set the region
g.region rast=Blue_DNs -p

# convert DNs to spectral Radiance values
r.mapcalc "Blue_Radiance = ( (10000 * IKONOS_Blue_DNs) / (728 * 71.3) )"
</source>

==== Planetary Reflectance ====

The equation to derive Reflectance values incorporates in addition:

<ul>
* The mathematical constant {{wikipedia|Pi}} <source lang="bash" enclose=none>PI=3.14159265358</source>.
* The Earth-Sun distance in astronomical units which depends on the acquisition's day of year (DOY -- also referred to as Julian day, {{wikipedia|Ordinal_date}}) and can be retrieved from the following spreadsheet <http://landsathandbook.gsfc.nasa.gov/excel_docs/d.xls>.
* The mean solar exoatmospheric irradiance in <math>\frac{W}{m^2*\mu m}</math>. See 3rd column of (interplated) values given above.
* The cosine of the ''Solar Zenith Angle'' (SZA) at the time of the acquisition. The SZA can be calculated from its complementary ''Solar Elevation Angle'' (SEA) given in the image acquisition's metadata.
</ul>

In the following example we accept as the acquisition's <source lang="bash" enclose=none>DOY=166</source> and <source lang="bash" enclose=none>SEA=52.78880</source>. Hence, we get the Earth-Sun distance <source lang="bash" enclose=none>ESD=1.0157675</source> and <source lang="bash" enclose=none>SZA = 37.21120 deg</source>.

Converting the in-Blue spectral band at-sensor Radiance in to Planerary Reflectance:

<source lang="bash" enclose=none>PI=3.14159265358; ESD=1.0157675; BAND_Esun=1930.9; SZA=37.21120</source>
{{cmd|r.mapcalc}} <source lang="bash" enclose=none>"Blue_Reflectance = ( ${PI} * Blue_Radiance * ${ESD}^2 ) / ( ${BAND_Esun} * cos(${SZA}) )"</source>

==== Automatising Conversions ====

The conversion process can be scripted to avoid repeating the same steps for each band separately.

===== Python =====

A custom python script, performing the operations of interest, might be like [https://github.com/NikosAlexandris/i.ikonos.toar i.ikonos.toar (for GRASS 7.x)]

===== Bash =====
In bash, such a script might be as the following example. '''Note,''' however, in this example script constants, band parameters and acquisition related metadata are hard-coded!

<source lang="bash">
#!/bin/bash

Pan_CalCoef=161 ; Pan_Width=403 ; Pan_Esun=1375.8
Blue_CalCoef=728 ; Blue_Width=71.3 ; Blue_Esun=1930.9
Green_CalCoef=720 ; Green_Width=88.6 ; Green_Esun=1854.8
Red_CalCoef=949 ; Red_Width=65.8 ; Red_Esun=1556.5
NIR_CalCoef=843 ; NIR_Width=95.4 ; NIR_Esun=1156.9

# set constants, band parameters and acquisition related metadata here!

# Pi, first 11 decimals
PI=3.14159265358

# Acquisition's Day of Year and estimated Earth-Sun Distance
DOY=166; ESD=1.0157675

# Sun Zenith Angle based on the acquisition's Sun Elevation Angle
SEA=52.78880; SZA=$(echo "90 - ${SEA}" | bc )

# bands to process
Spectral_Bands="Pan Blue Green Red NIR"

# loop over all bands
for BAND in ${Spectral_Bands}; do

echo "Processing the ${BAND} spectral band"

# set region
g.region rast=${BAND}_DNs

# set band parameters as variables
eval BAND_CalCoef="${BAND}_CalCoef"
eval BAND_Width="${BAND}_Width";
echo "Band Parameters set to CalCoef=${!BAND_CalCoef}, Bandwidth=${!BAND_Width}"

# conversion to Radiance
r.mapcalc "${BAND}_Radiance = ( ( 10^4 * ${BAND}_DNs ) / ( ${!BAND_CalCoef} * ${!BAND_Width} ) )"

# add info
r.support map=${BAND}_Radiance \
title="" \
units="W / m2 / μm / ster" \
description="At-sensor `echo ${BAND}` band spectral Radiance (W/m2/μm/sr)" \
source1='"IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance", by Martin Taylor, Geoeye'

# set Earth-Sun distance
eval BAND_Esun="${BAND}_Esun"; echo "Using Esun=${!BAND_Esun}"

# calculate ToAR
r.mapcalc "${BAND}_ToAR = \
( ${PI} * ${BAND}_Radiance * ${ESD}^2 ) / ( ${!BAND_Esun} * cos(${SZA}) )"

# add some metadata
r.support map=${BAND}_ToAR \
title="echo ${BAND} band (Top of Atmosphere Reflectance)" \
units="Unitless" \
description="Top of Atmosphere `echo ${BAND}` band spectral Reflectance (unitless)" \
source1='"IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance", by Martin Taylor, Geoeye' \
source2="e.g., the Image Provider!" \
history="PI=3.14159265358; ESD=1.0157675; BAND_Esun=1930.9; SZA=37.21120"

done
</source>

=== Atmospheric correction ===

See [[Atmospheric correction]]

== Post-Processing ==

Having beforehand satellite image data expressed in physical quantities (radiance or reflectance) is preferred to follow-up with digital image analysis techniques. A few common post-processing practices are Contrast-Enhancement, Pan-Sharpening and creating Pseudo- or True-Color Composites. Other well known enhancing manipualtions to support the analyses of satellite imagery, include deriving Vegetation Indices, transforming multi-spectral data based on [[Principal_Components_Analysis | PCA]] and Segmenting images.

=== Contrast Enhancement ===

''ToAdd''

=== Pan-Sharpening ===

[http://en.wikipedia.org/wiki/Pansharpened_image Pan-Sharpening] / [http://en.wikipedia.org/wiki/Image_fusion Fusion] is the process of merging high-resolution panchromatic and lower resolution multi-spectral imagery. [http://grass.osgeo.org/grass70/ GRASS 7] holds a dedicated pan-sharpening module, {{cmd|i.pansharpen}} which features three techniques for sharpening, namely the [http://wiki.awf.forst.uni-goettingen.de/wiki/index.php/Brovey_Transformation Brovey transformation], the classical IHS method and one that is based on [[Principal Components Analysis]] (PCA). Another algorithm deriving excellent detail and a realistic representation of original multispectral scene colors, is the High-Pass Filter Addition (HPFA) technique. It is available through the add-on {{AddonSrc|imagery|i.fusion.hpf|version=7}} (for GRASS 6, please refer to a bash shell script https://github.com/NikosAlexandris/i.fusion.hpf.sh which is, however, unmaintained).

One approach inside GRASS-GIS to get an acceptable color-balanced composite image after Pan-sharpening 11-bit IKONOS spectral bands comprises the following steps

<ul>
# rescale the 11-bit IKONOS spectral bands to 8-bit ranging in [0, 255] ({{cmd|r.rescale}})
# pan-sharpen with any of the featured methods (Brovey, IHS, PCA) ({{cmd|i.pansharpen}})
# color-balance by using the {{cmd|i.landsat.rgb}} module or manually adjusting the color tables of the bands of interest
# create a composite image by using the {{cmd|r.compose}} module
</ul>

==== Example Instructions for {{cmd|i.pansharpen}} ====

{{cmd|i.pansharpen}} works fine with 8-bit raster maps as an input. If the data to be processed are out of this range, that is out of <math>[0, 255]</math>, they can be rescaled to fit into this range by using GRASS' {{cmd|r.rescale}} module.

Given an IKONOS set of 11-bit spectral bands (Blue, Green, Red, NIR and Pan) ranging between <math>[0, 2047]</math>, and then querying for example the Blue band
<source lang="bash" enclose=none>r.info Blue_DNs -r</source>, would return

<source lang="bash">
min=0
max=2047
</source>

===== Rescaling single bands =====

Rescaling the Blue band to range between `[0, 255]`
<source lang="bash">
r.rescale in=Blue_DNs out=Blue_DNs_255 from=0,2047 to=0,255
</source>

The same step applies to both the rest of the multi-spectral bands and the Panchromatic band of interest. If working under Bash, To repeat the same command (given, it was the last on used in the GRASS' terminal), one can isntruct

<source lang="bash">
!!:gs/Blue/Green
</source>

This would substitute everywhere found on the last command the string "Blue" with the string "Green" and re-execute it. The same can be done for the rest of the bands, namely Red, NIR and Pan. Some attention is required to instruct substitution of the ''last'' used string.

===== Rescaling all bands at once =====

Of course, one can always use a ''for'' loop

<source lang="bash">
for DN in `g.mlist rast pat=*DNs`; do r.rescale in=${DN} out=${DN}_255 from=0,2047 to=0,255; done
</source>

===== Pan-Sharpening =====

As usual when working with GRASS, it is required to set the region of interest, i.e. <source lang="bash" enclose=none>g.region rast=Blue_DNs_255</source> to match the extent of the band(s) or else. The resolution itself is taken care in this particular case by the module and the resulting pan-sharpened raster maps will of the same high(er) resolution as the Panchromatic band.

An example command for an IHS-based Pan-Sharpening action might look like

<source lang="bash">
i.pansharpen pan=Pan_DNs_255 ms1=Blue_DNs_255 ms2=Green_DNs_255 ms3=Red_DNs_255 output=sharptest255 sharpen=ihs
</source>

===== Color Re-Balancnig =====

After the process completion, the module outputs

<source lang="bash">
...
The following pan-sharpened output maps have been generated:
sharptest255_red
sharptest255_green
sharptest255_blue

To visualize output, run: g.region -p rast=sharptest255.red
d.rgb r=sharptest255_red g=sharptest255_green b=sharptest255_blue

If desired, combine channels into a single RGB map with 'r.composite'.
</source>

Normally it should be enough to re-balance the colors after the pan-sharpening action by using for example the {{cmd|i.landsat.rgb}} module or manual adjustment of each of the three bands that would compose an RGB image.

<source lang="bash">
i.landsat.rgb r=sharptest255_red g=sharptest255_green b=sharptest255_blue -p
</source>

===== Automatising Sharpening & RGB Composition =====

An ''experimental'' bash script to automatise the sharpening/fusion and RGB composition process is demonstrated here. The script uses the Spectral Reflectance values (double precision values), gained from previous steps (see above), and produces Pan-sharpened images by applying all of the three methods that {{addon|i.pansharpen}} offers. In addition, it attempts to produce True-Color composite images, without and with re-balancing the color tables by using the {{cmd|i.landsat.rgb}} module.

Initial tests indicate that at least one of the three sharpening methods, combined with re-balancing, produces very clear and balanced True-Color composites. Note, there is no 100% confidence that it will produce nice looking True-Color composites. Some of the methods might simply work, others might deliver fancy-colored images. As usual, some manual action might be required to get the re-balancing to work as best as possible.

The script can be expanded in terms of using more inputs by utilising bash's positional parameters. However, care has to be taken to alter the instruction that concerns the conversion from double precision values to 8-bit integers.

<source lang="bash">
#!/bash/sh

# use in G7 only :-)

echo "Rescaling all images to 8-bit"
# convert to 8-bit first :-!
for Method_Input in \
"Pan ToAR" \
"Blue ToAR" \
"Green ToAR" \
"Red ToAR" \
"NIR ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo -e "\n" #echo $1 $2

# input BAND is...
eval BAND="${1}_${2}"

# set region
g.region rast=${BAND}

# integerise
r.mapcalc --v \
"${BAND}_integerised = round( 1000000 * ${BAND} )" # how many decimals?

# integerised BAND is...
eval BAND="${1}_${2}_integerised"

# get ${min} and ${max}
eval `r.info -r ${BAND}`

#
r.rescale in=${1}_${2}_integerised out=${1}_${2}_255 from=${min},${max} to=0,255

done ########################################################################
echo "Rescaling done!"

# pan-sharpen High Resolution imagery ---------------------------------------
# needs setting some naming conventions
for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# some echo
echo "Pan-Sharpening the <${2}> images based on the <${1}> method"

i.pansharpen \
sharpen=${1} \
pan=Pan_${2}_255 \
ms1=Blue_${2}_255 \
ms2=Green_${2}_255 \
ms3=Red_${2}_255 \
output=sharp_${1}_${2}_255

done ########################################################################

# compose BGRs --------------------------------------------------------------

# loop over Methods and Input Types, compose RGBs
for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# region
g.region rast=sharp_"${1}"_"${2}"_255_blue

# compose
r.composite --o \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue \
out=rgb_sharpened_"${1}"_"${2}"_255

done #######################################################################

# re-balance & re-compose colors --------------------------------------------

for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# re-balance
i.landsat.rgb -p \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue

# region -- Really necessary, again?
g.region rast=sharp_"${1}"_"${2}"_255_blue

# re-compose
r.composite --o \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue \
out=rgb_sharpened_"${1}"_"${2}"_255_rebalanced

done ########################################################################
</source>

==== Pan-Sharpening based on the HPFA technique ====

The technique, implemented via the GRASS add-on {{AddonSrc|imagery|i.fusion.hpf|version=7}}, as indicated above, involves a convolution using a High Pass Filter (HPF) on the high resolution data, then combining this with the lower resolution multispectral data.

''Source: "Optimizing the High-Pass Filter Addition Technique for Image Fusion", Ute G. Gangkofner, Pushkar S. Pradhan, and Derrold W. Holcomb (2008)''

The algorithm's steps are:

# Computing the ratio of the low (Multi-Spectral) to the high (Panchromatic) resolution
# High Pass Filtering the Panchromatic Image
# Resampling the Multi-Spectral image to the higher resolution
# Adding a weighted High-Pass-Filtered image to the upsampled Multi-Spectral image
# Optionally, matching the histogram of the Pan-Sharpened image to the one of the original Multi-Spectral image

=== Color Composites ===

''ToAdd''

== Vegetation Indices ==

''ToAdd''

== PCA ==

''ToAdd''

== IKONOS Image classification ==

See [[Image classification]]

== References / Sources ==

* [http://www.apollomapping.com/wp-content/user_uploads/2011/09/IKONOS_Esun_Calculations.pdf IKONOS Planetary Reflectance and Me an Solar Exoatmospheric Irradiance, by Martin Taylor]
* [http://web.unicen.edu.ar/crecic/docs/radrefl.pdf Ikonos DN Value Conversion to Planetary Reflectance Values, by David Fleming]
* [http://landsathandbook.gsfc.nasa.gov/data_prod/prog_sect11_3.html Landsat7 Science Data Users Handbook, Chapter 11, Section 3]
* [http://igett.delmar.edu/Resources/Remote%20Sensing%20Technology%20Training/Calculation-DN_to_Reflectance_Irish_20June08.pdf Calibrated Landsat Digital Number (DN) to Top of Atmosphere (TOA) Reflectance Conversion, by Richard Irish]
* from [http://landsat.usgs.gov/tools_access_all_faqs.php FAQs about the Landsat Missions]
** [http://landsat.usgs.gov/how_is_radiance_calculated.php How is radiance calculated?]
** [http://landsat.usgs.gov/at_sensor_reflectance_calculated.php How is at-sensor reflectance calculated?]

== See also ==

* GRASS-Wiki page about [http://grasswiki.osgeo.org/wiki/Image_processing Image Processing]

''ToAdd''

[[Category: Documentation]]
[[Category: FAQ]]
[[Category: IKONOS]]
[[Category: High Resolution]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Image processing]]

QuickBird

2016-08-14T08:10:09Z

⚠️NikosA: updated link to i.fusion.hpf

== Overview ==

QuickBird is a commercial earth observation satellite. Details about the sensor are provided at Digital Globe's [http://www.digitalglobe.com/sites/default/files/QuickBird-DS-QB-PROD.pdf QuickBird Data Sheet]. Wikipedia's article on {{wikipedia|QuickBird}} provides an informative overview as well.

=== Quick Facts ===

* Panchromatic (black and white) and Multi-Spectral imagery
* Multi-spectral bands are ordered as

band 1 = blue
2 = green
3 = red
4 = Near IR

* Scenes are 16.5km x 16.5km.

* Data is 11-bit integers stored in a 16-bit integer field. Thus band minimum = 0, band maximum = 2047. 

* Available pixel resolutions

<ul>
{|
! Product
! Black & White
! Multispectral
! Color & Pan-sharpened
|-
| Basic
| 61 cm to 72 cm as collected
| 2.44 m to 2.88 m as collected
| not available
|-
| Standard
| 60 cm or 70 cm
| 2.4 m or 2.8 m
| 60 cm or 70 cm
|-
| Orthorectified
| 60 cm or 70 cm
| 2.4 m or 2.8 m
| 60 cm or 70 cm
|}
</ul>

=== Notes ===

From the whitepaper [http://www.apollomapping.com/wp-content/user_uploads/2011/09/Radiance_Conversion_of_QuickBird_Data.pdf Radiance Conversion of QuickBird Data]:

<blockquote>QuickBird products are delivered to the customer as radiometrically corrected image pixels (qPixel,Band).
Radiometric correction includes a dark offset subtraction and a non-uniformity correction (detector-to-detector
relative gain). Corrected counts are specific to the QuickBird instrument and therefore QuickBird imagery MUST be
converted to spectral radiance before radiometric/spectral analysis or comparison with imagery from other sensors in
a radiometric/spectral manner.</blockquote>

== Availability (Sample Data) ==

* Search for commercial satellite image providers in the internet.
* Some sample QuickBird2 data are available at Digital Globe's collection of [http://www.digitalglobe.com/product-samples Imagery Product Samples]
* [http://www.landcover.org/ The Global Land Cover Facility (GLCF)] provides some [http://glcf.umd.edu/data/quickbird/ QuickBird] acqusitions for education or research purposes only.

== Modules overview ==

=== Outside of GRASS ===

* gdalinfo
* gdalwarp

=== Inside GRASS GIS ===

* {{cmd|r.in.gdal}}
* {{cmd|r.mapcalc}}
* {{cmd|r.colors}}
* {{cmd|i.landsat.rgb}} (GRASS 6.x) | {{cmd|i.colors.enhance|version=70}} (GRASS 7.x)
* {{cmd|i.pansharpen|version=70}} (GRASS 7.x)
* {{cmd|i.vi|version=70}} (GRASS 7.x)
* {{cmd|i.segment|version=70}} (GRASS 7.x)
* and more

== Pre-Processing ==

=== Overview ===

Typically, multispectral satellite data are converted into physical quantities such as ''Radiance'' or ''Reflectance'' before they are subjected in multispectral analysis techniques (image interpretation, band arithmetic, vegetation indices, matrix transformations, etc.). The latter can be differentiated in ''Top of Atmosphere Reflectance'' (ToAR) which does not account for atmospheric effects (absorption or scattering) and in ''Top of Canopy Reflectance'' (ToCR) which introduces a "correction" for atmospheric effects.

In order to derive Reflectance values, likewise as with remotely sensed data acquired by other sensors, QuickBird raw image digital numbers (DNs), also referred to as ''radiometrically corrected image pixels'' need to be converted to ''Top of Atmosphere Radiance'' values. Spectral Radiance values are then converted to Reflectance values based on various acquisition related parameters. Note, ''Spectal Radiance'' is the measure of the quantity of radiation that hits the sensor and typically expressed in <math>\frac{W}{m^2*sr*nm}</math>, that is ''watts'' per ''unit source area'', per ''unit solid angle'', and per ''unit wavelength''.

<ul>

* Converting QuickBird DNs to ''Top of Atmosphere'' Radiance, as described by Keith Krause (2005) in ''Radiometric Use of QuickBird Imagery, Technical Note'', can be done by using the following equation:

<math>L_{\lambda\text{Pixel, Band}} = \frac{K_{\text{Band}} * q_{\text{Pixel, Band}}}{\Delta\lambda_{\text{Band}}}</math>

 where:

<ul>
* <math>L_{\lambda\text{Pixel,Band}}</math>: Top-of-Atmosphere Spectral Radiance image pixels [W-m-2-sr-1-μm-1]
* <math>K_{\text{Band}}</math>: the absolute radiometric calibration factor [W-m-2-sr-1-count-1] for a given band
* <math>q_{\text{Pixel,Band}}</math>: radiometrically corrected image pixels [counts or Digital Numbers]
* <math>\Delta_{\lambda_{\text{Band}}}</math>: the effective bandwidth [μm] for a given band
</ul>

* Converting to Top of Atmosphere Reflectance, also referred to as Planetary Reflectance, can be done by using the following equation:

<math>\rho_p = \frac{\pi * L\lambda * d^2}{ESUN\lambda * cos(\Theta_S)}</math>

 where:

<ul>
* <math>\rho</math> - Unitless Planetary Reflectance
* <math>\pi</math> - mathematical constant (3.14159265358)
* <math>L\lambda</math> spectral Radiance at the sensor's aperture, from equation... '''''ToADD'''''
* <math>d</math> - Earth-Sun distance in astronomical units, interpolated values
* <math>Esun</math> - Mean solar exoatmospheric irradiance(s) (W/m2/μm), interpolated values
* <math>cos(\theta_s)</math> - {{wikipedia||Solar_zenith_angle}} from the image acquisition's metadata
</ul>

</ul>

=== Importing data ===

==== File Formats & Metadata ====

* GeoTIFF, NITF 2.1, NITF 2.0
* Image bits / pixel: 8 or 16 bits

Setting up a Location in GRASS GIS requires to know the Spatial Reference System in which the data of interest are projected. Retrieving the projection parameters of QuickBird imagery, can be done by using the `gdalinfo` program.

''ToAdd''
<ul>
* Various ways of Location creation based on georeferenced data
</ul>

==== Importing ====

Once inside an appropriate Location, importing the data is done with the {{cmd|r.in.gdal}} module.

For example, GeoTIFF files/bands, can be imported in one go by looping {{cmd|r.in.gdal}} over all of them

<source lang="bash">
for TIF in `echo *.tif`; do r.in.gdal in=${TIF} out=${TIF%%.*}; done
</source>

=== Cleanup ===
Rename the NIR band
{{cmd|g.rename}} QBird_Multichrom.4,QBird_Multichrom.NIR

It is very likely that the Red and Blue bands are swapped (in older datasets), so fix that by renaming them. (see [http://osdir.com/ml/gis.gdal.devel/2004-03/msg00025.html])
g.rename QBird_Multichrom.blue,QBird_Multichrom.realred
g.rename QBird_Multichrom.red,QBird_Multichrom.blue
g.rename QBird_Multichrom.realred,QBird_Multichrom.red

No data is set to 0, so convert that to NULL with {{cmd|r.null}}:
for BAND in red green blue NIR ; do
r.null QBird_Multichrom.$BAND setnull=0
echo $BAND:
r.info -r QBird_Multichrom.$BAND
echo
done

Set color tables appropriately for 11-bit data:
for BAND in red green blue NIR ; do
{{cmd|r.colors}} QBird_Multichrom.$BAND color=rules << EOF
0 black
2047 white
EOF
done

It is not necessarily needed, but if you wish to rescale to 8-bit data you can use {{cmd|r.rescale}} as follows:
for BAND in red green blue NIR ; do
r.rescale in=QBird_Multichrom.$BAND from=0,2047 \
out=QBird_Multichrom255.$BAND to=0,255
done

Display the 3-band RGB image:
{{cmd|d.rgb}} r=QBird_Multichrom.red g=QBird_Multichrom.green b=QBird_Multichrom.blue

If the image is excessively dark you might try using the {{cmd|i.landsat.rgb}} module to auto-balance the colors and redraw:
i.landsat.rgb r=QBird_Multichrom.red g=QBird_Multichrom.green b=QBird_Multichrom.blue
d.redraw

=== Deriving Physical Quantities ===

Converting Digital Numbers to Radiance/Reflectance requires knowledge about the sensor's specific spectral band parameters. These are, as extracted from the document ''Radiometric Use of QuickBird Imagery, Technical Note. 2005-11-07, by Keith Krause.''

<source lang=bash>
# Band-Averaged Solar Spectral Irradiance [W/sq.m./μm]
Pan_Esun=1381.79
Blue_Esun=1924.59
Green_Esun=1843.08
Red_Esun=1574.77
NIR_Esun=1113.71

# Revised K Factors & Effective Bandwidths

# 1st column: K Conversion Factors (& Pan TDI Level) for 16-Bit products [W/sq.m./sr/count]
# 2nd column: Effective Bandwidth [μm] per Spectral Band

K16_Pan10=0.08381880 ; Pan10_Width=0.398
K16_Pan13=0.06447600 ; Pan13_Width=0.398
K16_Pan18=0.04656600 ; Pan18_Width=0.398
K16_Pan24=0.03494440 ; Pan24_Width=0.398
K16_Pan32=0.02618840 ; Pan32_Width=0.398
K16_Blue=0.01604120 ; Blue_Width=0.068
K16_Green=0.01438470 ; Green_Width=0.099
K16_Red=0.01267350 ; Red_Width=0.071
K16_NIR=0.01542420 ; NIR_Width=0.114

# 1st column: k′ Conversion Factors for 8-Bit products
# 2nd column being identical (as above) not repeated!

K8_Pan10=1.02681367
K8_Pan13=1.02848939
K8_Pan18=1.02794702
K8_Pan24=1.02989685
K8_Pan32=1.02739898
K8_Blue=1.12097834
K8_Green=1.37652632
K8_Red=1.30924587
K8_NIR=0.98368622
</source>

Attention for 32-bit floating point calculations is requried. As stated, in the same document, 

''NOTE: conversion equations are to be performed on all pixels in a given band of a QuickBird image and should use 32-bit floating point calculations.''

==== Spectral Radiance ====

Converting a Blue Band (Digital Numbers) in to Spectral Radiance in the ''correct'' units to be further used for the conversion in to unitless Reflectance:

<source lang="bash">
# set the region
g.region rast=Blue_DNs -p

# convert DNs to spectral Radiance values
r.mapcalc "Blue_Radiance = '''ToAdd''' "
</source>

==== Planetary Reflectance ====

The equation to derive Reflectance values incorporates in addition:

<ul>
* The mathematical constant {{wikipedia|Pi}} <source lang="bash" enclose=none>PI=3.14159265358</source>.
* The Earth-Sun distance in astronomical units which depends on the acquisition's day of year (DOY -- also referred to as Julian day, {{wikipedia|Ordinal_date}}) and can be retrieved from the following spreadsheet <http://landsathandbook.gsfc.nasa.gov/excel_docs/d.xls>.
* The mean solar exoatmospheric irradiance in <math>\frac{W}{m^2*\mu m}</math>. See 3rd column of (interplated) values given above.
* The cosine of the ''Solar Zenith Angle'' (SZA) at the time of the acquisition. The SZA can be calculated from its complementary ''Solar Elevation Angle'' (SEA) given in the image acquisition's metadata.
</ul>

In the following example we accept as the acquisition's <source lang="bash" enclose=none>DOY=...</source> and <source lang="bash" enclose=none>SEA=...</source>. Hence, we get the Earth-Sun distance <source lang="bash" enclose=none>ESD=...</source> and <source lang="bash" enclose=none>SZA = ... deg</source>.

Converting the in-Blue spectral band at-sensor Radiance in to Planerary Reflectance:

<source lang="bash" enclose=none>PI=3.14159265358; ESD=...; BAND_Esun=...; SZA=...</source>
{{cmd|r.mapcalc}} <source lang="bash" enclose=none>"Blue_Reflectance = ( ${PI} * Blue_Radiance * ${ESD}^2 ) / ( ${BAND_Esun} * cos(${SZA}) )"</source>

==== Automatising Conversions ====

Scripting the conversion process avoids repeating the same steps for each band separately.

===== in Python =====

A custom python script, performing the operations of interest, might be like [https://github.com/NikosAlexandris/i.quickbird.toar i.quickbird.toar (for GRASS 7.x)]

===== in bash =====

Am example bash shell script, might look like the one that follows. '''Note,''' however, constants, band parameters and acquisition related metadata are hard-coded! Reviewing the code and altering appropriately is required, i.e. checking for the parameters <code>ESD</code>, <code>SEA</code>, <code>PanTDI</code>, <code>K_BAND</code>.

<source lang="bash">
#!bin/bash

# Pi, first 11 decimals
PI=3.14159265358

# HardCoded MetaData!

# Acquisition's Day of Year and estimated Earth-Sun Distance
DOY=274; ESD=1.001190

# Sun Zenith Angle based on the acquisition's Sun Elevation Angle
SEA=67.8; SZA=$(echo "90 - ${SEA}" | bc )

# which Pan TDI level?
PanTDI="10" # evaluate below...

# Spectral Irradiance [W-m-2-μm-1]

Pan_Esun=1381.79
Blue_Esun=1924.59
Green_Esun=1843.08
Red_Esun=1574.77
NIR_Esun=1113.71

# 1st column: K Conversion Factors for 16-Bit products
# 2nd column: Effective Bandwidth [μm] per Spectral Band

K16_Pan10=0.08381880; Pan10_Width=0.398
K16_Pan13=0.06447600; Pan13_Width=0.398
K16_Pan18=0.04656600; Pan18_Width=0.398
K16_Pan24=0.03494440; Pan24_Width=0.398
K16_Pan32=0.02618840; Pan32_Width=0.398
K16_Blue=0.01604120 ; Blue_Width=0.068
K16_Green=0.01438470; Green_Width=0.099
K16_Red=0.01267350 ; Red_Width=0.071
K16_NIR=0.01542420 ; NIR_Width=0.114

# 1st column: k′ Conversion Factors for 8-Bit products
# Effective Bandwidths [μm] per Spectral Band: as above

K8_Pan10=1.02681367
K8_Pan13=1.02848939
K8_Pan18=1.02794702
K8_Pan24=1.02989685
K8_Pan32=1.02739898
K8_Blue=1.12097834
K8_Green=1.37652632
K8_Red=1.30924587
K8_NIR=0.98368622

# Bands
Spectral_Bands="Pan Blue Green Red NIR"

# loop over all bands
for BAND in ${Spectral_Bands}; do

# which Pan TDI level?
if [[ ${BAND} == Pan ]]
then
eval Pan="Pan${PanTDI}"; echo "Note, processing for ${Pan}"
eval BAND="${Pan}"
fi

# some echo...
echo "Processing the ${BAND} spectral band"

# set band parameters as variables -- using K16 for 16-bit data!
eval K_BAND="K16_${BAND}"
eval BAND_Width="${BAND}_Width"
echo "Band Parameters set to K=${!K_BAND}, Bandwidth=${!BAND_Width}"

if [[ ${BAND} == Pan* ]]
then
BAND="Pan" ; echo "Processing the ${BAND} spectral band"
fi

# set region
g.region rast=${BAND}_DNs #-pg
echo "Region matching the ${BAND} spectral band"

# conversion to Radiance based on (1) -- attention: 32-bit calculations required
r.mapcalc "${BAND}_Radiance = ( double(${!K_BAND}) * ${BAND}_DNs ) / ${!BAND_Width}"
r.info -r "${BAND}_Radiance"

# add info
r.support map=${BAND}_Radiance \
title="" \
units="W / sq.m. / μm / ster" \
description="Top-of-Atmosphere `echo ${BAND}` band spectral Radiance [W/m^2/sr/μm]" \
source1='"Radiometric Use of QuickBird Imagery, Technical Note (2005)," by Keith Krause, Digital Globe'

eval BAND_Esun="${BAND}_Esun"; echo "Using Esun=${!BAND_Esun}"

# calculate ToAR -- ${BAND}_Radiance is already 32-bit -- see above!
r.mapcalc "${BAND}_ToAR = \
( ${PI} * ${BAND}_Radiance * ${ESD}^2 ) / ( ${!BAND_Esun} * cos(${SZA}) )"
r.info -r ${BAND}_ToAR

# add some metadata
r.support map=${BAND}_ToAR \
title="echo ${BAND} band (Top of Atmosphere Reflectance)" \
units="Unitless planetary reflectance" \
description="Top of Atmosphere `echo ${BAND}` band spectral Reflectance (unitless)" \
source1='"Radiometric Use of QuickBird Imagery, Technical Note (2005)," by Keith Krause, Digital Globe' \
source2="Digital Globe" \
history="PI=3.14159265358; K=${!K_BAND}; Bandwidth=${!BAND_Width}; ESD=${ESD}; Esun=${!BAND_Esun}; SZA=${SZA}"

done
</source>

=== Atmospheric correction ===

See [[Atmospheric correction]]

== Post-Processing ==

Having beforehand satellite image data expressed in physical quantities (radiance or reflectance) is preferred to follow-up with digital image analysis techniques.

== Pan Sharpening ==

In GRASS ver. 6.x, one can use the {{cmd|i.fusion.brovey}} module.

<source lang="bash">
i.fusion.brovey -q ms1=qbird_green ms2=qbird_nir ms3=qbird_red pan=qbird_pan outputprefix=brov
</source>

[[Category: Documentation]]
[[Category: Geodata]]
[[Category: High Resolution]]
[[Category: QuikBird]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Image processing]]

=== HPFA based sharpening ===

The following screenshots exemplify the High Pass Filtering Addition fusion technique applied in a fragment of the QuickBird acquisition "04APR05050541-X2AS_R1C1-000000186011_01_P001-Sri_Lanka-Kokilai_Lagoon" which is publicly available via the GLCF. An implementation of this technique for GRASS-GIS is available as a grass-addon {{AddonSrc|imagery|i.fusion.hpf|version=7}}.

{|
|[[File:Pan_04APR05050541-M2AS-000000186011_01_P001.jpg|thumb|400px|border|center|Fragment from the high resolution (0.7m) panchromatic image]]
|[[File:RGB_04APR05050541-M2AS-000000186011_01_P001.jpg|thumb|400px|border|center|Fragment from the low resolution (2.8m) multi-spectral RGB composite image]]
|}

{|
|[[File:RGB_HPF_Sharpened_Default_Parameters.jpg|thumb|400px|border|center|Fragment from the HPFA Pan-Sharpened RGB composite image. Sharpening performed with default parameters.]]
|[[File:RGB_HPF_Sharpened_Center_Low_Modulator_Min.jpg|thumb|400px|border|center|Fragment from another HPFA Pan-Sharpened RGB composite image. Sharpening performed with ''center=low'' and ''modulator=min'' parameters.]]
|}

WorldView

2016-08-14T05:11:28Z

⚠️NikosA: /* Pan-Sharpening */

= This page is under construction =

= Overview =

WorldView-1, WorldView-2 and WorldView-3 are commercial earth observation satellites. Details about the sensors are provided at Digital Globe's collection of [http://www.digitalglobe.com/resources/satellite-information Spacecraft Data Sheets].

It is recommended -- actually it is a necessity! -- to have a look at the documentation explaining [http://www.digitalglobe.com/sites/default/files/Imagery_Support_Data_Documentation%20%281%29.pdf Digital Globe's products metadata]. Note, all of DigitalGlobe’s satellites collect data using an 11 bit dynamic range. For technical reasons, products are delivered as either 16-bit or 8-bit data.

== Availability (Sample Data) ==

* Search for commercial satellite image providers in the internet.

== Modules overview ==

=== Outside of GRASS ===

Satellite imagery can be managed and, at some extent, pre-processed with various [http://www.gdal.org/gdal_utilities.html GDAL utilities].

* [http://www.gdal.org/gdalmanage.html gdalmanage]
* [http://www.gdal.org/gdalinfo.html gdalinfo]
* [http://www.gdal.org/gdal_translate.html gdal_translate]
* [http://www.gdal.org/gdalbuildvrt.html gdalbuildvrt]
* [http://www.gdal.org/gdalwarp.html gdalwarp]

=== Inside GRASS GIS ===

GRASS GIS features a complete set of modules and various add-ons enabling pre- and post-processing of WorldView satellite imagery. The following lists offer an overview of related modules and add-ons.

* {{cmd|r.in.gdal}}
* {{cmd|r.mapcalc}}
* {{cmd|r.colors}}
* {{cmd|i.landsat.rgb}}
* {{AddonSrc|imagery|i.fusion.hpf|version=7}}
* {{cmd|i.pansharpen}}
* [https://github.com/NikosAlexandris/i.worldview.toar i.worldview.toar]
* {{cmd|i.vi}}
* {{cmd|i.segment}}
* ...''more to add''

== Pre-Processing ==

=== Overview ===

Typically, multispectral satellite data are converted into physical quantities such as ''Radiance'' or ''Reflectance'' before they are subjected in multispectral analysis techniques (image interpretation, band arithmetic, vegetation indices, matrix transformations, etc.). The latter can be differentiated in ''Top of Atmosphere Reflectance'' (ToAR) which does not account for atmospheric effects (absorption or scattering) and in ''Top of Canopy Reflectance'' (ToCR) which introduces a "correction" for atmospheric effects.

In order to derive Reflectance values, likewise as with remotely sensed data acquired by other sensors, WorldView raw image digital numbers (DNs) need to be converted to ''at-sensor spectral Radiance'' values. At-sensor spectral Radiance values are an important input for the equation to derive Reflectance values. Note, ''Spectal Radiance'' is the measure of the quantity of radiation that hits the sensor and typically expressed in <math>\frac{W}{m^2*sr*nm}</math>, that is ''watts'' per ''unit source area'', per ''unit solid angle'', and per ''unit wavelength''.

<ul>

* Converting WorldView DNs to ''Top of Atmosphere'' Radiance, as described in [http://www.digitalglobe.com/downloads/Radiometric_Use_of_WorldView-2_Imagery.pdf Radiometric Use of WorldView-2 Imagery, Technical Note (2010)] by Todd Updike and Chris Comp, can be done by using the following equation:

<math>L_{\lambda\text{Pixel, Band}} = \frac{K_{\text{Band}} * q_{\text{Pixel, Band}}}{\Delta\lambda_{\text{Band}}}</math>

 where:

<ul>
* <math>L_{\lambda\text{Pixel,Band}}</math>: Top-of-Atmosphere Spectral Radiance image pixels [W-m-2-sr-1-μm-1]
* <math>K_{\text{Band}}</math>: the absolute radiometric calibration factor [W-m-2-sr-1-count-1] for a given band
* <math>q_{\text{Pixel,Band}}</math>: radiometrically corrected image pixels [counts or Digital Numbers]
* <math>\Delta_{\lambda_{\text{Band}}}</math>: the effective bandwidth [μm] for a given band
</ul>

* Converting to Top of Atmosphere Reflectance, also referred to as Planetary Reflectance, can be done by using the following equation:

<math>\rho_p = \frac{\pi * L\lambda * d^2}{ESUN\lambda * cos(\Theta_S)}</math>

 where:

<ul>
* <math>\rho</math> - Unitless Planetary Reflectance
* <math>\pi</math> - mathematical constant (3.14159265358)
* <math>L\lambda</math> spectral Radiance at the sensor's aperture, from equation... '''''ToADD'''''
* <math>d</math> - Earth-Sun distance in astronomical units, interpolated values
* <math>Esun</math> - Mean solar exoatmospheric irradiance(s) (W/m2/μm), interpolated values
* <math>cos(\theta_s)</math> - {{wikipedia||Solar_zenith_angle}} from the image acquisition's metadata
</ul>

</ul>

=== Importing data ===

==== File Formats & Metadata ====

''ToAdd''

==== Importing ====

''ToAdd''
<ul>
* Location creation based on georeferenced data
</ul>

Once inside a Location that is defined by the spatial reference system in which the bands of interest are projected, they can be imported with the {{cmd|r.in.gdal}} module.

For example, GeoTIFF files can be imported by looping {{cmd|r.in.gdal}} over all of them

<source lang="bash">
for TIF in `echo *.tif`; do r.in.gdal in=${TIF} out=${TIF%%.*}; done
</source>

To keep the raw material untouched, we create another Mapset inside the same Location and copy over the bands giving, ''optionally'' at the same time, a new name for each.

''ToAdd''

=== Deriving Physical Quantities ===

Conversions implemented in the GRASS module [https://github.com/NikosAlexandris/i.worldview.toar i.worldview.toar]

''ToCleanUp''

Converting Digital Numbers to Radiance/Reflectance requires knowledge about the sensor's specific spectral band parameters. In the following listed parameters (see below), the ''Effective Bandwidth''s and the ''Band-Averaged Solar Spectral Irradiance''s are extracted from the document [http://www.digitalglobe.com/downloads/Radiometric_Use_of_WorldView-2_Imagery.pdf Radiometric Use of WorldView-2 Imagery, Technical Note (2010)] by Todd Updike and Chris Comp. The Absolute Calibration Factors are extracted from a WorldView-2 product, specifically from an image metadata file (extension .IMD).

'''Note,''' however, as stated in the above document,

The absolute radiometric calibration factor is dependent on the specific band,
as well as the TDI exposure level, line rate, pixel aggregation, and bit depth
of the product. Based on these parameters, the appropriate value is provided in
the .IMD file. For this reason, care should be taken not to mix absolute radiometric
calibration factors between products that might have different collection conditions.

<source lang=bash>
Pan_CalCoef=0.05678345 ; Pan_Width=0.2846000 ; Pan_Esun=1580.8140
Coastal_CalCoef=0.009295654 ; Coastal_CalCoef=0.04730000 ; Coastal_Esun=1758.2229
Blue_CalCoef=0.01260825 ; Blue_Width=0.05430000 ; Blue_Esun=1974.2416
Green_CalCoef=0.009713071 ; Green_Width=0.06300000 ; Green_Esun=1856.4104
Yellow_CalCoef=0.005829815 ; Yellow_Width=0.03740000 ; Yellow_Esun=1738.4791
Red_CalCoef=0.01103623 ; Red_Width=0.05740000 ; Red_Esun=1559.4555
RedEdge_CalCoef=0.005188136 ; RedEdge_Width=0.03930000 ; Red_Edge_Esun=1342.0695
NIR1_CalCoef=0.01224380 ; NIR1_Width=0.09890000 ; NIR_Esun=1069.7302
NIR2_CalCoef=0.009042234 ; NIR2_Width=0.09960000 ; NIR2_Esun=861.2866
</source>

The following examples exemplify the conversion of raw Blue band digital numbers into Radiance and Reflectance.

==== Spectral Radiance ====

Converting a Blue Band (Digital Numbers) in to Spectral at-sensor Radiance in the ''correct'' units to be further used for the conversion in to unitless Reflectance:

<source lang="bash">
# set the region
g.region rast=Blue_DNs -p

# convert DNs to spectral Radiance values
r.mapcalc "Blue_Radiance = ( (10000 * WorldView_Blue_DNs) / (... * ...) )"
</source>

==== Planetary Reflectance ====

The equation to derive Reflectance values incorporates in addition:

<ul>
* The mathematical constant {{wikipedia|Pi}} <source lang="bash" enclose=none>PI=3.14159265358</source>.
* The Earth-Sun distance in astronomical units which depends on the acquisition's day of year (DOY -- also referred to as Julian day, {{wikipedia|Ordinal_date}}) and can be retrieved from the following spreadsheet <http://landsathandbook.gsfc.nasa.gov/excel_docs/d.xls>.
* The mean solar exoatmospheric irradiance in <math>\frac{W}{m^2*\mu m}</math>. See 3rd column of (interplated) values given above.
* The cosine of the ''Solar Zenith Angle'' (SZA) at the time of the acquisition. The SZA can be calculated from its complementary ''Solar Elevation Angle'' (SEA) given in the image acquisition's metadata.
</ul>

In the following example we accept as the acquisition's <source lang="bash" enclose=none>DOY=...</source> and <source lang="bash" enclose=none>SEA=...</source>. Hence, we get the Earth-Sun distance <source lang="bash" enclose=none>ESD=...</source> and <source lang="bash" enclose=none>SZA = ... deg</source>.

In any case, '''note''' that Top-of-Atmosphere Reflectance does not account for topographic, atmospheric, or BRDF differences.

Converting the in-Blue spectral band at-sensor Radiance in to Planerary Reflectance:

<source lang="bash" enclose=none>PI=...; ESD=...; BAND_Esun=...; SZA=...</source>
{{cmd|r.mapcalc}} <source lang="bash" enclose=none>"Blue_Reflectance = ( ${PI} * Blue_Radiance * ${ESD}^2 ) / ( ${BAND_Esun} * cos(${SZA}) )"</source>

==== Automatising Conversions ====

The conversion process can be scripted to avoid repeating the same steps for each band separately.

===== Python =====

A custom python script, performing the operations of interest, might be like [https://github.com/NikosAlexandris/i.worldview.toar i.worldview.toar (for GRASS 7.x)]

===== Bash =====
In bash, such a script might be as the following example. '''Note,''' however, in this example script constants, band parameters and acquisition related metadata are hard-coded!

<source lang="bash">
#!/bin/bash

# π, first 11 decimals
PI=3.14159265358

# HardCoded MetaData!

# Acquisition's Day of Year and estimated Earth-Sun Distance
DOY=100; ESD=1.00184

# Sun Zenith Angle based on the acquisition's Sun Elevation Angle
SEA=53.8; SZA=$(echo "90 - ${SEA}" | bc )

# some echo
echo "Acquisition-specific parameters"
echo "Day of Year: ${DOY}, Earth-Sun Distance: ${ESD}, Sun Zenith Angle: ${SZA}"
echo -e "\n"

# 1st column: Absolute Calibration Factors
# 2nd column: Spectral Band Effective Bandwidth, Δλ
# 3rd column: Band-Averaged Solar Spectral Irradiance [W/sq.m./micro-m]

K_Pan=0.05678345 ; Pan_Width=0.2846000 ; Pan_Esun=1580.8140
K_Coastal=0.009295654 ; Coastal_Width=0.04730000 ; Coastal_Esun=1758.2229
K_Blue=0.01260825 ; Blue_Width=0.05430000 ; Blue_Esun=1974.2416
K_Green=0.009713071 ; Green_Width=0.06300000 ; Green_Esun=1856.4104
K_Yellow=0.005829815 ; Yellow_Width=0.03740000 ; Yellow_Esun=1738.4791
K_Red=0.01103623 ; Red_Width=0.05740000 ; Red_Esun=1559.4555
K_RedEdge=0.005188136 ; RedEdge_Width=0.03930000 ; RedEdge_Esun=1342.0695
K_NIR1=0.01224380 ; NIR1_Width=0.09890000 ; NIR1_Esun=1069.7302
K_NIR2=0.009042234 ; NIR2_Width=0.09960000 ; NIR2_Esun=861.2866

# Bands
Spectral_Bands="Pan Coastal Blue Green Yellow Red RedEdge NIR1 NIR2"

# loop over all bands
for BAND in ${Spectral_Bands}; do

# some echo
echo -e "Processing the \"${BAND}\" spectral band"

# set band parameters as variables
eval BAND_Width="${BAND}_Width"
echo "* Band-specific radiometric parameters set to K=${!K_BAND}, Bandwidth=${!BAND_Width}"

# set region
g.region rast=${BAND}_DNs

# conversion to Radiance based on (1) -- attention: 32-bit calculations required
echo "* Converting ${BAND} Digital Numebrs to Radiance..." | tr -d "\n"
r.mapcalc "${BAND}_Radiance = ( double(${!K_BAND}) * ${BAND}_DNs ) / ${!BAND_Width}"

# report range
echo "* Reporting range of the ${BAND} spectral radiance: " | tr -d "\n"
# r.info -r "${BAND}_Radiance"
r.info -r "${BAND}_Radiance" | tr "\n" "," | cut -d"," -f1,2 | sed 's/,/,\ /'

# add info
r.support map=${BAND}_Radiance \
title="" \
units="W / sq.m. / μm / ster" \
description="Top-of-Atmosphere `echo ${BAND}` band spectral Radiance [W/m^2/sr/μm]" \
source1='"Radiometric Use of WorldView-2 Imagery, Technical Note (2010)", by Todd Updike & Chris Comp'

# Esun
eval BAND_Esun="${BAND}_Esun"

# some echo
echo "* Parameters required for the conversion: Esun= ${!BAND_Esun}, ESD= ${ESD}, ${SZA}"

# calculate ToAR
echo "* Converting ${BAND}_Radiance to Top of Atmosphere Reflectance..." | tr -d "\n"
r.mapcalc "${BAND}_ToAR = \
( ${PI} * ${BAND}_Radiance * ${ESD}^2 ) / ( ${!BAND_Esun} * cos(${SZA}) )"

# report range
echo "* Reporting range of the ${BAND} spectral radiance: " | tr -d "\n"
r.info -r ${BAND}_ToAR | tr "\n" "," | cut -d"," -f1,2 | sed 's/,/,\ /'

# add some metadata
r.support map=${BAND}_ToAR \
title="echo ${BAND} band (Top of Atmosphere Reflectance)" \
units="Unitless planetary reflectance" \
description="Top of Atmosphere `echo ${BAND}` band spectral Reflectance (unitless)" \
source1='"Radiometric Use of WorldView-2 Imagery, Technical Note (2010)", by Todd Updike & Chris Comp, Digital Globe' \
source2="Digital Globe" \
history="PI=3.14159265358; K=${!K_BAND}; Bandwidth=${!BAND_Width}; ESD=${ESD}; Esun=${!BAND_Esun}; SZA=${SZA}"

echo -e "\n"

done
</source>

=== Atmospheric correction ===

See [[Atmospheric correction]]

== Post-Processing ==

Having beforehand satellite image data expressed in physical quantities (radiance or reflectance) is preferred to follow-up with digital image analysis techniques. A few common post-processing practices are Contrast-Enhancement, Pan-Sharpening and creating Pseudo- or True-Color Composites. Other well known enhancing manipualtions to support the analyses of satellite imagery, include deriving Vegetation Indices, transforming multi-spectral data based on [[Principal_Components_Analysis | PCA]] and Segmenting images.

=== Contrast Enhancement ===

''ToAdd''

=== Pan-Sharpening ===

[http://en.wikipedia.org/wiki/Pansharpened_image Pan-Sharpening] / [http://en.wikipedia.org/wiki/Image_fusion Fusion] is the process of merging high-resolution panchromatic and lower resolution multi-spectral imagery. [http://grass.osgeo.org/grass70/ GRASS 7] holds a dedicated pan-sharpening module, {{cmd|i.pansharpen}} which features three techniques for sharpening, namely the [http://wiki.awf.forst.uni-goettingen.de/wiki/index.php/Brovey_Transformation Brovey transformation], the classical IHS method and one that is based on [[Principal Components Analysis]] (PCA).
Another algorithm deriving excellent detail and a realistic representation of original multispectral scene colors, is the High-Pass Filter Addition (HPFA) technique. It is implemented via the {{AddonSrc|imagery|i.fusion.hpf|version=7}} add-on (for GRASS 6, please refer to a bash shell script https://github.com/NikosAlexandris/i.fusion.hpf.sh which is, however, unmaintained).

One approach inside GRASS-GIS to get an acceptable color-balanced composite image after Pan-sharpening 11-bit IKONOS spectral bands comprises the following steps

<ul>
# rescale the 11-bit WorldView spectral bands to 8-bit ranging in [0, 255] ({{cmd|r.rescale}})
# pan-sharpen with any of the featured methods (Brovey, IHS, PCA) ({{cmd|i.pansharpen}})
# color-balance by using the {{cmd|i.landsat.rgb}} module or manually adjusting the color tables of the bands of interest
# create a composite image by using the {{cmd|r.compose}} module
</ul>

''ToAdd''

==== Automatising Sharpening & RGB Composition ====

An ''experimental'' bash script to automatise the sharpening/fusion and RGB composition process is demonstrated here. The script uses the Spectral Reflectance values (double precision values), gained from previous steps (see above), and produces Pan-sharpened images by applying all of the three methods that {{cmd|i.pansharpen}} offers. In addition, it attempts to produce True-Color composite images, without and with re-balancing the color tables by using the {{cmd|i.landsat.rgb}} module.

Initial tests indicate that at least one of the three sharpening methods, combined with re-balancing, produces very clear and balanced True-Color composites. Note, there is no 100% confidence that it will produce nice looking True-Color composites. Some of the methods might simply work, others might deliver fancy-colored images. As usual, some manual action might be required to get the re-balancing to work as best as possible.

The script can be expanded in terms of using more inputs by utilising bash's positional parameters. However, care has to be taken to alter the instruction that concerns the conversion from double precision values to 8-bit integers.

<source lang="bash">
...
</source>

=== Color Composites ===

''ToAdd''

== Vegetation Indices ==

''ToAdd''

== PCA ==

''ToAdd''

== WordView Image classification ==

See [[Image classification]]

== References / Sources ==

* ...
* [http://landsathandbook.gsfc.nasa.gov/data_prod/prog_sect11_3.html Landsat7 Science Data Users Handbook, Chapter 11, Section 3]
* [http://igett.delmar.edu/Resources/Remote%20Sensing%20Technology%20Training/Calculation-DN_to_Reflectance_Irish_20June08.pdf Calibrated Landsat Digital Number (DN) to Top of Atmosphere (TOA) Reflectance Conversion, by Richard Irish]
* from [http://landsat.usgs.gov/tools_access_all_faqs.php FAQs about the Landsat Missions]
** [http://landsat.usgs.gov/how_is_radiance_calculated.php How is radiance calculated?]
** [http://landsat.usgs.gov/at_sensor_reflectance_calculated.php How is at-sensor reflectance calculated?]

== See also ==

* GRASS-Wiki page about [http://grasswiki.osgeo.org/wiki/Image_processing Image Processing]

''ToAdd''

[[Category: Documentation]]
[[Category: FAQ]]
[[Category: WorldView]]
[[Category: High Resolution]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Image processing]]

WorldView

2016-08-14T05:10:22Z

⚠️NikosA: added link to https://github.com/NikosAlexandris/i.worldview.toar

= This page is under construction =

= Overview =

WorldView-1, WorldView-2 and WorldView-3 are commercial earth observation satellites. Details about the sensors are provided at Digital Globe's collection of [http://www.digitalglobe.com/resources/satellite-information Spacecraft Data Sheets].

It is recommended -- actually it is a necessity! -- to have a look at the documentation explaining [http://www.digitalglobe.com/sites/default/files/Imagery_Support_Data_Documentation%20%281%29.pdf Digital Globe's products metadata]. Note, all of DigitalGlobe’s satellites collect data using an 11 bit dynamic range. For technical reasons, products are delivered as either 16-bit or 8-bit data.

== Availability (Sample Data) ==

* Search for commercial satellite image providers in the internet.

== Modules overview ==

=== Outside of GRASS ===

Satellite imagery can be managed and, at some extent, pre-processed with various [http://www.gdal.org/gdal_utilities.html GDAL utilities].

* [http://www.gdal.org/gdalmanage.html gdalmanage]
* [http://www.gdal.org/gdalinfo.html gdalinfo]
* [http://www.gdal.org/gdal_translate.html gdal_translate]
* [http://www.gdal.org/gdalbuildvrt.html gdalbuildvrt]
* [http://www.gdal.org/gdalwarp.html gdalwarp]

=== Inside GRASS GIS ===

GRASS GIS features a complete set of modules and various add-ons enabling pre- and post-processing of WorldView satellite imagery. The following lists offer an overview of related modules and add-ons.

* {{cmd|r.in.gdal}}
* {{cmd|r.mapcalc}}
* {{cmd|r.colors}}
* {{cmd|i.landsat.rgb}}
* {{AddonSrc|imagery|i.fusion.hpf|version=7}}
* {{cmd|i.pansharpen}}
* [https://github.com/NikosAlexandris/i.worldview.toar i.worldview.toar]
* {{cmd|i.vi}}
* {{cmd|i.segment}}
* ...''more to add''

== Pre-Processing ==

=== Overview ===

Typically, multispectral satellite data are converted into physical quantities such as ''Radiance'' or ''Reflectance'' before they are subjected in multispectral analysis techniques (image interpretation, band arithmetic, vegetation indices, matrix transformations, etc.). The latter can be differentiated in ''Top of Atmosphere Reflectance'' (ToAR) which does not account for atmospheric effects (absorption or scattering) and in ''Top of Canopy Reflectance'' (ToCR) which introduces a "correction" for atmospheric effects.

In order to derive Reflectance values, likewise as with remotely sensed data acquired by other sensors, WorldView raw image digital numbers (DNs) need to be converted to ''at-sensor spectral Radiance'' values. At-sensor spectral Radiance values are an important input for the equation to derive Reflectance values. Note, ''Spectal Radiance'' is the measure of the quantity of radiation that hits the sensor and typically expressed in <math>\frac{W}{m^2*sr*nm}</math>, that is ''watts'' per ''unit source area'', per ''unit solid angle'', and per ''unit wavelength''.

<ul>

* Converting WorldView DNs to ''Top of Atmosphere'' Radiance, as described in [http://www.digitalglobe.com/downloads/Radiometric_Use_of_WorldView-2_Imagery.pdf Radiometric Use of WorldView-2 Imagery, Technical Note (2010)] by Todd Updike and Chris Comp, can be done by using the following equation:

<math>L_{\lambda\text{Pixel, Band}} = \frac{K_{\text{Band}} * q_{\text{Pixel, Band}}}{\Delta\lambda_{\text{Band}}}</math>

 where:

<ul>
* <math>L_{\lambda\text{Pixel,Band}}</math>: Top-of-Atmosphere Spectral Radiance image pixels [W-m-2-sr-1-μm-1]
* <math>K_{\text{Band}}</math>: the absolute radiometric calibration factor [W-m-2-sr-1-count-1] for a given band
* <math>q_{\text{Pixel,Band}}</math>: radiometrically corrected image pixels [counts or Digital Numbers]
* <math>\Delta_{\lambda_{\text{Band}}}</math>: the effective bandwidth [μm] for a given band
</ul>

* Converting to Top of Atmosphere Reflectance, also referred to as Planetary Reflectance, can be done by using the following equation:

<math>\rho_p = \frac{\pi * L\lambda * d^2}{ESUN\lambda * cos(\Theta_S)}</math>

 where:

<ul>
* <math>\rho</math> - Unitless Planetary Reflectance
* <math>\pi</math> - mathematical constant (3.14159265358)
* <math>L\lambda</math> spectral Radiance at the sensor's aperture, from equation... '''''ToADD'''''
* <math>d</math> - Earth-Sun distance in astronomical units, interpolated values
* <math>Esun</math> - Mean solar exoatmospheric irradiance(s) (W/m2/μm), interpolated values
* <math>cos(\theta_s)</math> - {{wikipedia||Solar_zenith_angle}} from the image acquisition's metadata
</ul>

</ul>

=== Importing data ===

==== File Formats & Metadata ====

''ToAdd''

==== Importing ====

''ToAdd''
<ul>
* Location creation based on georeferenced data
</ul>

Once inside a Location that is defined by the spatial reference system in which the bands of interest are projected, they can be imported with the {{cmd|r.in.gdal}} module.

For example, GeoTIFF files can be imported by looping {{cmd|r.in.gdal}} over all of them

<source lang="bash">
for TIF in `echo *.tif`; do r.in.gdal in=${TIF} out=${TIF%%.*}; done
</source>

To keep the raw material untouched, we create another Mapset inside the same Location and copy over the bands giving, ''optionally'' at the same time, a new name for each.

''ToAdd''

=== Deriving Physical Quantities ===

Conversions implemented in the GRASS module [https://github.com/NikosAlexandris/i.worldview.toar i.worldview.toar]

''ToCleanUp''

Converting Digital Numbers to Radiance/Reflectance requires knowledge about the sensor's specific spectral band parameters. In the following listed parameters (see below), the ''Effective Bandwidth''s and the ''Band-Averaged Solar Spectral Irradiance''s are extracted from the document [http://www.digitalglobe.com/downloads/Radiometric_Use_of_WorldView-2_Imagery.pdf Radiometric Use of WorldView-2 Imagery, Technical Note (2010)] by Todd Updike and Chris Comp. The Absolute Calibration Factors are extracted from a WorldView-2 product, specifically from an image metadata file (extension .IMD).

'''Note,''' however, as stated in the above document,

The absolute radiometric calibration factor is dependent on the specific band,
as well as the TDI exposure level, line rate, pixel aggregation, and bit depth
of the product. Based on these parameters, the appropriate value is provided in
the .IMD file. For this reason, care should be taken not to mix absolute radiometric
calibration factors between products that might have different collection conditions.

<source lang=bash>
Pan_CalCoef=0.05678345 ; Pan_Width=0.2846000 ; Pan_Esun=1580.8140
Coastal_CalCoef=0.009295654 ; Coastal_CalCoef=0.04730000 ; Coastal_Esun=1758.2229
Blue_CalCoef=0.01260825 ; Blue_Width=0.05430000 ; Blue_Esun=1974.2416
Green_CalCoef=0.009713071 ; Green_Width=0.06300000 ; Green_Esun=1856.4104
Yellow_CalCoef=0.005829815 ; Yellow_Width=0.03740000 ; Yellow_Esun=1738.4791
Red_CalCoef=0.01103623 ; Red_Width=0.05740000 ; Red_Esun=1559.4555
RedEdge_CalCoef=0.005188136 ; RedEdge_Width=0.03930000 ; Red_Edge_Esun=1342.0695
NIR1_CalCoef=0.01224380 ; NIR1_Width=0.09890000 ; NIR_Esun=1069.7302
NIR2_CalCoef=0.009042234 ; NIR2_Width=0.09960000 ; NIR2_Esun=861.2866
</source>

The following examples exemplify the conversion of raw Blue band digital numbers into Radiance and Reflectance.

==== Spectral Radiance ====

Converting a Blue Band (Digital Numbers) in to Spectral at-sensor Radiance in the ''correct'' units to be further used for the conversion in to unitless Reflectance:

<source lang="bash">
# set the region
g.region rast=Blue_DNs -p

# convert DNs to spectral Radiance values
r.mapcalc "Blue_Radiance = ( (10000 * WorldView_Blue_DNs) / (... * ...) )"
</source>

==== Planetary Reflectance ====

The equation to derive Reflectance values incorporates in addition:

<ul>
* The mathematical constant {{wikipedia|Pi}} <source lang="bash" enclose=none>PI=3.14159265358</source>.
* The Earth-Sun distance in astronomical units which depends on the acquisition's day of year (DOY -- also referred to as Julian day, {{wikipedia|Ordinal_date}}) and can be retrieved from the following spreadsheet <http://landsathandbook.gsfc.nasa.gov/excel_docs/d.xls>.
* The mean solar exoatmospheric irradiance in <math>\frac{W}{m^2*\mu m}</math>. See 3rd column of (interplated) values given above.
* The cosine of the ''Solar Zenith Angle'' (SZA) at the time of the acquisition. The SZA can be calculated from its complementary ''Solar Elevation Angle'' (SEA) given in the image acquisition's metadata.
</ul>

In the following example we accept as the acquisition's <source lang="bash" enclose=none>DOY=...</source> and <source lang="bash" enclose=none>SEA=...</source>. Hence, we get the Earth-Sun distance <source lang="bash" enclose=none>ESD=...</source> and <source lang="bash" enclose=none>SZA = ... deg</source>.

In any case, '''note''' that Top-of-Atmosphere Reflectance does not account for topographic, atmospheric, or BRDF differences.

Converting the in-Blue spectral band at-sensor Radiance in to Planerary Reflectance:

<source lang="bash" enclose=none>PI=...; ESD=...; BAND_Esun=...; SZA=...</source>
{{cmd|r.mapcalc}} <source lang="bash" enclose=none>"Blue_Reflectance = ( ${PI} * Blue_Radiance * ${ESD}^2 ) / ( ${BAND_Esun} * cos(${SZA}) )"</source>

==== Automatising Conversions ====

The conversion process can be scripted to avoid repeating the same steps for each band separately.

===== Python =====

A custom python script, performing the operations of interest, might be like [https://github.com/NikosAlexandris/i.worldview.toar i.worldview.toar (for GRASS 7.x)]

===== Bash =====
In bash, such a script might be as the following example. '''Note,''' however, in this example script constants, band parameters and acquisition related metadata are hard-coded!

<source lang="bash">
#!/bin/bash

# π, first 11 decimals
PI=3.14159265358

# HardCoded MetaData!

# Acquisition's Day of Year and estimated Earth-Sun Distance
DOY=100; ESD=1.00184

# Sun Zenith Angle based on the acquisition's Sun Elevation Angle
SEA=53.8; SZA=$(echo "90 - ${SEA}" | bc )

# some echo
echo "Acquisition-specific parameters"
echo "Day of Year: ${DOY}, Earth-Sun Distance: ${ESD}, Sun Zenith Angle: ${SZA}"
echo -e "\n"

# 1st column: Absolute Calibration Factors
# 2nd column: Spectral Band Effective Bandwidth, Δλ
# 3rd column: Band-Averaged Solar Spectral Irradiance [W/sq.m./micro-m]

K_Pan=0.05678345 ; Pan_Width=0.2846000 ; Pan_Esun=1580.8140
K_Coastal=0.009295654 ; Coastal_Width=0.04730000 ; Coastal_Esun=1758.2229
K_Blue=0.01260825 ; Blue_Width=0.05430000 ; Blue_Esun=1974.2416
K_Green=0.009713071 ; Green_Width=0.06300000 ; Green_Esun=1856.4104
K_Yellow=0.005829815 ; Yellow_Width=0.03740000 ; Yellow_Esun=1738.4791
K_Red=0.01103623 ; Red_Width=0.05740000 ; Red_Esun=1559.4555
K_RedEdge=0.005188136 ; RedEdge_Width=0.03930000 ; RedEdge_Esun=1342.0695
K_NIR1=0.01224380 ; NIR1_Width=0.09890000 ; NIR1_Esun=1069.7302
K_NIR2=0.009042234 ; NIR2_Width=0.09960000 ; NIR2_Esun=861.2866

# Bands
Spectral_Bands="Pan Coastal Blue Green Yellow Red RedEdge NIR1 NIR2"

# loop over all bands
for BAND in ${Spectral_Bands}; do

# some echo
echo -e "Processing the \"${BAND}\" spectral band"

# set band parameters as variables
eval BAND_Width="${BAND}_Width"
echo "* Band-specific radiometric parameters set to K=${!K_BAND}, Bandwidth=${!BAND_Width}"

# set region
g.region rast=${BAND}_DNs

# conversion to Radiance based on (1) -- attention: 32-bit calculations required
echo "* Converting ${BAND} Digital Numebrs to Radiance..." | tr -d "\n"
r.mapcalc "${BAND}_Radiance = ( double(${!K_BAND}) * ${BAND}_DNs ) / ${!BAND_Width}"

# report range
echo "* Reporting range of the ${BAND} spectral radiance: " | tr -d "\n"
# r.info -r "${BAND}_Radiance"
r.info -r "${BAND}_Radiance" | tr "\n" "," | cut -d"," -f1,2 | sed 's/,/,\ /'

# add info
r.support map=${BAND}_Radiance \
title="" \
units="W / sq.m. / μm / ster" \
description="Top-of-Atmosphere `echo ${BAND}` band spectral Radiance [W/m^2/sr/μm]" \
source1='"Radiometric Use of WorldView-2 Imagery, Technical Note (2010)", by Todd Updike & Chris Comp'

# Esun
eval BAND_Esun="${BAND}_Esun"

# some echo
echo "* Parameters required for the conversion: Esun= ${!BAND_Esun}, ESD= ${ESD}, ${SZA}"

# calculate ToAR
echo "* Converting ${BAND}_Radiance to Top of Atmosphere Reflectance..." | tr -d "\n"
r.mapcalc "${BAND}_ToAR = \
( ${PI} * ${BAND}_Radiance * ${ESD}^2 ) / ( ${!BAND_Esun} * cos(${SZA}) )"

# report range
echo "* Reporting range of the ${BAND} spectral radiance: " | tr -d "\n"
r.info -r ${BAND}_ToAR | tr "\n" "," | cut -d"," -f1,2 | sed 's/,/,\ /'

# add some metadata
r.support map=${BAND}_ToAR \
title="echo ${BAND} band (Top of Atmosphere Reflectance)" \
units="Unitless planetary reflectance" \
description="Top of Atmosphere `echo ${BAND}` band spectral Reflectance (unitless)" \
source1='"Radiometric Use of WorldView-2 Imagery, Technical Note (2010)", by Todd Updike & Chris Comp, Digital Globe' \
source2="Digital Globe" \
history="PI=3.14159265358; K=${!K_BAND}; Bandwidth=${!BAND_Width}; ESD=${ESD}; Esun=${!BAND_Esun}; SZA=${SZA}"

echo -e "\n"

done
</source>

=== Atmospheric correction ===

See [[Atmospheric correction]]

== Post-Processing ==

Having beforehand satellite image data expressed in physical quantities (radiance or reflectance) is preferred to follow-up with digital image analysis techniques. A few common post-processing practices are Contrast-Enhancement, Pan-Sharpening and creating Pseudo- or True-Color Composites. Other well known enhancing manipualtions to support the analyses of satellite imagery, include deriving Vegetation Indices, transforming multi-spectral data based on [[Principal_Components_Analysis | PCA]] and Segmenting images.

=== Contrast Enhancement ===

''ToAdd''

=== Pan-Sharpening ===

[http://en.wikipedia.org/wiki/Pansharpened_image Pan-Sharpening] / [http://en.wikipedia.org/wiki/Image_fusion Fusion] is the process of merging high-resolution panchromatic and lower resolution multi-spectral imagery. [http://grass.osgeo.org/grass70/ GRASS 7] holds a dedicated pan-sharpening module, {{cmd|i.pansharpen}} which features three techniques for sharpening, namely the [http://wiki.awf.forst.uni-goettingen.de/wiki/index.php/Brovey_Transformation Brovey transformation], the classical IHS method and one that is based on [[Principal Components Analysis]] (PCA).
Another algorithm deriving excellent detail and a realistic representation of original multispectral scene colors, is the High-Pass Filter Addition (HPFA) technique. It is implemented via the {{AddonSrc|imagery|i.fusion.hpf|version=7}} add-on (for GRASS 6, please refer to a bash shell script https://github.com/NikosAlexandris/i.fusion.hpf.sh which is, however, unmaintained).

One approach inside GRASS-GIS to get an acceptable color-balanced composite image after Pan-sharpening 11-bit IKONOS spectral bands comprises the following steps

<ul>
# rescale the 11-bit WorldView spectral bands to 8-bit ranging in [0, 255] ({{cmd|r.rescale}})
# pan-sharpen with any of the featured methods (Brovey, IHS, PCA) ({{cmd|i.pansharpen}})
# color-balance by using the {{cmd|i.landsat.rgb}} module or manually adjusting the color tables of the bands of interest
# create a composite image by using the {{cmd|r.compose}} module
</ul>

''ToAdd''

==== Automatising Sharpening & RGB Composition ====

An ''experimental'' bash script to automatise the sharpening/fusion and RGB composition process is demonstrated here. The script uses the Spectral Reflectance values (double precision values), gained from previous steps (see above), and produces Pan-sharpened images by applying all of the three methods that {{cmd|i.pansharpen}} offers. In addition, it attempts to produce True-Color composite images, without and with re-balancing the color tables by using the {{cmd|i.landsat.rgb}} module.

Initial tests indicate that at least one of the three sharpening methods, combined with re-balancing, produces very clear and balanced True-Color composites. Note, there is no 100% confidence that it will produce nice looking True-Color composites. Some of the methods might simply work, others might deliver fancy-colored images. As usual, some manual action might be required to get the re-balancing to work as best as possible.

The script can be expanded in terms of using more inputs by utilising bash's positional parameters. However, care has to be taken to alter the instruction that concerns the conversion from double precision values to 8-bit integers.

<source lang="bash">
...
</source>

=== Color Composites ===

''ToAdd''

== Vegetation Indices ==

''ToAdd''

== PCA ==

''ToAdd''

== WordView Image classification ==

See [[Image classification]]

== References / Sources ==

* ...
* [http://landsathandbook.gsfc.nasa.gov/data_prod/prog_sect11_3.html Landsat7 Science Data Users Handbook, Chapter 11, Section 3]
* [http://igett.delmar.edu/Resources/Remote%20Sensing%20Technology%20Training/Calculation-DN_to_Reflectance_Irish_20June08.pdf Calibrated Landsat Digital Number (DN) to Top of Atmosphere (TOA) Reflectance Conversion, by Richard Irish]
* from [http://landsat.usgs.gov/tools_access_all_faqs.php FAQs about the Landsat Missions]
** [http://landsat.usgs.gov/how_is_radiance_calculated.php How is radiance calculated?]
** [http://landsat.usgs.gov/at_sensor_reflectance_calculated.php How is at-sensor reflectance calculated?]

== See also ==

* GRASS-Wiki page about [http://grasswiki.osgeo.org/wiki/Image_processing Image Processing]

''ToAdd''

[[Category: Documentation]]
[[Category: FAQ]]
[[Category: WorldView]]
[[Category: High Resolution]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Image processing]]

WorldView

2016-08-14T05:08:29Z

⚠️NikosA: /* Inside GRASS GIS */

= This page is under construction =

= Overview =

WorldView-1, WorldView-2 and WorldView-3 are commercial earth observation satellites. Details about the sensors are provided at Digital Globe's collection of [http://www.digitalglobe.com/resources/satellite-information Spacecraft Data Sheets].

It is recommended -- actually it is a necessity! -- to have a look at the documentation explaining [http://www.digitalglobe.com/sites/default/files/Imagery_Support_Data_Documentation%20%281%29.pdf Digital Globe's products metadata]. Note, all of DigitalGlobe’s satellites collect data using an 11 bit dynamic range. For technical reasons, products are delivered as either 16-bit or 8-bit data.

== Availability (Sample Data) ==

* Search for commercial satellite image providers in the internet.

== Modules overview ==

=== Outside of GRASS ===

Satellite imagery can be managed and, at some extent, pre-processed with various [http://www.gdal.org/gdal_utilities.html GDAL utilities].

* [http://www.gdal.org/gdalmanage.html gdalmanage]
* [http://www.gdal.org/gdalinfo.html gdalinfo]
* [http://www.gdal.org/gdal_translate.html gdal_translate]
* [http://www.gdal.org/gdalbuildvrt.html gdalbuildvrt]
* [http://www.gdal.org/gdalwarp.html gdalwarp]

=== Inside GRASS GIS ===

GRASS GIS features a complete set of modules and various add-ons enabling pre- and post-processing of WorldView satellite imagery. The following lists offer an overview of related modules and add-ons.

* {{cmd|r.in.gdal}}
* {{cmd|r.mapcalc}}
* {{cmd|r.colors}}
* {{cmd|i.landsat.rgb}}
* {{AddonSrc|imagery|i.fusion.hpf|version=7}}
* {{cmd|i.pansharpen}}
* [https://github.com/NikosAlexandris/i.worldview.toar i.worldview.toar]
* {{cmd|i.vi}}
* {{cmd|i.segment}}
* ...''more to add''

== Pre-Processing ==

=== Overview ===

Typically, multispectral satellite data are converted into physical quantities such as ''Radiance'' or ''Reflectance'' before they are subjected in multispectral analysis techniques (image interpretation, band arithmetic, vegetation indices, matrix transformations, etc.). The latter can be differentiated in ''Top of Atmosphere Reflectance'' (ToAR) which does not account for atmospheric effects (absorption or scattering) and in ''Top of Canopy Reflectance'' (ToCR) which introduces a "correction" for atmospheric effects.

In order to derive Reflectance values, likewise as with remotely sensed data acquired by other sensors, WorldView raw image digital numbers (DNs) need to be converted to ''at-sensor spectral Radiance'' values. At-sensor spectral Radiance values are an important input for the equation to derive Reflectance values. Note, ''Spectal Radiance'' is the measure of the quantity of radiation that hits the sensor and typically expressed in <math>\frac{W}{m^2*sr*nm}</math>, that is ''watts'' per ''unit source area'', per ''unit solid angle'', and per ''unit wavelength''.

<ul>

* Converting WorldView DNs to ''Top of Atmosphere'' Radiance, as described in [http://www.digitalglobe.com/downloads/Radiometric_Use_of_WorldView-2_Imagery.pdf Radiometric Use of WorldView-2 Imagery, Technical Note (2010)] by Todd Updike and Chris Comp, can be done by using the following equation:

<math>L_{\lambda\text{Pixel, Band}} = \frac{K_{\text{Band}} * q_{\text{Pixel, Band}}}{\Delta\lambda_{\text{Band}}}</math>

 where:

<ul>
* <math>L_{\lambda\text{Pixel,Band}}</math>: Top-of-Atmosphere Spectral Radiance image pixels [W-m-2-sr-1-μm-1]
* <math>K_{\text{Band}}</math>: the absolute radiometric calibration factor [W-m-2-sr-1-count-1] for a given band
* <math>q_{\text{Pixel,Band}}</math>: radiometrically corrected image pixels [counts or Digital Numbers]
* <math>\Delta_{\lambda_{\text{Band}}}</math>: the effective bandwidth [μm] for a given band
</ul>

* Converting to Top of Atmosphere Reflectance, also referred to as Planetary Reflectance, can be done by using the following equation:

<math>\rho_p = \frac{\pi * L\lambda * d^2}{ESUN\lambda * cos(\Theta_S)}</math>

 where:

<ul>
* <math>\rho</math> - Unitless Planetary Reflectance
* <math>\pi</math> - mathematical constant (3.14159265358)
* <math>L\lambda</math> spectral Radiance at the sensor's aperture, from equation... '''''ToADD'''''
* <math>d</math> - Earth-Sun distance in astronomical units, interpolated values
* <math>Esun</math> - Mean solar exoatmospheric irradiance(s) (W/m2/μm), interpolated values
* <math>cos(\theta_s)</math> - {{wikipedia||Solar_zenith_angle}} from the image acquisition's metadata
</ul>

</ul>

=== Importing data ===

==== File Formats & Metadata ====

''ToAdd''

==== Importing ====

''ToAdd''
<ul>
* Location creation based on georeferenced data
</ul>

Once inside a Location that is defined by the spatial reference system in which the bands of interest are projected, they can be imported with the {{cmd|r.in.gdal}} module.

For example, GeoTIFF files can be imported by looping {{cmd|r.in.gdal}} over all of them

<source lang="bash">
for TIF in `echo *.tif`; do r.in.gdal in=${TIF} out=${TIF%%.*}; done
</source>

To keep the raw material untouched, we create another Mapset inside the same Location and copy over the bands giving, ''optionally'' at the same time, a new name for each.

''ToAdd''

=== Deriving Physical Quantities ===

''ToCleanUp''

Converting Digital Numbers to Radiance/Reflectance requires knowledge about the sensor's specific spectral band parameters. In the following listed parameters (see below), the ''Effective Bandwidth''s and the ''Band-Averaged Solar Spectral Irradiance''s are extracted from the document [http://www.digitalglobe.com/downloads/Radiometric_Use_of_WorldView-2_Imagery.pdf Radiometric Use of WorldView-2 Imagery, Technical Note (2010)] by Todd Updike and Chris Comp. The Absolute Calibration Factors are extracted from a WorldView-2 product, specifically from an image metadata file (extension .IMD).

'''Note,''' however, as stated in the above document,

The absolute radiometric calibration factor is dependent on the specific band,
as well as the TDI exposure level, line rate, pixel aggregation, and bit depth
of the product. Based on these parameters, the appropriate value is provided in
the .IMD file. For this reason, care should be taken not to mix absolute radiometric
calibration factors between products that might have different collection conditions.

<source lang=bash>
Pan_CalCoef=0.05678345 ; Pan_Width=0.2846000 ; Pan_Esun=1580.8140
Coastal_CalCoef=0.009295654 ; Coastal_CalCoef=0.04730000 ; Coastal_Esun=1758.2229
Blue_CalCoef=0.01260825 ; Blue_Width=0.05430000 ; Blue_Esun=1974.2416
Green_CalCoef=0.009713071 ; Green_Width=0.06300000 ; Green_Esun=1856.4104
Yellow_CalCoef=0.005829815 ; Yellow_Width=0.03740000 ; Yellow_Esun=1738.4791
Red_CalCoef=0.01103623 ; Red_Width=0.05740000 ; Red_Esun=1559.4555
RedEdge_CalCoef=0.005188136 ; RedEdge_Width=0.03930000 ; Red_Edge_Esun=1342.0695
NIR1_CalCoef=0.01224380 ; NIR1_Width=0.09890000 ; NIR_Esun=1069.7302
NIR2_CalCoef=0.009042234 ; NIR2_Width=0.09960000 ; NIR2_Esun=861.2866
</source>

The following examples exemplify the conversion of raw Blue band digital numbers into Radiance and Reflectance.

==== Spectral Radiance ====

Converting a Blue Band (Digital Numbers) in to Spectral at-sensor Radiance in the ''correct'' units to be further used for the conversion in to unitless Reflectance:

<source lang="bash">
# set the region
g.region rast=Blue_DNs -p

# convert DNs to spectral Radiance values
r.mapcalc "Blue_Radiance = ( (10000 * WorldView_Blue_DNs) / (... * ...) )"
</source>

==== Planetary Reflectance ====

The equation to derive Reflectance values incorporates in addition:

<ul>
* The mathematical constant {{wikipedia|Pi}} <source lang="bash" enclose=none>PI=3.14159265358</source>.
* The Earth-Sun distance in astronomical units which depends on the acquisition's day of year (DOY -- also referred to as Julian day, {{wikipedia|Ordinal_date}}) and can be retrieved from the following spreadsheet <http://landsathandbook.gsfc.nasa.gov/excel_docs/d.xls>.
* The mean solar exoatmospheric irradiance in <math>\frac{W}{m^2*\mu m}</math>. See 3rd column of (interplated) values given above.
* The cosine of the ''Solar Zenith Angle'' (SZA) at the time of the acquisition. The SZA can be calculated from its complementary ''Solar Elevation Angle'' (SEA) given in the image acquisition's metadata.
</ul>

In the following example we accept as the acquisition's <source lang="bash" enclose=none>DOY=...</source> and <source lang="bash" enclose=none>SEA=...</source>. Hence, we get the Earth-Sun distance <source lang="bash" enclose=none>ESD=...</source> and <source lang="bash" enclose=none>SZA = ... deg</source>.

In any case, '''note''' that Top-of-Atmosphere Reflectance does not account for topographic, atmospheric, or BRDF differences.

Converting the in-Blue spectral band at-sensor Radiance in to Planerary Reflectance:

<source lang="bash" enclose=none>PI=...; ESD=...; BAND_Esun=...; SZA=...</source>
{{cmd|r.mapcalc}} <source lang="bash" enclose=none>"Blue_Reflectance = ( ${PI} * Blue_Radiance * ${ESD}^2 ) / ( ${BAND_Esun} * cos(${SZA}) )"</source>

==== Automatising Conversions ====

The conversion process can be scripted to avoid repeating the same steps for each band separately.

===== Python =====

A custom python script, performing the operations of interest, might be like [https://github.com/NikosAlexandris/i.worldview.toar i.worldview.toar (for GRASS 7.x)]

===== Bash =====
In bash, such a script might be as the following example. '''Note,''' however, in this example script constants, band parameters and acquisition related metadata are hard-coded!

<source lang="bash">
#!/bin/bash

# π, first 11 decimals
PI=3.14159265358

# HardCoded MetaData!

# Acquisition's Day of Year and estimated Earth-Sun Distance
DOY=100; ESD=1.00184

# Sun Zenith Angle based on the acquisition's Sun Elevation Angle
SEA=53.8; SZA=$(echo "90 - ${SEA}" | bc )

# some echo
echo "Acquisition-specific parameters"
echo "Day of Year: ${DOY}, Earth-Sun Distance: ${ESD}, Sun Zenith Angle: ${SZA}"
echo -e "\n"

# 1st column: Absolute Calibration Factors
# 2nd column: Spectral Band Effective Bandwidth, Δλ
# 3rd column: Band-Averaged Solar Spectral Irradiance [W/sq.m./micro-m]

K_Pan=0.05678345 ; Pan_Width=0.2846000 ; Pan_Esun=1580.8140
K_Coastal=0.009295654 ; Coastal_Width=0.04730000 ; Coastal_Esun=1758.2229
K_Blue=0.01260825 ; Blue_Width=0.05430000 ; Blue_Esun=1974.2416
K_Green=0.009713071 ; Green_Width=0.06300000 ; Green_Esun=1856.4104
K_Yellow=0.005829815 ; Yellow_Width=0.03740000 ; Yellow_Esun=1738.4791
K_Red=0.01103623 ; Red_Width=0.05740000 ; Red_Esun=1559.4555
K_RedEdge=0.005188136 ; RedEdge_Width=0.03930000 ; RedEdge_Esun=1342.0695
K_NIR1=0.01224380 ; NIR1_Width=0.09890000 ; NIR1_Esun=1069.7302
K_NIR2=0.009042234 ; NIR2_Width=0.09960000 ; NIR2_Esun=861.2866

# Bands
Spectral_Bands="Pan Coastal Blue Green Yellow Red RedEdge NIR1 NIR2"

# loop over all bands
for BAND in ${Spectral_Bands}; do

# some echo
echo -e "Processing the \"${BAND}\" spectral band"

# set band parameters as variables
eval BAND_Width="${BAND}_Width"
echo "* Band-specific radiometric parameters set to K=${!K_BAND}, Bandwidth=${!BAND_Width}"

# set region
g.region rast=${BAND}_DNs

# conversion to Radiance based on (1) -- attention: 32-bit calculations required
echo "* Converting ${BAND} Digital Numebrs to Radiance..." | tr -d "\n"
r.mapcalc "${BAND}_Radiance = ( double(${!K_BAND}) * ${BAND}_DNs ) / ${!BAND_Width}"

# report range
echo "* Reporting range of the ${BAND} spectral radiance: " | tr -d "\n"
# r.info -r "${BAND}_Radiance"
r.info -r "${BAND}_Radiance" | tr "\n" "," | cut -d"," -f1,2 | sed 's/,/,\ /'

# add info
r.support map=${BAND}_Radiance \
title="" \
units="W / sq.m. / μm / ster" \
description="Top-of-Atmosphere `echo ${BAND}` band spectral Radiance [W/m^2/sr/μm]" \
source1='"Radiometric Use of WorldView-2 Imagery, Technical Note (2010)", by Todd Updike & Chris Comp'

# Esun
eval BAND_Esun="${BAND}_Esun"

# some echo
echo "* Parameters required for the conversion: Esun= ${!BAND_Esun}, ESD= ${ESD}, ${SZA}"

# calculate ToAR
echo "* Converting ${BAND}_Radiance to Top of Atmosphere Reflectance..." | tr -d "\n"
r.mapcalc "${BAND}_ToAR = \
( ${PI} * ${BAND}_Radiance * ${ESD}^2 ) / ( ${!BAND_Esun} * cos(${SZA}) )"

# report range
echo "* Reporting range of the ${BAND} spectral radiance: " | tr -d "\n"
r.info -r ${BAND}_ToAR | tr "\n" "," | cut -d"," -f1,2 | sed 's/,/,\ /'

# add some metadata
r.support map=${BAND}_ToAR \
title="echo ${BAND} band (Top of Atmosphere Reflectance)" \
units="Unitless planetary reflectance" \
description="Top of Atmosphere `echo ${BAND}` band spectral Reflectance (unitless)" \
source1='"Radiometric Use of WorldView-2 Imagery, Technical Note (2010)", by Todd Updike & Chris Comp, Digital Globe' \
source2="Digital Globe" \
history="PI=3.14159265358; K=${!K_BAND}; Bandwidth=${!BAND_Width}; ESD=${ESD}; Esun=${!BAND_Esun}; SZA=${SZA}"

echo -e "\n"

done
</source>

=== Atmospheric correction ===

See [[Atmospheric correction]]

== Post-Processing ==

Having beforehand satellite image data expressed in physical quantities (radiance or reflectance) is preferred to follow-up with digital image analysis techniques. A few common post-processing practices are Contrast-Enhancement, Pan-Sharpening and creating Pseudo- or True-Color Composites. Other well known enhancing manipualtions to support the analyses of satellite imagery, include deriving Vegetation Indices, transforming multi-spectral data based on [[Principal_Components_Analysis | PCA]] and Segmenting images.

=== Contrast Enhancement ===

''ToAdd''

=== Pan-Sharpening ===

[http://en.wikipedia.org/wiki/Pansharpened_image Pan-Sharpening] / [http://en.wikipedia.org/wiki/Image_fusion Fusion] is the process of merging high-resolution panchromatic and lower resolution multi-spectral imagery. [http://grass.osgeo.org/grass70/ GRASS 7] holds a dedicated pan-sharpening module, {{cmd|i.pansharpen}} which features three techniques for sharpening, namely the [http://wiki.awf.forst.uni-goettingen.de/wiki/index.php/Brovey_Transformation Brovey transformation], the classical IHS method and one that is based on [[Principal Components Analysis]] (PCA).
Another algorithm deriving excellent detail and a realistic representation of original multispectral scene colors, is the High-Pass Filter Addition (HPFA) technique. It is implemented via the {{AddonSrc|imagery|i.fusion.hpf|version=7}} add-on (for GRASS 6, please refer to a bash shell script https://github.com/NikosAlexandris/i.fusion.hpf.sh which is, however, unmaintained).

One approach inside GRASS-GIS to get an acceptable color-balanced composite image after Pan-sharpening 11-bit IKONOS spectral bands comprises the following steps

<ul>
# rescale the 11-bit WorldView spectral bands to 8-bit ranging in [0, 255] ({{cmd|r.rescale}})
# pan-sharpen with any of the featured methods (Brovey, IHS, PCA) ({{cmd|i.pansharpen}})
# color-balance by using the {{cmd|i.landsat.rgb}} module or manually adjusting the color tables of the bands of interest
# create a composite image by using the {{cmd|r.compose}} module
</ul>

''ToAdd''

==== Automatising Sharpening & RGB Composition ====

An ''experimental'' bash script to automatise the sharpening/fusion and RGB composition process is demonstrated here. The script uses the Spectral Reflectance values (double precision values), gained from previous steps (see above), and produces Pan-sharpened images by applying all of the three methods that {{cmd|i.pansharpen}} offers. In addition, it attempts to produce True-Color composite images, without and with re-balancing the color tables by using the {{cmd|i.landsat.rgb}} module.

Initial tests indicate that at least one of the three sharpening methods, combined with re-balancing, produces very clear and balanced True-Color composites. Note, there is no 100% confidence that it will produce nice looking True-Color composites. Some of the methods might simply work, others might deliver fancy-colored images. As usual, some manual action might be required to get the re-balancing to work as best as possible.

The script can be expanded in terms of using more inputs by utilising bash's positional parameters. However, care has to be taken to alter the instruction that concerns the conversion from double precision values to 8-bit integers.

<source lang="bash">
...
</source>

=== Color Composites ===

''ToAdd''

== Vegetation Indices ==

''ToAdd''

== PCA ==

''ToAdd''

== WordView Image classification ==

See [[Image classification]]

== References / Sources ==

* ...
* [http://landsathandbook.gsfc.nasa.gov/data_prod/prog_sect11_3.html Landsat7 Science Data Users Handbook, Chapter 11, Section 3]
* [http://igett.delmar.edu/Resources/Remote%20Sensing%20Technology%20Training/Calculation-DN_to_Reflectance_Irish_20June08.pdf Calibrated Landsat Digital Number (DN) to Top of Atmosphere (TOA) Reflectance Conversion, by Richard Irish]
* from [http://landsat.usgs.gov/tools_access_all_faqs.php FAQs about the Landsat Missions]
** [http://landsat.usgs.gov/how_is_radiance_calculated.php How is radiance calculated?]
** [http://landsat.usgs.gov/at_sensor_reflectance_calculated.php How is at-sensor reflectance calculated?]

== See also ==

* GRASS-Wiki page about [http://grasswiki.osgeo.org/wiki/Image_processing Image Processing]

''ToAdd''

[[Category: Documentation]]
[[Category: FAQ]]
[[Category: WorldView]]
[[Category: High Resolution]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Image processing]]

WorldView

2016-08-14T05:07:32Z

⚠️NikosA: added i.fusion.hpf

= This page is under construction =

= Overview =

WorldView-1, WorldView-2 and WorldView-3 are commercial earth observation satellites. Details about the sensors are provided at Digital Globe's collection of [http://www.digitalglobe.com/resources/satellite-information Spacecraft Data Sheets].

It is recommended -- actually it is a necessity! -- to have a look at the documentation explaining [http://www.digitalglobe.com/sites/default/files/Imagery_Support_Data_Documentation%20%281%29.pdf Digital Globe's products metadata]. Note, all of DigitalGlobe’s satellites collect data using an 11 bit dynamic range. For technical reasons, products are delivered as either 16-bit or 8-bit data.

== Availability (Sample Data) ==

* Search for commercial satellite image providers in the internet.

== Modules overview ==

=== Outside of GRASS ===

Satellite imagery can be managed and, at some extent, pre-processed with various [http://www.gdal.org/gdal_utilities.html GDAL utilities].

* [http://www.gdal.org/gdalmanage.html gdalmanage]
* [http://www.gdal.org/gdalinfo.html gdalinfo]
* [http://www.gdal.org/gdal_translate.html gdal_translate]
* [http://www.gdal.org/gdalbuildvrt.html gdalbuildvrt]
* [http://www.gdal.org/gdalwarp.html gdalwarp]

=== Inside GRASS GIS ===

GRASS GIS features a complete set of modules and various add-ons enabling pre- and post-processing of WorldView satellite imagery. The following lists offer an overview of related modules and add-ons.

* {{cmd|r.in.gdal}}
* {{cmd|r.mapcalc}}
* {{cmd|r.colors}}
* {{cmd|i.landsat.rgb}}
* {{cmd|i.pansharpen}}
* {{cmd|i.vi}}
* {{cmd|i.segment}}
* ...''more to add''

== Pre-Processing ==

=== Overview ===

Typically, multispectral satellite data are converted into physical quantities such as ''Radiance'' or ''Reflectance'' before they are subjected in multispectral analysis techniques (image interpretation, band arithmetic, vegetation indices, matrix transformations, etc.). The latter can be differentiated in ''Top of Atmosphere Reflectance'' (ToAR) which does not account for atmospheric effects (absorption or scattering) and in ''Top of Canopy Reflectance'' (ToCR) which introduces a "correction" for atmospheric effects.

In order to derive Reflectance values, likewise as with remotely sensed data acquired by other sensors, WorldView raw image digital numbers (DNs) need to be converted to ''at-sensor spectral Radiance'' values. At-sensor spectral Radiance values are an important input for the equation to derive Reflectance values. Note, ''Spectal Radiance'' is the measure of the quantity of radiation that hits the sensor and typically expressed in <math>\frac{W}{m^2*sr*nm}</math>, that is ''watts'' per ''unit source area'', per ''unit solid angle'', and per ''unit wavelength''.

<ul>

* Converting WorldView DNs to ''Top of Atmosphere'' Radiance, as described in [http://www.digitalglobe.com/downloads/Radiometric_Use_of_WorldView-2_Imagery.pdf Radiometric Use of WorldView-2 Imagery, Technical Note (2010)] by Todd Updike and Chris Comp, can be done by using the following equation:

<math>L_{\lambda\text{Pixel, Band}} = \frac{K_{\text{Band}} * q_{\text{Pixel, Band}}}{\Delta\lambda_{\text{Band}}}</math>

 where:

<ul>
* <math>L_{\lambda\text{Pixel,Band}}</math>: Top-of-Atmosphere Spectral Radiance image pixels [W-m-2-sr-1-μm-1]
* <math>K_{\text{Band}}</math>: the absolute radiometric calibration factor [W-m-2-sr-1-count-1] for a given band
* <math>q_{\text{Pixel,Band}}</math>: radiometrically corrected image pixels [counts or Digital Numbers]
* <math>\Delta_{\lambda_{\text{Band}}}</math>: the effective bandwidth [μm] for a given band
</ul>

* Converting to Top of Atmosphere Reflectance, also referred to as Planetary Reflectance, can be done by using the following equation:

<math>\rho_p = \frac{\pi * L\lambda * d^2}{ESUN\lambda * cos(\Theta_S)}</math>

 where:

<ul>
* <math>\rho</math> - Unitless Planetary Reflectance
* <math>\pi</math> - mathematical constant (3.14159265358)
* <math>L\lambda</math> spectral Radiance at the sensor's aperture, from equation... '''''ToADD'''''
* <math>d</math> - Earth-Sun distance in astronomical units, interpolated values
* <math>Esun</math> - Mean solar exoatmospheric irradiance(s) (W/m2/μm), interpolated values
* <math>cos(\theta_s)</math> - {{wikipedia||Solar_zenith_angle}} from the image acquisition's metadata
</ul>

</ul>

=== Importing data ===

==== File Formats & Metadata ====

''ToAdd''

==== Importing ====

''ToAdd''
<ul>
* Location creation based on georeferenced data
</ul>

Once inside a Location that is defined by the spatial reference system in which the bands of interest are projected, they can be imported with the {{cmd|r.in.gdal}} module.

For example, GeoTIFF files can be imported by looping {{cmd|r.in.gdal}} over all of them

<source lang="bash">
for TIF in `echo *.tif`; do r.in.gdal in=${TIF} out=${TIF%%.*}; done
</source>

To keep the raw material untouched, we create another Mapset inside the same Location and copy over the bands giving, ''optionally'' at the same time, a new name for each.

''ToAdd''

=== Deriving Physical Quantities ===

''ToCleanUp''

Converting Digital Numbers to Radiance/Reflectance requires knowledge about the sensor's specific spectral band parameters. In the following listed parameters (see below), the ''Effective Bandwidth''s and the ''Band-Averaged Solar Spectral Irradiance''s are extracted from the document [http://www.digitalglobe.com/downloads/Radiometric_Use_of_WorldView-2_Imagery.pdf Radiometric Use of WorldView-2 Imagery, Technical Note (2010)] by Todd Updike and Chris Comp. The Absolute Calibration Factors are extracted from a WorldView-2 product, specifically from an image metadata file (extension .IMD).

'''Note,''' however, as stated in the above document,

The absolute radiometric calibration factor is dependent on the specific band,
as well as the TDI exposure level, line rate, pixel aggregation, and bit depth
of the product. Based on these parameters, the appropriate value is provided in
the .IMD file. For this reason, care should be taken not to mix absolute radiometric
calibration factors between products that might have different collection conditions.

<source lang=bash>
Pan_CalCoef=0.05678345 ; Pan_Width=0.2846000 ; Pan_Esun=1580.8140
Coastal_CalCoef=0.009295654 ; Coastal_CalCoef=0.04730000 ; Coastal_Esun=1758.2229
Blue_CalCoef=0.01260825 ; Blue_Width=0.05430000 ; Blue_Esun=1974.2416
Green_CalCoef=0.009713071 ; Green_Width=0.06300000 ; Green_Esun=1856.4104
Yellow_CalCoef=0.005829815 ; Yellow_Width=0.03740000 ; Yellow_Esun=1738.4791
Red_CalCoef=0.01103623 ; Red_Width=0.05740000 ; Red_Esun=1559.4555
RedEdge_CalCoef=0.005188136 ; RedEdge_Width=0.03930000 ; Red_Edge_Esun=1342.0695
NIR1_CalCoef=0.01224380 ; NIR1_Width=0.09890000 ; NIR_Esun=1069.7302
NIR2_CalCoef=0.009042234 ; NIR2_Width=0.09960000 ; NIR2_Esun=861.2866
</source>

The following examples exemplify the conversion of raw Blue band digital numbers into Radiance and Reflectance.

==== Spectral Radiance ====

Converting a Blue Band (Digital Numbers) in to Spectral at-sensor Radiance in the ''correct'' units to be further used for the conversion in to unitless Reflectance:

<source lang="bash">
# set the region
g.region rast=Blue_DNs -p

# convert DNs to spectral Radiance values
r.mapcalc "Blue_Radiance = ( (10000 * WorldView_Blue_DNs) / (... * ...) )"
</source>

==== Planetary Reflectance ====

The equation to derive Reflectance values incorporates in addition:

<ul>
* The mathematical constant {{wikipedia|Pi}} <source lang="bash" enclose=none>PI=3.14159265358</source>.
* The Earth-Sun distance in astronomical units which depends on the acquisition's day of year (DOY -- also referred to as Julian day, {{wikipedia|Ordinal_date}}) and can be retrieved from the following spreadsheet <http://landsathandbook.gsfc.nasa.gov/excel_docs/d.xls>.
* The mean solar exoatmospheric irradiance in <math>\frac{W}{m^2*\mu m}</math>. See 3rd column of (interplated) values given above.
* The cosine of the ''Solar Zenith Angle'' (SZA) at the time of the acquisition. The SZA can be calculated from its complementary ''Solar Elevation Angle'' (SEA) given in the image acquisition's metadata.
</ul>

In the following example we accept as the acquisition's <source lang="bash" enclose=none>DOY=...</source> and <source lang="bash" enclose=none>SEA=...</source>. Hence, we get the Earth-Sun distance <source lang="bash" enclose=none>ESD=...</source> and <source lang="bash" enclose=none>SZA = ... deg</source>.

In any case, '''note''' that Top-of-Atmosphere Reflectance does not account for topographic, atmospheric, or BRDF differences.

Converting the in-Blue spectral band at-sensor Radiance in to Planerary Reflectance:

<source lang="bash" enclose=none>PI=...; ESD=...; BAND_Esun=...; SZA=...</source>
{{cmd|r.mapcalc}} <source lang="bash" enclose=none>"Blue_Reflectance = ( ${PI} * Blue_Radiance * ${ESD}^2 ) / ( ${BAND_Esun} * cos(${SZA}) )"</source>

==== Automatising Conversions ====

The conversion process can be scripted to avoid repeating the same steps for each band separately.

===== Python =====

A custom python script, performing the operations of interest, might be like [https://github.com/NikosAlexandris/i.worldview.toar i.worldview.toar (for GRASS 7.x)]

===== Bash =====
In bash, such a script might be as the following example. '''Note,''' however, in this example script constants, band parameters and acquisition related metadata are hard-coded!

<source lang="bash">
#!/bin/bash

# π, first 11 decimals
PI=3.14159265358

# HardCoded MetaData!

# Acquisition's Day of Year and estimated Earth-Sun Distance
DOY=100; ESD=1.00184

# Sun Zenith Angle based on the acquisition's Sun Elevation Angle
SEA=53.8; SZA=$(echo "90 - ${SEA}" | bc )

# some echo
echo "Acquisition-specific parameters"
echo "Day of Year: ${DOY}, Earth-Sun Distance: ${ESD}, Sun Zenith Angle: ${SZA}"
echo -e "\n"

# 1st column: Absolute Calibration Factors
# 2nd column: Spectral Band Effective Bandwidth, Δλ
# 3rd column: Band-Averaged Solar Spectral Irradiance [W/sq.m./micro-m]

K_Pan=0.05678345 ; Pan_Width=0.2846000 ; Pan_Esun=1580.8140
K_Coastal=0.009295654 ; Coastal_Width=0.04730000 ; Coastal_Esun=1758.2229
K_Blue=0.01260825 ; Blue_Width=0.05430000 ; Blue_Esun=1974.2416
K_Green=0.009713071 ; Green_Width=0.06300000 ; Green_Esun=1856.4104
K_Yellow=0.005829815 ; Yellow_Width=0.03740000 ; Yellow_Esun=1738.4791
K_Red=0.01103623 ; Red_Width=0.05740000 ; Red_Esun=1559.4555
K_RedEdge=0.005188136 ; RedEdge_Width=0.03930000 ; RedEdge_Esun=1342.0695
K_NIR1=0.01224380 ; NIR1_Width=0.09890000 ; NIR1_Esun=1069.7302
K_NIR2=0.009042234 ; NIR2_Width=0.09960000 ; NIR2_Esun=861.2866

# Bands
Spectral_Bands="Pan Coastal Blue Green Yellow Red RedEdge NIR1 NIR2"

# loop over all bands
for BAND in ${Spectral_Bands}; do

# some echo
echo -e "Processing the \"${BAND}\" spectral band"

# set band parameters as variables
eval BAND_Width="${BAND}_Width"
echo "* Band-specific radiometric parameters set to K=${!K_BAND}, Bandwidth=${!BAND_Width}"

# set region
g.region rast=${BAND}_DNs

# conversion to Radiance based on (1) -- attention: 32-bit calculations required
echo "* Converting ${BAND} Digital Numebrs to Radiance..." | tr -d "\n"
r.mapcalc "${BAND}_Radiance = ( double(${!K_BAND}) * ${BAND}_DNs ) / ${!BAND_Width}"

# report range
echo "* Reporting range of the ${BAND} spectral radiance: " | tr -d "\n"
# r.info -r "${BAND}_Radiance"
r.info -r "${BAND}_Radiance" | tr "\n" "," | cut -d"," -f1,2 | sed 's/,/,\ /'

# add info
r.support map=${BAND}_Radiance \
title="" \
units="W / sq.m. / μm / ster" \
description="Top-of-Atmosphere `echo ${BAND}` band spectral Radiance [W/m^2/sr/μm]" \
source1='"Radiometric Use of WorldView-2 Imagery, Technical Note (2010)", by Todd Updike & Chris Comp'

# Esun
eval BAND_Esun="${BAND}_Esun"

# some echo
echo "* Parameters required for the conversion: Esun= ${!BAND_Esun}, ESD= ${ESD}, ${SZA}"

# calculate ToAR
echo "* Converting ${BAND}_Radiance to Top of Atmosphere Reflectance..." | tr -d "\n"
r.mapcalc "${BAND}_ToAR = \
( ${PI} * ${BAND}_Radiance * ${ESD}^2 ) / ( ${!BAND_Esun} * cos(${SZA}) )"

# report range
echo "* Reporting range of the ${BAND} spectral radiance: " | tr -d "\n"
r.info -r ${BAND}_ToAR | tr "\n" "," | cut -d"," -f1,2 | sed 's/,/,\ /'

# add some metadata
r.support map=${BAND}_ToAR \
title="echo ${BAND} band (Top of Atmosphere Reflectance)" \
units="Unitless planetary reflectance" \
description="Top of Atmosphere `echo ${BAND}` band spectral Reflectance (unitless)" \
source1='"Radiometric Use of WorldView-2 Imagery, Technical Note (2010)", by Todd Updike & Chris Comp, Digital Globe' \
source2="Digital Globe" \
history="PI=3.14159265358; K=${!K_BAND}; Bandwidth=${!BAND_Width}; ESD=${ESD}; Esun=${!BAND_Esun}; SZA=${SZA}"

echo -e "\n"

done
</source>

=== Atmospheric correction ===

See [[Atmospheric correction]]

== Post-Processing ==

Having beforehand satellite image data expressed in physical quantities (radiance or reflectance) is preferred to follow-up with digital image analysis techniques. A few common post-processing practices are Contrast-Enhancement, Pan-Sharpening and creating Pseudo- or True-Color Composites. Other well known enhancing manipualtions to support the analyses of satellite imagery, include deriving Vegetation Indices, transforming multi-spectral data based on [[Principal_Components_Analysis | PCA]] and Segmenting images.

=== Contrast Enhancement ===

''ToAdd''

=== Pan-Sharpening ===

[http://en.wikipedia.org/wiki/Pansharpened_image Pan-Sharpening] / [http://en.wikipedia.org/wiki/Image_fusion Fusion] is the process of merging high-resolution panchromatic and lower resolution multi-spectral imagery. [http://grass.osgeo.org/grass70/ GRASS 7] holds a dedicated pan-sharpening module, {{cmd|i.pansharpen}} which features three techniques for sharpening, namely the [http://wiki.awf.forst.uni-goettingen.de/wiki/index.php/Brovey_Transformation Brovey transformation], the classical IHS method and one that is based on [[Principal Components Analysis]] (PCA).
Another algorithm deriving excellent detail and a realistic representation of original multispectral scene colors, is the High-Pass Filter Addition (HPFA) technique. It is implemented via the {{AddonSrc|imagery|i.fusion.hpf|version=7}} add-on (for GRASS 6, please refer to a bash shell script https://github.com/NikosAlexandris/i.fusion.hpf.sh which is, however, unmaintained).

One approach inside GRASS-GIS to get an acceptable color-balanced composite image after Pan-sharpening 11-bit IKONOS spectral bands comprises the following steps

<ul>
# rescale the 11-bit WorldView spectral bands to 8-bit ranging in [0, 255] ({{cmd|r.rescale}})
# pan-sharpen with any of the featured methods (Brovey, IHS, PCA) ({{cmd|i.pansharpen}})
# color-balance by using the {{cmd|i.landsat.rgb}} module or manually adjusting the color tables of the bands of interest
# create a composite image by using the {{cmd|r.compose}} module
</ul>

''ToAdd''

==== Automatising Sharpening & RGB Composition ====

An ''experimental'' bash script to automatise the sharpening/fusion and RGB composition process is demonstrated here. The script uses the Spectral Reflectance values (double precision values), gained from previous steps (see above), and produces Pan-sharpened images by applying all of the three methods that {{cmd|i.pansharpen}} offers. In addition, it attempts to produce True-Color composite images, without and with re-balancing the color tables by using the {{cmd|i.landsat.rgb}} module.

Initial tests indicate that at least one of the three sharpening methods, combined with re-balancing, produces very clear and balanced True-Color composites. Note, there is no 100% confidence that it will produce nice looking True-Color composites. Some of the methods might simply work, others might deliver fancy-colored images. As usual, some manual action might be required to get the re-balancing to work as best as possible.

The script can be expanded in terms of using more inputs by utilising bash's positional parameters. However, care has to be taken to alter the instruction that concerns the conversion from double precision values to 8-bit integers.

<source lang="bash">
...
</source>

=== Color Composites ===

''ToAdd''

== Vegetation Indices ==

''ToAdd''

== PCA ==

''ToAdd''

== WordView Image classification ==

See [[Image classification]]

== References / Sources ==

* ...
* [http://landsathandbook.gsfc.nasa.gov/data_prod/prog_sect11_3.html Landsat7 Science Data Users Handbook, Chapter 11, Section 3]
* [http://igett.delmar.edu/Resources/Remote%20Sensing%20Technology%20Training/Calculation-DN_to_Reflectance_Irish_20June08.pdf Calibrated Landsat Digital Number (DN) to Top of Atmosphere (TOA) Reflectance Conversion, by Richard Irish]
* from [http://landsat.usgs.gov/tools_access_all_faqs.php FAQs about the Landsat Missions]
** [http://landsat.usgs.gov/how_is_radiance_calculated.php How is radiance calculated?]
** [http://landsat.usgs.gov/at_sensor_reflectance_calculated.php How is at-sensor reflectance calculated?]

== See also ==

* GRASS-Wiki page about [http://grasswiki.osgeo.org/wiki/Image_processing Image Processing]

''ToAdd''

[[Category: Documentation]]
[[Category: FAQ]]
[[Category: WorldView]]
[[Category: High Resolution]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Image processing]]

IKONOS

2016-08-14T05:05:13Z

⚠️NikosA: updated link to i.ikonos.toar

= Overview =

IKONOS is a commercial earth observation satellite. Details about the sensor are provided at Digital Globe's [http://www.digitalglobe.com/sites/default/files/DG_IKONOS_DS.pdf IKONOS Data Sheet]. [http://en.wikipedia.org/wiki/Ikonos Wikipedia's article on IKONOS] provides a nice overview as well.

== Availability (Sample Data) ==

* Search for commercial satellite image providers in the internet.
* [http://www.landcover.org/ The Global Land Cover Facility (GLCF)] provides [ftp://ftp.glcf.umd.edu/glcf/China_earthquake_May_2008/IKONOS/ four openly available IKONOS scenes of western Sichuan].
* ISPRS provides a small [http://www.isprs.org/data/ikonos/ IKONOS data set], fragments from a Panchromatic image as well as from a Stereo product.

== Modules overview ==

=== Outside of GRASS ===

Satellite imagery can be managed and, at some extent, pre-processed with various [http://www.gdal.org/gdal_utilities.html GDAL utilities].

* [http://www.gdal.org/gdalmanage.html gdalmanage]
* [http://www.gdal.org/gdalinfo.html gdalinfo]
* [http://www.gdal.org/gdal_translate.html gdal_translate]
* [http://www.gdal.org/gdalbuildvrt.html gdalbuildvrt]
* [http://www.gdal.org/gdalwarp.html gdalwarp]

=== Inside GRASS GIS ===

GRASS GIS features a complete set of modules and various add-ons enabling pre- and post-processing of IKONOS satellite imagery. The following lists offer an overview of related modules and add-ons.

* {{cmd|r.in.gdal}}
* {{cmd|r.mapcalc}}
* {{cmd|r.colors}}
* {{cmd|i.landsat.rgb}}
* {{AddonSrc|imagery|i.fusion.hpf|version=7}}
* {{cmd|i.pansharpen}}
* [https://github.com/NikosAlexandris/i.ikonos.toar i.ikonos.toar]
* {{cmd|i.vi}}
* {{cmd|i.segment}}

== Pre-Processing ==

=== Overview ===

Typically, multispectral satellite data are converted into physical quantities such as ''Radiance'' or ''Reflectance'' before they are subjected in multispectral analysis techniques (image interpretation, band arithmetic, vegetation indices, matrix transformations, etc.). The latter can be differentiated in ''Top of Atmosphere Reflectance'' (ToAR) which does not account for atmospheric effects (absorption or scattering) and in ''Top of Canopy Reflectance'' (ToCR) which introduces a "correction" for atmospheric effects.

In order to derive Reflectance values, likewise as with remotely sensed data acquired by other sensors, IKONOS raw image digital numbers (DNs) need to be converted to ''at-sensor spectral Radiance'' values. At-sensor spectral Radiance values are an important input for the equation to derive Reflectance values. Note, ''Spectal Radiance'' is the measure of the quantity of radiation that hits the sensor and typically expressed in <math>\frac{W}{m^2*sr*nm}</math>, that is ''watts'' per ''unit source area'', per ''unit solid angle'', and per ''unit wavelength''.

<ul>
* Converting DNs to at-sensor Radiance can be done by using the following equation:

<math>L\lambda = \frac{10^4 * DN\lambda}{CalCoef\lambda * Bandwidth\lambda}</math>

* Converting to Top of Atmosphere Reflectance, also referred to as Planetary Reflectance, can be done by using the following equation:

<math>\rho_p = \frac{\pi * L\lambda * d^2}{ESUN\lambda * cos(\Theta_S)}</math>

 where:

<ul>
* <math>\rho</math> - Unitless Planetary Reflectance
* <math>\pi</math> - mathematical constant (3.14159265358)
* <math>L\lambda</math> spectral Radiance at the sensor's aperture, from equation... '''''ToADD'''''
* <math>d</math> - Earth-Sun distance in astronomical units, interpolated values
* <math>Esun</math> - Mean solar exoatmospheric irradiance(s) (W/m2/μm), interpolated values
* <math>cos(\theta_s)</math> - {{wikipedia||Solar_zenith_angle}} from the image acquisition's metadata
</ul>

</ul>

=== Importing data ===

==== File Formats & Metadata ====

''ToAdd''

==== Importing ====

In the following example the publicly available IKONOS acqisition [ftp://ftp.glcf.umd.edu/glcf/China_earthquake_May_2008/IKONOS/po_58204_0000000.20001116.China-Sichuan/ po_58204_0000000.20001116.China-Sichuan] is used.

''ToAdd''
<ul>
* Location creation based on georeferenced data
</ul>

Once inside a Location that is defined by the spatial reference system in which the bands of interest are projected, they can be imported with the {{cmd|r.in.gdal}} module.

For example, GeoTIFF files can be imported by looping {{cmd|r.in.gdal}} over all of them

<source lang="bash">
for TIF in `echo *.tif`; do r.in.gdal in=${TIF} out=${TIF%%.*}; done
</source>

To keep the raw material untouched, we create another Mapset inside the same Location and copy over the bands giving, ''optionally'' at the same time, a new name for each.

<source lang="bash">
g.mapset -c pre-processing
g.copy rast=po_58204_blu_0000000,Blue_DNs
g.copy rast=po_58204_grn_0000000,Green_DNs
g.copy rast=po_58204_red_0000000,Red_DNs
g.copy rast=po_58204_nir_0000000,NIR_DNs
g.copy rast=po_58204_pan_0000000,Pan_DNs
</source>

=== Deriving Physical Quantities ===

Conversions are implemented in a GRASS module available at https://github.com/NikosAlexandris/i.ikonos.toar.

''Details''

Converting Digital Numbers to Radiance/Reflectance requires knowledge about the sensor's specific spectral band parameters. Those are, as extracted from the document ''IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance'', by Martin Taylor (see references):

<source lang=bash>
Pan_CalCoef=161 ; Pan_Width=403 ; Pan_Esun=1375.8
Blue_CalCoef=728 ; Blue_Width=71.3 ; Blue_Esun=1930.9
Green_CalCoef=720 ; Green_Width=88.6 ; Green_Esun=1854.8
Red_CalCoef=949 ; Red_Width=65.8 ; Red_Esun=1556.5
NIR_CalCoef=843 ; NIR_Width=95.4 ; NIR_Esun=1156.9
</source>

The following examples exemplify the conversion of raw Blue band digital numbers into Radiance and Reflectance.

==== Spectral Radiance ====

Converting a Blue Band (Digital Numbers) in to Spectral at-sensor Radiance in the ''correct'' units to be further used for the conversion in to unitless Reflectance:

<source lang="bash">
# set the region
g.region rast=Blue_DNs -p

# convert DNs to spectral Radiance values
r.mapcalc "Blue_Radiance = ( (10000 * IKONOS_Blue_DNs) / (728 * 71.3) )"
</source>

==== Planetary Reflectance ====

The equation to derive Reflectance values incorporates in addition:

<ul>
* The mathematical constant {{wikipedia|Pi}} <source lang="bash" enclose=none>PI=3.14159265358</source>.
* The Earth-Sun distance in astronomical units which depends on the acquisition's day of year (DOY -- also referred to as Julian day, {{wikipedia|Ordinal_date}}) and can be retrieved from the following spreadsheet <http://landsathandbook.gsfc.nasa.gov/excel_docs/d.xls>.
* The mean solar exoatmospheric irradiance in <math>\frac{W}{m^2*\mu m}</math>. See 3rd column of (interplated) values given above.
* The cosine of the ''Solar Zenith Angle'' (SZA) at the time of the acquisition. The SZA can be calculated from its complementary ''Solar Elevation Angle'' (SEA) given in the image acquisition's metadata.
</ul>

In the following example we accept as the acquisition's <source lang="bash" enclose=none>DOY=166</source> and <source lang="bash" enclose=none>SEA=52.78880</source>. Hence, we get the Earth-Sun distance <source lang="bash" enclose=none>ESD=1.0157675</source> and <source lang="bash" enclose=none>SZA = 37.21120 deg</source>.

Converting the in-Blue spectral band at-sensor Radiance in to Planerary Reflectance:

<source lang="bash" enclose=none>PI=3.14159265358; ESD=1.0157675; BAND_Esun=1930.9; SZA=37.21120</source>
{{cmd|r.mapcalc}} <source lang="bash" enclose=none>"Blue_Reflectance = ( ${PI} * Blue_Radiance * ${ESD}^2 ) / ( ${BAND_Esun} * cos(${SZA}) )"</source>

==== Automatising Conversions ====

The conversion process can be scripted to avoid repeating the same steps for each band separately.

===== Python =====

A custom python script, performing the operations of interest, might be like [https://github.com/NikosAlexandris/i.ikonos.toar i.ikonos.toar (for GRASS 7.x)]

===== Bash =====
In bash, such a script might be as the following example. '''Note,''' however, in this example script constants, band parameters and acquisition related metadata are hard-coded!

<source lang="bash">
#!/bin/bash

Pan_CalCoef=161 ; Pan_Width=403 ; Pan_Esun=1375.8
Blue_CalCoef=728 ; Blue_Width=71.3 ; Blue_Esun=1930.9
Green_CalCoef=720 ; Green_Width=88.6 ; Green_Esun=1854.8
Red_CalCoef=949 ; Red_Width=65.8 ; Red_Esun=1556.5
NIR_CalCoef=843 ; NIR_Width=95.4 ; NIR_Esun=1156.9

# set constants, band parameters and acquisition related metadata here!

# Pi, first 11 decimals
PI=3.14159265358

# Acquisition's Day of Year and estimated Earth-Sun Distance
DOY=166; ESD=1.0157675

# Sun Zenith Angle based on the acquisition's Sun Elevation Angle
SEA=52.78880; SZA=$(echo "90 - ${SEA}" | bc )

# bands to process
Spectral_Bands="Pan Blue Green Red NIR"

# loop over all bands
for BAND in ${Spectral_Bands}; do

echo "Processing the ${BAND} spectral band"

# set region
g.region rast=${BAND}_DNs

# set band parameters as variables
eval BAND_CalCoef="${BAND}_CalCoef"
eval BAND_Width="${BAND}_Width";
echo "Band Parameters set to CalCoef=${!BAND_CalCoef}, Bandwidth=${!BAND_Width}"

# conversion to Radiance
r.mapcalc "${BAND}_Radiance = ( ( 10^4 * ${BAND}_DNs ) / ( ${!BAND_CalCoef} * ${!BAND_Width} ) )"

# add info
r.support map=${BAND}_Radiance \
title="" \
units="W / m2 / μm / ster" \
description="At-sensor `echo ${BAND}` band spectral Radiance (W/m2/μm/sr)" \
source1='"IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance", by Martin Taylor, Geoeye'

# set Earth-Sun distance
eval BAND_Esun="${BAND}_Esun"; echo "Using Esun=${!BAND_Esun}"

# calculate ToAR
r.mapcalc "${BAND}_ToAR = \
( ${PI} * ${BAND}_Radiance * ${ESD}^2 ) / ( ${!BAND_Esun} * cos(${SZA}) )"

# add some metadata
r.support map=${BAND}_ToAR \
title="echo ${BAND} band (Top of Atmosphere Reflectance)" \
units="Unitless" \
description="Top of Atmosphere `echo ${BAND}` band spectral Reflectance (unitless)" \
source1='"IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance", by Martin Taylor, Geoeye' \
source2="e.g., the Image Provider!" \
history="PI=3.14159265358; ESD=1.0157675; BAND_Esun=1930.9; SZA=37.21120"

done
</source>

=== Atmospheric correction ===

See [[Atmospheric correction]]

== Post-Processing ==

Having beforehand satellite image data expressed in physical quantities (radiance or reflectance) is preferred to follow-up with digital image analysis techniques. A few common post-processing practices are Contrast-Enhancement, Pan-Sharpening and creating Pseudo- or True-Color Composites. Other well known enhancing manipualtions to support the analyses of satellite imagery, include deriving Vegetation Indices, transforming multi-spectral data based on [[Principal_Components_Analysis | PCA]] and Segmenting images.

=== Contrast Enhancement ===

''ToAdd''

=== Pan-Sharpening ===

[http://en.wikipedia.org/wiki/Pansharpened_image Pan-Sharpening] / [http://en.wikipedia.org/wiki/Image_fusion Fusion] is the process of merging high-resolution panchromatic and lower resolution multi-spectral imagery. [http://grass.osgeo.org/grass70/ GRASS 7] holds a dedicated pan-sharpening module, {{cmd|i.pansharpen}} which features three techniques for sharpening, namely the [http://wiki.awf.forst.uni-goettingen.de/wiki/index.php/Brovey_Transformation Brovey transformation], the classical IHS method and one that is based on [[Principal Components Analysis]] (PCA). Another algorithm deriving excellent detail and a realistic representation of original multispectral scene colors, is the High-Pass Filter Addition (HPFA) technique. It is available through the add-on {{AddonSrc|imagery|i.fusion.hpf|version=7}} (for GRASS 6, please refer to a bash shell script https://github.com/NikosAlexandris/i.fusion.hpf.sh which is, however, unmaintained).

One approach inside GRASS-GIS to get an acceptable color-balanced composite image after Pan-sharpening 11-bit IKONOS spectral bands comprises the following steps

<ul>
# rescale the 11-bit IKONOS spectral bands to 8-bit ranging in [0, 255] ({{cmd|r.rescale}})
# pan-sharpen with any of the featured methods (Brovey, IHS, PCA) ({{cmd|i.pansharpen}})
# color-balance by using the {{cmd|i.landsat.rgb}} module or manually adjusting the color tables of the bands of interest
# create a composite image by using the {{cmd|r.compose}} module
</ul>

==== Example Instructions for {{cmd|i.pansharpen}} ====

{{cmd|i.pansharpen}} works fine with 8-bit raster maps as an input. If the data to be processed are out of this range, that is out of <math>[0, 255]</math>, they can be rescaled to fit into this range by using GRASS' {{cmd|r.rescale}} module.

Given an IKONOS set of 11-bit spectral bands (Blue, Green, Red, NIR and Pan) ranging between <math>[0, 2047]</math>, and then querying for example the Blue band
<source lang="bash" enclose=none>r.info Blue_DNs -r</source>, would return

<source lang="bash">
min=0
max=2047
</source>

===== Rescaling single bands =====

Rescaling the Blue band to range between `[0, 255]`
<source lang="bash">
r.rescale in=Blue_DNs out=Blue_DNs_255 from=0,2047 to=0,255
</source>

The same step applies to both the rest of the multi-spectral bands and the Panchromatic band of interest. If working under Bash, To repeat the same command (given, it was the last on used in the GRASS' terminal), one can isntruct

<source lang="bash">
!!:gs/Blue/Green
</source>

This would substitute everywhere found on the last command the string "Blue" with the string "Green" and re-execute it. The same can be done for the rest of the bands, namely Red, NIR and Pan. Some attention is required to instruct substitution of the ''last'' used string.

===== Rescaling all bands at once =====

Of course, one can always use a ''for'' loop

<source lang="bash">
for DN in `g.mlist rast pat=*DNs`; do r.rescale in=${DN} out=${DN}_255 from=0,2047 to=0,255; done
</source>

===== Pan-Sharpening =====

As usual when working with GRASS, it is required to set the region of interest, i.e. <source lang="bash" enclose=none>g.region rast=Blue_DNs_255</source> to match the extent of the band(s) or else. The resolution itself is taken care in this particular case by the module and the resulting pan-sharpened raster maps will of the same high(er) resolution as the Panchromatic band.

An example command for an IHS-based Pan-Sharpening action might look like

<source lang="bash">
i.pansharpen pan=Pan_DNs_255 ms1=Blue_DNs_255 ms2=Green_DNs_255 ms3=Red_DNs_255 output=sharptest255 sharpen=ihs
</source>

===== Color Re-Balancnig =====

After the process completion, the module outputs

<source lang="bash">
...
The following pan-sharpened output maps have been generated:
sharptest255_red
sharptest255_green
sharptest255_blue

To visualize output, run: g.region -p rast=sharptest255.red
d.rgb r=sharptest255_red g=sharptest255_green b=sharptest255_blue

If desired, combine channels into a single RGB map with 'r.composite'.
</source>

Normally it should be enough to re-balance the colors after the pan-sharpening action by using for example the {{cmd|i.landsat.rgb}} module or manual adjustment of each of the three bands that would compose an RGB image.

<source lang="bash">
i.landsat.rgb r=sharptest255_red g=sharptest255_green b=sharptest255_blue -p
</source>

===== Automatising Sharpening & RGB Composition =====

An ''experimental'' bash script to automatise the sharpening/fusion and RGB composition process is demonstrated here. The script uses the Spectral Reflectance values (double precision values), gained from previous steps (see above), and produces Pan-sharpened images by applying all of the three methods that {{addon|i.pansharpen}} offers. In addition, it attempts to produce True-Color composite images, without and with re-balancing the color tables by using the {{cmd|i.landsat.rgb}} module.

Initial tests indicate that at least one of the three sharpening methods, combined with re-balancing, produces very clear and balanced True-Color composites. Note, there is no 100% confidence that it will produce nice looking True-Color composites. Some of the methods might simply work, others might deliver fancy-colored images. As usual, some manual action might be required to get the re-balancing to work as best as possible.

The script can be expanded in terms of using more inputs by utilising bash's positional parameters. However, care has to be taken to alter the instruction that concerns the conversion from double precision values to 8-bit integers.

<source lang="bash">
#!/bash/sh

# use in G7 only :-)

echo "Rescaling all images to 8-bit"
# convert to 8-bit first :-!
for Method_Input in \
"Pan ToAR" \
"Blue ToAR" \
"Green ToAR" \
"Red ToAR" \
"NIR ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo -e "\n" #echo $1 $2

# input BAND is...
eval BAND="${1}_${2}"

# set region
g.region rast=${BAND}

# integerise
r.mapcalc --v \
"${BAND}_integerised = round( 1000000 * ${BAND} )" # how many decimals?

# integerised BAND is...
eval BAND="${1}_${2}_integerised"

# get ${min} and ${max}
eval `r.info -r ${BAND}`

#
r.rescale in=${1}_${2}_integerised out=${1}_${2}_255 from=${min},${max} to=0,255

done ########################################################################
echo "Rescaling done!"

# pan-sharpen High Resolution imagery ---------------------------------------
# needs setting some naming conventions
for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# some echo
echo "Pan-Sharpening the <${2}> images based on the <${1}> method"

i.pansharpen \
sharpen=${1} \
pan=Pan_${2}_255 \
ms1=Blue_${2}_255 \
ms2=Green_${2}_255 \
ms3=Red_${2}_255 \
output=sharp_${1}_${2}_255

done ########################################################################

# compose BGRs --------------------------------------------------------------

# loop over Methods and Input Types, compose RGBs
for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# region
g.region rast=sharp_"${1}"_"${2}"_255_blue

# compose
r.composite --o \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue \
out=rgb_sharpened_"${1}"_"${2}"_255

done #######################################################################

# re-balance & re-compose colors --------------------------------------------

for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# re-balance
i.landsat.rgb -p \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue

# region -- Really necessary, again?
g.region rast=sharp_"${1}"_"${2}"_255_blue

# re-compose
r.composite --o \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue \
out=rgb_sharpened_"${1}"_"${2}"_255_rebalanced

done ########################################################################
</source>

==== Pan-Sharpening based on the HPFA technique ====

The technique, implemented via the GRASS add-on {{AddonSrc|imagery|i.fusion.hpf|version=7}}, as indicated above, involves a convolution using a High Pass Filter (HPF) on the high resolution data, then combining this with the lower resolution multispectral data.

''Source: "Optimizing the High-Pass Filter Addition Technique for Image Fusion", Ute G. Gangkofner, Pushkar S. Pradhan, and Derrold W. Holcomb (2008)''

The algorithm's steps are:

# Computing the ratio of the low (Multi-Spectral) to the high (Panchromatic) resolution
# High Pass Filtering the Panchromatic Image
# Resampling the Multi-Spectral image to the higher resolution
# Adding a weighted High-Pass-Filtered image to the upsampled Multi-Spectral image
# Optionally, matching the histogram of the Pan-Sharpened image to the one of the original Multi-Spectral image

=== Color Composites ===

''ToAdd''

== Vegetation Indices ==

''ToAdd''

== PCA ==

''ToAdd''

== IKONOS Image classification ==

See [[Image classification]]

== References / Sources ==

* [http://www.apollomapping.com/wp-content/user_uploads/2011/09/IKONOS_Esun_Calculations.pdf IKONOS Planetary Reflectance and Me an Solar Exoatmospheric Irradiance, by Martin Taylor]
* [http://web.unicen.edu.ar/crecic/docs/radrefl.pdf Ikonos DN Value Conversion to Planetary Reflectance Values, by David Fleming]
* [http://landsathandbook.gsfc.nasa.gov/data_prod/prog_sect11_3.html Landsat7 Science Data Users Handbook, Chapter 11, Section 3]
* [http://igett.delmar.edu/Resources/Remote%20Sensing%20Technology%20Training/Calculation-DN_to_Reflectance_Irish_20June08.pdf Calibrated Landsat Digital Number (DN) to Top of Atmosphere (TOA) Reflectance Conversion, by Richard Irish]
* from [http://landsat.usgs.gov/tools_access_all_faqs.php FAQs about the Landsat Missions]
** [http://landsat.usgs.gov/how_is_radiance_calculated.php How is radiance calculated?]
** [http://landsat.usgs.gov/at_sensor_reflectance_calculated.php How is at-sensor reflectance calculated?]

== See also ==

* GRASS-Wiki page about [http://grasswiki.osgeo.org/wiki/Image_processing Image Processing]

''ToAdd''

[[Category: Documentation]]
[[Category: FAQ]]
[[Category: IKONOS]]
[[Category: High Resolution]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Image processing]]

IKONOS

2016-08-14T05:04:25Z

⚠️NikosA: updated list of related GRASS modules

= Overview =

IKONOS is a commercial earth observation satellite. Details about the sensor are provided at Digital Globe's [http://www.digitalglobe.com/sites/default/files/DG_IKONOS_DS.pdf IKONOS Data Sheet]. [http://en.wikipedia.org/wiki/Ikonos Wikipedia's article on IKONOS] provides a nice overview as well.

== Availability (Sample Data) ==

* Search for commercial satellite image providers in the internet.
* [http://www.landcover.org/ The Global Land Cover Facility (GLCF)] provides [ftp://ftp.glcf.umd.edu/glcf/China_earthquake_May_2008/IKONOS/ four openly available IKONOS scenes of western Sichuan].
* ISPRS provides a small [http://www.isprs.org/data/ikonos/ IKONOS data set], fragments from a Panchromatic image as well as from a Stereo product.

== Modules overview ==

=== Outside of GRASS ===

Satellite imagery can be managed and, at some extent, pre-processed with various [http://www.gdal.org/gdal_utilities.html GDAL utilities].

* [http://www.gdal.org/gdalmanage.html gdalmanage]
* [http://www.gdal.org/gdalinfo.html gdalinfo]
* [http://www.gdal.org/gdal_translate.html gdal_translate]
* [http://www.gdal.org/gdalbuildvrt.html gdalbuildvrt]
* [http://www.gdal.org/gdalwarp.html gdalwarp]

=== Inside GRASS GIS ===

GRASS GIS features a complete set of modules and various add-ons enabling pre- and post-processing of IKONOS satellite imagery. The following lists offer an overview of related modules and add-ons.

* {{cmd|r.in.gdal}}
* {{cmd|r.mapcalc}}
* {{cmd|r.colors}}
* {{cmd|i.landsat.rgb}}
* {{AddonSrc|imagery|i.fusion.hpf|version=7}}
* {{cmd|i.pansharpen}}
* https://github.com/NikosAlexandris/i.ikonos.toar
* {{cmd|i.vi}}
* {{cmd|i.segment}}

== Pre-Processing ==

=== Overview ===

Typically, multispectral satellite data are converted into physical quantities such as ''Radiance'' or ''Reflectance'' before they are subjected in multispectral analysis techniques (image interpretation, band arithmetic, vegetation indices, matrix transformations, etc.). The latter can be differentiated in ''Top of Atmosphere Reflectance'' (ToAR) which does not account for atmospheric effects (absorption or scattering) and in ''Top of Canopy Reflectance'' (ToCR) which introduces a "correction" for atmospheric effects.

In order to derive Reflectance values, likewise as with remotely sensed data acquired by other sensors, IKONOS raw image digital numbers (DNs) need to be converted to ''at-sensor spectral Radiance'' values. At-sensor spectral Radiance values are an important input for the equation to derive Reflectance values. Note, ''Spectal Radiance'' is the measure of the quantity of radiation that hits the sensor and typically expressed in <math>\frac{W}{m^2*sr*nm}</math>, that is ''watts'' per ''unit source area'', per ''unit solid angle'', and per ''unit wavelength''.

<ul>
* Converting DNs to at-sensor Radiance can be done by using the following equation:

<math>L\lambda = \frac{10^4 * DN\lambda}{CalCoef\lambda * Bandwidth\lambda}</math>

* Converting to Top of Atmosphere Reflectance, also referred to as Planetary Reflectance, can be done by using the following equation:

<math>\rho_p = \frac{\pi * L\lambda * d^2}{ESUN\lambda * cos(\Theta_S)}</math>

 where:

<ul>
* <math>\rho</math> - Unitless Planetary Reflectance
* <math>\pi</math> - mathematical constant (3.14159265358)
* <math>L\lambda</math> spectral Radiance at the sensor's aperture, from equation... '''''ToADD'''''
* <math>d</math> - Earth-Sun distance in astronomical units, interpolated values
* <math>Esun</math> - Mean solar exoatmospheric irradiance(s) (W/m2/μm), interpolated values
* <math>cos(\theta_s)</math> - {{wikipedia||Solar_zenith_angle}} from the image acquisition's metadata
</ul>

</ul>

=== Importing data ===

==== File Formats & Metadata ====

''ToAdd''

==== Importing ====

In the following example the publicly available IKONOS acqisition [ftp://ftp.glcf.umd.edu/glcf/China_earthquake_May_2008/IKONOS/po_58204_0000000.20001116.China-Sichuan/ po_58204_0000000.20001116.China-Sichuan] is used.

''ToAdd''
<ul>
* Location creation based on georeferenced data
</ul>

Once inside a Location that is defined by the spatial reference system in which the bands of interest are projected, they can be imported with the {{cmd|r.in.gdal}} module.

For example, GeoTIFF files can be imported by looping {{cmd|r.in.gdal}} over all of them

<source lang="bash">
for TIF in `echo *.tif`; do r.in.gdal in=${TIF} out=${TIF%%.*}; done
</source>

To keep the raw material untouched, we create another Mapset inside the same Location and copy over the bands giving, ''optionally'' at the same time, a new name for each.

<source lang="bash">
g.mapset -c pre-processing
g.copy rast=po_58204_blu_0000000,Blue_DNs
g.copy rast=po_58204_grn_0000000,Green_DNs
g.copy rast=po_58204_red_0000000,Red_DNs
g.copy rast=po_58204_nir_0000000,NIR_DNs
g.copy rast=po_58204_pan_0000000,Pan_DNs
</source>

=== Deriving Physical Quantities ===

Conversions are implemented in a GRASS module available at https://github.com/NikosAlexandris/i.ikonos.toar.

''Details''

Converting Digital Numbers to Radiance/Reflectance requires knowledge about the sensor's specific spectral band parameters. Those are, as extracted from the document ''IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance'', by Martin Taylor (see references):

<source lang=bash>
Pan_CalCoef=161 ; Pan_Width=403 ; Pan_Esun=1375.8
Blue_CalCoef=728 ; Blue_Width=71.3 ; Blue_Esun=1930.9
Green_CalCoef=720 ; Green_Width=88.6 ; Green_Esun=1854.8
Red_CalCoef=949 ; Red_Width=65.8 ; Red_Esun=1556.5
NIR_CalCoef=843 ; NIR_Width=95.4 ; NIR_Esun=1156.9
</source>

The following examples exemplify the conversion of raw Blue band digital numbers into Radiance and Reflectance.

==== Spectral Radiance ====

Converting a Blue Band (Digital Numbers) in to Spectral at-sensor Radiance in the ''correct'' units to be further used for the conversion in to unitless Reflectance:

<source lang="bash">
# set the region
g.region rast=Blue_DNs -p

# convert DNs to spectral Radiance values
r.mapcalc "Blue_Radiance = ( (10000 * IKONOS_Blue_DNs) / (728 * 71.3) )"
</source>

==== Planetary Reflectance ====

The equation to derive Reflectance values incorporates in addition:

<ul>
* The mathematical constant {{wikipedia|Pi}} <source lang="bash" enclose=none>PI=3.14159265358</source>.
* The Earth-Sun distance in astronomical units which depends on the acquisition's day of year (DOY -- also referred to as Julian day, {{wikipedia|Ordinal_date}}) and can be retrieved from the following spreadsheet <http://landsathandbook.gsfc.nasa.gov/excel_docs/d.xls>.
* The mean solar exoatmospheric irradiance in <math>\frac{W}{m^2*\mu m}</math>. See 3rd column of (interplated) values given above.
* The cosine of the ''Solar Zenith Angle'' (SZA) at the time of the acquisition. The SZA can be calculated from its complementary ''Solar Elevation Angle'' (SEA) given in the image acquisition's metadata.
</ul>

In the following example we accept as the acquisition's <source lang="bash" enclose=none>DOY=166</source> and <source lang="bash" enclose=none>SEA=52.78880</source>. Hence, we get the Earth-Sun distance <source lang="bash" enclose=none>ESD=1.0157675</source> and <source lang="bash" enclose=none>SZA = 37.21120 deg</source>.

Converting the in-Blue spectral band at-sensor Radiance in to Planerary Reflectance:

<source lang="bash" enclose=none>PI=3.14159265358; ESD=1.0157675; BAND_Esun=1930.9; SZA=37.21120</source>
{{cmd|r.mapcalc}} <source lang="bash" enclose=none>"Blue_Reflectance = ( ${PI} * Blue_Radiance * ${ESD}^2 ) / ( ${BAND_Esun} * cos(${SZA}) )"</source>

==== Automatising Conversions ====

The conversion process can be scripted to avoid repeating the same steps for each band separately.

===== Python =====

A custom python script, performing the operations of interest, might be like [https://github.com/NikosAlexandris/i.ikonos.toar i.ikonos.toar (for GRASS 7.x)]

===== Bash =====
In bash, such a script might be as the following example. '''Note,''' however, in this example script constants, band parameters and acquisition related metadata are hard-coded!

<source lang="bash">
#!/bin/bash

Pan_CalCoef=161 ; Pan_Width=403 ; Pan_Esun=1375.8
Blue_CalCoef=728 ; Blue_Width=71.3 ; Blue_Esun=1930.9
Green_CalCoef=720 ; Green_Width=88.6 ; Green_Esun=1854.8
Red_CalCoef=949 ; Red_Width=65.8 ; Red_Esun=1556.5
NIR_CalCoef=843 ; NIR_Width=95.4 ; NIR_Esun=1156.9

# set constants, band parameters and acquisition related metadata here!

# Pi, first 11 decimals
PI=3.14159265358

# Acquisition's Day of Year and estimated Earth-Sun Distance
DOY=166; ESD=1.0157675

# Sun Zenith Angle based on the acquisition's Sun Elevation Angle
SEA=52.78880; SZA=$(echo "90 - ${SEA}" | bc )

# bands to process
Spectral_Bands="Pan Blue Green Red NIR"

# loop over all bands
for BAND in ${Spectral_Bands}; do

echo "Processing the ${BAND} spectral band"

# set region
g.region rast=${BAND}_DNs

# set band parameters as variables
eval BAND_CalCoef="${BAND}_CalCoef"
eval BAND_Width="${BAND}_Width";
echo "Band Parameters set to CalCoef=${!BAND_CalCoef}, Bandwidth=${!BAND_Width}"

# conversion to Radiance
r.mapcalc "${BAND}_Radiance = ( ( 10^4 * ${BAND}_DNs ) / ( ${!BAND_CalCoef} * ${!BAND_Width} ) )"

# add info
r.support map=${BAND}_Radiance \
title="" \
units="W / m2 / μm / ster" \
description="At-sensor `echo ${BAND}` band spectral Radiance (W/m2/μm/sr)" \
source1='"IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance", by Martin Taylor, Geoeye'

# set Earth-Sun distance
eval BAND_Esun="${BAND}_Esun"; echo "Using Esun=${!BAND_Esun}"

# calculate ToAR
r.mapcalc "${BAND}_ToAR = \
( ${PI} * ${BAND}_Radiance * ${ESD}^2 ) / ( ${!BAND_Esun} * cos(${SZA}) )"

# add some metadata
r.support map=${BAND}_ToAR \
title="echo ${BAND} band (Top of Atmosphere Reflectance)" \
units="Unitless" \
description="Top of Atmosphere `echo ${BAND}` band spectral Reflectance (unitless)" \
source1='"IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance", by Martin Taylor, Geoeye' \
source2="e.g., the Image Provider!" \
history="PI=3.14159265358; ESD=1.0157675; BAND_Esun=1930.9; SZA=37.21120"

done
</source>

=== Atmospheric correction ===

See [[Atmospheric correction]]

== Post-Processing ==

Having beforehand satellite image data expressed in physical quantities (radiance or reflectance) is preferred to follow-up with digital image analysis techniques. A few common post-processing practices are Contrast-Enhancement, Pan-Sharpening and creating Pseudo- or True-Color Composites. Other well known enhancing manipualtions to support the analyses of satellite imagery, include deriving Vegetation Indices, transforming multi-spectral data based on [[Principal_Components_Analysis | PCA]] and Segmenting images.

=== Contrast Enhancement ===

''ToAdd''

=== Pan-Sharpening ===

[http://en.wikipedia.org/wiki/Pansharpened_image Pan-Sharpening] / [http://en.wikipedia.org/wiki/Image_fusion Fusion] is the process of merging high-resolution panchromatic and lower resolution multi-spectral imagery. [http://grass.osgeo.org/grass70/ GRASS 7] holds a dedicated pan-sharpening module, {{cmd|i.pansharpen}} which features three techniques for sharpening, namely the [http://wiki.awf.forst.uni-goettingen.de/wiki/index.php/Brovey_Transformation Brovey transformation], the classical IHS method and one that is based on [[Principal Components Analysis]] (PCA). Another algorithm deriving excellent detail and a realistic representation of original multispectral scene colors, is the High-Pass Filter Addition (HPFA) technique. It is available through the add-on {{AddonSrc|imagery|i.fusion.hpf|version=7}} (for GRASS 6, please refer to a bash shell script https://github.com/NikosAlexandris/i.fusion.hpf.sh which is, however, unmaintained).

One approach inside GRASS-GIS to get an acceptable color-balanced composite image after Pan-sharpening 11-bit IKONOS spectral bands comprises the following steps

<ul>
# rescale the 11-bit IKONOS spectral bands to 8-bit ranging in [0, 255] ({{cmd|r.rescale}})
# pan-sharpen with any of the featured methods (Brovey, IHS, PCA) ({{cmd|i.pansharpen}})
# color-balance by using the {{cmd|i.landsat.rgb}} module or manually adjusting the color tables of the bands of interest
# create a composite image by using the {{cmd|r.compose}} module
</ul>

==== Example Instructions for {{cmd|i.pansharpen}} ====

{{cmd|i.pansharpen}} works fine with 8-bit raster maps as an input. If the data to be processed are out of this range, that is out of <math>[0, 255]</math>, they can be rescaled to fit into this range by using GRASS' {{cmd|r.rescale}} module.

Given an IKONOS set of 11-bit spectral bands (Blue, Green, Red, NIR and Pan) ranging between <math>[0, 2047]</math>, and then querying for example the Blue band
<source lang="bash" enclose=none>r.info Blue_DNs -r</source>, would return

<source lang="bash">
min=0
max=2047
</source>

===== Rescaling single bands =====

Rescaling the Blue band to range between `[0, 255]`
<source lang="bash">
r.rescale in=Blue_DNs out=Blue_DNs_255 from=0,2047 to=0,255
</source>

The same step applies to both the rest of the multi-spectral bands and the Panchromatic band of interest. If working under Bash, To repeat the same command (given, it was the last on used in the GRASS' terminal), one can isntruct

<source lang="bash">
!!:gs/Blue/Green
</source>

This would substitute everywhere found on the last command the string "Blue" with the string "Green" and re-execute it. The same can be done for the rest of the bands, namely Red, NIR and Pan. Some attention is required to instruct substitution of the ''last'' used string.

===== Rescaling all bands at once =====

Of course, one can always use a ''for'' loop

<source lang="bash">
for DN in `g.mlist rast pat=*DNs`; do r.rescale in=${DN} out=${DN}_255 from=0,2047 to=0,255; done
</source>

===== Pan-Sharpening =====

As usual when working with GRASS, it is required to set the region of interest, i.e. <source lang="bash" enclose=none>g.region rast=Blue_DNs_255</source> to match the extent of the band(s) or else. The resolution itself is taken care in this particular case by the module and the resulting pan-sharpened raster maps will of the same high(er) resolution as the Panchromatic band.

An example command for an IHS-based Pan-Sharpening action might look like

<source lang="bash">
i.pansharpen pan=Pan_DNs_255 ms1=Blue_DNs_255 ms2=Green_DNs_255 ms3=Red_DNs_255 output=sharptest255 sharpen=ihs
</source>

===== Color Re-Balancnig =====

After the process completion, the module outputs

<source lang="bash">
...
The following pan-sharpened output maps have been generated:
sharptest255_red
sharptest255_green
sharptest255_blue

To visualize output, run: g.region -p rast=sharptest255.red
d.rgb r=sharptest255_red g=sharptest255_green b=sharptest255_blue

If desired, combine channels into a single RGB map with 'r.composite'.
</source>

Normally it should be enough to re-balance the colors after the pan-sharpening action by using for example the {{cmd|i.landsat.rgb}} module or manual adjustment of each of the three bands that would compose an RGB image.

<source lang="bash">
i.landsat.rgb r=sharptest255_red g=sharptest255_green b=sharptest255_blue -p
</source>

===== Automatising Sharpening & RGB Composition =====

An ''experimental'' bash script to automatise the sharpening/fusion and RGB composition process is demonstrated here. The script uses the Spectral Reflectance values (double precision values), gained from previous steps (see above), and produces Pan-sharpened images by applying all of the three methods that {{addon|i.pansharpen}} offers. In addition, it attempts to produce True-Color composite images, without and with re-balancing the color tables by using the {{cmd|i.landsat.rgb}} module.

Initial tests indicate that at least one of the three sharpening methods, combined with re-balancing, produces very clear and balanced True-Color composites. Note, there is no 100% confidence that it will produce nice looking True-Color composites. Some of the methods might simply work, others might deliver fancy-colored images. As usual, some manual action might be required to get the re-balancing to work as best as possible.

The script can be expanded in terms of using more inputs by utilising bash's positional parameters. However, care has to be taken to alter the instruction that concerns the conversion from double precision values to 8-bit integers.

<source lang="bash">
#!/bash/sh

# use in G7 only :-)

echo "Rescaling all images to 8-bit"
# convert to 8-bit first :-!
for Method_Input in \
"Pan ToAR" \
"Blue ToAR" \
"Green ToAR" \
"Red ToAR" \
"NIR ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo -e "\n" #echo $1 $2

# input BAND is...
eval BAND="${1}_${2}"

# set region
g.region rast=${BAND}

# integerise
r.mapcalc --v \
"${BAND}_integerised = round( 1000000 * ${BAND} )" # how many decimals?

# integerised BAND is...
eval BAND="${1}_${2}_integerised"

# get ${min} and ${max}
eval `r.info -r ${BAND}`

#
r.rescale in=${1}_${2}_integerised out=${1}_${2}_255 from=${min},${max} to=0,255

done ########################################################################
echo "Rescaling done!"

# pan-sharpen High Resolution imagery ---------------------------------------
# needs setting some naming conventions
for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# some echo
echo "Pan-Sharpening the <${2}> images based on the <${1}> method"

i.pansharpen \
sharpen=${1} \
pan=Pan_${2}_255 \
ms1=Blue_${2}_255 \
ms2=Green_${2}_255 \
ms3=Red_${2}_255 \
output=sharp_${1}_${2}_255

done ########################################################################

# compose BGRs --------------------------------------------------------------

# loop over Methods and Input Types, compose RGBs
for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# region
g.region rast=sharp_"${1}"_"${2}"_255_blue

# compose
r.composite --o \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue \
out=rgb_sharpened_"${1}"_"${2}"_255

done #######################################################################

# re-balance & re-compose colors --------------------------------------------

for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# re-balance
i.landsat.rgb -p \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue

# region -- Really necessary, again?
g.region rast=sharp_"${1}"_"${2}"_255_blue

# re-compose
r.composite --o \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue \
out=rgb_sharpened_"${1}"_"${2}"_255_rebalanced

done ########################################################################
</source>

==== Pan-Sharpening based on the HPFA technique ====

The technique, implemented via the GRASS add-on {{AddonSrc|imagery|i.fusion.hpf|version=7}}, as indicated above, involves a convolution using a High Pass Filter (HPF) on the high resolution data, then combining this with the lower resolution multispectral data.

''Source: "Optimizing the High-Pass Filter Addition Technique for Image Fusion", Ute G. Gangkofner, Pushkar S. Pradhan, and Derrold W. Holcomb (2008)''

The algorithm's steps are:

# Computing the ratio of the low (Multi-Spectral) to the high (Panchromatic) resolution
# High Pass Filtering the Panchromatic Image
# Resampling the Multi-Spectral image to the higher resolution
# Adding a weighted High-Pass-Filtered image to the upsampled Multi-Spectral image
# Optionally, matching the histogram of the Pan-Sharpened image to the one of the original Multi-Spectral image

=== Color Composites ===

''ToAdd''

== Vegetation Indices ==

''ToAdd''

== PCA ==

''ToAdd''

== IKONOS Image classification ==

See [[Image classification]]

== References / Sources ==

* [http://www.apollomapping.com/wp-content/user_uploads/2011/09/IKONOS_Esun_Calculations.pdf IKONOS Planetary Reflectance and Me an Solar Exoatmospheric Irradiance, by Martin Taylor]
* [http://web.unicen.edu.ar/crecic/docs/radrefl.pdf Ikonos DN Value Conversion to Planetary Reflectance Values, by David Fleming]
* [http://landsathandbook.gsfc.nasa.gov/data_prod/prog_sect11_3.html Landsat7 Science Data Users Handbook, Chapter 11, Section 3]
* [http://igett.delmar.edu/Resources/Remote%20Sensing%20Technology%20Training/Calculation-DN_to_Reflectance_Irish_20June08.pdf Calibrated Landsat Digital Number (DN) to Top of Atmosphere (TOA) Reflectance Conversion, by Richard Irish]
* from [http://landsat.usgs.gov/tools_access_all_faqs.php FAQs about the Landsat Missions]
** [http://landsat.usgs.gov/how_is_radiance_calculated.php How is radiance calculated?]
** [http://landsat.usgs.gov/at_sensor_reflectance_calculated.php How is at-sensor reflectance calculated?]

== See also ==

* GRASS-Wiki page about [http://grasswiki.osgeo.org/wiki/Image_processing Image Processing]

''ToAdd''

[[Category: Documentation]]
[[Category: FAQ]]
[[Category: IKONOS]]
[[Category: High Resolution]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Image processing]]

IKONOS

2016-08-14T05:01:28Z

⚠️NikosA: /* Deriving Physical Quantities */

= Overview =

IKONOS is a commercial earth observation satellite. Details about the sensor are provided at Digital Globe's [http://www.digitalglobe.com/sites/default/files/DG_IKONOS_DS.pdf IKONOS Data Sheet]. [http://en.wikipedia.org/wiki/Ikonos Wikipedia's article on IKONOS] provides a nice overview as well.

== Availability (Sample Data) ==

* Search for commercial satellite image providers in the internet.
* [http://www.landcover.org/ The Global Land Cover Facility (GLCF)] provides [ftp://ftp.glcf.umd.edu/glcf/China_earthquake_May_2008/IKONOS/ four openly available IKONOS scenes of western Sichuan].
* ISPRS provides a small [http://www.isprs.org/data/ikonos/ IKONOS data set], fragments from a Panchromatic image as well as from a Stereo product.

== Modules overview ==

=== Outside of GRASS ===

Satellite imagery can be managed and, at some extent, pre-processed with various [http://www.gdal.org/gdal_utilities.html GDAL utilities].

* [http://www.gdal.org/gdalmanage.html gdalmanage]
* [http://www.gdal.org/gdalinfo.html gdalinfo]
* [http://www.gdal.org/gdal_translate.html gdal_translate]
* [http://www.gdal.org/gdalbuildvrt.html gdalbuildvrt]
* [http://www.gdal.org/gdalwarp.html gdalwarp]

=== Inside GRASS GIS ===

GRASS GIS features a complete set of modules and various add-ons enabling pre- and post-processing of IKONOS satellite imagery. The following lists offer an overview of related modules and add-ons.

* {{cmd|r.in.gdal}}
* {{cmd|r.mapcalc}}
* {{cmd|r.colors}}
* {{cmd|i.landsat.rgb}}
* {{cmd|i.pansharpen}}
* {{cmd|i.vi}}
* {{cmd|i.segment}}
* ...''more to add''

== Pre-Processing ==

=== Overview ===

Typically, multispectral satellite data are converted into physical quantities such as ''Radiance'' or ''Reflectance'' before they are subjected in multispectral analysis techniques (image interpretation, band arithmetic, vegetation indices, matrix transformations, etc.). The latter can be differentiated in ''Top of Atmosphere Reflectance'' (ToAR) which does not account for atmospheric effects (absorption or scattering) and in ''Top of Canopy Reflectance'' (ToCR) which introduces a "correction" for atmospheric effects.

In order to derive Reflectance values, likewise as with remotely sensed data acquired by other sensors, IKONOS raw image digital numbers (DNs) need to be converted to ''at-sensor spectral Radiance'' values. At-sensor spectral Radiance values are an important input for the equation to derive Reflectance values. Note, ''Spectal Radiance'' is the measure of the quantity of radiation that hits the sensor and typically expressed in <math>\frac{W}{m^2*sr*nm}</math>, that is ''watts'' per ''unit source area'', per ''unit solid angle'', and per ''unit wavelength''.

<ul>
* Converting DNs to at-sensor Radiance can be done by using the following equation:

<math>L\lambda = \frac{10^4 * DN\lambda}{CalCoef\lambda * Bandwidth\lambda}</math>

* Converting to Top of Atmosphere Reflectance, also referred to as Planetary Reflectance, can be done by using the following equation:

<math>\rho_p = \frac{\pi * L\lambda * d^2}{ESUN\lambda * cos(\Theta_S)}</math>

 where:

<ul>
* <math>\rho</math> - Unitless Planetary Reflectance
* <math>\pi</math> - mathematical constant (3.14159265358)
* <math>L\lambda</math> spectral Radiance at the sensor's aperture, from equation... '''''ToADD'''''
* <math>d</math> - Earth-Sun distance in astronomical units, interpolated values
* <math>Esun</math> - Mean solar exoatmospheric irradiance(s) (W/m2/μm), interpolated values
* <math>cos(\theta_s)</math> - {{wikipedia||Solar_zenith_angle}} from the image acquisition's metadata
</ul>

</ul>

=== Importing data ===

==== File Formats & Metadata ====

''ToAdd''

==== Importing ====

In the following example the publicly available IKONOS acqisition [ftp://ftp.glcf.umd.edu/glcf/China_earthquake_May_2008/IKONOS/po_58204_0000000.20001116.China-Sichuan/ po_58204_0000000.20001116.China-Sichuan] is used.

''ToAdd''
<ul>
* Location creation based on georeferenced data
</ul>

Once inside a Location that is defined by the spatial reference system in which the bands of interest are projected, they can be imported with the {{cmd|r.in.gdal}} module.

For example, GeoTIFF files can be imported by looping {{cmd|r.in.gdal}} over all of them

<source lang="bash">
for TIF in `echo *.tif`; do r.in.gdal in=${TIF} out=${TIF%%.*}; done
</source>

To keep the raw material untouched, we create another Mapset inside the same Location and copy over the bands giving, ''optionally'' at the same time, a new name for each.

<source lang="bash">
g.mapset -c pre-processing
g.copy rast=po_58204_blu_0000000,Blue_DNs
g.copy rast=po_58204_grn_0000000,Green_DNs
g.copy rast=po_58204_red_0000000,Red_DNs
g.copy rast=po_58204_nir_0000000,NIR_DNs
g.copy rast=po_58204_pan_0000000,Pan_DNs
</source>

=== Deriving Physical Quantities ===

Conversions are implemented in a GRASS module available at https://github.com/NikosAlexandris/i.ikonos.toar.

''Details''

Converting Digital Numbers to Radiance/Reflectance requires knowledge about the sensor's specific spectral band parameters. Those are, as extracted from the document ''IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance'', by Martin Taylor (see references):

<source lang=bash>
Pan_CalCoef=161 ; Pan_Width=403 ; Pan_Esun=1375.8
Blue_CalCoef=728 ; Blue_Width=71.3 ; Blue_Esun=1930.9
Green_CalCoef=720 ; Green_Width=88.6 ; Green_Esun=1854.8
Red_CalCoef=949 ; Red_Width=65.8 ; Red_Esun=1556.5
NIR_CalCoef=843 ; NIR_Width=95.4 ; NIR_Esun=1156.9
</source>

The following examples exemplify the conversion of raw Blue band digital numbers into Radiance and Reflectance.

==== Spectral Radiance ====

Converting a Blue Band (Digital Numbers) in to Spectral at-sensor Radiance in the ''correct'' units to be further used for the conversion in to unitless Reflectance:

<source lang="bash">
# set the region
g.region rast=Blue_DNs -p

# convert DNs to spectral Radiance values
r.mapcalc "Blue_Radiance = ( (10000 * IKONOS_Blue_DNs) / (728 * 71.3) )"
</source>

==== Planetary Reflectance ====

The equation to derive Reflectance values incorporates in addition:

<ul>
* The mathematical constant {{wikipedia|Pi}} <source lang="bash" enclose=none>PI=3.14159265358</source>.
* The Earth-Sun distance in astronomical units which depends on the acquisition's day of year (DOY -- also referred to as Julian day, {{wikipedia|Ordinal_date}}) and can be retrieved from the following spreadsheet <http://landsathandbook.gsfc.nasa.gov/excel_docs/d.xls>.
* The mean solar exoatmospheric irradiance in <math>\frac{W}{m^2*\mu m}</math>. See 3rd column of (interplated) values given above.
* The cosine of the ''Solar Zenith Angle'' (SZA) at the time of the acquisition. The SZA can be calculated from its complementary ''Solar Elevation Angle'' (SEA) given in the image acquisition's metadata.
</ul>

In the following example we accept as the acquisition's <source lang="bash" enclose=none>DOY=166</source> and <source lang="bash" enclose=none>SEA=52.78880</source>. Hence, we get the Earth-Sun distance <source lang="bash" enclose=none>ESD=1.0157675</source> and <source lang="bash" enclose=none>SZA = 37.21120 deg</source>.

Converting the in-Blue spectral band at-sensor Radiance in to Planerary Reflectance:

<source lang="bash" enclose=none>PI=3.14159265358; ESD=1.0157675; BAND_Esun=1930.9; SZA=37.21120</source>
{{cmd|r.mapcalc}} <source lang="bash" enclose=none>"Blue_Reflectance = ( ${PI} * Blue_Radiance * ${ESD}^2 ) / ( ${BAND_Esun} * cos(${SZA}) )"</source>

==== Automatising Conversions ====

The conversion process can be scripted to avoid repeating the same steps for each band separately.

===== Python =====

A custom python script, performing the operations of interest, might be like [https://github.com/NikosAlexandris/i.ikonos.toar i.ikonos.toar (for GRASS 7.x)]

===== Bash =====
In bash, such a script might be as the following example. '''Note,''' however, in this example script constants, band parameters and acquisition related metadata are hard-coded!

<source lang="bash">
#!/bin/bash

Pan_CalCoef=161 ; Pan_Width=403 ; Pan_Esun=1375.8
Blue_CalCoef=728 ; Blue_Width=71.3 ; Blue_Esun=1930.9
Green_CalCoef=720 ; Green_Width=88.6 ; Green_Esun=1854.8
Red_CalCoef=949 ; Red_Width=65.8 ; Red_Esun=1556.5
NIR_CalCoef=843 ; NIR_Width=95.4 ; NIR_Esun=1156.9

# set constants, band parameters and acquisition related metadata here!

# Pi, first 11 decimals
PI=3.14159265358

# Acquisition's Day of Year and estimated Earth-Sun Distance
DOY=166; ESD=1.0157675

# Sun Zenith Angle based on the acquisition's Sun Elevation Angle
SEA=52.78880; SZA=$(echo "90 - ${SEA}" | bc )

# bands to process
Spectral_Bands="Pan Blue Green Red NIR"

# loop over all bands
for BAND in ${Spectral_Bands}; do

echo "Processing the ${BAND} spectral band"

# set region
g.region rast=${BAND}_DNs

# set band parameters as variables
eval BAND_CalCoef="${BAND}_CalCoef"
eval BAND_Width="${BAND}_Width";
echo "Band Parameters set to CalCoef=${!BAND_CalCoef}, Bandwidth=${!BAND_Width}"

# conversion to Radiance
r.mapcalc "${BAND}_Radiance = ( ( 10^4 * ${BAND}_DNs ) / ( ${!BAND_CalCoef} * ${!BAND_Width} ) )"

# add info
r.support map=${BAND}_Radiance \
title="" \
units="W / m2 / μm / ster" \
description="At-sensor `echo ${BAND}` band spectral Radiance (W/m2/μm/sr)" \
source1='"IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance", by Martin Taylor, Geoeye'

# set Earth-Sun distance
eval BAND_Esun="${BAND}_Esun"; echo "Using Esun=${!BAND_Esun}"

# calculate ToAR
r.mapcalc "${BAND}_ToAR = \
( ${PI} * ${BAND}_Radiance * ${ESD}^2 ) / ( ${!BAND_Esun} * cos(${SZA}) )"

# add some metadata
r.support map=${BAND}_ToAR \
title="echo ${BAND} band (Top of Atmosphere Reflectance)" \
units="Unitless" \
description="Top of Atmosphere `echo ${BAND}` band spectral Reflectance (unitless)" \
source1='"IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance", by Martin Taylor, Geoeye' \
source2="e.g., the Image Provider!" \
history="PI=3.14159265358; ESD=1.0157675; BAND_Esun=1930.9; SZA=37.21120"

done
</source>

=== Atmospheric correction ===

See [[Atmospheric correction]]

== Post-Processing ==

Having beforehand satellite image data expressed in physical quantities (radiance or reflectance) is preferred to follow-up with digital image analysis techniques. A few common post-processing practices are Contrast-Enhancement, Pan-Sharpening and creating Pseudo- or True-Color Composites. Other well known enhancing manipualtions to support the analyses of satellite imagery, include deriving Vegetation Indices, transforming multi-spectral data based on [[Principal_Components_Analysis | PCA]] and Segmenting images.

=== Contrast Enhancement ===

''ToAdd''

=== Pan-Sharpening ===

[http://en.wikipedia.org/wiki/Pansharpened_image Pan-Sharpening] / [http://en.wikipedia.org/wiki/Image_fusion Fusion] is the process of merging high-resolution panchromatic and lower resolution multi-spectral imagery. [http://grass.osgeo.org/grass70/ GRASS 7] holds a dedicated pan-sharpening module, {{cmd|i.pansharpen}} which features three techniques for sharpening, namely the [http://wiki.awf.forst.uni-goettingen.de/wiki/index.php/Brovey_Transformation Brovey transformation], the classical IHS method and one that is based on [[Principal Components Analysis]] (PCA). Another algorithm deriving excellent detail and a realistic representation of original multispectral scene colors, is the High-Pass Filter Addition (HPFA) technique. It is available through the add-on {{AddonSrc|imagery|i.fusion.hpf|version=7}} (for GRASS 6, please refer to a bash shell script https://github.com/NikosAlexandris/i.fusion.hpf.sh which is, however, unmaintained).

One approach inside GRASS-GIS to get an acceptable color-balanced composite image after Pan-sharpening 11-bit IKONOS spectral bands comprises the following steps

<ul>
# rescale the 11-bit IKONOS spectral bands to 8-bit ranging in [0, 255] ({{cmd|r.rescale}})
# pan-sharpen with any of the featured methods (Brovey, IHS, PCA) ({{cmd|i.pansharpen}})
# color-balance by using the {{cmd|i.landsat.rgb}} module or manually adjusting the color tables of the bands of interest
# create a composite image by using the {{cmd|r.compose}} module
</ul>

==== Example Instructions for {{cmd|i.pansharpen}} ====

{{cmd|i.pansharpen}} works fine with 8-bit raster maps as an input. If the data to be processed are out of this range, that is out of <math>[0, 255]</math>, they can be rescaled to fit into this range by using GRASS' {{cmd|r.rescale}} module.

Given an IKONOS set of 11-bit spectral bands (Blue, Green, Red, NIR and Pan) ranging between <math>[0, 2047]</math>, and then querying for example the Blue band
<source lang="bash" enclose=none>r.info Blue_DNs -r</source>, would return

<source lang="bash">
min=0
max=2047
</source>

===== Rescaling single bands =====

Rescaling the Blue band to range between `[0, 255]`
<source lang="bash">
r.rescale in=Blue_DNs out=Blue_DNs_255 from=0,2047 to=0,255
</source>

The same step applies to both the rest of the multi-spectral bands and the Panchromatic band of interest. If working under Bash, To repeat the same command (given, it was the last on used in the GRASS' terminal), one can isntruct

<source lang="bash">
!!:gs/Blue/Green
</source>

This would substitute everywhere found on the last command the string "Blue" with the string "Green" and re-execute it. The same can be done for the rest of the bands, namely Red, NIR and Pan. Some attention is required to instruct substitution of the ''last'' used string.

===== Rescaling all bands at once =====

Of course, one can always use a ''for'' loop

<source lang="bash">
for DN in `g.mlist rast pat=*DNs`; do r.rescale in=${DN} out=${DN}_255 from=0,2047 to=0,255; done
</source>

===== Pan-Sharpening =====

As usual when working with GRASS, it is required to set the region of interest, i.e. <source lang="bash" enclose=none>g.region rast=Blue_DNs_255</source> to match the extent of the band(s) or else. The resolution itself is taken care in this particular case by the module and the resulting pan-sharpened raster maps will of the same high(er) resolution as the Panchromatic band.

An example command for an IHS-based Pan-Sharpening action might look like

<source lang="bash">
i.pansharpen pan=Pan_DNs_255 ms1=Blue_DNs_255 ms2=Green_DNs_255 ms3=Red_DNs_255 output=sharptest255 sharpen=ihs
</source>

===== Color Re-Balancnig =====

After the process completion, the module outputs

<source lang="bash">
...
The following pan-sharpened output maps have been generated:
sharptest255_red
sharptest255_green
sharptest255_blue

To visualize output, run: g.region -p rast=sharptest255.red
d.rgb r=sharptest255_red g=sharptest255_green b=sharptest255_blue

If desired, combine channels into a single RGB map with 'r.composite'.
</source>

Normally it should be enough to re-balance the colors after the pan-sharpening action by using for example the {{cmd|i.landsat.rgb}} module or manual adjustment of each of the three bands that would compose an RGB image.

<source lang="bash">
i.landsat.rgb r=sharptest255_red g=sharptest255_green b=sharptest255_blue -p
</source>

===== Automatising Sharpening & RGB Composition =====

An ''experimental'' bash script to automatise the sharpening/fusion and RGB composition process is demonstrated here. The script uses the Spectral Reflectance values (double precision values), gained from previous steps (see above), and produces Pan-sharpened images by applying all of the three methods that {{addon|i.pansharpen}} offers. In addition, it attempts to produce True-Color composite images, without and with re-balancing the color tables by using the {{cmd|i.landsat.rgb}} module.

Initial tests indicate that at least one of the three sharpening methods, combined with re-balancing, produces very clear and balanced True-Color composites. Note, there is no 100% confidence that it will produce nice looking True-Color composites. Some of the methods might simply work, others might deliver fancy-colored images. As usual, some manual action might be required to get the re-balancing to work as best as possible.

The script can be expanded in terms of using more inputs by utilising bash's positional parameters. However, care has to be taken to alter the instruction that concerns the conversion from double precision values to 8-bit integers.

<source lang="bash">
#!/bash/sh

# use in G7 only :-)

echo "Rescaling all images to 8-bit"
# convert to 8-bit first :-!
for Method_Input in \
"Pan ToAR" \
"Blue ToAR" \
"Green ToAR" \
"Red ToAR" \
"NIR ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo -e "\n" #echo $1 $2

# input BAND is...
eval BAND="${1}_${2}"

# set region
g.region rast=${BAND}

# integerise
r.mapcalc --v \
"${BAND}_integerised = round( 1000000 * ${BAND} )" # how many decimals?

# integerised BAND is...
eval BAND="${1}_${2}_integerised"

# get ${min} and ${max}
eval `r.info -r ${BAND}`

#
r.rescale in=${1}_${2}_integerised out=${1}_${2}_255 from=${min},${max} to=0,255

done ########################################################################
echo "Rescaling done!"

# pan-sharpen High Resolution imagery ---------------------------------------
# needs setting some naming conventions
for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# some echo
echo "Pan-Sharpening the <${2}> images based on the <${1}> method"

i.pansharpen \
sharpen=${1} \
pan=Pan_${2}_255 \
ms1=Blue_${2}_255 \
ms2=Green_${2}_255 \
ms3=Red_${2}_255 \
output=sharp_${1}_${2}_255

done ########################################################################

# compose BGRs --------------------------------------------------------------

# loop over Methods and Input Types, compose RGBs
for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# region
g.region rast=sharp_"${1}"_"${2}"_255_blue

# compose
r.composite --o \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue \
out=rgb_sharpened_"${1}"_"${2}"_255

done #######################################################################

# re-balance & re-compose colors --------------------------------------------

for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# re-balance
i.landsat.rgb -p \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue

# region -- Really necessary, again?
g.region rast=sharp_"${1}"_"${2}"_255_blue

# re-compose
r.composite --o \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue \
out=rgb_sharpened_"${1}"_"${2}"_255_rebalanced

done ########################################################################
</source>

==== Pan-Sharpening based on the HPFA technique ====

The technique, implemented via the GRASS add-on {{AddonSrc|imagery|i.fusion.hpf|version=7}}, as indicated above, involves a convolution using a High Pass Filter (HPF) on the high resolution data, then combining this with the lower resolution multispectral data.

''Source: "Optimizing the High-Pass Filter Addition Technique for Image Fusion", Ute G. Gangkofner, Pushkar S. Pradhan, and Derrold W. Holcomb (2008)''

The algorithm's steps are:

# Computing the ratio of the low (Multi-Spectral) to the high (Panchromatic) resolution
# High Pass Filtering the Panchromatic Image
# Resampling the Multi-Spectral image to the higher resolution
# Adding a weighted High-Pass-Filtered image to the upsampled Multi-Spectral image
# Optionally, matching the histogram of the Pan-Sharpened image to the one of the original Multi-Spectral image

=== Color Composites ===

''ToAdd''

== Vegetation Indices ==

''ToAdd''

== PCA ==

''ToAdd''

== IKONOS Image classification ==

See [[Image classification]]

== References / Sources ==

* [http://www.apollomapping.com/wp-content/user_uploads/2011/09/IKONOS_Esun_Calculations.pdf IKONOS Planetary Reflectance and Me an Solar Exoatmospheric Irradiance, by Martin Taylor]
* [http://web.unicen.edu.ar/crecic/docs/radrefl.pdf Ikonos DN Value Conversion to Planetary Reflectance Values, by David Fleming]
* [http://landsathandbook.gsfc.nasa.gov/data_prod/prog_sect11_3.html Landsat7 Science Data Users Handbook, Chapter 11, Section 3]
* [http://igett.delmar.edu/Resources/Remote%20Sensing%20Technology%20Training/Calculation-DN_to_Reflectance_Irish_20June08.pdf Calibrated Landsat Digital Number (DN) to Top of Atmosphere (TOA) Reflectance Conversion, by Richard Irish]
* from [http://landsat.usgs.gov/tools_access_all_faqs.php FAQs about the Landsat Missions]
** [http://landsat.usgs.gov/how_is_radiance_calculated.php How is radiance calculated?]
** [http://landsat.usgs.gov/at_sensor_reflectance_calculated.php How is at-sensor reflectance calculated?]

== See also ==

* GRASS-Wiki page about [http://grasswiki.osgeo.org/wiki/Image_processing Image Processing]

''ToAdd''

[[Category: Documentation]]
[[Category: FAQ]]
[[Category: IKONOS]]
[[Category: High Resolution]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Image processing]]

IKONOS

2016-08-14T05:00:38Z

⚠️NikosA: i.ikonos.toar

= Overview =

IKONOS is a commercial earth observation satellite. Details about the sensor are provided at Digital Globe's [http://www.digitalglobe.com/sites/default/files/DG_IKONOS_DS.pdf IKONOS Data Sheet]. [http://en.wikipedia.org/wiki/Ikonos Wikipedia's article on IKONOS] provides a nice overview as well.

== Availability (Sample Data) ==

* Search for commercial satellite image providers in the internet.
* [http://www.landcover.org/ The Global Land Cover Facility (GLCF)] provides [ftp://ftp.glcf.umd.edu/glcf/China_earthquake_May_2008/IKONOS/ four openly available IKONOS scenes of western Sichuan].
* ISPRS provides a small [http://www.isprs.org/data/ikonos/ IKONOS data set], fragments from a Panchromatic image as well as from a Stereo product.

== Modules overview ==

=== Outside of GRASS ===

Satellite imagery can be managed and, at some extent, pre-processed with various [http://www.gdal.org/gdal_utilities.html GDAL utilities].

* [http://www.gdal.org/gdalmanage.html gdalmanage]
* [http://www.gdal.org/gdalinfo.html gdalinfo]
* [http://www.gdal.org/gdal_translate.html gdal_translate]
* [http://www.gdal.org/gdalbuildvrt.html gdalbuildvrt]
* [http://www.gdal.org/gdalwarp.html gdalwarp]

=== Inside GRASS GIS ===

GRASS GIS features a complete set of modules and various add-ons enabling pre- and post-processing of IKONOS satellite imagery. The following lists offer an overview of related modules and add-ons.

* {{cmd|r.in.gdal}}
* {{cmd|r.mapcalc}}
* {{cmd|r.colors}}
* {{cmd|i.landsat.rgb}}
* {{cmd|i.pansharpen}}
* {{cmd|i.vi}}
* {{cmd|i.segment}}
* ...''more to add''

== Pre-Processing ==

=== Overview ===

Typically, multispectral satellite data are converted into physical quantities such as ''Radiance'' or ''Reflectance'' before they are subjected in multispectral analysis techniques (image interpretation, band arithmetic, vegetation indices, matrix transformations, etc.). The latter can be differentiated in ''Top of Atmosphere Reflectance'' (ToAR) which does not account for atmospheric effects (absorption or scattering) and in ''Top of Canopy Reflectance'' (ToCR) which introduces a "correction" for atmospheric effects.

In order to derive Reflectance values, likewise as with remotely sensed data acquired by other sensors, IKONOS raw image digital numbers (DNs) need to be converted to ''at-sensor spectral Radiance'' values. At-sensor spectral Radiance values are an important input for the equation to derive Reflectance values. Note, ''Spectal Radiance'' is the measure of the quantity of radiation that hits the sensor and typically expressed in <math>\frac{W}{m^2*sr*nm}</math>, that is ''watts'' per ''unit source area'', per ''unit solid angle'', and per ''unit wavelength''.

<ul>
* Converting DNs to at-sensor Radiance can be done by using the following equation:

<math>L\lambda = \frac{10^4 * DN\lambda}{CalCoef\lambda * Bandwidth\lambda}</math>

* Converting to Top of Atmosphere Reflectance, also referred to as Planetary Reflectance, can be done by using the following equation:

<math>\rho_p = \frac{\pi * L\lambda * d^2}{ESUN\lambda * cos(\Theta_S)}</math>

 where:

<ul>
* <math>\rho</math> - Unitless Planetary Reflectance
* <math>\pi</math> - mathematical constant (3.14159265358)
* <math>L\lambda</math> spectral Radiance at the sensor's aperture, from equation... '''''ToADD'''''
* <math>d</math> - Earth-Sun distance in astronomical units, interpolated values
* <math>Esun</math> - Mean solar exoatmospheric irradiance(s) (W/m2/μm), interpolated values
* <math>cos(\theta_s)</math> - {{wikipedia||Solar_zenith_angle}} from the image acquisition's metadata
</ul>

</ul>

=== Importing data ===

==== File Formats & Metadata ====

''ToAdd''

==== Importing ====

In the following example the publicly available IKONOS acqisition [ftp://ftp.glcf.umd.edu/glcf/China_earthquake_May_2008/IKONOS/po_58204_0000000.20001116.China-Sichuan/ po_58204_0000000.20001116.China-Sichuan] is used.

''ToAdd''
<ul>
* Location creation based on georeferenced data
</ul>

Once inside a Location that is defined by the spatial reference system in which the bands of interest are projected, they can be imported with the {{cmd|r.in.gdal}} module.

For example, GeoTIFF files can be imported by looping {{cmd|r.in.gdal}} over all of them

<source lang="bash">
for TIF in `echo *.tif`; do r.in.gdal in=${TIF} out=${TIF%%.*}; done
</source>

To keep the raw material untouched, we create another Mapset inside the same Location and copy over the bands giving, ''optionally'' at the same time, a new name for each.

<source lang="bash">
g.mapset -c pre-processing
g.copy rast=po_58204_blu_0000000,Blue_DNs
g.copy rast=po_58204_grn_0000000,Green_DNs
g.copy rast=po_58204_red_0000000,Red_DNs
g.copy rast=po_58204_nir_0000000,NIR_DNs
g.copy rast=po_58204_pan_0000000,Pan_DNs
</source>

=== Deriving Physical Quantities ===

Conversions are implemented in a GRASS module available at https://github.com/NikosAlexandris/i.ikonos.toar. Detailed descriptions below.

Converting Digital Numbers to Radiance/Reflectance requires knowledge about the sensor's specific spectral band parameters. Those are, as extracted from the document ''IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance'', by Martin Taylor (see references):

<source lang=bash>
Pan_CalCoef=161 ; Pan_Width=403 ; Pan_Esun=1375.8
Blue_CalCoef=728 ; Blue_Width=71.3 ; Blue_Esun=1930.9
Green_CalCoef=720 ; Green_Width=88.6 ; Green_Esun=1854.8
Red_CalCoef=949 ; Red_Width=65.8 ; Red_Esun=1556.5
NIR_CalCoef=843 ; NIR_Width=95.4 ; NIR_Esun=1156.9
</source>

The following examples exemplify the conversion of raw Blue band digital numbers into Radiance and Reflectance.

==== Spectral Radiance ====

Converting a Blue Band (Digital Numbers) in to Spectral at-sensor Radiance in the ''correct'' units to be further used for the conversion in to unitless Reflectance:

<source lang="bash">
# set the region
g.region rast=Blue_DNs -p

# convert DNs to spectral Radiance values
r.mapcalc "Blue_Radiance = ( (10000 * IKONOS_Blue_DNs) / (728 * 71.3) )"
</source>

==== Planetary Reflectance ====

The equation to derive Reflectance values incorporates in addition:

<ul>
* The mathematical constant {{wikipedia|Pi}} <source lang="bash" enclose=none>PI=3.14159265358</source>.
* The Earth-Sun distance in astronomical units which depends on the acquisition's day of year (DOY -- also referred to as Julian day, {{wikipedia|Ordinal_date}}) and can be retrieved from the following spreadsheet <http://landsathandbook.gsfc.nasa.gov/excel_docs/d.xls>.
* The mean solar exoatmospheric irradiance in <math>\frac{W}{m^2*\mu m}</math>. See 3rd column of (interplated) values given above.
* The cosine of the ''Solar Zenith Angle'' (SZA) at the time of the acquisition. The SZA can be calculated from its complementary ''Solar Elevation Angle'' (SEA) given in the image acquisition's metadata.
</ul>

In the following example we accept as the acquisition's <source lang="bash" enclose=none>DOY=166</source> and <source lang="bash" enclose=none>SEA=52.78880</source>. Hence, we get the Earth-Sun distance <source lang="bash" enclose=none>ESD=1.0157675</source> and <source lang="bash" enclose=none>SZA = 37.21120 deg</source>.

Converting the in-Blue spectral band at-sensor Radiance in to Planerary Reflectance:

<source lang="bash" enclose=none>PI=3.14159265358; ESD=1.0157675; BAND_Esun=1930.9; SZA=37.21120</source>
{{cmd|r.mapcalc}} <source lang="bash" enclose=none>"Blue_Reflectance = ( ${PI} * Blue_Radiance * ${ESD}^2 ) / ( ${BAND_Esun} * cos(${SZA}) )"</source>

==== Automatising Conversions ====

The conversion process can be scripted to avoid repeating the same steps for each band separately.

===== Python =====

A custom python script, performing the operations of interest, might be like [https://github.com/NikosAlexandris/i.ikonos.toar i.ikonos.toar (for GRASS 7.x)]

===== Bash =====
In bash, such a script might be as the following example. '''Note,''' however, in this example script constants, band parameters and acquisition related metadata are hard-coded!

<source lang="bash">
#!/bin/bash

Pan_CalCoef=161 ; Pan_Width=403 ; Pan_Esun=1375.8
Blue_CalCoef=728 ; Blue_Width=71.3 ; Blue_Esun=1930.9
Green_CalCoef=720 ; Green_Width=88.6 ; Green_Esun=1854.8
Red_CalCoef=949 ; Red_Width=65.8 ; Red_Esun=1556.5
NIR_CalCoef=843 ; NIR_Width=95.4 ; NIR_Esun=1156.9

# set constants, band parameters and acquisition related metadata here!

# Pi, first 11 decimals
PI=3.14159265358

# Acquisition's Day of Year and estimated Earth-Sun Distance
DOY=166; ESD=1.0157675

# Sun Zenith Angle based on the acquisition's Sun Elevation Angle
SEA=52.78880; SZA=$(echo "90 - ${SEA}" | bc )

# bands to process
Spectral_Bands="Pan Blue Green Red NIR"

# loop over all bands
for BAND in ${Spectral_Bands}; do

echo "Processing the ${BAND} spectral band"

# set region
g.region rast=${BAND}_DNs

# set band parameters as variables
eval BAND_CalCoef="${BAND}_CalCoef"
eval BAND_Width="${BAND}_Width";
echo "Band Parameters set to CalCoef=${!BAND_CalCoef}, Bandwidth=${!BAND_Width}"

# conversion to Radiance
r.mapcalc "${BAND}_Radiance = ( ( 10^4 * ${BAND}_DNs ) / ( ${!BAND_CalCoef} * ${!BAND_Width} ) )"

# add info
r.support map=${BAND}_Radiance \
title="" \
units="W / m2 / μm / ster" \
description="At-sensor `echo ${BAND}` band spectral Radiance (W/m2/μm/sr)" \
source1='"IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance", by Martin Taylor, Geoeye'

# set Earth-Sun distance
eval BAND_Esun="${BAND}_Esun"; echo "Using Esun=${!BAND_Esun}"

# calculate ToAR
r.mapcalc "${BAND}_ToAR = \
( ${PI} * ${BAND}_Radiance * ${ESD}^2 ) / ( ${!BAND_Esun} * cos(${SZA}) )"

# add some metadata
r.support map=${BAND}_ToAR \
title="echo ${BAND} band (Top of Atmosphere Reflectance)" \
units="Unitless" \
description="Top of Atmosphere `echo ${BAND}` band spectral Reflectance (unitless)" \
source1='"IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance", by Martin Taylor, Geoeye' \
source2="e.g., the Image Provider!" \
history="PI=3.14159265358; ESD=1.0157675; BAND_Esun=1930.9; SZA=37.21120"

done
</source>

=== Atmospheric correction ===

See [[Atmospheric correction]]

== Post-Processing ==

Having beforehand satellite image data expressed in physical quantities (radiance or reflectance) is preferred to follow-up with digital image analysis techniques. A few common post-processing practices are Contrast-Enhancement, Pan-Sharpening and creating Pseudo- or True-Color Composites. Other well known enhancing manipualtions to support the analyses of satellite imagery, include deriving Vegetation Indices, transforming multi-spectral data based on [[Principal_Components_Analysis | PCA]] and Segmenting images.

=== Contrast Enhancement ===

''ToAdd''

=== Pan-Sharpening ===

[http://en.wikipedia.org/wiki/Pansharpened_image Pan-Sharpening] / [http://en.wikipedia.org/wiki/Image_fusion Fusion] is the process of merging high-resolution panchromatic and lower resolution multi-spectral imagery. [http://grass.osgeo.org/grass70/ GRASS 7] holds a dedicated pan-sharpening module, {{cmd|i.pansharpen}} which features three techniques for sharpening, namely the [http://wiki.awf.forst.uni-goettingen.de/wiki/index.php/Brovey_Transformation Brovey transformation], the classical IHS method and one that is based on [[Principal Components Analysis]] (PCA). Another algorithm deriving excellent detail and a realistic representation of original multispectral scene colors, is the High-Pass Filter Addition (HPFA) technique. It is available through the add-on {{AddonSrc|imagery|i.fusion.hpf|version=7}} (for GRASS 6, please refer to a bash shell script https://github.com/NikosAlexandris/i.fusion.hpf.sh which is, however, unmaintained).

One approach inside GRASS-GIS to get an acceptable color-balanced composite image after Pan-sharpening 11-bit IKONOS spectral bands comprises the following steps

<ul>
# rescale the 11-bit IKONOS spectral bands to 8-bit ranging in [0, 255] ({{cmd|r.rescale}})
# pan-sharpen with any of the featured methods (Brovey, IHS, PCA) ({{cmd|i.pansharpen}})
# color-balance by using the {{cmd|i.landsat.rgb}} module or manually adjusting the color tables of the bands of interest
# create a composite image by using the {{cmd|r.compose}} module
</ul>

==== Example Instructions for {{cmd|i.pansharpen}} ====

{{cmd|i.pansharpen}} works fine with 8-bit raster maps as an input. If the data to be processed are out of this range, that is out of <math>[0, 255]</math>, they can be rescaled to fit into this range by using GRASS' {{cmd|r.rescale}} module.

Given an IKONOS set of 11-bit spectral bands (Blue, Green, Red, NIR and Pan) ranging between <math>[0, 2047]</math>, and then querying for example the Blue band
<source lang="bash" enclose=none>r.info Blue_DNs -r</source>, would return

<source lang="bash">
min=0
max=2047
</source>

===== Rescaling single bands =====

Rescaling the Blue band to range between `[0, 255]`
<source lang="bash">
r.rescale in=Blue_DNs out=Blue_DNs_255 from=0,2047 to=0,255
</source>

The same step applies to both the rest of the multi-spectral bands and the Panchromatic band of interest. If working under Bash, To repeat the same command (given, it was the last on used in the GRASS' terminal), one can isntruct

<source lang="bash">
!!:gs/Blue/Green
</source>

This would substitute everywhere found on the last command the string "Blue" with the string "Green" and re-execute it. The same can be done for the rest of the bands, namely Red, NIR and Pan. Some attention is required to instruct substitution of the ''last'' used string.

===== Rescaling all bands at once =====

Of course, one can always use a ''for'' loop

<source lang="bash">
for DN in `g.mlist rast pat=*DNs`; do r.rescale in=${DN} out=${DN}_255 from=0,2047 to=0,255; done
</source>

===== Pan-Sharpening =====

As usual when working with GRASS, it is required to set the region of interest, i.e. <source lang="bash" enclose=none>g.region rast=Blue_DNs_255</source> to match the extent of the band(s) or else. The resolution itself is taken care in this particular case by the module and the resulting pan-sharpened raster maps will of the same high(er) resolution as the Panchromatic band.

An example command for an IHS-based Pan-Sharpening action might look like

<source lang="bash">
i.pansharpen pan=Pan_DNs_255 ms1=Blue_DNs_255 ms2=Green_DNs_255 ms3=Red_DNs_255 output=sharptest255 sharpen=ihs
</source>

===== Color Re-Balancnig =====

After the process completion, the module outputs

<source lang="bash">
...
The following pan-sharpened output maps have been generated:
sharptest255_red
sharptest255_green
sharptest255_blue

To visualize output, run: g.region -p rast=sharptest255.red
d.rgb r=sharptest255_red g=sharptest255_green b=sharptest255_blue

If desired, combine channels into a single RGB map with 'r.composite'.
</source>

Normally it should be enough to re-balance the colors after the pan-sharpening action by using for example the {{cmd|i.landsat.rgb}} module or manual adjustment of each of the three bands that would compose an RGB image.

<source lang="bash">
i.landsat.rgb r=sharptest255_red g=sharptest255_green b=sharptest255_blue -p
</source>

===== Automatising Sharpening & RGB Composition =====

An ''experimental'' bash script to automatise the sharpening/fusion and RGB composition process is demonstrated here. The script uses the Spectral Reflectance values (double precision values), gained from previous steps (see above), and produces Pan-sharpened images by applying all of the three methods that {{addon|i.pansharpen}} offers. In addition, it attempts to produce True-Color composite images, without and with re-balancing the color tables by using the {{cmd|i.landsat.rgb}} module.

Initial tests indicate that at least one of the three sharpening methods, combined with re-balancing, produces very clear and balanced True-Color composites. Note, there is no 100% confidence that it will produce nice looking True-Color composites. Some of the methods might simply work, others might deliver fancy-colored images. As usual, some manual action might be required to get the re-balancing to work as best as possible.

The script can be expanded in terms of using more inputs by utilising bash's positional parameters. However, care has to be taken to alter the instruction that concerns the conversion from double precision values to 8-bit integers.

<source lang="bash">
#!/bash/sh

# use in G7 only :-)

echo "Rescaling all images to 8-bit"
# convert to 8-bit first :-!
for Method_Input in \
"Pan ToAR" \
"Blue ToAR" \
"Green ToAR" \
"Red ToAR" \
"NIR ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo -e "\n" #echo $1 $2

# input BAND is...
eval BAND="${1}_${2}"

# set region
g.region rast=${BAND}

# integerise
r.mapcalc --v \
"${BAND}_integerised = round( 1000000 * ${BAND} )" # how many decimals?

# integerised BAND is...
eval BAND="${1}_${2}_integerised"

# get ${min} and ${max}
eval `r.info -r ${BAND}`

#
r.rescale in=${1}_${2}_integerised out=${1}_${2}_255 from=${min},${max} to=0,255

done ########################################################################
echo "Rescaling done!"

# pan-sharpen High Resolution imagery ---------------------------------------
# needs setting some naming conventions
for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# some echo
echo "Pan-Sharpening the <${2}> images based on the <${1}> method"

i.pansharpen \
sharpen=${1} \
pan=Pan_${2}_255 \
ms1=Blue_${2}_255 \
ms2=Green_${2}_255 \
ms3=Red_${2}_255 \
output=sharp_${1}_${2}_255

done ########################################################################

# compose BGRs --------------------------------------------------------------

# loop over Methods and Input Types, compose RGBs
for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# region
g.region rast=sharp_"${1}"_"${2}"_255_blue

# compose
r.composite --o \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue \
out=rgb_sharpened_"${1}"_"${2}"_255

done #######################################################################

# re-balance & re-compose colors --------------------------------------------

for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# re-balance
i.landsat.rgb -p \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue

# region -- Really necessary, again?
g.region rast=sharp_"${1}"_"${2}"_255_blue

# re-compose
r.composite --o \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue \
out=rgb_sharpened_"${1}"_"${2}"_255_rebalanced

done ########################################################################
</source>

==== Pan-Sharpening based on the HPFA technique ====

The technique, implemented via the GRASS add-on {{AddonSrc|imagery|i.fusion.hpf|version=7}}, as indicated above, involves a convolution using a High Pass Filter (HPF) on the high resolution data, then combining this with the lower resolution multispectral data.

''Source: "Optimizing the High-Pass Filter Addition Technique for Image Fusion", Ute G. Gangkofner, Pushkar S. Pradhan, and Derrold W. Holcomb (2008)''

The algorithm's steps are:

# Computing the ratio of the low (Multi-Spectral) to the high (Panchromatic) resolution
# High Pass Filtering the Panchromatic Image
# Resampling the Multi-Spectral image to the higher resolution
# Adding a weighted High-Pass-Filtered image to the upsampled Multi-Spectral image
# Optionally, matching the histogram of the Pan-Sharpened image to the one of the original Multi-Spectral image

=== Color Composites ===

''ToAdd''

== Vegetation Indices ==

''ToAdd''

== PCA ==

''ToAdd''

== IKONOS Image classification ==

See [[Image classification]]

== References / Sources ==

* [http://www.apollomapping.com/wp-content/user_uploads/2011/09/IKONOS_Esun_Calculations.pdf IKONOS Planetary Reflectance and Me an Solar Exoatmospheric Irradiance, by Martin Taylor]
* [http://web.unicen.edu.ar/crecic/docs/radrefl.pdf Ikonos DN Value Conversion to Planetary Reflectance Values, by David Fleming]
* [http://landsathandbook.gsfc.nasa.gov/data_prod/prog_sect11_3.html Landsat7 Science Data Users Handbook, Chapter 11, Section 3]
* [http://igett.delmar.edu/Resources/Remote%20Sensing%20Technology%20Training/Calculation-DN_to_Reflectance_Irish_20June08.pdf Calibrated Landsat Digital Number (DN) to Top of Atmosphere (TOA) Reflectance Conversion, by Richard Irish]
* from [http://landsat.usgs.gov/tools_access_all_faqs.php FAQs about the Landsat Missions]
** [http://landsat.usgs.gov/how_is_radiance_calculated.php How is radiance calculated?]
** [http://landsat.usgs.gov/at_sensor_reflectance_calculated.php How is at-sensor reflectance calculated?]

== See also ==

* GRASS-Wiki page about [http://grasswiki.osgeo.org/wiki/Image_processing Image Processing]

''ToAdd''

[[Category: Documentation]]
[[Category: FAQ]]
[[Category: IKONOS]]
[[Category: High Resolution]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Image processing]]

IKONOS

2016-08-14T04:58:15Z

⚠️NikosA: edits related to i.fusion.hpf

= Overview =

IKONOS is a commercial earth observation satellite. Details about the sensor are provided at Digital Globe's [http://www.digitalglobe.com/sites/default/files/DG_IKONOS_DS.pdf IKONOS Data Sheet]. [http://en.wikipedia.org/wiki/Ikonos Wikipedia's article on IKONOS] provides a nice overview as well.

== Availability (Sample Data) ==

* Search for commercial satellite image providers in the internet.
* [http://www.landcover.org/ The Global Land Cover Facility (GLCF)] provides [ftp://ftp.glcf.umd.edu/glcf/China_earthquake_May_2008/IKONOS/ four openly available IKONOS scenes of western Sichuan].
* ISPRS provides a small [http://www.isprs.org/data/ikonos/ IKONOS data set], fragments from a Panchromatic image as well as from a Stereo product.

== Modules overview ==

=== Outside of GRASS ===

Satellite imagery can be managed and, at some extent, pre-processed with various [http://www.gdal.org/gdal_utilities.html GDAL utilities].

* [http://www.gdal.org/gdalmanage.html gdalmanage]
* [http://www.gdal.org/gdalinfo.html gdalinfo]
* [http://www.gdal.org/gdal_translate.html gdal_translate]
* [http://www.gdal.org/gdalbuildvrt.html gdalbuildvrt]
* [http://www.gdal.org/gdalwarp.html gdalwarp]

=== Inside GRASS GIS ===

GRASS GIS features a complete set of modules and various add-ons enabling pre- and post-processing of IKONOS satellite imagery. The following lists offer an overview of related modules and add-ons.

* {{cmd|r.in.gdal}}
* {{cmd|r.mapcalc}}
* {{cmd|r.colors}}
* {{cmd|i.landsat.rgb}}
* {{cmd|i.pansharpen}}
* {{cmd|i.vi}}
* {{cmd|i.segment}}
* ...''more to add''

== Pre-Processing ==

=== Overview ===

Typically, multispectral satellite data are converted into physical quantities such as ''Radiance'' or ''Reflectance'' before they are subjected in multispectral analysis techniques (image interpretation, band arithmetic, vegetation indices, matrix transformations, etc.). The latter can be differentiated in ''Top of Atmosphere Reflectance'' (ToAR) which does not account for atmospheric effects (absorption or scattering) and in ''Top of Canopy Reflectance'' (ToCR) which introduces a "correction" for atmospheric effects.

In order to derive Reflectance values, likewise as with remotely sensed data acquired by other sensors, IKONOS raw image digital numbers (DNs) need to be converted to ''at-sensor spectral Radiance'' values. At-sensor spectral Radiance values are an important input for the equation to derive Reflectance values. Note, ''Spectal Radiance'' is the measure of the quantity of radiation that hits the sensor and typically expressed in <math>\frac{W}{m^2*sr*nm}</math>, that is ''watts'' per ''unit source area'', per ''unit solid angle'', and per ''unit wavelength''.

<ul>
* Converting DNs to at-sensor Radiance can be done by using the following equation:

<math>L\lambda = \frac{10^4 * DN\lambda}{CalCoef\lambda * Bandwidth\lambda}</math>

* Converting to Top of Atmosphere Reflectance, also referred to as Planetary Reflectance, can be done by using the following equation:

<math>\rho_p = \frac{\pi * L\lambda * d^2}{ESUN\lambda * cos(\Theta_S)}</math>

 where:

<ul>
* <math>\rho</math> - Unitless Planetary Reflectance
* <math>\pi</math> - mathematical constant (3.14159265358)
* <math>L\lambda</math> spectral Radiance at the sensor's aperture, from equation... '''''ToADD'''''
* <math>d</math> - Earth-Sun distance in astronomical units, interpolated values
* <math>Esun</math> - Mean solar exoatmospheric irradiance(s) (W/m2/μm), interpolated values
* <math>cos(\theta_s)</math> - {{wikipedia||Solar_zenith_angle}} from the image acquisition's metadata
</ul>

</ul>

=== Importing data ===

==== File Formats & Metadata ====

''ToAdd''

==== Importing ====

In the following example the publicly available IKONOS acqisition [ftp://ftp.glcf.umd.edu/glcf/China_earthquake_May_2008/IKONOS/po_58204_0000000.20001116.China-Sichuan/ po_58204_0000000.20001116.China-Sichuan] is used.

''ToAdd''
<ul>
* Location creation based on georeferenced data
</ul>

Once inside a Location that is defined by the spatial reference system in which the bands of interest are projected, they can be imported with the {{cmd|r.in.gdal}} module.

For example, GeoTIFF files can be imported by looping {{cmd|r.in.gdal}} over all of them

<source lang="bash">
for TIF in `echo *.tif`; do r.in.gdal in=${TIF} out=${TIF%%.*}; done
</source>

To keep the raw material untouched, we create another Mapset inside the same Location and copy over the bands giving, ''optionally'' at the same time, a new name for each.

<source lang="bash">
g.mapset -c pre-processing
g.copy rast=po_58204_blu_0000000,Blue_DNs
g.copy rast=po_58204_grn_0000000,Green_DNs
g.copy rast=po_58204_red_0000000,Red_DNs
g.copy rast=po_58204_nir_0000000,NIR_DNs
g.copy rast=po_58204_pan_0000000,Pan_DNs
</source>

=== Deriving Physical Quantities ===

Converting Digital Numbers to Radiance/Reflectance requires knowledge about the sensor's specific spectral band parameters. Those are, as extracted from the document ''IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance'', by Martin Taylor (see references):

<source lang=bash>
Pan_CalCoef=161 ; Pan_Width=403 ; Pan_Esun=1375.8
Blue_CalCoef=728 ; Blue_Width=71.3 ; Blue_Esun=1930.9
Green_CalCoef=720 ; Green_Width=88.6 ; Green_Esun=1854.8
Red_CalCoef=949 ; Red_Width=65.8 ; Red_Esun=1556.5
NIR_CalCoef=843 ; NIR_Width=95.4 ; NIR_Esun=1156.9
</source>

The following examples exemplify the conversion of raw Blue band digital numbers into Radiance and Reflectance.

==== Spectral Radiance ====

Converting a Blue Band (Digital Numbers) in to Spectral at-sensor Radiance in the ''correct'' units to be further used for the conversion in to unitless Reflectance:

<source lang="bash">
# set the region
g.region rast=Blue_DNs -p

# convert DNs to spectral Radiance values
r.mapcalc "Blue_Radiance = ( (10000 * IKONOS_Blue_DNs) / (728 * 71.3) )"
</source>

==== Planetary Reflectance ====

The equation to derive Reflectance values incorporates in addition:

<ul>
* The mathematical constant {{wikipedia|Pi}} <source lang="bash" enclose=none>PI=3.14159265358</source>.
* The Earth-Sun distance in astronomical units which depends on the acquisition's day of year (DOY -- also referred to as Julian day, {{wikipedia|Ordinal_date}}) and can be retrieved from the following spreadsheet <http://landsathandbook.gsfc.nasa.gov/excel_docs/d.xls>.
* The mean solar exoatmospheric irradiance in <math>\frac{W}{m^2*\mu m}</math>. See 3rd column of (interplated) values given above.
* The cosine of the ''Solar Zenith Angle'' (SZA) at the time of the acquisition. The SZA can be calculated from its complementary ''Solar Elevation Angle'' (SEA) given in the image acquisition's metadata.
</ul>

In the following example we accept as the acquisition's <source lang="bash" enclose=none>DOY=166</source> and <source lang="bash" enclose=none>SEA=52.78880</source>. Hence, we get the Earth-Sun distance <source lang="bash" enclose=none>ESD=1.0157675</source> and <source lang="bash" enclose=none>SZA = 37.21120 deg</source>.

Converting the in-Blue spectral band at-sensor Radiance in to Planerary Reflectance:

<source lang="bash" enclose=none>PI=3.14159265358; ESD=1.0157675; BAND_Esun=1930.9; SZA=37.21120</source>
{{cmd|r.mapcalc}} <source lang="bash" enclose=none>"Blue_Reflectance = ( ${PI} * Blue_Radiance * ${ESD}^2 ) / ( ${BAND_Esun} * cos(${SZA}) )"</source>

==== Automatising Conversions ====

The conversion process can be scripted to avoid repeating the same steps for each band separately.

===== Python =====

A custom python script, performing the operations of interest, might be like [https://github.com/NikosAlexandris/i.ikonos.toar i.ikonos.toar (for GRASS 7.x)]

===== Bash =====
In bash, such a script might be as the following example. '''Note,''' however, in this example script constants, band parameters and acquisition related metadata are hard-coded!

<source lang="bash">
#!/bin/bash

Pan_CalCoef=161 ; Pan_Width=403 ; Pan_Esun=1375.8
Blue_CalCoef=728 ; Blue_Width=71.3 ; Blue_Esun=1930.9
Green_CalCoef=720 ; Green_Width=88.6 ; Green_Esun=1854.8
Red_CalCoef=949 ; Red_Width=65.8 ; Red_Esun=1556.5
NIR_CalCoef=843 ; NIR_Width=95.4 ; NIR_Esun=1156.9

# set constants, band parameters and acquisition related metadata here!

# Pi, first 11 decimals
PI=3.14159265358

# Acquisition's Day of Year and estimated Earth-Sun Distance
DOY=166; ESD=1.0157675

# Sun Zenith Angle based on the acquisition's Sun Elevation Angle
SEA=52.78880; SZA=$(echo "90 - ${SEA}" | bc )

# bands to process
Spectral_Bands="Pan Blue Green Red NIR"

# loop over all bands
for BAND in ${Spectral_Bands}; do

echo "Processing the ${BAND} spectral band"

# set region
g.region rast=${BAND}_DNs

# set band parameters as variables
eval BAND_CalCoef="${BAND}_CalCoef"
eval BAND_Width="${BAND}_Width";
echo "Band Parameters set to CalCoef=${!BAND_CalCoef}, Bandwidth=${!BAND_Width}"

# conversion to Radiance
r.mapcalc "${BAND}_Radiance = ( ( 10^4 * ${BAND}_DNs ) / ( ${!BAND_CalCoef} * ${!BAND_Width} ) )"

# add info
r.support map=${BAND}_Radiance \
title="" \
units="W / m2 / μm / ster" \
description="At-sensor `echo ${BAND}` band spectral Radiance (W/m2/μm/sr)" \
source1='"IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance", by Martin Taylor, Geoeye'

# set Earth-Sun distance
eval BAND_Esun="${BAND}_Esun"; echo "Using Esun=${!BAND_Esun}"

# calculate ToAR
r.mapcalc "${BAND}_ToAR = \
( ${PI} * ${BAND}_Radiance * ${ESD}^2 ) / ( ${!BAND_Esun} * cos(${SZA}) )"

# add some metadata
r.support map=${BAND}_ToAR \
title="echo ${BAND} band (Top of Atmosphere Reflectance)" \
units="Unitless" \
description="Top of Atmosphere `echo ${BAND}` band spectral Reflectance (unitless)" \
source1='"IKONOS Planetary Reflectance and Mean Solar Exoatmospheric Irradiance", by Martin Taylor, Geoeye' \
source2="e.g., the Image Provider!" \
history="PI=3.14159265358; ESD=1.0157675; BAND_Esun=1930.9; SZA=37.21120"

done
</source>

=== Atmospheric correction ===

See [[Atmospheric correction]]

== Post-Processing ==

Having beforehand satellite image data expressed in physical quantities (radiance or reflectance) is preferred to follow-up with digital image analysis techniques. A few common post-processing practices are Contrast-Enhancement, Pan-Sharpening and creating Pseudo- or True-Color Composites. Other well known enhancing manipualtions to support the analyses of satellite imagery, include deriving Vegetation Indices, transforming multi-spectral data based on [[Principal_Components_Analysis | PCA]] and Segmenting images.

=== Contrast Enhancement ===

''ToAdd''

=== Pan-Sharpening ===

[http://en.wikipedia.org/wiki/Pansharpened_image Pan-Sharpening] / [http://en.wikipedia.org/wiki/Image_fusion Fusion] is the process of merging high-resolution panchromatic and lower resolution multi-spectral imagery. [http://grass.osgeo.org/grass70/ GRASS 7] holds a dedicated pan-sharpening module, {{cmd|i.pansharpen}} which features three techniques for sharpening, namely the [http://wiki.awf.forst.uni-goettingen.de/wiki/index.php/Brovey_Transformation Brovey transformation], the classical IHS method and one that is based on [[Principal Components Analysis]] (PCA). Another algorithm deriving excellent detail and a realistic representation of original multispectral scene colors, is the High-Pass Filter Addition (HPFA) technique. It is available through the add-on {{AddonSrc|imagery|i.fusion.hpf|version=7}} (for GRASS 6, please refer to a bash shell script https://github.com/NikosAlexandris/i.fusion.hpf.sh which is, however, unmaintained).

One approach inside GRASS-GIS to get an acceptable color-balanced composite image after Pan-sharpening 11-bit IKONOS spectral bands comprises the following steps

<ul>
# rescale the 11-bit IKONOS spectral bands to 8-bit ranging in [0, 255] ({{cmd|r.rescale}})
# pan-sharpen with any of the featured methods (Brovey, IHS, PCA) ({{cmd|i.pansharpen}})
# color-balance by using the {{cmd|i.landsat.rgb}} module or manually adjusting the color tables of the bands of interest
# create a composite image by using the {{cmd|r.compose}} module
</ul>

==== Example Instructions for {{cmd|i.pansharpen}} ====

{{cmd|i.pansharpen}} works fine with 8-bit raster maps as an input. If the data to be processed are out of this range, that is out of <math>[0, 255]</math>, they can be rescaled to fit into this range by using GRASS' {{cmd|r.rescale}} module.

Given an IKONOS set of 11-bit spectral bands (Blue, Green, Red, NIR and Pan) ranging between <math>[0, 2047]</math>, and then querying for example the Blue band
<source lang="bash" enclose=none>r.info Blue_DNs -r</source>, would return

<source lang="bash">
min=0
max=2047
</source>

===== Rescaling single bands =====

Rescaling the Blue band to range between `[0, 255]`
<source lang="bash">
r.rescale in=Blue_DNs out=Blue_DNs_255 from=0,2047 to=0,255
</source>

The same step applies to both the rest of the multi-spectral bands and the Panchromatic band of interest. If working under Bash, To repeat the same command (given, it was the last on used in the GRASS' terminal), one can isntruct

<source lang="bash">
!!:gs/Blue/Green
</source>

This would substitute everywhere found on the last command the string "Blue" with the string "Green" and re-execute it. The same can be done for the rest of the bands, namely Red, NIR and Pan. Some attention is required to instruct substitution of the ''last'' used string.

===== Rescaling all bands at once =====

Of course, one can always use a ''for'' loop

<source lang="bash">
for DN in `g.mlist rast pat=*DNs`; do r.rescale in=${DN} out=${DN}_255 from=0,2047 to=0,255; done
</source>

===== Pan-Sharpening =====

As usual when working with GRASS, it is required to set the region of interest, i.e. <source lang="bash" enclose=none>g.region rast=Blue_DNs_255</source> to match the extent of the band(s) or else. The resolution itself is taken care in this particular case by the module and the resulting pan-sharpened raster maps will of the same high(er) resolution as the Panchromatic band.

An example command for an IHS-based Pan-Sharpening action might look like

<source lang="bash">
i.pansharpen pan=Pan_DNs_255 ms1=Blue_DNs_255 ms2=Green_DNs_255 ms3=Red_DNs_255 output=sharptest255 sharpen=ihs
</source>

===== Color Re-Balancnig =====

After the process completion, the module outputs

<source lang="bash">
...
The following pan-sharpened output maps have been generated:
sharptest255_red
sharptest255_green
sharptest255_blue

To visualize output, run: g.region -p rast=sharptest255.red
d.rgb r=sharptest255_red g=sharptest255_green b=sharptest255_blue

If desired, combine channels into a single RGB map with 'r.composite'.
</source>

Normally it should be enough to re-balance the colors after the pan-sharpening action by using for example the {{cmd|i.landsat.rgb}} module or manual adjustment of each of the three bands that would compose an RGB image.

<source lang="bash">
i.landsat.rgb r=sharptest255_red g=sharptest255_green b=sharptest255_blue -p
</source>

===== Automatising Sharpening & RGB Composition =====

An ''experimental'' bash script to automatise the sharpening/fusion and RGB composition process is demonstrated here. The script uses the Spectral Reflectance values (double precision values), gained from previous steps (see above), and produces Pan-sharpened images by applying all of the three methods that {{addon|i.pansharpen}} offers. In addition, it attempts to produce True-Color composite images, without and with re-balancing the color tables by using the {{cmd|i.landsat.rgb}} module.

Initial tests indicate that at least one of the three sharpening methods, combined with re-balancing, produces very clear and balanced True-Color composites. Note, there is no 100% confidence that it will produce nice looking True-Color composites. Some of the methods might simply work, others might deliver fancy-colored images. As usual, some manual action might be required to get the re-balancing to work as best as possible.

The script can be expanded in terms of using more inputs by utilising bash's positional parameters. However, care has to be taken to alter the instruction that concerns the conversion from double precision values to 8-bit integers.

<source lang="bash">
#!/bash/sh

# use in G7 only :-)

echo "Rescaling all images to 8-bit"
# convert to 8-bit first :-!
for Method_Input in \
"Pan ToAR" \
"Blue ToAR" \
"Green ToAR" \
"Red ToAR" \
"NIR ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo -e "\n" #echo $1 $2

# input BAND is...
eval BAND="${1}_${2}"

# set region
g.region rast=${BAND}

# integerise
r.mapcalc --v \
"${BAND}_integerised = round( 1000000 * ${BAND} )" # how many decimals?

# integerised BAND is...
eval BAND="${1}_${2}_integerised"

# get ${min} and ${max}
eval `r.info -r ${BAND}`

#
r.rescale in=${1}_${2}_integerised out=${1}_${2}_255 from=${min},${max} to=0,255

done ########################################################################
echo "Rescaling done!"

# pan-sharpen High Resolution imagery ---------------------------------------
# needs setting some naming conventions
for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# some echo
echo "Pan-Sharpening the <${2}> images based on the <${1}> method"

i.pansharpen \
sharpen=${1} \
pan=Pan_${2}_255 \
ms1=Blue_${2}_255 \
ms2=Green_${2}_255 \
ms3=Red_${2}_255 \
output=sharp_${1}_${2}_255

done ########################################################################

# compose BGRs --------------------------------------------------------------

# loop over Methods and Input Types, compose RGBs
for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# region
g.region rast=sharp_"${1}"_"${2}"_255_blue

# compose
r.composite --o \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue \
out=rgb_sharpened_"${1}"_"${2}"_255

done #######################################################################

# re-balance & re-compose colors --------------------------------------------

for Method_Input in \
"ihs ToAR" \
"brovey ToAR" \
"pca ToAR"

do

# parse "${Method_STR}" and set positional parameters
set -- $Method_Input ; echo $1 $2

# re-balance
i.landsat.rgb -p \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue

# region -- Really necessary, again?
g.region rast=sharp_"${1}"_"${2}"_255_blue

# re-compose
r.composite --o \
r=sharp_"${1}"_"${2}"_255_red \
g=sharp_"${1}"_"${2}"_255_green \
b=sharp_"${1}"_"${2}"_255_blue \
out=rgb_sharpened_"${1}"_"${2}"_255_rebalanced

done ########################################################################
</source>

==== Pan-Sharpening based on the HPFA technique ====

The technique, implemented via the GRASS add-on {{AddonSrc|imagery|i.fusion.hpf|version=7}}, as indicated above, involves a convolution using a High Pass Filter (HPF) on the high resolution data, then combining this with the lower resolution multispectral data.

''Source: "Optimizing the High-Pass Filter Addition Technique for Image Fusion", Ute G. Gangkofner, Pushkar S. Pradhan, and Derrold W. Holcomb (2008)''

The algorithm's steps are:

# Computing the ratio of the low (Multi-Spectral) to the high (Panchromatic) resolution
# High Pass Filtering the Panchromatic Image
# Resampling the Multi-Spectral image to the higher resolution
# Adding a weighted High-Pass-Filtered image to the upsampled Multi-Spectral image
# Optionally, matching the histogram of the Pan-Sharpened image to the one of the original Multi-Spectral image

=== Color Composites ===

''ToAdd''

== Vegetation Indices ==

''ToAdd''

== PCA ==

''ToAdd''

== IKONOS Image classification ==

See [[Image classification]]

== References / Sources ==

* [http://www.apollomapping.com/wp-content/user_uploads/2011/09/IKONOS_Esun_Calculations.pdf IKONOS Planetary Reflectance and Me an Solar Exoatmospheric Irradiance, by Martin Taylor]
* [http://web.unicen.edu.ar/crecic/docs/radrefl.pdf Ikonos DN Value Conversion to Planetary Reflectance Values, by David Fleming]
* [http://landsathandbook.gsfc.nasa.gov/data_prod/prog_sect11_3.html Landsat7 Science Data Users Handbook, Chapter 11, Section 3]
* [http://igett.delmar.edu/Resources/Remote%20Sensing%20Technology%20Training/Calculation-DN_to_Reflectance_Irish_20June08.pdf Calibrated Landsat Digital Number (DN) to Top of Atmosphere (TOA) Reflectance Conversion, by Richard Irish]
* from [http://landsat.usgs.gov/tools_access_all_faqs.php FAQs about the Landsat Missions]
** [http://landsat.usgs.gov/how_is_radiance_calculated.php How is radiance calculated?]
** [http://landsat.usgs.gov/at_sensor_reflectance_calculated.php How is at-sensor reflectance calculated?]

== See also ==

* GRASS-Wiki page about [http://grasswiki.osgeo.org/wiki/Image_processing Image Processing]

''ToAdd''

[[Category: Documentation]]
[[Category: FAQ]]
[[Category: IKONOS]]
[[Category: High Resolution]]
[[Category: Import]]
[[Category: Digital Numbers]]
[[Category: Radiance]]
[[Category: Reflectance]]
[[Category: Image processing]]