GRASS-Wiki - User contributions [en]

Support

2012-02-29T17:56:17Z

⚠️Sprice: /* Commercial support: Freelancers who offer GRASS Support */

== GRASS Community ==

To reach the community, see [[GRASS Community]].

== Commercial support: Freelancers who offer GRASS Support ==

You need a person with programming skills? Check out this list:

* Seth Price <seth@pricepages.org> - low-level raster processing, code optimization, OpenCL
* you
* ...

== Commercial support: Companies and Institutions which offer GRASS Support ==

{| class="wikitable sortable" cellspacing="1" cellpadding="1" border="1"
|-
! scope="col" | Country
! scope="col" | Name
! scope="col" | Competences
! scope="col" | Site
|-
| Germany
| GDF Hannover
| Project management, Data analysis
| http://www.gdf-hannover.de
|-
| Italy
| Fondazione Edmund Mach
| Project management, Data analysis, Programming
| http://gis.cri.fmach.it
|-
| Germany
| Geoinformatik Büro Dassau
| Project management, Training, Data analysis, Programming
| http://www.gbd-consult.de
|-
| your country
| ...
| ...
| ...
|}

[[Category: FAQ]]

R.sun

2011-09-06T06:32:40Z

⚠️Sprice: /* OpenCL */

== Help page ==

* {{cmd|r.sun}} manual page

== Tips ==

The speed of r.sun is much higher if {{cmd|r.horizon}} is used first and the resulting maps are given as input to r.sun. Background: the horizon needs to be computed only one time before, not in every step within r.sun. See the example at the end of the {{cmd|r.sun}} help page.

=== Testing ===

Create an artificial surface containing a Gaussian mound:
{{AddonCmd|r.surf.volcano}} out=gauss method=gaussian kurtosis=1

Overlay some 200m contours to show underlying topography:
r.contour in=gauss out=gauss_200m_contours step=200
d.vect gauss_200m_contours color=white

Set map's color table to highlight detail:
r.colors rad_test.day355.beam col=bcyr -e
d.legend rad_test.day355.beam range=1300,1500

=== Time step ===

The following three images demonstrate the effects of using different time '''step''' parameters.

r.sun -s elevin="gauss" glob_rad="rad.global.30minT" day=180 step=0.5

[[File:Gauss_30min.png|400px|thumb|center|Default 30 minute time step over a Gaussian mound.]]

r.sun -s elevin="gauss" glob_rad="rad.global.15minT" day=180 step=0.25

[[File:Gauss_15min.png|400px|thumb|center|15 minute time step over a Gaussian mound.]]

r.sun -s elevin="gauss" glob_rad="rad.global.03minT" day=180 step=0.05

[[File:Gauss_03min.png|400px|thumb|center|3 minute time step over a Gaussian mound.]]

The 3 minute time step takes roughly ten times as long to run as the 30 minute timestep.

== Seed maps ==

Using seed maps can greatly (?, unclear) speed up processing. This is especially important if you will batch process e.g. for every day of the year.

* aspin= and slopein= options: create slope and aspect maps with the {{cmd|r.slope.aspect}} module.
: ''Caution: currently buggy''. See {{trac|498}}.
: Estimated speedup: ?%.

* horizon= and horizonstep= options: Pre-calculate horizon shadows by creating a series of horizon rasters with the {{cmd|r.horizon}} module.
: Results not as smooth as without using this option?? See {{trac|498}}.
: Estimated speedup: ?%.

* latin= and longin= options: Pre-calculate latitudes and longitudes for each raster cell.
The following script will create a raster containing latitude as the raster value:
<source lang="bash">
# create latitude map (WGS84)
g.region rast=elevation.dem
r.mapcalc one=1
r.out.xyz one | \
cut -f1,2 -d'|' | \
m.proj -oed --quiet | \
sed -e 's/[ \t]/|/g' | \
cut -f1-4 -d'|' | \
r.in.xyz in=- z=4 out=elevation.lat
g.remove one
</source>

Creating the longitude map is exactly the same but use column 3 of the {{cmd|m.proj}} output instead of column 4:
<source lang="bash">
# create longitude map (WGS84)
g.region rast=elevation.dem
r.mapcalc one=1
r.out.xyz one | \
cut -f1,2 -d'|' | \
m.proj -oed --quiet | \
sed -e 's/[ \t]/|/g' | \
cut -f1-4 -d'|' | \
r.in.xyz in=- z=3 out=elevation.lon
g.remove one
</source>

: Estimated speedup: 1.3% (in one test).
: See {{trac|498}}.

The above assumes that your projection's ellipsoid is WGS84. If it isn't, use other proj_out= terms with m.proj as required instead of the -o flag.

== Automation ==

It can be tedious to set up and run a long series of r.sun simulations for every day of the year. To automate this we can write a small shell script loop:

<source lang="bash">
for DAY in `seq 1 365` ; do
DAY_STR=`echo $DAY | awk '{printf("%.03d", $1)}'`
echo "Processing day $DAY_STR at `date` ..."

LINKE="`g.linke_by_day.py $DAY`"

r.sun -s elevin=elevation.dem day=$DAY lin=$LINKE step=0.05 \
beam_rad=rad_beam.$DAY_STR diff_rad=rad_diffuse.$DAY_STR \
refl_rad=rad_reflected.$DAY_STR glob_rad=rad_global.$DAY_STR \
insol_time=rad_insol_time.$DAY_STR
done
</source>

* [https://trac.osgeo.org/grass/browser/grass-addons/raster/r.sun.tools g.linke_by_day.py] is a small program which will return the Linke coefficient for any day of the year, interpolated from monthly values. You need to edit the script to fill in values appropriate for your study area.

* If you have a multi-core CPU and you'd like to speed things up, here is a small Bourne shell script implementing a poor-man's multi-processing trick:

<source lang="bash">
### r.sun mode 2 loop ###
BEGIN=1
END=365
STEP=1
NUM_CORES=4

for DAY in `seq $BEGIN $STEP $END` ; do
DAY_STR=`echo $DAY | awk '{printf("%.03d", $1)}'`
echo "Processing day $DAY_STR at `date` ..."

LINKE="`g.linke_by_day.py $DAY`"

CMD="r.sun -s elevin=elevation.dem day=$DAY lin=$LINKE step=0.05 \
beam_rad=rad_beam.$DAY_STR diff_rad=rad_diffuse.$DAY_STR \
refl_rad=rad_reflected.$DAY_STR glob_rad=rad_global.$DAY_STR \
insol_time=rad_insol_time.$DAY_STR --quiet"

# poor man's multi-threading for a multi-core CPU
MODULUS=`echo "$DAY $STEP $NUM_CORES" | awk '{print $1 % ($2 * $3)}'`
if [ "$MODULUS" = "$STEP" ] ; then
# stall to let the background jobs finish
$CMD
sleep 2
#while [ `pgrep -c r.sun` -ne 0 ] ; do
# sleep 5
#done
else
$CMD &
fi
done
</source>

* For a r.sun Mode 1 loop, see the [[OpenMP#Alternatives|OpenMP]] page.

== Overview of monthly maps ==

Given there are mean monthly global irradiation maps calculated, an overview map of all 12 maps can be drawn along with their legends (see also [http://grass.osgeo.org/wiki/Time_series#Common_legends_for_many_raster_maps Common legends for many raster maps]). The script assumes a common naming convention for all 12 monthly average maps (i.e. global_rad_avg.jan, global_rad_avg.feb, global_rad_avg.mar, etc.).

<source lang="bash">
#!/bash/sh
# script to draw monthly (mean) global solar irradiation maps in a 3x4 matrix

# set wide aspect ratio (16:9, e.g. 1366 width x 768 height)
d.monsize setm=x0 setw=1366 seth=768

# split in 12 frames
d.split.frame frames=12

# preferred font?
d.font DroidSans

for FRAME in "uno dec December" "dos jan January" "tres feb February" \
"cuatro mar March" "cinco apr April" "seis may May" \
"siete jun June" "ocho jul July" "nueve aug August" \
"diez sep September" "once oct October" "doce nov November"

do

# parse "${FRAMES_STR}" and set positional parameters
set -- $FRAME ; echo $1 $2 $3

# select FRAME
d.frame -s "${1}"

# draw the map
d.rast global_rad_avg."${2}"

# draw label on the left and vertically
d.text text="${3}" size=10 color=50:50:50 at=9,25 rotation=90

# draw legend
d.legend global_rad_avg."${2}" -s at=10,90,89,92

done
</source>

[[File:Mean_monthly_global_rad_maps_over_a_study_area_in_central_Greece.png|400px|thumb|center|Mean monthly global irradiation maps (n=38.918, s=38.531, w=21.793, e=22.436)]]
Same image also [http://grass.osgeo.org/grass-wiki/images/Mean_monthly_global_rad_maps_over_a_study_area_in_central_Greece_700px_height.jpg here (a bit of higher resolution)]

== Development ==

* Ongoing trials documented in [https://trac.osgeo.org/grass/ticket/498 trac ticket #498]

=== OpenCL ===
As part of the Google Summer of Code 2010 an {{wikipedia|OpenCL}} version has been written that allows ''r.sun'' to run on {{wikipedia|GPU}}s. This provides a massive speedup in processing time. The merging of this code into GRASS 7's r.sun is forthcoming (HB).
* To get your hands on the code now see http://github.com/mailseth/OpenCL-integration-for-GRASS---GDAL (functional prototype now available)
* To use OpenCL GPU functionality you'll want a graphics card like the ATI HD5770 or nVidia GTX260 or GTX460, or newer.
* OpenCL also allows for running on multi-core (or multi-processor) CPUs, for systems without GPUs. You can run OpenCL via multicore *or* GPU, but not both on the same job at the same time.
* Seth wrote: ''The OpenCL version of r.sun runs over 20x faster than the original version on my machine (2.26 GHz Mac Pro vs. GeForce GTX 285). However, it is hampered by the low memory on your GPU, so you may need to partition your raster.''

== ToDo ==

* Merge in OpenCL code
* Add support for [[OpenMP]] parallelization (if OpenCL doesn't make that redundant)

[[Category: Documentation]]
[[Category: Parallelization]]

R.sun

2010-08-27T16:13:50Z

⚠️Sprice: /* Development */ link to OpenCL development

== Help page ==

* {{cmd|r.sun}} manual page

== Tips ==

The speed of r.sun is much higher if {{cmd|r.horizon}} is used first and the resulting maps are given as input to r.sun. Background: the horizon needs to be computed only one time before, not in every step within r.sun. See the example at the end of the {{cmd|r.sun}} help page.

=== Testing ===

Create an artificial surface containing a Gaussian mound:
{{AddonCmd|r.surf.volcano}} out=gauss method=gaussian kurtosis=1

Overlay some 200m contours to show underlying topography:
r.contour in=gauss out=gauss_200m_contours step=200
d.vect gauss_200m_contours color=white

Set map's color table to highlight detail:
r.colors rad_test.day355.beam col=bcyr -e
d.legend rad_test.day355.beam range=1300,1500

=== Time step ===

The following three images demonstrate the effects of using different time '''step''' parameters.

r.sun -s elevin="gauss" glob_rad="rad.global.30minT" day=180 step=0.5

[[File:Gauss_30min.png|400px|thumb|center|Default 30 minute time step over a Gaussian mound.]]

r.sun -s elevin="gauss" glob_rad="rad.global.15minT" day=180 step=0.25

[[File:Gauss_15min.png|400px|thumb|center|15 minute time step over a Gaussian mound.]]

r.sun -s elevin="gauss" glob_rad="rad.global.03minT" day=180 step=0.05

[[File:Gauss_03min.png|400px|thumb|center|3 minute time step over a Gaussian mound.]]

The 3 minute time step takes roughly ten times as long to run as the 30 minute timestep.

== Seed maps ==

Using seed maps can greatly (?, unclear) speed up processing. This is especially important if you will batch process e.g. for every day of the year.

* aspin= and slopein= options: create slope and aspect maps with the {{cmd|r.slope.aspect}} module.
: ''Caution: currently buggy''. See {{trac|498}}.
: Estimated speedup: ?%.

* horizon= and horizonstep= options: Pre-calculate horizon shadows by creating a series of horizon rasters with the {{cmd|r.horizon}} module.
: Results not as smooth as without using this option?? See {{trac|498}}.
: Estimated speedup: ?%.

* latin= and longin= options: Pre-calculate latitudes and longitudes for each raster cell.
The following script will create a raster containing latitude as the raster value:
<source lang="bash">
# create latitude map (WGS84)
g.region rast=elevation.dem
r.mapcalc one=1
r.out.xyz one | \
cut -f1,2 -d'|' | \
m.proj -oed --quiet | \
sed -e 's/[ \t]/|/g' | \
cut -f1-4 -d'|' | \
r.in.xyz in=- z=4 out=elevation.lat
g.remove one
</source>

Creating the longitude map is exactly the same but use column 3 of the {{cmd|m.proj}} output instead of column 4:
<source lang="bash">
# create longitude map (WGS84)
g.region rast=elevation.dem
r.mapcalc one=1
r.out.xyz one | \
cut -f1,2 -d'|' | \
m.proj -oed --quiet | \
sed -e 's/[ \t]/|/g' | \
cut -f1-4 -d'|' | \
r.in.xyz in=- z=3 out=elevation.lon
g.remove one
</source>

: Estimated speedup: 1.3% (in one test).
: See {{trac|498}}.

The above assumes that your projection's ellipsoid is WGS84. If it isn't, use other proj_out= terms with m.proj as required instead of the -o flag.

== Automation ==

It can be tedious to set up and run a long series of r.sun simulations for every day of the year. To automate this we can write a small shell script loop:

<source lang="bash">
for DAY in `seq 1 365` ; do
DAY_STR=`echo $DAY | awk '{printf("%.03d", $1)}'`
echo "Processing day $DAY_STR at `date` ..."

LINKE="`g.linke_by_day.py $DAY`"

r.sun -s elevin=elevation.dem day=$DAY lin=$LINKE step=0.05 \
beam_rad=rad_beam.$DAY_STR diff_rad=rad_diffuse.$DAY_STR \
refl_rad=rad_reflected.$DAY_STR glob_rad=rad_global.$DAY_STR \
insol_time=rad_insol_time.$DAY_STR
done
</source>

* [https://trac.osgeo.org/grass/browser/grass-addons/raster/r.sun.tools g.linke_by_day.py] is a small program which will return the Linke coefficient for any day of the year, interpolated from monthly values. You need to edit the script to fill in values appropriate for your study area.

* If you have a multi-core CPU and you'd like to speed things up, here is a small Bourne shell script implementing a poor-man's multi-processing trick:

<source lang="bash">
### r.sun mode 2 loop ###
BEGIN=1
END=365
STEP=1
NUM_CORES=4

for DAY in `seq $BEGIN $STEP $END` ; do
DAY_STR=`echo $DAY | awk '{printf("%.03d", $1)}'`
echo "Processing day $DAY_STR at `date` ..."

LINKE="`g.linke_by_day.py $DAY`"

CMD="r.sun -s elevin=elevation.dem day=$DAY lin=$LINKE step=0.05 \
beam_rad=rad_beam.$DAY_STR diff_rad=rad_diffuse.$DAY_STR \
refl_rad=rad_reflected.$DAY_STR glob_rad=rad_global.$DAY_STR \
insol_time=rad_insol_time.$DAY_STR --quiet"

# poor man's multi-threading for a multi-core CPU
MODULUS=`echo "$DAY $STEP $NUM_CORES" | awk '{print $1 % ($2 * $3)}'`
if [ "$MODULUS" = "$STEP" ] ; then
# stall to let the background jobs finish
$CMD
sleep 2
#while [ `pgrep -c r.sun` -ne 0 ] ; do
# sleep 5
#done
else
$CMD &
fi
done
</source>

* For a r.sun Mode 1 loop, see the [[OpenMP#Alternatives|OpenMP]] page.

== Overview of monthly maps ==

Given there are mean monthly global irradiation maps calculated, an overview map of all 12 maps can be drawn along with their legends (see also [http://grass.osgeo.org/wiki/Time_series#Common_legends_for_many_raster_maps Common legends for many raster maps]). The script assumes a common naming convention for all 12 monthly average maps (i.e. global_rad_avg.jan, global_rad_avg.feb, global_rad_avg.mar, etc.).

<source lang="bash">
#!/bash/sh
# script to draw monthly (mean) global solar irradiation maps in a 3x4 matrix

# set wide aspect ratio (16:9, e.g. 1366 width x 768 height)
d.monsize setm=x0 setw=1366 seth=768

# split in 12 frames
d.split.frame frames=12

# preferred font?
d.font DroidSans

for FRAME in "uno dec December" "dos jan January" "tres feb February" \
"cuatro mar March" "cinco apr April" "seis may May" \
"siete jun June" "ocho jul July" "nueve aug August" \
"diez sep September" "once oct October" "doce nov November"

do

# parse "${FRAMES_STR}" and set positional parameters
set -- $FRAME ; echo $1 $2 $3

# select FRAME
d.frame -s "${1}"

# draw the map
d.rast global_rad_avg."${2}"

# draw label on the left and vertically
d.text text="${3}" size=10 color=50:50:50 at=9,25 rotation=90

# draw legend
d.legend global_rad_avg."${2}" -s at=10,90,89,92

done
</source>

[[File:Mean_monthly_global_rad_maps_over_a_study_area_in_central_Greece.png|400px|thumb|center|Mean monthly global irradiation maps (n=38.918, s=38.531, w=21.793, e=22.436)]]
Same image also [http://grass.osgeo.org/grass-wiki/images/Mean_monthly_global_rad_maps_over_a_study_area_in_central_Greece_700px_height.jpg here (a bit of higher resolution)]

== Development ==

* Ongoing trials documented in [https://trac.osgeo.org/grass/ticket/498 trac ticket #498]
* Is being written in OpenCL to be run on GPUs for a massive speedup. [http://github.com/mailseth/OpenCL-integration-for-GRASS---GDAL]

== ToDo ==

* Add support for [[OpenMP]] parallelization

[[Category: Documentation]]
[[Category: Parallelization]]

GPU

2010-04-02T17:06:00Z

⚠️Sprice: more float usage

Comments from the mailing list concerning GRASS and GPU parallelization:

* [http://lists.osgeo.org/pipermail/grass-dev/2008-March/thread.html#36083 Discussion] - GPU Parallelization (follow thread)
* [http://lists.osgeo.org/pipermail/grass-dev/2009-September/thread.html#46072 Discussion] - OpenCL Parallelization (follow thread)
* [http://lists.osgeo.org/pipermail/grass-dev/2008-March/036109.html Comment]
* [http://lists.osgeo.org/pipermail/grass-user/2009-December/053476.html Comment]

* As I understand it, CUDA is 100% dependent on the closed-source binary driver from nVidia and works on their video cards alone. Which is fine for today for people with nVidia hardware using their binary video card driver. If nVidia decides in a couple of years to stop supporting CUDA, your old card, your specific OS or distro, your OS or distro version+cpu type, or if they go out of business or are bought/sold to another company who is not interested, any code based on it becomes useless. For this reason code written for an open platform such as OpenCL, even if less advanced, seems to have a brighter long-term future. -- ''HB''

* Support for double precision floating point values must be retained for calculations which deal with positional data. For elevation and radiometric data floating point precision may be enough.

== Further reading ==

* LINUX Magazine March 10th, 2010
: "''GP-GPUs: OpenCL Is Ready For The Heavy Lifting''"
: http://www.linux-mag.com/id/7725

* http://www.hpcwire.com/offthewire/Khronos-Demonstrates-OpenCL-Momentum-at-SC09-70243307.html

* See the "'''Parallelization'''" category listing at the bottom of this page.

* OpenCL podcasts: http://www.macresearch.org/opencl

== Interesting Hardware ==

* [http://www.nvidia.com/object/product_geforce_gtx_480_us.html NVIDIA GeForce 480], packed with 3 billion transistors, 480 visual processing cores, 16 geometry units and 4 raster units. Multi-card SLI provides additional 90% performance boost.

== Modules of interest to be parallelized ==

The target version will be GRASS 7 (alias SVN trunk).

* {{cmd|v.in.ogr}} or <u>underlying vector library functions to build topology and spatial index</u>
* {{cmd|v.surf.rst}}
* {{cmd|v.vol.rst}}
** (''probably best to focus on the RST library first'')
* {{AddonCmd|r.viewshed}}
* {{cmd|r.sun}}
* {{cmd|r.proj}}
* {{cmd|v.proj}}
* {{cmd|v.net}}.* ???
* raster library
* ...
* <strike>{{cmd|r.mapcalc}}</strike> (already has pthreads support; probably I/O-bound)
* <strike>{{cmd|r.los}}</strike> (to be replaced by r.viewshed after last few bugs are fixed)
* <strike>{{cmd|i.rectify}}</strike> (internal code to be replaced by GDALwarp API)

[[Category: Parallelization]]

GPU

2010-04-02T17:04:31Z

⚠️Sprice: /* Further reading */ added podcasts

Comments from the mailing list concerning GRASS and GPU parallelization:

* [http://lists.osgeo.org/pipermail/grass-dev/2008-March/thread.html#36083 Discussion] - GPU Parallelization (follow thread)
* [http://lists.osgeo.org/pipermail/grass-dev/2009-September/thread.html#46072 Discussion] - OpenCL Parallelization (follow thread)
* [http://lists.osgeo.org/pipermail/grass-dev/2008-March/036109.html Comment]
* [http://lists.osgeo.org/pipermail/grass-user/2009-December/053476.html Comment]

* As I understand it, CUDA is 100% dependent on the closed-source binary driver from nVidia and works on their video cards alone. Which is fine for today for people with nVidia hardware using their binary video card driver. If nVidia decides in a couple of years to stop supporting CUDA, your old card, your specific OS or distro, your OS or distro version+cpu type, or if they go out of business or are bought/sold to another company who is not interested, any code based on it becomes useless. For this reason code written for an open platform such as OpenCL, even if less advanced, seems to have a brighter long-term future. -- ''HB''

* Support for double precision floating point values must be retained for calculations which deal with positional data. For elevation data floating point precision may be enough.

== Further reading ==

* LINUX Magazine March 10th, 2010
: "''GP-GPUs: OpenCL Is Ready For The Heavy Lifting''"
: http://www.linux-mag.com/id/7725

* http://www.hpcwire.com/offthewire/Khronos-Demonstrates-OpenCL-Momentum-at-SC09-70243307.html

* See the "'''Parallelization'''" category listing at the bottom of this page.

* OpenCL podcasts: http://www.macresearch.org/opencl

== Interesting Hardware ==

* [http://www.nvidia.com/object/product_geforce_gtx_480_us.html NVIDIA GeForce 480], packed with 3 billion transistors, 480 visual processing cores, 16 geometry units and 4 raster units. Multi-card SLI provides additional 90% performance boost.

== Modules of interest to be parallelized ==

The target version will be GRASS 7 (alias SVN trunk).

* {{cmd|v.in.ogr}} or <u>underlying vector library functions to build topology and spatial index</u>
* {{cmd|v.surf.rst}}
* {{cmd|v.vol.rst}}
** (''probably best to focus on the RST library first'')
* {{AddonCmd|r.viewshed}}
* {{cmd|r.sun}}
* {{cmd|r.proj}}
* {{cmd|v.proj}}
* {{cmd|v.net}}.* ???
* raster library
* ...
* <strike>{{cmd|r.mapcalc}}</strike> (already has pthreads support; probably I/O-bound)
* <strike>{{cmd|r.los}}</strike> (to be replaced by r.viewshed after last few bugs are fixed)
* <strike>{{cmd|i.rectify}}</strike> (internal code to be replaced by GDALwarp API)

[[Category: Parallelization]]