GRASS-Wiki - User contributions [en]

Image processing/Evapotranspiration

2023-12-18T08:01:50Z

⚠️Ychemin: /* Introduction */

== Introduction ==
The '''radiation''' is coming from the planet's star, Earth has around 1800 W/m2 of incoming radiation from the sun (also called irradiance) at exo-atmospheric altitude.

Atmospheric transfer of '''irradiance''' is subject to atmosphere particles interactions, scattering, diffraction, reflection, etc.

When reaching the planetary surface, is it partially reflected in the shortwave by '''Albedo'''.

Surface Albedo is the integrated 0.3-3 micro-meters reflectance, it reflects the shortwave energy coming from the sun by 5% on oceans, 15-25% on vegetation, 35-40% on sand/desert/beach, 60-80% on snow/clouds, roughly. This energy fraction is returned to atmosphere for complex interaction with gas particles again before leaving the atmosphere altogether.

[[File:EnergyBalance.png||500px||Alt="Chemin, 2006, PhD Thesis"]]

As for the shortwave surface balance, the '''energy''' is also received in the longwave, but interacts with the grey-body characteristics of the surface elements.

The blackbody to greybody fraction is ruled by emissivity, a hidden component in the thermal spectrum (search for Temperature- Emissivity Separation algorithms). On Earth, emissivity is always above 0.9 (mostly 0.96-0.98). The Stefan-Boltzman equation is dealing with blackbody energy, and multiplying it by Emissivity transforms it to greybody energy emitted.

Together, shortwave and longwave '''energy balance''' provide with the net radiation balance at the surface of the planetary body. This crucial term is the total energy available for thermodynamic fluxes to act on the surface of the planet.

On a planet like Earth where there is a triple phase of a given molecule (H20, in gas, liquid, solid), energy available is used to transfer phases from lower energy to higher energy (sublimation, liquefaction, evaporation). Thermal transfers also happen as conduction (soil, rocks) and convection (atmosphere, oceans).

The i.eb.* modules and models deal with:
* conduction {{cmd|i.eb.soilheatflux}} (for thermal conduction in soils)
* convection i.eb.h_* especially (for thermal convection in atmosphere)
* residual method estimating the energy left for evaporation processes ({{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} are examples).

Additionally, a helper module {{AddonCmd|i.eb.z0m}} is producing an estimation of the surface roughness, necessary in latent heat flux estimation.

For a detailed theory and modelling information behind these modules, please read the following document: [[File:Evaporation_manual_v0.6.7.pdf]]

== i.evapo.* modules/models are integrated '''models'''==

from the oldest that are not based on energy balance, but are computing some form of evapotranspiration (ETo: Evapotranspiration for a reference of 20cm well-watered grass, see FAO no56 report of Richard Allen)
* Penman-Monteith {{cmd|i.evapo.pm}}
* Priestley-Taylor {{cmd|i.evapo.pt}}
* Hargreaves {{cmd|i.evapo.mh}}

to newer that are in some form or another linked to thermal processes.
* Thermal index {{cmd|i.evapo.senay}}
* Biome thermal processes {{cmd|i.evapo.zk}} (global model based on typical info from biomes Albedo/soil heat flux)
* Thermodynamic processes {{cmd|i.evapo.potrad}} (potential ET if not water stress, based on astronomical equations only)

Most developed thermodynamic models are:
* TSEB (i.evapo.tseb: Two Source Energy Balance, Schmugge, Kustas, etc., 2000) Status: Incomplete Module
* SEBS (i.evapo.sebs: Zhu et al, 2002) Status: Incomplete Module
* SEBAL {{cmd|i.eb.h_sebal01}}, {{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} (Bastiaanssen et al, 1998, 2001)

and some newer models like RESET and others. The list of models using thermodynamic principles is large now, and any willing to submit a code for any of them is most welcome.

== Temporal Integration of Evapotranspiration ==

When integrated in time {{cmd|i.evapo.time}}, seasonal evapotranspiration can be computed and therefore permit a full monitoring of vegetation growth over the period. This is also a basis of crop yield estimation (see {{cmd|i.biomass}}).

[[File:I_evapo_time.png||800px||Alt="Temporal integration of ETa with data from a weather station"]]

== See also ==

* Manual pages:
** http://grass.osgeo.org/grass-stable/manuals/topic_energy_balance.html
** http://grass.osgeo.org/grass-stable/manuals/topic_evapotranspiration.html
** http://grass.osgeo.org/grass-stable/manuals/keywords.html#evaporative%20fraction
* [https://svn.osgeo.org/grass/grass-promo/tutorials/grass_landsat_ETa/ Evapotranspiration mapping with Landsat 5TM in GRASS GIS]

== Example script to process Landsat 8 Evapotranspiration ==
<source lang=shell>
#!/bin/bash

# This scripts imports, pre-processes and compute Evapotranspiration for Landsat 8
# Run in a Location compatible with your L8 dataset

# Atmospheric Transmissivity single-way
tsw=0.75
# Light Use Efficiency (please change dependent on your crop/plant)
lue=1.0

# Import files in GRASS
for file in *TIF
do
r.in.gdal -e input=$file output=$(echo $file | sed 's/.TIF//') --o
done
# Set region to newly imported files
g.region -s raster=$(ls *TIF | grep B1.TIF | sed 's/.TIF//')
LC08=$(ls *TIF | grep B1.TIF | sed 's/1.TIF//')

# Set metadata MTL file name
LC08met=$(ls *MTL.txt)

# Import DEM
r.import input=dem.tif output=dem extent=region --o
# Extract date
date=$(i.landsat.toar input=$LC08 output=l8 metfile=$LC08met lsatmet=date --o)
echo $date

# Apply Top of Atmosphere correction
i.landsat.toar input=$LC08 output=l8 metfile=$LC08met --o

# Set NODATA
for file in $(g.list type=raster pattern=LC08*)
do
r.null map=$file setnull=0
done

#Set computing area to B10 active pixels
for n in 1 2 3 4 5 6 7 8 9
do
r.mapcalc expression="temp=if(isnull(l810),null(),l8$n)" --o
g.rename raster=temp,l8$n --o
done

base=l8
# Create config files for 6s
year=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\1/')
mm=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\2/')
dd=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\3/')
doy=$( date -I -d "$year-$mm-$dd" | date +%j)
hh=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\1/')
mn=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\2/')
ss=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\3/')
time=$(echo "$hh + ($mn / 60.0) + ($ss / 3600.0)" | bc -l)
lat=$(echo $(cat *MTL* | grep CORNER_ | grep LAT | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
lon=$(echo $(cat *MTL* | grep CORNER_ | grep LON | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
echo "18 - geometrical conditions=Landsat 8" > 6s_conf.txt
echo "$mm $dd $hh.$mm $lon $lat - month day hh.ddd longitude latitude ("hh.ddd" is in GMT decimal hours)" >> 6s_conf.txt
echo "1 - atmospheric model=midlatitude summer" >> 6s_conf.txt
echo "1 - aerosols model=continental" >> 6s_conf.txt
echo "24 - visibility [km] (aerosol model concentration)" >> 6s_conf.txt
echo "-0.110 - mean target elevation above sea level [km]" >> 6s_conf.txt
echo "-1000 - sensor on board a satellite" >> 6s_conf.txt
#For Band 1:
cat 6s_conf.txt > $base\1_conf.txt
echo "115 - Coastal band of OLI Landsat 8" >> $base\1_conf.txt
#For Band 2:
cat 6s_conf.txt > $base\2_conf.txt
echo "116 - Blue band of OLI Landsat 8" >> $base\2_conf.txt
#For Band 3:
cat 6s_conf.txt > $base\3_conf.txt
echo "117 - Green band of OLI Landsat 8" >> $base\3_conf.txt
#For Band 4:
cat 6s_conf.txt > $base\4_conf.txt
echo "118 - Red band of OLI Landsat 8" >> $base\4_conf.txt
#For Band 5:
cat 6s_conf.txt > $base\5_conf.txt
echo "120 - NIR band of OLI Landsat 8" >> $base\5_conf.txt
#For Band 6:
cat 6s_conf.txt > $base\6_conf.txt
echo "122 - SWIR1 band of OLI Landsat 8" >> $base\6_conf.txt
#For Band 7:
cat 6s_conf.txt > $base\7_conf.txt
echo "123 - SWIR2 band of OLI Landsat 8" >> $base\7_conf.txt
#For Band 8:
cat 6s_conf.txt > $base\8_conf.txt
echo "119 - PAN band of OLI Landsat 8" >> $base\8_conf.txt
#For Band 9:
cat 6s_conf.txt > $base\9_conf.txt
echo "121 - Cirrus band of OLI Landsat 8" >> $base\9_conf.txt

# Compute Atmospheric correction
for file in $base\1 $base\2 $base\3 $base\4 $base\5 $base\6 $base\7 $base\8 $base\9
do
i.atcorr -r $file elevation=dem parameters=$file\_conf.txt output=s$file --o
r.mapcalc expression="temp=if(isnull(l810),null(),s$file)" --o
g.rename raster=temp,s$file --o
r.colors map=$file color=grey
done

# Create cloud Mask
g.rename raster=l83,l7.2
g.rename raster=l84,l7.3
g.rename raster=l85,l7.4
g.rename raster=l86,l7.5
g.rename raster=l810,l7.6
i.landsat.acca -5 -f input=l7. output=temp --o
g.rename raster=l7.2,l83
g.rename raster=l7.3,l84
g.rename raster=l7.4,l85
g.rename raster=l7.5,l86
g.rename raster=l7.6,l810
r.mapcalc expression="temp1=if(isnull(temp),0,1)" --o
r.mapcalc expression="maskVIS=if(isnull(l810),null(),temp1)" --o

# Apply mask
r.mask raster=maskVIS maskcats=0

# Latitude map
r.latlong input=l81 output=latitude --o

# Longitude map
r.latlong -l input=l81 output=longitude --o

# i.albedo
i.albedo -8 -c input=sl81,sl82,sl83,sl84,sl85,sl86,sl87 output=l8alb --o
r.colors -e map=l8alb color=grey

# i.vi viname=ndvi
i.vi viname=ndvi red=sl84 nir=sl85 output=l8ndvi --o
r.colors map=l8ndvi color=ndvi

# Update processing mask with water
r.mapcalc expression="maskWATER=if(l8ndvi<=0,0,maskVIS)" --o
r.mask -r
r.mask raster=maskWATER maskcats=0

# i.emissivity
i.emissivity input=l8ndvi output=l8emis --o

# i.vi viname=savi
i.vi viname=savi red=sl84 nir=sl85 output=l8savi --o
r.colors map=l8savi color=ndvi

# Daily Net Radiation (RNETD)
r.mapcalc expression="temp=if(isnull(l810),null(),dem)" --o
r.slope.aspect elevation=temp slope=slopedd aspect=aspectdd --o
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=rnetd insol_time=sunhours day=$doy nprocs=8 --o
r.mapcalc expression="l8rnetd=rnetd/sunhours" --o

# Instantaneous Net Radiation (RNET)
timelocal=$(echo "$time + (24.0 / 360.0) * $lon" | bc -l)
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=l8rnet day=$doy time=$timelocal nprocs=8 --o

# Soil Heat Flux (G0)
r.mapcalc expression="overpasstime=$time" --o
i.eb.soilheatflux albedo=l8alb ndvi=l8ndvi temperature=l810 netradiation=l8rnet localutctime=overpasstime output=l8g0 --o

# Sensible Heat Flux (H)
i.eb.z0m -p input=l8savi output=z0m --o
r.mapcalc expression="T0dem=0.00612*dem+l810" --o
i.eb.hsebal01 -a netradiation=l8rnet soilheatflux=l8g0 aerodynresistance=z0m temperaturemeansealevel=T0dem frictionvelocitystar=0.32407 vapourpressureactual=1.511 output=l8h --o

# Evaporative Fraction & Soil Moisture
i.eb.evapfr -m netradiation=l8rnet soilheatflux=l8g0 sensibleheatflux=l8h evaporativefraction=l8evapfr soilmoisture=l8sm --o

# Actual Evapotranspiration (ETa)
i.eb.eta netradiationdiurnal=l8rnetd evaporativefraction=l8evapfr temperature=l810 output=l8eta --o

# Biomass growth
r.mapcalc expression="l8fpar=1.257*l8ndvi-0.161" --o
r.mapcalc expression="doy=$doy" --o
r.mapcalc expression="tsw=$tsw" --o
r.mapcalc expression="lue=$lue" --o
i.biomass fpar=l8fpar lightuse_efficiency=lue latitude=latitude dayofyear=doy transmissivity_singleway=tsw water_availability=l8evapfr output=l8biom --o

# Export everything we need
for rstF in l8eta l8sm l8biom l8ndvi l8alb l8rnet l8g0 l8rnetd
do
r.null map=$rstF setnull=-100000-0
r.out.gdal -c -m input=$rstF output=$rstF\_$year$mm$dd.tif nodata=-32768 --o
done

</source>

[[Category: Documentation]]
[[Category: Evapotranspiration]]

File:Evaporation manual v0.6.7.pdf

2023-12-18T07:59:52Z

⚠️Ychemin: evapotranspiration modelling step by step manual

== Summary ==
evapotranspiration modelling step by step manual

Hyperion

2023-12-02T12:15:37Z

⚠️Ychemin: /* Signatures preparation and launching i.spec.sam */

== Hyperion Hyperspectral Data ==
[[File:eo1_satellite.jpg]][[File:HyperionR50G35B100.png||100px]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Hyperion Data Download ==
Level 1 GST is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either [https://earthexplorer.usgs.gov/ EarthExplorer] or [https://glovis.usgs.gov/ USGS Global Visualization Viewer (GloVis)].

{| class="wikitable"
|+ Level 1 GST specifications
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

=== Extracting and preparing data ===
The data is downloaded as a zip file (i.e. ''EO1H1260582015073110T1_1GST.zip'') uncompressing as a directory (i.e. ''EO1H1260582015073110T1_1GST/''). Inside exists another directory (i.e. ''EO1H1260582015073110T1/'').

The files inside are following this type of listing:
* EO1H1260582015073110T1_B001_L1GST.TIF
* EO1H1260582015073110T1_B002_L1GST.TIF
* ...
* EO1H1260582015073110T1_B241_L1GST.TIF
* EO1H1260582015073110T1_B242_L1GST.TIF
* EO1H1260582015073110T1_MTL_L1GST.TXT
* EO1H1260582015073110T1_PF2_01.fgdc
* README.txt

Of which ''EO1H1260582015073110T1_MTL_L1GST.TXT'' is a Landsat type metadata header file (''MTL type'') well known to Landsat users.

=== Data perculiarity ===
It seems that bands 1-8 are filled with NODATA, the same for bands 58 to 76, and again for 225 to 242.

For this reason, the pre-processing script below is avoiding the atmospheric correction of those bands.

This seems to be explained by the presence of two sensors inside the instrument, one dedicated to VNIR hyperspectral sensing and the second one dedicated to SWIR hyperspectral sensing. This is further reinforced by observing the overlapping of band wavelengths across bands 56-57 and 77-78.

== Import in GRASS, DN2Rad & Rad2Ref ==

<source lang="python">
#!/usr/bin/env python
"""
Process all HYPERION radiance .zip into reflectance.

Download all radiance zip files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 1st December, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes HYPERION images
# 1 - unzip *.zip files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import glob
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the HYPERION Directories
rsdatapath = "/home/yann/RSDATA/HYPERION"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "HYPERION"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .zip HYPERION radiance tracks downloaded
HYPERIONDirs = ["EO1H1260582015073110T1_1GST"] # , ""]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of HYPERION
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

# Define a KVP retrieval function
def findkvp(text, key):
for item in text.split("\n"):
if key in item:
return (item.strip())

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for HYPERIONDir in HYPERIONDirs:
# Unzip all of your HYPERION images in all your directories
#print("Unzip HYPERION files in ", HYPERIONDir)
#p = os.system("unzip "+HYPERIONDir+".zip")
# Go into the new created directory
path = rsdatapath+'/'+HYPERIONDir+'/'+HYPERIONDir.split('_')[0]
os.chdir(path)
# Select the first file in the directory
first = os.listdir(path)[0]
print(first)
# Create a temporary GRASS Location
p = os.system("grass -c "+first+" -e "+grassdata+location)
time.sleep(5)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer (unit is Km)
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=0.1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map (unit is km)
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import HYPERION in GRASS
count = 1
# To remove nodata from DN maps
rnull = Module('r.null')
# Define TIF list
ext=['TIF','tif']
listoftiffiles = [f for f in os.listdir(path) if any(f.endswith(e) for e in ext)]
# Process import
for file in listoftiffiles:
outfile='dn.'+str(count)
ringdal(input=file, output=outfile, quiet=QUIET)
rnull(map=outfile, setnull=0, quiet=QUIET)
count = count + 1
# Read HYPERION image CRS TODO check that "first" has full path attached
src = gdal.Open(first)
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Read MTL file to import scene characteristics
MTLfile = glob.glob('*MTL*', recursive=False)
print("Metadatafile is ",MTLfile)
with open(MTLfile[0], 'r') as file:
data = file.read()

gain_vnir = findkvp(data,"SCALING_FACTOR_VNIR").split("=")[1]
gain_swir = findkvp(data,"SCALING_FACTOR_SWIR").split("=")[1]
print("Gains VNIR %d & SWIR %d" % (int(gain_vnir), int(gain_swir)))
# Extract time mean values
starttimestamp = findkvp(data,"START_TIME").split("=")[1].strip()
endtimestamp = findkvp(data,"END_TIME").split("=")[1].strip()
starttime = starttimestamp.split(" ")[-1]
endtime = endtimestamp.split(" ")[-1]
decstarttime = float(starttime.split(":")[0])+float(starttime.split(":")[1])/60.0+float(starttime.split(":")[2])/3600.0
decendtime = float(endtime.split(":")[0])+float(endtime.split(":")[1])/60.0+float(endtime.split(":")[2])/3600.0
dectime = 0.5 * (decstarttime + decendtime)
print("timestamp ", str(dectime))
print(HYPERIONDir)
acqdate = findkvp(data, "ACQUISITION_DATE").split("=")[1].strip()
date = [acqdate.split("-")[0], acqdate.split("-")[1], acqdate.split("-")[2]]
print(date)
# HYPERION DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain for VNIR
for j in range(8, 57+1):
exp = str("rad."+str(j)+"=dn."+str(j)+"*10.0/"+str(gain_vnir))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Apply Gain for VNIR
for j in range(77, 224+1):
exp = str("rad."+str(j)+"=dn."+str(j)+"*10.0/"+str(gain_swir))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for HYPERION
# Sensor height (satellite is -1000)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# Create the handle for r.out.gdal
routgdal = Module('r.out.gdal')
# Process VNIR
# satellite band number (HYPERION VNIR [433-481])
satbandno = 433 # Band 8 of HYPERION is first for atmospheric correction
for idx in range(8,57+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 32
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = float(rinfo_out["min"])
maximum = float(rinfo_out["max"])
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

# Process SWIR
# satellite band number (HYPERION SWIR [482-628])
satbandno = 482 # Band 77 of HYPERION is first for atmospheric correction
for idx in range(77,224+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 33
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = float(rinfo_out["min"])
maximum = float(rinfo_out["max"])
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1
</source>

== Signatures preparation and launching i.spec.sam ==

<source lang="shell">
# Define the Hyperion files available for processing
listf="ref.8 ref.9 ref.10 ref.11 ref.12 ref.13 ref.14 ref.15 ref.16 ref.17 ref.18 ref.19 ref.20 ref.21 ref.22 ref.23 ref.24 ref.25 ref.26 ref.27 ref.28 ref.29 ref.30 ref.31 ref.32 ref.33 ref.34 ref.35 ref.36 ref.37 ref.38 ref.39 ref.40 ref.41 ref.42 ref.43 ref.44 ref.45 ref.46 ref.47 ref.48 ref.49 ref.50 ref.51 ref.52 ref.53 ref.54 ref.55 ref.56 ref.57 ref.77 ref.78 ref.79 ref.80 ref.81 ref.82 ref.83 ref.84 ref.85 ref.86 ref.87 ref.88 ref.89 ref.90 ref.91 ref.92 ref.93 ref.94 ref.95 ref.96 ref.97 ref.98 ref.99 ref.100 ref.101 ref.102 ref.103 ref.104 ref.105 ref.106 ref.107 ref.108 ref.109 ref.110 ref.111 ref.112 ref.113 ref.114 ref.115 ref.116 ref.117 ref.118 ref.119 ref.120 ref.121 ref.122 ref.123 ref.124 ref.125 ref.126 ref.127 ref.128 ref.129 ref.130 ref.131 ref.132 ref.133 ref.134 ref.135 ref.136 ref.137 ref.138 ref.139 ref.140 ref.141 ref.142 ref.143 ref.144 ref.145 ref.146 ref.147 ref.148 ref.149 ref.150 ref.151 ref.152 ref.153 ref.154 ref.155 ref.156 ref.157"

# Clean up .sign files
rm -f *.sign
# Name the signature file for i.spec.sam
sigfname="KL.sig"
rm -f $sigfname

echo "# Signatures for Hyperion image of KL" >> $sigfname

# Water in river
classname="water1"
coords="215455.8579625377,383556.12608450593"
fname="$classname.sign"
# Clean up temp file
rm -f temp
for file in $listf
do
r.what map=$file coordinates=$coords separator='comma' >> temp
done
# Define cols number in sig file
lno=$(cat temp | wc -l)
sed -i 's/$.*$,,$.*$/\2/' temp
rm -f temp1
for number in $(cat temp)
do
printf "%.8f " $number >> temp1
done
tr '\n' ' ' < temp1 > $fname
echo "# $classname" >> $sigfname

# Forest area
classname="forest1"
coords="199906.1669829222,331418.4535104364"
fname="$classname.sign"
# Clean up temp file
rm -f temp
for file in $listf
do
r.what map=$file coordinates=$coords separator='comma' >> temp
done
sed -i 's/$.*$,,$.*$/\2/' temp
rm -f temp1
for number in $(cat temp)
do
printf "%.8f " $number >> temp1
done
tr '\n' ' ' < temp1 > $fname
echo "# $classname" >> $sigfname

# Define rows number in sig file
signo=$(ls *.sign | wc -l)

# Complete the sig file
echo "# " >> $sigfname
echo "Matrix: $signo by $lno" >> $sigfname
n=0
for file in *.sign
do
echo "row$n: $(cat $file)" >> $sigfname
n=$(echo "$n + 1" | bc)
done
# i.spec.sam does not like a space at the end of a line
sed -i 's/\s*$//' $sigfname

# Define the imagery working group
listmaps=$(echo $listf | sed 's/\ /,/g')
i.group group=temp input=$listmaps --quiet

# Run a first set of Spectral Angle convergence
i.spec.sam group=temp input=$sigfname result=s --verbose

</source>

Hyperion

2023-12-02T11:50:24Z

⚠️Ychemin: /* Signatures preparation and launching i.spec.sam */

== Hyperion Hyperspectral Data ==
[[File:eo1_satellite.jpg]][[File:HyperionR50G35B100.png||100px]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Hyperion Data Download ==
Level 1 GST is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either [https://earthexplorer.usgs.gov/ EarthExplorer] or [https://glovis.usgs.gov/ USGS Global Visualization Viewer (GloVis)].

{| class="wikitable"
|+ Level 1 GST specifications
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

=== Extracting and preparing data ===
The data is downloaded as a zip file (i.e. ''EO1H1260582015073110T1_1GST.zip'') uncompressing as a directory (i.e. ''EO1H1260582015073110T1_1GST/''). Inside exists another directory (i.e. ''EO1H1260582015073110T1/'').

The files inside are following this type of listing:
* EO1H1260582015073110T1_B001_L1GST.TIF
* EO1H1260582015073110T1_B002_L1GST.TIF
* ...
* EO1H1260582015073110T1_B241_L1GST.TIF
* EO1H1260582015073110T1_B242_L1GST.TIF
* EO1H1260582015073110T1_MTL_L1GST.TXT
* EO1H1260582015073110T1_PF2_01.fgdc
* README.txt

Of which ''EO1H1260582015073110T1_MTL_L1GST.TXT'' is a Landsat type metadata header file (''MTL type'') well known to Landsat users.

=== Data perculiarity ===
It seems that bands 1-8 are filled with NODATA, the same for bands 58 to 76, and again for 225 to 242.

For this reason, the pre-processing script below is avoiding the atmospheric correction of those bands.

This seems to be explained by the presence of two sensors inside the instrument, one dedicated to VNIR hyperspectral sensing and the second one dedicated to SWIR hyperspectral sensing. This is further reinforced by observing the overlapping of band wavelengths across bands 56-57 and 77-78.

== Import in GRASS, DN2Rad & Rad2Ref ==

<source lang="python">
#!/usr/bin/env python
"""
Process all HYPERION radiance .zip into reflectance.

Download all radiance zip files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 1st December, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes HYPERION images
# 1 - unzip *.zip files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import glob
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the HYPERION Directories
rsdatapath = "/home/yann/RSDATA/HYPERION"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "HYPERION"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .zip HYPERION radiance tracks downloaded
HYPERIONDirs = ["EO1H1260582015073110T1_1GST"] # , ""]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of HYPERION
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

# Define a KVP retrieval function
def findkvp(text, key):
for item in text.split("\n"):
if key in item:
return (item.strip())

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for HYPERIONDir in HYPERIONDirs:
# Unzip all of your HYPERION images in all your directories
#print("Unzip HYPERION files in ", HYPERIONDir)
#p = os.system("unzip "+HYPERIONDir+".zip")
# Go into the new created directory
path = rsdatapath+'/'+HYPERIONDir+'/'+HYPERIONDir.split('_')[0]
os.chdir(path)
# Select the first file in the directory
first = os.listdir(path)[0]
print(first)
# Create a temporary GRASS Location
p = os.system("grass -c "+first+" -e "+grassdata+location)
time.sleep(5)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer (unit is Km)
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=0.1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map (unit is km)
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import HYPERION in GRASS
count = 1
# To remove nodata from DN maps
rnull = Module('r.null')
# Define TIF list
ext=['TIF','tif']
listoftiffiles = [f for f in os.listdir(path) if any(f.endswith(e) for e in ext)]
# Process import
for file in listoftiffiles:
outfile='dn.'+str(count)
ringdal(input=file, output=outfile, quiet=QUIET)
rnull(map=outfile, setnull=0, quiet=QUIET)
count = count + 1
# Read HYPERION image CRS TODO check that "first" has full path attached
src = gdal.Open(first)
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Read MTL file to import scene characteristics
MTLfile = glob.glob('*MTL*', recursive=False)
print("Metadatafile is ",MTLfile)
with open(MTLfile[0], 'r') as file:
data = file.read()

gain_vnir = findkvp(data,"SCALING_FACTOR_VNIR").split("=")[1]
gain_swir = findkvp(data,"SCALING_FACTOR_SWIR").split("=")[1]
print("Gains VNIR %d & SWIR %d" % (int(gain_vnir), int(gain_swir)))
# Extract time mean values
starttimestamp = findkvp(data,"START_TIME").split("=")[1].strip()
endtimestamp = findkvp(data,"END_TIME").split("=")[1].strip()
starttime = starttimestamp.split(" ")[-1]
endtime = endtimestamp.split(" ")[-1]
decstarttime = float(starttime.split(":")[0])+float(starttime.split(":")[1])/60.0+float(starttime.split(":")[2])/3600.0
decendtime = float(endtime.split(":")[0])+float(endtime.split(":")[1])/60.0+float(endtime.split(":")[2])/3600.0
dectime = 0.5 * (decstarttime + decendtime)
print("timestamp ", str(dectime))
print(HYPERIONDir)
acqdate = findkvp(data, "ACQUISITION_DATE").split("=")[1].strip()
date = [acqdate.split("-")[0], acqdate.split("-")[1], acqdate.split("-")[2]]
print(date)
# HYPERION DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain for VNIR
for j in range(8, 57+1):
exp = str("rad."+str(j)+"=dn."+str(j)+"*10.0/"+str(gain_vnir))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Apply Gain for VNIR
for j in range(77, 224+1):
exp = str("rad."+str(j)+"=dn."+str(j)+"*10.0/"+str(gain_swir))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for HYPERION
# Sensor height (satellite is -1000)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# Create the handle for r.out.gdal
routgdal = Module('r.out.gdal')
# Process VNIR
# satellite band number (HYPERION VNIR [433-481])
satbandno = 433 # Band 8 of HYPERION is first for atmospheric correction
for idx in range(8,57+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 32
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = float(rinfo_out["min"])
maximum = float(rinfo_out["max"])
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

# Process SWIR
# satellite band number (HYPERION SWIR [482-628])
satbandno = 482 # Band 77 of HYPERION is first for atmospheric correction
for idx in range(77,224+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 33
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = float(rinfo_out["min"])
maximum = float(rinfo_out["max"])
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1
</source>

== Signatures preparation and launching i.spec.sam ==

<source lang="shell">
# Define the Hyperion files available for processing
listf="ref.8 ref.9 ref.10 ref.11 ref.12 ref.13 ref.14 ref.15 ref.16 ref.17 ref.18 ref.19 ref.20 ref.21 ref.22 ref.23 ref.24 ref.25 ref.26 ref.27 ref.28 ref.29 ref.30 ref.31 ref.32 ref.33 ref.34 ref.35 ref.36 ref.37 ref.38 ref.39 ref.40 ref.41 ref.42 ref.43 ref.44 ref.45 ref.46 ref.47 ref.48 ref.49 ref.50 ref.51 ref.52 ref.53 ref.54 ref.55 ref.56 ref.57 ref.77 ref.78 ref.79 ref.80 ref.81 ref.82 ref.83 ref.84 ref.85 ref.86 ref.87 ref.88 ref.89 ref.90 ref.91 ref.92 ref.93 ref.94 ref.95 ref.96 ref.97 ref.98 ref.99 ref.100 ref.101 ref.102 ref.103 ref.104 ref.105 ref.106 ref.107 ref.108 ref.109 ref.110 ref.111 ref.112 ref.113 ref.114 ref.115 ref.116 ref.117 ref.118 ref.119 ref.120 ref.121 ref.122 ref.123 ref.124 ref.125 ref.126 ref.127 ref.128 ref.129 ref.130 ref.131 ref.132 ref.133 ref.134 ref.135 ref.136 ref.137 ref.138 ref.139 ref.140 ref.141 ref.142 ref.143 ref.144 ref.145 ref.146 ref.147 ref.148 ref.149 ref.150 ref.151 ref.152 ref.153 ref.154 ref.155 ref.156 ref.157"

# Clean up .sign files
rm -f *.sign
# Name the signature file for i.spec.sam
sigfname="KL.sig"
rm -f $sigfname

echo "# Signatures for Hyperion image of KL" >> $sigfname

# Water in river
classname="water1"
coords="215455.8579625377,383556.12608450593"
fname="$classname.sign"
# Clean up temp file
rm -f temp
for file in $listf
do
r.what map=$file coordinates=$coords separator='comma' >> temp
done
# Define cols number in sig file
lno=$(cat temp | wc -l)
sed -i 's/$.*$,,$.*$/\2/' temp
rm -f temp1
for number in $(cat temp)
do
printf "%.8f " $number >> temp1
done
tr '\n' ' ' < temp1 > $fname
echo "# $classname" >> $sigfname

# Forest area
classname="forest1"
coords="199906.1669829222,331418.4535104364"
fname="$classname.sign"
# Clean up temp file
rm -f temp
for file in $listf
do
r.what map=$file coordinates=$coords separator='comma' >> temp
done
sed -i 's/$.*$,,$.*$/\2/' temp
rm -f temp1
for number in $(cat temp)
do
printf "%.8f " $number >> temp1
done
tr '\n' ' ' < temp1 > $fname
echo "# $classname" >> $sigfname

# Define rows number in sig file
signo=$(ls *.sign | wc -l)

# Complete the sig file
echo "# " >> $sigfname
echo "Matrix: $signo by $lno" >> $sigfname
n=0
for file in *.sign
do
echo "row$n: $(cat $file)" >> $sigfname
n=$(echo "$n + 1" | bc)
done

# Define the imagery working group
listmaps=$(echo $listf | sed 's/\ /,/g')
i.group group=temp input=$listmaps --quiet

# Run a first set of Spectral Angle convergence
i.spec.sam group=temp input=$sigfname result=s
</source>

Hyperion

2023-12-02T11:07:57Z

⚠️Ychemin: /* Signatures preparation and launching i.spec.sam */

== Hyperion Hyperspectral Data ==
[[File:eo1_satellite.jpg]][[File:HyperionR50G35B100.png||100px]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Hyperion Data Download ==
Level 1 GST is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either [https://earthexplorer.usgs.gov/ EarthExplorer] or [https://glovis.usgs.gov/ USGS Global Visualization Viewer (GloVis)].

{| class="wikitable"
|+ Level 1 GST specifications
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

=== Extracting and preparing data ===
The data is downloaded as a zip file (i.e. ''EO1H1260582015073110T1_1GST.zip'') uncompressing as a directory (i.e. ''EO1H1260582015073110T1_1GST/''). Inside exists another directory (i.e. ''EO1H1260582015073110T1/'').

The files inside are following this type of listing:
* EO1H1260582015073110T1_B001_L1GST.TIF
* EO1H1260582015073110T1_B002_L1GST.TIF
* ...
* EO1H1260582015073110T1_B241_L1GST.TIF
* EO1H1260582015073110T1_B242_L1GST.TIF
* EO1H1260582015073110T1_MTL_L1GST.TXT
* EO1H1260582015073110T1_PF2_01.fgdc
* README.txt

Of which ''EO1H1260582015073110T1_MTL_L1GST.TXT'' is a Landsat type metadata header file (''MTL type'') well known to Landsat users.

=== Data perculiarity ===
It seems that bands 1-8 are filled with NODATA, the same for bands 58 to 76, and again for 225 to 242.

For this reason, the pre-processing script below is avoiding the atmospheric correction of those bands.

This seems to be explained by the presence of two sensors inside the instrument, one dedicated to VNIR hyperspectral sensing and the second one dedicated to SWIR hyperspectral sensing. This is further reinforced by observing the overlapping of band wavelengths across bands 56-57 and 77-78.

== Import in GRASS, DN2Rad & Rad2Ref ==

<source lang="python">
#!/usr/bin/env python
"""
Process all HYPERION radiance .zip into reflectance.

Download all radiance zip files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 1st December, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes HYPERION images
# 1 - unzip *.zip files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import glob
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the HYPERION Directories
rsdatapath = "/home/yann/RSDATA/HYPERION"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "HYPERION"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .zip HYPERION radiance tracks downloaded
HYPERIONDirs = ["EO1H1260582015073110T1_1GST"] # , ""]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of HYPERION
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

# Define a KVP retrieval function
def findkvp(text, key):
for item in text.split("\n"):
if key in item:
return (item.strip())

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for HYPERIONDir in HYPERIONDirs:
# Unzip all of your HYPERION images in all your directories
#print("Unzip HYPERION files in ", HYPERIONDir)
#p = os.system("unzip "+HYPERIONDir+".zip")
# Go into the new created directory
path = rsdatapath+'/'+HYPERIONDir+'/'+HYPERIONDir.split('_')[0]
os.chdir(path)
# Select the first file in the directory
first = os.listdir(path)[0]
print(first)
# Create a temporary GRASS Location
p = os.system("grass -c "+first+" -e "+grassdata+location)
time.sleep(5)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer (unit is Km)
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=0.1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map (unit is km)
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import HYPERION in GRASS
count = 1
# To remove nodata from DN maps
rnull = Module('r.null')
# Define TIF list
ext=['TIF','tif']
listoftiffiles = [f for f in os.listdir(path) if any(f.endswith(e) for e in ext)]
# Process import
for file in listoftiffiles:
outfile='dn.'+str(count)
ringdal(input=file, output=outfile, quiet=QUIET)
rnull(map=outfile, setnull=0, quiet=QUIET)
count = count + 1
# Read HYPERION image CRS TODO check that "first" has full path attached
src = gdal.Open(first)
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Read MTL file to import scene characteristics
MTLfile = glob.glob('*MTL*', recursive=False)
print("Metadatafile is ",MTLfile)
with open(MTLfile[0], 'r') as file:
data = file.read()

gain_vnir = findkvp(data,"SCALING_FACTOR_VNIR").split("=")[1]
gain_swir = findkvp(data,"SCALING_FACTOR_SWIR").split("=")[1]
print("Gains VNIR %d & SWIR %d" % (int(gain_vnir), int(gain_swir)))
# Extract time mean values
starttimestamp = findkvp(data,"START_TIME").split("=")[1].strip()
endtimestamp = findkvp(data,"END_TIME").split("=")[1].strip()
starttime = starttimestamp.split(" ")[-1]
endtime = endtimestamp.split(" ")[-1]
decstarttime = float(starttime.split(":")[0])+float(starttime.split(":")[1])/60.0+float(starttime.split(":")[2])/3600.0
decendtime = float(endtime.split(":")[0])+float(endtime.split(":")[1])/60.0+float(endtime.split(":")[2])/3600.0
dectime = 0.5 * (decstarttime + decendtime)
print("timestamp ", str(dectime))
print(HYPERIONDir)
acqdate = findkvp(data, "ACQUISITION_DATE").split("=")[1].strip()
date = [acqdate.split("-")[0], acqdate.split("-")[1], acqdate.split("-")[2]]
print(date)
# HYPERION DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain for VNIR
for j in range(8, 57+1):
exp = str("rad."+str(j)+"=dn."+str(j)+"*10.0/"+str(gain_vnir))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Apply Gain for VNIR
for j in range(77, 224+1):
exp = str("rad."+str(j)+"=dn."+str(j)+"*10.0/"+str(gain_swir))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for HYPERION
# Sensor height (satellite is -1000)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# Create the handle for r.out.gdal
routgdal = Module('r.out.gdal')
# Process VNIR
# satellite band number (HYPERION VNIR [433-481])
satbandno = 433 # Band 8 of HYPERION is first for atmospheric correction
for idx in range(8,57+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 32
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = float(rinfo_out["min"])
maximum = float(rinfo_out["max"])
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

# Process SWIR
# satellite band number (HYPERION SWIR [482-628])
satbandno = 482 # Band 77 of HYPERION is first for atmospheric correction
for idx in range(77,224+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 33
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = float(rinfo_out["min"])
maximum = float(rinfo_out["max"])
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1
</source>

== Signatures preparation and launching i.spec.sam ==

<source lang="shell">

# Define the Hyperion files available for processing
listf="ref.8 ref.9 ref.10 ref.11 ref.12 ref.13 ref.14 ref.15 ref.16 ref.17 ref.18 ref.19 ref.20 ref.21 ref.22 ref.23 ref.24 ref.25 ref.26 ref.27 ref.28 ref.29 ref.30 ref.31 ref.32 ref.33 ref.34 ref.35 ref.36 ref.37 ref.38 ref.39 ref.40 ref.41 ref.42 ref.43 ref.44 ref.45 ref.46 ref.47 ref.48 ref.49 ref.50 ref.51 ref.52 ref.53 ref.54 ref.55 ref.56 ref.57 ref.77 ref.78 ref.79 ref.80 ref.81 ref.82 ref.83 ref.84 ref.85 ref.86 ref.87 ref.88 ref.89 ref.90 ref.91 ref.92 ref.93 ref.94 ref.95 ref.96 ref.97 ref.98 ref.99 ref.100 ref.101 ref.102 ref.103 ref.104 ref.105 ref.106 ref.107 ref.108 ref.109 ref.110 ref.111 ref.112 ref.113 ref.114 ref.115 ref.116 ref.117 ref.118 ref.119 ref.120 ref.121 ref.122 ref.123 ref.124 ref.125 ref.126 ref.127 ref.128 ref.129 ref.130 ref.131 ref.132 ref.133 ref.134 ref.135 ref.136 ref.137 ref.138 ref.139 ref.140 ref.141 ref.142 ref.143 ref.144 ref.145 ref.146 ref.147 ref.148 ref.149 ref.150 ref.151 ref.152 ref.153 ref.154 ref.155 ref.156 ref.157"

# Clean up .sign files
rm -f *.sign
# Name the signature file for i.spec.sam
sigfname="KL.sig"
rm -f $sigfname

echo "# Signatures for Hyperion image of KL" >> $sigfname

# Water in river
classname="water1"
coords="215455.8579625377,383556.12608450593"
fname="$classname.sign"
# Clean up temp file
rm -f temp
for file in $listf
do
r.what map=$file coordinates=$coords separator='comma' >> temp
done
# Define cols number in sig file
lno=$(cat temp | wc -l)
sed -i 's/$.*$,,$.*$/\2/' temp
tr '\n' ' ' < temp > $fname
echo "# $classname" >> $sigfname

# Forest area
classname="forest1"
coords="199906.1669829222,331418.4535104364"
fname="$classname.sign"
# Clean up temp file
rm -f temp
for file in $listf
do
r.what map=$file coordinates=$coords separator='comma' >> temp
done
sed -i 's/$.*$,,$.*$/\2/' temp
tr '\n' ' ' < temp > $fname
echo "# $classname" >> $sigfname

# Define rows number in sig file
signo=$(ls *.sign | wc -l)

# Complete the sig file
echo "# " >> $sigfname
echo "Matrix $signo by $lno" >> $sigfname
n=0
for file in *.sign
do
echo "row$n: $(cat $file)" >> $sigfname
n=$(echo "$n + 1" | bc)
done

# Define the imagery working group
listmaps=$(echo $listf | sed 's/\ /,/g')
i.group group=temp input=$listmaps

# Run a first set of Spectral Angle convergence
i.spec.sam group=temp input=$sigfname result=s
</source>

Hyperion

2023-12-02T11:01:01Z

⚠️Ychemin: /* Hyperion Hyperspectral Data */

== Hyperion Hyperspectral Data ==
[[File:eo1_satellite.jpg]][[File:HyperionR50G35B100.png||100px]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Hyperion Data Download ==
Level 1 GST is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either [https://earthexplorer.usgs.gov/ EarthExplorer] or [https://glovis.usgs.gov/ USGS Global Visualization Viewer (GloVis)].

{| class="wikitable"
|+ Level 1 GST specifications
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

=== Extracting and preparing data ===
The data is downloaded as a zip file (i.e. ''EO1H1260582015073110T1_1GST.zip'') uncompressing as a directory (i.e. ''EO1H1260582015073110T1_1GST/''). Inside exists another directory (i.e. ''EO1H1260582015073110T1/'').

The files inside are following this type of listing:
* EO1H1260582015073110T1_B001_L1GST.TIF
* EO1H1260582015073110T1_B002_L1GST.TIF
* ...
* EO1H1260582015073110T1_B241_L1GST.TIF
* EO1H1260582015073110T1_B242_L1GST.TIF
* EO1H1260582015073110T1_MTL_L1GST.TXT
* EO1H1260582015073110T1_PF2_01.fgdc
* README.txt

Of which ''EO1H1260582015073110T1_MTL_L1GST.TXT'' is a Landsat type metadata header file (''MTL type'') well known to Landsat users.

=== Data perculiarity ===
It seems that bands 1-8 are filled with NODATA, the same for bands 58 to 76, and again for 225 to 242.

For this reason, the pre-processing script below is avoiding the atmospheric correction of those bands.

This seems to be explained by the presence of two sensors inside the instrument, one dedicated to VNIR hyperspectral sensing and the second one dedicated to SWIR hyperspectral sensing. This is further reinforced by observing the overlapping of band wavelengths across bands 56-57 and 77-78.

== Import in GRASS, DN2Rad & Rad2Ref ==

<source lang="python">
#!/usr/bin/env python
"""
Process all HYPERION radiance .zip into reflectance.

Download all radiance zip files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 1st December, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes HYPERION images
# 1 - unzip *.zip files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import glob
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the HYPERION Directories
rsdatapath = "/home/yann/RSDATA/HYPERION"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "HYPERION"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .zip HYPERION radiance tracks downloaded
HYPERIONDirs = ["EO1H1260582015073110T1_1GST"] # , ""]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of HYPERION
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

# Define a KVP retrieval function
def findkvp(text, key):
for item in text.split("\n"):
if key in item:
return (item.strip())

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for HYPERIONDir in HYPERIONDirs:
# Unzip all of your HYPERION images in all your directories
#print("Unzip HYPERION files in ", HYPERIONDir)
#p = os.system("unzip "+HYPERIONDir+".zip")
# Go into the new created directory
path = rsdatapath+'/'+HYPERIONDir+'/'+HYPERIONDir.split('_')[0]
os.chdir(path)
# Select the first file in the directory
first = os.listdir(path)[0]
print(first)
# Create a temporary GRASS Location
p = os.system("grass -c "+first+" -e "+grassdata+location)
time.sleep(5)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer (unit is Km)
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=0.1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map (unit is km)
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import HYPERION in GRASS
count = 1
# To remove nodata from DN maps
rnull = Module('r.null')
# Define TIF list
ext=['TIF','tif']
listoftiffiles = [f for f in os.listdir(path) if any(f.endswith(e) for e in ext)]
# Process import
for file in listoftiffiles:
outfile='dn.'+str(count)
ringdal(input=file, output=outfile, quiet=QUIET)
rnull(map=outfile, setnull=0, quiet=QUIET)
count = count + 1
# Read HYPERION image CRS TODO check that "first" has full path attached
src = gdal.Open(first)
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Read MTL file to import scene characteristics
MTLfile = glob.glob('*MTL*', recursive=False)
print("Metadatafile is ",MTLfile)
with open(MTLfile[0], 'r') as file:
data = file.read()

gain_vnir = findkvp(data,"SCALING_FACTOR_VNIR").split("=")[1]
gain_swir = findkvp(data,"SCALING_FACTOR_SWIR").split("=")[1]
print("Gains VNIR %d & SWIR %d" % (int(gain_vnir), int(gain_swir)))
# Extract time mean values
starttimestamp = findkvp(data,"START_TIME").split("=")[1].strip()
endtimestamp = findkvp(data,"END_TIME").split("=")[1].strip()
starttime = starttimestamp.split(" ")[-1]
endtime = endtimestamp.split(" ")[-1]
decstarttime = float(starttime.split(":")[0])+float(starttime.split(":")[1])/60.0+float(starttime.split(":")[2])/3600.0
decendtime = float(endtime.split(":")[0])+float(endtime.split(":")[1])/60.0+float(endtime.split(":")[2])/3600.0
dectime = 0.5 * (decstarttime + decendtime)
print("timestamp ", str(dectime))
print(HYPERIONDir)
acqdate = findkvp(data, "ACQUISITION_DATE").split("=")[1].strip()
date = [acqdate.split("-")[0], acqdate.split("-")[1], acqdate.split("-")[2]]
print(date)
# HYPERION DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain for VNIR
for j in range(8, 57+1):
exp = str("rad."+str(j)+"=dn."+str(j)+"*10.0/"+str(gain_vnir))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Apply Gain for VNIR
for j in range(77, 224+1):
exp = str("rad."+str(j)+"=dn."+str(j)+"*10.0/"+str(gain_swir))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for HYPERION
# Sensor height (satellite is -1000)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# Create the handle for r.out.gdal
routgdal = Module('r.out.gdal')
# Process VNIR
# satellite band number (HYPERION VNIR [433-481])
satbandno = 433 # Band 8 of HYPERION is first for atmospheric correction
for idx in range(8,57+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 32
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = float(rinfo_out["min"])
maximum = float(rinfo_out["max"])
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

# Process SWIR
# satellite band number (HYPERION SWIR [482-628])
satbandno = 482 # Band 77 of HYPERION is first for atmospheric correction
for idx in range(77,224+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 33
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = float(rinfo_out["min"])
maximum = float(rinfo_out["max"])
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1
</source>

== Signatures preparation and launching i.spec.sam ==

<source lang="shell">

# Define the Hyperion files available for processing
listf="ref.8 ref.9 ref.10 ref.11 ref.12 ref.13 ref.14 ref.15 ref.16 ref.17 ref.18 ref.19 ref.20 ref.21 ref.22 ref.23 ref.24 ref.25 ref.26 ref.27 ref.28 ref.29 ref.30 ref.31 ref.32 ref.33 ref.34 ref.35 ref.36 ref.37 ref.38 ref.39 ref.40 ref.41 ref.42 ref.43 ref.44 ref.45 ref.46 ref.47 ref.48 ref.49 ref.50 ref.51 ref.52 ref.53 ref.54 ref.55 ref.56 ref.57 ref.77 ref.78 ref.79 ref.80 ref.81 ref.82 ref.83 ref.84 ref.85 ref.86 ref.87 ref.88 ref.89 ref.90 ref.91 ref.92 ref.93 ref.94 ref.95 ref.96 ref.97 ref.98 ref.99 ref.100 ref.101 ref.102 ref.103 ref.104 ref.105 ref.106 ref.107 ref.108 ref.109 ref.110 ref.111 ref.112 ref.113 ref.114 ref.115 ref.116 ref.117 ref.118 ref.119 ref.120 ref.121 ref.122 ref.123 ref.124 ref.125 ref.126 ref.127 ref.128 ref.129 ref.130 ref.131 ref.132 ref.133 ref.134 ref.135 ref.136 ref.137 ref.138 ref.139 ref.140 ref.141 ref.142 ref.143 ref.144 ref.145 ref.146 ref.147 ref.148 ref.149 ref.150 ref.151 ref.152 ref.153 ref.154 ref.155 ref.156 ref.157"

# Clean up .sign files
rm -f *.sign
# Name the signature file for i.spec.sam
sigfname="KL.sig"
rm -f $sigfname

echo "# Signatures for Hyperion image of KL" >> $sigfname

# Water in river
classname="water1"
coords="215455.8579625377,383556.12608450593"
fname="$classname.sign"
# Clean up temp file
rm -f temp
for file in $listf
do
r.what map=$file coordinates=$coords separator='comma' >> temp
done
# Define cols number in sig file
lno=$(cat temp | wc -l)
sed -i 's/$.*$,,$.*$/\2/' temp
tr '\n' ' ' < temp > $fname
echo "# $classname" >> $sigfname

# Forest area
classname="forest1"
coords="199906.1669829222,331418.4535104364"
fname="$classname.sign"
# Clean up temp file
rm -f temp
for file in $listf
do
r.what map=$file coordinates=$coords separator='comma' >> temp
done
sed -i 's/$.*$,,$.*$/\2/' temp
tr '\n' ' ' < temp > $fname
echo "# $classname" >> $sigfname

# Define rows number in sig file
signo=$(ls *.sign | wc -l)

# Complete the sig file
echo "# " >> $sigfname
echo "Matrix $signo by $lno" >> $sigfname
n=0
for file in *.sign
do
echo "row$n: $(cat $file)" >> $sigfname
n=$(echo "$n + 1" | bc)
done

# Define the imagery working group
# If already existing and loaded, then empty it
i.group -l group=temp > temp
val=$(cat temp | wc -l)
if [ $val -ne "" ]
then i.group -r group=temp file=temp
fi
# Load it with the new set from $listf
i.group group=temp input=$(echo $listf | sed '/\ /,/')

# Run a first set of Spectral Angle convergence
i.spec.sam group=temp input=$sigfname result=s

</source>

Hyperion

2023-12-02T11:00:31Z

⚠️Ychemin:

== Hyperion Hyperspectral Data ==
[[File:eo1_satellite.jpg]][[File:HyperionR50G35B100.png]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Hyperion Data Download ==
Level 1 GST is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either [https://earthexplorer.usgs.gov/ EarthExplorer] or [https://glovis.usgs.gov/ USGS Global Visualization Viewer (GloVis)].

{| class="wikitable"
|+ Level 1 GST specifications
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

=== Extracting and preparing data ===
The data is downloaded as a zip file (i.e. ''EO1H1260582015073110T1_1GST.zip'') uncompressing as a directory (i.e. ''EO1H1260582015073110T1_1GST/''). Inside exists another directory (i.e. ''EO1H1260582015073110T1/'').

The files inside are following this type of listing:
* EO1H1260582015073110T1_B001_L1GST.TIF
* EO1H1260582015073110T1_B002_L1GST.TIF
* ...
* EO1H1260582015073110T1_B241_L1GST.TIF
* EO1H1260582015073110T1_B242_L1GST.TIF
* EO1H1260582015073110T1_MTL_L1GST.TXT
* EO1H1260582015073110T1_PF2_01.fgdc
* README.txt

Of which ''EO1H1260582015073110T1_MTL_L1GST.TXT'' is a Landsat type metadata header file (''MTL type'') well known to Landsat users.

=== Data perculiarity ===
It seems that bands 1-8 are filled with NODATA, the same for bands 58 to 76, and again for 225 to 242.

For this reason, the pre-processing script below is avoiding the atmospheric correction of those bands.

This seems to be explained by the presence of two sensors inside the instrument, one dedicated to VNIR hyperspectral sensing and the second one dedicated to SWIR hyperspectral sensing. This is further reinforced by observing the overlapping of band wavelengths across bands 56-57 and 77-78.

== Import in GRASS, DN2Rad & Rad2Ref ==

<source lang="python">
#!/usr/bin/env python
"""
Process all HYPERION radiance .zip into reflectance.

Download all radiance zip files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 1st December, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes HYPERION images
# 1 - unzip *.zip files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import glob
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the HYPERION Directories
rsdatapath = "/home/yann/RSDATA/HYPERION"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "HYPERION"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .zip HYPERION radiance tracks downloaded
HYPERIONDirs = ["EO1H1260582015073110T1_1GST"] # , ""]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of HYPERION
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

# Define a KVP retrieval function
def findkvp(text, key):
for item in text.split("\n"):
if key in item:
return (item.strip())

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for HYPERIONDir in HYPERIONDirs:
# Unzip all of your HYPERION images in all your directories
#print("Unzip HYPERION files in ", HYPERIONDir)
#p = os.system("unzip "+HYPERIONDir+".zip")
# Go into the new created directory
path = rsdatapath+'/'+HYPERIONDir+'/'+HYPERIONDir.split('_')[0]
os.chdir(path)
# Select the first file in the directory
first = os.listdir(path)[0]
print(first)
# Create a temporary GRASS Location
p = os.system("grass -c "+first+" -e "+grassdata+location)
time.sleep(5)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer (unit is Km)
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=0.1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map (unit is km)
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import HYPERION in GRASS
count = 1
# To remove nodata from DN maps
rnull = Module('r.null')
# Define TIF list
ext=['TIF','tif']
listoftiffiles = [f for f in os.listdir(path) if any(f.endswith(e) for e in ext)]
# Process import
for file in listoftiffiles:
outfile='dn.'+str(count)
ringdal(input=file, output=outfile, quiet=QUIET)
rnull(map=outfile, setnull=0, quiet=QUIET)
count = count + 1
# Read HYPERION image CRS TODO check that "first" has full path attached
src = gdal.Open(first)
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Read MTL file to import scene characteristics
MTLfile = glob.glob('*MTL*', recursive=False)
print("Metadatafile is ",MTLfile)
with open(MTLfile[0], 'r') as file:
data = file.read()

gain_vnir = findkvp(data,"SCALING_FACTOR_VNIR").split("=")[1]
gain_swir = findkvp(data,"SCALING_FACTOR_SWIR").split("=")[1]
print("Gains VNIR %d & SWIR %d" % (int(gain_vnir), int(gain_swir)))
# Extract time mean values
starttimestamp = findkvp(data,"START_TIME").split("=")[1].strip()
endtimestamp = findkvp(data,"END_TIME").split("=")[1].strip()
starttime = starttimestamp.split(" ")[-1]
endtime = endtimestamp.split(" ")[-1]
decstarttime = float(starttime.split(":")[0])+float(starttime.split(":")[1])/60.0+float(starttime.split(":")[2])/3600.0
decendtime = float(endtime.split(":")[0])+float(endtime.split(":")[1])/60.0+float(endtime.split(":")[2])/3600.0
dectime = 0.5 * (decstarttime + decendtime)
print("timestamp ", str(dectime))
print(HYPERIONDir)
acqdate = findkvp(data, "ACQUISITION_DATE").split("=")[1].strip()
date = [acqdate.split("-")[0], acqdate.split("-")[1], acqdate.split("-")[2]]
print(date)
# HYPERION DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain for VNIR
for j in range(8, 57+1):
exp = str("rad."+str(j)+"=dn."+str(j)+"*10.0/"+str(gain_vnir))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Apply Gain for VNIR
for j in range(77, 224+1):
exp = str("rad."+str(j)+"=dn."+str(j)+"*10.0/"+str(gain_swir))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for HYPERION
# Sensor height (satellite is -1000)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# Create the handle for r.out.gdal
routgdal = Module('r.out.gdal')
# Process VNIR
# satellite band number (HYPERION VNIR [433-481])
satbandno = 433 # Band 8 of HYPERION is first for atmospheric correction
for idx in range(8,57+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 32
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = float(rinfo_out["min"])
maximum = float(rinfo_out["max"])
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

# Process SWIR
# satellite band number (HYPERION SWIR [482-628])
satbandno = 482 # Band 77 of HYPERION is first for atmospheric correction
for idx in range(77,224+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 33
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = float(rinfo_out["min"])
maximum = float(rinfo_out["max"])
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1
</source>

== Signatures preparation and launching i.spec.sam ==

<source lang="shell">

# Define the Hyperion files available for processing
listf="ref.8 ref.9 ref.10 ref.11 ref.12 ref.13 ref.14 ref.15 ref.16 ref.17 ref.18 ref.19 ref.20 ref.21 ref.22 ref.23 ref.24 ref.25 ref.26 ref.27 ref.28 ref.29 ref.30 ref.31 ref.32 ref.33 ref.34 ref.35 ref.36 ref.37 ref.38 ref.39 ref.40 ref.41 ref.42 ref.43 ref.44 ref.45 ref.46 ref.47 ref.48 ref.49 ref.50 ref.51 ref.52 ref.53 ref.54 ref.55 ref.56 ref.57 ref.77 ref.78 ref.79 ref.80 ref.81 ref.82 ref.83 ref.84 ref.85 ref.86 ref.87 ref.88 ref.89 ref.90 ref.91 ref.92 ref.93 ref.94 ref.95 ref.96 ref.97 ref.98 ref.99 ref.100 ref.101 ref.102 ref.103 ref.104 ref.105 ref.106 ref.107 ref.108 ref.109 ref.110 ref.111 ref.112 ref.113 ref.114 ref.115 ref.116 ref.117 ref.118 ref.119 ref.120 ref.121 ref.122 ref.123 ref.124 ref.125 ref.126 ref.127 ref.128 ref.129 ref.130 ref.131 ref.132 ref.133 ref.134 ref.135 ref.136 ref.137 ref.138 ref.139 ref.140 ref.141 ref.142 ref.143 ref.144 ref.145 ref.146 ref.147 ref.148 ref.149 ref.150 ref.151 ref.152 ref.153 ref.154 ref.155 ref.156 ref.157"

# Clean up .sign files
rm -f *.sign
# Name the signature file for i.spec.sam
sigfname="KL.sig"
rm -f $sigfname

echo "# Signatures for Hyperion image of KL" >> $sigfname

# Water in river
classname="water1"
coords="215455.8579625377,383556.12608450593"
fname="$classname.sign"
# Clean up temp file
rm -f temp
for file in $listf
do
r.what map=$file coordinates=$coords separator='comma' >> temp
done
# Define cols number in sig file
lno=$(cat temp | wc -l)
sed -i 's/$.*$,,$.*$/\2/' temp
tr '\n' ' ' < temp > $fname
echo "# $classname" >> $sigfname

# Forest area
classname="forest1"
coords="199906.1669829222,331418.4535104364"
fname="$classname.sign"
# Clean up temp file
rm -f temp
for file in $listf
do
r.what map=$file coordinates=$coords separator='comma' >> temp
done
sed -i 's/$.*$,,$.*$/\2/' temp
tr '\n' ' ' < temp > $fname
echo "# $classname" >> $sigfname

# Define rows number in sig file
signo=$(ls *.sign | wc -l)

# Complete the sig file
echo "# " >> $sigfname
echo "Matrix $signo by $lno" >> $sigfname
n=0
for file in *.sign
do
echo "row$n: $(cat $file)" >> $sigfname
n=$(echo "$n + 1" | bc)
done

# Define the imagery working group
# If already existing and loaded, then empty it
i.group -l group=temp > temp
val=$(cat temp | wc -l)
if [ $val -ne "" ]
then i.group -r group=temp file=temp
fi
# Load it with the new set from $listf
i.group group=temp input=$(echo $listf | sed '/\ /,/')

# Run a first set of Spectral Angle convergence
i.spec.sam group=temp input=$sigfname result=s

</source>

File:HyperionR50G35B100.png

2023-12-02T10:59:42Z

⚠️Ychemin:

Hyperion

2023-12-02T10:56:52Z

⚠️Ychemin: Adding signatures to i.spec.sam

== Hyperion Hyperspectral Data ==
[[File:eo1_satellite.jpg]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Hyperion Data Download ==
Level 1 GST is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either [https://earthexplorer.usgs.gov/ EarthExplorer] or [https://glovis.usgs.gov/ USGS Global Visualization Viewer (GloVis)].

{| class="wikitable"
|+ Level 1 GST specifications
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

=== Extracting and preparing data ===
The data is downloaded as a zip file (i.e. ''EO1H1260582015073110T1_1GST.zip'') uncompressing as a directory (i.e. ''EO1H1260582015073110T1_1GST/''). Inside exists another directory (i.e. ''EO1H1260582015073110T1/'').

The files inside are following this type of listing:
* EO1H1260582015073110T1_B001_L1GST.TIF
* EO1H1260582015073110T1_B002_L1GST.TIF
* ...
* EO1H1260582015073110T1_B241_L1GST.TIF
* EO1H1260582015073110T1_B242_L1GST.TIF
* EO1H1260582015073110T1_MTL_L1GST.TXT
* EO1H1260582015073110T1_PF2_01.fgdc
* README.txt

Of which ''EO1H1260582015073110T1_MTL_L1GST.TXT'' is a Landsat type metadata header file (''MTL type'') well known to Landsat users.

=== Data perculiarity ===
It seems that bands 1-8 are filled with NODATA, the same for bands 58 to 76, and again for 225 to 242.

For this reason, the pre-processing script below is avoiding the atmospheric correction of those bands.

This seems to be explained by the presence of two sensors inside the instrument, one dedicated to VNIR hyperspectral sensing and the second one dedicated to SWIR hyperspectral sensing. This is further reinforced by observing the overlapping of band wavelengths across bands 56-57 and 77-78.

== Import in GRASS, DN2Rad & Rad2Ref ==

<source lang="python">
#!/usr/bin/env python
"""
Process all HYPERION radiance .zip into reflectance.

Download all radiance zip files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 1st December, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes HYPERION images
# 1 - unzip *.zip files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import glob
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the HYPERION Directories
rsdatapath = "/home/yann/RSDATA/HYPERION"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "HYPERION"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .zip HYPERION radiance tracks downloaded
HYPERIONDirs = ["EO1H1260582015073110T1_1GST"] # , ""]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of HYPERION
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

# Define a KVP retrieval function
def findkvp(text, key):
for item in text.split("\n"):
if key in item:
return (item.strip())

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for HYPERIONDir in HYPERIONDirs:
# Unzip all of your HYPERION images in all your directories
#print("Unzip HYPERION files in ", HYPERIONDir)
#p = os.system("unzip "+HYPERIONDir+".zip")
# Go into the new created directory
path = rsdatapath+'/'+HYPERIONDir+'/'+HYPERIONDir.split('_')[0]
os.chdir(path)
# Select the first file in the directory
first = os.listdir(path)[0]
print(first)
# Create a temporary GRASS Location
p = os.system("grass -c "+first+" -e "+grassdata+location)
time.sleep(5)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer (unit is Km)
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=0.1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map (unit is km)
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import HYPERION in GRASS
count = 1
# To remove nodata from DN maps
rnull = Module('r.null')
# Define TIF list
ext=['TIF','tif']
listoftiffiles = [f for f in os.listdir(path) if any(f.endswith(e) for e in ext)]
# Process import
for file in listoftiffiles:
outfile='dn.'+str(count)
ringdal(input=file, output=outfile, quiet=QUIET)
rnull(map=outfile, setnull=0, quiet=QUIET)
count = count + 1
# Read HYPERION image CRS TODO check that "first" has full path attached
src = gdal.Open(first)
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Read MTL file to import scene characteristics
MTLfile = glob.glob('*MTL*', recursive=False)
print("Metadatafile is ",MTLfile)
with open(MTLfile[0], 'r') as file:
data = file.read()

gain_vnir = findkvp(data,"SCALING_FACTOR_VNIR").split("=")[1]
gain_swir = findkvp(data,"SCALING_FACTOR_SWIR").split("=")[1]
print("Gains VNIR %d & SWIR %d" % (int(gain_vnir), int(gain_swir)))
# Extract time mean values
starttimestamp = findkvp(data,"START_TIME").split("=")[1].strip()
endtimestamp = findkvp(data,"END_TIME").split("=")[1].strip()
starttime = starttimestamp.split(" ")[-1]
endtime = endtimestamp.split(" ")[-1]
decstarttime = float(starttime.split(":")[0])+float(starttime.split(":")[1])/60.0+float(starttime.split(":")[2])/3600.0
decendtime = float(endtime.split(":")[0])+float(endtime.split(":")[1])/60.0+float(endtime.split(":")[2])/3600.0
dectime = 0.5 * (decstarttime + decendtime)
print("timestamp ", str(dectime))
print(HYPERIONDir)
acqdate = findkvp(data, "ACQUISITION_DATE").split("=")[1].strip()
date = [acqdate.split("-")[0], acqdate.split("-")[1], acqdate.split("-")[2]]
print(date)
# HYPERION DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain for VNIR
for j in range(8, 57+1):
exp = str("rad."+str(j)+"=dn."+str(j)+"*10.0/"+str(gain_vnir))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Apply Gain for VNIR
for j in range(77, 224+1):
exp = str("rad."+str(j)+"=dn."+str(j)+"*10.0/"+str(gain_swir))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for HYPERION
# Sensor height (satellite is -1000)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# Create the handle for r.out.gdal
routgdal = Module('r.out.gdal')
# Process VNIR
# satellite band number (HYPERION VNIR [433-481])
satbandno = 433 # Band 8 of HYPERION is first for atmospheric correction
for idx in range(8,57+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 32
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = float(rinfo_out["min"])
maximum = float(rinfo_out["max"])
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

# Process SWIR
# satellite band number (HYPERION SWIR [482-628])
satbandno = 482 # Band 77 of HYPERION is first for atmospheric correction
for idx in range(77,224+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 33
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = float(rinfo_out["min"])
maximum = float(rinfo_out["max"])
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1
</source>

== Signatures preparation and launching i.spec.sam ==

<source lang="shell">

# Define the Hyperion files available for processing
listf="ref.8 ref.9 ref.10 ref.11 ref.12 ref.13 ref.14 ref.15 ref.16 ref.17 ref.18 ref.19 ref.20 ref.21 ref.22 ref.23 ref.24 ref.25 ref.26 ref.27 ref.28 ref.29 ref.30 ref.31 ref.32 ref.33 ref.34 ref.35 ref.36 ref.37 ref.38 ref.39 ref.40 ref.41 ref.42 ref.43 ref.44 ref.45 ref.46 ref.47 ref.48 ref.49 ref.50 ref.51 ref.52 ref.53 ref.54 ref.55 ref.56 ref.57 ref.77 ref.78 ref.79 ref.80 ref.81 ref.82 ref.83 ref.84 ref.85 ref.86 ref.87 ref.88 ref.89 ref.90 ref.91 ref.92 ref.93 ref.94 ref.95 ref.96 ref.97 ref.98 ref.99 ref.100 ref.101 ref.102 ref.103 ref.104 ref.105 ref.106 ref.107 ref.108 ref.109 ref.110 ref.111 ref.112 ref.113 ref.114 ref.115 ref.116 ref.117 ref.118 ref.119 ref.120 ref.121 ref.122 ref.123 ref.124 ref.125 ref.126 ref.127 ref.128 ref.129 ref.130 ref.131 ref.132 ref.133 ref.134 ref.135 ref.136 ref.137 ref.138 ref.139 ref.140 ref.141 ref.142 ref.143 ref.144 ref.145 ref.146 ref.147 ref.148 ref.149 ref.150 ref.151 ref.152 ref.153 ref.154 ref.155 ref.156 ref.157"

# Clean up .sign files
rm -f *.sign
# Name the signature file for i.spec.sam
sigfname="KL.sig"
rm -f $sigfname

echo "# Signatures for Hyperion image of KL" >> $sigfname

# Water in river
classname="water1"
coords="215455.8579625377,383556.12608450593"
fname="$classname.sign"
# Clean up temp file
rm -f temp
for file in $listf
do
r.what map=$file coordinates=$coords separator='comma' >> temp
done
# Define cols number in sig file
lno=$(cat temp | wc -l)
sed -i 's/$.*$,,$.*$/\2/' temp
tr '\n' ' ' < temp > $fname
echo "# $classname" >> $sigfname

# Forest area
classname="forest1"
coords="199906.1669829222,331418.4535104364"
fname="$classname.sign"
# Clean up temp file
rm -f temp
for file in $listf
do
r.what map=$file coordinates=$coords separator='comma' >> temp
done
sed -i 's/$.*$,,$.*$/\2/' temp
tr '\n' ' ' < temp > $fname
echo "# $classname" >> $sigfname

# Define rows number in sig file
signo=$(ls *.sign | wc -l)

# Complete the sig file
echo "# " >> $sigfname
echo "Matrix $signo by $lno" >> $sigfname
n=0
for file in *.sign
do
echo "row$n: $(cat $file)" >> $sigfname
n=$(echo "$n + 1" | bc)
done

# Define the imagery working group
# If already existing and loaded, then empty it
i.group -l group=temp > temp
val=$(cat temp | wc -l)
if [ $val -ne "" ]
then i.group -r group=temp file=temp
fi
# Load it with the new set from $listf
i.group group=temp input=$(echo $listf | sed '/\ /,/')

# Run a first set of Spectral Angle convergence
i.spec.sam group=temp input=$sigfname result=s

</source>

Hyperion

2023-12-02T08:51:49Z

⚠️Ychemin: /* Import in GRASS, DN2Rad & Rad2Ref */

Hyperion

2023-12-02T08:49:27Z

⚠️Ychemin: /* Import in GRASS, DN2Rad & Rad2Ref */

== Hyperion Hyperspectral Data ==
[[File:eo1_satellite.jpg]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Hyperion Data Download ==
Level 1 GST is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either [https://earthexplorer.usgs.gov/ EarthExplorer] or [https://glovis.usgs.gov/ USGS Global Visualization Viewer (GloVis)].

{| class="wikitable"
|+ Level 1 GST specifications
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

=== Extracting and preparing data ===
The data is downloaded as a zip file (i.e. ''EO1H1260582015073110T1_1GST.zip'') uncompressing as a directory (i.e. ''EO1H1260582015073110T1_1GST/''). Inside exists another directory (i.e. ''EO1H1260582015073110T1/'').

The files inside are following this type of listing:
* EO1H1260582015073110T1_B001_L1GST.TIF
* EO1H1260582015073110T1_B002_L1GST.TIF
* ...
* EO1H1260582015073110T1_B241_L1GST.TIF
* EO1H1260582015073110T1_B242_L1GST.TIF
* EO1H1260582015073110T1_MTL_L1GST.TXT
* EO1H1260582015073110T1_PF2_01.fgdc
* README.txt

Of which ''EO1H1260582015073110T1_MTL_L1GST.TXT'' is a Landsat type metadata header file (''MTL type'') well known to Landsat users.

=== Data perculiarity ===
It seems that bands 1-8 are filled with NODATA, the same for bands 58 to 76, and again for 225 to 242.

For this reason, the pre-processing script below is avoiding the atmospheric correction of those bands.

This seems to be explained by the presence of two sensors inside the instrument, one dedicated to VNIR hyperspectral sensing and the second one dedicated to SWIR hyperspectral sensing. This is further reinforced by observing the overlapping of band wavelengths across bands 56-57 and 77-78.

== Import in GRASS, DN2Rad & Rad2Ref ==

<source lang="python">
#!/usr/bin/env python
"""
Process all HYPERION radiance .zip into reflectance.

Download all radiance zip files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 1st December, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes HYPERION images
# 1 - untar *.zip files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import glob
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the HYPERION Directories
rsdatapath = "/home/yann/RSDATA/HYPERION"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "HYPERION"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .zip HYPERION radiance tracks downloaded
HYPERIONDirs = ["EO1H1260582015073110T1_1GST"] # , ""]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of HYPERION
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

# Define a KVP retrieval function
def findkvp(text, key):
for item in text.split("\n"):
if key in item:
return (item.strip())

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for HYPERIONDir in HYPERIONDirs:
# Unzip all of your HYPERION images in all your directories
#print("Unzip HYPERION files in ", HYPERIONDir)
#p = os.system("unzip "+HYPERIONDir+".zip")
# Go into the new created directory
path = rsdatapath+'/'+HYPERIONDir+'/'+HYPERIONDir.split('_')[0]
os.chdir(path)
# Select the first file in the directory
first = os.listdir(path)[0]
print(first)
# Create a temporary GRASS Location
p = os.system("grass -c "+first+" -e "+grassdata+location)
time.sleep(5)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer (unit is Km)
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=0.1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map (unit is km)
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import HYPERION in GRASS
count = 1
# To remove nodata from DN maps
rnull = Module('r.null')
# Define TIF list
ext=['TIF','tif']
listoftiffiles = [f for f in os.listdir(path) if any(f.endswith(e) for e in ext)]
# Process import
for file in listoftiffiles:
outfile='dn.'+str(count)
ringdal(input=file, output=outfile, quiet=QUIET)
rnull(map=outfile, setnull=0, quiet=QUIET)
count = count + 1
# Read HYPERION image CRS TODO check that "first" has full path attached
src = gdal.Open(first)
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Read MTL file to import scene characteristics
MTLfile = glob.glob('*MTL*', recursive=False)
print("Metadatafile is ",MTLfile)
with open(MTLfile[0], 'r') as file:
data = file.read()

gain_vnir = findkvp(data,"SCALING_FACTOR_VNIR").split("=")[1]
gain_swir = findkvp(data,"SCALING_FACTOR_SWIR").split("=")[1]
print("Gains VNIR %d & SWIR %d" % (int(gain_vnir), int(gain_swir)))
# Extract time mean values
starttimestamp = findkvp(data,"START_TIME").split("=")[1].strip()
endtimestamp = findkvp(data,"END_TIME").split("=")[1].strip()
starttime = starttimestamp.split(" ")[-1]
endtime = endtimestamp.split(" ")[-1]
decstarttime = float(starttime.split(":")[0])+float(starttime.split(":")[1])/60.0+float(starttime.split(":")[2])/3600.0
decendtime = float(endtime.split(":")[0])+float(endtime.split(":")[1])/60.0+float(endtime.split(":")[2])/3600.0
dectime = 0.5 * (decstarttime + decendtime)
print("timestamp ", str(dectime))
print(HYPERIONDir)
acqdate = findkvp(data, "ACQUISITION_DATE").split("=")[1].strip()
date = [acqdate.split("-")[0], acqdate.split("-")[1], acqdate.split("-")[2]]
print(date)
# HYPERION DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain for VNIR
for j in range(8, 57+1):
exp = str("rad."+str(j)+"=dn."+str(j)+"*10.0/"+str(gain_vnir))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Apply Gain for VNIR
for j in range(77, 224+1):
exp = str("rad."+str(j)+"=dn."+str(j)+"*10.0/"+str(gain_swir))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for HYPERION
# Sensor height (satellite is -1000)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# Create the handle for r.out.gdal
routgdal = Module('r.out.gdal')
# Process VNIR
# satellite band number (HYPERION VNIR [433-481])
satbandno = 433 # Band 8 of HYPERION is first for atmospheric correction
for idx in range(8,57+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 32
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = float(rinfo_out["min"])
maximum = float(rinfo_out["max"])
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

# Process SWIR
# satellite band number (HYPERION SWIR [482-628])
satbandno = 482 # Band 77 of HYPERION is first for atmospheric correction
for idx in range(77,224+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 33
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1
</source>

Image processing/Evapotranspiration

2023-12-02T08:26:11Z

⚠️Ychemin: /* Introduction */

== Introduction ==
The '''radiation''' is coming from the planet's star, Earth has around 1800 W/m2 of incoming radiation from the sun (also called irradiance) at exo-atmospheric altitude.

Atmospheric transfer of '''irradiance''' is subject to atmosphere particles interactions, scattering, diffraction, reflection, etc.

When reaching the planetary surface, is it partially reflected in the shortwave by '''Albedo'''.

Surface Albedo is the integrated 0.3-3 micro-meters reflectance, it reflects the shortwave energy coming from the sun by 5% on oceans, 15-25% on vegetation, 35-40% on sand/desert/beach, 60-80% on snow/clouds, roughly. This energy fraction is returned to atmosphere for complex interaction with gas particles again before leaving the atmosphere altogether.

[[File:EnergyBalance.png||500px||Alt="Chemin, 2006, PhD Thesis"]]

As for the shortwave surface balance, the '''energy''' is also received in the longwave, but interacts with the grey-body characteristics of the surface elements.

The blackbody to greybody fraction is ruled by emissivity, a hidden component in the thermal spectrum (search for Temperature- Emissivity Separation algorithms). On Earth, emissivity is always above 0.9 (mostly 0.96-0.98). The Stefan-Boltzman equation is dealing with blackbody energy, and multiplying it by Emissivity transforms it to greybody energy emitted.

Together, shortwave and longwave '''energy balance''' provide with the net radiation balance at the surface of the planetary body. This crucial term is the total energy available for thermodynamic fluxes to act on the surface of the planet.

On a planet like Earth where there is a triple phase of a given molecule (H20, in gas, liquid, solid), energy available is used to transfer phases from lower energy to higher energy (sublimation, liquefaction, evaporation). Thermal transfers also happen as conduction (soil, rocks) and convection (atmosphere, oceans).

The i.eb.* modules and models deal with:
* conduction {{cmd|i.eb.soilheatflux}} (for thermal conduction in soils)
* convection i.eb.h_* especially (for thermal convection in atmosphere)
* residual method estimating the energy left for evaporation processes ({{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} are examples).

Additionally, a helper module {{AddonCmd|i.eb.z0m}} is producing an estimation of the surface roughness, necessary in latent heat flux estimation.

== i.evapo.* modules/models are integrated '''models'''==

from the oldest that are not based on energy balance, but are computing some form of evapotranspiration (ETo: Evapotranspiration for a reference of 20cm well-watered grass, see FAO no56 report of Richard Allen)
* Penman-Monteith {{cmd|i.evapo.pm}}
* Priestley-Taylor {{cmd|i.evapo.pt}}
* Hargreaves {{cmd|i.evapo.mh}}

to newer that are in some form or another linked to thermal processes.
* Thermal index {{cmd|i.evapo.senay}}
* Biome thermal processes {{cmd|i.evapo.zk}} (global model based on typical info from biomes Albedo/soil heat flux)
* Thermodynamic processes {{cmd|i.evapo.potrad}} (potential ET if not water stress, based on astronomical equations only)

Most developed thermodynamic models are:
* TSEB (i.evapo.tseb: Two Source Energy Balance, Schmugge, Kustas, etc., 2000) Status: Incomplete Module
* SEBS (i.evapo.sebs: Zhu et al, 2002) Status: Incomplete Module
* SEBAL {{cmd|i.eb.h_sebal01}}, {{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} (Bastiaanssen et al, 1998, 2001)

and some newer models like RESET and others. The list of models using thermodynamic principles is large now, and any willing to submit a code for any of them is most welcome.

== Temporal Integration of Evapotranspiration ==

When integrated in time {{cmd|i.evapo.time}}, seasonal evapotranspiration can be computed and therefore permit a full monitoring of vegetation growth over the period. This is also a basis of crop yield estimation (see {{cmd|i.biomass}}).

[[File:I_evapo_time.png||800px||Alt="Temporal integration of ETa with data from a weather station"]]

== See also ==

* Manual pages:
** http://grass.osgeo.org/grass-stable/manuals/topic_energy_balance.html
** http://grass.osgeo.org/grass-stable/manuals/topic_evapotranspiration.html
** http://grass.osgeo.org/grass-stable/manuals/keywords.html#evaporative%20fraction
* [https://svn.osgeo.org/grass/grass-promo/tutorials/grass_landsat_ETa/ Evapotranspiration mapping with Landsat 5TM in GRASS GIS]

== Example script to process Landsat 8 Evapotranspiration ==
<source lang=shell>
#!/bin/bash

# This scripts imports, pre-processes and compute Evapotranspiration for Landsat 8
# Run in a Location compatible with your L8 dataset

# Atmospheric Transmissivity single-way
tsw=0.75
# Light Use Efficiency (please change dependent on your crop/plant)
lue=1.0

# Import files in GRASS
for file in *TIF
do
r.in.gdal -e input=$file output=$(echo $file | sed 's/.TIF//') --o
done
# Set region to newly imported files
g.region -s raster=$(ls *TIF | grep B1.TIF | sed 's/.TIF//')
LC08=$(ls *TIF | grep B1.TIF | sed 's/1.TIF//')

# Set metadata MTL file name
LC08met=$(ls *MTL.txt)

# Import DEM
r.import input=dem.tif output=dem extent=region --o
# Extract date
date=$(i.landsat.toar input=$LC08 output=l8 metfile=$LC08met lsatmet=date --o)
echo $date

# Apply Top of Atmosphere correction
i.landsat.toar input=$LC08 output=l8 metfile=$LC08met --o

# Set NODATA
for file in $(g.list type=raster pattern=LC08*)
do
r.null map=$file setnull=0
done

#Set computing area to B10 active pixels
for n in 1 2 3 4 5 6 7 8 9
do
r.mapcalc expression="temp=if(isnull(l810),null(),l8$n)" --o
g.rename raster=temp,l8$n --o
done

base=l8
# Create config files for 6s
year=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\1/')
mm=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\2/')
dd=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\3/')
doy=$( date -I -d "$year-$mm-$dd" | date +%j)
hh=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\1/')
mn=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\2/')
ss=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\3/')
time=$(echo "$hh + ($mn / 60.0) + ($ss / 3600.0)" | bc -l)
lat=$(echo $(cat *MTL* | grep CORNER_ | grep LAT | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
lon=$(echo $(cat *MTL* | grep CORNER_ | grep LON | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
echo "18 - geometrical conditions=Landsat 8" > 6s_conf.txt
echo "$mm $dd $hh.$mm $lon $lat - month day hh.ddd longitude latitude ("hh.ddd" is in GMT decimal hours)" >> 6s_conf.txt
echo "1 - atmospheric model=midlatitude summer" >> 6s_conf.txt
echo "1 - aerosols model=continental" >> 6s_conf.txt
echo "24 - visibility [km] (aerosol model concentration)" >> 6s_conf.txt
echo "-0.110 - mean target elevation above sea level [km]" >> 6s_conf.txt
echo "-1000 - sensor on board a satellite" >> 6s_conf.txt
#For Band 1:
cat 6s_conf.txt > $base\1_conf.txt
echo "115 - Coastal band of OLI Landsat 8" >> $base\1_conf.txt
#For Band 2:
cat 6s_conf.txt > $base\2_conf.txt
echo "116 - Blue band of OLI Landsat 8" >> $base\2_conf.txt
#For Band 3:
cat 6s_conf.txt > $base\3_conf.txt
echo "117 - Green band of OLI Landsat 8" >> $base\3_conf.txt
#For Band 4:
cat 6s_conf.txt > $base\4_conf.txt
echo "118 - Red band of OLI Landsat 8" >> $base\4_conf.txt
#For Band 5:
cat 6s_conf.txt > $base\5_conf.txt
echo "120 - NIR band of OLI Landsat 8" >> $base\5_conf.txt
#For Band 6:
cat 6s_conf.txt > $base\6_conf.txt
echo "122 - SWIR1 band of OLI Landsat 8" >> $base\6_conf.txt
#For Band 7:
cat 6s_conf.txt > $base\7_conf.txt
echo "123 - SWIR2 band of OLI Landsat 8" >> $base\7_conf.txt
#For Band 8:
cat 6s_conf.txt > $base\8_conf.txt
echo "119 - PAN band of OLI Landsat 8" >> $base\8_conf.txt
#For Band 9:
cat 6s_conf.txt > $base\9_conf.txt
echo "121 - Cirrus band of OLI Landsat 8" >> $base\9_conf.txt

# Compute Atmospheric correction
for file in $base\1 $base\2 $base\3 $base\4 $base\5 $base\6 $base\7 $base\8 $base\9
do
i.atcorr -r $file elevation=dem parameters=$file\_conf.txt output=s$file --o
r.mapcalc expression="temp=if(isnull(l810),null(),s$file)" --o
g.rename raster=temp,s$file --o
r.colors map=$file color=grey
done

# Create cloud Mask
g.rename raster=l83,l7.2
g.rename raster=l84,l7.3
g.rename raster=l85,l7.4
g.rename raster=l86,l7.5
g.rename raster=l810,l7.6
i.landsat.acca -5 -f input=l7. output=temp --o
g.rename raster=l7.2,l83
g.rename raster=l7.3,l84
g.rename raster=l7.4,l85
g.rename raster=l7.5,l86
g.rename raster=l7.6,l810
r.mapcalc expression="temp1=if(isnull(temp),0,1)" --o
r.mapcalc expression="maskVIS=if(isnull(l810),null(),temp1)" --o

# Apply mask
r.mask raster=maskVIS maskcats=0

# Latitude map
r.latlong input=l81 output=latitude --o

# Longitude map
r.latlong -l input=l81 output=longitude --o

# i.albedo
i.albedo -8 -c input=sl81,sl82,sl83,sl84,sl85,sl86,sl87 output=l8alb --o
r.colors -e map=l8alb color=grey

# i.vi viname=ndvi
i.vi viname=ndvi red=sl84 nir=sl85 output=l8ndvi --o
r.colors map=l8ndvi color=ndvi

# Update processing mask with water
r.mapcalc expression="maskWATER=if(l8ndvi<=0,0,maskVIS)" --o
r.mask -r
r.mask raster=maskWATER maskcats=0

# i.emissivity
i.emissivity input=l8ndvi output=l8emis --o

# i.vi viname=savi
i.vi viname=savi red=sl84 nir=sl85 output=l8savi --o
r.colors map=l8savi color=ndvi

# Daily Net Radiation (RNETD)
r.mapcalc expression="temp=if(isnull(l810),null(),dem)" --o
r.slope.aspect elevation=temp slope=slopedd aspect=aspectdd --o
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=rnetd insol_time=sunhours day=$doy nprocs=8 --o
r.mapcalc expression="l8rnetd=rnetd/sunhours" --o

# Instantaneous Net Radiation (RNET)
timelocal=$(echo "$time + (24.0 / 360.0) * $lon" | bc -l)
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=l8rnet day=$doy time=$timelocal nprocs=8 --o

# Soil Heat Flux (G0)
r.mapcalc expression="overpasstime=$time" --o
i.eb.soilheatflux albedo=l8alb ndvi=l8ndvi temperature=l810 netradiation=l8rnet localutctime=overpasstime output=l8g0 --o

# Sensible Heat Flux (H)
i.eb.z0m -p input=l8savi output=z0m --o
r.mapcalc expression="T0dem=0.00612*dem+l810" --o
i.eb.hsebal01 -a netradiation=l8rnet soilheatflux=l8g0 aerodynresistance=z0m temperaturemeansealevel=T0dem frictionvelocitystar=0.32407 vapourpressureactual=1.511 output=l8h --o

# Evaporative Fraction & Soil Moisture
i.eb.evapfr -m netradiation=l8rnet soilheatflux=l8g0 sensibleheatflux=l8h evaporativefraction=l8evapfr soilmoisture=l8sm --o

# Actual Evapotranspiration (ETa)
i.eb.eta netradiationdiurnal=l8rnetd evaporativefraction=l8evapfr temperature=l810 output=l8eta --o

# Biomass growth
r.mapcalc expression="l8fpar=1.257*l8ndvi-0.161" --o
r.mapcalc expression="doy=$doy" --o
r.mapcalc expression="tsw=$tsw" --o
r.mapcalc expression="lue=$lue" --o
i.biomass fpar=l8fpar lightuse_efficiency=lue latitude=latitude dayofyear=doy transmissivity_singleway=tsw water_availability=l8evapfr output=l8biom --o

# Export everything we need
for rstF in l8eta l8sm l8biom l8ndvi l8alb l8rnet l8g0 l8rnetd
do
r.null map=$rstF setnull=-100000-0
r.out.gdal -c -m input=$rstF output=$rstF\_$year$mm$dd.tif nodata=-32768 --o
done

</source>

[[Category: Documentation]]
[[Category: Evapotranspiration]]

Image processing/Evapotranspiration

2023-12-02T08:25:00Z

⚠️Ychemin: /* See also */

== Introduction ==
The '''radiation''' is coming from the planet's star, Earth has around 1800 W/m2 of incoming radiation from the sun (also called irradiance) at exo-atmospheric altitude.

Atmospheric transfer of '''irradiance''' is subject to atmosphere particles interactions, scattering, diffraction, reflection, etc.

When reaching the planetary surface, is it partially reflected in the shortwave by '''Albedo'''. Surface Albedo is the integrated 0.3-3 micro-meters reflectance, it reflects the shortwave energy coming from the sun by 5% on oceans, 15-25% on vegetation, 35-40% on sand/desert/beach, 60-80% on snow/clouds, roughly. This energy fraction is returned to atmosphere for complex interaction with gas particles again before leaving the atmosphere altogether.

[[File:EnergyBalance.png||500px||Alt="Chemin, 2006, PhD Thesis"]]

As for the shortwave surface balance, the '''energy''' is also received in the longwave, but interacts with the grey-body characteristics of the surface elements. The blackbody to greybody fraction is ruled by emissivity, a hidden component in the thermal spectrum (search for Temperature- Emissivity Separation algorithms). On Earth, emissivity is always above 0.9 (mostly 0.96-0.98). The Stefan-Boltzman equation is dealing with blackbody energy, and multiplying it by Emissivity transforms it to greybody energy emitted.

Together, shortwave and longwave '''energy balance''' provide with the net radiation balance at the surface of the planetary body. This crucial term is the total energy available for thermodynamic fluxes to act on the surface of the planet. On a planet like Earth where there is a triple phase of a given molecule (H20, in gas, liquid, solid), energy available is used to transfer phases from lower energy to higher energy (sublimation, liquefaction, evaporation). Thermal transfers also happen as conduction (soil, rocks) and convection (atmosphere, oceans).

The i.eb.* modules and models deal with:
* conduction {{cmd|i.eb.soilheatflux}} (for thermal conduction in soils)
* convection i.eb.h_* especially (for thermal convection in atmosphere)
* residual method estimating the energy left for evaporation processes ({{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} are examples).

Additionally, a helper module {{AddonCmd|i.eb.z0m}} is producing an estimation of the surface roughness, necessary in latent heat flux estimation.

== i.evapo.* modules/models are integrated '''models'''==

from the oldest that are not based on energy balance, but are computing some form of evapotranspiration (ETo: Evapotranspiration for a reference of 20cm well-watered grass, see FAO no56 report of Richard Allen)
* Penman-Monteith {{cmd|i.evapo.pm}}
* Priestley-Taylor {{cmd|i.evapo.pt}}
* Hargreaves {{cmd|i.evapo.mh}}

to newer that are in some form or another linked to thermal processes.
* Thermal index {{cmd|i.evapo.senay}}
* Biome thermal processes {{cmd|i.evapo.zk}} (global model based on typical info from biomes Albedo/soil heat flux)
* Thermodynamic processes {{cmd|i.evapo.potrad}} (potential ET if not water stress, based on astronomical equations only)

Most developed thermodynamic models are:
* TSEB (i.evapo.tseb: Two Source Energy Balance, Schmugge, Kustas, etc., 2000) Status: Incomplete Module
* SEBS (i.evapo.sebs: Zhu et al, 2002) Status: Incomplete Module
* SEBAL {{cmd|i.eb.h_sebal01}}, {{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} (Bastiaanssen et al, 1998, 2001)

and some newer models like RESET and others. The list of models using thermodynamic principles is large now, and any willing to submit a code for any of them is most welcome.

== Temporal Integration of Evapotranspiration ==

When integrated in time {{cmd|i.evapo.time}}, seasonal evapotranspiration can be computed and therefore permit a full monitoring of vegetation growth over the period. This is also a basis of crop yield estimation (see {{cmd|i.biomass}}).

[[File:I_evapo_time.png||800px||Alt="Temporal integration of ETa with data from a weather station"]]

== See also ==

* Manual pages:
** http://grass.osgeo.org/grass-stable/manuals/topic_energy_balance.html
** http://grass.osgeo.org/grass-stable/manuals/topic_evapotranspiration.html
** http://grass.osgeo.org/grass-stable/manuals/keywords.html#evaporative%20fraction
* [https://svn.osgeo.org/grass/grass-promo/tutorials/grass_landsat_ETa/ Evapotranspiration mapping with Landsat 5TM in GRASS GIS]

== Example script to process Landsat 8 Evapotranspiration ==
<source lang=shell>
#!/bin/bash

# This scripts imports, pre-processes and compute Evapotranspiration for Landsat 8
# Run in a Location compatible with your L8 dataset

# Atmospheric Transmissivity single-way
tsw=0.75
# Light Use Efficiency (please change dependent on your crop/plant)
lue=1.0

# Import files in GRASS
for file in *TIF
do
r.in.gdal -e input=$file output=$(echo $file | sed 's/.TIF//') --o
done
# Set region to newly imported files
g.region -s raster=$(ls *TIF | grep B1.TIF | sed 's/.TIF//')
LC08=$(ls *TIF | grep B1.TIF | sed 's/1.TIF//')

# Set metadata MTL file name
LC08met=$(ls *MTL.txt)

# Import DEM
r.import input=dem.tif output=dem extent=region --o
# Extract date
date=$(i.landsat.toar input=$LC08 output=l8 metfile=$LC08met lsatmet=date --o)
echo $date

# Apply Top of Atmosphere correction
i.landsat.toar input=$LC08 output=l8 metfile=$LC08met --o

# Set NODATA
for file in $(g.list type=raster pattern=LC08*)
do
r.null map=$file setnull=0
done

#Set computing area to B10 active pixels
for n in 1 2 3 4 5 6 7 8 9
do
r.mapcalc expression="temp=if(isnull(l810),null(),l8$n)" --o
g.rename raster=temp,l8$n --o
done

base=l8
# Create config files for 6s
year=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\1/')
mm=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\2/')
dd=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\3/')
doy=$( date -I -d "$year-$mm-$dd" | date +%j)
hh=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\1/')
mn=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\2/')
ss=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\3/')
time=$(echo "$hh + ($mn / 60.0) + ($ss / 3600.0)" | bc -l)
lat=$(echo $(cat *MTL* | grep CORNER_ | grep LAT | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
lon=$(echo $(cat *MTL* | grep CORNER_ | grep LON | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
echo "18 - geometrical conditions=Landsat 8" > 6s_conf.txt
echo "$mm $dd $hh.$mm $lon $lat - month day hh.ddd longitude latitude ("hh.ddd" is in GMT decimal hours)" >> 6s_conf.txt
echo "1 - atmospheric model=midlatitude summer" >> 6s_conf.txt
echo "1 - aerosols model=continental" >> 6s_conf.txt
echo "24 - visibility [km] (aerosol model concentration)" >> 6s_conf.txt
echo "-0.110 - mean target elevation above sea level [km]" >> 6s_conf.txt
echo "-1000 - sensor on board a satellite" >> 6s_conf.txt
#For Band 1:
cat 6s_conf.txt > $base\1_conf.txt
echo "115 - Coastal band of OLI Landsat 8" >> $base\1_conf.txt
#For Band 2:
cat 6s_conf.txt > $base\2_conf.txt
echo "116 - Blue band of OLI Landsat 8" >> $base\2_conf.txt
#For Band 3:
cat 6s_conf.txt > $base\3_conf.txt
echo "117 - Green band of OLI Landsat 8" >> $base\3_conf.txt
#For Band 4:
cat 6s_conf.txt > $base\4_conf.txt
echo "118 - Red band of OLI Landsat 8" >> $base\4_conf.txt
#For Band 5:
cat 6s_conf.txt > $base\5_conf.txt
echo "120 - NIR band of OLI Landsat 8" >> $base\5_conf.txt
#For Band 6:
cat 6s_conf.txt > $base\6_conf.txt
echo "122 - SWIR1 band of OLI Landsat 8" >> $base\6_conf.txt
#For Band 7:
cat 6s_conf.txt > $base\7_conf.txt
echo "123 - SWIR2 band of OLI Landsat 8" >> $base\7_conf.txt
#For Band 8:
cat 6s_conf.txt > $base\8_conf.txt
echo "119 - PAN band of OLI Landsat 8" >> $base\8_conf.txt
#For Band 9:
cat 6s_conf.txt > $base\9_conf.txt
echo "121 - Cirrus band of OLI Landsat 8" >> $base\9_conf.txt

# Compute Atmospheric correction
for file in $base\1 $base\2 $base\3 $base\4 $base\5 $base\6 $base\7 $base\8 $base\9
do
i.atcorr -r $file elevation=dem parameters=$file\_conf.txt output=s$file --o
r.mapcalc expression="temp=if(isnull(l810),null(),s$file)" --o
g.rename raster=temp,s$file --o
r.colors map=$file color=grey
done

# Create cloud Mask
g.rename raster=l83,l7.2
g.rename raster=l84,l7.3
g.rename raster=l85,l7.4
g.rename raster=l86,l7.5
g.rename raster=l810,l7.6
i.landsat.acca -5 -f input=l7. output=temp --o
g.rename raster=l7.2,l83
g.rename raster=l7.3,l84
g.rename raster=l7.4,l85
g.rename raster=l7.5,l86
g.rename raster=l7.6,l810
r.mapcalc expression="temp1=if(isnull(temp),0,1)" --o
r.mapcalc expression="maskVIS=if(isnull(l810),null(),temp1)" --o

# Apply mask
r.mask raster=maskVIS maskcats=0

# Latitude map
r.latlong input=l81 output=latitude --o

# Longitude map
r.latlong -l input=l81 output=longitude --o

# i.albedo
i.albedo -8 -c input=sl81,sl82,sl83,sl84,sl85,sl86,sl87 output=l8alb --o
r.colors -e map=l8alb color=grey

# i.vi viname=ndvi
i.vi viname=ndvi red=sl84 nir=sl85 output=l8ndvi --o
r.colors map=l8ndvi color=ndvi

# Update processing mask with water
r.mapcalc expression="maskWATER=if(l8ndvi<=0,0,maskVIS)" --o
r.mask -r
r.mask raster=maskWATER maskcats=0

# i.emissivity
i.emissivity input=l8ndvi output=l8emis --o

# i.vi viname=savi
i.vi viname=savi red=sl84 nir=sl85 output=l8savi --o
r.colors map=l8savi color=ndvi

# Daily Net Radiation (RNETD)
r.mapcalc expression="temp=if(isnull(l810),null(),dem)" --o
r.slope.aspect elevation=temp slope=slopedd aspect=aspectdd --o
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=rnetd insol_time=sunhours day=$doy nprocs=8 --o
r.mapcalc expression="l8rnetd=rnetd/sunhours" --o

# Instantaneous Net Radiation (RNET)
timelocal=$(echo "$time + (24.0 / 360.0) * $lon" | bc -l)
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=l8rnet day=$doy time=$timelocal nprocs=8 --o

# Soil Heat Flux (G0)
r.mapcalc expression="overpasstime=$time" --o
i.eb.soilheatflux albedo=l8alb ndvi=l8ndvi temperature=l810 netradiation=l8rnet localutctime=overpasstime output=l8g0 --o

# Sensible Heat Flux (H)
i.eb.z0m -p input=l8savi output=z0m --o
r.mapcalc expression="T0dem=0.00612*dem+l810" --o
i.eb.hsebal01 -a netradiation=l8rnet soilheatflux=l8g0 aerodynresistance=z0m temperaturemeansealevel=T0dem frictionvelocitystar=0.32407 vapourpressureactual=1.511 output=l8h --o

# Evaporative Fraction & Soil Moisture
i.eb.evapfr -m netradiation=l8rnet soilheatflux=l8g0 sensibleheatflux=l8h evaporativefraction=l8evapfr soilmoisture=l8sm --o

# Actual Evapotranspiration (ETa)
i.eb.eta netradiationdiurnal=l8rnetd evaporativefraction=l8evapfr temperature=l810 output=l8eta --o

# Biomass growth
r.mapcalc expression="l8fpar=1.257*l8ndvi-0.161" --o
r.mapcalc expression="doy=$doy" --o
r.mapcalc expression="tsw=$tsw" --o
r.mapcalc expression="lue=$lue" --o
i.biomass fpar=l8fpar lightuse_efficiency=lue latitude=latitude dayofyear=doy transmissivity_singleway=tsw water_availability=l8evapfr output=l8biom --o

# Export everything we need
for rstF in l8eta l8sm l8biom l8ndvi l8alb l8rnet l8g0 l8rnetd
do
r.null map=$rstF setnull=-100000-0
r.out.gdal -c -m input=$rstF output=$rstF\_$year$mm$dd.tif nodata=-32768 --o
done

</source>

[[Category: Documentation]]
[[Category: Evapotranspiration]]

Image processing/Evapotranspiration

2023-12-02T08:21:40Z

⚠️Ychemin: /* Introduction */

== Introduction ==
The '''radiation''' is coming from the planet's star, Earth has around 1800 W/m2 of incoming radiation from the sun (also called irradiance) at exo-atmospheric altitude.

Atmospheric transfer of '''irradiance''' is subject to atmosphere particles interactions, scattering, diffraction, reflection, etc.

When reaching the planetary surface, is it partially reflected in the shortwave by '''Albedo'''. Surface Albedo is the integrated 0.3-3 micro-meters reflectance, it reflects the shortwave energy coming from the sun by 5% on oceans, 15-25% on vegetation, 35-40% on sand/desert/beach, 60-80% on snow/clouds, roughly. This energy fraction is returned to atmosphere for complex interaction with gas particles again before leaving the atmosphere altogether.

[[File:EnergyBalance.png||500px||Alt="Chemin, 2006, PhD Thesis"]]

As for the shortwave surface balance, the '''energy''' is also received in the longwave, but interacts with the grey-body characteristics of the surface elements. The blackbody to greybody fraction is ruled by emissivity, a hidden component in the thermal spectrum (search for Temperature- Emissivity Separation algorithms). On Earth, emissivity is always above 0.9 (mostly 0.96-0.98). The Stefan-Boltzman equation is dealing with blackbody energy, and multiplying it by Emissivity transforms it to greybody energy emitted.

Together, shortwave and longwave '''energy balance''' provide with the net radiation balance at the surface of the planetary body. This crucial term is the total energy available for thermodynamic fluxes to act on the surface of the planet. On a planet like Earth where there is a triple phase of a given molecule (H20, in gas, liquid, solid), energy available is used to transfer phases from lower energy to higher energy (sublimation, liquefaction, evaporation). Thermal transfers also happen as conduction (soil, rocks) and convection (atmosphere, oceans).

The i.eb.* modules and models deal with:
* conduction {{cmd|i.eb.soilheatflux}} (for thermal conduction in soils)
* convection i.eb.h_* especially (for thermal convection in atmosphere)
* residual method estimating the energy left for evaporation processes ({{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} are examples).

Additionally, a helper module {{AddonCmd|i.eb.z0m}} is producing an estimation of the surface roughness, necessary in latent heat flux estimation.

== i.evapo.* modules/models are integrated '''models'''==

from the oldest that are not based on energy balance, but are computing some form of evapotranspiration (ETo: Evapotranspiration for a reference of 20cm well-watered grass, see FAO no56 report of Richard Allen)
* Penman-Monteith {{cmd|i.evapo.pm}}
* Priestley-Taylor {{cmd|i.evapo.pt}}
* Hargreaves {{cmd|i.evapo.mh}}

to newer that are in some form or another linked to thermal processes.
* Thermal index {{cmd|i.evapo.senay}}
* Biome thermal processes {{cmd|i.evapo.zk}} (global model based on typical info from biomes Albedo/soil heat flux)
* Thermodynamic processes {{cmd|i.evapo.potrad}} (potential ET if not water stress, based on astronomical equations only)

Most developed thermodynamic models are:
* TSEB (i.evapo.tseb: Two Source Energy Balance, Schmugge, Kustas, etc., 2000) Status: Incomplete Module
* SEBS (i.evapo.sebs: Zhu et al, 2002) Status: Incomplete Module
* SEBAL {{cmd|i.eb.h_sebal01}}, {{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} (Bastiaanssen et al, 1998, 2001)

and some newer models like RESET and others. The list of models using thermodynamic principles is large now, and any willing to submit a code for any of them is most welcome.

== Temporal Integration of Evapotranspiration ==

When integrated in time {{cmd|i.evapo.time}}, seasonal evapotranspiration can be computed and therefore permit a full monitoring of vegetation growth over the period. This is also a basis of crop yield estimation (see {{cmd|i.biomass}}).

[[File:I_evapo_time.png||800px||Alt="Temporal integration of ETa with data from a weather station"]]

== See also ==

* Manual pages:
** http://grass.osgeo.org/grass70/manuals/topic_energy_balance.html
** http://grass.osgeo.org/grass70/manuals/topic_evapotranspiration.html
** http://grass.osgeo.org/grass70/manuals/keywords.html#evaporative%20fraction
* [https://svn.osgeo.org/grass/grass-promo/tutorials/grass_landsat_ETa/ Evapotranspiration mapping with Landsat 5TM in GRASS GIS]

== Example script to process Landsat 8 Evapotranspiration ==
<source lang=shell>
#!/bin/bash

# This scripts imports, pre-processes and compute Evapotranspiration for Landsat 8
# Run in a Location compatible with your L8 dataset

# Atmospheric Transmissivity single-way
tsw=0.75
# Light Use Efficiency (please change dependent on your crop/plant)
lue=1.0

# Import files in GRASS
for file in *TIF
do
r.in.gdal -e input=$file output=$(echo $file | sed 's/.TIF//') --o
done
# Set region to newly imported files
g.region -s raster=$(ls *TIF | grep B1.TIF | sed 's/.TIF//')
LC08=$(ls *TIF | grep B1.TIF | sed 's/1.TIF//')

# Set metadata MTL file name
LC08met=$(ls *MTL.txt)

# Import DEM
r.import input=dem.tif output=dem extent=region --o
# Extract date
date=$(i.landsat.toar input=$LC08 output=l8 metfile=$LC08met lsatmet=date --o)
echo $date

# Apply Top of Atmosphere correction
i.landsat.toar input=$LC08 output=l8 metfile=$LC08met --o

# Set NODATA
for file in $(g.list type=raster pattern=LC08*)
do
r.null map=$file setnull=0
done

#Set computing area to B10 active pixels
for n in 1 2 3 4 5 6 7 8 9
do
r.mapcalc expression="temp=if(isnull(l810),null(),l8$n)" --o
g.rename raster=temp,l8$n --o
done

base=l8
# Create config files for 6s
year=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\1/')
mm=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\2/')
dd=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\3/')
doy=$( date -I -d "$year-$mm-$dd" | date +%j)
hh=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\1/')
mn=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\2/')
ss=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\3/')
time=$(echo "$hh + ($mn / 60.0) + ($ss / 3600.0)" | bc -l)
lat=$(echo $(cat *MTL* | grep CORNER_ | grep LAT | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
lon=$(echo $(cat *MTL* | grep CORNER_ | grep LON | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
echo "18 - geometrical conditions=Landsat 8" > 6s_conf.txt
echo "$mm $dd $hh.$mm $lon $lat - month day hh.ddd longitude latitude ("hh.ddd" is in GMT decimal hours)" >> 6s_conf.txt
echo "1 - atmospheric model=midlatitude summer" >> 6s_conf.txt
echo "1 - aerosols model=continental" >> 6s_conf.txt
echo "24 - visibility [km] (aerosol model concentration)" >> 6s_conf.txt
echo "-0.110 - mean target elevation above sea level [km]" >> 6s_conf.txt
echo "-1000 - sensor on board a satellite" >> 6s_conf.txt
#For Band 1:
cat 6s_conf.txt > $base\1_conf.txt
echo "115 - Coastal band of OLI Landsat 8" >> $base\1_conf.txt
#For Band 2:
cat 6s_conf.txt > $base\2_conf.txt
echo "116 - Blue band of OLI Landsat 8" >> $base\2_conf.txt
#For Band 3:
cat 6s_conf.txt > $base\3_conf.txt
echo "117 - Green band of OLI Landsat 8" >> $base\3_conf.txt
#For Band 4:
cat 6s_conf.txt > $base\4_conf.txt
echo "118 - Red band of OLI Landsat 8" >> $base\4_conf.txt
#For Band 5:
cat 6s_conf.txt > $base\5_conf.txt
echo "120 - NIR band of OLI Landsat 8" >> $base\5_conf.txt
#For Band 6:
cat 6s_conf.txt > $base\6_conf.txt
echo "122 - SWIR1 band of OLI Landsat 8" >> $base\6_conf.txt
#For Band 7:
cat 6s_conf.txt > $base\7_conf.txt
echo "123 - SWIR2 band of OLI Landsat 8" >> $base\7_conf.txt
#For Band 8:
cat 6s_conf.txt > $base\8_conf.txt
echo "119 - PAN band of OLI Landsat 8" >> $base\8_conf.txt
#For Band 9:
cat 6s_conf.txt > $base\9_conf.txt
echo "121 - Cirrus band of OLI Landsat 8" >> $base\9_conf.txt

# Compute Atmospheric correction
for file in $base\1 $base\2 $base\3 $base\4 $base\5 $base\6 $base\7 $base\8 $base\9
do
i.atcorr -r $file elevation=dem parameters=$file\_conf.txt output=s$file --o
r.mapcalc expression="temp=if(isnull(l810),null(),s$file)" --o
g.rename raster=temp,s$file --o
r.colors map=$file color=grey
done

# Create cloud Mask
g.rename raster=l83,l7.2
g.rename raster=l84,l7.3
g.rename raster=l85,l7.4
g.rename raster=l86,l7.5
g.rename raster=l810,l7.6
i.landsat.acca -5 -f input=l7. output=temp --o
g.rename raster=l7.2,l83
g.rename raster=l7.3,l84
g.rename raster=l7.4,l85
g.rename raster=l7.5,l86
g.rename raster=l7.6,l810
r.mapcalc expression="temp1=if(isnull(temp),0,1)" --o
r.mapcalc expression="maskVIS=if(isnull(l810),null(),temp1)" --o

# Apply mask
r.mask raster=maskVIS maskcats=0

# Latitude map
r.latlong input=l81 output=latitude --o

# Longitude map
r.latlong -l input=l81 output=longitude --o

# i.albedo
i.albedo -8 -c input=sl81,sl82,sl83,sl84,sl85,sl86,sl87 output=l8alb --o
r.colors -e map=l8alb color=grey

# i.vi viname=ndvi
i.vi viname=ndvi red=sl84 nir=sl85 output=l8ndvi --o
r.colors map=l8ndvi color=ndvi

# Update processing mask with water
r.mapcalc expression="maskWATER=if(l8ndvi<=0,0,maskVIS)" --o
r.mask -r
r.mask raster=maskWATER maskcats=0

# i.emissivity
i.emissivity input=l8ndvi output=l8emis --o

# i.vi viname=savi
i.vi viname=savi red=sl84 nir=sl85 output=l8savi --o
r.colors map=l8savi color=ndvi

# Daily Net Radiation (RNETD)
r.mapcalc expression="temp=if(isnull(l810),null(),dem)" --o
r.slope.aspect elevation=temp slope=slopedd aspect=aspectdd --o
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=rnetd insol_time=sunhours day=$doy nprocs=8 --o
r.mapcalc expression="l8rnetd=rnetd/sunhours" --o

# Instantaneous Net Radiation (RNET)
timelocal=$(echo "$time + (24.0 / 360.0) * $lon" | bc -l)
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=l8rnet day=$doy time=$timelocal nprocs=8 --o

# Soil Heat Flux (G0)
r.mapcalc expression="overpasstime=$time" --o
i.eb.soilheatflux albedo=l8alb ndvi=l8ndvi temperature=l810 netradiation=l8rnet localutctime=overpasstime output=l8g0 --o

# Sensible Heat Flux (H)
i.eb.z0m -p input=l8savi output=z0m --o
r.mapcalc expression="T0dem=0.00612*dem+l810" --o
i.eb.hsebal01 -a netradiation=l8rnet soilheatflux=l8g0 aerodynresistance=z0m temperaturemeansealevel=T0dem frictionvelocitystar=0.32407 vapourpressureactual=1.511 output=l8h --o

# Evaporative Fraction & Soil Moisture
i.eb.evapfr -m netradiation=l8rnet soilheatflux=l8g0 sensibleheatflux=l8h evaporativefraction=l8evapfr soilmoisture=l8sm --o

# Actual Evapotranspiration (ETa)
i.eb.eta netradiationdiurnal=l8rnetd evaporativefraction=l8evapfr temperature=l810 output=l8eta --o

# Biomass growth
r.mapcalc expression="l8fpar=1.257*l8ndvi-0.161" --o
r.mapcalc expression="doy=$doy" --o
r.mapcalc expression="tsw=$tsw" --o
r.mapcalc expression="lue=$lue" --o
i.biomass fpar=l8fpar lightuse_efficiency=lue latitude=latitude dayofyear=doy transmissivity_singleway=tsw water_availability=l8evapfr output=l8biom --o

# Export everything we need
for rstF in l8eta l8sm l8biom l8ndvi l8alb l8rnet l8g0 l8rnetd
do
r.null map=$rstF setnull=-100000-0
r.out.gdal -c -m input=$rstF output=$rstF\_$year$mm$dd.tif nodata=-32768 --o
done

</source>

[[Category: Documentation]]
[[Category: Evapotranspiration]]

Image processing/Evapotranspiration

2023-12-02T08:17:32Z

⚠️Ychemin:

== Introduction ==
The '''radiation''' is coming from the planet's star, Earth has around 1800 W/m2 of incoming radiation from the sun (also called irradiance) at exo-atmospheric altitude.

Atmospheric transfer of '''irradiance''' is subject to atmosphere particles interactions, scattering, diffraction, reflection, etc.

When reaching the planetary surface, is it partially reflected in the shortwave by '''Albedo'''. Surface Albedo is the integrated 0.3-3 micro-meters reflectance, it reflects the shortwave energy coming from the sun by 5% on oceans, 15-25% on vegetation, 35-40% on sand/desert/beach, 60-80% on snow/clouds, roughly. This energy fraction is returned to atmosphere for complex interaction with gas particles again before leaving the atmosphere altogether.

[[File:EnergyBalance.png||500px||Alt="Chemin, 2006, PhD Thesis"]]

As for the shortwave surface balance, the '''energy''' is also received in the longwave, but interacts with the grey-body characteristics of the surface elements. The blackbody to greybody fraction is ruled by emissivity, a hidden component in the thermal spectrum (search for Temperature- Emissivity Separation algorithms). On Earth, emissivity is always above 0.9 (mostly 0.96-0.98). The Stefan-Boltzman equation is dealing with blackbody energy, and multiplying it by Emissivity transforms it to greybody energy emitted.

Together, shortwave and longwave '''energy balance''' provide with the net radiation balance at the surface of the planetary body. This crucial term is the total energy available for thermodynamic fluxes to act on the surface of the planet. On a planet like Earth where there is a triple phase of a given molecule (H20, in gas, liquid, solid), energy available is used to transfer phases from lower energy to higher energy (sublimation, liquefaction, evaporation). Thermal transfers also happen as conduction (soil, rocks) and convection (atmosphere, oceans). The i.eb.* modules and models deal with conduction (i.eb.g0, for thermal conduction in soils), convection (i.eb.h_* especially, for thermal convection in atmosphere) and by residual method estimate the energy left for evaporation processes ({{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} are examples). {{AddonCmd|i.eb.z0m}} is producing an estimation of the surface roughness.

== i.evapo.* modules/models are integrated '''models'''==

from the oldest that are not based on energy balance, but are computing some form of evapotranspiration (ETo: Evapotranspiration for a reference of 20cm well-watered grass, see FAO no56 report of Richard Allen)
* Penman-Monteith {{cmd|i.evapo.pm}}
* Priestley-Taylor {{cmd|i.evapo.pt}}
* Hargreaves {{cmd|i.evapo.mh}}

to newer that are in some form or another linked to thermal processes.
* Thermal index {{cmd|i.evapo.senay}}
* Biome thermal processes {{cmd|i.evapo.zk}} (global model based on typical info from biomes Albedo/soil heat flux)
* Thermodynamic processes {{cmd|i.evapo.potrad}} (potential ET if not water stress, based on astronomical equations only)

Most developed thermodynamic models are:
* TSEB (i.evapo.tseb: Two Source Energy Balance, Schmugge, Kustas, etc., 2000) Status: Incomplete Module
* SEBS (i.evapo.sebs: Zhu et al, 2002) Status: Incomplete Module
* SEBAL {{cmd|i.eb.h_sebal01}}, {{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} (Bastiaanssen et al, 1998, 2001)

and some newer models like RESET and others. The list of models using thermodynamic principles is large now, and any willing to submit a code for any of them is most welcome.

== Temporal Integration of Evapotranspiration ==

When integrated in time {{cmd|i.evapo.time}}, seasonal evapotranspiration can be computed and therefore permit a full monitoring of vegetation growth over the period. This is also a basis of crop yield estimation (see {{cmd|i.biomass}}).

[[File:I_evapo_time.png||800px||Alt="Temporal integration of ETa with data from a weather station"]]

== See also ==

* Manual pages:
** http://grass.osgeo.org/grass70/manuals/topic_energy_balance.html
** http://grass.osgeo.org/grass70/manuals/topic_evapotranspiration.html
** http://grass.osgeo.org/grass70/manuals/keywords.html#evaporative%20fraction
* [https://svn.osgeo.org/grass/grass-promo/tutorials/grass_landsat_ETa/ Evapotranspiration mapping with Landsat 5TM in GRASS GIS]

== Example script to process Landsat 8 Evapotranspiration ==
<source lang=shell>
#!/bin/bash

# This scripts imports, pre-processes and compute Evapotranspiration for Landsat 8
# Run in a Location compatible with your L8 dataset

# Atmospheric Transmissivity single-way
tsw=0.75
# Light Use Efficiency (please change dependent on your crop/plant)
lue=1.0

# Import files in GRASS
for file in *TIF
do
r.in.gdal -e input=$file output=$(echo $file | sed 's/.TIF//') --o
done
# Set region to newly imported files
g.region -s raster=$(ls *TIF | grep B1.TIF | sed 's/.TIF//')
LC08=$(ls *TIF | grep B1.TIF | sed 's/1.TIF//')

# Set metadata MTL file name
LC08met=$(ls *MTL.txt)

# Import DEM
r.import input=dem.tif output=dem extent=region --o
# Extract date
date=$(i.landsat.toar input=$LC08 output=l8 metfile=$LC08met lsatmet=date --o)
echo $date

# Apply Top of Atmosphere correction
i.landsat.toar input=$LC08 output=l8 metfile=$LC08met --o

# Set NODATA
for file in $(g.list type=raster pattern=LC08*)
do
r.null map=$file setnull=0
done

#Set computing area to B10 active pixels
for n in 1 2 3 4 5 6 7 8 9
do
r.mapcalc expression="temp=if(isnull(l810),null(),l8$n)" --o
g.rename raster=temp,l8$n --o
done

base=l8
# Create config files for 6s
year=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\1/')
mm=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\2/')
dd=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\3/')
doy=$( date -I -d "$year-$mm-$dd" | date +%j)
hh=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\1/')
mn=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\2/')
ss=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\3/')
time=$(echo "$hh + ($mn / 60.0) + ($ss / 3600.0)" | bc -l)
lat=$(echo $(cat *MTL* | grep CORNER_ | grep LAT | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
lon=$(echo $(cat *MTL* | grep CORNER_ | grep LON | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
echo "18 - geometrical conditions=Landsat 8" > 6s_conf.txt
echo "$mm $dd $hh.$mm $lon $lat - month day hh.ddd longitude latitude ("hh.ddd" is in GMT decimal hours)" >> 6s_conf.txt
echo "1 - atmospheric model=midlatitude summer" >> 6s_conf.txt
echo "1 - aerosols model=continental" >> 6s_conf.txt
echo "24 - visibility [km] (aerosol model concentration)" >> 6s_conf.txt
echo "-0.110 - mean target elevation above sea level [km]" >> 6s_conf.txt
echo "-1000 - sensor on board a satellite" >> 6s_conf.txt
#For Band 1:
cat 6s_conf.txt > $base\1_conf.txt
echo "115 - Coastal band of OLI Landsat 8" >> $base\1_conf.txt
#For Band 2:
cat 6s_conf.txt > $base\2_conf.txt
echo "116 - Blue band of OLI Landsat 8" >> $base\2_conf.txt
#For Band 3:
cat 6s_conf.txt > $base\3_conf.txt
echo "117 - Green band of OLI Landsat 8" >> $base\3_conf.txt
#For Band 4:
cat 6s_conf.txt > $base\4_conf.txt
echo "118 - Red band of OLI Landsat 8" >> $base\4_conf.txt
#For Band 5:
cat 6s_conf.txt > $base\5_conf.txt
echo "120 - NIR band of OLI Landsat 8" >> $base\5_conf.txt
#For Band 6:
cat 6s_conf.txt > $base\6_conf.txt
echo "122 - SWIR1 band of OLI Landsat 8" >> $base\6_conf.txt
#For Band 7:
cat 6s_conf.txt > $base\7_conf.txt
echo "123 - SWIR2 band of OLI Landsat 8" >> $base\7_conf.txt
#For Band 8:
cat 6s_conf.txt > $base\8_conf.txt
echo "119 - PAN band of OLI Landsat 8" >> $base\8_conf.txt
#For Band 9:
cat 6s_conf.txt > $base\9_conf.txt
echo "121 - Cirrus band of OLI Landsat 8" >> $base\9_conf.txt

# Compute Atmospheric correction
for file in $base\1 $base\2 $base\3 $base\4 $base\5 $base\6 $base\7 $base\8 $base\9
do
i.atcorr -r $file elevation=dem parameters=$file\_conf.txt output=s$file --o
r.mapcalc expression="temp=if(isnull(l810),null(),s$file)" --o
g.rename raster=temp,s$file --o
r.colors map=$file color=grey
done

# Create cloud Mask
g.rename raster=l83,l7.2
g.rename raster=l84,l7.3
g.rename raster=l85,l7.4
g.rename raster=l86,l7.5
g.rename raster=l810,l7.6
i.landsat.acca -5 -f input=l7. output=temp --o
g.rename raster=l7.2,l83
g.rename raster=l7.3,l84
g.rename raster=l7.4,l85
g.rename raster=l7.5,l86
g.rename raster=l7.6,l810
r.mapcalc expression="temp1=if(isnull(temp),0,1)" --o
r.mapcalc expression="maskVIS=if(isnull(l810),null(),temp1)" --o

# Apply mask
r.mask raster=maskVIS maskcats=0

# Latitude map
r.latlong input=l81 output=latitude --o

# Longitude map
r.latlong -l input=l81 output=longitude --o

# i.albedo
i.albedo -8 -c input=sl81,sl82,sl83,sl84,sl85,sl86,sl87 output=l8alb --o
r.colors -e map=l8alb color=grey

# i.vi viname=ndvi
i.vi viname=ndvi red=sl84 nir=sl85 output=l8ndvi --o
r.colors map=l8ndvi color=ndvi

# Update processing mask with water
r.mapcalc expression="maskWATER=if(l8ndvi<=0,0,maskVIS)" --o
r.mask -r
r.mask raster=maskWATER maskcats=0

# i.emissivity
i.emissivity input=l8ndvi output=l8emis --o

# i.vi viname=savi
i.vi viname=savi red=sl84 nir=sl85 output=l8savi --o
r.colors map=l8savi color=ndvi

# Daily Net Radiation (RNETD)
r.mapcalc expression="temp=if(isnull(l810),null(),dem)" --o
r.slope.aspect elevation=temp slope=slopedd aspect=aspectdd --o
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=rnetd insol_time=sunhours day=$doy nprocs=8 --o
r.mapcalc expression="l8rnetd=rnetd/sunhours" --o

# Instantaneous Net Radiation (RNET)
timelocal=$(echo "$time + (24.0 / 360.0) * $lon" | bc -l)
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=l8rnet day=$doy time=$timelocal nprocs=8 --o

# Soil Heat Flux (G0)
r.mapcalc expression="overpasstime=$time" --o
i.eb.soilheatflux albedo=l8alb ndvi=l8ndvi temperature=l810 netradiation=l8rnet localutctime=overpasstime output=l8g0 --o

# Sensible Heat Flux (H)
i.eb.z0m -p input=l8savi output=z0m --o
r.mapcalc expression="T0dem=0.00612*dem+l810" --o
i.eb.hsebal01 -a netradiation=l8rnet soilheatflux=l8g0 aerodynresistance=z0m temperaturemeansealevel=T0dem frictionvelocitystar=0.32407 vapourpressureactual=1.511 output=l8h --o

# Evaporative Fraction & Soil Moisture
i.eb.evapfr -m netradiation=l8rnet soilheatflux=l8g0 sensibleheatflux=l8h evaporativefraction=l8evapfr soilmoisture=l8sm --o

# Actual Evapotranspiration (ETa)
i.eb.eta netradiationdiurnal=l8rnetd evaporativefraction=l8evapfr temperature=l810 output=l8eta --o

# Biomass growth
r.mapcalc expression="l8fpar=1.257*l8ndvi-0.161" --o
r.mapcalc expression="doy=$doy" --o
r.mapcalc expression="tsw=$tsw" --o
r.mapcalc expression="lue=$lue" --o
i.biomass fpar=l8fpar lightuse_efficiency=lue latitude=latitude dayofyear=doy transmissivity_singleway=tsw water_availability=l8evapfr output=l8biom --o

# Export everything we need
for rstF in l8eta l8sm l8biom l8ndvi l8alb l8rnet l8g0 l8rnetd
do
r.null map=$rstF setnull=-100000-0
r.out.gdal -c -m input=$rstF output=$rstF\_$year$mm$dd.tif nodata=-32768 --o
done

</source>

[[Category: Documentation]]
[[Category: Evapotranspiration]]

Image processing/Evapotranspiration

2023-12-02T08:16:21Z

⚠️Ychemin: /* i.evapo.* modules/models are integrated models */

[[File:EnergyBalance.png||500px||Alt="Chemin, 2006, PhD Thesis"]]

== Introduction ==
The '''radiation''' is coming from the planet's star, Earth has around 1800 W/m2 of incoming radiation from the sun (also called irradiance) at exo-atmospheric altitude.

Atmospheric transfer of '''irradiance''' is subject to atmosphere particles interactions, scattering, diffraction, reflection, etc.

When reaching the planetary surface, is it partially reflected in the shortwave by '''Albedo'''. Surface Albedo is the integrated 0.3-3 micro-meters reflectance, it reflects the shortwave energy coming from the sun by 5% on oceans, 15-25% on vegetation, 35-40% on sand/desert/beach, 60-80% on snow/clouds, roughly. This energy fraction is returned to atmosphere for complex interaction with gas particles again before leaving the atmosphere altogether.

As for the shortwave surface balance, the '''energy''' is also received in the longwave, but interacts with the grey-body characteristics of the surface elements. The blackbody to greybody fraction is ruled by emissivity, a hidden component in the thermal spectrum (search for Temperature- Emissivity Separation algorithms). On Earth, emissivity is always above 0.9 (mostly 0.96-0.98). The Stefan-Boltzman equation is dealing with blackbody energy, and multiplying it by Emissivity transforms it to greybody energy emitted.

Together, shortwave and longwave '''energy balance''' provide with the net radiation balance at the surface of the planetary body. This crucial term is the total energy available for thermodynamic fluxes to act on the surface of the planet. On a planet like Earth where there is a triple phase of a given molecule (H20, in gas, liquid, solid), energy available is used to transfer phases from lower energy to higher energy (sublimation, liquefaction, evaporation). Thermal transfers also happen as conduction (soil, rocks) and convection (atmosphere, oceans). The i.eb.* modules and models deal with conduction (i.eb.g0, for thermal conduction in soils), convection (i.eb.h_* especially, for thermal convection in atmosphere) and by residual method estimate the energy left for evaporation processes ({{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} are examples). {{AddonCmd|i.eb.z0m}} is producing an estimation of the surface roughness.

== i.evapo.* modules/models are integrated '''models'''==

from the oldest that are not based on energy balance, but are computing some form of evapotranspiration (ETo: Evapotranspiration for a reference of 20cm well-watered grass, see FAO no56 report of Richard Allen)
* Penman-Monteith {{cmd|i.evapo.pm}}
* Priestley-Taylor {{cmd|i.evapo.pt}}
* Hargreaves {{cmd|i.evapo.mh}}

to newer that are in some form or another linked to thermal processes.
* Thermal index {{cmd|i.evapo.senay}}
* Biome thermal processes {{cmd|i.evapo.zk}} (global model based on typical info from biomes Albedo/soil heat flux)
* Thermodynamic processes {{cmd|i.evapo.potrad}} (potential ET if not water stress, based on astronomical equations only)

Most developed thermodynamic models are:
* TSEB (i.evapo.tseb: Two Source Energy Balance, Schmugge, Kustas, etc., 2000) Status: Incomplete Module
* SEBS (i.evapo.sebs: Zhu et al, 2002) Status: Incomplete Module
* SEBAL {{cmd|i.eb.h_sebal01}}, {{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} (Bastiaanssen et al, 1998, 2001)

and some newer models like RESET and others. The list of models using thermodynamic principles is large now, and any willing to submit a code for any of them is most welcome.

== Temporal Integration of Evapotranspiration ==

When integrated in time {{cmd|i.evapo.time}}, seasonal evapotranspiration can be computed and therefore permit a full monitoring of vegetation growth over the period. This is also a basis of crop yield estimation (see {{cmd|i.biomass}}).

[[File:I_evapo_time.png||800px||Alt="Temporal integration of ETa with data from a weather station"]]

== See also ==

* Manual pages:
** http://grass.osgeo.org/grass70/manuals/topic_energy_balance.html
** http://grass.osgeo.org/grass70/manuals/topic_evapotranspiration.html
** http://grass.osgeo.org/grass70/manuals/keywords.html#evaporative%20fraction
* [https://svn.osgeo.org/grass/grass-promo/tutorials/grass_landsat_ETa/ Evapotranspiration mapping with Landsat 5TM in GRASS GIS]

== Example script to process Landsat 8 Evapotranspiration ==
<source lang=shell>
#!/bin/bash

# This scripts imports, pre-processes and compute Evapotranspiration for Landsat 8
# Run in a Location compatible with your L8 dataset

# Atmospheric Transmissivity single-way
tsw=0.75
# Light Use Efficiency (please change dependent on your crop/plant)
lue=1.0

# Import files in GRASS
for file in *TIF
do
r.in.gdal -e input=$file output=$(echo $file | sed 's/.TIF//') --o
done
# Set region to newly imported files
g.region -s raster=$(ls *TIF | grep B1.TIF | sed 's/.TIF//')
LC08=$(ls *TIF | grep B1.TIF | sed 's/1.TIF//')

# Set metadata MTL file name
LC08met=$(ls *MTL.txt)

# Import DEM
r.import input=dem.tif output=dem extent=region --o
# Extract date
date=$(i.landsat.toar input=$LC08 output=l8 metfile=$LC08met lsatmet=date --o)
echo $date

# Apply Top of Atmosphere correction
i.landsat.toar input=$LC08 output=l8 metfile=$LC08met --o

# Set NODATA
for file in $(g.list type=raster pattern=LC08*)
do
r.null map=$file setnull=0
done

#Set computing area to B10 active pixels
for n in 1 2 3 4 5 6 7 8 9
do
r.mapcalc expression="temp=if(isnull(l810),null(),l8$n)" --o
g.rename raster=temp,l8$n --o
done

base=l8
# Create config files for 6s
year=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\1/')
mm=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\2/')
dd=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\3/')
doy=$( date -I -d "$year-$mm-$dd" | date +%j)
hh=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\1/')
mn=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\2/')
ss=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\3/')
time=$(echo "$hh + ($mn / 60.0) + ($ss / 3600.0)" | bc -l)
lat=$(echo $(cat *MTL* | grep CORNER_ | grep LAT | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
lon=$(echo $(cat *MTL* | grep CORNER_ | grep LON | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
echo "18 - geometrical conditions=Landsat 8" > 6s_conf.txt
echo "$mm $dd $hh.$mm $lon $lat - month day hh.ddd longitude latitude ("hh.ddd" is in GMT decimal hours)" >> 6s_conf.txt
echo "1 - atmospheric model=midlatitude summer" >> 6s_conf.txt
echo "1 - aerosols model=continental" >> 6s_conf.txt
echo "24 - visibility [km] (aerosol model concentration)" >> 6s_conf.txt
echo "-0.110 - mean target elevation above sea level [km]" >> 6s_conf.txt
echo "-1000 - sensor on board a satellite" >> 6s_conf.txt
#For Band 1:
cat 6s_conf.txt > $base\1_conf.txt
echo "115 - Coastal band of OLI Landsat 8" >> $base\1_conf.txt
#For Band 2:
cat 6s_conf.txt > $base\2_conf.txt
echo "116 - Blue band of OLI Landsat 8" >> $base\2_conf.txt
#For Band 3:
cat 6s_conf.txt > $base\3_conf.txt
echo "117 - Green band of OLI Landsat 8" >> $base\3_conf.txt
#For Band 4:
cat 6s_conf.txt > $base\4_conf.txt
echo "118 - Red band of OLI Landsat 8" >> $base\4_conf.txt
#For Band 5:
cat 6s_conf.txt > $base\5_conf.txt
echo "120 - NIR band of OLI Landsat 8" >> $base\5_conf.txt
#For Band 6:
cat 6s_conf.txt > $base\6_conf.txt
echo "122 - SWIR1 band of OLI Landsat 8" >> $base\6_conf.txt
#For Band 7:
cat 6s_conf.txt > $base\7_conf.txt
echo "123 - SWIR2 band of OLI Landsat 8" >> $base\7_conf.txt
#For Band 8:
cat 6s_conf.txt > $base\8_conf.txt
echo "119 - PAN band of OLI Landsat 8" >> $base\8_conf.txt
#For Band 9:
cat 6s_conf.txt > $base\9_conf.txt
echo "121 - Cirrus band of OLI Landsat 8" >> $base\9_conf.txt

# Compute Atmospheric correction
for file in $base\1 $base\2 $base\3 $base\4 $base\5 $base\6 $base\7 $base\8 $base\9
do
i.atcorr -r $file elevation=dem parameters=$file\_conf.txt output=s$file --o
r.mapcalc expression="temp=if(isnull(l810),null(),s$file)" --o
g.rename raster=temp,s$file --o
r.colors map=$file color=grey
done

# Create cloud Mask
g.rename raster=l83,l7.2
g.rename raster=l84,l7.3
g.rename raster=l85,l7.4
g.rename raster=l86,l7.5
g.rename raster=l810,l7.6
i.landsat.acca -5 -f input=l7. output=temp --o
g.rename raster=l7.2,l83
g.rename raster=l7.3,l84
g.rename raster=l7.4,l85
g.rename raster=l7.5,l86
g.rename raster=l7.6,l810
r.mapcalc expression="temp1=if(isnull(temp),0,1)" --o
r.mapcalc expression="maskVIS=if(isnull(l810),null(),temp1)" --o

# Apply mask
r.mask raster=maskVIS maskcats=0

# Latitude map
r.latlong input=l81 output=latitude --o

# Longitude map
r.latlong -l input=l81 output=longitude --o

# i.albedo
i.albedo -8 -c input=sl81,sl82,sl83,sl84,sl85,sl86,sl87 output=l8alb --o
r.colors -e map=l8alb color=grey

# i.vi viname=ndvi
i.vi viname=ndvi red=sl84 nir=sl85 output=l8ndvi --o
r.colors map=l8ndvi color=ndvi

# Update processing mask with water
r.mapcalc expression="maskWATER=if(l8ndvi<=0,0,maskVIS)" --o
r.mask -r
r.mask raster=maskWATER maskcats=0

# i.emissivity
i.emissivity input=l8ndvi output=l8emis --o

# i.vi viname=savi
i.vi viname=savi red=sl84 nir=sl85 output=l8savi --o
r.colors map=l8savi color=ndvi

# Daily Net Radiation (RNETD)
r.mapcalc expression="temp=if(isnull(l810),null(),dem)" --o
r.slope.aspect elevation=temp slope=slopedd aspect=aspectdd --o
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=rnetd insol_time=sunhours day=$doy nprocs=8 --o
r.mapcalc expression="l8rnetd=rnetd/sunhours" --o

# Instantaneous Net Radiation (RNET)
timelocal=$(echo "$time + (24.0 / 360.0) * $lon" | bc -l)
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=l8rnet day=$doy time=$timelocal nprocs=8 --o

# Soil Heat Flux (G0)
r.mapcalc expression="overpasstime=$time" --o
i.eb.soilheatflux albedo=l8alb ndvi=l8ndvi temperature=l810 netradiation=l8rnet localutctime=overpasstime output=l8g0 --o

# Sensible Heat Flux (H)
i.eb.z0m -p input=l8savi output=z0m --o
r.mapcalc expression="T0dem=0.00612*dem+l810" --o
i.eb.hsebal01 -a netradiation=l8rnet soilheatflux=l8g0 aerodynresistance=z0m temperaturemeansealevel=T0dem frictionvelocitystar=0.32407 vapourpressureactual=1.511 output=l8h --o

# Evaporative Fraction & Soil Moisture
i.eb.evapfr -m netradiation=l8rnet soilheatflux=l8g0 sensibleheatflux=l8h evaporativefraction=l8evapfr soilmoisture=l8sm --o

# Actual Evapotranspiration (ETa)
i.eb.eta netradiationdiurnal=l8rnetd evaporativefraction=l8evapfr temperature=l810 output=l8eta --o

# Biomass growth
r.mapcalc expression="l8fpar=1.257*l8ndvi-0.161" --o
r.mapcalc expression="doy=$doy" --o
r.mapcalc expression="tsw=$tsw" --o
r.mapcalc expression="lue=$lue" --o
i.biomass fpar=l8fpar lightuse_efficiency=lue latitude=latitude dayofyear=doy transmissivity_singleway=tsw water_availability=l8evapfr output=l8biom --o

# Export everything we need
for rstF in l8eta l8sm l8biom l8ndvi l8alb l8rnet l8g0 l8rnetd
do
r.null map=$rstF setnull=-100000-0
r.out.gdal -c -m input=$rstF output=$rstF\_$year$mm$dd.tif nodata=-32768 --o
done

</source>

[[Category: Documentation]]
[[Category: Evapotranspiration]]

Image processing/Evapotranspiration

2023-12-02T08:13:20Z

⚠️Ychemin:

[[File:EnergyBalance.png||500px||Alt="Chemin, 2006, PhD Thesis"]]

== Introduction ==
The '''radiation''' is coming from the planet's star, Earth has around 1800 W/m2 of incoming radiation from the sun (also called irradiance) at exo-atmospheric altitude.

Atmospheric transfer of '''irradiance''' is subject to atmosphere particles interactions, scattering, diffraction, reflection, etc.

When reaching the planetary surface, is it partially reflected in the shortwave by '''Albedo'''. Surface Albedo is the integrated 0.3-3 micro-meters reflectance, it reflects the shortwave energy coming from the sun by 5% on oceans, 15-25% on vegetation, 35-40% on sand/desert/beach, 60-80% on snow/clouds, roughly. This energy fraction is returned to atmosphere for complex interaction with gas particles again before leaving the atmosphere altogether.

As for the shortwave surface balance, the '''energy''' is also received in the longwave, but interacts with the grey-body characteristics of the surface elements. The blackbody to greybody fraction is ruled by emissivity, a hidden component in the thermal spectrum (search for Temperature- Emissivity Separation algorithms). On Earth, emissivity is always above 0.9 (mostly 0.96-0.98). The Stefan-Boltzman equation is dealing with blackbody energy, and multiplying it by Emissivity transforms it to greybody energy emitted.

Together, shortwave and longwave '''energy balance''' provide with the net radiation balance at the surface of the planetary body. This crucial term is the total energy available for thermodynamic fluxes to act on the surface of the planet. On a planet like Earth where there is a triple phase of a given molecule (H20, in gas, liquid, solid), energy available is used to transfer phases from lower energy to higher energy (sublimation, liquefaction, evaporation). Thermal transfers also happen as conduction (soil, rocks) and convection (atmosphere, oceans). The i.eb.* modules and models deal with conduction (i.eb.g0, for thermal conduction in soils), convection (i.eb.h_* especially, for thermal convection in atmosphere) and by residual method estimate the energy left for evaporation processes ({{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} are examples). {{AddonCmd|i.eb.z0m}} is producing an estimation of the surface roughness.

== i.evapo.* modules/models are integrated '''models'''==

from the oldest that are not based on energy balance, but are computing some form of evapotranspiration (ETo: Evapotranspiration for a reference of 20cm well-watered grass, see FAO no56 report of Richard Allen)
* Penman-Monteith {{cmd|i.evapo.pm}}
* Priestley-Taylor {{cmd|i.evapo.pt}}
* Hargreaves {{cmd|i.evapo.mh}}

to newer that are in some form or another linked to thermal processes.
* Thermal index {{cmd|i.evapo.senay}}
* Biome thermal processes {{cmd|i.evapo.zk}} (global model based on typical info from biomes Albedo/soil heat flux)
* Thermodynamic processes {{cmd|i.evapo.potrad}} (potential ET if not water stress, based on astronomical equations only)

Most developed thermodynamic models are:
* TSEB (i.evapo.tseb: Two Source Energy Balance, Schmugge, Kustas, etc., 2000)
* SEBS (i.evapo.sebs: Zhu et al, 2002)
* SEBAL {{cmd|i.eb.h_sebal01}}, {{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} (Bastiaanssen et al, 1998, 2001)

and some newer models like RESET and others. The list of models using thermodynamic principles is large now, and any willing to submit a code for any of them is most welcome.

== Temporal Integration of Evapotranspiration ==

When integrated in time {{cmd|i.evapo.time}}, seasonal evapotranspiration can be computed and therefore permit a full monitoring of vegetation growth over the period. This is also a basis of crop yield estimation (see {{cmd|i.biomass}}).

[[File:I_evapo_time.png||800px||Alt="Temporal integration of ETa with data from a weather station"]]

== See also ==

* Manual pages:
** http://grass.osgeo.org/grass70/manuals/topic_energy_balance.html
** http://grass.osgeo.org/grass70/manuals/topic_evapotranspiration.html
** http://grass.osgeo.org/grass70/manuals/keywords.html#evaporative%20fraction
* [https://svn.osgeo.org/grass/grass-promo/tutorials/grass_landsat_ETa/ Evapotranspiration mapping with Landsat 5TM in GRASS GIS]

== Example script to process Landsat 8 Evapotranspiration ==
<source lang=shell>
#!/bin/bash

# This scripts imports, pre-processes and compute Evapotranspiration for Landsat 8
# Run in a Location compatible with your L8 dataset

# Atmospheric Transmissivity single-way
tsw=0.75
# Light Use Efficiency (please change dependent on your crop/plant)
lue=1.0

# Import files in GRASS
for file in *TIF
do
r.in.gdal -e input=$file output=$(echo $file | sed 's/.TIF//') --o
done
# Set region to newly imported files
g.region -s raster=$(ls *TIF | grep B1.TIF | sed 's/.TIF//')
LC08=$(ls *TIF | grep B1.TIF | sed 's/1.TIF//')

# Set metadata MTL file name
LC08met=$(ls *MTL.txt)

# Import DEM
r.import input=dem.tif output=dem extent=region --o
# Extract date
date=$(i.landsat.toar input=$LC08 output=l8 metfile=$LC08met lsatmet=date --o)
echo $date

# Apply Top of Atmosphere correction
i.landsat.toar input=$LC08 output=l8 metfile=$LC08met --o

# Set NODATA
for file in $(g.list type=raster pattern=LC08*)
do
r.null map=$file setnull=0
done

#Set computing area to B10 active pixels
for n in 1 2 3 4 5 6 7 8 9
do
r.mapcalc expression="temp=if(isnull(l810),null(),l8$n)" --o
g.rename raster=temp,l8$n --o
done

base=l8
# Create config files for 6s
year=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\1/')
mm=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\2/')
dd=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\3/')
doy=$( date -I -d "$year-$mm-$dd" | date +%j)
hh=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\1/')
mn=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\2/')
ss=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\3/')
time=$(echo "$hh + ($mn / 60.0) + ($ss / 3600.0)" | bc -l)
lat=$(echo $(cat *MTL* | grep CORNER_ | grep LAT | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
lon=$(echo $(cat *MTL* | grep CORNER_ | grep LON | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
echo "18 - geometrical conditions=Landsat 8" > 6s_conf.txt
echo "$mm $dd $hh.$mm $lon $lat - month day hh.ddd longitude latitude ("hh.ddd" is in GMT decimal hours)" >> 6s_conf.txt
echo "1 - atmospheric model=midlatitude summer" >> 6s_conf.txt
echo "1 - aerosols model=continental" >> 6s_conf.txt
echo "24 - visibility [km] (aerosol model concentration)" >> 6s_conf.txt
echo "-0.110 - mean target elevation above sea level [km]" >> 6s_conf.txt
echo "-1000 - sensor on board a satellite" >> 6s_conf.txt
#For Band 1:
cat 6s_conf.txt > $base\1_conf.txt
echo "115 - Coastal band of OLI Landsat 8" >> $base\1_conf.txt
#For Band 2:
cat 6s_conf.txt > $base\2_conf.txt
echo "116 - Blue band of OLI Landsat 8" >> $base\2_conf.txt
#For Band 3:
cat 6s_conf.txt > $base\3_conf.txt
echo "117 - Green band of OLI Landsat 8" >> $base\3_conf.txt
#For Band 4:
cat 6s_conf.txt > $base\4_conf.txt
echo "118 - Red band of OLI Landsat 8" >> $base\4_conf.txt
#For Band 5:
cat 6s_conf.txt > $base\5_conf.txt
echo "120 - NIR band of OLI Landsat 8" >> $base\5_conf.txt
#For Band 6:
cat 6s_conf.txt > $base\6_conf.txt
echo "122 - SWIR1 band of OLI Landsat 8" >> $base\6_conf.txt
#For Band 7:
cat 6s_conf.txt > $base\7_conf.txt
echo "123 - SWIR2 band of OLI Landsat 8" >> $base\7_conf.txt
#For Band 8:
cat 6s_conf.txt > $base\8_conf.txt
echo "119 - PAN band of OLI Landsat 8" >> $base\8_conf.txt
#For Band 9:
cat 6s_conf.txt > $base\9_conf.txt
echo "121 - Cirrus band of OLI Landsat 8" >> $base\9_conf.txt

# Compute Atmospheric correction
for file in $base\1 $base\2 $base\3 $base\4 $base\5 $base\6 $base\7 $base\8 $base\9
do
i.atcorr -r $file elevation=dem parameters=$file\_conf.txt output=s$file --o
r.mapcalc expression="temp=if(isnull(l810),null(),s$file)" --o
g.rename raster=temp,s$file --o
r.colors map=$file color=grey
done

# Create cloud Mask
g.rename raster=l83,l7.2
g.rename raster=l84,l7.3
g.rename raster=l85,l7.4
g.rename raster=l86,l7.5
g.rename raster=l810,l7.6
i.landsat.acca -5 -f input=l7. output=temp --o
g.rename raster=l7.2,l83
g.rename raster=l7.3,l84
g.rename raster=l7.4,l85
g.rename raster=l7.5,l86
g.rename raster=l7.6,l810
r.mapcalc expression="temp1=if(isnull(temp),0,1)" --o
r.mapcalc expression="maskVIS=if(isnull(l810),null(),temp1)" --o

# Apply mask
r.mask raster=maskVIS maskcats=0

# Latitude map
r.latlong input=l81 output=latitude --o

# Longitude map
r.latlong -l input=l81 output=longitude --o

# i.albedo
i.albedo -8 -c input=sl81,sl82,sl83,sl84,sl85,sl86,sl87 output=l8alb --o
r.colors -e map=l8alb color=grey

# i.vi viname=ndvi
i.vi viname=ndvi red=sl84 nir=sl85 output=l8ndvi --o
r.colors map=l8ndvi color=ndvi

# Update processing mask with water
r.mapcalc expression="maskWATER=if(l8ndvi<=0,0,maskVIS)" --o
r.mask -r
r.mask raster=maskWATER maskcats=0

# i.emissivity
i.emissivity input=l8ndvi output=l8emis --o

# i.vi viname=savi
i.vi viname=savi red=sl84 nir=sl85 output=l8savi --o
r.colors map=l8savi color=ndvi

# Daily Net Radiation (RNETD)
r.mapcalc expression="temp=if(isnull(l810),null(),dem)" --o
r.slope.aspect elevation=temp slope=slopedd aspect=aspectdd --o
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=rnetd insol_time=sunhours day=$doy nprocs=8 --o
r.mapcalc expression="l8rnetd=rnetd/sunhours" --o

# Instantaneous Net Radiation (RNET)
timelocal=$(echo "$time + (24.0 / 360.0) * $lon" | bc -l)
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=l8rnet day=$doy time=$timelocal nprocs=8 --o

# Soil Heat Flux (G0)
r.mapcalc expression="overpasstime=$time" --o
i.eb.soilheatflux albedo=l8alb ndvi=l8ndvi temperature=l810 netradiation=l8rnet localutctime=overpasstime output=l8g0 --o

# Sensible Heat Flux (H)
i.eb.z0m -p input=l8savi output=z0m --o
r.mapcalc expression="T0dem=0.00612*dem+l810" --o
i.eb.hsebal01 -a netradiation=l8rnet soilheatflux=l8g0 aerodynresistance=z0m temperaturemeansealevel=T0dem frictionvelocitystar=0.32407 vapourpressureactual=1.511 output=l8h --o

# Evaporative Fraction & Soil Moisture
i.eb.evapfr -m netradiation=l8rnet soilheatflux=l8g0 sensibleheatflux=l8h evaporativefraction=l8evapfr soilmoisture=l8sm --o

# Actual Evapotranspiration (ETa)
i.eb.eta netradiationdiurnal=l8rnetd evaporativefraction=l8evapfr temperature=l810 output=l8eta --o

# Biomass growth
r.mapcalc expression="l8fpar=1.257*l8ndvi-0.161" --o
r.mapcalc expression="doy=$doy" --o
r.mapcalc expression="tsw=$tsw" --o
r.mapcalc expression="lue=$lue" --o
i.biomass fpar=l8fpar lightuse_efficiency=lue latitude=latitude dayofyear=doy transmissivity_singleway=tsw water_availability=l8evapfr output=l8biom --o

# Export everything we need
for rstF in l8eta l8sm l8biom l8ndvi l8alb l8rnet l8g0 l8rnetd
do
r.null map=$rstF setnull=-100000-0
r.out.gdal -c -m input=$rstF output=$rstF\_$year$mm$dd.tif nodata=-32768 --o
done

</source>

[[Category: Documentation]]
[[Category: Evapotranspiration]]

Image processing/Evapotranspiration

2023-12-02T08:11:41Z

⚠️Ychemin:

[[File:EnergyBalance.png||500px||Alt="Chemin, 2006, PhD Thesis"]]

The '''radiation''' is coming from the planet's star, Earth has around 1800 W/m2 of incoming radiation from the sun (also called irradiance) at exo-atmospheric altitude.

Atmospheric transfer of '''irradiance''' is subject to atmosphere particles interactions, scattering, diffraction, reflection, etc.

When reaching the planetary surface, is it partially reflected in the shortwave by '''Albedo'''. Surface Albedo is the integrated 0.3-3 micro-meters reflectance, it reflects the shortwave energy coming from the sun by 5% on oceans, 15-25% on vegetation, 35-40% on sand/desert/beach, 60-80% on snow/clouds, roughly. This energy fraction is returned to atmosphere for complex interaction with gas particles again before leaving the atmosphere altogether.

As for the shortwave surface balance, the '''energy''' is also received in the longwave, but interacts with the grey-body characteristics of the surface elements. The blackbody to greybody fraction is ruled by emissivity, a hidden component in the thermal spectrum (search for Temperature- Emissivity Separation algorithms). On Earth, emissivity is always above 0.9 (mostly 0.96-0.98). The Stefan-Boltzman equation is dealing with blackbody energy, and multiplying it by Emissivity transforms it to greybody energy emitted.

Together, shortwave and longwave '''energy balance''' provide with the net radiation balance at the surface of the planetary body. This crucial term is the total energy available for thermodynamic fluxes to act on the surface of the planet. On a planet like Earth where there is a triple phase of a given molecule (H20, in gas, liquid, solid), energy available is used to transfer phases from lower energy to higher energy (sublimation, liquefaction, evaporation). Thermal transfers also happen as conduction (soil, rocks) and convection (atmosphere, oceans). The i.eb.* modules and models deal with conduction (i.eb.g0, for thermal conduction in soils), convection (i.eb.h_* especially, for thermal convection in atmosphere) and by residual method estimate the energy left for evaporation processes ({{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} are examples). {{AddonCmd|i.eb.z0m}} is producing an estimation of the surface roughness.

i.evapo.* modules/models are integrated '''models'''.

from the oldest that are not based on energy balance, but are computing some form of evapotranspiration (ETo: Evapotranspiration for a reference of 20cm well-watered grass, see FAO no56 report of Richard Allen)
* Penman-Monteith {{cmd|i.evapo.pm}}
* Priestley-Taylor {{cmd|i.evapo.pt}}
* Hargreaves {{cmd|i.evapo.mh}}

to newer that are in some form or another linked to thermal processes.
* Thermal index {{cmd|i.evapo.senay}}
* Biome thermal processes {{cmd|i.evapo.zk}} (global model based on typical info from biomes Albedo/soil heat flux)
* Thermodynamic processes {{cmd|i.evapo.potrad}} (potential ET if not water stress, based on astronomical equations only)

Most developed thermodynamic models are:
* TSEB (i.evapo.tseb: Two Source Energy Balance, Schmugge, Kustas, etc., 2000)
* SEBS (i.evapo.sebs: Zhu et al, 2002)
* SEBAL {{cmd|i.eb.h_sebal01}}, {{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} (Bastiaanssen et al, 1998, 2001)

and some newer models like RESET and others. The list of models using thermodynamic principles is large now, and any willing to submit a code for any of them is most welcome.

When integrated in time {{cmd|i.evapo.time}}, seasonal evapotranspiration can be computed and therefore permit a full monitoring of vegetation growth over the period. This is also a basis of crop yield estimation (see {{cmd|i.biomass}}).

[[File:I_evapo_time.png||800px||Alt="Temporal integration of ETa with data from a weather station"]]

== See also ==

* Manual pages:
** http://grass.osgeo.org/grass70/manuals/topic_energy_balance.html
** http://grass.osgeo.org/grass70/manuals/topic_evapotranspiration.html
** http://grass.osgeo.org/grass70/manuals/keywords.html#evaporative%20fraction
* [https://svn.osgeo.org/grass/grass-promo/tutorials/grass_landsat_ETa/ Evapotranspiration mapping with Landsat 5TM in GRASS GIS]

== Example script to process Landsat 8 Evapotranspiration ==
<source lang=shell>
#!/bin/bash

# This scripts imports, pre-processes and compute Evapotranspiration for Landsat 8
# Run in a Location compatible with your L8 dataset

# Atmospheric Transmissivity single-way
tsw=0.75
# Light Use Efficiency (please change dependent on your crop/plant)
lue=1.0

# Import files in GRASS
for file in *TIF
do
r.in.gdal -e input=$file output=$(echo $file | sed 's/.TIF//') --o
done
# Set region to newly imported files
g.region -s raster=$(ls *TIF | grep B1.TIF | sed 's/.TIF//')
LC08=$(ls *TIF | grep B1.TIF | sed 's/1.TIF//')

# Set metadata MTL file name
LC08met=$(ls *MTL.txt)

# Import DEM
r.import input=dem.tif output=dem extent=region --o
# Extract date
date=$(i.landsat.toar input=$LC08 output=l8 metfile=$LC08met lsatmet=date --o)
echo $date

# Apply Top of Atmosphere correction
i.landsat.toar input=$LC08 output=l8 metfile=$LC08met --o

# Set NODATA
for file in $(g.list type=raster pattern=LC08*)
do
r.null map=$file setnull=0
done

#Set computing area to B10 active pixels
for n in 1 2 3 4 5 6 7 8 9
do
r.mapcalc expression="temp=if(isnull(l810),null(),l8$n)" --o
g.rename raster=temp,l8$n --o
done

base=l8
# Create config files for 6s
year=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\1/')
mm=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\2/')
dd=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\3/')
doy=$( date -I -d "$year-$mm-$dd" | date +%j)
hh=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\1/')
mn=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\2/')
ss=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\3/')
time=$(echo "$hh + ($mn / 60.0) + ($ss / 3600.0)" | bc -l)
lat=$(echo $(cat *MTL* | grep CORNER_ | grep LAT | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
lon=$(echo $(cat *MTL* | grep CORNER_ | grep LON | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
echo "18 - geometrical conditions=Landsat 8" > 6s_conf.txt
echo "$mm $dd $hh.$mm $lon $lat - month day hh.ddd longitude latitude ("hh.ddd" is in GMT decimal hours)" >> 6s_conf.txt
echo "1 - atmospheric model=midlatitude summer" >> 6s_conf.txt
echo "1 - aerosols model=continental" >> 6s_conf.txt
echo "24 - visibility [km] (aerosol model concentration)" >> 6s_conf.txt
echo "-0.110 - mean target elevation above sea level [km]" >> 6s_conf.txt
echo "-1000 - sensor on board a satellite" >> 6s_conf.txt
#For Band 1:
cat 6s_conf.txt > $base\1_conf.txt
echo "115 - Coastal band of OLI Landsat 8" >> $base\1_conf.txt
#For Band 2:
cat 6s_conf.txt > $base\2_conf.txt
echo "116 - Blue band of OLI Landsat 8" >> $base\2_conf.txt
#For Band 3:
cat 6s_conf.txt > $base\3_conf.txt
echo "117 - Green band of OLI Landsat 8" >> $base\3_conf.txt
#For Band 4:
cat 6s_conf.txt > $base\4_conf.txt
echo "118 - Red band of OLI Landsat 8" >> $base\4_conf.txt
#For Band 5:
cat 6s_conf.txt > $base\5_conf.txt
echo "120 - NIR band of OLI Landsat 8" >> $base\5_conf.txt
#For Band 6:
cat 6s_conf.txt > $base\6_conf.txt
echo "122 - SWIR1 band of OLI Landsat 8" >> $base\6_conf.txt
#For Band 7:
cat 6s_conf.txt > $base\7_conf.txt
echo "123 - SWIR2 band of OLI Landsat 8" >> $base\7_conf.txt
#For Band 8:
cat 6s_conf.txt > $base\8_conf.txt
echo "119 - PAN band of OLI Landsat 8" >> $base\8_conf.txt
#For Band 9:
cat 6s_conf.txt > $base\9_conf.txt
echo "121 - Cirrus band of OLI Landsat 8" >> $base\9_conf.txt

# Compute Atmospheric correction
for file in $base\1 $base\2 $base\3 $base\4 $base\5 $base\6 $base\7 $base\8 $base\9
do
i.atcorr -r $file elevation=dem parameters=$file\_conf.txt output=s$file --o
r.mapcalc expression="temp=if(isnull(l810),null(),s$file)" --o
g.rename raster=temp,s$file --o
r.colors map=$file color=grey
done

# Create cloud Mask
g.rename raster=l83,l7.2
g.rename raster=l84,l7.3
g.rename raster=l85,l7.4
g.rename raster=l86,l7.5
g.rename raster=l810,l7.6
i.landsat.acca -5 -f input=l7. output=temp --o
g.rename raster=l7.2,l83
g.rename raster=l7.3,l84
g.rename raster=l7.4,l85
g.rename raster=l7.5,l86
g.rename raster=l7.6,l810
r.mapcalc expression="temp1=if(isnull(temp),0,1)" --o
r.mapcalc expression="maskVIS=if(isnull(l810),null(),temp1)" --o

# Apply mask
r.mask raster=maskVIS maskcats=0

# Latitude map
r.latlong input=l81 output=latitude --o

# Longitude map
r.latlong -l input=l81 output=longitude --o

# i.albedo
i.albedo -8 -c input=sl81,sl82,sl83,sl84,sl85,sl86,sl87 output=l8alb --o
r.colors -e map=l8alb color=grey

# i.vi viname=ndvi
i.vi viname=ndvi red=sl84 nir=sl85 output=l8ndvi --o
r.colors map=l8ndvi color=ndvi

# Update processing mask with water
r.mapcalc expression="maskWATER=if(l8ndvi<=0,0,maskVIS)" --o
r.mask -r
r.mask raster=maskWATER maskcats=0

# i.emissivity
i.emissivity input=l8ndvi output=l8emis --o

# i.vi viname=savi
i.vi viname=savi red=sl84 nir=sl85 output=l8savi --o
r.colors map=l8savi color=ndvi

# Daily Net Radiation (RNETD)
r.mapcalc expression="temp=if(isnull(l810),null(),dem)" --o
r.slope.aspect elevation=temp slope=slopedd aspect=aspectdd --o
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=rnetd insol_time=sunhours day=$doy nprocs=8 --o
r.mapcalc expression="l8rnetd=rnetd/sunhours" --o

# Instantaneous Net Radiation (RNET)
timelocal=$(echo "$time + (24.0 / 360.0) * $lon" | bc -l)
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=l8rnet day=$doy time=$timelocal nprocs=8 --o

# Soil Heat Flux (G0)
r.mapcalc expression="overpasstime=$time" --o
i.eb.soilheatflux albedo=l8alb ndvi=l8ndvi temperature=l810 netradiation=l8rnet localutctime=overpasstime output=l8g0 --o

# Sensible Heat Flux (H)
i.eb.z0m -p input=l8savi output=z0m --o
r.mapcalc expression="T0dem=0.00612*dem+l810" --o
i.eb.hsebal01 -a netradiation=l8rnet soilheatflux=l8g0 aerodynresistance=z0m temperaturemeansealevel=T0dem frictionvelocitystar=0.32407 vapourpressureactual=1.511 output=l8h --o

# Evaporative Fraction & Soil Moisture
i.eb.evapfr -m netradiation=l8rnet soilheatflux=l8g0 sensibleheatflux=l8h evaporativefraction=l8evapfr soilmoisture=l8sm --o

# Actual Evapotranspiration (ETa)
i.eb.eta netradiationdiurnal=l8rnetd evaporativefraction=l8evapfr temperature=l810 output=l8eta --o

# Biomass growth
r.mapcalc expression="l8fpar=1.257*l8ndvi-0.161" --o
r.mapcalc expression="doy=$doy" --o
r.mapcalc expression="tsw=$tsw" --o
r.mapcalc expression="lue=$lue" --o
i.biomass fpar=l8fpar lightuse_efficiency=lue latitude=latitude dayofyear=doy transmissivity_singleway=tsw water_availability=l8evapfr output=l8biom --o

# Export everything we need
for rstF in l8eta l8sm l8biom l8ndvi l8alb l8rnet l8g0 l8rnetd
do
r.null map=$rstF setnull=-100000-0
r.out.gdal -c -m input=$rstF output=$rstF\_$year$mm$dd.tif nodata=-32768 --o
done

</source>

[[Category: Documentation]]
[[Category: Evapotranspiration]]

Image processing/Evapotranspiration

2023-12-02T08:05:14Z

⚠️Ychemin:

[[File:EnergyBalance.png||500px||Alt="Chemin, 2006, PhD Thesis"]]

The '''radiation''' is coming from the planet's star, Earth has around 1800 W/m2 of incoming radiation from the sun (also called irradiance) at exo-atmospheric altitude.

Atmospheric transfer of '''irradiance''' is subject to atmosphere particles interactions, scattering, diffraction, reflection, etc.

When reaching the planetary surface, is it partially reflected in the shortwave by '''Albedo'''. Surface Albedo is the integrated 0.3-3 micro-meters reflectance, it reflects the shortwave energy coming from the sun by 5% on oceans, 15-25% on vegetation, 35-40% on sand/desert/beach, 60-80% on snow/clouds, roughly. This energy fraction is returned to atmosphere for complex interaction with gas particles again before leaving the atmosphere altogether.

As for the shortwave surface balance, the '''energy''' is also received in the longwave, but interacts with the grey-body characteristics of the surface elements. The blackbody to greybody fraction is ruled by emissivity, a hidden component in the thermal spectrum (search for Temperature- Emissivity Separation algorithms). On Earth, emissivity is always above 0.9 (mostly 0.96-0.98). The Stefan-Boltzman equation is dealing with blackbody energy, and multiplying it by Emissivity transforms it to greybody energy emitted.

Together, shortwave and longwave '''energy balance''' provide with the net radiation balance at the surface of the planetary body. This crucial term is the total energy available for thermodynamic fluxes to act on the surface of the planet. On a planet like Earth where there is a triple phase of a given molecule (H20, in gas, liquid, solid), energy available is used to transfer phases from lower energy to higher energy (sublimation, liquefaction, evaporation). Thermal transfers also happen as conduction (soil, rocks) and convection (atmosphere, oceans). The i.eb.* modules and models deal with conduction (i.eb.g0, for thermal conduction in soils), convection (i.eb.h_* especially, for thermal convection in atmosphere) and by residual method estimate the energy left for evaporation processes ({{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} are examples). {{AddonCmd|i.eb.z0m}} is producing an estimation of the surface roughness.

i.evapo.* modules/models are integrated '''models''', from the oldest (Penman-Monteith {{cmd|i.evapo.pm}}, Priestley-Taylor {{cmd|i.evapo.pt}}, Hargreaves {{cmd|i.evapo.mh}}, etc) that are not based on energy balance, but are computing some form of evapotranspiration (ETo: Evapotranspiration for a reference of 20cm well-watered grass, see FAO no56 report of Richard Allen) to newer that area in some form or another linked to thermal processes ({{cmd|i.evapo.senay}}: uses a thermal index), biome thermal processes ({{cmd|i.evapo.zk}}: global model based on typical info from biomes Albedo/soil heat flux) or more thermodynamic processes ({{cmd|i.evapo.potrad}}: potential ET if not water stress, based on astronomical equations only). Most developed thermodynamic models are TSEB (i.evapo.tseb: Two Source Energy Balance, Schmugge, Kustas, etc., 2000), SEBS (i.evapo.sebs: Zhu et al, 2002), SEBAL ({{cmd|i.eb.h_sebal01}}, {{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}}: Bastiaanssen et al, 1998, 2001), and some newer models like RESET and others. The list of models using thermodynamic principles is large now, and any willing to submit a code for any of them is most welcome.

When integrated in time {{cmd|i.evapo.time}}, seasonal evapotranspiration can be computed and therefore permit a full monitoring of vegetation growth over the period. This is also a basis of crop yield estimation (see {{cmd|i.biomass}}.

[[File:I_evapo_time.png||800px||Alt="Temporal integration of ETa with data from a weather station"]]

== See also ==

* Manual pages:
** http://grass.osgeo.org/grass70/manuals/topic_energy_balance.html
** http://grass.osgeo.org/grass70/manuals/topic_evapotranspiration.html
** http://grass.osgeo.org/grass70/manuals/keywords.html#evaporative%20fraction
* [https://svn.osgeo.org/grass/grass-promo/tutorials/grass_landsat_ETa/ Evapotranspiration mapping with Landsat 5TM in GRASS GIS]

== Example script to process Landsat 8 Evapotranspiration ==
<source lang=shell>
#!/bin/bash

# This scripts imports, pre-processes and compute Evapotranspiration for Landsat 8
# Run in a Location compatible with your L8 dataset

# Atmospheric Transmissivity single-way
tsw=0.75
# Light Use Efficiency (please change dependent on your crop/plant)
lue=1.0

# Import files in GRASS
for file in *TIF
do
r.in.gdal -e input=$file output=$(echo $file | sed 's/.TIF//') --o
done
# Set region to newly imported files
g.region -s raster=$(ls *TIF | grep B1.TIF | sed 's/.TIF//')
LC08=$(ls *TIF | grep B1.TIF | sed 's/1.TIF//')

# Set metadata MTL file name
LC08met=$(ls *MTL.txt)

# Import DEM
r.import input=dem.tif output=dem extent=region --o
# Extract date
date=$(i.landsat.toar input=$LC08 output=l8 metfile=$LC08met lsatmet=date --o)
echo $date

# Apply Top of Atmosphere correction
i.landsat.toar input=$LC08 output=l8 metfile=$LC08met --o

# Set NODATA
for file in $(g.list type=raster pattern=LC08*)
do
r.null map=$file setnull=0
done

#Set computing area to B10 active pixels
for n in 1 2 3 4 5 6 7 8 9
do
r.mapcalc expression="temp=if(isnull(l810),null(),l8$n)" --o
g.rename raster=temp,l8$n --o
done

base=l8
# Create config files for 6s
year=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\1/')
mm=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\2/')
dd=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\3/')
doy=$( date -I -d "$year-$mm-$dd" | date +%j)
hh=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\1/')
mn=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\2/')
ss=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\3/')
time=$(echo "$hh + ($mn / 60.0) + ($ss / 3600.0)" | bc -l)
lat=$(echo $(cat *MTL* | grep CORNER_ | grep LAT | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
lon=$(echo $(cat *MTL* | grep CORNER_ | grep LON | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
echo "18 - geometrical conditions=Landsat 8" > 6s_conf.txt
echo "$mm $dd $hh.$mm $lon $lat - month day hh.ddd longitude latitude ("hh.ddd" is in GMT decimal hours)" >> 6s_conf.txt
echo "1 - atmospheric model=midlatitude summer" >> 6s_conf.txt
echo "1 - aerosols model=continental" >> 6s_conf.txt
echo "24 - visibility [km] (aerosol model concentration)" >> 6s_conf.txt
echo "-0.110 - mean target elevation above sea level [km]" >> 6s_conf.txt
echo "-1000 - sensor on board a satellite" >> 6s_conf.txt
#For Band 1:
cat 6s_conf.txt > $base\1_conf.txt
echo "115 - Coastal band of OLI Landsat 8" >> $base\1_conf.txt
#For Band 2:
cat 6s_conf.txt > $base\2_conf.txt
echo "116 - Blue band of OLI Landsat 8" >> $base\2_conf.txt
#For Band 3:
cat 6s_conf.txt > $base\3_conf.txt
echo "117 - Green band of OLI Landsat 8" >> $base\3_conf.txt
#For Band 4:
cat 6s_conf.txt > $base\4_conf.txt
echo "118 - Red band of OLI Landsat 8" >> $base\4_conf.txt
#For Band 5:
cat 6s_conf.txt > $base\5_conf.txt
echo "120 - NIR band of OLI Landsat 8" >> $base\5_conf.txt
#For Band 6:
cat 6s_conf.txt > $base\6_conf.txt
echo "122 - SWIR1 band of OLI Landsat 8" >> $base\6_conf.txt
#For Band 7:
cat 6s_conf.txt > $base\7_conf.txt
echo "123 - SWIR2 band of OLI Landsat 8" >> $base\7_conf.txt
#For Band 8:
cat 6s_conf.txt > $base\8_conf.txt
echo "119 - PAN band of OLI Landsat 8" >> $base\8_conf.txt
#For Band 9:
cat 6s_conf.txt > $base\9_conf.txt
echo "121 - Cirrus band of OLI Landsat 8" >> $base\9_conf.txt

# Compute Atmospheric correction
for file in $base\1 $base\2 $base\3 $base\4 $base\5 $base\6 $base\7 $base\8 $base\9
do
i.atcorr -r $file elevation=dem parameters=$file\_conf.txt output=s$file --o
r.mapcalc expression="temp=if(isnull(l810),null(),s$file)" --o
g.rename raster=temp,s$file --o
r.colors map=$file color=grey
done

# Create cloud Mask
g.rename raster=l83,l7.2
g.rename raster=l84,l7.3
g.rename raster=l85,l7.4
g.rename raster=l86,l7.5
g.rename raster=l810,l7.6
i.landsat.acca -5 -f input=l7. output=temp --o
g.rename raster=l7.2,l83
g.rename raster=l7.3,l84
g.rename raster=l7.4,l85
g.rename raster=l7.5,l86
g.rename raster=l7.6,l810
r.mapcalc expression="temp1=if(isnull(temp),0,1)" --o
r.mapcalc expression="maskVIS=if(isnull(l810),null(),temp1)" --o

# Apply mask
r.mask raster=maskVIS maskcats=0

# Latitude map
r.latlong input=l81 output=latitude --o

# Longitude map
r.latlong -l input=l81 output=longitude --o

# i.albedo
i.albedo -8 -c input=sl81,sl82,sl83,sl84,sl85,sl86,sl87 output=l8alb --o
r.colors -e map=l8alb color=grey

# i.vi viname=ndvi
i.vi viname=ndvi red=sl84 nir=sl85 output=l8ndvi --o
r.colors map=l8ndvi color=ndvi

# Update processing mask with water
r.mapcalc expression="maskWATER=if(l8ndvi<=0,0,maskVIS)" --o
r.mask -r
r.mask raster=maskWATER maskcats=0

# i.emissivity
i.emissivity input=l8ndvi output=l8emis --o

# i.vi viname=savi
i.vi viname=savi red=sl84 nir=sl85 output=l8savi --o
r.colors map=l8savi color=ndvi

# Daily Net Radiation (RNETD)
r.mapcalc expression="temp=if(isnull(l810),null(),dem)" --o
r.slope.aspect elevation=temp slope=slopedd aspect=aspectdd --o
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=rnetd insol_time=sunhours day=$doy nprocs=8 --o
r.mapcalc expression="l8rnetd=rnetd/sunhours" --o

# Instantaneous Net Radiation (RNET)
timelocal=$(echo "$time + (24.0 / 360.0) * $lon" | bc -l)
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=l8rnet day=$doy time=$timelocal nprocs=8 --o

# Soil Heat Flux (G0)
r.mapcalc expression="overpasstime=$time" --o
i.eb.soilheatflux albedo=l8alb ndvi=l8ndvi temperature=l810 netradiation=l8rnet localutctime=overpasstime output=l8g0 --o

# Sensible Heat Flux (H)
i.eb.z0m -p input=l8savi output=z0m --o
r.mapcalc expression="T0dem=0.00612*dem+l810" --o
i.eb.hsebal01 -a netradiation=l8rnet soilheatflux=l8g0 aerodynresistance=z0m temperaturemeansealevel=T0dem frictionvelocitystar=0.32407 vapourpressureactual=1.511 output=l8h --o

# Evaporative Fraction & Soil Moisture
i.eb.evapfr -m netradiation=l8rnet soilheatflux=l8g0 sensibleheatflux=l8h evaporativefraction=l8evapfr soilmoisture=l8sm --o

# Actual Evapotranspiration (ETa)
i.eb.eta netradiationdiurnal=l8rnetd evaporativefraction=l8evapfr temperature=l810 output=l8eta --o

# Biomass growth
r.mapcalc expression="l8fpar=1.257*l8ndvi-0.161" --o
r.mapcalc expression="doy=$doy" --o
r.mapcalc expression="tsw=$tsw" --o
r.mapcalc expression="lue=$lue" --o
i.biomass fpar=l8fpar lightuse_efficiency=lue latitude=latitude dayofyear=doy transmissivity_singleway=tsw water_availability=l8evapfr output=l8biom --o

# Export everything we need
for rstF in l8eta l8sm l8biom l8ndvi l8alb l8rnet l8g0 l8rnetd
do
r.null map=$rstF setnull=-100000-0
r.out.gdal -c -m input=$rstF output=$rstF\_$year$mm$dd.tif nodata=-32768 --o
done

</source>

[[Category: Documentation]]
[[Category: Evapotranspiration]]

Image processing/Evapotranspiration

2023-12-02T08:04:10Z

⚠️Ychemin:

[[File:EnergyBalance.png||500px||Alt="Chemin, 2006, PhD Thesis"]]

The '''radiation''' is coming from the planet's star, Earth has around 1800 W/m2 of incoming radiation from the sun (also called irradiance) at exo-atmospheric altitude.

Atmospheric transfer of '''irradiance''' is subject to atmosphere particles interactions, scattering, diffraction, reflection, etc.

When reaching the planetary surface, is it partially reflected in the shortwave by '''Albedo'''. Surface Albedo is the integrated 0.3-3 micro-meters reflectance, it reflects the shortwave energy coming from the sun by 5% on oceans, 15-25% on vegetation, 35-40% on sand/desert/beach, 60-80% on snow/clouds, roughly. This energy fraction is returned to atmosphere for complex interaction with gas particles again before leaving the atmosphere altogether.

As for the shortwave surface balance, the '''energy''' is also received in the longwave, but interacts with the grey-body characteristics of the surface elements. The blackbody to greybody fraction is ruled by emissivity, a hidden component in the thermal spectrum (search for Temperature- Emissivity Separation algorithms). On Earth, emissivity is always above 0.9 (mostly 0.96-0.98). The Stefan-Boltzman equation is dealing with blackbody energy, and multiplying it by Emissivity transforms it to greybody energy emitted.

Together, shortwave and longwave '''energy balance''' provide with the net radiation balance at the surface of the planetary body. This crucial term is the total energy available for thermodynamic fluxes to act on the surface of the planet. On a planet like Earth where there is a triple phase of a given molecule (H20, in gas, liquid, solid), energy available is used to transfer phases from lower energy to higher energy (sublimation, liquefaction, evaporation). Thermal transfers also happen as conduction (soil, rocks) and convection (atmosphere, oceans). The i.eb.* modules and models deal with conduction (i.eb.g0, for thermal conduction in soils), convection (i.eb.h_* especially, for thermal convection in atmosphere) and by residual method estimate the energy left for evaporation processes ({{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} are examples). {{AddonCmd|i.eb.z0m}} is producing an estimation of the surface roughness.

i.evapo.* modules/models are integrated '''models''', from the oldest (Penman-Monteith {{cmd|i.evapo.pm}}, Priestley-Taylor {{cmd|i.evapo.pt}}, Hargreaves {{cmd|i.evapo.mh}}, etc) that are not based on energy balance, but are computing some form of evapotranspiration (ETo: Evapotranspiration for a reference of 20cm well-watered grass, see FAO no56 report of Richard Allen) to newer that area in some form or another linked to thermal processes ({{cmd|i.evapo.senay}}: uses a thermal index), biome thermal processes ({{cmd|i.evapo.zk}}: global model based on typical info from biomes Albedo/soil heat flux) or more thermodynamic processes ({{cmd|i.evapo.potrad}}: potential ET if not water stress, based on astronomical equations only). Most developed thermodynamic models are TSEB (i.evapo.tseb: Two Source Energy Balance, Schmugge, Kustas, etc., 2000), SEBS (i.evapo.sebs: Zhu et al, 2002), SEBAL ({{cmd|i.eb.h_sebal01}}, {{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}}: Bastiaanssen et al, 1998, 2001), and some newer models like RESET and others. The list of models using thermodynamic principles is large now, and any willing to submit a code for any of them is most welcome.

When integrated in time {{cmd||i.evapo.time}}, seasonal evapotranspiration can be computed and therefore permit a full monitoring of vegetation growth over the period. This is also a basis of crop yield estimation (see {{cmd||i.biomass}}.

[[File:I_evapo_time.png||800px||Alt="Temporal integration of ETa with data from a weather station"]]

== See also ==

* Manual pages:
** http://grass.osgeo.org/grass70/manuals/topic_energy_balance.html
** http://grass.osgeo.org/grass70/manuals/topic_evapotranspiration.html
** http://grass.osgeo.org/grass70/manuals/keywords.html#evaporative%20fraction
* [https://svn.osgeo.org/grass/grass-promo/tutorials/grass_landsat_ETa/ Evapotranspiration mapping with Landsat 5TM in GRASS GIS]

== Example script to process Landsat 8 Evapotranspiration ==
<source lang=shell>
#!/bin/bash

# This scripts imports, pre-processes and compute Evapotranspiration for Landsat 8
# Run in a Location compatible with your L8 dataset

# Atmospheric Transmissivity single-way
tsw=0.75
# Light Use Efficiency (please change dependent on your crop/plant)
lue=1.0

# Import files in GRASS
for file in *TIF
do
r.in.gdal -e input=$file output=$(echo $file | sed 's/.TIF//') --o
done
# Set region to newly imported files
g.region -s raster=$(ls *TIF | grep B1.TIF | sed 's/.TIF//')
LC08=$(ls *TIF | grep B1.TIF | sed 's/1.TIF//')

# Set metadata MTL file name
LC08met=$(ls *MTL.txt)

# Import DEM
r.import input=dem.tif output=dem extent=region --o
# Extract date
date=$(i.landsat.toar input=$LC08 output=l8 metfile=$LC08met lsatmet=date --o)
echo $date

# Apply Top of Atmosphere correction
i.landsat.toar input=$LC08 output=l8 metfile=$LC08met --o

# Set NODATA
for file in $(g.list type=raster pattern=LC08*)
do
r.null map=$file setnull=0
done

#Set computing area to B10 active pixels
for n in 1 2 3 4 5 6 7 8 9
do
r.mapcalc expression="temp=if(isnull(l810),null(),l8$n)" --o
g.rename raster=temp,l8$n --o
done

base=l8
# Create config files for 6s
year=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\1/')
mm=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\2/')
dd=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\3/')
doy=$( date -I -d "$year-$mm-$dd" | date +%j)
hh=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\1/')
mn=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\2/')
ss=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\3/')
time=$(echo "$hh + ($mn / 60.0) + ($ss / 3600.0)" | bc -l)
lat=$(echo $(cat *MTL* | grep CORNER_ | grep LAT | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
lon=$(echo $(cat *MTL* | grep CORNER_ | grep LON | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
echo "18 - geometrical conditions=Landsat 8" > 6s_conf.txt
echo "$mm $dd $hh.$mm $lon $lat - month day hh.ddd longitude latitude ("hh.ddd" is in GMT decimal hours)" >> 6s_conf.txt
echo "1 - atmospheric model=midlatitude summer" >> 6s_conf.txt
echo "1 - aerosols model=continental" >> 6s_conf.txt
echo "24 - visibility [km] (aerosol model concentration)" >> 6s_conf.txt
echo "-0.110 - mean target elevation above sea level [km]" >> 6s_conf.txt
echo "-1000 - sensor on board a satellite" >> 6s_conf.txt
#For Band 1:
cat 6s_conf.txt > $base\1_conf.txt
echo "115 - Coastal band of OLI Landsat 8" >> $base\1_conf.txt
#For Band 2:
cat 6s_conf.txt > $base\2_conf.txt
echo "116 - Blue band of OLI Landsat 8" >> $base\2_conf.txt
#For Band 3:
cat 6s_conf.txt > $base\3_conf.txt
echo "117 - Green band of OLI Landsat 8" >> $base\3_conf.txt
#For Band 4:
cat 6s_conf.txt > $base\4_conf.txt
echo "118 - Red band of OLI Landsat 8" >> $base\4_conf.txt
#For Band 5:
cat 6s_conf.txt > $base\5_conf.txt
echo "120 - NIR band of OLI Landsat 8" >> $base\5_conf.txt
#For Band 6:
cat 6s_conf.txt > $base\6_conf.txt
echo "122 - SWIR1 band of OLI Landsat 8" >> $base\6_conf.txt
#For Band 7:
cat 6s_conf.txt > $base\7_conf.txt
echo "123 - SWIR2 band of OLI Landsat 8" >> $base\7_conf.txt
#For Band 8:
cat 6s_conf.txt > $base\8_conf.txt
echo "119 - PAN band of OLI Landsat 8" >> $base\8_conf.txt
#For Band 9:
cat 6s_conf.txt > $base\9_conf.txt
echo "121 - Cirrus band of OLI Landsat 8" >> $base\9_conf.txt

# Compute Atmospheric correction
for file in $base\1 $base\2 $base\3 $base\4 $base\5 $base\6 $base\7 $base\8 $base\9
do
i.atcorr -r $file elevation=dem parameters=$file\_conf.txt output=s$file --o
r.mapcalc expression="temp=if(isnull(l810),null(),s$file)" --o
g.rename raster=temp,s$file --o
r.colors map=$file color=grey
done

# Create cloud Mask
g.rename raster=l83,l7.2
g.rename raster=l84,l7.3
g.rename raster=l85,l7.4
g.rename raster=l86,l7.5
g.rename raster=l810,l7.6
i.landsat.acca -5 -f input=l7. output=temp --o
g.rename raster=l7.2,l83
g.rename raster=l7.3,l84
g.rename raster=l7.4,l85
g.rename raster=l7.5,l86
g.rename raster=l7.6,l810
r.mapcalc expression="temp1=if(isnull(temp),0,1)" --o
r.mapcalc expression="maskVIS=if(isnull(l810),null(),temp1)" --o

# Apply mask
r.mask raster=maskVIS maskcats=0

# Latitude map
r.latlong input=l81 output=latitude --o

# Longitude map
r.latlong -l input=l81 output=longitude --o

# i.albedo
i.albedo -8 -c input=sl81,sl82,sl83,sl84,sl85,sl86,sl87 output=l8alb --o
r.colors -e map=l8alb color=grey

# i.vi viname=ndvi
i.vi viname=ndvi red=sl84 nir=sl85 output=l8ndvi --o
r.colors map=l8ndvi color=ndvi

# Update processing mask with water
r.mapcalc expression="maskWATER=if(l8ndvi<=0,0,maskVIS)" --o
r.mask -r
r.mask raster=maskWATER maskcats=0

# i.emissivity
i.emissivity input=l8ndvi output=l8emis --o

# i.vi viname=savi
i.vi viname=savi red=sl84 nir=sl85 output=l8savi --o
r.colors map=l8savi color=ndvi

# Daily Net Radiation (RNETD)
r.mapcalc expression="temp=if(isnull(l810),null(),dem)" --o
r.slope.aspect elevation=temp slope=slopedd aspect=aspectdd --o
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=rnetd insol_time=sunhours day=$doy nprocs=8 --o
r.mapcalc expression="l8rnetd=rnetd/sunhours" --o

# Instantaneous Net Radiation (RNET)
timelocal=$(echo "$time + (24.0 / 360.0) * $lon" | bc -l)
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=l8rnet day=$doy time=$timelocal nprocs=8 --o

# Soil Heat Flux (G0)
r.mapcalc expression="overpasstime=$time" --o
i.eb.soilheatflux albedo=l8alb ndvi=l8ndvi temperature=l810 netradiation=l8rnet localutctime=overpasstime output=l8g0 --o

# Sensible Heat Flux (H)
i.eb.z0m -p input=l8savi output=z0m --o
r.mapcalc expression="T0dem=0.00612*dem+l810" --o
i.eb.hsebal01 -a netradiation=l8rnet soilheatflux=l8g0 aerodynresistance=z0m temperaturemeansealevel=T0dem frictionvelocitystar=0.32407 vapourpressureactual=1.511 output=l8h --o

# Evaporative Fraction & Soil Moisture
i.eb.evapfr -m netradiation=l8rnet soilheatflux=l8g0 sensibleheatflux=l8h evaporativefraction=l8evapfr soilmoisture=l8sm --o

# Actual Evapotranspiration (ETa)
i.eb.eta netradiationdiurnal=l8rnetd evaporativefraction=l8evapfr temperature=l810 output=l8eta --o

# Biomass growth
r.mapcalc expression="l8fpar=1.257*l8ndvi-0.161" --o
r.mapcalc expression="doy=$doy" --o
r.mapcalc expression="tsw=$tsw" --o
r.mapcalc expression="lue=$lue" --o
i.biomass fpar=l8fpar lightuse_efficiency=lue latitude=latitude dayofyear=doy transmissivity_singleway=tsw water_availability=l8evapfr output=l8biom --o

# Export everything we need
for rstF in l8eta l8sm l8biom l8ndvi l8alb l8rnet l8g0 l8rnetd
do
r.null map=$rstF setnull=-100000-0
r.out.gdal -c -m input=$rstF output=$rstF\_$year$mm$dd.tif nodata=-32768 --o
done

</source>

[[Category: Documentation]]
[[Category: Evapotranspiration]]

Image processing/Evapotranspiration

2023-12-02T08:03:43Z

⚠️Ychemin:

[[File:EnergyBalance.png||800px||Alt="Chemin, 2006, PhD Thesis"]]

The '''radiation''' is coming from the planet's star, Earth has around 1800 W/m2 of incoming radiation from the sun (also called irradiance) at exo-atmospheric altitude.

Atmospheric transfer of '''irradiance''' is subject to atmosphere particles interactions, scattering, diffraction, reflection, etc.

When reaching the planetary surface, is it partially reflected in the shortwave by '''Albedo'''. Surface Albedo is the integrated 0.3-3 micro-meters reflectance, it reflects the shortwave energy coming from the sun by 5% on oceans, 15-25% on vegetation, 35-40% on sand/desert/beach, 60-80% on snow/clouds, roughly. This energy fraction is returned to atmosphere for complex interaction with gas particles again before leaving the atmosphere altogether.

As for the shortwave surface balance, the '''energy''' is also received in the longwave, but interacts with the grey-body characteristics of the surface elements. The blackbody to greybody fraction is ruled by emissivity, a hidden component in the thermal spectrum (search for Temperature- Emissivity Separation algorithms). On Earth, emissivity is always above 0.9 (mostly 0.96-0.98). The Stefan-Boltzman equation is dealing with blackbody energy, and multiplying it by Emissivity transforms it to greybody energy emitted.

Together, shortwave and longwave '''energy balance''' provide with the net radiation balance at the surface of the planetary body. This crucial term is the total energy available for thermodynamic fluxes to act on the surface of the planet. On a planet like Earth where there is a triple phase of a given molecule (H20, in gas, liquid, solid), energy available is used to transfer phases from lower energy to higher energy (sublimation, liquefaction, evaporation). Thermal transfers also happen as conduction (soil, rocks) and convection (atmosphere, oceans). The i.eb.* modules and models deal with conduction (i.eb.g0, for thermal conduction in soils), convection (i.eb.h_* especially, for thermal convection in atmosphere) and by residual method estimate the energy left for evaporation processes ({{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} are examples). {{AddonCmd|i.eb.z0m}} is producing an estimation of the surface roughness.

i.evapo.* modules/models are integrated '''models''', from the oldest (Penman-Monteith {{cmd|i.evapo.pm}}, Priestley-Taylor {{cmd|i.evapo.pt}}, Hargreaves {{cmd|i.evapo.mh}}, etc) that are not based on energy balance, but are computing some form of evapotranspiration (ETo: Evapotranspiration for a reference of 20cm well-watered grass, see FAO no56 report of Richard Allen) to newer that area in some form or another linked to thermal processes ({{cmd|i.evapo.senay}}: uses a thermal index), biome thermal processes ({{cmd|i.evapo.zk}}: global model based on typical info from biomes Albedo/soil heat flux) or more thermodynamic processes ({{cmd|i.evapo.potrad}}: potential ET if not water stress, based on astronomical equations only). Most developed thermodynamic models are TSEB (i.evapo.tseb: Two Source Energy Balance, Schmugge, Kustas, etc., 2000), SEBS (i.evapo.sebs: Zhu et al, 2002), SEBAL ({{cmd|i.eb.h_sebal01}}, {{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}}: Bastiaanssen et al, 1998, 2001), and some newer models like RESET and others. The list of models using thermodynamic principles is large now, and any willing to submit a code for any of them is most welcome.

When integrated in time {{cmd||i.evapo.time}}, seasonal evapotranspiration can be computed and therefore permit a full monitoring of vegetation growth over the period. This is also a basis of crop yield estimation (see {{cmd||i.biomass}}.

[[File:I_evapo_time.png||500px||Alt="Temporal integration of ETa with data from a weather station"]]

== See also ==

* Manual pages:
** http://grass.osgeo.org/grass70/manuals/topic_energy_balance.html
** http://grass.osgeo.org/grass70/manuals/topic_evapotranspiration.html
** http://grass.osgeo.org/grass70/manuals/keywords.html#evaporative%20fraction
* [https://svn.osgeo.org/grass/grass-promo/tutorials/grass_landsat_ETa/ Evapotranspiration mapping with Landsat 5TM in GRASS GIS]

== Example script to process Landsat 8 Evapotranspiration ==
<source lang=shell>
#!/bin/bash

# This scripts imports, pre-processes and compute Evapotranspiration for Landsat 8
# Run in a Location compatible with your L8 dataset

# Atmospheric Transmissivity single-way
tsw=0.75
# Light Use Efficiency (please change dependent on your crop/plant)
lue=1.0

# Import files in GRASS
for file in *TIF
do
r.in.gdal -e input=$file output=$(echo $file | sed 's/.TIF//') --o
done
# Set region to newly imported files
g.region -s raster=$(ls *TIF | grep B1.TIF | sed 's/.TIF//')
LC08=$(ls *TIF | grep B1.TIF | sed 's/1.TIF//')

# Set metadata MTL file name
LC08met=$(ls *MTL.txt)

# Import DEM
r.import input=dem.tif output=dem extent=region --o
# Extract date
date=$(i.landsat.toar input=$LC08 output=l8 metfile=$LC08met lsatmet=date --o)
echo $date

# Apply Top of Atmosphere correction
i.landsat.toar input=$LC08 output=l8 metfile=$LC08met --o

# Set NODATA
for file in $(g.list type=raster pattern=LC08*)
do
r.null map=$file setnull=0
done

#Set computing area to B10 active pixels
for n in 1 2 3 4 5 6 7 8 9
do
r.mapcalc expression="temp=if(isnull(l810),null(),l8$n)" --o
g.rename raster=temp,l8$n --o
done

base=l8
# Create config files for 6s
year=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\1/')
mm=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\2/')
dd=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\3/')
doy=$( date -I -d "$year-$mm-$dd" | date +%j)
hh=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\1/')
mn=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\2/')
ss=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\3/')
time=$(echo "$hh + ($mn / 60.0) + ($ss / 3600.0)" | bc -l)
lat=$(echo $(cat *MTL* | grep CORNER_ | grep LAT | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
lon=$(echo $(cat *MTL* | grep CORNER_ | grep LON | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
echo "18 - geometrical conditions=Landsat 8" > 6s_conf.txt
echo "$mm $dd $hh.$mm $lon $lat - month day hh.ddd longitude latitude ("hh.ddd" is in GMT decimal hours)" >> 6s_conf.txt
echo "1 - atmospheric model=midlatitude summer" >> 6s_conf.txt
echo "1 - aerosols model=continental" >> 6s_conf.txt
echo "24 - visibility [km] (aerosol model concentration)" >> 6s_conf.txt
echo "-0.110 - mean target elevation above sea level [km]" >> 6s_conf.txt
echo "-1000 - sensor on board a satellite" >> 6s_conf.txt
#For Band 1:
cat 6s_conf.txt > $base\1_conf.txt
echo "115 - Coastal band of OLI Landsat 8" >> $base\1_conf.txt
#For Band 2:
cat 6s_conf.txt > $base\2_conf.txt
echo "116 - Blue band of OLI Landsat 8" >> $base\2_conf.txt
#For Band 3:
cat 6s_conf.txt > $base\3_conf.txt
echo "117 - Green band of OLI Landsat 8" >> $base\3_conf.txt
#For Band 4:
cat 6s_conf.txt > $base\4_conf.txt
echo "118 - Red band of OLI Landsat 8" >> $base\4_conf.txt
#For Band 5:
cat 6s_conf.txt > $base\5_conf.txt
echo "120 - NIR band of OLI Landsat 8" >> $base\5_conf.txt
#For Band 6:
cat 6s_conf.txt > $base\6_conf.txt
echo "122 - SWIR1 band of OLI Landsat 8" >> $base\6_conf.txt
#For Band 7:
cat 6s_conf.txt > $base\7_conf.txt
echo "123 - SWIR2 band of OLI Landsat 8" >> $base\7_conf.txt
#For Band 8:
cat 6s_conf.txt > $base\8_conf.txt
echo "119 - PAN band of OLI Landsat 8" >> $base\8_conf.txt
#For Band 9:
cat 6s_conf.txt > $base\9_conf.txt
echo "121 - Cirrus band of OLI Landsat 8" >> $base\9_conf.txt

# Compute Atmospheric correction
for file in $base\1 $base\2 $base\3 $base\4 $base\5 $base\6 $base\7 $base\8 $base\9
do
i.atcorr -r $file elevation=dem parameters=$file\_conf.txt output=s$file --o
r.mapcalc expression="temp=if(isnull(l810),null(),s$file)" --o
g.rename raster=temp,s$file --o
r.colors map=$file color=grey
done

# Create cloud Mask
g.rename raster=l83,l7.2
g.rename raster=l84,l7.3
g.rename raster=l85,l7.4
g.rename raster=l86,l7.5
g.rename raster=l810,l7.6
i.landsat.acca -5 -f input=l7. output=temp --o
g.rename raster=l7.2,l83
g.rename raster=l7.3,l84
g.rename raster=l7.4,l85
g.rename raster=l7.5,l86
g.rename raster=l7.6,l810
r.mapcalc expression="temp1=if(isnull(temp),0,1)" --o
r.mapcalc expression="maskVIS=if(isnull(l810),null(),temp1)" --o

# Apply mask
r.mask raster=maskVIS maskcats=0

# Latitude map
r.latlong input=l81 output=latitude --o

# Longitude map
r.latlong -l input=l81 output=longitude --o

# i.albedo
i.albedo -8 -c input=sl81,sl82,sl83,sl84,sl85,sl86,sl87 output=l8alb --o
r.colors -e map=l8alb color=grey

# i.vi viname=ndvi
i.vi viname=ndvi red=sl84 nir=sl85 output=l8ndvi --o
r.colors map=l8ndvi color=ndvi

# Update processing mask with water
r.mapcalc expression="maskWATER=if(l8ndvi<=0,0,maskVIS)" --o
r.mask -r
r.mask raster=maskWATER maskcats=0

# i.emissivity
i.emissivity input=l8ndvi output=l8emis --o

# i.vi viname=savi
i.vi viname=savi red=sl84 nir=sl85 output=l8savi --o
r.colors map=l8savi color=ndvi

# Daily Net Radiation (RNETD)
r.mapcalc expression="temp=if(isnull(l810),null(),dem)" --o
r.slope.aspect elevation=temp slope=slopedd aspect=aspectdd --o
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=rnetd insol_time=sunhours day=$doy nprocs=8 --o
r.mapcalc expression="l8rnetd=rnetd/sunhours" --o

# Instantaneous Net Radiation (RNET)
timelocal=$(echo "$time + (24.0 / 360.0) * $lon" | bc -l)
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=l8rnet day=$doy time=$timelocal nprocs=8 --o

# Soil Heat Flux (G0)
r.mapcalc expression="overpasstime=$time" --o
i.eb.soilheatflux albedo=l8alb ndvi=l8ndvi temperature=l810 netradiation=l8rnet localutctime=overpasstime output=l8g0 --o

# Sensible Heat Flux (H)
i.eb.z0m -p input=l8savi output=z0m --o
r.mapcalc expression="T0dem=0.00612*dem+l810" --o
i.eb.hsebal01 -a netradiation=l8rnet soilheatflux=l8g0 aerodynresistance=z0m temperaturemeansealevel=T0dem frictionvelocitystar=0.32407 vapourpressureactual=1.511 output=l8h --o

# Evaporative Fraction & Soil Moisture
i.eb.evapfr -m netradiation=l8rnet soilheatflux=l8g0 sensibleheatflux=l8h evaporativefraction=l8evapfr soilmoisture=l8sm --o

# Actual Evapotranspiration (ETa)
i.eb.eta netradiationdiurnal=l8rnetd evaporativefraction=l8evapfr temperature=l810 output=l8eta --o

# Biomass growth
r.mapcalc expression="l8fpar=1.257*l8ndvi-0.161" --o
r.mapcalc expression="doy=$doy" --o
r.mapcalc expression="tsw=$tsw" --o
r.mapcalc expression="lue=$lue" --o
i.biomass fpar=l8fpar lightuse_efficiency=lue latitude=latitude dayofyear=doy transmissivity_singleway=tsw water_availability=l8evapfr output=l8biom --o

# Export everything we need
for rstF in l8eta l8sm l8biom l8ndvi l8alb l8rnet l8g0 l8rnetd
do
r.null map=$rstF setnull=-100000-0
r.out.gdal -c -m input=$rstF output=$rstF\_$year$mm$dd.tif nodata=-32768 --o
done

</source>

[[Category: Documentation]]
[[Category: Evapotranspiration]]

File:I evapo time.png

2023-12-02T07:57:27Z

⚠️Ychemin:

Image processing/Evapotranspiration

2023-12-02T07:36:08Z

⚠️Ychemin:

[[File:EnergyBalance.png||800px]]

The '''radiation''' is coming from the planet's star, Earth has around 1800 W/m2 of incoming radiation from the sun (also called irradiance) at exo-atmospheric altitude.

Atmospheric transfer of '''irradiance''' is subject to atmosphere particles interactions, scattering, diffraction, reflection, etc.

When reaching the planetary surface, is it partially reflected in the shortwave by '''Albedo'''. Surface Albedo is the integrated 0.3-3 micro-meters reflectance, it reflects the shortwave energy coming from the sun by 5% on oceans, 15-25% on vegetation, 35-40% on sand/desert/beach, 60-80% on snow/clouds, roughly. This energy fraction is returned to atmosphere for complex interaction with gas particles again before leaving the atmosphere altogether.

As for the shortwave surface balance, the '''energy''' is also received in the longwave, but interacts with the grey-body characteristics of the surface elements. The blackbody to greybody fraction is ruled by emissivity, a hidden component in the thermal spectrum (search for Temperature- Emissivity Separation algorithms). On Earth, emissivity is always above 0.9 (mostly 0.96-0.98). The Stefan-Boltzman equation is dealing with blackbody energy, and multiplying it by Emissivity transforms it to greybody energy emitted.

Together, shortwave and longwave '''energy balance''' provide with the net radiation balance at the surface of the planetary body. This crucial term is the total energy available for thermodynamic fluxes to act on the surface of the planet. On a planet like Earth where there is a triple phase of a given molecule (H20, in gas, liquid, solid), energy available is used to transfer phases from lower energy to higher energy (sublimation, liquefaction, evaporation). Thermal transfers also happen as conduction (soil, rocks) and convection (atmosphere, oceans). The i.eb.* modules and models deal with conduction (i.eb.g0, for thermal conduction in soils), convection (i.eb.h_* especially, for thermal convection in atmosphere) and by residual method estimate the energy left for evaporation processes ({{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} are examples). {{AddonCmd|i.eb.z0m}} is producing an estimation of the surface roughness.

i.evapo.* modules/models are integrated '''models''', from the oldest (Penman-Monteith {{cmd|i.evapo.pm}}, Priestley-Taylor {{cmd|i.evapo.pt}}, Hargreaves {{cmd|i.evapo.mh}}, etc) that are not based on energy balance, but are computing some form of evapotranspiration (ETo: Evapotranspiration for a reference of 20cm well-watered grass, see FAO no56 report of Richard Allen) to newer that area in some form or another linked to thermal processes ({{cmd|i.evapo.senay}}: uses a thermal index), biome thermal processes ({{cmd|i.evapo.zk}}: global model based on typical info from biomes Albedo/soil heat flux) or more thermodynamic processes ({{cmd|i.evapo.potrad}}: potential ET if not water stress, based on astronomical equations only). Most developed thermodynamic models are TSEB (i.evapo.tseb: Two Source Energy Balance, Schmugge, Kustas, etc., 2000), SEBS (i.evapo.sebs: Zhu et al, 2002), SEBAL ({{cmd|i.eb.h_sebal01}}, {{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}}: Bastiaanssen et al, 1998, 2001), and some newer models like RESET and others. The list of models using thermodynamic principles is large now, and any willing to submit a code for any of them is most welcome.

== See also ==

* Manual pages:
** http://grass.osgeo.org/grass70/manuals/topic_energy_balance.html
** http://grass.osgeo.org/grass70/manuals/topic_evapotranspiration.html
** http://grass.osgeo.org/grass70/manuals/keywords.html#evaporative%20fraction
* [https://svn.osgeo.org/grass/grass-promo/tutorials/grass_landsat_ETa/ Evapotranspiration mapping with Landsat 5TM in GRASS GIS]

== Example script to process Landsat 8 Evapotranspiration ==
<source lang=shell>
#!/bin/bash

# This scripts imports, pre-processes and compute Evapotranspiration for Landsat 8
# Run in a Location compatible with your L8 dataset

# Atmospheric Transmissivity single-way
tsw=0.75
# Light Use Efficiency (please change dependent on your crop/plant)
lue=1.0

# Import files in GRASS
for file in *TIF
do
r.in.gdal -e input=$file output=$(echo $file | sed 's/.TIF//') --o
done
# Set region to newly imported files
g.region -s raster=$(ls *TIF | grep B1.TIF | sed 's/.TIF//')
LC08=$(ls *TIF | grep B1.TIF | sed 's/1.TIF//')

# Set metadata MTL file name
LC08met=$(ls *MTL.txt)

# Import DEM
r.import input=dem.tif output=dem extent=region --o
# Extract date
date=$(i.landsat.toar input=$LC08 output=l8 metfile=$LC08met lsatmet=date --o)
echo $date

# Apply Top of Atmosphere correction
i.landsat.toar input=$LC08 output=l8 metfile=$LC08met --o

# Set NODATA
for file in $(g.list type=raster pattern=LC08*)
do
r.null map=$file setnull=0
done

#Set computing area to B10 active pixels
for n in 1 2 3 4 5 6 7 8 9
do
r.mapcalc expression="temp=if(isnull(l810),null(),l8$n)" --o
g.rename raster=temp,l8$n --o
done

base=l8
# Create config files for 6s
year=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\1/')
mm=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\2/')
dd=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\3/')
doy=$( date -I -d "$year-$mm-$dd" | date +%j)
hh=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\1/')
mn=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\2/')
ss=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\3/')
time=$(echo "$hh + ($mn / 60.0) + ($ss / 3600.0)" | bc -l)
lat=$(echo $(cat *MTL* | grep CORNER_ | grep LAT | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
lon=$(echo $(cat *MTL* | grep CORNER_ | grep LON | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
echo "18 - geometrical conditions=Landsat 8" > 6s_conf.txt
echo "$mm $dd $hh.$mm $lon $lat - month day hh.ddd longitude latitude ("hh.ddd" is in GMT decimal hours)" >> 6s_conf.txt
echo "1 - atmospheric model=midlatitude summer" >> 6s_conf.txt
echo "1 - aerosols model=continental" >> 6s_conf.txt
echo "24 - visibility [km] (aerosol model concentration)" >> 6s_conf.txt
echo "-0.110 - mean target elevation above sea level [km]" >> 6s_conf.txt
echo "-1000 - sensor on board a satellite" >> 6s_conf.txt
#For Band 1:
cat 6s_conf.txt > $base\1_conf.txt
echo "115 - Coastal band of OLI Landsat 8" >> $base\1_conf.txt
#For Band 2:
cat 6s_conf.txt > $base\2_conf.txt
echo "116 - Blue band of OLI Landsat 8" >> $base\2_conf.txt
#For Band 3:
cat 6s_conf.txt > $base\3_conf.txt
echo "117 - Green band of OLI Landsat 8" >> $base\3_conf.txt
#For Band 4:
cat 6s_conf.txt > $base\4_conf.txt
echo "118 - Red band of OLI Landsat 8" >> $base\4_conf.txt
#For Band 5:
cat 6s_conf.txt > $base\5_conf.txt
echo "120 - NIR band of OLI Landsat 8" >> $base\5_conf.txt
#For Band 6:
cat 6s_conf.txt > $base\6_conf.txt
echo "122 - SWIR1 band of OLI Landsat 8" >> $base\6_conf.txt
#For Band 7:
cat 6s_conf.txt > $base\7_conf.txt
echo "123 - SWIR2 band of OLI Landsat 8" >> $base\7_conf.txt
#For Band 8:
cat 6s_conf.txt > $base\8_conf.txt
echo "119 - PAN band of OLI Landsat 8" >> $base\8_conf.txt
#For Band 9:
cat 6s_conf.txt > $base\9_conf.txt
echo "121 - Cirrus band of OLI Landsat 8" >> $base\9_conf.txt

# Compute Atmospheric correction
for file in $base\1 $base\2 $base\3 $base\4 $base\5 $base\6 $base\7 $base\8 $base\9
do
i.atcorr -r $file elevation=dem parameters=$file\_conf.txt output=s$file --o
r.mapcalc expression="temp=if(isnull(l810),null(),s$file)" --o
g.rename raster=temp,s$file --o
r.colors map=$file color=grey
done

# Create cloud Mask
g.rename raster=l83,l7.2
g.rename raster=l84,l7.3
g.rename raster=l85,l7.4
g.rename raster=l86,l7.5
g.rename raster=l810,l7.6
i.landsat.acca -5 -f input=l7. output=temp --o
g.rename raster=l7.2,l83
g.rename raster=l7.3,l84
g.rename raster=l7.4,l85
g.rename raster=l7.5,l86
g.rename raster=l7.6,l810
r.mapcalc expression="temp1=if(isnull(temp),0,1)" --o
r.mapcalc expression="maskVIS=if(isnull(l810),null(),temp1)" --o

# Apply mask
r.mask raster=maskVIS maskcats=0

# Latitude map
r.latlong input=l81 output=latitude --o

# Longitude map
r.latlong -l input=l81 output=longitude --o

# i.albedo
i.albedo -8 -c input=sl81,sl82,sl83,sl84,sl85,sl86,sl87 output=l8alb --o
r.colors -e map=l8alb color=grey

# i.vi viname=ndvi
i.vi viname=ndvi red=sl84 nir=sl85 output=l8ndvi --o
r.colors map=l8ndvi color=ndvi

# Update processing mask with water
r.mapcalc expression="maskWATER=if(l8ndvi<=0,0,maskVIS)" --o
r.mask -r
r.mask raster=maskWATER maskcats=0

# i.emissivity
i.emissivity input=l8ndvi output=l8emis --o

# i.vi viname=savi
i.vi viname=savi red=sl84 nir=sl85 output=l8savi --o
r.colors map=l8savi color=ndvi

# Daily Net Radiation (RNETD)
r.mapcalc expression="temp=if(isnull(l810),null(),dem)" --o
r.slope.aspect elevation=temp slope=slopedd aspect=aspectdd --o
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=rnetd insol_time=sunhours day=$doy nprocs=8 --o
r.mapcalc expression="l8rnetd=rnetd/sunhours" --o

# Instantaneous Net Radiation (RNET)
timelocal=$(echo "$time + (24.0 / 360.0) * $lon" | bc -l)
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=l8rnet day=$doy time=$timelocal nprocs=8 --o

# Soil Heat Flux (G0)
r.mapcalc expression="overpasstime=$time" --o
i.eb.soilheatflux albedo=l8alb ndvi=l8ndvi temperature=l810 netradiation=l8rnet localutctime=overpasstime output=l8g0 --o

# Sensible Heat Flux (H)
i.eb.z0m -p input=l8savi output=z0m --o
r.mapcalc expression="T0dem=0.00612*dem+l810" --o
i.eb.hsebal01 -a netradiation=l8rnet soilheatflux=l8g0 aerodynresistance=z0m temperaturemeansealevel=T0dem frictionvelocitystar=0.32407 vapourpressureactual=1.511 output=l8h --o

# Evaporative Fraction & Soil Moisture
i.eb.evapfr -m netradiation=l8rnet soilheatflux=l8g0 sensibleheatflux=l8h evaporativefraction=l8evapfr soilmoisture=l8sm --o

# Actual Evapotranspiration (ETa)
i.eb.eta netradiationdiurnal=l8rnetd evaporativefraction=l8evapfr temperature=l810 output=l8eta --o

# Biomass growth
r.mapcalc expression="l8fpar=1.257*l8ndvi-0.161" --o
r.mapcalc expression="doy=$doy" --o
r.mapcalc expression="tsw=$tsw" --o
r.mapcalc expression="lue=$lue" --o
i.biomass fpar=l8fpar lightuse_efficiency=lue latitude=latitude dayofyear=doy transmissivity_singleway=tsw water_availability=l8evapfr output=l8biom --o

# Export everything we need
for rstF in l8eta l8sm l8biom l8ndvi l8alb l8rnet l8g0 l8rnetd
do
r.null map=$rstF setnull=-100000-0
r.out.gdal -c -m input=$rstF output=$rstF\_$year$mm$dd.tif nodata=-32768 --o
done

</source>

[[Category: Documentation]]
[[Category: Evapotranspiration]]

Image processing/Evapotranspiration

2023-12-02T07:33:53Z

⚠️Ychemin:

[[File:EnergyBalance.png]]

The '''radiation''' is coming from the planet's star, Earth has around 1800 W/m2 of incoming radiation from the sun (also called irradiance) at exo-atmospheric altitude.

Atmospheric transfer of '''irradiance''' is subject to atmosphere particles interactions, scattering, diffraction, reflection, etc.

When reaching the planetary surface, is it partially reflected in the shortwave by '''Albedo'''. Surface Albedo is the integrated 0.3-3 micro-meters reflectance, it reflects the shortwave energy coming from the sun by 5% on oceans, 15-25% on vegetation, 35-40% on sand/desert/beach, 60-80% on snow/clouds, roughly. This energy fraction is returned to atmosphere for complex interaction with gas particles again before leaving the atmosphere altogether.

As for the shortwave surface balance, the '''energy''' is also received in the longwave, but interacts with the grey-body characteristics of the surface elements. The blackbody to greybody fraction is ruled by emissivity, a hidden component in the thermal spectrum (search for Temperature- Emissivity Separation algorithms). On Earth, emissivity is always above 0.9 (mostly 0.96-0.98). The Stefan-Boltzman equation is dealing with blackbody energy, and multiplying it by Emissivity transforms it to greybody energy emitted.

Together, shortwave and longwave '''energy balance''' provide with the net radiation balance at the surface of the planetary body. This crucial term is the total energy available for thermodynamic fluxes to act on the surface of the planet. On a planet like Earth where there is a triple phase of a given molecule (H20, in gas, liquid, solid), energy available is used to transfer phases from lower energy to higher energy (sublimation, liquefaction, evaporation). Thermal transfers also happen as conduction (soil, rocks) and convection (atmosphere, oceans). The i.eb.* modules and models deal with conduction (i.eb.g0, for thermal conduction in soils), convection (i.eb.h_* especially, for thermal convection in atmosphere) and by residual method estimate the energy left for evaporation processes ({{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} are examples). {{AddonCmd|i.eb.z0m}} is producing an estimation of the surface roughness.

i.evapo.* modules/models are integrated '''models''', from the oldest (Penman-Monteith {{cmd|i.evapo.pm}}, Priestley-Taylor {{cmd|i.evapo.pt}}, Hargreaves {{cmd|i.evapo.mh}}, etc) that are not based on energy balance, but are computing some form of evapotranspiration (ETo: Evapotranspiration for a reference of 20cm well-watered grass, see FAO no56 report of Richard Allen) to newer that area in some form or another linked to thermal processes ({{cmd|i.evapo.senay}}: uses a thermal index), biome thermal processes ({{cmd|i.evapo.zk}}: global model based on typical info from biomes Albedo/soil heat flux) or more thermodynamic processes ({{cmd|i.evapo.potrad}}: potential ET if not water stress, based on astronomical equations only). Most developed thermodynamic models are TSEB (i.evapo.tseb: Two Source Energy Balance, Schmugge, Kustas, etc., 2000), SEBS (i.evapo.sebs: Zhu et al, 2002), SEBAL ({{cmd|i.eb.h_sebal01}}, {{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}}: Bastiaanssen et al, 1998, 2001), and some newer models like RESET and others. The list of models using thermodynamic principles is large now, and any willing to submit a code for any of them is most welcome.

== See also ==

* Manual pages:
** http://grass.osgeo.org/grass70/manuals/topic_energy_balance.html
** http://grass.osgeo.org/grass70/manuals/topic_evapotranspiration.html
** http://grass.osgeo.org/grass70/manuals/keywords.html#evaporative%20fraction
* [https://svn.osgeo.org/grass/grass-promo/tutorials/grass_landsat_ETa/ Evapotranspiration mapping with Landsat 5TM in GRASS GIS]

== Example script to process Landsat 8 Evapotranspiration ==
<source lang=shell>
#!/bin/bash

# This scripts imports, pre-processes and compute Evapotranspiration for Landsat 8
# Run in a Location compatible with your L8 dataset

# Atmospheric Transmissivity single-way
tsw=0.75
# Light Use Efficiency (please change dependent on your crop/plant)
lue=1.0

# Import files in GRASS
for file in *TIF
do
r.in.gdal -e input=$file output=$(echo $file | sed 's/.TIF//') --o
done
# Set region to newly imported files
g.region -s raster=$(ls *TIF | grep B1.TIF | sed 's/.TIF//')
LC08=$(ls *TIF | grep B1.TIF | sed 's/1.TIF//')

# Set metadata MTL file name
LC08met=$(ls *MTL.txt)

# Import DEM
r.import input=dem.tif output=dem extent=region --o
# Extract date
date=$(i.landsat.toar input=$LC08 output=l8 metfile=$LC08met lsatmet=date --o)
echo $date

# Apply Top of Atmosphere correction
i.landsat.toar input=$LC08 output=l8 metfile=$LC08met --o

# Set NODATA
for file in $(g.list type=raster pattern=LC08*)
do
r.null map=$file setnull=0
done

#Set computing area to B10 active pixels
for n in 1 2 3 4 5 6 7 8 9
do
r.mapcalc expression="temp=if(isnull(l810),null(),l8$n)" --o
g.rename raster=temp,l8$n --o
done

base=l8
# Create config files for 6s
year=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\1/')
mm=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\2/')
dd=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\3/')
doy=$( date -I -d "$year-$mm-$dd" | date +%j)
hh=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\1/')
mn=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\2/')
ss=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\3/')
time=$(echo "$hh + ($mn / 60.0) + ($ss / 3600.0)" | bc -l)
lat=$(echo $(cat *MTL* | grep CORNER_ | grep LAT | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
lon=$(echo $(cat *MTL* | grep CORNER_ | grep LON | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
echo "18 - geometrical conditions=Landsat 8" > 6s_conf.txt
echo "$mm $dd $hh.$mm $lon $lat - month day hh.ddd longitude latitude ("hh.ddd" is in GMT decimal hours)" >> 6s_conf.txt
echo "1 - atmospheric model=midlatitude summer" >> 6s_conf.txt
echo "1 - aerosols model=continental" >> 6s_conf.txt
echo "24 - visibility [km] (aerosol model concentration)" >> 6s_conf.txt
echo "-0.110 - mean target elevation above sea level [km]" >> 6s_conf.txt
echo "-1000 - sensor on board a satellite" >> 6s_conf.txt
#For Band 1:
cat 6s_conf.txt > $base\1_conf.txt
echo "115 - Coastal band of OLI Landsat 8" >> $base\1_conf.txt
#For Band 2:
cat 6s_conf.txt > $base\2_conf.txt
echo "116 - Blue band of OLI Landsat 8" >> $base\2_conf.txt
#For Band 3:
cat 6s_conf.txt > $base\3_conf.txt
echo "117 - Green band of OLI Landsat 8" >> $base\3_conf.txt
#For Band 4:
cat 6s_conf.txt > $base\4_conf.txt
echo "118 - Red band of OLI Landsat 8" >> $base\4_conf.txt
#For Band 5:
cat 6s_conf.txt > $base\5_conf.txt
echo "120 - NIR band of OLI Landsat 8" >> $base\5_conf.txt
#For Band 6:
cat 6s_conf.txt > $base\6_conf.txt
echo "122 - SWIR1 band of OLI Landsat 8" >> $base\6_conf.txt
#For Band 7:
cat 6s_conf.txt > $base\7_conf.txt
echo "123 - SWIR2 band of OLI Landsat 8" >> $base\7_conf.txt
#For Band 8:
cat 6s_conf.txt > $base\8_conf.txt
echo "119 - PAN band of OLI Landsat 8" >> $base\8_conf.txt
#For Band 9:
cat 6s_conf.txt > $base\9_conf.txt
echo "121 - Cirrus band of OLI Landsat 8" >> $base\9_conf.txt

# Compute Atmospheric correction
for file in $base\1 $base\2 $base\3 $base\4 $base\5 $base\6 $base\7 $base\8 $base\9
do
i.atcorr -r $file elevation=dem parameters=$file\_conf.txt output=s$file --o
r.mapcalc expression="temp=if(isnull(l810),null(),s$file)" --o
g.rename raster=temp,s$file --o
r.colors map=$file color=grey
done

# Create cloud Mask
g.rename raster=l83,l7.2
g.rename raster=l84,l7.3
g.rename raster=l85,l7.4
g.rename raster=l86,l7.5
g.rename raster=l810,l7.6
i.landsat.acca -5 -f input=l7. output=temp --o
g.rename raster=l7.2,l83
g.rename raster=l7.3,l84
g.rename raster=l7.4,l85
g.rename raster=l7.5,l86
g.rename raster=l7.6,l810
r.mapcalc expression="temp1=if(isnull(temp),0,1)" --o
r.mapcalc expression="maskVIS=if(isnull(l810),null(),temp1)" --o

# Apply mask
r.mask raster=maskVIS maskcats=0

# Latitude map
r.latlong input=l81 output=latitude --o

# Longitude map
r.latlong -l input=l81 output=longitude --o

# i.albedo
i.albedo -8 -c input=sl81,sl82,sl83,sl84,sl85,sl86,sl87 output=l8alb --o
r.colors -e map=l8alb color=grey

# i.vi viname=ndvi
i.vi viname=ndvi red=sl84 nir=sl85 output=l8ndvi --o
r.colors map=l8ndvi color=ndvi

# Update processing mask with water
r.mapcalc expression="maskWATER=if(l8ndvi<=0,0,maskVIS)" --o
r.mask -r
r.mask raster=maskWATER maskcats=0

# i.emissivity
i.emissivity input=l8ndvi output=l8emis --o

# i.vi viname=savi
i.vi viname=savi red=sl84 nir=sl85 output=l8savi --o
r.colors map=l8savi color=ndvi

# Daily Net Radiation (RNETD)
r.mapcalc expression="temp=if(isnull(l810),null(),dem)" --o
r.slope.aspect elevation=temp slope=slopedd aspect=aspectdd --o
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=rnetd insol_time=sunhours day=$doy nprocs=8 --o
r.mapcalc expression="l8rnetd=rnetd/sunhours" --o

# Instantaneous Net Radiation (RNET)
timelocal=$(echo "$time + (24.0 / 360.0) * $lon" | bc -l)
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=l8rnet day=$doy time=$timelocal nprocs=8 --o

# Soil Heat Flux (G0)
r.mapcalc expression="overpasstime=$time" --o
i.eb.soilheatflux albedo=l8alb ndvi=l8ndvi temperature=l810 netradiation=l8rnet localutctime=overpasstime output=l8g0 --o

# Sensible Heat Flux (H)
i.eb.z0m -p input=l8savi output=z0m --o
r.mapcalc expression="T0dem=0.00612*dem+l810" --o
i.eb.hsebal01 -a netradiation=l8rnet soilheatflux=l8g0 aerodynresistance=z0m temperaturemeansealevel=T0dem frictionvelocitystar=0.32407 vapourpressureactual=1.511 output=l8h --o

# Evaporative Fraction & Soil Moisture
i.eb.evapfr -m netradiation=l8rnet soilheatflux=l8g0 sensibleheatflux=l8h evaporativefraction=l8evapfr soilmoisture=l8sm --o

# Actual Evapotranspiration (ETa)
i.eb.eta netradiationdiurnal=l8rnetd evaporativefraction=l8evapfr temperature=l810 output=l8eta --o

# Biomass growth
r.mapcalc expression="l8fpar=1.257*l8ndvi-0.161" --o
r.mapcalc expression="doy=$doy" --o
r.mapcalc expression="tsw=$tsw" --o
r.mapcalc expression="lue=$lue" --o
i.biomass fpar=l8fpar lightuse_efficiency=lue latitude=latitude dayofyear=doy transmissivity_singleway=tsw water_availability=l8evapfr output=l8biom --o

# Export everything we need
for rstF in l8eta l8sm l8biom l8ndvi l8alb l8rnet l8g0 l8rnetd
do
r.null map=$rstF setnull=-100000-0
r.out.gdal -c -m input=$rstF output=$rstF\_$year$mm$dd.tif nodata=-32768 --o
done

</source>

[[Category: Documentation]]
[[Category: Evapotranspiration]]

File:EnergyBalance.png

2023-12-02T07:33:23Z

⚠️Ychemin:

Image processing/Evapotranspiration

2023-12-02T07:20:45Z

⚠️Ychemin: Edit Evapotranspiration script for Landsat 8

The '''radiation''' is coming from the planet's star, Earth has around 1800 W/m2 of incoming radiation from the sun (also called irradiance) at exo-atmospheric altitude.

Atmospheric transfer of '''irradiance''' is subject to atmosphere particles interactions, scattering, diffraction, reflection, etc.

When reaching the planetary surface, is it partially reflected in the shortwave by '''Albedo'''. Surface Albedo is the integrated 0.3-3 micro-meters reflectance, it reflects the shortwave energy coming from the sun by 5% on oceans, 15-25% on vegetation, 35-40% on sand/desert/beach, 60-80% on snow/clouds, roughly. This energy fraction is returned to atmosphere for complex interaction with gas particles again before leaving the atmosphere altogether.

As for the shortwave surface balance, the '''energy''' is also received in the longwave, but interacts with the grey-body characteristics of the surface elements. The blackbody to greybody fraction is ruled by emissivity, a hidden component in the thermal spectrum (search for Temperature- Emissivity Separation algorithms). On Earth, emissivity is always above 0.9 (mostly 0.96-0.98). The Stefan-Boltzman equation is dealing with blackbody energy, and multiplying it by Emissivity transforms it to greybody energy emitted.

Together, shortwave and longwave '''energy balance''' provide with the net radiation balance at the surface of the planetary body. This crucial term is the total energy available for thermodynamic fluxes to act on the surface of the planet. On a planet like Earth where there is a triple phase of a given molecule (H20, in gas, liquid, solid), energy available is used to transfer phases from lower energy to higher energy (sublimation, liquefaction, evaporation). Thermal transfers also happen as conduction (soil, rocks) and convection (atmosphere, oceans). The i.eb.* modules and models deal with conduction (i.eb.g0, for thermal conduction in soils), convection (i.eb.h_* especially, for thermal convection in atmosphere) and by residual method estimate the energy left for evaporation processes ({{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}} are examples). {{AddonCmd|i.eb.z0m}} is producing an estimation of the surface roughness.

i.evapo.* modules/models are integrated '''models''', from the oldest (Penman-Monteith {{cmd|i.evapo.pm}}, Priestley-Taylor {{cmd|i.evapo.pt}}, Hargreaves {{cmd|i.evapo.mh}}, etc) that are not based on energy balance, but are computing some form of evapotranspiration (ETo: Evapotranspiration for a reference of 20cm well-watered grass, see FAO no56 report of Richard Allen) to newer that area in some form or another linked to thermal processes ({{cmd|i.evapo.senay}}: uses a thermal index), biome thermal processes ({{cmd|i.evapo.zk}}: global model based on typical info from biomes Albedo/soil heat flux) or more thermodynamic processes ({{cmd|i.evapo.potrad}}: potential ET if not water stress, based on astronomical equations only). Most developed thermodynamic models are TSEB (i.evapo.tseb: Two Source Energy Balance, Schmugge, Kustas, etc., 2000), SEBS (i.evapo.sebs: Zhu et al, 2002), SEBAL ({{cmd|i.eb.h_sebal01}}, {{cmd|i.eb.evapfr}}, {{cmd|i.eb.eta}}: Bastiaanssen et al, 1998, 2001), and some newer models like RESET and others. The list of models using thermodynamic principles is large now, and any willing to submit a code for any of them is most welcome.

== See also ==

* Manual pages:
** http://grass.osgeo.org/grass70/manuals/topic_energy_balance.html
** http://grass.osgeo.org/grass70/manuals/topic_evapotranspiration.html
** http://grass.osgeo.org/grass70/manuals/keywords.html#evaporative%20fraction
* [https://svn.osgeo.org/grass/grass-promo/tutorials/grass_landsat_ETa/ Evapotranspiration mapping with Landsat 5TM in GRASS GIS]

== Example script to process Landsat 8 Evapotranspiration ==
<source lang=shell>
#!/bin/bash

# This scripts imports, pre-processes and compute Evapotranspiration for Landsat 8
# Run in a Location compatible with your L8 dataset

# Atmospheric Transmissivity single-way
tsw=0.75
# Light Use Efficiency (please change dependent on your crop/plant)
lue=1.0

# Import files in GRASS
for file in *TIF
do
r.in.gdal -e input=$file output=$(echo $file | sed 's/.TIF//') --o
done
# Set region to newly imported files
g.region -s raster=$(ls *TIF | grep B1.TIF | sed 's/.TIF//')
LC08=$(ls *TIF | grep B1.TIF | sed 's/1.TIF//')

# Set metadata MTL file name
LC08met=$(ls *MTL.txt)

# Import DEM
r.import input=dem.tif output=dem extent=region --o
# Extract date
date=$(i.landsat.toar input=$LC08 output=l8 metfile=$LC08met lsatmet=date --o)
echo $date

# Apply Top of Atmosphere correction
i.landsat.toar input=$LC08 output=l8 metfile=$LC08met --o

# Set NODATA
for file in $(g.list type=raster pattern=LC08*)
do
r.null map=$file setnull=0
done

#Set computing area to B10 active pixels
for n in 1 2 3 4 5 6 7 8 9
do
r.mapcalc expression="temp=if(isnull(l810),null(),l8$n)" --o
g.rename raster=temp,l8$n --o
done

base=l8
# Create config files for 6s
year=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\1/')
mm=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\2/')
dd=$(cat *MTL* | grep DATE_ACQUIRED | grep -oP '(=)[^;]+' | sed 's/=\ $.*$/\1/' | sed 's/$.*$-$.*$-$.*$/\3/')
doy=$( date -I -d "$year-$mm-$dd" | date +%j)
hh=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\1/')
mn=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\2/')
ss=$(cat *MTL* | grep TIME | grep -oP '(=)[^;]+' | sed 's/=\ \"$.*$:$.*$:$.*$\.$.*$/\3/')
time=$(echo "$hh + ($mn / 60.0) + ($ss / 3600.0)" | bc -l)
lat=$(echo $(cat *MTL* | grep CORNER_ | grep LAT | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
lon=$(echo $(cat *MTL* | grep CORNER_ | grep LON | grep -oP '(=)[^;]+' | sed 's/=//' | awk '{s+=$1} END {print s}') "/4.0" | bc -l)
echo "18 - geometrical conditions=Landsat 8" > 6s_conf.txt
echo "$mm $dd $hh.$mm $lon $lat - month day hh.ddd longitude latitude ("hh.ddd" is in GMT decimal hours)" >> 6s_conf.txt
echo "1 - atmospheric model=midlatitude summer" >> 6s_conf.txt
echo "1 - aerosols model=continental" >> 6s_conf.txt
echo "24 - visibility [km] (aerosol model concentration)" >> 6s_conf.txt
echo "-0.110 - mean target elevation above sea level [km]" >> 6s_conf.txt
echo "-1000 - sensor on board a satellite" >> 6s_conf.txt
#For Band 1:
cat 6s_conf.txt > $base\1_conf.txt
echo "115 - Coastal band of OLI Landsat 8" >> $base\1_conf.txt
#For Band 2:
cat 6s_conf.txt > $base\2_conf.txt
echo "116 - Blue band of OLI Landsat 8" >> $base\2_conf.txt
#For Band 3:
cat 6s_conf.txt > $base\3_conf.txt
echo "117 - Green band of OLI Landsat 8" >> $base\3_conf.txt
#For Band 4:
cat 6s_conf.txt > $base\4_conf.txt
echo "118 - Red band of OLI Landsat 8" >> $base\4_conf.txt
#For Band 5:
cat 6s_conf.txt > $base\5_conf.txt
echo "120 - NIR band of OLI Landsat 8" >> $base\5_conf.txt
#For Band 6:
cat 6s_conf.txt > $base\6_conf.txt
echo "122 - SWIR1 band of OLI Landsat 8" >> $base\6_conf.txt
#For Band 7:
cat 6s_conf.txt > $base\7_conf.txt
echo "123 - SWIR2 band of OLI Landsat 8" >> $base\7_conf.txt
#For Band 8:
cat 6s_conf.txt > $base\8_conf.txt
echo "119 - PAN band of OLI Landsat 8" >> $base\8_conf.txt
#For Band 9:
cat 6s_conf.txt > $base\9_conf.txt
echo "121 - Cirrus band of OLI Landsat 8" >> $base\9_conf.txt

# Compute Atmospheric correction
for file in $base\1 $base\2 $base\3 $base\4 $base\5 $base\6 $base\7 $base\8 $base\9
do
i.atcorr -r $file elevation=dem parameters=$file\_conf.txt output=s$file --o
r.mapcalc expression="temp=if(isnull(l810),null(),s$file)" --o
g.rename raster=temp,s$file --o
r.colors map=$file color=grey
done

# Create cloud Mask
g.rename raster=l83,l7.2
g.rename raster=l84,l7.3
g.rename raster=l85,l7.4
g.rename raster=l86,l7.5
g.rename raster=l810,l7.6
i.landsat.acca -5 -f input=l7. output=temp --o
g.rename raster=l7.2,l83
g.rename raster=l7.3,l84
g.rename raster=l7.4,l85
g.rename raster=l7.5,l86
g.rename raster=l7.6,l810
r.mapcalc expression="temp1=if(isnull(temp),0,1)" --o
r.mapcalc expression="maskVIS=if(isnull(l810),null(),temp1)" --o

# Apply mask
r.mask raster=maskVIS maskcats=0

# Latitude map
r.latlong input=l81 output=latitude --o

# Longitude map
r.latlong -l input=l81 output=longitude --o

# i.albedo
i.albedo -8 -c input=sl81,sl82,sl83,sl84,sl85,sl86,sl87 output=l8alb --o
r.colors -e map=l8alb color=grey

# i.vi viname=ndvi
i.vi viname=ndvi red=sl84 nir=sl85 output=l8ndvi --o
r.colors map=l8ndvi color=ndvi

# Update processing mask with water
r.mapcalc expression="maskWATER=if(l8ndvi<=0,0,maskVIS)" --o
r.mask -r
r.mask raster=maskWATER maskcats=0

# i.emissivity
i.emissivity input=l8ndvi output=l8emis --o

# i.vi viname=savi
i.vi viname=savi red=sl84 nir=sl85 output=l8savi --o
r.colors map=l8savi color=ndvi

# Daily Net Radiation (RNETD)
r.mapcalc expression="temp=if(isnull(l810),null(),dem)" --o
r.slope.aspect elevation=temp slope=slopedd aspect=aspectdd --o
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=rnetd insol_time=sunhours day=$doy nprocs=8 --o
r.mapcalc expression="l8rnetd=rnetd/sunhours" --o

# Instantaneous Net Radiation (RNET)
timelocal=$(echo "$time + (24.0 / 360.0) * $lon" | bc -l)
r.sun elevation=temp aspect=aspectdd slope=slopedd albedo=l8alb glob_rad=l8rnet day=$doy time=$timelocal nprocs=8 --o

# Soil Heat Flux (G0)
r.mapcalc expression="overpasstime=$time" --o
i.eb.soilheatflux albedo=l8alb ndvi=l8ndvi temperature=l810 netradiation=l8rnet localutctime=overpasstime output=l8g0 --o

# Sensible Heat Flux (H)
i.eb.z0m -p input=l8savi output=z0m --o
r.mapcalc expression="T0dem=0.00612*dem+l810" --o
i.eb.hsebal01 -a netradiation=l8rnet soilheatflux=l8g0 aerodynresistance=z0m temperaturemeansealevel=T0dem frictionvelocitystar=0.32407 vapourpressureactual=1.511 output=l8h --o

# Evaporative Fraction & Soil Moisture
i.eb.evapfr -m netradiation=l8rnet soilheatflux=l8g0 sensibleheatflux=l8h evaporativefraction=l8evapfr soilmoisture=l8sm --o

# Actual Evapotranspiration (ETa)
i.eb.eta netradiationdiurnal=l8rnetd evaporativefraction=l8evapfr temperature=l810 output=l8eta --o

# Biomass growth
r.mapcalc expression="l8fpar=1.257*l8ndvi-0.161" --o
r.mapcalc expression="doy=$doy" --o
r.mapcalc expression="tsw=$tsw" --o
r.mapcalc expression="lue=$lue" --o
i.biomass fpar=l8fpar lightuse_efficiency=lue latitude=latitude dayofyear=doy transmissivity_singleway=tsw water_availability=l8evapfr output=l8biom --o

# Export everything we need
for rstF in l8eta l8sm l8biom l8ndvi l8alb l8rnet l8g0 l8rnetd
do
r.null map=$rstF setnull=-100000-0
r.out.gdal -c -m input=$rstF output=$rstF\_$year$mm$dd.tif nodata=-32768 --o
done

</source>

[[Category: Documentation]]
[[Category: Evapotranspiration]]

Hyperion

2023-12-02T07:01:48Z

⚠️Ychemin: /* Data Download */

== Hyperion Hyperspectral Data ==
[[File:eo1_satellite.jpg]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Hyperion Data Download ==
Level 1 GST is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either [https://earthexplorer.usgs.gov/ EarthExplorer] or [https://glovis.usgs.gov/ USGS Global Visualization Viewer (GloVis)].

{| class="wikitable"
|+ Level 1 GST specifications
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

=== Extracting and preparing data ===
The data is downloaded as a zip file (i.e. ''EO1H1260582015073110T1_1GST.zip'') uncompressing as a directory (i.e. ''EO1H1260582015073110T1_1GST/''). Inside exists another directory (i.e. ''EO1H1260582015073110T1/'').

The files inside are following this type of listing:
* EO1H1260582015073110T1_B001_L1GST.TIF
* EO1H1260582015073110T1_B002_L1GST.TIF
* ...
* EO1H1260582015073110T1_B241_L1GST.TIF
* EO1H1260582015073110T1_B242_L1GST.TIF
* EO1H1260582015073110T1_MTL_L1GST.TXT
* EO1H1260582015073110T1_PF2_01.fgdc
* README.txt

Of which ''EO1H1260582015073110T1_MTL_L1GST.TXT'' is a Landsat type metadata header file (''MTL type'') well known to Landsat users.

=== Data perculiarity ===
It seems that bands 1-8 are filled with NODATA, the same for bands 58 to 76, and again for 225 to 242.

For this reason, the pre-processing script below is avoiding the atmospheric correction of those bands.

This seems to be explained by the presence of two sensors inside the instrument, one dedicated to VNIR hyperspectral sensing and the second one dedicated to SWIR hyperspectral sensing. This is further reinforced by observing the overlapping of band wavelengths across bands 56-57 and 77-78.

== Import in GRASS, DN2Rad & Rad2Ref ==

<source lang="python">
#!/usr/bin/env python
"""
Process all HYPERION radiance .zip into reflectance.

Download all radiance zip files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 1st December, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes HYPERION images
# 1 - unzip *.zip files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import glob
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the HYPERION Directories
rsdatapath = "/home/yann/RSDATA/HYPERION"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "HYPERION"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .zip HYPERION radiance tracks downloaded
HYPERIONDirs = ["EO1H1260582015073110T1_1GST"] # , ""]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of HYPERION
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

# Define a KVP retrieval function
def findkvp(text, key):
for item in text.split("\n"):
if key in item:
return (item.strip())

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for HYPERIONDir in HYPERIONDirs:
# Unzip all of your HYPERION images in all your directories
print("Unzip HYPERION files in ", HYPERIONDir)
p = os.system("unzip "+HYPERIONDir+".zip")
# Go into the new created directory
path = rsdatapath+'/'+HYPERIONDir+'/'+HYPERIONDir.split('_')[0]
os.chdir(path)
# Select the first file in the directory
first = os.listdir(path)[0]
print(first)
# Create a temporary GRASS Location
p = os.system("grass -c "+first+" -e "+grassdata+location)
time.sleep(5)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer (unit is Km)
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=0.1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map (unit is km)
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import HYPERION in GRASS
count = 0
# To remove nodata from DN maps
rnull = Module('r.null')
# Define TIF list
ext=['TIF','tif']
listoftiffiles = [f for f in os.listdir(path) if any(f.endswith(e) for e in ext)]
# Process import
for file in listoftiffiles:
outfile='dn.'+str(count)
ringdal(input=file, output=outfile, quiet=QUIET)
rnull(map=outfile, setnull=0, quiet=QUIET)
count = count + 1
# Read HYPERION image CRS TODO check that "first" has full path attached
src = gdal.Open(first)
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Read MTL file to import scene characteristics
MTLfile = glob.glob('*MTL*', recursive=False)
print("Metadatafile is ",MTLfile)
with open(MTLfile[0], 'r') as file:
data = file.read()

gain_vnir = findkvp(data,"SCALING_FACTOR_VNIR").split("=")[1]
gain_swir = findkvp(data,"SCALING_FACTOR_SWIR").split("=")[1]
print("Gains VNIR %d & SWIR %d" % (int(gain_vnir), int(gain_swir)))
# Extract time mean values
starttimestamp = findkvp(data,"START_TIME").split("=")[1].strip()
endtimestamp = findkvp(data,"END_TIME").split("=")[1].strip()
starttime = starttimestamp.split(" ")[-1]
endtime = endtimestamp.split(" ")[-1]
decstarttime = float(starttime.split(":")[0])+float(starttime.split(":")[1])/60.0+float(starttime.split(":")[2])/3600.0
decendtime = float(endtime.split(":")[0])+float(endtime.split(":")[1])/60.0+float(endtime.split(":")[2])/3600.0
dectime = 0.5 * (decstarttime + decendtime)
print("timestamp ", str(dectime))
print(HYPERIONDir)
acqdate = findkvp(data, "ACQUISITION_DATE").split("=")[1].strip()
date = [acqdate.split("-")[0], acqdate.split("-")[1], acqdate.split("-")[2]]
print(date)
# HYPERION DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for HYPERION
# Sensor height (satellite is -1000)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# Create the handle for r.out.gdal
routgdal = Module('r.out.gdal')

# Process VNIR
# satellite band number (HYPERION VNIR [433-481])
satbandno = 433 # Band 8 of HYPERION is first for atmospheric correction
for idx in range(8,57+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 32
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

# Process SWIR
# satellite band number (HYPERION SWIR [482-628])
satbandno = 482 # Band 77 of HYPERION is first for atmospheric correction
for idx in range(77,224+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 33
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

</source>

Hyperion

2023-12-02T07:00:37Z

⚠️Ychemin: /* Import in GRASS, DN2Rad & Rad2Ref */

== Hyperion Hyperspectral Data ==
[[File:eo1_satellite.jpg]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Data Download ==
Level 1 GST is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either [https://earthexplorer.usgs.gov/ EarthExplorer] or [https://glovis.usgs.gov/ USGS Global Visualization Viewer (GloVis)].

{| class="wikitable"
|+ Level 1 GST specifications
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

=== Extracting and preparing data ===
The data is downloaded as a zip file (i.e. ''EO1H1260582015073110T1_1GST.zip'') uncompressing as a directory (i.e. ''EO1H1260582015073110T1_1GST/''). Inside exists another directory (i.e. ''EO1H1260582015073110T1/'').

The files inside are following this type of listing:
* EO1H1260582015073110T1_B001_L1GST.TIF
* EO1H1260582015073110T1_B002_L1GST.TIF
* ...
* EO1H1260582015073110T1_B241_L1GST.TIF
* EO1H1260582015073110T1_B242_L1GST.TIF
* EO1H1260582015073110T1_MTL_L1GST.TXT
* EO1H1260582015073110T1_PF2_01.fgdc
* README.txt

Of which ''EO1H1260582015073110T1_MTL_L1GST.TXT'' is a Landsat type metadata header file (''MTL type'') well known to Landsat users.

=== Data perculiarity ===
It seems that bands 1-8 are filled with NODATA, the same for bands 58 to 76, and again for 225 to 242.

For this reason, the pre-processing script below is avoiding the atmospheric correction of those bands.

This seems to be explained by the presence of two sensors inside the instrument, one dedicated to VNIR hyperspectral sensing and the second one dedicated to SWIR hyperspectral sensing. This is further reinforced by observing the overlapping of band wavelengths across bands 56-57 and 77-78.

== Import in GRASS, DN2Rad & Rad2Ref ==

<source lang="python">
#!/usr/bin/env python
"""
Process all HYPERION radiance .zip into reflectance.

Download all radiance zip files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 1st December, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes HYPERION images
# 1 - unzip *.zip files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import glob
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the HYPERION Directories
rsdatapath = "/home/yann/RSDATA/HYPERION"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "HYPERION"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .zip HYPERION radiance tracks downloaded
HYPERIONDirs = ["EO1H1260582015073110T1_1GST"] # , ""]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of HYPERION
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

# Define a KVP retrieval function
def findkvp(text, key):
for item in text.split("\n"):
if key in item:
return (item.strip())

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for HYPERIONDir in HYPERIONDirs:
# Unzip all of your HYPERION images in all your directories
print("Unzip HYPERION files in ", HYPERIONDir)
p = os.system("unzip "+HYPERIONDir+".zip")
# Go into the new created directory
path = rsdatapath+'/'+HYPERIONDir+'/'+HYPERIONDir.split('_')[0]
os.chdir(path)
# Select the first file in the directory
first = os.listdir(path)[0]
print(first)
# Create a temporary GRASS Location
p = os.system("grass -c "+first+" -e "+grassdata+location)
time.sleep(5)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer (unit is Km)
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=0.1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map (unit is km)
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import HYPERION in GRASS
count = 0
# To remove nodata from DN maps
rnull = Module('r.null')
# Define TIF list
ext=['TIF','tif']
listoftiffiles = [f for f in os.listdir(path) if any(f.endswith(e) for e in ext)]
# Process import
for file in listoftiffiles:
outfile='dn.'+str(count)
ringdal(input=file, output=outfile, quiet=QUIET)
rnull(map=outfile, setnull=0, quiet=QUIET)
count = count + 1
# Read HYPERION image CRS TODO check that "first" has full path attached
src = gdal.Open(first)
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Read MTL file to import scene characteristics
MTLfile = glob.glob('*MTL*', recursive=False)
print("Metadatafile is ",MTLfile)
with open(MTLfile[0], 'r') as file:
data = file.read()

gain_vnir = findkvp(data,"SCALING_FACTOR_VNIR").split("=")[1]
gain_swir = findkvp(data,"SCALING_FACTOR_SWIR").split("=")[1]
print("Gains VNIR %d & SWIR %d" % (int(gain_vnir), int(gain_swir)))
# Extract time mean values
starttimestamp = findkvp(data,"START_TIME").split("=")[1].strip()
endtimestamp = findkvp(data,"END_TIME").split("=")[1].strip()
starttime = starttimestamp.split(" ")[-1]
endtime = endtimestamp.split(" ")[-1]
decstarttime = float(starttime.split(":")[0])+float(starttime.split(":")[1])/60.0+float(starttime.split(":")[2])/3600.0
decendtime = float(endtime.split(":")[0])+float(endtime.split(":")[1])/60.0+float(endtime.split(":")[2])/3600.0
dectime = 0.5 * (decstarttime + decendtime)
print("timestamp ", str(dectime))
print(HYPERIONDir)
acqdate = findkvp(data, "ACQUISITION_DATE").split("=")[1].strip()
date = [acqdate.split("-")[0], acqdate.split("-")[1], acqdate.split("-")[2]]
print(date)
# HYPERION DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for HYPERION
# Sensor height (satellite is -1000)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# Create the handle for r.out.gdal
routgdal = Module('r.out.gdal')

# Process VNIR
# satellite band number (HYPERION VNIR [433-481])
satbandno = 433 # Band 8 of HYPERION is first for atmospheric correction
for idx in range(8,57+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 32
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

# Process SWIR
# satellite band number (HYPERION SWIR [482-628])
satbandno = 482 # Band 77 of HYPERION is first for atmospheric correction
for idx in range(77,224+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 33
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

</source>

Hyperion

2023-12-02T06:57:51Z

⚠️Ychemin: /* Import in GRASS, DN2Rad & Rad2Ref */

== Hyperion Hyperspectral Data ==
[[File:eo1_satellite.jpg]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Data Download ==
Level 1 GST is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either [https://earthexplorer.usgs.gov/ EarthExplorer] or [https://glovis.usgs.gov/ USGS Global Visualization Viewer (GloVis)].

{| class="wikitable"
|+ Level 1 GST specifications
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

=== Extracting and preparing data ===
The data is downloaded as a zip file (i.e. ''EO1H1260582015073110T1_1GST.zip'') uncompressing as a directory (i.e. ''EO1H1260582015073110T1_1GST/''). Inside exists another directory (i.e. ''EO1H1260582015073110T1/'').

The files inside are following this type of listing:
* EO1H1260582015073110T1_B001_L1GST.TIF
* EO1H1260582015073110T1_B002_L1GST.TIF
* ...
* EO1H1260582015073110T1_B241_L1GST.TIF
* EO1H1260582015073110T1_B242_L1GST.TIF
* EO1H1260582015073110T1_MTL_L1GST.TXT
* EO1H1260582015073110T1_PF2_01.fgdc
* README.txt

Of which ''EO1H1260582015073110T1_MTL_L1GST.TXT'' is a Landsat type metadata header file (''MTL type'') well known to Landsat users.

=== Data perculiarity ===
It seems that bands 1-8 are filled with NODATA, the same for bands 58 to 76, and again for 225 to 242.

For this reason, the pre-processing script below is avoiding the atmospheric correction of those bands.

This seems to be explained by the presence of two sensors inside the instrument, one dedicated to VNIR hyperspectral sensing and the second one dedicated to SWIR hyperspectral sensing. This is further reinforced by observing the overlapping of band wavelengths across bands 56-57 and 77-78.

== Import in GRASS, DN2Rad & Rad2Ref ==

<source lang="python">
#!/usr/bin/env python
"""
Process all HYPERION radiance .zip into reflectance.

Download all radiance zip files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 1st December, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes HYPERION images
# 1 - unzip *.zip files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import glob
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the HYPERION Directories
rsdatapath = "/home/yann/RSDATA/HYPERION"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "HYPERION"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .zip HYPERION radiance tracks downloaded
HYPERIONDirs = ["EO1H1260582015073110T1_1GST"] # , ""]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of HYPERION
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

# Define a KVP retrieval function
def findkvp(text, key):
for item in text.split("\n"):
if key in item:
return (item.strip())

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for HYPERIONDir in HYPERIONDirs:
# Unzip all of your HYPERION images in all your directories
#print("Unzip HYPERION files in ", HYPERIONDir)
#p = os.system("unzip "+HYPERIONDir+".zip")
# Go into the new created directory
path = rsdatapath+'/'+HYPERIONDir+'/'+HYPERIONDir.split('_')[0]
os.chdir(path)
# Select the first file in the directory
first = os.listdir(path)[0]
print(first)
# Create a temporary GRASS Location
p = os.system("grass -c "+first+" -e "+grassdata+location)
time.sleep(5)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer (unit is Km)
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=0.1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map (unit is km)
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import HYPERION in GRASS
count = 0
# To remove nodata from DN maps
rnull = Module('r.null')
# Define TIF list
ext=['TIF','tif']
listoftiffiles = [f for f in os.listdir(path) if any(f.endswith(e) for e in ext)]
# Process import
for file in listoftiffiles:
outfile='dn.'+str(count)
ringdal(input=file, output=outfile, quiet=QUIET)
rnull(map=outfile, setnull=0, quiet=QUIET)
count = count + 1
# Read HYPERION image CRS TODO check that "first" has full path attached
src = gdal.Open(first)
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Read MTL file to import scene characteristics
MTLfile = glob.glob('*MTL*', recursive=False)
print("Metadatafile is ",MTLfile)
with open(MTLfile[0], 'r') as file:
data = file.read()

gain_vnir = findkvp(data,"SCALING_FACTOR_VNIR").split("=")[1]
gain_swir = findkvp(data,"SCALING_FACTOR_SWIR").split("=")[1]
print("Gains VNIR %d & SWIR %d" % (int(gain_vnir), int(gain_swir)))
# Extract time mean values
starttimestamp = findkvp(data,"START_TIME").split("=")[1].strip()
endtimestamp = findkvp(data,"END_TIME").split("=")[1].strip()
starttime = starttimestamp.split(" ")[-1]
endtime = endtimestamp.split(" ")[-1]
decstarttime = float(starttime.split(":")[0])+float(starttime.split(":")[1])/60.0+float(starttime.split(":")[2])/3600.0
decendtime = float(endtime.split(":")[0])+float(endtime.split(":")[1])/60.0+float(endtime.split(":")[2])/3600.0
dectime = 0.5 * (decstarttime + decendtime)
print("timestamp ", str(dectime))
print(HYPERIONDir)
acqdate = findkvp(data, "ACQUISITION_DATE").split("=")[1].strip()
date = [acqdate.split("-")[0], acqdate.split("-")[1], acqdate.split("-")[2]]
print(date)
# HYPERION DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for HYPERION
# Sensor height (satellite is -1000)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# Create the handle for r.out.gdal
routgdal = Module('r.out.gdal')
exit()
# Process VNIR
# satellite band number (HYPERION VNIR [433-481])
satbandno = 433 # Band 8 of HYPERION is first for atmospheric correction
for idx in range(8,57+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 32
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

# Process SWIR
# satellite band number (HYPERION SWIR [482-628])
satbandno = 482 # Band 77 of HYPERION is first for atmospheric correction
for idx in range(77,224+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 33
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

</source>

Hyperion

2023-12-02T06:57:12Z

⚠️Ychemin: /* Data Download */

== Hyperion Hyperspectral Data ==
[[File:eo1_satellite.jpg]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Data Download ==
Level 1 GST is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either [https://earthexplorer.usgs.gov/ EarthExplorer] or [https://glovis.usgs.gov/ USGS Global Visualization Viewer (GloVis)].

{| class="wikitable"
|+ Level 1 GST specifications
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

=== Extracting and preparing data ===
The data is downloaded as a zip file (i.e. ''EO1H1260582015073110T1_1GST.zip'') uncompressing as a directory (i.e. ''EO1H1260582015073110T1_1GST/''). Inside exists another directory (i.e. ''EO1H1260582015073110T1/'').

The files inside are following this type of listing:
* EO1H1260582015073110T1_B001_L1GST.TIF
* EO1H1260582015073110T1_B002_L1GST.TIF
* ...
* EO1H1260582015073110T1_B241_L1GST.TIF
* EO1H1260582015073110T1_B242_L1GST.TIF
* EO1H1260582015073110T1_MTL_L1GST.TXT
* EO1H1260582015073110T1_PF2_01.fgdc
* README.txt

Of which ''EO1H1260582015073110T1_MTL_L1GST.TXT'' is a Landsat type metadata header file (''MTL type'') well known to Landsat users.

=== Data perculiarity ===
It seems that bands 1-8 are filled with NODATA, the same for bands 58 to 76, and again for 225 to 242.

For this reason, the pre-processing script below is avoiding the atmospheric correction of those bands.

This seems to be explained by the presence of two sensors inside the instrument, one dedicated to VNIR hyperspectral sensing and the second one dedicated to SWIR hyperspectral sensing. This is further reinforced by observing the overlapping of band wavelengths across bands 56-57 and 77-78.

== Import in GRASS, DN2Rad & Rad2Ref ==

<source lang="python">
#!/usr/bin/env python
"""
Process all HYPERION radiance .zip into reflectance.

Download all radiance zip files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 1st December, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes HYPERION images
# 1 - untar *.zip files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import glob
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the HYPERION Directories
rsdatapath = "/home/yann/RSDATA/HYPERION"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "HYPERION"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .zip HYPERION radiance tracks downloaded
HYPERIONDirs = ["EO1H1260582015073110T1_1GST"] # , ""]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of HYPERION
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

# Define a KVP retrieval function
def findkvp(text, key):
for item in text.split("\n"):
if key in item:
return (item.strip())

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for HYPERIONDir in HYPERIONDirs:
# Unzip all of your HYPERION images in all your directories
#print("Unzip HYPERION files in ", HYPERIONDir)
#p = os.system("unzip "+HYPERIONDir+".zip")
# Go into the new created directory
path = rsdatapath+'/'+HYPERIONDir+'/'+HYPERIONDir.split('_')[0]
os.chdir(path)
# Select the first file in the directory
first = os.listdir(path)[0]
print(first)
# Create a temporary GRASS Location
p = os.system("grass -c "+first+" -e "+grassdata+location)
time.sleep(5)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer (unit is Km)
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=0.1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map (unit is km)
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import HYPERION in GRASS
count = 0
# To remove nodata from DN maps
rnull = Module('r.null')
# Define TIF list
ext=['TIF','tif']
listoftiffiles = [f for f in os.listdir(path) if any(f.endswith(e) for e in ext)]
# Process import
for file in listoftiffiles:
outfile='dn.'+str(count)
ringdal(input=file, output=outfile, quiet=QUIET)
rnull(map=outfile, setnull=0, quiet=QUIET)
count = count + 1
# Read HYPERION image CRS TODO check that "first" has full path attached
src = gdal.Open(first)
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Read MTL file to import scene characteristics
MTLfile = glob.glob('*MTL*', recursive=False)
print("Metadatafile is ",MTLfile)
with open(MTLfile[0], 'r') as file:
data = file.read()

gain_vnir = findkvp(data,"SCALING_FACTOR_VNIR").split("=")[1]
gain_swir = findkvp(data,"SCALING_FACTOR_SWIR").split("=")[1]
print("Gains VNIR %d & SWIR %d" % (int(gain_vnir), int(gain_swir)))
# Extract time mean values
starttimestamp = findkvp(data,"START_TIME").split("=")[1].strip()
endtimestamp = findkvp(data,"END_TIME").split("=")[1].strip()
starttime = starttimestamp.split(" ")[-1]
endtime = endtimestamp.split(" ")[-1]
decstarttime = float(starttime.split(":")[0])+float(starttime.split(":")[1])/60.0+float(starttime.split(":")[2])/3600.0
decendtime = float(endtime.split(":")[0])+float(endtime.split(":")[1])/60.0+float(endtime.split(":")[2])/3600.0
dectime = 0.5 * (decstarttime + decendtime)
print("timestamp ", str(dectime))
print(HYPERIONDir)
acqdate = findkvp(data, "ACQUISITION_DATE").split("=")[1].strip()
date = [acqdate.split("-")[0], acqdate.split("-")[1], acqdate.split("-")[2]]
print(date)
# HYPERION DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for HYPERION
# Sensor height (satellite is -1000)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# Create the handle for r.out.gdal
routgdal = Module('r.out.gdal')
exit()
# Process VNIR
# satellite band number (HYPERION VNIR [433-481])
satbandno = 433 # Band 8 of HYPERION is first for atmospheric correction
for idx in range(8,57+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 32
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

# Process SWIR
# satellite band number (HYPERION SWIR [482-628])
satbandno = 482 # Band 77 of HYPERION is first for atmospheric correction
for idx in range(77,224+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 33
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

</source>

Hyperion

2023-12-02T06:55:57Z

⚠️Ychemin: /* Extracting and preparing data */

== Hyperion Hyperspectral Data ==
[[File:eo1_satellite.jpg]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Data Download ==
Level 1 GST is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either [https://earthexplorer.usgs.gov/ EarthExplorer] or [https://glovis.usgs.gov/ USGS Global Visualization Viewer (GloVis)].

{| class="wikitable"
|+ Level 1 GST specifications
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

=== Extracting and preparing data ===
The data is downloaded as a zip file (i.e. ''EO1H1260582015073110T1_1GST.zip'') uncompressing as a directory (i.e. ''EO1H1260582015073110T1_1GST/''). Inside exists another directory (i.e. ''EO1H1260582015073110T1/'').

The files inside are following this type of listing:
* EO1H1260582015073110T1_B001_L1GST.TIF
* EO1H1260582015073110T1_B002_L1GST.TIF
* ...
* EO1H1260582015073110T1_B241_L1GST.TIF
* EO1H1260582015073110T1_B242_L1GST.TIF
* EO1H1260582015073110T1_MTL_L1GST.TXT
* EO1H1260582015073110T1_PF2_01.fgdc
* README.txt

Of which ''EO1H1260582015073110T1_MTL_L1GST.TXT'' is a Landsat type metadata header file (''MTL type'') well known to Landsat users.

It seems that bands 1-8 are filled with NODATA, the same for bands 58 to 76, and again for 225 to 242. For this reason, the pre-processing script below is avoiding the atmospheric correction of those bands. This seems to be explained by the presence of two sensors inside the instrument, one dedicated to VNIR hyperspectral sensing and the second one dedicated to SWIR hyperspectral sensing. This is further reinforced by observing the overlapping of band wavelengths across bands 56-57 and 77-78.

== Import in GRASS, DN2Rad & Rad2Ref ==

<source lang="python">
#!/usr/bin/env python
"""
Process all HYPERION radiance .zip into reflectance.

Download all radiance zip files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 1st December, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes HYPERION images
# 1 - untar *.zip files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import glob
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the HYPERION Directories
rsdatapath = "/home/yann/RSDATA/HYPERION"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "HYPERION"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .zip HYPERION radiance tracks downloaded
HYPERIONDirs = ["EO1H1260582015073110T1_1GST"] # , ""]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of HYPERION
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

# Define a KVP retrieval function
def findkvp(text, key):
for item in text.split("\n"):
if key in item:
return (item.strip())

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for HYPERIONDir in HYPERIONDirs:
# Unzip all of your HYPERION images in all your directories
#print("Unzip HYPERION files in ", HYPERIONDir)
#p = os.system("unzip "+HYPERIONDir+".zip")
# Go into the new created directory
path = rsdatapath+'/'+HYPERIONDir+'/'+HYPERIONDir.split('_')[0]
os.chdir(path)
# Select the first file in the directory
first = os.listdir(path)[0]
print(first)
# Create a temporary GRASS Location
p = os.system("grass -c "+first+" -e "+grassdata+location)
time.sleep(5)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer (unit is Km)
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=0.1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map (unit is km)
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import HYPERION in GRASS
count = 0
# To remove nodata from DN maps
rnull = Module('r.null')
# Define TIF list
ext=['TIF','tif']
listoftiffiles = [f for f in os.listdir(path) if any(f.endswith(e) for e in ext)]
# Process import
for file in listoftiffiles:
outfile='dn.'+str(count)
ringdal(input=file, output=outfile, quiet=QUIET)
rnull(map=outfile, setnull=0, quiet=QUIET)
count = count + 1
# Read HYPERION image CRS TODO check that "first" has full path attached
src = gdal.Open(first)
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Read MTL file to import scene characteristics
MTLfile = glob.glob('*MTL*', recursive=False)
print("Metadatafile is ",MTLfile)
with open(MTLfile[0], 'r') as file:
data = file.read()

gain_vnir = findkvp(data,"SCALING_FACTOR_VNIR").split("=")[1]
gain_swir = findkvp(data,"SCALING_FACTOR_SWIR").split("=")[1]
print("Gains VNIR %d & SWIR %d" % (int(gain_vnir), int(gain_swir)))
# Extract time mean values
starttimestamp = findkvp(data,"START_TIME").split("=")[1].strip()
endtimestamp = findkvp(data,"END_TIME").split("=")[1].strip()
starttime = starttimestamp.split(" ")[-1]
endtime = endtimestamp.split(" ")[-1]
decstarttime = float(starttime.split(":")[0])+float(starttime.split(":")[1])/60.0+float(starttime.split(":")[2])/3600.0
decendtime = float(endtime.split(":")[0])+float(endtime.split(":")[1])/60.0+float(endtime.split(":")[2])/3600.0
dectime = 0.5 * (decstarttime + decendtime)
print("timestamp ", str(dectime))
print(HYPERIONDir)
acqdate = findkvp(data, "ACQUISITION_DATE").split("=")[1].strip()
date = [acqdate.split("-")[0], acqdate.split("-")[1], acqdate.split("-")[2]]
print(date)
# HYPERION DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for HYPERION
# Sensor height (satellite is -1000)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# Create the handle for r.out.gdal
routgdal = Module('r.out.gdal')
exit()
# Process VNIR
# satellite band number (HYPERION VNIR [433-481])
satbandno = 433 # Band 8 of HYPERION is first for atmospheric correction
for idx in range(8,57+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 32
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

# Process SWIR
# satellite band number (HYPERION SWIR [482-628])
satbandno = 482 # Band 77 of HYPERION is first for atmospheric correction
for idx in range(77,224+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 33
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

</source>

Hyperion

2023-12-02T06:50:18Z

⚠️Ychemin: /* Data Download */

== Hyperion Hyperspectral Data ==
[[File:eo1_satellite.jpg]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Data Download ==
Level 1 GST is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either [https://earthexplorer.usgs.gov/ EarthExplorer] or [https://glovis.usgs.gov/ USGS Global Visualization Viewer (GloVis)].

{| class="wikitable"
|+ Level 1 GST specifications
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

=== Extracting and preparing data ===
The data is downloaded as a zip file (i.e. ''EO1H1260582015073110T1_1GST.zip'') uncompressing as a directory (i.e. ''EO1H1260582015073110T1_1GST/''). Inside exists another directory (i.e. ''EO1H1260582015073110T1/'').

The files inside are following this type of listing:
* EO1H1260582015073110T1_B001_L1GST.TIF
* EO1H1260582015073110T1_B002_L1GST.TIF
* ...
* EO1H1260582015073110T1_B241_L1GST.TIF
* EO1H1260582015073110T1_B242_L1GST.TIF
* EO1H1260582015073110T1_MTL_L1GST.TXT
* EO1H1260582015073110T1_PF2_01.fgdc
* README.txt

Of which ''EO1H1260582015073110T1_MTL_L1GST.TXT'' is a Landsat type metadata header file (''MTL type'') well known to Landsat users.

== Import in GRASS, DN2Rad & Rad2Ref ==

<source lang="python">
#!/usr/bin/env python
"""
Process all HYPERION radiance .zip into reflectance.

Download all radiance zip files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 1st December, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes HYPERION images
# 1 - untar *.zip files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import glob
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the HYPERION Directories
rsdatapath = "/home/yann/RSDATA/HYPERION"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "HYPERION"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .zip HYPERION radiance tracks downloaded
HYPERIONDirs = ["EO1H1260582015073110T1_1GST"] # , ""]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of HYPERION
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

# Define a KVP retrieval function
def findkvp(text, key):
for item in text.split("\n"):
if key in item:
return (item.strip())

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for HYPERIONDir in HYPERIONDirs:
# Unzip all of your HYPERION images in all your directories
#print("Unzip HYPERION files in ", HYPERIONDir)
#p = os.system("unzip "+HYPERIONDir+".zip")
# Go into the new created directory
path = rsdatapath+'/'+HYPERIONDir+'/'+HYPERIONDir.split('_')[0]
os.chdir(path)
# Select the first file in the directory
first = os.listdir(path)[0]
print(first)
# Create a temporary GRASS Location
p = os.system("grass -c "+first+" -e "+grassdata+location)
time.sleep(5)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer (unit is Km)
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=0.1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map (unit is km)
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import HYPERION in GRASS
count = 0
# To remove nodata from DN maps
rnull = Module('r.null')
# Define TIF list
ext=['TIF','tif']
listoftiffiles = [f for f in os.listdir(path) if any(f.endswith(e) for e in ext)]
# Process import
for file in listoftiffiles:
outfile='dn.'+str(count)
ringdal(input=file, output=outfile, quiet=QUIET)
rnull(map=outfile, setnull=0, quiet=QUIET)
count = count + 1
# Read HYPERION image CRS TODO check that "first" has full path attached
src = gdal.Open(first)
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Read MTL file to import scene characteristics
MTLfile = glob.glob('*MTL*', recursive=False)
print("Metadatafile is ",MTLfile)
with open(MTLfile[0], 'r') as file:
data = file.read()

gain_vnir = findkvp(data,"SCALING_FACTOR_VNIR").split("=")[1]
gain_swir = findkvp(data,"SCALING_FACTOR_SWIR").split("=")[1]
print("Gains VNIR %d & SWIR %d" % (int(gain_vnir), int(gain_swir)))
# Extract time mean values
starttimestamp = findkvp(data,"START_TIME").split("=")[1].strip()
endtimestamp = findkvp(data,"END_TIME").split("=")[1].strip()
starttime = starttimestamp.split(" ")[-1]
endtime = endtimestamp.split(" ")[-1]
decstarttime = float(starttime.split(":")[0])+float(starttime.split(":")[1])/60.0+float(starttime.split(":")[2])/3600.0
decendtime = float(endtime.split(":")[0])+float(endtime.split(":")[1])/60.0+float(endtime.split(":")[2])/3600.0
dectime = 0.5 * (decstarttime + decendtime)
print("timestamp ", str(dectime))
print(HYPERIONDir)
acqdate = findkvp(data, "ACQUISITION_DATE").split("=")[1].strip()
date = [acqdate.split("-")[0], acqdate.split("-")[1], acqdate.split("-")[2]]
print(date)
# HYPERION DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for HYPERION
# Sensor height (satellite is -1000)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# Create the handle for r.out.gdal
routgdal = Module('r.out.gdal')
exit()
# Process VNIR
# satellite band number (HYPERION VNIR [433-481])
satbandno = 433 # Band 8 of HYPERION is first for atmospheric correction
for idx in range(8,57+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 32
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

# Process SWIR
# satellite band number (HYPERION SWIR [482-628])
satbandno = 482 # Band 77 of HYPERION is first for atmospheric correction
for idx in range(77,224+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 33
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

</source>

Hyperion

2023-12-02T06:42:16Z

⚠️Ychemin: Add pyGRASS script

== Hyperion Hyperspectral Data ==
[[File:eo1_satellite.jpg]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Data Download ==
Level 1 GST is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either [https://earthexplorer.usgs.gov/ EarthExplorer] or [https://glovis.usgs.gov/ USGS Global Visualization Viewer (GloVis)].

{| class="wikitable"
|+ Level 1 GST specifications
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

== Import in GRASS, DN2Rad & Rad2Ref ==

<source lang="python">
#!/usr/bin/env python
"""
Process all HYPERION radiance .zip into reflectance.

Download all radiance zip files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 1st December, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes HYPERION images
# 1 - untar *.zip files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import glob
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the HYPERION Directories
rsdatapath = "/home/yann/RSDATA/HYPERION"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "HYPERION"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .zip HYPERION radiance tracks downloaded
HYPERIONDirs = ["EO1H1260582015073110T1_1GST"] # , ""]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of HYPERION
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

# Define a KVP retrieval function
def findkvp(text, key):
for item in text.split("\n"):
if key in item:
return (item.strip())

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for HYPERIONDir in HYPERIONDirs:
# Unzip all of your HYPERION images in all your directories
#print("Unzip HYPERION files in ", HYPERIONDir)
#p = os.system("unzip "+HYPERIONDir+".zip")
# Go into the new created directory
path = rsdatapath+'/'+HYPERIONDir+'/'+HYPERIONDir.split('_')[0]
os.chdir(path)
# Select the first file in the directory
first = os.listdir(path)[0]
print(first)
# Create a temporary GRASS Location
p = os.system("grass -c "+first+" -e "+grassdata+location)
time.sleep(5)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer (unit is Km)
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=0.1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map (unit is km)
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import HYPERION in GRASS
count = 0
# To remove nodata from DN maps
rnull = Module('r.null')
# Define TIF list
ext=['TIF','tif']
listoftiffiles = [f for f in os.listdir(path) if any(f.endswith(e) for e in ext)]
# Process import
for file in listoftiffiles:
outfile='dn.'+str(count)
ringdal(input=file, output=outfile, quiet=QUIET)
rnull(map=outfile, setnull=0, quiet=QUIET)
count = count + 1
# Read HYPERION image CRS TODO check that "first" has full path attached
src = gdal.Open(first)
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Read MTL file to import scene characteristics
MTLfile = glob.glob('*MTL*', recursive=False)
print("Metadatafile is ",MTLfile)
with open(MTLfile[0], 'r') as file:
data = file.read()

gain_vnir = findkvp(data,"SCALING_FACTOR_VNIR").split("=")[1]
gain_swir = findkvp(data,"SCALING_FACTOR_SWIR").split("=")[1]
print("Gains VNIR %d & SWIR %d" % (int(gain_vnir), int(gain_swir)))
# Extract time mean values
starttimestamp = findkvp(data,"START_TIME").split("=")[1].strip()
endtimestamp = findkvp(data,"END_TIME").split("=")[1].strip()
starttime = starttimestamp.split(" ")[-1]
endtime = endtimestamp.split(" ")[-1]
decstarttime = float(starttime.split(":")[0])+float(starttime.split(":")[1])/60.0+float(starttime.split(":")[2])/3600.0
decendtime = float(endtime.split(":")[0])+float(endtime.split(":")[1])/60.0+float(endtime.split(":")[2])/3600.0
dectime = 0.5 * (decstarttime + decendtime)
print("timestamp ", str(dectime))
print(HYPERIONDir)
acqdate = findkvp(data, "ACQUISITION_DATE").split("=")[1].strip()
date = [acqdate.split("-")[0], acqdate.split("-")[1], acqdate.split("-")[2]]
print(date)
# HYPERION DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for HYPERION
# Sensor height (satellite is -1000)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# Create the handle for r.out.gdal
routgdal = Module('r.out.gdal')
exit()
# Process VNIR
# satellite band number (HYPERION VNIR [433-481])
satbandno = 433 # Band 8 of HYPERION is first for atmospheric correction
for idx in range(8,57+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 32
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

# Process SWIR
# satellite band number (HYPERION SWIR [482-628])
satbandno = 482 # Band 77 of HYPERION is first for atmospheric correction
for idx in range(77,224+1):
# Geometrical conditions (HYPERION VNIR = 32, SWIR = 33)
geom = 33
b = "rad." + str(idx)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=HYPERION
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, a satellite)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# i th band of HYPERION (atcorr internal no)
param.append(str(satbandno))
prm = "param_HYPERION.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal(input=b_out, output=HYPERIONDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1

</source>

Hyperion

2023-12-02T06:39:16Z

⚠️Ychemin: /* Hyperion Hyperspectral Satellite */

Hyperion

2023-12-02T06:38:32Z

⚠️Ychemin: /* Data Download */

== Hyperion Hyperspectral Satellite ==
[[File:eo1_satellite.jpg]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Data Download ==
Level 1 GST is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either [https://earthexplorer.usgs.gov/ EarthExplorer] or [https://glovis.usgs.gov/ USGS Global Visualization Viewer (GloVis)].

{| class="wikitable"
|+ Level 1 GST specifications
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

Hyperion

2023-12-02T06:36:57Z

⚠️Ychemin: /* Data Download */

== Hyperion Hyperspectral Satellite ==
[[File:eo1_satellite.jpg]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Data Download ==
Level 1 GST is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either EarthExplorer or USGS Global Visualization Viewer (GloVis).

{| class="wikitable"
|+ Level 1 GST specifications
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

Hyperion

2023-12-02T06:36:35Z

⚠️Ychemin: /* Data Download */

== Hyperion Hyperspectral Satellite ==
[[File:eo1_satellite.jpg]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Data Download ==
Level 1GST is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either EarthExplorer or USGS Global Visualization Viewer (GloVis).

{| class="wikitable"
|+ Level GST specifications
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

Hyperion

2023-12-02T06:35:43Z

⚠️Ychemin: /* Data Download */

== Hyperion Hyperspectral Satellite ==
[[File:eo1_satellite.jpg]]
=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Data Download ==
Level 1Gst is terrain corrected and provided in 16-bit radiance values. The data are available in Geographic Tagged Image-File Format (GeoTIFF) and are distributed via download at no charge through either EarthExplorer or USGS Global Visualization Viewer (GloVis).

{| class="wikitable"
|+ Caption text
|-
| Map projection || UTM (default zone of the scene center coordinates)
|-
| Horizontal Datum || WGS84
|-
| Resampling method || CC (cubic convolution)
|-
| Image orientation || Map (north up)
|-
| Pixel size || 30 meter (10 meter pan band)
|-
| Format || GeoTIFF
|-
| Output media || Download (no charge)
|}

Hyperion

2023-12-02T06:29:55Z

⚠️Ychemin: /* Hyperion Hyperspectral Satellite */

File:Eo1 satellite.jpg

2023-12-02T06:29:21Z

⚠️Ychemin:

Hyperion

2023-12-02T06:26:42Z

⚠️Ychemin: Created page with "== Hyperion Hyperspectral Satellite == === EO-1 platform === The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]). The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM). The original EO-1 Mission was success..."

== Hyperion Hyperspectral Satellite ==

=== EO-1 platform ===
The NASA EO-1 satellite was launched on November 21, 2000 as part of a one-year technology validation/demonstration mission ([https://www.usgs.gov/centers/eros/science/earth-observing-1-eo-1 EO-1 Mission Page]).

The mission was to use the Advanced Land Imager (ALI) instrument on EO-1 to validate and demonstrate technology for the Landsat Data Continuity Mission (LDCM).

The original EO-1 Mission was successfully completed in November 2001. As the end of the Mission approached, the remote sensing research and scientific communities expressed high interest in continued acquisition of image data from EO-1. Based on this user interest and willingness to assist in funding continued operations, an agreement was reached between NASA and the USGS to allow continuation of the EO-1 Program as an Extended Mission.

On February 22nd, 2017, the Earth-Observing One (EO-1) satellite was decommissioned.

=== Sensors ===
* Advanced Land Imager (ALI)
* Hyperion
* Linear Etalon Imaging Spectrometer Array (LEISA) Atmospheric Corrector (LAC)

The EO-1 Extended Mission was chartered to collect and distribute ALI multispectral and Hyperion hyperspectral products in response to Data Acquisition Requests (DARs). Under the Extended Mission provisions, image data acquired by EO-1 were archived and distributed by the USGS Center for Earth Resources Observation and Science (EROS) and placed in the public domain.

== Data Download ==

AVIRIS

2023-12-02T06:13:14Z

⚠️Ychemin: /* AVIRIS Hyperspectral Sensor */

== AVIRIS Hyperspectral Sensor ==
[[File:aviris_text2.png]]
=== Introduction ===
'''AVIRIS''' is an acronym for the Airborne Visible InfraRed Imaging Spectrometer ([https://aviris.jpl.nasa.gov/ AVIRIS Main page]).

Its optical sensor creates calibrated images of the upwelling spectral radiance in 224 contiguous spectral channels/bands with wavelengths from 400 to 2500 nanometers (nm).

=== Airplanes ===
'''AVIRIS''' has been flown on four aircraft platforms:
* NASA's ER-2 jet
* Twin Otter International's turboprop
* Scaled Composites' Proteus
* NASA's WB-57.

Flight details:
* The ER-2 flies at approximately 20km above sea level, at about 730 km/hr.
* The Twin Otter aircraft flies at 4km above ground level at 130km/hr.

=== Data Download ===
Downloading data from '''AVIRIS''' flights is straightforward using the [https://aviris.jpl.nasa.gov/dataportal/ AVIRIS Data Portal].

== Import AVIRIS in GRASS ==
=== Data Downloaded ===

'''AVIRIS''' is downloaded as a .tar.gz file (i.e. ''f180116t01p00r07.tar.gz'') uncompressing as a directory (i.e. ''f180116t01p00r07rdn_e/'') containing with a set files like this:

==== List of files ====
* AVIRIS_OrthoProcessing_Info.txt
* f180116t01p00r07rdn_e_eph
* f180116t01p00r07rdn_e.gain
* f180116t01p00r07rdn_e_lonlat_eph
* ''f180116t01p00r07rdn_e_obs''
* f180116t01p00r07rdn_e_obs.hdr
* ''f180116t01p00r07rdn_e_obs_ort''
* f180116t01p00r07rdn_e_obs_ort.hdr
* ''f180116t01p00r07rdn_e_ort_glt''
* f180116t01p00r07rdn_e_ort_glt.hdr
* ''f180116t01p00r07rdn_e_ort_igm''
* f180116t01p00r07rdn_e_ort_igm.hdr
* f180116t01p00r07rdn_e_ort.plog
* f180116t01p00r07rdn_e.rccf18
* ''f180116t01p00r07rdn_e_sc01_ort_img''
* f180116t01p00r07rdn_e_sc01_ort_img.hdr
* f180116t01p00r07rdn_e.spc

All files twinned with a .hdr file, and in italics in the list above, are ENVI style binary imagery datasets (''BSQ format''). Towards the end of the list, the file matching the pattern ''sc01'' is the largest file, and the one holding the actual hyperspectral data.

==== DN to Radiance Conversion ====

The DN to Radiance correction is done by applying a gain to the stored data in the BSQ file. In this case, the gain file in ''f180116t01p00r07rdn_e.gain''. Extracting the gain values can be done like this in Python:

<source lang="python">
gain = []
with open(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e.gain") as file:
for line in file:
gain.append(line.strip().split()[0])
</source>

After that, the DN to Radiance correction can be done like this:

<source lang="python">
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules import Module

rmapcalc = Module('r.mapcalc')
# AVIRIS DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
</source>

=== Import in GRASS, gain correction (DN2Rad) & atmospheric correction (Rad2Ref) ===
<source lang="python">
#!/usr/bin/env python
"""
Process all AVIRIS radiance tar.gz into reflectance.

Download all radiance tar.gz files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 24th October, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes AVIRIS images
# 1 - untar *.tar.gz files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the AVIRIS Directories
rsdatapath = "/home/yann/RSDATA/AVIRIS"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "AVIRIS"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .tar.gz AVIRIS radiance tracks downloaded
AVIRISDirs = ["f180116t01p00r07"] # , "f100517t01p00r11"]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of AVIRIS
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

def read_ephemeris_file(file_path):
"""
Read the ephemeris _eph file and extract flight altitude.

Parameters
----------
file_path : STRING
the ephemeris *_eph file.

Returns
-------
data : FLOAT
the average height of sensor during flight.
"""
data = []
# Define the format of a single record in the binary file
record_format = 'd' * 6 # Six double-precision floating-point numbers
with open(file_path, 'rb') as f:
while True:
record = f.read(struct.calcsize(record_format))
if not record:
break # Reached end of file
values = struct.unpack(record_format, record)
data.append(values)
return data

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for AVIRISDir in AVIRISDirs:
# Untar all of your AVIRIS images in all your directories
print("Untar AVIRIS files in ", AVIRISDir)
#p = os.system("tar xvf "+AVIRISDir+".tar.gz")
with open(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e.gain") as file:
for line in file:
gain.append(line.strip().split()[0])
# Process a North alignment of the original file TODO add datatype=float
kwargs = {'format': 'GTiff', 'geoloc': True, 'options': 'overwrite'}
# gdal.UseExceptions()
#ds = gdal.Warp("del.tif", AVIRISDir + "rdn_e/" + AVIRISDir + "rdn_e_sc01_ort_img", **kwargs)
#del ds
# Create a temporary GRASS Location
p = os.system("grass -c del.tif -e "+grassdata+location)
time.sleep(20)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM file is not empty
# ringdal(input=AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e_ort_igm", output="dem",
# overwrite=OVR)
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import AVIRIS in GRASS
ringdal(input="del.tif", output="dn", quiet=QUIET)
# Remove nodata from DN maps
rnull = Module('r.null')
for idx in range(224):
rnull(map="dn."+str(idx+1), setnull=[-50, 0], quiet=QUIET)
# Read rotated AVIRIS image
src = gdal.Open("del.tif")
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Import Auxiliary layers, rotate it first
ds = gdal.Warp("del1.tif", AVIRISDir + "rdn_e/" + AVIRISDir +
"rdn_e_obs_ort", **kwargs)
del ds
# Import in GRASS TODO force Float data
ringdal(input="del1.tif", output="obs", flags="e", overwrite=OVR, quiet=QUIET)
# Remove nodata from UTC map
rnull(map="obs.10", setnull=[-9999, 0], quiet=QUIET)
# Extract stats
rinfo = Module('r.info', map="obs.10", flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
dectime = rinfo_out["mean"]
print("timestamp ", str(dectime))
print(AVIRISDir)
date = ["20"+AVIRISDir[1:3], AVIRISDir[3:5], AVIRISDir[5:7]]
print(date)
# AVIRIS DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for AVIRIS
# Geometrical conditions (AVIRIS = 31)
geom = 31
# Sensor height (satellite is -1000)
eph_data = read_ephemeris_file(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e_eph")
# Extract average altitude
add = 0
for j in range(len(eph_data)):
add += eph_data[j][2]
height = add/len(eph_data)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1*height/1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# satellite band number (AVIRIS [209-432])
satbandno = 209 # Band 1 of AVIRIS is first for atmospheric correction
for idx in range(224):
b = "rad." + str(idx+1)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx+1), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=AVIRIS
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, sensor on board a plane)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# puw (water vapor content between the aircraft and the surface)
param.append("-1.0\n")
# po3 (ozone content between the aircraft and the surface)
param.append("-1.0\n")
# taerp (the aerosol optical thickness at 550nm between the aircraft and the surface)
param.append("-1.0\n")
# i th band of AVIRIS (atcorr internal no)
param.append(str(satbandno))
prm = "param_AVIRIS.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal = Module('r.out.gdal')
routgdal(input=b_out, output=AVIRISDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1
</source>

AVIRIS

2023-12-02T06:12:46Z

⚠️Ychemin: /* AVIRIS Hyperspectral Sensor */

== AVIRIS Hyperspectral Sensor ==
[[aviris_text2.png]]
=== Introduction ===
'''AVIRIS''' is an acronym for the Airborne Visible InfraRed Imaging Spectrometer ([https://aviris.jpl.nasa.gov/ AVIRIS Main page]).

Its optical sensor creates calibrated images of the upwelling spectral radiance in 224 contiguous spectral channels/bands with wavelengths from 400 to 2500 nanometers (nm).

=== Airplanes ===
'''AVIRIS''' has been flown on four aircraft platforms:
* NASA's ER-2 jet
* Twin Otter International's turboprop
* Scaled Composites' Proteus
* NASA's WB-57.

Flight details:
* The ER-2 flies at approximately 20km above sea level, at about 730 km/hr.
* The Twin Otter aircraft flies at 4km above ground level at 130km/hr.

=== Data Download ===
Downloading data from '''AVIRIS''' flights is straightforward using the [https://aviris.jpl.nasa.gov/dataportal/ AVIRIS Data Portal].

== Import AVIRIS in GRASS ==
=== Data Downloaded ===

'''AVIRIS''' is downloaded as a .tar.gz file (i.e. ''f180116t01p00r07.tar.gz'') uncompressing as a directory (i.e. ''f180116t01p00r07rdn_e/'') containing with a set files like this:

==== List of files ====
* AVIRIS_OrthoProcessing_Info.txt
* f180116t01p00r07rdn_e_eph
* f180116t01p00r07rdn_e.gain
* f180116t01p00r07rdn_e_lonlat_eph
* ''f180116t01p00r07rdn_e_obs''
* f180116t01p00r07rdn_e_obs.hdr
* ''f180116t01p00r07rdn_e_obs_ort''
* f180116t01p00r07rdn_e_obs_ort.hdr
* ''f180116t01p00r07rdn_e_ort_glt''
* f180116t01p00r07rdn_e_ort_glt.hdr
* ''f180116t01p00r07rdn_e_ort_igm''
* f180116t01p00r07rdn_e_ort_igm.hdr
* f180116t01p00r07rdn_e_ort.plog
* f180116t01p00r07rdn_e.rccf18
* ''f180116t01p00r07rdn_e_sc01_ort_img''
* f180116t01p00r07rdn_e_sc01_ort_img.hdr
* f180116t01p00r07rdn_e.spc

All files twinned with a .hdr file, and in italics in the list above, are ENVI style binary imagery datasets (''BSQ format''). Towards the end of the list, the file matching the pattern ''sc01'' is the largest file, and the one holding the actual hyperspectral data.

==== DN to Radiance Conversion ====

The DN to Radiance correction is done by applying a gain to the stored data in the BSQ file. In this case, the gain file in ''f180116t01p00r07rdn_e.gain''. Extracting the gain values can be done like this in Python:

<source lang="python">
gain = []
with open(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e.gain") as file:
for line in file:
gain.append(line.strip().split()[0])
</source>

After that, the DN to Radiance correction can be done like this:

<source lang="python">
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules import Module

rmapcalc = Module('r.mapcalc')
# AVIRIS DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
</source>

=== Import in GRASS, gain correction (DN2Rad) & atmospheric correction (Rad2Ref) ===
<source lang="python">
#!/usr/bin/env python
"""
Process all AVIRIS radiance tar.gz into reflectance.

Download all radiance tar.gz files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 24th October, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes AVIRIS images
# 1 - untar *.tar.gz files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the AVIRIS Directories
rsdatapath = "/home/yann/RSDATA/AVIRIS"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "AVIRIS"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .tar.gz AVIRIS radiance tracks downloaded
AVIRISDirs = ["f180116t01p00r07"] # , "f100517t01p00r11"]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of AVIRIS
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

def read_ephemeris_file(file_path):
"""
Read the ephemeris _eph file and extract flight altitude.

Parameters
----------
file_path : STRING
the ephemeris *_eph file.

Returns
-------
data : FLOAT
the average height of sensor during flight.
"""
data = []
# Define the format of a single record in the binary file
record_format = 'd' * 6 # Six double-precision floating-point numbers
with open(file_path, 'rb') as f:
while True:
record = f.read(struct.calcsize(record_format))
if not record:
break # Reached end of file
values = struct.unpack(record_format, record)
data.append(values)
return data

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for AVIRISDir in AVIRISDirs:
# Untar all of your AVIRIS images in all your directories
print("Untar AVIRIS files in ", AVIRISDir)
#p = os.system("tar xvf "+AVIRISDir+".tar.gz")
with open(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e.gain") as file:
for line in file:
gain.append(line.strip().split()[0])
# Process a North alignment of the original file TODO add datatype=float
kwargs = {'format': 'GTiff', 'geoloc': True, 'options': 'overwrite'}
# gdal.UseExceptions()
#ds = gdal.Warp("del.tif", AVIRISDir + "rdn_e/" + AVIRISDir + "rdn_e_sc01_ort_img", **kwargs)
#del ds
# Create a temporary GRASS Location
p = os.system("grass -c del.tif -e "+grassdata+location)
time.sleep(20)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM file is not empty
# ringdal(input=AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e_ort_igm", output="dem",
# overwrite=OVR)
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import AVIRIS in GRASS
ringdal(input="del.tif", output="dn", quiet=QUIET)
# Remove nodata from DN maps
rnull = Module('r.null')
for idx in range(224):
rnull(map="dn."+str(idx+1), setnull=[-50, 0], quiet=QUIET)
# Read rotated AVIRIS image
src = gdal.Open("del.tif")
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Import Auxiliary layers, rotate it first
ds = gdal.Warp("del1.tif", AVIRISDir + "rdn_e/" + AVIRISDir +
"rdn_e_obs_ort", **kwargs)
del ds
# Import in GRASS TODO force Float data
ringdal(input="del1.tif", output="obs", flags="e", overwrite=OVR, quiet=QUIET)
# Remove nodata from UTC map
rnull(map="obs.10", setnull=[-9999, 0], quiet=QUIET)
# Extract stats
rinfo = Module('r.info', map="obs.10", flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
dectime = rinfo_out["mean"]
print("timestamp ", str(dectime))
print(AVIRISDir)
date = ["20"+AVIRISDir[1:3], AVIRISDir[3:5], AVIRISDir[5:7]]
print(date)
# AVIRIS DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for AVIRIS
# Geometrical conditions (AVIRIS = 31)
geom = 31
# Sensor height (satellite is -1000)
eph_data = read_ephemeris_file(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e_eph")
# Extract average altitude
add = 0
for j in range(len(eph_data)):
add += eph_data[j][2]
height = add/len(eph_data)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1*height/1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# satellite band number (AVIRIS [209-432])
satbandno = 209 # Band 1 of AVIRIS is first for atmospheric correction
for idx in range(224):
b = "rad." + str(idx+1)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx+1), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=AVIRIS
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, sensor on board a plane)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# puw (water vapor content between the aircraft and the surface)
param.append("-1.0\n")
# po3 (ozone content between the aircraft and the surface)
param.append("-1.0\n")
# taerp (the aerosol optical thickness at 550nm between the aircraft and the surface)
param.append("-1.0\n")
# i th band of AVIRIS (atcorr internal no)
param.append(str(satbandno))
prm = "param_AVIRIS.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal = Module('r.out.gdal')
routgdal(input=b_out, output=AVIRISDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1
</source>

AVIRIS

2023-12-02T06:11:42Z

⚠️Ychemin: /* AVIRIS Hyperspectral Sensor */

== AVIRIS Hyperspectral Sensor ==
[[AVIRIS_text2.png]]
=== Introduction ===
'''AVIRIS''' is an acronym for the Airborne Visible InfraRed Imaging Spectrometer ([https://aviris.jpl.nasa.gov/ AVIRIS Main page]).

Its optical sensor creates calibrated images of the upwelling spectral radiance in 224 contiguous spectral channels/bands with wavelengths from 400 to 2500 nanometers (nm).

=== Airplanes ===
'''AVIRIS''' has been flown on four aircraft platforms:
* NASA's ER-2 jet
* Twin Otter International's turboprop
* Scaled Composites' Proteus
* NASA's WB-57.

Flight details:
* The ER-2 flies at approximately 20km above sea level, at about 730 km/hr.
* The Twin Otter aircraft flies at 4km above ground level at 130km/hr.

=== Data Download ===
Downloading data from '''AVIRIS''' flights is straightforward using the [https://aviris.jpl.nasa.gov/dataportal/ AVIRIS Data Portal].

== Import AVIRIS in GRASS ==
=== Data Downloaded ===

'''AVIRIS''' is downloaded as a .tar.gz file (i.e. ''f180116t01p00r07.tar.gz'') uncompressing as a directory (i.e. ''f180116t01p00r07rdn_e/'') containing with a set files like this:

==== List of files ====
* AVIRIS_OrthoProcessing_Info.txt
* f180116t01p00r07rdn_e_eph
* f180116t01p00r07rdn_e.gain
* f180116t01p00r07rdn_e_lonlat_eph
* ''f180116t01p00r07rdn_e_obs''
* f180116t01p00r07rdn_e_obs.hdr
* ''f180116t01p00r07rdn_e_obs_ort''
* f180116t01p00r07rdn_e_obs_ort.hdr
* ''f180116t01p00r07rdn_e_ort_glt''
* f180116t01p00r07rdn_e_ort_glt.hdr
* ''f180116t01p00r07rdn_e_ort_igm''
* f180116t01p00r07rdn_e_ort_igm.hdr
* f180116t01p00r07rdn_e_ort.plog
* f180116t01p00r07rdn_e.rccf18
* ''f180116t01p00r07rdn_e_sc01_ort_img''
* f180116t01p00r07rdn_e_sc01_ort_img.hdr
* f180116t01p00r07rdn_e.spc

All files twinned with a .hdr file, and in italics in the list above, are ENVI style binary imagery datasets (''BSQ format''). Towards the end of the list, the file matching the pattern ''sc01'' is the largest file, and the one holding the actual hyperspectral data.

==== DN to Radiance Conversion ====

The DN to Radiance correction is done by applying a gain to the stored data in the BSQ file. In this case, the gain file in ''f180116t01p00r07rdn_e.gain''. Extracting the gain values can be done like this in Python:

<source lang="python">
gain = []
with open(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e.gain") as file:
for line in file:
gain.append(line.strip().split()[0])
</source>

After that, the DN to Radiance correction can be done like this:

<source lang="python">
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules import Module

rmapcalc = Module('r.mapcalc')
# AVIRIS DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
</source>

=== Import in GRASS, gain correction (DN2Rad) & atmospheric correction (Rad2Ref) ===
<source lang="python">
#!/usr/bin/env python
"""
Process all AVIRIS radiance tar.gz into reflectance.

Download all radiance tar.gz files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 24th October, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes AVIRIS images
# 1 - untar *.tar.gz files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the AVIRIS Directories
rsdatapath = "/home/yann/RSDATA/AVIRIS"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "AVIRIS"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .tar.gz AVIRIS radiance tracks downloaded
AVIRISDirs = ["f180116t01p00r07"] # , "f100517t01p00r11"]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of AVIRIS
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

def read_ephemeris_file(file_path):
"""
Read the ephemeris _eph file and extract flight altitude.

Parameters
----------
file_path : STRING
the ephemeris *_eph file.

Returns
-------
data : FLOAT
the average height of sensor during flight.
"""
data = []
# Define the format of a single record in the binary file
record_format = 'd' * 6 # Six double-precision floating-point numbers
with open(file_path, 'rb') as f:
while True:
record = f.read(struct.calcsize(record_format))
if not record:
break # Reached end of file
values = struct.unpack(record_format, record)
data.append(values)
return data

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for AVIRISDir in AVIRISDirs:
# Untar all of your AVIRIS images in all your directories
print("Untar AVIRIS files in ", AVIRISDir)
#p = os.system("tar xvf "+AVIRISDir+".tar.gz")
with open(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e.gain") as file:
for line in file:
gain.append(line.strip().split()[0])
# Process a North alignment of the original file TODO add datatype=float
kwargs = {'format': 'GTiff', 'geoloc': True, 'options': 'overwrite'}
# gdal.UseExceptions()
#ds = gdal.Warp("del.tif", AVIRISDir + "rdn_e/" + AVIRISDir + "rdn_e_sc01_ort_img", **kwargs)
#del ds
# Create a temporary GRASS Location
p = os.system("grass -c del.tif -e "+grassdata+location)
time.sleep(20)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM file is not empty
# ringdal(input=AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e_ort_igm", output="dem",
# overwrite=OVR)
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import AVIRIS in GRASS
ringdal(input="del.tif", output="dn", quiet=QUIET)
# Remove nodata from DN maps
rnull = Module('r.null')
for idx in range(224):
rnull(map="dn."+str(idx+1), setnull=[-50, 0], quiet=QUIET)
# Read rotated AVIRIS image
src = gdal.Open("del.tif")
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Import Auxiliary layers, rotate it first
ds = gdal.Warp("del1.tif", AVIRISDir + "rdn_e/" + AVIRISDir +
"rdn_e_obs_ort", **kwargs)
del ds
# Import in GRASS TODO force Float data
ringdal(input="del1.tif", output="obs", flags="e", overwrite=OVR, quiet=QUIET)
# Remove nodata from UTC map
rnull(map="obs.10", setnull=[-9999, 0], quiet=QUIET)
# Extract stats
rinfo = Module('r.info', map="obs.10", flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
dectime = rinfo_out["mean"]
print("timestamp ", str(dectime))
print(AVIRISDir)
date = ["20"+AVIRISDir[1:3], AVIRISDir[3:5], AVIRISDir[5:7]]
print(date)
# AVIRIS DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for AVIRIS
# Geometrical conditions (AVIRIS = 31)
geom = 31
# Sensor height (satellite is -1000)
eph_data = read_ephemeris_file(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e_eph")
# Extract average altitude
add = 0
for j in range(len(eph_data)):
add += eph_data[j][2]
height = add/len(eph_data)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1*height/1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# satellite band number (AVIRIS [209-432])
satbandno = 209 # Band 1 of AVIRIS is first for atmospheric correction
for idx in range(224):
b = "rad." + str(idx+1)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx+1), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=AVIRIS
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, sensor on board a plane)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# puw (water vapor content between the aircraft and the surface)
param.append("-1.0\n")
# po3 (ozone content between the aircraft and the surface)
param.append("-1.0\n")
# taerp (the aerosol optical thickness at 550nm between the aircraft and the surface)
param.append("-1.0\n")
# i th band of AVIRIS (atcorr internal no)
param.append(str(satbandno))
prm = "param_AVIRIS.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal = Module('r.out.gdal')
routgdal(input=b_out, output=AVIRISDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1
</source>

File:Aviris text2.png

2023-12-02T06:10:40Z

⚠️Ychemin: Logo AVIRIS

== Summary ==
Logo AVIRIS

AVIRIS

2023-12-02T06:08:18Z

⚠️Ychemin:

== AVIRIS Hyperspectral Sensor ==

=== Introduction ===
'''AVIRIS''' is an acronym for the Airborne Visible InfraRed Imaging Spectrometer ([https://aviris.jpl.nasa.gov/ AVIRIS Main page]).

Its optical sensor creates calibrated images of the upwelling spectral radiance in 224 contiguous spectral channels/bands with wavelengths from 400 to 2500 nanometers (nm).

=== Airplanes ===
'''AVIRIS''' has been flown on four aircraft platforms:
* NASA's ER-2 jet
* Twin Otter International's turboprop
* Scaled Composites' Proteus
* NASA's WB-57.

Flight details:
* The ER-2 flies at approximately 20km above sea level, at about 730 km/hr.
* The Twin Otter aircraft flies at 4km above ground level at 130km/hr.

=== Data Download ===
Downloading data from '''AVIRIS''' flights is straightforward using the [https://aviris.jpl.nasa.gov/dataportal/ AVIRIS Data Portal].

== Import AVIRIS in GRASS ==
=== Data Downloaded ===

'''AVIRIS''' is downloaded as a .tar.gz file (i.e. ''f180116t01p00r07.tar.gz'') uncompressing as a directory (i.e. ''f180116t01p00r07rdn_e/'') containing with a set files like this:

==== List of files ====
* AVIRIS_OrthoProcessing_Info.txt
* f180116t01p00r07rdn_e_eph
* f180116t01p00r07rdn_e.gain
* f180116t01p00r07rdn_e_lonlat_eph
* ''f180116t01p00r07rdn_e_obs''
* f180116t01p00r07rdn_e_obs.hdr
* ''f180116t01p00r07rdn_e_obs_ort''
* f180116t01p00r07rdn_e_obs_ort.hdr
* ''f180116t01p00r07rdn_e_ort_glt''
* f180116t01p00r07rdn_e_ort_glt.hdr
* ''f180116t01p00r07rdn_e_ort_igm''
* f180116t01p00r07rdn_e_ort_igm.hdr
* f180116t01p00r07rdn_e_ort.plog
* f180116t01p00r07rdn_e.rccf18
* ''f180116t01p00r07rdn_e_sc01_ort_img''
* f180116t01p00r07rdn_e_sc01_ort_img.hdr
* f180116t01p00r07rdn_e.spc

All files twinned with a .hdr file, and in italics in the list above, are ENVI style binary imagery datasets (''BSQ format''). Towards the end of the list, the file matching the pattern ''sc01'' is the largest file, and the one holding the actual hyperspectral data.

==== DN to Radiance Conversion ====

The DN to Radiance correction is done by applying a gain to the stored data in the BSQ file. In this case, the gain file in ''f180116t01p00r07rdn_e.gain''. Extracting the gain values can be done like this in Python:

<source lang="python">
gain = []
with open(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e.gain") as file:
for line in file:
gain.append(line.strip().split()[0])
</source>

After that, the DN to Radiance correction can be done like this:

<source lang="python">
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules import Module

rmapcalc = Module('r.mapcalc')
# AVIRIS DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
</source>

=== Import in GRASS, gain correction (DN2Rad) & atmospheric correction (Rad2Ref) ===
<source lang="python">
#!/usr/bin/env python
"""
Process all AVIRIS radiance tar.gz into reflectance.

Download all radiance tar.gz files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 24th October, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes AVIRIS images
# 1 - untar *.tar.gz files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the AVIRIS Directories
rsdatapath = "/home/yann/RSDATA/AVIRIS"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "AVIRIS"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .tar.gz AVIRIS radiance tracks downloaded
AVIRISDirs = ["f180116t01p00r07"] # , "f100517t01p00r11"]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of AVIRIS
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

def read_ephemeris_file(file_path):
"""
Read the ephemeris _eph file and extract flight altitude.

Parameters
----------
file_path : STRING
the ephemeris *_eph file.

Returns
-------
data : FLOAT
the average height of sensor during flight.
"""
data = []
# Define the format of a single record in the binary file
record_format = 'd' * 6 # Six double-precision floating-point numbers
with open(file_path, 'rb') as f:
while True:
record = f.read(struct.calcsize(record_format))
if not record:
break # Reached end of file
values = struct.unpack(record_format, record)
data.append(values)
return data

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for AVIRISDir in AVIRISDirs:
# Untar all of your AVIRIS images in all your directories
print("Untar AVIRIS files in ", AVIRISDir)
#p = os.system("tar xvf "+AVIRISDir+".tar.gz")
with open(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e.gain") as file:
for line in file:
gain.append(line.strip().split()[0])
# Process a North alignment of the original file TODO add datatype=float
kwargs = {'format': 'GTiff', 'geoloc': True, 'options': 'overwrite'}
# gdal.UseExceptions()
#ds = gdal.Warp("del.tif", AVIRISDir + "rdn_e/" + AVIRISDir + "rdn_e_sc01_ort_img", **kwargs)
#del ds
# Create a temporary GRASS Location
p = os.system("grass -c del.tif -e "+grassdata+location)
time.sleep(20)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM file is not empty
# ringdal(input=AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e_ort_igm", output="dem",
# overwrite=OVR)
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import AVIRIS in GRASS
ringdal(input="del.tif", output="dn", quiet=QUIET)
# Remove nodata from DN maps
rnull = Module('r.null')
for idx in range(224):
rnull(map="dn."+str(idx+1), setnull=[-50, 0], quiet=QUIET)
# Read rotated AVIRIS image
src = gdal.Open("del.tif")
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Import Auxiliary layers, rotate it first
ds = gdal.Warp("del1.tif", AVIRISDir + "rdn_e/" + AVIRISDir +
"rdn_e_obs_ort", **kwargs)
del ds
# Import in GRASS TODO force Float data
ringdal(input="del1.tif", output="obs", flags="e", overwrite=OVR, quiet=QUIET)
# Remove nodata from UTC map
rnull(map="obs.10", setnull=[-9999, 0], quiet=QUIET)
# Extract stats
rinfo = Module('r.info', map="obs.10", flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
dectime = rinfo_out["mean"]
print("timestamp ", str(dectime))
print(AVIRISDir)
date = ["20"+AVIRISDir[1:3], AVIRISDir[3:5], AVIRISDir[5:7]]
print(date)
# AVIRIS DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for AVIRIS
# Geometrical conditions (AVIRIS = 31)
geom = 31
# Sensor height (satellite is -1000)
eph_data = read_ephemeris_file(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e_eph")
# Extract average altitude
add = 0
for j in range(len(eph_data)):
add += eph_data[j][2]
height = add/len(eph_data)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1*height/1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# satellite band number (AVIRIS [209-432])
satbandno = 209 # Band 1 of AVIRIS is first for atmospheric correction
for idx in range(224):
b = "rad." + str(idx+1)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx+1), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=AVIRIS
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, sensor on board a plane)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# puw (water vapor content between the aircraft and the surface)
param.append("-1.0\n")
# po3 (ozone content between the aircraft and the surface)
param.append("-1.0\n")
# taerp (the aerosol optical thickness at 550nm between the aircraft and the surface)
param.append("-1.0\n")
# i th band of AVIRIS (atcorr internal no)
param.append(str(satbandno))
prm = "param_AVIRIS.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal = Module('r.out.gdal')
routgdal(input=b_out, output=AVIRISDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1
</source>

AVIRIS

2023-12-02T06:06:31Z

⚠️Ychemin:

== AVIRIS Hyperspectral Sensor ==
=== Introduction ===
'''AVIRIS''' is an acronym for the Airborne Visible InfraRed Imaging Spectrometer ([https://aviris.jpl.nasa.gov/ AVIRIS Main page]).

Its optical sensor creates calibrated images of the upwelling spectral radiance in 224 contiguous spectral channels/bands with wavelengths from 400 to 2500 nanometers (nm).

=== Airplanes ===
'''AVIRIS''' has been flown on four aircraft platforms:
* NASA's ER-2 jet
* Twin Otter International's turboprop
* Scaled Composites' Proteus
* NASA's WB-57.

Flight details:
* The ER-2 flies at approximately 20km above sea level, at about 730 km/hr.
* The Twin Otter aircraft flies at 4km above ground level at 130km/hr.

=== Data Download ===
Downloading data from '''AVIRIS''' flights is straightforward using the [https://aviris.jpl.nasa.gov/dataportal/ AVIRIS Data Portal].

== Import AVIRIS in GRASS ==
=== Data Downloaded ===

'''AVIRIS''' is downloaded as a .tar.gz file (i.e. ''f180116t01p00r07.tar.gz'') uncompressing as a directory (i.e. ''f180116t01p00r07rdn_e/'') containing with a set files like this:

==== List of files ====
* AVIRIS_OrthoProcessing_Info.txt
* f180116t01p00r07rdn_e_eph
* f180116t01p00r07rdn_e.gain
* f180116t01p00r07rdn_e_lonlat_eph
* ''f180116t01p00r07rdn_e_obs''
* f180116t01p00r07rdn_e_obs.hdr
* ''f180116t01p00r07rdn_e_obs_ort''
* f180116t01p00r07rdn_e_obs_ort.hdr
* ''f180116t01p00r07rdn_e_ort_glt''
* f180116t01p00r07rdn_e_ort_glt.hdr
* ''f180116t01p00r07rdn_e_ort_igm''
* f180116t01p00r07rdn_e_ort_igm.hdr
* f180116t01p00r07rdn_e_ort.plog
* f180116t01p00r07rdn_e.rccf18
* ''f180116t01p00r07rdn_e_sc01_ort_img''
* f180116t01p00r07rdn_e_sc01_ort_img.hdr
* f180116t01p00r07rdn_e.spc

All files twinned with a .hdr file, and in italics in the list above, are ENVI style binary imagery datasets (''BSQ format''). Towards the end of the list, the file matching the pattern ''sc01'' is the largest file, and the one holding the actual hyperspectral data.

==== DN to Radiance Conversion ====

The DN to Radiance correction is done by applying a gain to the stored data in the BSQ file. In this case, the gain file in ''f180116t01p00r07rdn_e.gain''. Extracting the gain values can be done like this in Python:

<source lang="python">
gain = []
with open(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e.gain") as file:
for line in file:
gain.append(line.strip().split()[0])
</source>

After that, the DN to Radiance correction can be done like this:

<source lang="python">
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules import Module

rmapcalc = Module('r.mapcalc')
# AVIRIS DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
</source>

=== Import in GRASS, gain correction (DN2Rad) & atmospheric correction (Rad2Ref) ===
<source lang="python">
#!/usr/bin/env python
"""
Process all AVIRIS radiance tar.gz into reflectance.

Download all radiance tar.gz files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 24th October, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes AVIRIS images
# 1 - untar *.tar.gz files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the AVIRIS Directories
rsdatapath = "/home/yann/RSDATA/AVIRIS"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "AVIRIS"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .tar.gz AVIRIS radiance tracks downloaded
AVIRISDirs = ["f180116t01p00r07"] # , "f100517t01p00r11"]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of AVIRIS
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

def read_ephemeris_file(file_path):
"""
Read the ephemeris _eph file and extract flight altitude.

Parameters
----------
file_path : STRING
the ephemeris *_eph file.

Returns
-------
data : FLOAT
the average height of sensor during flight.
"""
data = []
# Define the format of a single record in the binary file
record_format = 'd' * 6 # Six double-precision floating-point numbers
with open(file_path, 'rb') as f:
while True:
record = f.read(struct.calcsize(record_format))
if not record:
break # Reached end of file
values = struct.unpack(record_format, record)
data.append(values)
return data

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for AVIRISDir in AVIRISDirs:
# Untar all of your AVIRIS images in all your directories
print("Untar AVIRIS files in ", AVIRISDir)
#p = os.system("tar xvf "+AVIRISDir+".tar.gz")
with open(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e.gain") as file:
for line in file:
gain.append(line.strip().split()[0])
# Process a North alignment of the original file TODO add datatype=float
kwargs = {'format': 'GTiff', 'geoloc': True, 'options': 'overwrite'}
# gdal.UseExceptions()
#ds = gdal.Warp("del.tif", AVIRISDir + "rdn_e/" + AVIRISDir + "rdn_e_sc01_ort_img", **kwargs)
#del ds
# Create a temporary GRASS Location
p = os.system("grass -c del.tif -e "+grassdata+location)
time.sleep(20)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM file is not empty
# ringdal(input=AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e_ort_igm", output="dem",
# overwrite=OVR)
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import AVIRIS in GRASS
ringdal(input="del.tif", output="dn", quiet=QUIET)
# Remove nodata from DN maps
rnull = Module('r.null')
for idx in range(224):
rnull(map="dn."+str(idx+1), setnull=[-50, 0], quiet=QUIET)
# Read rotated AVIRIS image
src = gdal.Open("del.tif")
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Import Auxiliary layers, rotate it first
ds = gdal.Warp("del1.tif", AVIRISDir + "rdn_e/" + AVIRISDir +
"rdn_e_obs_ort", **kwargs)
del ds
# Import in GRASS TODO force Float data
ringdal(input="del1.tif", output="obs", flags="e", overwrite=OVR, quiet=QUIET)
# Remove nodata from UTC map
rnull(map="obs.10", setnull=[-9999, 0], quiet=QUIET)
# Extract stats
rinfo = Module('r.info', map="obs.10", flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
dectime = rinfo_out["mean"]
print("timestamp ", str(dectime))
print(AVIRISDir)
date = ["20"+AVIRISDir[1:3], AVIRISDir[3:5], AVIRISDir[5:7]]
print(date)
# AVIRIS DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for AVIRIS
# Geometrical conditions (AVIRIS = 31)
geom = 31
# Sensor height (satellite is -1000)
eph_data = read_ephemeris_file(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e_eph")
# Extract average altitude
add = 0
for j in range(len(eph_data)):
add += eph_data[j][2]
height = add/len(eph_data)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1*height/1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# satellite band number (AVIRIS [209-432])
satbandno = 209 # Band 1 of AVIRIS is first for atmospheric correction
for idx in range(224):
b = "rad." + str(idx+1)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx+1), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=AVIRIS
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, sensor on board a plane)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# puw (water vapor content between the aircraft and the surface)
param.append("-1.0\n")
# po3 (ozone content between the aircraft and the surface)
param.append("-1.0\n")
# taerp (the aerosol optical thickness at 550nm between the aircraft and the surface)
param.append("-1.0\n")
# i th band of AVIRIS (atcorr internal no)
param.append(str(satbandno))
prm = "param_AVIRIS.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal = Module('r.out.gdal')
routgdal(input=b_out, output=AVIRISDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1
</source>

AVIRIS

2023-12-02T06:04:35Z

⚠️Ychemin: /* Files of interest */

== AVIRIS Hyperspectral Sensor ==
=== Introduction ===
'''AVIRIS''' is an acronym for the Airborne Visible InfraRed Imaging Spectrometer ([https://aviris.jpl.nasa.gov/ AVIRIS Main page]).

Its optical sensor creates calibrated images of the upwelling spectral radiance in 224 contiguous spectral channels/bands with wavelengths from 400 to 2500 nanometers (nm).

=== Airplanes ===
'''AVIRIS''' has been flown on four aircraft platforms:
* NASA's ER-2 jet
* Twin Otter International's turboprop
* Scaled Composites' Proteus
* NASA's WB-57.

Flight details:
* The ER-2 flies at approximately 20km above sea level, at about 730 km/hr.
* The Twin Otter aircraft flies at 4km above ground level at 130km/hr.

=== Data Download ===
Downloading data from '''AVIRIS''' flights is straightforward using the [https://aviris.jpl.nasa.gov/dataportal/ AVIRIS Data Portal].

== Import AVIRIS in GRASS ==
=== Data Downloaded ===

'''AVIRIS''' is downloaded as a .tar.gz file (i.e. ''f180116t01p00r07.tar.gz'') uncompressing as a directory (i.e. ''f180116t01p00r07rdn_e/'') containing with a set files like this:

==== List of files ====
* AVIRIS_OrthoProcessing_Info.txt
* f180116t01p00r07rdn_e_eph
* f180116t01p00r07rdn_e.gain
* f180116t01p00r07rdn_e_lonlat_eph
* ''f180116t01p00r07rdn_e_obs''
* f180116t01p00r07rdn_e_obs.hdr
* ''f180116t01p00r07rdn_e_obs_ort''
* f180116t01p00r07rdn_e_obs_ort.hdr
* ''f180116t01p00r07rdn_e_ort_glt''
* f180116t01p00r07rdn_e_ort_glt.hdr
* ''f180116t01p00r07rdn_e_ort_igm''
* f180116t01p00r07rdn_e_ort_igm.hdr
* f180116t01p00r07rdn_e_ort.plog
* f180116t01p00r07rdn_e.rccf18
* ''f180116t01p00r07rdn_e_sc01_ort_img''
* f180116t01p00r07rdn_e_sc01_ort_img.hdr
* f180116t01p00r07rdn_e.spc

All files twinned with a .hdr file, and in italics in the list above, are ENVI style binary imagery datasets (''BSQ format''). Towards the end of the list, the file matching the pattern ''sc01'' is the largest file, and the one holding the actual hyperspectral data.

==== DN to Radiance Conversion ====

The DN to Radiance correction is done by applying a gain to the stored data in the BSQ file. In this case, the gain file in ''f180116t01p00r07rdn_e.gain''. Extracting the gain values can be done like this in Python:

<source lang="python">
gain = []
with open(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e.gain") as file:
for line in file:
gain.append(line.strip().split()[0])
</source>

After that, the DN to Radiance correction can be done like this:

<source lang="python">
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules import Module

rmapcalc = Module('r.mapcalc')
# AVIRIS DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
</source>

=== Importing script ===

=== PyGRASS script to import and atmospherically correct AVIRIS ===
<source lang="python">
#!/usr/bin/env python
"""
Process all AVIRIS radiance tar.gz into reflectance.

Download all radiance tar.gz files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 24th October, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes AVIRIS images
# 1 - untar *.tar.gz files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the AVIRIS Directories
rsdatapath = "/home/yann/RSDATA/AVIRIS"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "AVIRIS"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .tar.gz AVIRIS radiance tracks downloaded
AVIRISDirs = ["f180116t01p00r07"] # , "f100517t01p00r11"]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of AVIRIS
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

def read_ephemeris_file(file_path):
"""
Read the ephemeris _eph file and extract flight altitude.

Parameters
----------
file_path : STRING
the ephemeris *_eph file.

Returns
-------
data : FLOAT
the average height of sensor during flight.
"""
data = []
# Define the format of a single record in the binary file
record_format = 'd' * 6 # Six double-precision floating-point numbers
with open(file_path, 'rb') as f:
while True:
record = f.read(struct.calcsize(record_format))
if not record:
break # Reached end of file
values = struct.unpack(record_format, record)
data.append(values)
return data

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for AVIRISDir in AVIRISDirs:
# Untar all of your AVIRIS images in all your directories
print("Untar AVIRIS files in ", AVIRISDir)
#p = os.system("tar xvf "+AVIRISDir+".tar.gz")
with open(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e.gain") as file:
for line in file:
gain.append(line.strip().split()[0])
# Process a North alignment of the original file TODO add datatype=float
kwargs = {'format': 'GTiff', 'geoloc': True, 'options': 'overwrite'}
# gdal.UseExceptions()
#ds = gdal.Warp("del.tif", AVIRISDir + "rdn_e/" + AVIRISDir + "rdn_e_sc01_ort_img", **kwargs)
#del ds
# Create a temporary GRASS Location
p = os.system("grass -c del.tif -e "+grassdata+location)
time.sleep(20)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM file is not empty
# ringdal(input=AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e_ort_igm", output="dem",
# overwrite=OVR)
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import AVIRIS in GRASS
ringdal(input="del.tif", output="dn", quiet=QUIET)
# Remove nodata from DN maps
rnull = Module('r.null')
for idx in range(224):
rnull(map="dn."+str(idx+1), setnull=[-50, 0], quiet=QUIET)
# Read rotated AVIRIS image
src = gdal.Open("del.tif")
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Import Auxiliary layers, rotate it first
ds = gdal.Warp("del1.tif", AVIRISDir + "rdn_e/" + AVIRISDir +
"rdn_e_obs_ort", **kwargs)
del ds
# Import in GRASS TODO force Float data
ringdal(input="del1.tif", output="obs", flags="e", overwrite=OVR, quiet=QUIET)
# Remove nodata from UTC map
rnull(map="obs.10", setnull=[-9999, 0], quiet=QUIET)
# Extract stats
rinfo = Module('r.info', map="obs.10", flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
dectime = rinfo_out["mean"]
print("timestamp ", str(dectime))
print(AVIRISDir)
date = ["20"+AVIRISDir[1:3], AVIRISDir[3:5], AVIRISDir[5:7]]
print(date)
# AVIRIS DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for AVIRIS
# Geometrical conditions (AVIRIS = 31)
geom = 31
# Sensor height (satellite is -1000)
eph_data = read_ephemeris_file(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e_eph")
# Extract average altitude
add = 0
for j in range(len(eph_data)):
add += eph_data[j][2]
height = add/len(eph_data)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1*height/1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# satellite band number (AVIRIS [209-432])
satbandno = 209 # Band 1 of AVIRIS is first for atmospheric correction
for idx in range(224):
b = "rad." + str(idx+1)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx+1), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=AVIRIS
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, sensor on board a plane)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# puw (water vapor content between the aircraft and the surface)
param.append("-1.0\n")
# po3 (ozone content between the aircraft and the surface)
param.append("-1.0\n")
# taerp (the aerosol optical thickness at 550nm between the aircraft and the surface)
param.append("-1.0\n")
# i th band of AVIRIS (atcorr internal no)
param.append(str(satbandno))
prm = "param_AVIRIS.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal = Module('r.out.gdal')
routgdal(input=b_out, output=AVIRISDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1
</source>

AVIRIS

2023-12-02T06:02:44Z

⚠️Ychemin: /* Data Downloaded */

== AVIRIS Hyperspectral Sensor ==
=== Introduction ===
'''AVIRIS''' is an acronym for the Airborne Visible InfraRed Imaging Spectrometer ([https://aviris.jpl.nasa.gov/ AVIRIS Main page]).

Its optical sensor creates calibrated images of the upwelling spectral radiance in 224 contiguous spectral channels/bands with wavelengths from 400 to 2500 nanometers (nm).

=== Airplanes ===
'''AVIRIS''' has been flown on four aircraft platforms:
* NASA's ER-2 jet
* Twin Otter International's turboprop
* Scaled Composites' Proteus
* NASA's WB-57.

Flight details:
* The ER-2 flies at approximately 20km above sea level, at about 730 km/hr.
* The Twin Otter aircraft flies at 4km above ground level at 130km/hr.

=== Data Download ===
Downloading data from '''AVIRIS''' flights is straightforward using the [https://aviris.jpl.nasa.gov/dataportal/ AVIRIS Data Portal].

== Import AVIRIS in GRASS ==
=== Data Downloaded ===

'''AVIRIS''' is downloaded as a .tar.gz file (i.e. ''f180116t01p00r07.tar.gz'') uncompressing as a directory (i.e. ''f180116t01p00r07rdn_e/'') containing with a set files like this:

==== List of files ====
* AVIRIS_OrthoProcessing_Info.txt
* f180116t01p00r07rdn_e_eph
* f180116t01p00r07rdn_e.gain
* f180116t01p00r07rdn_e_lonlat_eph
* ''f180116t01p00r07rdn_e_obs''
* f180116t01p00r07rdn_e_obs.hdr
* ''f180116t01p00r07rdn_e_obs_ort''
* f180116t01p00r07rdn_e_obs_ort.hdr
* ''f180116t01p00r07rdn_e_ort_glt''
* f180116t01p00r07rdn_e_ort_glt.hdr
* ''f180116t01p00r07rdn_e_ort_igm''
* f180116t01p00r07rdn_e_ort_igm.hdr
* f180116t01p00r07rdn_e_ort.plog
* f180116t01p00r07rdn_e.rccf18
* ''f180116t01p00r07rdn_e_sc01_ort_img''
* f180116t01p00r07rdn_e_sc01_ort_img.hdr
* f180116t01p00r07rdn_e.spc

==== Files of interest ====
All files twinned with a .hdr file, and in italics in the list above, are ENVI style binary imagery datasets (''BSQ format''). Towards the end of the list, the file matching the pattern ''sc01'' is the largest file, and the one holding the actual hyperspectral data.

The DN to Radiance correction is done by applying a gain to the stored data in the BSQ file. In this case, the gain file in ''f180116t01p00r07rdn_e.gain''. Extracting the gain values can be done like this in Python:

<source lang="python">
gain = []
with open(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e.gain") as file:
for line in file:
gain.append(line.strip().split()[0])
</source>

After that, the DN to Radiance correction can be done like this:

<source lang="python">
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules import Module

rmapcalc = Module('r.mapcalc')
# AVIRIS DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
</source>

=== Importing script ===

=== PyGRASS script to import and atmospherically correct AVIRIS ===
<source lang="python">
#!/usr/bin/env python
"""
Process all AVIRIS radiance tar.gz into reflectance.

Download all radiance tar.gz files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 24th October, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes AVIRIS images
# 1 - untar *.tar.gz files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the AVIRIS Directories
rsdatapath = "/home/yann/RSDATA/AVIRIS"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "AVIRIS"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .tar.gz AVIRIS radiance tracks downloaded
AVIRISDirs = ["f180116t01p00r07"] # , "f100517t01p00r11"]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of AVIRIS
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

def read_ephemeris_file(file_path):
"""
Read the ephemeris _eph file and extract flight altitude.

Parameters
----------
file_path : STRING
the ephemeris *_eph file.

Returns
-------
data : FLOAT
the average height of sensor during flight.
"""
data = []
# Define the format of a single record in the binary file
record_format = 'd' * 6 # Six double-precision floating-point numbers
with open(file_path, 'rb') as f:
while True:
record = f.read(struct.calcsize(record_format))
if not record:
break # Reached end of file
values = struct.unpack(record_format, record)
data.append(values)
return data

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for AVIRISDir in AVIRISDirs:
# Untar all of your AVIRIS images in all your directories
print("Untar AVIRIS files in ", AVIRISDir)
#p = os.system("tar xvf "+AVIRISDir+".tar.gz")
with open(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e.gain") as file:
for line in file:
gain.append(line.strip().split()[0])
# Process a North alignment of the original file TODO add datatype=float
kwargs = {'format': 'GTiff', 'geoloc': True, 'options': 'overwrite'}
# gdal.UseExceptions()
#ds = gdal.Warp("del.tif", AVIRISDir + "rdn_e/" + AVIRISDir + "rdn_e_sc01_ort_img", **kwargs)
#del ds
# Create a temporary GRASS Location
p = os.system("grass -c del.tif -e "+grassdata+location)
time.sleep(20)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM file is not empty
# ringdal(input=AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e_ort_igm", output="dem",
# overwrite=OVR)
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import AVIRIS in GRASS
ringdal(input="del.tif", output="dn", quiet=QUIET)
# Remove nodata from DN maps
rnull = Module('r.null')
for idx in range(224):
rnull(map="dn."+str(idx+1), setnull=[-50, 0], quiet=QUIET)
# Read rotated AVIRIS image
src = gdal.Open("del.tif")
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Import Auxiliary layers, rotate it first
ds = gdal.Warp("del1.tif", AVIRISDir + "rdn_e/" + AVIRISDir +
"rdn_e_obs_ort", **kwargs)
del ds
# Import in GRASS TODO force Float data
ringdal(input="del1.tif", output="obs", flags="e", overwrite=OVR, quiet=QUIET)
# Remove nodata from UTC map
rnull(map="obs.10", setnull=[-9999, 0], quiet=QUIET)
# Extract stats
rinfo = Module('r.info', map="obs.10", flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
dectime = rinfo_out["mean"]
print("timestamp ", str(dectime))
print(AVIRISDir)
date = ["20"+AVIRISDir[1:3], AVIRISDir[3:5], AVIRISDir[5:7]]
print(date)
# AVIRIS DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for AVIRIS
# Geometrical conditions (AVIRIS = 31)
geom = 31
# Sensor height (satellite is -1000)
eph_data = read_ephemeris_file(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e_eph")
# Extract average altitude
add = 0
for j in range(len(eph_data)):
add += eph_data[j][2]
height = add/len(eph_data)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1*height/1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# satellite band number (AVIRIS [209-432])
satbandno = 209 # Band 1 of AVIRIS is first for atmospheric correction
for idx in range(224):
b = "rad." + str(idx+1)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx+1), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=AVIRIS
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, sensor on board a plane)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# puw (water vapor content between the aircraft and the surface)
param.append("-1.0\n")
# po3 (ozone content between the aircraft and the surface)
param.append("-1.0\n")
# taerp (the aerosol optical thickness at 550nm between the aircraft and the surface)
param.append("-1.0\n")
# i th band of AVIRIS (atcorr internal no)
param.append(str(satbandno))
prm = "param_AVIRIS.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal = Module('r.out.gdal')
routgdal(input=b_out, output=AVIRISDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1
</source>

AVIRIS

2023-12-02T05:54:42Z

⚠️Ychemin: /* Interface to copying maps (g.copy) */

== AVIRIS Hyperspectral Sensor ==
=== Introduction ===
'''AVIRIS''' is an acronym for the Airborne Visible InfraRed Imaging Spectrometer ([https://aviris.jpl.nasa.gov/ AVIRIS Main page]).

Its optical sensor creates calibrated images of the upwelling spectral radiance in 224 contiguous spectral channels/bands with wavelengths from 400 to 2500 nanometers (nm).

=== Airplanes ===
'''AVIRIS''' has been flown on four aircraft platforms:
* NASA's ER-2 jet
* Twin Otter International's turboprop
* Scaled Composites' Proteus
* NASA's WB-57.

Flight details:
* The ER-2 flies at approximately 20km above sea level, at about 730 km/hr.
* The Twin Otter aircraft flies at 4km above ground level at 130km/hr.

=== Data Download ===
Downloading data from '''AVIRIS''' flights is straightforward using the [https://aviris.jpl.nasa.gov/dataportal/ AVIRIS Data Portal].

== Import AVIRIS in GRASS ==
=== Data Downloaded ===

'''AVIRIS''' is downloaded as a .tar.gz file (i.e. ''f180116t01p00r07.tar.gz'') uncompressing as a directory (i.e. ''f180116t01p00r07rdn_e/'') containing with a set files like this:

* AVIRIS_OrthoProcessing_Info.txt
* f180116t01p00r07rdn_e_eph
* f180116t01p00r07rdn_e.gain
* f180116t01p00r07rdn_e_lonlat_eph
* ''f180116t01p00r07rdn_e_obs''
* f180116t01p00r07rdn_e_obs.hdr
* ''f180116t01p00r07rdn_e_obs_ort''
* f180116t01p00r07rdn_e_obs_ort.hdr
* ''f180116t01p00r07rdn_e_ort_glt''
* f180116t01p00r07rdn_e_ort_glt.hdr
* ''f180116t01p00r07rdn_e_ort_igm''
* f180116t01p00r07rdn_e_ort_igm.hdr
* f180116t01p00r07rdn_e_ort.plog
* f180116t01p00r07rdn_e.rccf18
* ''f180116t01p00r07rdn_e_sc01_ort_img''
* f180116t01p00r07rdn_e_sc01_ort_img.hdr
* f180116t01p00r07rdn_e.spc

All files twinned with a .hdr file, and in italics in the list above, are ENVI style binary imagery datasets (''BSQ format''). Towards the end of the list, the file matching the pattern ''sc01'' is the largest file, and the one holding the actual hyperspectral data.

=== Importing script ===

=== PyGRASS script to import and atmospherically correct AVIRIS ===
<source lang="python">
#!/usr/bin/env python
"""
Process all AVIRIS radiance tar.gz into reflectance.

Download all radiance tar.gz files into rsdatapath
Make sure GRASS GIS and python-gdal are installed
run this code after specifying your directories

# Date: 24th October, 2023
# Public domain, GRASS GIS

# PURPOSE
# This script processes AVIRIS images
# 1 - untar *.tar.gz files
# 2 - Connect files in GRASS GIS Location of your choice
# 3 - DN to Radiance (Apply Gain)
# 4 - Radiance to Surface reflectance (i.atcorr)
"""
# Load necessary libraries
import os
import sys
import time
import struct
from subprocess import PIPE
# from osgeo import ogr
from osgeo import osr
from osgeo import gdal
# define GRASS-Python environment
os.environ['GISBASE'] = "/usr/local/grass84"
grass_pydir = os.path.join("/usr/local/grass84", "etc", "python")
sys.path.append(grass_pydir)
from grass.script import setup as gsetup
from grass.pygrass.modules.shortcuts import raster as r
from grass.pygrass.modules.shortcuts import imagery as i
from grass.pygrass.modules import Module
from grass.script.utils import parse_key_val

# QUIET REPORTS
QUIET = True

# OVERWRITE EXISTING FILES
OVR = True

# Setup the path to the AVIRIS Directories
rsdatapath = "/home/yann/RSDATA/AVIRIS"

# Setup your GRASS GIS working directory
grassdata = "/home/yann/grassdata/"
location = "AVIRIS"
print(location)
mapset = "PERMANENT"

# DEM input for atmospheric correction
inDEM = "/home/yann/RSDATA/SRTM/dem.vrt"

# Visibility distance [Km]
vis = 18

# List of .tar.gz AVIRIS radiance tracks downloaded
AVIRISDirs = ["f180116t01p00r07"] # , "f100517t01p00r11"]

# CONFIGURATION OF 6S
# https://grass.osgeo.org/grass84/manuals/i.atcorr.html

# Atmospheric mode
# none = 0
# tropical = 1
# midlatitude summer = 2
# midlatitude winter = 3
atm_mode = 1
# Aerosol model
# none = 0
# continental = 1
# maritime = 2
# urban = 3
aerosol_mode = 2

# END OF USER CHANGES
# DO NOT CHANGE ANYTHING AFTER THIS LINE !

# Change directory to find the tar.gz files of AVIRIS
os.chdir(rsdatapath)

# Remove previous temporary GRASS Location
p = os.system("rm -Rf "+grassdata+location)

def read_ephemeris_file(file_path):
"""
Read the ephemeris _eph file and extract flight altitude.

Parameters
----------
file_path : STRING
the ephemeris *_eph file.

Returns
-------
data : FLOAT
the average height of sensor during flight.
"""
data = []
# Define the format of a single record in the binary file
record_format = 'd' * 6 # Six double-precision floating-point numbers
with open(file_path, 'rb') as f:
while True:
record = f.read(struct.calcsize(record_format))
if not record:
break # Reached end of file
values = struct.unpack(record_format, record)
data.append(values)
return data

# START PROCESS
# Read Gain from File
gain = []
# Start processing each radiance track bundlefrom DN to Ref
for AVIRISDir in AVIRISDirs:
# Untar all of your AVIRIS images in all your directories
print("Untar AVIRIS files in ", AVIRISDir)
#p = os.system("tar xvf "+AVIRISDir+".tar.gz")
with open(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e.gain") as file:
for line in file:
gain.append(line.strip().split()[0])
# Process a North alignment of the original file TODO add datatype=float
kwargs = {'format': 'GTiff', 'geoloc': True, 'options': 'overwrite'}
# gdal.UseExceptions()
#ds = gdal.Warp("del.tif", AVIRISDir + "rdn_e/" + AVIRISDir + "rdn_e_sc01_ort_img", **kwargs)
#del ds
# Create a temporary GRASS Location
p = os.system("grass -c del.tif -e "+grassdata+location)
time.sleep(20)
session = gsetup.init(grassdata, location, mapset)
print("Import in GRASS GIS")
# Import DEM for atmospheric correction
ringdal = Module('r.in.gdal')
# In case the DEM file is not empty
# ringdal(input=AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e_ort_igm", output="dem",
# overwrite=OVR)
# In case the DEM is available to import from srtm.csi.cgiar.org
#rimport = Module('r.import')
#rimport(input=inDEM, output="dem", extent="region", overwrite=OVR)
# The least effective, make your own flat DEM layer
rmapcalc = Module('r.mapcalc')
rmapcalc(expression="dem=1", overwrite=OVR)
rmapcalc.wait()
# Create a visibility (AOD) map
rmapcalc(expression="vis=18", overwrite=OVR)
rmapcalc.wait()
# Import AVIRIS in GRASS
ringdal(input="del.tif", output="dn", quiet=QUIET)
# Remove nodata from DN maps
rnull = Module('r.null')
for idx in range(224):
rnull(map="dn."+str(idx+1), setnull=[-50, 0], quiet=QUIET)
# Read rotated AVIRIS image
src = gdal.Open("del.tif")
ulx_, xres, xskew, uly_, yskew, yres = src.GetGeoTransform()
lrx_ = ulx_ + (src.RasterXSize * xres)
lry_ = uly_ + (src.RasterYSize * yres)
# Setup the source projection
source = osr.SpatialReference()
source.ImportFromWkt(src.GetProjection())
del src
# The target projection is ll wgs84
target = osr.SpatialReference()
target.ImportFromEPSG(4326)
# Create the transform
transform = osr.CoordinateTransformation(source, target)
# Transform the points
uly, ulx, _ = transform.TransformPoint(ulx_, uly_)
lry, lrx, _ = transform.TransformPoint(lrx_, lry_)
# Import Auxiliary layers, rotate it first
ds = gdal.Warp("del1.tif", AVIRISDir + "rdn_e/" + AVIRISDir +
"rdn_e_obs_ort", **kwargs)
del ds
# Import in GRASS TODO force Float data
ringdal(input="del1.tif", output="obs", flags="e", overwrite=OVR, quiet=QUIET)
# Remove nodata from UTC map
rnull(map="obs.10", setnull=[-9999, 0], quiet=QUIET)
# Extract stats
rinfo = Module('r.info', map="obs.10", flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
dectime = rinfo_out["mean"]
print("timestamp ", str(dectime))
print(AVIRISDir)
date = ["20"+AVIRISDir[1:3], AVIRISDir[3:5], AVIRISDir[5:7]]
print(date)
# AVIRIS DN->Rad->Ref
print("Convert DN to Rad")
# Apply Gain from file
for j in range(len(gain)):
exp = str("rad."+str(j+1)+"=dn."+str(j+1)+"*10.0/"+str(gain[j]))
#print(exp)
rmapcalc(expression=exp, overwrite=OVR, quiet=QUIET)
rmapcalc.wait()
# Atmospheric Correction
print("Atmospheric Correction")
# Basic script for i.atcorr for AVIRIS
# Geometrical conditions (AVIRIS = 31)
geom = 31
# Sensor height (satellite is -1000)
eph_data = read_ephemeris_file(AVIRISDir+"rdn_e/"+AVIRISDir+"rdn_e_eph")
# Extract average altitude
add = 0
for j in range(len(eph_data)):
add += eph_data[j][2]
height = add/len(eph_data)
# Define the sensor height as Km and negative (for 6S)
sens_height = -1*height/1000.0
# Here we suppose you have altitude (DEM) and Visibility (VIS) maps ready
# ---------------------------------------------
# Altitude dummy value (in Km should be negative in this param file)
# (overwritten by DEM raster input)
alt = -0.600
# datetime of satellite overpass (month, day, GMT decimal hour)
mdh = str(date[1])+" "+str(date[2])+" "+str("%.3f" % float(dectime))
print(mdh)
# Central Lat/Long
Lon = ulx+(lrx-ulx)/2.0
Lat = lry+(uly-lry)/2.0
print(Lon, Lat)
# satellite band number (AVIRIS [209-432])
satbandno = 209 # Band 1 of AVIRIS is first for atmospheric correction
for idx in range(224):
b = "rad." + str(idx+1)
b_out = b.replace("rad.", "ref.")
# Extract min max radiance stats
rinfo = Module('r.info', map="rad."+str(idx+1), flags="s", stdout_=PIPE)
rinfo_out = parse_key_val(rinfo.outputs.stdout)
# Extract mean values from Dict
minimum = rinfo_out["min"]
maximum = rinfo_out["max"]
if (minimum < 0.0):
minimum = 0.0
param = []
# geometrical conditions=AVIRIS
param.append(str(geom)+"\n")
# month day hh.ddd longitude latitude (hh.ddd is in decimal hours GMT)
param.append(str(mdh)+" "+str("%.2f" % Lon)+" "+str("%.2f" % Lat)+"\n")
# atmospheric mode=tropical
param.append(str(atm_mode)+"\n")
# aerosols model=maritime
param.append(str(aerosol_mode)+"\n")
# visibility [km] (aerosol model concentration), not used (raster input)\n")
param.append(str(vis)+"\n")
# mean target elevation above sea level [km] (here 600m asl), not used (raster input)
param.append(str(alt)+"\n")
# sensor height (here, sensor on board a plane)
param.append(str("%.2f" % round(sens_height,2))+"\n")
# puw (water vapor content between the aircraft and the surface)
param.append("-1.0\n")
# po3 (ozone content between the aircraft and the surface)
param.append("-1.0\n")
# taerp (the aerosol optical thickness at 550nm between the aircraft and the surface)
param.append("-1.0\n")
# i th band of AVIRIS (atcorr internal no)
param.append(str(satbandno))
prm = "param_AVIRIS.txt"
f = open(prm, "w")
f.writelines(param)
f.close()
print(b)
print(b_out)
iatcorr = Module('i.atcorr')
iatcorr(input=b, elevation="dem", visibility="vis", parameters=prm,
output=b_out, range=[minimum, maximum], rescale=[0.0,1.0], quiet=QUIET, overwrite=OVR)
routgdal = Module('r.out.gdal')
routgdal(input=b_out, output=AVIRISDir+"_"+b_out+".tif", overwrite=OVR, quiet=QUIET)
satbandno = satbandno + 1
</source>