R.modis GSoC 2011

Student Name:	Luca Delucchi, University of Genova
Organization:	OSGeo - Open Source Geospatial Foundation
Mentor Name:	Markus Neteler
Title:	r.in.modis for GRASS GIS
SVN:	r.modis

Abstract

MODIS data are free geodata and they are crucial to derive ecological indicators. MODIS is flown on two NASA satellite systems, Terra and Aqua, which significantly improve the current data availability for scientific purposes. I am facing the challenge to develop a module which will be able to:

• import data from a downloaded file
• directly download MODIS data from a FTP server and then load the file(s) in a GRASS GIS location.

Application

Idea

The idea was a little bit changed from the initial idea that I wrote in the GSOC proposal. I spoke to my mentor and we think that the best solution is to create three modules:

• r.modis.download: to download the MODIS products using pyModis library. It will be able to download more tiles and different range of times
• r.modis.import: to import Level 3 MODIS products, it will use pyModis library to parse the XML files and MODIS Reprojection Tool. It will be able to import single tile or creates a mosaic for more tiles of the same day
• r.modis.process: to process and import MODIS raw data using MODIS Reprojection Tool Swath and geolocation angles

Project Plan

May 23 - 29	Wiki documentation, implementing of new functions in the pyModis library, start r.modis.download
May 30 - June 5	finishing and testing r.modis.download, start r.modis.import
June 6 - July 3	finishing and testing r.modis.import, I hope also to start r.modis.process
July 4 - July 31	finishing and testing r.modis.process
August 1 - 16	documentation, testing and bug fixing
August 17 - 22	final reviews

How to test

Note: GRASS 7 required.

Install r.modis.* modules via g.extension:

g.extension extension=r.modis

Please report any bugs, wishes via trac:

component: Raster
keywords: modis

See list of known bugs.

Weekly reports

#0 2011-05-27

1) What do I have completed this week?

I spent nine wonderful days in Prague, I attended before the meeting Geoinformatics FCE CTU 2011 and after the first Community spring. Nine days with a lot of main developers and power users of GRASS community I worked on pyModis, the library that I'll use in r.modis.download to download MODIS file:

• fixed a bug with Stefano Cavallari [1]
• implemented a new class to parse XML files of MODIS products [2]
• fixed a bug about name of downloaded file, now using os.path.join() [3]
• fixed a bug in the dayDownload function [4]

I also started to implement r.modis.download.

2) What am I going to achieve for next week?

Finish r.modis.download, if I'll have time I'd like to develop a class in pyModis to write a XML file for the mosaik file, that I'll use in r.modis.import

3) Is there any blocking issue?

No, everything is under control

#1 2011-06-03

1) What do I have completed this week?

During this week I could work well, without problems. I finished r.modis.download, now I'm looking for more info about Makefile, after I'll solve this little blocking issue I'll commit in the svn repository the first version of r.modis.download. I started to write documentation files for GRASS modules as HTML file like the other modules. I also worked on pyModis, I added some functions and a new class to launch Modis Reprojection Tool (MRT) software to convert a HDF file into a GeoTIFF file. This class it will be used in r.modis.import that I started to do. Here some changes

• an option to force the last day, this is useful with sixteen days data because delta could download more data that the user want [5]

MN mentor comment: please explain this a bit more

The library look in the folders called with the days started from today option and it go back of number delta... this work well with the daily products but with the sixteen days it could download more data, with endday option you can stop the download in the right day -- lucadelu, 20:28 4 June 2011 (UTC)

• fixed a bug when pyModis writes the list of downloaded files [6]
• added new functions for reading every variables from xml file of MODIS data [7] and one for creating a configuration file for MRT Tool software [8]
• started new class to convert HDF to GeoTITF file format [9]

2) What am I going to achieve for next week?

Release the first version of r.modis.download on the grass-addons svn repository. Working on r.modis.import

3) Is there any blocking issue?

I have to study Makefile system in GRASS GIS to compile pyModis like a library.

#2 2011-06-10

1) What do I have completed this week?

I released first version of r.modis.download to download MODIS products from NASA FTP server [10]

I continued to work on r.modis.import and pyModis [11] [12]

I had a meeting with Markus where we discussed these topics

• r.modis.download: he suggested some changes for username and password options
• r.modis.import: we spoke about the parameters to use for the conversion of hdf file with MRT resample

2) What am I going to achieve for next week?

Fix the problems of r.modis.download and continue with r.modis.import

3) Is there any blocking issue?

No

#3 2011-06-17

1) What do I have completed this week?

I fix some problems on r.modis:

• on r.modis.download [13]
  • remove the options username and password replace by option setting. This options could be the path of a file with username and password or the two options could be set by standard input
  • change the documentation
• on rmodislib:
  • on product class add some attributes to products dictionary and add a function to return the product from the code [14]
  • add a function get_proj to return projection paramaters [15]
  • add class datum to return the datum in the format for MRT software [16] [17]
  • add class projection to return the projection in the format for MRT software [18]
• continue to work on r.modis.import, I hope to release the first version next week

I work also on pyModis:

• close some bugs [19] [20]
• now the class convertModis can conver an hdf to tif file [21]
• add documentation for class and module [22]

2) What am I going to achieve for next week?

I'll try to release a working version of r.modis.import at least for a single tile, and start to work on mosaic creation

3) Is there any blocking issue?

No

#4 2011-06-24

1) What do I have completed this week?

I worked on r.modis.import, now I have a working copy. Currently it support only Lat/Long projection and not the mosaic

2) What am I going to achieve for next week?

I'll continue to work or r.modis.import to support more projections and maybe to create mosaic. I'll sure release something on GRASS ADDONS svn

3) Is there any blocking issue?

No

#5 2011-07-01

1) What do I have completed this week?

I release the first version of r.modis.import [23]; as input it support single tile and a list of HDF files, it can also create daily mosaic, it was tested only with location Lat/Long

On pyModis I did this changes:

• some english fix [24], and other little fix [25]
• add createMosaic function to create mosaic [26]; the method write_mosaic_xml works but it will be completed next weeks, missing some not essential attributes

2) What am I going to achieve for next week?

I will test the import in other projection system and I will start to study for the third module, r.modis.process. For this module I have to study the manual and do some test before starting to write code

3) Is there any blocking issue?

No

#6 2011-07-08

1) What do I have completed this week?

This week I had few time to spent in GSoC.

I read documentation about PROJ.4 to support more projection system in r.modis.import

I started to read documentation about MRT Swath

2) What am I going to achieve for next week?

Support more projection system r.modis.import

Finish to read MRT Swath documentation e start developing of r.modis.process

3) Is there any blocking issue?

No

#7 2011-07-15 Mid-term evaluations

1) What do I have completed this week?

This week I worked on the support for more projection system [27], actually it's possible to use some projection systems from the supported by MRT

I also started to work on r.modis.process, the first piece is the implementation of a class in pyModis to read and convert the raw MODIS data using MRTSwath

2) What am I going to achieve for next week?

I'll continue to work on r.modis.process, I hope to finish the class in pyModis and after start r.modis.process module

3) Is there any blocking issue?

No

#8 2011-07-22

1) What do I have completed this week?

I created a class, processModis in pyModis to process MODIS raw data using swath2grid. This class is similar to convertModis class [28].

I added also a function inside the parseModis class to create the parameter file for swath2grid [29].

2) What am I going to achieve for next week?

I would write the module r.modis.process

3) Is there any blocking issue?

No, maybe a little problem with the output of swath2grid. I have to investigate better about this

#9 2011-07-29

1) What do I have completed this week?

I continue to investigate the problem with swath2grid. I also try to find another solution using pytables but also with this there are some problems.

2) What am I going to achieve for next week?

I hope to find a good way to work with MODIS raw data

3) Is there any blocking issue?

I have a problem to convert raw data, I hope to find a solution early

#10 2011-08-05

1) What do I have completed this week?

I continued to investigate the problem with swath2grid. I also work on r.modis.import [30] and r.modis.process; now it seems to works but the result are very bad (screenshot)

2) What am I going to achieve for next week?

I contacted the NASA for investigating the error, I hope for an answer

3) Is there any blocking issue?

I have a problem to convert raw data, I hope to find a solution early

#11 2011-08-12

1) What do I have completed this week?

I worked on r.modis.import, now it support the analysis of QA map to select only good pixel. Now it works only for LST product, I have to implement the also for NDVI and Snow products. [31]

I fix some sentences of r.modis.download [32]

2) What am I going to achieve for next week?

Finish to implement NDVI and Snow products, I have to complete the function for write XML file of mosaic.

3) Is there any blocking issue?

I continue to have a problem to convert raw data, NASA didn't answer yes

#13 2011-08-26 Final evaluation

Passed.

Installation in GRASS 7

Run in a GRASS 7 session:

 g.extension r.modis