GRASS GSoC 2024 EODAG Support - Revision history

Hamed5001: /* Documentation & GitHub Fork */

2025-04-08T23:19:51Z

Documentation & GitHub Fork

← Older revision		Revision as of 23:19, 8 April 2025
Line 506:		Line 506:
	\|-		\|-
	\| i.landsat.download		\| i.landsat.download
	\| [https://grass.osgeo.org/grass84/manuals/addons/i.~~sentinel~~.download.html i.landsat.download manual]		\| [https://grass.osgeo.org/grass84/manuals/addons/i.landsat.download.html i.landsat.download manual]
	\|}		\|}

Hamed5001: /* Timeline */

2025-04-04T07:10:06Z

Timeline

← Older revision		Revision as of 07:10, 4 April 2025
Line 55:		Line 55:
	# Testing, refactoring and bug fixing		# Testing, refactoring and bug fixing
	# Work on feedback		# Work on feedback
	\|\|OK<br>OK<br>~~POSTPONED~~<br>OK<br>OK<br>		\|\|OK<br>OK<br>OK<br>OK<br>OK<br>
	\|-		\|-
	\| '''Week 5''' (June 23 - June 29) \|\|		\| '''Week 5''' (June 23 - June 29) \|\|

Hamed5001: /* Timeline */

2024-09-07T12:43:04Z

Timeline

← Older revision		Revision as of 12:43, 7 September 2024
Line 100:		Line 100:
	\|\|OK<br>OK		\|\|OK<br>OK
	\|-		\|-
	\| '''Evaluation Period''' \|\| August 26 - September 2 \|\| Submit code and final evaluation \|\| ~~In Progress~~		\| '''Evaluation Period''' \|\| August 26 - September 2 \|\| Submit code and final evaluation \|\| OK
	\|}		\|}

Hamed5001: /* Future Work */

2024-08-25T19:02:19Z

Future Work

← Older revision		Revision as of 19:02, 25 August 2024
Line 392:		Line 392:

	== Future Work ==		== Future Work ==
	* ~~There is potential to integrate~~ the <code>i.eodag</code> tool into <code>i.modis.download</code> too. This could be achieved by either adding <code>i.eodag</code> as an option or reimplementing <code>i.modis.download</code> to rely on <code>i.eodag</code> as the primary source for '''MODIS''' data.		* Investigate the possibility of the integration of the <code>i.eodag</code> tool into <code>i.modis.download</code> too. This could be achieved by either adding <code>i.eodag</code> as an option or reimplementing <code>i.modis.download</code> to rely on <code>i.eodag</code> as the primary source for '''MODIS''' data.

	* Investigate how '''EODAG product types''' can be matched to GRASS GIS '''image band libraries'''. [https://github.com/HamedElgizery/grass-addons/issues/7 Read More]		* Investigate how '''EODAG product types''' can be matched to GRASS GIS '''image band libraries'''. [https://github.com/HamedElgizery/grass-addons/issues/7 Read More]

			* Consider adding an option for adding labels for downloaded scenes with the <code>r.support</code> tool.

	* There is room for improvement in data querying and filtering, particularly for i.sentinel.download, which supports over '''20 products''', each with its own set of auxiliary metadata. A useful enhancement would be to provide users with an option to retrieve a list of available filters for each product type, enabling more refined searches. Although the filtering method is already implemented and functional, adding a method to access the metadata (queryables / filters) list for each product type would be beneficial. Since <code>i.eodag</code> currently offers metadata lists for each product type, this feature can be implemented in <code>i.sentinel.download</code> by leveraging the <code>i.eodag</code> tool.		* There is room for improvement in data querying and filtering, particularly for i.sentinel.download, which supports over '''20 products''', each with its own set of auxiliary metadata. A useful enhancement would be to provide users with an option to retrieve a list of available filters for each product type, enabling more refined searches. Although the filtering method is already implemented and functional, adding a method to access the metadata (queryables / filters) list for each product type would be beneficial. Since <code>i.eodag</code> currently offers metadata lists for each product type, this feature can be implemented in <code>i.sentinel.download</code> by leveraging the <code>i.eodag</code> tool.

Hamed5001: /* Future Work */

2024-08-24T00:38:35Z

Future Work

← Older revision		Revision as of 00:38, 24 August 2024
Line 392:		Line 392:

	== Future Work ==		== Future Work ==
	* There is potential to integrate the <code>i.eodag</code> ~~sub-package~~ into ~~the~~ <code>i.modis.download</code> ~~sub-package~~. This could be achieved by either adding <code>i.eodag</code> as an option or reimplementing <code>i.modis.download</code> to rely on <code>i.eodag</code> as the primary source for '''MODIS''' ~~geospatial~~ data.		* There is potential to integrate the <code>i.eodag</code> tool into <code>i.modis.download</code> too. This could be achieved by either adding <code>i.eodag</code> as an option or reimplementing <code>i.modis.download</code> to rely on <code>i.eodag</code> as the primary source for '''MODIS''' data.

	* Investigate how '''EODAG product types''' can be matched to GRASS GIS i'''~~mage~~ band libraries'''. [https://github.com/HamedElgizery/grass-addons/issues/7 Read More]		* Investigate how '''EODAG product types''' can be matched to GRASS GIS '''image band libraries'''. [https://github.com/HamedElgizery/grass-addons/issues/7 Read More]

	* There is room for improvement in data querying and filtering, particularly for i.sentinel.download, which supports over '''20 products''', each with its own set of auxiliary metadata. A useful enhancement would be to provide users with an option to retrieve a list of available filters for each product type, enabling more refined searches. Although the filtering method is already implemented and functional, adding a method to access the metadata (queryables / filters) list for each product type would be beneficial. Since <code>i.eodag</code> currently offers metadata lists for each product type, this feature can be implemented in <code>i.sentinel.download</code> by leveraging the <code>i.eodag</code> ~~sub-package~~.		* There is room for improvement in data querying and filtering, particularly for i.sentinel.download, which supports over '''20 products''', each with its own set of auxiliary metadata. A useful enhancement would be to provide users with an option to retrieve a list of available filters for each product type, enabling more refined searches. Although the filtering method is already implemented and functional, adding a method to access the metadata (queryables / filters) list for each product type would be beneficial. Since <code>i.eodag</code> currently offers metadata lists for each product type, this feature can be implemented in <code>i.sentinel.download</code> by leveraging the <code>i.eodag</code> tool.

	* The process of listing queryable options in <code>i.eodag</code> could be enhanced by extracting them directly from a ~~scene~~ instance rather than relying solely on the '''EODAG API,''' which may sometimes be outdated.		* The process of listing queryable options in <code>i.eodag</code> could be enhanced by extracting them directly from a data product instance rather than relying solely on the '''EODAG API,''' which may sometimes be outdated.

	== Links ==		== Links ==

Hamed5001: /* Conclusion */

2024-08-24T00:35:40Z

Conclusion

← Older revision		Revision as of 00:35, 24 August 2024
Line 385:		Line 385:
	== Conclusion ==		== Conclusion ==
	To wrap it up, I’m glad to have successfully achieved the three main targets of the project, with the invaluable support from my mentors!		To wrap it up, I’m glad to have successfully achieved the three main targets of the project, with the invaluable support from my mentors!
	* Integrated '''EODAG API''' to GRASS GIS through new ~~the~~ <code>i.eodag</code> ~~sub-package~~, serving as a unified portal for downloading a variety of ~~geospatial~~ data products.		* Integrated '''EODAG API''' to GRASS GIS through a new tool <code>i.eodag</code>, serving as a unified portal for downloading a variety of Earth Observation data products.
	* Reimplemented <code>i.landsat.download</code> using the new <code>i.eodag</code> ~~sub-package~~.		* Reimplemented <code>i.landsat.download</code> using the new <code>i.eodag</code> tool.
	* Reimplemented <code>i.sentinel.download</code> using the new <code>i.eodag</code> ~~sub-package~~.		* Reimplemented <code>i.sentinel.download</code> using the new <code>i.eodag</code> tool.

	I am incredibly grateful for the support I received this summer and for the opportunity to contribute to GRASS GIS through GSoC 2024. This experience has been rewarding, and I am excited to continue enhancing <code>i.eodag</code>. While we've made significant progress, there are still exciting opportunities for further improvement, as outlined in the Future Work section below.		I am incredibly grateful for the support I received this summer and for the opportunity to contribute to GRASS GIS through GSoC 2024. This experience has been rewarding, and I am excited to continue enhancing <code>i.eodag</code>. While we've made significant progress, there are still exciting opportunities for further improvement, as outlined in the Future Work section below.

Hamed5001: Apply suggestions

2024-08-24T00:34:19Z

Apply suggestions

Hamed5001: Apply suggestions from review

2024-08-24T00:29:26Z

Apply suggestions from review

← Older revision		Revision as of 00:29, 24 August 2024
Line 350:		Line 350:

	== The state of integration BEFORE the start of GSoC ==		== The state of integration BEFORE the start of GSoC ==
	~~Previously~~, ~~each geospatial~~ download ~~module~~ relied on different APIs, necessitating significant maintenance to accommodate updates from both the APIs themselves and their respective providers. Specifically:		Before this project, all of MODIS, Landsat and Sentinel download tools relied on different APIs, necessitating significant maintenance to accommodate for updates from both the APIs themselves and their respective providers. Specifically:

	*[https://grass.osgeo.org/grass84/manuals/addons/i.landsat.download.html i.landsat.download]:		*[https://grass.osgeo.org/grass84/manuals/addons/i.landsat.download.html i.landsat.download]:
	** Utilized the [https://github.com/yannforget/landsatxplore landsatxplore] Python library to download Landsat ~~geospatial scenes~~."		** Utilized the [https://github.com/yannforget/landsatxplore landsatxplore] Python library to download Landsat data from USGS. This library is no longer maintained by the original author.
	*[https://grass.osgeo.org/grass84/manuals/addons/i.sentinel.download.html i.sentinel.download] utilized three distinct data sources:		*[https://grass.osgeo.org/grass84/manuals/addons/i.sentinel.download.html i.sentinel.download] utilized three distinct data sources:
	** [https://github.com/sentinelsat/sentinelsat?tab=readme-ov-file sentinelsat] for downloading Sentinel data from the [https://scihub.copernicus.eu/ Copernicus Open Access Hub]. However, after the data migration to the [https://dataspace.copernicus.eu/ Copernicus Data Space Ecosystem], Sentinel products were no longer available through this source, rendering portions of the previous implementation non-functional.		** [https://github.com/sentinelsat/sentinelsat?tab=readme-ov-file sentinelsat] for downloading Sentinel data from the [https://scihub.copernicus.eu/ Copernicus Open Access Hub]. However, after the data migration to the [https://dataspace.copernicus.eu/ Copernicus Data Space Ecosystem], Sentinel products were no longer available through this source, rendering portions of the previous implementation non-functional.
	** [https://github.com/yannforget/landsatxplore landsatxplore] to access Sentinel data from [https://earthexplorer.usgs.gov/ USGS].		** [https://github.com/yannforget/landsatxplore landsatxplore] to access Sentinel data from [https://earthexplorer.usgs.gov/ USGS].
	** Manual data requests via the [https://cloud.google.com/solutions/geospatial?hl=en Google Cloud Storage] API.		** Manual data requests via the [https://cloud.google.com/solutions/geospatial?hl=en Google Cloud Storage] API.

	Maintaining the aforementioned geospatial sources presented significant challenges. Specifically, the i.landsat.download module required updates to ensure compatibility with changes in the Landsatxplore API. Additionally, a substantial portion of the i.sentinel.download module became non-functional following a data migration. Regular maintenance and periodic fixes were necessary, which placed a considerable burden on the project.

	== The state of integration AFTER GSoC ==		== The state of integration AFTER GSoC ==

Hamed5001: Apply suggestions

2024-08-24T00:25:53Z

Apply suggestions

← Older revision		Revision as of 00:25, 24 August 2024
Line 347:		Line 347:
	'''Community''': GRASS GIS - OSGeo		'''Community''': GRASS GIS - OSGeo
	== Abstract ==		== Abstract ==
	This project introduces a new ~~subpackage~~ for GRASS GIS, <code>i.eodag</code>, which integrates the EODAG API with GRASS GIS, serving as a central portal for downloading ~~geospatial scenes~~. ~~Previously~~, downloading ~~these scenes~~ required three separate modules: <code>i.landsat.download</code>, <code>i.sentinel.download</code>, and <code>i.modis.download</code>. Each module relied on different APIs, leading to significant maintenance efforts. The <code>i.eodag</code> ~~subpackage~~ simplifies this process by providing a unified source for all geospatial products, reducing the maintenance burden to a single API—the EODAG API. Additionally, <code>i.landsat.download</code> and <code>i.sentinel.download</code> have been reimplemented ~~using the new~~ <code>i.eodag</code> ~~subpackage~~, with their logic abstracted to improve modularity and maintainability.		This project introduces a new extension for GRASS GIS, <code>i.eodag</code>, which integrates the EODAG API with GRASS GIS, serving as a central portal for downloading Earth Observation data and products. Before this project, downloading such data required three separate modules: <code>i.landsat.download</code>, <code>i.sentinel.download</code>, and <code>i.modis.download</code>. Each module relied on different APIs, leading to significant maintenance efforts. The <code>i.eodag</code> extension simplifies this process by providing a unified source for all geospatial products, reducing the maintenance burden to a single API—the [https://eodag.readthedocs.io EODAG API]. Additionally, <code>i.landsat.download</code> and <code>i.sentinel.download</code> have been reimplemented to internally use <code>i.eodag</code>, with their logic abstracted to improve modularity and maintainability.

	== The state of integration BEFORE the start of GSoC ==		== The state of integration BEFORE the start of GSoC ==

Hamed5001: /* Conclusion */

2024-08-23T17:26:49Z

Conclusion

← Older revision		Revision as of 17:26, 23 August 2024
Line 388:		Line 388:
	To wrap it up, I’m glad to have successfully achieved the three main targets of the project, with the invaluable support from my mentors!		To wrap it up, I’m glad to have successfully achieved the three main targets of the project, with the invaluable support from my mentors!
	* Integrated '''EODAG API''' to GRASS GIS through new the <code>i.eodag</code> sub-package, serving as a unified portal for downloading a variety of geospatial data products.		* Integrated '''EODAG API''' to GRASS GIS through new the <code>i.eodag</code> sub-package, serving as a unified portal for downloading a variety of geospatial data products.
	* Reimplemented <code>i.landsat.download</code> using ~~the~~ the new <code>i.eodag</code> sub-package.		* Reimplemented <code>i.landsat.download</code> using the new <code>i.eodag</code> sub-package.
	* Reimplemented <code>i.sentinel.download</code> using ~~the~~ the new <code>i.eodag</code> sub-package.		* Reimplemented <code>i.sentinel.download</code> using the new <code>i.eodag</code> sub-package.

	I am incredibly grateful for the support I received this summer and for the opportunity to contribute to GRASS GIS through GSoC 2024. This experience has been rewarding, and I am excited to continue enhancing <code>i.eodag</code>. While we've made significant progress, there are still exciting opportunities for further improvement, as outlined in the Future Work section below.		I am incredibly grateful for the support I received this summer and for the opportunity to contribute to GRASS GIS through GSoC 2024. This experience has been rewarding, and I am excited to continue enhancing <code>i.eodag</code>. While we've made significant progress, there are still exciting opportunities for further improvement, as outlined in the Future Work section below.

← Older revision		Revision as of 00:34, 24 August 2024
Line 360:		Line 360:

	== The state of integration AFTER GSoC ==		== The state of integration AFTER GSoC ==
	#~~Implemented~~ a new module, <code>i.eodag</code>, which serves as a unified portal for downloading a variety of ~~geospatial~~ data products, extending beyond just Sentinel and Landsat. This integration allows GRASS GIS to rely on a single API, '''EODAG''', instead of managing multiple APIs like '''[https://github.com/yannforget/landsatxplore ~~Landsatxplore~~]''', '''[https://github.com/sentinelsat/sentinelsat ~~Sentinelsat~~]''', and '''[https://cloud.google.com/solutions/geospatial?hl=en Google Cloud Storage]'''.		#I implemented a new module, [https://www.google.com/url?q=https://grass.osgeo.org/grass84/manuals/addons/i.eodag.html&sa=D&source=docs&ust=1724462440593459&usg=AOvVaw3a3jNg0Qah1KVNbas67qFB <code>i.eodag</code>], which serves as a unified portal for downloading a variety of Earth Observation data products, extending beyond just Sentinel and Landsat. This integration allows GRASS GIS to rely on a single API, '''EODAG''', instead of managing multiple APIs like '''[https://github.com/yannforget/landsatxplore landsatxplore]''', '''[https://github.com/sentinelsat/sentinelsat sentinelsat]''', and '''[https://cloud.google.com/solutions/geospatial?hl=en Google Cloud Storage]'''. Some of the features implemented in i.eodag:
	##Supports generic searching without specifying a provider.		##Supports generic searching without specifying a provider.
	##Supports limiting searching to a single provider.		##Supports limiting searching to a single provider.
Line 369:		Line 369:
	##Saves search results in a '''GeoJSON''' file, including all scenes' metadata.		##Saves search results in a '''GeoJSON''' file, including all scenes' metadata.
	##Reads previously generated '''GeoJSON''' files for further filtering.		##Reads previously generated '''GeoJSON''' files for further filtering.
	##Downloads scenes saved in a GeoJSON file.		##Downloads scenes saved in a '''GeoJSON''' file.
	##Reads a text file with a list of IDs and searches for them.		##Reads a text file with a list of IDs and searches for them.
	##Supports '''TOTP''' Authentication for [https://creodias.eu/ Creodias] provider.		##Supports '''TOTP''' Authentication for [https://creodias.eu/ Creodias] provider.
Line 378:		Line 378:
	##Supports querying/filtering with both '''OR''' and '''AND''' relations, as well as '''relational operators''' (e.g., 'lt', 'gt', etc.).		##Supports querying/filtering with both '''OR''' and '''AND''' relations, as well as '''relational operators''' (e.g., 'lt', 'gt', etc.).
	##Supports printing of relevant EODAG metadata e.g. available providers, available products and their corresponding queryables (aka. products' metadata).		##Supports printing of relevant EODAG metadata e.g. available providers, available products and their corresponding queryables (aka. products' metadata).
	##For the full list of features and options, please refer to the [https://grass.osgeo.org/grass84/manuals/addons/i.eodag.html i.eodag manual.]		##For the full list of features and options, as well as usage examples, please refer to the [https://grass.osgeo.org/grass84/manuals/addons/i.eodag.html i.eodag manual.]
	#~~Added~~ a comprehensive test suite for <code>i.eodag</code>, featuring resilient and robust test cases. These tests are designed to be flexible, accommodating different EODAG versions and future developments.		#I added a comprehensive test suite for <code>i.eodag</code>, featuring resilient and robust test cases. These tests are designed to be flexible, accommodating different EODAG versions and future developments.
	#~~Reimplemented~~ <code>i.landsat.download</code> using <code>i.eodag</code>, while maintaining backward compatibility. The reimplementation abstracted the logic, making the ~~modules~~ more modular and extensible for future updates.		#I reimplemented <code>i.landsat.download</code> using <code>i.eodag</code>, while maintaining backward compatibility. The reimplementation abstracted the logic, making the tool more modular and extensible for future updates.
	#~~Reimplemented~~ <code>i.sentinel.download</code> using ~~the new~~ <code>i.eodag</code> ~~subpackage~~, reducing the need for maintaining multiple APIs. This reimplementation simplified and abstracted the logic effectively. ~~Some~~		#I reimplemented <code>i.sentinel.download</code> using <code>i.eodag</code>, reducing the need for maintaining multiple APIs. This reimplementation simplified and abstracted the logic effectively.

	== Conclusion ==		== Conclusion ==