Replacement raster format - Revision history

⚠️Landa: moved to trac

2010-04-24T19:56:31Z

moved to trac

← Older revision		Revision as of 19:56, 24 April 2010
Line 1:		Line 1:
	{{~~MoveToTrac~~}}		{{MovedToTrac\|Grass7/RasterLib}}
	~~GRASS's long-standing raster format is overdue for a major overhaul.<BR>~~
	~~Below you will find some ideas and roadmaps for future work.<BR>~~
	~~The idea of this page is to collect ideas and flesh out a~~
	~~specification so that when the change occurs, all the~~
	~~components will be in place, pitfalls expected, and~~
	~~the implimentation, when it comes, quick and painless. Most~~
	~~importanly it can serve to keep interested parties informed~~
	~~and working together instead of in parallel forks.<BR>~~
	~~Any changes to the data format will necessitate a bump in~~
	~~major version number (i.e. from GRASS 6 to GRASS 7) so if~~
	~~possible changes should happen in the same development cycle,~~
	~~and relatively minor changes should be held back in~~
	~~experimental status until a major change is committed.~~

	~~__TOC__~~

	~~see also [[GRASS 7 ideas collection#Raster]]~~

	~~==== Core raster format ====~~
	~~''Lead developer: Glynn Clements''~~

	* Storage in tiles instead of by row.
	** Reasoning: Glynn said on the mailing list: "In most cases, single-level tiled storage will give you close to the same performance with a lot less complexity."
	** Function needed to check whether tiles are all null, or all the same value.
	** What tile size should be used? Could be user/program specified, or standard value of something like 64x64. If there isn't a fixed value then there should be utility program that can convert tiled rasters to different tile sizes. -
	*** However, row-oriented storage has the advantage that you can easily skip entire rows when downsampling. Using e.g. 64x1 "tiles" would provide both optimisations, but at the expense of reduced compression, as you need a pointer (8 bytes) for each tile, and the compression has to be restarted for each tile.
	~~[[User:Ferrouswheel\|Ferrouswheel]]~~
	* Merge NULL file into main data array.

	~~==== Directory structure ====~~

	* Centralize map components in "<tt>$MAPSET/raster/$MAPNAME/*</tt>" instead of many "<tt>$MAPSET/cell/$MAPNAME</tt>", etc. directories.

	~~Many library functions and modules will need to be updated.~~
	~~The GRASS 6 vector format has already been ported to this structure.~~

	* demo 2-way conversion scripts: [http://wald.intevation.org/tracker/index.php?func=detail&aid=372&group_id=21&atid=205 Gforge patch #372]

	~~==== Meta-data support ====~~

	~~The existing raster meta-data handling is rather weak.~~
	~~(currently stored in <tt>$MAPSET/hist/$MAPNAME</tt>)~~
	~~Total replacement will be the best option.~~

	~~Brad Douglas suggests:~~
	~~''It would be very advantageous to at least support metadata as specified~~
	~~in [http://www.fgdc.gov/standards/projects/FGDC-standards-projects/csdgm_rs_ex/MetadataRemoteSensingExtens.pdf FGDC-STD-012-2002].~~
	~~XML is an ideal file format.''~~

	~~Nick Lawrence suggests:~~
	~~The GIMP project is examining the concept of a general mult-layer bitmap format.~~

	~~[http://create.freedesktop.org/wiki/index.php/General_multilayered_bitmap_exchange_format OpenRaster - Create Wiki]~~
	~~[http://pippin.gimp.org/OpenRaster/ OpenRaster Sandbox]~~

	~~==== Recent discussions ====~~

	* http://www.nabble.com/-GRASS5--Raster-files-suggestion%3A-new-directory-layout-td8588651.html
	* http://lists.osgeo.org/pipermail/grass-dev/2008-April/036994.html
	* http://lists.osgeo.org/pipermail/grass-dev/2008-April/037578.html

	~~Other suggestions:~~
	* [[Time series in GRASS]] support

	~~[[Category:Development]]~~

⚠️Glynn: Disadvantages of tiled storage

2008-10-04T07:43:18Z

Disadvantages of tiled storage

← Older revision		Revision as of 07:43, 4 October 2008
Line 24:		Line 24:
	** Reasoning: Glynn said on the mailing list: "In most cases, single-level tiled storage will give you close to the same performance with a lot less complexity."		** Reasoning: Glynn said on the mailing list: "In most cases, single-level tiled storage will give you close to the same performance with a lot less complexity."
	** Function needed to check whether tiles are all null, or all the same value.		** Function needed to check whether tiles are all null, or all the same value.
	** What tile size should be used? Could be user/program specified, or standard value of something like 64x64. If there isn't a fixed value then there should be utility program that can convert tiled rasters to different tile sizes. - [[User:Ferrouswheel\|Ferrouswheel]]		** What tile size should be used? Could be user/program specified, or standard value of something like 64x64. If there isn't a fixed value then there should be utility program that can convert tiled rasters to different tile sizes. -
			*** However, row-oriented storage has the advantage that you can easily skip entire rows when downsampling. Using e.g. 64x1 "tiles" would provide both optimisations, but at the expense of reduced compression, as you need a pointer (8 bytes) for each tile, and the compression has to be restarted for each tile.
			[[User:Ferrouswheel\|Ferrouswheel]]
	* Merge NULL file into main data array.		* Merge NULL file into main data array.

⚠️Landa: /* Recent discussions */ + link to ML

2008-04-28T13:50:21Z

Recent discussions: + link to ML

← Older revision		Revision as of 13:50, 28 April 2008
Line 57:		Line 57:
	* http://www.nabble.com/-GRASS5--Raster-files-suggestion%3A-new-directory-layout-td8588651.html		* http://www.nabble.com/-GRASS5--Raster-files-suggestion%3A-new-directory-layout-td8588651.html
	* http://lists.osgeo.org/pipermail/grass-dev/2008-April/036994.html		* http://lists.osgeo.org/pipermail/grass-dev/2008-April/036994.html
			* http://lists.osgeo.org/pipermail/grass-dev/2008-April/037578.html

	Other suggestions:		Other suggestions:

Neteler: +Recent discussions

2008-04-22T14:27:53Z

+Recent discussions

← Older revision		Revision as of 14:27, 22 April 2008
Line 53:		Line 53:
	[http://pippin.gimp.org/OpenRaster/ OpenRaster Sandbox]		[http://pippin.gimp.org/OpenRaster/ OpenRaster Sandbox]

			==== Recent discussions ====

			* http://www.nabble.com/-GRASS5--Raster-files-suggestion%3A-new-directory-layout-td8588651.html
			* http://lists.osgeo.org/pipermail/grass-dev/2008-April/036994.html

	Other suggestions:		Other suggestions:

⚠️Landa: +category MoveToTrac

2008-04-13T00:06:57Z

+category MoveToTrac

← Older revision		Revision as of 00:06, 13 April 2008
Line 1:		Line 1:
			{{MoveToTrac}}
	GRASS's long-standing raster format is overdue for a major overhaul.<BR>		GRASS's long-standing raster format is overdue for a major overhaul.<BR>
	Below you will find some ideas and roadmaps for future work.<BR>		Below you will find some ideas and roadmaps for future work.<BR>

⚠️HamishBowman: /* Directory structure */ convert script

2007-10-24T08:34:48Z

Directory structure: convert script

← Older revision		Revision as of 08:34, 24 October 2007
Line 28:		Line 28:
	==== Directory structure ====		==== Directory structure ====

	* Centralize map components in "<tt>$MAPSET/raster/$MAPNAME/*</tt>" instead of many "<tt>$MAPSET/cell/$MAPNAME</tt>",etc. directories~~.<BR><BR>Many library functions and modules will need to be updated.<BR>The GRASS 6 vector format has already been ported to this structure~~.		* Centralize map components in "<tt>$MAPSET/raster/$MAPNAME/*</tt>" instead of many "<tt>$MAPSET/cell/$MAPNAME</tt>", etc. directories.

			Many library functions and modules will need to be updated.
			The GRASS 6 vector format has already been ported to this structure.

			* demo 2-way conversion scripts: [http://wald.intevation.org/tracker/index.php?func=detail&aid=372&group_id=21&atid=205 Gforge patch #372]

	==== Meta-data support ====		==== Meta-data support ====

Neteler at 19:48, 22 October 2007

2007-10-22T19:48:23Z

← Older revision		Revision as of 19:48, 22 October 2007
Line 14:		Line 14:

	__TOC__		__TOC__

			see also [[GRASS 7 ideas collection#Raster]]

	==== Core raster format ====		==== Core raster format ====
Line 46:		Line 48:
	[http://pippin.gimp.org/OpenRaster/ OpenRaster Sandbox]		[http://pippin.gimp.org/OpenRaster/ OpenRaster Sandbox]


			Other suggestions:
			* [[Time series in GRASS]] support

	[[Category:Development]]		[[Category:Development]]

⚠️Nick Lawrence at 23:49, 19 April 2007

2007-04-19T23:49:43Z

← Older revision		Revision as of 23:49, 19 April 2007
Line 39:		Line 39:
	in [http://www.fgdc.gov/standards/projects/FGDC-standards-projects/csdgm_rs_ex/MetadataRemoteSensingExtens.pdf FGDC-STD-012-2002].		in [http://www.fgdc.gov/standards/projects/FGDC-standards-projects/csdgm_rs_ex/MetadataRemoteSensingExtens.pdf FGDC-STD-012-2002].
	XML is an ideal file format.''		XML is an ideal file format.''

			Nick Lawrence suggests:
			The GIMP project is examining the concept of a general mult-layer bitmap format.

			[http://create.freedesktop.org/wiki/index.php/General_multilayered_bitmap_exchange_format OpenRaster - Create Wiki]
			[http://pippin.gimp.org/OpenRaster/ OpenRaster Sandbox]


	[[Category:Development]]		[[Category:Development]]

Neteler: +cat

2006-06-16T19:51:07Z

+cat

← Older revision		Revision as of 19:51, 16 June 2006
Line 39:		Line 39:
	in [http://www.fgdc.gov/standards/projects/FGDC-standards-projects/csdgm_rs_ex/MetadataRemoteSensingExtens.pdf FGDC-STD-012-2002].		in [http://www.fgdc.gov/standards/projects/FGDC-standards-projects/csdgm_rs_ex/MetadataRemoteSensingExtens.pdf FGDC-STD-012-2002].
	XML is an ideal file format.''		XML is an ideal file format.''


			[[Category:Development]]

⚠️Ferrouswheel: /* Core raster format */

2006-06-16T02:51:53Z

Core raster format

← Older revision		Revision as of 02:51, 16 June 2006
Line 21:		Line 21:
	** Reasoning: Glynn said on the mailing list: "In most cases, single-level tiled storage will give you close to the same performance with a lot less complexity."		** Reasoning: Glynn said on the mailing list: "In most cases, single-level tiled storage will give you close to the same performance with a lot less complexity."
	** Function needed to check whether tiles are all null, or all the same value.		** Function needed to check whether tiles are all null, or all the same value.
	** What tile size should be used? Could be user/program specified, or standard value of something like 64x64. If there isn't a fixed value then there should be utility program that can convert tiled rasters to different tile sizes.		** What tile size should be used? Could be user/program specified, or standard value of something like 64x64. If there isn't a fixed value then there should be utility program that can convert tiled rasters to different tile sizes. - [[User:Ferrouswheel\|Ferrouswheel]]
	** [[User:Ferrouswheel\|Ferrouswheel]]
	* Merge NULL file into main data array.		* Merge NULL file into main data array.