GRASS GIS Performance: Difference between revisions
(fixes and updates) |
|||
| Line 29: | Line 29: | ||
'''DBF-Backend''': GRASS 4.x - 6.x use by default the DBF backend. While there is no explicitly stated maximum number of allowed attribute columns, Web sources report a maximum '''between 128 and 1023/24'''. Trials with GRASS 6.4.2 in 2012 result in write failure if > 2000 attribute columns are used. Export to DBF-based ESRI Shapefile provides a warning if more that '''255''' attributes are used: Other software tools may ignore all further attributes, hence a maximum of '''128''' columns may be prudent. | '''DBF-Backend''': GRASS 4.x - 6.x use by default the DBF backend. While there is no explicitly stated maximum number of allowed attribute columns, Web sources report a maximum '''between 128 and 1023/24'''. Trials with GRASS 6.4.2 in 2012 result in write failure if > 2000 attribute columns are used. Export to DBF-based ESRI Shapefile provides a warning if more that '''255''' attributes are used: Other software tools may ignore all further attributes, hence a maximum of '''128''' columns may be prudent. | ||
'''SQLite-Backend''': GRASS 7.x uses by default the SQLite backend. The default maximum number of attribute columns is '''2000''' according to the [http://www.sqlite.org/limits.html specifications]. This number can be increased by compiling SQlite with changed settings. | '''SQLite-Backend''': GRASS 7.x uses by default the SQLite backend. The default maximum number of attribute columns is '''2000''' according to the [http://www.sqlite.org/limits.html#max_column specifications]. This number can be increased by compiling SQlite with changed settings. | ||
'''MySQL-Backend''': The default maximum number of attribute columns is '''4096''' according to the [http://wiki.postgresql.org/wiki/FAQ#What_is_the_maximum_size_for_a_row.2C_a_table.2C_and_a_database.3F specifications]. | '''MySQL-Backend''': The default maximum number of attribute columns is '''4096''' according to the [http://wiki.postgresql.org/wiki/FAQ#What_is_the_maximum_size_for_a_row.2C_a_table.2C_and_a_database.3F specifications]. | ||
Revision as of 15:28, 28 August 2012
GRASS GIS Performance
GRASS GIS is noted for being ready for massive data analysis. This page contains an yet incomplete collection of performance indicators.
Architecture
GRASS GIS is fully 32bit and 64bit compliant. See also the Software requirements specification.
Number of opened input files
There are only operating system constraints of the number of input files which can be opened simultaneously. Commonly the limit is 1024 files. In operating systems like Linux this limit can be overcome with the "ulimit" settings.
See also
Memory management
Due to the modular architecture of GRASS GIS the overhead is minimal.
See also
Vector management
Maximum Number of Attribute Columns
The maximum number of attribute columns of a table connected to a vector map is defined by the capabilities of the the selected database backend (set with db.connect).
DBF-Backend: GRASS 4.x - 6.x use by default the DBF backend. While there is no explicitly stated maximum number of allowed attribute columns, Web sources report a maximum between 128 and 1023/24. Trials with GRASS 6.4.2 in 2012 result in write failure if > 2000 attribute columns are used. Export to DBF-based ESRI Shapefile provides a warning if more that 255 attributes are used: Other software tools may ignore all further attributes, hence a maximum of 128 columns may be prudent.
SQLite-Backend: GRASS 7.x uses by default the SQLite backend. The default maximum number of attribute columns is 2000 according to the specifications. This number can be increased by compiling SQlite with changed settings.
MySQL-Backend: The default maximum number of attribute columns is 4096 according to the specifications.
PostgreSQL-Backend: The default maximum number of attribute columns is 250-1600 according to the specifications depending on column types.
Oracle: The default maximum number of attribute columns is 1000 according to the specifications.
Large file support
Large raster data processing
GRASS GIS 7 supports the off_t type, hence it can address an enormous amount of raster data.
See also
Some benchmarks:
- Import of ECAD 6.0 Tmean dataset: 22650 layers in single netCDF file: import takes 300 Seconds while reading file via NFS (i.e. 75 maps per second)
Large vector data processing
GRASS GIS 7 supports the off_t type, hence it can address an enormous amount of vector data. Currently multi-billion vector points have been managed (citation) without topology (since not needed). In all GRASS versions, the limit with topology is at time 2^31 - 1 (about 2 billion) features.
Parallelization
In GRASS 7, a few modules have been parallelized with openMP. However, if data can be processed in chunks, GRASS GIS can be used on clusters.
See also: