# Difference between revisions of "Limiting precision in vector nodes"

(+URL GeographicLib: Support for high precision arithmetic) |
|||

Line 13: | Line 13: | ||

* [http://www.ibm.com/developerworks/library/j-jtp0114/ Tricks and traps with floating point and decimal numbers] by Brian Goetz | * [http://www.ibm.com/developerworks/library/j-jtp0114/ Tricks and traps with floating point and decimal numbers] by Brian Goetz | ||

* [http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html What Every Computer Scientist Should Know About Floating-Point Arithmetic] by David Goldberg | * [http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html What Every Computer Scientist Should Know About Floating-Point Arithmetic] by David Goldberg | ||

− | + | * [https://geographiclib.sourceforge.io/html/highprec.html GeographicLib: Support for high precision arithmetic] | |

[[Category: FAQ]] | [[Category: FAQ]] | ||

[[Category: Vector]] | [[Category: Vector]] |

## Latest revision as of 15:45, 23 April 2018

There are specifications (e.g. the Greek Cadastral System) which ask the digitizer to create their data with (for example) three digits behind the decimal place. As a general rule for the overwhelming majority of modern computers, exact representation of IEEE floating point numbers is not guaranteed in binary data storage (see links below), but if it's what the spec says it's what has to be attempted anyway.

Here is an example using the Spearfish sample dataset in GRASS 6.4 to reduce the node coordinates of vector polygons (*areas*) to three decimal digit precision:

v.out.ascii in=quads format=standard dp=3 | v.in.ascii out=quads_dp3 format=standard

Topology *should* be preserved, but it is worth paying close attention to the verification and topology-building messages upon import. (in case of errors make sure that the original map did not have them too)

Any connected databases will need to be copied or reconnected. (see the v.db.connect module)