Test Suite

GRASS Test Suite

We aim at creating a comprehensive test suite for GRASS modules and libraries.

Background

See what has been done so far by Sören Gebbert and others here: Development#QA and the good [| VKT test suite]

Keep an eye on GRASS_7_ideas_collection#Complete_GRASS_Test_Suite:_see_activity_on_Test_Suite_development_page GRASS 7 ideas collection

Main picture

We plan to run unittests and integration tests for both libraries and modules. The test suite will be run after compilation, with a command like:

 $ make tests [ proj [ libs | modules | all ] ]

Options:

• proj: run the tests with a [list of] CRS, so that reprojection and map unit handling are tested.
• libs: run tests for libs only
• modules: run tests for modules only, assuming that libs are error-free
• all: run tests on libs first, then if they pass, test the modules. It's the opposite of regression tests.

Tests

Modules

Tests are targeted to single modules/library functions, as independent as possible from other GRASS modules.

Where independence is impossible, then try to run called modules first and check if they break. -- how is it possible to call a module in the test, and by that way call its test too?

Tests are in each module's folder (written in Python), and in each library's folder (written in Python using ctypes).

Framework to generate and compare grass data types: raster, vector, raster3d, general, db, icon, imagery, d.*?

wxGUI testing should be tested separately.

Automated tests on server generates HTML report. Test several platforms.

Test framework:

The test framework is something like:

• BaseTest:
  • Handles module test results codes and messages
  • Creates a HTML report from these

• RasterTest(BaseTest):
  • Raster data generator functions
  • Raster data comparison functions

• VectorTest(BaseTest):
  • ...

• ...

Using these classes, each module will have its own test class, like:

• RMapcalcTest(RasterTest):
  • generates data
  • runs r.mapcalc
  • compare results with the expected ones
  • output the results of the test as messages, to be prettified in HTML

Timeline and status

TBD

Interested people

• Sören Gebbert
• Anne Ghisla
• Add your name here