Test Suite - Revision history

Neteler:

2014-12-27T21:26:33Z

<strike>

← Older revision Revision as of 21:26, 27 December 2014
Line 1: Line 1:

''This page is ~~depreciated~~ and serves just as a historical reference. Visit [http://grass.osgeo.org/grass71/manuals/libpython/gunittest_testing.html documentation about testing] for current information.''
''This page is deprecated and serves just as a historical reference.''

'''Visit the [http://grass.osgeo.org/grass71/manuals/libpython/gunittest_testing.html gunittest documentation about testing] for current information.'''

__TOC__
__TOC__

<strike>

= GRASS GIS 7 Test Suite =
= GRASS GIS 7 Test Suite =

Line 526: Line 529:

* [[User:Pietro|Pietro Zambelli]]
* [[User:Pietro|Pietro Zambelli]]

* This page does not reflect the current state. If you are interested, read the documentation, try to run some tests, try to write some tests, and write to grass-dev mailing list.
* This page does not reflect the current state. If you are interested, read the documentation, try to run some tests, try to write some tests, and write to grass-dev mailing list.

</strike>

[[Category: Development]]
[[Category: Development]]

[[Category: Testing]]
[[Category: Testing]]

⚠️Wenzeslaus: obsolete page, link documentation

2014-12-27T14:30:20Z

obsolete page, link documentation

← Older revision Revision as of 14:30, 27 December 2014
Line 1: Line 1:

''This page is depreciated and serves just as a historical reference. Visit [http://grass.osgeo.org/grass71/manuals/libpython/gunittest_testing.html documentation about testing] for current information.''

__TOC__
__TOC__

Line 4: Line 6:

We aim at creating a comprehensive test suite for GRASS GIS 7 modules and libraries.
We aim at creating a comprehensive test suite for GRASS GIS 7 modules and libraries.

~~New 2014: see http://trac.osgeo.org/grass/wiki/GSoC/2014/TestingFrameworkForGRASS~~

== Background ==
== Background ==

Line 517: Line 517:

The Make system should execute the Python test framework main Python file in each Module or Library directory.
The Make system should execute the Python test framework main Python file in each Module or Library directory.

I guess test specific entries must be added to the modules and library Makefiles?
I guess test specific entries must be added to the modules and library Makefiles?

~~== Timeline and status ==~~

~~{|class="wikitable sortable"~~

~~|'''Task''' || '''Status''' || '''Developer(s)'''~~

|-

~~| Implementation of modules' tests || Progress || Sören~~

|-

~~| Implementation of r3.* tests || Progress || Sören~~

|-

~~| Implementation of test framework || Starting || Anne~~

|-

~~| Make integration || To be done || -~~

|-

|}

== Interested people ==
== Interested people ==

Line 540: Line 525:

* [[User:Wenzeslaus|Vaclav Petras]]
* [[User:Wenzeslaus|Vaclav Petras]]

* [[User:Pietro|Pietro Zambelli]]
* [[User:Pietro|Pietro Zambelli]]

* ~~Add your name here~~
* This page does not reflect the current state. If you are interested, read the documentation, try to run some tests, try to write some tests, and write to grass-dev mailing list.

[[Category: Development]]
[[Category: Development]]

[[Category: Testing]]
[[Category: Testing]]

Neteler: +New 2014: see http://trac.osgeo.org/grass/wiki/GSoC/2014/TestingFrameworkForGRASS

2014-09-25T07:56:35Z

+New 2014: see http://trac.osgeo.org/grass/wiki/GSoC/2014/TestingFrameworkForGRASS

← Older revision Revision as of 07:56, 25 September 2014
Line 4: Line 4:

We aim at creating a comprehensive test suite for GRASS GIS 7 modules and libraries.
We aim at creating a comprehensive test suite for GRASS GIS 7 modules and libraries.

New 2014: see http://trac.osgeo.org/grass/wiki/GSoC/2014/TestingFrameworkForGRASS

== Background ==
== Background ==

⚠️Pietro: add my name to the list of interested people

2013-10-29T09:53:09Z

add my name to the list of interested people

← Older revision Revision as of 09:53, 29 October 2013
Line 537: Line 537:

* [[User:Landa|Martin Landa]]
* [[User:Landa|Martin Landa]]

* [[User:Wenzeslaus|Vaclav Petras]]
* [[User:Wenzeslaus|Vaclav Petras]]

* [[User:Pietro|Pietro Zambelli]]

* Add your name here
* Add your name here

[[Category: Development]]
[[Category: Development]]

[[Category: Testing]]
[[Category: Testing]]

⚠️Wenzeslaus: /* Interested people */

2013-10-28T20:16:19Z

Interested people

← Older revision Revision as of 20:16, 28 October 2013
Line 536: Line 536:

* [[User:AnneGhisla| Anne Ghisla]]
* [[User:AnneGhisla| Anne Ghisla]]

* [[User:Landa|Martin Landa]]
* [[User:Landa|Martin Landa]]

* [[User:Wenzeslaus|Vaclav Petras]]

* Add your name here
* Add your name here

[[Category: Development]]
[[Category: Development]]

[[Category: Testing]]
[[Category: Testing]]

⚠️HamishBowman: /* Background */ wikifix

2013-06-15T12:02:50Z

Background: wikifix

← Older revision Revision as of 12:02, 15 June 2013
Line 7: Line 7:

== Background ==
== Background ==

See what has been done so far by Sören Gebbert and others here: [[Development#QA]] and the good [[http://www.vtk.org/Wiki/images/b/bc/Testing.cover.txt | VKT test suite]]
See what has been done so far by Sören Gebbert and others here: [[Development#QA]] and the good [http://www.vtk.org/Wiki/images/b/bc/Testing.cover.txt VKT test suite]

Keep an eye on [[GRASS_7_ideas_collection#Complete_GRASS_Test_Suite:_see_activity_on_Test_Suite_development_page | GRASS 7 ideas collection]]
Keep an eye on [[GRASS_7_ideas_collection#Complete_GRASS_Test_Suite:_see_activity_on_Test_Suite_development_page | GRASS 7 ideas collection]]

Neteler: /* GRASS Test Suite */

2013-06-15T08:48:10Z

GRASS Test Suite

← Older revision Revision as of 08:48, 15 June 2013
Line 1: Line 1:

__TOC__
__TOC__

= GRASS Test Suite =
= GRASS GIS 7 Test Suite =

We aim at creating a comprehensive test suite for GRASS modules and libraries.
We aim at creating a comprehensive test suite for GRASS GIS 7 modules and libraries.

== Background ==
== Background ==

Line 13: Line 13:

== Main picture ==
== Main picture ==

We plan to run ~~unittests~~ and integration tests for both libraries and modules. The test suite will be run after compilation, with a command like:
We plan to run unit tests and integration tests for both libraries and modules. The test suite will be run after compilation, with a command like:

   $ make tests [ proj ]
   $ make tests [ proj ]

⚠️AnneGhisla: /* Timeline and status */ small table with progress so far

2011-07-07T12:44:11Z

Timeline and status: small table with progress so far

← Older revision Revision as of 12:44, 7 July 2011
Line 518: Line 518:

== Timeline and status ==
== Timeline and status ==

~~TBD~~
{|class="wikitable sortable"

|'''Task''' || '''Status''' || '''Developer(s)'''

|-

| Implementation of modules' tests || Progress || Sören

|-

| Implementation of r3.* tests || Progress || Sören

|-

| Implementation of test framework || Starting || Anne

|-

| Make integration || To be done || -

|-

|}

== Interested people ==
== Interested people ==

⚠️Huhabla: /* Main picture */

2011-07-05T09:51:18Z

Main picture

← Older revision Revision as of 09:51, 5 July 2011
Line 22: Line 22:

Tests are executed recursively. If the "make tests" command is executed in the root source folder all libraries and modules are tested. To test only the libraries you need to switch in the '''lib''' directory and execute "make tests". If you want to test only raster modules switch into the '''raster''' directory and run "make tests", same for other modules directories. To test a single module switch into the module directory and run "make tests".
Tests are executed recursively. If the "make tests" command is executed in the root source folder all libraries and modules are tested. To test only the libraries you need to switch in the '''lib''' directory and execute "make tests". If you want to test only raster modules switch into the '''raster''' directory and run "make tests", same for other modules directories. To test a single module switch into the module directory and run "make tests".

Testing should work in parallel, so that make -j6 tests will execute as maximum 6 tests in parallel. The framework should take care that parallel testing test runs can be performed.

== Tests ==
== Tests ==

⚠️Huhabla at 09:26, 5 July 2011

2011-07-05T09:26:03Z

← Older revision Revision as of 09:26, 5 July 2011
Line 64: Line 64:

*** In case the precision is not provided the default behavior of the export module is used
*** In case the precision is not provided the default behavior of the export module is used

Reference data for validation must be located in the module/library directory. The reference data names must be identical with the generated data (files, maps, ...) except that reference data always has a '''.ref''' suffix.
Reference data for validation must be located in the module/library reference directory. The reference data names must be identical with the generated data (files, maps, ...) except that reference data always has a '''.ref''' suffix.

Tests are in each module's and in each library's folder. To test library functions special modules must be implemented. Library test modules test the library functions directly and should be written in C. Have a look at the test directories in the g3d, gpde and gmath libraries of grass7.
Tests are in each module's and in each library's folder. To test library functions special modules must be implemented. Library test modules test the library functions directly and should be written in C. Have a look at the test directories in the g3d, gpde and gmath libraries of grass7.

Line 337: Line 337:

* Creation of a test mapset for each test case in a specific test location located in the grass sources
* Creation of a test mapset for each test case in a specific test location located in the grass sources

* Setting the environment variables to the test location and grass installation (grass environment to run modules)
* Setting the environment variables to the test location and grass installation (grass environment to run modules)

* For each test case a separate temporary directory should be created in which the test is executed. After testing the temporary directory get removed. The temporary directory should be located in the temporary mapset which will be automatically removed after testing.

* Parsing and interpretation and execution of test scripts
* Parsing and interpretation and execution of test scripts

* Support of several test scripts in a single module directory
* Support of several test scripts in a single module directory

Line 394: Line 395:

   def ExecuteAll(self):
   def ExecuteAll(self):

   """
   """

* Create a temporary test directory in the temporary mapset

   * Calls ExecuteTest or ExecutePreProcess for each entry in the execution order list
   * Calls ExecuteTest or ExecutePreProcess for each entry in the execution order list

   * Summarize the HTML and text summary content for each entry in the list
   * Summarize the HTML and text summary content for each entry in the list

* Clean up the temporary directories generate for each command

   """
   """

   pass
   pass

← Older revision		Revision as of 21:26, 27 December 2014
Line 1:		Line 1:
	''This page is ~~depreciated~~ and serves just as a historical reference. Visit [http://grass.osgeo.org/grass71/manuals/libpython/gunittest_testing.html documentation about testing] for current information.''		''This page is deprecated and serves just as a historical reference.''

			'''Visit the [http://grass.osgeo.org/grass71/manuals/libpython/gunittest_testing.html gunittest documentation about testing] for current information.'''

	__TOC__		__TOC__

			<strike>
	= GRASS GIS 7 Test Suite =		= GRASS GIS 7 Test Suite =

Line 526:		Line 529:
	* [[User:Pietro\|Pietro Zambelli]]		* [[User:Pietro\|Pietro Zambelli]]
	* This page does not reflect the current state. If you are interested, read the documentation, try to run some tests, try to write some tests, and write to grass-dev mailing list.		* This page does not reflect the current state. If you are interested, read the documentation, try to run some tests, try to write some tests, and write to grass-dev mailing list.

			</strike>

	[[Category: Development]]		[[Category: Development]]
	[[Category: Testing]]		[[Category: Testing]]

← Older revision		Revision as of 14:30, 27 December 2014
Line 1:		Line 1:
			''This page is depreciated and serves just as a historical reference. Visit [http://grass.osgeo.org/grass71/manuals/libpython/gunittest_testing.html documentation about testing] for current information.''

	__TOC__		__TOC__

Line 4:		Line 6:

	We aim at creating a comprehensive test suite for GRASS GIS 7 modules and libraries.		We aim at creating a comprehensive test suite for GRASS GIS 7 modules and libraries.

	~~New 2014: see http://trac.osgeo.org/grass/wiki/GSoC/2014/TestingFrameworkForGRASS~~

	== Background ==		== Background ==
Line 517:		Line 517:
	The Make system should execute the Python test framework main Python file in each Module or Library directory.		The Make system should execute the Python test framework main Python file in each Module or Library directory.
	I guess test specific entries must be added to the modules and library Makefiles?		I guess test specific entries must be added to the modules and library Makefiles?

	~~== Timeline and status ==~~

	~~{\|class="wikitable sortable"~~
	~~\|'''Task''' \|\| '''Status''' \|\| '''Developer(s)'''~~
	\|-
	~~\| Implementation of modules' tests \|\| Progress \|\| Sören~~
	\|-
	~~\| Implementation of r3.* tests \|\| Progress \|\| Sören~~
	\|-
	~~\| Implementation of test framework \|\| Starting \|\| Anne~~
	\|-
	~~\| Make integration \|\| To be done \|\| -~~
	\|-
	\|}

	== Interested people ==		== Interested people ==
Line 540:		Line 525:
	* [[User:Wenzeslaus\|Vaclav Petras]]		* [[User:Wenzeslaus\|Vaclav Petras]]
	* [[User:Pietro\|Pietro Zambelli]]		* [[User:Pietro\|Pietro Zambelli]]
	* ~~Add your name here~~		* This page does not reflect the current state. If you are interested, read the documentation, try to run some tests, try to write some tests, and write to grass-dev mailing list.

	[[Category: Development]]		[[Category: Development]]
	[[Category: Testing]]		[[Category: Testing]]

← Older revision		Revision as of 07:56, 25 September 2014
Line 4:		Line 4:

	We aim at creating a comprehensive test suite for GRASS GIS 7 modules and libraries.		We aim at creating a comprehensive test suite for GRASS GIS 7 modules and libraries.

			New 2014: see http://trac.osgeo.org/grass/wiki/GSoC/2014/TestingFrameworkForGRASS

	== Background ==		== Background ==

← Older revision		Revision as of 09:53, 29 October 2013
Line 537:		Line 537:
	* [[User:Landa\|Martin Landa]]		* [[User:Landa\|Martin Landa]]
	* [[User:Wenzeslaus\|Vaclav Petras]]		* [[User:Wenzeslaus\|Vaclav Petras]]
			* [[User:Pietro\|Pietro Zambelli]]
	* Add your name here		* Add your name here

	[[Category: Development]]		[[Category: Development]]
	[[Category: Testing]]		[[Category: Testing]]

← Older revision		Revision as of 20:16, 28 October 2013
Line 536:		Line 536:
	* [[User:AnneGhisla\| Anne Ghisla]]		* [[User:AnneGhisla\| Anne Ghisla]]
	* [[User:Landa\|Martin Landa]]		* [[User:Landa\|Martin Landa]]
			* [[User:Wenzeslaus\|Vaclav Petras]]
	* Add your name here		* Add your name here

	[[Category: Development]]		[[Category: Development]]
	[[Category: Testing]]		[[Category: Testing]]

← Older revision		Revision as of 12:02, 15 June 2013
Line 7:		Line 7:
	== Background ==		== Background ==

	See what has been done so far by Sören Gebbert and others here: [[Development#QA]] and the good [[http://www.vtk.org/Wiki/images/b/bc/Testing.cover.txt \| VKT test suite]]		See what has been done so far by Sören Gebbert and others here: [[Development#QA]] and the good [http://www.vtk.org/Wiki/images/b/bc/Testing.cover.txt VKT test suite]

	Keep an eye on [[GRASS_7_ideas_collection#Complete_GRASS_Test_Suite:_see_activity_on_Test_Suite_development_page \| GRASS 7 ideas collection]]		Keep an eye on [[GRASS_7_ideas_collection#Complete_GRASS_Test_Suite:_see_activity_on_Test_Suite_development_page \| GRASS 7 ideas collection]]

← Older revision		Revision as of 08:48, 15 June 2013
Line 1:		Line 1:
	__TOC__		__TOC__

	= GRASS Test Suite =		= GRASS GIS 7 Test Suite =

	We aim at creating a comprehensive test suite for GRASS modules and libraries.		We aim at creating a comprehensive test suite for GRASS GIS 7 modules and libraries.

	== Background ==		== Background ==
Line 13:		Line 13:
	== Main picture ==		== Main picture ==

	We plan to run ~~unittests~~ and integration tests for both libraries and modules. The test suite will be run after compilation, with a command like:		We plan to run unit tests and integration tests for both libraries and modules. The test suite will be run after compilation, with a command like:

	$ make tests [ proj ]		$ make tests [ proj ]

← Older revision		Revision as of 12:44, 7 July 2011
Line 518:		Line 518:
	== Timeline and status ==		== Timeline and status ==

	~~TBD~~		{\|class="wikitable sortable"
			\|'''Task''' \|\| '''Status''' \|\| '''Developer(s)'''
			\|-
			\| Implementation of modules' tests \|\| Progress \|\| Sören
			\|-
			\| Implementation of r3.* tests \|\| Progress \|\| Sören
			\|-
			\| Implementation of test framework \|\| Starting \|\| Anne
			\|-
			\| Make integration \|\| To be done \|\| -
			\|-
			\|}

	== Interested people ==		== Interested people ==

← Older revision		Revision as of 09:51, 5 July 2011
Line 22:		Line 22:

	Tests are executed recursively. If the "make tests" command is executed in the root source folder all libraries and modules are tested. To test only the libraries you need to switch in the '''lib''' directory and execute "make tests". If you want to test only raster modules switch into the '''raster''' directory and run "make tests", same for other modules directories. To test a single module switch into the module directory and run "make tests".		Tests are executed recursively. If the "make tests" command is executed in the root source folder all libraries and modules are tested. To test only the libraries you need to switch in the '''lib''' directory and execute "make tests". If you want to test only raster modules switch into the '''raster''' directory and run "make tests", same for other modules directories. To test a single module switch into the module directory and run "make tests".

			Testing should work in parallel, so that make -j6 tests will execute as maximum 6 tests in parallel. The framework should take care that parallel testing test runs can be performed.

	== Tests ==		== Tests ==

← Older revision		Revision as of 09:26, 5 July 2011
Line 64:		Line 64:
	*** In case the precision is not provided the default behavior of the export module is used		*** In case the precision is not provided the default behavior of the export module is used

	Reference data for validation must be located in the module/library directory. The reference data names must be identical with the generated data (files, maps, ...) except that reference data always has a '''.ref''' suffix.		Reference data for validation must be located in the module/library reference directory. The reference data names must be identical with the generated data (files, maps, ...) except that reference data always has a '''.ref''' suffix.

	Tests are in each module's and in each library's folder. To test library functions special modules must be implemented. Library test modules test the library functions directly and should be written in C. Have a look at the test directories in the g3d, gpde and gmath libraries of grass7.		Tests are in each module's and in each library's folder. To test library functions special modules must be implemented. Library test modules test the library functions directly and should be written in C. Have a look at the test directories in the g3d, gpde and gmath libraries of grass7.
Line 337:		Line 337:
	* Creation of a test mapset for each test case in a specific test location located in the grass sources		* Creation of a test mapset for each test case in a specific test location located in the grass sources
	* Setting the environment variables to the test location and grass installation (grass environment to run modules)		* Setting the environment variables to the test location and grass installation (grass environment to run modules)
			* For each test case a separate temporary directory should be created in which the test is executed. After testing the temporary directory get removed. The temporary directory should be located in the temporary mapset which will be automatically removed after testing.
	* Parsing and interpretation and execution of test scripts		* Parsing and interpretation and execution of test scripts
	* Support of several test scripts in a single module directory		* Support of several test scripts in a single module directory
Line 394:		Line 395:
	def ExecuteAll(self):		def ExecuteAll(self):
	"""		"""
			* Create a temporary test directory in the temporary mapset
	* Calls ExecuteTest or ExecutePreProcess for each entry in the execution order list		* Calls ExecuteTest or ExecutePreProcess for each entry in the execution order list
	* Summarize the HTML and text summary content for each entry in the list		* Summarize the HTML and text summary content for each entry in the list
			* Clean up the temporary directories generate for each command
	"""		"""
	pass		pass