3. Regression Testing

The `testing/' directory contains several files to support regression testing, as well as a subdirectory `testing/tests/'. The significant files in this subdirectory have the extension `.tst'. These files are parsed line-by-line, with the first word of a line indicating the action to be taken. Note the first word is case-sensitive. The defined actions are as follows:

path NEWPATH

Add NEWPATH to the search path for modules.

module MODULE

Execute subsequent queries against module MODULE, until another occurrence of module.

expect NUMBER

Expect at least NUMBER solutions to the next query. Fail the test if fewer are presented.

maximum NUMBER

Expect at most NUMBER solutions to the next query. Fail the test if more are presented.

query

The next line of the file contains a query to execute. Many queries can be presented in one test file. Note that each query must be terminated by a fullstop (.).

Thus a test file could contain the following:

path ../examples/utility
module lists
query
append (2::nil) (3::nil) L.
expect 5
maximum 5
query
append L1 L2 [1,2,3,4].
module reverse
query
reverse (1::2::3::4::nil) L.

Tests are run by the shell script `testing/runtest'. The script is invoked as testing/runtest <testfile> [check | commit]. <testfile> is the `.tst' file containing the queries to be executed. If the second parameter is check, then the test will be run, and its status reported. If the second parameter is commit, then the results of executing the queries in the test file will be considered correct, and will overwrite any previous results.

The top-level makefile contains a target, tests, which will execute all tests in the testing/tests/ subdirectory. To use this target, type make tests in the top-level Teyjus directory.

In the process of running a test, the test file is interpreted, and the results of the presented queries are collected. The test's result file (with extension `.res') is then referenced. If the result file does not yet exist, the new results are stored in the result file. If the result file already exists, the new results are compared to the result file. The test is successful only if no differences are found.

If a test is unsuccessful, but the new results are correct, use the commit option to force the testing system to consider the results correct, and record them as such.

This document was generated by Gopalan Nadathur on October, 20 2007 using texi2html 1.76.