GENESIS: Documentation

Related Documentation:

Neurospaces Tester

The Neurospaces tester framework provides a general purpose framework for regressing testing.

It includes:

Despite the wealth of functionality, using and configuring the tester is quite simple (see below for more). The latest version of the tester framework can be found in the DeveloperPackage. The same framework is used for regression testing of all GENESIS related software.

The Neurospaces tester is written in Perl. Knowledge of Perl is not necessary to be able to work with the tester. Basic knowledge of Perl might be handy, and is now explained.

Tester Calling

The most convenient way to call the tester is by doing “make check”. For GENESIS software packages, this calls neurospaces_harness, a Perl script that runs all the tester specifications found in tests/specifications.

Tester Configuration

The Neurospaces tester needs to know certain things about your machine before tests can be executed. These things are in the tester configuration file, named tests.config, located in your current directory.

It is normal that a software package has multiple tests.config files, e.g. in several directories.

The tests.config file defines a Perl hash (we plan to also support JSON). In the test specifications, this hash is available in the variable main::config. So to access the core_directory entry in the hash, you have


Here is an example script:

#!/usr/bin/perl -w  
use strict;  
my $package_name = "heccer";  
my $package_label = "build-8";  
my $package_version = "build-8-0";  
my $monotone_id = ‘mtn automate get_current_revision_id‘;  
chomp $monotone_id;  
my $config  
    = {  
         core_directory => ’./’,  
         description => ’Configure the tester when run from this directory’,  
         outputs_dir => ’./tests/html’,  
         package => {  
            label => $package_label,  
            name => $package_name,  
            version => $package_version,  
            version_control_id => $monotone_id,  
         tests_directory => ’./tests/specifications’,  
return $config;


Tester Specifications

A tester specification is contained in a file with a name that has a suffix of .t. The tester specification tells the tester what application to run, with what command line, command line options, and under what environment. It is possible to change directory before the application gets executed, or to change to a chrooted environment.

After running the application, the tester communicates with the application over stdin/stdout/stderr (general socket communication under development). The output read from stdout/sterr is checked with expected output (literal output or regular expressions), and mismatches are reported.

Multiple writes to and reads from the application are possible.

If, for whatever reason, the same application is started in more than one test specification, the running application is ‘recycled’ (unless the side_effects option is used, see below).

If there was configuration in effect for an application specific library, the library is checked for changes after a test has been performed. Changes to the library are not allowed.

Test Specifications

A test specification is a file with Perl code that returns a Perl hash (as noted above, we plan to also support JSON):

#!/usr/bin/perl -w  
use strict;  
my $test  
    = {  
. . .  
. . .  
return $test;  

The content of the hash ref encodes all the tests in the specification. The following example is taken from the Model Container software package:

#!/usr/bin/perl -w  
use strict;  
my $test  
    = {  
         command_definitions => [  
            arguments => [  
            command => ’./neurospacesparse’,  
            command_tests => [  
                  description => "Is neurospaces startup successful ?",  
                  read => ’neurospaces ’,  
                  timeout => 3,  
                  write => undef,  
                  description => "Does the version information match with \  
                     model-container-build-8 ?",  
                  read => "model-container-build-8",  
                  write => "version",  
               description => "check version information",  
            description => "run-time versioning",  
            name => ’version.t’,  
return $test;

A test specification contains the following keys/values:

Command Definitions

  1. Arguments and command keys are the shell command line to run, and to connect stdin and stdout.
  2. command_tests See below.
  3. description Gives the purpose of the test in a few words (not a paragraph).

Command Tests

The description key is mandatory, the other keys are optional.

  1. description Describes the purpose of the test in a few words (not an entire paragraph).
  2. read Expected output from the application.
  3. write Writes output to stdin of the application before any output is generated.
  4. comment A text comment.
  5. side_effects Avoids recycling of application commands, even if the options are the same.
  6. preparation and reparation Two commands to bring the application environment into a particalur state and restore the previous state after the test has been performed. Do not use this to change the application state, use the write key for that.
  7. disabled Explains why a test has been disabled. Omitting this key or setting this key to a false value or the emtpy string, enables the test.

Tester HTML


Tester Tutorial