GENESIS: Documentation Related Documentation:

The Developer Package Maintaining many software projects quickly becomes a lot of work. It also makes it difficult for new people to step into software development, for example, because dependency tracking between software components must be done manually. The Developer package automates the management of multiple software projects. It automates updates to local source code, synchronization between distributed repositories, compilation, testing and installation, packaging and releases, serving monotone repositories and other things. Besides monotone it also has experimental support for integration with mercurial.

The Developer package provides developer utilities that comply with GENESIS development standards. At its core is the neurospacesxbuild script used for automated software installation and maintenance on a configurable set of software packages. Because this script has many options, the most common operations are provided using simpler frontends. Other scripts are related to version identification of the software, source code documentation comments, testing, and release management. The following sections include simple and detailed examples.

Downloading the Developer Package The Developer Package can be downloaded from the GENESIS package archive website.

Simple Examples Simple examples can be used by developers who do not want to know about the intricate details of the Developer tools. Most developers fall into this category. Software developers and laboratories who have added their own private software components should be OK using the simple examples and can skip reading the detailed examples.

Detailed Examples The detailed examples provide:

Steps to build a release for the example <your-software> package.

Examples of the release procedure.

Utilities Some utilities currently depend on monotone for source code version control due to their configuration. It is possible to work with other version control systems.

Main script

neurospacesxbuild: Performs various back-end operations on a set of software packages. It is configurable in its packages and operations. The command:

xxxneurospaces_buildx--help
xxxxxxx

describes how the script works and lists default actions and build options. The flags at the end of the output should be inspected.

In brief: neurospacesxbuild operates in one of two modes determined by where the script finds the source code (“neurospacesxbuild --help” gives more details). As indicated in the following figure, neurospacesxbuild creates an extensible associative array (hash table) that can be visualized as a matrix of packages (rows) and build operations (columns). The script traverses the matrix by row, i.e. packages first, and performs successive build operations for a given package. The figure below lists the default build options. Options to neurospacesxbuild select or deselect cells in the matrix according to the configuration specified by any options to the neurospacesxbuild command. Note: Addition of a new component to GENESIS is the equivalent of adding a new column to the neurospacesxbuild associative matrix.

PIC Installation Order of Components: This is the left -> right component order in the neurospacesxbuild associative array:

Developer Component: Described in this documentation.

Model Container: Provides a highly optimized solver-independent internal storage format for models.

Exchange: Provides exchange in common descriptive standards such as NeuroML and NineML.

Experiment: Experimental Protocols.

Heccer: The current standard numerical solver.

SSP: The simple Scheduler in Perl is currently the standard scheduler for GENESIS.

NS-SLI: The GENESIS 2 Backward Compatibility Module. Required to run GENESIS 2 scripts.

G-Shell: Runs interactive GENESIS sessions.

GENESIS Documentation System: A modular documentation system designed to maintain geographically distributed software systems.

Note: When installing GENESIS manually, it is recommended to follow the installation order given above.

Front-end Developer Scripts The following utilities take optional arguments of “--regex” to select the packages they operate on, and “--verbose” to run the command in a more verbose mode. They follow the order of the Developer Workflow. The information they return are determined by the configuration of the Developer package.

Note: These Developer package commands are run from a command line prompt, i.e. outside the GENESIS environment.

Write/Update Code:

neurospacesxcreatexdirectories Create the correct directory layout required for GENESIS development.

neurospacesxdownload Download GENESIS components from a central archive.

neurospacesxinit Create source code repositories.

neurospacesxpull Download the source code or patches from a remote repository. This places the newest version of the source code in your local repository.

neurospacesxpush Upload source code or patches to a remote repository.

neurospacesxserve Starts serving the source code repositories such that other people can pull and sync to your machine (note that this locks all your local databases).

neurospacesxstatus Check for local source code modification (no network required). neurospacesxsync Synchronize local source code modification with a repository.

neurospacesxupdate Makes the local source code up to date using the repositories locally stored on your PC (so this is an entirely local operation). If necessary, this command also automatically merges parallel development branches.

neurospacesxupgrade Pulls code from the GENESIS repositories, calls neurospacesxupdate and neurospacesxinstall

neurospacesxtoolsxpropagate Propagates updates to source code of shared tools from source to target packages.

Document Code API:

mcad2doxy Convert obsolete multicad documentation to doxygen format, this has been used to convert the developer documentation of some of the software components to doxygen format.

neurospacesxdocs Builds package specific documentation on your local machine.

neurospacesxwebsitexprepare Prepare a version of the website on your local PC, and optionally upload it. This command builds a subset of documentation maintainted with the command neurospacesxdocs.

Build/Check:

neurospacesxconfigure (Re)configures the simulator software (requires packages to be installed already). neurospacesxinstall Install new simulator executable binaries and scripts.

neurospacesxuninstall Uninstalls  the  simulator

software (including the Developer package commands, to reinstall, go to the Developer package source code directory and run “make

&& sudo make install”).

neurospacesxclean Clean local source code and documentation directories.

The neurospacesxupgrade command makes the local source code up to date with the source code served by the remote servers. It thereby combines steps 1 and 3 explained above. This command is similar in operation to the invocation of the commands neurospacesxuninstall, neurospacesxpull, neurospacesxclean, neurospacesxupdate, neurospacesxinstall

Regression Testing:

nstestxquery Queries test specifications of a package.

neurospacesxharness Run the test specifications found in the current directory, see also the Neurospaces tester.

neurospacesxcheck Check for correctness of the installed software.

Simple Example: Enable the distcheck target of the make files, in addition to the default actions:

xxx$xneurospaces_buildx--distcheckx--help
xxxxxxx

Internal/Public Release:

nspkg-deb Automated debian package builder.

neurospacesxpackages Show which software components are enabled on your local machine.

Simple Example: This command invokes

xxx$xneurospaces_buildx--help-packages
xxxxxxx

neurospacesxdist Build a release given a milestone label and version number. This script also inserts the milestone label and version number into the Tag database which is a YAML file and can be found at: /etc/neurospaces/tagxdatabase. The following two utilities query this database:

td-labels Shows all release labels (major and version numbers).

td-majors Shows all major release labels.

For updating of version keywords the following are invoked automatically by the neurospacesxbuild script:

releasexextract Extract release information from a monotone repository. When a Tag is set, that will be the result, otherwise the SHA of the current base revision will be the result.

releasexexpand Do keyword expansion, see the manpage in the source for more details.

Simple Example: What shell commands will be run after options have been given, without really running the commands.

xxx$xneurospaces_buildx--verbosex--distcheckx--dry-runx--developer

Simple Example: How are operations mapped to packages.

xxx$xneurospaces_buildx--help-all

Detailed Example: Steps in building a release for the example

<your-software> package.

Tag the code: The GENESIS convention for creating these tags, which must be unique, is to concatenate an identifier from the name of the lab of origin, the purpose and/or type of software (e.g. build, passive, active, python, userdocs, des, network, i64, purkinje, integration, pools, etc), with an appended numerical identifier. Here, for example we use the generic tag

“<mylab-mysoftware-vnum>”.

xxx$xneurospaces_buildx--tagx<mylab-mysoftware-vnum>x--verbosex--verbosex--verbosex--no-compilex--no-configurex--no-installx--regexx<my-software>x--developer

Build the release for the tagged code:

xxx$xneurospaces_buildx--distx--verbosex--verbosex--verbosex--no-compilex--no-configurex--no-installx--regexx<my-software>x--developer

Or with one command line:

xxx$xneurospaces_buildx--tagx<mylab-mysoftware-vnum>x--distx--verbosex--verbosex--verbosex--no-compilex--no-configurex--no-installx--regexx<my-software>x--developer

Upload the tarballs:

   xxx$xneurospaces_buildx--src-tagx<mylab-mysoftware-vnum>x--upload-serverxftp://upload.sourceforge.net/incomingx--verbosex--verbosex--verbosex--no-compilex--no-configurex--no-installx--regexx<my-software>x--developer

Do not forget to edit release notes, tag the files as “Any”, “Source.gz”, etc. Due to the crappy Sourceforge interfaces, this has to be done manually (anyone have any ideas?).

Integrate Software Components:

neurospacesxversions Shows which versions of the GENESIS packages are installed.

neurospacesxcron A cron job based tester script.

Simple Example: What operations on known packages exist.

         xxx$xneurospaces_buildx--help-operations
         xxxxxxx

Detailed Examples of the Release Procedure After putting a Tag on the code, you first must check if the package can be built from the tarball. The autotools distcheck target does this (see below). An additional check is to build the tarball, put it somewhere on your filesystem, and then build the check target from the tarball. For example, first build the tarball and put it somewhere on your filesystem (e.g. /tmp/uploads):

xxxneurospaces_buildx--developerx--verbosex--upload-serverx--no-installx--regexxstudio

Then build the check target from the tarball:

xxxneurospaces_buildx--verbosex--checkx--regexxstudiox--src-tagxpython-5xx--src-dirx/tmp/uploadsx--verbosex--verbosex--unpackx--no-install

In a one line summary you need to insert a --tag, and --distcheck the current code, then make them available on Sourceforge. In one line of code:

xxxneurospaces_buildx--developerx--distcheckx--verbosex--tagxnetwork-5x--src-tagxnetwork-5x--upload-serverxhttps://frs.sourceforge.net/uploadsx--verbose

If you don’t want to do the upload, for a single package:

xxxneurospaces_buildx--tagxnetwork-5x--distcheckx--regexx'model-container'x--developerx--verbose

If that works, you then want to do the upload:

xxxneurospaces_buildx--src-tagxnetwork-5x--upload-serverxhttps://frs.sourceforge.net/uploadsx--regexx'model-container'x--developerx--verbose

The --verbose option is there to let you know what is going on during this lengthy procedure.

The same procedure is used to build a release of either individual or all packages. An official public release is made available for download on Sourceforge. Intermediate releases are for internal use only.

Because official releases are built using automake’s distcheck target, they must pass the tests of the package (on the machine used for the build). So they are considered to be alpha releases (ie. internally well tested).

The release procedure normally checks for package correctness using the automake distcheck target, meaning that both install and uninstall targets work and are each other’s complement. Official releases must always succeed on this target. Intermediate releases can fail.

Other Examples

Uninstall 4 packages on your developer machine:

xxx$xneurospaces_buildx--verbosex--verbosex--verbosex--no-compilex--no-configurex--uninstallx--regexx'(my-software|experiment|heccer|ssp|studio)'x--developer

Build checked releases for four packages:

xxx$xneurospaces_buildx--verbosex--verbosex--verbosex--no-compilex--no-configurex--no-installx--distcheckx--regexx'(my-software|experiment|heccer|ssp|studio)'x--developer

After a modification of the developer package, reinstall it:

xxx$xneurospaces_buildx--enablexdeveloperx--regexxdeveloperx--developerx--verbosex--verbosex--verbose

Releasing the developer package on Sourceforge:

xxx$xneurospaces_buildx--tagxbuild-25x--distx--src-tagxbuild-25x--upload-serverxftp://upload.sourceforge.net/incomingx--enablexdeveloperx--regexx'(developer)'x--developerx--verbose