Related Documentation:

• GENESIS 3 project TODO list.
• TODOs for the GENESIS 3 Publication System.

GENESIS: Documentation

1 The TODO and the DONE List

This document contains a list of functions and descriptions that need to be implemented or incorporated in the software or software infrastructure. The level of detail of the descriptions can vary between items. After an item is described, it can be followed by a checklist with information of the level of implementation. For example, subitems in the checklist can be marked with the word ‘DONE’.

This document can grow when people add new functions that await implementation, and it can be consolidated by moving items that have been completed from the TODO list to the DONE list. Items are never removed from the DONE list.

This TODO list was built on December 28, 2015, 8:13.

2 G-3 Release TODO List

2.1 Add Network Modeling Support

Network modeling support still requires substantial development on one software component, and also impacts other software components.

• DES
  • instantiation of DES queuer stand-alone (DONE)
  • instantiation of DES distributor stand-alone (DONE)
  • tests of DES stand-alone (DONE)
• model-container
  • instantiation of projections/connections by model-container (DONE)
  • inspection of projections/connections by model-container API (DONE)
• SSP
  • instantiation of DES queuer from SSP (DONE)
  • instantiation of DES distributor from SSP (DONE)
  • translation of model-container connections to DES connections (DONE)
  • tests for SSP
• G-Shell
  • add integration commands to G-Shell
  • tests for G-Shell
• backward compatibility
  • integration of planar/volume connect command
  • tests for backward compatibility

2.2 ns-sli installed G-2 scripts

• The tests scripts of ns-sli are not included in the tarball. As a consequence they are not inluded in the rpm nor deb packages.
• Change loop: Mando’s change solved this, Hugo undid the change for reasons mentioned in an email from Hugo to Mando.

2.3 symmetric compartments (2)

Connect the NEURON compartmental solver to the model-container.

2.4 Slow operations on MAC OSX

Heavy memory operations such as pqsetall run slowly on Mac OSX to the point of causing several tests to fail. Test errors are inconsistent, and different tests will fail on different test runs.

2.5 Stalling error on MAC OSX

There is an issue with stalling on the Mac OSX Leopard. The result is a stalled process with no CPU activity. The error is inconsistent and only seems to occur when the target machine is running other high memory tasks. From examining the state of the process during the lock it appears to be a system lock on a print statement.

The stall error currently occurs during the nesting.t test sometime after pqsetall has been called.

A similar problem was found here on the Apple forums.

The same type of stall occurred previously when running the command ./neurospacesparse -v 1 -q legacy/networks/supernetwork.ndf in the model container function QueryHandlerPrintSpikeRecieverCount() . This stall error has not been observed in well over a year when running this command and currently does not produce it.

A comment has been placed in the section of code in the model container where the error occurs at line 6133 in querymachine.c.

Previous measures to diagnose were to use the Instruments application to see exactly where the process was locking, which lead to discovering the information on the Apple forums.

7/25/2010: Found the point of the stall during a test run to be at the ystem $shell_command;all in neurospaces_harness.

8/30/2010: Made some changes in an attempt to remove the stalling, instead get an error at ”select$DARWIN_EXTSN” as opposed to the low level write statement ”write$NOCANCEL$UNIX2003”. Stall remains so won’t check in the changes since it did not fix the problem.

8/31/2010: Using ptkdb and running different operating systems side by side I’ve narrowed down the possible error to the perl Expect module, which seems to behave a bit differently on Mac OSX compared to CentOS and Ubuntu. For the tests which start with a command that outputs expected output with a following test case that does not write, only reads this output, the output of the prompt starting the program along with the programs output is concatenated together. This causes the test to fail, but since this read only happens on the first test specification, subsequent tests pass.

2.6 Simulator Performance Benchmarking

Performance Benchmarking is traditionally classified in phases of model-construction time and simulation run-time. Test the performance of the Purkinje cell in each of the following environments:

• GENESIS 2 (old GENESIS 2 enviroment). What is the time taken by the simulator to run 1 second of simulation in G-2?
• NS-SLI (GENESIS 3–GENESIS 2 backward compatibility). What is the time taken when running 1 second of simulation in G-3 using the the NS-SLI module to read the original G-2 scripts that encode the Purkinje cell model?
• SSP (GENESIS 3 batch simulation environment). What is the time taken when running 1 second of simulation in G-3 from a UNIX shell command line with the Purkinje cell model encoded in NDF format?
• G-Shell (GENESIS 3 interactive simulation environment). What is the time taken when running 1 second of simulation in G-3 from the interactive G-3 shell with the Purkinje cell model encoded in NDF format?

2.7 Demonstration Scripts

The following scripts have been selected for use as demonstration scripts:

• Purkinje Cell Scripts from Erik De Schutter: some of the scripts are used for regression tests (and work correctly). Other Purkinje cell scripts still need validation.
• Rallpacks: for benchmarking it would be useful to implement scripts for the rallpack standard. To show the relationships between the different GENESIS 3 interfaces, it is useful to implement these scripts in the different interfaces currently available (Perl, Python, NS-SLI, NDF, Experiment, Heccer, G-Shell).

  For visibility and good integration with the rest of the system, it would be best to integrate these scripts with the regression tester, such that they are validated and documented on a daily basis.

  • Rallpack 1: Passive unbranched cable.
  • Rallpack 2: Branched cable.
  • Rallpack 3: Squid axon/Active cable.
  • Integrate with Tests.

2.8 Cleanup Packages Archive

There are redundant packages and directories in http://repo-genesis3.cbi.utsa.edu/deb/ and other download directories. Ideally there is only one directory for each set of deb packages, and all the deb packages are listed there as one flat list.

2.9 Release Notes

• See also release-notes.tex
• Document that describes the release.
  • Compile a list of G-3 capabilities, compile them from each of the components.
  • Statements about:
    • Backward compatibility.
      • Clarify the level of backward compatibility.
      • Emphasize on commitment of backward compatibility.
      • Support of conversions from G2 to G3 for the next two years.
      • explain policies for feature requests.
      • Rallpacks.
      • Does the purkinje cell and its various stimulation protocols.
      • please try your simulations and report problems
    • NeuroML support (passive compartments).
    • PyNN support, placeholder for an interface to PyNN through the pynn_load command
  • Installation.
    • Compile a list of dependencies (DONE, see installation documentation).
    • Create an integrated installation tar ball or so (DONE, on repo).
    • add deb and rpm packages (DONE, on the repo, see other point).
    • Release levels: G-3 Hacker (coherent release for technical users), G-3 Intermediate (also for script users, includes python for the creation of GUIs and projects), G-3 Full (include complete GUI).
    • Release plan: alpha, beta, full release.
    • Synchronize userdocs and wiki installation docs (DONE).
  • Mailing lists, discussion boards and other genesis related systems.
    • What are they used for?
    • What do we want them to be used for? Why?
    • Add  hg repository (unnecessary?).
    • Add bulletin boards and mailing lists.
    • Add NS related systems.
      • www.neurospaces.org
      • Blog.
  • alpha: Mid-October.
  • beta1: Mid-January.
  • beta2: Mid-March.
  • full: Mid-July.

2.10 Announce GENESIS 3 alpha1

Write an announcement of the GENESIS 3 alpha1 release to the connectionist, compneuro, genesis-users email lists. Signed with Hugo Cornelis, Allan Coop, Mando Rodriguez, Dave Beeman, James Bower.

2.11 Report Document for Testers

Construct a report document with instructions for alpha and beta testers. The instructions consist of installation instructions and a questionnaire about basic application functionality.

Questionnaire must be put on the website.

2.12 Packaging and Distribution (for the alpha1 Release)

Packaging involves the creation of tarballs with source code to the creation of binary packages for different OSs.

For packaging and distribution of the alpha release of GENESIS 3.0, each software component must implement the following makefile targets:

• dist: for creation of the tarball with source code. Create an installer script that builds the tarball of each package (neurospaces_dist) (DONE).
• pkg-deb: for creation of a binary archive for debian based Linux distributions (including Ubuntu). - (95%)
• pkg-rpm: for creation of a binary archive for rpm based Linux distributions (such as Redhat and Fedora). - (DONE)
• MAC .dmg file. - (DONE)

Communicate with the previous debian maintainers about this. They are Sam Hocevar (Debian packages) sam+deb at zoy.org and Matt Zimmerman mdz at csh.rit.edu

2.13 Daily Releases and Release Labels

The installer scripts can update the release labels in the source code automatically, and check them in into the source code repositories. The checkin should be done on a branch ’dist’ to prevent contamination of the default ’0’ branch with versions that reconfigure the software with new release labels.

The checkin should be made visible from the repository server (ie. the tester should push its changes from its repository to the server repository).

Currently date based. This does not work with distributed version control. The only way to version files in a distributed environment is using the version control identifiers.

2.14 Source code Configuration

Fix automake scripts to detect different operating systems and build makefiles accordingly:

• Fedora 11: Compensate for libhistory being built into the installed libc.
• BSD: Not currently supported.
• Solaris 10: Not currently supported.

2.15 GUI: Construct Model

• Import from External Library:
  • Displaying NDF library is working. (DONE)
  • Get the GUI read from external libraries on the net.
  • Add support for different file types like SWC. (25% done)
• Create Model: Create interface to allow a user to create a simple model via text and/or graphics.
• Explore Model: Create a graphics widget which displays the morphology when clicking ’explore model’. Remove Studio from output menu.
• Load Project: Get GUI to load a project based on a yaml file which includes all relevant input and result files (DONE).
• Save Project: Get save model to work properly (save a project file as well as all included files) (DONE)
• Load Tutorial: Display a list of clickable tutorial links and have the GUI open them in a web browser of html widget. (50% done)
• Create Project: Present a wizard so a user may click on menus to initlialize a new project. (75% done)

2.16 GUI: Run Simulation (2)

• STOP: Get ‘stop’ simulation to properly work (not sure how this will work since the G-Shell is stop and wait).
• SAVE: Save model state.
• LOAD: Get GUI to load a previous save state. (20% done)
• Runtime Option: create a menu for adding runtime options.

2.17 GUI: Output

• Matlab: Create export menu/functionality for matlab.
• xmgrace: Create export menu/functionality for xmgrace.
• Mathematica: Create export menu/functionality for Mathematica.

2.18 GUI: Iterators

Add menus/functionality to the iterator menu.

Put a couple of iterator scripts of the purkinje cell comparison and the mutual information projects in a directory. The Iterators menu lists all the scripts found in this directory, clicking on one of them runs the appropriate script. The script generates SSP configuration files that can be loaded into the GUI.

2.19 GUI: Other

• Screenshot functionality for plot and explore model. (90% done)

2.20 Documentation Framing

Look and feel of documentation website.

Check what options of the framing functions of tex4ht can be used.

2.21 gshell python

Document the python dependencies of the gshell.

Currently the gshell requires Inline::Python as a recommended option to be installed.

2.22 symmetric compartments (1)

Redundant with the other entry of symmetric compartments.

Implement symmetric compartments. We will have to decide how to do this: core implementation, or converting each symmetric compartment to two asymmetric compartments. This will also likely involve core design decisions, because the model-container does not distinguish between symmetric and asymmetric compartments (only the solvers do).

3 DONE List

3.1 Packaging and Distribution (for the alpha1 Release)

Postponed:

An additional package must be created that has all other software components of GENESIS 3 as dependencies. An installer script must be created that builds all the OS specific archives and puts them in a common directory (neurospaces_dist_archives).

3.2 Model Container

Complete the implementation of the ndf_save command. Add tests for saving a full model, saving a library of models and saving a modularized part of a model.

• there is now an export library function in the model-container that exports all the components of a model in the private model section of an NDF file.
• The model is then trivially instantiated in the public model section.
• There is also an export all function in the model-container that can save a modularized part of a model (but you loose the relationships between individual model components).
• The gshell ndf_save command is trivially linked to the export library function.

Algorithmically:

• The export library traverses the model tree in post-order, thereby exporting untagged components, and tagging exported ones.
• The prototypes in the tree are always exported as references to other components using the ’CHILD’ token.
• The names of the model components are generated on the fly, except for those components visible to the user (in the public_models section). The public names remain the same (they are the ’interface’ to the model).
• The library is essentially a flat representation of the model, the model hierarchy is only generated using the insertion of ’CHILD’ren.

3.3 SSP Optimizer

The SSP optimizer uses the perl Inline module to inline C code with perl code. The Inline module creates a directory hierarchy inside the current directory to do its work. The Inline module must be configured to do this in a standard directory, where all users share the result.

The installation of SSP must run SSP to do the compilation of the optimizer, and install it afterwards. This is to avoid race conditions between simultaneously running ssp processes.