Related Documentation:

• TODOs for the first release of GENESIS 3.
• TODOs for the GENESIS 3 Publication System.

GENESIS: Documentation

1 The TODO and the DONE List

This document contains a list of functions and descriptions that need to be implemented or incorporated in the software or software infrastructure. The level of detail of the descriptions can vary between items. After an item is described, it can be followed by a checklist with information of the level of implementation. For example, subitems in the checklist can be marked with the word ‘DONE’.

This document can grow when people add new functions that await implementation, and it can be consolidated by moving items that have been completed from the TODO list to the top of the DONE list. Items are never removed from the DONE list.

This TODO list was built on December 28, 2015, 8:28.

2 TODO List

2.1 heccer compile

Looks like ’neurospaces_install heccer’ currently fails because the python code tries to link with the model-container archives in the source directories instead of the ones that have been installed.

2.2 replace jodconverter with unoconv

Seems more flexible, regular command line utility, see also http://lists.debian.org/debian-accessibility/2011/10/msg00001.html and follow.

Needs package unoconv on ubuntu 12.04.

2.3 remove the application_classes entry from SSP configuration files

The entry has always the default value.

2.4 Convert the heccer configuration in an SSP file to a readable format

For use in SSP configuration files and in the gshell.

2.5 testing reconfiguration

Mando, I have no idea what is going on, but it seems autoconf related and related to uninstallation of STARTUP_FILES. You?

I don’t know how this would influence the consistency of the tester output, for instance if it leaves foreign files that contaminate the filesystem. I also don’t know how long this has been happening on my machine because this type of error is not reported by neurospaces_cron when not run from a crontab.

2.6 testing reconfiguration

neurospaces_clean has a –distclean option, but I don’t think it has ever been tested. We should incorporate it into neurospaces_cron and neurospaces_upgrade.

2.7 ns-sli warning registry

The ns-sli does more soundness checks on a model and the simulation run-time and gives many more warnings than G-2. Typical examples include:

• When running tests/scripts/test-traub91-v0/traub91_asym_simple3.g:

  *** Warning: the setmethod command is deprecated in G-3

• When running tests/scripts/PurkM9_model/CURRENT9.g:

  Warning: ParameterResolveToPidinStack() does not perform any operations.

• When running tests/scripts/test-traub91-v0/traub91_asym_simple3.g:

  *** Warning: zero initial value for the concentration of the following:   /cell/apical_18/Ca_conc

• When running tests/scripts/test-traub91-v0/traub91_asym_simple3.g:

  Warning: Tabchannel parameter xmin has already been set to 0.000000 for K_C   Warning: Tabchannel parameter xmax has already been set to 1000.000000 for K_C

• During execution of tests/scripts/rallpack3-simple.g (1):

  warning: cell found during solver_mathcomponent_typer()   warning:     /axon/solve   warning: cell found during solver_mathcomponent_processor()   warning:     /axon/solve   warning: cell found during solver_mathcomponent_finalizer()   warning:     /axon/solve

• During execution of tests/scripts/rallpack3-simple.g (2):

  Warning: Heccer object /axon exists, resetting it instead.

• During readcell tests/scripts/test-RSnet2/RScell.p in tests/scripts/test-RSnet2/RScell-G3-nohole.g:

  *** Warning: during readcell: channel /library/Na_pyr_dp has no Gbar / gmax defined   *** Warning: during readcell: channel /library/Kdr_pyr_dp has no Gbar / gmax defined   *** Warning: during readcell: channel /library/KM_pyr_dp has no Gbar / gmax defined

The implementation of a warning registry that registers warnings without displaying them avoids annoying a user. A new SLI command ’show_warnings’ shows all the registered warnings.

2.8 website images

The documentation system uses ht4tex which is the successor of latex2html.

It looks like a set of configuration files of dvipng determine the quality of the generated images. Each linux distro puts these files in different locations.

So far it is not clear how exactly to modifiy the configuration to improve the quality of the images, neither to the choice of distribution we would like to support.

Proposal to support at least one user-oriented operating system for generating higher quality images, and I still think we should do this. Then we can understand how well this works.

The genesis website runs on CentOS, not user-oriented. The best candidate is maybe ubuntu.

2.9 gshell output_resolution and output_interval

The problem comes when I want to set an output interval (i.e. a time step for a G2 clock) that is not related to the integration time step. Presently one can give a gshell command, such as:

to output every fifth integration step, but the resolution has to be an integer multiplier of simulation time step. I had a case where I wanted to compare to a G2 result where the simulation clock was 20 microsec and the output clock was 50 microsec, and it was not posssible. Furthermore, it is only possible to set the output resolution if the output generator is running in ’steps’ output_mode, which means that the file output will list step number instead of simulation time. Perhaps that should be an option, but one usually cares about the simulation time, not the step number, nor the relation between the output interval and the simulation time step. In fact, if we at some point incorporate variable time step solver methods, it wouldn’t be possible to relate the ”output resolution” to the time step.

So, my suggestion is to replace ’output_resolution’ with ’output_interval’, and that this should be an option within g-tube also.

Another slightly related issue that came up in the benchmarking is that the rallpack axon model produces two output files, one for each end of the axon. Alternatively, it could have used asc_file to produce a single file with three columns for time, axon0, axonx values. But neither option is presently possible with G3. (I have not yet implemented the G3plot capability to plot multiple columns yet, but it is on the list.) These options should be part of the gshell/g-tube output model.

2.10 tester cron job mail reporting

The email reporting facility of the tester needs to be refactored and cleaned up. It currently is highly sensitive to small misconfiguration errors, see also the entry of 29th of August in system-administration-log.

It is likely that the best way forward is to refactor things across the different scripts by creating a couple of perl packages.

2.11 tester framework and compilation errors

The script neurospaces_build does not detect compilation errors. Maybe because the option keep-going is set, needs further investigation. The priority of this problem is very low. Compilation errors are assumed to be detected on the machine of checkin rather than the automated tester.

2.12 when adding a new software component

• Figure out the workflow from adding the experiment component.
• Default monotone branch pattern: The neurospaces_build script should set a default branch pattern during the execution of the procedure to add a new software component.
• It’s also possible to have the public keys in a text file so that when a ne repo is initialized, those public keys are automatically put in

See also the document that describes how to add a new software component.

2.13 ns-sli installation

The ns-sli module install files under /usr/local/ns-sli rather than /usr/local/neurospaces/ns-sli. This installation needs to be ligned up with the installation of other modules such as gshell.

2.14 morphology_summarize

To characterize a cell morphology it would be good to extend the morphology_summarize command to include the average (+/- SD, SE ?) number of branch point from the soma to each tip and the number of primary, secondary, tertiary . . . etc, branch points.

2.15 GShell – Endogeneous Activation

Define GShell tests for endogeneous activation.

2.16 building documentation

Correct build of the documentation depends on a working G-3 installation. It looks as if userdocs_cron should be integrated into neurospaces_cron. Both the build of the documenation and the execution of the tests should be configurable from the neurospaces_cron options. The userdocs_cron script should call the neurospaces_docs script.

See also chat between Mando and me date 23th of August 2010.

2.17 ns-sli: follow-up after the October developers meeting 1

Add monotone version information identifiers to the gshell command ’list components’.

2.18 ns-sli: follow-up after the October developers meeting 1

Implement the following SLI commands and objects:

• scaletabchan: also try to implement a tagging mechanism that allows to browse the ’history of a tabchan’ (Hugo).
• spikegen: likely requires a trivial mapping to the spikegen token in the model-container (DONE).
• delta compression of ascii files: requires the glue of the Experiment::Output object to create a filesystem fifo with a dummy name, bind it to a perl oneliner and pipe it to bzip2 to write compressed output (Mando).
• randomspike: likely needs network support to be finished first (Hugo).
• pulsegen: likely needs network support to be finished first (Mando).
• asc_file: has several common fields currently not supported by the backward-compatibility module: float_format, append, initialize, leave_open (Mando).

2.19 ns-sli: follow-up after the October developers meeting 2

Others that need implementation:

• Table object functions and modes, some are spike related, some are input and output related.
• Analysis functions related to table objects.
• Interspike and peristim objects (low priority).
• Createmap needs proper testing.
• Volumeconnect needs to be linked to the volumeconnect algorithm in the model-container.
• Script_out: requires a hook in ssp.
• Disk_in and disk_out equivalents. Likely the delta-compressor and the annotations provided by DAVIS are already sufficient.

2.20 Model container and readline library

The model-container does not use the readline library for its querymachine. This function was present in the past and should be restored because it helps in querying models via the querymachine interface.

There is logic to detect readline in the filesystem present in the configure script, but somehow this part of the configuration does not propagate through to the compilation phase of the model-container.

2.21 File permission errors

Certain tests output to a hardcoded filename, such as /tmp/output. When one user runs the tests this file is given permissions for their user id. If a second user tries to run the same test, it writes to the same filename, the test will fail and the test will bail out with an error (as was displayed by the regression tester machine). This bottlenecks the number of users that can reliably install a developer installation on the same machine.

2.22 Mercurial

The source code of most GENESIS 3 software components are maintained using the Monotone version control system. The installer scripts integrate with Monotone to provide for automated building and testing.

The G-Tube is maintained using the Mercurial version control system. We must continue and finish the integration of the Mercurial version control system and the DeveloperScript neurospaces_build.

2.23 Gating Variables with Coupled Equations

There is currently no way to solve coupled equations of gating variables.

dm/dt = A * m + B * n dn/dt = C * m + D * n

In GENESIS 2, this is possible using forward integration methods only.

2.24 Level 4 Documentation: Technical Specifications

For each software component a set of (html) documents must be generated that specify the capabilities of the component. For Unix shell scripts, technical specifications can be generated from the usage messages generated by a “--help” argument.

• Developer package:
  • neurospaces_build
  • neurospaces_cron
  • release-expand
  • neurospaces_harness
  • tests_2_html
  • td-labels
  • td-majors
• model-container:
  • neurospacesparse
  • morphology2ndf
• experiment:
• heccer:
  • heccer
• ssp:
  • ssp
  • ssp_directory
• studio:
  • neurospaces
• ns-sli:
  • ns-sli
  • convert
• exchange:
  • neurospaces_exchange
• gshell:
  • genesis-g3
• userdocs:
  • userdocs-build
  • userdocs-check
  • userdocs-create
  • userdocs-rename
  • userdocs-snippet
  • userdocs-sync
  • userdocs-tag-filter
  • userdocs-tag-replace-items
  • userdocs-version

2.25 Level 3-7 Documentation: Creation Guide

Documentation from Levels 3–7 are ”automatically” generated. What is required is a description for developers of how to create these levels of documentation and how to add them to the appropriate places where they can be accessed by the GENESIS Documentation System. These are level 2 documents.

• Level 3: explain the html-upload target, and how the html-pages are copied to a directory visible to apache.
• Level 4: Each software package creates its own technical specifications, that are copied to a directory visible to apache.
• Level 5: Only contains links to external websites. As an example the document effic84.
• Level 6: Explains the proper format for inserting doxygen comments, doxygen configuration, and doxygen output formats.
• Level 7: Inline source code documentation, and Xrefactory configuration.

2.26 Documentation Browsing

Tag all the documentation with keywords to define flows through the documentation and ease browsing and reading of related documents.

Should be provided via a browser-based framework to give ‘professional’ look to documentation. See Documentation Framing (following).

3 DONE List

3.1 userdocs-sync error messages

Does not happen anymore, likely due to the recent rewrites of some of the userdocs scripts.

The userdocs-sync command in the g-tube source directory gives error messages about the descriptor.yml files part of the g-tube source code. This is a bug because userdocs-sync is supposed to only run in the userdocs source directory.

3.2 Populate the Model Library

Dave has been working on this, and populated the library with useful models. See the CNS 2011 report

Build up a library of cells that can be used with G2 or G3.

• Purkinje cell model edsjb1994 (present in the model container).
• Purkinje cell model Sergio (hopefully by email).
• Purkinje cell model Allan (may be included in dash).
• Rapp’s passive model (must be reconstructed from the paper).
• Different species purkinje cell morphologies (gp, rat, turtle, finch (present in project-browser).
• Rallpacks 1, 2, 3 (present in NS-SLI).
• Traub91 (present in NS-SLI).

3.3 Progress Reports to the Supplement

The supplement funding period has past

Every three months a report about progress on the work of the administrative supplement must be submitted to NIH.

This can be done by picking items from the DONE list related to the work proposed in the administrative supplement.

First move conceptualized items from the supplement to this TODO list.

3.4 gshell help components

Diagnosis of problem and documented procedure for troubleshooting the Inline::Python module is found in GENESIS dependencies document, in the Note about Inline::Python section. (Working)

Fails on Mac OSX after reinstallation of GENESIS developer package for Python module:

3.5 Incorrect path to NDF model file corrupts G-Shell

3.6 gshell list_elements

The gshell command ’list_elements’ sometimes loses part of the output. In the following snippet parts of the ’km’ and ’kh’ channels have disappeared:

This is obviously wrong behavior. The querymachine command ’expand’ can be used to compare with the correct output.

3.7 documentation synchronization

When the internet is down, the userdocs-sync script continues its execution after a synchronization failure.

3.8 userdocs-check

Integrate userdocs-check with userdocs-sync.

3.9 Split I/O Objects from the Heccer package

The Heccer package currently contains the Heccer software component and various Input and Output Simulation objects. It is better to have an I/O package separate from the Heccer package.

3.10 Perl Inline code directory

Create a consolidated directory for inline perl code so that Perl does not constantly create ”_Inline” directories in every working directory that ssp and the gshell are called in.

3.11 Descriptor file tags recognized by neurospaces_cron (2)

The following tag should be recognized by the documentation build system (cf. draft, published):

• - local Private (local) documentation publication.

The local tag is currently implemented by invoking the following command from a UNIX shell:

This command builds a local website of all the documents that have the tag ’local’.

This command builds a local website with all the published and the local documents.

3.12 Implement more consistent G-Shell command names

Assigned to Hugo (G-Shell) and Mando (G-Tube).

Also doing similar commands for inputclass_templates.

3.13 Bibliographies

Publication documentation does not currently support bibtex references. e.g.

Ka channel.

References appear as [?] in text and the References section is missing.

This probably requires the following sequence to be implemented in userdocs-build:

3.14 bibliographies

Documentation that use citations are currently not supported by the documentation system. Such documents are built differently on different machines.

Seems at present nobody knows what the original problem was, but this description being in the done list means that the problem has been solved.

3.15 Fix trash cleanup in automated tester.

(DONE)

The automated tester (neurospaces_cron) currently has a problem with not deleting all of the files it generates in /tmp. Over time the “files” grow and consume large amounts of disk space. The trash file removal must be fixed so that it keeps from contaminating the file system further. It must also be improved by making the trash list an configurable input file that can be passed as an argument to the neurospaces_cron script.

3.16 ns-sli .simrc

The backward-compatibility module expects the .simrc file in the users home directory. The default place of this file should be a place in /usr/local. (DONE)

This file may also be renamed to .ns-sli.

3.17 Descriptor file tags recognized by neurospaces_cron (1)

Extend the number of file formats for documentation that are recognized by the documentation build system when a document is placed in a document folder, as for example, currently happens for PDF formated documents. The following tags should be recognized for the alpha release.

• - html Hypertext markup language. (DONE)
• - png Portable network graphics. (DONE)
• - ps Postscript. (DONE)

This means that tags given above, should also be placed in the descriptor.yml file and recognized.

3.18 Unify the Methods of Handling of Inputs to a Model

There are several fundamentally different ways to specify inputs to a simulated model from the G-Shell. For example, a voltage clamp protocol is specified by creating input objects (pclamp object). A current injection and endogeneous stimulation is specified by setting parameters.

3.19 GUI: Other

• Finish yaml loader to load the clickable graphic complete with roll overs dynamically (DONE).
• Have the GUI use a ‘project’ file which will track various aspects of a users project. (DONE)
• Get compileall script to work and make python byte code for various distros (DONE).
• Have GUI track Workflow state and guide user accordingly. (DONE)

3.20 GUI: Output

• Plot: Get plot to take parameters from the simulation to fill in the legend, X and Y axis as well as the title of a graph. - (DONE)

3.21 GUI: Run Simulation (1)

• CHECK: Appears to be working. (DONE)
• RUN: Appears to be working. (DONE)
• RESET: Get reset functionality to work. (DONE)

3.22 GUI: Design Experiment

• Choose Inputs: create a menu of selectable inputs from the loaded model. (DONE).
• Choose Outputs: create a menu of selectable outputs from the loaded model (DONE).

3.23 Find out about Dependencies

Write a document that briefly describes the dependencies. These dependencies are known:

• Perl dependencies:
  • YAML
  • Clone
  • Data::Utilities
• GUI dependencies:
  • wxWidgets
  • pyYAML

Compile other dependencies from the NS wiki.

3.24 G-Shell enhancements/bugs on Mac

1. genesis > help ndf_save         ---         *** Error: no help for topic ndf_save yet         genesis > ndf_save mypc.ndf         Bus error         dhcp-129-111-247-96:0 adcmobile$ genesis-g3         cat: neurospaces/config.h: No such file or directory         Welcome to the GENESIS 3 shell         genesis > ndf_load cells/purkinje/edsjb1994.ndf         genesis > ndf_save         Bus error         dhcp-129-111-247-96:0 adcmobile$

2. After a ce command there should be no requirement for the absolute path argument for commands such as set_model_parameter and show_parameter, etc.

   genesis > ndf_load cells/purkinje/edsjb1994.ndf   genesis > pwe   /   genesis > show_parameter /Purkinje/segments/b3s45[10] RM   value = 3   genesis > ce /Purkinje/segments/b3s45[10]   genesis > show_parameter RM   symbol not found   genesis > show_parameter /Purkinje/segments/b3s45[10] RM   value = 3

   The use of the command show_parameter above is invalid. Use of the current element as the default element for the GENESIS 2 setfield command interferes with future extensibility of this command and will not be implemented.

   This can be easily worked around using the ’.’ to refer to the current working element. The following snippet illustrates this use:

   genesis > ndf_load cells/purkinje/edsjb1994.ndf   genesis > ce /Purkinje/segments/   genesis > pwe   /Purkinje/segments   genesis > ce b3s45[10]   genesis > pwe   /Purkinje/segments/b3s45[10]   genesis > show_parameter . RM   value = 3   genesis > ce ../   genesis > pwe   /Purkinje/segments   genesis > show_parameter b3s45[10] RM   value = 3

3.25 Allow use of token names for model components

This requires component names to be stringified in the NDF file format. E.g. cell, neuron, channel, etc.

3.26 Current Clamp Protocol

Make a current clamp protocol available to the gshell. Based on the voltage clamp protocol, add an inputclass, set the clamp value and connect it to a variable.

3.27 Documentation Summary Tag

Add support for a summary descriptor attribute that will append some text to end of a link in the documentation conents page. (DONE)

3.28 Use neurospaces_build to serve the source code

Right now custom scripts are used to serve the GENESIS 3 source code to the internet. This leads to duplication of configuration data such as filenames. The installer script neurospaces_build has dedicated function to serve the source code without configuration data duplication. Rather than using custom scripts, the neurospaces_build script should be used to serve the source code to the outside world. (DONE)

3.29 gshell studio

Disable the studio in the gshell, it should only be loaded on explicit request (DONE).

3.30 make dist for ns-sli

Building a tarball of the backward compatibility modules is broken. This needs to be fixed before a debian package can be built. (DONE)

3.31 Repository Server

Monotone sometimes hangs on the repository server. Upgrade the repository server (DONE).

3.32 Automated Regression Tester

After an update of the model-container, monotone reports that there are files missing from the local model-container workspace. This inhibits further updates such that the tester starts testing old software. An example monotone checkin that produces this problem is the model-container checkin of version d4cbeb603334d009ba2797976db15f53db5046fe. This version contains a white-space only change but corrupts the model-container workspace on the tester machine.

3.33 ns-sli showfield

Implement ns-sli showfield. (DONE)

3.34 G-Shell

• Implement the ssp_save and ssp_load commands.
• Complete the link with python.

3.35 configure.ac and neurospaces.h

The configure script of Heccer and NS-SLI does not find a valid neurospaces.h header even when one is installed in a standard location.

3.36 asc_file object

The asc_file object test in ns-sli needs to be completed.

3.37 Installation documentation

Consolidate and correct the installation documentation for software developers.

3.38 Repository Upgrade

Upgrade the source code repository (also test machine) to the latest stable Debian version. This was a duplicate of 3.31.

3.39 Monotone Clone Operation Integration with the Installer Scripts

Integrate ‘mtn clone’ into the installer scripts (DONE).

3.40 Convert Developer Documentation to Doxygen

• Finish mcad2doxy convertor (DONE).
• Convert and Check Output (DONE).
• Configure and Makefile Integration (DONE).

3.41 Backwards Compatibility for Single Neurons

• Ca pools (DONE).
• ascii_out (DONE).
• readcell (DONE).
• hsolve (DONE).

3.42 Webcheck Output

Correct the errors reported by webcheck. (DONE)

3.43 Webcheck

Go through the Webcheck output and identify critical and easy to fix problems. (DONE)