PGENESIS Reference Manual

• Latest serial GENESIS Reference Manual
• pgenesis shell script used to run PGENESIS
• parallel routines jump: paron, raddmsg, rshowmsg, rvolumeconnect, rvolumedelay, rvolumeweight, @, async, waiton, barrier, barrierall
• Parallel I/O Issues
• Scheduler considerations
  
• pitfalls and holes

Startup

To use any of the capabilities of the parallel library, one must first start it up. This will also spawn the requested number of worker nodes on architectures that support process-spawning.

  paron                Starts up the parallel library.

There are several commands for obtaining configuration information:

  mynode               number of this node in this zone 
  nnodes               number of nodes in this zone 
  myzone               number of this node's zone 
  nzones               number of zones 
  ntotalnodes          number of nodes in all zones 
  mytotalnode          unique number over all zones for this node 
  mypvmid              task identifier used by PVM for this node 
  npvmcpu              number of cpus used by PVM in the parallel machine 

The ability to run parallel threads can be turned on or off (default is on).

  threadson            Re-enables parallelism. 
  threadsoff           Disables parallelism.
  clearthreads         Clears all pending parallel setup commands or
                         remote procedure calls
  clearthread          Clears at most one pending parallel setup command
                         or remote procedure call

Adding Messages

It is possible to create arbitrary messages between elements on different nodes using the raddmsg command:

  raddmsg              Adds message between the listed sources elements and
                       the listed destination elements (which may be designated
                       to be on other nodes by means of the '@' notation).

The following routine displays inter-node messages correctly (and suppresses the display of the postmaster messages used to implement the inter-node messages).

  rshowmsg             Connects one group of elements in a volume to another,
                       using source and destination element lists and masks.

Synaptic Connections

There are several routines which allow one to set up multiple synaptic connections across nodes. They are analogues of the regular GENESIS routines for setting up synapses.

  rvolumeconnect       Connects one group of elements in a volume to another,
                       using source and destination element lists and masks.
  rvolumedelay         Sets delays of a group of synapses receiving input
                       from a list of presynaptic elements in a volume.
  rvolumeweight        Sets weights of a group of synapses receiving input
                       from a list of presynaptic elements in a volume.

Remote Command Execution and Synchronization

  command@nodelist     Executes command on specified nodes synchronously
                       (i.e., does not return until remote commands have
                        completed and returned result)
  async command@nodelist
                       Executes command on specified nodes asynchronously
                       (i.e., returns without waiting for result)
  waiton               Wait for completion of a specified async command,
                        or wait for completion of all async commands.
  barrier              Wait for all nodes in my zone to reach this point.
  barrierall           Wait for all nodes in alls zones to reach this point.

Unsupported and Dangerous Operations

It is extremely easy to reach deadlock in parallel programs, one way to reduce the chances of this is frequent use of barriers and sparse use of asynchronous commands. However, barriers are expensive to execute and can reduce parallelism, so they should be placed judiciously in scripts.

The serial GENESIS stop command should be used only with extreme care in zones containing more than one node. PGENESIS executes an implicit barrier before performing a simulation step. If any nodes enter the barrier then all nodes must, otherwise deadlock will result. It is very difficult to satisfy this requirment when the stop command is issued.

Issuing step commands must be done with care. Since the step command executes an implicit barrier, failure to follow the following rule can result in deadlock. The two safe methods to issue step commands are:

1. step commands are issued exclusively locally (i.e., no use of the @ operator with step)
2. remote simulation step commands (e.g., step@all) be issued by at most one node in a zone.

Holes in the documentation

Revised: October 1996
URL: http://www.psc.edu/general/software/packages/pgenesis/ref/Parallel.html