Imcat man pages

rshloop(1) --- Imcat Users Guide --- section: imcattools_rshloop --- last changed: Thu Jul 10 HST

NAME rshloop - execute shell commands in parallel. SYNOPSIS rshloop [-u] [-d] [-v] [-q] [-s slavelist] [-f ifmt] [-i initcom] ni commandformatstring DESCRIPTION 'rshloop' executes shell commands in parallel across a network. It is currently functionally equivalent, and has identical arguments, to 'pvmloop'. 'rshloop' first reads a table of slaves from 'rshloopslaves.lst' (or from 'slavelist' with the -s option). The first line of this table should contain ns, the number of slaves to be used, followed by ns lines containing two strings: nodetag and nodename. For example: 3 01 node01 02 node02 03 node03 You may specify any node, including the local node, multiple times. This will result in multiple processes running on each physical node. 'rshloop' then creates a set of temporary FIFO pipes /tmp/pid.s.fifo, one per slave. 'rshloop' then initiates a sequence of nb batches of nc <= ns commands until all of the ni commands have been exhausted. If the node is the local node, the form of the command actually executed is runcom command s | /tmp/pid.s.fifo & where 'command' is generated from the 'commandformatstring' as described below and s is the the slave number. The command 'runcom' executes the command given as its first argument and saves the stdout and stderr, if not already redirected elsewhere to temporary files. Once the command has terminated, it sends a message to its stdout containing any stderr and stdout from the command and then cleans up after itself. If the node is not the local node, the command actually executed is rsh remotenode 'runcom command s' | /tmp/pid.s.fifo & Once a batch of commands has been initiated, rshloop opens all of the FIFOs for reading (otherwise the commands will block). It then loops over the nc processes and collects the output. The commandformatstring has a syntax similar to a printf format string. On each iteration it is processed and each occurence of %i is replaced by the iteration number i = 0...ni-1, %I is replaced by a fixed length textual representation of i %n is replaced by the node number, and %N is replaced by the nodetag, and %% is replaced by %. The result may be a compound command of subprocesses linked by the pipe symbol '|', and the output of the command may be redirected into a disk file on the slave (in which case the master process will receive, and generate, no standard output). EXAMPLES These examples assume the rshloopslaves.lst file above, and that each slave has a disk named /dnn, where nn is the nodetag. To check on the status of the slaves: rshloop 3 w To clear a scratch directory on each slave: rshloop 3 'rm /d%N/tmp/*' To generate a set of 1000 Monte Carlo simulations with some command 'monty' which takes as an argument a seed (given here by the iteration number): rshloop 1000 "monty -seed %i > /d%N/tmp/monty%I.dat" This would cause the following commands to be executed: monty -seed 0 > /d01/tmp/monty000.dat (on node01) monty -seed 1 > /d02/tmp/monty001.dat (on node02) monty -seed 2 > /d03/tmp/monty002.dat (on node03) monty -seed 3 > /d01/tmp/monty003.dat (on node01) ...... monty -seed 998 > /d01/tmp/monty998.dat (on node03) monty -seed 999 > /d01/tmp/monty999.dat (on node01) OPTIONS With -u flag we output this man page and exit. With -d flag we just output the series of commands that would otherwise be executed and exit. This is highly recommended with commands that delete files etc. Use -v flag to invoke verbose mode, or -q to run quietly. Use -f ifmt to specify a format string for the iteration number, otherwise we use '%.nd' specification where n is just large enough to hold the number ni-1. With '-i inticom' option the shell command string 'initcom' is executed on the master node using system(initcom) before any other output is collected. This allows you to generate a header, to which the output of the slave processes can be prepended. For example, rshloop -q -i 'lc -C -n x < /dev/null' 10 'makerandcat 1000 -seed 2 -dim 1 | lc -o' generates a catalogue containing 10 x 1000 random numbers. The effect of the 'lc -o' call is to chop the headers off the slave output. Similarly you can use 'imhead -g ....' to generate a fits header and pipe the output of the slaves through 'imhead -d'. BUGS 'rshloop' does not die if the remote shell commands fail. 'rshloop' does not do any load balancing. FILES rshloopslaves.lst SEE ALSO pvmloop(1), pvmserver(1), topvm(1). AUTHOR Nick Kaiser --- kaiser@hawaii.edu

back to imcattools_rshloop back to imcat