Sequencer Command Line Tools¶

The Sequencer package provides the following command line tools:

seqrun.py: Executes the Sequencer scripts given in the command line. It’s not interactive, therefore it breaks and barks at you at the slightest provocation (i.e. at any non catched exception).
seqdraw.py: Produces a graphical representation of the given Sequencer scripts, it generates output in many formats (the formats supported by graphviz).
seqcli.py: It is a command line, shell-like, program that allows to interact with the ELT Sequencer.

seqrun.py¶

Executes the Sequencer scripts given in the command line. Usage:

$ seqrun.py -h
usage: seqrun.py [-h] [-l LOG_LEVEL] ...

Execute sequences

positional arguments:
modules               specific module files to run

optional arguments:
-h, --help            show this help message and exit
-l LOG_LEVEL, --log-level LOG_LEVEL

It is non-interactive so it breaks at the slightes provocation. An example:

# specify two modules to execute in sequence
$ seqrun.py seq.samples.a seq.samples.b

seqdraw.py¶

Since sequences are basically graphs it has been considered useful to have a tool that allows to visually represent a given sequence. This should be used as a checking step before executing the sequence in order to visually verify it

$ seqdraw.py -h
usage: seqdraw.py [-h] [-l LOG_LEVEL] file modules [modules ...]

Draw sequences

positional arguments:
  file                  output file(s) name
  modules               specific modules to draw

optional arguments:
  -h, --help            show this help message and exit
  -l LOG_LEVEL, --log-level LOG_LEVEL

This command will create sequence files(s) from the sequence(s) given. Usage example:

$ seqdraw.py output.png seq.samples.a
$ seqdraw.py output.jpeg seq.samples.b

seqcli.py¶

Purpose¶

seqcli is a command line shell-like program that allows to interact with the ELT Sequencer.

It can load and execute Sequences interactively.

It was written as a testing tool, it shows. But has evolved enough to be a useful frontend to load and run sequences.

seqcli is network based, it waits for a connection on a specific network port. Once a client connects to it can send commands to interact with the server.

Warning

Being a simplistic tool, no provissions are made for multiple users connecting to the same seqcli instance. Just don’t do it.

Basic Usage¶

Starting up¶

To star the seqcli server just type:

$ seqcli.py

This starts the seqcli server listening on port 8000.

Command line options¶

The seqcli server accepts the following command line options:

--serve-cli: [HOST:]PORT serves the CLI in the given host+port.
-l: Define logging level (ERROR, INFO, DEBUG )
--log-level: same as -l option.

Basic interactions¶

In order to interact with seqcli two shell windows are needed.

In one window seqcli is running sequences and displaying their output.

The second shell window, where the client is running, sends command to seqcli and gets replies.

Connecting¶

Any network client will work (e.g. telnet to port 8000). My personal preference is nc. In order to connect to the server one can execute:

$ rlwrap nc localhost 8000
Welcome to the CLI interface of echo!
Try:
    * 'help' to display the help message
    * 'list' to display the command list.
>>>

Loading sequences¶

At the prompt one can send commands to the server. The following will load a sequence module (if found):

>>> load seq.samples.a
>>> tplist
>>> tplist
( 0) - seq.samples.a: Simple example.

The tplist command shows the modules loaded, in the order they were loaded (0-index) and the module’s documentation if any.

One can load as many modules as desired, or load the same module again. Just because:

>>> load seq.samples.b
>>> load seq.samples.a
>>> tplist
( 0) - seq.samples.a: Simple example.
( 1) - seq.samples.b: Exdcutes Tpl.a, Tpl.b and Tpl.c in parallel.
( 2) - seq.samples.a: Simple example.

Executing Sequences¶

The run command executes the loaded sequences. It receives an index argument. The index indicates which of the loaded sequences shall be executed or pass -1 as index in order to run them all. In the follwoing example the first loaded sequence is executed. Its output is shown in the seqcli window:

>>> run 0
-> run 0

# The seqcli window shows the following in this case.
A
B

Sequence status¶

Sequences are modelled as DAGs. The state of the nodes in the DAG can be examined with the nodes command. The output comes on the seqcli window. For the above example:

N-- start_ObWrapper_qq33yXD start FINISHED
N-- end_ObWrapper_qq33yXD end FINISHED
T+- Sequence_7mGzYVB Sequence FINISHED
        N-- start_Sequence_7mGzYVB start FINISHED
        N-- end_Sequence_7mGzYVB end FINISHED
        N-- a_QN60G a FINISHED
        N-- b_nPOD b FINISHED

To decipher the above ouput, every line is formatted as:

Lines are indented according to its level in the DAG. Every node in the DAG is prefixed by its type, as follows:

N
This for Actions.

T
Refers to a Sequence.

P
Inidicates a Parallel sequence.

L
To indicate a Loop.

The node_id and node_name are automatically assigned if not given by the programmer.

The start_ObWrapper_xxx and end_ObWrapper nodes are the global start and end created internally by the running engine. Below them come the sequences the user has decided to execute.

In the example above all nodes are FINISHED, no error was detected.

Error Handling¶

When a Sequencer Item generates an error, i.e. raises a python exception, then the whole sequence is CANCELLED, and the offending node is marked as FINISHED|ERROR. The following example shows such a sequence:

>>>
>>> load seq.samples.test_err
>>> run 0
# ---------- seqcli output ----
.....
File "/anaconda3/envs/seq/lib/python3.7/site-packages/seq/samples/test_err.py", line 28, in b
x = 1 / 0
ZeroDivisionError: division by zero

The exception stacktrace is shown in seqcli’s window clearly showing the offending method and line where the exception occurrs (division by zero). The output of nodes command:

N-- start_ObWrapper_Yw813J2 start FINISHED
N-- end_ObWrapper_Yw813J2 end CANCELLED
T+- Sequence_xAgRAlq Sequence CANCELLED|ERROR
        N-- start_Sequence_xAgRAlq start FINISHED
        N-- end_Sequence_xAgRAlq end CANCELLED
        N-- Tpl.a_k7Pnx Tpl.a FINISHED
        N-- b Tpl.b FINISHED|ERROR
        N-- Tpl.c_YWk62 Tpl.c CANCELLED

Where the node b has state FINISHED|ERROR.

The resume command will SKIP the failed node (since it is finished anyway) and continue the sequence. In this example just the Tpl.c node will be executed.

>>> resume
>>> nodes
# -------- nodes command output
N-- start_ObWrapper_Yw813J2 start FINISHED
N-- end_ObWrapper_Yw813J2 end FINISHED
T+- Sequence_xAgRAlq Sequence FINISHED|ERROR
        N-- start_Sequence_xAgRAlq start FINISHED
        N-- end_Sequence_xAgRAlq end FINISHED
        N-- Tpl.a_k7Pnx Tpl.a FINISHED
        N-- b Tpl.b FINISHED|ERROR
        N-- Tpl.c_YWk62 Tpl.c FINISHED

It can be seen that after the resume command was issued the sequence is completed, however the item Sequence_xxxx shows state FINISHED|ERROR since one of its nodes b is also in FINISHED|ERROR state.

Another option is to issue the retry command. Which will restart the sequence from the failed node b.

Commands Reference¶

The following are the commands recognized by the seqcli server.

abort
Aborts a running sequence.

exit
Exits the program.

help
Explains how to get help

list
List valid commands

load <mod>
Loads a Sequencer script

nodes
Shows the state of the running (or last execute) sequencer script(s).

pause
TBI

ps
TBI

resume
Restart a sequence from the next valid state. AKA skip and continue.

retry
Restart the sequence by retrying the last failed node.

run <idx>
Executes the loaded sequence at index <idx>. if idx is -1 then executes all loaded sequences.

tplist
Lists loaded sequences