Tutorial

The purpose of the sequencer is to execute Sequencer scripts, either for engineering or science purposes.

The unit of execution is a single step which is just a normal python function or method with no input parameters.

Sequences are modeled as Directed Acyclic Graph (DAG). Each node in the graph can either be a simple action which just invokes a single sequencer step or a more complex node which contains a complete sequencer script.

This allows sequences to be grouped and nested freely. Ultimately they will execute steps.

The Sequencer API allows to create these graphs.

Building Sequences

Sequencer scripts are modeled as DAGs, the Sequencer API allows to create nodes in the DAG. There are different node types that determine the way its children are scheduled for execution (e.g. Parallel, Sequential), you are free to select, mix and match the node type(s) that suits better your needs.

The Sequencer Engine expects to find either a module method or a specific class name in a module in order to construct a Sequencer script from it. The conventions are the following.

1. A module level create_sequence() function. See Tutorial 1 (a.py).

2. A class named Tpl which must provide a static method create_sequence. See Parallel tasks sample.

In either case, the return value is the root node of the graph being implemented.

The first convention is tried first, if no create_sequence() function is found, the second convention is attempted.

For very simple scripts, following the first convention is perfectly fine. For more complex scripts, e.g. a Sequencer script goging to be parametrized to control mutipe devices at the same time, the class approach is recomended.

Node Dependencies

Regarding the order of execution, the DAG edges allows to represent the nodes dependencies. Each node in the DAG depends, from its parent nodes. Meaning that a node will not be started until every node that precedes it has finished its own execution.

Node dependencies determines the execution order of the sequence steps. A node wont’t run until all its dependencies have finished. In the graph, a node dependency is seen as an incoming edge.

Chaining the basic node types Sequence and Parallel defines a dependency hierarchy.

Sequence nodes are executed one after the other, therefore each node depends on its predecessor. On the other hand, Parallel nodes indicates that all nodes it contains shall be executed together. By combining and chaining this two basic node types it is possible to express any dependency graph.

Very simple sequences

Let’s start with a simple sequence.

As mentioned before the simplest step is a python coroutine with no input parameters which is used to create an Action node.

Note

The no input parameter rule can be bypassed with strategies shows in Passing Arguments to Actions

In this case, we define a sequence that executes two steps, one after the other, namely do_a() and do_b(). Source codes is shown below.

Tutorial 1 (a.py)

#!/usr/bin/env python
"""
Simple example (A)

Executes function a() and b()
"""
import logging
import asyncio
from seq.lib.nodes import Sequence
from seq.lib import getUserLogger


#LOGGER = logging.getLogger(__name__)
LOGGER = getUserLogger()

async def a():
    """Simply do_a"""
    await asyncio.sleep(1)
    LOGGER.info("Ax")
    return "A"

async def b():
    """do_b example
    """
    LOGGER.info("Bx")
    return "B"

def create_sequence(*args, **kw):
    """Builds my example sequence"""
    myname = kw.pop('name', "Example A")
    return Sequence.create(a, b, name=myname, **kw)

Important

The Sequencer Engine is based on the asyncio library, therefore it is biased towards coroutines, but they are not mandatory as shown in Loop example.

These are some simple rules to create sequences:

• A step (Action) is a python coroutine with no input parameters. See Passing Arguments to Actions to break this rule.

• In this case, the sequence is created using the Sequence.create (Sequence) constructor, which receives the steps to be executed (in the given order).

• The python module which contains the sequence must define a create_sequence function as shown in the example. It returns a Sequence node that holds the nodes it will execute.

Important

The Sequence.create constructor provides syntax sugar in order to support passing coroutines as the sequence’s graph nodes. In such a case a node of type seq.nodes.Action is automagically created and inserted in the graph.

This simple sequence 01 is graphically shown below. It can be imported from python as:

>>> import seq.samples.a

simple sequence 01

Notice that only two nodes were specified in the create_sequence() function. However the simple sequence 01 figure shows four nodes. The sequencer engine adds a start node (black circle) and end node (double black circle) to every node container type, i.e. those nodes that have children: Parallel, Sequence and Loop.

Note

The start and end node, among other things, makes easy to chain nodes together by linking the end node of a container with the initial node of the next.

Executing Tasks in Parallel

One is not limited to create just linear sequences. Parallel activities (pseudo parallel) can be created using the Parallel.create() constructor. It receives the same parameters as the Sequence node constructor. When executed, the sequencer engine processes the Parallel nodes children in parallel.

Parallel tasks sample

"""
Parallel nodes example.
"""
import asyncio
import random
import time
import logging
from seq.lib.nodes import Parallel, ActionInThread
Logger = logging.getLogger(__name__)

class Tpl:
    """A sample Sequence"""

    def a(self):
        """sleeps randomly"""
        t = random.randrange(5)
        time.sleep(t)
        Logger.info("... done A")

    async def b(self):
        """sleeps randomly"""
        t = random.randrange(5)
        await asyncio.sleep(t)
        Logger.info("... done B")

    @staticmethod
    def create(**kw):
        """Builds my sequence"""
        a = Tpl()
        p = Parallel.create( ActionInThread(a.a), a.b, **kw)
        return p

Which is represented graphically as follows.

Parallel Sequence

Points to notice:

1. In this case the Sequencer Engine discover a class named Tpl and calls its create method (@staticmethod as the convention mandates).

2. The example Parallel Sequence also shows that steps are not limited to coroutines. Just wrap it in ActionInThread node.

3. There is no problem mixing normal routines and asynchronous code. The sequencer will send the normal code to a separate thread and execute it there.

In order to avoid normal methods or functions to potentially block the asyncio loop (by holding the CPU) they must be executed on their own Thread.

This is achieved with the ActionInThread node. In the example the b() method is wrapped in such node.

Executing Tasks in a Loop

The Loop node allows to repeat a set of steps while a condition is True.

Loop example

"""
Implements a loop.
The condition checks Loop's index < 3.
"""
import asyncio
import logging
import random
from seq.lib.nodes import Loop

logger = logging.getLogger(__name__)


class Tpl:  # Mandatory class name
    async def a(self):
        """sleeps up to 1 second"""
        t = random.random()  # 0..1
        await asyncio.sleep(t)
        logger.info(".. done A: %d", Loop.index.get())

    async def b(self):
        """sleeps up to 1 second"""
        t = random.random()  # 0..1
        await asyncio.sleep(t)
        logger.info(" .. done B: %d", Loop.index.get())

    async def c(self):
        pass

    async def check_condition(self):
        """
        The magic of contextvars in asyncio
        Loop.index is local to each asyncio task
        """
        logger.info("Loop index: %d", Loop.index.get())
        return Loop.index.get() < 3

    @staticmethod
    def create(**kw):
        t = Tpl()
        l = Loop.create(t.a, t.b, t.c,
                        condition=t.check_condition, **kw)
        return l

Which is represented graphically as follows.

Loop example

The code in Loop example shows the Loop’s node constructor which takes a variable number of arguments comprising the Loop’s body, i.e. the part that is repeated, do_a(), do_b() and do_c() in this case. The condition node is specified with the condition keyword.

The Loop’s index is kept in the context variable index, meaning it can be accessed as Loop.index.get() as the my_condition() function shows.

In order to separate the index value of the different loops that might be occurring at the same time the Loop’s index is implemented as an asyncio context variable. Therefore to get its value one has to call its get() method as the my_condition() function shows.

Embedding Sequencer Scripts

Sequences can be reused or embedded in order to produce more complex activities. The following example uses the sequences “Tut_01” and “Tut_02” to create a new sequence that executes them in Parallel and just for the kicks adds an step from a local class.

Important

Embedding a Sequence entails to import the module and instantiate its Sequencer script (either with create_sequence() or by Tpl.create(). But how do you know which one to call? You don’t have to know. You let the OB object do it for you (or the seq_factory() function) as:

from seq.lib.nodes import seq_factory
from seq.samples.a

mynode = seq_factory(a)

Sequence embedding example

#!/usr/bin/env python
"""
Simple example.

Uses nodes from template defined in module 'a'.
It also uses the 'a' template as a whole.
"""
from seq.lib.nodes import Parallel, seq_factory
from seq.lib.ob import OB

from seq.samples import a
from seq.samples import b

class Tpl:
    async def one(self):
        print("one")
        return 0

    async def two(self):
        print("two")
        return 99

    @staticmethod
    def create():
        aa = OB.create_sequence(a, name="A")
        bb = OB.create_sequence(b, name="B")
        
        #aa = seq_factory(a, name="A")
        #bb = seq_factory(b, name="B")
        s = Tpl()
        return Parallel.create(aa, bb, s.one)

Some points to note:

• Use seq.lib.nodes.seq_factory() to select the right method to instantiate a predefined sequencer script, so it can be reused.

Observation Blocks and Templates

Observation Blocks are defined trough JSON files. A JSON file can store simple data structures and objects.

From the point of view of the Sequencer, an Observation Block is a sequence of templates that needs to be executed in the specified order. Therefore the sequencer is only concerned with the “templates” section of the JSON file. The Sample Observation Block file shows a (hopefully) simple Observation Block.

Sample Observation Block file

{
    "obId": 0,
    "itemType": "OB",
    "name": "obex 2",
    "executionTime": 0,
    "runId": "string",
    "instrument": "string",
    "ipVersion": "string",
    "obsDescription": {
        "name": "My humble II OB(A)",
        "userComments": "A",
        "instrumentComments": "AA"
    },
    "constraints": {
        "name": "string",
        "seeing": 0,
        "airmass": 0,
        "moonDistance": 0,
        "waterVapour": 0,
        "atm": "string",
        "fli": 0,
        "strehlRatio": 0,
        "skyTransparency": "string",
        "twilight": 0
    },
    "target": {
        "name": "string",
        "ra": "string",
        "dec": "string",
        "coordinateSystem": "ICRS",
        "comment": "string"
    },
    "templates": [
        {
            "templateName": "seq.samples.tpa",
            "type": "string",
            "parameters": [
                {
                    "name": "par_b",
                    "type": "integer",
                    "value": 0
                },
                {
                    "name": "par_c",
                    "type": "number",
                    "value": 77
                }
            ]
        },
        {
            "templateName": "seq.samples.tpa",
            "type": "string",
            "parameters": [
                {
                    "name": "par_b",
                    "type": "integer",
                    "value": 0
                },
                {
                    "name": "par_c",
                    "type": "number",
                    "value": 10
                }
            ]
        }
    ],
    "pi": {
        "firstName": "I",
        "lastName": "Condor",
        "email": "user@example.com"
    }
}

For now, we only care about the templates section. The Python modules implementing the desired actions are defined with the templateName keyword. Many templates can be specified this way. Moreover, many instances of the same template can be requested, The samples JSON file shows this case with the seq.samples.tpa Python module.

Note

templateName value must be a valid Python module expressed in dot notation. Just as it is written to be imported from the python prompt. Therefore the said Python module must be located somewhere along the PYTHONPATH.

The parameters section describes the runtime parameters (name, type, value) to be handed over to the Template instance and they can be accessed from the Template’s python code.

Accessing Template Variables

It is possible to access the value of the Template variables Inside the Python code implementing a Template.

In order to access to template parameters like the ones defined in Sample Observation Block file one has to use the function get_param(). It knows the current template and request the parameters of interest. In the case of the example, the code to access par_b and par_c is as follows.

Access Template variables in Python

#!/usr/bin/env python
"""
Shows variables


"""
import sys, inspect
import logging
import asyncio
from seq.lib.nodes import Sequence, Parallel, Template
from seq.lib.nodes import Action as _a
from seq.lib.nodes import ActionInThread as _ait
from seq.lib import logConfig
from seq.lib.nodes import get_param

import tkinter as tk
from tkinter import simpledialog


logger = logging.getLogger(__name__)

__seq_params__ ={"par_b": 11, "par_c":22}

class Tpl:

    async def do_sum(self):
        """Adds two integers"""
        parb = int(get_param("par_b"))
        parc = int(get_param("par_c"))

        logger.debug("Node sum")
        logger.info(
            "xSUM {} + {} = {}".format(parb,parc,parb+parc))
        await asyncio.sleep(1)
                
    async def delay(self):
        """Waits a sec"""
        logger.debug("sleep a bit")
        await asyncio.sleep(1)
            
    @staticmethod
    def create(*args, **kw):
        "Adds two variables"
        a = Tpl()
        return Sequence.create(
            a.do_sum,
            a.delay,
            name="TPL_SUM Example", **kw)

The code in Access Template variables in Python, in particular, the highlited lines shows the way to obtain a reference to the current_template object and to request from it the parameters, namely par_b and par_c.

Notice the parameter values are retrieved as string and have to be converted to integer in this case.

Inserting a basic dialog window

A basic dialog window is displayed when a specified condition is not met. It gives the user the possibility to perform an additional step to overcome the unfulfilled condition and proceed or to stop the sequence execution. An instance of BasicDialog shall be inserted in a sequence as below with a condition and an extra step provided by the user:

Insertion of a basic dialog window in a sequence

#!/usr/bin/env python
"""Usage of a dialog node node"""
import logging
import asyncio
from seq.lib.nodes import Sequence, BasicDialog
from seq.lib import logConfig


LOGGER = logging.getLogger(__name__)

async def a():
    LOGGER.info("a")
    return "A"

async def b():
    LOGGER.info("b")

async def test_condition():
    await asyncio.sleep(0.5)
    return False

async def extra_step():
    await asyncio.sleep(3)
    print("What an extra step!")
    return True

def create_sequence(*args, **kw):
    """Builds my sequence"""
    logConfig(level=logging.INFO)
    c = BasicDialog(name="some_dialog", description="my first dialog", condition_label = "test condition", check_condition=test_condition, step_label="perform extra step", condition_extra_step=extra_step)
    return Sequence.create(a,c,b, **kw)

Conclusion

This finishes the basic tutorial. One can create any type of flow using the node types shown. More details about node’s attributes and context are given in A Deeper Look.