1. Introduction

This User Manual describes how to build C++ applications for the ELT using the Rapid Application Development (RAD) toolkit.

RAD is an application framework that enables the development of event-driven applications for the ELT based on call-backs or state machines.

RAD assumes the following application stack:

An applications based on RAD runs on a Server or a Virtual Machine (VM) installed with the ELT Development Environment.

The Development Environment is based on Linux Cent OS and includes the GNU C++ compiler, waf building tool, and many other libraries such the Google Unit Tests, Robot framework for the integration tests, etc. (see: “Guide to Developing Software for the EELT”, https://pdm.eso.org/kronodoc/HQ/ESO-288431).

The Software Platform is a set of libraries, running on top of the Development Environment, that provides common services such as: Error Handling, Logging, Messaging, Configuration, In-memory DB (Online-DB), Alarms, etc.

The official ELT Software Platform is called CII: the Core Integration Infrastructure. Since CII is not available yet, the following off-theshelf libraries have been used:

Service	Description
Error Handling	Exceptions
Configuration	Based on files using YAML
Messaging	Req/Rep and Pub/Sub using ZMQ (De-)Serialization using Google ProtoBuf
Logging	Based on EasyLogging
Online-DB	Redis in-memory key/value DB

Warning

RAD will be ported to CII as soon as a stable version of CII will be available. Application based on RAD without CII will have to be ported to the new RAD-CII version.

The rest of the document is organized as following:

• Description of how to configure an account
• RAD directory structure and how to retrieve / compile it
• Development steps to follow and code generation from templates
• Description of the interface module
• Tutorial 1 on developing a C++ callback-based application
• Tutorial 2 on developing a C++ Stae Machine-based application
• Tutorial 3 on developing Integration Tests based on Robot Framework
• Additional information on templates
• Examples

1.1. Configuring PREFIX/INTROOT

To configure environment variables LMOD (https://europeansouthernobservatory.sharepoint.com/sites/EELT-ICS-SWFW/Wiki/EELT%20LMOD.aspx) is used. It replaces the VLT PECS tool.

LMOD is based on LUA language. The condifuration of the env. variables can be stored in the ~/modulefiles/private.lua file. For example:

local home = os.getenv("HOME")

local introot = pathJoin(home, "MYPROJECT-INTROOT")
setenv ("INTROOT", introot)
setenv ("PREFIX", introot)

local introotbin = pathJoin(introot, "bin")
prepend_path("PATH", introotbin)

local introotlib64 = pathJoin(introot, "lib64")
prepend_path("LD_LIBRARY_PATH", introotlib64)

local introotlib = pathJoin(introot, "lib")
prepend_path("LD_LIBRARY_PATH", introotlib)

Note

• PREFIX is needed by waf to know where to install. This usually coincide with INTROOT.
• INTROOT is needed by RAD to check where to load the configuration files from

1.2. RAD Directory Structure and SVN

RAD is archived in SVN at: http://svnhq9.hq.eso.org/p9/trunk/EELT/ICS/ifw/tools/rad

In order to compile RAD, the following library (scxml4cpp) is also needed: http://svnhq9.hq.eso.org/p9/trunk/EELT/ICS/ifw/tools/scxml4cpp RAD integration tests are archived at: http://svnhq9.hq.eso.org/p9/trunk/EELT/ICS/ifwTests/rad

RAD is organized in the following directories:

Directory	Description
rad/codegen	Code generator to create the event classes
rad/cpp/core	Library providing event services: -Event handling (AnyEvent, Event, Dispatcher, GetPayload, Traits) -Callback errors (Errors) -Logging and Assert (Logger, Assert) -UNIX signal handling (Signal) -Timers (Timer)
rad/cpp/services	Library providing DB and msgs services: -Access to OnlineDB (DbAdapter, DbAdapterRedis) -Request/reply (MsgReplier, MsgRequestor, MsgRequestorRaw, MsgRequest, MsgHandler) -Pub/Sub (TopicPub, TopicSub, TopicHandler) -Error handling (Exceptions)
rad/cpp/sm	Library providing State Machine services: -Actions (ActionCallback, ActionGroup, ActionMgr) -Activities (Activity, ActivityPthread) -State Machine façade (SMAdapter, SMEvent) -Req/Rep helpers (SMRequestor, SMRequestorRaw)
rad/cpp/utils	Static helper methods (Helper)
rad/cpp/templates	Cookiecutters templates to create: -Application modules based on RAD (rad-cpptpl-appl) -Interface modules (rad-cpptpl-applif) -Integration test modules (rad-robtpltest)
rad/cpp/_examples	Examples of RAD based applications -Interface module for the example applications (exif) -Utility to send commands (exsend) -RAD based application example (server). -Experimental RAD based application (server2) NOT to be used.
scxml4cpp/engine	State Machine engine library (SCXML interpreter)
scxml4cpp/parser	State Machine parser library (SCXML parser)

1.3. Retrieving, Building, and Installing RAD

At the moment, RAD is not part of the development environment, therefore it has to be retrieved from SVN and copied (as external) in your project directory.

For example, if your project is called MYPROJECT:

> cd MYPROJECT
> mkdir tools
> cd tools
> svn co http://svnhq9.hq.eso.org/p9/trunk/EELT/ICS/ifw/tools

In addition your project wscript file (MYPROJECT/wscript) must be updated to look as follow:

from wtools import project
project.declare_project(name='MYPROJECT', version='1.0-dev',

requires='cxx qt5 boost python pyqt5 protoc', # Required features the project needs

recurse='tools', # Recurse into project's packages
boost_libs='program_options system regex filesystem')

The project can now be reconfigured and compiled:

> cd MYPROJECT
> waf configure
> waf

To install it, make sure the PREFIX env. variable is set to the directory you want (usually the INTROOT):

> waf install

2. Development Steps

In order to develop an application based on RAD, the following steps can be performed:

1. Add RAD to your waf project ◦ Add to a waf project root directory the tools subdirectory with ifw/tools/rad and ifw/tools/scxml4cpp as SVN externals
2. Generate an Interface module ◦ Using the templates in rad/cpp/templates/config/rad-cpptpl-applif
3. Generate the Application module ◦ Using the templates in rad/cpp/templates/config/rad-cpptpl-appl
4. Generate Integration Test module ◦ Using the templates in rad/cpp/templates/config/rad-robtpl-test
5. Build and install the generated artifacts
6. Test the generated artifacts
7. Customize Application, Test, and Interface modules
8. Go To step 5

Note

• Step 1 can be skipped for IFW applications/subsystems since RAD is already part of IFW.
• Step 2 can be skipped if the interface module already exists.

2.1. Code Generation from Templates

For the moment only code generation from (textual) templates is supported.

It is based on the COOKIECUTTER tools and it allows to generate the initial version of the Application, Interface, and Integration Test modules.

The templates are archived in the rad/cpp/templates/config directory. There is one directory per template:

• rad-cpptpl-applif to generate the interface module
• rad-cpptpl-appl to generate the application module
• rad-robtpl-test to generate the integration test module

2.2. Interface Module

The interface module contains the definition of data structures exchanged by the application via request/reply (parameters) and pub/sub (topics).

Data structures are defined using Google Protocol Buffers and they are stored in .proto file

(s) in the directory: < interface_module>/interface/<interface_module>/<filename>.proto

The .proto files are compiled by Google protoc compiler that generates the .pb.cpp and .pb.h files in the <project>/build/ directory. The protoc compiler is invoked by waf every time you compile (and the .proto files have been modified).

Generated files contain the C++ classes representing the data structures. These classes provide the methods to deserialize (parse) message payloads and to serialize.

For example the exif module contains the interface used by RAD test applications. It contains the followign .proto files: hellorad.proto, requests.proto and topics.proto.

The requests.proto file contains the definition data structures used to send requests and replies, for example the Init command (without parameters) and the related reply (with a string parameter):

syntax = "proto3"

package exif;

message ReqInit {
}

message RepInit {
    string reply = 1;
}

This is complied by the protoc compiler which generates, in the build directory the following C++ files:

<project>/build/…/exif.pb.cpp

<project>/build/…/exif.ph.h

These files are used by the application to send/receive commands/replies.

2.3. Tutorial 1 - Creating Callback Application reusing an existing Interface Module

The following commands create a C++ callback based application that reuses an existing interface module.

> mkdir rad_example
> cd rad_example
> svn co http://svnhq9.hq.eso.org/p9/trunk/EELT/ICS/ifw . <- Appl part of IFW project
> waf configure
> cd tools/rad/cpp/_examples
> cookiecutter ../templates/config/rad-cpptpl-appl
module_name [hello]: hello                                <- module name
application_name [hello]: hello                           <- application binary name
package_name [examples]:                                  <- name of package containing the module
interface_name [helloif]: exif                            <- reusing existing interface module
'exif' libs [tools.rad.cpp._examples.exif]:               <- path to the interface module library (wscript)
Select state_machine:                                     <- 1 = State Machine appl., 2 = Callbacak appl.

1. - y

2. - n

..

Choose from 1, 2 [1]: 2                                   <- Callback based application
> waf install
> redis-server &                                          <- Start the Online-DB if not already running
> dbbrowser &                                             <- Start the Online-DB browser to monitor the application
> hello -c hello/config.yaml -l DEBUG &                   <- Start the application
> exSend 10 127.0.0.1 5599 exif.ReqStatus ""              <- Send command to the application

The generated files are described in the table below.

File	Class	Description
hello/wscript	n/a	WAF file to build the module.
hello/config/hello/hello.yaml	n/a	YAML application configuration.
hello/src/config.[hpp|cpp]	Config	Class to handle the (read-only) application configuration.
hello/src/dataContext.[hpp|cpp]	DataContext	Class to handle the (read/write) run-time data generated by the application.
hello/src/dbInterface.[hpp|cpp]	DbInterface	Interface class read/write from/to the Online-DB.
hello/src/events.rad.ev	n/a	File describing all the events handled by the application.
hello/src/msgParsers.[hpp|cpp]	MsgParsers TopicParsers	Classes to parse requests and topics and map them into the events described in events.rad.ev.
hello/src/callbacks.[hpp|cpp]	Callbacks	Class containing the implementation of the callbacks.
hello/src/main.cpp	n/a	Contains the main() function that instantiates all the objects and starts the event loop.

wscript File

The wscript module file contains the information needed by waf (and wtools) to build the application:

from wtools.module import declare_cprogram

declare_cprogram(target='hello',                   <- name of the compiled binary

features='radgen',                                 <- enable events code generation from .rad.ev files

use='BOOST                                         <- RAD uses many boost libraries

protobuf                                           <- Google Protocol Buffers library

tools.rad.cpp.core

tools.rad.cpp.services                             <- RAD libraries

tools.rad.cpp.utils

libzmq libczmq                                     <- ZMQ libraries

hiredis                                            <- REDIS client library (Online-DB)

yaml-cpp                                           <- YAML library (configuration)

tools.rad.cpp._examples.exif')                     <- Interface module library containing the generated Google Protocol Buffer classes

Config Class

The Config class deals with (read only) application configuration. It has:

• One attribute per configuration parameter
• One Get method per configuration parameter

Application can be configured by:

1. Initialize in Config constructor its attributes using hard coded default values defined in Config.hpp
2. Reading environment variables (via the rad::Helper::GetEnvVar ) in the Config constructor to initialize (some of) its attributes.
3. Reading the command line options (via boost::program_config) in the Config::ParseOptions() method to initialize (some of) its attributes.
4. Reading the configuration file parameters (via YAML library) in the Config::LoadConfig() method.

The developer can extend this class with additional configuration parameters.

The Config object is created in the main() at the very beginning:

int main(int argc, char \*argv[]) {

RAD_LOG_TO_CONSOLE(true);                            <- Use RAD to set logging

RAD_LOG_SETLEVEL("INFO");                            <- log level

RAD_LOG_INFO() << "Application hello started.";

try {

  hello::Config config;                              <- Create Config object

  if (config.ParseOptions(argc, argv) == -1) {       <- Parse command line params

    // request for help
    return EXIT_SUCCESS;

  }

config.LoadConfig();                                 <- Load configuration file

…

DbInterface Class

The DbInterface class deals with reading/writing from/to the key/value Online DB.

The generated class, has Get and Set methods to read/write the application state and configuration:

• GetControlState(), GetControlSubstate(), …
• SetControlState(), SetControlSubstate(), …
• SetConfig(Config& cfg)

The developer should add all the Get/Set methods required by the application.

The DbInterface constructor takes as parameter:

• A string representing the prefix to be added to all the “keys” before writing in DB
• A reference to an object that allows to talk to the Online DB. This object should implement the rad::DbAdapter interface. RAD provides a class which is specialized to talk to Redis DB: rad::DbAdapterRedis

The DbInterface object is an attribute of the DataContext (see next slide).

DataContext Class

The DataContext class:

• Contains runtime data handled by the application (to be added by the developer)
• Provide access to the Online DB by having as attribute an instance of the DbInterface. This allows to read/write the run-time data from/to Online-DB. It provides, for example, the DataContext::UpdateDb() method to write the application configuration information in the Online-DB.
• Contains a reference to the Config object to be able to access the configuration information (DataContext::GetConfig() method) and to reload the configuration from file (DataContext::ReloadConfig() method). The DataContext object is created in the main() function after the Config object:

…

rad::DbAdapterRedis redis_db; // Runtime DB

hello::DataContext data_ctx(config, redis_db); // Runtime data context

…

Events

RAD based applications reacts to events. Events can be:

• Requests
• Replies
• Topics
• Timeouts
• Unix signals (CTRL-C, etc.)
• Internal events (events generated by the application itself)

The events supported by the application should be added by the developer to the events.rad.ev text file (or, more in general, text files with extension .rad.ev).

Similarly to the .proto files, also .rad.ev files are transformed into C++ code by a “compiler” called radgen. radgen is invoked by waf as part of the build process every time the .rad.ev files are modified. The generated code (events.rad.cpp and events.rad.hpp files) is located in the <project>/build directory.

The events.rad.ev file contains:

version: "1.0"

namespace: Events                               <- Prefix to be used in the State Machine model

includes:
- exif/requests.pb.h                            <- Definition of the types of the event payloads

events:

Init:                                           <- Name of the event: "Events.Init"
    doc: Event for the ReqInit request message. <- (optional) comment
    payload: exif::ReqInit                      <- Payload data type (interface module in this case)
    origin: request                             <- Whether the event is triggered by a command
    context: statemachine                       <- Whether it should be dispatched to the State Machine

CtrlC:                                          <- Name of the event: "Events.CtrlC" , no payload
    context: statemachine

…

The events.rad.[hpp|cpp] files, generated from events.rad.ev, contains:

#ifndef EVENTS_EVENTS_RAD_HPP\_

#define EVENTS_EVENTS_RAD_HPP\_

#include <rad/AnyEvent.hpp>

#include <rad/MsgRequest.hpp>

#include <exif/requests.pb.h>

namespace Events {

  std::vector<rad::EventInfo> listEvents(); // Lists available/known events.

  // Event for the ReqInit request message.

  class Init final : public rad::AnyEvent {

  public:

    static constexpr char const* id = "Events.Init";

    static constexpr rad::EventInfo::Context ctx = rad::EventInfo::Context::statemachine;

    using payload_t = rad::MsgRequest<exif::ReqInit>;

    explicit Init(rad::MsgRequest<exif::ReqInit> const&);

…

Event Loop

The event loop is responsible for continuously listen to requests, replies, topics, timeouts, UNIX signals, etc. and invoke the associated RAD callback:

• The event loop is implemented using boost::asio.
• Boost::asio event loop invokes RAD callbacks.
• RAD callbacks creates Application Events objects (defined in the events.rad.ev file) using the MsgParsers and TopicParsers classes and dispatch them, via rad::Dispatcher, to the Application Callbacks or to the State Machine engine.

OS/ZMQ	BOOST	RAD	Application	Application
Low level “events”	boost::asio Event loop	RAD callbacks	Application Events (defined in the .rad.ev files)	Application Callbacks (or State Machine)
-ZMQ messages (requests, replies, topics)
-Unix signals
-Timeouts

MsgParsers Class

The MsgParsers and TopicParsers classes parse incoming requests, replies, and topics ZMQ messages received by the application and create the corresponding Application Events, defined in the events.rad.ev file.

The created Application Event is then dispatched via the rad::Dispatcher to the registered Application Callback.

A parser looks like:

class MsgParsers final : public rad::MsgHandler {

public:

  MsgParsers(rad::Dispatcher& event_dispatcher);  <- takes a reference to the rad::Dispatcher object


  void handle(const std::string& identity,        <- invoked by rad::MsgReplier when a request is received; it parses the requests and invoke

             const std::string& payload_type,

             rad::Dispatcher

             void const\* msg, size_t msg_size) override;

private:

  rad::Dispatcher& m_dispatcher;

};

void MsgParsers::handle(const std::string& identity,

                        const std::string& payload_type,

                        void const\* msg,

                        size_t msg_size) {

  RAD_LOG_TRACE();

  RAD_LOG_DEBUG() << "dispatching payload type <" << payload_type << "> from <" << identity << ">";

  if (payload_type == "exif.ReqInit") {

    Events::Init::payload_t payload;

    payload.SetOriginatorId(identity);

    m_dispatcher.dispatch(Events::Init(payload));

  } else if (payload_type == "exif.ReqExit") {

….

  }

}

The developer has to add in the MsgParsers::handle() method the parsing of new commands and in TopicParsers::handle() for the topics.

Callbacks Class

The Application Callbacks have to be implemented by the developer.

The application template provides some basic callbacks in the Callbacks class:

Callbacks::Callbacks(boost::asio::io_service& ios,

                     rad::Dispatcher& dispatcher,

                     rad::MsgReplier& msgReplier, DataContext& data)

: m_io_service(ios),

  m_msg_replier(msgReplier),

  m_signal(ios, dispatcher, rad::UniqueEvent(new Events::CtrlC())),

  m_data(data) {

    using std::placeholders::_1;                                                         <- Callback registration

    dispatcher.registerHandler("Events.Init", std::bind(&Callbacks::Init, this, \_1));

…

}

void Callbacks::Init(const rad::AnyEvent& lastEvent) { <- Callback for the Init cmd exif::RepInit rep;

rep.set_reply("OK");

size_t nBytes =
m_msg_replier.Send(rad::Helper::GetRequestId<Events::Init>(lastEvent),
rep); RAD_ASSERT(nBytes > 0);

}

main.c file

 …

// Create events and related objects

rad::Dispatcher event_dispatcher; <- Used to dispatch Events to the
Callbacks // event loop

boost::asio::io_service io_service; <- boost::asio Event Loop

// incoming commands handlers

rad::MsgReplier in_req_handler( <- RAD object to receive

config.GetMsgReplierEndpoint(), io_service, <- ZMQ requests

std::unique_ptr<rad::MsgHandler>(

new hello::MsgParsers(event_dispatcher)));

// Callback related objects

hello::Callbacks cbs(io_service, event_dispatcher, <- Object with the
callbacks

in_req_handler, data_ctx);

in_req_handler.Start(); <- Start receiving ZMQ requests

io_service.run(); <- Start the Event Loop (stopped by ReqExit or Ctrl-C
callbacks)

in_req_handler.Stop(); <- Stop receiving ZMQ requests

…

Adding a New Command

To summarize, adding the processing of a new command implies:

1. Update the Requests.proto file in the interface module with the new data structures representing the command (exif.ReqMyNewCommad) and the reply (exif.RepMyNewReply)
2. Update the events.rad.ev file with the Application Event associated to the command (Events.MyNewEvent)
3. Update the MsgParsers::handle() method with the parsing of the new command
4. Update the Callbacks class by adding

◦ the new Application Callback method

◦ register the new callback method in the constructor of the Callbacks class

2.4. Tutorial 2 - Creating State Machine Application reusing and Interface Module

The following commands create a C++ state machine based application that reuses an existing interface module.

> mkdir rad_example

> cd rad_example

> svn co http://svnhq9.hq.eso.org/p9/trunk/EELT/ICS/ifw .             <- Appl part of IFW project > waf configure

> cd tools/rad/cpp/_examples

> cookiecutter ../templates/config/rad-cpptpl-appl
  module_name [hello]: hello                                          <- module name
  application_name [hello]: hello                                     <- application binary name
  package_name [examples]:                                            <- name of package containing the module
  interface_name [helloif]: exif                                      <- reusing existing interface module 'exif'
  libs [tools.rad.cpp._examples.exif]:                                <- path to the interface module library (wscript)
  Select state_machine:                                               <- 1 = State Machine appl., 2 = Callbacak appl.

1. - y

2. - n

Choose from 1, 2 [1]: 1                                               <- State Machine based application

> waf install

> redis-server &                                                      <- Start the Online-DB if not already running

> dbbrowser &                                                         <- Start the Online-DB browser to monitor the application

> hello -c hello/config.yaml -l DEBUG &                               <- Start the application

> exSend 10 127.0.0.1 5599 exif.ReqStatus ""                          <- Send command to the application

The list of generated files are similar to Tutorial 1 without the callbacks.[hpp|cpp] files but with, in addition, the following files:

File	Class	Description
hello/config/hello/sm.xml	n/a	SCXML state machine model.
hello/src/actionsStd.	ActionsStd	Class containing the methods implementing the actions defined in the state machine model.
[hpp|cpp]
hello/src/actionMgr.	ActionMgr	Class to handle the (read/write) run-time data generated by the application.
[hpp|cpp]
hello/src/main.cpp	n/a	Contains the mail() function the Instantiates all the objects and starts the event loop.

2.4.1. State Machine Model

The State Machine model is described using SCXML notation define here: https://www.w3.org/TR/scxml/ A basic State Machine model (from the template) is in:

<module>/config/<module>/sm.xml

In the application configuration file <module>/config/<module>/config.yaml the following entry is added to be able to load the State Machine model:

cfg.sm.scxml : “hello/sm.xml”

In principle the sm.xml model could be generated:

• from UML/SysML State Diagram using COMODO tool (see WSF2 user manual)
• (under investigation by Jens) SCXML plugin for Qt designer

The SCXML state machine model is executed by the scxml4cpp SCXML interpreter:

http://svnhq9.hq.eso.org/p9/trunk/EELT/ICS/ifw/tools/scxml4cpp which contains 2 libraries: the interpreter (scxml4cpp/engine) and the parser (scxml4cpp/parser). The parser is based on a generic XML parser called xerces-c.

In addition, RAD provides a library to interface to the scxml4cpp engine called “sm” (state machine).

wscript file is therefore updated to use these additional libraries:

• tools.rad.cpp.sm
• tools.scxml4cpp.engine
• tools.scxml4cpp.parser
• xerces-c

2.4.2. Actions

In a Callback Application, the processing of an event triggers the invocation of a callback (a method of a class). There is one-to-one relation between event and callback: event -> callback.

In a State Machine Application, the processing of an event triggers the invocation and action (a method of a class). The difference is that which action (callback) to invoke depends from the event and the current state. The relation is two-to-one: (event, current state) -> action.

Example of actions to process some standard events are in actionsStd class.

void ActionsStd::Init(const rad::AnyEvent& last_event) {

  exif::RepInit rep;

  rep.set_reply("OK");

  size_t nBytes = m_msg_replier.Send(

  rad::Helper::GetRequestId < Events::Init > (last_event), rep);

  RAD_ASSERT(nBytes > 0);

}

Developer can create similar action classes with new action methods (callbacks).

2.4.3. Activities

Actions can be used to implement short tasks since they are blocking the main thread (until the class method returns). For example: actionsStd::Status() to implement the Status command.

For long lasting tasks, it is possible to use (do-) Activities. In RAD an activity is mapped to a thread. An activity is started, by the State Machine Engine, when entering a state and it is stopped when exiting the state.

To create an Activity, the developer has to specialize the rad::Activity (or the rad::ActivityPthread) class and implement the run() method.

2.4.4. ActionMgr

The ActionMgr class is responsible to instantiate the action and activity classes.

void ActionMgr::CreateActions(boost::asio::io_service& ios,

                              rad::Dispatcher& dispatcher,

                              rad::MsgReplier& msg_replier,

                              DataContext& the_data) {

  ActionsStd\* actions_std = new ActionsStd(ios,
                                            dispatcher,
                                            msg_replier,
                                            the_data);     <- instantiate action class

  if (actions_std == nullptr) {

    RAD_LOG_ERROR() << "Cannot create actions_std object.";

    return;

  }

  AddActionGroup(actions_std);

…

  the_action = new rad::ActionCallback("ActionInit",       <- create binding between action method and the State Machine (SCXML) model
                                       std::bind(&ActionsStd::Init, actions_std, \_1));

  AddAction(the_action);

….

}

In the SCXML model, the action is defined as follow:

<transition event="Events.Init" target="Ready">

<customActionDomain:ActionInit name="ActionInit"/>                       <- in the SCXML model
</transition>

In this example there are no activities, but if there were some (see server example) they would be instantiated here:

void ActionMgr::CreateActivities(DataContext& the_data,
                                 rad::SMAdapter& sm) {

  RAD_LOG_TRACE();

  scxml4cpp::Activity\* theActivity = nullptr;

  theActivity = new ActivityMoving("ActivityMoving", sm, theData);
  AddActivity(theActivity);

}

In the SCXML model, the activity ActivityMoving within the state Moving, would be defined as follow:

<state id="Moving">
  <invoke id="ActivityMoving"/>                                         <- in the SCXML model
</state>

2.4.5. main.c file

// SM event queue and context scxml4cpp::EventQueue external_events;    <- objects needed by the SM engine
scxml4cpp::Context state_machine_ctx;

// State Machine façade <- RAD adapter to the SM engine
rad::SMAdapter state_machine(io_service, event_dispatcher, &state_machine_ctx, external_events);

// Actions and Activities hello::ActionMgr action_mgr;                  <- creates actions manager, actions, activities objects
action_mgr.CreateActions(io_service,
                         event_dispatcher,
                         in_req_handler,
                         data_ctx);

action_mgr.CreateActivities(data_ctx,
                            state_machine);

// Load SM model
state_machine.Load(config.GetSmScxmlFilename(),
                   &action_mgr.GetActions(),
                   &action_mgr.GetActivities());

// Register Events callbacks
std::vector<rad::EventInfo> events = Events::listEvents();               <- get list of events to be sent to the SM

state_machine.RegisterEvents(events);                                    <- tells rad::Dispatecher() to inject the events to the SM

// Register SM Status & Event notification callbacks

hello::ActionsStd\* actionsStd = dynamic_cast<hello::ActionsStd*>(action_mgr.FindActionGroup("ActionsStd"));
RAD_ASSERTPTR(actionsStd);

state_machine.AddStatusListener(actionsStd);                             <- register a callback invoked by the SM upon change of state

state_machine.AddEventListener(actionsStd);                              <- register a callback invoked by the SM upon processing event

state_machine.Start();                                                   <- start the SM engine (i.e. start processing injected events)

in_req_handler.Start();

io_service.run();

in_req_handler.Stop();

state_machine.Stop();                                                    <- stop the SM engine

2.4.6. Adding a New Command

To summarize, adding the processing of a new command implies:

1. Update the Requests.proto file in the interface module with the new data structures representing the command (exif.ReqMyNewCommad) and the reply (exif.RepMyNewReply)
2. Update the events.rad.ev file with the Application Event associated to the command (Events.MyNewEvent)
3. Update the MsgParsers::handle() method with the parsing of the new command
4. Update the State Machine model (sm.xml)
5. Update the action class (e.g. ActionsStd) by adding a new method
6. Update the ActionMgr class by adding the binding to the new method

2.5. Tutorial 3 - Integration Tests

This tutorial shows how to create integration tests for a RAD based application.

> mkdir rad_test                                                 <- the directory should be outside the waf project tree
> cd rad_test
> cookiecutter
rad_example/tools/rad/cpp/templates/config/rad-robtpl-test/
module_name [hello]: test_hello                                  <- name of integration test module
module_to_test [hello]: hello                                    <- name of the module to test
application_to_test [hello]: hello                               <- name of the binary to test
interface_prefix [helloif]: exif                                 <- name of the interface module used by the app
> robot test_hello/src/stdcmds.robot                             <- run the integration tests

Note

• the template assumes that exSend can be used to send commands (i.e. it assumes exif interface).
• the application under test (“hello” in this case) must be installed before running the test.

The commands above generates the files listed in the following table.

File	Description
test_hello/stdcmds.robot	Robot file containing the test cases: -Startup application -Send a command to the application -Shutdown the application
test_hello/utilities.txt	Robot file containing the definition of some custom variables and keywords.

2.6. Interface Template (rad-cpptpl-applif)

The interface module contains the Google Protocol Buffers data type definitions (.proto files) that are used to describe the parameters of requests, replies, and topics.

One of this interface module can contain the interface of more than application.

To create a helloif interface module in PROJECT1/pkg1/ directory execute the following commands (press enter to accept the default values):

> cd PROJECT1/pkg1
> cookiecutter PROJECT1/tools/rad/cpp/templates/config/rad-cpptpl-applif/
module_name [helloif]:
library_name [helloif]:
package_name [examples]: pkg1

The first option allows to specify the name of the interface module. This is used to create the module directory.

The second option allows to specify the name of the library for the interface module. This is used to give a unique name to the library.

The third option allows to specify the name of the package containing the interface module. This is used to identify the parent of the module to be able to generate the doxygen documentation.

What is generated is the helloif directory with the following content:

File	Description
helloif/wscript	waf build file that builds (invoking the protoc compiler) all the .proto files in interface/ directory and create the realted C++ library helloif.
helloif/interface/helloif/Requests.proto	File containing the definition of the standard commands/replies and the related parameters.

2.7. Application Template (rad-cpptpl-appl)

To create a new hello application based on RAD in PROJECT1/pkg1/ directory execute the following commands (press enter to accept the default values):

> cd PROJECT1/pkg1
> cookiecutter
PROJECT1/tools/rad/cpp/templates/config/rad-cpptpl-appl/
module_name [hello]:
application_name [hello]:
package_name [examples]: pkg1
interface_name [helloif]:
libs [tools.rad.cpp._examples.helloif]: pkg1.helloif
Select state_machine:

1. - y

2. - n

..

Choose from 1, 2 [1]: 2

The first option allows to specify the name of the application module. This is used to create the module directory, set the headers guards, and namespaces.

The second option allows to specify the name of the application binary. It can be same as the module name but it has to be unique within the INTROOT/bin directory.

The third option allows to specify the name of the package containing the application module. This is used to identify the parent of the module to be able to generate the doxygen documentation.

The fourth option allows to specify the name of the interface used by the application. This is used to prefix the requests/replies/topics. The fifth option allows to specify the full path to the interface library used by the application. The interface library contains the Google Protocol Buffers data types (see section above on the interface module).

The last option allows to select whether the application should be based on State Machine (option 1) or on simple callbacks (option 2).

Note that if you want to be able to send commands to your application you need to create kind of msgSend utility. To start with you can reuse the exSend tool developed to run the RAD examples. In this case you should select exif as interface name and tools.rad.cpp._examples.exif when running cookiecutters template.

What is generated is the hello directory with the content that vaires depending on whether the application is based on call-backs or on State Machines.

For application based on call-backs, the following files are generated:

File	Description
hello/wscript	waf build file that builds the application.
hello/config/hello/config.yaml	Application configuration file in YAML format. By default it contains the following key/values: msg.req.endpoint: “tcp://*:5577” # IP address and port used to accept requests db.timeout_sec :2 #timeout in seconds when connecting to runtime DB In case of applications based on State Machine it contains also the path to the SCXML model: sm.scxml : “hello/sm.xml”
hello/src/Callbacks.hpp|cpp	The methods of the Callbacks class are the call-backs invoked when an event is generated. For example, when the ReqInit command is received, the corresponding Events.Init is created and the Callbacks.Init() method is invoked.
hello/src/Config.hpp|cpp	The Config class represents the application read-only configuration. It groups three types of configurations: the default hard-coded values defined in Config.hpp the command line options (Config::ParseOptions() method) the YAML configuration file (Config::LoadConfig() method) The configuration values can be accessed via the Config::GetXXX() methods. Note that the keys used in the YAML configuration file are defined in DbInterface.hpp and are
	the same used to read/write in the runtime DB.
hello/src/DbInterface.hpp|cpp	The DbInterface class is used to read/write key/values in the runtime DB. The keyes are defined in the header file.
hello/src/DataContext.hpp|cpp	The DataContext class is used to share runtime read/write information between call-backs, actions, activities. It provides access to the Config object and to the DbInterface.
hello/src/MsgParsers.hpp|cpp	The MsgParsers and TopicParsers classes provide the methods to parse incoming ZMQ messages, create the corresponding events (defined in the Events.rad.ev file) and propagate the events to the call-backs or to the State Machine engine.
hello/src/Events.rad.ev	This DSL file described the events handled by the applications. There should be one event per request and per topic type to be able to process incoming commands and topics. In addition it should list events generated by timers, Linux signals, or by actions and activities.
hello/src/main.cpp	This file contain the main () function that: instantiates the objects described above nd starts the event loop register the call-backs methods (or the State Machine actions) with the related events starts the event loop.

For application based on State Machines call-backs, the following additional files are generated and the Callbacks.hpp|.cpp are removed:

File	Description
hello/config/hello/sm.yaml	This file is the SCXML model of the State Machine.
hello/src/ActionsStd.hpp|cpp	The ActionsStd class replaces the Callbacks class. It contains the actions invoked by the State Machine engine upon the reception of an event.
hello/src/ActionMgr.hpp|cpp	The ActionMgr class instantiates the actions and do-activities defined in the SCXML model. For example it create an ActionsStd object.

2.8. Integration Test Template (rad-cpptpl-applif)

The integration test template rad-cpptpl-applif can be used to create integration test modules based on the ELT Test Framework (ELT) and Robot framework tool.

To create an integration test module for the hello application in the tests/ directory execute the following commands (press enter to accept the default values):

> cd PROJECT1
> mkdir tests
> cd tests
> cookiecutter PROJECT1/tools/rad/cpp/templates/config/rad-robtpl-test/
module_name [hello]:
application_name [hello]:
interface_prefix [helloif::]:

The first option allows to specify the name of the module or the integration test.

The second option allows to specify the name of the application to be tested.

The third option allows to specify the prefix (namespace) to be used when sending the commands to the application under test.

What is generated is the hello directory with the following content:

File	Description
hello/etf.yaml	The ETF configuration file containing the specification of which plugins to use to run the tests (Robot framework our case) and the list of test files to execute (hello.robot).
hello/src/hello.robot	This file contains the definition of test cases to verify the proper execution of the standard commands such as Stop, Init, Enable, Disable, etc..
hello/src/utilities.txt	This file contains some pre-defined Robot keywords that can be used to send commands to the application. It also contain some configuration information like IP address and port of the application to be tested.

In order to run the test directly using Robot the following commands can be executed:

> cd PROJECT1/tests/hello/src
> robot hello.robot

In order to run the test using ETF the following commands can be executed:

> cd tests/hello/

Note that the tests is using exSend utility to send the commands. Therefore the test will run only if your application is using exif interface module.

2.9. Customizing Application Interface

In order to add a new command to the application the following steps have to be performed:

• Add the request and related reply in the interface module (e.g. helloif/interface/helloif/Requests.proto file)

• Add the event corresponding to the request in the event definition file (e.g. hello/src/Events.rad.ev file)

• In case of call-back based application ◦ add a new method in the Callbacks class or add a new call-back class

  ◦ update the main() function with the registration of the new call-back

• In case of State Machine based application ◦ update the SCXML model with the transition dealing with the new request (e.g. hello/config/hello/sm.yaml)

  ◦ add a new method in the ActionsStd class or add a new actions class ◦ update the ActionsMgr class with the registration of the new action

2.10. Building and Installing Applications

RAD templates provide the wscript used by waf and wtools building system. To build and install the hello application:

> cd PROJECT1/pkg1/hello
> waf install

Make sure that the PREFIX environment variable is set to the installation directory (which could coincide with the INTROOT).

2.11. Starting-up and Shutting-down Applications

Before starting your application, make sure the following environment variables have been properly set:

Variable	Description
INTROOT	Should point where waf is installing the applications and the configuration files. This is used by the application when loading the configuration files (config.yaml and sm.xml)
DB_HOST	Optional variable defining the IP address and port to connect to the runtime DB. For example: “127.0.0.1:6379”

If you want to use the runtime DB, Redis server has to be started with the following command:

> redis-server

A textual redis client can be started as follow:

> redis-cli

or a graphical one:

> dbbrowser &

At this point the application can be started:

> hello -c hello/config.yaml &

The default command line optiotions are as follow:

   -h [ --help ] Print help messages
   -n [ --proc-name ] arg Process name
   -l [ --log-level ] arg Log level: ERROR, WARNING, STATE, EVENT, ACTION, INFO, DEBUG, TRACE
   -c [ --config ] arg Configuration filename
   -d [ --db-host ] arg In-memory DB host (ipaddr:port)

To get the status of the application (assuming the application was created with the exif as interface module):

> exSend 5000 127.0.0.1 5577 exif.ReqStatus ""

3. Integration Tests

RAD has an integration test module based on the server example (described below).

SVN: http://svnhq9.hq.eso.org/p9/branches/hlfw/rap/tests/rad

4. Libraries

RAD applications use the services provided by RAD libraries.

The API of RAD libraries is described in doxygen documentation.

5. Examples

This section contains some example applications created using RAD.

Real applications based on RAD can be found in SVN at: http://svnhq9.hq.eso.org/p9/trunk/EELT/CS/TCS/M1LCS/Software

5.1. exif

This is an example of interface module with the definition of the commands, reples, topics used by the example applications.

SVN: http://svnhq9.hq.eso.org/p9/trunk/EELT/ICS/ifw/tools/rad/cpp/_examples/exif

It contains Requests.proto for the definition of the requests/replies and Topics.proto for the definition of the pub/sub topics.

5.2. exsend

This is an example of how to build a utility application (similar to the VLT msgSend) able to send requests and receive replies defined in exif interface module. This application is using some RAD libraries but it is not generated from the RAD templates.

SVN: http://svnhq9.hq.eso.org/p9/trunk/EELT/ICS/ifw/tools/rad/cpp/_examples/exsend The exsend module implements the exSend application that can be used as follow:

exSend <timeout> <IP> <port> <command> <parameters>

where:

   <timeout> reply timeout in msec
   <IP> IP address
   <port> port <command> command to be sent to the server (exif.ReqInit, exif.ReqExit, ...)
   <parameters> parameters of the command

for example to query the status (with 5 sec timeout) of an application running on the same local host on port 5577:

exSend 5000 127.0.0.1 5577 exif.ReqStatus ""

5.3. server

This is an example that shows how to create RAD based applications that uses request/reply, pub/sub, timers, Linux signals. It uses the interface defined in exif interface module and can be controlled by sending the commands via the exSend application (see exsend module).

SVN: http://svnhq9.hq.eso.org/p9/trunk/EELT/ICS/ifw/tools/rad/cpp/_examples/server

The server module has two possible configuration files and state machines:

server/config/radServer/config.yaml server/config/radServer/sm.xml

server/config/radServer/config1.yaml server/config/radServer/sm1.xml

The first configuration (config.yaml and sm.xnl) is used to instantiate a prsControl application that is able to process exif.ReqPreset commands. When a exif.ReqPreset command is received, the application executes the ActionPreset::Start action which sends a exif.ReqMove to a second application (altazControl) that simulate the movement of the axes of a telescope. While waiting for the completion of the preset, it monitors the axes position by subscribing to topic XYMeas topic published on port 5560. The topic is processed by the XYMeas topic is processed by the ActionPreset::Monitor action. See the picture below for a more complete overview of the behaviour of prsControl application.

The second application (altazControl) is configured using config1.yaml and sm1.xml files. It receives the exif.ReqMove command and executes the ActionsMove::Start action and starts a do-activity: the ActivityMoving thread. The thread simulates the movement of the axes and publishes the intermediate positions via the XYMeas topic. When the target position is reached, the do-activity terminates and a reply (exif.RepMove) to the originator of the exif.ReqMove command is sent by the ActionsMove::Done action. See the picture below for a more complete overview of the behaviour of prsControl application. See the picture below for a more complete overview of the behaviour of altazControl application.

Note that both applications store the configuration, status, and telescope position information in the Redis runtime DB.

The sequence of messages to initialize, enable, and start the preset is shown below.

The sequence of messages to preset the axes is shown below.

In order to run the server example refer to the RAD integration test section.

5.4. hellorad + server

This is an example that shows how a Python client can talk to a C++ server. The client sends a ReqTest request containing “Ping pong” text, the server receives the requests and replies with a RepTest reply. To run the example first start the server:

radServer -c radServer/config.yaml -l DEBUG &

and the start the client (http://svnhq9.hq.eso.org/p9/trunk/EELT/ICS/ifw/tools/rad/py/_examples/hellorad):

hellorad client --req-endpoint='tcp://localhost:5577'