4. MAL Mappings

Document ID:

Revision:

2.10

Last modification:

March 18, 2024

Status:

Released

Repository:

https://gitlab.eso.org/cii/info/cii-docs

File:

mal_mappings.rst

Project:

ELT CII

Owner:

Marcus Schilling
Document History

Revision

Date

Changed/ reviewed

Section(s)

Modification

0.1

2018-07-21

msekoranja

All

Created.

0.9

2018-08-03

msekoranja

3

Added Python sections, properties.

1.0

2018-08-10

msekoranja

All

Minor updates, release.

2.0

2019-02-02

msekoranja

All

Updated property names, removed duplicated parts from MAL API manual and Transfer document.

2.1

2019-04-16

msekoranja

3

References

Updated OPC UA C++ properties.

References update

2.2

2020-03-24

msekoranja

3

Added ZPB C++/Java send queue size property.

2.3

2021-09-07

dkumar

3

Added OPCUA logging description for open62541

2.4

2021-09-24

dkumar

3

Added ZPB C++/Java disable type check property.

2.5

2022-04-26

dkumar

3

Added ZPB C++ enable ephemeral port check

2.6

2022-06-23

dkumar

3

Added MudPi C++ broadcast interface property

2.7

2022-06-30

mschilli

3

Added URI subsections

2.8

2023-07-04

nkornwei

4.3.1

Updated URI subsection

2.9

2024-03-14

mschilli

4.3.1.2

participant name - new property

2.10

18.03.2024

mschilli

0

Public doc

Confidentiality

This document is classified as Public.

Scope

This document is a manual for the Middleware mappings of the ELT Core Integration Infrastructure software.

Audience

This document is aimed at Users and Maintainers of the ELT Core Integration Infrastructure software.

Glossary of Terms

API

Application Programmers Interface

CII

Core Integration Infrastructure

DDS

Data Distribution Service

MAL

Middleware Abstraction Layer

MUDPI

Multicast UDP Interface

URI

Uniform Resource Identifier

ZPB

ZeroMQ and Protocol Buffers

OPC UA

OPC (Object Linking and Embedding for Process Control) Unified Architecture

References

  1. Cosylab, MAL Design document, CSL-DOC-17-147260 v1.5

  2. Cosylab, ELT CII MAL Transfer document, CSL-DOC-18-168015, version 1.8

  3. Cosylab, Python MAL Bindings Design document, CSL-DOC-18-152412, version 1.3

  4. Cosylab, MAL API User Manual document, CSL-MAN-17-150198, version 2.1

4.1. Overview

This document describes configuration options for the following MAL mappings: DDS, ZPB and OPC UA. All the details on the MAL API and its functionality are covered by the MAL design document [1] and MAL User Manual [4].

4.2. Installation

See MAL Mappings Transfer document [2] on how to install MAL mappings.

4.3. C++

4.3.1. DDS

4.3.1.1. URIs

The underlying middleware is multicast. Discovery is by domain id, and topic-oriented. Publisher and Subscriber use the same URI to indicate which topic they both join. Peers (publishers or subscribers) in different domains will not discover each other. Request-Reply is not supported (any more).

  • Publisher: dds.ps://<DOMAIN_ID>/DDS-TOPIC-NAME

  • Subscriber: same as Publisher

  • Requester: Not supported

When IPv6 is enabled (through native configuration of the middleware), no changes to the URIs are needed.

Details

CII URI syntax is of the following form: <scheme>:[//<authority>]</path>[?<query>][#fragment]. The Authority is used to indicate the DDS domain, while Query and fragment URI components are not used in DDS MAL URIs, therefore the DDS MAL URI is simplified: <scheme>://<domain_id>/path

Schema component of DDS MAL URI uses dds.ps string for specifying a DDS middleware publish subscribe profile. The string dds.rr for a DDS middleware request reply profile is no longer supported. A mandatory URI path component needs to hold the name of the topic (for publish subscribe profile). The domain_id is optional, if no specified it is set by the DDS QoS XML, or defaults to domain id 0.

Examples of MAL DDS URIs:

  • dds.ps://100/Sample - publisher for topicName = Sample in domain id 100

  • dds.ps://100/Sample - subscriber connected to topicName = Sample in domain id 100

4.3.1.2. Mal Specific Properties

The following table enumerates mapping specific properties:

Name

Type

Description

Used by

Default

dds.log4cplus.filename

string

logging configuration file used by DDS MAL

MAL

(none)

dds.qos.profile.library

string

DDS QoS profile library.

MAL/ Pub/Sub

(none)

dds.qos.profile.name

string

DDS QoS profile name.

MAL/ Pub/Sub

(none)

dds.qos.profile.name. participant

string

DDS QoS profile name for participant.

Pub/Sub

(none)

dds.qos.profile.name. topic

string

DDS QoS profile name for topic.

Pub/Sub

(none)

dds.qos.profile.name. writer

string

DDS QoS profile name for writer.

Pub

(none)

dds.qos.profile.name. reader

string

DDS QoS profile name for reader.

Sub

(none)

dds.qos.profile.name. subscriber

string

DDS QoS profile name for subscriber.

Sub

(none)

dds.qos.profile.name. publisher

string

DDS QoS profile name for publisher.

Pub

(none)

dds.wait_for_ack. timeout_ms

integer

DDS wait for ack. before tearing down writer and publisher.

MAL

5000ms

dds.qos.participant. name

string

QoS override: name to assign to participant

Pub/Sub

(none)

dds.qos.participant. interfaceWhiteList

string

QoS override: list of NICs for participant

Pub/Sub

(none)

Table 1 C++ DDS MAL mapping specific properties

DDS QoS profile name selection

Custom QoS profiles are enabled only when a user sets the “dds.qos.profile.library” mal property pointing to the file containing the QoS profiles. If the user sets only the “dds.qos.profile.name” mal property, then the same QoS profile name will be used for the creation of the following DDS entities: participant, topic, writer, publisher, reader, and subscriber. User has the option to override the default QoS profile name when creating a participant (“dds.qos.profile.name.participant”), a topic(“dds.qos.profile.name.topic”), a writer (“dds.qos.profile.name.writer”), a reader(“dds.qos.profile.name.reader”), a subscriber(“dds.qos.profile.name.subscriber”), and a publisher(“dds.qos.profile.name.publisher”). If the user does not override the profile name for a specific DDS entity, the default profile name is used.

DDS Wait for acknowledgment timeout property

If this property is not provided, a default timeout of 5000 ms will be assumed as the timeout value. If the timeout value is larger than 0 then writer->wait_for_acknowledgements() and publisher->wait_for_acknowledgements() will be called before tearing down the writers and the publisher. This means that developers can still suppress calling wait_for_acknowledgements if they specify the new property with a timeout value of 0, otherwise the call to wait_for_acks will always happen from now on.

QoS overrides

The MAL DDS offers to modify a few QoS attributes after they were read from an external QoS file.

  • Participant name: The participant name is displayed in tools like dds_monitor.

  • Participant interface whitelist: Network interfaces the participant uses (whitespace-separated list).

Note: MAL holds one participant for every domain-id. Accordingly, all entitites (publishers, subscribers) in one domain share the same name and the same interfaces.

Note: These attributes only take effect on creation of a participant, i.e. the moment when the first entity (publisher, subscriber) for any given domain-id gets instantiated.

4.3.2. ZPB

4.3.2.1. URIs

The underlying middleware is peer-to-peer. Discovery is based on host/port, plus name-filtering per port. A publisher exposes a topic-name on a local port, a subscriber needs to know where the publisher is. A server exposes one or more server-interfaces on a local port, a requester needs to know where the server is.

  • Publisher: zpb.ps://*:PORT/ZPB-TOPIC-NAME

  • Subscriber: zpb.ps://PUBLISHER-HOST:PUBLISHER-PORT/TOPIC-NAME

  • Requester: zpb.rr://PUBLISHER-HOST:PUBLISHER-PORT/SERVER-NAME/SERVER-IF-NAME

ZMQ supports IPv6, but it needs to be enabled on the ZMQ socket as an option (currently not done by the ZPB-MAL). Once it is enabled on the sockets, IPv6 addresses need to be used in the URIs.

Details

ZPB MAL uses zpb.ps for publish-subscribe and zpb.rr for request-reply string for its URI schema component. A server-based authority component must be specified in the URI, without unsupported user-info element. When binding to the socket, i.e. in case of a publisher and request-reply server, an asterisk wildcard character is allowed to specify ANY network interface to bind to. A mandatory URI path component needs to hold the name of the topic or service. A path can be hierarchical. An empty URI path is allowed when creating a server: a path component is used as a service name prefix for all services registered to that server. Query and fragment URI components are not used.

There are helper methods available to create ZPB URIs, e.g. ZpbUtil.createPsUri(String host, int port, String topicName) and ZpbUtil.createRrUri(String host, int port, String instanceName) .

Examples of ZPB URIs:

  • zpb.ps://*:12345/Sample - publisher bound on ANY interface on port 12345, topicName = Sample:

  • zpb.ps://eltmgr01:12345/Sample - subscriber connected to eltmgr01, port = 12345, topicName = Sample:

  • zpb.rr://localhost:5555/RobotControl - server bound to localhost interface, port = 5555, service name prefix = RobotControl:

  • zpb.rr://10.0.0.1:5555/RobotControl/Robot1 - client connected to 10.0.0.1, port = 5555, serviceName = RobotControl/Robot1:

4.3.2.2. Mal Specific Properties

The following table enumerates mapping specific properties:

Name

Type

Description

Used by

Default

zpb.ps.slow JoinerDelay Ms

integer

Slow joiner delay. Time in millisecond s to wait after publisher has been instantiate d. This mitigates slow joiner delay problem but does NOT guarantee that first N messages will not be lost.

Publisher

0 (no delay)

zpb.schedul edExecutorT hreads

integer

Number of thread pool threads to handle internal timer based tasks (e.g. QoS deadline timeouts).

MAL

3

zpb.rr.call TimeoutSec

(C++ only)

integer

Client asynchronou s method call timeout in seconds.

Client

0 (no timeout)

zpb.ps.zmq. sndhwm

integer

Publisher send queue size. Use to limit memory usage when dealing with large messages. 0 stands for unlimited.

Publisher

1000 (ZMQ timeout)

zpb.ps. disableType HashCheck

boolean

Disable subscriber’s check for type hash correctness

Subscribers

false

zpb.rr. zmqReconnect IvlMs

integer

Set reconnection interval in ms

Client

100 ms

zpb.rr. zmqReconnect IvlMaxMs

integer

Set maximum reconnection interval in ms

Client

0 ms

zpb.rr. enableEpheme ralPortCheck

boolean

Throw an ex exception if ephemeral po rt is used, otherwise log warning

Server

false

Table 2 C++ ZPB MAL mapping specific properties

4.3.3. OPC UA

4.3.3.1. URIs

The underlying middleware is connection-oriented. Discovery via host-port. The URI encodes a mapping between ICD-entity and OPC-UA node, expressed as a semicolon-separated list of tuples (namespaceId, objectId), where objectId is either a nodeId or a methodId, e.g. “2,HelloWorld/ScalarTypes/Int64;2,HelloWorld/ScalarTypes/Double”. The order of the tuples must match the order of the fields in the ICD struct.

  • Publisher: not applicable

  • Subscriber: same as Requester

  • Requester: opcua.rr://SERVER-HOST:SERVER-PORT/NAMESPACE#MAPPING

OPC-UA supports IPv6. To use it, the SERVER-HOST must be specified according to Format for Literal IPv6 Addresses.

4.3.3.2. Mal Specific Properties

The following table enumerates mapping specific properties:

Name

Type

Description

Used by

Default

opc.asyncLo opExecution PeriodMs

integer

The sleep period of the asynchronou s process loop.

MAL

50

opc.asyncCa llSubmitTim eoutMs

integer

Timeout for submitting an asynchronou s call.

Client

1000

opc.asyncCa llRetryPeri odMs

integer

The retry period for failed asynchronou s calls.

MAL

250

opc.log4cpl us.filename

string

log4cplus file name.

MAL

undefined

opc.ps.poll ingPeriodMs

integer

Multi-node subscriber polling period in millisecond s.

Subscriber,

MrvSubscrib er

1000

opc.ps.outs tandingPubl ishRequests

integer

Number of outstanding publish requests.

Publisher, Subscriber, MrvSubscrib er

10

opc.ps. usePollSubsc riber

boolean

Force subscriber to poll for value updates instead of a monitor subscription

Subscriber, MrvSubscrib er

false

opc.ps.reque stedPublishi ngIntervalMs

integer

Server-side sampling interval of a subscripti on (i.e. an interval to sample/poll/ monitor an item)

Subscriber, MrvSubscrib er

5000

Table 3 C++ OPC UA MAL mapping specific properties

4.3.3.3. Data Access

The DA profile is implemented using special dedicated MAL request-response DataAccess interface. The interface is defined as:

public interface DataAccess extends RrEntity {
   <T extends DataEntity<T>> T read(Class<T> dataClass, SampleInfo sample_info);
   <T extends DataEntity<T>> void write(T value);
}

Synchronous (DataAccessSync) and asynchronous (DataAccessAsync) versions are available that directly issue OPC-UA DA read and write requests to the OPC-UA server. OPC UA nodes need to be specified at client instantiation time using URI fragment component using the following format: opcua.rr://<host>:<port>/namespace#nsId1,nodeId1;nsId2,nodeId2;…;nsIdN,nodeIdN;

IDs can be either a string or an integer ID, where nsId stands for namespace ID and nodeId for node ID. Node values are then mapped to the structure of type T at the read method call, or structure of type T mapped to the nodes values at the write method call. Note that order and type of the node values and structure fields must match. In case of mismatch, an exception is thrown. In case of an error or timeout (specified as QoS), an exception is thrown. Connection management is handled by MAL transparently, as for every request-reply interface: connection process is initiated asynchronously and automatically and in case of disconnection a connection is restored automatically.

The SampleInfo class passed with read() provides the source timestamp, one for each node in the request. An example of use is below:

with factory.getClient(uri,DataAccessSync,mal.rr.qos.ReplyTime(THREE_SECONDS),mal.MalProperties()) as da:
   try:
         sample = da.getMal().createDataEntity(dataEntity)
         sample_info=SampleInfo()
         da.read(sample, sample_info)
         log.info('READ %s info nodes' % (sample_info.GetNodeCount()))
         x=0
         while x < sample_info.GetNodeCount():
            log.info('Timestamp[%s]=%s' % (x,sample_info.GetTimestampForNode(x)))
            x += 1
   except:
         raise

The OPC UA MAL library provides a predefined set of structures that hold a value of the supported types (see design document). User defined structures must be defined using ICD and their OPC UA specific structures generated by the icd-gen tooling.

In addition, MAL Publisher interface is also implemented as DA write operation, i.e. publish call actually calls DA write and is instantiated with the same URI semantic.

4.3.3.4. Logging

Additional info for opcua logging:

Open62541 library (client part) will log events via log4cplus malOpcuaOpen62541 logger. If a user does not provide a log4cplus configuration with the malOpcuaOpen62541 logger or a root logger, no log output will be generated from the open62541 library. Bellow is an example configuration pertaining to the open62541 library logging: malOpcuaOpen62541 logger configured for only ERROR and FATAL logging with additivity set to false (events won’t be propagated to the root logger). With this configuration all other mal loggers will log with TRACE level. User needs to provide the location of the log4cplus configuration for mal-opcua using mal property opc.log4cplus.filename (see table 3).

log4cplus.rootLogger=TRACE, stdout
log4cplus.logger.malOpcuaOpen62541=ERROR, stdout
log4cplus.additivity.malOpcuaOpen62541=false

log4cplus.appender.stdout=log4cplus::ConsoleAppender
log4cplus.appender.stdout.layout=log4cplus::PatternLayout
log4cplus.appender.stdout.layout.ConversionPattern=%5p [%t] (%F:%L) - %m%n

4.3.4. MUDPI

4.3.4.1. URIs

The underlying middleware is packet-oriented, packets are addressed via host-port. The Publisher needs to know the address of the subscriber. The Publisher binds to a random local port. The publisher sends unicast or multicast depending on the address encoded in the URI (multicast typically: 224.0.0.0 - 239.255.255.255). In case of unicast, the subscriber must (obviously) run on the host described the URI. In case of multicast, the publisher can choose the outgoing_interface by passing a property to getPublisher(…), see section Mal Specific Properties below.

  • Publisher: mudpi.ps://SUBSCRIBER-HOST:SUBSCRIBER-PORT/MUDPI-TOPIC-NAME

  • Subscriber: same as Publisher

  • Requester: mudpi.rr://SERVER-HOST:SERVER-PORT/MUDPI-SERVER-IF-NAME

Examples of URIs

  • mudpi.ps:///127.0.0.1:12781/TELIF - unicast on loopback interface

  • mudpi.ps://224.0.0.1:12000/TELIF - multicast in subnet

  • mudpi.ps://134.171.3.160:12000/TELIF - unicast to host 134.171.3.160

4.3.4.2. Mal Specific Properties

The following table enumerates mapping specific properties:

Name

Type

Description

Used by

Default

mudpi.ps. interfaceName

string

Broadcast interface name for mudpi sender Calling setInterface

Publisher

not set (empty string)

4.4. Java

4.4.1. DDS

4.4.1.1. Mal Specific Properties

For mapping specific properties refer to Table 1 C++ DDS MAL mapping specific properties.

4.4.2. ZPB

4.4.2.1. Mal Specific Properties

For mapping specific properties refer to Table 2 C++ ZPB MAL mapping specific properties.

4.4.3. OPC UA

4.4.3.1. Mal Specific Properties

The following table enumerates mapping specific properties:

Name

Type

Description

Used by

Default

opc.schedul edExecutorT hreads

integer

Number of thread pool threads to handle internal timer based tasks (e.g. QoS deadline timeouts).

MAL

3

opc.ps.poll ingPeriodMs

integer

Multi-node subscriber polling period in millisecond s.

Subscriber,

MrvSubscrib er

1000

opc.ps.reqP ubRateMs

integer

OPC UA stack subscriptio n requested publishing interval in millisecond s.

MAL

1000

opc.ps.queu eSize

integer

OPC UA stack subscriptio n queue size.

MAL

3

Table 4 Java OPC UA MAL mapping specific properties

4.5. Python

Python MAL API builds on MAL C++ API foundation. There is no native Python MAL mapping implementation, everything is provided via language mapping to C++.

Python applications use Python MAL API only and are not aware of the actual MAL implementation details.

Refer to Python Bindings Design [3] and MAL API User Manual [4] documents for detailed information.