11. Telemetry

Document ID:

Revision:

2.0

Last modification:

January 24, 2025

Status:

DRAFT

Repository:

https://gitlab.eso.org/cii/srv/cii-srv

File:

telemetry.rst

Project:

ELT CII

Owner:

Marcus Schilling
Document History

Revision

Date

Changed/ reviewed

Section(s)

Modification

0.1

09.03.2020

smarincek, jpribosek, dsoklic

All

Document creation.

1.0

06.05.2020

dsoklic/bte rpinc

All

Review updates. Released document.

1.1

17.6.2020

jpribosek

All

Updating manual after receiving support questions.

1.2

7.7.2020

jpribosek

All

Updating manual according to raised LAT issues.

1.3

30.7.2020

jpribosek

All

Updating manual according to raised LAT issues.

1.4

27.8.2020

jpribosek

All

Updating manual according to raised LAT issues.

1.5

3.9.2020

jpribosek

All

Updating manual according to raised LAT issues. Added additional section for setting up OLDB simulator.

1.6

24.9.2020

jpribosek

All

Updating manual according to raised LAT issues.

1.7

06.10.2020

jpribosek

All

Updating manual according to the review comments.

1.8

18.03.2024

mschilli

0

CII v4: public doc

2.0

24.01.2025

mschilli

All

CII v5.0: document restarted

Confidentiality

This document is classified as Public.

Scope

This document is a manual for the Telemetry system of the ELT Core Integration Infrastructure software.

Audience

This document is aimed at Users and Maintainers of the ELT Core Integration Infrastructure software.

Glossary of Terms

API

Application Programming Interface

CII

Core Integration Infrastructure

CLI

Command Line Interface

DP

Data Point

EA

Engineering Archive

OLDB

Online Database

References

  1. Logging Manual: https://www.eso.org/~eltmgr/CII/latest/manuals/html/docs/log.html

11.1. Overview

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

NOTE: DRAFT STATUS

THIS DOCUMENT IS UNDERGOING A COMPLETE REWRITE.

MANY SECTIONS ARE INCOMPLETE AT THIS TIME.

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

This document is a user manual for usage of the CII Telemetry system. It explains how to use the Telemetry API and the available tools.

All examples in this manual will be also presented in the telemetry-examples module.

11.2. Introduction

The purpose of the CII Telemetry system is archiving periodic, event-based and ad-hoc data that will be used for calculation of statistics, and the long-term trends and performance of the ELT telescope.

11.2.1. Dependencies

11.2.1.1. Engineering Archive

The Engineering Archive (EA) stores the data that the Telemetry Archiver receives and decides to archive. The Engineering Archive is, in fact, a set of services. The EA has been initially designed to use Persistence interface to access Elasticsearch and Hadoop, but can be modified to use other forms of storage.

11.2.1.2. Configuration Service

The CII Telemetry Service uses CII Configuration Service to store its configuration. The Configuration service must be accessible for the Telemetry Archiver to successfully initialize.

11.2.1.3. OLDB Service

The Online Database service is the main source of data that provides data point updates which are checked against the capturing rules defined by the Telemetry Archiver (TA) and later archived in the archiving system. For testing purposes, the OLDB service could be replaced with the OLDB simulator that generates the data point value updates based on the capturing rules.

11.2.2. Components

11.2.2.1. Telemetry APIs

Telemetry Client API provides an interface for easier interaction with the archiving system. The API could be used by the command line tools and other CII Applications. Telemetry Client API consists of:

  • Recording API - API used for storing data to the Engineering Archive.

  • Query API - API used for querying and retrieving data from the Engineering Archive.

  • Management API - API used for managing data in the Engineering Archive.

11.2.2.2. Telemetry CLI tools

CLI tools for interacting with the Telemetry Archiver System.

11.2.2.3. Telemetry Archivers

PENDING

11.3. Telemetry APIs (Common)

11.3.1. wscript

def configure(cnf):
  cnf.check_python_module('cii.telem')

def configure(cnf):
   cnf.check_wdep(
      wdep_name='cii-services-telemetry.cpp.telem',
      uselib_store='cii-services-telemetry.cpp.telem'
   )

11.3.2. imports/includes

::

import cii.telem

::

#include <ciiTelem.hpp>

11.4. Telemetry Recording API

The Recording API is available in C++ and Python.

11.4.1. Data Entitites

11.4.1.1. Time Series Metadata

Field

Type

Description

Example

tsid

string

Eternal ID

abc-xyz-123-127

btype

string

CII Basic Type

UINT16

units

string

SI Unit

m/s

json

string

full metadata

<json text>

tsname

string

Time Series Name

/tel/m2/coeff

comment

string

human readable

<text>

origin

string

producer

m2-sa-est

serial

string

device S/N

inventory-ms123

role

string

purpose

cabinet heat

11.4.1.2. Time Series Point

Field

Type

Description

Example

value

string|int|bool

measured value

127.0

mtime

string

measure time

12:00:00:00.000

11.4.2. Telemetry Recording API Example

# Acquire API
# ----------------
cii.telem.TelemManager.get_instance()
recorder = telman.create_telem_recorder("johnny", telman.backends.log)

# Specify metadata
# ----------------
tsmeta = tel.TsMetadata()
tsmeta.tsname  = "/elt/tel/m2/lsv/m2saest/mon/sv_force_correction"
tsmeta.units   = "coeff"
tsmeta.comment = "Tel M2 force correction"
tsmeta.origin  = "m2saest"
tsmeta.btype   = "INT32"
recorder.record_metadata (tsmeta)

# Record points
# -------------
recorder.record_point (tel.TsPoint(123))
recorder.record_point (tel.TsPoint(456))
recorder.record_point (tel.TsPoint(789))

std::shared_ptr<cii::telem::TelemManager> shared_tm = cii::telem::TelemManager::GetInstance();

// Acquire API
// -----------
std::shared_ptr<cii::telem::TelemRecorder> recorder;
recorder = telman.CreateTelemRecorder("johnny", telman.backends.log);

 // Specify metadata
 // ----------------
 cii::telem::TsMetadata tsmeta {};
 tsmeta.tsname  = "/elt/tel/m2/lsv/m2saest/mon/sv_force_correction";
 tsmeta.units   = "coeff";
 tsmeta.comment = "Tel M2 force correction";
 tsmeta.origin  = "m2saest";
 tsmeta.btype   = "INT32";
 recorder->RecordMetadata (tsmeta);

 // Record points
 // -------------
 recorder->RecordPoint (cii::telem::TsPoint(123));
 recorder->RecordPoint (cii::telem::TsPoint(456));
 recorder->RecordPoint (cii::telem::TsPoint(789));

11.5. Telemetry Query API

The Query API is only available in Python

11.5.1. Telemetry Query API Example

import cii.telem as tel
tm = tel.TelemManager.get_instance()

# Not available in this version

11.5.2. Querying data

11.5.3. Reading data

11.6. Telemetry Bulk API

11.6.1. Storing a list of data

11.6.2. Downloading list of data

11.7. Telemetry Management API

11.7.1. Moving data

11.7.2. Deleting data

11.8. Telemetry CLI tools

This section describes how to use Telemetry Archiver CLI tools. These command line tools simplify basic operations without needing to write custom applications for these tasks.

PENDING

PENDING

PENDING

PENDING

PENDING

11.9. Telemetry Archiver

11.9.1. Data Capture Configuration

Before the Telemetry Archiver can archive data packets, it must be configured. The data capture configuration can be done using the Configuration GUI.

11.9.2. Telemetry OLDB Subscription

CII Telemetry Archiver subscribes to OLDB data points via OLDB client and archives the data point values according to the data capture configurations that are stored in configuration service. Each data capture configuration is used to provide archiving options for one data point. It is possible to define multiple data capture rules for the same data point URI on the same CII Telemetry Archiver - e.g. data point could be archived on both value change and quality flag change on the same Telemetry Archiver.

Each Telemetry Archiver has defined its unique specified range in the form of a URI (i.e. serviceRange) that defines the location in configuration service from where the data capture configurations will be loaded from. For example, if the serviceRange field in Telemetry Archiver configuration is set to cii.config:///telemetry/service1, then all data capture configurations placed under /telemetry/service1/datacapture/ URI on configuration service will be used (an example of valid uri would then be cii.config:///telemetry/service1/datacapture/c722835e-cb22-4819-adb6-7cfe60c1ff39). Extra note: data capture configurations should be placed under <SERVICE RANGE>/datacapture/ uri.
For each data point that will be archived a data capture configuration must be created. Since the data capture configurations are going to be stored in the CII Configuration service a CIIBasicDataTypes can be used for configuration fields. The data capture settings are described in Appendix B.

The changes of the data capture configurations are only propagated when the end-user calls the refreshConfiguration method on the Service Management API (see section 5.4). This will reload the specified configuration (or all configurations) without interrupting the ongoing archiving tasks except as required by the modified configuration. The configuration service will not be continuously polled for data capture configuration changes. When the Telemetry Archiver is started (initialized), the data capture configurations are downloaded via CII config client from the location specified in serviceRange. The Telemetry OLDB Subscription subscribes to data point changes via OLDB client. Internally the OLDB client maintains a pool of data point subscriptions.

Telemetry Archivers do not know about one another and it’s the end user’s responsibility to distribute data points across them depending on the expected service workload. This allows easy horizontal scaling of the Telemetry Archivers depending on the amount of data that we would like to archive. If more throughput is needed, additional Telemetry Archivers can be deployed.
Each Telemetry Archiver contains its configuration stored in the CII Configuration Service
and connects to the CII Configuration Service using the CII Configuration Client. The client by default uses the ciiconfservicehost as the host location of CII Configuration Service (ciiconfservicehost is the hostname of the CII Configuration Service that should be resolved either by local DNS or custom /etc/hosts file). The default Telemetry Archiver configuration is stored under the cii.config://*/telemetry/config/defaultarchiverconfig.
Multiple service range entries can be defined per one Telemetry Archiver configuration.
To specify the custom configuration for another Telemetry Archiver, a custom Telemetry Archiver configuration YAML must be provided. Appendix C contains a description of all configuration fields and an example of a custom YAML configuration.
Figure -3 shows an example of 2 Telemetry Archivers. Telemetry Archiver 1 uses the default configuration and the Telemetry Archiver 2 uses custom configuration. They both communicate with the configuration service to retrieve the service range and data capture configurations. Each Telemetry Archiver configuration specifies service ranges under which the data capture configurations are found. The data capture configuration, which is in subpath of specified service range, specifies the details about the behavior of the data capture (OLDB data point URI, delta value, …).

When the Telemetry Archiver starts up, the data point values (CiiOldbDpValue) of the subscribed data points (as defined in data capture configurations) are downloaded from OLDB and stored in memory. Every time the data point value of the subscribed data point changes, they are checked against the data capture configurations to determine if the data point is suitable for archiving.

The subscribed data point will be archived depending on the following cases:

  • Value change: if the data point value changed and if the difference between the old and new data point value has changed for more than the specified delta value and if duration since the last archive is bigger than the minimum interval duration (set in data capture configuration) the data point will be archived.

  • Metadata change: if the data capture configuration is capturing based on the metadata change, the data point value will be archived whenever the metadata has changed

  • Quality change: if the data capture configuration is capturing based on the quality change and the data point quality flag has changed, the data point value will be archived.

  • Maximum interval: the purpose of the maximum interval is to archive the data point value in the absence of a data point value change event. If the maximum interval is set, a periodic task will be submitted to Telemetry Archiver’s scheduled executor that will try to archive the last data point value according to the specified period. Timestamp since the last archive event for the given data capture configuration will be stored in the memory of the archiving worker.

  • Minimum interval: the purpose of the minimum interval is to not archive too many data point value changes in a short timeframe. If the minimum interval is set and the duration between now and time, since the last value was archived, is bigger than the minimum interval duration, the data point value will be archived

It is possible to combine the above rules to make more fine-grained archiving behavior. For example, one can create two data capture configurations:

  1. Data capture configuration with value archiving behavior and maximum time interval set to 10 seconds.

  2. Data capture configuration with quality archiving behavior.

11.9.3. Telemetry Archiver Deployment

PENDING