7. Online Database¶
Revision: |
1.3 |
|---|---|
Status: |
Released |
Repository: |
|
Project: |
ELT CII |
Folder: |
trunk/deliverables/phase8 |
Document ID: |
CSL-DOC-19-176283 |
File: |
MAN-ELT_OLDB.docx |
Owner: |
Jan Pribošek |
Last modification: |
July 17, 2020 |
Created: |
July 5, 2019 |
Prepared by |
Reviewed by |
Approved by |
|---|---|---|
Jan Pribošek (CSL SWE) Marko Novak (CSL) Jernej Strniša (CSL) |
Miha Vitorovič (CSL) Borut Terpinc (CSL SWE) |
Gregor Čuk (CSL) |
Revision |
Date |
Changed/rev iewed |
Section(s) |
Modificatio n |
|---|---|---|---|---|
0.1 |
05.07.2019 |
jpribosek, manovak |
5 |
Document creation. GUI Section. |
0.2 |
19.07.2019 |
jpribosek, jstrnisa |
1-4 Appendix A Appendix B |
Added client API sections. |
0.3 |
05.08.2019 |
mvitorovic bterpinc |
All |
Document review and fixes. |
1.0 |
24.09.2019 |
jpribosek |
All |
Final updates, release. |
1.1 |
20.1.2020 |
jstrnisa |
All |
Major update of all sections after ESO topical review |
1.2 |
13.7.2020 |
bterpinc |
3.1 4.1 5.6 |
Statement about pub/sub queue adjustment added. Note about the synchronous methods added. Additional External Redis serves explanation added. |
1.3 |
16.7.2020 |
bteprinc |
7 |
CLI tools update |
Confidentiality
This document is classified as a confidential document. As such, it or parts thereof must not be made accessible to anyone not listed in the Audience section, neither in electronic nor in any other form.
Scope
This document is manual for the Online Database system used by applications in the ELT Core Integration Infrastructure Software.
Audience
This document is aimed at those Cosylab and ESO employees involved with the ELT Core Integration Infrastructure Software project, as well as other Users and Maintainers of the ELT Core Integration Infrastructure Software.
Glossary of Terms
API |
Application Programming Interface |
|---|---|
CII |
Core Integration Infrastructure |
CLI |
Command Line Interface |
DP |
Data Point |
GUI |
Graphical User Interface |
ES |
ElasticSearch |
JSON |
Javascript object notation |
OLDB |
Online Database |
SVN |
Subversion |
YAML |
YAML Ain’t Markup Language |
References
ESO, Core Integration Infrastructure Requirements Specification, ESO-192922 Version 6
Cosylab, ELT CII Online Database Transfer Document, CSL-DOC-19-176283, Version 1.4
Cosylab, CII Configuration User’s Manual, CSL-DOC-19-176283, Version 1.4
Redis main page: https://redis.io/
HDFS documentation: https://hadoop.apache.org/docs/r1.2.1/index.html
Octave documentation version 5.1.0: https://octave.org/doc/v5.1.0/
Redis INFO command https://redis.io/commands/info
7.1. Overview¶
7.2. Introduction¶
The Core Integration Infrastructure (CII) Online Database (OLDB) provides distributed data publishing and access to actual or live data for user interface and control applications that do not have low-latency or real-time performance requirements. The term “online” refers to the fact that the database provides current and live values for the data points of the control system.
Figure 2‑1: OLDB API interactions
A data point represents a single value in the OLDB and consists of a value, timestamp, quality, and metadata. Data points are identified by a unique URI (section 6.2.1).
Data point value is the actual data the data point is holding. It can be any of the predefined types (Described in Appendix A). The type of the data point can be a primitive type (e.g. integer, double, string) or a more complex type (vector or 2D matrix).
The timestamp is the timestamp of the last data point value modification.
The quality represents the state of the data point. It can be OK, SUSPECT and BAD. Only OK data points can be read and written to (except when correcting the quality to OK). The meaning of each quality state is as follows:
OK: the value of the Property is valid
BAD: the value is not OK and should not be used
SUSPECT: there is a value, but it is not clear if it is reliable. For example, the readout of a sensor is inconsistent with other readings and the sensor might be faulty.
Metadata is the meta information about the data point and consists of information like DP data type, formula, quality expression and other fields specific to DP data type (for a detailed description of metadata content see section 6.2.3). Metadata is maintained by CII Configuration service [3], but can also be manipulated through the OLDB Client.
Data points can define a calculation formula (included in metadata) that determines the value of the data point. The formula can include URIs of other data points and so the value of the data point can depend on values of other data points. A data point with formula is called calculated data point and cannot be written to by users, since its value is determined by the formula.
OLDB Client API provides a means to manipulate data points (create, read, write and delete) through the OLDB client. Users can access and manipulate OLDB through the Client API library (srv-oldb) or OLDB GUI and CLI applications. The srv-oldb library is written in Java, C++ and Python. The inline code examples and class definitions in this document are written in Java. For example in C++ and Python see Chapter 4 where complete examples are listed in all three languages.
OLDB uses Redis key-value store [4] and HDFS file storage [5] for storing the data. Redis is used for small data point values while large DP that exceed the limit defined in the OLDB configuration (section 3.3.1) are saved in HDFS. The Redis storage is further divided into two parts:
default storage: This is the default location for data points. All small sized data points are stored here.
in-memory-only storage: This is a special storage where only predetermined data points (defined in data point configuration) are stored. It is intended for larger values that need quicker access than HDFS can provide.
See [2] for a detailed description of data point configuration and how to manipulate it.
Both default and in-memory-only storages are horizontally scalable and can consist of any number of Redis servers either single instance or clusters.
The lifetime of the data points is not bounded by the lifetime of the OLDB client instance because their values and configurations are saved in the OLDB storage and Configuration service respectably. If the Redis storage is configured to be persistent, the data points also persist after Redis server shutdown.
7.3. Prerequisites¶
This section describes the prerequisites for using the OLDB library and applications. Note that the preparation of the environment (i.e. installation of required modules, configuration, running of services) is not in the scope of this document. Consult your administrator and the OLDB TRD document [2] for information on how to establish the environment for usage of the OLDB library and applications.
7.3.1. WAF Modules¶
For using the OLDB library and CLI application the following modules must be built and installed with WAF in the specified order:
elt-mal
client-api
srv-config
oldb-client
srv-oldb
For using the OLDB GUI application in addition to the above two more modules must be built and installed:
elt-qt-widgets
oldb-gui
Before building anything, Proper .pc files must be put in the “/eelt/System/pkgconfig” folder. These files are used by the WAF wscript to properly resolve the dependencies. Please refer to OLDB TRD for details [2].
7.3.2. Services¶
7.3.2.1. Redis¶
OLDB uses Redis [4] key-value store as its main storage of data point values. At a minimum the OLDB needs at least one default storage server, at least one in-memory-only (External Redis Server - 5.6) storage and exactly one pub/sub server. One Redis instance can serve as all three servers so minimally one Redis instance must be running for OLDB to work.
The number of default and in-memory-only storage servers is unbounded. Only one Redis server can be designated to be a pub/sub server and can be either a single instance or cluster.
See the OLDB TRD document [2] on how to setup the Redis server
7.3.2.2. HDFS¶
HDFS storage is used for large-sized data points (the ones that are not defined to be stored in in-memory-only storage). OLDB needs exactly one HDFS storage server running to work properly. The installation and configuration of the HDFS storage are out of the scope of this document. Guides for doing this can be found in [2].
7.3.2.3. Calculation Service¶
For the OLDB framework to work as intended, the calculation service (OLDB service) must be running. The calculation service is responsible for calculating the value and quality of calculated data points. The calculation service consists of any number of calculation node processes and a scheduler process. Each calculation node is responsible for the calculation of a subset of all existing data points and the scheduler process is responsible for distributing the calculated data points among the calculation nodes.
All the calculation nodes must be running before the scheduler process. The scheduler must be configured with addresses of existing calculation nodes so it can connect to them.
To add a calculation node, the scheduler configuration must be updated with the new node’s address and the scheduler process must be restarted. The old nodes will retain their calculated data points and continue to recalculate their values. Note that during scheduler downtime newly created calculated data points will not be distributed and calculated. They will be distributed however when the scheduler process restarts.
If a calculation node shuts down the subset of calculated data points assigned to it will not be reassigned to another calculation node and will not be calculated. For this to happen the scheduler process must be restarted.
The proper way to shut down the calculation service is to first stop the scheduler process and then the calculation nodes.
7.3.2.4. Configuration Service¶
OLDB uses CII Configuration Service [3] to store its configuration. The Configuration service must be accessible for the OLDB Client and Calculation Service to successfully initialize.
7.3.3. Configuration¶
Both the OLDB Client and the calculation service must be configured to work properly. The configuration storage is handled by the CII Configuration Service [3]. Both OLDB and Calculation Service configurations are stored in a versioned way.
7.3.3.1. OLDB Client Configuration¶
The OLDB client configuration is stored in the CiiOldbConfigClass object. The configuration is fetched from the CII Configuration Service by the OLDB client at start-up.
The configuration class contains the following fields:
redisServers: a MdStringArray class containing a list of strings that contains parameters for default storage servers. Each string corresponds to one server and is formatted as <sequential-number>:<hostname>:<port>:<server-type> where server-type is either SINGLE or CLUSTER and describes the type of the server (either single instance or cluster). The <sequential-number> should be the sequential number of servers in the list starting at 0.
redisServersExternal: a MdStringArray class containing a list of strings that contains parameters for in-memory-only storage servers. Each string corresponds to one server and is formatted as <alias>:<hostname>:<port>:<server-type> where server-type is either SINGLE or CLUSTER and describes the type of the server (either single instance or cluster).
pubSubServerInfo: a MdString class containing a string that contains parameters for the OLDB pub/sub server. The string is formatted in the same manner as strings in the redisServers field, except that the <sequential-number> has no significance since there is always one server.
pubSubHandlerProccesingInterval: a MdInt32 containing an integer that represents the interval in milliseconds at which a pub/sub message handler should handle messages. To prevent the incorrect ordering of command execution, the incoming pub/sub messages are ordered by timestamp in unbounded queues and processed every pubSubHandlerProccesingInterval milliseconds. The performance of the pub-sub system will degrade with larger pubSubHandlerProccesingInterval value, be but the correct order of message execution will be more guaranteed. The recommended value for this field is 10. The size of the pub/sub event queue is unbounded, and thou it is not adjustable. Note: This reason for this setting is the proper timestamp ordering of the incoming messages.
valueSizeLimitRedisExt: a MdInt32 containing an integer that represents the value size limit in bytes for in-memory-only servers. A data point value written to these servers is sliced and distributed among several key-value pairs so that each chunk does not exceed the valueSizeLimitRedisExt. If this value is too small write operations on large data points will be slower. It should not be greater than 512 Mb (512 * 1024 * 1024 bytes) since this is the limit of one key-value pair on Redis. The recommended value for this field is 128 Mb (128 * 1024 * 1024 bytes). The value must be set in bytes (e.g., 134217728).
valueSizeLimitHDFS: a MdInt32 containing an integer that represents the value size limit in bytes for default servers. If a data point value written to the default server exceeds this limit, it is redirected to the HDFS storage. In this case, the data point information holds only the reference to its location. This value should not be greater than 512 Mb (512 * 1024 * 1024 bytes) since this is the limit of one key-value pair on Redis. The recommended value for this field is 100Kb (100 * 1024 bytes) The value must be set in bytes (e.g., 102400).
extRedisKeyExpireTime: a MdInt32 containing an integer that represents the expire time in milliseconds of the keys on the in-memory-only servers. Whenever a data point is written to in-memory-only storage, the old value keys are not rewritten (some client could at that time be reading them) but only set to expire (are deleted) after the extRedisKeyExpireTime. This value should always be bigger than a time of a write operation on any data point, else a read operation could return an error or malformed value. The recommended value is 5000.
largeStorageLocation: a MdString with the address of HDFS storage.
largeStorageEndpoint: a MdString with the endpoint of HDFS storage.
Note that the OLDB client configuration cannot be changed dynamically. If the configuration has been changed, the OLDB client must be restarted for the changes to take effect.
The OLDB configuration is stored in CII Config Service under the URI:
cii.config://remote/oldb/configurations/oldbclientconfig
For testing purposes, a default remote configuration can be deployed on the Configuration Service using the oldb-initEs script. This script will initialize testing OLDB configuration data. Every time the script is run, it will create a new version. The OLDB client always reads the last version of the configuration at start up.
oldb-initEs
7.3.3.2. Calculation Service Configuration¶
The scheduler process must be configured before it can be run. The configuration is done through the CII Configuration service [3] using the CiiOldbSchedulerConfig. The only field this configuration needs is a list of string URIs of calculation nodes that the scheduler should handle.
The calculation service configuration is stored in CII Config Service under the URI:
cii.config://remote/oldb/calculationnodes
7.4. OLDB Library Usage¶
This section explains through examples of how to use the OLDB API library in an application. The OLDB API library provides the user the functionality to create, read, write, query, subscribe to and delete data points in the CII OLDB. The API is distributed among two classes. The CiiOldb and CiiOldbDataPoint.
7.4.1. Includes/Imports¶
For basic usage of the OLDB library, the user needs the CiiOldb client class, the CiiOldbDataPoint and the CiiOldbDpValue class. The first two classes contain the API for manipulating data points and the last one is a container for the data point value triplet (i.e. value, quality, timestamp). Next the language specific URI class is needed. If data points are created with provided metadata, respective metadata classes need to be imported. See section 6.2.3 for the list of metadata classes.
7.4.1.1. Java¶
import java.net.URI;
import elt.oldb.client.CiiOldb;
import elt.oldb.client.CiiOldbDataPoint;
import elt.oldb.client.CiiOldbDpValue;
//Metadata classes
import elt.config.classes.meta.MdOldbNumber;
7.4.1.2. C++¶
#include <ciiOldbDpValue.hpp>
#include <ciiOldbFactory.hpp>
//Metadata classes
#include <meta/MdOldbNumber.hpp>
7.4.1.3. Python¶
import elt.oldb
import elt.config.Uri
For subscribing to data point changes, the CiiOldbSubscription interface or a class implementing this interface must be imported.
7.4.1.4. Java¶
import elt.oldb.client.CiiOldbDpSubscription;
7.4.1.5. C++¶
#include <ciiOldbSubscription.hpp>
7.4.1.6. Python¶
Included in elt.oldb module.
Since OLDB API also throws exceptions, these need to be imported to. For Java and C++ these exceptions can be found in elt.oldb.exceptions package and <ciiOldbExceptions.hpp> header. For Python these exceptions are included in elt.oldb module.
Linked library includes can be seen in code examples of 4.1.2, 4.1.3 and 4.1.4
7.4.1.7. Basic Example¶
The examples in this section can be found in the oldb-examples project (BasicExample.java, sample-app.cpp, cii-oldb-examples-sample-app-py.py).
In basic example (Listing 4‑1, Listing 4‑2, Listing 4‑3) four constant (without formulas) data points of type DOUBLE, STRING, MATRIX2D_DOUBLE and VECTOR_INT32 are created. Multiple different creation methods are used to create these data points (see appendix C.1 on creation methods). These data points are then read to confirm their initial values. New values are then written to the data points.
Then the data points are queried using a glob expression and again with additional filter arguments (query for the specific type and value range).
Then the children of specific directory parent in the URI hierarchy are queried.
At last, the data points are deleted and OLDB client is closed.
Note: All OLDB client methods are synchronous towards the Redis server.
Listing 4‑1: Java Basic Example
package sampleapp;
import java.net.URI;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import elt.common.AttributeType;
import elt.config.classes.meta.MdOldbNumber;
import elt.config.client.CiiConfigClient;
import elt.config.exceptions.CiiConfigDeleteException;
import elt.config.exceptions.CiiConfigInitializationError;
import elt.config.exceptions.CiiConfigSaveException;
import elt.config.exceptions.CiiConfigWriteDisabledException;
import elt.error.CiiInvalidTypeException;
import elt.error.CiiInvalidURIException;
import elt.oldb.client.CiiOldb;
import elt.oldb.client.CiiOldbDataPoint;
import elt.oldb.client.CiiOldbDpSubscription;
import elt.oldb.client.CiiOldbDpValue;
import elt.oldb.client.impl.CiiOldbImp;
import elt.oldb.client.util.CiiOldbUriUtils;
import elt.oldb.exceptions.CiiOldbConfigError;
import elt.oldb.exceptions.CiiOldbCyclicDependencyException;
import elt.oldb.exceptions.CiiOldbDeletedException;
import elt.oldb.exceptions.CiiOldbDpConstraintsException;
import elt.oldb.exceptions.CiiOldbDpExistsException;
import elt.oldb.exceptions.CiiOldbDpQualityException;
import elt.oldb.exceptions.CiiOldbDpUndefinedException;
import elt.oldb.exceptions.CiiOldbExpressionException;
import elt.oldb.exceptions.CiiOldbIllegalOperationException;
import elt.oldb.exceptions.CiiOldbInitializationError;
import elt.oldb.exceptions.CiiOldbStorageError;
import elt.oldb.exceptions.CiiOldbURIException;
import elt.oldb.exceptions.CiiOldbUndefinedMetadataException;
import elt.oldb.exceptions.CiiOldbWriteDisabledException;
/**
* @copyright (c) Copyright ESO 2019 All Rights Reserved ESO (eso.org) is an
* Intergovernmental Organisation, and therefore special legal
* conditions apply.
* @ingroup srv-oldb-java-sample-app
*/
public class SampleApplication {
public static void main(String[] args) {
CiiOldb oldbClient = null;
CiiConfigClient configClient = null;
try {
// Initialize OLDB and Config clients
oldbClient = CiiOldb.getInstance();
configClient = CiiConfigClient.getInstance();
// Define URIs
URI doubleDpURI =
URI.create("cii.oldb:/sampleroot/child/device/doubledp");
URI matrixDpURI = URI.create("cii.oldb:/sampleroot/child/matrixdp");
URI stringDpURI = URI.create("cii.oldb:/sampleroot/child/stringdp");
URI vectorDpURI = URI.create("cii.oldb:/sampleroot/child/vectordp");
oldbClient.deleteDataPoint(doubleDpURI);
oldbClient.deleteDataPoint(matrixDpURI);
oldbClient.deleteDataPoint(stringDpURI);
oldbClient.deleteDataPoint(vectorDpURI);
// CREATE DATA POINTS
// Create DOUBLE data point with provided metadata instance name.
// First we create the metadata and save it to Config.
MdOldbNumber<Double> doubleDpMeta =
new MdOldbNumber<Double>("customDoubleDpMeta",
"metadata of a constant DP", 0.0, null, null, 0.0, 10.0, null);
URI doubleDpMetaURI = URI
.create(CiiOldbUriUtils.CONFIG_METADATA_URI +
doubleDpMeta.getMetadataInstanceName());
configClient.saveMetadata(doubleDpMetaURI, doubleDpMeta);
CiiOldbDataPoint<Double> doubleDp = (CiiOldbDataPoint<Double>)
oldbClient.createDataPoint(doubleDpURI,
doubleDpMeta.getMetadataInstanceName(),
doubleDpMeta.getClass().getSimpleName());
// Create MATRIX2D_DOUBLE data point with default matrix metadata
// OldbDoubleMatrixStd and provided initial value.
// The default metadata must exist on Config Service.
// Run oldb-initEs script to generate
// default metadata instances.
CiiOldbDataPoint<List<Double>> matrixDp =
oldbClient.createDataPointByValue(matrixDpURI,
new ArrayList<>(50), Double.class, true);
// Create STRING data point with default string metadata
// OldbStringStd and provided initial value.
CiiOldbDataPoint<String> stringDp =
oldbClient.createDataPointByValue(stringDpURI, "ABCDEF");
// Create VECTOR_INT32 data point with provided metadata instance name.
// Metadata with this instance name must exist in Config Service.
CiiOldbDataPoint<List<Integer>> vectorDp =
(CiiOldbDataPoint<List<Integer>>) oldbClient
.createDataPoint(vectorDpURI, "OldbInt32ArrayStd", "MdOldbArray");
// Read data point values.
CiiOldbDpValue<Double> constDpValue = doubleDp.readValue();
CiiOldbDpValue<List<Double>> matrixDpValue = matrixDp.readValue();
CiiOldbDpValue<String> stringDpValue = stringDp.readValue();
CiiOldbDpValue<List<Integer>> vectorDpValue = vectorDp.readValue();
// Write to data points
List<Double> newMatrixValue = new ArrayList<>();
for (int i = 0; i < 2500; i++) {
newMatrixValue.add(new Double(i));
}
doubleDp.writeValue(2.0);
matrixDp.writeValue(newMatrixValue);
stringDp.writeValue("12345");
vectorDp.writeValue(new ArrayList<>(Arrays.asList(2, 4, 6)));
// Read the data point values.
constDpValue = doubleDp.readValue();
matrixDpValue = matrixDp.readValue();
stringDpValue = stringDp.readValue();
vectorDpValue = vectorDp.readValue();
// get Multiple data points satisfying an URI glob expression.
List<CiiOldbDataPoint<?>> searchResult = oldbClient
.getDataPoints(new ArrayList<URI>(Arrays.asList(URI.create("cii.oldb:/sampleroot/**"))));
// get Multiple DOUBLE data points with value between 1.0 and 3.0
List<CiiOldbDataPoint<Double>> filteredSearchResult =
oldbClient.getDataPoints(
new ArrayList<URI>(
Arrays.asList(URI.create("cii.oldb:/sampleroot/**"))),
AttributeType.DOUBLE, 1.0, 3.0);
// get the children of directory URI cii.oldb:/sampleroot. Returns a map
// specifying whether
// a child is a data point or just a directory.
Map<URI, Boolean> children =
oldbClient.getChildren(URI.create("cii.oldb:/sampleroot"));
// Delete data points
oldbClient.deleteDataPoint(doubleDpURI);
oldbClient.deleteDataPoint(matrixDpURI);
oldbClient.deleteDataPoint(stringDpURI);
oldbClient.deleteDataPoint(vectorDpURI);
configClient.deleteMetadata(
URI.create(CiiOldbUriUtils.CONFIG_METADATA_URI +
doubleDpMeta.getMetadataInstanceName()),
doubleDpMeta.getMetadataInstanceName(), -1);
// Close the clients
configClient.close();
oldbClient.close();
} catch (CiiConfigInitializationError | CiiOldbDpConstraintsException | CiiInvalidTypeException
| CiiInvalidURIException | CiiOldbDpExistsException | CiiOldbCyclicDependencyException
| CiiOldbExpressionException | CiiOldbDpQualityException | CiiOldbUndefinedMetadataException
| CiiOldbStorageError | CiiOldbConfigError | CiiOldbDpUndefinedException | CiiOldbDeletedException
| CiiOldbWriteDisabledException | CiiConfigWriteDisabledException | CiiOldbInitializationError
| CiiOldbURIException | CiiConfigDeleteException | CiiConfigSaveException e) {
e.printStackTrace();
if (configClient != null) {
configClient.close();
}
if (oldbClient != null) {
oldbClient.close();
}
}
}
}
Listing 4‑2: C++ Basic Example
/**
* @copyright (c) Copyright ESO 2019 All Rights Reserved
* ESO (eso.org) is an Intergovernmental Organisation, and therefore special legal conditions apply.
* @ingroup oldb-examples
*/
#include <mal/utility/Uri.hpp>
#include <ciiOldbFactory.hpp>
#include <ciiOldbDpValue.hpp>
#include <ciiOldbExceptions.hpp>
int main(int ac, char *av[]) {
try {
// Initialize OLDB
auto oldb_client = ::elt::oldb::CiiOldbFactory::GetInstance();
::elt::oldb::CiiOldbGlobal::SetWriteEnabled(true);
std::string RND_PREFIX = ::elt::oldb::CiiOldbUtil::NewUUID();
// Define URIs
::elt::mal::Uri double_dp_uri{"cii.oldb:/" + RND_PREFIX + "/sampleroot/child/device/doubledp"};
::elt::mal::Uri matrix_dp_uri{"cii.oldb:/" + RND_PREFIX + "/sampleroot/child/matrixdp"};
::elt::mal::Uri string_dp_uri{"cii.oldb:/" + RND_PREFIX + "/sampleroot/child/stringdp"};
::elt::mal::Uri vector_dp_uri{"cii.oldb:/" + RND_PREFIX + "/sampleroot/child/vectordp"};
// CREATE DATA POINTS
// Create DOUBLE data point with provided metadata instance.
// Metadata must first be created and saved to Config Service.
const std::string double_dp_meta_instance_name = RND_PREFIX + "customDoubleDpMeta";
auto double_dp_meta = CiiDataPointMetadataFactory::getNewMetadataInstance<
::elt::config::classes::meta::MdOldb<double>>(double_dp_meta_instance_name);
double_dp_meta->setComment("metadata of a constant DP");
double_dp_meta->set_default_value(0.0);
double_dp_meta->set_min_limit(0.0);
double_dp_meta->set_max_limit(10.0);
// meta data needs to be created before calling CreateDataPoint methods
::elt::oldb::CiiOldbUtil::CreateDataPointInfo(double_dp_uri, *double_dp_meta);
auto double_dp = oldb_client->CreateDataPoint<double>(
double_dp_uri, double_dp_meta_instance_name);
// Create MATRIX2D_DOUBLE data point with default matrix metadata OldbDoubleMatrixStd
// and provided initial value. The default metadata must exist on Config Service.
// Run oldb-initEs script to generate default metadata instances.
std::vector<double> matrix_double =
{1.0, 20.0, 30.0, 40.0, 50.0, 4.4, 45.3, 34.4, 445.3, 301.3,
2.0, 33.3, 34.3, 33.3, 33.3, 5.5, 32.2, 33.4, 222.3, 203.1};
auto mdi_matrix_double = ::elt::config::CiiConfigClient::retrieveMetadata(
::elt::oldb::CiiOldbGlobal::GetTargetConfigStorage(),
::elt::oldb::CiiOldbGlobal::MDI_MATRIX_DOUBLE,
::elt::oldb::CiiOldbGlobal::CONFIG_VERSION);
::elt::oldb::CiiOldbUtil::CreateDataPointInfo(matrix_dp_uri, *mdi_matrix_double);
bool is_matrix = true;
auto matrix_dp = oldb_client->CreateDataPointByValue(matrix_dp_uri, matrix_double, is_matrix);
// Create STRING data point with default string metadata OldbStringStd and
// provided initial value.
auto mdi_string = ::elt::config::CiiConfigClient::retrieveMetadata(
::elt::oldb::CiiOldbGlobal::GetTargetConfigStorage(),
::elt::oldb::CiiOldbGlobal::MDI_STRING,
::elt::oldb::CiiOldbGlobal::CONFIG_VERSION);
::elt::oldb::CiiOldbUtil::CreateDataPointInfo(string_dp_uri, *mdi_string);
auto string_dp = oldb_client->CreateDataPointByValue(
string_dp_uri, std::string("ABCDEF"));
// Create VECTOR_INT32 data point with provided metadata instance name.
// Metadata with this instance name must exist in Config Service.
auto mdi_vector_int32 = ::elt::config::CiiConfigClient::retrieveMetadata(
::elt::oldb::CiiOldbGlobal::GetTargetConfigStorage(),
::elt::oldb::CiiOldbGlobal::MDI_VECTOR_INT32,
::elt::oldb::CiiOldbGlobal::CONFIG_VERSION);
::elt::oldb::CiiOldbUtil::CreateDataPointInfo(vector_dp_uri, *mdi_vector_int32);
auto vector_dp_base = oldb_client->CreateDataPoint(vector_dp_uri,
::elt::oldb::CiiOldbGlobal::MDI_VECTOR_INT32);
auto vector_dp =
std::dynamic_pointer_cast<::elt::oldb::CiiOldbDataPoint<std::vector<std::int32_t>>>(
vector_dp_base);
if (!vector_dp) {
throw ::elt::oldb::CiiOldbException("Casting to data point of vector int32 type failed");
}
// Read data point values.
auto const_dp_value = double_dp->ReadValue();
auto matrix_dp_value = matrix_dp->ReadValue();
auto string_dp_value = string_dp->ReadValue();
// vectors do not have default value when created, needs a write before read
vector_dp->WriteValue(std::vector<std::int32_t>{2, 4, 6});
auto vector_dp_value = vector_dp->ReadValue();
// Write to data points
std::vector<double> new_matrix_value(2500);
for (std::int32_t i = 0; i < 2500; ++i) {
new_matrix_value.push_back(static_cast<double>(i));
}
double_dp->WriteValue(2.0);
matrix_dp->WriteValue(new_matrix_value);
string_dp->WriteValue("12345");
vector_dp->WriteValue(std::vector<std::int32_t>{2, 4, 6});
// Read the data point values.
const_dp_value = double_dp->ReadValue();
matrix_dp_value = matrix_dp->ReadValue();
string_dp_value = string_dp->ReadValue();
vector_dp_value = vector_dp->ReadValue();
// Get multiple data points satisfying an URI glob expression
std::vector<::elt::mal::Uri> uris = {::elt::mal::Uri("cii.oldb:/sampleroot/**")};
auto search_result = oldb_client->GetDataPoints(uris);
// Get multiple DOUBLE data points with value between 1.0 and 3.0
auto filtered_search_result = oldb_client->GetDataPoints<double>(
uris, ::elt::common::CiiBasicDataType::DOUBLE, 1.0, 3.0);
// Get the children of directory URI cii.oldb:/sampleroot. Returns a map specifying whether
// a child is a data point or just a directory
auto children = oldb_client->GetChildren(::elt::mal::Uri("cii.oldb:/sampleroot"));
// Delete data points
oldb_client->DeleteDataPoint(double_dp_uri);
oldb_client->DeleteDataPoint(matrix_dp_uri);
oldb_client->DeleteDataPoint(string_dp_uri);
oldb_client->DeleteDataPoint(vector_dp_uri);
return 0;
} catch (const ::elt::oldb::CiiOldbException& ex) {
std::cerr << "CiiOldbException occured while executing sample code. What: "
<< ex.what() << '\n';
}
return -1;
}
Listing 4‑3: Python Basic Example
#!/usr/bin/env python
"""
@copyright (c) Copyright ESO 2019 All Rights Reserved
ESO (eso.org) is an Intergovernmental organisation,
and therefore special legal conditions apply.
@ingroup client-apis-python
@author Cosylab
"""
#This script shows basic usage of OLDB API.
import os, sys, traceback, time, threading, uuid
import elt.config
import elt.oldb
# Define URIs
RND_PREFIX = str(uuid.uuid4())
double_dp_uri = elt.config.Uri("cii.oldb:/%s/sampleroot/child/device/doubledp" % RND_PREFIX)
string_dp_uri = elt.config.Uri("cii.oldb:/%s/sampleroot/child/device/stringdp" % RND_PREFIX)
vector_dp_uri = elt.config.Uri("cii.oldb:/%s/sampleroot/child/device/vectordp" % RND_PREFIX)
matrix_dp_uri = elt.config.Uri("cii.oldb:/%s/sampleroot/child/device/matrixdp" % RND_PREFIX)
# Initialize OLDB client
oldb_client = elt.oldb.CiiOldbFactory.get_instance()
def _main():
# enable writing
elt.oldb.CiiOldbGlobal.set_write_enabled(True)
# create DOUBLE data point with provided metadata instance
# Metadata must first be created and saved to Config Service.
double_dp_meta = \
elt.oldb.typesupport.DOUBLE.get_new_number_metadata_instance("%scustomDoubleDpMeta" % RND_PREFIX)
double_dp_meta.set_comment("metadata of a constant DP")
double_dp_meta.set_min_limit(0.0)
double_dp_meta.set_max_limit(10.0)
double_dp_meta.set_default_value(0.0)
# Create & Save Metadata
elt.oldb.CiiOldbUtil.create_data_point_info(double_dp_uri, double_dp_meta);
# Create DOUBLE data point
double_dp = oldb_client.create_data_point(double_dp_uri, double_dp_meta.get_instance_name())
# Create MATRIX2D_DOUBLE data point with default matrix metadata OldbDoubleMatrixStd
# and provided initial value. The default metadata must exist on Config Service.
matrix_double = elt.oldb.VectorDOUBLE([
1.0, 20.0, 30.0, 40.0, 50.0, 4.4, 45.3, 34.4, 445.3, 301.3,
2.0, 33.3, 34.3, 33.3, 33.3, 5.5, 32.2, 33.4, 222.3, 203.1 ])
mdi_matrix_double = elt.config.CiiConfigClient.retrieve_metadata(
elt.oldb.CiiOldbGlobal.get_target_config_storage(),
elt.oldb.CiiOldbGlobal.MDI_MATRIX_DOUBLE,
elt.oldb.CiiOldbGlobal.CONFIG_VERSION)
elt.oldb.CiiOldbUtil.create_data_point_info(matrix_dp_uri, mdi_matrix_double)
is_matrix = True
matrix_dp = oldb_client.create_data_point_by_value(matrix_dp_uri, matrix_double, is_matrix)
# Create STRING data point with default string metadata OldbStringStd anf provided initial value
mdi_string = elt.config.CiiConfigClient.retrieve_metadata(
elt.oldb.CiiOldbGlobal.get_target_config_storage(),
elt.oldb.CiiOldbGlobal.MDI_STRING,
elt.oldb.CiiOldbGlobal.CONFIG_VERSION)
elt.oldb.CiiOldbUtil.create_data_point_info(string_dp_uri, mdi_string)
string_dp = oldb_client.create_data_point_by_value(string_dp_uri, "ABCDEF")
# Create VECTOR_INT32 data point with default vector metadata OldbDoubleVectorStd
mdi_vector32 = elt.config.CiiConfigClient.retrieve_metadata(
elt.oldb.CiiOldbGlobal.get_target_config_storage(),
elt.oldb.CiiOldbGlobal.MDI_VECTOR_INT32,
elt.oldb.CiiOldbGlobal.CONFIG_VERSION)
elt.oldb.CiiOldbUtil.create_data_point_info(vector_dp_uri, mdi_vector32)
vector_dp = oldb_client.create_data_point(vector_dp_uri, elt.oldb.CiiOldbGlobal.MDI_VECTOR_INT32)
# Read values
const_dp_value = double_dp.read_value()
matrix_dp_value = matrix_dp.read_value()
string_dp_value = string_dp.read_value()
# vectors do not have default value when created, needs a write before read
vector_dp.write_value([2, 4, 6])
vector_dp_value = vector_dp.read_value()
print('Initial Value of double_dp: ', const_dp_value.get_value())
print('Initial Value of string_dp: ', string_dp_value.get_value())
print('First Value of vector_dp: ', vector_dp_value.get_value())
# Write new values to the Data points
new_matrix_value = elt.oldb.VectorDOUBLE()
for x in range(250):
new_matrix_value.append(x)
double_dp.write_value(2.7)
matrix_dp.write_value(new_matrix_value)
string_dp.write_value('New string value')
vector_dp.write_value([7, 8, 9, 10])
# Read values
const_dp_value = double_dp.read_value()
matrix_dp_value = matrix_dp.read_value()
string_dp_value = string_dp.read_value()
vector_dp_value = vector_dp.read_value()
print('New Value of double_dp: ', const_dp_value.get_value())
print('New value of matrix_dp: ', matrix_dp_value.get_value())
print('New Value of string_dp: ', string_dp_value.get_value())
print('New Value of vector_dp: ', vector_dp_value.get_value())
# Get Multiple data points satisfying URI glob expression
uris = [ elt.config.Uri('cii.oldb:/%s/sampleroot/**' % RND_PREFIX) ]
search_result = oldb_client.get_data_points(uris)
print('Search Result: ', search_result)
# Get Multiple DOUBLE data points with value between 1.0 and 3.0
filtered_search_result = oldb_client.get_data_points(uris, elt.config.CiiBasicDataType.DOUBLE,
1.0, 3.0)
print('Filtered search result: ', filtered_search_result)
# Get the children of directory URI cii.oldb:/sampleroot. Returns a map specifying whether
# a child is a data point or just a directory
children = oldb_client.get_children(elt.config.Uri('cii.oldb:/sampleroot'))
# Delete data points
oldb_client.delete_data_point(double_dp_uri)
oldb_client.delete_data_point(matrix_dp_uri)
oldb_client.delete_data_point(string_dp_uri)
oldb_client.delete_data_point(vector_dp_uri)
# Delete meta data
elt.config.CiiConfigClient.delete_metadata(elt.oldb.CiiOldbGlobal.get_target_config_storage(),
double_dp_meta.get_instance_name())
def main():
"""main method wrapper"""
result = 0
try:
result = _main()
except Exception as e:
print(e)
traceback.print_tb(sys.exc_info()[2])
result = 5
return result
if __name__ == '__main__':
sys.exit(main())
7.4.1.8. Subscribing Example¶
The examples in this section can be found in the oldb-examples project (SubscriptionExample.java, subscription-sample-app.cpp, cii-oldb-examples-subscription-sample-app-py.py).
In the subscribing example (Listing 4‑4, Listing 4‑5, Listing 4‑6) a data point of type DOUBLE is created and a subscription is added to it to detect value changes and deletion.
Listing 4‑4: Java Subscribing Example
package sampleapp;
import java.net.URI;
import elt.oldb.client.CiiOldb;
import elt.oldb.client.CiiOldbDataPoint;
import elt.oldb.client.CiiOldbDpSubscription;
import elt.oldb.client.CiiOldbDpValue;
import elt.oldb.exceptions.CiiOldbConfigError;
import elt.oldb.exceptions.CiiOldbCyclicDependencyException;
import elt.oldb.exceptions.CiiOldbDeletedException;
import elt.oldb.exceptions.CiiOldbDpConstraintsException;
import elt.oldb.exceptions.CiiOldbDpExistsException;
import elt.oldb.exceptions.CiiOldbDpQualityException;
import elt.oldb.exceptions.CiiOldbExpressionException;
import elt.oldb.exceptions.CiiOldbIllegalOperationException;
import elt.oldb.exceptions.CiiOldbInitializationError;
import elt.oldb.exceptions.CiiOldbStorageError;
import elt.oldb.exceptions.CiiOldbURIException;
import elt.oldb.exceptions.CiiOldbUndefinedMetadataException;
import elt.oldb.exceptions.CiiOldbWriteDisabledException;
public class SubscriptionExampleApplication {
public static void main(String[] args) {
CiiOldb oldbClient = null;
try {
// Initialize OLDB and Config clients
oldbClient = CiiOldb.getInstance();
// Define URIs
URI doubleDpURI = URI.create("cii.oldb:/sampleroot/child/device/doubledp");
CiiOldbDataPoint<Double> doubleDp = oldbClient.createDataPointByValue(doubleDpURI, 1.0);
// Subscribe to a data point value changes.
doubleDp.subscribe(new CiiOldbDpSubscription() {
@Override
public void newValue(CiiOldbDpValue<?> value, URI uri) {
try {
System.out.println("newValue: " + uri + " " + value.getValue());
} catch (CiiOldbDpQualityException e) {
// Should not happen
}
}
@Override
public void dpRemoved(URI uri) {
System.out.println("dpRemoved: " + uri);
}
});
// Write to data points
doubleDp.writeValue(2.0);
// Delete data points
oldbClient.deleteDataPoint(doubleDpURI);
//Close the clients
oldbClient.close();
} catch (CiiOldbInitializationError | CiiOldbDpExistsException | CiiOldbCyclicDependencyException
| CiiOldbExpressionException | CiiOldbDpQualityException | CiiOldbDpConstraintsException
| CiiOldbUndefinedMetadataException | CiiOldbStorageError | CiiOldbConfigError | CiiOldbURIException
| CiiOldbWriteDisabledException | CiiOldbDeletedException e) {
e.printStackTrace();
if (oldbClient != null) {
oldbClient.close();
}
}
}
}
Listing 4‑5: C++ Subscribing Example
/**
* @copyright (c) Copyright ESO 2019 All Rights Reserved
* ESO (eso.org) is an Intergovernmental Organisation, and therefore special legal conditions apply.
* @ingroup oldb-examples
*/
#include <mal/utility/Uri.hpp>
#include <ciiOldbFactory.hpp>
#include <ciiOldbSubscription.hpp>
#include <ciiOldbDpValue.hpp>
#include <ciiOldbExceptions.hpp>
namespace elt {
namespace oldb {
namespace app {
class AppOldbDpSubscription : public CiiOldbDpSubscription<double> {
public:
AppOldbDpSubscription():
CiiOldbDpSubscription<double>(::elt::common::CiiBasicDataType::DOUBLE) {}
void DpRemoved(::elt::mal::Uri uri) override {
std::cout << "dpRemoved:" << uri.string() << '\n';
}
void NewValue(std::shared_ptr<CiiOldbDpValue<double>> value, ::elt::mal::Uri uri) override {
std::cout << "newValue:" << uri.string() << " " << value->GetValue() << '\n';
}
};
} // namespace app
} // namespace oldb
} // namespace elt
int main(int ac, char *av[]) {
try {
// Initialize OLDB
auto oldb_client = ::elt::oldb::CiiOldbFactory::GetInstance();
::elt::oldb::CiiOldbGlobal::SetWriteEnabled(true);
// Define URIs
std::string RND_PREFIX = ::elt::oldb::CiiOldbUtil::NewUUID();
::elt::mal::Uri double_dp_uri{"cii.oldb:/" + RND_PREFIX + "sampleroot/child/device/doubledp" };
// CREATE DATA POINTS
// meta data needs to be created before calling CreateDataPoint methods
auto mdi_double = ::elt::config::CiiConfigClient::retrieveMetadata(
::elt::oldb::CiiOldbGlobal::GetTargetConfigStorage(),
::elt::oldb::CiiOldbGlobal::MDI_DOUBLE,
::elt::oldb::CiiOldbGlobal::CONFIG_VERSION);
::elt::oldb::CiiOldbUtil::CreateDataPointInfo(double_dp_uri, *mdi_double);
auto double_dp = oldb_client->CreateDataPointByValue(double_dp_uri, 1.0);
// Subscribe to a data point value changes.
auto subscription = std::make_shared<::elt::oldb::app::AppOldbDpSubscription>();
double_dp->Subscribe(subscription);
// Subscription takes affect after some period after Subscribe
std::this_thread::sleep_for(std::chrono::seconds(2));
// Write to data points
double_dp->WriteValue(2.0);
// Takes time for subscription listener to be called
std::this_thread::sleep_for(std::chrono::seconds(2));
// Delete data points
oldb_client->DeleteDataPoint(double_dp_uri);
std::this_thread::sleep_for(std::chrono::seconds(3));
return 0;
} catch (const ::elt::oldb::CiiOldbException& ex) {
std::cerr << "CiiOldbException occured while executing sample code. What: "
<< ex.what() << '\n';
}
return -1;
}
Listing 4‑6: Python Subscribing Example
#!/usr/bin/env python
"""
@copyright (c) Copyright ESO 2019 All Rights Reserved
ESO (eso.org) is an Intergovernmental organisation,
and therefore special legal conditions apply.
@author Cosylab
"""
#This script presents DP subscription example.
import os, sys, traceback, time, threading, uuid
import elt.config
import elt.oldb
RND_PREFIX = str(uuid.uuid4())
# Initialize OLDB client
oldb_client = elt.oldb.CiiOldbFactory.get_instance()
# Define URIs
double_dp_uri = elt.config.Uri("cii.oldb:/%s/sampleroot/child/device/double_dp" % RND_PREFIX)
class AppOldbDpSubscription:
"""Subscription listener implementation, must implement
new_value and dp_removed methods
"""
def new_value(self, value, uri):
print('new_value, value=%s, uri=%s' % (value.get_value(), uri.string()))
def dp_removed(self, uri):
print('dp_removed, uri=%s' % (uri.string(),))
class WorkThread(threading.Thread):
def run(self):
dp = oldb_client.get_data_point(double_dp_uri)
for x in range(1, 3):
dp.write_value(x*.5)
time.sleep(0.2)
del dp
oldb_client.delete_data_point(double_dp_uri)
# Give subscription system time to pass the message around
time.sleep(1.0)
def _main():
# enable writing
elt.oldb.CiiOldbGlobal.set_write_enabled(True)
# create data points
# metadata needs to be created before calling cereate_data_point method
mdi_double = elt.config.CiiConfigClient.retrieve_metadata(
elt.oldb.CiiOldbGlobal.get_target_config_storage(),
elt.oldb.CiiOldbGlobal.MDI_DOUBLE,
elt.oldb.CiiOldbGlobal.CONFIG_VERSION)
elt.oldb.CiiOldbUtil.create_data_point_info(double_dp_uri, mdi_double)
double_dp = oldb_client.create_data_point_by_value(double_dp_uri, 1.0)
# Subscribe to data point. Pass instance of SubscriptionListener() to the subscription.
listener = elt.oldb.typesupport.DOUBLE.get_new_subscription_instance(AppOldbDpSubscription())
double_dp.subscribe(listener)
# Do not write to the data points from the same thread. Deadlock because of GIL.
# launch work thread that manipulates data
workThread = WorkThread()
workThread.start()
workThread.join()
def main():
"""main method wrapper"""
result = 0
try:
result = _main()
except Exception as e:
print(e)
traceback.print_tb(sys.exc_info()[2])
result = 5
return result
if __name__ == '__main__':
sys.exit(main())
7.4.1.9. Advanced Example¶
The examples in this section can be found in the oldb-examples project (AdvancedExample.java, advanced-sample-app.cpp, cii-oldb-examples-advanced-sample-app-py.py).
The advanced example (Listing 4‑8, Listing 4‑9, Listing 4‑10) covers the advanced topics of working with the OLDB library. Note that the calculation service must be running for this example to produce expected results. In the example the following actions are executed:
A constant DOUBLE data point is created (inputDubleDp),
a DOUBLE data point with quality expression is created (squareDoubleDp),
a constant VECTOR_DOUBLE data point is created (inputVectorDp),
a calculated DOUBLE data point is created (sumOfSquaresDp), It calculates the sum of squares of inputVectorDp elements. It uses a custom function which is defined in the $ INTROOT/etc/oldb/octave/oldb_custom.m file (Listing 4‑7),
a value that causes the quality expression evaluate to BAD is written to squareDoubleDp,
the value of squareDoubleDp is rewritten to change the quality back to OK,
the storage location of inputVectorDp value is changed to in-memory-only storage,
the metadata of squareDoubleDp is updated with formula so that squareDoubleDp becomes a calculated data point. It calculates the square of inputDubleDp value,
write operations are disabled and check is done that it is indeed impossible to write to data points,
data points are deleted.
Listing 4‑7: Custom Formula Function.
function returnval = SUM_SQUARE(v)
s = 0
for i = v
s = s + i^2
endfor
returnval = s
endfunction
Listing 4‑8: Java Advanced Example
package sampleapp;
import java.net.URI;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import elt.config.classes.CiiOldbDataPointInfo;
import elt.config.classes.meta.MdOldbNumber;
import elt.config.classes.meta.MdString;
import elt.config.client.CiiConfigClient;
import elt.config.exceptions.CiiConfigInitializationError;
import elt.config.exceptions.CiiConfigNoTcException;
import elt.config.exceptions.CiiConfigSaveException;
import elt.config.exceptions.CiiConfigUpdateException;
import elt.config.exceptions.CiiConfigWriteDisabledException;
import elt.error.CiiInvalidTypeException;
import elt.error.CiiInvalidURIException;
import elt.oldb.client.CiiOldb;
import elt.oldb.client.CiiOldbDataPoint;
import elt.oldb.client.CiiOldbDpValue;
import elt.oldb.client.impl.CiiOldbImp;
import elt.oldb.client.util.CiiOldbUriUtils;
import elt.oldb.exceptions.CiiOldbConfigError;
import elt.oldb.exceptions.CiiOldbCyclicDependencyException;
import elt.oldb.exceptions.CiiOldbDeletedException;
import elt.oldb.exceptions.CiiOldbDpConstraintsException;
import elt.oldb.exceptions.CiiOldbDpExistsException;
import elt.oldb.exceptions.CiiOldbDpQualityException;
import elt.oldb.exceptions.CiiOldbDpUndefinedException;
import elt.oldb.exceptions.CiiOldbExpressionException;
import elt.oldb.exceptions.CiiOldbIllegalOperationException;
import elt.oldb.exceptions.CiiOldbInitializationError;
import elt.oldb.exceptions.CiiOldbInvalidTypeException;
import elt.oldb.exceptions.CiiOldbStorageError;
import elt.oldb.exceptions.CiiOldbURIException;
import elt.oldb.exceptions.CiiOldbUndefinedMetadataException;
import elt.oldb.exceptions.CiiOldbWriteDisabledException;
public class AdvancedSampleApp {
public static void main(String[] args) {
CiiOldb oldbClient = null;
CiiConfigClient configClient = null;
URI inputDoubleUri = null;
URI squareDoubleUri = null;
URI inputVectorUri = null;
URI sumofsquaresDoubleUri = null;
try {
// Initialize OLDB and Config clients
oldbClient = CiiOldb.getInstance();
configClient = CiiConfigClient.getInstance();
inputDoubleUri = URI.create("cii.oldb:/sampleroot/inputdouble");
squareDoubleUri = URI.create("cii.oldb:/sampleroot/squaredouble");
inputVectorUri = URI.create("cii.oldb:/sampleroot/inputdector");
sumofsquaresDoubleUri =
URI.create("cii.oldb:/sampleroot/sumofsquaresdouble");
oldbClient.deleteDataPoint(squareDoubleUri);
oldbClient.deleteDataPoint(inputDoubleUri);
oldbClient.deleteDataPoint(sumofsquaresDoubleUri);
oldbClient.deleteDataPoint(inputVectorUri);
// Create a constant DOUBLE data point
CiiOldbDataPoint<Double> inputDubleDp =
oldbClient.createDataPointByValue(inputDoubleUri, 1.1);
// Create a DOUBLE data point with a quality expression
MdOldbNumber<Double> squareMd =
new MdOldbNumber<Double>("squareOfSumMd", "", 2.0, null,
"if (CURR_VAL() > 1.0) SETQUAL(\"OK\") else SETQUAL(\"BAD\") endif", null, null, null);
URI squareMdURI = URI.create(CiiOldbUriUtils.CONFIG_METADATA_URI + squareMd.getMetadataInstanceName());
configClient.saveMetadata(squareMdURI, squareMd);
CiiOldbDataPoint<Double> squareDoubleDp = (CiiOldbDataPoint<Double>) oldbClient.createDataPoint(
squareDoubleUri, squareMd.getMetadataInstanceName(), squareMd.getClass().getSimpleName());
// Create a VECTOR_DOUBLE data point
CiiOldbDataPoint<List<Double>> inputVectorDp = oldbClient.createDataPointByValue(inputVectorUri,
new ArrayList<>(Arrays.asList(1.0, 2.0, 3.0)), Double.class);
// Create a DOUBLE data poitn with formula
MdOldbNumber<Double> sumOfSquaresMd = new MdOldbNumber<Double>("sumOfSquares", "", 0.0,
String.format("SUM_SQUARE(${%s})", inputVectorUri.toString()), null, null, null, null);
URI squareOfSumMdURI = URI
.create(CiiOldbUriUtils.CONFIG_METADATA_URI + sumOfSquaresMd.getMetadataInstanceName());
configClient.saveMetadata(squareOfSumMdURI, sumOfSquaresMd);
CiiOldbDataPoint<Double> sumOfSquaresDp = (CiiOldbDataPoint<Double>) oldbClient.createDataPoint(
sumofsquaresDoubleUri, sumOfSquaresMd.getMetadataInstanceName(),
sumOfSquaresMd.getClass().getSimpleName());
try {
Thread.sleep(100);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// Read the values of data points to confirm the calculation have been done.
CiiOldbDpValue<Double> squareDpValue = squareDoubleDp.readValue();
System.out.println(squareDoubleUri);
System.out.println("value :" + squareDpValue.getValue());
System.out.println("quality :" + squareDpValue.getQuality());
CiiOldbDpValue<Double> sumOfSquaresDpValue = sumOfSquaresDp.readValue();
System.out.println(sumofsquaresDoubleUri);
System.out.println("value :" + sumOfSquaresDpValue.getValue());
System.out.println("quality :" + sumOfSquaresDpValue.getQuality());
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// Write a value that causes the quality expression to evaluate quality of data
// point to BAD
squareDoubleDp.writeValue(0.1);
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// Check that the quality of data point is BAD.
squareDpValue = squareDoubleDp.readValue();
System.out.println(squareDoubleUri);
System.out.println("quality :" + squareDpValue.getQuality());
try {
squareDpValue.getValue();
} catch (CiiOldbDpQualityException e) {
System.out.println(
"Cannot call getValue because the quality is bad. The value can be retrieved from exception: "
+ e.getValue());
}
// Write a value that causes the quality expression to evaluate quality of data
// point to OK
squareDoubleDp.writeValue(1.3);
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// Check that the quality of data point is back to OK.
squareDpValue = squareDoubleDp.readValue();
System.out.println(squareDoubleUri);
System.out.println("value :" + squareDpValue.getValue());
System.out.println("quality :" + squareDpValue.getQuality());
// Change the storage location of the data point to in-memory-only storage
URI inputVectorConfigURI = URI.create(CiiOldbUriUtils.CONFIG_INFO_URI_ROOT + inputVectorUri.getPath());
CiiOldbDataPointInfo infoConfig = configClient.retrieveConfig(inputVectorConfigURI)
.getData(CiiOldbDataPointInfo.class);
MdString serverAlias = new MdString("testExtRedisServer");
serverAlias.setVersion(1);
infoConfig.setServerAlias(serverAlias);
configClient.updateConfig(inputVectorConfigURI, infoConfig);
try {
Thread.sleep(100);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
inputVectorDp = (CiiOldbDataPoint<List<Double>>) oldbClient.getDataPoint(inputVectorUri);
inputVectorDp.writeValue(new ArrayList<>(Arrays.asList(2.0, 3.0, 4.0)));
// Update a constant data point metadata to change it into calculated data point
// (add a formula to metadata)
squareMd.setFormula(String.format("${%s}^2", inputDoubleUri.toString()));
configClient.updateMetadata(
CiiOldbUriUtils.createMetaDataConfigUri(squareMd.getMetadataInstanceName()),
squareMd.getMetadataInstanceName(), -1, squareMd);
squareDoubleDp.setMetadata(squareMd.getMetadataInstanceName());
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// Read the data point to confirm that the value is now calculated
squareDpValue = squareDoubleDp.readValue();
System.out.println(squareDoubleUri);
System.out.println("value :" + squareDpValue.getValue());
System.out.println("quality :" + squareDpValue.getQuality());
// Disable write operations on data points
oldbClient.setWriteEnabled(false);
try {
inputDubleDp.writeValue(0.2);
} catch (CiiOldbWriteDisabledException e) {
System.out.println("Write on DP not permited");
}
oldbClient.setWriteEnabled(true);
// Delete data points
oldbClient.deleteDataPoint(squareDoubleUri);
oldbClient.deleteDataPoint(inputDoubleUri);
oldbClient.deleteDataPoint(sumofsquaresDoubleUri);
oldbClient.deleteDataPoint(inputVectorUri);
// close OLDB and Config clients
configClient.close();
oldbClient.close();
} catch (CiiConfigInitializationError | CiiOldbInitializationError | CiiOldbDpExistsException
| CiiOldbCyclicDependencyException | CiiOldbExpressionException | CiiOldbDpQualityException
| CiiOldbDpConstraintsException | CiiOldbUndefinedMetadataException | CiiOldbStorageError
| CiiOldbConfigError | CiiOldbURIException | CiiOldbWriteDisabledException | CiiOldbDeletedException
| CiiInvalidTypeException | CiiConfigSaveException | CiiConfigWriteDisabledException
| CiiInvalidURIException | CiiOldbDpUndefinedException | CiiOldbInvalidTypeException
| CiiConfigNoTcException | CiiConfigUpdateException e) {
e.printStackTrace();
if (configClient != null) {
configClient.close();
}
if (oldbClient != null) {
oldbClient.close();
}
}
}
}
Listing 4‑9: C++ Advanced Example
/**
* @copyright (c) Copyright ESO 2019 All Rights Reserved
* ESO (eso.org) is an Intergovernmental Organisation, and therefore special legal conditions apply.
* @ingroup oldb-examples
*/
#include <mal/utility/Uri.hpp>
#include <ciiOldbFactory.hpp>
#include <ciiOldbExceptions.hpp>
int main(int ac, char *av[]) {
try {
// Initialize OLDB
auto oldb_client = ::elt::oldb::CiiOldbFactory::GetInstance();
::elt::oldb::CiiOldbGlobal::SetWriteEnabled(true);
std::string RND_PREFIX = ::elt::oldb::CiiOldbUtil::NewUUID();
// Define URIs
::elt::mal::Uri input_double_uri{
"cii.oldb:/" + RND_PREFIX + "/sampleroot/child/device/doubledp"};
::elt::mal::Uri square_double_uri{"cii.oldb:/" + RND_PREFIX + "/sampleroot/squaredouble"};
::elt::mal::Uri input_vector_uri{"cii.oldb:/" + RND_PREFIX + "/sampleroot/inputvector"};
::elt::mal::Uri sum_of_squares_double_uri{
"cii.oldb:/" + RND_PREFIX + "/sampleroot/sumofsquaresdouble"};
// should not be any data points with this uri since uri has random part
// (consistent with java sample)
try {
oldb_client->DeleteDataPoint(square_double_uri);
} catch (const ::elt::oldb::CiiOldbException &) {}
try {
oldb_client->DeleteDataPoint(input_double_uri);
} catch (const ::elt::oldb::CiiOldbException &) {}
try {
oldb_client->DeleteDataPoint(sum_of_squares_double_uri);
} catch (const ::elt::oldb::CiiOldbException &) {}
try {
oldb_client->DeleteDataPoint(input_vector_uri);
} catch (const ::elt::oldb::CiiOldbException &) {}
// Create a constant DOUBLE data point
// meta data needs to be created before calling CreateDataPoint methods
auto mdi_double = ::elt::config::CiiConfigClient::retrieveMetadata(
::elt::oldb::CiiOldbGlobal::GetTargetConfigStorage(),
::elt::oldb::CiiOldbGlobal::MDI_DOUBLE,
::elt::oldb::CiiOldbGlobal::CONFIG_VERSION);
::elt::oldb::CiiOldbUtil::CreateDataPointInfo(input_double_uri, *mdi_double);
auto input_double_dp = oldb_client->CreateDataPointByValue(input_double_uri, 1.1);
// Create a DOUBLE data point with a quality expression
std::string square_md_instance_name = "squareOfSumMd";
auto square_md = CiiDataPointMetadataFactory::getNewMetadataInstance<
::elt::config::classes::meta::MdOldb<double>>(square_md_instance_name);
square_md->set_default_value(2.0);
square_md->set_qualityExpression(
"if (CURR_VAL() > 1.0) SETQUAL(\"OK\") else SETQUAL(\"BAD\") endif");
// meta data needs to be created before calling CreateDataPoint
::elt::oldb::CiiOldbUtil::CreateDataPointInfo(square_double_uri, *square_md);
auto square_double_dp = oldb_client->CreateDataPoint<double>(
square_double_uri, square_md_instance_name);
// Create a VECTOR_DOUBLE data point
std::vector<double> vector_double = { 1.0, 2.0, 3.0 };
auto mdi_vector_double = ::elt::config::CiiConfigClient::retrieveMetadata(
::elt::oldb::CiiOldbGlobal::GetTargetConfigStorage(),
::elt::oldb::CiiOldbGlobal::MDI_VECTOR_DOUBLE,
::elt::oldb::CiiOldbGlobal::CONFIG_VERSION);
::elt::oldb::CiiOldbUtil::CreateDataPointInfo(input_vector_uri, *mdi_vector_double);
bool is_matrix = false;
auto input_vector_dp = oldb_client->CreateDataPointByValue(
input_vector_uri, vector_double, is_matrix);
// Create a DOUBLE data point with formula
std::string sum_of_squares_md_instance_name = "sumOfSquares";
auto sum_of_squares_md = CiiDataPointMetadataFactory::getNewMetadataInstance<
::elt::config::classes::meta::MdOldb<double>>(sum_of_squares_md_instance_name);
std::string formula = "SUM_SQUARE(${" + input_vector_uri.string() + "})";
sum_of_squares_md->set_default_value(0.0);
sum_of_squares_md->set_formula(formula);
::elt::oldb::CiiOldbUtil::CreateDataPointInfo(sum_of_squares_double_uri, *sum_of_squares_md);
auto sum_of_squares_dp = oldb_client->CreateDataPoint<double>(
sum_of_squares_double_uri, sum_of_squares_md_instance_name);
std::this_thread::sleep_for(std::chrono::milliseconds(100));
// Read the values of data points to confirm the calculation have been done.
auto square_dp_value = square_double_dp->ReadValue();
std::cout << square_double_uri.string() << '\n';
std::cout << "value: " << square_dp_value->GetValue() << '\n';
std::cout << "quality:"
<< ::elt::oldb::CiiOldbUtil::ToQualityStr(square_dp_value->GetQuality()) << "\n\n";
auto sum_of_squares_dp_value = sum_of_squares_dp->ReadValue();
std::cout << sum_of_squares_double_uri.string() << '\n';
std::cout << "value: " << sum_of_squares_dp_value->GetValue() << '\n';
std::cout << "quality:"
<< ::elt::oldb::CiiOldbUtil::ToQualityStr(sum_of_squares_dp_value->GetQuality()) << "\n\n";
std::this_thread::sleep_for(std::chrono::milliseconds(2000));
// Write a value that causes the quality expression to evaluate quality of data point to BAD
square_double_dp->WriteValue(0.1);
std::this_thread::sleep_for(std::chrono::milliseconds(2000));
// Check that the quality of data point is BAD.
square_dp_value = square_double_dp->ReadValue();
std::cout << square_double_uri.string() << '\n';
std::cout << "quality:"
<< ::elt::oldb::CiiOldbUtil::ToQualityStr(square_dp_value->GetQuality()) << "\n\n";
try {
square_dp_value->GetValue();
} catch(const ::elt::oldb::CiiOldbDpQualityWithValueException<double>& ex) {
std::cerr << "Cannot call getValue because the quality is bad. "
"The value can be retrieved from exception: " << ex.GetValue() << '\n';
}
// Write a value that causes the quality expression to evaluate quality of data point to OK
square_double_dp->WriteValue(1.3);
std::this_thread::sleep_for(std::chrono::milliseconds(2000));
// Check that the quality of data point is back to OK.
square_dp_value = square_double_dp->ReadValue();
std::cout << square_double_uri.string() << '\n';
std::cout << "value: " << square_dp_value->GetValue() << '\n';
std::cout << "quality:"
<< ::elt::oldb::CiiOldbUtil::ToQualityStr(square_dp_value->GetQuality()) << "\n\n";
// Change the storage location of the data point to in-memory-only storage
::elt::mal::Uri input_vector_config_uri(
::elt::oldb::CiiOldbGlobal::GetTargetConfigDataPointsUriPrefix()
+ input_vector_uri.path().to_string());
auto info_config =
elt::config::CiiConfigClient::getConfigData<::elt::config::classes::CiiOldbDataPointInfo>(
input_vector_config_uri, ::elt::oldb::CiiOldbGlobal::CONFIG_VERSION);
info_config->set_serverAlias("testExtRedisServer");
::elt::config::CiiConfigClient::saveTargetConfig(input_vector_config_uri, *info_config);
std::this_thread::sleep_for(std::chrono::milliseconds(100));
input_vector_dp = oldb_client->GetDataPoint<std::vector<double>>(
input_vector_uri, ::elt::common::CiiBasicDataType::VECTOR_DOUBLE);
input_vector_dp->WriteValue(std::vector<double> { 2.0, 3.0, 4.0 });
// Update a constant data point metadata to change it into calculated data point
// (add a formula to metadata)
formula = "${" + input_double_uri.string() + "}^2";
square_md->set_formula(formula);
square_double_dp->SetMetadata(square_md_instance_name);
std::this_thread::sleep_for(std::chrono::milliseconds(1000));
// Read the data point to confirm that the value is now calculated
square_dp_value = square_double_dp->ReadValue();
std::cout << square_double_uri.string() << '\n';
std::cout << "value: " << square_dp_value->GetValue() << '\n';
std::cout << "quality:"
<< ::elt::oldb::CiiOldbUtil::ToQualityStr(square_dp_value->GetQuality()) << "\n\n";
// Disable write operations on data points
::elt::oldb::CiiOldbGlobal::SetWriteEnabled(false);
try {
input_double_dp->WriteValue(0.2);
} catch(const ::elt::oldb::CiiOldbWriteDisabledException&) {
std::cerr << "Write on DP not permitted\n";
}
::elt::oldb::CiiOldbGlobal::SetWriteEnabled(true);
// Delete data points
oldb_client->DeleteDataPoint(square_double_uri);
oldb_client->DeleteDataPoint(input_double_uri);
oldb_client->DeleteDataPoint(sum_of_squares_double_uri);
oldb_client->DeleteDataPoint(input_vector_uri);
return 0;
} catch (const ::elt::oldb::CiiOldbException& ex) {
std::cerr << "CiiOldbException occured while executing sample code. What: "
<< ex.what() << '\n';
} catch (const std::exception& ex) {
std::cerr << "Unknown exception occured while executing sample code. What: "
<< ex.what() << '\n';
}
return -1;
}
Listing 4‑10: Python Advanced Example
#!/usr/bin/env python
"""
@copyright (c) Copyright ESO 2019 All Rights Reserved
ESO (eso.org) is an Intergovernmental organisation,
and therefore special legal conditions apply.
@ingroup client-apis-python
@author Cosylab
"""
# Make sure OLDB services are running before executing this script.
import os, sys, traceback, time, threading, uuid
import elt.config
import elt.oldb
RND_PREFIX = str(uuid.uuid4())
# Define URIs
input_double_uri = elt.config.Uri("cii.oldb:/%s/sampleroot/child/device/doubledp" % RND_PREFIX)
square_double_uri = elt.config.Uri("cii.oldb:/%s/sampleroot/child/device/squaredouble" % RND_PREFIX)
input_vector_uri = elt.config.Uri("cii.oldb:/%s/sampleroot/child/device/inputvector" % RND_PREFIX)
sum_of_squares_uri = elt.config.Uri("cii.oldb:/%s/sampleroot/child/device/sumofsquaredouble" % RND_PREFIX)
def _main():
# Initialize OLDB
elt.oldb.CiiOldbGlobal.set_write_enabled(True)
elt.oldb.CiiOldbGlobal.set_calc_enabled(True)
# Initialize OLDB client
oldb_client = elt.oldb.CiiOldbFactory.get_instance()
# Remove Data points if they already exist
try:
oldb_client.delete_data_point(input_double_uri)
except:
pass
try:
oldb_client.delete_data_point(square_double_uri)
except:
pass
try:
oldb_client.delete_data_point(input_vector_uri)
except:
pass
try:
oldb_client.delete_data_point(sum_of_squares_uri)
except:
pass
# Create a constant DOUBLE data point
# meta data needs to be created before calling create_data_point method
mdi_double = elt.config.CiiConfigClient.retrieve_metadata(
elt.oldb.CiiOldbGlobal.get_target_config_storage(),
elt.oldb.CiiOldbGlobal.MDI_DOUBLE,
elt.oldb.CiiOldbGlobal.CONFIG_VERSION)
elt.oldb.CiiOldbUtil.create_data_point_info(input_double_uri, mdi_double)
input_double_dp = oldb_client.create_data_point_by_value(input_double_uri, 1.1)
# Create a DOUBLE data point with quality expression
square_md = elt.oldb.typesupport.DOUBLE.get_new_number_metadata_instance('squareOfSumMd')
square_md.set_default_value(2.0)
square_md.set_quality_expression('if (CURR_VAL() > 1.0) SETQUAL("OK") else SETQUAL("BAD") endif')
# meta data needs to be created before calling create_data_point
elt.oldb.CiiOldbUtil.create_data_point_info(square_double_uri, square_md)
square_double_dp = oldb_client.create_data_point(square_double_uri, square_md.get_instance_name())
# Create VECTOR_DOUBLE data point
vector_double = elt.oldb.VectorDOUBLE([ 1.0, 2.0, 3.0 ])
mdi_vector_double = elt.config.CiiConfigClient.retrieve_metadata(
elt.oldb.CiiOldbGlobal.get_target_config_storage(),
elt.oldb.CiiOldbGlobal.MDI_VECTOR_DOUBLE,
elt.oldb.CiiOldbGlobal.CONFIG_VERSION)
elt.oldb.CiiOldbUtil.create_data_point_info(input_vector_uri, mdi_vector_double)
input_vector_dp = oldb_client.create_data_point_by_value(input_vector_uri, vector_double, False)
# Create DOUBLE data point with formula
sum_of_squares_md = elt.oldb.typesupport.DOUBLE.get_new_number_metadata_instance('sumOfSquares')
formula = 'SUM_SQUARE(${%s})' % (input_vector_uri.string(),)
sum_of_squares_md.set_default_value(0.0)
sum_of_squares_md.set_formula(formula)
# Create & Save Metadata
elt.oldb.CiiOldbUtil.create_data_point_info(sum_of_squares_uri, sum_of_squares_md)
sum_of_squares_dp = oldb_client.create_data_point(sum_of_squares_uri, sum_of_squares_md.get_instance_name())
# Wait for calculations to finish
time.sleep(3.0)
square_dp_value = square_double_dp.read_value()
print('%s, value: %s, quality: %s' % (square_double_uri.string(), square_dp_value.get_value(),
square_dp_value.get_quality()))
sum_of_squares_dp_value = sum_of_squares_dp.read_value()
print('%s, value: %s, quality: %s' % (sum_of_squares_uri.string(), sum_of_squares_dp_value.get_value(),
sum_of_squares_dp_value.get_quality()))
# Write a value that causes the quality expression to evaluate quality of data point to BAD
square_double_dp.write_value(0.1)
time.sleep(2.0)
try:
square_dp_value = square_double_dp.read_value()
print('%s, quality: %s' % (square_double_uri, square_dp_value.get_quality()))
square_dp_value.get_value()
except elt.oldb.CiiOldbDpQualityException as e:
print("Cannot call get_value because quality is bad.")
# Write a value that causes the quality expression to evaluate quality of data point back to OK
square_double_dp.write_value(1.3)
time.sleep(2.0)
# Check that quality of data point is back to OK
square_dp_value = square_double_dp.read_value()
print('%s, value: %s, quality: %s' % (square_double_uri, square_dp_value.get_value(),
square_dp_value.get_quality()))
# Change the storage location of the data point to in-memory-only storage
input_vector_config_uri = elt.config.Uri(
elt.oldb.CiiOldbGlobal.get_target_config_data_points_uri_prefix() +
input_vector_uri.path())
info_config = elt.config.CiiConfigClient.retrieve_config(input_vector_config_uri,
elt.oldb.CiiOldbGlobal.CONFIG_VERSION)
info_config.set_server_alias("testExtRedisServer")
elt.config.CiiConfigClient.save_target_config(input_vector_config_uri, info_config)
time.sleep(0.1)
input_vector_dp = oldb_client.get_data_point(input_vector_uri)
input_vector_dp.write_value([2.0, 3.0, 4.0])
# Update a constant data point metadata to change it into calculated data point
# (add formula to metadata)
formula = '${%s}^2' % (input_double_uri.string(),)
square_md.set_formula(formula)
square_double_dp.set_metadata(square_md.get_instance_name())
time.sleep(2.0)
# Read the data point to confirm that the value is now calculated
square_dp_value = square_double_dp.read_value()
print('%s, value: %s, quality: %s' % (square_double_uri.string(), square_dp_value.get_value(),
square_dp_value.get_quality()))
# Disable write operations on data points
elt.oldb.CiiOldbGlobal.set_write_enabled(False)
try:
input_double_dp.write_value(7.7)
except elt.oldb.CiiOldbWriteDisabledException as e:
print('Write on DP not permitted')
# Enable write operations
elt.oldb.CiiOldbGlobal.set_write_enabled(True)
# Delete data points
oldb_client.delete_data_point(square_double_uri)
oldb_client.delete_data_point(input_double_uri)
oldb_client.delete_data_point(sum_of_squares_uri)
oldb_client.delete_data_point(input_vector_uri)
# Delete meta data
elt.config.CiiConfigClient.delete_metadata(elt.oldb.CiiOldbGlobal.get_target_config_storage(),
square_md.get_instance_name())
elt.config.CiiConfigClient.delete_metadata(elt.oldb.CiiOldbGlobal.get_target_config_storage(),
sum_of_squares_md.get_instance_name())
def main():
"""main method wrapper"""
result = 0
try:
result = _main()
except Exception as e:
print(e)
traceback.print_tb(sys.exc_info()[2])
result = 5
return result
if __name__ == '__main__':
sys.exit(main())
7.5. Advanced Topics¶
This section describes in detail the advanced functionalities of OLDB API library.
7.5.1. OLDB Statistics¶
Basic traffic statistics can be obtained for the OLDB Client and Calculation service.
7.5.1.1. OLDB Client Statistics¶
For OLDB client the basic operation statistics can be obtained directly from Redis by using the redis-cli command line client application. Note that Redis must be installed on the machine so that the redis-cli application is available. See [2] for how to install Redis. Redis client app can be run in the terminal. It can be given options -h (for hostname) and -p (for the port) to connect to specific Redis server. By default it will connect to localhost:6379.
redis-cli -h 10.71.0.247
10.71.0.247:6379>
Do get basic statistics about the read and write operations the INFO command with commandstats option should be run (Listing 5‑1).
Listing 5‑1: INFO commandstats result
10.71.0.247:6379> INFO commandstats
# Commandstats
cmdstat_expire:calls=54,usec=754,usec_per_call=13.96
cmdstat_subscribe:calls=690,usec=9138,usec_per_call=13.24
cmdstat_exists:calls=296,usec=2683,usec_per_call=9.06
cmdstat_get:calls=948,usec=8600,usec_per_call=9.07
cmdstat_mget:calls=11,usec=88,usec_per_call=8.00
cmdstat_mset:calls=59,usec=662,usec_per_call=11.22
cmdstat_del:calls=190,usec=1708,usec_per_call=8.99
cmdstat_command:calls=3,usec=3138,usec_per_call=1046.00
cmdstat_set:calls=29601,usec=411882,usec_per_call=13.91
cmdstat_publish:calls=29992,usec=318727,usec_per_call=10.63
cmdstat_info:calls=3,usec=273,usec_per_call=91.00
The most important fields in the (Listing 5‑1) are:
cmdstat_get:calls – number of read operations.
cmdstat_set:calls – number of write operations.
cmdstat_del:calls – number of delete operations.
See [7] for full explanation of the INFO command and how to get other information about the Redis server (e.g. master – slave statistics for cluster servers).
7.5.1.2. Calculation Service Statistics¶
Calculation service exposes basic statistic using the Java MBeans JMX interface. Statistics can be obtained using various JMX tools. The example shows usage of jconsole. The user should start the jconsole with:
jconsole&
Calculation service can be chosen from the list of processes and connected to by clicking on the Connect button (Figure 5‑1). In this example, a calculation node process is chosen. The same statistics can be found for the Scheduler process.
Figure 5‑1: Connect to Calculation Service with Jconsole
After the connection is made, the console with statistics is displayed. The console displays heap usage, CPU usage, number of threads, and similar java statistics. The specific Calculation service statistics are exposed under the elt.oldb.calc.service.stat namespace.
To obtain statistics, the user should click on the MBeans tab and browse the elt.oldb.calc.service.stat namespace (Figure 5‑2). The statistics show up-time of the calculation node and number of data points calculated by this calculation node (the number of calculation operations made).
Figure 5‑2: Calculation Service Statistics
7.5.2. Disabling Writes to Data Points¶
Write operations on data points can be disabled or enabled by calling the setWriteEnabled method on the OLDB client. If disabled, all calls of writeValue on data points will throw a CiiOldbWriteDisabledException. Note that the write disabling only effects the same process environment in which it was done. Two OLDB clients on different machines (or in different processes on the same machine) can have different settings regarding the permission of write operations. Write operations are by default enabled.
//Disable write operations on data points
oldbClient.setWriteEnabled(false);
try {
inputDubleDp.writeValue(0.2);
} catch (CiiOldbWriteDisabledException e) {
System.out.println("Write on DP not permited");
}
oldbClient.setWriteEnabled(true);
7.5.3. Custom functions in Data Point Formulas¶
Formulas of calculated data points can include any functions from the Octave language, but also user defined functions. The user can include their custom Octave functions in the $INTROOT/etc/oldb/octave/oldb_custom.m file.
function returnval = SUM_SQUARE(v)
s = 0
for i = v
s = s + i^2
endfor
returnval = s
endfunction
These functions must be included in the file before the Calculation Service is started. Changes to oldb_custom.m file will not take effect if the Calculation Service is already running.
The custom functions can then be included in data point formulas.
//Create a DOUBLE data poitn with formula
MdOldbNumber<Double> sumOfSquaresMd = new MdOldbNumber<Double>("sumOfSquares", "", 0.0, String.format("SUM_SQUARE(${%s})", inputVectorUri.toString()), null, null, null, null);
URI squareOfSumMdURI = URI.create(CiiOldbImp.CONFIG_METADATA_URI + sumOfSquaresMd.getMetadataInstanceName());
configClient.saveMetadata(squareOfSumMdURI, sumOfSquaresMd);
CiiOldbDataPoint<Double> sumOfSquaresDp = oldbClient.createDataPoint(sumofsquaresDoubleUri, sumOfSquaresMd);
To see how to write Octave functions see [6].
7.5.4. Manipulating Data Point Metadata¶
Metadata of the data point can be updated dynamically. Certain metadata attributes (e.g. limits or formula) can be changed or the data point’s metadata instance can be replaced altogether (change of the metadata instance name). This can be done by calling the setMetadata method on the data point which accepts a string metadata instance name. The data point metadata can be changed in two ways:
The attributes of the metadata of this data point have been changed (the provided metadata instance name is the same as the one the data point already has). These changes must be saved to Configuration Service before calling this method for it to have any effect.
The metadata instance of the data point has been replaced (the provided metadata instance name is different than the one the data point already has). The new metadata must already exist and must be of the same type as the old one. This change will be propagated to the data point configuration (Section 5.5).
The change will be propagated to the calculation service so that if the change is relevant to it (e.g. a data point has become a calculated data point), it can take proper actions (e.g. add it to a calculation node).
//Update a constant data point metadata to change it into calculated data point
//(add a formula to metadata)
squareMd.setFormula(String.format("${%s}^2", inputDoubleUri.toString()));
configClient.updateMetadata(
CiiOldbUriUtils.createMetaDataConfigUri(squareMd.getMetadataInstanceName()),
squareMd.getMetadataInstanceName(), -1, squareMd);
squareDoubleDp.setMetadata(squareMd);
Note that the type of data point cannot be changed. Trying to update metadata with a different type of metadata will result in CiiOldbInvalidTypeException. That is, the class of the metadata cannot change once it has been defined in the data point at creation. If there is a need for data point type change the data point must be deleted and created again with the updated metadata.
The change will however not be propagated to other instances of the data point. For the change to take effect on these data points must be fetched again from the OLDB.
7.5.5. Data Point Configuration¶
Each data point has a configuration in Configuration Service which is created automatically at data point creation. It is primarily used for internal management so that the user needs not to concern themselves with it except in a case where the storage location needs to be changed.
The data point configuration is stored in the CiiOldbDataPointInfo object. This configuration class contains the following fields.
uri: the MdString class containing the URI of the data point as the string
metadataInstanceName: the MdString class containing the data point’s metadata instance name as a string.
metadataClassName: the MdString class containing the data point’s metadata class name as a string (See Table 6‑1 for a list of OLDB metadata classes).
serverAlias: The MdString class containing the alias for external Redis server (in-memory-only data storage) on which this data point value is stored. All newly created data points have this field null which means that their value is stored in default storage (default Redis server or HDFS).
The only field relevant to the user is the serverAlias, for it can be used to change the storage location of the data point (Section 5.7).
7.5.6. External Redis Servers¶
External Redis servers are used for data points with larger values that need quicker access than an HDFS storage is able to provide. They are regular Redis servers (can be single instances or clusters), but data point values sliced and saved under multiple key-value pairs. The number of slices depends on the data size limit of each slice which is defined with the valueSizeLimitRedisExt field in the OLDB configuration (Section 3.3.1). The list of the external Redis servers that OLDB client will connect to is written in the OLDB configuration. This setting is read at OLDB client start up (See redisServersExternal filed in Section 3.3.1). There each external server is formatted as <alias>:<hostname>:<port>:<server-type> where server-type is either SINGLE or CLUSTER and describes the type of the server. This alias, which is defined by the administrator can be used to assign a data point to the external server. A single specific data point can be assigned to specific Redis External Server. The data point setting serverAlias is used to control this behavior (5.5).
As the in-memory-only servers are intended to be used for high throughput, the properly tuned Redis setting need to be applied. The setting can be set to single or cluster Redis, but when low latency is required a cluster can cause additional delay. By design, Redis is single-threaded, which means that every write or read to the server, will block it. In case the cluster is used, the time to copy the data between the cluster parts need to be considered. If large blocks of data are used, these times can be very long.
The data points that have value written to the external Redis server have also a key-value field on the default Redis storage. This key-value doesn’t represent the actual data point value, but it is only the address of the external server (alias of the external Redis).
7.5.7. Changing the Storage Location of the Data Point¶
To change the Storage location of the data point, the user must update the data point configuration and change the serverAlias field. If this field is set to null the data point value will be moved to default Redis servers, otherwise, it will be moved to external Redis server with this alias. In code, this can be done using the ConfigClient API from the Configuration Service [3]. Example of changing data point storage location can be seen in Listing 5‑2.
Listing 5‑2: Example of changing data point storage location
//Change the storage location of the data point to in-memory-only storage
URI inputVectorConfigURI = URI.create(CiiOldbImp.CONFIG_INFO_URI_ROOT + inputVectorUri.getPath());
CiiOldbDataPointInfo infoConfig = configClient.retrieveConfig(inputVectorConfigURI).getData(CiiOldbDataPointInfo.class);
MdString serverAlias = new MdString("testExtRedisServer");
serverAlias.setVersion(1);
infoConfig.setServerAlias(serverAlias);
configClient.updateTargetConfig(inputVectorConfigURI, infoConfig);
try {
Thread.sleep(100);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
//The data point needs to be reread and written to so that the change takes effect
inputVectorDp = (CiiOldbDataPoint<List<Double>>) oldbClient.getDataPoint(inputVectorUri);
inputVectorDp.writeValue(new ArrayList<>(Arrays.asList(2.0, 3.0, 4.0)));
Note: the in-code configuration change does not alert data points. For the configuration change to take effect in the OLDB storages, the data point must be fetched again from the OLDB and a value needs to be written to it. After that also other instances of data points will read and write to the right storage location.
It is possible to alert all clients using a data point instance of the config change with Config GUI or CLI tool.
If the configuration is changed through the Config GUI [3], there is a “notify oldb” button to send a notification to all data points.
The CLI tool config-oldb-update notifies the clients of data point config changes. It takes one parameter (the config URI under which the data point configuration is saved in the configuration service). The data point configuration URI can be constructed from the data point URI
Example:If the data point URI is cii.oldb:/sampleroot/inputvector, the URI of configuration is cii.config:/remote/oldb/datapoints/sampleroot/inputvector.
./config-oldb-update cii.config://remote/oldb/datapoints/sampleroot/inputvector
7.6. OLDB API LIBRARY¶
This section explains the details of the srv-oldb library. The OLDB API is split between two classes. The CiiOldb and CiiOldbDataPoint.
7.6.1. CiiOldb¶
The core OLDB Client API is the singleton CiiOldb class. The CiiOldb class contains methods for creating retrieving and deleting data points. See Appendix C for a description of CiiOldb API.
7.6.2. CiiOldbDataPoint¶
The other part of the OLDB Client API is the CiiOldbDataPoint class (Figure 3). It represents the data point in the OLDB and allows reading from and writing to it. The CiiOldbDataPoint class includes three main fields:
URI: the unique data point identifier.
CiiOldbDpValue: the value class containing the data point data value, timestamp and quality. It is parameterized with the type of the data value.
CiiOldbMetaData: the metadata class that contains data point type, formula, quality expression and other data type specific information.
The CiiOldbDataPoint is a generic class parameterized with the programming language type of the data value (for Java and C++).
Figure 3: CiiOldbDataPoint class diagram
7.6.2.1. Data Point URI¶
Data point URI is the unique hierarchy identifier of the data point and contains OLDB schema cii.oldb:/ and the path to the data point (e.g. cii.oldb:/root/child/dp1).
Note: To make OLDB hierarchy working as expected, the data point URI should not contain any triple underscore (“___”) character. Because internally all “/” characters are converted to triple underscores, a triple underscores in a level name would result in an additional level.
URI uri = point.getUri();
Note: The OLDB is case insensitive so two URIs differing only in case are considered the same. All URIs are internally stored in lower case.
Note: It is valid that a data point URI path is a sub-path of another data point URI path since the Configuration Service does not hold the URIs in a hierarchy structure. So for example, if there a data point with URI cii.oldb:/some/path/a, a data point with cii.oldb:/some/path/a/b can be created.
7.6.2.2. Data Point Value¶
The CiiOldbDpValue contains three pieces of information that dynamically change during the data point lifetime:
Value: The actual data value of the data point. Data types of values are listed in Appendix A.
Quality: The CiiOldbDpQuality enumeration that represents the quality state of the data point. Possible values are OK, SUSPECT or BAD.
Timestamp: The moment in time when the data point was last modified.
The meanings of the quality state are as follows:
OK: the value of the Property is valid
BAD: the value is not OK and should not be used
SUSPECT: there is a value, but it is not clear if it is reliable. For example, the readout of a sensor is inconsistent with other readings and the sensor might be faulty.
If the data point quality is not OK, data point value cannot be read and written to. Note that the difference between BAD and SUSPECT is just semantic. The OLDB library threats these two states the same. If anything goes wrong with the data point internally (e.g. error in the data point formula) the quality is set to BAD. It is up to the user to set the quality to SUSPECT if appropriate.
The CiiOldbDpValue is a generic class parameterized with the programming language type of the data value (for Java and C++, but not Python because it is not strongly typed).
CiiOldbDpValue<T> value = point.readValue();
7.6.2.3. Metadata¶
The MdOldb is the class containing all the metadata information about the data point. It is not expected that this information would change often but it is possible to manipulate it through the OLDB API (Section 5.4). It is a subtype of MdBase from the Cii Config Service [3] so it contains all the fields contained in the MdBase. The OLDB relevant fields of the MdBase are genType (the type of the data point value) and defaultValue (the default value of the data point). The OLDB specific fields contained in the base class MdOldb are data point formula and qualityExpression.
For other fields in MdBase class consult the [3] document.
The CiiOldbDataPoint class contains a subtype of MdOldb specific to the data point type (Table 6‑1). The user of the CiiOldbDataPoint must know the type of the data point and the metadata class corresponding to it so they can retrieve the metadata object, as seen in the following example:
MdOldbString metadata = point.getMetadata(MdOldbString.class);
If at runtime the type of the data point is not known, the user can determine it by retrieving the metadata as the MdOldb base type, retrieving the generic type from it and casting the metadata in the right class.
MdOldb metadata = point.getMetadata(MdOldb.class);
AttributeType type = metadata.getGenType();
if (type == AttributeType.STRING) {
metadata = (MOldbString) metdata;
}
All the built-in OLDB metadata classes are listed in Table 6‑1. All classes derive from MdOldb (which derives from config MdBase) class which defines basic OLDB fields (quality and formula). The metadata class associated with the data type can be read in Appendix B.
Note that it is not meant to extend the existing metadata classes and define new “types” of data points since the data types defined by Core Integration Infrastructure are covered with existing metadata. It is in theory possible to extend OLDB metadata classes, but it means extending the OLDB design and coding of new extended classes.
Table 6‑1: Built-in metadata classes
Class |
Description |
Specific Fields |
|---|---|---|
MdOldbNumber |
Metadata for all scalar number types. Parameterized with the language mapping of the type |
minLimit (data point value type): the lower limit of the data point value. maxLimit (data point value type): the upper limit of the data point value. units (string): the units of the data point. |
MdOldbString |
Metadata for string data types. Not parameterized. |
allowedValues (list): all the values the data point value is allowed to have. If the list is empty or null, all values are allowed. size (integer): the length of the string value. This is not a limit just informational filed. |
MdOldbArray |
Metadata for all vector data points. Parameterized with the type of the elements. |
|
MdOldbMatrix |
Metadata for all matrix data points. Parameterized with the type of the elements. This is a subtype of MdOldbArray* |
width (integer): The width of the matrix (the number of elements in each matrix row). |
MdOldbBinary |
Metadata for byte array data points. |
*Note: The matrix type data point actually holds the matrix flattened in to a vector. The metadata then holds the width of the matrix (number of elements in each row) so the user can reconstruct the matrix (e.g. if data point value is a vector of length 25 and the width property of the metadata is 5, the data point represents a 5x5 matrix). The length of the vector value must be divisible by the width otherwise the CiiConfigDataLimitsException is thrown.
7.6.2.3.1. Data Point Type¶
The Core Integration Infrastructure defines the types of all data values. All data point values are one of these predefined types. The predefined type effectively defines the type of the data point. Programmatically the data point type is defined in the AttributeType enumeration class (elt-common). See Appendix A for table of types relevant for data point values and their mappings to specific programming language types. The data point type is stored in the data point metadata in the genType field. Data point type cannot change during the lifetime of a data point. Data point with specific URI must be deleted and created again if the type needs to be changed.
AttributeType type = metadata.getGenType();
7.6.2.3.2. Data Point Formula¶
Data point formula is a mathematical expression by which the value of the data point can be calculated. If data point defines a formula, it is a calculated data point. Calculated data points cannot be directly written to (trying to do so will result in the CiiOldbIllegalOperationException). They are handled by the Calculation Service. The data point formula is defined in the data point metadata.
String formula = metadata.getFormula();
Since the Calculation Service uses the Octave language, the data point formula must be a valid Octave expression [6] and must return a value of the data point type. With that satisfied, the formula can contain all Octave language functions, Calculation Service build-in functions and user defined functions.
See [6] for definitions of Octave language functions.
Calculation Service defines its own built-in functions for accessing the data point’s current and previous values and previous and current timestamp:
PREV_VAL()
CURR_VAL()
PREV_TIMESTAMP()
CURR_TIMESTAMP()
See Section 5.3 for how to define custom Octave functions to use in the data point formulas.
Data point formula can contain references to other data points. This makes the data point dependent on other data points. The dependent data point value will be recalculated when one of its dependencies change. A valid reference to another data point is the data point URI enclosed in curly brackets with a preceding dollar sign (e.g. ${…}). The engine accepts either the full URI or only a relative path of the data point.
Example:
${/root/child/device/dp1} + 3 * sqrt(${cii.oldb:/root/child/device/dp2})
Data point formula must not establish cyclic dependencies. Doing that will result in a CiiOldbCyclicDependencyException at the data point creation or metadata update.
Any syntax errors in the data point formula will result in data point quality to be set to BAD by the Calculation Service.
Note: Octave numbers are by definition float values so if a formula of a data point is a constant number and the user expects a different type it should be converted by an Octave conversion function (e.g. int32(100)).
7.6.2.3.3. Data Point Quality Expression¶
The quality expression is an expression which defines the data point quality. The data point quality represents the quality state of the data point and can have values OK, SUSPECT or BAD.
Data point quality evaluation is done by the Calculation Service (Section 3.2.3) and must be a valid Octave expression. The data point quality expression must not return any value but must set the quality of the data point by using the Calculation Service built-in function SETQUAL(quality) with the parameter either OK, SUSPECT or BAD.
The data point quality expression can contain Octave language functions, Calculation Service built-in functions listed above (with the addition of SETQUAL function) and custom defined functions (Section 5.3).
Example:
if (CURR_VAL() > 0) SETQUAL(OK) else SETQUAL(BAD) endif
The data point quality expression must not include references to any other data points. Any syntax error in the quality expression will result in data point quality to be set to BAD by the Calculation Service.
7.7. OLDB CLI¶
This section describes how to use OLDB command line application. The oldb-cli program can be used for read, write and subscribe operations and with data points that have textual outputs. The output of the tool can be parsed and used in other scripts.
7.7.1. Read operations¶
The tool reads the current value of the data point at the given data point URI and displays it on the standard output. Depending on the options used it can also display data point’s quality expression and formula as well. Arrays and matrices are displayed in the standard JSON format. It is also possible to write the value to a file (Section 7.1.3).
The syntax for read operations is:
oldb-cli read <data point URI> [-f <output file>|-e|-vf|-time|-quality|-value]
The -h argument prints help description to standard output.
7.7.1.1. Standard Usage¶
To read a value of a data point, use the client together with an URI that points to the desired data point. If the data type of value is BINARY or its string representation exceeds 500 characters, the value won’t be displayed on the standard output and you will have to use the output file option to retrieve it (Section 7.1.3), otherwise an error will be thrown. To output just a single part of the (timestamp/quality/value) group, the options can be used (-time, -quality, -value).
Example:
$ oldb-cli read cii.oldb:///glob/root/child/testintdp
Timestamp: 2020-01-13T14:49:54.702Z
Quality: OK
Value: 13
7.7.1.2. Retrieving Value Formula and Quality Expression¶
To also retrieve value formula or quality expression of a data point use the options -vf (for value formula) and/or -e (for quality expression).
Example:
$ oldb-cli read cii.oldb:/glob/root/child/testintdp -vf -e
Quality expression: if (CURR_VAL() > 0) SETQUAL(OK) else SETQUAL(BAD) endif
Value formula: ${/root/child/device/DP1} + 3 * ${/root/child/device/DP2}
Timestamp: 2020-01-13T14:49:54.702Z
Quality: OK
Value: 13
7.7.1.3. Saving value to a File¶
To save the data point’s value to a file use the -f option, followed by a path to file to write to. Unless the datatype of value is BINARY or its string representation exceeds 500 characters, the value will still get displayed on standard output as well.
Example:
Saving to file:
$ oldb-cli read cii.oldb:/glob/root/child/teststringdp -f testStringDpValue
Timestamp: 2020-01-07T19:09:13.117Z
Quality: OK
Value: ABCDEF
Reading the value from file:
$ cat testStringDpValue
ABCDEF
7.7.2. Write operation¶
The tool gets the data point at the given URI and writes a new value to it. Data point’s quality, timestamp, formula and quality expression can all be changed through the tool with the right options. Arrays and matrices must be given in the standard JSON format. Values can be given through a file too.
The syntax for writing is:
oldb-cli write <data point uri> <value> [-f] [-h] [-q <quality>] [-e <quality expression>] [-t <timestamp>] [-vf]
The -h argument prints help description to standard output.
7.7.2.1. Standard Usage¶
To write a value to data point, use the client together with an URI that points to the desired data point and value to be written. If the datatype of the value to be written is BINARY, the value must be given through the -f option (Section 7.2.5).
Example:
$ oldb-cli write cii.oldb:/glob/root/child/testintdp 1234
Timestamp: 2020-01-14T15:34:52.392Z
Quality: OK
Value: 1234
7.7.2.2. Setting Quality and Timestamp¶
When setting a data point’s value, the quality and timestamp can also be set.
To set the quality use the -q option followed by the desired quality (OK/SUSPECT/BAD).
Example:
$ oldb-cli write cii.oldb:/glob/root/child/testintdp 1234 -q BAD
Timestamp: 2020-01-14T15:34:52.392Z
Quality: OK
Value: 1234
To set a timestamp to the data point use the -t option, followed by the timestamp in ISO 8601 extended date-time format.
Example:
$ oldb-cli write cii.oldb:/glob/root/child/testintdp 1234 -t 2020-01-07T19:09:13.117Z
Timestamp: 2020-01-07T19:09:13.117Z
Quality: OK
Value: 1234
Note: all three data point properties (value, quality, timestamp) can be set in one operation using all the above options.
7.7.2.3. Setting Formula¶
If you want to set a formula to a data point, replace the value argument with the value formula, and use the -vf option. The functionality will fist read the metadata instance name that belongs to the given data point and then writes the formula to the metadata instance.
Note: Formula is a part of the data point metadata instance. As one metadata instance can belong to many datapoints, this change can affect multiple data points.
Example:
oldb-cli write cii.oldb:/glob/root/child/testintdp3 '${cii.oldb:/glob/root/child/testintdp2} + 3 * sqrt(${cii.oldb:/glob/root/child/testintdp})' -vf
7.7.2.4. Setting Quality Expression¶
If instead of setting a fixed quality, you want to set a quality expression, you can do so by using the -e option followed by the desired quality expression. In this case, the value argument can be omitted.
Example:
oldb-cli write cii.oldb:/glob/root/child/testintdp2 -e 'if (CURR_VAL() > 0) SETQUAL(OK) else SETQUAL(BAD) endif'
7.7.2.5. Writing Value from a File¶
To write a value from a file, replace the value argument with a path to the file containing the value and use the -f option.
Example:
Writing value from a file:
$ oldb-cli write cii.oldb:/glob/root/child/testintdp /home/esodev/values/testIntDpValue -f
Timestamp: 2020-01-14T15:40:34.783Z
Quality: OK
Value: 25
7.7.2.6. Writing Value directly from the formula.¶
Value to specific data point can be written directly from the expression formula. To use the formula, the value must start with the “=” character.
Example:
Writing values from a formula:
$ oldb-cli write cii.oldb:/glob/root/child/testdoubledp '=e(3)'
Timestamp: 2020-07-17T08:48:22.399Z
Quality: OK
Value: 20.085536923187668
$ oldb-cli write cii.oldb:/glob/root/child/testintdp '=${cii.oldb:/glob/root/child/testintdp2} + ${cii.oldb:/glob/root/child/testintdp3}'
Timestamp: 2020-07-17T08:50:00.445Z
Quality: OK
Value: 4435
7.7.3. Subscribe operations¶
The tool retrieves the data point at the given URI, subscribes to it and displays value changes to the standard output until user interrupts it or the data point is deleted. Arrays and matrices are displayed in the standard JSON format.
The syntax for subscribe functionality is:
$ oldb-cli subscribe <data point uri> [-f <output file prefix>] [-h]
7.7.3.1. Standard Usage¶
To subscribe to a data point, use the client together with an URI that points to the desired data point. If the data type of value is BINARY or its string representation exceeds 500 characters, the value won’t be displayed on the standard output and you will have to use the output file prefix option (Section 7.3.2) to have changes written to files.
Example:
$ oldb-cli subscribe cii.oldb:/glob/root/child/testintdp
Subscribed to cii.oldb:/glob/root/child/testintdp
To stop the subscription interrupt the program with Ctrl-C.
Timestamp: 2020-01-10T13:04:52.788Z
Quality: OK
Value: 30
Timestamp: 2020-01-10T13:05:11.474Z
Quality: OK
Value: 25
Timestamp: 2020-01-10T13:05:19.885Z
Quality: OK
Value: 20
Timestamp: 2020-01-10T13:05:27.098Z
Quality: OK
Value: 15
Timestamp: 2020-01-10T13:05:34.134Z
Quality: OK
Value: 10
(base) [esodev@localhost values]$
7.7.3.2. Output to Files¶
To write the received values to files, use the -f option with a file name prefix. The files will then be placed in the current directory and named as <prefix_timestamp> (for each value update a new file is created).
Example:
$ oldb-cli subscribe cii.oldb:/glob/root/child/testintdp -f testIntDpValues
Subscribed to cii.oldb:/glob/root/child/testintdp
To stop the subscription interrupt the program with Ctrl-C.
Timestamp: 2020-01-10T13:04:52.788Z
Quality: OK
Value: 30
Timestamp: 2020-01-10T13:05:11.474Z
Quality: OK
Value: 25
Timestamp: 2020-01-10T13:05:19.885Z
Quality: OK
Value: 20
Timestamp: 2020-01-10T13:05:27.098Z
Quality: OK
Value: 15
Timestamp: 2020-01-10T13:05:34.134Z
Quality: OK
Value: 10
^C(base) [esodev@localhost values]$ ls
testIntDpValues_2020-01-10T13:04:52.788Z testIntDpValues_2020-01-10T13:05:11.474Z testIntDpValues_2020-01-10T13:05:19.885Z testIntDpValues_2020-01-10T13:05:27.098Z testIntDpValues_2020-01-10T13:05:34.134Z
The values are stored inside the files.
Example:
[esodev@localhost values]$ cat testIntDpValues_2020-01-10T13:05:19.885Z
20
[esodev@localhost values]$
7.8. GUI¶
This section describes the components of OLDB GUI. OLDB GUI has functionalities to read, write, create, delete and update data points. The GUI can also be used to read metadata of data points, but changing of metadata is not supported. Config GUI [3] must be used to change the metadata instances, which the OLDB uses.
7.8.1. General¶
OLDB GUI is a graphical application that can be used for managing the data points and displaying their values. The data points are retrieved from the remote OLDB service.
After the OLDB GUI application has been successfully installed (See [2] on how to install OLDB GUI application), it can be executed by issuing the following command from the shell:
oldb-gui [path_to_application_config_file] [path_to_log_config_file]
The oldb_gui executable takes 2 optional arguments:
path_to_application_config_file: a path to the file that contains the settings for the OLDB GUI application. An example value for the attribute: ../config/app-config.ini. A detailed description of the application configuration file can be found in section 8.2.
path_to_log_config_file: a path to the file that contains the settings for the log4cplus logging library. If it is not provided, the basic log4cplus configuration is used. Usually, the configuration file for the log4cplus library is named log4cplus.properties. Example value for the attribute: ../log4cplus.properties.
7.8.2. Configuration File¶
The configuration file for the OLDB GUI application contains the information needed to connect to the remote configuration service (i.e. the data source). The configuration file contains 3 sections:
DataSource: this section contains all the data needed to connect to the primary configuration service. This service is used for retrieving the metadata about every selected data point. The hostname attribute provides the address of the service (in a form of an IP address or a host name) and the port attribute provides a port on which the service listens for incoming connections.
DataSourceBackup: this section contains the same parameters as DataSource, only the values are set to the ones of the backup configuration service. In case you don’t have a backup configuration service, insert the values of the primary configuration service into this section.
MatrixEdit: this section determines the max number of rows (maxHeight) and columns (maxWidth) that a given vector or matrix can have in order for its data to be displayed and edited directly in the OLDB GUI (for vectors, the maxWidth defines the size limit). Vectors and matrices that are larger than the MatrixEdit values can only be downloaded to the local disk and uploaded from the local disk.
If “path_to_application_config_file” is not provided to the application, default values will be assigned to respective parameters:
in DataSource section, the hostname attribute will get the ciiconfservicehost value assigned and the port attribute will get the 9116 value assigned,
in DataSourceBackup section, the hostname attribute will get the ciiconfservicehost2 value assigned and the port attribute will get the 9116 value assigned,
in MatrixEdit section, the maxWidth and MaxHeight attributes will both get value of 100.
Note: If “path_to_application_config_file” is not provided, both ciiconfservicehost and ciiconfservicehost2 must be set in order to run the GUI. These hostnames must be set in the /etc/hosts file.
7.8.3. Functionality¶
7.8.3.1. Main Window¶
The main window is displayed when we execute the OLDB GUI. The application screenshot is shown in Figure 8‑1. As we can see from the figure, the functionality of the OLDB GUI is related to the management of data points.
Figure 8‑1: Main window
The GUI is divided into 2 parts. The left part displays the list of folders and data points retrieved from the remote service, search bar and the buttons for adding and deleting data points. The user can select the folders and data points by clicking them.
The right part is used for displaying the data about the selected data point: its value and metadata.
7.8.3.2. Menu¶
The OLDB GUI application only contains two menu entries, the Subscriptions (Figure 8‑2) and Options. The Subscription menu entry contains the 3 items for managing data point subscriptions (i.e. the windows for continuously displaying the latest data point values):
Load Subscriptions: loads the list of data point subscriptions from a file on disk (through file explorer) and opens them in windows in saved positions. All subscriptions are started (the value updates will be visible). If a subscription window is already displayed for a data point with a given URI, a new subscription window won’t be displayed once more.
Save Subscriptions: saves the list of currently displayed data point subscription windows (and their position on the screen) into a CSV file on disk. A file explorer window is opened and the user can enter the file name and location.
Close All Subscriptions: closes all the data point subscription windows that are currently displayed (unsubscribes from the data points).
Figure 8‑2: Application menu
The Options menu contains a checkbox item “write enabled”. This checkbox must be checked to enable write operations on data points. If this checkbox is not checked, writing to data points will not be possible. It is not checked by default.
7.8.4. Hierarchy view¶
The hierarchical view displays the hierarchical structure of folders and data points. It enables the user to do the following actions:
Browse and view the data points that are stored in the remote service.
Add new data points.
Delete existing data points.
Trigger the loading of data point values.
Selecting Folder
The user selects a folder by clicking the inner node in the data point hierarchy tree (Figure 8‑3). If the leaf tree node is clicked, the data point is selected. The folders are created automatically, based on the data points’ URIs.
When a folder is selected, all the data in the data point frame is cleared. The folders don’t have any values or metadata, they are only used for organizing data points. When a folder is selected the user can create a new data point by clicking on the Add button below the hierarchy tree.
Figure 8‑3: Selecting folder
Selecting Data Point
The user selects a data point by clicking the leaf node in the data point hierarchy tree (Figure 8‑4). The data points are organized in the tree structure based on their URIs.
For the easier searching of data points, a search filter bar is located at the top of the hierarchy tree. The user can input a part of the URI and when he/she hits the Enter key, all the data points whose name doesn’t contain the string input are filtered out (the string comparison is case insensitive). In order to display all the data points again, the user needs to clear the filtering text and hit the Enter key again.
When a configuration is selected, the following data is loaded in the data point frame of the GUI:
data point value, quality and timestamp: the most recent value, quality and timestamp of the selected data point.
the list of data point’s metadata attributes and their values.
Figure 8‑4: Data point selected
Note that it is not possible to change data point metadata through OLDB GUI application. The user should use the Config GUI application to change the metadata.
The data point URI is at the moment not showed anywhere in the data point frame. The user can reconstruct the data point URI from the tree path.
Adding a New Data Point
The adding of new data point is initiated by clicking the Add button below the hierarchy tree. The button is only enabled if a folder is selected.
When the Add button is clicked, a dialog for inserting a data point is displayed (Figure 8‑5). The dialog contains the following fields:
URI prefix: a read-only field that contains the URI prefix
URI suffix: the input field where the user can input the URI suffix or the name of the data point. The final URI of the configuration is formed by concatenating the URI prefix and suffix.
Metadata instance: here, the user selects the metadata instance for the newly created data point.
After the user clicks the OK button, the dialog is closed and a new data point is created on the remote server.
Figure 8‑5: Adding new data point
Deleting an Existing Data Point
The deleting of existing data point is initiated by clicking the Delete button below the hierarchy tree. The button is only enabled if a data point is selected.
When the Delete button is clicked, a confirmation dialog is displayed where the user can confirm or cancel the deletion. On user confirmation, the data point is deleted on the remote server.
Figure 8‑6: Data point deletion confirmation dialog
Data Point Value and Metadata Frame
The data point value and metadata frame occupy the right side of the application window. It displays the value and all the metadata values for the selected data point. It enables the user to do the following actions:
Refresh data point values.
Update data point values.
Subscribe to data point value changes.
Manually Refreshing Data Point Value
The manual refreshing of the value for the selected data point is triggered by clicking the Read button just below the area for displaying the data point value. After doing this, a request is sent to a remote server, the most recent data point value is retrieved and displayed in the GUI.
Setting the Data Point Value
There are 3 groups of data point values:
primitive values: the setting of the primitive value for the data point is performed using the input data dialog (Figure 8‑7) that is displayed when clicking the modify button next to the text field for displaying the data point value. The dialog contains a simple input field where the user can input the data value. After the value is entered and the dialog is closed (by clicking OK), the Write button must be clicked for value to be written to the OLDB storage.
vector & matrix values: they are displayed in a tabular form (Figure 8‑8). The table values are set by clicking different table cells. After doing this, the same dialog displays as with the primitive values. Here, the user inputs the value for the table cell. In order for the modified values to be sent to the remote service, the Write button needs to be clicked.
binary data values: the binary data values don’t get displayed in the OLDB GUI. It must be stored to or loaded from the local disk.
Whenever the user modifies the data point value in the GUI the timestamp field updates to “now”, so that when Write button is clicked the timestamp written to the OLDB is of the last change. The user can also modify the timestamp field in the same manner than the value.
Figure 8‑7: Data input dialog
Figure 8‑8: Vector & matrix data point value
The vector & matrix values as well as binary data values can also be loaded from a local disk and be stored to a local disk:
the loading from a local disk can be started by clicking the Upload button. In the dialog that gets shown, the user selects the file on local disk, from which the data point values will be loaded. After the file has been loaded it can be sent to the remote service by clicking the Write button.
the storing to the local disk can be started by clicking the Download button. In the dialog that gets shown, the user selects the file on local disk, to which the data point values will be stored.
For the vector & matrix data the user can also change the number of rows and columns. The row-and-column-setting dialog (Figure 8‑10) is displayed by right-clicking the table and selecting the Resize table item from the context menu (Figure 8‑9). If the matrix size is made smaller, any values already entered will be deleted for the removed columns or rows. If the matrix is made larger, the additional cells are empty.
Figure 8‑9: “Resize table” context menu
Figure 8‑10: Dialog for setting the number of rows and columns.
Subscribing to Data Point Value Changes
Subscribing to the value changes of the selected data point is performed by clicking the Subscribe button that is located under the widget for displaying data point value. At this point the new window dialog is displayed that displays the most recent value of the given data point. Whenever the data point value changes on the server it is automatically updated in the subscription dialog. Multiple subscription dialogs can be displayed at the same time, meaning that the user can monitor the values of multiple data points. For a given data point, however, only a single subscription window can be displayed, having duplicate subscription windows for the same data point is not possible.
The user can organize the subscription windows in any way they like, they can even store the position of the subscription windows into a file. Later, the user can use the file to restore the windows. This enables the user to have a pre-defined sets of data points that they can monitor (see section 8.3.2 on how this is done).
Figure 8‑11: Arrangement of multiple subscription windows on the screen
7.9. Data Point Value Type to Language Types Mapping¶
Below is the mapping table between data point types and data types of all programming languages that the OLDB can use.
Table 8‑1: Data point types mapping table
ELT CII BASIC TYPE |
Java |
C++ |
Python |
|
|---|---|---|---|---|
INT8 |
Byte |
std::int8_t |
int |
|
INT16 |
Short |
std::int16_t |
int |
|
INT32 |
Integer |
std::int32_t |
int |
|
INT64 |
Long |
std::int64_t |
int |
|
UINT8 |
Byte |
std::uint8_t |
int |
|
UINT16 |
Short |
std::uint16_t |
int |
|
UINT32 |
Integer |
std::uint32_t |
int |
|
UINT64 |
Long |
std::uint64_t |
int |
|
SINGLE |
Float |
float |
float |
|
DOUBLE |
Double |
double |
float |
|
BOOLEAN |
Boolean |
bool |
bool |
|
STRING |
String |
std::string |
str |
|
BINARY |
java.nio.ByteBuffer |
std::vector<std::uint8_t> |
bytes |
|
VECTOR_INT32 |
List<Integer> |
std::vector<std::int32_t> |
list |
|
VECTOR_SINGLE |
List<Float> |
std::vector<float> |
list |
|
VECTOR_DOUBLE |
List<Double> |
std::vector<double> |
list |
|
VECTOR_STRING |
List<String> |
std::vector<std::string> |
List |
|
MATRIX2D_INT32 |
List<Integer> |
std::vector<std::int32_t> |
list |
|
MATRIX2D_SINGLE |
List<Float> |
std::vector<float> |
list |
|
MATRIX2D_DOUBLE |
List<Double> |
std::vector<double> |
List |
7.10. Default Metadata Instance Names¶
ELT CII BASIC TYPE |
Metadata Instance Name |
Metadata Class name |
|---|---|---|
INT8 |
OldbInt8Std |
MdOldbNumber |
INT16 |
OldbInt16Std |
MdOldbNumber |
INT32 |
OldbInt32Std |
MdOldbNumber |
INT64 |
OldbInt64Std |
MdOldbNumber |
UINT8 |
OldbUInt8Std |
MdOldbNumber |
UINT16 |
OldbUInt16Std |
MdOldbNumber |
UINT32 |
OldbUInt32Std |
MdOldbNumber |
UINT64 |
OldbUInt64Std |
MdOldbNumber |
SINGLE |
OldbSingleStd |
MdOldbNumber |
DOUBLE |
OldbDoubleStd |
MdOldbNumber |
STRING |
OldbStringStd |
MdOldbString |
BINARY |
OldbBinaryStd |
MdOldbBinary |
VECTOR_INT32 |
OldbInt32ArrayStd |
MdOldbArray |
VECTOR_SINGLE |
OldbFloatArrayStd |
MdOldbArray |
VECTOR_DOUBLE |
OldbDoubleArrayStd |
MdOldbArray |
VECTOR_STRING |
OldbStringArrayStd |
MdOldbArray |
MATRIX2D_INT32 |
OldbInt32MatrixStd |
MdOldbMatrix |
MATRIX2D_SINGLE |
OldbFloatMatrixStd |
MdOldbMatrix |
MATRIX2D_DOUBLE |
OldbDoubleMatrixStd |
MdOldbMatrix |
7.11. OLDB API LISTING¶
This a listing of OLDB API methods with short description for both CiiOldb and CiiOldbDataPoint classes. The method signatures are written in Java. The signatures are equal (with some exceptions in some of createDataPoint methods) in all three languages except for the native data types of each language (e.g. for string, double, integer). For data type mapping between languages see Table 8‑1. For URI variables java uses java.net.URI, C++ uses <mal/utility/Uri.hpp> and Python uses elt.config.Uri.
Note that all OLDB API methods are synchronous (blocking) methods.
For detail explanation of exceptions thrown by these methods, consult the oldb-client source code.
7.11.1. CiiOldb¶
static CiiOldb getInstance() |
|---|
The method returns the singleton instance implementation of the OLDB client CiiOldb. |
CiiOldbDataPoint<?> createDataPoint(URI uri, String metadataInstanceName, String metadataClassName) |
|---|
The method creates a data point at the defined URI with defined metadata. The data point data type and other restrictions are defined by its metadata which must already exist. Once the data point is created its data type cannot be changed. Note: This method signature is different in C++ and Python where the String metadataClassName parameter is omitted. This is because of Java language specifics. |
CiiOldbDataPoint<String> createDataPointByValue(URI uri, String value) |
|---|
Creates The method creates a data point of type STRING at the defined URI with defined value. It uses the default OLDB string metadata with metadata instance name “OldbStringStd”. The metadata must already exist. |
CiiOldbDataPoint<Byte> createDataPointByValue(URI uri, Byte value) |
|---|
Creates The method creates a data point of type INT8 at the defined URI with defined value. It uses the default OLDB string metadata with metadata instance name “OldbInt32Std”. The metadata must already exist. |
CiiOldbDataPoint<Short> createDataPointByValue(URI uri, Short value) |
|---|
Creates The method creates a data point of type INT16 at the defined URI with defined value. It uses the default OLDB string metadata with metadata instance name “OldbInt16Std”. The metadata must already exist. |
CiiOldbDataPoint<Integer> createDataPointByValue(URI uri, Integer value) |
|---|
Creates The method creates a data point of type INT32 at the defined URI with defined value. It uses the default OLDB string metadata with metadata instance name “OldbInt32Std”. The metadata must already exist. |
CiiOldbDataPoint<Float> createDataPointByValue(URI uri, Float value) |
|---|
Creates The method creates a data point of type SINGLE at the defined URI with defined value. It uses the default OLDB string metadata with metadata instance name “OldbSingleStd”. The metadata must already exist. |
CiiOldbDataPoint<Double> createDataPointByValue(URI uri, Short value) |
|---|
Creates The method creates a data point of type DOUBLE at the defined URI with defined value. It uses the default OLDB string metadata with metadata instance name “OldbDoubleStd”. The metadata must already exist. |
CiiOldbDataPoint<ByteBuffer> createDataPointByValue(URI uri, ByteBuffer value) |
|---|
Creates The method creates a data point of type BINARY at the defined URI with defined value. It uses the default OLDB string metadata with metadata instance name “OldbBinaryStd”. The metadata must already exist. |
CiiOldbDataPoint<List<T>> createDataPointByValue(URI uri, List<T> value, Class<T> clazz, boolean isMatrix) |
|---|
Creates The method creates a data point of matrix or vector type at the defined URI with defined value. It uses the default OLDB string metadata with metadata instance name “Oldb<element-type>ArrayStd”. The metadata must already exist. The <element-type> is “Int32”, “Single”, “Double” or “String” (vector only). The isMatrix parameter defines whether a matrix or vector data point is created. The clazz parameter defines the class of the elements of matrix/vector. Note: The Class<T> clazz parameter is only needed in Java version of the method because of language specifics. |
CiiOldbDataPoint<List<T>> createDataPointByValue(URI uri, List<T> value, Class<T> clazz) |
|---|
Creates The method creates a data point of vector type at the defined URI with defined value. It uses the default OLDB string metadata with metadata instance name “MdO<element-type>ArrayStd”. The metadata must already exist. The <element-type> is “Int32”, “Single”, “Double” or “String”. The clazz parameter defines the class of the elements of vector. Note: The Class<T> clazz parameter is only needed in Java version of the method because of language specifics. |
void deleteDataPoint(URI uri) |
|---|
The method deletes a data point referenced by the URI. |
Map<URI, Boolean> getChildren(URI uri) |
||||||
|---|---|---|---|---|---|---|
Finds all direct children of the given URI. Returns the map of all found URI with boolean value indicating whether the URI represents a data point (true if data point, false if only folder). Since OLDB allows to create a data point with an URI that is a sub-path of another URI, there can be both folder and data point URIs present. In that case the boolean value is true. Note: Because OLDB is case insensitive all URIs returned will be in lower case. Example Consider the following OLDB URIs: cii.oldb:/root/child/device/dp1
cii.oldb:/root/child/dp2
cii.oldb:/root/child/dp2/dp3
A call to the getChildren method with URI = cii.oldb:/root/child will return the map in Table 8‑2. Table 8‑2: Example of getChildren call result
Note that the path cii.oldb:/root/child/dp2 hierarchically represents both data point and a folder. In this case the getChildren returns TRUE for this path. This means that a TRUE for an URI does not mean that the URI doesn’t also represent a folder. |
CiiOldbDataPoint<?> getDataPoint(URI uri) |
|---|
Returns a data point at a specified URI. Note that the caller of the getDataPoint method must know the data point type to cast the received object to the right type (only required for the strongly typed languages like Java and C++). The type of the data point can be obtained from metadata (Section 6.2.3). Note: this method does not provide transactional safety. There is no guarantee that the data point is not deleted the moment after it was retrieved (data point object of a deleted data point is not usable). |
List<CiiOldbDataPoint<?>> getDataPoints(List<URI> uris) |
|---|
The method returns a list of data points matching the specified URIs. The URIs in the list can be written as a simple glob pattern using three special characters. * for any number of characters restricted to the URI hierarchy level ** for any number of characters spanning multiple levels ^ for exactly one character Example: If we have a simple OLDB that contains data points with the following URIs: cii.oldb:/root/child/device/dp1 cii.oldb:/ root/child/device/dp21 cii.oldb:/ root/child/device/dp23 cii.oldb:/ root/child1/motor/dp2 Then calls to getDataPoints with these glob patterns would result in following data points: cii.oldb:/root/child/device/dp2* -> dp21, dp23 cii.oldb:/root/child/** -> dp1, dp21, dp23 cii.oldb:/root/child/device/dp^ -> dp1 Note: this method does not provide transactional safety. There is no guarantee that a data point is not deleted the moment after it was retrieved (data point object of a deleted data point is not usable). |
List<CiiOldbDataPoint<T>> getDataPoints(List<URI> uris, AttributeType dataType, T minValue, T maxValue) |
|---|
The method returns a list of data points matching the specified URIs, filtered by type and min and max limits. As with the getDataPoints above, this method supports the simple glob patterns. Note: this method does not provide transactional safety. There is no guarantee that a data point is not deleted the moment after it was retrieved (data point object of a deleted data point is not usable). |
void setWriteEnabled(boolean writeEnabled) |
|---|
Enables or disables all writes to data points. |
void close(); |
|---|
Closes connections and cleans up resources. |
Below are the Java and C++ CiiOldb API listings. The Python client uses Python bindings and has therefore the same API as the C++ client.
Listing 8‑1: Java CiiOldb API
static CiiOldb getInstance() throws CiiOldbInitializationError {
return CiiOldbImp.getInstance();
}
CiiOldbDataPoint<?> createDataPoint(URI uri, String metadataInstanceName, String metadataClassName) throws CiiOldbDpExistsException, CiiOldbCyclicDependencyException, CiiOldbExpressionException,
CiiOldbDpConstraintsException, CiiOldbUndefinedMetadataException, CiiOldbStorageError, CiiOldbConfigError, CiiOldbURIException;
CiiOldbDataPoint<String> createDataPointByValue(URI uri, String value) throws CiiOldbDpExistsException, CiiOldbCyclicDependencyException, CiiOldbExpressionException, CiiOldbDpConstraintsException,
CiiOldbUndefinedMetadataException, CiiOldbStorageError, CiiOldbConfigError, CiiOldbURIException;
CiiOldbDataPoint<Byte> createDataPointByValue(URI uri, Byte value) throws CiiOldbDpExistsException, CiiOldbCyclicDependencyException, CiiOldbExpressionException, CiiOldbDpConstraintsException,
CiiOldbUndefinedMetadataException, CiiOldbStorageError, CiiOldbConfigError, CiiOldbURIException;
CiiOldbDataPoint<Short> createDataPointByValue(URI uri, Short value) throws CiiOldbDpExistsException, CiiOldbCyclicDependencyException, CiiOldbExpressionException, CiiOldbDpConstraintsException,
CiiOldbUndefinedMetadataException, CiiOldbStorageError, CiiOldbConfigError, CiiOldbURIException;
CiiOldbDataPoint<Integer> createDataPointByValue(URI uri, Integer value) throws CiiOldbDpExistsException, CiiOldbCyclicDependencyException, CiiOldbExpressionException, CiiOldbDpConstraintsException,
CiiOldbUndefinedMetadataException, CiiOldbStorageError, CiiOldbConfigError, CiiOldbURIException;
CiiOldbDataPoint<Long> createDataPointByValue(URI uri, Long value) throws CiiOldbDpExistsException, CiiOldbCyclicDependencyException, CiiOldbExpressionException, CiiOldbDpConstraintsException,
CiiOldbUndefinedMetadataException, CiiOldbStorageError, CiiOldbConfigError, CiiOldbURIException;
CiiOldbDataPoint<Float> createDataPointByValue(URI uri, Float value) throws CiiOldbDpExistsException, CiiOldbCyclicDependencyException, CiiOldbExpressionException, CiiOldbDpConstraintsException,
CiiOldbUndefinedMetadataException, CiiOldbStorageError, CiiOldbConfigError, CiiOldbURIException;
CiiOldbDataPoint<Double> createDataPointByValue(URI uri, Double value) throws CiiOldbDpExistsException, CiiOldbCyclicDependencyException, CiiOldbExpressionException, CiiOldbDpConstraintsException,
CiiOldbUndefinedMetadataException, CiiOldbStorageError, CiiOldbConfigError, CiiOldbURIException;
CiiOldbDataPoint<ByteBuffer> createDataPointByValue(URI uri, ByteBuffer value) throws CiiOldbDpExistsException, CiiOldbCyclicDependencyException, CiiOldbExpressionException, CiiOldbDpConstraintsException,
CiiOldbUndefinedMetadataException, CiiOldbStorageError, CiiOldbConfigError, CiiOldbURIException;
<T> CiiOldbDataPoint<List<T>> createDataPointByValue(URI uri, List<T> value, Class<T> clazz, boolean isMatrix) throws CiiOldbDpExistsException, CiiOldbCyclicDependencyException, CiiOldbExpressionException,
CiiOldbDpConstraintsException, CiiOldbUndefinedMetadataException, CiiOldbStorageError, CiiOldbConfigError, CiiOldbURIException, CiiInvalidTypeException;
<T> CiiOldbDataPoint<List<T>> createDataPointByValue(URI uri, List<T> value, Class<T> clazz) throws CiiOldbDpExistsException, CiiOldbCyclicDependencyException, CiiOldbExpressionException,
CiiOldbDpConstraintsException, CiiOldbUndefinedMetadataException, CiiOldbStorageError, CiiOldbConfigError, CiiOldbURIException, CiiInvalidTypeException;
void deleteDataPoint(URI uri) throws CiiOldbStorageError, CiiOldbURIException, CiiOldbConfigError;
Map<URI, Boolean> getChildren(URI uri) throws CiiOldbDpUndefinedException, CiiOldbURIException;
CiiOldbDataPoint<?> getDataPoint(URI uri) throws CiiOldbURIException, CiiOldbDpUndefinedException, CiiOldbUndefinedMetadataException, CiiOldbStorageError;
List<CiiOldbDataPoint<?>> getDataPoints(List<URI> uris)
throws CiiOldbUndefinedMetadataException;
<T> List<CiiOldbDataPoint<T>> getDataPoints(List<URI> uris, AttributeType dataType, T minValue, T maxValue) throws CiiOldbDpConstraintsException, CiiOldbDpUndefinedException, CiiOldbDpQualityException,
CiiOldbUndefinedMetadataException;
void setWriteEnabled(boolean writeEnabled);
void close();
Listing 8‑2: C++ CiiOldb API
std::shared_ptr<CiiOldbTypedDataBase> CreateDataPoint(const ::elt::mal::Uri& uri, const std::string& meta_data_instance_name);
template<typename T>
std::shared_ptr<CiiOldbDataPoint<T>> CreateDataPointByValue(
const ::elt::mal::Uri& uri, const T& value);
template<typename T>
std::shared_ptr<CiiOldbDataPoint<std::vector<T>>> CreateDataPointByValue(
const ::elt::mal::Uri& uri, const std::vector<T>& value, bool is_matrix);
std::shared_ptr<CiiOldbTypedDataBase> GetDataPoint(const ::elt::mal::Uri& uri) const;
template<typename T> std::shared_ptr<CiiOldbDataPoint<T>> GetDataPoint(
const ::elt::mal::Uri& uri, ::elt::common::CiiBasicDataType data_type) const;
std::vector<std::shared_ptr<CiiOldbTypedDataBase>> GetDataPoints(
const std::vector<::elt::mal::Uri>& uris) const;
template<typename T> std::vector<std::shared_ptr<CiiOldbDataPoint<T>>> GetDataPoints(const std::vector<::elt::mal::Uri>& uris,
::elt::common::CiiBasicDataType data_type, T min_value, T max_value) const;
std::map<::elt::mal::Uri, bool> GetChildren(const ::elt::mal::Uri& uri) const;
static void SetWriteEnabled(bool enabled) noexcept;
static bool IsWriteEnabled() noexcept;
void DeleteDataPoint(const ::elt::mal::Uri& uri);
7.11.2. CiiOldbDataPoint¶
CiiOldbDpValue<T> readValue() |
|---|
Returns a copy of this Data Point value. Every call to readValue retrieves an up to date value from the OLDB, but the CiiOldbDpValue object is a snapshot at the time of retrieval and does not update automatically. The user should call readValue to get the updated value. |
void writeValue(T value) |
|---|
Writes the value to the data point. The quality is unchanged and timestamp is now. |
writeValue(T value, Instant timestamp) |
|---|
Writes the value and the timestamp to the data point. The quality is unchanged. |
void writeValue(T value, Instant timestamp, CiiOldbDpQuality quality) |
|---|
Writes all three fields of the data point value to the data point. |
void subscribe(CiiOldbDpSubscription callback) |
|---|
Subscribes a callback to the data point to listen to value changes. |
void unsubscribe(CiiOldbDpSubscription callback) |
|---|
Unsubscribes a callback from the data point. |
void setQuality(CiiOldbDpQuality quality) |
|---|
Sets the data point quality. |
<U extends MdOldb<T>> U getMetadata(Class<U> clazz) |
|---|
Returns the metadata of the data point. Caller must provide the metadata class (extends MdOldb). |
<U extends MdOldb> void setMetadata(String metadataInstanceName) |
|---|
Updates the data point metadata with the metadata which instance name is provided. The changes made in the metadata must be saved to config service otherwise this call will have no effect. If the instance name is different from the one this data point already has (new metadata instance name has been assigned to the data point), the new metadata must respect the data point type. |
Below are the Java and C++ CiiOldbDataPoint API listings. The Python client uses Python bindings and has therefore the same API as the C++ client.
Listing 8‑3: Java CiiOldbDatapoint API
public abstract CiiOldbDpValue<T> readValue() throws CiiOldbDpUndefinedException, CiiOldbDeletedException, CiiOldbStorageError;
public abstract void writeValue(T value) throws CiiOldbDpQualityException, CiiOldbDpConstraintsException, CiiOldbWriteDisabledException, CiiOldbDeletedException, CiiOldbStorageError;
public abstract void writeValue(T value, Instant timestamp) throws CiiOldbDpConstraintsException, CiiOldbWriteDisabledException, CiiOldbDpQualityException, CiiOldbDeletedException, CiiOldbStorageError;
public abstract void writeValue(T value, Instant timestamp, CiiOldbDpQuality quality) throws CiiOldbWriteDisabledException, CiiOldbIllegalOperationException, CiiOldbDeletedException,
CiiOldbStorageError, CiiOldbDpConstraintsException;
public abstract void subscribe(CiiOldbDpSubscription callback) throws CiiOldbDeletedException;
public abstract void unsubscribe(CiiOldbDpSubscription callback) throws CiiOldbDeletedException;
public abstract void setQuality(CiiOldbDpQuality quality) throws CiiOldbDeletedException, CiiOldbStorageError, CiiOldbWriteDisabledException, CiiOldbIllegalOperationException;
public <U extends MdOldb<T>> U getMetadata(Class<U> clazz);
public abstract <U extends MdOldb> void setMetadata(String metadataInstanceName)
throws CiiOldbInvalidTypeException, CiiOldbCyclicDependencyException, CiiOldbExpressionException,CiiOldbUndefinedMetadataException, CiiOldbInitializationError, CiiOldbDeletedException, CiiOldbURIException;
public URI getUri();
Listing 8‑4: C++ CiiOldbDataPoint API
std::shared_ptr<CiiOldbDpValue<T>> ReadValue(bool check_bad_quality = true);
void WriteValue(const T& value, int64_t timestamp = CiiOldbUtil::Now());
void WriteValue(const T& value, int64_t timestamp, CiiOldbDpQuality quality,
const std::string *overridden_server_alias = nullptr,
const std::size_t *overriden_value_size_limit_hdfs = nullptr);
void WriteValue(std::istream& value, std::uint64_t size, int64_t timestamp = CiiOldbUtil::Now());
void WriteValue(std::istream& value, std::uint64_t size, int64_t timestamp,
CiiOldbDpQuality quality);
void Subscribe(const std::shared_ptr<CiiOldbDpSubscription<T>>& listener);
void Unsubscribe(const std::shared_ptr<CiiOldbDpSubscription<T>>& listener);
std::shared_ptr<::elt::config::classes::meta::MdOldb<T>> GetMetadata() const;
void SetMetadata(const std::string& meta_data_instance_name);
void SetQuality(CiiOldbDpQuality quality) override;
elt::common::CiiBasicDataType GetType() const override;