Components

Figure Fig. 1 show the application components provided by this software package.

Fig. 1 Package and main components overview

daqOcmServer

Coordinates the full Data Acquisition life-cycle.

Main responsibilities:

• Act as interface for creating Data Acquisitions.

• Provide life-cycle control of Data Acquisitions.

daqOcmCtl

Controls daqOcmServer from command line.

daqDpmServer

Data Product Manager server component.

Main responsibilities:

• Queues Data Acquisitions and schedules them for merging.

• Copy inputs for Data Product from source nodes to local storage.

• Execute daqDpmMerge to create Data Product.

• Deliver Data Product to specified receivers (normally OLAS).

daqDpmMerge

Standalone (command line) executable.

Main responsibilities:

• Create Data Product from Data Product Specification referencing and containing input sources (FITS files and keywords).

Software Context

This section describes the software context of OCM and DPM, focusing on provided and used software interfaces. Fig. 2 shows the components with their relations to interfaces which also specifies their dependencies. The individual interfaces are described for daqOcmServer in Table 2 and daqDpmServer in Table 3.

Fig. 2 Software context for daqOcmServer and daqDpmServer. The interface category refers to whether the interface is used (or provided) persistently or in a transitory manner e.g. because interface is used only for the duration of an operation. Non-conjugated ports (without ~) indicate that the interface is provided whereas conjugated ports (with ~) indicate a interface is used.

Table 2 Interface Description: daqOcmServer

Interface	Description
stdif	Standard interface implemented and provided by OCM which enables operational life-cycle control and supervision.
daqif.OcmDaqControl	Interface implemented and provided by OCM to manage Data Acquisitions.
~oldb	OCM has a persistent dependency to the Online Database to update it with current configuration and application status.
~metadaqif	OCM use the metadaqif to acquire data from metadata sources for the duration of each Data Acquisition.
~recif	OCM use the recif to acquire data from primary data sources for the duration of each Data Acquisition.
~daqif.DpmDaqControl	If DPM is operational OCM will send commands to schedule the individual Data Acquisitions to be merged into Data Products.

Table 3 Interface Description: daqDpmServer

Interface	Description
daqif.DpmDaqControl	Interface provided by DPM for scheduling Data Acquisition to be merged into Data Product.
~oldb	DPM has a persistent dependency to the Online Database (redis-server) to update it with current configuration, application status and status of all Data Acquisitions.
remote-storage	Interface to remote file systems used to fetch the individual files to be merged into Data Products or send final Data Product to a receiver.
archive	Interface used to deliver Data Products to OLAS.
storage	DPM stores all files on a file system before merging is performed as well as the resulting Data Products.
daqDpmMerge	To perform the merge daqDpmServer will execute daqDpmMerge as a subprocess and monitor it using standard I/O pipes. To abort merging signals will be used.

Deployment

Reference

Two reference models for deployment is provided below in Fig. 3 and Fig. 4. The node Instrument Workstation below refers to any instrument provided node. There are no particular requirements or constraints unless daqDpmServer is deployed there.

Both daqOcmServer and daqDpmServer have a private workspace on the local file system that persist independently of the deployed process and is used for state persistence and recovery. For additional details see sections for OCM and DPM respectively.

Fig. 3 Reference deployment showing how daqDpmServer is deployed on the Data Handling Server Workstation. Advantage of this deployment is that daqDpmServer does not need to transfer the Data Product to DHS Workstation. Both daqDpmServer and daqOcmServer have a private workspace on the local file system where they are deployed.

Fig. 4 Alternative deployment showing how daqDpmServer is deployed on an Instrument Workstation. In this deployment daqDpmServer needs to transfer the Data Product to DHS Workstation for archiving. Both daqDpmServer and daqOcmServer have a private workspace on the local file system where they are deployed.

Deployment Constraints

Software Context also show dependencies to neighbouring components or systems, as anonymous interfaces, as well as the temporal dimension with the interface category. If a dependency is transitory because it e.g. is only required to complete an operation, for the duration of the operation, this will also be noted below.

The following constraints needs to be taken into consideration when deploying each component:

daqOcmServer

Apart from satisfying runtime dependencies and services enumerated in Table 2 there are no specific deployment constraints for OCM. The only persistent service used by daqOcmServer is the oldb (redis server).

To acquire data from a data source daqOcmServer requires request/reply access for the duration of StateAcquiring. Once data has been acquired there is no connection or dependency towards any data source.

daqDpmServer

daqDpmServer and its subprocesses may consume a lot of I/O resources. Both network I/O when transferring source files to the local node or delivering the Data Product to receivers and disk I/O when Data Product is created from the input source files. As such this needs to be dimensioned.

daqDpmServer is special in that it does not require an operational ICS to perform its function. This is also the reason why it does not provide the standard interface stdif, because it does not require state management; if it is running it is operational. However, since daqOcmServer continuously receives Data Acquisition status updates from daqDpmServer it is strongly recommended to operate daqDpmServer with daqOcmServer running to have consistency between them (state NotOperational is also ok).

daqDpmServer can also be deployed multiple times on the same host, each with their own private workspace. It is e.g. a valid option to deploy multiple instances on the same OLAS workstation, that may be shared by multiple instruments. In this scenario it is recommended that each daqDpmServer runs as an instrument specific user. There is no coordination between daqDpmServer instances and each may compete with available resources.

To create data products from the individually created FITS files daqDpmServer requires access to the files using either locally mounted file system access or remotely with properly configured unattended rsync access. At this time it is recommended to use public key authentication.

Example configuration:

# ~/.ssh/config

# DPM access to hosts providing FITS files
Host 10.0.129.1
    IdentityFile /path/to/public/key/id_rsa
Host 10.0.129.10
    IdentityFile /path/to/public/key/id_rsa