Metadata Collector

Overview

The Metadata Collector (MDC) is a reusable component that can receive requests from e.g. the science instrument Observation Coordination Manager (OCM) to produce FITS files containing keywords and binary tables. These FITS files are then merged by the science instrument Data Product Manager (DPM) to produce a single file that is delivered to the archive system.

The MDC needs to be customized both at compile-time and run-time. A tutorial on how to implement a custom Metadata Collector is provided in the Metadata Collector Tutorial.

Basic Operation

In contrast to the Telemetry Recorder, the Metadata Collector does NOT support commands Run, Idle and Recover. However, being an RTC Component, it still supports basic commands Init, Enable, Disable, Reset and Update, which are required to bring the component to state On:Operational and to trigger configuration updates.

To manage metadata acquisition sessions, the Metadata Collector implements ECS interface metadaqif, and thus provides additional commands StartDaq, StopDaq, AbortDaq and GetDaqStatus. These commands can be used in state On:Operational to control and monitor individual metadata acquisition sessions. A detailed description of the metadaqif can be found here.

Note that the Metadata Collector can only handle a single recording session at a time. If a client attempts to start a new session while another session is still ongoing, the StartDaq command will be rejected.

Once a recording session is started and the component is in state On:Operational:Acquiring, it will ingest data from various sources and write them to a set of output files located in folder $DATAROOT/component_id/session_id/.

In contrast to the Telemetry Recorder, a recording session cannot conclude on its own, meaning that each recording session must be finished with a StopDaq or AbortDaq command. The generated files can be accessed once the session has concluded.

In case a recording session fails, the component will auto-transition to state On:Operational:Idle and mark the session as failed. This can be observed with command GetDaqStatus.

Common Recording Infrastructure

The Telemetry Recorder and also the Metadata Collector make use of the same common recording infrastructure that is explained in the Telemetry Recorder section.

Limitations and Known Issues

At this point, the recording infrastructure is not optimised for high performance data recording, which means that there is still room for optimisation to make recording viable at fast loop rates.

Currently, the only output format are FITS binary tables. Additional recording unit types that can record FITS keywords and other formats will be added in a sub-sequent release.

Not all event types are supported at the moment e.g. support for certain configuration and coordination event types is still missing. Please use the generic JsonEventRecordingUnit to record such events until the missing parts are implemented in a sub-sequent version.