Creating a Custom Telemetry Recorder

This example shows how to implement a Telemetry Recorder. A Telemetry Recorder allows recording shared memory queues, events and datapoints which are recorded by so called recording units. To create a Telemetry Recorder component these recording units have to be defined.

Provided Example

The provided example implements a Telemetry Recorder for one of the example topics as well as recording events. It is not intended to be used in an actual implementation, but rather as a template for an actual Telemetry Recorder.

Note

For simplicity reasons, the example uses the file based implementation of OLDB, Persistent and Runtime Configuration Repositories; as well as file based service discovery. This means that the underlying format of configuration and data points is different than the one used when the above mentioned services are used with the standard back-ends such as CII configuration service, CII OLDB, etc.

Source Code Location

The example source code can be found in the following sub-directory of the rtctk project:

_examples/exampleTelRec

Modules

The provided example is composed into the following waf modules:

  • app - The application including default configuration

  • scripts - Helper scripts for deployment and control

Dependencies

The provided example component depends on the following modules:

  • rtctk.componentFramework.rtcComponent - basic RTC Component functionality

  • rtctk.componentFramework.services.dataRecording - common recording infrastructure

  • rtctk.reusableComponents.telRec.lib - business logic class with implementation

In addition, the following applications are used:

  • rtctkClient - to send basic commands to the application

  • ExampleShmPub - an example SHM publisher from the DataTask example to provide example data

Running the Example Telemetry Recorder

The exampleTelRec can be run in isolation or as part of an SRTC system. To record telemetry data a shared memory queue needs to be created. This can be achieved using a Shared Memory Publisher (shmPub) or a Telemetry Subscriber (telSub). The queue name and the destination file name need to be provided through the Runtime Configuration Repository.

To make the example as simple as possible a script rtctkExampleTelRec.sh is provided to start an example shared memory publisher and an example recorder.

rtctkExampleTelRec.sh

After installing the RTC Tk, run the example using the following sequence of commands:

# Deploy and start the example applications
rtctkExampleTelRec.sh deploy
rtctkExampleTelRec.sh start

# Use the client to step through the life-cycle of the respective component instance
rtctkExampleTelRec.sh send Init
rtctkExampleTelRec.sh send Enable
rtctkExampleTelRec.sh send Run

# all commands until here can also be run using
# rtctkExampleTelRec.sh run


rtctkExampleTelRec.sh send Idle
rtctkExampleTelRec.sh send Disable
rtctkExampleTelRec.sh send Reset

# Gracefully terminate the applcations and clean-up
rtctkExampleTelRec.sh stop
rtctkExampleTelRec.sh undeploy

The recorded data will be stored in the $DATAROOT directory. The sub-directory can be customised by changing the session_id setting in the Runtime Configuration Repository.

Note

The commands indicated above, e.g. when using run, may generate the following output:

rtctkExampleTelRec: no process found
rtctkExampleShmPub: no process found

This is expected and should not be treated as an indication of failure.

Development Guide

This section explains how to create a simple Telemetry Recorder from scratch. For more detailed description see the Telemetry Recorder section.

Recording Units

A TelemetryRecorder hosts one or many recording units. The Rtc Toolkit provides recording units for IPCQ, Event, and DataPoint recording. Each recording unit has it’s own state and might wait for a condition before starting. E.g. the IPCQ recording unit supports waiting for a specific sample id before actually writing the data. It is also possible to follow another recording unit (leader), so that the recording is done while the other recording unit is recording.

RecordingInfo

To allow recording of arbitrary structs (like IPCQ topics), the RecordingInfo trait class is used. To record a topic or event type, this class needs to be specialised to allow the recorder to extract the data from these types. In the example this looks as follows:

template<>
struct rtctk::componentFramework::RecordingInfo<ScaoLoopTopic> {
    using DataType = ScaoLoopTopic;

    using Slopes = AsSpanT<const decltype(std::declval<ScaoLoopTopic>().wfs.slopes)>;
    using Intensities = AsSpanT<const decltype(std::declval<ScaoLoopTopic>().wfs.intensities)>;
    using Commands = AsSpanT<const decltype(std::declval<ScaoLoopTopic>().commands)>;

    using Output = std::tuple<uint32_t, Slopes, Intensities, Commands>;
    using Recorder = FitsRecorder<uint32_t, Slopes, Intensities, Commands>;

    static constexpr Recorder::ColumnDescription COLUMNS = Recorder::ColumnDescription {
        { {"sample_id"sv, ""sv},
        {"slopes"sv, ""sv},
        {"intensities"sv, ""sv},
        {"commands"sv, ""sv} }
    };

    static Output AsTuple(const DataType& data) {
        return std::make_tuple(
            static_cast<uint32_t>(data.sample_id),
            ToSpan(data.wfs.slopes),
            ToSpan(data.wfs.intensities), ToSpan(data.commands));
    }

    static constexpr uint32_t SampleId(const DataType& data) {
        return data.sample_id;
    }
};

The parts that need to be defined are:

  • DataType: the type that is being recorded.

  • Output: needs to be a tuple representation of the DataType.

  • Recorder: the default recorder to use for this DataType.

  • COLUMNS: contains the name and unit of the FITS binary table columns

  • AsTuple: A function that performs the conversion to the tuple defined in Output.

  • SampleId: A function that returns the SampleId of the DataType.

In this example the Slopes, Intensities, and Commands aliases are used to make the other type definitions simpler. The data fields of the topic are converted to spans (using the helper type trait AsSpanT and the ToSpan function) which prevents unnecessary copies.

Business Logic Factory

The RTC Toolkit provides a reusable BusinessLogic class in rtctk::telemetryRecorder::BusinessLogic to manage multiple Recording Units. Since construction of the recording units is deferred to ActivityInitialising, instrument RTC developers need to provide a nested factory method that defines which recording units a component shall have. This factory is then passed to the generic component-runner method.

// main entry point for user code
void RtcComponentMain(Args const& args) {

    // will be invoked at component startup
    auto bl_factory = [](std::string const& name, ServiceContainer& services) {

        // inner factory will be invoked in ActivityInitialising
        auto ru_factory = [](std::string const& name, ServiceContainer& services) {

            auto& rtr = services.Get<RuntimeRepoIf>();
            auto& oldb = services.Get<OldbIf>();
            auto& es = services.Get<EventServiceIf>();

            RecUnitListType rec_units;

            AddRecUnit<IpcqRecordingUnit<ScaoLoopTopic>>(
                    rec_units, name, "unit_1", rtr, oldb);

            AddRecUnit<JsonEventRecordingUnit<>>(
                    rec_units, name, "unit_2", rtr, oldb, es, "test_topic");

            return rec_units;
        };

        // returns the biz logic object
        return std::make_unique<BusinessLogic>(name, services, std::move(ru_factory));
    };

    // invoke component runner function
    RunAsRtcComponent<BusinessLogic>(args, std::move(bl_factory));
}

This creates a new TelemetryRecorder with two recording units:

  • IpcqRecordingUnit for a ScaoLoopTopic

  • JsonEventRecordingUnit, which can record any events on the test_topic topic

Each unit has a unique name, that is used to retrieve configuration for that unit from the Runtime Configuration Repository and to publish the current state of that unit to the OLDB. To add more recording units, it is sufficient to add other AddRecUnit calls in the same fashion.

Hint

Do not forget to add the RecordingInfo and the settings in the Runtime Repository, whenever applicable.