The Data Acquisition Process

Contents

• The Data Acquisition Process

  • Introduction

    • Stylistic Conventions

    • Process Overview

  • StateAcquiring

    • Overview

    • Starting

    • Stopping

    • Aborting

  • StateMerging

    • Overview

    • NotScheduled

    • Scheduled

    • Transferring

    • Merging

    • Releasing

Introduction

This section provides an overview of the Data Acquisition process, both in terms of what is currently implemented in OCM and what is planned for with DPM.

For an overview of conceptual model, processes and the relationships between components see the section Overview.

Stylistic Conventions

The following visual convention is used for states and transitions.

Normal

Normal state.

Transitional

Transitional states are states where the Data Acquisition remain during handling of a command, if it cannot be completed immediately.

Command()

Transition that occurred when initiating the handling of a command. That it is a request/command can be identified by the suffix “()”.

Event

Transition that occurred due to an internally generated event, internal or external. This can also occur due to completion of the handling a command and is where the reply is sent.

Process Overview

Each successful Data Acquisition includes the following states (represented in the MAL API with DaqState) which spans the following high-level activities:

StateAcquiring

From the initial point it is created in OCM through the phase where data actually has been acquired by data sources.

StateMerging

Through the phase where a Data Product is created from the acquired data and released to the Online Archive System (OLAS) or other configured recipients.

StateCompleted

To the final state for a Data Acquisition. The two possible substates are:

• Completed and

• Aborted,

which indicates a completed or user aborted Data Acquisition respectively.

These states, each with their own substates with an overview of transitions are visualized in Fig. 10.

Fig. 10 SysML diagram of Data Acquisition states and transitions. Transitions in bold indicate the transition path for a successful Data Acquisition. Purple states are transitional states used for the duration of an operation. The orthogonal region with states NotError and Error is exclusively a means to model the error flag used in the implementation. Expect further refinement in StateMerging in following releases.

Important

The possible final states for a Data Acquisition is either Completed or Aborted. OCM and DPM will never abort a successfully started Data Acquisition on its own volition; only at the request of a client (normally the instrument operator).

The only scenario where OCM aborts is if StartDaq() fails, in which case the potentially partially started Data Acquisition is forcibly aborted.

Note

Error is possible in any state which is indicated with an error flag in the Data Acquisition status structure DaqStatus (this can be considered an orthogonal error state as shown in Fig. 10). This means for example that a Data Acquisition may be Aborted with or without error.

Errors may prevent forward progress in some cases and in other cases errors may be ignored by forcing forward progress with commands such as ForceStopDaq() (and accepting degraded Data Product as a result).

The enumeration DaqState defines the top-level states and all the sub-states are enumerated together in DaqSubState. See section OCM Data Acquisition Control for description of the states.

The next sections provide further details of the Data Acquisition life-cycle and how to interact with OCM for control.

StateAcquiring

This section documents the details of the top-level StateAcquiring state of a Data Acquisition; the observable states, the transitions and how it relates to the OCM command interface described here.

Warning

Although the OCM API disambiguates on which Data Acquisition to operate on (by requiring user to provide the Data Acquisition identifier), the interface to detector data sources recif does not, and will always operate on its current recording whether it is correct or not. This means that users must be extra careful and e.g. make sure to only have one active Data Acquisition at a time which use the same primary data source.

Overview

The following diagram shows an overview for a nominal Data Acquisition where commands succeed:

Note

For brevity ForceAbortDaq() is not shown in the diagram as it performs the same transitions as AbortDaq() except that when error occcurs it still performs the transition.

Fig. 11 Data Acquisition state and transition overview

The transition for the event Stopping is performed automatically by OCM when all primary data sources stop. If e.g. detector is configured to integrate for 10 seconds it will automatically stop, which is then observed by OCM, which then triggers the transition to Stopping state where metadata sources are being stopped.

Starting

The following diagram show the initial states of a Data Acquisition created with StartDaq(), from the initial state NotStarted which is the point where the Data Acquisition has been created and registered internally in OCM but has not yet initiated any actions. When ready OCM transitions to state Starting in which all data sources are requested to start their data acquisition. When all sources acknowledge successfully the Data Acquisition transitions to Acquiring.

Note

StartDaq() creates a new data acquisition and starts it in one command, so the state NotStarted is never observed from outside. If a use-case requires it this can be changed to a two-step command, one that creates the Data Acquisition and one command that starts it.

Important

If StartDaq() does not succeed the user should clean up the failed data acquisition by aborting it. This will abort any partially started acquisitions for the configured sources. If a data source is not responding or otherwise report error, this will cause AbortDaq() to fail. For these cases ForceAbortDaq() can be used to force the transition to Aborted.

Stopping

The following diagram show from which states StopDaq() and ForceStopDaq() is valid. The diagram shows only StopDaq() but is valid also for ForceStopDaq().

Using ForceStopDaq() command the only difference is that the transition to Stopped is forcefully performed even in the presence of errors from e.g. data sources.

Note

If StopDaq() fails the Data Acquisition remains in Stopping state. At this point it is possible to retry StopDaq() or force it with ForceStopDaq().

Important

Since ForceStopDaq() stops a Data Acquisition even if data sources fail to stop, it means that user might have to perform manual error recovery on the faulty components.

Aborting

The following diagram show from which states AbortDaq() and ForceAbortDaq() is valid. The diagram shows only AbortDaq() but is valid also for ForceAbortDaq().

Using ForceAbortDaq() command the only difference is that the transition to Aborted is forcefully performed even in the presence of errors from e.g. data sources.

Note

If AbortDaq() fails the Data Acquisition remains in Aborting state. At this point it is possible to retry AbortDaq() or force it with ForceAbortDaq().

Important

Since ForceAbortDaq() aborts a Data Acquisition even if data sources fail to abort, it means that user might have to perform manual error recovery on the faulty components.

StateMerging

This section documents the details of the top-level state StateMerging state of a Data Acquisition. Compared to StateAcquiring the successful sequence is always autonomous and can complete unattended. The exception to this is if the Data Acquisition should be aborted in which case the request AbortDaq() can be sent to OCM, which will delegate to DPM if required.

If a failure occurs e.g. because of misconfiguration so that file transfer fails, DPM will attempt again next time it is started. As a last resort manual recovery can be attempted.

Important

Releasing to OLAS or other receivers is not yet implemented.

Note

It is worth clarifying that clients are not required and are never expected to have to interact directly with DPM.

Overview

Fig. 12 Data Acquisition state and transition overview for StateMerging

NotScheduled

OCM will attempt to schedule Data Acquisition for merging. If DPM is offline or otherwise unreachable it will remain in this state.

As Data Acquisition has not yet been scheduled it is possible to abort the Data Acquisition without a connection to daqDpmServer.

Scheduled

Responsibility for completing the Data Acquisition is from this point on DPM and authoratitive Data Acquisition status originates from DPM, but still published by OCM.

If a request to abort Data Acquisition is made the normal behaviour is to forward the request to DPM. If DPM is offline the Data Acquisition can only be aborted with ForceAbortDaq(), but this will be unknown to DPM:

Warning

There is a risk of Data Acquisition state inconsistency if Data Acquisition is forcibly aborted. As DPM is offline or unreachable it may independently of OCM complete the merge process. As such it is possible the Data Acquisition status is inconsistent or may change after new information from DPM is available again.

Transferring

Files are transferred to the local daqDpmServer workspace. This time there is no optimization implemented for the case the file is available from local file system mount.

Merging

This is the state where the final Data Product is created from all the previously acquired data. An overview of that process is provided in Data Product Creation.

Releasing

Data Product is released to configured receivers.