Data Interface Dictionary

The DID module of the DIT Component (ifw/dit/did), provides tools to handle Data Interface Dictionaries (DIDs).

This mainly consists in the following operations:

  • Loading dictionaries.
  • Looking up keywords.
  • Filtering out sets of keywords.

The Doxygen documentation for the “dit::did” module is available here.

At this point in time, it is only foreseen to use DIDs to assist generating ‘ESO compliant’ FITS files.

Data Interface Dictionary - Format

The DID format is based on YAML.

An example of a DID is shown in the following (extract from “primary.did”):

#===============================================================================
Header:
  Name: ifw/dit/stddid/primary.did
  Scope: FITS
  CreationDate: 2018-11-12
  Version: "@(#) $Id$"
  Include: ()
  Description: |
    Dictionary defining the non-hierarchical keyword cards used in ESO FITS files.
    DICB: Reviewed for compliance with ESO-044156/6.
#===============================================================================
SIMPLE:
  Class: (FITS)
  Type: Boolean
  Format: c
  Default:
  Unit:
  Range: ()
  Comment: Standard FITS
  Tags: ()
  Description: |
    SIMPLE must be equal to 'T' to conform to FITS.
#-------------------------------------------------------------------------------
NAXIS:
  Class: (FITS)
  Type: Int32
  Format: d
  Default:
  Unit:
  Range: ()
  Comment: Nb of axes in data array
  Tags: ()
  Description: |
    Number of axes of the data array (limited to less than 999).
    A value of 0 signifies that no data follow the
    header in the HDU (header and data unit).
#-------------------------------------------------------------------------------

An example of a keyword definition of an ESO hierarchical keyword, is shown here:

INS(#) FILT# ID:
  Class: (FITS)
  Type: String
  Format: s
  Default: None
  Unit: None
  Range: ()
  Comment: Filter ID
  Tags: ()
  Description: |
    ESO identification number for the filter.
    Format: #n where n is a unique number for the
    observatory.

As can be seen, it is possible to specify that a keyword has an index, appending “#” to the keyword component.

If the “#” is contained in parenthesis, the index is optional. The above listed keyword can thus be looked up as e.g. “INS FILT2 ID” or “INS1 FILT2 ID”.

In the SVN tree, in the module hosting DID documents shall typically be stored in the SVN tree as follows ((example):

<SVN path>/<project>/<package>/<module>/resource/dictionary/<package>/<module>/<DID name>.did

Note, it is proposed to use extension “.did” for DIDs.

An example of how to configure a module, which only provides DID files, can be found here:

http://svnhq9.hq.eso.org/p9/trunk/EELT/ICS/ifw/dit/stddid

The properties of the fields of the DID header, are as follows:

Field: “Header”:

Marks the beginning of the header segment.

Field: “Name”:

The name of the DID, typically the name in the module under the “resource/dictionary” directory, e.g. “<package>/<module>/<DID name>.did”.

Field: “Scope”:

User defined. Could be e.g. “FCF FITS” or “FCF Ops Logs”.

Field: “CreationDate”:

ISO 8601 date stamp for when the DID was created.

Field: “Version”:

Preferably, version string, automatically updated by the software configuration management system.

Field: “Include”:

List of DIDs to be loaded. Shall be given, enclosed in parenthesis, i.e. “(<DID>[,<DID>])”, e.g. “(pkg1/mod1/example1.did, pkg2/mod2/example2.did”). The DIDs will be loaded in the order they are listed, starting from the left. After loading/parsing/installing included DIDs, the records of the DID being loaded, are handled. In case of multiple key definitions, the last occurrence takes precende. This feature is referred to as “DID inheritance”.

Field: “Description”:

Description of the DID. The following mark-up will be supported by the (future) DID Editor, to make the descriptions more readable:

  • Heading 1: “#<heading>#”.
  • Heading 2: “##<heading>##”.
  • Bold: “*<word>[*<word>]*”.
  • Underline: “_<word>[_<word>]_”.
  • Italics: “-<word>[-<word>]-“.
  • Bulleted list: “** <item>[\n** <item>”.
  • Numbered list: “## <item>[\n## <item>”.
  • Dashed list: “— <item>[\n— <item>]”.

More text mark-up options may be added later.

The properties of the fields of the DID Keyword Record, are as follows:

Field: “<keyword>”:

The keyword itself.

No specific casing is assumed, but a hierarchical structure of the keys is supported. E.g. “Detector#.Window#.StartX” or “INS# FILT# ID”. The DID Class interprets “.” (dot) and ” ” (space) as separators of the elements constituting a keyword.

The hash sign (“#”) shown above, indicates that the component of the keyword has an index. The indeces must be in the internal “[0; oo[“.

A future extension will allow to specify that a component of a keyword, may have an index as follows: “INS(#) FILT(#) ID”. This means that the keyword components may be used with an index or not. If the hash sign is defined like this “INS#”, that component must have an index, otherwise it is illegal.

Field: “Class”:

Indicates the context in which the keyword is used. For the moment it is only foreseen to use the keyword records for generating FITS files, and the field should therefore be “(FITS)”.

Field: “Type”:

The data type of the key. The following types/type names are handled:

  • Boolean.
  • Int8.
  • UInt8.
  • Int16.
  • UInt16.
  • Int32.
  • UInt32.
  • Int64.
  • UInt64.
  • Float.
  • Double.
  • String.
  • NonBasic (to be defined when CII released).
  • TimeDate (to be defined when CII released).
  • TimeStamp (to be defined when CII released).
  • Blob (to be defined when CII released).

Field: “Format”:

Format of the value when generating a keyword line. Based on the C ‘printf()’ formatting, leaving out the “%”. Only limited support at this point.

Field: “Default”:

Default value for the key. If not applicable, it should be left empty.

Field: “Unit”:

The unit of the value represented by the keyword. No specific checks of the value of this field is provided, but it is recommended to use SI units when possible.

Field: “Range”:

The valid range or ranges of the keyword value. Example of range definitions:

  1. “((0;30))”: Interval [0; 30] is the valid range of the keyword.
  2. “((1;5),(10;20))”: Internals [1; 5] and [10; 20] are valid ranges.

If not used, the field value can be left empty.

Field: “Comment”:

The comment field as it will be written when generating a keyword line.

Field: “Tags”:

An optional, variable length list of tags, used to identify the key. E.g. “(Tag1, Tag2)”.

Field: “Description”:

Detailed description of the Keyword Record. See Description Field of the DID Header above.

Note

At this point, no ‘formal’ validation of the correctness of a DID via schema validation is provided. This may be provided in a future release. The validation and checks carried out by the “dit::did::Did” class, and associated helper classes, should normally be enough to verify the proper syntax and semantics of the document.

Note

For now, no tool is provided to create and modify DIDs. It is foreseen, in the near future, to provide a GUI application, which can be used for managing the DIDs. Without such a tool, creating the DIDs is a bit ‘cumbersome’.

Header Files, Library & Namespace

The following header files are provided by the “dit::dit” module:

Header File: “dit/did/Header.hpp”:

Provides classes/definitions for handling of the DID Header.

Header File: “dit/did/Record.hpp”:

Classes for handling the information for one Keyword Record.

Header File: “dit/did/Did.hpp”:

Provides the class used to handle one DID, which implies loading and looking up keyword information.

The library for the “dit::did” module is named: “ifwDitDid”.

The namespace prefix is “dit::did”.

Classes

The DID module provides the following classes:

Class File: “dit::did::Header”:

Class to handle the DID header.

Class File: “dit::did::Range”:

Used to handle the Range Field of a Key Record.

Class File: “dit::did::FormatSpec”:

Used to parse and handle the Format Field of a Key Record.

Class File: “dit::did::Record”:

Used to handle one Keyword Record.

Class File: “dit::did::Did”:

Class to handle one DID, which may include other DIDs.

Example

The usage of the library is straightforward, but the following small example, provides an impression how to use it:

#include "dit/did/Did.hpp"

dit::did::Did did;
did.Load("prj/pkg/mod/Test.did");

// Print out the headers of the DIDs loaded.
dit::did::Header::DidHdrField hdr_fields = static_cast<dit::did::Header::DidHdrField>
        (dit::did::Header::DID_HDR_FIELD_NAME |
         dit::did::Header::DID_HDR_FIELD_SCOPE |
         dit::did::Header::DID_HDR_FIELD_CRE_DATE |
         dit::did::Header::DID_HDR_FIELD_INCLUDE |
         dit::did::Header::DID_HDR_FIELD_DESCRIPTION);
for (auto it = did.GetHeaders().begin(); it != did.GetHeaders().end(); it++) {
    std::cout << std::endl << it->Print(hdr_fields) << std::endl;
}

// Print out entire DID, including header of main DID loaded.
std::cout << std::endl << did.Print(dit::did::Did::DID_HDR_LAST) << std::endl;

// Look up a single Keyword Record.
dit::did::Record record;
did.LookUp("INS1.LAMP3.NAME", record);
std::cout << std::endl << record.Print()) << std::endl;

// Look up multiple records.
ctd::defines::List<dit::did::Record> records;
did.LookUp("INS1.LAMP2.*", records);
std::cout << std::endl << dit::did::Record::PrintRecords(records)) << std::endl;