The QC1 database

Design specifications for table definition files (TDF)

Reinhard Hanuschik

Version 1.1 (2002-10-21)
 

1. Purpose

The QC1 database consists of a number of tables per instrument. The content of these tables is similar to the content of the QC1 data tables in ASCII format which are maintained at the moment on the UNIX machines. The structure of these tables is also very similar to the existing ones, with some added keys (= columns) hosting technical information and data like filenames, useful for association with other data types or databases.

The format of the database tables is described by table definition files (TDF). These form the backbone of the QC1 database structure which is otherwise flat. I.e. it has a set of QC1 tables but (at least at the moment) no relations between them. So the TDFs, once finished, completely describe the database structure.

They have a second important purpose: they serve as configuration files for the qc1 database interfaces. E.g., it is here where the user defines which keys, in which order, are visible in the qc1_plotter interface.
 

2. Format

There is a set of QC1 tables which describe per instrument and mode, a set of QC1 data. E.g. FORS1 has

fors1_bias
fors1_dark
fors1_img_scrflat
fors1_img_skyflat
fors1_img_zerop
fors1_img_fixp
fors1_img_colext
fors1_lss_wave

These are mapped one by one into corresponding TDFs for the QC1 database. These files are then used to construct the database tables, and to provide self-documentation. E.g., in some interfaces, the selected keys are displayed together with the description provided in the corresponding TDF. The TDFs themselves become part of the database.

By agreement with Stefano, the TDFs per instrument can either be provided as single or multiple excel table (.xls), or as flat ASCII tables.
 

3. Content

As a database table, each TDF itself has rows and columns which become entries and keys.

The set of columns is always the same across all TDFs. It has:
 
field_name  self-explaining if possible; if keyword_name exists, used a translation
keyword_name short version of related FITS keyword or QC keyword, if available; e.g. HIERARCH.ESO.INS.TEMP1.MEAN would translate into ins_temp1_mean
field_type  varchar/float/integer/datetime
plot_flag  used for qc1_plotter interface; the qc1_browser also uses this keyword for default selection.
Possible values:
empty - not offered in qc1_plotter, unselected in qc1_browser
XD - key used as X axis default (one and only one)
YD - Y axis default (one and only one)
XY - clickable in interface
F - clickable for filtering
Combinations XDY, XYD, XY, FXY are allowed.
keyword_type type of key; used for sorting by qc1_browser; possible values:
general
qc1
instrument
sort_flag  integer number; used for sorting by qc1_browser; within all keys per keyword_type, sorting is done along sort_flag
length length for varchar types
default_value default values: NULL or specific value
example example
description used for self-documentation

The set of entries differs from TDF to TDF. While the set of general keywords is the same in most cases (if not all), the qc1 keywords are of course specific. The instrument keywords will again largely be the same across one instrument mode.
 

3.1 General keywords

The essential set of general keywords is the following:

calibfile
pipefile
arcfile
origfile
cdbfile
mjd_obs
tpl_start
civil_date
validity_flag
timestamp

-calibfile:  the file name of the master calibration file (our convention)
-pipefile: the pipefile name of the master calibration file
-arcfile: the arcfile  name of the (first) associated raw file
-origfile: the origfile name of the (first) associated raw file
-cdbfile: the cdbfile name of the master calibration file (given on ingestion, only known for downloaded files!)

-tpl_start: tpl.start keyword from the raw files
-civil_date: format yyy-mm-dd (e.g. 2002-10-22)
-validity_flag: yes by default; no by updating
-timestamp: of last modification

Filenames: For the purpose of being able to cross-correlate (join) information from the qc1 database with e.g. the raw data archive, it is extremely useful to have, as part of the qc1 database, information about the file names associated with the qc1 parameters. Of course this information is not always present, but we should try hard to retrieve it for historical data, and to provide for future data by default.
 

3.2 QC1 keywords

These are just inherited from the UNIX qc1 tables.
 

3.3 Instrument keywords

These contain all relevant instrument information, in most cases just taken from the UNIX qc1 tables.
 

4. Reference


The complete set of TDFs for UVES (echelle mode) is available as single excel file under the URL  http://www.eso.org/~qc/docu/GENERAL/uves_qc.xls.