QC AT ESO

The main goals of ESO data quality control (QC) are:

• to know the instrument status (health checks)
• to monitor it over time (trending)
• to deliver data of known and defined quality (certification).

The QC process uses data processing pipelines. These support all VLT/VLTI instruments and all major modes. More ...

The QC process is fed by the data transfer from Paranal into the archive which takes place within minutes. The calibration data are then processed incrementally at ESO Headquarters (typically once per hour). The quality information is extracted into QC1 parameters. The most important parameters are displayed graphically on the Health Check monitor. Also, their values are scored against configured thresholds, to discover anomalies. The quality scores are also displayed graphically and are evaluated both by the QC scientists and the Paranal staff astronomers.

The complete trending history of each VLT instrument is also maintained on the Health Check monitor.

The QC0 process checks that

• raw data have a reasonable exposure level
• comply formally with the ESO standards
• calibration data are complete
• user-defined observational constraints have been respected during the observation

Typically the QC0 checks are done on raw data. QC0 is entirely done by Paranal staff astronomers.

Quality Control Level 1 consists of quality checks on pipeline-processed data. Quality is measured by numbers, the QC1 parameters. Main goals of the QC1 process are

• to monitor instrument stability and performance
• to monitor the atmosphere
• to monitor pipeline performance.

The checks are done on calibration data. Scientific usefulness cannot be assessed with this process.

The QC1 parameters are used

• to assess the quality of master calibration files (e.g. noise properties, gradients),
• to compare with expected values (e.g. gain value),
• to assess the performance of the instrument (e.g. throughput from photometric zeropoints),
• to monitor efficiency of master creation process (e.g. reduction of random noise).

The result of the QC1 process is used to

• decide whether master creation data are certified or rejected,
• assess the quality of the reduction process.

The QC1 parameters are archived. They are directly available for studies through the QC1 database interface (although in many cases the trending pages may be more interesting, see below).

Trending is monitoring QC1 parameters over time (e.g. effective resolution as function of time). A core task of QC Garching is to create trending pages (also called Health Check monitor) which are used to monitor the instrument evolution and performance.

The most recent values of important QC1 parameters are scored against configured thresholds, to discover anomalies. The quality scores are also displayed graphically and are evaluated both by the QC scientists and the Paranal staff astronomers.

The names of the pipeline products follow a certain naming convention. This is done to have human-readable rather than technical file names. The naming scheme has been chosen to have the type of product, its date of creation, and its pipeline-relevant parameter values all recognizable from the product name. With the begin of period 76 (October 2005), both science and calibration product files are renamed. The general format of user-oriented product file names is the same for all VLT/VLTI data.

The AMBER HC plots are available on the HC home page.

Instrument code

All AMBER pipeline product names start with AM.

Type of product

The type of product is coded in the fits key HIERARCH.ESO.PRO.CATG. A short form of this is coded in the name using a four character scheme:

Science products (if processed) have the following scheme:
AM_S<PPP>_<OBS_ID>_<DATE_TIME>_<parameter_string>.<ext>
e.g. AM_SSFR_200147608_2005-02-26T03:47:46.826_GMR_24403509_U2U3.fits
where:

• S<PPP> is the four letter code uniquely describing the type of SCIENCE product (the first letter is always S for SCIENCE). Each OB consists of several files containing fringes (5 of them in general), the merged file is a concatenation of all the frames of these individual files
• OBS_ID is the OB number read from the OBS.ID fits key
• DATE_TIME is the UT date and time of the first raw frame measured for the OB (e.g. 2005-04-01T04:31:12); this information is required since OBs can be executed several times
• parameter_string is a string containing the instrument setup information (e.g. grism mode (high-medium or low resolution), P2VM ID, telescope stations)
• ext is either fits or tfits (for fits tables)