The Health Check monitor is developed and maintained by QC Garching. It provides information about the health of the instrument and about calibration quality.

Update information and process monitor

Last update: timestamp of creation of this page and plot (UT)

Now: current timestamp (UT) (press Ctrl+Shift+R to enforce reloading of the page, it may be cached)

(0d 20h:05m ago): difference of both timestamps (requires javascript enabled)

QC pipeline: this is the name, version and installation date of the pipeline being currently in use for processing at QC (for comparison to the version installed on Paranal)

	transfer	monitors the health of the data transfer system (DTS) from Paranal to Garching Archive (ngas); green if no CALIB file delayed by more than 1 hour, otherwise red; red alert may explain processing problems
	ngas	monitors the QC access to the archive (ngas); green if download possible, otherwise red; red alert may explain processing problems

What to do if HealthCheck monitor page is outdated? (intended for our Paranal partners)

The Health Check monitor pages are refreshed upon discovery of new data, but at least once in 24 hours. If a page is found to be older than 24 hours (as indicated by the "... ago" field) this might indicate a problem (this field is calculated by your browser).

The first thing to check is: does this problem show up only for the current report, or for all reports of a given instruments, or for all instruments? In the latter case, there is most likely a network or a database problem. It may be known already, check first with DHA. If the problem is specific to one instrument, send a mail ("contact") if you expect the QC scientist to be able to read it; to the general list qc@eso.org, if not. On a weekend generic problems will most likely be picked by the Garching maintenance teams, they provide service 7 days a week during daytime. Again ask DHA for support, they know whom to contact.

If there is a problem with one report only, this is most likely due to the process on the QC machine. Pls. contact the QC scientist in charge (qc_<instr_name>@eso.org).

Forced refresh

On Paranal, you can force a refresh of the HC monitor using the button 'forced refresh'. You would use it when you have taken recent calibrations and want to see the results on the corresponding HC monitor, without waiting for the hourly cronjob on the Garching machines.

You first authenticate (ask shiftleader for user and password). Then enter your email address which will be used to send a notification about start and end of the processing. Follow up the processing on a link that is filled with the processing log.

Once you have launched a request, a cronjob will pick this signal within a minute and start the processing. ALL calibration data for that instrument that have arrived in the archive since the last regular processing (less than an hour ago) will be processed. Depending on the number of files and on the performance of the pipeline, the processing may take a few minutes or longer.

It might happen that not all of the new files are already available in NGAS. This depends on the DTS and takes typically a couple of minutes. The log file will tell you so. You might then want to start the same process again, or just wait until the regular cronjob takes over. At the end the log file will tell you about the status of the new files and the names of the HC plots that have been updated.

Please note that the HC plot update will be immediate after termination of the job on the HQ server, but might take another few minutes to be replicated to the PL server. In urgent cases, you may want to use the www.eso.org URL for checking the HC plot, instead of the www.pl.eso.org address.

In exceptional cases it might happen that the forced refresh is in conflict with the regular processing job being called once per hour. In that case the forced job will be terminated since it doesn't make sense, and a notification will be sent to you.

Navigation

This navigation links to:

• this file;
• the process description (users guide);
• a file with additional information (time zone information, link to ASM, monitor of process health).

Group navigation. Plots are often grouped together. For instance, plots with detector properties could be grouped like:

same group:	bias		dark	linearity&gain	fp-noise	contamination

  daily/often; important to check
This marks the most important plots. These are the ones which are fed daily by dedicated health check calibrations. These are also the ones which should be checked daily. The other ones are also of interest but are not filled so frequently (nevertheless all are updated at least once a day or when new data are detected).

The second line displays the report scores (see here).

News. There are three kinds of 'news' (messages on the HC monitor): general news (applicable to the HC monitor in general), instrument news (applicable to all HC reports of a particular instrument), and report news (messages about current issues). The report news can be managed interactively, in the same way as the calChecker analysis cases:

Report news:

2012-11-26 The low flat lamp issue is followed up in PPRS-47984 [whummel@eso.org]

• click 'edit'
• provide your credentials (if unknown or forgotten, contact qc@eso.org)
• provide a new comment, edit or delete an existing comment
• for new or updated entries, provide a date and your email in addition to your comment.

The HC comment editor is available to both parties of the closed QC loop: QC and SciOps. In particular, daytime or nighttime astronomers on Paranal are encouraged to use the editor to exchange information with their colleagues.

Note that the report news have a validity period of 14 days. It is assumed that any quality issue related to a HC report is either solved, or becomes obsolete after that time. The report news are not designed to store permanent information.

You can also use the link edit COMMENTS ... on the 'score overview' page and select any HC report of a given instrument for comment editing. This is particularly useful if e.g. a new quality issue has occurred and a related comment should be entered for several affected HC reports.

An updated report comment will be visible immediately on the corresponding HC report. You may need to enforce page refresh with CTRL+R.

Note that report news will not immediately be visible on the Paranal ('PL') mirror site, but have to wait for replication (up to 15 minutes).

DATE*:		2012-12-09 report | NLT	2012-12-10 report | NLT	2012-12-11 report | NLT	2012-12-12 report | NLT	2012-12-13 report | NLT	2012-12-14 report | NLT
Raw CAL displays:		raw	raw	raw	raw	raw	raw
Product quality:		products	products	products	products	products	products

'Raw CAL displays' and 'Product quality' links. With the same look and feel as on the calChecker, this panel displays links to the pages with a) screenshots of raw CALIB files (links raw), and b) complete information about the data quality (links products). Find more information about the data quality links in the calChecker .

There are also links to the raw data reports (report) and the nightlogs for the last 7 days (NLT). The nightlog section requires a password for login.

Quick-look navigation, history, tutorial, contact. The HC plots have a link labelled scores&comments... . It has as counterpart on the QUICK-LOOK version: HC plot. By using these links, you switch forth and back between the full version with graphics, and the score version for quick-look.

scores&comments... |FULL | history ... | plot tutorial ... | contact                 daily/often; important to check     [?] *Date on this monitor changes at 21:00 UT

Full report. Some HC plots have an additional FULL version covering the whole data history. They are linked as orange square FULL.

History. This link opens the set of HISTORY plots (if available) in a separate window.

close window	history:	2003	2004	2005	2006					2007					2008					HEALTH	FULL
		...	...	...

"HEALTH" is the HealthCheck plot (focusing on the most recent data).
"FULL" is an optional link to the FULL report combining all historical data points in one plot (if available).

Tutorial. This link provides additional information about the QC items, the data, the algorithms etc. These pages are in the instrument QC section.

Contact. Send an email to the QC scientist responsible for this particular HC page.

Date change. Date on this monitor changes at 21:00 UT. This is 9 hours later than the Paranal convention changing date at 12:00 UT. This is done (1) to align with calChecker, (2) to have nighttime observations and the daytime calibrations under the same date. Effectively a new date starts shortly before the new night.

Result table

It has two parts:

• information and research
• statistical results and downloads

The first part has a link to display (e.g. for printing) the HC plot ("Print") and links to the corresponding QC1 database table for browsing and plotting ("Information and research"), sometimes with other useful links.

The second part has the result table. Find in this table more details about the applied statistics and the current trend. Here you can usually also download

Plot symbols

The symbols used in the plots are displayed in the column "Symbols". An attempt has been made to display them in the HTML pages as they are shown on the python plots. Some symbols show up here as number in brackets, like [20]. The table below lists all possible plot codes and reveals that [20] stands for a filled triangle.

Statistical functions

Average:

method
MEAN	average (with clipping if combined with certain threshold option, see below)
MEDIAN	median (with clipping if combined with certain threshold option, see below)
FIXED=val1	fixed value
NONE	nothing applied

The average values are listed in the result table (together with their units and the number of data points evaluated). They are indicated in the plots by a solid line. For MEAN and MEDIAN, they are clipped (outliers beyond the thresholds are removed after first calculation, then the calculation is repeated with the cleaned data set) if combined with certain threshold options (see below).

Thresholds:

method		implementation	clipping
VAL=val1,val2	fixed thresholds	thresholds are independent of average; val1, val2 listed in the result table	no
none	nothing applied		no

The threshold values are listed in the result table. These values are indicated in the plots by broken lines, and are used to detect and mark statistical outliers. VAL thresholding is static and might represent e.g. a specification or an alarm level based on experience or specifications.

Outliers. Statistical outliers are marked if thresholds are defined and plotted. Marking is done by a red asterisk (to become blue if the symbol color is red).

Data points outside the Y range. These data points are marked by a red arrow.

LAST data point. If configured, all data points from the last date are highlighted by a larger circle.

Data source:

method
QC1DB	QC1 database (final, certified QC1 parameters from pipeline processing by QC)

Scores and quick-look

Quick-look version. Clicking on scores&comments brings you to the score overview (QUICK-LOOK page). It displays the score result table:

FF level (scores, last 7 days up to 2010-01-31*)
	1: Med1 3|	2: Med2 2|
	4: IFU1 4|	5: IFU2 2|	6: Argus 3|

It displays scores for each HC report. Scores are flags assessing the compliance of data points with configured thresholds. A green score symbol indicates that the corresponding instrument component performs OK in the monitored time range, a red one may indicate a problem.

Each plot in the report has one score table cell. Each cell displays a set of scores. Scores are displayed for the last 7 days only, up to the specified date. This time range is aligned with the calChecker page which also has a standard depth of 7 days. Note that the specified date corresponds to the last date with an available measurement (i.e. with a processed calibration file) which is not necessarily today. If you move your mouse over a score symbol, the corresponding date will display as tool tip. If you click on a score symbol, the full score report for this product will display.

Scores are based on QC1 parameters. They are checked against an upper and lower threshold. If compliant, the score is 0, otherwise 1. An outlier is scored as 1.

All scores are aggregated to plot scores and report scores. If no outlier is found, the corresponding plot is scored OK and marked green. Outliers get a NOK score and are marked red:

2: GLOW 2| This plot has two QC data points in the monitored time range, both scored OK.   2: GLOW 2| ... a NOK score.

Comments.

The comment section has two parts:

• the (general and report) NEWS section replicated from the main HC report page
• quality comments as entered by the QC scientist upon certification

Quality comments: Plot Date* AB name score product quality comment Plot 4 2010-01-22 GIRAF.2010-01-23T06:50:12.384_tpl.ab saturated Plot 4 2010-01-23 GIRAF.2010-01-24T01:37:34.812_tpl.ab saturated Plot 4 2010-01-23 GIRAF.2010-01-24T06:02:36.832_tpl.ab ok Plot 5 2010-01-23 GIRAF.2010-01-24T07:45:59.520_tpl.ab ok "Date" links to complete AB product page; "AB" links to score report

Only those scores display here which got a comment (which usually but not exclusively refers to a red score). These comments are intended to better understand a red score.

Plot scores. If the last QC data point for a plot is found NOK, the total score for the plot is NOK and marked red (see below). If any other QC data point is found red but the last one is green, the total score for the plot is OK and marked green (since it is assumed that the problem is solved in the meanwhile).

2: GLOW 2| Both data points are scored OK. The plot score is OK.   2: GLOW 2| Same plot, with one data point scored NOK (but not the last one) . The plot score is OK. 2: GLOW 2| Same plot, with the last data point scored NOK. The plot score is NOK. 3: ALL_GLOW multiple dataset Note: Plots with multiple data sets cannot be scored.

Report scores. Total scores per plot are accumulated to report scores and displayed at larger size (see example above). Report scores can be OK, NOK or grey ("not implemented"). If no report score is red (some may be grey), the plot score is OK and displays green. A single red plot score will turn the report score also red. Reports without scoring are marked grey and are not evaluated further.

Other possible tags:

Accumulated report scores are repeated in the group navigation bar, for quick reference:

same group:	bias		dark	linearity&gain	fp-noise	contamination

Group scores. Likewise, report scores are accumulated to group scores. They are displayed in the vertical navigation bar at left:

FLAMES/GIRAFFE:
score overview
detector
grating stability
lamp stability
flat level
wavelength calib

If you want to monitor the current health of an instrument, you start with its navigation bar. Either all items are green, or you find at least a red one. In that case, follow any red score to the group level, find the corresponding report and plot scores. There you may want to check the detailed HC plot. All in all 2 levels to see the QC data point which causes the alert, and 3 to see all details.

Shiftleader's page

ALL INSTRUMENTS

Designed as the highest-level overview, it offers a monitor of all instrument scores. It comes in three versions: a snapshot of the current situation, and history overviews of the last 7 and 30 days.