Common DFOS tools:
Documentation
dfos = Data Flow Operations System, the common tool set for DFO
  make printable v4.0:
- enabled for OPSHUB environments		 see also:
- link to HELP page
 




[ used databases ] databases none
[ used dfos tools ] dfos tools called by processAB
[ output used by ] output monitor page: $DFO_MON_DIR/status_<date>.html, linked to dfoMonitor;
[ upload/download ] upload/download download: historical scores and comments from qcweb server
javascript javascripts jquery-latest.js, jquerydataTables.js under ${JSCRIPT_URL}
topics: description | summary&project information | condor section | recursive mode
Features: dependencies | calibration map | scores | earlier/later labels | products | CalSelector
Result table: navigation | navigation, sorting and filtering | usage | configuration

getStatusAB (OPSHUB environment)

This is the tool for the AB monitor. This documentation refers to the OPSHUB environment. Find the documentation for the DFOS and PHOENIX environment here.

You may also want to refer to the quick help provided by the HELP link on the AB monitor.

OPS
HUB		 this page applies for the OPSHUB environment

[ top ] Description

This tool creates the 'AB monitor'. The AB monitor provides a graphical overview of the ABs (Association Blocks = processing jobs), their content and their status. The tool is called by the workflow tool distillery. The AB monitor pages are linked to the projects monitor (dfoMonitor).

The tool supports CALIB and SCIENCE data in the same way. It has information about the current project, and displays historical QC scores and certification information (collected from the qcweb server).

The tool scans all ABs for the specified DATE and PROJECT under $DFO_AB_DIR. It displays the process status, the recipe, the RAW_TYPE, and the setup keys. It links to available products, association logs, processing logs, and finally to the fits products.

In CONDOR mode, it displays the queue status.

[ top ] Summary and project information. In addition to the AB specific information, there is also global information about the number of all/successful/failed ABs and about the currently configured pipeline version:

number of ABs (all | success | failed | created): 10 | 10 | 0 | 0         pipeline version: giraf-2.16.3

There are all kinds of links to data directories for the specific instrument chosen for the PROJECT, and the project definition for the PROJECT itself (here: for GIRAFFE_PROJECT2):

GIRAFFE quick links (book-keeping): AB | JOBS | DFS_LOG | MONITOR | CONFIG
Data directories for GIRAFFE: RAW | CAL | PRODUCTS
Local OCA rules for GIRAFFE: CREATE_CASCADE | CALSELECTOR

PROJECT log file: GIRAFFE_PROJECT2
Project definition for GIRAFFE_PROJECT2:

PROJECT_NAME INSTRUMENT MODE DATE SELECT AB_METHOD CALSEL_MODE OCA_RLS_METHOD ACCEPT_060
GIRAFFE_PROJECT2 GIRAFFE SCIENCE 2018-06-19 ANY CALSELECTOR MASTER GIRAFFE_CALSELECTOR_rules NO

If a project has more than one date, then there is one line per configured date, making cross-navigation between dates easy.


[ top ] Condor section (DRS_TYPE=CON only)

Condor jobs for opshub1@godot:
2013-02-15 09:44:17 0 jobs; 0 completed, 0 removed, 0 idle, 0 running, 0 held, 0 suspended
Last cascade:
/qc/opshub1/condor/CALIB_2013-02-13-1360863054.70915699
 		Dagman for opshub1@godot:
No job running.

The condor section has, in the first field, an overview of the current condor jobs for $PROJ_INSTRUMENT, a link to the last executed cascade, and links to error files (if any) in the condor job directory.

The second field has an overview of the currently active condor jobs for $PROJ_INSTRUMENT.

For DRS_TYPE=CPL and INT the condor table and the dagman table are disabled.

[ top ] Recursive mode. With CONDOR, the tool executes in recursive mode (option -r). It is then looping in the background (running every 60 seconds).

[ top ] Features: dependencies | calibration map | earlier/later labels | historical scores | marking of calSelector issues for science ABs | calibmap links | product links | browser refresh

The AB monitor has several features which are described in the following.

[ top ] Dependency marking. Those ABs which are contained in other ABs in $DFO_AB_DIR are marked by a little blue square:

   GIRAF.2010-08-27T13:32:41.034_tpl.ab CAL11 compl. OK   gimasterflat FFLAT Medusa2_H665.0 OK P_LOG! 2.8 products   (0/6) OK

With this marking it is easier to see if the failure, or rejection, of an AB might have consequences for other ABs.

[ top ] Calibration map. The RAW_TYPE is linked to the DFOS_OPS calibration map overview page on the QC web pages.

[ top ] Earlier/later labels. In the OPSHUB environment, a date is defined either by the SCIENCE data, or (if the project is entirely about CALIB data) by the project date. In particular for SCIENCE projects, this means that the associated CALIB ABs could come from earlier dates, the SCIENCE date, or from later dates. If earlier or later, these ABs are marked as such, for easier overview.

[ top ] Historical scores and certification comments. These two columns collect the scores and comments as downloaded from the QC web server, for easy reference. Note these scores and comments are NOT created by the distillery tool. They are available only for CALIB data.

[ top ] Science ABs: marking of calSelector issues. If created by the AB_METHOD CALSELECTOR, the SCIENCE ABs have also links to the CalSelector XML (X) and text files (T).

If the CalSelector association has a 'false' flag in its XML file (for completeness issues), this is highlighted by the tool with the X with a red background. If the result flag is 'true' (Raw2Master and complete) the background is green. The text file has a red background if CalSelector has switched to Raw2Raw instead of Raw2Master, otherwise it is green.

[ top ] Links to products. The column "PRODUCTS" has a link to $DFS_PRODUCT/$PROJ_INSTRUMENT/$RAW_TYPE/$DATE, the place where the pipelines deliver their products. You can inspect them here, or apply your analysis scripts.

[ top ] Browser refresh. The output page usually refreshes itself within 60 seconds. This is useful for incremental processing or during certification when the page is rather dynamic. This behaviour can become annoying if you want to investigate an issue, e.g. during certification. You can then turn this behaviour temporarily off, by clicking

browser_refresh: on (every 60 sec | stop | on); tool_refresh: off

[ top ] Navigation. There is top/bottom navigation.

  CAL report NLT | QC processing 

The CAL link refers to the historical calChecker page for that date (which probably has some explanation if there is an association issue, otherwise ignore). 'report' links to the data report on the qcweb server, NLT is the night log link. The 'QC processing' links to the historical AB monitor for this specific instrument and date on the QC web server. This is the complete overview of all processed calibration ABs for that date, and of all associated (not processed) science ABs. This overview might differ from the current project page but is sometimes useful for comparison.

[ top ] Table sorting. This feature allows comfortable client-side sorting of an HTML table. It is provided by the java script 'dataTables' and is provided by the jQuery library (http://tablesorter.com/docs/). Both are installed under http://www.eso.org/observing/dfo/quality/ALL/jscript (find more technical hints there). Click on any of the main table columns and get the whole table sorted instantaneously. You can sort in asc or desc directions. For a few columns the sorting is disabled, for obvious reasons. Possible applications are quick overviews of all ABs per setting, or per recipe, per RAW_TYPE etc.

Filtering and searching. You can do two kinds of search. For the normal search, enter a string or a combination of strings. For the dependency search, use the column INDEX. Its values come in two flavours: SCI and CAL. For the dependency search, enter e.g. SCI01? and see all ABs displayed which contributed to SCI01, plus SCI01 itself. The question mark stands for "left dependencies", i.e. all ABs which are organized left of the science AB in the cascade (parent ABs) (see here). Search for parent ABs also works for CAL ABs: e.g., find a standard star AB and enter its INDEX with '?' to see all its parent ABs. Child ABs (depending ones, "right" dependencies) are displayed if you enter an exclamation mark (e.g. CAL10!).

INDEX searches work on the full data set displayed on the page, but not beyond.

For these kind of searches it might be desirable to turn off the automatic browser refresh.

Below is an overview of all columns of the AB monitor. All have tooltips on hovering; most of them are sortable, marked by .

  BQS AB NAME INDEX COMPL. AB LOG RECIPE RAW_TYPE SETUP AB STATUS P LOG T_EXEC
PRODUCTS SAME DATE? HISTORICAL
SCORE 		HISTORICAL
CERTIFICATION
created by/ retrieved from vultur_exec_cascade (CONDOR script) createAB, or downloaded from qcweb   AB key COMPLETENESS createAB, CalSelector AB key RECIPE AB key RAW_TYPE createJob processAB; AB key PROCESS_STATUS processAB, esorex processAB, esorex; AB key TEXEC processAB getStatusAB downloaded from qcweb server downloaded from qcweb server
linking to $DFO_AB_DIR     association log; SCIENCE: xml, txt files, all under $DFO_AB_DIR;   calibration cascade on www.eso.org/qc     processing log, under $DFS_LOG   product fits files in $DFS_PRODUCT area   $DFO_AB_DIR/<ab>.html and <ab>.tlog  

[ top ] Usage. The tool is automatically called from processAB and distillery.

It can also be called from the command line. Then use the syntax displayed in the upper left box ("Refresh:"), e.g. 'getStatusAB -d 2018-05-24 -p GIRAFFE_PROJECT2'.

As with all dfos tools, type

getStatusAB -h for on-line help,

getStatusAB -v for the version number.

[ top ] Configuration

The tool configuration is done entirely within distillery, do not touch.

[ top ]
Last update: April 26, 2021 by rhanusch