make printable
Common DFOS tools:
|
| dfos = Data Flow Operations System, the common tool set for DFO |
| make printable |
Note: The tool is distributed through the utilities package (utilPack). |
This tool checks for inconsistencies between the dp_headers and the data_products tables in the observations database. Such inconsistencies may rarely occur if e.g. the replication process is broken. Since the result of a scan of dp_headers ("header download") is used as reference list for fits data downloads, a file missing in dp_headers will also not be downloaded as fits file.
The tool can also check (if called with option -C) the existance of all headers in the DFO_HDR_DIR directory. It is called with that option at the begin of the certifyProducts workflow. It can then not only catch missing headers, but also call 'autoDaily -D' and thereby createAB, processAB and processQC the missing ABs, without creating unnecessary overheads and inconsistencies with the already processed ABs for that date.
It takes into account the dataclient behaviour for the last "broken" template (which is moved to the next date) and does not fire false alerts if it finds those headers.
Type checkConsist -h for on-line help, checkConsist -v for the version number, and
checkConsist -d 2024-06-10
to start checking date 2024-06-10, and
checkConsist -d 2024-06-10 -C
to extend the check to the downloaded headers (operationally this is done within certifyProducts).
The tool comes as part of utilPack. Use dfosExplorer, or dfosInstall -t utilPack to install.
None.
1. observations..dp_headers is scanned. All files within the specified date are searched, $DFO_OFFSET is taken into account.
2. observations..data_products is scanned. All files within the specified date are searched, $DFO_OFFSET is taken into account.
3. Both file lists are compared, inconsistencies are written into $TMP_DIR/con_mail and emailed to $OP_ACCOUNT as an alert. The email is only sent if the problem is detected for the first time (to avoid hourly repetitions since this tool is usually embedded in autoDaily).
The problem fix has to be initiated at a case-by-case basis.
With option -C, there is an additional test of the results of the dp_headers scan versus the content of the DFO_HDR_DIR directory.