Phase 3 Release Validator

On 16 September 2013 the Phase 3 system has been upgraded including a new version of the Phase 3 release validator. Please update to the new version (v2.0.0) if you downloaded the Phase 3 release validator before.

Local validation of the Phase 3 release is an essential preparatory step before uploading the data to ESO. In this process the internal consistency and the metadata content of the data products will be checked against the ESO/EDP data format standard. For this purpose ESO provides a dedicated Java tool–the Phase 3 Release Validator.

Please make sure to obtain the latest versions of the Phase 3 validator (v2.0.0) and fitsverify (v4.16) for data release preparation.

Software Download

The current Phase 3 validator, version number 2.0.0, implements the ESO Science Data Product Standard, Issue 5, Date: 11/01/2013.

The application version can be checked using the following command:

$ java -jar ./validator.jar -v
Starting up validation tool.
application.name = Phase 3 Release Validation Tool
application.command = java -jar validator.jar -r<reldir> -m{create|update} [other-options]
application.version = 2.0.0

Change History

Main changes in Phase 3 validator v2.0.0 with respect to the previous version v1.1.1:

  • Phase 3 validator v2.0.0 supports recording the processing provenance of data products using one dedicated FITS binary table extension (cf. ESO SDP Standard, Issue 5, Sect. 2.4.2, for details);
  • Performance improvements;
  • Bugs fixed;
  • Requires Java 7 (see below: System Requirements)

Previous Versions

Main changes in Phase 3 validator v1.1.1 with respect to the previous version:

  • Phase 3 validator v1.1.1 supports the submission of science catalogue data;
  • Support the submission of the Phase 3 release description as PDF document;
  • Improved checks on FITS keywords and their content to identify problems as early as possible in the validation process;
  • Support for PRODCATG tokens in uppercase;
  • User documentation updated (see below)

System Requirements

The validator has been developed and tested under Scientific Linux 5.5 (i386, 32-bit, based on Red Hat Enterprise 5). It requires Java version jre1.7.0_21 or higher. The validator is expected to run under Linux; other platforms are not supported at the moment.

FITSVERIFY

The FITS file format-verification tool fitsverify, which is maintained and distributed by NASA/HEASARC, must be installed on the system running the Phase 3 validator. Phase 3 data must pass fitsverify without any warning to be accepted. fitsverify can be obtained from http://heasarc.gsfc.nasa.gov/docs/software/ftools/fitsverify/

The Phase 3 infrastructure employs fitsverify version 4.16, which is currently the lastest version (October 2010). It is not recommended to run the Phase 3 validator in combination with any other tool or software version, because it was not tested and one cannot assure that everything would work correctly.

This binary is compiled for Linux (i386, 32-bit) and would not work on other operating systems like Mac OS X. You may download the source code from

http://heasarc.gsfc.nasa.gov/docs/software/ftools/fitsverify/

and compile it for your operating system. We cannot assure though that everything would work, as the tool was not tested for different operating systems.

Network connectivity

The validator must be able to establish a network connection to the ESO Phase 3 system at runtime.

Calling Sequence

Run the validator from the command line:

java -jar validator.jar -r <dir> -m create|update

Output

The validator displays a summary of the release content and produces one of the following status messages, either

OK - RELEASE CAN BE UPLOADED TO ESO

or

ERROR - PLEASE FIX THE ERRORS BEFORE UPLOADING THE RELEASE

The validator produces three files in the current working directory:

  1. validator.error - list of short error messages describing the errors found in validating the release. This file is empty if no errors are found.
  2. validator.toc - contains the status (release ok/error), a summary of the release in terms of number and type of validate files, number and type of errors found, number of parsed datasets and provenance information, and a dump of the parsed metadata for datasets, provenance, and product categories.
  3. validator.log - detailed list of messages generated during the validation.

User Documentation