Phase 3 Policies for ESO VISTA Public Surveys

Following from the agreement signed by the ESO Director General and the PIs of the VISTA public surveys, the delivery of the data products to the ESO archive from an ESO Public Survey is the responsibility of the PI, which certifies the scientific quality and accuracy of the data products.

Survey products will be delivered to the ESO archive in a format specified in the ESO Science Data Products standard [PDF], which is accessible from the top-level Phase 3 web page. Please be aware of the improvement and changes on the data interface associated with the Phase 3 upgrade; more information available at these web pages:
http://www.eso.org/sci/observing/phase3/upgrade_summary.html
and
http://www.eso.org/sci/observing/phase3/faq.html

VISTA Cycle 2

Upcoming Phase 3 Submission

The first Phase 3 submission for VISTA Cycle 2 public surveys products including OB-level images, weightmaps, and single-band source lists will be open between

1 September 2018 and 31 October 2018,

after which the data will be validated, archived and published for access by the ESO community in due course.

The PIs of the VISTA Cycle 2 public surveys are invited to submit their data products including all observations, which have been completed on a tile-by-tile basis before the reference date October 1st, 2017.

VISTA Cycle 1

According to the general Phase 3 policy for ESO public surveys, the PIs of the six VISTA Cycle 1 public surveys are invited to submit their data products based on observations that were completed six months before the opening date for submission.

  • The first Phase 3 submission of images, weight-maps and source lists from the VISTA public surveys occurred in April 2011.
  • The first Phase 3 submission of catalogue data from the VISTA public surveys occurred in May/June 2012.
  • The second Phase 3 submission of images, weight-maps and source lists from the VISTA public surveys occurred in July 2012.
  • The third Phase 3 submission including image tiles, source lists and catalogue data occured in June 2014
  • The fourth Phase 3 submission including image tiles, source lists and catalogue data occured in April-May 2016

Upcoming Phase 3 Submission

The next Phase 3 submission for VISTA Cycle 1 public surveys products including image tiles, source lists and catalogue data will be open between

1 September 2018 and 31 October 2018,

after which the data will be validated, archived and published for access by the ESO community in due course.

The PIs of the six VISTA Cycle 1 public surveys are invited to submit their data products including all observations which have been completed before the reference date October 1st, 2017.

Summary of VISTA public survey deliverables

The table hereafter lists a short description of the data product deliverables. A final release (after fifth year) is expected for all surveys.

Summary of the deliverables for VISTA Public Surveys

Survey

Deliverable #1

Tiles and confidence map

Deliverable #2

Single band source lists for each tile

Deliverable #3

Co-added deep stacked tiles

Deliverable #4

Aperture-matched multi-band catalogue

Deliverable #5

Multi-epoch catalogues / light-curves

Deliverable #6

Catalogue of Variables

UltraVISTA1 Y, J, H, Ks, NB118 yes yes yes– from the deep stack no no
VHS2 Y, J, H, Ks yes no yes no no
VIDEO Z, Y, J, H, Ks yes yes yes– from the deep stack yes– in Ks band no
VVV Z, Y, J, H, Ks yes

yes– stacking all Ks epochs obtained so far

 

yes yes yes
VIKING2 Z, Y, J, H, Ks yes yes– J band only yes no no
VMC Y, J, Ks yes yes yes yes yes

Notes:

Calibrations: Deliverables #1, and #2 have single-night based calibrations.

Calibrations: Deliverables #3, #4, and #5 must have uniform photometric and astrometric calibrations that are checked globally.

1For UltraVISTA, Deliverable #1 refers to the reduced images corresponding to the individual 1-hour integrations (ie. single OB products).

2For VHS and VIKING, the Deliverable #1, the image tile, already has the depth and S/N targeted for that pointing position on the sky. The Survey release would then provide the revised astrometry and photometry globally calibrated. It may imply an update of the metadata associated with the deliverable #1, rather than a resubmission of the FITS files for these tiles.

Content of VISTA public survey catalogues

Multi-band source catalogues including all the photometric bands relevant to each project are primary deliverables for all VISTA public surveys. Variability surveys (e.g. VVV, also VMC) deliver photometric data points as a function of epoch, a.k.a. light-curves, and specific measured or derived parameters of photometric variables, like amplitude of variation, period, object type, etc. The precise content of the catalogues to be delivered will depend on the scientific goals and exploitation possibilities of each VISTA public survey programme. The submission of further catalogue data beyond the core deliverables indicated above is most welcome during Phase 3. All data must be accompanied by the necessary documentation.

  • We request from the PI to define one unique source identifier (ID), preferentially according to the IAU recommendations for nomenclature using the schema
PREFIX hhmmss.s(s)±ddmmss.s(s)

where PREFIX denotes the survey programme. The consistent definition of (IAU) source names throughout related publications is important to ESO archive users because it enables the efficient exploitation of the data using the catalogue query interface.

In case of degenerate catalogues, where IAU names are not guaranteed to be unique, the unique source ID should be defined in terms of a positive integer number (>0) within a dedicated column. The IAU name should be included in an additional column.

The source ID serves as a reference key and must be recorded in any catalogue that contains measurements for the given source, including multi-epoch photometric data collections (light-curves) and catalogues of variables.

  • Any source catalogue must include Right Ascension and Declination in decimal degrees (J2000) to specify the celestial position of each source.
  • Source positions, apparent magnitudes and colours should refer to the survey-wide (a.k.a. global) astrometric and photometric system established by cross-calibrating the available data using overlapping tiles and across different bands. This implies that magnitudes and colour indices should be corrected for seeing variations across different observations. Single-band catalogues, termed source lists in the context of Phase 3, usually do not qualify for submission as survey source catalogue.
  • The catalogue columns that contains the best estimate of the total flux for a point-like source should be indicated in the release description. The detailed nature of the total flux estimator depends on the scientific goals of the particular survey.
  • Multi-band source catalogues should generally contain colour indices for each source in order to facilitate efficient queries including colour constraints. The survey team is free to identify suitable colours according to their scientific objectives. The colour definitions should be documented in the data release description associated to each catalogue.
  • If some data fields within a catalogue are missing i.e. there is no valid measurement it must be encoded in terms of special values, in particular using the IEEE NaN ('Not a Number') for floating point type of data.

Large Area Surveys

For the large area surveys (VHS, VIKING, VVV, VMC) the catalogue data should be submitted in a tile-by-tile fashion, meaning that the entire catalogue is segmented into N separate FITS data files, where each file corresponds to one VISTA survey tile, plus one metadata file describing the entire catalogue.

Multiple detections of the same source, which occur in the overlap regions of adjacent survey tiles, are expected to be merged in the final multi-band photometric catalogue. Unless the uniqueness of the sources in the band-merged catalogue is being guaranteed, duplicate entries should be consistently flagged throughout the entire catalogue by including a dedicated column ('PRIMARY_SOURCE'), which is set to 1 if the source is a primary one and to 0 otherwise.

De-reddened magnitudes, i.e. corrected for galactic extinction, based on the survey team’s best model should be listed in the VVV and VMC source catalogue in addition to the uncorrected (apparent) magnitudes.

Deep Surveys

UltraVISTA is expected to submit one multi-band source catalogue (1 file) collectively covering the deep and ultra-deep regions of the survey. In case of VIDEO one multi-band source catalogue (1 file) per survey region (VIDEO-XMM3, CDF-S, ELAIS-S1) is delivered.

 

Processing provenance

Data providers are requested to include consistent and complete processing provenance information in each science data product file (cf. ESO SDP Standard, Issue 5, Sect. 2.4). PROV references should point to the precursor science data files which have been included into the generation of the given product provided that the referenced files already exist in the ESO archive or they are being submitted as part of the same data release. The correctness of PROV pointers is an important criterion for data product acceptance.