Ignacio Vera (ESO), Adam Dobrzycki (ESO), Myha Vuong (ESO), Cristiano da Rocha (ESO)
Abstract
The ESO Archive holds ca. 20 million FITS files: raw observations taken at the La Silla Paranal Observatory in Chile, data from APEX and WFCAM telescopes, pipeline-processed data generated by the Quality Control and Data Processing Group in ESO Garching and, since recently, reduced data delivered by the PI's through the ESO Phase 3
infrastructure.
A metadata repository has been developed by the ESO Archive, to hold all the FITS file headers with up-to-date information using data warehouse technology. Presently, the repository contains more that 10 billion keywords from headers of all ESO FITS files. We have added to the repository a mechanism for keeping track of header ingestion and modification, allowing to build incremental applications on top of it. The aim is to provide a framework allowing for creation of fast and good quality metadata query services.
We present hFits, a tool for data publishing allowing for metadata enhancement. The tool reads from the metadata repository and inserts the metadata into the conventional relational database systems using a simple configuration framework. It utilises the metadata repository tracking mechanism to incrementally refresh the services and
transparently propagate any metadata updates. It supports the use of user defined functions where, for example, WCS coordinates can be calculated or related metadata can be extracted from other information systems, and for each new header file it provides to the archived file the access attributes following ESO data access policy, publishing the
data to the community.
Paper ID: P156
Poster Instructions
|
