Niall Gaffney (STScI), Stefano Casertano (STScI), Brian Ferguson (STScI)
Abstract
The Hubble Legacy Archive (HLA) is a project to add value to the Hubble
Space Telescope data archive by producing and delivering science-ready
drizzled data products and source lists derived from these products.
Initially, ACS, NICMOS, and WFCP2 data were combined using
instrument-specific pipelines based on scripts developed to process the
ACS GOODS data and a separate set of scripts to generate source
extractor and DAOPhot source lists.
We present the re-engineered pipeline based on existing and improved
algorithms with the aim of improving processing quality,
cross-instrument portability, data flow management, and software
maintenance. The new pipeline, initially designed for WFC3 data,
isolates instrument-specific processing and is easily extendable to
other instruments and to generating wide-area mosaics. Significant
improvements have been made in image combination using improved
alignment, source detection, and background equalization routines. It
integrates improved alignment procedures, better noise model, and source
list generation within a single code base. Wherever practical, Pyraf
based routines have been replaced with non-IRAF based python libraries
(e.g. Numpy and pyFits). The data formats have been modified to handle
better and more consistent propagation of information from individual
exposures to the combined products. A new exposure layer stores the
effective exposure time for each pixel in the sky which is key in
properly interpreting combined images from diverse data that were not
initially planned to be mosaiced. We worked to improve the validity of
the metadata within our FITS headers for these products relative to
standard IRAF/Pyraf processing. Any keywords that pertain to individual
exposures have been removed from the primary and extension headers and
placed in a table extension for more direct and efficient perusal. This
mechanism also allows for more detailed information on the processing of
individual images to be stored and propagated providing a more
hierarchical metadata storage system than key value pair FITS headers
provide. In this poster we will discuss the changes to the pipeline
processing and source list generation and the lessons learned which may
be applicable to other archive projects as well as discuss our new
metadata curation and preservation process.Paper ID: P046
Poster Instructions
|
