Introduction to CPL
Like all major astronomical facilities, the telescopes operated by ESO require a large amount of automatic data processing in order to put it into a form that it can be easily dealt with by astronomers. This data processing is usually carried out by complex software that implement sophisticated algorithms and deal with vast quantities of data. They also typically run on "remotely" located hardware, unattended for the most part by human operators. As such, the requirement for robustness, automation and precision is of the utmost importance.
Due to the huge scale of ESO's Paranal and La Silla operations, there are many instruments, collecting data at a multitude of different operating modes; from fibre- and slit-spectrographs to imagers to interferometry. Most of these require a software pipeline to handle their initial data processing requirements. Currently, such pipelines are operational for fifteen instruments. Initially they have been written on the basis of three different software packages leading to considerable overlap and duplication of functionality. However, with the growing number and complexity of ESO instruments, the need to consolidate the common functionality of these pipelines became clear quickly.
History of CPL
In June 2001, Ncholas Devillard and Ralf Palsa first proposed a common software library in order to ease and accelerate the development efforts for the different VLT instrument pipelines. This software library, named the Common Pipeline Library, would essentially be built up from already existing code. In particular, the Eclipse library (used for ISAAC and NACO pipelines) and concepts of the VIMOS data reduction software would be the main pillars of the CPL software.
In September 2001, Michele Peron formed a CPL project team, consisting of
Nicholas Devillard and Yves Jung (working for ISAAC,
NAOS/CONICA),
together with Ralf Palsa and Carlo Izzo (working for VIMOS, FORS1/2),
as well as Pascal Ballester and Cyrus Sabet from the VLTI pipeline
project. Klaus Banse served as mediator and chairman. In 2002, Nicholas Devillard
left and
Michael Kiesgen [until June 2004] became a member of the team in the context
of his work on the Giraffe pipeline.
Finally, Lars Lundin,
Andrea Modigliani and Derek J. McKay [until April 2005] joined the CPL
team in the course of the year 2003. In 2004 came
Nuria P. F. McKay [until late 2004] and
Jonas Møller-Larsen became a team member in 2005 [until late 2007].
In 2006 Sandra Castro replaced Ralf Palsa in the CPL group.
In July 2008 Yves Jung left the CPL team and his place was
taken up by Lander de Bilbao [until December 2010]. In 2009 Sandra Castro and Ralf Palsa switched again
places in the team.
Currently, Ralf Palsa, Enrique Garcia and Lars Lundin, together with Klaus Banse, make up the CPL team in ESO's SDD/PSD department.
A preliminary version of the CPL was released in May 2002. Building on this basic version, the first official release of the CPL was made available to the public by ESO in December 2003, version 2.0 in April 2005. Since then the CPL is ready for distribution to consortia for pipeline development. The most recent version of CPL is version 5.3.1 released in April 2011.
The CPL Project
As mentioned above, the CPL group of ESO has been working on this "Common Pipeline Library" (CPL) with the aim to increase the functionality available to pipeline developers, to provide a more rigorously-tested set of functions within this available set and, hence, ease the development and maintenance load on the software developers who are supporting the ever increasing family of ESO instrumentation.
The Common Pipeline Library consists of a set of ISO-C libraries,
which have been developed to standardise the way VLT instrument
pipelines are built, to shorten their development cycle and to ease
their maintenance.
The CPL was not designed as a general purpose image processing library (although it accomplishes that task too) but, rather, its design reflects two primary ESO requirements. The first of these is to provide an interface to the VLT pipeline runtime-environment. The second, is to provide a set of software tools, which make possible the rapid construction of astronomical data-reduction tasks.
The CPL provides:
- many useful low-level data types (images, tables, matrices, strings, property lists, ...),
- medium-level data access methods (a simple data abstraction layer for FITS files),
- table organisation and manipulation,
- keyword/value handling and management,
- a standardised application interface for pipeline recipes, and,
- support for dynamic loading of recipe modules (using programs such as EsoRex).
Despite the current bias towards instrument pipeline development, the library also provides a variety of general-purpose image- and signal-processing functions, making it an excellent framework for the creation of more generic data-handling packages.
Installation
Full instructions on how to obtain, build and install the CPL are given on the CPL downloads page.Note that even if this page is kept up to date as much as possible, it might not be fully applicable for a particular release. Patch releases are particularly susceptible to this. Users are therefore advised to read the installation instructions delivered with the CPL distribution that they download or otherwise obtain. These release specific instructions can be found in file README located in the top level directory of the unpacked CPL source tree.