VLT Instrument Pipelines

Latest News:

Feb 22, 2023: ERIS pipeline v1.5.3 is availble

More information on the ERIS release is provided here.

Nov 17, 2023: MATISSE pipeline v2.0.0 is availble

More information on the MATISSE release is provided here.

Oct 23, 2023: KMOS pipeline v4.4.1 is availble

More information on the KMOS release is provided here.

Oct 20, 2023: UVES pipeline v6.4.1 is availble

More information on the UVES release is provided here.

Sep 29, 2023: Updated ESPRESSO pipeline

More information on the ESPRESSO release is provided here.

May 31, 2023: P-111 Instrument Pipeline Packages released!

For period 111 updated versions of all Instrument Pipeline Packages are now available.

  • For the installation of RPM packages (Fedora 35 to 37, CentOS 7 and Scientific Linux 7) continue here
  • For the installation of MacPorts packages continue here
  • For the source kit installation go here

Known Issues:

  • Please note that pipeline packages including EDPS workflows are not available for CentOS 7 and
    Scientific Linux 7 because the required software dependencies cannot be fulfilled on these platforms.
  • Please note that for macOS 13 in genearal and Apple Silicon (M1, M2) based Mac hardware MacPorts
    packages are provided through MacPorts source installations. For technical reasons binary packages
    cannot be provided for these configurations. Except for a small time overhead during the installation,
    this should have no impact on the user experience.
  • EsoReflex will not work with Java 17 or newer. With Java 17 becoming the default Java installation
    on recent systems (e.g. Fedora 37), please make sure that on those systems you also have Java 11
    installed for use with EsoReflex! For RPM or MacPorts installations of ESO pipeline packages this
    dependency will automatically be satisfied. For source kit installations Java 11 must be available
    before pipeline kits are installed.


To get announcements of new pipeline releases, send an email to eso-pipelines-announce-join@eso.org.

To receive the ESO Science Newsletter subscribe here.



Getting Started

  1. Install the pipeline(s) using RPM packages, MacPorts packages or Source Kits.
  2. Detailed usage information can be found in the individual user manuals and tutorials available here

Four Ways to Run a Pipeline

  1. EsoReflex is the recommended environment to reduce ESO data. It automatically organizes input files according to their category and runs the entire reduction chain at the push of a button. It supports break points in the reduction sequence in order to inspect and interact with intermediate and final products and rerun the corresponding step if necessary.
  2. Gasgano is a Java-based data file organizer developed and maintained by ESO. It can be used to manage and organize in a systematic way the astronomical data observed and produced by all VLT compliant telescopes. Gasgano is a graphics-based software tool for organising and viewing data files produced by the VLT Control System. Gasgano offers functionalities for data viewing, grouping, sorting, classification, searching, and filtering of data. And, of course, Gasgano will run any requested CPL recipe on the selected data.
  3. EsoRex, a command-line utility for running pipeline recipes is also available. It may be embedded by users at their home institute into data reduction scripts for the automation of processing tasks.
  4. EDPS is the next generation ESO data processing environment, which will eventually replace EsoReflex. At this time EDPS is an early release version used for quality control at the observatory and comes without a GUI. This, however, will change in the future. A subset of pipelines provide already support for EDPS. These pipelines are marked in the table below. Please follow the link there for information about EDPS and how to use it.
The underlying algorithms and recipes are the same for a given instrument pipeline, irrespective if EsoReflex, Gasgano or Esorex is used.

Installation Instructions

Fedora, CentOS, Scientific Linux and macOS

Currently ESO provides binary packages for:

  • Fedora (35 to 37)
  • CentOS 7
  • Scientific Linux 7
  • macOS (11, and 12; Intel hardware only)
Users of those systems are encouraged to install the Instrument Pipeline packages using the RPM packages or the MacPorts packages respectively.
Please follow the installation instructions for RPM packages or MacPorts packages.

ESO Pipeline Packages for Ubuntu/Debian systems

For Ubuntu/Debian systems a subset of pipeline packages is provided by the Debian Astro project. For Debian systems the pipeline packages are available directly from the official repositories using the Debian package managing tools. For Ubuntu systems the ESO pipeline packages are available in the universe repository, which may have to be enabled first. Please refer to the system documentation or Ubuntu forums on how to setup the repository. For both, Debian and Ubuntu, the ESO pipeline packages can be found by querying the repositories for package names starting with cpl-plugin.

Please note that ESO does not directly support these packages. In case of problems with these packages please report issues to Debian and Ubuntu using their normal bug reporting channels. ESO also has no influence on which instrument pipeline packages or which version of a pipeline package are made available by the Debian Astro project. Naturally these packages will only be created after ESO publicly releases a new pipeline version. Consequently, one should not expect that the most recent pipeline version offered through this website is instantly available in the Debian and Ubuntu repositories.

Windows WSL

While an installation of ESO pipeline packages using the Windows Subsystem for Linux (WSL) is not officially supported, it is possible to use it for running ESO pipelines also on Microsoft Windows systems. For details and best effort support go here.

Source Kit package installation

Users of operating systems for which no RPM or MacPorts packages are currently available should follow the instructions in this section.

Pre-requisites

Please consult the Software Prerequisites page before installing the pipelines or EsoReflex from source.

If you use conda environments it is strongly recommended that you disable any active conda environment prior to running any of the two installer scripts described in the following sections!

EsoReflex based Pipeline Installations

For an installation of pipeline packages with EsoReflex support please follow the instructions in this section.

  • Download the install_esoreflex script.

    Important: As of November 2021 there is a new version of the script. Please make sure you use that one.

  • If necessary, make it executable: chmod u+x install_esoreflex
  • Execute the command and follow the instructions: ./install_esoreflex
  • If you make use of any of these workflows in a scientific publication, please consider citing EsoReflex. You may reference the following paper: Automated Data Reduction Workflows for Astronomy: The ESO Reflex Environment, 2013, Freudling, W., et al., Astron.Astrophys., 559, A96 (Bibtex entry)

Installations without EsoReflex

If you want to install pipeline packages without EsoReflex support please follow the instruction in this section. It applies to all pipeline packages.

  • Download the desired pipeline kit(s) from the Source Kit column in the table below into an empty, temporary directory.
  • Download the Source Kit installer into the same directory.
  • If necessary, make it executable: chmod u+x install_pipelinekit
  • Execute the command: ./install_pipelinekit *-kit-*.*
  • After the installation the temporary directory with its contents can be deleted.

Instrument Pipeline Packages

Instrument Release Notes Source Kit User Manual Cookbook Additional Documents Additional Datasets EsoReflex Tutorials/Cookbooks EDPS workflow Status
AMBER 2023-05-19 4.4.4 4.4.4
End of maintenance
CR2RES 2024-02-08 1.4.2 1.4.2
Tutorial: 1.4.2
Demo Data: 1.2.0
Active
CRIRES 2023-05-19 2.3.17 2.3.17
Cookbook
End of maintenance
DETMON 2023-05-10 1.3.13 1.3.5
Operational on hold
EFOSC 2023-04-24 2.3.9 1.1
Demo Data
Tutorial: 2.3.9 (EFOSC-IMG)
Tutorial: 2.3.9 (EFOSC-MOS)
Demo Data: 0.1
End of maintenance
ERIS 2024-02-20 1.5.3 1.5.3 (ERIS-NIX)
1.5.3 (ERIS-SPIFFIER)
Tutorial: 1.5.3 (ERIS-NIX)
Tutorial: 1.5.3 (ERIS-SPIFFIER)
Demo Data: 0.5
Active
ESOTK 2023-02-03 0.9.6 0.9.6
Tutorial: 0.9.6
Demo Data: 0.9
Active
ESPRESSO-DAS 2023-04-01 1.3.7 1.3.7
Tutorial: 1.3.7
Demo Data: 0.9.5
Active
ESPRESSO 2023-09-28 3.1.0 3.1.0
Static Calibration WAVE & DLL Matrixes (432 MB)
Tutorial: 3.1.0
Demo Data: 0.6.0
yes Active
FORS 2023-05-17 5.6.5 5.17
Demo Data (29 MB)
Tutorial: 1.6 (FORS-IMG)
Tutorial: 1.9 (FORS-PMOS)
Tutorial: 1.27 (FORS-SPEC)
Demo Data: 0.9
Operational on hold
GIRAFFE 2023-04-27 2.16.11 2.16.8
Cookbook
Tutorial: 1.4
Demo Data: 0.3
Operational on hold
GRAVITY 2022-10-12 1.6.0 1.6.0
Tutorial: 1.6.0
Demo Data: 0.8
Active
HARPS 2023-02-03 3.0.0 3.0.0
Static Calibration WAVE & DLL Matrixes (17 MB)
Tutorial: 3.0.0
Demo Data: 0.1.0
Active
HAWKI 2023-05-10 2.4.13 2.4.13
Tutorial: 2.4.13
Demo Data: 0.5
Operational on hold
ISAAC 2023-05-19 6.2.4 6.2.4
Static Calibration Files (50 MB)
End of maintenance
KMOS 2023-12-04 4.4.2 3.6
Tutorial: 5.1
Demo Data: 2.2.0
yes Active
MATISSE 2023-11-17 2.0.0 2.0.0
Tutorial: 2.0.0
Demo Data: 0.4
Active
MIDI 2023-05-19 2.9.5 2.9.5
End of maintenance
MOLECFIT 2023-05-10 4.3.1 4.3.1
Tutorial: 4.3.1
Demo Data: 1.5
Active
MUSE 2023-05-31 2.8.9 2.8.9
MUSE IFU 6 trace tables
Leagacy MUSE static calibrations
Tutorial: 17.0 (muse)
Tutorial: 5.0 (muse-zap)
Demo Data: 1.6
Active
NACO 2023-05-19 4.4.12 4.4.12
End of maintenance
NIRPS 2023-09-05 3.0.4 3.0.4
Static Calibration WAVE & DLL Matrixes (62 MB)
Extra raw calibrations (7,5GB)
Tutorial: 3.0.4
Demo Data: 0.2.0
Active
SINFONI 2023-05-19 3.3.4 3.3.4
ADA IV 2006 paper
Calibration Database Example (255 MB)
Demonstration Package (1.2 GB)
Tutorial: 3.3.4
Demo Data: 0.3
End of maintenance
SOFI 2023-03-08 1.5.15 1.2
End of maintenance
SPHERE 2023-05-10 0.47.1 1.47
Tutorial: 1.3 (ifs)
Tutorial: 1.3 (ird-img)
Tutorial: 1.2 (ird-lss)
Tutorial: 1.2 (zpl-img)
Tutorial: 1.3 (zpl-pol)
Demo Data: 1.5
Active
UVES 2023-10-20 6.4.1 6.4.1 (UVES)
6.4.1 (UVES-FIBRE)
Demonstration Package (2.0 GB)
Old Standard Star Calibration Data
Tutorial: 6.4.1 (UVES)
Tutorial: 6.4.1 (UVES-FIBRE)
Demo Data: 4.5
yes Operational on hold
VIRCAM 2023-05-19 2.3.12 2.3.12
Tutorial: 2.3.12
Demo Data: 0.1
End of maintenance
VIMOS 2023-05-19 4.1.8 4.1.8
Tutorial: 4.1.8 (VIMOS-IFU)
Tutorial: 4.1.8 (VIMOS-IMA)
Tutorial: 4.1.8 (VIMOS-MOS)
Demo Data: 0.5
End of maintenance
VISIR 2023-05-17 4.4.4 1.11
Tutorial: 1.6
Demo Data: 0.4
Operational on hold
XSHOOTER 2023-05-10 3.6.3 3.6.3
Additional NIR telluric model calatog (190 MB)
Tutorial: 3.6.3
Demo Data: 1.3
Operational on hold

Pipelines Status

The pipeline status determines the level of ESO support in fixing the software. For any help with installation and usage of a pipeline, please contact the helpdesk ticketing system. ESO internal (operations) users should request support from the pipeline developers team using JIRA. Here are the possible entries:
  • Active: Pipelines that are affected by a data product upgrade are considered active in the sense that a project team is actively working on them. These pipelines are fully supported and have the highest priority for support. The support consists in prompt evaluation of all requests.
  • Operational on hold: Pipelines that are on hold are not under active development and are of lower priority for support. Pipelines are put into this status when they are considered to be essentially completed, i.e. no enhancements or upgrades are currently under way. The software support consists in porting these pipelines to new CPL versions, carrying out emergency repairs and fixing trivial problems. The User Support Department will evaluate and answer if possible the requests for help received via the helpdesk ticketing system, but by default there will be no support available from pipeline developers.
  • End of maintenance: Pipelines in this status are the lowest priority for support. The only work performed on these pipelines is to make sure they can still run in the current ESO environment. The User Support Department evaluates and answers if possible the requests for help received via the helpdesk ticketing system. Tickets reporting problems on the pipeline will not receive any software support.

 

Additional tools for data visualisation and analysis.
Package
Release Information Release File
Includes CPL,Gasgano,EsoRex,
and static calibration data
Users Manual and Documentation Data Sets
Sky Correction and Modeling Tools Molecfit (legacy versions)
Skycorr
Skycalc

The skytools webpages provide access to the most recent information and software packages.

Please note that the Molecfit packages available on the skytools webpages are no longer supported!
Please use the new version available here instead.

 

Publications, Reference Documents, and Links


A list of Data Reduction Frequently Asked Questions is available.

The ESO Quality Control Group processes VLT data and monitors the instrument and data quality. They provide an overview of data and data processing on their web pages.

If you make use of EsoReflex workflows in a scientific publication, please consider citing EsoReflex. You may reference the following paper: Automated Data Reduction Workflows for Astronomy: The ESO Reflex Environment, 2013, Freudling, W., et al., Astron.Astrophys., 559, A96 (Bibtex entry)

A collection of Science Software related documentation (Software resources, Development standards, etc.) is available here