ESO Pipelines and EsoReflex Software Prerequisites for Source-based Installations

Software prerequisistes for source-based installation

This page provides information about the required software needed to install the ESO pipelines and EsoReflex from source. Note that if you have a Fedora Linux, Scientific Linux or macOS system it is strongly recommended to perform a binary installation as explained in RPM installation and MacPorts installation.

Note that if you are using scripts install_esoreflex or install_pipelinekit you are basically using source-based installations.

Software prerequisites for pipeline source installations

  • POSIX (Unix-like) operating system. The following standard Unix command line programs should be available: make, awk, tr, cat, sed, grep, mkdir, rm, cksum, basename, touch, tar.
  • A C++11 and C99 compiler. It is recommended to use the GNU compiler (version 4.8 or later), but clang has also been tested. Note that if you are compiling the UVES pipeline, clang is known to give wrong results in some algorithms, so only gcc should be used.
  • gzip data compression program
  • bzip2 data compression program
  • perl interpreter
  • pkg-config utility (optional)

Using distribution package managers

You must have sudo/root access to install the packages through the yum package manager. Contact your system administrator for that. Here we assume that sudo access has been properly configured for you in the system. Note that you might be prompted with your password.

  • Fedora 26 to 29 official repositories. Execute the following commands in a terminal:
    sudo dnf install gcc-c++ zlib-devel make gzip bzip2 tar perl gawk sed grep coreutils pkgconfig
  • Red Hat EL, CentOS, Scientific Linux 7.x or later official repositories. Execute the following commands in a terminal:
    sudo yum install gcc-c++ zlib-devel make gzip bzip2 tar perl gawk sed grep coreutils pkgconfig
  • Ubuntu 15.10 to 18.10 and Debian 8 or later official repositories. Execute in a terminal the following commands (you might be prompted for your password):
    sudo apt-get install g++ zlib1g-dev make gzip bzip2 tar perl gawk sed grep coreutils pkg-config
  • macOS MacPorts. If you have MacPorts installed, issue the following command:
    sudo port install pkgconfig autoconf automake libtool bzip2
    Take into account that Xcode needs also to be installed (a prerequisiste also for MacPorts itself).

Software prerequisites for GRAVITY pipeline source installations

Using distribution package managers

You must have sudo/root access to install the packages through the yum package manager. Contact your system administrator for that. Here we assume that sudo access has been properly configured for you in the system. Note that you might be prompted with your password.

  • Fedora 26 to 29 official repositories. Execute the following commands in a terminal:
    sudo dnf install yorick yorick-optimpack yorick-yeti
  • Red Hat EL, CentOS, Scientific Linux 6.x or later official repositories. Execute the following commands in a terminal:
    sudo yum install yorick yorick-optimpack yorick-yeti
  • Ubuntu 15.10 to 18.10 and Debian 8 or later official repositories. Execute in a terminal the following commands (you might be prompted for your password):
    sudo apt-get install yorick yorick-optimpack yorick-yeti
  • macOS MacPorts. If you have MacPorts installed, issue the following command:
    sudo port install yorick yorick-optimpack yorick-yeti

Software prerequisites for KMOS pipeline source installations

  • Fortran compiler.

Using distribution package managers

You must have sudo/root access to install the packages through the yum package manager. Contact your system administrator for that. Here we assume that sudo access has been properly configured for you in the system. Note that you might be prompted with your password.

  • Fedora 26 to 29 official repositories. Execute the following commands in a terminal:
    sudo dnf install gcc-gfortran libgfortran-static
  • Red Hat EL, CentOS, Scientific Linux 6.x or later official repositories. Execute the following commands in a terminal:
    sudo yum install gcc-gfortran libgfortran-static
  • Ubuntu 15.10 to 18.10 and Debian 8 or later official repositories. Execute in a terminal the following commands (you might be prompted for your password):
    sudo apt-get install gfortran
  • macOS MacPorts. If you have MacPorts installed, issue the following command:
    sudo port install gcc7 +gfortran
    Note that if you have other gcc version already installed, for instance gcc 6, you should install the fortran variant for that compiler, i. e.,
    gcc6 +gfortran

Software prerequisites for EsoReflex source installations

In addition to the pre-requisites listed above, the following software is needed to execute EsoReflex properly:

Note that you should make sure all the prerequisite binaries are available under one of the paths indicated by the following command:

getconf PATH

If this is not the case, then you will need to add the appropriate paths to the esoreflex.path parameter in the installed esoreflex.rc configuration file, after EsoReflex is installed.

EsoReflex 2.9.0 and newer supports Python based recipes as long as EsoRex is compiled with one of the following prerequisites:

  • libffi (optional, version 3.0 or later)
  • libffcall (optional, version 2.0 or later)

Note that these are not necessary if running workflows that do not rely on Python based recipes. Therefore, at the moment this is only applicable to contributed workflows, e.g. for MUSE.

Using distribution package managers

You must have sudo/root access to install the packages through the yum package manager. Contact your system administrator for that. Here we assume that sudo access has been properly configured for you in the system. Note that you might be prompted with your password.

  • Fedora 26 to 29 official repositories. Execute the following commands in a terminal:
    sudo dnf install python2-matplotlib python2-matplotlib-wx wxPython python-astropy numpy java-openjdk python2 libffi-devel

    You will still have to install the fv binary manually. Follow the instructions here. If possible, install fv so that it is accessible through /usr/bin.

  • Red Hat EL, CentOS, Scientific Linux 6.x or later official repositories. Execute the following commands in a terminal:
    sudo yum install python-matplotlib-wx wxPython pyfits numpy java-1.8.0-openjdk python libffi-devel
    If the operating system is 64 bits:
    update-alternatives --set java java-1.8.0-openjdk.x86_64
    If it is 32 bits:
    update-alternatives --set java /usr/lib/jvm/jre-1.8.0-openjdk/bin/java

    The wxPython backend for Matplotlib is not available by default on the CentOS 7 platform. You will have to build this yourself if you plan to run a workflow that requires this feature. Please refer to the Matplotlib installation instructions for building with the wxPython backend enabled. Alternatively install from the RPM repositories instead.

    You will still have to install the fv binary manually. Follow the instructions here. If possible, install fv so that it is accessible through /usr/bin.

    To enable Python based recipes, EsoRex needs to be able to find libffi. Unfortunately, on older versions of Red Hat, Centos and Scientific Linux 6, the libffi is not automatically visible to the EsoRex compilation if using the install_esoreflex script. The easiest approach to solving this is to run commands similar to the following (Note that you may need to adjust the version numbers in the paths):

    sudo ln /usr/lib64/libffi-3.0.5/include/ffi.h /usr/include/ffi.h
    sudo ln /usr/lib64/libffi-3.0.5/include/ffitarget.h /usr/include/ffitarget.h
  • Ubuntu 15.10 to 18.10 and Debian 8 or later official repositories. Execute in a terminal the following commands (you might be prompted for your password):
    sudo apt-get install python-matplotlib python-wxgtk3.0 python-astropy python-numpy default-jre libffi-dev

    You will still have to install the fv binary manually. Follow the instructions here. If possible, install fv so that it is accessible through /usr/bin.

  • macOS MacPorts. If you have MacPorts installed, issue the following command:
    sudo port install py27-matplotlib py27-wxpython-2.8 py27-numpy py27-astropy xorg-server libffi
    sudo port select --set python python27
    Please make sure that the version of python used is the one from MacPorts by adding /opt/local/bin first in your path. This depends on your shell. For bash:
    export PATH=/opt/local/bin:$PATH
    In some shells you might have to do a rehash:
    rehash

    If using the libffi option, you will have to make sure that EsoRex is able to find, compile and link against libffi.

    You would have to install a Java 8 virtual machine separately as well.

    You will still have to install the fv binary manually. Follow the instructions here and install in the default location (Applications folder). Remember you will need to modify the esoreflex.rc configuration file once EsoReflex is installed to add the additional path /Applications/fv.app/Contents/MacOS to the esoreflex.path configuration parameter.

    To enable Python based recipes, EsoRex needs to be able to find libffi. Unfortunately, the version in MacPorts is not automatically visible to the EsoRex compilation if using the install_esoreflex script. The easiest approach to solving this is to run commands similar to the following (Note that you may need to adjust the version numbers in the library file name and paths):

    sudo mkdir -p /usr/local/lib /usr/local/include
    sudo ln /opt/local/lib/libffi.6.dylib /usr/local/lib/libffi.6.dylib
    sudo ln /opt/local/lib/libffi-3.2.1/include/ffi.h /usr/local/include/ffi.h
    sudo ln /opt/local/lib/libffi-3.2.1/include/ffitarget.h /usr/local/include/ffitarget.h

Software prerequisites for Gasgano source installations

Note that Gasgano installation is disabled when using the install_esoreflex. If you are using the install_pipelinekit script then it is enable by default, but can be disabled with -ignore-gasgano. Note that if you are installing JDK in macOS, you should make it known to the installation procedure with this extra command:
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/
for OS X 10.7 Lion and OS X 10.8 Mountain Lion.

Using distribution package managers

You must have sudo/root access to install the packages through the yum package manager. Contact your system administrator for that. Here we assume that sudo access has been properly configured for you in the system. Note that you might be prompted with your password.

  • Fedora 26 to 29 official repositories. Execute the following commands in a terminal:
    sudo dnf install java-1.8.0-openjdk-devel
  • Ubuntu 15.10 to 18.10 and Debian 8 or later official repositories. Execute in a terminal the following commands (you might be prompted for your password):
    sudo apt-get install default-jdk

Installation procedure

If you are not using macports for macOS 10.11 - 10.14 or RPMs for Fedora, CentOS, Scientific Linux, then the install_esoreflex script installation method is the recommented way to install pipelines with Reflex support by following these instructions:

  1. Check that you have all the software prerequisites for pipeline source installation as well as the software prerequisites for EsoReflex
  2. Download install_esoreflex
  3. Make it executable:
    chmod u+x install_esoreflex
  4. Execute the installation script. This will install EsoReflex, the supported pipelines and demo data. The script will ask for an installation directory (install_dir) as well as the download and data directories.
    ./install_esoreflex
    If the software prerequisites are not met the script will fail, or in the case of optional packages (python modules) it will prompt whether you want to continue. If this happens, go and review again the Reflex software prerequisites.
  5. If any of the prerequisite binaries (indicated in Reflex software prerequisites ) do not exist in one of the paths indicated by the command,
    getconf PATH
    then you need to update the EsoReflex configuration. Open the installed install_dir/etc/esoreflex.rc file (where install_dir is the previously selected installation directory) and add the appropriate paths as a colon separated list to the esoreflex.path parameter.
  6. Execute EsoReflex:
    install_dir/bin/esoreflex
This script will allow to install the pipelines with available workflows as well as EsoReflex itself. It requires an Internet connection, since it retrieves all the packages directly from the net. Please read section Software prerequisites to make sure that you have the needed software before running the script. The script will detect if there is any missing software, but it won't attempt to install it.

The script also has limited support for reinstallation of software if new versions of the pipelines or EsoReflex are available.

Manual Installation of all the software

This installation procedure installs the pipeline and the workflow first, and then EsoReflex is used on top of an existing pipeline installation. It is a more advanced setup and therefore it is recommended only for advanced users.
  1. Download and make a regular installation of the pipeline. Note that if you already have an installed version which includes the workflow, it is also possible to use it.
  2. Make sure that EsoRex can access the uves recipes using the following command:
    esorex --recipes
  3. Download and install EsoReflex
  4. Start EsoReflex and open the workflow located in the installation directory of the pipeline:
    installation_dir/share/reflex/workflows/pipeline-x.x.x
  5. Demo data can be found in this table under column "Demo Data Sets".
  6. If you have run EsoReflex in the same machine and account where another EsoReflex installation was previously made, you might have to remove directories ~/KeplerData and ~/.kepler to clean the cache.

Bug Reports

If you experience an unexpected behaviour of any component of EsoReflex or the pipeline recipes package, please, first refer to the list of known problems and limitations in the workflow tutorial, the pipeline manual, or the EsoReflex user manual, in this order.

For any other issues or requests, please, send a report to the ESO User Support Department, describing:        

  • the EsoReflex and pipeline version, and the version of other components (e.g., Gasgano, EsoRex, ...) you are using
  • the version of your OS and C compiler
  • the version of python and required python modules
  • the exact sequence of actions that were performed before the problem occurred
  • what were precisely the symptoms and the possible error message(s)
  • whether the problem is repeatable