Installing PyCPL¶
Installation from Pre-Built Packages¶
Note
Pre-built, binary packages for Fedora 36 – 37, macOS 11 and macOS 12 (Intel only) are in preparation. This section will be updated with the installation instructions as soon as binary packages of PyCPL and PyEsoRex are available.
Installation from Sources¶
While a system wide installation of PyCPL and PyEsoRex is of course supported, the following installation instructions will show an installation using a Python virtual environment. It is expected that this is a more common approach followed by individual users when installing python packages.
It is however assumed that the dependencies needed to build and use PyCPL and PyEsoRex can be installed using the system’s package management system or by installing an ESO instrument pipeline source kit.
Software Prerequisites¶
In order to build, install and use PyCPL and PyEsoRex the following software packages, with the given minimum version (if indicated), is needed:
C++ compiler supporting C++17 (GCC 9.5, Clang 5)
CMake 3.12
Python 3.8 (including the development package, i.e. headers and libraries)
Python packages:
setuptools 45.0.0
pybind11 2.8.0
astropy
pandas (optional)
Sphinx (optional, for generating the API documentation)
pytest (optional, for running the testsuite)
scipy (optional, for running the testsuite)
CPL 7.2.2
Depending on whether PyCPL and PyEsoRex will be installed system wide, local to the user, or in a Python virtual environment (or alike) the required packages must be available in the respective environment. While the compiler, Python itself and CPL are generally expected to be provided by the system, or possibly the user’s local environment. CMake and the required Python packages however may be installed into the target virtual environment only, in case a virtual environment approach is chosen.
Getting the Source Code¶
The most recent version of the PyCPL and PyEsoRex source distributions is available here:
Older releases of PyCPL and PyEsoRex can be found in the PyCPL archive and the PyEsoRex archive.
Users with access to the ESO Gitlab may also clone the PyCPL and PyEsoRex repositories directly (ESO internal access only):
$ git clone https://gitlab.eso.org/cpl/pycpl $ git clone https://gitlab.eso.org/cpl/pyesorex
Installing Development Tools¶
The following shows how to install the standard dependencies for an ESO pipeline source installation. These dependencies are also needed to build PyCPL. These dependencies can be installed using the package manager of the different systems. Please skip to the section that applies to your environment.
- Fedora 35 and newer:
$ sudo dnf install gcc gcc-c++ zlib-devel make gzip bzip2 tar perl gawk sed grep coreutils pkgconfig $ sudo dnf install cmake python3-devel
- Ubuntu 20.04 and newer:
$ sudo apt install gcc g++ zlib1g-dev make gzip bzip2 tar perl gawk sed grep coreutils pkg-config $ sudo apt install cmake python3-dev python3-venv
- macOS 11 and newer:
$ sudo port install pkgconfig autoconf automake libtool bzip2 $ sudo port install cmake python310
To set Python 3.10 as the default Python version to use on macOS, run
$ sudo port select --set python python310 $ sudo port select --set python3 python310
Installing CPL¶
There are several ways to install CPL. Depending on the use case one may choose one way or the other. If CPL is already installed on your system, because you have already installed an ESO instrument pipeline for instance, it is recommended that you use this CPL installation for building PyCPL, provided that the existing CPL installation includes the CPL header files, and meets the PyCPL requirements.
Attention
Regardless of which of the following methods you use to install CPL, the root directory of the CPL installation will later be needed to correctly build PyCPL.
Installing CPL from the ESO repositories¶
Binary packages of CPL are available from the ESO repositories for a number of different platforms. The information for which platforms binary packages of CPL are currently available can be found on the VLT Instrument Pipelines page.
Detailed instruction on how to configure and use the ESO repositories are available at the pipeline webpage for Linux (RPM) and macOS systems.
With the repositories configured you will have to install the CPL packages:
Fedora 35 and newer:
$ sudo dnf install cpl-devel
macOS 11 and newer:
$ sudo port install cpl
This will install CPL, including the required CPL headers, into /usr
on Linux systems, and into /opt/local on macOS systems.
All pipeline packages which are available from these repositories can be used together with PyCPL.
Installing CPL using a Pipeline Source Kit¶
When installing an ESO instrument pipeline using one of the pipeline source kits which are available from the VLT Instrument Pipelines page, the CPL which is part of the source kit will be automatically installed.
For use with PyCPL the source kit maybe be installed using either the pipeline installer script, or the Reflex installer script. Instructions on how to install an ESO pipeline source kit using any of the two installer scripts can be found on the VLT Instrument Pipelines page.
The list of available pipeline source kits can be found here. All kits shown in this table provide a CPL which meets the PyCPL requirements.
Using this method CPL will be installed into the directory that is given when the installer scripts ask for an installation directory.
Installing CPL from Sources¶
It is also possible to install CPL using the CPL source distribution. This is the recommended way for developers working on an instrument pipeline project, or expert users who are familiar with the CPL build system and the needed developer tools. Using this method should hardly be necessary for a normal user.
To install CPL, and possibly its dependencies, using their respective source distributions, please refer to the installation instructions coming with the individual source packages. The installation instructions of CPL are included in the source package, which is available from the CPL download page
Choosing this method to install CPL gives you complete freedom where CPL, and possibly its dependencies, will be installed. It is however up to you to know where to point the PyCPL build system so that it finds the CPL installation.
Creating a Python Virtual Environment¶
To prepare for building and installing PyCPL and PyEsoRex a Python virtual environment may be created. To do so choose a location where the virtual environment is going to be installed. If you have used virtual environments before, you may have such a location already and it may be used to host the Python virtual environment for PyCPL and PyEsoRex as well.
In the following <venv-root-dir> will have to be replaced by the path you have chosen
as the installation directory of the virtual environment.
$ python3 -m venv <venv-root-dir>/pycpl
Installing PyCPL and PyEsoRex¶
Once the required dependencies are installed the PyCPL and the PyEsoRex source distributions can be installed.
Preparing the Environment¶
The first step is to setup the environment so that PyCPL uses the correct CPL
installation. In order to point the PyCPL build system to the appropriate
CPL installation the environmet variable CPLDIR has to be set to the
root directory of the CPL installation. For instance, if the CPL libraries
are installed in the directory /usr/lib, then CPLDIR must be set to
/usr. Hints on where CPL might be installed are given in the Section
Installing CPL.
$ export CPLDIR=<cpl-root-dir>
Note
If in doubt where CPL is installed, try to find the file libcplcore.so
or libcplcore.dylib depending whether you use a Linux system or a macOS system,
respectively.
Make sure the libraries from the CPL installation located in CPLDIR are preferred
when looking for libraries. This is particularly important if you have more than one CPL
installation on your system, but it is also recommended if there is only a single CPL
installation.
This is done by adding the CPL library directory to the beginning of the library search path. The environment variable names differ, depending on the OS you are using:
Linux:
$ export LD_LIBRARY_PATH=$CPLDIR/lib64:$CPLDIR/lib:$LD_LIBRARY_PATH
macOS:
$ export DYLD_LIBRARY_PATH=$CPLDIR/lib:$DYLD_LIBRARY_PATH
Note
If you exit from your shell session and start a new one later you will have to set these previously defined environment variables again before you can continue with the installtion.
The same applies if you are using PyCPL or PyEsoRex in a newly started shell environment.
The next step is to activate the Python virtual environment:
$ . <venv-root-dir>/pycpl/bin/activate
At this point it is recommended to update pip, setuptools, wheel and build, in particular if your virtual environment was created from a somewhat older Python installation:
(pycpl)$ python3 -m pip install --upgrade pip setuptools build wheel
As a last step before actually building PyCPL it may be necessary to change the default PyCPL recipe search path. By default PyCPL expects recipes to be found in a subdirectory of the CPL installation. This default may however not be suitable, for instance if CPL is installed into a system location where a normal user is usually not able to put his own recipes.
Attention
If the default recipe directory does not exist, a warning will be issued when PyCPL is built and when the Python module is imported. Usually this default recipe location does not exist if CPL is installed stand-alone, i.e. not as part of a pipeline source kit installation.
This default location can be changed when building PyCPL using the environment
variable PYCPL_RECIPE_DIR and point it to an appropriate, existing directory.
Typical locations where recipes are installed may depend on the method used
to install pipeline packages, and the operating system. For instance, using the
package managers to install pipeline packages from the ESO repositories will
install recipes into /usr/lib64/esopipes-plugins and
/opt/local/lib/esopipes-plugins on 64-bit Fedora systems and macOS, respectively.
If pipeline packages are installed using a source kit the recipe location will be
<prefix>/lib/esopipes-plugins, where prefix is the root directory of the
installation given by the user during the installation process.
Note
It is of course always possible to change this default recipe
location later, at runtime by changing the recipe search path from the
Python interpreter prompt, a script, or through a PyEsoRex command
line option. Setting PYCPL_RECIPE_DIR will only change the built-in
default. However, this path will persist as a default setting until
PyCPL is eventually re-compiled. Thus, a reasonable location should
be chosen.
Assuming that all pipeline recipes will be installed locally in a user account
into the existing location $HOME/pipelines/lib/esopipes-plugins the
default PyCPL recipe location can be changed setting the variable like this:
(pycpl)$ export PYCPL_RECIPE_DIR=$HOME/pipelines/lib/esopipes-plugins
The environment variable PYCPL_RECIPE_DIR is only used for building PyCPL.
Once the installation of PyCPL is finished, the variable may be unset again.
Building PyCPL and PyEsoRex¶
With the environment set PyCPL and PyEsoRex can be build and installed from the source distribution packages in the one of the following ways:
- Option 1:
The easiest way to install the latest available version of PyCPL and PyEsoRex, without the need for downloading the source packages first, you can use the following command:
(pycpl)$ python3 -m pip install --extra-index-url https://ftp.eso.org/pub/dfs/pipelines/libraries pycpl pyesorex
- Option 2:
To install PyCPL and PyEsoRex after getting the source distributions from the locations given in Getting the Source Code and placing them into a temporary directory
<tmpdir>, use the command:(pycpl)$ python3 -m pip install <tmpdir>/pycpl-1.0.0.tar.gz <tmpdir>/pyesorex-1.0.0.tar.gz
- Option 3:
Installing PyCPL or PyEsoRex from a working copy of their respective GitLab repository, or a directory containing the unpacked source distribution package of PyCPL or PyEsoRex, the following needs to be done. For instance in the case of PyCPL:
(pycpl)$ cd <tmpdir>/pycpl=1.0.0 (pycpl)$ python3 -m pip install .
At this point the installation of PyCPL and PyEsoRex is finished and you may unset the
environment variable PYCPL_RECIPE_DIR as it is no longer needed:
(pycpl)$ unset PYCPL_RECIPE_DIR
Verifying the Installation¶
Finally verify that the installation actually succeeded:
(pycpl)$ cd <tmpdir>
(pycpl)$ python3
From the Python interpreter prompt import the cpl and the
pyesorex modules:
>>> import cpl
>>> import pyesorex
If this succeeds without any messages printed to the terminal then you have successfully installed PyCPL and PyEsoRex API!
To verify that the PyEsoRex CLI is correctly installed:
(pycpl)$ pyesorex --version
or
(pycpl)$ pyesorex --help
