Migration to DFS2018 / CentOS

Version: 2018-06-06

Before the update

  1. Install the new versions of autoDaily, dfoMonitor, pet.py, qc_rawdisp.py, and qcdate. All are functional also in the pre-DFS2018 environment.

  2. Save the crontab e.g. via: crontab -l > my_cron

  3. Just before the upgrade:
    terminate firefox
    install an empty crontab.

After CentOs has been installed

  1. Before first login:
    on your personal Linux desktop machine edit the file
    ${HOME}/.ssh/known_hosts
    and remove all lines containing the name of the just migrated machine, e.g. "muc03". Also delete left over lines from a previous migration, i.e. lines containing "muc03n".
  2. First log in: the password has been reset to "pippo". When you first log in, you have to give this password and then you are required to set a new password (which can be the same password that you previously used).
  3. Check that DFS and DFOS environment variables have been set:
    > env | grep DFS
    > env | grep DFO
  4. Check that TMP_DIR is existing. If not, create it.
  5. Firefox: make sure that the "old" firefox is correctly found.
    In .dfosrc, add a new environment variable and source .dfosrc:
    export DFO_BROWSER_PATH="/opt/firefox"
    Make an alias to the "old" firefox, e.g. in $HOME/.pecs/misc-all.ali:
    alias firefox='/opt/firefox/firefox'.
    Source this file and verify that the "old" firefox is available. Typing
    > firefox -v
    should return version 24.8.0. Only then start firefox.
    (The firefox call also gives a warning message which is not relevant.)
  6. Setting environment for Python. This is needed for all accounts so that qc1Ingest functions. Insert in .qcrc somewhere to the end, e.g. after sourcing .dfosrc:
    source activate /opt/qc
    and source .qcrc. If you want to quit this environment you may type on the command line
    > source deactivate
    IMPORTANT: make sure that .qcrc is not sourced again. This will manipulate the PATH so that the "wrong" Python is found. In your crontabs, it is sufficient to source .bashrc (if needed). And also avoid sourcing of .qcrc in your QC scripts. The call of processQC within Condor does already the necessary sourcing.
  7. Check path to isql:
    > which isql
    should return
    /opt/sybase/bin/isql
    or
    /opt/sybase-16.0/OCS-16_0/bin/isql
    If not, please check set up of PATH (likely in ${HOME}/.qcrc). A statement like
    export PATH=/opt/python/bin:${PATH}
    should be removed or or should be executed after
    export PATH=/opt/sybase/bin/isql:${PATH}
  8. Esorex configuration: remove old pipeline versions and add the current ones in ${HOME}/.esorex/esorex.rc
  9. Esorex configuration: some configuration keys are no longer supported. In order to avoid warnings by esorex, remove from ${HOME}/.esorex.esorex.rc:
    esorex.caller.help, esorex.caller.version, esorex.caller.man-page, and esorex.caller.create-config
  10. CPL: if calls of pipeline recipes result in CPL errors then the LD_LIBRARY_PATH likely points to the VLT2017 CPL (which is incompatible with DFS2018 release of pipelines). Try setting explicitly in .dfosrc:
    export LD_LIBRARY_PATH=/opt/sybase/lib:/opt/dfs/lib
  11. Install crontab again from saved file, e.g.: crontab my_cron
  12. xview: this tool is not available any longer, please use display instead.
  13. MIDAS: change of version to MIDVERS=17FEBpl1.2
  14. Python QC scripts: the recommended interpreter is: #!/usr/bin/env python .
  15. Python QC scripts: the package pyfits is included in "our" environment (see 6.) but you may want to replace it for the future because development on it has been stopped. The simplest way is to replace
    import pyfits
    with
    from astropy.io import fits as pyfits
    in your scripts.
  16. Python QC scripts: the package asciidata is not available. The functionality can be replaced by astropy.io.ascii. See me for details if needed.
  17. If you find that the plot legend is arbitrarily placed: force it to e.g. the upper right corner by
    from matplotlib import rcParams
    rcParams['legend.loc'] = 'upper right'
  18. Python QC scripts: If you are using something like
    ax = pylab.axes(...)
    ...
    pylab.axes(ax)

    you may want to replace the second statement in order to avoid anoying warnings:
    pylab.sca(ax)
    If you are not knowing what I am talking about then don't worry ...