Common DFOS tools:
Documentation

dfos = Data Flow Operations System, the common tool set for DFO
*make printable

Maintenance

This is a description of the maintenance of the dfos system. It applies to both subcomponenents, dfos and tqs.

Where?

Maintainer: The DFOS system has originally been developed by Reinhard Hanuschik, with contributions from other QC group members. It is currently maintained by Burkhard Wolff. Send all requests to him.

Maintenance site: The operational account for DFOS maintenance is kmos@muc01 (originally it was giraffe@muc02). The root for dfos maintenance is $HOME/distrib (for distribution).

directory
content
$HOME/distrib/bin
all dfos sources
$HOME/distrib/version
all versions of dfos sources
$HOME/distrib/history
short history description which is displayed under 'history' on the dfos main page
$HOME/distrib/config
template configuration files which are contained in the distribution; same substructure as ($DFO_CONFIG_DIR)
$HOME/distrib/doc all help files (the ones under $DFO_DOC_DIR)
$HOME/distrib/makefiles all makefiles (as contained in the distribution); substructure: .../<tool name>/makefile

How can I access?

Usually not, unless you are the maintainer. In an emergency, get authorization from B. Wolff. Login as qcshift@muc02 and from there to the DFOS maintenance account kmos@muc01 (just type 'kmos').

How is it maintained?

Save current and earlier versions. A cronjob $HOME/crontabs/cronDfosVersion is called once a day and puts a copy of the latest version of each dfos tool (under $HOME/distrib/bin) into $HOME/distrib/versions. That version gets the name <tool_name>_v<version>. Effectively this is a version control system.

Distribute a new version.

For each tool, there is a little shell script under $HOME/distrib called make_tar_<tool_name>. That script finds all components of the distribution package (e.g. tool source, template config file, help file, history file, makefile; maybe other tools) and adds a README file. All components are put in a tarball named <tool_name>.tar and ftp'ed to the QC internal account (http://www.eso.org/~qc/dfos/tools). Note that it takes a couple of minutes after the ftp for the pages to show the new content (cronjob running to update server content).

The following steps should be checked for a new release (all on maintenance site):

Then:

For any issues related to web server maintenance, please refer to Michael Naumann.

The tar files are linked to the dfos main pages http://www.eso.org/~qc/dfos/details.html and http://www.eso.org/~qc/tqs/details_tqs.html.

The installation of a new tool is described here.

Development style and rules

The development and maintenance of the common dfos tool suite is of key importance for day-to-day QC operations. Therefore it is restricted.

Most dfos tools are written as bash shell scripts. The scripting approach provides extreme flexibility which is required due to the ever-changing operational conditions. On the short side, tools are generally not optimized for performance, but for stability.

Attempts are made to develop tools with the same structure, the same syntax and the same style. For instance, there are many different ways to code a DO loop or an IF statement in a shell script. Some developers prefer a concise style. dfos has a "transparent" style, simple-minded, inelegant, but (hopefully) easy to understand by others who may need to maintain the system.

The tools try to follow the workflow. They have an initial section, containing the definition of variables (either hard-coded or read from a configuration file); evaluation of command options; procedures and helper scripts. Sections are made visible in the code. The main part of the code with the workflow steps always starts with section 1.0. An attempt has been made to use parameter and option names which are common among the tools.


Last update: April 26, 2021 by bwolff