NGAS Release Cycle
ESO Garching, 2007-10-05
J.Knudstrup
Revision:
1.0
This document contains a description of how an NGAS release
cycle is carried out (or was carried out until V3.1.1).
The release cycle has evolved over time, for instance in
the beginning Linux and NGAS were installed manually and a snapshot ISO image created
from this via Mindi/Mondo and installed on all machines. Now the OS is
installed via a kickstart installation (from DVD) and NGAS via an installation
tool from CDROM, USB stick etc, or from a WEB address.
This document describes the latest status of the
release preparation as it has been carried out for NGAS V3.X.
The document does not contain a description of the NGAS
environment per se, but this information could be added if of interest. Also
not contained in this document are the preparation and handling of new configuration
files and updating of existing configuration parameters.
Note, there was an earlier initiative to write down
the installation procedure and to document the NGAS operational environment, but
this was never completed due to the change from using Mondo/Mindi to a kickstart
based installation. This draft document is named: “NGAS System Installation
& Configuration Manual/VLT-MAN-19400-3104”, but is mostly obsolete.
To prepare an NGAS
distribution, do as follows:
$ngasmgr> cd
/opsw/packages/ngams/ngamsTest/
$ngasmgr> python
ngamsTest.py -notifemail jknudstr@eso.org
An example of such a test report is shown below:
Date:
Mon, 18 Jun 2007 10:37:37 GMT
From:
NGAS user <ngas@eso.org>
Message-Id:
<200706181037.l5IAbbTt012829@ngasdev3.hq.eso.org>
Subject:
NG/AMS FUNCTIONAL TESTS REPORT
To:
undisclosed-recipients:;
NG/AMS
FUNCTIONAL TESTS - TEST REPORT
Date:
2007-06-18T09:55:16.585
Host: ngasdev3
NG/AMS Version: v3.1.1/Beta1/2007-06-15T17:00:00
Running
Test Suite: ngamsArchiveClientTest
(#1/30) - 14.9 s - SUCCESS
Running
Test Suite: ngamsArchiveCmdTest
(#2/30) - 488.5s - SUCCESS
Running
Test Suite: ngamsArchiveStressTest
(#3/30) - 84.3 s - SUCCESS
Running
Test Suite: ngamsAuthorizationTest
(#4/30) - 22.2 s - SUCCESS
Running
Test Suite: ngamsCClientTest
(#5/30) - 287.8s - SUCCESS
Running
Test Suite: ngamsCheckFileCmdTest
(#6/30) - 44.0 s - SUCCESS
Running
Test Suite: ngamsCloneCmdTest
(#7/30) - 312.5s - SUCCESS
Running
Test Suite: ngamsConfigCmdTest
(#8/30) - 8.6 s - SUCCESS
Running
Test Suite: ngamsConfigHandlingTest
(#9/30) - 39.6 s - SUCCESS
Running
Test Suite: ngamsDataCheckingThreadTest
(#10/30) - 18.4 s - SUCCESS
Running
Test Suite: ngamsDbSnapShotTest
(#11/30) - 97.1 s - SUCCESS
Running
Test Suite: ngamsDiscardCmdTest
(#12/30) - 76.3 s - SUCCESS
Running
Test Suite: ngamsEmailNotificationTest
(#13/30) - 1.2 s - SUCCESS
Running
Test Suite: ngamsExitCmdTest
(#14/30) - 11.5 s - SUCCESS
Running
Test Suite: ngamsHelpCmdTest
(#15/30) - 11.4 s - SUCCESS
Running
Test Suite: ngamsIdleSuspensionTest
(#16/30) - 293.2s - SUCCESS
Running
Test Suite: ngamsInitCmdTest
(#17/30) - 15.3 s - SUCCESS
Running
Test Suite: ngamsJanitorThreadTest
(#18/30) - 1.2 s - SUCCESS
Running
Test Suite: ngamsLabelCmdTest
(#19/30) - 1.2 s - SUCCESS
Running
Test Suite: ngamsOfflineCmdTest
(#20/30) - 8.6 s - SUCCESS
Running
Test Suite: ngamsOnlineCmdTest
(#21/30) - 10.2 s - SUCCESS
Running
Test Suite: ngamsPClientTest
(#22/30) - 217.7s - SUCCESS
Running
Test Suite: ngamsRegisterCmdTest
(#23/30) - 9.3 s - SUCCESS
Running
Test Suite: ngamsRemDiskCmdTest
(#24/30) - 76.7 s - SUCCESS
Running
Test Suite: ngamsRemFileCmdTest
(#25/30) - 62.2 s - SUCCESS
Running
Test Suite: ngamsRestoreCmdTest
(#26/30) - 1.2 s - SUCCESS
Running
Test Suite: ngamsRetrieveCmdTest
(#27/30) - 269.6s - SUCCESS
Running
Test Suite: ngamsServerTest
(#28/30) - 1.2 s - SUCCESS
Running
Test Suite: ngamsStatusCmdTest
(#29/30) - 53.9 s - SUCCESS
Running
Test Suite: ngamsSubscriptionTest
(#30/30) - 1.2 s - SUCCESS
Executed
30 Test Suites in 2541.162 seconds
If tests fail, the detailed information about the cause of the problem will be contained in the report
In
order to tag an NGAS release, the tool “ngasFreezeRelease” is used. It should
be invoked as follows:
$ngasmgr> python
/opsw/packages/ngasInstall/src/ngasFreezeRelease.py
It
will prompt for a release tag for the pertinent modules. This could be e.g. “NGAS-V3_1_1Beta1”;
the latest release tag will be displayed and can be chosen if desirable. If the
same tag is chosen, the CVS module in question will be retagged.
After
having tagged all CVS modules constituting an NGAS distribution, the release
tarball should be created. This is done by means of the tool “ngasCreateDist”.
This should be invoked as follows:
$ngasmgr> python
/opsw/packages/ngasInstall/src/ngasCreateDist.py
It
will prompt the user for some information about the release (name etc.).
The
resulting release tarball will appear as, e.g.:
/opsw/packages/ngasInstall/tar-balls/V3_1_1-2007-07-12T08_42_49-Linux_SL43.tar.gz
This
should then be published or stored in the appropriate location.
JKN: until now for convenience, I
published it under my private ESO WEB area:
http://www.eso.org/~jknudstr/NGAS/RELEASES/V3_1_1/
The contents under http://www.eso.org/~jknudstr/NGAS/RELEASES/,
was synchronised to SOS’ more official repository for the NGAS distributions:
http://aclxint.hq.eso.org/addons/ngas/ngas_software/CURRENT/ [1]
From this site, SOS was installing NGAS
on the operational nodes. At LS and PL, NGAS was installed from a CDROM (the
NGAS installation tool supports both).
Just for completenes, the crontab to
mirror the NGAS release area onto the SOS NGAS SW area, was as follows:
30 * * * * rm
-f /home/jknudstr/public_html/NGAS/RELEASES/*~; \
/usr/local/bin/rsync
-ave ssh –delete \
/home/jknudstr/public_html/NGAS/RELEASES/
\
ngas@aclxint:/home/ngas/ngas_software/
> /dev/null
In
order to install an NGAS Node, the tool “ngasInstallSys” is used; the
latest/appropriate version is available within the NGAS distribution as a
standalone tool:
http://www.eso.org/~jknudstr/NGAS/RELEASES/V3_1_1/ngasInstallSys
It
can be executed without the availability of further NGAS SW. It only needs
Python to execute.
In
order to facilitate the installation procedure, pre-defined installation
commands are provided within the release, e.g.:
http://www.eso.org/~jknudstr/NGAS/RELEASES/V3_1_1/INSTALL-CMD-LS-AHU
As
can be seen from the name, imaginatively, this command is used to install the
Archive Handling Unit (AHU) at La Silla.
The
content of such a command is e.g. (from the example above)[2]:
###############################################################################
# Installation UNIX shell script to install an NGAS system of type:
#
# LS, AHU
#
###############################################################################
###############################################################################
# Base URL from where to obtain the distribution.
###############################################################################
#SRC_URL=http://aclxint.hq.eso.org/addons/ngas/ngas_software/CURRENT/
SRC_URL=file://$PWD/
###############################################################################
# Various parameters to update for the specific context.
###############################################################################
INST_EMAIL=ngast@eso.org
WATCHDOG_EMAIL=ngast@eso.org
NGAMS_CFG_FILE_SRC=/opsw/packages/ngasCfg/cfg/NgamsCfg_LS_AHU.xml
NGAMS_CFG_FILE=/opsw/packages/cfg/ngamsServer.conf
NGAMS_DB_CFG_ID=FILE-BASED-CFG
PAR_ZOPE_URLS=http://$HOSTNAME:8080/NGAS/OLASLS
#ALIASES_FILE=/opsw/packages/ngasSys/common/LS_etc_aliases
NGAS_UTILS_RESOURCE_FILE=/opsw/packages/ngasSys/common/ngas_DHA_LS
###############################################################################
…
As
can be seen from the “SRC_URL” above, the LS and PL systems are installed from
a CDROM. I.e., the data for the distribution is picked up locally. The other
URL is the one used at HQ to access the distribution files.
The
other parameters are as follows:
· INST_EMAIL: Email recipient that will receive the final report indiacting the status of the installation procedure and all actions executed.
· WATCHDOG_EMAIL: Email recipient that will receive notification when the NGAS Server Watchdog restarts the NGAS Server.
· NGAMS_CFG_FILE_SRC: Source configuration file to be used. Must be specified and must at least contain the parameters for connecting to the RDBMS.
· NGAMS_CFG_FILE: The configuration file used as input parameter to the “ngamsServer” application; this is a link pointing to the configuration file above.
· NGAMS_DB_CFG_ID: Indicates if the configuration information is stored in the DB or in a local file. In case of former, this should be the ID of the DB configuration. If only a local configuration file is used, this should be “FILE-BASED-CFG”.
· PAR_ZOPE_URLS: Indicates the parameters for the NGAS Zope Watchdog to monitor.
· NGAS_UTILS_RESOURCE_FILE: Resource file used by the NGAS Utilities.
Usually
it should be possible to use one of the predefined installation commands. If
necessary, the best suitable one can be taken and updated to adapt the system
for specific purposes.
As
example of an installation report is shown below:
Subject:
NGAS INSTALLATION REPORT
From:
ngasmgr@eso.org
Date:
Tue, 26 Jun 2007 12:45:44 +0200 (MEST)
To:
undisclosed-recipients:;
================================================================================
NGAS
INSTALLATION UTILITY
--------------------------------------------------------------------------------
Date: 2007-06-26T10:43:11 UTC
User: ngasmgr
Host: ngasdev3
Status: OK
================================================================================
INFO>
Executing shell command: ps -efww | grep ngamsServer | grep
/opsw/util/python/bin/python | grep -v root
INFO>
Running installation tool with command line options:
--installation-root=/opsw/packages/,
--src-url=http://www.eso.org/~jknudstr/NGAS/RELEASES/V3_1_1/, --post-installation-pi=Install_Post_Inst,
--notif-email=jknudstr@eso.org,
--Install_Configure_NGAS=NGAMS_CFG_FILE_SRC=/opsw/packages/ngasCfg/cfg/NgamsCfg.Example.xml,NGAMS_CFG_FILE=/opsw/packages/cfg/ngamsServer.conf,NOTIF_EMAIL=jknudstr@eso.org,
--Install_Configure_Users,
--Install_Configure_Zope=PAR_ZOPE_URLS=http://ngasdev3.hq.eso.org:8080/NGAS/ESOECF,
--Install_Configure_aliases=/opsw/packages/ngasSys/common/DEV_etc_aliases,
--Install_Configure_pydoc, --Install_cfitsio, --Install_chksum, --Install_frameIngest,
--Install_ngams_bins, --Install_ngasPlugIns,
--Install_Configure_ngasUtils=RESOURCE_FILE=/opsw/packages/ngasSys/common/ngas_DEV
INFO>
Installing NGAS Package ...
INFO>
Selected NGAS Package Name: NGAS-CURRENT
INFO>
Executing shell command: rm -rf /tmp/NGAS-CURRENT.tar*
INFO>
Retrieving package from URL:
http://www.eso.org/~jknudstr/NGAS/RELEASES/V3_1_1//NGAS-CURRENT.tar.gz into
target file: /tmp/NGAS-CURRENT.tar.gz ...
INFO>
Executing shell command: gunzip /tmp/NGAS-CURRENT.tar.gz
INFO>
Entering path: /opsw/packages/
INFO>
Executing shell command: rm -rf /opsw/packages//ngasInstall/plug-ins/*.py
INFO>
Executing shell command: tar xvf /tmp/NGAS-CURRENT.tar
INFO>
Entering path: /opsw/packages/ngams
INFO>
Executing shell command: ./bootstrap
INFO>
Executing shell command: ./configure --prefix=/opsw/packages/
INFO>
Entering path: ngamsCClient
INFO>
Executing shell command: make clean all install
INFO>
Installed NGAS Package
INFO>
Entering runInstallPlugIns() ...
INFO>
Executing Installation Plug-In: Install_Configure_Users
INFO>
Executed Installation Plug-In: Install_Configure_Users
INFO>
Not executing Installation Plug-In Install_Python
INFO>
Executing Installation Plug-In: Install_Configure_aliases
INFO>
Executed Installation Plug-In: Install_Configure_aliases
INFO>
Executing Installation Plug-In: Install_Configure_ngasUtils
INFO>
Executed Installation Plug-In: Install_Configure_ngasUtils
INFO>
Executing Installation Plug-In: Install_Configure_pydoc
INFO>
Executed Installation Plug-In: Install_Configure_pydoc
INFO>
Executing Installation Plug-In: Install_frameIngest
INFO>
Executed Installation Plug-In: Install_frameIngest
INFO>
Executing Installation Plug-In: Install_ngams_bins
INFO>
Executed Installation Plug-In: Install_ngams_bins
INFO>
Not executing Installation Plug-In Install_SysFiles
INFO>
Not executing Installation Plug-In Install_Sybase
INFO>
Executing Installation Plug-In: Install_chksum
INFO>
Executed Installation Plug-In: Install_chksum
INFO>
Executing Installation Plug-In: Install_cfitsio
INFO>
Executed Installation Plug-In: Install_cfitsio
INFO>
Executing Installation Plug-In: Install_ngasPlugIns
INFO>
Executed Installation Plug-In: Install_ngasPlugIns
INFO>
Executing Installation Plug-In: Install_Configure_NGAS
INFO>
Executed Installation Plug-In: Install_Configure_NGAS
INFO>
Not executing Installation Plug-In Install_Configure_fstab
INFO>
Executing Installation Plug-In: Install_Configure_Zope
INFO>
Executed Installation Plug-In: Install_Configure_Zope
INFO>
Not executing Installation Plug-In Install_Post_Inst
INFO>
Not executing Installation Plug-In Install_Zope
INFO>
Leaving runInstallPlugIns()
INFO>
Executing Installation Plug-In: Install_Post_Inst
INFO>
Executed Installation Plug-In: Install_Post_Inst
INFO>
Executed system installation/configuration
INFO>
Entering submitReport() ...
================================================================================
After
the node has been installed it can be verified manually in the following way:
$ngasmgr> sudo /etc/init.d/ngamsServer
start
$ngasmgr> ngamsStatus
Status of request:
Host: ngasdev3
Port: 7777
Command: STATUS
Date: 2007-10-05T12:39:53.832
Error Code: 0
Host ID: ngasdev3
Message: Successfully handled command STATUS
Status: SUCCESS
State: ONLINE
Sub-State: IDLE
NG/AMS Version: v3.1.1/2007-07-12T10:30:00
Other commands could be
executed for instance to archive data and retrieve files for the sake of making
a rough verification of the node.
The
installation can also be verified by executing the automatic NGAS tests; see
above (“Preparing the Distribution”). However, in order to execute the tests,
the operational DB servers must be commented out from the Sybase interfaces
file. Also, the alias in “/etc/aliases” for the user “ngasmgr” must be unset. The
test tool will inform about this if one or both preconditions are not
fulfilled.