TOC PREV NEXT INDEX

Put your logo here!


2.4 Making a simple test

This section shows the main features of the tat test driver by developing a simple CCS test.

The purpose of this test is to test the msgSend utility: we want to check that msgSend is able to send the message LCCGVER (retrieve LCC version) to an LCU.

The tat test driver is the main end-user program made available by the tat module: it is also named tat. Tat looks for its directives into the file named TestList that should be in the module test directory.

2.4.1 Environment configuration

To execute msgSend, we need two environments: a workstation one and a LCU one.

Let's call WSName the environment variable for the WS environment and LCUName the environment variable for the LCU environment.

For each environment designated by the environment variable <envVar>, tat requires that a corresponding directory named ENVIRONMENTS/<envVar> exists in the current test directory. These directories will stay empty because this test does not need customized environments.

$ mkdir ENVIRONMENTS/WSName
$ mkdir ENVIRONMENTS/LCUName

Make these environment name variables known to tat by adding the following two lines in TestList and let's call ccsTest the test case we are developing:

# TestList for msgSend
ENVIRONMENT WSName WS
ENVIRONMENT LCUName LCU
ccsTest

2.4.2 Environment allocation and creation

The option makeEnv of the tat command allocates and creates the environments:

$ tat makeEnv

By default, tat makeEnv does not print any output. Only in case of error, messages are printed on the standard output.

The allocated environment names can be found in the .testSession file:

$ cat .testSession
LCU LCUName lte42
WS WSNAme wte4901



Running tat makeEnv opens a test session: the allocated environment are booked until they are explicitely released with the command tat cleanEnv.

2.4.3 Writing the test

Our test script needs only to call msgSend. Note that by default, tat assigns to RTAPENV the value of the first workstation environment declared in TestList.


#!/bin/sh
#
# ccsTest
msgSend $LCUName lccServer LCCGVER ""

2.4.4 Generating the test reference

To generate the test reference, type:

$ tat -g ccsTest
Reference file ccsTest.ref generated
Reference file generated.

Tat has generated the reference file ccsTest.ref in the ref subdirectory:


$ cat ref/ccsTest.ref
MESSAGEBUFFER:
"@(#) LCC 3.19 11 Jun 1996"

2.4.5 Running the test

Now you can run again the test just by typing:

$ tat ccsTest
TEST ccsTest PASSED.
PASSED.

When executing tat with a test script as parameter, tat assumes that it has already created all the needed environments. Note that this simple test does not require you to recreate the environment, otherwise you should run tat cleanEnv, tat makeEnv and tat ccsTest.

The output of the test is redirected to the file ccsTest.out in the subdirectory ./tatlogs/run<pid>:

$ cat tatlogs/run<pid>/scanTest.out
scanTest OK

2.4.6 Cleaning the environments

To clean environments, run the command:

$ tat cleanEnv

As for tat makeEnv this command does not generate output. Only in case of problems, errors messages are printed on the standard output.

After having run tat cleanEnv, you cannot run your test again: the test session is closed. If you try, tat issues the following error message:

$ tat ccsTest
No test session active
Differences found in ccsTest.diff
TEST ccsTest FAILED.
FAILED.

2.4.7 Running the test with one command

The above described tat options allow the software developer to develop his test without having to recreate the environments everytime the test is run. Once the test reference has been generated and once this test reference is reproductible, the whole test can be run with the following command:

$ tat
TEST ccsTest PASSED.
PASSED.

2.5 Making more complicate tests

This section shows more advanced features, namely environment configuration.

Tat allows to configure local RTAP workstation environments, remote RTAP workstation environments, qsemu workstation environments (without RTAP) and LCU environments.

To show this, we will make a simple CCS/LCU test: writing a value in the WS environment database and executing a VxWorks binary on the LCU. Let's call WSName the environment configuration for the WS environment and LCUName the environment variable for the LCU environment.

The TestList file needs only to contain (assuming that the WS test script is called envTest.sh and the VxWorks script is called envTest.vw):

ENVIRONMENT WSName WS
ENVIRONMENT LCUName LCU
envTest.sh
envTest.vw

2.5.1 workstation environment configuration

· create a directory ./ENVIRONMENTS/WSName/dbl
· create a file named ./ENVIRONMENTS/WSName/dbl/DATABASE.db whose contents is:
//****************************************************************
// CCS DEFINITIONS
//****************************************************************

// Loads classes definition from standard file
#include "CCS.db"

//****************************************************************
// APPLICATION DEFINITIONS
//****************************************************************

// Add hereafter branches defined by your applications
// and that you want to load
#include "ENV.db"

· create a file named ./ENVIRONMENTS/WSName/dbl/ENV.db whose contents is:
// Loads classes definition from standard file
#include "CCS.db"

// for envTest

POINT NULL_CLASS Appl_data:envTest
BEGIN
ALIAS envTest

ATTRIBUTE int32 scalar_int32
END

· in the Makefile template ./ENVIRONMENTS/dbl/Makefile, add the following lines:
# On-Line Database Files
# ----------------------
DBL_CLASSES =
DBL_BCF = ENV DATABASE
DBL_LOAD = DATABASE
DBL_FLAGS = -v0

2.5.2 LCU environment configuration

We need a VxWorks binary named envTest:

· in the current test directory, create a file name envTest.c that is just printing "test OK"
· in the current test directory, create the corresponding Makefile for VxWorks.
· create the directory ./ENVIRONMENTS/LCUName
· create a file named ./ENVIRONMENTS/LCUName/userScript containing the following line:
# envTest.vw userScript
lcubootAutoLoad 1,"envTest"

2.5.3 Environment allocation and creation

When running tat makeEnv, tat will do the following:

· runs make clean all in the current directory and generates ../bin/envTest
· allocates the environments specified in TestList
· creates the local RTAP workstation environment by merging standard files from $VLTROOT/templates/forEnvs/RTAP/ with customized files from ./ENVIRONMENTS/WSName into $VLTDATA/ENVIROMENTS/w<HOST><xx> where w<HOST><xx> is the allocated RTAP environment
· starts the local RTAP workstation environment w<HOST><xx>
· creates the LCU environment by merging standard files from $VLTROOT/templates/forEnvs/LCU/ with customized files from ./ENVIRONMENTS/LCUName into $VLTDATA/ENVIRONMENTS/l<LCU> where l<LCU> is the allocated LCU environment
· reboot the LCU by executing the standard bootScript and then ./ENVIRONMENTS/LCUName/userScript.

If you run tat with the verbose mode, you get the following output:

$ tat -v makeEnv

Allocated WS environment for WSName: wte4901
Allocated LCU for LCUName: lte42
Creating target directory for WSName.
Starting autoloading environment for WSName.
Starting real environment for WSname.
WSName environment successfully created.
Creating target directory for LCUName.
LCUName rebooting vxWorks.
Executing bootScript of LCUName.
LCUName environment successfully created.

If you only want to know which environments have been allocated, look at the .testSession file:

$ cat .testSession
LCU LCUName lte42
WS WSNAme wte4901

2.5.4 Writing the tests

Our test needs to:

· write the local workstation database: this can be easily done with the shell by using the dbWrite utility on the workstation side
· execute the envTest program on the LCU: this requires to connect to the LCU, to send the execution command, to test the output of this command and to logout from the LCU. This is made easy by the followin feature of tat: if (and only if) the test script has the .vw suffix, tat assumes that it is aVxWorks script and executes it on the LCU (tat will automatically filter the name of the LCU and the VxWorks output lines containing the string "value =").

# envTest.sh (workstation side)
# 1. write the DB point created by ENV.db
dbWrite "@WSName:Appl_data:envTest" 666
if [ $? -ne 0 ]
then echo "dbWrite failed"; exit 1
else echo "dbWrite OK"
fi
# 2. execute the expect program
expect -f envTest.tcl


Here is the VxWorks script envTest.vw:

# envTest.vw
#
envTest

Note that the workstation script uses only WSName to reference the allocated environments and that the VxWorks script uses by default the environment variable LCU: all these variables are set by tat when running a test; environments variables specified in TestList are set to the corresponding allocated environments; LCU is always set to the LCU host name (not the LCU environment name) of the first LCU environment allocated by tat. RTAPENV is also set by tat to the first WS environment allocated.

2.5.5 Generating the test reference

Run the commands:

$ tat -g envTest.sh
Reference envTest.sh.ref generated
Reference file generated.

$ tat -g envTest.vw
Reference envTest.vw.ref generated
Reference file generated.

Tat has generated the reference files in the ref subdirectory:

$ cat ref/envTest.sh.ref
dbWrite OK

$ cat ref/envTest.vw.ref
test OK

The output of the test is redirected to the file envTest.out in the directory /tmp:

$ cat tatlogs/run<pid>/envTest.sh.out
dbWrite OK
envTest VxWorks OK

Instead of calling tat twice, you can also run everything in one go with the following command:

$ tat
TEST envTest.sh PASSED.
TEST envTest.vw PASSED.
PASSED.

Running tat without any option is equivalent to the following operations sequence:

· tat makeEnv: generation of test software, allocation and creation of all environments specified in TestList
· tat <test>: execution of all test scripts specified in TestList
· tat cleanEnv: cleaning of the test software and releasing of all environments specified in TestList.

2.6 Other types of environment supported

2.6.1 Remote workstation environment support

Tat allows to work with remote RTAP workstation environments. A remote workstation environment is specified in TestList with the RWS keyword instead of WS for a local workstation environment. Example:

# This is a remote workstation environment specification
ENVIRONMENT RemoteEnvName RWS

Remote environments are located on the machine specified by the RHOST environment variable. TestList syntax does not allow to specify the remote machine: all remote environments allocated during a test session are on the same machine. If the environment variable RHOST is not defined, remote environments are created by default on the local host.

2.6.2 CCS lite support

Tat allows also to work with CCS lite (i.e.without RTAP): in this case, a local CCS lite environment is specified in TestList.lite with the QS keyword. Example:

# This is a CCS lite environment specification
ENVIRONMENT QsemuEnvName QS

Note the major differences between CCS and CCS lite environment support:

feature
CCS
CCS lite
tat input file
TestList
TestList.lite
local WS environment type
WS
QS
concurrent local WS environment ?
yes
yes
remote WS environment type
RWS
not available

LCU environments are supported in both environments.

If the test has exactly the same output in both environemnts , you can have the same test script and the same reference file. If the test does not have the same output in both environments, you should have two different tests (two differents tests scripts and two different references).

2.7 More on environment configuration

Standard environments are defined by the module envs and corresponding templates can be found under $VLTROOT/templates/forEnvs/. Tat use the vcc services to manage environments (creation, starting, deletion).

2.7.1 WS/RWS environment configuration

The ENVIRONMENTS_RTAP man page lists all the file that can be customized. It is not recommended to customize any RTAP database file (RtapDiskDb, RtapHdrSnap, RtapRamSnap and RtapSnapCtrl). Note that if you want to modify the list of processes to be started by RtapScheduler you should modify the file RtapEnvTable.normal. Files already existing under $MODROOT/dbl need not to be duplicated under the ./ENVIRONMENTS directory: tat defines automatically MODROOT and takes care that the corresponding make db uses also $MODROOT/dbl files.

2.7.2 QS environment configuration

If you want to modify the list of processes to be started by ccsScheduler you should modify the file CcsEnvTable. Files already existing under $MODROOT/dbl need not to be duplicated under the ./ENVIRONMENTS directory: tat defines automatically MODROOT and takes care that the corresponding make db uses also $MODROOT/dbl files.

2.7.3 Scan system configuration

It is possible to configure the database both on WS side and LCU side for the scan system. To do this, create a file named ./ENVIRONMENTS/<envName>/dbl/USER.db.scan from the standard template USER.db and replace environment names by the environment variable references (without leading `$').

Example:

· If your TestList file looks like this one:
# TestList for scan system test
ENVIRONMENT WSname WS
ENVIRONMENT LCUName LCU
scanTest
· the file ./ENVIRONMENTS/WSName/dbl/USER.db.scan should contain:
POINT "<VLT scan dev>" "ccs_config:scan config:LAN:LCUName"
BEGIN
ALIAS "LCUName"
END
· the file ./ENVIRONMENTS/LCUName/dbl/USER.db.scan should contain:
POINT "<VLT scan dev" "LAN:WSName"
BEGIN
ALIAS "WSName"
END
· Running tat makEnv executes the following steps:
· execution of make clean all
· allocation of one local RTAP environment for WSName and of one LCU environment for LCUName
· creation of ./ENVIRONMENTS/WSName/USER.db from ./ENVIRONMENTS/WSName/USER.db.scan by replacing occurences of WSName with the allocated environment
· creation of ./ENVIRONMENTS/LCUName/USER.db from ./ENVIRONMENTS/LCUName/USER.db.scan by replacing occurences of LCUName with the allocated environment
· for each environment, merging of the customized files with the standard templates; creation of the target directories and starting of the environments.

Note that tat does not take care of the global configuration of the scan system: additional configuration steps should be added to the user test script (scanTest).

2.7.4 LCU environment configuration

The ENVIRONMENTS_LCU man page lists all the files that can be customized. tat uses vcc to manage LCU environments which uses itself the lcuboot module: any lcuboot statement is supported in the userScript file (note that you cannot modify bootScript).

If not already defined, tat always defines MODROOT to the current module under test: this feature allows userScript to load a VxWorks binary from the current module by using lcuboot services.

Since lcuboot takes cares of the general LCU setup, userScript shall never hard-code any environment variable already defined (as VLTROOT, MODROOT, HOSTNAME, HOSTENV, HOSTIPADDR, hotTCPPort, LOCALHOST, LOCALENV, LOCALIPADDR, localTCPPort, BOOTROOT, etc.).

2.8 Automatic compilation and cleaning

Before running the makeEnv step, tat executes a make clean all to build the test binaries and adds to the PATH environment variable the bin subdirectory of the current module. The output of this compilation step can be found in file make.log in the current test directory. Afer running the cleanEnv step, tat executes a make clean.

The make clean step can be disabled by using the -nc option: in this case, tat only executes make all before the makeEnv step and nothing after the cleanEnv step.

If defined, the environment variable TAT_MAKE_ARGS is added to the make [clean] all command line of the makeEnv step.

Note however that tat does not compile the module itself: this is left to the user.

Those are the rule for tat output files cleaning:

1. tat clean the environments under $VLTDATA/ENVIRONMENTS with cleanEnv if interrupted
2. If the test is PASSED the related ./tatlogs/run<pid> is removed exept in case tat was executed with -nc option
3. If the test is FAILED the directory ./tatlogs/run<pid> is not removed
4. The standard make clean used under the directory test remove completely ./tatlogs. When tat execute make clean the argument MAKE_FROMTAT=defined is passed to the makefile to avoid the tatlogs cleaning.

2.9 Automatic filtering

Before writing the test output into ./tatlogs/run<pid>/<test>.out, tat executes the following operations:

· if the file TestList.grep exists, the initial test output is filtered by running grep -v -f TestList output. Every line of the test output containing patterns appearing in TestList.grep is removed.
· if the file TestList.sed exists, the initial test output is filtered by running sed -f TestList.sed output. Everyline of the test output is processed by the sed program TestList.sed.

These automatic filtering capabilities allow to remove from the output what should not be part of the test reference because it is not reproducable (for example, time stamps).

Note that environments names allocated by tat are always filtered at run time: ./tatlogs/run<pid>/<test>.out and ./ref/<test>.ref always contain environment variables specified in TestList instead of environments names allocated at run time.

2.10 Setting a test time out

It is possible to set a time out for a test: this feature is needed when running a large test suite to prevent one looping or hanging test to stop the whole test suite.

By setting the environment variable TEST_WAIT (in seconds), each test specified in TestList will be killed if it has been running for more than $TEST_WAIT seconds. Note that TEST_WAIT is global for all tests listed in a given TestList. TEST_WAIT does not take into account the time needed by the makeEnv and cleanEnv steps: it concerns only the time needed by the test script itself. If TEST_WAIT is not set, there is no timeout and a test may loop or hang a very long time.

2.11 Using a dedicated LCU

Some tests requires to work with specific hardware only available on a given LCU. To make a test with a dedicated LCU whose environment name is <lcuEnv>, do the following steps:

· if the LCU comes from the tat LCU pool, lock it:
$ touch $LCUROOT/LOCK.<lcuEnv>
$ echo "$HOST $HOME" >> $LCUROOT/LOCK.<lcuEnv>
· create your private LCU pool by redefining LCUROOT in your login session:
$ mkdir $HOME/myLCUROOT
$ setenv LCUROOT $HOME/myLCUROOT
$ touch $LCUROOT/<lcuEnv>
· when the test is over, if the LCU comes from the LCU pool, unlock it:
$ unsetenv LCUROOT
# in a new login session
$ rm $LCUROOT/LOCK.<lcuEnv>

3 REFERENCE

3.1 tat(1)

NAME
tat - automated test driver


SYNOPSIS
tat [-v ] [-nc] [-g] [all | makeEnv | cleanEnv | [ <testid> ...] ]


DESCRIPTION
Looks on the current directory, normally "/test", for a file named
TestList on a full CCS installation ($RTAPROOT defined)
TestList.lite on a CCSlite installation ($RTAPROOT, $NOCCS not defined)
TestList.NOCCS on a NOCCS installation ($NOCCS defined)
which specifies the WS/RWS/QS/LCU environments to be created and the
tests to be run. tat executes the following steps:

- the environment variable SHLIB_PATH/LD_LIBRARY_PATH is redefined
to module lib directory first
- the environment variable PATH is redefined to module binaries first
- 'make clean all' is executed in ws and lcu test directories
to build the test binaries
- environments are allocated and created
- tests scripts are run (if needed, test scripts are made executable.
a test script may actually be any executable file in PATH).
- environments are deleted and released.
- 'make clean' is executed in ws and lcu test directories

The standard output and the standard error of test scripts are first
processed by :

- grep -v -f TestList.grep <output>
- sed -f TestList.sed <output>

and then are redirected into ./tatlogs/run<process_id>/<testid>.out
If this file is the same as ./ref/<testid>.ref, tat prints
"TEST <testid> PASSED" otherwise tat prints "TEST <testid> FAILED".
Note that for each allocated environment, tat subsitutes the allocated
environment name by the corresponding environment variable in the file
./tatlogs/run<process_id>/<testid>.out and ./ref/<testid>.ref.

Note the maching between the test driver steps and the log files:

- output of test script: ./tatlogs/run<process_id>/<testid>.out
- differences between reference file and output file:
./tatlogs/run<process_id>/<testid>.diff
In case of failures during environments creation, error messages are saved
under ./tatlogs/run<process_id>/<env> along whith the whole environment
for further checking, the environment under $VLTDATA/ENVIRONMENT is
always removed


TESTLIST SYNTAX
- ENVIRONMENT <envName> <type> specifies a environment to be created;

type QS : qsemu local environment (CCS lite only)
type WS : local RTAP environment on the local machine
type RWS: remote RTAP environment on a remote machine
type LCU: LCU environment

- <testName> specifies the script to be called to run the test (no
parameters allowed). The script must be in the current directory.
If the script is not executable, 'tat [-g]' will make it.
If the testscript has the ".vw" suffix, tat assumes that it's
a VxWorks script and it's executed on the LCU environment
listed in TestList. Tat filters automatically the name from the
LCU and lines containing "value = " from the VxWorks shell output.

-<envName> is the name of the environment variable defining the environment
name: it is NOT the name of the environment. Environments are dynamically
allocated: there is no way to use a given RTAP environment or use a given
LCU environment. During test execution, RTAPENV is assigned to the first
WS environment described in TestList and LCU is assigned to the first LCU
described in TestList (the value of LCU is the name of the LCU not the
name of the LCU environment).

Commented line begin with the # character.


ENVIRONMENT CREATION
Environments are built from scratch. The module test subdirectory shall
contain a ./ENVIRONMENTS/<envName> subdirectory for every environment
listed in TestList. Note that even if such a directory is empty, it shall
still exist; in this case, standard templates are used. Otherwise files
needed to create the environment referenced by <envName> are copied from
./ENVIRONMENTS/<envName>.

For a WS/RWS environment, all the files under ./ENVIRONMENTS/<envName>
are taken into account and have priority. This includes
RtapEnvTable.startup (vccEnvInit), RtapEnvTable.normal (vccEnvStart)
dbl/Makefile, dbl/*.db, dbl/*.class, etc. In addition, files under
<module>[/ws]/dbl are also taken into account by dbl: instead of being
copied into the target directory, they are searched by dbl when executing
'make db' using the MODINC environment variable.

If these files don't exist, files are taken from the following directory:
$VLTROOT/templates/forEnvs/RTAP.

For a LCU environment, all the files under ./ENVIRONMENTS/<envName>
are taken into account and have priority. This includes userScript,
logFile, rebootFile, devicesFile, dbl/Makefile, dbl/*.db, etc.
The only exception is that bootScript should NOT exist under
./ENVIRONMENTS/<envName>: if you want to load specific modules, add them
to userScript using lcuboot services. bootScript is always generated
at run time and cannot be directly modified. bootScript executes
userScript before starting LCC.

If these files don't exist, files are taken from the following directory:
$VLTROOT/templates/forEnvs/LCU.

Note that userScript should not contain any reference to the WS/RWS
environment: the statements defining VLTROOT, VLTDATA, HOSTNAME, HOSTENV,
HOSTIPADDR, hostTCPPort, LOCALHOST, LOCALENV, LOCALIPADDR, localTCPPort,
etc., are automatically generated by the lcuboot into bootScript.


SCAN SYSTEM CONFIGURATION
If the file ./ENVIRONMENTS/<envName>/dbl/USER.db.scan contains references
to the environment variables for the environments (<envName>), these
references are subsituted with the allocated environment names before
building the DB. Note that :
- this works only with USER.db.scan (not with USER.db !)
- a reference is just the environment variable name without any prefix
or suffix (no '$' !).


TEST OF VXWORKS BINARIES
If not already defined, tat defines MODROOT and it is possible to load
additional vxWorks modules using lcuboot functions by adding them
to userScript. Since lcuboot adds $MODROOT/bin to BINPATH), MODROOT is set
to <module>/lcu for a module having both "ws" and "lcu" subdirectories,
otherwise MODROOT is set to <module>.


OPTIONS
makeEnv: run 'make clean all ' in ws and lcu test directories; allocates
all the WS and LCU environments and creates them: RTAP environments have
been created and started, LCU have been allocated and rebooted.
'makeEnv' step fails the target directory is not writable by the current
user.

<testName>: run only the script <testName>. This option assumes that
'tat makeEnv' has been run before. tat redefines the environment variables
specified in TestList so that the environments created by makeEnv can be
used. If 'tat makeEnv' has not been run, the message " cannot open
.testSession" is displayed and tat exits. Several test id. may be
specified. If needed, the test script is made executable.

cleanEnv: frees and cleans all the environments allocated by makeEnv
option. RTAP environments are shut down, corresponding directories are
deleted. LCU boot directories are deleted; run 'make clean'.

all: this is the option by default. If no test session is active,
it's equivalent to the following options sequence:

- 'make clean all'
- makeEnv
- run all tests
- cleanEnv.
- 'make clean'

If a test session is active, this option executes all test listed
in TestList.

-v: verbose mode. Displays allocate environment and detailed
steps of environment creation and deletion.

-g: generates the test reference in ./ref/<test>.ref. A former reference
file is overwritten. The only other valid option with -g are 'all', '-g'
and <testName>. If <testName> is specified, the makeEnv and cleanEnv steps
are not executed. By default, all tests specified in TestFile are executed.

-nc: Run 'make all' instead of 'make clean all' before environments
allocation and creation; do not run 'make clean' after test execution.

-l: allows to loop on a set of test directories by using the
the same workstation environment and the same LCU environment.
All test directories must be under the current directory and are all
taken into account. All test directories MUST have the same environment
variable names for WS/LCU environments (./ENVIRONMENTS/<envVar>) in
TestList. In the first test directory, tat runs 'makeEnv'; in the last
test directory, tat runs 'cleanEnv'; in all test directories, the LCU is
rebooted once and all tests are run. Verbose option should by used to print
current directory under test. Note that it is not yet possible
to specify the first directory to be tested and that environments may not
be properly released in case of interruption or internal error.


RETURN VALUES
0: tat succeeded in executing the test (wether tests have failed
or not)
1: tat failed to execute one test and managed to release allocated
environments
2: tat interrupted


ENVIRONMENT VARIABLES
If not defined, tat define MODROOT (see test of VxWorks binaries).
NO_ERR_DISPLAY is always set to TRUE.

LCUROOT is the directory of the LCUs pool.

RTAP_WAIT specifies the timeout for a RTAP environment starting;
default is 60 seconds (for both local and remote environments).

Remote environments are only supported on one machine defined
by the RHOST environment variable. Default value of RHOST is $HOST.

VLT_VCCBOOTTIMNE specifies the timeout for a LCU to boot vxWorks or
to execute its bootscript. Default value is 300 seconds. Can be overriden
only with greater values.

If defined, TAT_MAKE_ARGS is passed as additional argument to the
'make [clean] all' command.

TEST_WAIT specifies the timeout for a single test in seconds:
if not defined. there is no timeout. If defined, every test in
TestList running more than $TEST_WAIT seconds is killed by tat.


CAUTIONS
Each sub directory of ./ENVIRONMENTS should be owner writable.

Environments allocated by running 'tat makeEnv' are booked until
the command 'tat cleanEnv' is run. When developing a test, special
care should be taken not to book "forever" environment ressources.

Do not mix TestList.grep and TestList.sed syntax: it's the best way
to get a empty reference file.

RtapEnvTable is not taken into account: use RtapEnvTable.normal
Do not customize bootScript: use userScript.

Never hard code in your test reference file, any host name or address,
environment name, port number or even user name or file name refering
to your test directory.

It should be mandatory for the user to re-test once its module tests have
been archived, possibly from a different computer using different
environments.


BUGS
userScript cannot load VxWorks binaries from a NFS mounted directory.

On Solaris 2.4, local test directory must not be already automounted:
do a "cd" to the file system whose name is listed in the "Mounted on"
column of df output.

Files under $MODROOT/dbl are _not_ taken into account by makeEnv step
if they already exist under $VLTROOT/dbl or $INTROOT/dbl.

If environment variables defined for WS/RWS/LCU environments have
similar patterns so that one name is a sub pattern of another name
(ex: WSname and RWSname), the scan system configuration fails.

Tabs instead of blanks in TestList triggers a syntax error.

An echo statement in a init file (.cshrc for example) makes tat fails
because vccEnvCreate fails.

Tat does not work if executed in background (it seems that RtapUnlockExe
hangs in this case).

Tat must use the sequencer tcl shell (seqSh) because it uses vccLib
procedures: this means that it is not possible to use tat to test the
sequencer with Purify.


SEE ALSO
tatEnvStatus(1), vccEnv(1), envsCreate(1), ENVIRONMENTS_RTAP(5),
ENVIRONMENTS_LCU(5), ENVIRONMENTS_QSEMU(5).



- - - - - -
Last change: 28/10/98-08:36


4 INSTALLATION GUIDE

4.1 tat installation

If your VLTROOT dates after june 1996, skip this section: tat should be already available.

If you need to install the tat tool, retrieve the module tat from the cmm archive, then remember that:

· tat requires the sequencer, vcc and lcuboot modules
· tat uses the tcl like expect interpreter, installed with other tcl interpreters.

To install a new version of tat, just run make clean all man install from tat/src/ .

4.2 tat configuration

tat allocates dynamically WS and LCU environments. These environments are taken from pools that need to be configured by hand and inserted in the ACC data base (see below).

4.2.1 Remote shell privilege

Each tat user should be able to execute a remote shell on the current workstation (even if he does not use any remote environment):/etc/hosts.equiv or $HOME/.rhosts should take this into account. So, add in the appropriate file the record:

<ws_name> <username>

where <ws_name> is the name (or fully qualified name) of the machine where <username> wants to run tat. Do the same for all the machines you want to use.

4.2.2 RTAP environment pool

The RTAP environment pool is local on every host. RTAP environments shall be named <string><number> with the following rules: if the host hame starts with the `w' character, <string> is set to the first 6 characters of host name, otherwise <string> is the result of the concatenation of the `w' character with the first 5 characters of host name. <number> ranges always from `0' to `9'. Since environment name length is limited to 7 characters, this means that there is a maximum of 10 tat workstation environment per host.

For example:

- if your host name is te49 and you want to have 5 RTAP environments in the pool, these environments shall be named wte490, wte491, wte492, wte493 and wte494.

- if your host is named wt0tcs and you want to have 5 RTAP environments in the pool, these environments shall be named wt0tcs0, wt0tcs1, wt0tcs2, wt0tcs3, wt0tcs4 and wt0tcs5.

Each environment must be added to:

· /etc/services.
From a communication point of view, each environment is identified by the node on which is running and a TCP/IP port number. The same number can be used on different nodes for the same type of environment. Currently we use:
· one number in the range 2001-2999 for each RTAP environment.
So, add in the /etc/services as many records as tat environments like:
wte490 2220/tcp
· /etc/$RTAPROOT/RtapEnvList. If you want to use remote environment, do not forget to add them to /etc/$RTAPROOT/RtapEnvList.
Each RTAP environment needs to know where other RTAP, QSEMU, LCU environments are located, they can be both local or remote to the workstation.
So add in the /etc/$RTAPROOT/RtapEnvList as many records as tat environments like:
wte490 /vltdata/ENVIRONMENTS/wte490 (if the environment is local)
wte350 te35 (if the environment is on a different workstation1)
· the VLT central configuration database: see [3] for more information. When adding workstation environmnents to this database, the attribute attic must be set to 1.
So, for each tat environment, add in the $VLTDATA/msql/accData.sql database, in the table "prog_environment " a record like:
INSERT INTO prog_environment VALUES ('wte490' ,'','RTAP',2220,'te49','','','',1 ) \g
Be careful: in the list "VALUES" the fourth field is the TCP port you inserted in the /etc/services file. The numbers must match!
Reminder: after each modification to the accData.sql file, you should run "accLoadData" to make the modification effective.

4.2.3 CCS Lite environment pool

The CCS Lite environment pool is local on every host. CCS Lite environments shall be named <string><letter> with the following rules: <string> is set as for RTAP environments, <letter> ranges always from `a' to `j'. As for RTAP, there is a maximum of 10 CCS lite tat workstation environment per host.

For example:

- if your host name is te49 and you want to have 5 CCS lite environments in the pool, these environments shall be named wte49a, wte49b, wte49c, wte49d and wte49e.

Each environment must be added to:

· /etc/services.
From a communication point of view, each environment is identified by the node on which is running and a TCP/IP port number. The same number can be used on different nodes for the same type of environment. Currently we use:
· one number in the range 2001-2999 for each QSEMU environment.
So, add in the /etc/services as many records as tat environments like:
wte49a 2240/tcp
· $VLTDATA/config/CcsEnvList.
Each QSEMU environment needs to know where other RTAP, QSEMU, LCU environments are located, they can be both local or remote to the workstation.
So add in the $VLTDATA/config/CcsEnvList as many records as tat environments like:
wte49a /vltdata/ENVIRONMENTS/wte49a (if the environment is local)
wte35a te35 (if the environment is on a different workstation2)
· the VLT central configuration database: see [3] for more information. When adding workstation environmnents to this database, the attribute attic must be set to 1.
So, for each tat environment, add in the $VLTDATA/msql/accData.sql database, in the table "prog_environment " a record like:
INSERT INTO prog_environment VALUES ('wte49a' ,'','RTAP',2240,'te49','','','',1 ) \g
Be careful: in the list "VALUES" the fourth field is the TCP port you inserted in the /etc/services file. The numbers must match!
Reminder: after each modification to the accData.sql file, you should run "accLoadData" to make the modification effective.

4.2.4 LCU environment pool3

The LCU pool is global to your local network because a LCU is not dedicated to one host; it is not managed by the VLT central configuration database. You need to have one directory NFS mounted on every workstation where tat tests will be run; this directory should be writable for every tat user:

$ mkdir <LCUpoolPath>
$ chmod g+w <LCUpathPath>

For each LCU dedicated to automated testing, execute:

$ touch <LCUpoolPath>/<LCU environment name>

Note: LCU environment name is the name of the LCU prefixed with an `l'.

Each environment must be added to:

· /etc/services.
From a communication point of view, each environment is identified by the node on which is running and a TCP/IP port number. The same number can be used on different nodes for the same type of environment. Currently we use:
· 2160 for LCU environments;

Add in your standard environment definition on each workstation:

$ setenv LCUROOT <LCUpoolPath>

5 TROUBLE SHOOTING GUIDE

To make automatic testing in the VLT common software environment is not so easy. Usually you need one workstation environment using RTAP and one LCU environment. And everythind needs to work together at the same time...This section tries to give some hints when something does not work as expected.

5.1 Environment allocation problem

The messages "No more LCU available under $LCUROOT" and "No more WS environment available under $HOST:$VLTDATA/ENVIRONMENTS" are self explanatory. But the reason may not be obvious.

Running the command tatEnvStatus lists WS or LCU environments still locked but unused since the corresponding .testSession file does not exist any more. It may also be possible that somebody has forgotten to run tat cleanEnv (in this case tatEnvStatus cannot know it since the corresponding file .testSession still exists): take a look at the last modification time of the environments locks (/tmp/<envId>.lock for WS and $LCUROOT/LOCK.<envId> for LCU).

In any case, to manually unlock and stop environments, do the following:

· WS case:
$ vccEnvStop -e <envId>
$ rm /tmp/<envId>.lock
$ vccEnvDelete -e <envId>

· LCU case:
$ rm $LCUROOT/LOCK.<envId>
$ vccEnvDelete -e <envId>

5.2 File permission problems

Tat fails to create an environment if it cannot modify permissions of environment files. To avoid this problem, one should never change permisssions of the directories $VLTDATA/ENVIRONMENTS/<envId>. One should also avoid to create files by hand in those directories with unsufficient permissions: these directory trees should be world writable and remain so.

5.3 RTAP problems

Experience shows that most of RTAP starting problems are related with operating system resources availibility. Since RTAP uses a lot of IPC resources (messages queues, semaphores and shared memory), if you have a problem when creating a RTAP environment, check that :

· the kernel is configured as described in [1], especially with respect to the number of concurrent running environments.
· shared memory segments are available. If there are shared memory segments still listed with ipcs -m but not used , use ipcrm -m to remove them.

5.4 tat interruption

If tat is interrupted or fails with an "Error: ..." message, tat releases also the allocated environment resources and prints the message "Cleaning ...". However if tat is interrupted during this cleaning step, tat stops and some environments may not be cleaned or freed: refer to above section 5.1 to release environments by hand.

Try to interrupt tat only ONCE by typing <CTRL C> only ONCE otherwise environments will not be properly removed. It will always some take time for tat to stop but it will stop: please be patient...

5.5 Know issues

5.5.1 NFS related problems

· the possibility to load a VxWorks binary from $MODROOT on the LCU using userScript does not work if the $MODROOT directory is not on a local file system.
· tat does not work at all on Solaris 2.4 if run from a local test directory that is already mounted by automount: tat can only work if the file system name returned by pwd matches the one listed in the "Mounted on" column of the df command output: do a cd to the right file system and then to your test directory.

5.5.2 LCU booting problem

tat does not stop as soon as the LCU boot script encounters an error. In case of unexpected errors on the LCU, the output of the booting script (./tatlogs/run<pid>/<lcuEnv>/.reboot.log) should be carefully checked.

The environment variable VLT_VCCBOOTTIME specifies the general time out in seconds used by tat when waiting for the LCU to be ready. This variable value may be modified but now below 300 because a LCU booting can take some time.

5.5.3 Database configuration problem

tat does not take into account a file under $MODROOT/dbl if this file already exists under $VLTROOT or $INTROOT: the file taken into account will be the one under $VLTROOT or $INTROOT.

5.5.4 Interpreters usage

VLT programming standards requires that only the Bourne-Shell shall be used to write test scripts. If you really need to use another shell, take care that this shell behaves the same way as the Bourne shell with regards to environment variables.

1
This case arises when for instance the tat environment wte490 on workstation te49 wants to communicate with a remote environment wte350 on the workstation te35.

2
This case arises when for instance the tat environment wte49a on workstation te49 wants to communicate with a remote environment wte35a on the workstation te35.

3
This paragraph is actually out of date. Refer to [1], chapter 4, for more up-to-date information.



Quadralay Corporation
http://www.webworks.com
Voice: (512) 719-3399
Fax: (512) 719-3606
sales@webworks.com
TOC PREV NEXT INDEX