2 USER'S GUIDE

This chapter provides a functional description of the tat module.

2.1 Overview

What is automatic testing? Several conditions are required to automatize a test. First, the test should be executed by typing one (and only one) command. This command should prepare the general environment setup needed by the test, execute the test and tell wether the test has succeeded or not. To be able to determine automatically wether the test has succeeded or failed, the test should, generally speaking, be reproductible. That means that the output of test (possibly after some automatic filtering) should be reproductible.

A automatic test developed with tat can run without any modifications on any host using any LCU(s). tat enables also simultaneous executions of the same test on the same host.

tat has been specifically designed for VLT software tests: it does handle any test based on batch programs. Some cases of GUI programs may also be handled by tat but this feature is not in the scope of the present issue of this document.

2.2 Concepts

The tat module works with the following concepts:

- the test driver

- the single test

- the environment, as defined by CCS.

None of these concepts are revolutionary or even new: some customized test drivers have already been developed for CCS modules and the tat test driver owes much eccsTestDriver.

2.2.1 The test driver

The test driver is the conductor of the automated tests. It does - almost - everything: its features

make up a good overview of the test software:

1. the test driver looks for a file named TestList specifying environments and tests

2. the test driver executes make clean all to generate the test software

3. the test driver allocates and creates environments

4. the test driver executes the test, compares the test output with the test reference

5. the test driver executes make clean

6. the test driver deletes and frees environments.

2.2.2 Environments

Environments should not be hard-coded in test scripts. If so, it would be practically impossible to execute a test on another computer with another LCU (because the LCU environment name is based on the LCU name). As a consequence, environments shall only be refered to with environment variables: this is the rule for all environments, whether they are local WS, remote WS or LCU.

When starting a test, the test drivers allocates environments from environment pools dedicated to tat and assigns the chosen environment names to the corresponding environment variables. This method has also the advantage to free the developer to look for given environments before running each test. And it is also possible to execute the same test on the same machine.

Environments may be customized by creating files in ./ENVIRONMENTS/<envName>: when creating environments, tat will first search files into these directories before searching in the standard directories under $VLTROOT.

At the end of the test, the test driver deletes the environments that it has created and gives them back to the pool, so that they can be reused by anybody.

2.2.3 The TestList file

TestList contains two types of directives:

· environment directive

· test directive

An environment directive is a line containing the following arguments:

ENVIRONMENT <envVar> <envType>

<envVar> is the name of the environment variable that tat will assign the allocated environment name to; <envType> is the type of the environment that tat will allocate and create; it should be one of the following:

· WS for a local RTAP workstation environment

· QS for a CCS lite local workstation environment

· RWS for a remote RTAP workstation environment

· LCU for a LCU environment

A test directive is a line containing the name of a executable command to be called. This command cannot have parameters and identifies the test.

Note that environments are created at the beginning and accessible to all tests. Using <envVar> every test can select the needed environment(s).

If there are no CCS environment directive, the user should take care that its test is reproducable by:

· creating from scratch the needed environment (input data, etc...) at the beginning of the test

· removing the needed environment at the end the test.

2.2.4 the single test

For tat, a test in a executable command (usually a shell script) called by the test driver: this script shall not interfere with the test driver; it should not create environment or modify them if this should be done by the test driver itself.

A test may call any program, script or utility provided that this utility is non interactive and portable among supported platforms: these prerequisites ensure that the test output is repeatable and may used to make a test reference. The test driver calls the single test and records the output in ./tatlogs/run<pid>/<test>.out. A special case is when the test driver is instructed to build the reference file: the output of each single test is recorded in ./ref/<test>.ref. The output may also be filtered to remove non repeatable output (like dates, times, etc.) by the test driver.

2.3 Usage

Tat has been designed to serve:

- development by:

· creating the environment once at the beginning: tat makeEnv

· repeating as many time as needed the sequence : editing the test and running the test: tat <test>

· generating the reference file: tat -g <test>

· releasing the environments: tat cleanEnv.

- integration: by running tat on a sequence of modules.

Quadralay Corporation http://www.webworks.com Voice: (512) 719-3399 Fax: (512) 719-3606 sales@webworks.com