| |
|
|
|
2.8 MESSAGE SYSTEM
2.8.1 Purpose
The CCS message system provides a similar access to the basic communication facilities, independently of the location of the communicating processes: local or remote WS or LCU. However, the internal mechanisms used to send messages to a local destination differ from the ones used to reach a remote destination. On the WS, the CCS Message system is built on top of the Rtap program-to-program communication mechanism. Based on Unix message queues for local communications and ARPA/Berkeley socket connection for remote communications, it provides a sequenced, reliable, two-way-connection. TCP, the underlying protocol for LAN communications ensures that data is not lost or duplicated. The complete functionality of the message system is fully supported under CCS_LITE.
2.8.2 Basic Concepts
Interprocess communication is based on the exchange of messages. A message is a formatted stream of bytes. It contains references to the source process, the destination process, some structural information, and the user data. User messages transported by the CCS message system fall into two main types:
Replies always refer to a previous command.
The CCS message system makes use of a private module cmd, to perform the syntax checks on commands to be sent.
2.8.2.1 Protocol
A generalized protocol is defined for the program-to-program communication level, both between workstations and LCUs. Each application shall follow it.
1. Communications are always initiated with a command sent from an originating application to a destination application.
2. The send operation performs standard checks on the message (including on its content if requested) and attempts to queue the message at the destination side. The send process is blocked until an acknowledgement is received or a timeout has elapsed to certify that the message has been correctly delivered to the destination or that the transmission has failed. The acknowledgement and the timeout are generated by the internal transport layers.
3. Applications can ask for messages at any time, activating a receive operation. The receive removes from the queue, the first message according to the current filtering option. Both commands and replies are received in this way.
If the message is a command, and the booking checks are activated, it is checked against the current booking situation:
· if the check is negative: a reply reporting the error is automatically sent back to the originating application. This reply terminates the command logically. Nothing is returned to the destination application and the message space is available to accept the next commands.
· if the command is accepted: the incoming command is passed to the destination application that processes it.
4. The destination application shall use the check subroutines together with the parsing subroutines (See section Functions) to decode commands. It then processes the request expressed by the command and produces as many replies as needed. If the command cannot be satisfied within a short delay (typically a few seconds), a first immediate reply can be returned to report that the command has been accepted (The protocol has been simplified, and this acceptance reply is no more mandatory). If the execution of the command requires several replies, the last one shall be clearly indicated. In case the request cannot be satisfied, the reply shall report an error message handled as a special type ERROR_REPLY. In that case, the ERROR_REPLY concludes the treatment of the command, and is automatically flagged as lastReply.
2.8.2.2 Architecture
Messages are exchanged between processes. A process belongs to an environment, that can be located either on a WS or a LCU. Each process has a queue for incoming messages. For each machine, both WS and LCU, a server is provided to handle messages coming from other machines and directed to processes located in that machine.
If the destination process is on the same environment as the sender, the message is put directly in the destination queue.
If the destination process is on another environment (on the same host or on another one), the message is sent to the server on the destination machine. The server puts the message in the destination queue and, if the message is a command, returns an acknowledgment message.
2.8.2.3 Security
With the JANUARY 2006 release, some security checks have been added to the communication mechanism between environments. In particular, any attempt to initiate communications with a remote environment will be checked on the destination side against the configuration declared in the CcsEnvList file on that side. This file is located in $VLTDATA/config, and is normally created and populated by the vcc tools using configuration data extracted from the ACC Database. The file contains one entry per environment telling on which host it is supposed to run. In case the host is on the local WS, an additional entry provides the local environment directory. The security check verifies that the message comes from the IP address corresponding to the host configured in the CcsEnvList file. If both match, the communication is established and further messages from that environment will be processed without checking the IP address. In case of mismatch, the message is rejected and a log is produced, reporting the attempt.
2.8.2.4 Queue Handling
Standard Unix message queues are used for communications. The CCS Message System makes use of the Rtap enhancements and also provides some extension to perform filtering on the received messages:
· If no filtering is requested, the input queue is managed as a FIFO, and messages are received in chronological sequence.
· If filtering is applied, messages are always received internally on a priority basis by a process, ensuring that replies are delivered faster than commands. The subsequent handling of a message within a process depends on whether it matches the filter or not. If not it is put on an internal, process specific queue until a request is performed with the corresponding filter, otherwise it is processed immediately. Until the APRIL 2004 release the internal queue was used only in case the filtering criteria couldn't be satisfied with the IPC resources management capabilities. However, since the Unix message queues have a fixed maximum size of 64 kB, messages not matching the filter could quickly saturate that amount of memory, potentially leading to deadlocks. Since the APRIL 2004 release, whenever filtering is used, any message not matching the criteria is moved to an internal storage area, releasing queue resources for other incoming messages, and thus greatly reducing the risks of deadlocks. This mechanism is almost transparent for the application: the required memory (in the application space) being managed on demand by the message system itself.
2.8.2.5 Message Structure
A message is formed by identification information and user data. The user data can be another message.
· identification of the sender and of the destination in terms of process and environment, including the identification in the sense of the booking system.
· message type: command/reply/last reply (in case of multiple replies)/error replies/obituaries/event
· User data: According to the command the message is referring to, user data can be ASCII formatted data, formatted binary data organised like C-style structures, and free-formatted binary data.
· ASCII formatted data is organized as a string where parameters are position dependent. Compulsory separators provide the position information. Commas are used to separate values for different parameters, spaces as separators between multiple values for the same parameter.
· Validity checks are performed only on formatted data (ASCII or C-style structures), free formatted binary data does not support any syntax check.
· As an alternative, User data can be another message. This allows message nesting. Typical uses of it: requeueing of a message, communication handlers, etc. The only limitation to nesting is the total message length.
The message structure can be summarized by the following rules:
message = identification information + (user data | message)
user data = formatted ASCII data |
formatted ASCII data = [parameter [ {separator + parameter} ] ]
formatted binary = C style structures
free-formatted binary = byte string
2.8.2.6 Monitoring the Input Queue
Normally a process expects input only from its message queue, which is established on registration to a CCS environment. But sometimes it also needs to be able to accept asynchronous input from additional sources like a pipe or a file. In order to avoid polling the message queue and loading the System, the CCS Message System provides the function msgMonitorQueue() allowing to monitor the queue by msgSelect(), which is built on top of the standard OS function select().
msgMonitorQueue() returns a file descriptor, which may be used in the read mask of the msgSelect() function.
For more detailed information as well as an example of how to use this function see man-page of msgMonitorQueue().
2.8.2.7 Command Definition Table and Syntax Checks.
The definition of normal commands an application can understand is left to the application designer. However, mandatory commands are defined for classes of applications like LCU applications, INS, etc. A command definition table must be produced for each individual process accepting commands, Command Interpreter or other. This table is used to preprocess commands typed from the user interface or sent by the sequencer. A description of the table and an example of such a table are given in the LCC User manual.
For each command the table shall contain, in particular:
· The default reply used in simulation (this includes the possibility to refer to on-line data base variables).
A command definition table must be built for each application. Rules as defined for the present release of CCS/LCC are the following:
One entry per command, attributes are given after the KEYWORDS. Keywords are typed in upper case and are followed by "=". Spaces are allowed before and after the "=" character. All keywords, except for optional keywords, are mandatory and must be given in the order as defined below.
d. COMMAND=command name (7 chars max). Command names are not case sensitive and are always processed as ASCII upper case only (allowed character set: A-Z and 0-9, first character must be a letter).
e. SYNONYMS=A list of accepted synonyms, any number of characters, separated by commas. They are only used when commands are typed from UIF and they are replaced by the equivalent command when the message is sent. This list is optional. It is there just in case the 7 characters allowed for the command are not enough to give meaningful names in all circumstances.
f. FORMAT=How parameters are formatted: "A" for ASCII, "C" for structured binary, "B" for unformatted binary.
With ASCII format all parameters shall be sent as strings in the message body.
With structured binary format the parameters shall be sent in binary format of the type defined by the keyword PAR_TYPE.
With unformatted binary format the parameters can be sent in any format. No syntax checking will be done.
The receiving process must be able to handle the format of the message body.
g. PARAMETERS=A list of parameters: Parameters are position dependent. This list determines the sequence. The keyword "PARAMETERS=" just introduces the list. The list contains the following entries:
i. PAR_NAME=parameter name. Start a new parameter description. The parameter name shall be short, but a maximum length of 256 characters is allowed. Allowed character set: A-Z, 0-9, underscore and decimal point.
ii. PAR_UNIT=parameter unit.(Official SI units and allowed multiples)
Defines the unit of the parameter in the message body. A user interface or any other process accepting other units shall convert to the unit specified here before sending the command. PAR_UNIT is not applicable for parameter of type LOGICAL.
iii. PAR_TYPE=parameter type: STRING, INTEGER, REAL, LOGICAL
INTEGER is 32 bits, REAL is 64 bits (double), LOGICAL can have the value ccsTRUE or ccsFALSE.
iv. PAR_RANGE=parameter range (static) Interval or enumeration
If interval, given with INTERVAL MIN=xxx;MAX=xxx
If enumeration, given with ENUM val1,val2,.....,valn
v. PAR_OPTIONAL=YES, NO. The flag indicates if the parameter is optional. If the flag is set and a default value is defined, the default value is sent in the message buffer if the parameter is not given in the input. If the flag is set and no default value is defined, no value is sent if the parameter is not given in the input. The left-out parameter will in the message buffer be indicated by two consecutive commas (e.g. if for a command with the parameters par1, par2 and par3, par2 is optional, the message body will be: par1,..,par3) or if it is the last parameter with one comma less (e.g. if par 3 is optional, the message body will be: par1,par2. The receiving side must also be able to accept the message with a trailing comma par1,par2,).
Only commands with ASCII format parameters can have optional parameters.
vi. PAR_DEF_VAL=default value if the parameter can be defaulted. Any parameter can have a default value. If no value is given for the parameter in the input, the default value if it is defined, shall be put in the message body.
Parameter of type LOGICAL can only have the default value FALSE corresponding to ccsFALSE, TRUE, corresponding to ccsTRUE, is not accepted. The default value for LOGICAL is valid also when not specified.
vii. PAR_REPETITION_FACTOR=Optional keyword indicating that the parameter is repeated a fixed number of times. The parameter must be repeated exactly the number of times specified. If the parameter is given in the input less number of times, the missing repetitions are replaced by the default value. A parameter with fixed repetition rate cannot be optional. Repeated parameters are separated by space in the message buffer.
Example: A command has three parameters par1, par2 and par3. Par2 is repeated twice. The message buffer will be
par1,par2 par2,par3
PAR_REPETITION_FACTOR is not applicable for parameter of type LOGICAL.
viii. PAR_MAX_REPETITION=Optional keyword indicating that the parameter can be repeated a variable number of times. The value gives the maximum number of repetitions, but it can also be repeated less times than this value. The keyword is not allowed if the keyword PAR_REPETITION_FACTOR is defined for the command. Repeated parameters are separated by space in the message buffer. A parameter with this keyword will only be sent the number of times given in the input and cannot be defaulted.
Example: A command has three parameters, par1, par2 and par3. Par3 has PAR_MAX_REPETITION=10. The message body will be, if par3 is input 4 times:
par1,par2,par3 par3 par3 par3
or if par3 is input twice:
par1,par2,par3 par3.
Only commands with ASCII format parameters can have parameters with variable repetition.
PAR_MAX_REPETITION is not applicable for parameter of type LOGICAL.
h. REPLY_FORMAT=How the reply is formatted: "A" for ASCII, "C" for structured binary, "B" for unformatted binary.
i. REPLY_PARAMETERS=An optional keyword with a list of parameters: Parameters are position dependent. This list determines the sequence. The keyword "REPLY PARAMETERS=" just introduces the list.
The reply parameter specification is optional. The rule is that parameters of the reply normally shall be specified. Most replies have a defined format. In case it is not possible to specify the reply, the definition can be left out.
The list contains the following entries:
iv. PAR_DEF_VAL=default value used only in simulation mode and cannot be used by applications. The value defined will be returned by the CCS simulator. The default value can be given as a value or as a database reference. The database reference shall be given as a database symbolic address starting with the character ":" for an absolute address or with specification of the environment as @env:absolute address or with the view specifier <relative> or <alias>.
v. PAR_REPETITION_FACTOR=optional keyword indicating that a parameter is repeated the number of times given by the value. The parameter must be repeated exactly the number of times specified.
j. REPLY_LENGTH=optional parameter for unformatted binary replies, defining the length of the default reply.
k. DISPLAY_FORMAT=An optional formatted string containing an ASCII string to be displayed when the reply is received. The string can contain c-type printf() formatting, where to plug in the parameters of the reply. The reply received must contain the number of parameters of the format specified in the string. Example: A reply of type formatted binary containing two values, one integer and one real, could have the following display format: "Temperature at sensor %i is %f degrees C".
The display format specification is optional. The rule is that a display format normally shall be specified. Most replies have a defined format. In case it is not possible to specify the reply, the definition can be left out.
If display format is not specified, the reply buffer shall be displayed in the format received.
2.8.2.8 Special Commands
In addition to the normal commands some special commands are predefined, which are handled automatically by all applications linked with the message system library. They are sent like any other command and trapped inside the CCS layers on the receiving side.
Enabling autologging of messages received and sent by an application.This command is trapped by the msgRecvMsg() function.
Disabling autologging of messages received and sent by an application.This command is trapped by the msgRecvMsg() function.
Verifying whether a process is able to send or receive messages.This command is trapped by the msgRecvMsg() function.
Sending the SIGUSR2 signal to the requested task causing the corresponding kill handler to terminate the task.This command is trapped by the msgServer.
Sending the SIGUSR1 signal to the requested task causing the corresponding break handler to perform the BREAK command. This command is trapped by the msgServer.
For detailed information about the configuration of the signal handlers connected to an application, see the ccsInit() function manpage.
2.8.3 Transfer of Data via Sockets
The CCS Message System also provides an interface enabling the direct use of sockets for transfer of large amounts of data between two applications.This is reserved for internal use, in particular by the dxf module. Applications shall use the higher level interface functions provided by dxf.
The message system implements the following functions:
A transfer of data between two processes (in the following called process A and process B) using this interface is realized as follows:
· and accepts the connection request after a positive reply on the msgCONNECT_CMD command has been received
2. The processes have to synchronize and call the corresponding CCS Message System functions in order to exchange data. There is no protocol defined for the actual transfer of data. The way a transfer is initiated/performed is entirely up to the designer of the applications using this interface. A procedure could be
· The sending process first sends a reply containing information about the amount of data available (if necessary) and then writes the data on the socket using msgSendData().
The command msgCONNECT_CMD is defined in the CDT-file msgSocket.icdt, which has to be included in the CDT of each application calling msgAcceptConnection().
For more detailed information as well as an example of how to use this interface see the man-pages of the functions mentioned above.
2.8.4 Programmatic Interface
The following functions are provided as part of the CCS Message System Library:
msgAcceptConnection() establishes a socket connection
msgCompareProcids() compares two process IDs
msgGetPidByName() returns the OS process ID of a process specified by its process name
msgMonitorQueue() enables/disables asynchronous monitoring of the message queue
msgOpenConnection() creates a socket connection between two pocesses
msgPing() checks whether a process can send/receive messages
msgParseMsg() returns pointers to the different parts of a received message
msgRecvData() reads data from a socket
msgRecvMsg() retrieves the next message in the input queue
msgScheduleProcess() schedules a process
msgSelect() synchronous I/O multiplexing
msgSendData() writes data on a socket
msgSendCommand() sends a message of type COMMAND
msgSendReply() sends a message of type REPLY or ERROR_REPLY
msgSetFilter() prepares a filter valid for a msgRecvMsg()
In order to use the message system facilities, an application has first to create the message queue to accept incoming messages and to register with an active environment. These operations are automatically performed by the ccsInit function, which must be called at the beginning of any program, before any other ccs function is called.
The basic steps involved in messages operations are:
· Define the destination process and environment. Normally the destination process must be active before any message can be sent to it. If necessary, first schedule the destination process: msgScheduleProcess().
· Send the message: msgSendCommand(), and handle possible errors. During debugging phase, it is recommended, whenever possible, to request a syntax check before sending the command (The command definition table must already have been loaded. cf. cmdSetup utility).
· Wait for a reply: msgRecvMsg(). If required, use filtering capabilities to return only when a reply from the command destination process is available. If necessary, handle timeout and related error returns.
· If the reply is of type msgTYPE_ERROR_REPLY, use the error number and mnemonic to check, if an error recovery procedure can be attempted or not. In case of successful recovery, reset the error stack, log a message to trace recovery and proceed. Otherwise add a new error to the stack meaningful for the next higher level and return it.
· If necessary, make a copy of the received message to avoid unexpected over-writing of the buffer by further access to the message system possibly hidden within some CCS/Rtap subroutines.
· If the command can be processed and satisfied, immediately return a/the reply(ies) containing the result of the command: msgSendReply(). Within the last reply the lastReply flag must be set.
· if expected, return an immediate reply: msgSendReply() reporting acceptance of the command and indicating that other reply(ies) will follow. This is not mandatory, it depends on the protocol agreed during design of the sending and receiving applications.
· Process the command and return as many replies as needed to satisfy the command. There must be a reply concluding the treatment of the command. That reply must be flagged as lastReply.
· If an error occurs during command processing, setup an error structure and return a final error reply: msgSendReply().
2.8.5 Daemons
The three following processes are required for the message system to operate properly. See below how to configure the environment to have them started.
· cmdManager, sending default replies for simulated processes. These replies are generated as defined in the command definition table belonging to the simulated process.
2.8.6 Utilities
The following utilities are part of the CCS Message System distribution:
· msgSend, enabling a user to send a single command and receive the corresponding reply(ies) in a script or interactively via the shell.
· msgSchedule, enabling a user to schedule a process on any active environment know to the local one in a script or interactively via the shell.
2.8.7 Examples
The following example, representing the source code of the program used to check the CCS installation procedure, shows how to send a command, receive a single reply, and handle errors occurring.
/***********************************************************************
* -------- -------- --------------------------------------------
* gfilippi 07/12/94 msgSetFilter added
/************************************************************************
* wsappSetValue - set an LCU variable according to user input
* This program is part of the VLT Release 1 installation verification test.
* It provides also an example of a Workstation application communicating
* via CCS Message System to an LCU application (lcuappStoreData). This
* example uses also the CCS Error System and the CCS Logging System.
* The program prompt the user for an integer number. As it is typed in,
* the number is formatted into a message (SETVAL) and sent to the LCU partner
* application (lcuappStoreData), that should stere the number into the
* LCU database. The reply message from the LCU closes the loop and a
* `q' as answer forces an EXIT message to be sent to the LCU, that
* should terminate, and as the reply from the LCU the process comes back
* this process terminate as well.
* <LCU_name> LCU environment name where lcuappStoreData runs
* RTAPENV must be defined to the WS local environment.
* wsappShowValue, lcuappStoreData
*************************************************************************/
int main(int argc, char ** argv)
ccsPROCNAME myProcess = "wsappSetValue";
ccsPROCNAME LCUproc = "lcuapp";
char sccsId[80] = "WS application example - Version 1.2, Nov 94";
printf("%s - %s \n", myProcess, sccsId); /* on the terminal */
logData (myProcess, WSAPP_LOG_START, sccsId); /* on the log file */
printf("\n\tusage: %s <LCU> \n\n",argv[0]);
strcpy((char *)LCUenv, argv[1]);
if (ccsInit(argv[0],0,NULL,NULL,&error) != SUCCESS)
if (ccsGetMyProcId(env, &orgId, proc, &error) != SUCCESS)
if (msgSetFilter(NULL, NULL, msgANY_MESSAGE, 0, "", &myfilter, &error) != SUCCESS)
* Get value from user and send it to LCU until `q' is pressed
printf("\nEnter a number to send to the LCU %s\nEnter `q' to quit\n\n", LCUenv);
* `q' has been typed: send EXIT to LCU, close db and exit
if (msgSendCommand("EXIT", 0, LCUenv, LCUproc, numString,
if (msgRecvMsg(&myfilter, 3000, &myMsg, &error) != SUCCESS)
if (ccsExit(&error) != SUCCESS)
else if (sscanf(numString, "%d", &number) == 1)
* a valid integer has been typed: send it to LCU using SETVAL and continue
if (msgSendCommand("SETVAL", 0, LCUenv, LCUproc, numString,
strlen(numString) + 1, 0, &error) != SUCCESS)
if (msgRecvMsg(&myfilter, 3000, &myMsg, &error) != SUCCESS)
* unrecognized answer: notify the user and continue
printf("Input error, Entry has to be either number or `q'");
errAdd(&error, "wsapp", wsappINVALID_INPUT, __FILE__, "");
* normal termination: close environment and exit
if (ccsExit(&error) != SUCCESS)
printf("\n\n %s terminated.\n\n", myProcess); /* on the terminal */
logData (myProcess, WSAPP_LOG_END, "terminated"); /* on the log file */
if (ccsExit(&error) == FAILURE)
printf("CLEAN: Unable to close connection.\n");
In the following the include files needed for generation of this application are listed:
/************************************************************************
* -------- -------- ----------------------------------------------
/************************************************************************
* Example of Workstation application
*----------------------------------------------------------------------
* System Headers required by all applications in this module
#include "rtapClasses/rtTypes.h" /* RTAP */
* to make the code more readable the following macros for error treatment
* - CLEAN used when ccsInit has been performed
* - ABORT used when ccsInit has not yet been performed
* For each "XXXXX" macro a "xxxxx" label defines in the code the
* appropriate actions needed for that specific error condition.
* The macro PRINT_ERROR produces debugging information.
printf("File %s line %d\n", __FILE__, __LINE__); \
printf("Failure: %s\n", reason); \
printf("Error structure\n"); \
printf(" EnvName : %s\n", error.envName); \
printf(" StackId : %x\n", error.stackId.id); \
printf(" SequenceNbr: %d\n", error.sequenceNumber);\
printf(" ModuleId : %s\n", error.moduleId); \
printf(" LocationId : %s\n", error.location); \
printf(" ErrorNumber: %d\n", error.errorNumber); \
printf(" Parameters : %s\n", error.runTimePar); \
#define CLEAN(reason) { PRINT_ERROR(reason); goto clean; }
#define ABORT(reason) { PRINT_ERROR(reason); goto abort; }
#define WSAPP_LOG_OFFSET 100 /* starting from 100 because appl-logs */
#define WSAPP_LOG_START (WSAPP_LOG_OFFSET + 1) /* process start */
#define WSAPP_LOG_END (WSAPP_LOG_OFFSET + 2) /* process end */
/****************************************************************
# Error Include File Created on Jun 7 19:17:47 1994
# This file has been generated by utility errEditor
# !!!!!!!!!!!!!! DO NOT MANUALLY EDIT THIS FILE !!!!!!!!!!!!!!
****************************************************************/
2.8.8 Configuration
The following operations, normally performed during CCS configuration, are required, so that the CCS Message System operates properly:
For the local as well as for all remote environments the local one wants to communicate with one entry has to be added in $RTAPROOT/etc/RtapEnvList (alternatively $VLTDATA/config/CcsEnvList for CCS_LITE), which contains the names of all RTAP/CCS_LITE environments and associated directories on the local host, and all environments and associated host names on a network.
environment name<space>root directory defines the environment's home directory
environment name<space>host name defines the host where the environment
Environment names must be unique over a network, and directory names must be unique on a local host.
All host names must also be declared in the /etc/services file together with a tcp port number associated with the corresponding service. The tcp port numbers are centrally administered and must not be selected or changed by users.
The proper behavior of the message system (valid for full CCS & CCS_Lite) requires that the owners of the processes sending and receiving commands & replies are both existing as Unix users (entry in /etc/passwd) on both sides of the communication channel.
N.B. In particular, this applies to the user "rtap". For historical reasons, this particular user must also be declared on CCS_Lite installations
For the processes msgServer and RtapQServer the following entries have to be added to the RtapEnvTable located in the environment's directory:
3 1 N P N A Y 100 128 RtapQServer
0 2 N N N A N 100 128 msgServer
For CCS_LITE, the following entries must be defined in the CcsEnvTable located in the environment's directory:
0 3 N N N N N 100 128 msgServer
The environment variable RTAPENV must be set to the name of the local environment.
 Quadralay Corporation http://www.webworks.com Voice: (512) 719-3399 Fax: (512) 719-3606 sales@webworks.com |
| |
|
|
|